SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A. W. Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
H. Wang, M. Gurbuzbalaban, L. Zhu, U. Simsekli, and M. A. Erdogdu. Convergence rates of stochastic gradient descent under infinite noise variance. Advances in Neural Information Processing Systems, 34:18866-18877, 2021.
J. Zhang, S. P. Karimireddy, A. Veit, S. Kim, S. Reddi, S. Kumar, and S. Sra. Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems, 33: 15383-15393, 2020.
Organization In Appendix A, we state some elementary probabilistic facts. The next two sections focus on proving our lemma on noisy location estimation. In Appendix B, we prove some critical lemmas used in the proof, and in Appendix C, we present the complete version of our location estimation algorithm, while addressing some typos in the main text. We mention some typos in footnotes and correct other minor typos without a mention.
Moving forward, in Appendix D, we introduce an algorithm and prove a hardness result for the specific version of list-decodable mean estimation we consider, which differs from prior work. Finally, in Appendix E, we state the final guarantees we can get for the problem of list-decodable stochastic optimization, incorporating our lemma from Appendix D.
A Elementary Probability Facts
In this section, we recall some elementary lemmas from probability theory.
Lemma A.1 (Hoeffding). Let  $X_{1}, \ldots, X_{n}$  be independent random variables such that  $X_{i} \in [a_{i}, b_{i}]$ . Let  $S_{n} := \frac{1}{n} \sum_{i=1}^{n} X_{i}$ , then for all  $t > 0$
$$
\mathbf {P r} [ | S _ {n} - \mathbf {E} [ S _ {n} ] | \geq t ] \leq \exp \left(- \frac {2 n ^ {2} t ^ {2}}{\sum_ {i = 1} ^ {n} (b _ {i} - a _ {i}) ^ {2}}\right) .
$$
Lemma A.2 (Multivariate Chebyshev). Let  $X_{1},\ldots ,X_{m}$  be independent random variables drawn from  $D$  where  $D$  is a distribution over  $\mathbb{R}^d$  such that  $\mathbf{E}_{X\sim D}[X] = 0$  and  $\mathbf{E}_{X\sim D}[\| X\| ^2 ]\leq \sigma^2$  .Let  $S_{m}\coloneqq \frac{1}{m}\sum_{i = 1}^{m}X_{i}$  , then for all  $t > 0$
$$
\Pr [ \| S _ {m} \| \geq t ] \leq \sigma^ {2} / m t ^ {2}.
$$
Proof. We first prove the following upper bound,
$$
\begin{array}{l} \mathbf {P r} \left[ \forall v \| v \| = 1. \left| \frac {1}{m} \sum_ {i \in [ m ]} X _ {i} \cdot v \right| > t \right] <   \frac {\mathbf {E} [ | \sum_ {i \in [ m ]} X _ {i} \cdot v | ^ {2} ]}{m ^ {2} t ^ {2}} <   \frac {\mathbf {E} [ \sum_ {i , j} (X _ {i} \cdot v) (X _ {j} \cdot v) ]}{m ^ {2} t ^ {2}} \\ <   \frac {\sum_ {i , j} \mathbf {E} [ (X _ {i} \cdot v) (X _ {j} \cdot v) ]}{m ^ {2} t ^ {2}} <   \frac {\sum_ {i} \mathbf {E} [ (X _ {i} \cdot v) ^ {2} ]}{m ^ {2} t ^ {2}} <   \frac {\sum_ {i} \mathbf {E} [ \| X _ {i} \| ^ {2} ]}{m ^ {2} t ^ {2}} \\ <   \frac {m \sigma^ {2}}{m ^ {2} t ^ {2}} = \frac {\sigma^ {2}}{m t ^ {2}}. \\ \end{array}
$$
Since the inequality holds for all unit  $v$ , it also holds for the unit  $v$  in the direction of  $S_{m}$ , completing the proof.
Fact A.3 (Inflation via conditional probability). Let  $y$  be a random variable with mean  $\mu$  and variance  $\sigma^2$  and let  $\xi$  be an arbitrary random variable independent of  $y$ , then
$$
\begin{array}{l} \Pr [ \xi \in (a, b) ] <   (1 + 2 / A ^ {2}) \Pr [ \xi + y \in (a + \mu - \sigma A, b + \mu + \sigma A) \\ <   \left(1 + 2 / A ^ {2}\right) \Pr [ \xi + y \in \left(a - | \mu | - | \sigma A |, b + | \mu | + | \sigma A |\right) ]. \\ \end{array}
$$
Proof. To do this, we inflate the intervals and use conditional probabilities.
$$
\begin{array}{l} \mathbf {P r} [ \xi \in (a, b) ] = \mathbf {P r} [ \xi + y \in (a + \mu - \sigma A, b + \mu + \sigma A) \mid | y - \mu | <   \sigma A ] \\ = \frac {\operatorname {P r} [ \xi + y \in (a + \mu - \sigma A , b + \mu + \sigma A) \text {a n d} | y - \mu | <   \sigma A ]}{\operatorname {P r} [ | y - \mu | <   \sigma A ]} \\ <   (1 - 1 / A ^ {2}) ^ {- 1} \Pr [ \xi + y \in (a + \mu - \sigma A, b + \mu + \sigma A) \text {a n d} | y - \mu | <   \sigma A ] \\ <   \left(1 + 2 / A ^ {2}\right) \mathbf {P r} [ \xi + y \in (a + \mu - \sigma A, b + \mu + \sigma A) ]. \\ \end{array}
$$
The second inequality above follows from observing that we are simply lengthening the interval.
We will often use the second version for ease of analysis.
B Useful Lemmas
In this section, we present some helpful lemmas for the algorithm on noisy one-dimensional location estimation.
To recap the setting: We can access samples from distributions  $\xi + y$  and  $\xi + y' + t$ . Here,  $\mathbf{Pr}[\xi = 0] > \alpha$ ,  $y$  and  $y'$  are distributions with zero mean and bounded variance, and  $t \in \mathbb{R}$  is an unknown translation. Our objective is to estimate the value of  $t$ .
B.1 Useful Lemma for Rough Estimation
Our algorithm for one-dimensional location estimation consists of two steps. In the first step, we obtain an initial estimate of the shift between the two distributions by computing pairwise differences of samples drawn from each distribution. This involves taking the median of the distribution of  $x + y$ , where  $x$  is symmetric and  $y$  has mean 0 and bounded variance.
The following lemma demonstrates that the median of this distribution is at most  $O(\sigma \alpha^{-1/2})$ , where  $\sigma$  is the standard deviation of  $y$ . Furthermore, this guarantee cannot be improved.
Fact B.1 (Median of Symmetric + Bounded-variance Distribution). Let  $x$  be a random variable symmetric around 0 such that  $\alpha \in (0,1)$ ,  $\mathbf{Pr}[x = 0] \geq \alpha$ . Let  $y$  be a random variable with mean 0 and variance  $\sigma^2$ . If  $S$  is a set of  $O(1 / \alpha^2 \log(1 / \delta))$  samples drawn from the distribution of  $x + y$ ,  $|\mathrm{median}(S)| \leq O(\sigma \alpha^{-1/2})$ .
This guarantee is tight in the sense that there exist distributions for  $x$  and  $y$  satisfying the above constraints, such that  $\mathrm{median}(x + y)$  can be as large as  $\Omega (\sigma \alpha^{-1 / 2})$ .
Proof. We show that  $\mathbf{Pr}[x + y < -O(\sigma / \sqrt{\alpha})] < 0.5$  and  $\mathbf{Pr}[x + y > O(\sigma / \sqrt{\alpha})] < 0.5$ , as a result,  $|\mathrm{median}(x + y)| < O(\sigma / \sqrt{\alpha})$ . We will later transfer this guarantee to the uniform distribution over the samples.
Applying Fact A.3 to the first probability, we see that  $\mathbf{Pr}[x + y < -O(\sigma / \sqrt{\alpha})] < (1 + \alpha)\mathbf{Pr}[x < -O(\sigma / \sqrt{\alpha})]$ .
Since  $\mathbf{Pr}[x = 0] \geq \alpha$ , we see that  $\mathbf{Pr}[x < 0] \leq 1/2 - \alpha$ ,
and so  $\mathbf{Pr}[x + y < -O(\sigma / \sqrt{\alpha})] < (1 + \alpha)(0.5 - \alpha) = 0.5 - \alpha + 0.5\alpha - \alpha^2 = 0.5 - 0.5\alpha - \alpha^2 < 0.5.$
The upper bound follows similarly.
Since  $\mathbf{Pr}[x = 0] \geq \alpha$  and  $\mathbf{Pr}[|y| < O(\sigma / \sqrt{\alpha})] \geq 1 - \alpha$ , we see  $\mathbf{Pr}[|x + y| < O(\sigma / \sqrt{\alpha})] \geq \alpha / 2$ . Hoeffding's inequality (Lemma A.1) now implies that the empirical median also satisfies the above upper bound as long as the number of samples is greater than  $O(1) / \alpha^2 \log(1 / \delta)$ .
To see that this is tight, consider the distribution centered at 0, whose density function is  $2 / (y + 2)^3$  in the range  $[1,\infty)$ , and is 0 otherwise.
Call this  $D_{y^{-3}}$ . Observe that  $\mathbf{Pr}_{D_{y^{-3}}}[z > t] < O(1)\int_t^\infty y^{-3}dy = C / t^2$
Let  $x$  be a symmetric distribution whose distribution takes the value 0 with probability  $\alpha$  and takes the values  $\pm \alpha^{-1/2} 100C^{1/2} + 10$  with probability  $0.5(1 - \alpha)$ .
We show that the median of the distribution of  $x + y$  where  $y$  is drawn from  $D_{y^{-3}}$ , is larger than  $\Omega(\alpha^{-1/2})$ .
To see this, we show that the probability that  $x + y$  takes a value smaller than  $100\alpha^{-1/2}C^{1/2}$  is less than half, implying that the median has to be larger than this quantity.
$x$  takes three values. Note that  $y + 100\alpha^{-1/2}C^{1/2} + 12$  places no mass in the region  $(- \infty, 100\alpha^{-1/2}C^{1/2} + 10]$ . So to estimate the probability that  $x + y$  takes a value smaller than  $100\alpha^{-1/2}C^{1/2} + 10$ , we only need to consider contributions from the other two possible values. By
choosing  $\alpha$  small enough, so that  $100\alpha^{-1 / 2}C^{1 / 2} > 10$ , we see
$$
\begin{array}{l} \Pr [ x + y <   1 0 0 \alpha^ {- 1 / 2} C ^ {1 / 2} + 1 0 ] \\ <   0. 5 (1 - \alpha) \Pr [ y <   2 0 0 \alpha^ {- 1 / 2} C ^ {1 / 2} + 2 0 ] + \alpha \Pr [ y <   1 0 0 \alpha^ {- 1 / 2} C ^ {1 / 2} + 1 0 ] \\ <   0. 5 (1 - \alpha) (1 - C / (2 0 0 \alpha^ {- 1 / 2} C ^ {1 / 2} + 2 0) ^ {2}) + \alpha (1 - C / (1 0 0 \alpha^ {- 1 / 2} C ^ {1 / 2} + 1 0) ^ {2}) \\ <   0. 5 (1 - \alpha) \left(1 - \alpha^ {1 / 2} / (4 0 0) ^ {2}\right) + \alpha \\ <   0. 5 + 0. 5 \alpha - \alpha^ {1 / 2} / 8 \cdot \left(4 0 0 ^ {2}\right). \\ \end{array}
$$
We are done when  $0.5\alpha - \alpha^{1/2} / 8 \cdot (400^2) < 0$ , this happens for  $\alpha^{1/2} < 2 / (8 \cdot (400^2))$ .
B.2 Useful Lemma for Finer Estimation
In the second step of our location-estimation lemma, we refine the estimate of  $t$ . To do this, we first re-center the distributions based on our rough estimate, so that the shift after re-centering is bounded. Then, we identify an interval  $I$  centered around 0 such that, when conditioning on  $\xi + z$  falling within this interval, the expected value of  $\xi + z$  remains the same as when conditioning on  $\xi$  falling within the same interval. This expectation will help us get an improved estimate, which we use to get an improved re-centering of our original distributions, and repeat the process.
To identify such an interval, we search for a pair of bounded-length intervals equidistant from the origin (for e.g.  $(-10\sigma, -5\sigma)$  and  $(5\sigma, 10\sigma)$ ) that contain very little probability mass. By doing so, when  $z$  is added to  $\xi$ , the amount of probability mass shifted into the interval  $(-5\sigma, 5\sigma)$ $z$  remains small.
In this subsection, we prove Lemma B.2, which states that any positive sequence which has a finite sum must eventually have one small element. The lemma also gives a concrete upper bound on which element of the sequence satisfies this property.
Lemma B.2.  $a_i \geq 0$  for all  $i$  and  $\sum_{i=0}^{\infty} a_i < C$  for some constant  $C$ . Also, suppose we have  $\eta \in (0,1)$  and  $L \in \mathbb{R}$  such that  $L \geq 1$ . Then there is an integer  $i$  such that  $L \leq i \leq (C / a_0 + L^\eta)^{1/\eta}$  and  $ia_i < \eta \sum_{j=0}^{i} a_j$ .
Consider a partition of the reals into length  $L$  intervals. Here,  $L$  simply denotes the length of the intervals and the lower bound to  $i$ , not the smoothness parameter.
In our proof, we will use Lemma B.2 on the sequence  $a_i$ , where  $a_i$  corresponds to an upper bound on the mass of  $\xi$  contained in the  $i$ -th intervals equidistant from the origin on either side, and the mass that crosses them (i.e., the mass of  $\xi$  that is moved either inside or out of the interval when  $z$  is added to it).
We need the following calculation to prove Lemma B.2.
Notation: For integer  $i \geq 1$  and  $\eta \in (0,1)$ , define  $(i - \eta)! \coloneqq \Pi_{j=1}^{i}(j - \eta)$ .
Fact B.3. Let  $A_{k} \coloneqq 1 + \sum_{t = 1}^{k - 1}\frac{\eta(t - 1)!}{(t - \eta)!}$ . Then, for  $k \geq 2$ ,  $A_{k} = (k - 1)! / (k - 1 - \eta)!$ .
Proof. We prove this by induction. By definition, our hypothesis holds for  $A_{2}$  because  $A_{2} = 1 + \eta / (1 - \eta) = 1 / (1 - \eta) = (2 - 1)! / (2 - 1 - \eta)!$ . Suppose it holds for all  $2 \leq t \leq k$ . We then show that it holds for  $t = k + 1$ .
$$
\begin{array}{l} A _ {k + 1} = 1 + \sum_ {t = 1} ^ {k} \frac {\eta (t - 1) !}{(t - \eta) !} = A _ {k} + \frac {\eta (k - 1) !}{(k - \eta) !} \\ = \frac {(k - 1) !}{(k - 1 - \eta) !} + \frac {\eta (k - 1) !}{(k - \eta) !} = \frac {(k - 1) !}{(k - 1 - \eta) !} \left(1 + \frac {\eta}{k - \eta}\right) \\ = \frac {(k - 1) !}{(k - 1 - \eta) !} \frac {k}{k - \eta} = \frac {k !}{(k - \eta) !}. \\ \end{array}
$$
Proof of Lemma B.2 Let  $U = (C / a_0 + L^\eta)^{1 / \eta}$  and suppose towards a contradiction that there is no such  $i$  that satisfies the lemma. That means, we will assume that  $ia_i \geq \eta \sum_{j=0}^i a_j$  for all integers  $i \in [1, U]$ ; or equivalently, we assume  $a_i \geq \frac{\eta}{i - \eta} \sum_{j=0}^{i-1} a_j$ . We then show that this implies  $i^{1 - \eta}a_i \geq \eta a_0$  for all  $i$  in the range.
Consider the inductive hypothesis on  $t$  given by  $a_{t} \geq \eta \frac{(t - 1)!}{(t - \eta)!} \cdot a_{0}$ . The base case when  $t = 1$  is true since  $a_{1} \geq \eta a_{0} / (1 - \eta)$  by our assumption. Suppose the inductive hypothesis holds for integers  $t \in [1, k - 1]$ . We show this for  $t = k$  below. By starting with our assumption then applying the inductive hypothesis, we have that
$$
\begin{array}{l} a _ {k} \geq \frac {\eta}{k - \eta} \sum_ {t = 0} ^ {k - 1} a _ {t} \\ \geq \frac {a _ {0} \eta}{k - \eta} \left(1 + \sum_ {t = 1} ^ {k - 1} \frac {\eta (t - 1) !}{(t - \eta) !}\right) \\ = a _ {0} \eta \frac {(k - 1) !}{(k - \eta) !}. \\ \end{array}
$$
The final equality follows from Fact B.3 which states that  $(k - 1)! / (k - 1 - \eta)! = 1 + \sum_{t = 1}^{k - 1}\frac{\eta(t - 1)!}{(t - \eta)!}$ . Simplifying this further, we see that since  $(i - \eta)\geq i\exp (-\eta /i)$  for all  $i\in [1,k]$ ,
$$
\begin{array}{l} a _ {k} \geq a _ {0} \eta \frac {(k - 1) !}{(k - \eta) !} \\ \geq a _ {0} \eta \frac {(k - 1) !}{k ! \exp (- \eta / k)} \\ \geq a _ {0} \eta (1 / k) (1 / \exp (- \eta (\sum_ {i = 1} ^ {k} 1 / i))) \\ \geq a _ {0} \eta (1 / k) (1 / \exp (- \eta \log (k) / 2 0)) \\ \geq \left(a _ {0} / 2\right) \eta \left(1 / k ^ {1 - \eta / 2 0}\right). \\ \end{array}
$$
$$
\begin{array}{l} C > \sum_ {i = L} ^ {U} a _ {i} > a _ {0} \eta \sum_ {i = L} ^ {U} \left(1 / i ^ {1 - \eta / 2 0}\right) \\ > a _ {0} \eta \int_ {L} ^ {U} (1 / x ^ {1 - \eta}) d x \\ = a _ {0} \left(U ^ {\eta} - L ^ {\eta}\right). \\ \end{array}
$$
By definition,  $U = (C / a_0 + L^\eta)^{1 / \eta}$  so the above inequality cannot be true and thus we have arrived at a contradiction. Then it follows that there exists an  $i$  such that  $1 \leq L \leq i \leq (C / a_0 + L^\eta)^{1 / \eta}$  and  $ia_i < \eta \sum_{j=0}^{i} a_j$ .
C Noisy Location Estimation
In this section, we state and prove the guarantees of our algorithms for noisy location estimation (Lemma 3.1 and Lemma 3.5).
C.1 One-dimensional Noisy Location Estimation
Throughout the technical summary and some parts of the proof, we make the assumption that the variables  $y$  and  $y^\prime$  were bounded. Extending this assumption to bounded-variance distributions requires significant effort.
Our algorithm for one-dimensional noisy location estimation (Algorithm 1) can be thought of as a two-step process. The first step involves a rough initial estimation algorithm, while the second step employs an iterative algorithm that progressively refines the estimate by a factor of  $\eta$  in each iteration.
Due to space limitations and for ease of exposition, the algorithm we present in the main body is a sketch of the refinement procedure.
In this Algorithm 1, we introduce the definition of  $\hat{P}$  (the empirical estimate of  $\tilde{P}(\cdot)$ ), which is an upper bound on the probability mentioned earlier. This probability can be calculated using samples from  $\xi + z$  and  $\xi + z'$ . Additionally, we incorporate the iterative refinement process within the algorithm.
Algorithm 4 One-dimensional Location Estimation: Shift1D  $(S_{1}, S_{2}, \eta, \sigma, \alpha)$
Input: Sample sets  $S_{1}, S_{2} \subset \mathbb{R}^{d}$  of size  $m, \alpha, \eta \in (0, 1), \sigma > 0$
1. Let  $T = O(\log_{1/\eta}(1/\alpha))$ . For  $j \in \{1,2\}$ , partition  $S_j$  into  $T$  equal pieces,  $S_j^{(i)}$  for  $i \in [T]$ .
2.  $D = \{a - b\mid a\in S_1^{(1)},b\in S_2^{(1)}\}$
3.  $t^{\prime}(1)\coloneqq \mathrm{median}(D)$
4. Set  $A = O(1 / \sqrt{\alpha})$
5. Repeat steps 6 to 12, for  $i$  going from 2 to  $T$ :
6.  $S_{1}^{(i)}\coloneqq S_{1}^{(i)} - t_{r}^{\prime}(i - 1)$
7. For  $j \in \{1, 2\}$
$$
\hat{P} (k)\leq \eta \sum_{j\in \{1,2\}}\mathbf{Pr}_{x\sim S_{j}^{(i)}}[|x|\in A\sigma k]\pm O(\eta /i).
$$
Lemma C.1 (One-dimensional location-estimation). There is an algorithm (Algorithm 1) which, given  $(1 / \eta^5)(O(1) / \alpha)^{2 / \eta}\log (1 / \eta \alpha \delta)$  samples  $\{\xi_i + y_i + t\}_{i = 1}^m$  and  $\{\tilde{\xi}_i + y_i'\}_{i = 1}^m$  where  $\xi_{i}$  and  $\tilde{\xi}_i$  are both drawn from  $D_{\xi}$ ,  $y_{i}$  and  $y_{i}'$  are drawn from distinct distributions with mean-zero with variance bounded above by  $\sigma$  and  $t\in \mathbb{R}$  is an unknown translation, runs in time  $\mathrm{poly}(1 / \eta ,(O(1) / \alpha)^{1 / \eta},\log (1 / \delta \eta \alpha))$  and recovers  $t^\prime$  such that  $|t - t^{\prime}|\leq O(\eta \sigma)$ .
Proof. Our proof is based on the following claims:
Claim C.2 (Rough Estimate). There is an algorithm which, given  $m = O((1 / \alpha^4)\log (1 / \delta))$  samples of the kind  $\xi +y + t$  and  $\xi +y^{\prime}$  , where  $t\in \mathbb{R}$  is an unknown translation, returns  $t_r^\prime$  satisfying  $|t_r^\prime -t| <   O(\sigma \alpha^{-1 / 2})$
Claim C.3 (Fine Estimate). Suppose  $z, z'$  have means bounded from above by  $A\sigma$  and variances at most  $\sigma^2$  and suppose  $\alpha \in (0,1)$  and  $\eta \in (0,1/2)$ . Then in  $\mathrm{poly}((O(1)/\alpha\eta)^{1/\eta}, \log(1/\delta\eta\alpha))$  samples and  $\mathrm{poly}((O(1)/\alpha\eta)^{1/\eta}, \log(1/\delta\eta\alpha))$  time, it is possible to recover  $k \in [1/\eta\alpha, (O(1)/\eta\alpha)^{O(1/\eta)}]$  such that
$$
\widehat {\mathbf {E}} [ \xi + z \mid | \xi + z | \leq A \sigma k ] - \widehat {\mathbf {E}} [ \xi + z ^ {\prime} \mid | \xi + z ^ {\prime} | \leq A \sigma k ] = \mathbf {E} [ z ] - \mathbf {E} [ z ^ {\prime} ] \pm \eta (A \sigma).
$$
Using Claim 3.2, we first identify a rough estimate  $t_r'$  satisfying  $|t_r' - t| < O(\sigma \alpha^{-1/2})$ . This allows us to re-center  $y'$ . Let the re-centered distribution be denoted by  $z' = y'$  and  $z = y + t - t_r'$ . Then  $z$  and  $z'$  are such that  $\mathbf{E}[z]$  and  $\mathbf{E}[z']$  are both at most  $O(\sigma \alpha^{-1/2})$  in magnitude, and have variance at most  $\sigma^2$ .
Claim 3.4 then allows us to estimate  $t_f^\prime$  such that  $\left|(E[z] - E[z^{\prime}]) - t_f^{\prime}\right| = \left|t - t_r^{\prime} - t_f^{\prime}\right|\leq \eta O(\sigma \alpha^{-1 / 2})$
Setting  $t' = t_r' + t_f'$ , we see that our estimate  $t'$  is now  $\eta$  times closer to  $t$  compared to  $t_r'$ .
To refine this estimate further, we can obtain fresh samples and re-center using  $t'$  instead of  $t_{\tau}^{\prime}$ . Repeating this process  $O(\log_{1 / \eta}(1 / \alpha)) = O(\log_{\eta}(\alpha))$  times is sufficient to obtain an estimate that incurs an error of  $\eta \cdot \eta^{\log_{\eta}(\alpha^{1 / 2})} \cdot O(\sigma \alpha^{-1 / 2}) \leq O(\eta \sigma)$ .
This results in a runtime and sample complexity that is only  $O(\log_{1/\eta}(1/\alpha))$  times the runtime and sample complexity required by Claim 3.4. This amounts to the final runtime and sample complexity being  $\text{poly}((O(1)/\alpha\eta)^{1/\eta}, \log(1/\delta\eta\alpha))$ .
We now prove Claim 3.2 and Claim 3.4.
Claim 3.2 shows that the median of the distribution of pairwise differences of  $\xi + y + t$  and  $\xi + y'$  estimates the mean up to an error of  $\sigma \alpha^{-1/2}$ .
Proof of Claim 3.2 Let  $\tilde{\xi}$  be a random variable with the same distribution as  $\xi$  and independently drawn. We have independent samples from the distributions of  $\xi + y + t$  and  $\xi + y'$ . Applying Fact B.1 to these distributions, we see that if we have at least  $O(1 / \alpha^4)\log (1 / \delta)$  samples from the distribution of  $(\xi - \tilde{\xi}) + (y - y') + t$ , these samples will have a median of  $t \pm O(\sigma / \sqrt{\alpha})$ .
Proof of Claim 3.4 To identify such a  $k$ , the idea is to ensure that  $\mathbf{E}[\xi + z \mid |\xi + z| \leq A\sigma k] = \mathbf{E}[\xi + z \mid |\xi| \leq A\sigma k] \pm O(A\eta\sigma) = \mathbf{E}[\xi \mid |\xi| \leq A\sigma k] + \mathbf{E}[z] \pm O(A\eta\sigma)$ , and similarly for  $z'$ . The theorem follows by taking the difference of these equations.
Before we proceed, we will need the following definitions: let  $P(i,z)$  be defined as follows:
$$
\begin{array}{l} P (i, z) := \Pr [ | \xi | \in A \sigma (i - 1, i + 1) ] \\ + \Pr [ | \xi | <   A i \sigma , | \xi + z | > A i \sigma ] + \Pr [ | \xi | > A i \sigma , | \xi + z | <   A i \sigma ]. \\ \end{array}
$$
This will help us bound the final error terms that arise in the calculation. We will need the following upper bound on  $P(i,z) + P(i,z')$ .
Claim C.4. There exists a function  $\tilde{P}:\mathbb{N}\to \mathbb{R}^{+}$  satisfying:
1. For all  $i \in \mathbb{N}$ ,  $\tilde{P}(i) \geq P(i, z) + P(i, z')$  which can be computed using samples from  $\xi + z$  and  $\xi + z'$ .
2. There is a  $k \in [(1 / \alpha \eta), (C / \alpha + 1 / (\alpha \eta)^{\eta})^{1 / \eta}]$  such that  $k\tilde{P}(k) < \eta \sum_{j=0}^{k} \tilde{P}(k)$ .
3.  $\sum_{j=0}^{k} \tilde{P}(k) = O(\mathbf{Pr}[|\xi + z| \leq A\sigma k] + \mathbf{Pr}[|\xi + z'| \leq A\sigma k])$ .
4. With probability  $1 - \delta$ , for all  $i < (O(1) / \alpha)^{1 / \eta} / \eta$ ,  $\tilde{P}(i)$  can be estimated to an accuracy of  $O(\eta / i)$  by using  $(O(1) / \alpha)^{2 / \eta} \log (1 / \eta \alpha \delta) / \eta^5$  samples from  $\xi + z$  and  $\xi + z'$ .
We defer the proof of Claim C.4 to Appendix C.2, and continue with our proof showing that  $\mathbf{E}[\xi + z \mid |\xi + z| \leq A\sigma k] \approx \mathbf{E}[\xi + z \mid |\xi| \leq A\sigma k]$  for  $k$  satisfying the conclusions of Claim C.4. To this end, observe the following for  $f(\xi, z)$  being either 1 or  $\xi + z$ .
$$
\begin{array}{l} \left. \left| \mathbf {E} [ f (\xi , z) \mathbf {1} (| \xi | \leq \sigma i) ] - \mathbf {E} [ f (\xi , z) \mathbf {1} (| \xi + z | \leq \sigma i) ] \right| \right. \\ \leq \left| \mathbf {E} [ f (\xi , z) \mathbf {1} (| \xi + z | > \sigma i) \mathbf {1} (| \xi | \leq \sigma i) ] \right| + \left| \mathbf {E} [ f (\xi , z) \mathbf {1} (| \xi + z | \leq \sigma i) \mathbf {1} (| \xi | > \sigma i) ] \right|. \\ \end{array}
$$
By setting  $f(\xi, z) \coloneqq 1$  and considering the case where  $i = k$  satisfies the conclusions of Claim C.4, we can bound the "error terms"
$\mathbf{Pr}[|\xi + z| \leq A\sigma k$  and  $|\xi| > A\sigma k]$  and  $\mathbf{Pr}[|\xi + z| > A\sigma k$  and  $|\xi| \leq A\sigma k]$  in terms of  $\tilde{P}(k)$ .
Furthermore,  $\tilde{P}(k)$  itself is upper bounded by  $O(\eta / k)(\mathbf{Pr}[|\xi + z| \leq A\sigma k] + \mathbf{Pr}[|\xi + z'| \leq A\sigma k])$  as per Item 2 and Item 3. Putting these facts together, we have that
$$
\begin{array}{l} | \mathbf {P r} [ | \xi | \leq A \sigma k ] - \mathbf {P r} [ | \xi + z | \leq A \sigma k ] | \\ = O (\eta / k) \left(\mathbf {P r} [ | \xi + z | \leq A \sigma k ] + \mathbf {P r} [ | \xi + z ^ {\prime} | \leq A \sigma k ]\right). \\ \end{array}
$$
A similar claim holds for the distribution over  $z'$ . An application of the triangle inequality now implies
$$
\begin{array}{l} \left| \Pr \left[ \left| \xi + z ^ {\prime} \right| \leq A \sigma k \right] - \Pr \left[ \left| \xi + z \right| \leq A \sigma k \right] \right| \\ = O (\eta / k) \left(\mathbf {P r} [ | \xi + z | \leq A \sigma k ] + \mathbf {P r} [ | \xi + z ^ {\prime} | \leq A \sigma k ]\right). \\ \end{array}
$$
If  $|A - B| < \tau (A + B)$  it follows that  $(1 - \tau) / (1 + \tau) < A / B < (1 + \tau) / (1 - \tau)$ . For  $\tau \in (0, 1/2]$ , this means  $A = \Theta(B)$ . Applying this to our case, we can conclude that  $\mathbf{Pr}[|\xi + z'| \leq A\sigma k] = \Theta(\mathbf{Pr}[|\xi + z| \leq A\sigma k])$ . Substituting this equivalence back into the previous expression, we obtain:
$$
\left. \left| \mathbf {P r} [ | \xi | \leq A \sigma k ] - \mathbf {P r} [ | \xi + z | \leq A \sigma k ] \right| = O (\eta / k) \left(\mathbf {P r} [ | \xi + z | \leq A \sigma k ]\right). \right. \tag {1}
$$