SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Similarly, when  $f(\xi, z) \coloneqq \xi + z$ , we need to control the error terms:  $\mathbf{E}[(\xi + z) \mathbf{1}(|\xi + z| \leq A\sigma k) \mathbf{1}(|\xi| > A\sigma k)]$  and  $\mathbf{E}[(\xi + z) \mathbf{1}(|\xi + z| > A\sigma k) \mathbf{1}(|\xi| \leq A\sigma k)]$ .
Observe that  $(\xi + z)\mathbf{1}(|\xi + z| \leq A\sigma k)\mathbf{1}(|\xi| > A\sigma k)$  has a nonzero value with probability at most  $\mathbf{E}[\mathbf{1}(|\xi + z| \leq A\sigma k)\mathbf{1}(|\xi| > A\sigma k)] < \tilde{P}(k)$ . Also, the magnitude of  $(\xi + z)$  in this event is at most  $A\sigma k$ . Putting these together, we get that
$$
| \mathbf {E} [ (\xi + z) \mathbf {1} (| \xi + z | \leq A \sigma k) \mathbf {1} (| \xi | > A \sigma k) ] | <   A \sigma k \tilde {P} (k) <   O (A \sigma \eta) \sum_ {j = 1} ^ {k} \tilde {P} (j).
$$
Unfortunately, we cannot use the same argument to bound  $\mathbf{E}[(\xi + z) \mathbf{1}(|\xi + z| > A\sigma k) \mathbf{1}(|\xi| \leq A\sigma k)]$ , since  $|\xi + z|$  is no longer bounded by  $A\sigma k$  in this event. However, we can break the sum  $\xi + z$  as follows:  $\xi + z = \xi + z\mathbf{1}(|z| > A\sigma k) + z\mathbf{1}(|z| \leq A\sigma k)$ . This allows us to get the following bound:
$$
\begin{array}{l} \mathbf {E} \left[ (\xi + z) \mathbf {1} (| \xi + z | > A \sigma k) \mathbf {1} (| \xi | \leq A \sigma k) \right] \\ <   2 A \sigma k \tilde {P} (k) + \mathbf {E} [ z \mathbf {1} (| z | > A \sigma k) \mathbf {1} (| \xi + z | > A \sigma k) \mathbf {1} (| \xi | \leq A \sigma k) ] \\ <   2 A \sigma k \tilde {P} (k) + \mathbf {E} [ z \mathbf {1} (| z | > A \sigma k) ] \\ <   2 A \sigma k \tilde {P} (k) + A \sigma / k \\ <   O (A \eta \sigma \sum_ {j = 1} ^ {k} \tilde {P} (j)) + O (A \eta \sigma \alpha), \\ \end{array}
$$
where the third inequality follows by an application of Chebyshev's inequality, and the final inequality follows by choosing  $k \geq 1 / (\eta \alpha)$ .
Putting everything together, we see
$$
\mathbf {E} [ (\xi + z) \mathbf {1} (| \xi + z | \leq A \sigma k) ] = \mathbf {E} [ (\xi + z) \mathbf {1} (| \xi | \leq A \sigma k) ] \pm O (A \sigma \eta \alpha + A \sigma \eta \sum_ {j = 1} ^ {k} \tilde {P} (j)). \tag {2}
$$
To finally compute the conditional probability, we use Equation (1) and Equation (2) to get
$$
\begin{array}{l} \mathbf {E} [ (\xi + z) \mid | \xi | \leq A \sigma k ] = \frac {\mathbf {E} [ (\xi + z) \mathbf {1} (| \xi + z | \leq A \sigma k) ] \pm O (A \sigma \eta \alpha + A \sigma \eta \sum_ {j = 1} ^ {k} \tilde {P} (j))}{\mathbf {P r} [ | \xi | \leq A \sigma k ]} \\ = \frac {\mathbf {E} [ (\xi + z) \mathbf {1} (| \xi + z | \leq A \sigma k) ] \pm O (A \sigma \eta \alpha + A \sigma \eta \sum_ {j = 1} ^ {k} \tilde {P} (j))}{(1 + \Theta (\eta / k)) \mathbf {P r} [ | \xi + z | \leq A \sigma k ]} \\ = (1 - \Theta (\eta / k)) \mathbf {E} [ (\xi + z) | | \xi + z | \leq A \sigma k ] \\ \pm O (1) \frac {A \sigma \eta \alpha + A \sigma \eta \mathbf {P r} [ | \xi + z | \leq A \sigma k ]}{\mathbf {P r} [ | \xi + z | \leq A \sigma k ]} \\ = \mathbf {E} [ (\xi + z) \mid | \xi + z | \leq A \sigma k ] \pm O (A \eta \sigma), \\ \end{array}
$$
where the second inequality is a consequence of Item 3, and the last is due to the fact that  $\mathbf{Pr}[|\xi + z| \leq A\sigma k] \geq \alpha / 2$  whenever  $k > 2$ , which follows from an application of Fact A.3 while noting the fact that  $\mathbf{Pr}[\xi = 0] \geq \alpha$ .
Taking a difference for the above calculations for  $z$  and  $z'$ , we see that,
$$
\mathbf {E} [ (\xi + z) \mid | \xi + z | \leq A \sigma k ] - \mathbf {E} [ (\xi + z ^ {\prime}) \mid | \xi + z ^ {\prime} | \leq A \sigma k ] = \mathbf {E} [ z ] - \mathbf {E} [ z ^ {\prime} ] \pm O (A \eta \sigma).
$$
Consider this final error, and let  $O(A\eta \sigma) < CA\eta \sigma$  for some constant  $C$ . Repeating the above argument initially setting  $\eta = \eta' / C$ , where  $C$  is the constant gives us the guarantee we need.
Finally, we estimate the runtime and sample complexity of our algorithm. The main bottleneck in our algorithm is the repeated estimation of  $\tilde{P}(i)$  and estimation of  $\mathbf{E}[(\xi + z) \mid |\xi + z| \leq A\sigma k]$ .
According to Item 4, each time we estimate  $\tilde{P}(i)$  to the desired accuracy, we draw  $\mathrm{poly}(1/\eta, (O(1)/\alpha)^{1/\eta}, \log(1/\delta \eta \alpha))$  samples.
An application of Hoeffding's inequality (Lemma A.1) then allows us to estimate the conditional expectation  $\mathbf{E}[(\xi + z) \mid |\xi + z| \leq A\sigma k]$  to an accuracy of  $\eta A\sigma$  by drawing  $\mathrm{poly}(1/\eta, (O(1)/\alpha)^{1/\eta}, \log(1/\delta \eta \alpha))$  samples as well. The exponential dependence here comes from the exponential upper bound on  $k$ .
C.2 Proof of Claim C.4
In this section, we prove the existence of  $\tilde{P} (\cdot)$  which is an upper bound on  $P(i,z) + P(i,z^{\prime})$ , which we can estimate using samples from  $\xi +z$  and  $\xi +z^{\prime}$ .
Proof of Claim C.4
Proof of Item 1:
Recall the definition of  $P(i,z)$
For Item 1 to hold, we need to define  $\tilde{P}(i)$  to be an upper bound on  $P(i,z) + P(i,z')$  which can be computed using samples from  $\xi + z$  and  $\xi + z'$ . To this end, we bound  $P(i,z)$  as follows. First, note that we can adjust the endpoints of the intervals to get
$$
\begin{array}{l} P (i, z) <   3 \Pr [ | \xi | \in A \sigma (i - 1, i + 1) ] \\ + \Pr [ | \xi | <   A (i - 1) \sigma , | \xi + z | > A i \sigma ] + \Pr [ | \xi | > A (i + 1) \sigma , | \xi + z | <   A i \sigma ]. \\ \end{array}
$$
Then, we partition the ranges in the definition above into intervals of length  $A\sigma$  to get:
$$
\begin{array}{l} P (i, z) <   3 \Pr [ | \xi | \in A \sigma (i - 1, i + 1) ] \\ + \sum_ {j = 1} ^ {i - 2} \mathbf {P r} [ | \xi | \in A j \sigma + [ 0, A \sigma), | \xi + z | > A i \sigma ] \\ + \sum_ {j = 1} ^ {i - 1} \mathbf {P r} [ | \xi | > A (i + 1) \sigma , | \xi + z | \in A j \sigma + [ 0, A \sigma) ]. \\ \end{array}
$$
Next, an application of the triangle inequality to  $|\xi| \in A j\sigma + [0, A\sigma)$  and  $|\xi + z| > A i\sigma$  implies that  $|z| \geq A(i - j - 1)\sigma$ . Similarly, the same kind of argument when  $|\xi| > A(i + 1)\sigma$  and  $|\xi + z| \in A j\sigma + [0, A\sigma)$  demonstrates that  $|-z| = |\xi + z - \xi| \geq A(i - j)\sigma$ . We then use Fact A.3 to move from  $|\xi + z|$  to  $|\xi|$  in the third term.
$$
\begin{array}{l} P (i, z) <   3 \Pr [ | \xi | \in A \sigma (i - 1, i + 1) ] \\ + \sum_ {j = 1} ^ {i - 2} \Pr [ | \xi | \in A j \sigma + A \sigma [ 0, 1), | z | \geq (i - j - 1) A \sigma ] \\ + O (1) \sum_ {j = 1} ^ {i - 1} \mathbf {P r} [ | \xi | \in A j \sigma + A \sigma [ - 2, 3), | z | \geq (i - j) A \sigma ]. \\ \end{array}
$$
An application of Chebyshev's inequality to  $z$ , using the independence of  $z$  and  $\xi$ , gives that
$$
\begin{array}{l} P (i, z) <   3 \Pr [ | \xi | \in A \sigma (i - 1, i + 1) ] \\ + O (1) \sum_ {j = 1} ^ {i - 2} \left(1 / (i - j - 1) ^ {2}\right) \Pr [ | \xi | \in A j \sigma + A \sigma [ 0, 1) ] \\ + O (1) \sum_ {j = 1} ^ {i - 1} \left(1 / (i - j) ^ {2}\right) \Pr [ | \xi | \in A j \sigma + A \sigma [ - 2, 3) ]. \\ \end{array}
$$
Another application of Fact A.3 applied to  $(\xi + z) - z$  then gives us
$$
\begin{array}{l} P (i, z) <   3 \Pr [ | \xi + z | \in A \sigma (i - 5, i + 5) ] \\ + O (1) \sum_ {j = 1} ^ {i - 2} \left(1 / (i - j - 1) ^ {2}\right) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 2, 3) ] \\ + O (1) \sum_ {j = 1} ^ {i - 1} \left(1 / (i - j) ^ {2}\right) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5 ]). \\ \end{array}
$$
Finally, extending all intervals so that they match, and observing that  $\sum_{j=1}^{i-2}(1/(i-j-1)^2)\mathbf{Pr}[|\xi + z| \in Aj\sigma + A\sigma[-2,3)] \leq \sum_{j=1}^{i-1}(1/(i-j)^2)\mathbf{Pr}[|\xi + z| < Aj\sigma + A\sigma[-4,5])$ , we get
$$
\begin{array}{l} P (i, z) <   O (1) \Pr [ | \xi + z | \in A \sigma (i - 5, i + 5) ] \\ + O (1) \sum_ {j = 1} ^ {i - 1} \left(1 / (i - j) ^ {2}\right) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ]. \\ \end{array}
$$
We now let  $\tilde{P}(i,z)$  denote the final upper bound on  $P(i,z)$ . The value of having  $\tilde{P}(i,z)$  is that it can be computed using samples from  $\xi + z$ .
$$
\begin{array}{l} \tilde {P} (i, z) := O (1) \mathbf {P r} [ | \xi + z | \in A \sigma (i - 5, i + 5) ] \\ + O (1) \sum_ {j = 1} ^ {i - 1} \left(1 / (i - j) ^ {2}\right) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ]. \\ \end{array}
$$
We defined  $\tilde{P}(i) = \tilde{P}(i,z) + \tilde{P}(i,z')$ .
Proof of Item 2:
First observe that  $\sum_{i=0}^{\infty} \tilde{P}(i) < C$  for some constant  $C$ . It is clear that this is true of the first term, since every interval will get over-counted at most 10 times. To see that the second term can be bounded, observe that
$$
\begin{array}{l} \sum_ {i = 1} ^ {\infty} \sum_ {j = 1} ^ {i - 1} (1 / (i - j) ^ {2}) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \\ <   \sum_ {j = 1} ^ {\infty} \sum_ {i = 1} ^ {\infty} (1 / (i - j) ^ {2}) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \\ <   \sum_ {j = 1} ^ {\infty} \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \sum_ {i = 1} ^ {\infty} (1 / (i - j) ^ {2}) = O (1). \\ \end{array}
$$
The first inequality follows by extending the limits of summation.
The final inequality follows from the fact that the total probability is at most 1, every interval of size  $\sigma$  gets over-counted at most finitely many times, and the fact that  $\sum_{k=1}^{\infty} 1 / k^2 = O(1)$ .
Item 2 now follows from the fact that  $\tilde{P}(i)$ ,  $i \geq 0$  is a non-negative sequence that sums to a finite quantity, and  $\tilde{P}(0) \geq \alpha/2$ , since the interval  $\tilde{P}(0)$  upper bounds is contains at least a constant fraction of the mass of  $\xi$  at 0 that is moved by  $z, z'$ , and  $\mathbf{Pr}[\xi = 0] \geq \alpha$ .
Applying Lemma B.2, we get our result.
Proof of Item 3:
Let  $k$  be such that Item 2 holds, i.e.  $k\tilde{P}(k) < \eta \sum_{j=1}^{k} \tilde{P}(k)$ , then the goal is to show  $\sum_{j=1}^{k} \tilde{P}(k) = O(\mathbf{Pr}[|\xi + z| < A\sigma k] + \mathbf{Pr}[|\xi + z'| < A\sigma k])$ .
We first consider the sum over  $i$ , of  $\tilde{P}(i, z)$ . It is easy to see that this is
$$
\begin{array}{l} \sum_ {i = 1} ^ {k} \tilde {P} (i, z) = O (1) \Pr [ | \xi + z | \leq A \sigma (k + 5) ] \\ + O (1) \sum_ {i = 1} ^ {k} \sum_ {j = 1} ^ {i - 1} \left(1 / (i - j) ^ {2}\right) \Pr \left[ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \right]. \\ \end{array}
$$
The first term on the RHS is almost what we want. We now show how to bound the second term,
$$
\begin{array}{l} \sum_ {i = 1} ^ {k} \sum_ {j = 1} ^ {i - 1} (1 / (i - j) ^ {2}) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \\ <   \sum_ {j = 1} ^ {k - 1} \sum_ {i = 0; i \neq j} ^ {k} (1 / (i - j) ^ {2}) \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \\ = \sum_ {j = 1} ^ {k - 1} \mathbf {P r} [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \sum_ {i = 0; i \neq j} ^ {k} (1 / (i - j) ^ {2}) \\ <   O (1) \sum_ {j = 1} ^ {k - 1} \Pr [ | \xi + z | \in A j \sigma + A \sigma [ - 4, 5) ] \\ <   O (1) \Pr [ | \xi + z | <   A (k + 5) \sigma ]. \\ \end{array}
$$
The first inequality holds since any pair of  $(i,j)$  that has a nonzero term in the first sum will also occur in the second sum, and all terms are non-negative.
The second equality is just pulling the common  $j$  term out.
The third inequality follows from the fact that  $\sum_{i=1}^{\infty} 1 / i^{2} = O(1)$ .
The fourth inequality follows from the fact that each  $\sigma$ -length interval is overcounted at most a constant number of times.
This allows us to bound  $\sum_{i=1}^{k} \tilde{P}(i, z)$  by  $O(\mathbf{Pr}[|\xi + z| < A\sigma(k + 5)])$  overall. Similarly for  $\sum_{i=1}^{k} \tilde{P}(i, z')$ , we can obtain a bound of  $O(\mathbf{Pr}[|\xi + z'| < A\sigma(k + 5)])$ . Putting these together, we see  $\sum_{i=1}^{k} \tilde{P}(i) \leq O(\mathbf{Pr}[|\xi + z| < A\sigma(k + 5)] + \mathbf{Pr}[|\xi + z'| < A\sigma(k + 5)])$ . Finally, to get the upper bound claimed in Item 3, observe that
$$
\begin{array}{l} \Pr [ | \xi + z | <   A \sigma (k + 5) ] + \Pr [ | \xi + z ^ {\prime} | <   A \sigma (k + 5) ] \\ = \Pr [ | \xi + z | <   A \sigma (k - 4) ] + \Pr [ | \xi + z ^ {\prime} | <   A \sigma (k - 4) ] \\ + \Pr [ | \xi + z | \in A \sigma (k - 4, k + 5) ] + \Pr [ | \xi + z ^ {\prime} | \in A \sigma (k - 4, k + 5) ] \\ \leq \Pr [ | \xi + z | <   A \sigma (k - 4) ] + \Pr [ | \xi + z ^ {\prime} | <   A \sigma (k - 4) ] \\ + \tilde {P} (k) \\ \leq \Pr [ | \xi + z | <   A \sigma (k - 4) ] + \Pr [ | \xi + z ^ {\prime} | <   A \sigma (k - 4) ] \\ + O (\eta / k) \left(\Pr \left[ | \xi + z | <   A \sigma (k + 5) \right] + \Pr \left[ | \xi + z ^ {\prime} | <   A \sigma (k + 5) \right]\right). \\ \end{array}
$$
Rearranging the inequality and by scaling  $\eta$  such that  $O(\eta /k)\leq 1 / 2$ , we see that  $\mathbf{Pr}[\| \xi +z| < A\sigma (k + 5)] + \mathbf{Pr}[\| \xi +z' | < A\sigma (k + 5)] = O(\mathbf{Pr}[\| \xi +z| < A\sigma (k - 4)] + \mathbf{Pr}[\| \xi +z' | < A\sigma (k - 4)]) = O(\mathbf{Pr}[\| \xi +z| < A\sigma k] + \mathbf{Pr}[\| \xi +z' | < A\sigma k])$ , completing our proof of Item 3.
Proof of Item 4:
Finally, to see Item 4 holds, observe that  $0 < \tilde{P}(i) < O(1)$ . Let  $B = (1/\eta)(O(1)/\alpha)^{1/\eta}$  denote the maximum index before which we can find a  $k$  such that  $k\tilde{P}(k) \leq \eta \sum_{i=0}^{k} \tilde{P}(i)$ . Now, To estimate  $\tilde{P}(i)$  empirically, we partition the interval  $(-BA\sigma, BA\sigma)$  into  $B$  intervals of length  $A\sigma$  each, and estimate the probability of  $\xi + z$  falling in each interval. If we estimate each of these probabilities to an accuracy of  $\eta/(100B)$ , we can estimate  $\tilde{P}(i)$  to an accuracy of  $O(\eta/i)$ .
An application of Hoeffding's inequality (Lemma A.1) tells us that each estimate will require  $O(B^2 / \eta^2 \log(1/\delta))$  samples. Taking a union bound over all these intervals, we see that we will require  $O(B^2 / \eta^2 \log(B/\delta))$  samples.
Finally, another union bound over each  $i \in [0, B]$  implies that we will need  $O(B^2 / \eta^2 \log (B^2 / \delta))$  samples. Substituting the value of  $B$  back in, we see that this amounts to requiring  $(1 / \eta^5)(O(1) / \alpha)^{2 / \eta} \log (1 / \eta \alpha \delta)$  samples.
Estimating  $\tilde{P}(i)$  will take time polynomial in the number of samples, and so we take time  $\mathrm{poly}((O(1) / \alpha)^{1/\eta}, 1/\eta, \log(1/\eta\alpha\delta))$ .
C.3 High-dimensional Noisy Location Estimation
In this section, we explain how to use our one-dimensional location estimation algorithm to get an algorithm for noisy location estimation in  $d$  dimensions.
The algorithm performs one-dimensional location estimation coordinate-wise, after a random rotation.
We need to perform such a rotation to ensure that every coordinate has a known variance bound of  $\sigma / \sqrt{d}$ .
Algorithm 5 High-dimensional Location Estimation: ShiftHighD(S1, S2, η, σ, α)
input: Sample sets  $S_{1}, S_{2} \subset \mathbb{R}^{d}$  of size  $m$ ,  $\eta \in (0,1)$ ,  $\sigma > 0$ ,  $\alpha$
1. Sample  $R_{i,j}$  i.i.d. from the uniform distribution over  $\{\pm 1 / \sqrt{d}\}$  for  $i, j \in [d]$
2. Represent  $S_{1}$  and  $S_{2}$  in the basis given by the rows of  $R$ :  $r_{1},\ldots ,r_{d}$ .
3. for  $i\in [d]$  do
$$
\mid v _ {i} ^ {\prime} := \operatorname {S h i f t 1 D} \left(S _ {1} \cdot e _ {i}, S _ {2} \cdot e _ {i}, \eta , O \left(\sigma / \sqrt {d}\right), \alpha\right)
$$
4. Change the representation of  $v'$  back to the standard basis.
5. Probability Amplification: Repeat steps 1-4,  $T \coloneqq O(\log (1 / \delta))$  times to get  $C \coloneqq \{v_1',\dots ,v_T'\}$
6. Find a ball of radius  $O(\eta \sigma)$  centered at one of the  $v_{i}^{\prime}$  containing  $> 90\%$  of  $C$ . If such a vector exists, set  $v^{\prime}$  to be this vector. Otherwise set  $v^{\prime}$  to be an arbitrary element of  $C$ .
5. Return  $v'$ .
Lemma C.5 (Location Estimation). Let  $D_{y_i}$  for  $i \in \{1,2\}$  be the distributions of  $\xi + z_i$  for  $i \in \{1,2\}$  where  $\xi$  is drawn from  $D_{\xi}$  and  $\mathbf{Pr}_{\xi \sim D_{\xi}}[\xi = 0] \geq \alpha$  and  $z_i \sim D_i$  are distributions over  $\mathbb{R}^d$  satisfying  $\mathbf{E}_{D_i}[x] = 0$  and  $\mathbf{E}_{D_i}[\| x\|^2] \leq \sigma^2$ . Let  $v \in \mathbb{R}^d$  be an unknown shift, and  $D_{y_2,v}$  denote the distribution of  $y_2$  shifted by  $v$ . There is an algorithm (Algorithm 5), which draws  $\mathrm{poly}(1/\eta, (O(1)/\alpha)^{1/\eta}, \log(1/\delta\eta\alpha))$  samples each from  $D_{y_1}$  and  $D_{y_2,v}$  runs in time  $\mathrm{poly}(d,1/\eta, (O(1)/\alpha)^{1/\eta}, \log(1/\delta\eta\alpha))$  and returns  $v'$  satisfying  $\| v' - v \| \leq O(\eta\sigma)$  with probability  $1 - \delta$ .
Proof. Consider a matrix  $R$  whose entries  $R_{i,j}$  are independently drawn from the uniform distribution over  $\pm 1 / \sqrt{d}$ . and whose diagonals are  $1 / \sqrt{d}$ .
Our goal is to show that with probability at least  $99\%$ , the standard deviation of each coordinate of  $Rz$  is bounded by  $O(\sigma / \sqrt{d})$ , i.e., the standard deviation of  $Rz \cdot e_i$  is at most  $O(\sigma / \sqrt{d})$  for all integer  $i$  in  $[d]$ .
We can then amplify this probability to ensure that the algorithm fails with a probability that is exponentially small.
To see this, observe that  $Rz\cdot e_i = r_i\cdot z$  , and so  $\mathbf{E}_z[r_i\cdot z] = 0$
$$
\begin{array}{l} \mathbf {E} _ {z} [ (r _ {i} \cdot z) ^ {2} ] = \sum_ {p \in [ d ], q \in [ d ]} R _ {i, p} R _ {i, q} \mathbf {E} [ z _ {p} z _ {q} ] \\ = \sum_ {i = 1} ^ {d} \mathbf {E} \left[ z _ {i} ^ {2} \right] / d + 2 \sum_ {p, q \in [ d ], p <   q} R _ {i, p} R _ {i, q} \mathbf {E} \left[ z _ {p} z _ {q} \right] \\ \leq (\sigma^ {2} / d) + 2 \sum_ {p, q \in [ d ], p <   q} R _ {i, p} R _ {i, q} {\bf E} [ z _ {p} z _ {q} ]. \\ \end{array}
$$
We now bound the second term with probability  $99\%$  via applying Chebyshev's inequality. Observe that  $\mathbf{E}[z_p z_q] \leq \sqrt{\mathbf{E}[z_p^2] \mathbf{E}[z_q^2]} \leq \sigma^2$ . Since  $R_{i,p}$  and  $R_{i,q}$  are drawn independently and  $p \neq q$ , we see that the variables  $R_{i,p} R_{i,q}$  and  $R_{i,l} R_{i,m}$  pairwise independent for pairs  $(p, q) \neq (l, m)$ , this implies  $\mathbf{Pr}[\left|\sum_{p,q \in [d], p < q} R_{i,p} R_{i,q} \mathbf{E}[z_p z_q]\right| > T] \leq \frac{O(\sigma^4)}{d d^2 T^2}$ . By choosing  $T = O(\sigma^2 / d)$ , we see that the right-hand side above is at most  $0.001 / d$ .
A union bound over all the coordinates then tells us that with probability  $99\%$ , the variance of each coordinate is at most  $O(\sigma^2 / d)$ .
Then, for each coordinate  $i$ , we can identify  $v_{i}^{\prime} = v_{i} \pm O(\eta \sigma / \sqrt{d})$  through an application of Lemma 3.1. Putting these together with probability at least  $99\%$ , we find  $v^{\prime}$  satisfying  $\| v^{\prime} - v \|^{2} \leq O(\eta^{2}\sigma^{2})$ .
Changing between these basis representations maintains the quality of our estimate since the new basis contains unit vectors nearly orthogonal to each other. With high probability, the inner products between these are around  $O(1 / \sqrt{d})$  for every pairwise comparison, so  $R$  approximates a random rotation.
Probability Amplification: The current guarantee ensures that we obtain a good candidate with a constant probability of success. However, for the final algorithmic guarantee, we need a higher probability of success. To achieve this, we modify the algorithm as follows:
1. Run the algorithm  $T$  times, each time returning a candidate  $v_{i}^{\prime}$  that is, with probability 99%, within  $O(\eta \sigma)$  distance from the true solution.
2. Construct a list of candidates  $C = \{v_1',\dots ,v_T'\}$
3. Identify a ball of radius  $O(\sigma \eta)$  centered at one of the  $v_{i}^{\prime}$  that contains at least  $90\%$  of the remaining points.
4. Return the corresponding  $v_{i}^{\prime}$  as the final output.
5. If no such  $v_{i}^{\prime}$  exists, return any vector from  $C$ .
Let  $E$  denote the event that a point is within  $O(\eta \sigma)$  to the true solution.
This will succeed with probability  $1 - \exp(-T)$ . To see why, observe the chance that we recover  $(2/3)T$  vectors outside the event  $E$  is less than  $(0.01)^{2/3T}\binom{T}{2/3T} < (0.047)^T\binom{T}{T/2} < (0.047)^T(2^T/\sqrt{T}) < (0.095)^T$ .
D List-Decodable Mean Estimation
This section presents an algorithm for list-decodable mean estimation when the inlier distribution follows  $\mathcal{D}_{\sigma}$ . Here,  $\mathcal{D}_{\sigma}$  represents a set of distributions over  $\mathbb{R}^d$  defined as  $\mathcal{D}_{\sigma} \coloneqq \{D \mid \mathbf{E}_D[|x - \mathbf{E}_D[x]|^2] \leq \sigma^2\}$ . In our setting, we receive samples from  $\xi + z$ , where  $\mathbf{Pr}[\xi = 0] > \alpha$ , where  $\alpha$  can be close to 0. Our objective is to estimate the mean with a high degree of precision.
Note that the guarantees provided by prior work do not directly apply to our setting. Prior work examines a more aggressive setting where arbitrary outliers are drawn with a probability of  $1 - \alpha$ . These outliers might not have the additive structure we have.
Recall the definition of an  $(\alpha, \beta, s)$ -LDME algorithm:
Definition D.1 (Algorithm for List-Decodable Mean Estimation). Algorithm  $\mathcal{A}$  is an  $(\alpha, \beta, s)$ -LDME algorithm for  $\mathcal{D}$  (a set of candidate inlier distributions) if with probability  $1 - \delta_{\mathcal{A}}$ , it returns a list  $\mathcal{L}$  of size  $s$  such that  $\min_{\hat{\mu} \in \mathcal{L}} \| \hat{\mu} - \mathbf{E}_{x \sim D}[x] \| \leq \beta$  for  $D \in \mathcal{D}$  when given  $m_{\mathcal{A}}$  samples of the kind  $z + \xi$  for  $z \sim D$  and  $\mathbf{Pr}[\xi = 0] \geq \alpha$ . If  $1 - \alpha$  is a sufficiently small constant less than  $1/2$ , then  $s = 1$ .
We now prove Fact 2.1 which we restate below for convenience.
Fact D.2 (List-decoding algorithm). There is an  $(\alpha, \eta \sigma, \tilde{O}((1 / \alpha)^{2 / \eta^2}))$ -LDME algorithm for the inlier distribution belonging to  $\mathcal{D}_{\sigma}$  which runs in time  $\tilde{O}(d(1 / \alpha)^{2 / \eta^2})$  and succeeds with probability  $1 - \delta$ . Conversely, any algorithm which returns a list, one of which makes an error of at most  $O(\eta \sigma)$  in  $\ell_2$  norm to the true mean, must have a list whose size grows exponentially in  $1 / \eta$ .
Proof. Algorithm: Consider the following algorithm:
1. If  $\alpha < c$  and  $\eta > \sqrt{\alpha}$ : Run any stability-based robust mean estimation algorithm from Diakonikolas et al. [2020b] and return a singleton list containing the output of the algorithm.
2. Otherwise, for integer each  $i \in [1, 100(1 / \alpha)^{2 / \eta^2} \log (1 / \delta)^2]$  sample  $1 / \eta^2$  samples and let their mean be  $\mu_i$ .
3. Return the list  $\mathcal{L} = \{\mu_i\mid i\in [1,100(1 / \alpha)^{2 / \eta^2}\log (1 / \delta)^2 ]\}$
If the algorithm returns in the first step, then the guarantees follow from the guarantees of the algorithm for robust mean estimation from Diakonikolas et al. [2020b] (Proposition 1.5 on page 4).
Otherwise, observe that the probability that every one of  $1 / \eta^2$  samples drawn is an inlier, is  $\alpha^{1 / \eta^2}$ .
Hence, with probability  $1 - \delta$  we see that if we draw  $1 / \eta^2$  samples  $O((1 / \alpha)^{2 / \eta^2}\log (1 / \delta)^2)$  times, there are at least  $O(\log (1 / \delta))$  sets of samples containing only inliers. Then, the mean of one of these concentrates to an error of  $O(\eta \sigma)$  by an application of Lemma A.2. More precisely, Lemma A.2 ensures that with probability  $99\%$ , the mean of a set of  $1 / \eta^2$  inliers concentrates up to an error of  $O(\eta \sigma)$ . Repeating this  $\log (1 / \delta)$  times, we get our result.