SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Using Fact D.1 with  $y_{1} = X, y_{2} = y, \mu_{1} = \mu_{2} = 0, \Sigma_{11} = \mathbf{I}_{d}, \Sigma_{12} = \beta, \Sigma_{21} = \beta^{\top}, \Sigma_{22} = \sigma_{y}^{2} = \sigma^{2} + \| \beta \|_{2}^{2}$ .
Fact D.1. If  $\begin{bmatrix} y_1 \\ y_2 \end{bmatrix} \sim \mathcal{N}\left(\begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}, \begin{bmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{bmatrix}\right)$ , then  $y_1|y_2 \sim \mathcal{N}(\bar{\mu}, \bar{\Sigma})$ , with  $\bar{\mu} = \mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(y_2 - \mu_2)$  and  $\Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$ .
For the distribution  $G_{I}$ , the claim follows by definition of conditional probability and law of total probability
$$
p _ {X | y \in I} (x) = \frac {\int_ {a ^ {\prime} \in I} p _ {X | y = a ^ {\prime}} (x) p _ {y} (a ^ {\prime}) \mathrm {d} a ^ {\prime}}{\mathbf {P r} [ y \in I ]},
$$
and then use the expression for the pdf of  $G_{a'}$  that we showed earlier.
D.1.1 Properties of Conditional Distribution
In Section 3, we mainly focused on the case where the underlying distribution is  $\mathcal{N}(\mu, \mathbf{I}_d)$ . For the purposes of our robust regression algorithm, we show in this section that the goodness conditions are satisfied by the distribution  $G_I$  and get as a corollary a robust mean estimator for  $\mu_{G_I}$ .
Lemma D.2 (Properties of  $G_I$ ). Let  $I = [a, b]$  be an interval of length  $\ell$ . Further suppose that  $\max(|a|, |b|) \leq 2\sigma_y$ . Let  $X \sim G_I$  and let  $\mu$  and  $\pmb{\Sigma}$  be the mean and covariance of  $X$ . Then:
1.  $\left\| \mu - a\beta / \sigma_y^2 \right\|_2 \leq \ell \| \beta \|_2 / \sigma_y^2$  and  $\| \mu \|_2 \leq 4 \| \beta \|_2 / \sigma_y$
2.  $\| \mathbf{\Sigma} - \mathbf{I}\|_{\mathrm{op}}\leq 9\| \beta \| _2^2 /\sigma_y^2$
3.  $\| X - \mu \|_{\psi_2} \lesssim 1 + \| \beta \|_2 / \sigma_y$ .
Proof. For any  $z \in I$ , we know that the mean of  $G_{z}$  is equal to  $z\beta/\sigma_{y}^{2}$  by Claim 4.1. Since  $G_{I}$  is a convex combination of  $G_{z}$  for  $z \in I$ , it follows that  $\mu$  is a convex combination of  $a\beta/\sigma_{y}^{2}$  and  $b\beta/\sigma_{y}^{2}$ . Therefore, the mean  $\mu$  satisfies  $\| \mu - a\beta/\sigma_{y}^{2} \| \leq |b - a||\beta|_{2}/\sigma_{y}^{2}$ .
Similarly, for any  $z \in I$ ,  $\mathbf{E}_{W \sim G_z}[WW^\top] = \mathbf{I}_d - \frac{1}{\sigma_y^2} \beta \beta^\top + \frac{z^2}{\sigma_y^4} \beta \beta^\top$  by Claim 4.1. Thus the second moment of  $X \sim G_I$  is equal to  $\mathbf{E}_{X \sim G_I}[XX^\top] = \mathbf{I}_d - \frac{1}{\sigma_y^2} \beta \beta^\top + \frac{\alpha a^2 + (1 - \alpha) b^2}{\sigma_y^4} \beta \beta^\top$  for  $\alpha \in [0,1]$ . Thus, we have that
$$
\left\| \boldsymbol {\Sigma} - \mathbf {I} _ {d} \right\| _ {\mathrm {o p}} = \left\| \mathbf {E} _ {X \sim G _ {I}} [ X X ^ {\top} ] - \mu \mu^ {\top} - \mathbf {I} _ {d} \right\| _ {\mathrm {o p}}
$$
$$
\begin{array}{l} \leq \| \mu \| _ {2} ^ {2} + \frac {1}{\sigma_ {y} ^ {2}} \| \beta \| _ {2} ^ {2} + \max  (a ^ {2}, b ^ {2}) \| \beta \| _ {2} ^ {2} / \sigma_ {y} ^ {4} \\ \leq 2 \max (a ^ {2}, b ^ {2}) \| \beta \| _ {2} ^ {2} / \sigma_ {y} ^ {4} + \frac {\| \beta \| _ {2} ^ {2}}{\sigma_ {y} ^ {2}} \leq 9 \| \beta \| _ {2} ^ {2} / \sigma_ {y} ^ {2}. \\ \end{array}
$$
The proof of the last claim is based on the fact that each component of the mixture  $G_{I}$  is Gaussian (and thus sub-Gaussian). For a single one-dimensional Gaussian  $Y \sim \mathcal{N}(0, \sigma^2)$ , the sub-Gaussian norm is constant, in particular  $\| Y \|_{\psi_2} = \sqrt{2 / \pi}$ .
$$
\begin{array}{l} \mathbf {E} _ {X \sim G _ {I}} \left[ | v ^ {\top} (x - \mu_ {G _ {I}}) | ^ {p} \right] ^ {1 / p} = \left(\mathbf {E} _ {a \sim I} \mathbf {E} _ {X \sim G _ {a}} \left[ | v ^ {\top} (x - \mu_ {G _ {I}}) | ^ {p} \right]\right) ^ {1 / p} \\ = \max  _ {a \in I} \mathbf {E} _ {X \sim G _ {a}} \left[ | v ^ {\top} (x - \mu_ {G _ {I}}) | ^ {p} \right] ^ {1 / p} \\ \leq \max  _ {a \in I} \mathbf {E} _ {X \sim G _ {a}} \left[ | v ^ {\top} (x - \mu_ {G _ {a}}) | ^ {p} \right] ^ {1 / p} + \| \mu_ {G _ {a}} - \mu_ {G _ {I}} \| _ {2} \\ \leq \sqrt {2 p / \pi} + \ell \| \beta \| _ {2} / \sigma_ {y} ^ {2}. \\ \end{array}
$$
Hence,  $\left\| v^{\top}(X - \mu_{G_I})\right\|_{\psi_2} \leq \sqrt{2 / \pi} + \ell \left\| \beta \right\|_2 / \sigma_y^2$  for every direction  $v \in S^{d-1}$ .
D.2 Robustness of Goodness Conditions to Spectral Noise
We now show that a large set of i.i.d. samples from the conditional distribution  $G_{I}$  satisfies the goodness condition provided that the interval is close to the origin and the norm of  $\beta$  is small.
Lemma D.3. Let  $I = [a,b]$  be an interval of length  $\ell$ . Let  $\epsilon_0$  be a small enough constant. Let  $\epsilon \in (0,\epsilon_0)$ . Further suppose that  $\max(|a|, |b|) \leq 2\sigma_y$ ,  $\|\beta\|_2 \lesssim \sigma \epsilon \log(1/\epsilon)$ , and  $\alpha \geq \epsilon^2$ .
Let  $S$  be a set of  $n$  i.i.d. points from the distribution  $G_{I}$ . If  $n \gg \frac{k^{3}(d\log(k / \epsilon) + \log(1 / \tau))}{\min(\alpha^{2},\epsilon^{2} / \log^{2}(1 / \epsilon))}$  then with probability  $1 - \tau$ , the uniform distribution on  $S$  satisfies  $(\epsilon, \alpha, k)$ -goodness condition (Definition 2.1) with respect to  $\mu_{G_I}$ .
Proof. We show this for each one of the conditions separately.
Condition (1) We first calculate the true probability of  $v^{\top}(X - \mu_{G_I})$  being larger than  $\gamma$ . The distribution of  $v^{\top}(X - \mu_{G_a})$  is  $\mathcal{N}(0, \tilde{\sigma}_v)$  where  $\tilde{\sigma}_v \coloneqq 1 - (v^{\top}\beta)^2 / \sigma_y^2$ . Using Taylor approximation for the cdf of the projected points,
$$
\operatorname * {P r} _ {X \sim G _ {a}} \left[ v ^ {\top} \left(X - \mu_ {G _ {a}}\right) > \tilde {\sigma} _ {v} t \right] \leq \frac {1}{2} - \frac {t}{\sqrt {2 \pi}} + \frac {t ^ {3}}{2 \sqrt {2 \pi}}.
$$
$$
\begin{array}{l} \max  _ {v \in \mathcal {S} ^ {d - 1}} \Pr_ {X \sim G _ {I}} \left[ v ^ {\top} \left(X - \mu_ {G _ {I}}\right) > t \tilde {\sigma} _ {v} + \max  _ {a \in I} \| \mu_ {G _ {a}} - \mu_ {G _ {I}} \| _ {2} \right] \\ \leq \max  _ {a \in I} \mathbf {P r} _ {X \sim G _ {a}} \left[ v ^ {\top} \left(X - \mu_ {G _ {I}}\right) > t \tilde {\sigma} + \max  _ {a \in I} \| \mu_ {G _ {a}} - \mu_ {G _ {I}} \| _ {2} \right] \\ \leq \max  _ {a \in I} \mathbf {P r} _ {X \sim G _ {a}} \left[ v ^ {\top} \left(X - \mu_ {G _ {a}}\right) > t \tilde {\sigma} \right] \\ \leq \frac {1}{2} - \frac {t}{\sqrt {2 \pi}} + \frac {t ^ {3}}{2 \sqrt {2 \pi}}. \\ \end{array}
$$
Applying VC inequality as in Lemma B.20, we obtain that the empirical probability over a set of size  $n \gtrsim (d + \log(1 / \tau)) / \epsilon^2$  samples will satisfy that with probability  $1 - \tau$ ,
$$
\max  _ {v \in \mathcal {S} ^ {d - 1}} \mathbf {P r} _ {X \sim S} \left[ v ^ {\top} \left(X - \mu_ {G _ {I}}\right) > t \max  _ {v \in \mathcal {S} ^ {d - 1}} \tilde {\sigma} _ {v} + \max  _ {a \in I} \| \mu_ {G _ {a}} - \mu_ {G _ {I}} \| \right] \leq \frac {1}{2} - \frac {t}{\sqrt {2 \pi}} + \frac {t ^ {3}}{2 \sqrt {2 \pi}} + \epsilon .
$$
The expression on the right is less than  $1 / 2$  for  $t\gtrsim \epsilon$ . Thus, we obtain that Condition (1) is satisfied with  $\gamma_{1}\lesssim \epsilon +\max_{a\in I}\| \mu_{G_{a}} - \mu_{G_{I}}\|_{2}\lesssim \epsilon +\ell \beta /\sigma_{y}^{2}$ .
Conditions (2.a) and (2.b) The proof of these two conditions for the Gaussian case only uses its sub-Gaussian concentration. From Lemma D.2, we have that  $\| X - \mu_{G_I}\|_{\psi_2} \leq \sqrt{2 / \pi} + \ell \| \beta \|_2 / \sigma_y^2$  which is less than a constant when  $\ell \lesssim \sigma$  and  $\| \beta \|_2 \lesssim \sigma$ .
Hence, identically to Lemma B.21, we have that if  $n \gg (d + \log(1/\tau)) / \alpha^2$ , for any set  $S' \subset S$  with  $|S'| \geq (1 - \alpha)|S|$ , the sample mean  $\mu_{S'}$  and the sample second moment  $\overline{\Sigma}_{S'}$  satisfy that  $\| \mu_{S'} - \mu_{G_I} \|_2 \lesssim \alpha \sqrt{\log(1 / \alpha)}$  and  $\| \overline{\Sigma}_{S'} - \Sigma_{G_I} \|_{\mathrm{op}} \lesssim \alpha \log(1 / \alpha)$ .
Using the triangle inequality, we have that  $\| \overline{\mathbf{\Sigma}}_{S'} - \mathbf{I}\|_{\mathrm{op}} \leq \| \overline{\mathbf{\Sigma}}_{S'} - \mathbf{I}\|_{\mathrm{op}} + \| \overline{\mathbf{\Sigma}}_{S'} - \mathbf{\Sigma}_{G_I}\|_{\mathrm{op}} \lesssim \alpha \log (1 / \alpha) + \epsilon^2 \log^2 (1 / \epsilon) \lesssim \alpha \log (1 / \alpha)$ , where we used Lemma D.2 and the assumptions on  $\| \beta \|_2$  and  $\alpha$ . Thus, Conditions (2.a) and (2.b) are satisfied.
Condition (2.c) We will follow the same notation as in Lemma B.20 and claim 4.1. Observe that the proof of Lemma B.20 relied on Equation (4). Our goal will be to show that a similar right tail concentration inequality is satisfied for the distribution  $G_{I}$ . Since  $G_{I}$  is a convex combination of  $G_{z}$  for  $z \in [a, b]$ , it suffices to show that each  $G_{z}$  satisfies Equation (4) after small changes.
Let  $X \sim G_{z}$ . Then  $x = \Sigma_{z}^{1/2}y + \mu_{z}$  for  $y \sim \mathcal{N}(0, \mathbf{I})$  and  $\mu_{z} = z\beta/\sigma_{y}^{2}$  and  $\Sigma_{z} = \mathbf{I} - \beta\beta^{\top}/\sigma_{y}^{2}$ . We will show that the polynomial  $p_{\mathbf{U}}(x) = \|\mathbf{U}(x - \mu_{I})\|_{2}^{2}$  satisfies similar right tail concentration inequality as  $\|\mathbf{U}y\|_{2}^{2}$  in Lemma B.20. Applying triangle inequality,
$$
\begin{array}{l} p _ {\mathbf {U}} (x) = \| \mathbf {U} (x - \mu_ {I}) \| _ {2} ^ {2} \leq 2 \| \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} y \| _ {2} ^ {2} + 2 \| \mathbf {U} (\mu_ {z} - \mu_ {I}) \| _ {2} ^ {2} \\ \leq 2 \| \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} y \| _ {2} ^ {2} + 2 \| \mathbf {U} \| _ {\mathrm {o p}} ^ {2} \| \mu_ {z} - \mu_ {I} \| _ {2} ^ {2}. \\ \end{array}
$$
By Lemma D.2,  $\| \mu_z - \mu_I\| _2^2$ , is at most  $\ell^2\| \beta \| _2^2 /\sigma_y^4\leq 1$  under the assumptions. Thus, we obtain the following deterministic expression:
$$
p _ {\mathbf {U}} (x) \leq 2 \| \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} y \| _ {2} ^ {2} + 2 \operatorname {t r} \left(\mathbf {U} ^ {\top} \mathbf {U}\right). \tag {33}
$$
The expectation of  $\| \mathbf{U}\pmb{\Sigma}_z^{1 / 2}y\| _2^2$  is  $\mathrm{tr}(\pmb{\Sigma}_z^{1 / 2}\mathbf{U}^\top \mathbf{U}\pmb{\Sigma}_z^{1 / 2}) = \mathrm{tr}(\mathbf{U}^\top \mathbf{U}\pmb{\Sigma}_z)$ , which is less than  $\mathrm{tr}(\mathbf{U}^\top \mathbf{U})$  since  $\pmb{\Sigma}\preceq \mathbf{I}$ . Combining this with Equation (33), we obtain
$$
\mathbb {P} \left(p _ {\mathbf {U}} (x) \geq 1 0 0 \operatorname {t r} \left(\mathbf {U} ^ {\top} \mathbf {U}\right)\right) \leq \mathbb {P} \left(\left\| \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} y \right\| _ {2} ^ {2} - \mathbf {E} \left[ \left\| \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} y \right\| _ {2} ^ {2} \right] \geq 4 8 \operatorname {t r} \left(\mathbf {U} ^ {\top} \mathbf {U}\right)\right). \tag {34}
$$
Finally, the right tail of  $\| \mathbf{U}\pmb{\Sigma}_z^{1 / 2}y\| _2^2 -\mathbf{E}[\| \mathbf{U}\pmb{\Sigma}_z^{1 / 2}y\| _2^2 ]$  depends on  $\| \pmb{\Sigma}_z^{1 / 2}\mathbf{U}^\top \mathbf{U}\pmb{\Sigma}_z^{1 / 2}\|_{\mathrm{F}}$  and  $\| \pmb{\Sigma}_z^{1 / 2}\mathbf{U}^\top \mathbf{U}\pmb{\Sigma}_z^{1 / 2}\|_{\mathrm{op}}$  by the Hanson Wright inequality. Both of these expressions are again bounded by  $\| \mathbf{U}^\top \mathbf{U}\|_{\mathrm{F}}$  and  $\| \mathbf{U}^\top \mathbf{U}\|_{\mathrm{op}}$ , respectively as follows:
$$
\| \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \mathbf {U} ^ {\top} \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \| _ {\mathrm {F}} \leq \| \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \| _ {\mathrm {o p}} \| \mathbf {U} ^ {\top} \mathbf {U} \| _ {\mathrm {F}} \| \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \| _ {\mathrm {o p}} \leq \| \mathbf {U} ^ {\top} \mathbf {U} \| _ {\mathrm {F}}
$$
$$
\left\| \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \mathbf {U} ^ {\top} \mathbf {U} \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \right\| _ {\mathrm {o p}} \leq \left\| \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \right\| _ {\mathrm {o p}} \left\| \mathbf {U} ^ {\top} \mathbf {U} \right\| _ {\mathrm {o p}} \left\| \boldsymbol {\Sigma} _ {z} ^ {1 / 2} \right\| _ {\mathrm {o p}} \leq \left\| \mathbf {U} ^ {\top} \mathbf {U} \right\| _ {\mathrm {o p}},
$$
where we use  $\| \Sigma_z\|_{\mathrm{op}}\leq 1$  and Fact B.9. Hence, applying Hanson-Wright inequality to  $\| \mathbf{U}\pmb{\Sigma}_z^{1 / 2}y\| _2^2$  along with the above relations, Equation (34) implies that  $p_{\mathbf{U}}(x)$  satisfies Equation (4) with  $(\epsilon /k)^5$  in the right hand side. The rest of the proof goes through similar to Lemma B.20.
Thus, Theorem 3.1 is applicable and we obtain the following algorithm.
Corollary D.4. Let  $\epsilon_0$  be a small enough constant. Let  $\epsilon \in (0,\epsilon_0),c\in (0,1),\delta \in (0,1)$ . Let  $S\in \mathbb{R}^d$  be an  $\epsilon$ -corrupted set of  $n\gg \frac{d + \log(1 / \tau)}{\epsilon^2}$  polylog(d/ε) points from the distribution  $G_{I}$  of Claim 4.1 with  $\ell \leq \sigma /\log (1 / \epsilon)$  and  $\| \beta \| _2\leq \sigma \epsilon \log (1 / \epsilon)$ . Then, the Algorithm 5 produces an estimate  $\widehat{\mu}$  such that  $\| \widehat{\mu} -\mu_{G_I}\| _2\leq \epsilon$  with probability at least  $1 - \tau$  in time  $(nd + 1 / \epsilon^{2 + c})\mathrm{polylog}(n,d,1 / \epsilon ,1 / \tau)$ .
D.3 Proof of Theorem 1.4
We now provide the details missing from Section 4. The proof sketch there relied heavily on the events defined in Equation (1), restated below:
(iii)  $\| \widehat{\beta}_I - \mu_{G_I}\| _2\lesssim \epsilon /c.$  (35)
The details missing from the proof sketch in Section 4 are: (i) the events in Equation (35) hold, the events in Equation (35) suffice for the algorithm's output to be correct, and the proof of Lemma 4.2. We already explained why (i) and (ii) hold. The third holds by Corollary D.4.
We begin by formally showing that if the events in Equation (1) hold, then the algorithm's output is correct.
Claim D.5. Consider the setting and notation of Theorem 1.4 and Section 4. Let  $c \in (0,1)$ . Assuming that the following three events hold
(i)  $|\widehat{\sigma}_y^2 -\sigma_y^2 |\lesssim \sigma_y^2\epsilon \log (1 / \epsilon).$
(ii)  $|a| > 0.0001\hat{\sigma}_{y}$ .
(iii)  $\| \widehat{\beta}_I - \mu_{G_I}\| _2\lesssim \epsilon /c.$
Then,  $\| \widehat{\beta} -\beta \| _2\lesssim \sigma \epsilon /c$
Proof. We want to show that given Items (i) to (iii), we have that  $\| \widehat{\beta} -\beta \| _2\lesssim \sigma \epsilon /c$ , where  $\widehat{\beta} := (\widehat{\sigma}_y^2 /a)\widehat{\beta}_I$  is the algorithm's output vector. Using the triangle inequality,
$$
\left\| \widehat {\beta} - \beta \right\| _ {2} = \left\| \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \widehat {\beta} _ {I} - \beta \right\| _ {2} \leq \left\| \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \widehat {\beta} _ {I} - \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \mu_ {G _ {I}} \right\| _ {2} + \left\| \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \mu_ {G _ {I}} - \beta \right\| _ {2}. \tag {36}
$$
We upper bound each term by  $O(\sigma \epsilon / c)$  separately. For the first term we have that
$$
\begin{array}{l} \left\| \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \widehat {\beta} _ {I} - \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \mu_ {G _ {I}} \right\| _ {2} = \frac {\widehat {\sigma} _ {y} ^ {2}}{| a |} \left\| \widehat {\beta} _ {I} - \mu_ {G _ {I}} \right\| _ {2} \\ \lesssim \widehat {\sigma} _ {y} \left\| \widehat {\beta} _ {I} - \mu_ {G I} \right\| _ {2} \quad (\text {b y}) \\ \lesssim \widehat {\sigma} _ {y} \epsilon / c \quad (\text {b y}) \\ \lesssim \sigma_ {y} \epsilon / c \quad (\text {b y I t e m (i) a n d} \epsilon <   1 / 2) \\ \leq \left(\sigma + \| \beta \| _ {2}\right) \epsilon / c \quad \left(\sigma_ {y} ^ {2} := \sigma^ {2} + \| \beta \| _ {2} ^ {2}\right) \\ \lesssim (\sigma + \sigma \epsilon \log (1 / \epsilon)) \epsilon / c \quad \text {(b y a s s u m p t i o n} \| \beta \| _ {2} \lesssim \sigma \epsilon \log (1 / \epsilon)) \\ \lesssim \sigma \epsilon / c. \quad (\epsilon <   1 / 2) \\ \end{array}
$$
For the second term of Equation (36), the triangle inequality implies that
$$
\left\| \frac {\widehat {\sigma} _ {y} ^ {2}}{a} \mu_ {G _ {I}} - \beta \right\| _ {2} \leq \left\| \frac {\sigma_ {y} ^ {2}}{a} \mu_ {G _ {I}} - \beta \right\| _ {2} + \frac {| \widehat {\sigma} _ {y} ^ {2} - \sigma_ {y} ^ {2} |}{| a |} \| \mu_ {G _ {I}} \| _ {2}.
$$
We bound the first term above as follows:
$$
\begin{array}{l} \left\| \frac {\sigma_ {y} ^ {2}}{a} \mu_ {G _ {I}} - \beta \right\| _ {2} = \frac {\sigma_ {y} ^ {2}}{| a |} \left\| \mu_ {G _ {I}} - \frac {a}{\sigma_ {y} ^ {2}} \beta \right\| _ {2} \lesssim \sigma_ {y} \left\| \mu_ {G _ {I}} - \frac {a}{\sigma_ {y} ^ {2}} \beta \right\| _ {2} \tag {by Item (i)} \\ \leq \sigma_ {y} \frac {\ell \| \beta \| _ {2}}{\sigma_ {y} ^ {2}} \quad (\text {b y}) \\ \begin{array}{l} \leq \frac {\ell \| \beta \| _ {2}}{\sigma} \lesssim \sigma \epsilon . \\ \hskip 1 4. 2 2 6 3 7 8 p t (\text {b y}) \end{array} \\ \end{array}
$$
The second term can be controlled as follows:
$$
\begin{array}{l} \frac {\left| \hat {\sigma} _ {y} ^ {2} - \sigma_ {y} ^ {2} \right|}{\left| a \right|} \| \mu_ {G _ {I}} \| _ {2} \lesssim \sigma_ {y} \epsilon \log (1 / \epsilon) \| \mu_ {G _ {I}} \| _ {2} \tag {Items(i) and (ii)} \\ \lesssim \sigma_ {y} \epsilon \log (1 / \epsilon) \frac {(a + \ell) \| \beta \| _ {2}}{\sigma_ {y} ^ {2}} \quad (\text {b y}) \\ \end{array}
$$
$$
\begin{array}{l} \lesssim \frac {\sigma_ {y} \epsilon \log (1 / \epsilon) \widehat {\sigma} _ {y} \| \beta \| _ {2}}{\sigma_ {y} ^ {2}} \quad (a + \ell \lesssim \widehat {\sigma} _ {y} \text {b y d e f i n i t i o n o f} a, \ell) \\ \lesssim \epsilon \log (1 / \epsilon) \| \beta \| _ {2} \quad (\text {b y I t e m (i)}) \\ \lesssim \sigma \epsilon . \quad (\text {b y} \\ \end{array}
$$
Finally, we provide the proof of Lemma 4.2 below.
Proof. We start with the first claim.
$$
\begin{array}{l} \operatorname * {P r} _ {(X, y) \sim G} [ y \in I ] = \operatorname * {P r} _ {y \sim \mathcal {N} (0, \sigma_ {y} ^ {2})} [ y \in I ] \\ = \int_ {y \in \mathbb {R}} \mathbb {1} (y \in I) \phi (y; 0, \sigma_ {y} ^ {2}) d y \quad (\phi (y; 0, \sigma_ {y} ^ {2}) \text {d e n o t e s t h e p d f o f} \mathcal {N} (0, \sigma_ {y} ^ {2})) \\ \geq \int_ {y \in \left[ - 1. 1 \sigma_ {y}, 1. 1 \sigma_ {y} \right]} \mathbb {1} (y \in I) \phi (y; 0, \sigma_ {y} ^ {2}) d y \\ \geq \left(0. 2 / \sigma_ {y}\right) \int_ {y \in \left[ - 1. 1 \sigma_ {y}, 1. 1 \sigma_ {y} \right]} \mathbb {1} (y \in I) \mathrm {d} y \\ \geq 0. 2 \ell / \sigma_ {y}, \\ \end{array}
$$
where we the second line uses that  $I \subset [-1.1\sigma_y, 1.1\sigma_y]$  because its center belongs in  $[- \widehat{\sigma}_y, \widehat{\sigma}_y]$  and its length is  $\ell = \widehat{\sigma}_y / \log(1/\epsilon)$ , and  $\widehat{\sigma}_y$  is very close to  $\sigma_y$  (Item (i)), and  $\epsilon \ll 1$ . In the third line we used that the pdf of  $\mathcal{N}(0, \sigma_y^2)$  is pointwise bigger than a small multiple of  $1/\sigma_y$  in the range  $[-1.1\sigma_y^2, 1.1\sigma_y^2]$ .
We now prove the second claim. First,
$$
\begin{array}{l} \Pr_ {(X, y) \sim B} [ y \in I ] \leq \Pr_ {(X, y) \sim B} [ y \in I, y \in [ - \widehat {\sigma} _ {y} - \ell , \widehat {\sigma} _ {y} + \ell ] ] + \Pr_ {(X, y) \sim B} [ y \in I, y \notin [ - \widehat {\sigma} _ {y} - \ell , \widehat {\sigma} _ {y} + \ell ] ] (37) \\ \leq \Pr_ {(X, y) \sim B} [ y \in I, y \in \left[ - \widehat {\sigma} _ {y} - \ell , \widehat {\sigma} _ {y} + \ell \right] ] (38) \\ \leq \Pr_ {(X, y) \sim B} [ y \in I \mid y \in \left[ - \widehat {\sigma} _ {y} - \ell , \widehat {\sigma} _ {y} + \ell \right] ]. (39) \\ \end{array}
$$
Now let  $B'$  be the conditional distribution  $B$  given  $y \in [-\widehat{\sigma}_y - \ell, \widehat{\sigma}_y + \ell]$  for saving space. Recall that the interval  $I$  is itself a random interval. If we take expectation with respect to  $I$  of both sides of Equation (39) we have that
$$
\begin{array}{l} \underset {I, (X, y)} {\mathbf {P r}} \left[ y \in I \right] \leq \underset {I} {\mathbf {E}} \left[ \underset {(x, y) \sim B ^ {\prime}} {\mathbf {E}} [ \mathbb {1} (y \in I) ] \right] = \underset {(x, y) \sim B ^ {\prime}} {\mathbf {E}} \left[ \underset {I} {\mathbf {E}} [ \mathbb {1} (y \in I) ] \right] \\ \leq \mathbf {E} _ {(x, y) \sim B ^ {\prime}} \left[ \frac {\ell}{2 . 2 \widehat {\sigma} _ {y}} \right] \leq \frac {\ell}{2 . 2 \widehat {\sigma} _ {y}} \lesssim \frac {\ell}{\sigma_ {y}}. \\ \end{array}
$$
where we changed the order of expectations, and used that given a fixed point  $y \in [-\widehat{\sigma}_y - \ell, \widehat{\sigma}_y + \ell]$ , the probability that this  $y$  is being hit by our random interval  $I$  is at most the length  $\ell$  of  $I$  divided by the length of  $[- \widehat{\sigma}_y - \ell, \widehat{\sigma}_y + \ell]$  (which as we have shown before is subset of  $[-1.1\sigma_y, 1.1\sigma_y]$ . In the last inequality we relate  $\widehat{\sigma}_y$  and  $\sigma_y$  once more using Item (i).
E Adaptation of Algorithm 1 to the Streaming Setting
In this section, we give a brief sketch of how the robust mean estimation algorithm of Theorem 1.3 can be adapted to the streaming setting using the techniques of [DKPP22]. The adaptation follows closely [DKPP22] and we thus omit the details.
We recall the data access model for the streaming setting:
Definition E.1 (Single-Pass Streaming Model). Let  $S$  be a fixed set. In the one-pass streaming model, the elements of  $S$  are revealed one at a time to the algorithm, and the algorithm is allowed a single pass over these points.
The samples that form the set  $S$  are still samples from an  $\epsilon$ -contaminated version of  $\mathcal{N}(\mu, \mathbf{I})$  as before. The adaptation of our main result to that model is the following:
Theorem E.2 (Almost Linear-Time and Streaming Algorithm for Robust Mean Estimation). Let  $\epsilon_0$  be a sufficiently small positive constant. There is an algorithm that, given parameters  $\epsilon \in (0,\epsilon_0)$ ,  $c\in (0,1)$ ,  $\delta \in (0,1)$ , reads a stream of  $\mathrm{poly}(d / \epsilon)$  points from an  $\epsilon$ -corrupted version of  $\mathcal{N}(\mu ,\mathbf{I}_d)$  (cf. Definition 1.1), computes an estimate  $\widehat{\mu}$  such that  $\| \widehat{\mu} -\mu \| _2 = O(\epsilon /c)$  with probability at least  $1 - \delta$ . Moreover, the algorithm runs in time  $(nd + 1 / \epsilon^{2 + c})\mathrm{polylog}(d / \epsilon \delta)$ , and uses additional memory  $\tilde{O} (d + \mathrm{poly}(1 / \epsilon))$ .
We also restrict our attention to Algorithm 1, which is the first stage of the overall algorithm. This is because the second stage, Algorithm 4, is only utilized when the dimension has been reduced to just a polylog  $(d / \epsilon)$ , thus one can simulate the offline algorithm in the streaming setting by storing the whole dataset, which has size  $O(\mathrm{polylog}(d / \epsilon) / \epsilon^2)$  (by "size" we mean the number of scalar numbers that the dataset consists of).