SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left. \operatorname {t r} \left(P \left(\hat {\Sigma} - \Sigma\right) P Q\right) = \operatorname {t r} \left(\left(\hat {\Sigma} - \Sigma\right) P Q P\right) \leq 0. 1 \| \Sigma^ {1 / 2} P Q P \Sigma^ {1 / 2} \| _ {F}, \right.
$$
where the last step applies Condition (L.2) of Definition 2.1 with  $V = PQP$ . Finally, using the cyclic properties of the trace operator,
$$
\begin{array}{l} \left\| \Sigma^ {1 / 2} P Q P \Sigma^ {1 / 2} \right\| _ {\mathrm {F}} ^ {2} = \operatorname {t r} \left(\Sigma^ {1 / 2} P Q P \Sigma P Q P \Sigma^ {1 / 2}\right) = \operatorname {t r} (Q P \Sigma P Q P \Sigma P) \\ = \| Q P \Sigma P \| _ {\mathrm {F}} ^ {2} \leq \| Q \| _ {\mathrm {F}} ^ {2} \| P \Sigma P \| _ {\mathrm {o p}} ^ {2} \leq \| P \Sigma P \| _ {\mathrm {o p}} ^ {2}, \\ \end{array}
$$
where the first inequality uses Fact B.3 and the second inequality uses  $\| Q\|_{\mathrm{F}}\leq 1$ . Thus we have the following:
$$
\begin{array}{l} \left\| P \left(\hat {\Sigma} - \Sigma\right) P \right\| _ {F} = \sup  _ {Q: \| Q \| _ {F} \leq 1} \operatorname {t r} \left(P \left(\hat {\Sigma} - \Sigma\right) P Q\right) \\ = \sup  _ {\text {s y m m e t r i c} Q: \| Q \| _ {\mathrm {F}} \leq 1} \operatorname {t r} \left(P \left(\hat {\Sigma} - \Sigma\right) P Q\right) \\ \leq 0. 1 \| P \Sigma P \| _ {\mathrm {o p}}. \\ \end{array}
$$
where the second equality is due to the fact that  $P\left(\hat{\Sigma} - \Sigma\right)P$  is itself symmetric.
Proof. To simplify notation, let  $\widehat{\Sigma} \coloneqq \mathbf{Cov}_{X \sim A'}[X]$  and  $\overline{\Sigma} \coloneqq \mathbf{E}_{X \sim A'}[(X - \mu)(X - \mu)^\top]$ . Observe that  $\overline{\Sigma} = \widehat{\Sigma} + (\mu - \hat{\mu})(\mu - \hat{\mu})^\top$ . We apply triangle inequality and Corollary D.2 with to obtain the following:
$$
\begin{array}{l} \| P (\widehat {\Sigma} - \Sigma) P \| _ {F} \leq \| P (\overline {{\Sigma}} - \Sigma) P \| _ {F} + \| P (\mu - \hat {\mu}) (\mu - \hat {\mu}) ^ {\top} P \| _ {F} \\ = \| P (\bar {\Sigma} - \Sigma) P \| _ {F} + \| P (\mu - \hat {\mu}) \| _ {2} ^ {2} \\ \leq 0. 1 \| P \Sigma P \| _ {\mathrm {o p}} + 0. 1 \| P \Sigma P \| _ {\mathrm {o p}} \\ \leq 0. 2 \| P \Sigma P \| _ {\mathrm {o p}}, \tag {6} \\ \end{array}
$$
where the first inequality follows from the triangle inequality and the second inequality uses Corollary D.2.
We now analyze the desired expression by using the triangle inequality and the lemma assumption as follows:
$$
\| P \Sigma P - L \| _ {F} \leq \| P \widehat {\Sigma} P - L \| _ {F} + \| P (\widehat {\Sigma} - \Sigma) P \| _ {F} \leq r + 0. 2 \| P \Sigma P \| _ {\mathrm {o p}},
$$
where the first term was bounded by  $r$  by assumption and the second term uses the bound (6). Thus it suffices to show that  $\| P\Sigma P\|_{\mathrm{op}} = O(r + \| L\|_{\mathrm{op}})$ . To this end, we again use triangle inequality as follows:
$$
\begin{array}{l} \| P \Sigma P \| _ {\mathrm {o p}} \leq \| P \widehat {\Sigma} P - L \| _ {\mathrm {o p}} + \| L \| _ {\mathrm {o p}} + \| P (\widehat {\Sigma} - \Sigma) P \| _ {\mathrm {o p}} \\ \leq r + \| L \| _ {\mathrm {o p}} + 0. 2 \| P \Sigma P \| _ {\mathrm {o p}}, \tag {7} \\ \end{array}
$$
where the second inequality uses the bound (6). Rearranging the above display inequality, we obtain  $\| P\Sigma P\|_{\mathrm{op}} = O(r + \| L\|_{\mathrm{op}})$
E Analysis of a Single Recursive Call of the Algorithm: Proofs from Section 3
In this section, we present the details that were omitted from Section 3. We recall Assumption 3.1 from the main body.
-  $S = \{x_{i}\}_{i=1}^{n}$  is a set of  $n$  uncontaminated samples, which is assumed to be  $(\eta, 2\epsilon_{0})$ -stable with respect to the inlier distribution  $D$  having mean and covariance  $\mu, \Sigma$  (c.f. Definition 2.1). We assume  $\eta \leq 0.001$ ,  $\epsilon_{0} = 0.01$ , and  $\mathbf{Var}_{X \sim D}[X^{\top}AX] \leq C_{1}(\| \Sigma^{1/2}A\Sigma^{1/2}\|_{F}^{2} + \| \Sigma^{1/2}A\mu \|_{2}^{2})$  for all symmetric  $d \times d$  matrices  $A$  and a constant  $C_{1}$ .
-  $T$  is the input set to the current recursive call of the algorithm (after the adversarial corruptions), which satisfies  $|S \cap T| \geq (1 - 2\epsilon_0)|S|$  and  $|T| \leq (1 / \alpha)|S|$ .
- We denote  $H = \mathbf{E}_{X \sim T}\left[X X^{\top}\right]$ .
- We denote by  $\widetilde{S}, \widetilde{T}$  the versions of  $S$  and  $T$  normalized by  $H^{\dagger/2} \colon \widetilde{S} = \{H^{\dagger/2}x : x \in S\}$  and  $\widetilde{T} = \{H^{\dagger/2}x : x \in T\}$ . We use the notation  $\widetilde{x}$  for elements in  $\widetilde{S}$  and  $\widetilde{T}$ , and  $x$  for elements in  $S$  and  $T$ . Similarly, we use the notation  $\widetilde{X}$  for random variables with support in  $\widetilde{S}$  or  $\widetilde{T}$ .
- The mean and covariance of the inlier distribution  $D$  after transformation with  $H^{\dagger /2}$  are denoted by  $\widetilde{\mu} \coloneqq H^{\dagger /2}\mu$ ,  $\widetilde{\Sigma} \coloneqq H^{\dagger /2}\Sigma H^{\dagger /2}$ . We denote the empirical mean and covariance of the transformed inliers in  $T$  by  $\hat{\mu} \coloneqq \mathbf{E}_{\widetilde{X} \sim \widetilde{S} \cap \widetilde{T}}[\widetilde{X}]$  and  $\hat{\Sigma} \coloneqq \mathbf{Cov}_{\widetilde{X} \sim \widetilde{S} \cap \widetilde{T}}[\widetilde{X}]$ .
E.1 Normalization and Proof of Lemma 3.2
We start with a simple result stating that after normalization with  $H^{\dagger /2}$ , the mean and covariance of the inliers, both empirical and population-level, are bounded. This is intuitive since the inliers constitute  $\Omega (\alpha)$ -fraction of the overall samples and the second moment of the complete set after normalization is bounded by identity.
Lemma E.1 (Normalization). Make Assumption 3.1 and recall the notations  $\widetilde{\mu},\widetilde{\Sigma},\hat{\mu},\hat{\Sigma}$  that were defined in Assumption 3.1. We have that:
1.  $\| \hat{\Sigma}\|_{\mathrm{op}}\leq 2 / \alpha$
2.  $\| \hat{\mu}\| _2\leq \sqrt{2 / \alpha}.$
3.  $\| \widetilde{\Sigma}\|_{\mathrm{op}}\leq 3 / \alpha$
4.  $\| \widetilde{\mu}\| _2\leq \sqrt{3 / \alpha}$
5.  $\| \mathbf{E}_{\widetilde{X}\sim \widetilde{S}}[\widetilde{X} ]\| _2\leq 2 / \sqrt{\alpha}$  and  $\| \mathbf{Cov}_{\widetilde{X}\sim \widetilde{S}}[\widetilde{X} ]\|_{\mathrm{op}}\leq 4 / \alpha .$
6. For every matrix  $A$  with  $\| A\|_{\mathrm{F}}\leq 1$ ,  $|\mathbf{E}_{\widetilde{X}\sim \widetilde{S}}[\widetilde{X}^{\top}A\widetilde{X}] - \mathbf{E}_{X\sim D}[(H^{\dagger /2}X)^{\top}A(H^{\dagger /2}X)]| < 1 / \alpha$ .
Proof. We prove each part below:
Establishing Item 1 The transformation  $H$  is such that  $\mathbf{E}_{\widetilde{X} \sim \widetilde{T}}[\widetilde{X} \widetilde{X}^{\top}] = \mathbf{E}_{X \sim T}[H^{\dagger/2}XX^{\top}H^{\dagger/2}] = H^{\dagger/2}HH^{\dagger/2} = \mathbb{P}_H$ , which has operator norm at most 1. By assumption, we have that  $|\widetilde{S} \cap \widetilde{T}| \geq (1 - 2\epsilon_0)\alpha |T| \geq 0.5\alpha |T|$ . Thus, we obtain the following inequality:
$$
\frac {1}{\left| \widetilde {S} \cap \widetilde {T} \right|} \sum_ {\widetilde {x} \in \widetilde {S} \cap \widetilde {T}} \widetilde {x} \widetilde {x} ^ {\top} \preceq \frac {1}{\left| \widetilde {S} \cap \widetilde {T} \right|} \sum_ {\widetilde {x} \in \widetilde {T}} \widetilde {x} \widetilde {x} ^ {\top} = \frac {| \widetilde {T} |}{\left| \widetilde {S} \cap \widetilde {T} \right|} \mathbb {P} _ {H} \preceq \frac {2}{\alpha} \mathbb {P} _ {H}. \tag {8}
$$
By applying (8), we obtain
$$
\hat {\Sigma} = \frac {1}{\left| \widetilde {S} \cap \widetilde {T} \right|} \sum_ {\widetilde {x} \in \widetilde {S} \cap \widetilde {T}} (\widetilde {x} - \hat {\mu}) (\widetilde {x} - \hat {\mu}) ^ {\top} \preceq \frac {1}{\left| \widetilde {S} \cap \widetilde {T} \right|} \sum_ {\widetilde {x} \in \widetilde {S} \cap \widetilde {T}} \widetilde {x} \widetilde {x} ^ {\top} \preceq \frac {2}{\alpha} \mathbb {P} _ {H},
$$
implying that  $\| \hat{\Sigma}\|_{\mathrm{op}}\leq 2 / \alpha$  , since  $\mathbb{P}_H$  is just a projection matrix with  $0 / 1$  eigenvalues.
Establishing Item 2 Since, for any random vector  $X$ , we have that  $\mathbf{E}[X] \mathbf{E}[X]^{\top} \preceq \mathbf{E}[XX^{\top}]$  (which is equivalent to  $\mathbf{Cov}[X] \succeq 0$ ), we obtain
$$
\hat {\mu} \hat {\mu} ^ {\top} \preceq \frac {1}{| \widetilde {S} \cap \widetilde {T} |} \sum_ {\widetilde {x} \in \widetilde {S} \cap \widetilde {T}} \widetilde {x} \widetilde {x} ^ {\top} \preceq \frac {2}{\alpha} \mathbb {P} _ {H},
$$
where the last inequality uses (8). This implies that  $\| \hat{\mu}\| _2^2\leq 2 / \alpha$
Establishing Item 3 The goal is to bound the population-level covariance  $\|\widetilde{\Sigma}\|_{\mathrm{op}}$ . To this end, we will use the bounds from Items 1 and 2 which bounds their empirical versions and relate the empirical versions to the population ones via the deterministic conditions.
Consider an arbitrary subset  $\widetilde{S}_1$  of  $\widetilde{S}$  satisfying  $\left|\widetilde{S}_1\right| \geq \left|\widetilde{S}\right|(1 - 2\epsilon_0)$ . We first note that by Corollary D.2, we have that
$$
\left\| \widetilde {\Sigma} - \underset {\widetilde {X} \sim \widetilde {S} _ {1}} {\mathbf {E}} [ (\widetilde {X} - \widetilde {\mu}) (\widetilde {X} - \widetilde {\mu}) ^ {\top} ] \right\| _ {\mathrm {F}} \leq 0. 1 \left\| \widetilde {\Sigma} \right\| _ {\mathrm {o p}} \quad \text {a n d} \quad \left\| \widetilde {\mu} - \underset {\widetilde {X} \sim \widetilde {S} _ {1}} {\mathbf {E}} [ \widetilde {X} ] \right\| _ {2} \leq 0. 1 \sqrt {\left\| \widetilde {\Sigma} \right\| _ {\mathrm {o p}}} . \tag {9}
$$
Now define  $\widetilde{\Sigma}_1\coloneqq \mathbf{E}_{\widetilde{X}\sim \widetilde{S}\cap \widetilde{T}}[(\widetilde{X} -\widetilde{\mu})(\widetilde{X} -\widetilde{\mu})^\top ]$  , the centered second moment matrix of  $\widetilde{S}\cap \widetilde{T}$  which satisfies  $\widetilde{\Sigma}_1 = \hat{\Sigma} +(\widetilde{\mu} -\hat{\mu})(\widetilde{\mu} -\hat{\mu})^\top$  . We have that
$$
\begin{array}{l} \| \widetilde {\Sigma} \| _ {\mathrm {o p}} \leq \| \widetilde {\Sigma} - \hat {\Sigma} \| _ {\mathrm {F}} + \| \hat {\Sigma} \| _ {\mathrm {o p}} \quad (\text {t r i a n g l e i n e q u a l i t y}) \\ \leq \| \tilde {\Sigma} - \tilde {\Sigma} _ {1} \| _ {F} + \| \tilde {\mu} - \hat {\mu} \| _ {2} ^ {2} + \| \hat {\Sigma} \| _ {\mathrm {o p}} (\text {t r i a n g l e i n e q u a l i t y a n d} \tilde {\Sigma} _ {1} = \hat {\Sigma} + (\mu - \hat {\mu}) (\mu - \hat {\mu}) ^ {\top}) \\ \leq 0. 2 \| \widetilde {\Sigma} \| _ {\mathrm {o p}} + \| \widehat {\Sigma} \| _ {\mathrm {o p}}, \\ \end{array}
$$
where in the last line we use (9) for  $\widetilde{S}_1 = \widetilde{S} \cap \widetilde{T}$  and the fact that  $\left|\widetilde{S} \cap \widetilde{T}\right| \geq (1 - 2\epsilon_0)|\widetilde{S}|$ . Rearranging, we have that  $\|\widetilde{\Sigma}\|_{\mathrm{op}} < 1.25\|\hat{\Sigma}\|_{\mathrm{op}}$  for which we can use Item 1 to further upper bound it by  $3 / \alpha$ .
Establishing Item 4 We use a similar argument as in Item 3:
$$
\| \tilde {\mu} \| _ {2} \leq \| \hat {\mu} \| _ {2} + \| \tilde {\mu} - \hat {\mu} \| _ {2} \leq \| \hat {\mu} \| _ {2} + 0. 1 \sqrt {\| \tilde {\Sigma} \| _ {\mathrm {o p}}} \leq \sqrt {3 / \alpha}, \tag {10}
$$
where the first step uses the triangle inequality, the second inequality uses Equation (9) for  $\widetilde{S}_1 = \widetilde{S} \cap \widetilde{T}$ , and the last inequality uses Items 2 and 3.
Establishing Item 5 For the covariance condition, we have the following:
$$
\begin{array}{l} \left\| \mathbf {C o v} _ {\widetilde {X} \sim \widetilde {S}} [ \widetilde {X} ] \right\| _ {\mathrm {o p}} \leq \left\| \mathbf {E} _ {\widetilde {X} \sim \widetilde {S}} \left[ (\widetilde {X} - \widetilde {\mu}) (\widetilde {X} - \widetilde {\mu}) ^ {\top} \right] \right\| _ {\mathrm {o p}} \quad \text {(u s i n g} \mathbf {C o v} (\widetilde {X}) \preceq \mathbf {E} [ \widetilde {X} \widetilde {X} ^ {\top} ]) \\ \leq \left\| _ {\widetilde {X} \sim \widetilde {S}} \left[ (\widetilde {X} - \widetilde {\mu}) (\widetilde {X} - \widetilde {\mu}) ^ {\top} \right] - \widetilde {\Sigma} \right\| _ {\mathrm {o p}} + \| \widetilde {\Sigma} \| _ {\mathrm {o p}} \quad \text {(t r i a n g l e i n e q u a l i t y)} \\ \leq 1. 1 \lVert \widetilde {\boldsymbol {\Sigma}} \rVert_ {\mathrm {o p}}, \\ \end{array}
$$
where the last step upper bounds the first term using Equation (9) for  $\widetilde{S}_1 = \widetilde{S}$ , which trivially satisfies  $|\widetilde{S} |\geq |\widetilde{S} |(1 - 2\epsilon_0)$ . The overall expression is upper bounded by  $(1.1\times 3) / \alpha$  by Item 3.
The mean condition has a similar proof:
$$
\left\| \mathbf {E} _ {\widetilde {X} \sim \widetilde {S}} [ \widetilde {X} ] \right\| _ {2} \leq \left\| \mathbf {E} _ {\widetilde {X} \sim \widetilde {S}} [ \widetilde {X} ] - \widetilde {\mu} \right\| _ {2} + \| \widetilde {\mu} \| _ {2} \leq 0. 1 \sqrt {\| \widetilde {\Sigma} \| _ {\mathrm {o p}}} + \sqrt {3 / \alpha} \leq 2 / \sqrt {\alpha},
$$
where we use Equations (9) and (10).
where the inequality (a) uses the fact that  $\| uv^{\top} - wz^{\top}\|_{\mathrm{F}}^{2}\leq \| u - w\|_{2}^{2} + \| v - z\|_{2}^{2}$  for the last term.
The previous result implies that the mean and covariance of the inlier distribution  $D$  after transformation with  $H^{\dagger /2}$  is bounded. Since the variance of degree-two polynomials under  $D$  is bounded by assumption, we obtain the following bound on the variance:
Proof. By Assumption 3.1, we have that  $\mathbf{Var}_{X\sim D}[X^{\top}BX]\leq C_{1}(\| \Sigma^{1 / 2}B\Sigma^{1 / 2}\|_{F}^{2} + \| \Sigma^{1 / 2}B\mu \|_{2}^{2})$  for all  $B\in \mathbb{R}^{d\times d}$ . Applying this to  $B = H^{\dagger /2}AH^{\dagger /2}$ , we have that
$$
\begin{array}{l} \mathop {\operatorname {V a r}} _ {X \sim D} [ (H ^ {\dagger / 2} X) ^ {\top} A (H ^ {\dagger / 2} X) ] \leq C _ {1} (\| \Sigma^ {1 / 2} H ^ {\dagger / 2} A H ^ {\dagger / 2} \Sigma^ {1 / 2} \| _ {F} ^ {2} + \| \Sigma^ {1 / 2} H ^ {\dagger / 2} A H ^ {\dagger / 2} \mu \| _ {2} ^ {2}) \\ = C _ {1} \left(\| \widetilde {\Sigma} ^ {1 / 2} A \widetilde {\Sigma} ^ {1 / 2} \| _ {F} ^ {2} + \| \widetilde {\Sigma} ^ {1 / 2} A \widetilde {\mu} \| _ {2} ^ {2}\right), \tag {11} \\ \end{array}
$$
where we use that  $\| \widetilde{\Sigma}^{1 / 2}A\widetilde{\Sigma}^{1 / 2}\| _F^2 = \| \Sigma^{1 / 2}H^{\dagger /2}AH^{\dagger /2}\Sigma^{1 / 2}\| _F^2$  and  $\| \Sigma^{1 / 2}H^{\dagger /2}AH^{\dagger /2}\mu \| _2^2 =$ $\| \widetilde{\Sigma}^{1 / 2}A\widetilde{\mu}\| _2^2$  since  $\widetilde{\Sigma} = H^{\dagger /2}\Sigma H^{\dagger /2}$  and  $\widetilde{\mu} = H^{\dagger /2}\mu$ . By Lemma E.1,  $\|\widetilde{\Sigma}\|_{\mathrm{op}}\leq 3/\alpha$ . Since  $\| A\|_{\mathrm{F}} = 1$ , we have that the first term in (11) is bounded as follows:  $\| \widetilde{\Sigma}^{1 / 2}A\widetilde{\Sigma}^{1 / 2}\|_{\mathrm{F}}^{2}\leq$ $\| \widetilde{\Sigma}\|_{\mathrm{op}}^{2}\| A\|_{\mathrm{F}}^{2}\leq 9 / \alpha^{2}$ , using the fact that  $\| AB\|_{\mathrm{F}}\leq \| A\|_{\mathrm{op}}\| B\|_{\mathrm{F}}$  (Fact B.3). We also have that  $\| \widetilde{\Sigma}^{1 / 2}A\widetilde{\mu}\| _2^2\leq \| \widetilde{\Sigma}\|_{\mathrm{op}}\| A\|_{\mathrm{op}}^2\| \widetilde{\mu}\| _2^2\leq 9 / \alpha^2$ , where we the bounds from Lemma E.1. The desired conclusion then follows by applying these upper bounds in (11).
Establishing Item 6 Consider an arbitrary square matrix  $A$  with  $\| A\|_{\mathrm{F}}\leq 1$ . We have that
E.2 Certificate Lemma: Proof of Lemma 3.4
Lemma E.2 (Case when we stop and return; Formal version of Lemma 3.4). Make Assumption 3.1. Let  $w \in \mathbb{R}^{d^2}$  be the leading eigenvector of the Cov  $\widetilde{X} \sim \widetilde{T}[\widetilde{X}^{\otimes 2}]$  with  $\| w \|_2 = 1$ , and let  $A \in \mathbb{R}^{d \times d}$  be  $w^\sharp$ . Note that  $\| w \|_2 = 1$  implies  $\| A \|_{\mathrm{F}} = 1$ . For every subset  $\widetilde{T}' \subseteq \widetilde{T}$  with  $|\widetilde{T}'| \geq (\alpha / 2)|\widetilde{T}|$ , we have that
$$
\left\| _ {\tilde {X} \sim \tilde {T} ^ {\prime}} \mathbf {C o v} [ \widetilde {X} ] - \mathbb {P} _ {H} \right\| _ {\mathrm {F}} ^ {2} \lesssim \frac {1}{\alpha} \mathbf {V a r} _ {\tilde {X} \sim \tilde {T}} [ \widetilde {X} ^ {\top} A \widetilde {X} ] + \frac {1}{\alpha^ {2}}.
$$
In particular, we have that  $\left\| H^{\dagger /2}\Sigma H^{\dagger /2} - \mathbb{P}_H\right\|_{\mathrm{F}}^2\lesssim \frac{1}{\alpha}\mathbf{Var}_{\widetilde{X}\sim \widetilde{T}}[\widetilde{X}^\top A\widetilde{X} ] + \frac{1}{\alpha^2}$  and  $\left\| H^{\dagger /2}\Sigma H^{\dagger /2} - \mathbb{P}_{\Sigma}\right\|_{\mathrm{F}}^{2}\lesssim \frac{1}{\alpha}\mathbf{Var}_{\widetilde{X}\sim \widetilde{T}}[\widetilde{X}^{\top}A\widetilde{X} ] + \frac{1}{\alpha^{2}}.$
Proof. We will apply the main certificate lemma, Lemma D.4, with  $P = H^{\dagger /2}$ . The desired result then follows by noting that  $\mathbf{E}_{X\sim T}[PXX^{\top}P] = H^{\dagger /2}HH^{\dagger /2} = \mathbb{P}_H$  and the operator norm of  $\mathbb{P}_H$  is at most 1. We will use the following result, proved in Appendix C, to upper bound the Frobenius norm of  $H^{\dagger /2}\Sigma H^{\dagger /2} - \mathbb{P}_{\Sigma}$ :
Before providing the proof sketch of Lemma C.3 below, we show that it suffices to prove the desired result. Since  $H$  is the second moment matrix of  $T$ , we have that  $\ker(H) \subseteq \ker(\sum_{x \in S \cap T} xx^{\top})$ , which is further contained in  $\ker(\Sigma)$  by Condition (L.5) in Definition 2.1. We apply Lemma C.3 with  $B = \Sigma$  and  $A = H$ , which gives the desired result.
We now give a brief proof sketch of Lemma C.3. By triangle inequality, it suffices to show that  $\| \mathbb{P}_A - \mathbb{P}_B\|_{\mathrm{F}}\leq \| ABA - \mathbb{P}_A\|_{\mathrm{F}}$ . Since  $\ker (A)\subset \ker (B),\mathbb{P}_A - \mathbb{P}_B$  is again a projection matrix of rank equal to  $\operatorname {rank}(A) - \operatorname {rank}(B)$  and thus its Frobenius norm is equal to square root of  $\operatorname {rank}(A) - \operatorname {rank}(B)$ . On the other hand, the matrices  $ABA$  and  $\mathbb{P}_A$  have a rank difference of at least  $\operatorname {rank}(A) - \operatorname {rank}(B)$ . Combining this observation with the fact that  $\mathbb{P}_A$  has binary eigenvalues, we can lower bound  $\| ABA - \mathbb{P}_A\|_{\mathrm{F}}$  by square root of  $\operatorname {rank}(A) - \operatorname {rank}(B)$ .
E.3 Filtering: Proof of Lemma 3.5
We first show that if we take the median of a quadratic polynomial  $p$  over the corrupted set, then the sample mean of  $p$  over the inliers is not too far from the median if the left and right quantiles are close.
Lemma E.3 (Quantiles of quadratics after normalization). Make Assumption 3.1. Let  $A$  be an arbitrary symmetric matrix with  $\| A \|_{\mathrm{F}} = 1$ . Define  $\widetilde{y}_{\mathrm{median}} = \mathrm{Median}\left(\left\{\widetilde{x}^{\top}A\widetilde{x} \mid \widetilde{x} \in \widetilde{T}\right\}\right)$ . Let  $m_1$  be any number less than  $|S| / 3$ . Denote by  $q_i$  the  $i$ -th smallest point of  $\left\{\widetilde{x}^{\top}A\widetilde{x} \mid \widetilde{x} \in \widetilde{T}\right\}$ .
Suppose that  $q_{|T| - m_1} - q_{m_1} \leq R$  for  $R \coloneqq C(1 / \alpha) \log (1 / \eta)$  with  $C > 100\sqrt{C_1}$  (recall that the absolute constant  $C_1$  is defined in Assumption 3.1). Then,  $\left|\mathbf{E}_{\widetilde{X} \sim \widetilde{S}}[\widetilde{X}^\top A\widetilde{X}] - \widetilde{y}_{\mathrm{median}}\right| \leq 2R$ .
Proof. Let  $\mu' = \mathbf{E}_{\widetilde{X} \sim \widetilde{S}}[\widetilde{X}^\top A \widetilde{X}]$ . (Not to be confused with  $\hat{\mu}$  and  $\widetilde{\mu}$  in Assumption 3.1, which are expectations of  $\widetilde{X}$  in the samples and at the population level, instead of the quadratic form in the definition of  $\mu'$ .)
Given that  $\widetilde{y}_{\mathrm{median}}$  (by definition) lies within the interval  $[q_{m_1}, q_{|T|-m_1}]$  which has length at most  $R$ , it suffices to argue that  $\mu'$  also lies within distance  $R$  of that interval. Namely,  $\mu' \geq q_{m_1} - R$  and  $\mu' \leq q_{|T|-m_1} + R$ .
Let  $\sigma^2 \coloneqq \mathbf{Var}_{X \sim D}[(H^{\dagger/2}X)^\top A(H^{\dagger/2}X)]$ . The  $(\eta, 2\epsilon_0)$ -stability of  $S$  in Assumption 3.1 (see Condition (L.3)) implies that all but an  $\eta$ -fraction of  $\{\widetilde{x}^\top A\widetilde{x} : \widetilde{x} \in \widetilde{S} \cap \widetilde{T}\}$  lies in the interval
$\mathbf{E}_{X \sim D}[(H^{\dagger/2}X)^{\top}A(H^{\dagger/2}X)] \pm 10\sigma \log(1/\eta)$  since the size of the set  $\widetilde{S} \cap \widetilde{T}$  is at least  $|S|(1 - 2\epsilon_0)$  (by Assumption 3.1).
Thus, for at least  $(1 - \eta)|S \cap T| \geq |S|(1 - 2\epsilon_0 - \eta)|S|$  points in  $T$ , the value of  $\widetilde{x}^\top A\widetilde{x}$  lies in the interval  $\mathbf{E}_{X \sim D}[(H^{\dagger/2}X)^\top A(H^{\dagger/2}X)] \pm 10\sigma \log(1/\eta)$ . Since  $\eta \leq 0.01$  and  $\epsilon_0 < 1/4$ , the number of such points is at least  $|S|/3$ .
Rephrasing, there is an interval  $[y_{\mathrm{left}}, y_{\mathrm{right}}]$  that contains at least  $|S| / 3$  points in  $\widetilde{T}$ , where
$$
y _ {\mathrm {l e f t}} = \underset {X \sim D} {\mathbf {E}} [ (H ^ {\dagger / 2} X) ^ {\top} A (H ^ {\dagger / 2} X) ] - 1 0 \sigma \log (1 / \eta),
$$
$$
y _ {\mathrm {r i g h t}} = \underset {X \sim D} {\mathbf {E}} [ (H ^ {\dagger / 2} X) ^ {\top} A (H ^ {\dagger / 2} X) ] + 1 0 \sigma \log (1 / \eta).
$$
Therefore, there are at most  $|T| - |S| / 3$  points in  $\widetilde{T}$  that are less than  $y_{\mathrm{left}}$ , implying that  $q_{|T| - |S| / 3} \geq y_{\mathrm{left}}$ . Furthermore, since, by assumption we have  $m_1 \leq |S| / 3$ , this implies that  $q_{|T| - m_1} \geq y_{\mathrm{left}}$ . By a symmetric argument, we also have that  $q_{m_1} \leq y_{\mathrm{right}}$ .
Next, recall that by Item 6 of Lemma E.1, we have that  $|\mathbf{E}_{X\sim D}[(H^{\dagger /2}X)^{\top}A(H^{\dagger /2}X)] - \mu^{\prime}| = |\mathbf{E}_{X\sim D}[(H^{\dagger /2}X)^{\top}A(H^{\dagger /2}X)] - \mathbf{E}_{\widetilde{X}\sim \widetilde{S}}[\widetilde{X}^{\top}A\widetilde{X} ]| < 1 / \alpha$ . Thus, we have  $q_{|T| - m_1}\geq y_{\mathrm{left}}\geq \mu ' - 10\sigma \log (1 / \eta) - 1 / \alpha$ . Rearranging, we get  $\mu^{\prime}\leq q_{|T| - m_{1}} + 10\sigma \log (1 / \eta) + 1 / \alpha$ . Symmetrically, we get that  $\mu^{\prime}\geq q_{m_1} - 10\sigma \log (1 / \eta) - 1 / \alpha$ .
Finally, we argue that  $10\sigma \log (1 / \eta) + 1 / \alpha \leq R$ , showing that  $\mu^{\prime}\leq q_{|T| - m_1} + R$  and  $\mu^{\prime}\geq q_{m_1} - R$ , which as we argued at the beginning of the proof is sufficient to show the lemma statement. To argue this, by Lemma 3.2, we have that  $\sigma \leq 5\sqrt{C_1} /\alpha$ . Thus, it suffices to choose, as in the lemma statement, that  $R = (C / \alpha)\log (1 / \eta)$  for some sufficiently large  $C > 0$ .
The above result implies that if we calculate the "squared deviation" of the projections of inliers centered around the empirical median of the corrupted set, then it will be of the order of the empirical variance of inliers up to a factor of  $R^2$  (by triangle inequality). Moreover, the empirical variance of the inliers is of the order  $O(1 / \alpha^2)$  by Lemma 3.2. Thus if the variance of the corrupted set is much larger than the upper bound, then we can assign score to each point, the function  $f$  below, such that the contribution from outliers is much larger than the inliers.
Lemma E.4 (Filtering; Formal version of Lemma 3.5). Make Assumption 3.1. Let  $A$  be an arbitrary symmetric matrix with  $\| A \|_{\mathrm{F}} = 1$ . Let  $R = C(1 / \alpha) \log(1 / \eta)$  for  $C \geq 100 \sqrt{C_1}$ . Define  $\widetilde{y}_{\mathrm{median}} = \mathrm{Median}(\{\widetilde{x}^{\top} A \widetilde{x} \mid \widetilde{x} \in \widetilde{T}\})$ . Define the function  $f(\widetilde{x}) := (\widetilde{x}^{\top} A \widetilde{x} - \widetilde{y}_{\mathrm{median}})^2$ . Let  $m_1$  be a number less than  $|S| / 3$ . Denote by  $q_i$  the  $i$ -th smallest point of  $\{\widetilde{x}^{\top} A \widetilde{x} \mid \widetilde{x} \in \widetilde{T}\}$ .