SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The constants  $1/3$  and  $2/9$  are chosen for ease of presentation. For all constructions of Hessian oracles in this paper, we take the straightforward relaxation  $\| \widetilde{H}_t - \nabla^2 f(x_t) \|_{\mathrm{op}} \leq \| \widetilde{H}_t - \nabla^2 f(x_t) \|_F$  and upper bound Hessian inexactness using Frobenius norm. Proof of a simplified version of Proposition 2.2 with a weaker high probability bound that is sufficient for our purposes is provided in Appendix A.2 for completeness.
Robust Mean Estimation. Recent algorithms in robust mean estimation give dimension-independent error in the presence of outliers under strong contamination model.
We use the following results, see, e.g., [DKP20], where the upper bound  $\sigma$  on the spectral norm of covariance matrix is unknown to the algorithm.
Proposition 2.3 (Robust Mean Estimation). Fix any  $0 < \xi < 1$ . Let  $S$  be a multiset of  $n = O((k\log k + \log (1 / \xi)) / \epsilon)$  i.i.d. samples from a distribution on  $\mathbb{R}^k$  with mean  $\mu_S$  and covariance  $\Sigma$ . Let  $T\subset \mathbb{R}^k$  be an  $\epsilon$ -corrupted version of  $S$  as in Definition [1.1]. There exists an algorithm (Algorithm A.2) such that, with probability at least  $1 - \xi$ , on input  $\epsilon$  and  $T$  (but not  $\| \Sigma \|_{\mathrm{op}}$ ) returns a vector  $\widehat{\mu}$  in polynomial time so that  $\| \mu_S - \widehat{\mu}\| = O(\sqrt{\|\Sigma\|_{\mathrm{op}}\epsilon})$ .
Algorithm A.2 is given in Appendix A.3. Proposition 2.3 states that for Algorithm A.2 to succeed with high probability,  $\widetilde{O}(k / \epsilon)$  i.i.d. samples need to be drawn from a  $k$ -dimensional distribution of bounded covariance. State of the art algorithms for robust mean estimation can be implemented in near-linear time, requiring only a logarithmic number of passes on the data, see, e.g, [Dia+22; CDG19; DHL19]. Any of these faster algorithms could be used for our purposes.
With the above results, the remaining technical component for applying the robust estimation subroutine (Algorithm A.2) in this paper is handling the dependence across iterations.
Because we will run RobustMeanEstimation in each iteration of our optimization algorithm, the gradients  $\{\nabla f_i(x_t)\}_{i=1}^n$  and Hessians  $\{\nabla^2 f_i(x_t)\}_{i=1}^n$  can no longer be considered as independently drawn from a distribution after the first iteration. Although they are i.i.d. for fixed  $x$ , the dependence on previous iterations through  $x$  will break the independence assumption. Therefore, we will need a union bound over all  $x_t$  to handle dependence across different iterations  $t$ . We deal with this technicality in Appendix B.
3 General Robust Nonconvex Optimization
In this section, we establish a general result that uses Algorithm A.1 to obtain approximate SOSPs in the presence of outliers under strong contamination. The inexact gradient and inexact Hessian oracles are constructed with the robust mean estimation subroutine (Algorithm A.2). We consider stochastic optimization tasks in Definition 1.2 satisfying Assumption 1.4. We construct the inexact gradient and Hessian oracle required by Algorithm A.1 as follows:
$$
\widetilde {g} _ {t} \leftarrow \text {R o b u s t M e a n E s t i m a t i o n} \left(\left\{\nabla f _ {i} \left(x _ {t}\right) \right\} _ {i = 1} ^ {n}, 4 \epsilon\right)
$$
$$
\widetilde {H} _ {t} \leftarrow \text {R o b u s t M e a n E s t i m a t i o n} \left(\left\{\nabla^ {2} f _ {i} \left(x _ {t}\right) \right\} _ {i = 1} ^ {n}, 4 \epsilon\right)
$$
Then we have the following guarantee:
Theorem 3.1. Suppose we are given  $\epsilon$ -corrupted set of functions  $\{f_i\}_{i=1}^n$  for sample size  $n$ , generated according to Definition 1.2. Suppose Assumption 1.4 holds in a bounded region  $\mathcal{B} \subset \mathbb{R}^D$  of diameter  $\gamma$  with gradient and Hessian covariance bound  $\sigma_g$  and  $\sigma_H$  respectively, and we have an arbitrary initialization  $x_0 \in \mathcal{B}$ . Algorithm A.1 initialized at  $x_0$  outputs an  $(\epsilon_g, \epsilon_H)$ -approximate SOSP for a sufficiently large sample with probability at least  $1 - \xi$  if the following conditions hold:
(I) All iterates  $x_{t}$  in Algorithm A.1 stay inside the bounded region  $\mathcal{B}$ .
(II) For an absolute constant  $c > 0$ , it holds that  $\sigma_g\sqrt{\epsilon} \leq c\epsilon_g$  and  $\sigma_H\sqrt{\epsilon} \leq c\epsilon_H$ .
The algorithm uses  $n = \widetilde{O}(D^2/\epsilon)$  samples, where  $\widetilde{O}(\cdot)$  hides logarithmic dependence on  $D, \epsilon, L_{D_g}, L_{D_H}, B_{D_g}, B_{D_H}, \gamma/\sigma_h, \gamma/\sigma_g$ , and  $1/\xi$ . The algorithm runs in time polynomial in the above parameters.
Note that we are able to obtain dimension-independent errors  $\epsilon_{g}$  and  $\epsilon_{H}$ , provided that  $\sigma_{g}$  and  $\sigma_{H}$  are dimension-independent.
3.1 Low Rank Matrix Sensing Problems
In this section, we study the problem of outlier-robust low rank matrix sensing as formally defined in Definition 1.6. We first apply the above framework to obtain an approximate SOSP in Section 3.1.2. Then we make use of the approximate SOSP to obtain a solution that is close to the ground-truth matrix  $M^{*}$  in Section 3.1.3; this demonstrates the usefulness of approximate SOSPs.
3.1.1 Main results for Robust Low Rank Matrix Sensing
The following are the main results that we obtain in this section:
Theorem 3.2 (Main Theorem Under Noiseless Measurements). Consider the noiseless setting as in Theorem 1.7 with  $\sigma = 0$ . For some sample size  $n = \widetilde{O}((d^2 r^2 + dr \log(\Gamma/\xi)) / \epsilon)$  and with probability at least  $1 - \xi$ , there exists an algorithm that outputs a solution that is  $\iota$ -close to  $M^*$  in Frobenius norm in  $O(r^2 \kappa^3 \log(1/\xi) + \kappa \log(\sigma_r^\star/\iota))$  calls to the robust mean estimation subroutine (Algorithm A.2).
This result achieves exact recovery of  $M^{*}$ , despite the strong contamination of samples. Each iteration involves a subroutine call to robust mean estimation. Algorithm A.2 presented here is one simple example of robust mean estimation; there are refinements [Dia+22; DHL19] that run in nearly linear time, so the total computation utilizing those more efficient algorithms indeed requires  $\widetilde{O}\left(r^{2}\kappa^{3}\right)$  passes of data (computed gradients and Hessians).
Theorem 3.3 (Main Theorem Under Noisy Measurements). Consider the same setting as in Theorem 1.7 with  $\sigma \neq 0$ . There exists a sample size  $n = \tilde{O}((d^2 r^2 + dr \log(\Gamma/\xi)) / \epsilon)$  such that
- if  $\sigma \leq r\Gamma$ , then with probability at least  $1 - \xi$ , there exists an algorithm that outputs a solution  $\widehat{M}$  in  $\widetilde{O}(r^2\kappa^3)$  calls to robust mean estimation routine A.2 with error  $\| \widehat{M} - M^* \|_F = O(\kappa \sigma \sqrt{\epsilon})$ ;
- if  $\sigma \geq r\Gamma$ , then with probability at least  $1 - \xi$ , there exists a (different) algorithm that outputs a solution  $\widehat{M}$  in one call to robust mean estimation routine [A.2] with error  $\| \widehat{M} - M^{*} \|_{F} = O(\sigma \sqrt{\epsilon})$ .
We prove Theorem 3.3 in Appendix C.4 and instead focus on the noiseless measurements with  $\sigma = 0$  when we develop our algorithms in this section; the two share many common techniques. In the remaining part of Section 3.1, we use  $\mathcal{G}_0$  in Definition 1.6 for the data generation process.
We now describe how we obtain the solution via nonconvex optimization. Consider the following objective function for (uncorrupted) matrix sensing:
$$
\min  _ {\substack {M \in \mathbb {R} ^ {d \times d} \\ \operatorname {rank}(M) = r}} \frac {1}{2} \underset {(A _ {i}, y _ {i}) \sim \mathcal {G} _ {0}} {\mathbb {E}} (\langle M, A _ {i} \rangle - y _ {i}) ^ {2}. \tag{2}
$$
We can write  $M = UU^{\top}$  for some  $U \in \mathbb{R}^{d \times r}$  to reparameterize the objective function. Let
$$
f _ {i} (U) := \frac {1}{2} \left(\langle U U ^ {\top}, A _ {i} \rangle - y _ {i}\right) ^ {2}. \tag {3}
$$
We can compute
$$
\bar {f} (U) := \underset {\left(A _ {i}, y _ {i}\right) \sim \mathcal {G} _ {0}} {\mathbb {E}} f _ {i} (U) = \frac {1}{2} \operatorname {V a r} \left\langle U U ^ {\top} - M ^ {*}, A _ {i} \right\rangle = \frac {1}{2} \left\| U U ^ {\top} - M ^ {*} \right\| _ {F} ^ {2}. \tag {4}
$$
We seek to solve the following optimization problem under the corruption model in Definition 1.2:
$$
\min  _ {U \in \mathbb {R} ^ {d \times r}} \bar {f} (U). \tag {5}
$$
The gradient Lipschitz constant and Hessian Lipschitz constant of  $\bar{f}$  are given by the following result. Fact 3.4 ([Jin+17], Lemma 6). For any  $\Gamma >\sigma_1^\star$ ,  $\bar{f} (U)$  has gradient Lipschitz constant  $L_{g} = 16\Gamma$  and Hessian Lipschitz constant  $L_{H} = 24\Gamma^{\frac{1}{2}}$  inside the region  $\{U:\| U\|_{\mathrm{op}}^2 < \Gamma \}$ .
3.1.2 Global Convergence to an Approximate SOSP
In this section, we apply our framework Theorem 3.1 to obtain global convergence from an arbitrary initialization to an approximate SOSP, by providing problem-specific analysis to guarantee that both Assumption 1.4 and algorithmic assumptions (I) and (II) required by Theorem 3.1 are satisfied.
Theorem 3.5 (Global Convergence to a SOSP). Consider the noiseless setting as in Theorem1.7 with  $\sigma = 0$  and  $\epsilon = O(1 / (\kappa^3 r^3))$ . Assume we have an arbitrary initialization  $U_{0}$  inside  $\{U:\| U\|_{\mathrm{op}}^2\leq \Gamma \}$ . There exists a sample size  $n = \widetilde{O}\left((d^{2}r^{2} + dr\log (\Gamma /\xi)) / \epsilon\right)$  such that with probability at least  $1 - \xi$ , Algorithm A.1 initialized at  $U_{0}$  outputs  $a\left(\frac{1}{24}\sigma_r^{\star 3 / 2},\frac{1}{3}\sigma_r^\star\right)$ -approximate SOSP using at most  $O\big(r^{2}\kappa^{3}\log (1 / \xi)\big)$  calls to robust mean estimation subroutine (Algorithm A.2).
Proof of Theorem 3.5 To apply Theorem 3.1, we verify Assumption 1.4 first. To verify (i), for all  $U$  and  $A_{i}$ ,  $f_{i}(U) = \frac{1}{2}\left(\langle U U^{\top}, A_{i} \rangle - y_{i}\right)^{2} \geq 0$ , so  $f^{*} = 0$  is a uniform lower bound. We verify (ii) in Appendix C.2; conceptually, by Fact 3.4,  $\bar{f}$  is gradient and Hessian Lipschitz; both gradient and Hessian of  $f_{i}$  are sub-exponential and concentrate around those of  $\bar{f}$ . To check (iii), we calculate the gradients and Hessians of  $f_{i}$  in Appendix C.1.1 and bound their covariances from above in Appendix C.1.2 and C.1.3. The result is summarized in the following lemma. Note that the domain of the target function in Algorithm A.1 and Theorem 3.1 is the Euclidean space  $\mathbb{R}^{D}$ , so we vectorize  $U$  and let  $D = dr$ . The gradient becomes a vector in  $\mathbb{R}^{dr}$  and the Hessian becomes a matrix in  $\mathbb{R}^{dr \times dr}$ .
Lemma 3.6 (Gradient and Hessian Covariance Bounds). For all  $U \in \mathbb{R}^{d \times r}$  with  $\|U\|_{\mathrm{op}}^2 \leq \Gamma$  and  $f_i$  defined in Equation (3), it holds
$$
\left\| \operatorname {C o v} \left(\operatorname {v e c} \left(\nabla f _ {i} (U)\right)\right) \right\| _ {\mathrm {o p}} \leq 8 \left\| U U ^ {\top} - M ^ {*} \right\| _ {F} ^ {2} \| U \| _ {\mathrm {o p}} ^ {2} \leq 3 2 r ^ {2} \Gamma^ {3} \tag {6}
$$
$$
\left\| \operatorname {C o v} \left(\operatorname {v e c} \left(H _ {i}\right)\right) \right\| _ {\mathrm {o p}} \leq 1 6 r \left\| U U ^ {\top} - M ^ {*} \right\| _ {F} ^ {2} + 1 2 8 \| U \| _ {\mathrm {o p}} ^ {4} \leq 1 9 2 r ^ {3} \Gamma^ {2} \tag {7}
$$
We proceed to verify the algorithmic assumptions in Theorem 3.1. For the assumption (I), we prove the following Lemma in Appendix C.2 to show that all iterates stay inside the bounded region in which we compute the covariance bounds.
Lemma 3.7. All iterates of Algorithm  $\boxed{A.1}$  stay inside the region  $\{U: \| U\|_{\mathrm{op}}^2 \leq \Gamma\}$ .
To verify Theorem 3.1 (II), we let  $\epsilon_g = \frac{1}{32}\sigma_r^{\star 3/2}, \epsilon_H = \frac{1}{4}\sigma_r^\star$  and  $\sigma_g = 8r\Gamma^{1.5}, \sigma_H = 16r^{1.5}\Gamma$ . So if we assume  $\epsilon = O(1 / (\kappa^3 r^3))$ , then for the absolute constant  $c$  in Theorem 3.1 it holds that
$$
\sigma_ {g} \sqrt {\epsilon} \leq c \epsilon_ {g} \quad \sigma_ {H} \sqrt {\epsilon} \leq c \epsilon_ {H}.
$$
Hence, Theorem 3.1 applies and Algorithm A.1 outputs an  $(\epsilon_g,\epsilon_H)$ -approximate SOSP with high probability in polynomial time. To bound the runtime, since  $\bar{f} (U_0) = 1 / 2\left\| U_0U_0^\top -M^*\right\| _F^2 = O(r^2\Gamma^2)$  for an arbitrary initialization  $U_{0}$  with  $\| U_0\|_{\mathrm{op}}^2 < \Gamma$ , the initial distance can be bounded by  $O(r^{2}\Gamma^{2})$ . Setting  $L_{g} = 16\Gamma ,L_{H} = 24\Gamma^{1 / 2},\bar{f} (U_0) = O(r^2\Gamma^2),f^* = 0$  and thus  $C_{\epsilon} = O(\sigma_r^{\star 3} / \Gamma)$ , Proposition 2.2 implies that Algorithm A.1 outputs a  $(\frac{1}{24}\sigma_r^{\star 3 / 2},\frac{1}{3}\sigma_r^{\star})$ -approximate second order stationary point  $U_{SOSP}$  in  $O(r^{2}\kappa^{3}\log (1 / \xi))$  steps with high probability.
3.1.3 Local Linear Convergence
In this section, we describe a local search algorithm that takes a  $(\frac{1}{24}\sigma_r^{\star 3 / 2},\frac{1}{3}\sigma_r^{\star})$  -approximate second-order stationary point as its initialization and achieves exact recovery even in the presence of outliers.
Algorithm 3.1: Local Inexact Gradient Descent
Data: The initialization  $U_{SOSP}$  is a  $\left(\frac{1}{24}\sigma_r^{\star 3/2}, \frac{1}{3}\sigma_r^{\star}\right)$ -approximate SOSP, corruption fraction is  $\epsilon$ , corrupted samples are  $\{(A_i, y_i)\}_{i=1}^n$ , target distance to optima is  $\iota$
Result:  $U$  that is  $\iota$ -close in Frobenius norm to some global minimum
1  $\eta = 1 / \Gamma$ $U_{0} = U_{SOSP}$
2 for  $t = 0,1,\ldots$  do
3  $\begin{array}{rl} & {\widetilde{g}_t\coloneqq \mathbf{RobustMeanEstimation}(\{\nabla f_i(U_t)\}_{i = 1}^n,4\epsilon)}\\ & {U_{t + 1}\gets U_t - \eta \widetilde{g}_t} \end{array}$  4
Theorem 3.8 (Local Linear Convergence). Consider the same noiseless setting as in Theorem [1.7]. Assume we already found a  $(\frac{1}{24}\sigma_r^{\star 3/2}, \frac{1}{3}\sigma_r^{\star})$ -approximate SOSP  $U_{SOSP}$  of  $\bar{f}$ . Then there exists a sample size  $n = \widetilde{O}(dr\log(1/\xi)/\epsilon)$  such that with probability at least  $1 - \xi$ , Algorithm [3.1] initialized at  $U_{SOSP}$  outputs a solution that is  $\iota$ -close to some global minimum in Frobenius norm after  $O(\kappa\log(\sigma_r^{\star}/\iota))$  calls to robust mean estimation subroutine (Algorithm [A.2]). Moreover, all iterates  $U_t$  are  $\frac{1}{3}\sigma_r^{\star 1/2}$ -close to some global minimum in Frobenius norm.
Proof Sketch. First we use known properties of  $\bar{f} = \mathbb{E}_{(A_i,y_i)\sim \mathcal{G}_0}f_i$  from the literature [Jin+17] to show approximate SOSPs of  $\bar{f}$  in particular our initialization  $U_{SOSP}$  are in a small neighborhood of the global minima of  $\bar{f}$ . In that neighborhood, it was also known that  $\bar{f}$  satisfies some local regularity conditions that enable gradient descent's linear convergence.
However, the algorithm only has access to the inexact gradient from the robust mean estimation subroutine (line 3 in Algorithm 3.1) and, therefore, we need to establish linear convergence of inexact gradient descent. We achieve this with the following iterative argument: As the iterate gets closer to global optima, the covariance bound of sample gradient in Equation 6 gets closer to 0. Because the accuracy of the robust mean estimation scales with the covariance bound (see Proposition 2.3), a more accurate estimate of population gradient  $\nabla \bar{f}$  can be obtained via the robust estimation subroutine (Algorithm A.2). This, in turn, allows for an improved inexact gradient descent step, driving the algorithm towards an iterate that is even closer to global optima.
See Appendix C.3 for the complete proof.
4 Statistical Query Lower Bound for Low Rank Matrix Sensing
Our general algorithm (Theorem 3.1) leads to an efficient algorithm for robust low rank matrix sensing with sample complexity  $O(d^{2}r^{2} / \epsilon)$  (Theorem 3.3). Interestingly, the sample complexity of the underlying robust estimation problem — ignoring computational considerations — is  $\Theta (dr / \epsilon)$ . The information-theoretic upper bound of  $O(dr / \epsilon)$  can be achieved by an exponential (in the dimension) time algorithm (generalizing the Tukey median to our regression setting); see, e.g., Theorem 3.5 in [Gao20].
Given this discrepancy, it is natural to ask whether the sample complexity achieved by our algorithm can be improved via a different computationally efficient method. In this section, we provide evidence that this may not be possible. In more detail, we establish a near-optimal information-computation tradeoff for the problem, within the class of Statistical Query (SQ) algorithms. To formally state our lower bound, we require basic background on SQ algorithms.
Basics on SQ Model. SQ algorithms are a class of algorithms that, instead of access to samples from some distribution  $\mathcal{P}$ , are allowed to query expectations of bounded functions over  $\mathcal{P}$ .
Definition 4.1 (SQ Algorithms and STAT Oracle [Kea98]). Let  $\mathcal{P}$  be a distribution on  $\mathbb{R}^{d^2 + 1}$ . A Statistical Query (SQ) is a bounded function  $q: \mathbb{R}^{d^2 + 1} \to [-1, 1]$ . For  $\tau > 0$ , the  $\mathrm{STAT}(\tau)$  oracle
responds to the query  $q$  with a value  $v$  such that  $|v - \mathbb{E}_{X \sim \mathcal{P}}[q(X)]| \leq \tau$ . An SQ algorithm is an algorithm whose objective is to learn some information about an unknown distribution  $\mathcal{P}$  by making adaptive calls to the corresponding  $\mathrm{STAT}(\tau)$  oracle.
In this section, we consider  $\mathcal{P}$  as the unknown corrupted distribution where  $(A_i, y_i)$  are drawn. The SQ algorithm tries to learn the ground truth matrix  $M^*$  from this corrupted distribution; the goal of the lower bound result is to show that this is hard.
The SQ model has the capability to implement a diverse set of algorithmic techniques in machine learning such as spectral techniques, moment and tensor methods, local search (e.g., Expectation Maximization), and several others [Fel+17]. A lower bound on the SQ complexity of a problem provides evidence of hardness for the problem. [Bre+21] established that (under certain assumptions) an SQ lower bound also implies a qualitatively similar lower bound in the low-degree polynomial testing model. This connection can be used to show a similar lower bound for low-degree polynomials.
Our main result here is a near-optimal SQ lower bound for robust low rank matrix sensing that applies even for rank  $r = 1$ , i.e., when the ground truth matrix is  $M^{*} = uu^{\top}$  for some  $u \in \mathbb{R}^d$ . The choice of rank  $r = 1$  yields the strongest possible lower bound in our setting because it is the easiest parameter regime: Recall that the sample complexity of our algorithm is  $\tilde{O}(d^2 r^2)$  as in Theorems 3.2 and 3.3 and the main message of our SQ lower bound is to provide evidence that the  $d^2$  factor is necessary for computationally efficient algorithms even if  $r = 1$ .
Theorem 4.2 (SQ Lower Bound for Robust Rank-One Matrix Sensing). Let  $\epsilon \in (0,1/2)$  be the fraction of corruptions and let  $c \in (0,1/2)$ . Assume the dimension  $d \in \mathbb{N}$  is sufficiently large. Consider the  $\epsilon$ -corrupted rank-one matrix sensing problem with ground-truth matrix  $M^{*} = uu^{\top}$  and noise  $\sigma^{2} = O(1)$ . Any SQ algorithm that outputs  $\widehat{u}$  with  $\| \widehat{u} - u \| = O(\epsilon^{1/4})$  either requires  $2^{\Omega(d^{c})} / d^{2-4c}$  queries or makes at least one query to  $\mathrm{STAT}\left(e^{O(1/\sqrt{\epsilon})} / O\left(d^{1-2c}\right)\right)$ .
In other words, we show that, when provided with SQ access to an  $\epsilon$ -corrupted distribution, approximating  $u$  is impossible unless employing a statistical query of higher precision than what can be achieved with a strictly sub-quadratic number (e.g.,  $d^{1.99}$ ) of samples. Note that the SQ oracle  $\mathrm{STAT}(e^{O(1 / \sqrt{\epsilon})} / O(d^{1 - 2c}))$  can be simulated with  $O(d^{2 - 4c}) / e^{O(1 / \epsilon)}$  samples, and this bound is tight in general. Informally speaking, this theorem implies that improving the sample complexity from  $d^2$  to  $d^{2 - 4c}$  requires exponentially many queries. This result can be viewed as a near-optimal information-computation tradeoff for the problem, within the class of SQ algorithms.
The proof follows a similar analysis as in [DKS19; Dia+21], using one-dimensional moment matching to construct a family of corrupted conditional distributions, which induce a family of corrupted joint distributions that are SQ-hard to learn. We provide the details of the proof in Appendix D. Apart from the formal proof, in Appendix E we also informally discuss the intuition for why some simple algorithms that require  $O(d)$  samples do not provide dimension-independent error guarantees.
Shuyao Li was supported in part by NSF Awards DMS-2023239, NSF Award CCF-2007757 and the U. S. Office of Naval Research under award number N00014-22-1-2348. Yu Cheng was supported in part by NSF Award CCF-2307106. Ilias Diakonikolas was supported in part by NSF Medium Award CCF-2107079, NSF Award CCF-1652862 (CAREER), a Sloan Research Fellowship, and a DARPA Learning with Less Labels (LwLL) grant. Jelena Diakonikolas was supported in part by NSF Award CCF-2007757 and by the U. S. Office of Naval Research under award number N00014-22-1-2348. Rong Ge was supported in part by NSF Award DMS-2031849, CCF-1845171 (CAREER) and a Sloan Research Fellowship. Stephen Wright was supported in part by NSF Awards DMS-2023239 and CCF-2224213 and AFOSR via subcontract UTA20-001224 from UT-Austin.
[Pas+10] P. Paschou, J. Lewis, A. Javed, and P. Drineas. "Ancestry Informative Markers for Fine-Scale Individual Assignment to Worldwide Populations". In: Journal of Medical Genetics 47 (2010), pp. 835-847.
[Pra+20] A. Prasad, A. S. Suggala, S. Balakrishnan, and P. Ravikumar. "Robust estimation via robust gradient estimation". In: Journal of the Royal Statistical Society. Series B. Statistical Methodology 82.3 (2020), pp. 601-627.
[Ros+02] N. Rosenberg, J. Pritchard, J. Weber, H. Cann, K. Kidd, L. Zhivotovsky, and M. Feldman. "Genetic structure of human populations". In: Science 298 (2002), pp. 2381-2385.
[RRT17] M. Raginsky, A. Rakhlin, and M. Telgarsky. “Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis”. In: Proceedings of the 2017 Conference on Learning Theory. Ed. by S. Kale and O. Shamir. Vol. 65. Proceedings of Machine Learning Research. PMLR, July 2017, pp. 1674–1703.
[SKL17] J. Steinhardt, P. W. Koh, and P. S. Liang. "Certified Defenses for Data Poisoning Attacks". In: Advances in Neural Information Processing Systems 30. 2017, pp. 3520-3532.
[SQW16] J. Sun, Q. Qu, and J. Wright. “A geometric analysis of phase retrieval”. In: Information Theory (ISIT), 2016 IEEE International Symposium on. IEEE. 2016, pp. 2379–2383.
[SQW17] J. Sun, Q. Qu, and J. Wright. "Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture". In: IEEE Trans. Inf. Theor. 63.2 (Feb. 2017), pp. 853-884.
[Tuk75] J. Tukey. “Mathematics and picturing of data”. In: Proceedings of ICM. Vol. 6. 1975, pp. 523–531.
[WM22] J. Wright and Y. Ma. High-dimensional data analysis with low-dimensional models: Principles, computation, and applications. Cambridge University Press, 2022.
[Yin+19] D. Yin, Y. Chen, R. Kannan, and P. Bartlett. “Defending against saddle point attack in Byzantine-robust distributed learning”. In: International Conference on Machine Learning. PMLR. 2019, pp. 7074–7084.