SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\tau_ {j} <   \frac {d + r \delta}{r - d + 1}.
$$
Once again, the lemma does not assume anything about  $j$  being in  $S$ .
Proof. First, observe that for any  $j \in [n]$ , we have
$$
\sum_ {i \in S} \tau_ {i j} ^ {2} = \sum_ {i \in S} v _ {j} ^ {T} M ^ {- 1} v _ {i} v _ {i} ^ {T} M ^ {- 1} v _ {j} = v _ {j} ^ {T} M ^ {- 1} M M ^ {- 1} v _ {j} = \tau_ {j}.
$$
Combining this observation with Lemma 3.3 and summing over  $i \in S$  (with repetitions, if  $S$  is a multi-set), we have
$$
(1 + \tau_ {j}) (r - \sum_ {i \in S} \tau_ {i}) + \tau_ {j} <   r (1 + \delta).
$$
Now using Lemma 3.2, we get
$$
(1 + \tau_ {j}) (r - d) + \tau_ {j} <   r + r \delta ,
$$
and simplifying this completes the proof of the lemma.
Since Lemma 3.4 does not make any additional assumptions about  $j$ , we immediately have:
Corollary 3.5. The following properties hold for the output of the Local search procedures.
1. For LocalSearch-NR, the output  $S$  satisfies: for all  $j \in [n] \setminus S$ ,  $\tau_j < \frac{d + r\delta}{r - d + 1}$ .
2. For LocalSearch-R, the output  $S$  satisfies: for all  $j \in [n]$ ,  $\tau_j < \frac{d + r\delta}{r - d + 1}$ .
3.3 Volumetric Spanners: Spanning Subsets in the  $\ell_2$  Norm
We use Lemma 3.1 and Corollary 3.5 to obtain the following.
Theorem 3.6 ( $\ell_2$ -volumetric spanner). For any set  $X = \{v_{1}, v_{2}, \ldots, v_{n}\}$  of  $n \geq d$  vectors in  $\mathbb{R}^d$  and parameter  $r \geq d$ , LocalSearch-NR outputs a (max\{1,  $\left(\frac{d + r\delta}{r - d + 1}\right)^{1/2}$ )-approximate  $\ell_2$ -volumetric spanner of  $X$  of size  $r$  in  $O\left(\frac{d}{\delta}\log r\right)$  iterations of Local Search.
In particular, setting  $r = 3d$  and  $\delta = 1/3$ , LocalSearch-NR returns an  $\ell_2$ -volumetric spanner of size 3d in  $O(d\log d)$  iterations of Local Search. Furthermore, setting  $r = 2d - 1$  and  $\delta = 0$ , LocalSearch-NR implies the existence of an  $\ell_2$ -volumetric spanner of size  $2d - 1$ .
Proof. Let  $S$  be the output of LocalSearch-NR with the parameters  $r, \delta$  on  $X$ . Let  $U$  be the matrix whose columns are  $\{v_{i} : i \in S\}$ . We show how to express any  $v_{j} \in X$  as  $U\alpha$ , where  $\alpha \in \mathbb{R}^{r}$  is a coefficient vector with  $\| \alpha \|_{2}$  being small.
For any  $j \in S$ ,  $v_{j}$  can be clearly written with  $\alpha$  being a vector that is 1 in the row corresponding to  $v_{j}$  and 0 otherwise, thus  $\| \alpha \| = 1$ . For any  $j \notin S$ , by definition, the solution to  $U\alpha = v_{j}$  is  $\alpha = U^{\dagger}v_{j}$ , where  $U^{\dagger}$  is the Moore-Penrose pseudoinverse. Thus, we have
$$
\left\| \alpha \right\| _ {2} ^ {2} = v _ {j} ^ {T} (U ^ {\dagger}) ^ {T} U ^ {\dagger} v _ {j} = v _ {j} ^ {T} (U U ^ {T}) ^ {- 1} v _ {j} = \tau_ {j}.
$$
Here we are using standard properties of the pseudoinverse. (These can be proved easily using the SVD). Hence, by Corollary 3.5, we have  $\| \alpha \|_2 \leq \left( \frac{d + r\delta}{r - d + 1} \right)^{1/2}$ .
3.4 Spanning Subsets in the  $\ell_p$  Norm
We now extend our methods above for all  $\ell_p$ -norms, for  $p \in [1, \infty)$ . As outlined in Section 1.1, we see three distinct behaviors. We begin now with the lower bound for  $p = 1$ .
$\ell_1$ -volumetric spanner. For the case  $p = 1$ , we show that small sized spanning subsets do not exist for non-trivial approximation factors.
Our construction is based on "almost orthogonal" sets of vectors.
Lemma 3.7. There exists a set of  $m = \exp(\Omega(d))$  unit vectors  $v_1, \ldots, v_m \in \mathbb{R}^d$  such that for every pair of  $i, j \in [m]$ ,  $|\langle v_i, v_j \rangle| \leq c \sqrt{\frac{\log m}{d}}$  for some fixed constant  $c$ .
An example construction of almost orthogonal vectors is a collection of random vectors where each coordinate of each vector is picked uniformly at random from  $\{\frac{1}{\sqrt{d}}, \frac{-1}{\sqrt{d}}\}$  (e.g., see [Dasgupta et al., 2009]).
Theorem 3.8 (Lower bound for  $\ell_1$ -volumetric spanners). For any  $n \leq \exp(\Omega(d))$ , there exists a set of  $n$  vectors in  $\mathbb{R}^d$  that has no  $o(\sqrt{\frac{d}{\log n}})$ -approximate  $\ell_1$ -volumetric spanner of size at most  $n - 1$ .
In other words, unless the spanning subset contains all vectors, it is not possible to get an  $\ell_1$ -volumetric spanner with approximation factor  $o\left(\sqrt{\frac{d}{\log n}}\right)$ .
Proof. Let  $X = \{v_{1},\ldots ,v_{n}\}$  be a set of  $n$  almost orthonormal vectors as in Lemma 3.7. Suppose for the sake of contradiction, that there exists a spanning subset indexed by  $S$  that is a strict subset of  $[n]$ . Note that for every  $i\in [n]\setminus S$  and  $j\in S$ ,  $|\langle v_i,v_j\rangle |\leq c\sqrt{\frac{\log n}{d}}$ . So, for any representation of  $v_{i}$  in terms of vectors in  $S$ , i.e.,  $v_{i} = \sum_{j\in S}\alpha_{j}v_{j}$ ,
$$
1 = \langle v _ {i}, v _ {i} \rangle = \sum_ {j \in S} \alpha_ {j} \langle v _ {i}, v _ {j} \rangle \leq \| \alpha \| _ {1} \cdot c \sqrt {\frac {\log n}{d}}.
$$
Hence,  $\| \alpha \|_1 \geq \frac{1}{c} \sqrt{\frac{d}{\log n}}$ , as long as  $|S| < n$ .
Note that the lower bound nearly matches the easy upper bound that one obtains from  $\ell_2$  volumetric spanners (Theorem 3.6), described below:
Corollary 3.9. For any set of vectors  $X = \{v_{1}, v_{2}, \ldots, v_{n}\}$ , an  $\ell_{2}$ -volumetric spanner is also a  $2\sqrt{d}$ -approximate  $\ell_{1}$ -volumetric spanner. Consequently, such a spanner of size  $O(d)$  exists and can be found in  $O(d\log d)$  iterations of Local Search.
The proof follows from the fact that if  $\| \alpha \|_2 \leq 1$ ,  $\| \alpha \|_1 \leq \sqrt{3d}$ , for  $\alpha \in \mathbb{R}^{3d}$  (which is a consequence of the Cauchy-Schwarz inequality). Note that the existence and construction of an  $\ell_2$  volumetric spanner of size  $3d$  was shown in Theorem 3.6.
$\ell_p$ -volumetric spanner for  $p \in (1,2)$ . Next, we apply the same argument as above for the case  $p \in (1,2)$ . Here, we see that the lower bound is not so strong: one can obtain a trade-off between the size of the spanner and the approximation. Once again, the solution returned by LocalSearch-NR is an almost optimal construction for spanning subsets in the  $\ell_p$  norm. The proofs are deferred to Section B of the Supplement.
Theorem 3.10 (Lower bound for  $\ell_p$ -volumetric spanners for  $p \in (1,2)$ ). For any value of  $n \leq e^{\Omega(d)}$  and  $1 < p < 2$ , there exists a set of  $n$  vectors in  $\mathbb{R}^d$  that has no  $o(r^{\frac{1}{p} - 1} \cdot (\frac{d}{\log n})^{\frac{1}{2}})$ -approximate  $\ell_p$ -volumetric spanner of size at most  $r$ .
In particular, a (1-approximate)  $\ell_p$ -volumetric spanner of  $V$ , has size  $\Omega\left(\left(\frac{d}{\log n}\right)^{\frac{p}{2p - 2}}\right)$ .
Next, we show that local search outputs almost optimal  $\ell_p$ -volumetric spanners.
Theorem 3.11 (Construction of  $\ell_p$ -volumetric spanners for  $p \in (1,2)$ ). For any set of vectors  $X = \{v_1, v_2, \ldots, v_n\} \subset \mathbb{R}^d$  and  $p \in (1,2)$ , LocalSearch-NR outputs an  $O(r^{\frac{1}{p} - 1} \cdot d^{\frac{1}{2}})$ -approximate  $\ell_p$ -volumetric spanner of  $X$  of size  $r$ .
In particular, the local search algorithm outputs a 1-approximate  $\ell_p$ -volumetric spanner when  $r = O(d^{\frac{p}{2p - 2}})$ .
$\ell_{p}$ -volumetric spanner for  $p > 2$ . The result for  $p > 2$  simply follows from the results for  $\ell_{2}$ -norm and the fact that  $\|x\|_p \leq \|x\|_2$  for any  $p \geq 2$  when  $\|x\|_2 \leq 1$ .
Corollary 3.12 ( $\ell_p$ -volumetric spanner for  $p > 2$ ). For any set of  $n$  vectors  $X = \{v_1, v_2, \ldots, v_n\} \subset \mathbb{R}^d$ , LocalSearch-NR outputs a 1-approximate  $\ell_p$ -volumetric spanner of  $X$  of size  $r = 3d$  in  $O\left(\frac{d}{\delta} \log d\right)$  iterations of Local Search.
4 Applications of Local Search and Volumetric Spanners
We now give an application of our Local Search algorithms and volumetric spanners to the problem of finding coresets for the MVEE problem. For other applications, refer to Section A.
Definition 4.1 (Minimum volume enclosing ellipsoid (MVEE)). Given a set of points  $X = \{v_{1}, v_{2}, \ldots, v_{n}\} \subseteq \mathbb{R}^{d}$ , define  $\mathcal{E}(X)$  to be the ellipsoid of the minimum volume containing the points  $X \cup (-X)$ , where  $(-X) := \{-v : v \in X\}$ .
While the MVEE problem is well-defined for general sets of points, we are restricting to sets that are symmetric about the origin. It is well-known (see Todd [2016]) that the general case can be reduced to the symmetric one. Thus for any  $X$ ,  $\mathcal{E}(X)$  is centered at the origin. Since  $\mathcal{E}$  is convex, one can also define  $\mathcal{E}(X)$  to be the ellipsoid of the least volume containing  $\mathrm{conv}(\pm v_1,\pm v_2,\ldots ,\pm v_n)$ , where  $\mathrm{conv}(\cdot)$  refers to the convex hull.
As defined in Section 2.1, a coreset is a subset of  $X$  that preserves the volume of the MVEE.
Theorem 4.2. Consider a set of vectors  $X = \{v_{1},\dots ,v_{n}\} \subset \mathbb{R}^{d}$ . Let  $S$  be the output of the algorithm LocalSearch-R on  $X$ , with
$$
r = \lceil (1 + \frac {4}{\epsilon}) d \rceil , \quad \delta = \frac {\epsilon d}{4 r}. \tag {5}
$$
Then  $S$  is a coreset for the MVEE problem on  $X$ .
To formulate the MVEE problem, recall that any ellipsoid  $\mathcal{E}$  can be defined using a positive semidefinite (PSD) matrix  $H$ , as
$$
\mathcal {E} = \{x: x ^ {T} H x \leq d \},
$$
and for  $\mathcal{E}$  defined as such, we have  $\mathrm{vol}(\mathcal{E}) = \operatorname*{det}(H^{-1})$ , up to a factor that only depends on the dimension  $d$  (i.e., is independent of the choice of the ellipsoid). Thus, to find  $\mathcal{E}$ , we can consider the following optimization problem.
$$
(M V E E): \min  - \ln \det (H) \quad \text {s u b j e c t}
$$
$$
v _ {i} ^ {T} H v _ {i} \leq d \quad \forall i \in [ n ],
$$
$$
H \succeq 0.
$$
It is well known (e.g., Boyd et al. [2004]) that this is a convex optimization problem. For any  $\lambda \in \mathbb{R}^n$  with  $\lambda_i \geq 0$  for all  $i \in [n]$ , the Lagrangian for this problem can be defined as:
$$
\mathcal {L} (H; \lambda) = - \ln \det  (H) + \sum_ {i \in [ n ]} \lambda_ {i} (v _ {i} ^ {T} H v _ {i} - d).
$$
Let OPT be the optimal value of the problem MVEE defined above. For any  $\lambda$  with non-negative coordinates, we have
$$
\mathrm {O P T} \geq \min  _ {H} \mathcal {L} (H; \lambda),
$$
where the minimization is over the feasible set for MVEE; this is because over the feasible set, the second term of the definition of  $\mathcal{L}(H;\lambda)$  is  $\leq 0$ . We can then remove the feasibility constraint, and conclude that
$$
\mathrm{OPT}\geq \min_{H\succeq 0}\mathcal{L}(H;\lambda),
$$
as this only makes the minimum smaller. For any given  $\lambda$  with non-negative coordinates, the minimizing  $H$  can now be found by setting the derivative to 0,
$$
- H ^ {- 1} + \sum_ {i \in [ n ]} \lambda_ {i} v _ {i} v _ {i} ^ {T} = 0 \iff H = \left(\sum_ {i} \lambda_ {i} v _ {i} v _ {i} ^ {T}\right) ^ {- 1}.
$$
There is a mild technicality here: if  $\lambda$  is chosen such that  $\sum_{i}\lambda_{i}v_{i}v_{i}^{T}$  is not invertible, then  $\min_{H\succeq 0}(H;\lambda) = -\infty$ . We will only consider  $\lambda$  for which this is not the case. Thus, we have that for any  $\lambda$  with non-negative coordinates for which  $\sum_{i}\lambda_{i}v_{i}v_{i}^{T}$  is invertible,
$$
\mathrm {O P T} \geq \ln \det  \left(\sum_ {i} \lambda_ {i} v _ {i} v _ {i} ^ {T}\right) + d - d \sum_ {i} \lambda_ {i}. \tag {6}
$$
We are now ready to prove Theorem 4.2 on small-sized coresets.
Proof of Theorem 4.2. Let  $X = \{v_{1}, v_{2}, \ldots, v_{n}\}$  be the set of given points, and let  $S$  be the output of the algorithm LocalSearch-R on  $X$ , with  $r, \delta$  chosen later in (5). By definition,  $S$  is a multi-set of size  $r$ , and we define  $T$  to be its support,  $\operatorname{supp}(S)$ . We prove that  $T$  is a coreset for the MVEE problem on  $X$ .
To do so, define  $\mathrm{OPT}_X$  and  $\mathrm{OPT}_T$  to be the optimum values of the optimization problem MVEE defined earlier on sets  $X$  and  $T$  respectively. Since the problem on  $T$  has fewer constraints, we have  $\mathrm{OPT}_T \leq \mathrm{OPT}_X$ , and thus we focus on showing that  $\mathrm{OPT}_X \leq d\ln(1 + \epsilon) + \mathrm{OPT}_T$ . This will imply the desired bound on the volumes.
Let  $S$  be the multi-set returned by the algorithm LocalSearch-R, and let  $M := \sum_{i \in S} v_i v_i^T$ . Define  $\lambda_i = n_i / r$ , where  $n_i$  is the number of times  $i$  appears in  $S$ . By definition, we have that  $\sum_{i \in [n]} \lambda_i = 1$ . Further, if we define  $H := (\sum_{i \in [n]} \lambda_i v_i v_i^T)^{-1}$ , we have  $H^{-1} = \frac{1}{r} \cdot M$ .
Now, using Corollary 3.5, we have that for all  $j \in [n]$ ,
$$
v _ {j} ^ {T} M ^ {- 1} v _ {j} <   \frac {d + r \delta}{r - d + 1} \Rightarrow v _ {j} ^ {T} H v _ {j} <   \frac {r (d + r \delta)}{r - d + 1} = d \left(1 + \frac {d - 1}{r - d + 1}\right) \left(1 + \frac {r \delta}{d}\right).
$$
Our choice of parameters will be such that both the terms in the parentheses are  $(1 + \epsilon /4)$ . For this, we can choose  $r,\delta$  as in (5).
Thus, we have that  $H' = \frac{H}{(1 + \epsilon)}$  is a feasible solution to the optimization problem MVEE on  $X$ . This implies that  $\mathrm{OPT}_X \leq -\ln \det(H / (1 + \epsilon)) = d \ln (1 + \epsilon) - \ln \det(H)$ . The last equality follows because  $\det(H / c) = \det(H) / c^d$  for a  $d$ -dimensional  $H$ , and any constant  $c$ .
Next, using the fact that the  $\lambda_{i}$  are supported on  $T = \operatorname{supp}(S)$ , we can use (6) to conclude that  $\mathrm{OPT}_T \geq \ln \det(H^{-1}) = -\ln \det(H)$ , where we also used the fact that  $\sum_{i} \lambda_{i} = 1$ .
Together, these imply that  $\mathrm{OPT}_X\leq d\ln (1 + \epsilon) + \mathrm{OPT}_T$  , as desired.
We show that a one-swap local search procedure can be used to obtain an efficient construction of volumetric spanners, also known as well-conditioned spanning subsets. This improves (and simplifies) two lines of work that have used this notion in applications ranging from bandit algorithms to matrix sketching and low rank approximation. We then show that the local search algorithm also yields nearly tight results for an  $\ell_p$  analog of volumetric spanners. Finally, we obtain  $O(d / \epsilon)$  sized coresets for the classic problem of minimum volume enclosing ellipsoid, improving previous results by a  $d\log \log d$  term.
An interesting open problem arising from our work is that of developing an efficient algorithm to find a  $\ell_2$  volumetric spanner of size  $2d - 1$ . While our local search approach demonstrates the existence of such spanner, it does not find it in polynomial time. Another interesting question is about further improving the gap: is it possible to obtain a volumetric spanner of size smaller than  $2d - 1$  for all instances?
The first author thanks the National Science Foundation for supporting his work via awards CCF-2008688 and CCF-2047288.
D. Achlioptas and F. McSherry. Fast computation of low-rank matrix approximations. Journal of the ACM (JACM), 54(2):9-es, 2007.
P. K. Agarwal and N. H. Mustafa.  $k$ -means projective clustering. In Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 155-165, 2004.
P. K. Agarwal and C. M. Procopiuc. Approximation algorithms for projective clustering. Journal of Algorithms, 46(2):115-139, 2003.
P. K. Agarwal, S. Har-Peled, K. R. Varadarajan, et al. Geometric approximation via coresets. Combinatorial and computational geometry, 52(1):1-30, 2005.
J. Altschuler, A. Bhaskara, G. Fu, V. Mirrokni, A. Rostamizadeh, and M. Zadimoghaddam. Greedy column subset selection: New bounds and distributed algorithms. In International conference on machine learning, pages 2539-2548, 2016.
N. Anari and T.-D. Vuong. From sampling to optimization on discrete domains with applications to determinant maximization. In Conference on Learning Theory, pages 5596-5618, 2022.
B. Awerbuch and R. Kleinberg. Online linear optimization and adaptive routing. Journal of Computer and System Sciences, 74(1):97-114, 2008.
K. Ball. Ellipsoids of maximal volume in convex bodies. Geometriae Dedicata, 41(2):241-250, 1992.
J. D. Batson, D. A. Spielman, and N. Srivastava. Twice-ramanujan sparsifiers. In Proceedings of the forty-first annual ACM symposium on Theory of computing, pages 255–262, 2009.
C. Boutsidis, M. W. Mahoney, and P. Drineas. Unsupervised feature selection for principal components analysis. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 61-69, 2008.
C. Boutsidis, M. W. Mahoney, and P. Drineas. An improved approximation algorithm for the column subset selection problem. In Proceedings of the twentieth annual ACM-SIAM symposium on Discrete algorithms, pages 968-977, 2009.
S. Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.
A. Civril and M. Magdon-Ismail. On selecting a maximum volume sub-matrix of a matrix and related problems. Theoretical Computer Science, 410(47-49):4801-4811, 2009.
M. B. Cohen, B. Cousins, Y. T. Lee, and X. Yang. A near-optimal algorithm for approximating the john ellipsoid. In Conference on Learning Theory, pages 849-873, 2019.
V. Cohen-Addad, D. Saulpic, and C. Schwiegelshohn. A new coreset framework for clustering. In S. Khuller and V. V. Williams, editors, 53rd Annual ACM SIGACT Symposium on Theory of Computing (STOC), pages 169–182, 2021.
A. Dasgupta, P. Drineas, B. Harb, R. Kumar, and M. W. Mahoney. Sampling algorithms and coresets for  $\ell_p$  regression. SIAM Journal on Computing, 38(5):2060-2078, 2009.
A. Deshpande, L. Rademacher, S. S. Vempala, and G. Wang. Matrix approximation and projective clustering via volume sampling. Theory of Computing, 2(1):225-247, 2006.
P. Drineas, M. W. Mahoney, and S. Muthukrishnan. Subspace sampling and relative-error matrix approximation: Column-based methods. In 9th International Workshop on Approximation Algorithms for Combinatorial Optimization Problems, APPROX 2006 and 10th International Workshop on Randomization and Computation, RANDOM 2006, pages 316-326. Springer, 2006.
A. Frieze, R. Kannan, and S. Vempala. Fast monte-carlo algorithms for finding low-rank approximations. Journal of the ACM (JACM), 51(6):1025-1041, 2004.
P. Gritzmann, V. Klee, and D. Larman. Largest  $j$ -simplices in  $n$ -polytopes. Discrete & Computational Geometry, 13:477-515, 1995.
E. Hazan and Z. Karnin. Volumetric spanners: an efficient exploration basis for learning. Journal of Machine Learning Research, 2016.
L. G. Khachiyan and M. J. Todd. On the complexity of approximating the maximal inscribed ellipsoid for a polytope. Technical report, Cornell University Operations Research and Industrial Engineering, 1990.
G. Kindler, A. Naor, and G. Schechtman. The UGC hardness threshold of the  $L_{p}$  grothendieck problem. Mathematics of Operations Research, 35(2):267-283, 2010.
P. Kumar and E. A. Yildirim. Minimum-volume enclosing ellipsoids and core sets. Journal of Optimization Theory and applications, 126(1):1-21, 2005.
J. Lindenstrauss and L. Tzafriri. Classical Banach Spaces I: Sequence Spaces. Ergebnisse der Mathematik und ihrer Grenzgebiete. 2. Folge. Springer Berlin Heidelberg, 2013.
V. Madan, M. Singh, U. Tantipongpipat, and W. Xie. Combinatorial algorithms for optimal design. In Conference on Learning Theory, pages 2210-2258, 2019.
S. Mahabadi, P. Indyk, S. O. Gharan, and A. Rezaei. Composable core-sets for determinant maximization: A simple near-optimal algorithm. In International Conference on Machine Learning, pages 4254–4263, 2019.
B. Mirzasoleiman, A. Karbasi, R. Sarkar, and A. Krause. Distributed submodular maximization: Identifying representative elements in massive data. Advances in Neural Information Processing Systems, 26, 2013.
A. Nikolov. Randomized rounding for the largest simplex problem. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 861-870, 2015.
B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37(23):3311-3325, 1997. ISSN 0042-6989.
F. Pukelsheim. Optimal design of experiments. SIAM, 2006.
M. Rudelson. Contact points of convex bodies. *Israel Journal of Mathematics*, 101:93-124, 1997.
N. Srivastava. On contact points of convex bodies. In Geometric Aspects of Functional Analysis: Israel Seminar 2006-2010, pages 393-412, 2012.
M. J. Todd. Minimum-volume ellipsoids: Theory and algorithms. SIAM, 2016.
E. Welzl. Smallest enclosing disks (balls and ellipsoids). In New Results and New Trends in Computer Science, volume 555 of Lecture Notes in Computer Science, pages 359-370. Springer, 1991.
D. P. Woodruff and T. Yasuda. New subset selection algorithms for low rank approximation: Offline and online. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing, pages 1802-1813, 2023.
A Other Applications of Volumetric Spanners
We now show some direct applications of our construction of volumetric spanners.
A.1 Oblivious  $\ell_p$  Subspace Embeddings
Oblivious subspace embeddings (OSEs) are a well studied tool in matrix approximation, where the goal is to show that there exist sketching matrices that preserve the norm (say the  $\ell_p$  norm) of all vectors in an unknown subspace with high probability. The constructions and analyses of OSEs rely on the existence of a well-conditioned spanning set for the vectors of interest. The following follows directly from Theorem 3.6 (note that we are only using our result for  $\ell_2$ ).
Theorem A.1 (Improvement of Theorem 1.11 in [Woodruff and Yasuda, 2023]). Let  $p \in (1,\infty)$  and let  $A \in \mathbb{R}^{n \times d}$ . There exists a matrix  $R \in \mathbb{R}^{d \times s}$  for  $s = 3d$  such that  $\| ARe_i\|_p = 1$  for every  $i \in [s]$ , and for every  $x \in \mathbb{R}^d$ ,  $\| Ax\|_p = 1$ , there exists a  $y \in \mathbb{R}^s$  such that  $Ax = ARy$  and  $\| y\|_2 \leq 1$ .
The Theorem follows by considering the set
$$
X = \left\{A x: \| A x \| _ {p} = 1 \right\},
$$
and considering a well conditioned spanning subset in the  $\ell_2$  norm. Theorem 3.6 shows the existence of such a subset with  $s = 3d$ , thus the theorem follows.
However, note that the proof is non-constructive. In order to make it polynomial time, we need to show that the local search procedure can be implemented efficiently. For  $p \geq 2$ , this may be possible via the classic result of Kindler et al. [2010] on  $\ell_p$  variants of the Grothendieck inequality, but we note that the applications in [Woodruff and Yasuda, 2023] require only the existential statement.
A.2 Entrywise Huber Low Rank Approximation
The Huber loss is a classic method introduced as a robust analog to the least squares error. A significant amount of work has been dedicated to finding low-rank approximations of a matrix with the objective of minimizing the entry-wise Huber loss. The following offers a slight improvement over [Woodruff and Yasuda, 2023].
Theorem A.2. Let  $A \in \mathbb{R}^{n \times d}$  and let  $k \geq 1$ . There exists a polynomial time algorithm that outputs a subset  $S \subset [d]$  of columns in  $A$  of size  $O(k \log d)$  and  $X \in \mathbb{R}^{S \times d}$  such that
$$
\| A - A | ^ {S} X \| _ {H} \leq O (k) \min  _ {\operatorname {r a n k} \left(A _ {k}\right) \leq k} \| A - A _ {k} \| _ {H},
$$
where  $A|^{S}$  denotes the matrix whose columns are the columns of  $A$  indexed by  $S$  and  $\| \cdot \|_{H}$  denotes the entrywise Huber loss.
Note that the size of  $S$  is reduced from  $O(k\log \log k\log d)$  to  $O(k\log d)$ . The proof of Theorem A.2 follows from Theorem 1.6 in [Woodruff and Yasuda, 2023] and our improved construction for  $\ell_2$ -volumetric spanner, i.e.,  $O(1)$ -approximate spanning subset of size  $O(d)$  (see Theorem 3.6).
A.3 Average Top  $k$  Subspace Embedding
For a given vector  $v \in \mathbb{R}^n$ , the average top  $k$  loss is defined as