SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left\| v \right\| _ {\mathrm {A T} _ {k}} := \frac {1}{k} \sum_ {i \in [ k ]} | v _ {[ i ]} |,
$$
where  $v_{i}$  denotes the  $i$ th largest coordinate in  $v$ .
Using the result of [Woodruff and Yasuda, 2023] relating the problem of average top  $k$  subspace embedding to  $\ell_2$ -volumetric spanners as a black-box, we have the following theorems for small  $k$  (i.e.,  $k \leq 3d$ ) and large  $k$  (i.e.,  $k > 3d$ ) respectively.
Theorem A.3 (small  $k$ ). Let  $A \in \mathbb{R}^{n \times d}$  and let  $k \leq 3d$ . There exists a set  $S \subset [n]$  of size  $O(d)$  such that for all  $x \in \mathbb{R}^d$ ,
$$
\| A | _ {S} x \| _ {\mathrm {A T} _ {k}} \leq \| A x \| _ {\mathrm {A T} _ {k}} \leq O (\sqrt {k d}) \cdot \| A | _ {S} x \| _ {\mathrm {A T} _ {k}},
$$
where  $A|_{S}$  denotes the set of rows in  $A$  indexed by  $S$ .
Theorem A.4 (large  $k$ ). Let  $A \in \mathbb{R}^{n \times d}$  and let  $k \geq k_0$  where  $k_0 = O(d + \frac{1}{\delta})$ . Let  $P_1, \ldots, P_{\frac{k}{t}}$  be a random partition of  $[n]$  into  $\frac{k}{t}$  groups where  $t = O(d + \log \frac{1}{\delta})$ . For every  $i \in [\frac{k}{t}]$ , there exists a set  $S_i \subset N_i$  of size  $O(d)$  such that with probability at least  $1 - \delta$ , for all  $x \in \mathbb{R}^d$ ,
$$
\| A | _ {S} x \| _ {\mathrm {A T} _ {k}} \leq \| A x \| _ {\mathrm {A T} _ {k}} \leq O (\sqrt {t d}) \cdot \| A | _ {S} x \| _ {\mathrm {A T} _ {k}},
$$
where  $S \coloneqq \bigcup_{i \in [\frac{k}{s}]} S_i$  and  $A|_S$  denotes the set of rows in  $A$  indexed by  $S$ .
In both regimes, compared to [Woodruff and Yasuda, 2023], our improved bounds for  $\ell_2$ -volumetric spanner saves a factor of  $\log \log d$  in the number of rows and a factor of  $\sqrt{\log\log d}$  in the distortion. The proofs of above theorems respectively follow from Theorem 3.11 and 3.12 of [Woodruff and Yasuda, 2023] together with our Theorem 3.6.
A.4 Cascaded Norm Subspace Embedding
Next, we explore the implications of the improved bound of  $\ell_2$ -volumetric spanner (i.e., Theorem 3.6) for embedding a subspace of matrices under  $(\| \cdot \|_{\infty}, \| \cdot \|)$ - Cascaded norm, which first evaluates an arbitrary norm of the rows and then returns the maximum value over the  $n$  rows.
The following is a consequence of Theorem 3.13 in [Woodruff and Yasuda, 2023] and our Theorem 3.6. We describe our result for the  $(\| \cdot \|_{\infty},\| \cdot \|)$ - Cascaded norm, which first evaluates an arbitrary norm of rows and then returns the maximum value over the  $n$  rows.
Theorem A.5 (( $\| \|\_{\infty}, \| \|$ )-subspace embedding). Let  $A \in \mathbb{R}^{n \times d}$  and let  $\|\|$  be any norm on  $\mathbb{R}^m$ . There exists a set  $S \subset [n]$  of size at most 3d such that for every  $X \in \mathbb{R}^{d \times m}$ ,
$$
\| A | _ {S} X \| _ {(\| \bigstar \| _ {\infty}, \| \bigstar \|)} \leq \| A X \| _ {(\| \bigstar \| _ {\infty}, \| \bigstar \|)} \leq O (\sqrt {d}) \| A | _ {S} X \| _ {(\| \bigstar \| _ {\infty}, \| \bigstar \|)}
$$
Proof. Note that the second part follows from the symmetry of  $M$  (and thus also  $M^{-1}$ ). To see the first part, note that we can write  $v_{i}^{T}M^{-1}v_{i} = \langle M^{-1},v_{i}v_{i}^{T}\rangle$ , where  $\langle U,V\rangle$  refers to the entry-wise inner product between matrices  $U,V$ , which also equals  $\operatorname{Tr}(U^T V)$ . Using this,
$$
\sum_ {i \in S} \tau_ {i} = \sum_ {i \in S} \langle M ^ {- 1}, v _ {i} v _ {i} ^ {T} \rangle = \langle M ^ {- 1}, M \rangle = \operatorname {T r} (I) = d.
$$
In the last equality, we used the symmetry of  $M$ .
B.2 Proof of Theorem 3.10
Proof. The proof follows from the same argument as before. Consider a set of  $n > r$  almost orthonormal vectors  $X = \{v_{1},\dots ,v_{n}\} \subset \mathbb{R}^{d}$  from Lemma 3.7.
Consider an index  $i \in [n] \setminus S$  and let  $v_{i} = \sum_{j \in S} \alpha_{j} v_{j}$ . By Lemma 2.5, for any  $p > 1$ ,
$$
\left\| \alpha \right\| _ {p} \geq r ^ {\frac {1}{p} - 1} \cdot \left\| \alpha \right\| _ {1} = r ^ {\frac {1}{p} - 1} \cdot \frac {1}{c} \left(\frac {d}{\log n}\right) ^ {\frac {1}{2}}.
$$
In particular, to get a 1-approximate  $\ell_p$ -volumetric spanner, i.e.,  $\| \alpha \|_p = 1$ , the spanning subset must have size  $r = \Omega \left( \left( \frac{d}{\log n} \right)^{\frac{p}{2p - 2}} \right)$ .
B.3 Proof of Theorem 3.11
Proof. By Corollary 3.5, the local search outputs a set of vectors in  $X$  indexed by the set  $S \subset [n]$  of size  $r > d$  such that for every  $i \in [n] \setminus S$ ,  $v_{i}$  can be written as a linear combination of the vectors in
the spanner,  $v_{i} = \sum_{j\in S}\alpha_{j}v_{j}$ , such that  $\| \alpha \| _2\leq \left(\frac{d + r\delta}{r - d + 1}\right)^{\frac{1}{2}}$ . By Lemma 2.5 and setting  $\delta = d / r$  for any  $1 < p < 2$ ,
$$
\| \alpha \| _ {p} \leq r ^ {\frac {1}{p} - \frac {1}{2}} \cdot \left(\frac {d + r \delta}{r - d + 1}\right) ^ {\frac {1}{2}} = O \left(r ^ {\frac {1}{p} - 1} \cdot (d + r \delta) ^ {\frac {1}{2}}\right) = O \left(r ^ {\frac {1}{p} - 1} \cdot d ^ {\frac {1}{2}}\right).
$$
In particular, if we set  $r = O(d^{\frac{p}{2 - 2p}})$ , the subset of vectors  $S$  returned by LocalSearch-NR is an (exact)  $\ell_p$ -volumetric spanner; i.e., for every  $i \in [n] \setminus S$ ,  $\| \alpha \|_p \leq 1$ .
Finally, the runtime analysis follows immediately from Lemma 3.1.