SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbf {P} \left(\left\| \frac {1}{| \operatorname {s g n} |} \sum_ {i \in \operatorname {s g n}} \tilde {Z} _ {i} \right\| \leq \mathbf {E} \left\| \frac {1}{| \operatorname {s g n} |} \sum_ {i \in \operatorname {s g n}} \tilde {Z} _ {i} \right\| + u _ {\operatorname {s g n}}\right)
$$
for both  $\mathrm{sgn} \in \{O, E\}$  and where  $u_{\mathrm{sgn}} = u - \mathbf{E}\left\| \frac{1}{|\mathrm{sgn}|}\sum_{i \in \mathrm{sgn}}\tilde{Z}_i\right\|$ . Introduce now the change of variables  $(1 + 9\varepsilon)t_{\mathrm{sgn}} = u_{\mathrm{sgn}} - \eta\left(\mathbf{E}\left\| \frac{1}{|\mathrm{sgn}|}\sum_{i \in \mathrm{sgn}}\tilde{Z}_i\right\|\right)$ .
We invoke Theorem 4.1 twice; we do so once for the even blocks and once for the odd blocks. This yields for  $\mathrm{sgn} \in \{O, E\}$  that for any  $\varepsilon \in (0, \infty)$ ,  $\eta \in (0, 1]$  and  $t_{\mathrm{sgn}} \geq 0$  we have that:
$$
\begin{array}{l} \mathbf {P} \left(\left\| \frac {1}{| \operatorname {s g n} |} \sum_ {i \in \operatorname {s g n}} \tilde {Z} _ {i} \right\| \leq (1 + \eta) \mathbf {E} \left\| \frac {1}{| \operatorname {s g n} |} \sum_ {i \in \operatorname {s g n}} \tilde {Z} _ {i} \right\| + (1 + 9 \varepsilon) t _ {\operatorname {s g n}}\right) \\ \leq \exp \left(- \frac {\left| \operatorname {s g n} \right| t _ {\operatorname {s g n}} ^ {2}}{(2 + \eta) \Lambda_ {\operatorname {s g n}}}\right) + C _ {\varepsilon , \eta , s} \sum_ {a _ {i} \subset \operatorname {s g n}} \frac {\mathbf {E} \left\| \sum_ {j \in a _ {i}} Z _ {j} \right\| ^ {s}}{\left| \operatorname {s g n} \right| ^ {s} t _ {\operatorname {s g n}} ^ {s}} \tag {A.1} \\ \end{array}
$$
with  $\Lambda_{\mathrm{sgn}}$  as announced and where we used that  $\sum_{j\in a_i}Z_j$  is equal to  $\sum_{j\in a_i}\tilde{Z}_j$  in distribution for every  $a_i\subset [n]$ .
Consequently, with  $u = (1 + 9\varepsilon)t_{\mathrm{sgn}} + (1 + \eta)\mathbf{E}\left\| \frac{1}{|\mathrm{sgn}|}\sum_{i\in \mathrm{sgn}}\tilde{Z}_i\right\|$  we find that (A.1) and Corollary 4.1 together yield that:
$$
\begin{array}{l} \mathbf {P} \left(\left\| \frac {1}{n} \sum_ {i = 1} ^ {n} Z _ {i} \right\| \geq u\right) \leq \sum_ {i = 2} ^ {2 m - 1} \beta_ {Z} (a _ {i}) \\ + \sum_ {\operatorname {s g n} \in \{O, E \}} \exp \left(- \frac {| \operatorname {s g n} | t _ {\operatorname {s g n}} ^ {2}}{(2 + \eta) \Lambda_ {\operatorname {s g n}}}\right) + C _ {\varepsilon , \eta , s} \sum_ {a _ {i} \subset \operatorname {s g n}} \frac {\mathbf {E} \left\| \sum_ {j \in a _ {i}} Z _ {j} \right\| ^ {s}}{| \operatorname {s g n} | ^ {s} t _ {\operatorname {s g n}} ^ {s}} \tag {A.2} \\ \end{array}
$$
where summation over  $a_{i} \subset \operatorname{sgn}$  is restricted to  $a_{1:2m}$ . If for some  $c > 0$  we choose
$$
t _ {\mathrm {s g n}} \geq \max  _ {\operatorname {s g n} \in \{O, E \}} \sqrt {\Lambda_ {\mathrm {s g n}}} \times \sqrt {\frac {c \vee (2 + \eta) \log (1 / \delta)}{| \operatorname {s g n} |}}
$$
and let  $\sqrt{r} \geq \max_{\mathrm{sgn} \in \{O, E\}} \frac{\mathbf{E}\left\|\frac{1}{\sqrt{\left|\mathrm{sgn}\right|}}\sum_{i \in \mathrm{sgn}}\tilde{Z}_i\right\|}{\sqrt{\Lambda_{\mathrm{sgn}}}}$  then (A.2) reads for either  $\mathrm{sgn} \in \{O, E\}$ :
$$
\begin{array}{l} \mathbf {P} \left(\left\| \frac {1}{n} \sum_ {i = 1} ^ {n} Z _ {i} \right\| \geq \max  _ {\operatorname {s g n} \in \{O, E \}} \sqrt {\frac {\Lambda_ {\operatorname {s g n}}}{| \operatorname {s g n} |}} \left((1 + \eta) \sqrt {r} + (1 + 9 \varepsilon) \times \sqrt {c \vee (2 + \eta) \log (1 / \delta)}\right)\right) \\ \leq \sum_ {i = 1} ^ {2 m} \beta_ {Z} (a _ {i}) + 2 \delta + \frac {C _ {\varepsilon , \eta , s}}{c ^ {s / 2} | O | ^ {s / 2}} \sum_ {a _ {i} \subset O} \frac {\mathbf {E} \left\| \sum_ {j \in a _ {i}} Z _ {j} \right\| ^ {s}}{\Lambda_ {O} ^ {s / 2}} + \frac {C _ {\varepsilon , \eta , s}}{c ^ {s / 2} | E | ^ {s / 2}} \sum_ {a _ {i} \subset E} \frac {\mathbf {E} \left\| \sum_ {j \in a _ {i}} Z _ {j} \right\| ^ {s}}{\Lambda_ {E} ^ {s / 2}}. \\ \end{array}
$$
The result follows if we choose  $c = r\eta^2 /(1 + 9\varepsilon)^2$
A.2 Proof of Theorem 4.3
We will argue by truncation. This rests on the observation that for any sequence  $F_{1:n}$  of measurable sets:
$$
\sum_ {i \in a} X _ {i} X _ {i} ^ {\mathsf {T}} \succeq \sum_ {i \in a} X _ {i} X _ {i} ^ {\mathsf {T}} \mathbf {1} _ {F _ {i}}. \tag {A.3}
$$
where as before we let  $a \subset [n]$ . The idea is that once we truncate we do not have to worry about the upper tail of the empirical covariance matrix. We begin by showing below that a well-chosen level of truncation still preserves most of the mass of the lower tail.
A.2.1 Truncation step
We will want to choose the events  $F_{i}$  to be constant across each block. We require the following simple observation. It states that the moment condition  $\mathbf{E}\langle v,X_i\rangle^4\leq \mathsf{h}^2\langle v,\mathbf{E}[X_iX_i^\top ]v\rangle$  extends naturally to the situation where each block is our basic unit of randomness.
Lemma A.1. Under the hypotheses of Theorem 4.3 for every subset  $a \subset [n]$  we have that:
$$
\mathbf {E} \left(\frac {1}{| a |} \sum_ {i \in a} \langle v, X _ {i} \rangle^ {2}\right) ^ {2} \leq \mathsf {h} ^ {2} \left(\frac {1}{| a |} \sum_ {i \in a} \mathbf {E} \langle v, X _ {i} \rangle^ {2}\right).
$$
Proof. By direct calculation:
$$
\begin{array}{l} \mathbf {E} \left(\sum_ {i \in a} \langle v, X _ {i} \rangle^ {2}\right) ^ {2} \leq \left(\sum_ {i \in a} \sqrt {\mathbf {E} \langle v , X _ {i} \rangle^ {4}}\right) ^ {2} \quad (\text {C a u c h y - S c h w a r z}) \\ \leq \mathsf {h} ^ {2} \left(\sum_ {i \in a} \sqrt {\mathbf {E} \langle v , X _ {i} \rangle^ {2}}\right) ^ {2} \quad \left(\mathbf {E} \langle v, X _ {i} \rangle^ {4} \leq \mathsf {h} ^ {2} \langle v, \mathbf {E} [ X _ {i} X _ {i} ^ {\top} ] v \rangle\right) \tag {A.4} \\ \leq \mathsf {h} ^ {2} | a | \sum_ {i \in a} \mathbf {E} \langle v, X _ {i} \rangle^ {2}. \quad \text {(C a u c h y - S c h w a r z)} \\ \end{array}
$$
To finish the proof, divide both sides of (A.4) by  $|a|^2$ .
As announced above, let now the  $F_{i}$  be constant on each block  $a\subset [n]$ . In what follows, we abuse notation in the obvious way and write  $F_{a} = F_{i}$  for any  $i\in a$ . The next lemma shows that suitable truncation preserves most of the mass of the lower tail.
Lemma A.2. Impose the hypotheses of Theorem 4.3, fix  $\tau > 0$  and let  $F_{a} \triangleq \left\{\frac{1}{|a|}\sum_{i\in a}\langle v,X_i\rangle^2\leq \tau^2\right\}$ . For every subset  $a\subset [n]$  we have that:
$$
\left(1 - \frac {\mathsf {h} ^ {2}}{\tau^ {2}}\right) \mathbf {E} \left[ \sum_ {i \in a} \langle v, X _ {i} \rangle^ {2} \right] \leq \sum_ {i \in a} \mathbf {E} \langle v, X _ {i} \rangle^ {2} \mathbf {1} _ {F _ {a}}.
$$
Proof. As announced, fix  $\tau > 0$  and let  $F_{a} \triangleq \left\{\frac{1}{|a|} \sum_{i \in a} \langle v, X_{i} \rangle^{2} \leq \tau \right\}$ . We have that:
$$
\begin{array}{l} \frac {1}{| a |} \mathbf {E} \left[ \sum_ {i \in a} \langle v, X _ {i} \rangle^ {2} - \sum_ {i \in a} \langle v, X _ {i} \rangle^ {2} \mathbf {1} _ {F _ {a}} \right] \\ = \mathbf {E} \left[ \frac {1}{| a |} \sum_ {i \in a} \langle v, X _ {i} \rangle^ {2} \mathbf {1} _ {F _ {a} ^ {c}} \right] \\ \leq \sqrt {\mathbf {E} \left[ \frac {1}{| a |} \sum_ {i \in a} \langle v , X _ {i} \rangle^ {2} \right] ^ {2} \mathbf {P} \left(F _ {a} ^ {c}\right)} \quad (\text {C a u c h y - S c h w a r z}) \\ \leq h \sqrt {\mathbf {E} \left[ \frac {1}{| a |} \sum_ {i \in a} \langle v , X _ {i} \rangle^ {2} \right]} \sqrt {\mathbf {P} \left(F _ {a} ^ {c}\right)} (A.5) \\ \leq h \sqrt {\mathbf {E} \left[ \frac {1}{| a |} \sum_ {i \in a} \langle v , X _ {i} \rangle^ {2} \right]} \frac {\sqrt {\mathbf {E} \left[ \frac {1}{| a |} \sum_ {i \in a} \langle v , X _ {i} \rangle^ {2} \right] ^ {2}}}{\tau^ {2}} (\text {M a r k o v ’ s I n e q u a l i t y}) \\ \leq \frac {h ^ {2}}{\tau^ {2}} \mathbf {E} \left[ \frac {1}{| a |} \sum_ {i \in a} \langle v, X _ {i} \rangle^ {2} \right]. (Lemma A.1) \\ \end{array}
$$
To finish the proof, multiply both sides of (A.5) above by  $|a|$  and re-arrange.
We now proceed with the truncation argument. Define the function
$$
\phi_ {\tau} (u) \triangleq \left\{ \begin{array}{l l} u ^ {2} & \text {i f} | u | \leq \tau , \\ \tau^ {2} & \text {i f} | u | > \tau . \end{array} \right.
$$
The truncation is combined with an approximation. Namely, notice that we have
$$
\begin{array}{l} \frac {1}{n} \sum_ {i = 1} ^ {n} \langle v, X _ {i} \rangle^ {2} \geq \frac {1}{n} \sum_ {i = 1} ^ {n} \phi_ {\tau} (\langle v, X _ {i} \rangle) \\ = \frac {1}{n} \mathbf {E} \sum_ {i = 1} ^ {n} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \frac {1}{n} \left[ \mathbf {E} \sum_ {i = 1} ^ {n} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \sum_ {i = 1} ^ {n} \phi_ {\tau} (\langle v, X _ {i} \rangle) \right] \tag {A.6} \\ \geq \mathbf {E} \sum_ {i = 1} ^ {n} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in O} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \sum_ {i \in O} \phi_ {\tau} (\langle v, X _ {i} \rangle) \right] \\ - \sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in E} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \sum_ {i \in E} \phi_ {\tau} (\langle v, X _ {i} \rangle) \right] \\ \end{array}
$$
where the last step uses the assumption that  $v \in \partial \Sigma_{X}$ .
By invoking Lemma A.2 and noticing that  $\phi_{\tau}$  can be lower-bounded via the indicators of the  $F_{a}$  we have that:
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {E} \phi_ {\tau} (\langle v, X _ {i} \rangle) \geq \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {E} \langle v, X _ {i} \rangle^ {2} \mathbf {1} _ {F _ {a}} \geq \left(1 - \frac {\mathrm {h} ^ {2}}{\tau^ {2}}\right) \left[ \frac {1}{n} \sum_ {i = 1} ^ {n} \mathbf {E} \langle v, X _ {i} \rangle^ {2} \right] = \left(1 - \frac {\mathrm {h} ^ {2}}{\tau^ {2}}\right) \tag {A.7}
$$
since  $v \in \partial \Sigma_{X}$ . Consequently by combining (A.6) and (A.7) we obtain the deterministic decomposition:
$$
\begin{array}{l} \frac {1}{n} \sum_ {i = 1} ^ {n} \langle v, X _ {i} \rangle^ {2} \geq \left(1 - \frac {\mathsf {h} ^ {2}}{\tau^ {2}}\right) - \sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in O} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \sum_ {i \in O} \phi_ {\tau} (\langle v, X _ {i} \rangle) \right] \\ - \sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in E} \phi_ {\tau} (\langle v, X _ {i} \rangle) - \sum_ {i \in E} \phi_ {\tau} (\langle v, X _ {i} \rangle) \right] \tag {A.8} \\ \end{array}
$$
for every  $v\in \partial \Sigma_X$
The decomposition (A.8) sets the stage for the last step of the proof: we will apply the block decoupling result of Proposition 4.1 to replace the empirical processes appearing on the right of (A.8) and pay an additional failure probability of  $\sum_{i=2}^{2m-1} \beta_X(a_i)$ .
Consequently, it only remains to control the supremum of the following truncated and decoupled empirical process  $(\mathrm{sgn}\in \{O,E\})$  ..
$$
\sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right]. \tag {A.9}
$$
A.2.2 Approximation step: bounding (A.9)
We now intend to invoke Talagrand's inequality for independent but not necessarily equally distributed variables [see Klein and Rio, 2005, Theorem 1.1]. To do so, we first need to control the expectation and then the weak variance of the process (A.9). For future reference, we also point out that by construction:
$$
\left| \sum_ {i \in a} \mathbf {E} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right| \leq 2 \tau^ {2} | a |. \tag {A.10}
$$
We now turn to controlling the expectation of (A.9). Using that the odd (resp. even) blocks of the decoupled process are independent, a straightforward symmetrization argument yields [see e.g. Wainwright, 2019, Proposition 4.11]:
$$
\mathbf {E} \sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] \leq \frac {2}{n} \mathbf {E} \left[ \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {j \in [ 2 m ], a _ {j} \subset \operatorname {s g n}} \eta_ {j} \sum_ {i \in a _ {j}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] \tag {A.11}
$$
for a sequence  $\eta_j, j \in [2m]$  of Rademacher random variables. Since the map  $(\mathbb{R}^d, \| \cdot \|_2) \ni x \mapsto \sum_{j=1}^{d} \phi_\tau(x_j) \in (\mathbb{R}, |\cdot|)$  is  $2\sqrt{d}\tau$ -Lipschitz, the vector version of the Ledoux-Talagrand contraction principle now yields [see Maurer, 2016, Theorem 2]:
$$
2 \mathbf {E} \left[ \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {j \in [ 2 m ], a _ {j} \subset \operatorname {s g n}} \eta_ {j} \sum_ {i \in a _ {j}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] \leq 4 \tau \sqrt {2 \max  _ {j \in [ 2 m ]} | a _ {j} |} \mathbf {E} \left[ \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {i \in \operatorname {s g n}} \eta_ {i} ^ {\prime} \langle v, \tilde {X} _ {i} \rangle \right] \tag {A.12}
$$
for another sequence  $\eta_i^\prime$ $i\in [n]$  of Rademacher random variables.
Obviously,
$$
\mathbf {E} \left[ \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {i \in \operatorname {s g n}} \eta_ {i} \langle v, \tilde {X} _ {i} \rangle \right] = \mathbf {E} \left\| \sum_ {i \in \operatorname {s g n}} \eta_ {i} \Sigma_ {X} ^ {- 1 / 2} \tilde {X} _ {i} \right\| _ {2} \leq \sqrt {\mathbf {E} \left\| \sum_ {i \in \operatorname {s g n}} \Sigma_ {X} ^ {- 1 / 2} \tilde {X} _ {i} \right\| _ {2} ^ {2}} \leq \sqrt {n d _ {\mathrm {X}}}. \tag {A.13}
$$
Combining (A.11), (A.12) and (A.13) we find that:
$$
\mathbf {E} \sup  _ {v \in \partial \Sigma_ {X}} \frac {1}{n} \left[ \mathbf {E} \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] \leq 4 \tau \sqrt {2 n d _ {\mathrm {X}} \max  _ {j \in [ 2 m ]} | a _ {j} |} \tag {A.14}
$$
As announced, we also need an upper bound on the variance of the truncated empirical process (A.9):
$$
\begin{array}{l} \sup  _ {v \in \partial \Sigma_ {X}} \mathbf {V a r} \left[ \mathbf {E} \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \sum_ {i \in \operatorname {s g n}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] \\ = \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {j \in [ 2 m ], a _ {j} \subset \operatorname {s g n}} \mathbf {V a r} \left[ \mathbf {E} \sum_ {i \in a _ {j}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \sum_ {i \in a} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] \quad (\text {B l o c k w i s e I n d e p .}) \\ \leq \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {j \in [ 2 m ], a _ {j} \subset \operatorname {s g n}} \mathbf {E} \left[ \sum_ {i \in a _ {j}} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) \right] ^ {2} \quad (J e n s e n's I n e q.) \\ \leq \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {j \in [ 2 m ], a _ {j} \subset \operatorname {s g n}} \tau^ {2} | a _ {j} | \sum_ {i \in a _ {j}} \mathbf {E} \langle v, \tilde {X} _ {i} \rangle^ {2} \quad (| \phi_ {\tau} (\cdot) | \leq \tau^ {2}) \\ \leq \tau^ {2} \max  _ {a \subset \operatorname {s g n}} | a | \sup  _ {v \in \partial \Sigma_ {X}} \sum_ {i \in \operatorname {s g n}} \mathbf {E} \langle v, \tilde {X} _ {i} \rangle^ {2} \\ \leq \tau^ {2} n \max  _ {j \in [ 2 m ]} | a _ {j} |. \tag {A.15} \\ \end{array}
$$
Combining Klein and Rio's version of Talagrand's inequality with our estimates (A.10), (A.14) and (A.15), we find for any  $u \in [0, \infty)$  that:
$$
\begin{array}{l} \mathbf {P} \left(\sup  _ {v \in \partial \Sigma_ {X}} \left[ \sum_ {i \in \operatorname {s g n}} \left(\mathbf {E} \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle) - \phi_ {\tau} (\langle v, \tilde {X} _ {i} \rangle)\right) \right] \geq 4 \tau \sqrt {2 n d _ {\times} \max  _ {a \subset \operatorname {s g n}} | a |} + u\right) \\ \leq \exp \left(\frac {- u ^ {2}}{2 v + 6 \tau^ {2} \max  _ {j \in [ 2 m ]} | a _ {j} | u}\right) \tag {A.16} \\ \end{array}
$$
with  $\nu = \tau^2 n\max_{j\in [2m]}|a_j| + 8\sqrt{2nd_X\max_{a\subset sgn}|a|}$
A.2.3 Finishing the proof of Theorem 4.3
Fix a positive constant  $\alpha \in \mathbb{R}$  to be determined later. We now set  $u = \alpha n$ . Let us further fix a positive constant  $\varepsilon \in \mathbb{R}$  and assume that  $n \geq (1 / \varepsilon^2)d_{\mathsf{X}}\max_{j \in [2m]}|a_j|$ . Under these additional hypotheses:
$$
4 \tau \sqrt {2 n d _ {\times} \max  _ {j \in [ 2 m ]} | a _ {j} |} \leq 4 \tau n \sqrt {2} \varepsilon \tag {A.17}
$$
$$
\begin{array}{l} 2 v + 6 \tau^ {2} \max  _ {j \in [ 2 m ]} | a _ {j} | u = 2 \tau^ {2} n \max  _ {j \in [ 2 m ]} | a _ {j} | + 1 6 \sqrt {2 n d _ {\chi} \max  _ {j \in [ 2 m ]} | a _ {j} |} + 6 \tau^ {2} \max  _ {j \in [ 2 m ]} | a _ {j} | u \tag {A.18} \\ \leq \tau^ {2} \max  _ {j \in [ 2 m ]} | a _ {j} | (2 n + 6 \alpha n) + 1 6 \tau n \sqrt {2} \varepsilon \\ \end{array}
$$
We now decouple using Proposition 4.1 and instantiate our upper bound (A.16) to control (A.8). This step combined with the estimates (A.17) and (A.18) yields:
$$
\begin{array}{l} \mathbf {P} \left(\forall v \in \partial \Sigma_ {X}: \frac {1}{n} \sum_ {i = 1} ^ {n} \langle v, X _ {i} \rangle^ {2} \geq \left(1 - \frac {\mathrm {h} ^ {2}}{\tau^ {2}}\right) - 8 \tau \sqrt {2} \varepsilon - 2 \alpha\right) \\ \geq 1 - 2 \exp \left(- \frac {\alpha^ {2} n}{\tau^ {2} \max  _ {j \in [ 2 m ]} | a _ {j} | (2 + 6 \alpha) + 1 6 \tau \sqrt {2} \varepsilon}\right) - \sum_ {i = 2} ^ {2 m - 1} \beta_ {X} \left(a _ {i}\right). \tag {A.19} \\ \end{array}
$$
By choosing  $\alpha$  and  $\varepsilon$  sufficiently small and  $\tau$  sufficiently large, we find that there exists a universal positive constant  $C\in \mathbb{R}$  such that if
$$
n \geq C \max  _ {j \in [ 2 m ]} | a _ {j} | \left(d _ {\mathbf {X}} + \mathsf {h} ^ {2} \log (1 / \delta)\right) \quad \text {a n d} \quad \sum_ {i = 2} ^ {2 m - 1} \beta_ {X} \left(a _ {i}\right) \leq \frac {\delta}{2}
$$
$$
\mathbf {P} \left(\forall v \in \partial \Sigma_ {X}: \frac {1}{n} \sum_ {i = 1} ^ {n} \langle v, X _ {i} \rangle^ {2} \geq \frac {1}{2}\right) \geq 1 - \delta .
$$
The result follows by rescaling to arbitrary  $v\in \mathbb{R}^{d_{\mathrm{x}}}$
A.3 Finishing the of Theorem 3.1
The proof of Theorem 3.1 now easily follows from Theorem 4.3 and Theorem 4.2. We translate the necessary conditions below.
First, we note that in light of Theorem 4.3 if (4.7) holds then we may estimate  $\tilde{\Sigma}_n^{-1} \preceq cI_{d_x}$  for some universal positive constant  $c > 0$ . We remark that (4.7) holds by (3.3) and (3.5). Consequently, it suffices to control the random walk (2.6).
Second, we turn to the control of the random walk (2.6), which is provided by Theorem 4.2, by instantiating it to the case of the Euclidean space  $(\mathbb{R}^{d_{\Upsilon}\times d_{\mathsf{X}}},\| \cdot \| _F)$ . We identify this space by linear isometry (vec) with  $(\mathbb{R}^{d_{\Upsilon}d_{\mathsf{X}}},\| \cdot \| _2)$  and hence in the notation of Theorem 4.2 we have that the weak variances are:
$$
\Lambda_ {O} = \frac {1}{| O |} \left\| \sum_ {i \in [ 2 m ], i \in 2 \mathbb {N} - 1} \mathbf {E} \left[ \operatorname {v e c} \left(\sum_ {j \in a _ {i}} Z _ {j}\right) \operatorname {v e c} \left(\sum_ {j \in a _ {i}} Z _ {j}\right) ^ {\mathsf {T}} \right] \right\| _ {\mathrm {o p}};
$$
$$
\Lambda_ {E} = \frac {1}{| E |} \left\| \sum_ {i \in [ 2 m ], i \in 2 \mathbb {N}} \mathbf {E} \left[ \operatorname {v e c} \left(\sum_ {j \in a _ {i}} Z _ {j}\right) \operatorname {v e c} \left(\sum_ {j \in a _ {i}} Z _ {j}\right) ^ {\mathsf {T}} \right] \right\| _ {\mathrm {o p}}.
$$
Using monotonicity of the operator norm over symmetric positive semidefinite matrices and the condition (3.4) we also have for either  $\mathrm{sgn}\in \{O,E\}$ :
$$
c ^ {\prime} \sigma^ {2} \leq \Lambda_ {\mathrm {s g n}} \leq \sigma^ {2} \tag {A.20}
$$
for some universal positive constant  $c'$ . Condition (3.4) also implies that the constant  $r$  in Theorem 4.2 can be chosen as  $c''\mathrm{edim}(\Sigma)$  for some universal positive constant  $c''$ . The final step is to note that the last two terms on the right hand side of (4.6) are less than  $c'''\delta$  for some third universal positive constant  $c'''$  if the respective second parts of (3.3) and (3.4) hold.