SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Efficient algorithms for adversarial contexts Our algorithm for adversarially-chosen agent contexts suffers exponential-in-  $d$  per-round computational complexity, which makes it unsuitable for use in settings with high-dimensional contexts. Deriving polynomial-time algorithms with sublinear strategic regret for this setting is an exciting (but challenging) direction for future research.
More than two actions Finally, it would be interesting to extend our algorithms for strategic learning under bandit feedback to the setting in which the principal has three or more actions at their disposal. While prior work [29] implies an impossibility result for strategic regret minimization with three or more actions, other (relaxed) notions of optimality (e.g., sublinear Stackelberg regret; recall Definition 2.2) may still be possible.
KH is supported in part by an NDSEG Fellowship. KH and ZSW are supported in part by the NSF FAI Award #1939606. For part of this work, CP was supported by a FODSI postdoctoral fellowship from UC Berkeley. The authors would like to thank the anonymous NeurIPS reviewers for valuable feedback.
[1] Algorithmic hiring: Complex hiring by numbers? URL https://hiring.monster.com/resources/recruiting-strategies/workforce-planning/hiring-algorithms/.
[2] Using white font on a cv to trick ats. URL https://www.thecvstore.net/blog/cv-ats-white-font/#:~text=Even%20if%20you%20get%20past,harm%20your%20future%20employment%20prospects.
[3] Saba Ahmadi, Hedyeh Beyhaghi, Avrim Blum, and Keziah Naggita. The strategic perceptron. In Proceedings of the 22nd ACM Conference on Economics and Computation, pages 6-25, 2021.
[4] Saba Ahmadi, Avrim Blum, and Kunhe Yang. Fundamental bounds on online strategic classification. In Proceedings of the 24th ACM Conference on Economics and Computation, EC 2023, London, United Kingdom, July 9-12, 2023, pages 22-58. ACM, 2023.
[5] András Antos, Gábor Bartók, Dávid Pál, and Csaba Szepesvári. Toward a classification of finite partial-monitoring games. Theoretical Computer Science, 473:77-99, 2013.
[6] Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E Schapire. The nonstochastic multiarmed bandit problem. SIAM journal on computing, 32(1):48-77, 2002.
[7] Maria-Florina Balcan, Avrim Blum, Nika Haghtalab, and Ariel D Procaccia. Commitment without regrets: Online learning in stackelberg security games. In Proceedings of the sixteenth ACM conference on economics and computation, pages 61–78, 2015.
[8] Hamsa Bastani, Mohsen Bayati, and Khashayar Khosravi. Mostly exploration-free algorithms for contextual bandits. Management Science, 67(3):1329-1349, 2021. doi: 10.1287/mnsc.2020.3605. URL https://doi.org/10.1287/mnsc.2020.3605.
[9] Yahav Bechavod, Katrina Ligett, Aaron Roth, Bo Waggoner, and Steven Z Wu. Equal opportunity in online classification with partial feedback. Advances in Neural Information Processing Systems, 32, 2019.
[10] Yahav Bechavod, Katrina Ligett, Steven Wu, and Juba Ziani. Gaming helps! learning from strategic interactions in natural dynamics. In International Conference on Artificial Intelligence and Statistics, pages 1234–1242. PMLR, 2021.
[11] Yahav Bechavod, Chara Podimata, Steven Wu, and Juba Ziani. Information discrepancy in strategic learning. In International Conference on Machine Learning, pages 1691-1715. PMLR, 2022.
[12] Jillian Berman. Do ai-powered lending algorithms silently discriminate? this initiative aims to find out, Nov 2021. URL https://www.marketwatch.com/story/do-ai-powered-lending-algorithms-silently-discriminate-this-initiative\ -aims-to-find-out-11637246524.
[13] Martino Bernasconi, Matteo Castiglioni, Andrea Celli, Alberto Marchesi, Francesco Trovò, and Nicola Gatti. Optimal rates and efficient algorithms for online bayesian persuasion. In International Conference on Machine Learning, pages 2164-2183. PMLR, 2023.
[14] Alberto Bietti, Alekh Agarwal, and John Langford. A contextual bandit bake-off. The Journal of Machine Learning Research, 22(1):5928-5976, 2021.
[15] Avrim Blum, John Hopcroft, and Ravindran Kannan. Foundations of data science. Cambridge University Press, 2020.
[50] Mark Sellke and Aleksandrs Slivkins. The price of incentivizing exploration: A characterization via thompson sampling and sample complexity. In Proceedings of the 22nd ACM Conference on Economics and Computation, pages 795-796, 2021.
[51] Ohad Shamir. A variant of azuma's inequality for martingales with subgaussian tails. arXiv preprint arXiv:1110.2392, 2011.
[52] Yonadav Shavit, Benjamin Edelman, and Brian Axelrod. Causal strategic linear regression. In International Conference on Machine Learning, pages 8676-8686. PMLR, 2020.
[53] Vidyashankar Sivakumar, Zhiwei Steven Wu, and Arindam Banerjee. Structured linear contextual bandits: A sharp and geometric smoothed analysis. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 9026-9035. PMLR, 2020. URL http://proceedings.mlr.press/v119/sivakumar20a.html.
[54] Joel A Tropp. User-friendly tail bounds for sums of random matrices. Foundations of computational mathematics, 12:389-434, 2012.
[55] Geng Zhao, Banghua Zhu, Jiantao Jiao, and Michael Jordan. Online learning in stackelberg games with an omniscient follower. In International Conference on Machine Learning, pages 42304-42316. PMLR, 2023.
[56] Banghua Zhu, Stephen Bates, Zhuoran Yang, Yixin Wang, Jiantao Jiao, and Michael I. Jordan. The sample complexity of online contract design. In Proceedings of the 24th ACM Conference on Economics and Computation, EC 2023, London, United Kingdom, July 9-12, 2023, page 1188. ACM, 2023.
A Useful concentration inequalities
Theorem A.1 (Matrix Azuma, Tropp [54]). Consider a self-adjoint matrix martingale  $\{Y_s : s = 1, \dots, t\}$  in dimension  $d$ , and let  $\{X_s\}_{s \in [t]}$  be the associated difference sequence satisfying  $\mathbb{E}_{s-1} X_s = 0_{d \times d}$  and  $X_s^2 \preceq A_s^2$  for some fixed sequence  $\{A_s\}_{s \in [t]}$  of self-adjoint matrices. Then for all  $\alpha > 0$ ,
$$
\mathbb {P} \left(\lambda_ {\max } \left(Y _ {t} - \mathbb {E} Y _ {t}\right) \geq \alpha\right) \leq d \cdot \exp \left(- \alpha^ {2} / 8 \sigma^ {2}\right),
$$
where  $\sigma^2 \coloneqq \left\| \sum_{s=1}^{t} A_s^2 \right\|_2$ .
Theorem A.2 (A variant of Azuma's inequality for martingales with subgaussian tails, Shamir [51]). Let  $Z_{1}, Z_{2}, \ldots, Z_{t}$  be a martingale difference sequence with respect to a sequence  $W_{1}, W_{2}, \ldots, W_{t}$ , and suppose there are constants  $b > 1$ ,  $c > 0$  such that for any  $s$  and any  $\alpha > 0$ , it holds that
$$
\max  \left\{\mathbb {P} \left(Z _ {s} > \alpha \mid X _ {1}, \dots , X _ {s - 1}\right), \mathbb {P} \left(Z _ {s} <   - \alpha \mid X _ {1}, \dots , X _ {s - 1}\right) \right\} \leq b \cdot \exp \left(- c \alpha^ {2}\right).
$$
Then for any  $\gamma > 0$ , it holds with probability  $1 - \gamma$  that
$$
\sum_ {s = 1} ^ {t} Z _ {s} \leq \sqrt {\frac {2 8 b \log (1 / \gamma)}{c T}}.
$$
Theorem B.1. Let  $f_{U^d} : \mathcal{X} \to \mathbb{R}_{\geq 0}$  denote the density function of the uniform distribution over the  $d$ -dimensional unit sphere. If agent contexts are drawn from a distribution over the  $d$ -dimensional unit sphere with density function  $f : \mathcal{X} \to \mathbb{R}_{\geq 0}$  such that  $\frac{f(\mathbf{x})}{f_{U^d}(\mathbf{x})} \geq c_0 > 0, \forall \mathbf{x} \in \mathcal{X}$ , then Algorithm 1 achieves the following performance guarantee:
$$
\mathrm {R e g} (T) \leq 4 d + \frac {8}{c _ {0} \cdot c _ {1} (\delta , d) \cdot c _ {2} (\delta , d)} \sqrt {1 4 d \sigma^ {2} T \log (4 d T / \gamma)}
$$
with probability  $1 - \gamma$ , where  $0 < c_{1}(\delta, d) \coloneqq \mathbb{P}_{\mathbf{x} \sim U^{d}}(\mathbf{x}[1] \geq \delta)$  and  $0 < c_{2}(\delta, d) \coloneqq \mathbb{E}_{\mathbf{x} \sim U^{d}}[\mathbf{x}[2]^{2} | \mathbf{x}[1] \geq \delta]$ .
Proof. We start from the definition of strategic regret. Note that under apple tasting feedback,  $\pmb{\theta}^{(0)} = \mathbf{0}$ .
$$
\begin{array}{l} \operatorname {R e g} (T) := \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\theta} ^ {(a _ {t} ^ {*})} - \boldsymbol {\theta} ^ {(a _ {t})}, \mathbf {x} _ {t} \right\rangle \\ = \sum_ {t = 1} ^ {T} \left\langle \hat {\boldsymbol {\theta}} _ {t} ^ {(a _ {t} ^ {*})} - \hat {\boldsymbol {\theta}} _ {t} ^ {(a _ {t})}, \mathbf {x} _ {t} \right\rangle + \left\langle \boldsymbol {\theta} ^ {(a _ {t} ^ {*})} - \hat {\boldsymbol {\theta}} _ {t} ^ {(a _ {t} ^ {*})}, \mathbf {x} _ {t} \right\rangle + \left\langle \hat {\boldsymbol {\theta}} _ {t} ^ {(a _ {t})} - \boldsymbol {\theta} ^ {(a _ {t})}, \mathbf {x} _ {t} \right\rangle \\ \leq \sum_ {t = 1} ^ {T} \left| \left\langle \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {t} ^ {(1)}, \mathbf {x} _ {t} \right\rangle \right| + \left| \left\langle \boldsymbol {\theta} ^ {(0)} - \hat {\boldsymbol {\theta}} _ {t} ^ {(0)}, \mathbf {x} _ {t} \right\rangle \right| \\ \leq \sum_ {t = 1} ^ {T} \left\| \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {t} ^ {(1)} \right\| _ {2} \| \mathbf {x} _ {t} \| _ {2} + \left\| \boldsymbol {\theta} ^ {(0)} - \hat {\boldsymbol {\theta}} _ {t} ^ {(0)} \right\| _ {2} \| \mathbf {x} _ {t} \| _ {2} \\ \leq 2 d + \sum_ {t = 2 d + 1} ^ {T} \left\| \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {t} ^ {(1)} \right\| _ {2} + \left\| \boldsymbol {\theta} ^ {(0)} - \hat {\boldsymbol {\theta}} _ {t} ^ {(0)} \right\| _ {2} \\ \end{array}
$$
where the first inequality follows from Lemma B.2, the second inequality follows from the Cauchy-Schwarz, and the third inequality follows from the fact that the instantaneous regret at each time-step is at most 2 and we use the first  $d$  rounds to bootstrap our OLS. The result follows by Lemma B.3, a union bound, and the fact that  $\sum_{d+1}^{T} \sqrt{\frac{1}{t}} \leq 2\sqrt{T}$ .
$$
\left. \left\langle \widehat {\boldsymbol {\theta}} _ {t} ^ {\left(a _ {t} ^ {*}\right)} - \widehat {\boldsymbol {\theta}} _ {t} ^ {\left(a _ {t}\right)}, \mathbf {x} _ {t} \right\rangle \leq 0 \right.
$$
Proof. If  $a_{t} = a_{t}^{*}$ , the condition is satisfied trivially. If  $a_{t} \neq a_{t}^{*}$ , then either (1)  $\langle \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}, \mathbf{x}_t' \rangle - \delta \| \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}\|_2 \geq 0$  and  $\langle \pmb{\theta}^{(1)} - \pmb{\theta}^{(0)} \rangle < 0$  or (2)  $\langle \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}, \mathbf{x}_t' \rangle - \delta \| \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}\|_2 < 0$  and  $\langle \pmb{\theta}^{(1)} - \pmb{\theta}^{(0)} \rangle \geq 0$ .
Case 1:  $\langle \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)},\mathbf{x}_t^\prime \rangle -\delta \| \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}\| _2\geq 0$  and  $\langle \pmb {\theta}^{(1)} - \pmb{\theta}^{(0)}\rangle <  0.$ $a_{t}^{*} = 0,a_{t} = 1)$  By Definition 2.1, we can rewrite
$$
\langle \widehat {\boldsymbol {\theta}} _ {t} ^ {(1)} - \widehat {\boldsymbol {\theta}} _ {t} ^ {(0)}, \mathbf {x} _ {t} ^ {\prime} \rangle - \delta \| \widehat {\boldsymbol {\theta}} _ {t} ^ {(1)} - \widehat {\boldsymbol {\theta}} _ {t} ^ {(0)} \| _ {2} \geq 0
$$
$$
\left\langle \widehat {\boldsymbol {\theta}} _ {t} ^ {(1)} - \widehat {\boldsymbol {\theta}} _ {t} ^ {(0)}, \mathbf {x} _ {t} \right\rangle + \left(\delta^ {\prime} - \delta\right) \| \widehat {\boldsymbol {\theta}} _ {t} ^ {(1)} - \widehat {\boldsymbol {\theta}} _ {t} ^ {(0)} \| _ {2} \geq 0
$$
for some  $\delta' \leq \delta$ . Since  $(\delta' - \delta) \| \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)} \|_2 \leq 0$ ,  $\langle \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}, \mathbf{x}_t \rangle \geq 0$  must hold.
Case 2:  $\langle \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)},\mathbf{x}_t^{\prime}\rangle -\delta \| \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)}\| _2 <   0$  and  $\langle \pmb {\theta}^{(1)} - \pmb{\theta}^{(0)}\rangle \geq 0$ $(a_{t}^{*} = 1,a_{t} = 0)$
Since modification did not help agent  $t$  receive action  $a_{t} = 1$ , we can conclude that  $\langle \widehat{\pmb{\theta}}_t^{(1)} - \widehat{\pmb{\theta}}_t^{(0)},\mathbf{x}_t\rangle < 0$ .
Lemma B.3. Let  $f_{U^d} : \mathcal{X} \to \mathbb{R}_{\geq 0}$  denote the density function of the uniform distribution over the  $d$ -dimensional unit sphere. If  $T \geq d$  and agent contexts are drawn from a distribution over the  $d$ -dimensional unit sphere with density function  $f : \mathcal{X} \to \mathbb{R}_{\geq 0}$  such that  $\frac{f(\mathbf{x})}{f_{U^d}(\mathbf{x})} \geq c_0 \in \mathbb{R}_{>0}$ ,  $\forall \mathbf{x} \in \mathcal{X}$ , then the following guarantee holds under apple tasting feedback.
$$
\| \pmb {\theta} ^ {(1)} - \hat {\pmb {\theta}} _ {t + 1} ^ {(1)} \| _ {2} \leq \frac {2}{c _ {0} \cdot c _ {1} (\delta , d) \cdot c _ {2} (\delta , d)} \sqrt {\frac {1 4 d \sigma^ {2} \log (2 d / \gamma_ {t})}{t}}
$$
with probability  $1 - \gamma_{t}$
Proof. Let  $\mathcal{I}_s^{(1)} = \{\langle \hat{\pmb{\theta}}_s^{(1)},\mathbf{x}_s\rangle \geq \delta \| \hat{\pmb{\theta}}_s^{(1)}\| _2 + r_0\}$ . Then, from the definition of  $\pmb{\theta}_{t + 1}^{(1)}$  we have:
$$
\begin{array}{l} \hat {\boldsymbol {\theta}} _ {t + 1} ^ {(1)} := \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} r _ {s} (1) \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \quad \text {(c l o s e d f o r m s o l u t i o n o f O L S)} \\ = \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \left(\mathbf {x} _ {s} ^ {\top} \boldsymbol {\theta} ^ {(1)} + \epsilon_ {s}\right) \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \quad (\text {p l u g i n} r _ {s} (1)) \\ = \boldsymbol {\theta} ^ {(1)} + \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \epsilon_ {s} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} \\ \end{array}
$$
Re-arranging the above and taking the  $\ell_2$  norm on both sides we get:
$$
\begin{array}{l} \left\| \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {t + 1} ^ {(1)} \right\| _ {2} = \left\| \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \epsilon_ {s} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} \right\| _ {2} \\ \leq \left\| \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\}\right) ^ {- 1} \right\| _ {2} \left\| \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \epsilon_ {s} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right\| _ {2} \quad (\text {C a u c h y - S c h w a r z}) \\ = \frac {\left\| \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \epsilon_ {s} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right\| _ {2}}{\sigma_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\}\right)} \\ = \frac {\left\| \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \epsilon_ {s} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right\| _ {2}}{\lambda_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\}\right)} \\ \end{array}
$$
where for a matrix  $M$ ,  $\sigma_{\mathrm{min}}$  is the smallest singular value  $\sigma_{\mathrm{min}}(M) \coloneqq \min_{\|x\|=1} \|Mx\|$  and  $\lambda_{\mathrm{min}}$  is the smallest eigenvalue. Note that the two are equal since the matrix  $\sum_{s=1}^{t} \mathbf{x}_s \mathbf{x}_s^\top \mathbb{1}\{\mathcal{I}_s^{(1)}\}$  is PSD as the sum of PSD matrices (outer products induce PSD matrices). The final bound is obtained by applying Lemma B.4, Lemma B.5, and a union bound.
Lemma B.4. The following bound holds on the  $\ell_2$ -norm of  $\sum_{s=1}^{t} \mathbf{x}_s \epsilon_s \mathbb{1}\{\mathcal{I}_s^{(1)}\}$  with probability  $1 - \gamma_t$ :
$$
\left\| \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \epsilon_ {s} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right\| _ {2} \leq 2 \sqrt {1 4 d \sigma^ {2} t \log (d / \gamma_ {t})}
$$
Proof. Let  $\mathbf{x}[i]$  denote the  $i$ -th coordinate of a vector  $\mathbf{x}$ . Observe that  $\sum_{s=1}^{t} \epsilon_s \mathbf{x}_s[i] \mathbb{1}\{\mathcal{I}_s^{(1)}\}$  is a sum of martingale differences with  $Z_s := \epsilon_s \mathbf{x}_s[i] \mathbb{1}\{\mathcal{I}_s^{(1)}\}$ ,  $X_s := \sum_{s'=1}^{s} \epsilon_{s'} \mathbf{x}_{s'}[i] \mathbb{1}\{\mathcal{I}_{s'}^{(1)}\}$ , and
$$
\max  \left\{\mathbb {P} \left(Z _ {s} > \alpha \mid X _ {1}, \dots , X _ {s - 1}\right), \mathbb {P} \left(Z _ {s} <   - \alpha \mid X _ {1}, \dots , X _ {s - 1}\right) \right\} \leq \exp \left(- \alpha^ {2} / 2 \sigma^ {2}\right).
$$
By Theorem A.2,
$$
\sum_ {s = 1} ^ {t} \epsilon_ {s} \mathbf {x} _ {s} [ i ] \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \leq 2 \sqrt {1 4 \sigma^ {2} t \log (1 / \gamma_ {i})}
$$
with probability  $1 - \gamma_{i}$ . The desired result follows via a union bound and algebraic manipulation.
Lemma B.5. The following bound holds on the minimum eigenvalue of  $\sum_{s=1}^{t} \mathbf{x}_s \mathbf{x}_s^\top \mathbb{1}\left\{\mathcal{I}_s^{(1)}\right\}$  with probability  $1 - \gamma_t$ :
$$
\lambda_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\}\right) \geq \frac {t}{c _ {0} \cdot c _ {1} (\delta , d) \cdot c _ {2} (\delta , d)} + 4 \sqrt {2 t \log (d / \gamma_ {t})}
$$
$$
\begin{array}{l} \lambda_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \}\right) \\ \geq \lambda_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} - \mathbb {E} _ {s - 1} [ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} ]\right) + \lambda_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbb {E} _ {s - 1} [ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} ]\right) \\ \geq \lambda_ {m i n} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} - \mathbb {E} _ {s - 1} [ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} ]\right) + \sum_ {s = 1} ^ {t} \lambda_ {m i n} \left(\mathbb {E} _ {s - 1} [ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} ]\right) (2) \\ \end{array}
$$
where the inequalities follow from the fact that  $\lambda_{min}(A + B) \geq \lambda_{min}(A) + \lambda_{min}(B)$  for two Hermitian matrices  $A, B$ . Note that the outer products form Hermitian matrices. Let  $Y_{t} := \sum_{s=1}^{t} \mathbf{x}_{s} \mathbf{x}_{s}^{\top} \mathbb{1}\{\mathcal{I}_{s}^{(1)}\} - \mathbb{E}_{s-1}[\mathbf{x}_{s} \mathbf{x}_{s}^{\top} \mathbb{1}\{\mathcal{I}_{s}^{(1)}\}]$ . Note that by the tower rule,  $\mathbb{E}Y_{t} = \mathbb{E}Y_{0} = 0$ . Let  $-X_{s} := \mathbb{E}_{s-1}[\mathbf{x}_{s} \mathbf{x}_{s}^{\top} \mathbb{1}\{\mathcal{I}_{s}^{(1)}\}]$ , then  $\mathbb{E}_{s-1}[-X_{s}] = 0$ , and  $(-X_{s})^{2} \preceq 4I_{d}$ . By Theorem A.1,
$$
\mathbb {P} \left(\lambda_ {\max } \left(- Y _ {t}\right) \geq \alpha\right) \leq d \cdot \exp \left(- \alpha^ {2} / 3 2 t\right).
$$
Since  $-\lambda_{max}(-Y_t) = \lambda_{min}(Y_t)$ ,
$$
\mathbb {P} \left(\lambda_ {\max } \left(Y _ {t}\right) \leq \alpha\right) \leq d \cdot \exp \left(- \alpha^ {2} / 3 2 t\right).
$$
Therefore,  $\lambda_{min}(Y_t) \geq 4\sqrt{2t\log(d / \gamma_t)}$  with probability  $1 - \gamma_t$ . We now turn our attention to lower bounding  $\lambda_{min}(\mathbb{E}_{s-1}[\mathbf{x}_s\mathbf{x}_s^\top \mathbb{1}\{\mathcal{I}_s^{(1)}\}])$ .
$$
\begin{array}{l} \lambda_ {m i n} \left(\mathbb {E} _ {s - 1} \left[ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right]\right) := \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \boldsymbol {\omega} ^ {\top} \mathbb {E} _ {s - 1} \left[ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right] \boldsymbol {\omega} \\ = \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \boldsymbol {\omega} ^ {\top} \left(\int \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} f (\mathbf {x} _ {s}) d \mathbf {x} _ {s}\right) \boldsymbol {\omega} \\ = \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \boldsymbol {\omega} ^ {\top} \left(\int \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} f (\mathbf {x} _ {s}) \cdot \frac {f _ {U ^ {d}} (\mathbf {x} _ {s})}{f (\mathbf {x} _ {s})} d \mathbf {x} _ {s}\right) \boldsymbol {\omega} \\ \geq c _ {0} \cdot \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \boldsymbol {\omega} ^ {\top} \mathbb {E} _ {s - 1, U ^ {d}} [ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{\mathcal {I} _ {s} ^ {(1)} \} ] \boldsymbol {\omega} \\ = c _ {0} \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \mathbb {E} _ {s - 1, U ^ {d}} [ \langle \boldsymbol {\omega}, \mathbf {x} _ {s} \rangle^ {2} | \langle \widehat {\boldsymbol {\beta}} _ {s}, \mathbf {x} _ {s} \rangle \geq \delta \| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2} ] \cdot \underbrace {\mathbb {P} _ {s - 1 , U ^ {d}} (\langle \widehat {\boldsymbol {\beta}} _ {s} , \mathbf {x} _ {s} \rangle \geq \delta \| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2})} _ {c _ {1} (\delta , d)} \\ = c _ {0} \cdot c _ {1} (\delta , d) \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \mathbb {E} _ {s - 1, U ^ {d}} [ \langle \boldsymbol {\omega}, \mathbf {x} _ {s} \rangle^ {2} | \langle \widehat {\boldsymbol {\beta}} _ {s}, \mathbf {x} _ {s} \rangle \geq \delta \| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2} ] \tag {3} \\ \end{array}
$$
Throughout the remainder of the proof, we surpress the dependence on  $U^d$  and note that unless stated otherwise, all expectations are taken with respect to  $U^d$ . Let  $B_s \in \mathbb{R}^{d \times d}$  be the orthonormal matrix such that the first column is  $\widehat{\beta}_s / \| \widehat{\beta}_s\|_2$ . Note that  $B_s e_1 = \widehat{\beta}_s / \| \widehat{\beta}_s\|_2$  and  $B_s \mathbf{x} \sim U^d$ .
$$
\begin{array}{l} \mathbb {E} _ {s - 1} \left[ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} | \langle \widehat {\boldsymbol {\beta}} _ {s}, \mathbf {x} _ {s} \rangle \geq \delta \| \widehat {\boldsymbol {\beta}} _ {s} \| \right] = \mathbb {E} _ {s - 1} \left[ \left(B _ {s} \mathbf {x} _ {s}\right) \left(B _ {s} \mathbf {x} _ {s}\right) ^ {\top} | \langle \widehat {\boldsymbol {\beta}} _ {s}, B _ {s} \mathbf {x} _ {s} \rangle \geq \delta \right] \\ = B _ {s} \mathbb {E} _ {s - 1} [ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} | \mathbf {x} _ {s} ^ {\top} B _ {s} ^ {\top} \| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2} B _ {s} e _ {1} \geq \delta \cdot \| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2} ] B _ {s} ^ {\top} \\ = B _ {s} \mathbb {E} _ {s - 1} \left[ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mid \mathbf {x} _ {s} [ 1 ] \geq \delta \right] B _ {s} ^ {\top} \\ \end{array}
$$
Observe that for  $j \neq 1, i \neq j, \mathbb{E}[\mathbf{x}_s[j]\mathbf{x}_s[i]|\mathbf{x}_s[1] \geq \delta] = 0$ . Therefore,