SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} _ {s - 1} \left[ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \right] \langle \widehat {\beta} _ {s}, \mathbf {x} _ {s} \rangle \geq \delta \| \widehat {\beta} _ {s} \| = B _ {s} (\mathbb {E} [ \mathbf {x} _ {s} [ 2 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta ] I _ {d} \\ + \left(\mathbb {E} \left[ \mathbf {x} _ {s} [ 1 ] ^ {2} \mid \mathbf {x} _ {s} [ 1 ] \geq \delta \right] - \mathbb {E} \left[ \mathbf {x} _ {s} [ 2 ] ^ {2} \mid \mathbf {x} _ {s} [ 1 ] \geq \delta \right]\right) \mathbf {e} _ {1} \mathbf {e} _ {1} ^ {\top}) B _ {s} ^ {\top} \\ = \mathbb {E} [ \mathbf {x} _ {s} [ 2 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta ] I _ {d} \\ + \left(\mathbb {E} [ \mathbf {x} _ {s} [ 1 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta \right] - \mathbb {E} [ \mathbf {x} _ {s} [ 2 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta ]) \frac {\widehat {\boldsymbol {\beta}} _ {s}}{\| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2}} \left(\frac {\widehat {\boldsymbol {\beta}} _ {s}}{\| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2}}\right) ^ {\top} \\ \end{array}
$$
$$
\begin{array}{l} \lambda_ {m i n} \left(\mathbb {E} _ {s - 1} \left[ \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \left\{\mathcal {I} _ {s} ^ {(1)} \right\} \right]\right) \geq c _ {0} \cdot c _ {1} (\delta , d) \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \left(\mathbb {E} \left[ \mathbf {x} _ {s} [ 2 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta \right] \| \boldsymbol {\omega} \| _ {2} \right. \\ \left. + \left(\mathbb {E} [ \mathbf {x} _ {s} [ 1 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta ] - \mathbb {E} [ \mathbf {x} _ {s} [ 2 ] ^ {2} | \mathbf {x} _ {s} [ 1 ] \geq \delta ]\right) \left\langle \omega , \frac {\widehat {\boldsymbol {\beta}} _ {s}}{\| \widehat {\boldsymbol {\beta}} _ {s} \| _ {2}} \right\rangle^ {2}\right) \\ \geq c _ {0} \cdot c _ {1} (\delta , d) \cdot c _ {2} (\delta , d) \\ \end{array}
$$
Lemma B.6. For sufficiently large values of  $d$ ,
$$
c _ {1} (\delta , d) \geq \Theta \left(\frac {(1 - \delta) ^ {d / 2}}{d}\right).
$$
Proof. Lemma B.6 is obtained via a similar argument to Theorem 2.7 in Blum et al. [15]. As in Blum et al. [15], we are interested in the volume of a hyperspherical cap. However, we are interested in a lower-bound, not an upper-bound (as is the case in [15]). Let  $\mathrm{A}$  denote the portion of the  $d$ -dimensional hypersphere with  $\mathbf{x}[1] \geq \frac{\sqrt{c}}{d - 1}$  and let  $\mathrm{H}$  denote the upper hemisphere.
$$
c _ {1} (\delta , d) := \mathbb {P} _ {\mathbf {x} \sim U ^ {d}} (\mathbf {x} [ 1 ] \geq \delta) = \frac {v o l (A)}{v o l (H)}
$$
In order to lower-bound  $c_{1}(\delta, d)$ , it suffices to lower bound  $vol(A)$  and upper-bound  $vol(H)$ . In what follows, let  $V(d)$  denote the volume of the  $d$ -dimensional hypersphere with radius 1.
Lower-bounding  $vol(A)$ : As in [15], to calculate the volume of A, we integrate an incremental volume that is a disk of width  $d\mathbf{x}[1]$  and whose face is a ball of dimension  $d - 1$  and radius  $\sqrt{1 - \mathbf{x}[1]^2}$ . The surface area of the disk is  $(1 - \mathbf{x}[1]^2)^{\frac{d - 1}{2}}V(d - 1)$  and the volume above the slice  $\mathbf{x}[1] \geq \delta$  is
$$
v o l (A) = \int_ {\delta} ^ {1} \left(1 - \mathbf {x} [ 1 ] ^ {2}\right) ^ {\frac {d - 1}{2}} V (d - 1) d \mathbf {x} [ 1 ]
$$
To get a lower bound on the integral, we use the fact that  $1 - x^{2} \geq 1 - x$  for  $x \in [0,1]$ . The integral now takes the form
$$
v o l (A) \geq \int_ {\delta} ^ {1} (1 - \mathbf {x} [ 1 ]) ^ {\frac {d - 1}{2}} V (d - 1) d \mathbf {x} [ 1 ] = \frac {V (d - 1)}{d + 1} \cdot 2 (1 - \delta) ^ {\frac {d + 1}{2}}
$$
Upper-bounding  $vol(H)$ : We can obtain an exact expression for  $vol(H)$  in terms of  $V(d - 1)$  using the recursive relationship between  $V(d)$  and  $V(d - 1)$ :
$$
v o l (H) = \frac {1}{2} V (d) = \frac {\sqrt {\pi}}{2} \frac {\Gamma (\frac {d}{2} + \frac {1}{2})}{\Gamma (\frac {d}{2} + 1)} V (d - 1)
$$
Plugging in our bounds for  $vol(A)$ ,  $vol(H)$  and simplifying, we see that
$$
c _ {1} (\delta , d) \geq \frac {(1 - \delta) ^ {\frac {d + 1}{2}}}{\sqrt {\pi} (d + 1)} \frac {\Gamma (\frac {d}{2} + 1)}{\Gamma (\frac {d}{2} + \frac {1}{2})} = \Theta \left(\frac {(1 - \delta) ^ {\frac {d + 1}{2}}}{d + 1}\right)
$$
where the equality follows from Stirling's approximation.
Lemma B.7. The following bound holds on  $c_{2}(\delta, d)$ :
$$
c _ {2} (\delta , d) \geq \frac {1}{3 d} \left(\frac {3}{4} - \frac {1}{2} \delta - \frac {1}{4} \delta^ {2}\right) ^ {3}.
$$
Proof. We begin by computing  $\mathbb{E}[\mathbf{x}[2]^2 |\mathbf{x}[1] = \delta ']$ , for  $\delta^{\prime}\in (0,1)$ . If  $\mathbf{x}[1] = \delta '$ , then  $\mathbf{x}[2]^2 +\ldots +$ $\mathbf{x}[d]^2\leq 1 - (\delta ')^2$ . Using this fact, we see that
$$
\mathbb {E} [ \mathbf {x} [ 2 ] ^ {2} | \mathbf {x} [ 1 ] = \delta^ {\prime} ] = \frac {1}{d} \mathbb {E} _ {r \sim U n i f [ 0, 1 - (\delta^ {\prime}) ^ {2} ]} [ r ^ {2} ] = \frac {1}{3 d} (1 - (\delta^ {\prime}) ^ {2}) ^ {3}.
$$
Since  $\mathbb{E}[\mathbf{x}[2]^2 |\mathbf{x}[1]\geq \delta ]\geq \mathbb{E}[\mathbf{x}[2]^2 |\mathbf{x}[1] = \delta +\frac{1 - \delta}{2} ],$
$$
\mathbb {E} [ \mathbf {x} [ 2 ] ^ {2} | \mathbf {x} [ 1 ] \geq \delta ] \geq \mathbb {E} \left[ \mathbf {x} [ 2 ] ^ {2} | \mathbf {x} [ 1 ] = \frac {\delta + 1}{2} \right] = \frac {1}{3 d} \left(\frac {3}{4} - \frac {1}{2} \delta - \frac {1}{4} \delta^ {2}\right) ^ {3}
$$
B.2 Proof of Proposition 3.4
Proposition B.8. For any sequence of linear threshold policies  $\beta_{1},\ldots ,\beta_{T}$
$$
\mathbb {E} _ {\mathbf {x} _ {1}, \dots , \mathbf {x} _ {T} \sim U ^ {d}} \left[ \sum_ {t = 1} ^ {T} \mathbb {1} \{\langle \mathbf {x} _ {t}, \boldsymbol {\beta} _ {t} \rangle \geq \delta \| \boldsymbol {\beta} _ {t} \| _ {2} \} \right] = T \cdot \mathbb {P} _ {\mathbf {x} \sim U ^ {d}} (\mathbf {x} [ 1 ] \geq \delta)
$$
Proof. Let  $B_t \in \mathbb{R}^{d \times d}$  be the orthonormal matrix such that the first column is  $\beta_t / \| \beta_t \|_2$ . Note that  $B_t \mathbf{x} \sim U^d$  if  $\mathbf{x} \sim U^d$  and  $B_t e_1 = \beta_t / \| \beta_t \|_2$ .
$$
\begin{array}{l} \mathbb {E} _ {\mathbf {x} _ {1}, \dots , \mathbf {x} _ {T} \sim U ^ {d}} [ \sum_ {t = 1} ^ {T} \mathbb {1} \{\langle \mathbf {x} _ {t}, \boldsymbol {\beta} _ {t} \rangle \geq \delta \| \boldsymbol {\beta} _ {t} \| _ {2} \} ] = \sum_ {t = 1} ^ {T} \mathbb {P} _ {\mathbf {x} _ {t} \sim U ^ {d}} (\langle \mathbf {x} _ {t}, \boldsymbol {\beta} _ {t} \rangle \geq \delta \| \boldsymbol {\beta} _ {t} \| _ {2}) \\ = \sum_ {t = 1} ^ {T} \mathbb {P} _ {\mathbf {x} _ {t} \sim U ^ {d}} \left(\left\langle B _ {t} \mathbf {x} _ {t}, \boldsymbol {\beta} _ {t} \right\rangle \geq \delta \| \boldsymbol {\beta} _ {t} \| _ {2}\right) \\ = \sum_ {t = 1} ^ {T} \mathbb {P} _ {\mathbf {x} _ {t} \sim U ^ {d}} \left(\mathbf {x} _ {t} ^ {\top} B _ {t} ^ {\top} \| \boldsymbol {\beta} _ {t} \| _ {2} B _ {t} e _ {1} \geq \delta \| \boldsymbol {\beta} _ {t} \| _ {2}\right) \\ = \sum_ {t = 1} ^ {T} \mathbb {P} _ {\mathbf {x} _ {t} \sim U ^ {d}} \left(\mathbf {x} _ {t} ^ {\top} I _ {d} e _ {1} \geq \delta \| _ {2}\right) \\ = T \cdot \mathbb {P} _ {\mathbf {x} \sim U ^ {d}} (\mathbf {x} [ 1 ] \geq \delta \| _ {2}) \\ \end{array}
$$
B.3 Explore-Then-Commit Analysis
Theorem B.9. Let  $f_{U^d} : \mathcal{X} \to \mathbb{R}_{\geq 0}$  denote the density function of the uniform distribution over the  $d$ -dimensional unit sphere. If agent contexts are drawn from a distribution over the  $d$ -dimensional unit sphere with density function  $f : \mathcal{X} \to \mathbb{R}_{\geq 0}$  such that  $\frac{f(\mathbf{x})}{f_U(\mathbf{x})} \geq c_0 > 0, \forall \mathbf{x} \in \mathcal{X}$ , then Algorithm 2 achieves the following performance guarantee
$$
\operatorname {R e g} _ {\mathrm {E T C}} (T) \leq \frac {8 \cdot 6 3 ^ {1 / 3}}{c _ {0}} d \sigma^ {2 / 3} T ^ {2 / 3} \log^ {1 / 3} (4 d / \gamma)
$$
with probability  $1 - \gamma$  if  $T_0\coloneqq 4\cdot 63^{1 / 3}\sigma^{2 / 3}dT^{2 / 3}\log^{1 / 3}(4d / \gamma)$
$$
\begin{array}{l} \operatorname {R e g} _ {\mathrm {E T C}} (T) := \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\theta} ^ {\left(a _ {t} ^ {*}\right)} - \boldsymbol {\theta} ^ {\left(a _ {t}\right)}, \mathbf {x} _ {t} \right\rangle \\ \leq T _ {0} + \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\theta} ^ {\left(a _ {t} ^ {*}\right)} - \boldsymbol {\theta} ^ {\left(a _ {t}\right)}, \mathbf {x} _ {t} \right\rangle \\ = T _ {0} + \sum_ {t = T _ {0} + 1} ^ {T} \left\langle \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(a _ {t} ^ {*})} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(a _ {t})}, \mathbf {x} _ {t} \right\rangle + \left\langle \boldsymbol {\theta} ^ {(a _ {t} ^ {*})} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(a _ {t} ^ {*})}, \mathbf {x} _ {t} \right\rangle + \left\langle \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(a _ {t})} - \boldsymbol {\theta} ^ {(a _ {t})}, \mathbf {x} _ {t} \right\rangle \\ \leq T _ {0} + \sum_ {t = T _ {0} + 1} ^ {T} | \langle \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(1)}, \mathbf {x} _ {t} \rangle | + | \langle \boldsymbol {\theta} ^ {(0)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(0)}, \mathbf {x} _ {t} \rangle | \\ \leq T _ {0} + \sum_ {t = T _ {0} + 1} ^ {T} \| \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(1)} \| _ {2} \| \mathbf {x} _ {t} \| _ {2} + \| \boldsymbol {\theta} ^ {(0)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(0)} \| _ {2} \| \mathbf {x} _ {t} \| _ {2} \\ \leq T _ {0} + T \cdot \| \boldsymbol {\theta} ^ {(1)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(1)} \| _ {2} + T \cdot \| \boldsymbol {\theta} ^ {(0)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(0)} \| _ {2} \\ \leq T _ {0} + T \cdot \frac {2 4 d}{c _ {0}} \sqrt {\frac {7 d \sigma^ {2} \log (4 d / \gamma)}{T _ {0}}} \\ \end{array}
$$
with probability  $1 - \gamma$ , where the last inequality follows from Lemma B.10 and a union bound. The result follows from picking  $T_0 = 4 \cdot 63^{1/3} d\sigma^{2/3} T^{2/3} \log^{1/3}(4d / \gamma)$ .
Lemma B.10. Let  $f_{U^d} : \mathcal{X} \to \mathbb{R}_{\geq 0}$  denote the density function of the uniform distribution over the  $d$ -dimensional unit sphere. If  $T \geq 2d$  and agent contexts are drawn from a distribution over the
$d$ -dimensional unit sphere with density function  $f: \mathcal{X} \to \mathbb{R}_{\geq 0}$  such that  $\frac{f(\mathbf{x})}{f_U(\mathbf{x})} \geq c_0 > 0$ ,  $\forall \mathbf{x} \in \mathcal{X}$ , then the following guarantee holds for  $a \in \{0,1\}$ .
$$
\| \pmb {\theta} ^ {(a)} - \hat {\pmb {\theta}} _ {t + 1} ^ {(a)} \| _ {2} \leq \frac {1 2 d}{c _ {0}} \sqrt {\frac {7 d \sigma^ {2} \log (2 d / \gamma_ {t})}{T _ {0}}}
$$
$$
\begin{array}{l} \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(a)} := \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) ^ {- 1} \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} r _ {s + k} ^ {(a)} \\ = \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) ^ {- 1} \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \left(\mathbf {x} _ {s + k} ^ {\top} \boldsymbol {\theta} ^ {(a)} + \epsilon_ {s + k}\right) \\ = \pmb {\theta} ^ {(a)} + \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) ^ {- 1} \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \epsilon_ {s + k} \\ \end{array}
$$
where  $k = 0$  if  $a = 0$  and  $k = T_0$  if  $a = 1$ . Therefore,
$$
\begin{array}{l} \left\| \boldsymbol {\theta} ^ {(a)} - \hat {\boldsymbol {\theta}} _ {T _ {0} / 2} ^ {(a)} \right\| _ {2} = \left\| \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) ^ {- 1} \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \epsilon_ {s + k} \right\| _ {2} \\ \leq \left\| \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) ^ {- 1} \right\| _ {2} \left\| \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \epsilon_ {s + k} \right\| _ {2} \\ = \frac {\left\| \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \boldsymbol {\epsilon} _ {s + k} \right\| _ {2}}{\sigma_ {m i n} (\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top})} \\ = \frac {\left\| \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \boldsymbol {\epsilon} _ {s + k} \right\| _ {2}}{\lambda_ {m i n} \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right)} \\ \end{array}
$$
The desired result is obtained by applying Lemma B.11, Lemma B.12, and a union bound.
Lemma B.11. The following bound holds on the  $\ell_2$ -norm of  $\sum_{s=1}^{T_0/2} \mathbf{x}_{s+k} \epsilon_{s+k}$  with probability  $1 - \gamma$ :
$$
\left\| \sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \epsilon_ {s + k} \right\| _ {2} \leq 2 \sqrt {7 d \sigma^ {2} T _ {0} \log (d / \gamma)}
$$
Proof. Observe that  $\sum_{s=1}^{T_0/2} \epsilon_{k+s} \mathbf{x}_{k+s}[i]$  is a sum of martingale differences with  $Z_{k+s} := \epsilon_{k+s} \mathbf{x}_{k+s}[i]$ ,  $X_{k+s} := \sum_{s'=1}^{s} \epsilon_{k+s'} \mathbf{x}_{k+s'}[i]$ , and
$\max \{\mathbb{P}(Z_{k + s} > \alpha |X_{k + 1},\ldots ,X_{k + s - 1}),\mathbb{P}(Z_{k + s} < - \alpha |X_{k + 1},\ldots ,X_{k + s - 1})\} \leq \cdot \exp (-\alpha^2 /2\sigma^2).$
$$
\sum_ {s = 1} ^ {T _ {0} / 2} \epsilon_ {k + s} \mathbf {x} _ {k + s} [ i ] \leq 2 \sqrt {7 \sigma^ {2} T _ {0} \log (1 / \gamma_ {i})}
$$
Lemma B.12. The following bound holds on the minimum eigenvalue of  $\sum_{s=1}^{T_0/2} \mathbf{x}_{s+k} \mathbf{x}_{s+k}^\top$  with probability  $1 - \gamma$ :
$$
\lambda_ {m i n} \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) \geq \frac {T _ {0}}{6 d} + 4 \sqrt {T _ {0} \log (d / \gamma)}
$$
$$
\begin{array}{l} \lambda_ {m i n} \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top}\right) \geq \lambda_ {m i n} \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} - \mathbb {E} [ \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} ]\right) + \lambda_ {m i n} \left(\sum_ {s = 1} ^ {T _ {0} / 2} \mathbb {E} [ \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} ]\right) \\ \geq \lambda_ {m i n} (\sum_ {s = 1} ^ {T _ {0} / 2} \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} - \mathbb {E} [ \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} ]) + \sum_ {s = 1} ^ {T _ {0} / 2} \lambda_ {m i n} (\mathbb {E} [ \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} ]) \\ \end{array}
$$
where the inequalities follow from the fact that  $\lambda_{min}(A + B) \geq \lambda_{min}(A) + \lambda_{min}(B)$  for two Hermitian matrices  $A, B$ . Let  $Y_{T_0/2} \coloneqq \sum_{s=1}^{T_0/2} \mathbf{x}_{s+k} \mathbf{x}_{s+k}^\top - \mathbb{E}[\mathbf{x}_{s+k} \mathbf{x}_{s+k}^\top]$ . Note that  $\mathbb{E}Y_{T_0/2} = \mathbb{E}Y_0 = 0$ ,  $-X_{s+k} \coloneqq \mathbb{E}[\mathbf{x}_{s+k} \mathbf{x}_{s+k}^\top]$ ,  $\mathbb{E}[-X_{s+k}] = 0$ , and  $(-X_{s+k})^2 \preceq 4I_d$ . By Theorem A.1,
$$
\mathbb {P} \left(\lambda_ {\max } \left(- Y _ {T _ {0} / 2}\right) \geq \alpha\right) \leq d \cdot \exp \left(- \alpha^ {2} / 1 6 T _ {0}\right).
$$
Since  $-\lambda_{max}(-Y_{T_0 / 2}) = \lambda_{min}(Y_{T_0 / 2}),$
$$
\mathbb {P} \left(\lambda_ {\max } \left(Y _ {T _ {0} / 2}\right) \leq \alpha\right) \leq d \cdot \exp \left(- \alpha^ {2} / 1 6 T _ {0}\right).
$$
Therefore,  $\lambda_{min}(Y_{T_0 / 2})\geq 4\sqrt{T_0\log(d / \gamma)}$  with probability  $1 - \gamma$ . We now turn our attention to lower bounding  $\lambda_{min}(\mathbb{E}[\mathbf{x}_{s + k}\mathbf{x}_{s + k}^{\top}])$ .
$$
\begin{array}{l} \lambda_ {m i n} \left(\mathbb {E} \left[ \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} \right]\right) := \min  _ {\boldsymbol {\omega} \in S ^ {d - 1}} \boldsymbol {\omega} ^ {\top} \mathbb {E} \left[ \mathbf {x} _ {s + k} \mathbf {x} _ {s + k} ^ {\top} \right] \boldsymbol {\omega} \\ = \min  _ {\omega \in S ^ {d - 1}} \omega^ {\top} \frac {1}{3 d} I _ {d} \omega \\ = \frac {1}{3 d} \\ \end{array}
$$
Theorem B.13. Let  $\operatorname{Reg}_{\mathrm{OLS}}(T)$  be the strategic regret of Algorithm 1 and  $\operatorname{Reg}_{\mathrm{ETC}}(T)$  be the strategic regret of Algorithm 2. The expected strategic regret of Algorithm 3 is
$$
\mathbb {E} [ \operatorname {R e g} (T) ] \leq 4 \cdot \min  \{\mathbb {E} [ \operatorname {R e g} _ {\mathrm {O L S}} (T) ], \mathbb {E} [ \operatorname {R e g} _ {\mathrm {E T C}} (T) ] \}
$$
Proof. Case 1:  $T < \tau^{*}$  From Theorem B.9, we know that
$$
\operatorname {R e g} _ {\mathrm {E T C}} (\tau_ {i}) \leq \frac {8 \cdot 6 3 ^ {1 / 3}}{c _ {0}} d \sigma^ {2 / 3} \tau_ {i} ^ {2 / 3} \log^ {1 / 3} (4 d \tau_ {i} ^ {2})
$$
with probability  $1 - 1 / \tau_i^2$ . Therefore,
$$
\mathbb {E} [ \mathrm {R e g} _ {\mathrm {E T C}} (\tau_ {i}) ] \leq \frac {8 \cdot 6 3 ^ {1 / 3}}{c _ {0}} d \sigma^ {2 / 3} \tau_ {i} ^ {2 / 3} \log^ {1 / 3} (4 d \tau_ {i} ^ {2}) + \frac {2}{\tau_ {i}}
$$
Observe that  $\sum_{j=1}^{i-1} \mathbb{E}[\mathrm{Reg}_{\mathrm{ETC}}(\tau_j)] \leq \mathbb{E}[\mathrm{Reg}_{\mathrm{ETC}}(\tau_i)]$ . Suppose  $\tau_{i-1} \leq T \leq \tau_i$  for some  $i$ . Under such a scenario,
$$
\begin{array}{l} \mathbb {E} [ \operatorname {R e g} (T) ] \leq 2 \mathbb {E} [ \operatorname {R e g} _ {\mathrm {E T C}} (\tau_ {i}) ] \\ \leq 2 \mathbb {E} [ \operatorname {R e g} _ {\mathrm {E T C}} (2 T) ] \\ \leq 4 \mathbb {E} [ \operatorname {R e g} _ {\mathrm {E T C}} (T) ] \\ \end{array}
$$
Case 2:  $T \geq \tau^{*}$  Let  $t^{*}$  denote the actual switching time of Algorithm 3.
$$
\operatorname {R e g} (T) := \sum_ {t = 1} ^ {t ^ {*}} \left\langle \boldsymbol {\theta} ^ {\left(a _ {t} ^ {*}\right)} - \boldsymbol {\theta} ^ {\left(a _ {t}\right)}, \mathbf {x} _ {t} \right\rangle + \sum_ {t = t ^ {*} + 1} ^ {T} \left\langle \boldsymbol {\theta} ^ {\left(a _ {t} ^ {*}\right)} - \boldsymbol {\theta} ^ {\left(a _ {t}\right)}, \mathbf {x} _ {t} \right\rangle
$$
$$
\begin{array}{l} \mathbb {E} [ \operatorname {R e g} (T) ] \leq 2 \cdot \mathbb {E} [ \operatorname {R e g} _ {\mathrm {E T C}} (t ^ {*}) ] + \mathbb {E} [ \operatorname {R e g} _ {\mathrm {O L S}} (T - t ^ {*}) ] \\ \leq 2 \cdot \mathbb {E} [ \operatorname {R e g} _ {\mathrm {O L S}} (t ^ {*}) ] + \mathbb {E} [ \operatorname {R e g} _ {\mathrm {O L S}} (T) ] \\ \leq 2 \cdot \mathbb {E} \left[ \operatorname {R e g} _ {\mathrm {O L S}} \left(\tau^ {*}\right) \right] + \mathbb {E} \left[ \operatorname {R e g} _ {\mathrm {O L S}} (T) \right] \\ \leq 3 \cdot \mathbb {E} [ \operatorname {R e g} _ {\mathrm {O L S}} (T) ] \\ \end{array}
$$
where the first line follows from case 1, the second line follows from the fact that  $t^* \leq \tau^*$  (and so  $\mathbb{E}[\mathrm{Reg}_{\mathrm{ETC}}(t^{*})] \leq \mathbb{E}[\mathrm{Reg}_{\mathrm{OLS}}(t^{*})]$ ), the third line follows from the fact that  $t^* \leq \tau^*$ , and the fourth line follows from the fact that  $T \geq \tau^*$ .
B.5 Inconsistency of OLS when using all data
Theorem B.14.  $\begin{array}{rl}\lim_{t\to \infty}\hat{\pmb{\theta}}_{t + 1}^{(1)} & = \pmb{\theta}_{*}^{(1)} \end{array}$  if and only if  $\begin{array}{r}\lim_{t\to \infty}\sum_{s = 1}^{t}\mathbf{x}_s'\mathbf{x}_s'\top \mathbb{1}\{a_s = 1\} \end{array}$
$$
\begin{array}{l} \lim  _ {t \rightarrow \infty} \hat {\boldsymbol {\theta}} _ {t + 1} ^ {(1)} := \lim  _ {t \rightarrow \infty} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\prime \top} \mathbb {1} \{a _ {s} = 1 \}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} r _ {s} ^ {(1)} \mathbb {1} \{a _ {s} = 1 \} \\ = \lim  _ {t \rightarrow \infty} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\prime \top} \mathbb {1} \left\{a _ {s} = 1 \right\}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \left(\mathbf {x} _ {s} ^ {\top} \boldsymbol {\theta} _ {*} ^ {(1)} + \epsilon_ {s}\right) \mathbb {1} \left\{a _ {s} = 1 \right\} \\ = \lim  _ {t \rightarrow \infty} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\prime \top} \mathbb {1} \left\{a _ {s} = 1 \right\}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\top} \boldsymbol {\theta} _ {*} ^ {(1)} \mathbb {1} \left\{a _ {s} = 1 \right\} \\ + \lim  _ {t \rightarrow \infty} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\prime \top} \mathbb {1} \{a _ {s} = 1 \}\right) ^ {- 1} \sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \epsilon_ {s} \mathbb {1} \{a _ {s} = 1 \} \\ = \lim  _ {t \rightarrow \infty} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\prime \top} \mathbb {1} \{a _ {s} = 1 \}\right) ^ {- 1} \left(\sum_ {s = 1} ^ {t} \mathbf {x} _ {s} ^ {\prime} \mathbf {x} _ {s} ^ {\top} \mathbb {1} \{a _ {s} = 1 \}\right) \boldsymbol {\theta} _ {*} ^ {(1)} \\ \end{array}
$$
Theorem C.1. Algorithm 4 with  $\eta = \sqrt{\frac{\log(|\mathcal{E}|)}{T\lambda^2|\mathcal{E}|}}$ ,  $\gamma = 2\eta \lambda |\mathcal{E}|$ , and  $\varepsilon = \left(\frac{d\sigma\log T}{T}\right)^{1 / (d + 2)}$  incurs expected strategic regret:
$$
\mathbb {E} [ \operatorname {R e g} (T) ] \leq 6 T ^ {(d + 1) / (d + 2)} (d \sigma \log T) ^ {1 / (d + 2)} = \widetilde {\mathcal {O}} \left(T ^ {(d + 1) / (d + 2)}\right).
$$
Proof. Let  $a_{t,e}$  correspond to the action chosen by a grid point  $e \in \mathcal{E}$ . We simplify notation to  $a_{t} = a_{t,e_{t}}$  to be the action chosen by the sampled grid point  $e_{t}$  at round  $t$ . For the purposes of the analysis, we also define  $\ell_t(e) = 1 + \lambda - r_t(a_{t,e_t})$ .
We first analyze the difference between the loss of the algorithm and the best-fixed point on the grid  $e^{*}$ , i.e.,
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {R e g} ^ {\varepsilon} (T) \right] = \max  _ {e ^ {*} \in \mathcal {E}} \mathbb {E} \left[ \sum_ {t \in [ T ]} r _ {t} \left(a _ {t, e ^ {*}}\right) \right] - \mathbb {E} \left[ \sum_ {t \in [ T ]} r _ {t} \left(a _ {t}\right) \right] \\ = \mathbb {E} \left[ \sum_ {t \in [ T ]} \ell_ {t} \left(e _ {t}\right) \right] - \min  _ {e ^ {*} \in \mathcal {E}} \mathbb {E} \left[ \sum_ {t \in [ T ]} \ell_ {t} \left(e ^ {*}\right) \right] \\ \end{array}
$$
where the equivalence between working with  $\ell_t(\cdot)$  as opposed to  $r_t(\cdot)$  holds because  $\ell_t(\cdot)$  are just a shift from  $r_t(\cdot)$  that is the same across all rounds and experts. For the regret of the algorithm, we show that:
$$
\mathbb {E} \left[ \operatorname {R e g} ^ {\varepsilon} (T) \right] = O \left(T \cdot d \cdot \left(\frac {1}{\varepsilon}\right) ^ {2 d} \cdot \log \left(\frac {1}{\varepsilon}\right)\right) \tag {4}
$$
We define the "good" event as the event that the reward is in  $[0,1]$  for every round  $t$ :  $\mathcal{C} = \{r_t \in [0,1], \forall t \in [T]\}$ . Note that this depends on the noise of the round  $\varepsilon_t$ . We will call the complement of the "good" event, the "bad" event  $\neg \mathcal{C}$ . The regret of the algorithm depends on both  $\mathcal{C}$  and  $\neg \mathcal{C}$  as follows:
$$
\mathbb {E} \left[ \operatorname {R e g} ^ {\varepsilon} (T) \right] = \mathbb {E} \left[ \operatorname {R e g} ^ {\varepsilon} (T) | \mathcal {C} \right] \cdot \Pr [ \mathcal {C} ] + \mathbb {E} \left[ \operatorname {R e g} ^ {\varepsilon} (T) | \neg \mathcal {C} \right] \cdot \Pr [ \neg \mathcal {C} ] \leq \mathbb {E} \left[ \operatorname {R e g} ^ {\varepsilon} (T) | \mathcal {C} \right] + T \cdot \Pr [ \neg \mathcal {C} ] \tag {5}
$$
where the inequality is due to the fact that  $\operatorname*{Pr}[\mathcal{C}] \leq 1$  and that in the worst case, the algorithm must pick up a loss of 1 at each round.
We now upper bound the probability with which the bad event happens.