SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where we write  $C = \max_{t\geq 1}\frac{2}{|\frac{1}{4e\sqrt{\pi}} - \frac{1}{t^2}|} + 1$ . The first inequality is due to Lemma A.2, the second inequality is due to Lemma A.6, and the last inequality follows from  $\Delta_t\leq 2$  for all  $t\in [T]$ . To alleviate the burden on notation, we write  $g_{t} = \min \{2,(2\sqrt{d\log t} +1)C\widehat{\beta}_{t}\| \mathbf{x}_{t}\|_{\mathbf{V}_{t}^{-1}}\}$  and an index set  $\mathfrak{T} = \{t\in [T]:\| \mathbf{x}_t / \overline{\sigma}_t\|_{\mathbf{V}_t^{-1}}\geq 1\}$ . It follows that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} g _ {t} = \sum_ {t = 1} ^ {T} \min  \left\{2, (2 \sqrt {d \log t} + 1) C \widehat {\beta} _ {t} \bar {\sigma} _ {t} \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\} \\ = \sum_ {t \in \mathcal {T}} \min  \left\{2, \left(2 \sqrt {d \log t} + 1\right) C \widehat {\beta} _ {t} \bar {\sigma} _ {t} \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\} \\ + \sum_ {t \notin \mathcal {T}} \min  \left\{2, (2 \sqrt {d \log t} + 1) C \widehat {\beta} _ {t} \overline {{\sigma}} _ {t} \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\} \\ \leq 2 | \mathfrak {T} | + \sum_ {t \notin \mathfrak {T}} (2 \sqrt {d \log t} + 1) C \widehat {\beta} _ {t} \overline {{\sigma}} _ {t} \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \\ = 2 \left| \mathcal {T} \right| + 3 C \sum_ {t \notin \mathcal {T}} \widehat {\beta} _ {t} \sqrt {d \log t} \bar {\sigma} _ {t} \min  \{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} \\ \leq 4 d \log \left(1 + T / \left(d \lambda \alpha^ {2}\right)\right) + 3 C \sum_ {t \notin \mathcal {T}} \widehat {\beta} _ {t} \sqrt {d \log t} \bar {\sigma} _ {t} \min  \left\{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\}, \tag {A.7} \\ \end{array}
$$
where the first inequality follows from the definition of  $g_{t}$ , the last equality is due to  $\| \mathbf{x}_t / \overline{\sigma}_t\|_{\mathbf{V}_t^{-1}} < 1$  for all  $t \notin \mathcal{T}$ , and the second inequality is due to
$$
\begin{array}{l} | \mathfrak {T} | \leq \sum_ {t \in \mathfrak {T}} \min  \{1, \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} ^ {2} \} \\ \leq \sum_ {t = 1} ^ {T} \min  \left\{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} ^ {2} \right\} \\ \leq 2 d \log (1 + T / (d \lambda \alpha^ {2})). \\ \end{array}
$$
The last inequality is due to Lemma C.2 and  $\| \mathbf{x}_t / \overline{\sigma}_t\| \leq 1 / \alpha$ . Further, we decompose  $\overline{\mathcal{T}} = [T]\backslash \mathcal{T} = \overline{\mathcal{T}}_1\cup \overline{\mathcal{T}}_2$  where
$$
\overline {{\mathcal {T}}} _ {1} = \left\{t \in \overline {{\mathcal {T}}} : \overline {{\sigma}} _ {t} = \sigma_ {t} \text {o r} \overline {{\sigma}} _ {t} = \alpha \right\}, \qquad \overline {{\mathcal {T}}} _ {1} = \left\{t \in \overline {{\mathcal {T}}} : \overline {{\sigma}} _ {t} = \gamma \| \mathbf {x} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} ^ {1 / 2} \right\}.
$$
The second term in (A.7) can be controlled through a decomposition
$$
\begin{array}{l} \sum_ {t \notin \mathfrak {T}} \widehat {\beta} _ {t} \sqrt {d \log t} \bar {\sigma} _ {t} \min  \{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} = \sum_ {t \in \overline {{\mathfrak {T}}} _ {1}} \widehat {\beta} _ {t} \sqrt {d \log t} \bar {\sigma} _ {t} \min  \{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} \\ + \sum_ {t \in \overline {{\mathcal {T}}} _ {2}} \widehat {\beta} _ {t} \sqrt {d \log t} \bar {\sigma} _ {t} \min  \{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \}, \\ \end{array}
$$
For  $t\in \overline{\mathcal{T}}_1$  , we have
$$
\begin{array}{l} \sum_ {t \in \overline {{\mathcal {T}}} _ {1}} \widehat {\beta} _ {t} \sqrt {d \log t} \bar {\sigma} _ {t} \min  \{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} \leq \sum_ {t \in \overline {{\mathcal {T}}} _ {1}} \widehat {\beta} _ {t} \sqrt {d \log t} (\sigma_ {t} + \alpha) \min  \{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} \\ \leq \sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} \sqrt {d \log t} \left(\sigma_ {t} + \alpha\right) \min  \left\{1, \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\} \\ \leq \sqrt {2 d \sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} \left(\sigma_ {t} ^ {2} + \alpha^ {2}\right) \log T} \sqrt {\sum_ {t = 1} ^ {T} \min  \left\{1 , \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} ^ {2} \right\}} \\ \leq 2 \sqrt {d \sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} \left(\sigma_ {t} ^ {2} + \alpha^ {2}\right) \log T} \sqrt {d \log \left(1 + T / \left(d \lambda \alpha^ {2}\right)\right)}, \\ \end{array}
$$
where the first inequality is due to Cauchy-Schwarz inequality, and the second inequality is due to Lemma C.2 and  $\| \mathbf{x}_t / \overline{\sigma}_t\| \leq 1 / \alpha$ . For  $t\in \overline{\mathbb{T}}_2$ , we have  $\overline{\sigma}_t = \gamma^2\| \mathbf{x}_t / \overline{\sigma}_t\|_{\mathbf{V}_t^{-1}}$ , and then
$$
\begin{array}{l} \sum_ {t \in \overline {{\mathcal {T}}} _ {2}} \widehat {\beta} _ {t} \sqrt {d \log t} \overline {{\sigma}} _ {t} \min \{1, \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} = \sum_ {t \in \overline {{\mathcal {T}}} _ {2}} \widehat {\beta} _ {t} \sqrt {d \log t} \overline {{\sigma}} _ {t} \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \\ \leq \gamma^ {2} \sum_ {t \in \overline {{\mathbf {J}}} _ {2}} \widehat {\beta} _ {t} \sqrt {d \log t} \| \mathbf {x} _ {t} / \overline {{\sigma}} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} ^ {2} \\ \leq \gamma^ {2} \sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} \sqrt {d \log t} \min  \left\{1, \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} ^ {2} \right\} \\ \leq 2 \gamma^ {2} d \sqrt {d \log T} \log (1 + T / (d \lambda \alpha^ {2})) (\max  _ {t \in [ T ]} \widehat {\beta} _ {t}). \\ \end{array}
$$
Hence, combining the inequalities above, we have
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} g _ {t} \leq 4 d \log (1 + T / (d \lambda \alpha^ {2})) + 3 C \cdot 2 \sqrt {d \sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} (\sigma_ {t} ^ {2} + \alpha^ {2}) \log T} \sqrt {d \log (1 + T / (d \lambda \alpha^ {2}))} \\ + 3 C \cdot 2 \gamma^ {2} d \sqrt {d \log T} \log (1 + T / (d \lambda \alpha^ {2})) (\max  _ {t \in [ T ]} \widehat {\beta} _ {t}), \\ \end{array}
$$
and an upper bound on expected regret follows noting that  $\sum_{t=1}^{T} \min \{2, (2\sqrt{d\log t} + 1)C\widehat{\beta}_t\| \mathbf{x}_t\|_{\mathbf{V}_t^{-1}}\}$  enjoys a deterministic upper bound:
$$
\begin{array}{l} \mathbb {E} [ \mathcal {R} (T) ] \leq \sum_ {t = 1} ^ {T} g _ {t} + \delta T + 4 \\ \leq 4 d \log (1 + T / (d \lambda \alpha^ {2})) + 3 C \cdot 2 \sqrt {d \sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} (\sigma_ {t} ^ {2} + \alpha^ {2}) \log T} \sqrt {d \log (1 + T / (d \lambda \alpha^ {2}))} \\ + 3 C \cdot 2 \gamma^ {2} d \sqrt {d \log T} \log (1 + T / (d \lambda \alpha^ {2})) (\max  _ {t \in [ T ]} \widehat {\beta} _ {t}) + \delta T + 4 \\ \end{array}
$$
Take  $\lambda = d$ ,  $\alpha = 1 / \sqrt{T}$ , and  $\gamma = \sqrt{R} /d^{1 / 4}$ , it follows that
$$
\log (1 + T / (d \lambda \alpha^ {2})) \leq \log (T ^ {2} / d ^ {2} + 1)
$$
$$
\begin{array}{l} \widehat {\beta} _ {t} = 1 2 \sqrt {d \log (1 + t / (d \lambda \alpha^ {2})) \log (3 2 (1 + \log (\gamma^ {2} / \alpha)) t ^ {2} / \delta)} \\ + 3 0 R \log (3 2 (1 + \log (\gamma^ {2} / \alpha)) t ^ {2} / \delta) / \gamma^ {2} + \sqrt {\lambda} \\ \leq 1 2 \sqrt {d \log (1 + t T / d ^ {2}) \log (3 2 (1 + \log (R \sqrt {T} / \sqrt {d})) t ^ {2} / \delta)} \\ + 3 0 \sqrt {d} \log (3 2 (1 + \log (R \sqrt {T} / \sqrt {d})) t ^ {2} / \delta) + \sqrt {d}. \\ \end{array}
$$
Combining Lemma A.2 and the inequalities above with  $\delta = \frac{1}{T}$ , we conclude that
$$
\begin{array}{l} \mathbb {E} [ \Re (T) ] \lesssim d \log T + d \log T \sqrt {\sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} \left(\sigma_ {t} ^ {2} + \alpha^ {2}\right)} + \gamma^ {2} \sqrt {d ^ {3} \log^ {3} T} \left(\max  _ {t \in [ T ]} \widehat {\beta} _ {t}\right) \\ \lesssim d \log T + d \log T \sqrt {\sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} \left(\sigma_ {t} ^ {2} + \alpha^ {2}\right)} + d \log T \sqrt {d \log T} \\ \lesssim d \log T \sqrt {\sum_ {t = 1} ^ {T} \widehat {\beta} _ {t} ^ {2} \left(\sigma_ {t} ^ {2} + 1 / T\right)} + d \log T \sqrt {d \log T} \\ \lesssim d ^ {3 / 2} \log T \sqrt {\sum_ {t = 1} ^ {T} \sigma_ {t} ^ {2}} + d ^ {3 / 2} \log T. \\ \end{array}
$$
Recall that we have
$$
\mathbb {E} [ \mathcal {R} (T) ] \leq \sum_ {t = 1} ^ {T} \mathbb {E} [ \min \{2, (2 \sqrt {d \log t} + 1) C \widehat {\beta_ {t}} \| \mathbf {x} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \} | \mathcal {H} _ {t} ] + \sum_ {t = 1} ^ {T} \frac {2}{t ^ {2}},
$$
For notational convenience, we further define  $S_{t} = \sum_{s=1}^{t} D_{t}$  where  $D_{t} = \Delta_{t} - g_{t} - \frac{2}{t^{2}}$ , and
$$
g _ {t} = \min  \{2, (2 \sqrt {d \log t} + 1) C \widehat {\beta} _ {t} \| \mathbf {x} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \}.
$$
It follows that  $\mathbb{E}[D_t\mid \mathcal{H}_t]\leq 0$  for all  $t\in [T]$ , and  $\{S_t\}_{t\geq 1}$  therefore forms a super-martingale process with respect to the filtrations  $\{\mathcal{H}_t\} _t$ . Note that  $|\bar{D}_t|\leq 2g_t + 2\leq 4$ , and with Azuma-Hoeffding inequality we have with probability at least  $1 - \delta /2$  that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} G _ {t} \leq \sum_ {t = 1} ^ {T} g _ {t} + \sum_ {t = 1} ^ {T} \frac {2}{t ^ {2}} + \sqrt {2 \sum_ {t = 1} ^ {T} 4 \log (2 / \delta)} \\ \leq \sum_ {t = 1} ^ {T} \min  \left\{2, (2 \sqrt {d \log t} + 1) C \widehat {\beta} _ {t} \| \mathbf {x} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\} + \sum_ {t = 1} ^ {T} \frac {2}{t ^ {2}} + 2 \sqrt {2 T \log (2 / \delta)} \\ = \sum_ {t = 1} ^ {T} \min  \left\{2, \left(2 \sqrt {d \log t} + 1\right) C \widehat {\beta_ {t}} \bar {\sigma} _ {t} \| \mathbf {x} _ {t} / \bar {\sigma} _ {t} \| _ {\mathbf {V} _ {t} ^ {- 1}} \right\} + \sum_ {t = 1} ^ {T} \frac {2}{t ^ {2}} + 2 \sqrt {2 T \log (2 / \delta)} \\ \lesssim d ^ {3 / 2} \log T \sqrt {\sum_ {t = 1} ^ {T} \sigma_ {t} ^ {2}} + d ^ {3 / 2} \log T + \sqrt {T}. \\ \end{array}
$$
B Regret Analysis of Algorithm 3
B.1 Clarification of notation
Analogous to Appendix A, we first introduce two concentration events  $\mathcal{E}_{t,\ell}^{\theta}$  and  $\mathcal{E}_{t,\ell}^{\mathrm{TS}}$  for each  $t\in [T]$  and  $\ell \in [L]$ , i.e.,
$$
\mathcal {E} _ {t, \ell} ^ {\boldsymbol {\theta}} = \{\left| \mathbf {x} ^ {\mathsf {T}} \widehat {\boldsymbol {\theta}} _ {t, \ell} - \mathbf {x} ^ {\mathsf {T}} \boldsymbol {\theta} ^ {*} \right| \leq \widetilde {\beta} _ {t, \ell} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}, \forall \mathbf {x} \in \mathcal {X} _ {t} \}
$$
$$
\mathcal {E} _ {t, \ell} ^ {\mathrm {T S}} = \{| \mathbf {x} ^ {\intercal} \pmb {\theta} _ {t, \ell} ^ {\mathrm {T S}} - \mathbf {x} ^ {\intercal} \widehat {\pmb {\theta}} _ {t, \ell} | \leq \sqrt {2 d \log (4 t ^ {2} L / \delta)} \widehat {\beta} _ {t, \ell} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}, \forall \mathbf {x} \in \mathfrak {X} _ {t} \}.
$$
These events are also characterized as the inclusion of  $\widehat{\theta}_{t,\ell}$  and  $\theta_{t,\ell}^{\mathrm{TS}}$  in the corresponding confidence ellipse, respectively. It is noteworthy that the event  $\mathcal{E}_{t,\ell}^{\theta}$  is defined using
$$
\widetilde {\beta} _ {t, \ell} = 1 6 \cdot 2 ^ {- \ell} \sqrt {\sum_ {s \in \Psi_ {t , \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \log \left(8 t ^ {2} L / \delta\right)} + 6 \cdot 2 ^ {- \ell} R \log \left(8 t ^ {2} L / \delta\right) + 2 ^ {- \ell + 1} \tag {B.1}
$$
rather than the confidence radius  $\widehat{\beta}_{t,\ell}$ .
Subsequently, we introduce a Freedman-type concentration inequality pertinent to vector-valued martingales. This theorem, distinct from Theorem A.1, necessitates a nuanced control on the uncertainty term  $\| \mathbf{y}_t\|_{\mathbf{U}_{t - 1}^{-1}}$  for all  $t\geq 1$ . Consequently, it furnishes an even more stringent bound in comparison to Theorem A.1.
Theorem B.1 (Zhao et al. (2023), Theorem 2.1). Let  $\{\mathcal{H}_t\}_{t=1}^{\infty}$  be a filtration,  $\{\mathbf{y}_t, \varrho_t\}_{t \geq 1}$  a stochastic process so that  $\mathbf{y}_t \in \mathbb{R}^d$  is  $\mathcal{H}_t$ -measurable and  $\varrho_t \in \mathbb{R}$  is  $\mathcal{H}_{t+1}$ -measurable. For  $t \geq 1$  let  $r_t = \langle \mathbf{y}_t, \boldsymbol{\theta}^* \rangle + \varrho_t$  and suppose that  $\varrho_t, \mathbf{y}_t$  satisfy
$$
| \varrho_ {t} | \leq R, \quad \mathbb {E} [ \varrho_ {t} \mid \mathcal {H} _ {t} ] = 0, \quad \sum_ {s = 1} ^ {t} \mathbb {E} [ \varrho_ {s} ^ {2} \mid \mathcal {H} _ {s} ] \leq v _ {t}
$$
for some constants  $R, \{v_t\}_t > 0$  and  $\pmb{\theta}^{*} \in \mathbb{R}^{d}$ . Then, for any  $0 < \delta < 1$ , with probability at least  $1 - \delta$  we have for all  $t > 0$  that
where  $\mathbf{U}_t = \lambda \mathbf{I}_d + \sum_{s=1}^t \mathbf{y}_s \mathbf{y}_s^\top$ ,  $\pmb{\nu}_t = \sum_{s=1}^t r_s \mathbf{y}_s$ ,  $\pmb{\theta}_t = \mathbf{U}_t^{-1} \pmb{\nu}_t$ , and
$$
\beta_ {t} = 1 6 \rho \sqrt {v _ {t} \log (4 t ^ {2} / \delta)} + 6 \rho R \log (4 t ^ {2} / \delta),
$$
where  $\rho \geq \sup_{t\geq 1}\| \mathbf{y}_t\|_{\mathbf{U}_{t - 1}^{-1}}$
B.2 Proof of supporting lemmata
Lemmas B.2 and B.3 show that the concentration events  $\mathcal{E}_{t,\ell}^{\theta}$  and  $\mathcal{E}_{t,\ell}^{\mathrm{TS}}$  hold with high probability for all  $t\in [T]$  and  $\ell \in [L]$ .
Lemma B.2. For any  $\delta \in (0,1)$ , if we define  $\widetilde{\beta}_{t,\ell}$  for all  $t \geq 1$  and  $\ell \in [L]$  as in (B.1), then the estimate  $\widehat{\theta}_{t,\ell}$  given by the associated weighted regressions in Algorithm 3 satisfy
$$
\mathbb {P} \left(\left| \mathbf {x} ^ {\mathsf {T}} \widehat {\boldsymbol {\theta}} _ {t, \ell} - \mathbf {x} ^ {\mathsf {T}} \boldsymbol {\theta} ^ {*} \right| \leq \widetilde {\beta} _ {t, \ell} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}, \forall \mathbf {x} \in \mathcal {X} _ {t}, \forall t \in \Psi_ {t + 1, \ell}, \forall \ell \in [ L ]\right) \geq 1 - \frac {\delta}{2}.
$$
Proof. Notice that for any fixed  $\ell \in [L]$ ,  $t \in \Psi_{t+1,\ell}$ , and  $\mathbf{x} \in \mathcal{X}_t$  that
$$
\left| \mathbf {x} ^ {\mathsf {T}} \widehat {\boldsymbol {\theta}} _ {t, \ell} - \mathbf {x} ^ {\mathsf {T}} \boldsymbol {\theta} ^ {*} \right| \leq \left\| \widehat {\boldsymbol {\theta}} _ {t, \ell} - \boldsymbol {\theta} ^ {*} \right\| _ {\mathbf {V} _ {t, \ell}} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}.
$$
Following the assumption on  $\varepsilon_{t}$ , it holds that
$$
| w _ {t} \varepsilon_ {t} | \leq | \varepsilon_ {t} | \leq R, \quad \mathbb {E} [ w _ {t} \varepsilon_ {t} \mid \mathcal {H} _ {t} ] = 0, \quad \mathbb {E} [ w _ {t} ^ {2} \varepsilon_ {t} ^ {2} \mid \mathcal {H} _ {t} ] \leq w _ {t} ^ {2} \sigma_ {t} ^ {2}, \quad \| w _ {t} \mathbf {x} _ {t} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}} = 2 ^ {- \ell},
$$
where  $w_{t} = 2^{-\ell} / \| \mathbf{x}_{t}\|_{\mathbf{V}_{t,\ell}^{-1}}\leq 1$  Apply Theorem B.1 with stochastic process  $\{w_t\mathbf{x}_t,w_t\varepsilon_t\}$  to get that with probability at least  $1 - \frac{\delta}{2L}$  for all  $t\in \Psi_{T + 1,\ell}$
$$
\left\| \widehat {\boldsymbol {\theta}} _ {t, \ell} - \boldsymbol {\theta} ^ {*} \right\| _ {\mathbf {V} _ {t, \ell}} \leq \widetilde {\beta} _ {t, \ell},
$$
$$
\widetilde {\beta} _ {t, \ell} = 1 6 \cdot 2 ^ {- \ell} \sqrt {\sum_ {s \in \Psi_ {t , \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \log (8 t ^ {2} L / \delta)} + 6 \cdot 2 ^ {- \ell} R \log (8 t ^ {2} L / \delta) + 2 ^ {- \ell + 1}.
$$
It follows from a union bound that with probability at least  $1 - \delta /2$  that for all  $\ell \in [L],t\in \Psi_{t + 1,\ell}$  and  $\mathbf{x}\in \mathcal{X}_t$
$$
\left| \mathbf {x} ^ {\mathsf {T}} \widehat {\boldsymbol {\theta}} _ {t, \ell} - \mathbf {x} ^ {\mathsf {T}} \boldsymbol {\theta} ^ {*} \right| \leq \widetilde {\beta} _ {t, \ell} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}.
$$
Lemma B.3. For any  $\delta \in (0,1)$  and  $\widehat{\beta}_{t,\ell}$  defined in (5.3) for all  $t\geq 1$  and  $\ell \in [L]$ , the sampled variables  $\pmb{\theta}_{t,\ell}^{\mathrm{TS}}$  in Algorithm 3 satisfy
$$
\mathbb {P} \left(| \mathbf {x} ^ {\mathsf {T}} \boldsymbol {\theta} _ {t, \ell} ^ {\mathrm {T S}} - \mathbf {x} ^ {\mathsf {T}} \widehat {\boldsymbol {\theta}} _ {t, \ell} | \leq \sqrt {2 d \log (4 t ^ {2} L / \delta)} \cdot \widehat {\beta} _ {t, \ell} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}, \forall \mathbf {x} \in \mathcal {X} _ {t}, \forall t \in \Psi_ {t + 1, \ell}, \forall \ell \in [ L ]\right) \geq 1 - \frac {\delta}{2}.
$$
Proof. Note that  $\pmb{\theta}_{t,\ell}^{\mathrm{TS}}\sim \mathcal{N}(\widehat{\pmb{\theta}}_{t,\ell},\pmb{\Sigma}_{t,\ell})$  where  $\pmb {\Sigma}_t = \widehat{\beta}_{t,\ell}^2\mathbf{V}_{t,\ell}^{-1}$ , we have with probability at least  $1 - \frac{\delta}{4t^2L}$
$$
\begin{array}{l} | \mathbf {x} ^ {\mathsf {T}} \pmb {\theta} _ {t, \ell} ^ {\mathrm {T s}} - \mathbf {x} ^ {\mathsf {T}} \widehat {\pmb {\theta}} _ {t, \ell} | = | \mathbf {x} ^ {\mathsf {T}} \mathbf {V} _ {t, \ell} ^ {- 1 / 2} \mathbf {V} _ {t, \ell} ^ {1 / 2} (\pmb {\theta} _ {t, \ell} ^ {\mathrm {T s}} - \widehat {\pmb {\theta}} _ {t, \ell}) | \\ \leq \widehat {\beta} _ {t, \ell} \left\| \frac {\mathbf {V} _ {t , \ell} ^ {1 / 2} \left(\boldsymbol {\theta} _ {t , \ell} ^ {\mathrm {T S}} - \widehat {\boldsymbol {\theta}} _ {t , \ell}\right)}{\widehat {\beta} _ {t , \ell}} \right\| _ {2} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}} \\ \leq \widehat {\beta} _ {t, \ell} \sqrt {2 d \log (4 t ^ {2} L / \delta)} \| \mathbf {x} \| _ {\mathbf {V} _ {t, \ell} ^ {- 1}}, \\ \end{array}
$$
where the last inequality follows the concentration inequality in Lemma C.1 for standard normal distribution and that
$$
\left\| \frac {\mathbf {V} _ {t , \ell} ^ {1 / 2} (\boldsymbol {\theta} _ {t , \ell} ^ {\mathrm {T s}} - \widehat {\boldsymbol {\theta}} _ {t , \ell})}{\widehat {\beta} _ {t , \ell}} \right\| _ {2} \sim \mathcal {N} (0, 1).
$$
A union bound over all  $\ell \in [L]$  and  $t\geq 1$  yields the desired result, where the probability bound follows  $\sum_{t = 1}^{\infty}1 / t^{2} <   2$
Next, we show that the empirical estimator  $\zeta_{t,\ell}$  provides an accurate estimate of the summation of weighted variance up to a logarithmic factor. The proof is analogous to Zhao et al. (2023), and we provide it here for the sake of maintaining a self-contained discourse.
Lemma B.4. For all  $t \geq 1$  and  $\ell \in [L]$  such that  $2^{\ell} \geq 64\sqrt{\log(4(t + 1)^{2}L / \delta)}$ , if  $\mathcal{E}_{t,\ell}^{\theta}$  is satisfied and  $\zeta_{t,\ell} = \sum_{s \in \Psi_{t,\ell}} w_s^2 (r_s - \langle \mathbf{x}_s, \widehat{\pmb{\theta}}_{s,\ell} \rangle)^2$ , then the following inequalities hold:
$$
\sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \leq 8 \zeta_ {t + 1, \ell} + 6 R ^ {2} \log (4 (t + 1) ^ {2} L / \delta) + 2 ^ {- 2 \ell + 4},
$$
$$
\zeta_ {t + 1, \ell} \leq \frac {3}{2} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} + \frac {7}{3} R ^ {2} \log (4 t ^ {2} L / \delta) + 2 ^ {- 2 \ell}.
$$
Proof. For any  $\ell \in [L]$  and  $t\in [T]$ , we have  $\sum_{s\in \Psi_{t + 1,\ell}}w_i^2\varepsilon_s^2$  being an unbiased estimator of  $\sum_{s\in \Psi_{t + 1,\ell}}w_i^2\sigma_s^2$  conditioned on past observations. More specifically, we have  $\mathbb{E}[\varepsilon_t^2\mid \mathbf{x}_{1:t},r_{1:t - 1}] - \mathbb{E}[\sigma_t^2\mid \mathbf{x}_{1:t},r_{1:t - 1}] = 0$  for all  $t\geq 1$  and
$$
\begin{array}{l} \sum_ {s \in \Psi_ {t + 1, \ell}} \mathbb {E} \left[ w _ {s} ^ {2} \left(\varepsilon_ {s} ^ {2} - \sigma_ {s} ^ {2}\right) ^ {2} \mid \mathbf {x} _ {1: t}, r _ {1: t - 1} \right] \leq \sum_ {s \in \Psi_ {t + 1, \ell}} \mathbb {E} \left[ w _ {s} ^ {2} \varepsilon_ {s} ^ {4} \mid \mathbf {x} _ {1: t}, r _ {1: t - 1} \right] \\ \leq R ^ {2} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2}, \\ \end{array}
$$
where the first inequality follows from  $\mathbb{Var}(w_s\varepsilon_s^2\mid \mathbf{x}_{1:t},r_{1:t - 1})\leq \mathbb{E}[w_s^2\varepsilon_s^4\mid \mathbf{x}_{1:t},r_{1:t - 1}]$ . Following Freedman inequality, it holds that with probability at least  $1 - 2\delta /L$  that for all  $t\geq 1$
$$
\begin{array}{l} \left| \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \varepsilon_ {s} ^ {2} - \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \right| \leq \sqrt {2 R ^ {2} \sum_ {s \in \Psi_ {t + 1 , \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \log \left(4 t ^ {2} L / \delta\right)} + \frac {4}{3} R ^ {2} \log \left(4 t ^ {2} L / \delta\right) \tag {B.2} \\ \leq \frac {1}{2} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} + \frac {7}{3} R ^ {2} \log (4 t ^ {2} L / \delta), \\ \end{array}
$$
where the second inequality follows from Young's inequality.
$$
\begin{array}{l} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \leq 2 \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \varepsilon_ {s} ^ {2} + \frac {1 4}{3} R ^ {2} \log (4 t ^ {2} L / \delta) \\ \leq 4 \sum_ {s \in \Psi_ {t, \ell}} w _ {s} ^ {2} (r _ {s} - \langle \mathbf {x} _ {s}, \widehat {\boldsymbol {\theta}} _ {t, \ell} \rangle) ^ {2} + \frac {1 4}{3} R ^ {2} \log (4 t ^ {2} L / \delta) \\ + 4 \sum_ {s \in \Psi_ {t, \ell}} w _ {s} ^ {2} \left(\left(r _ {s} - \left\langle \mathbf {x} _ {s}, \widehat {\boldsymbol {\theta}} _ {t, \ell} \right\rangle\right) - \varepsilon_ {s}\right) ^ {2}, \\ \end{array}
$$
where the second inequality follows from  $(x + y)^2 \leq 2x^2 + 2y^2$ . Notice that the last term is bounded by
$$
\begin{array}{l} \sum_ {s \in \Psi_ {t, \ell}} w _ {s} ^ {2} ((r _ {s} - \langle \mathbf {x} _ {s}, \widehat {\boldsymbol {\theta}} _ {t, \ell} \rangle) - \varepsilon_ {s}) ^ {2} = \sum_ {s \in \Psi_ {t, \ell}} w _ {s} ^ {2} \langle \mathbf {x} _ {s}, \widehat {\boldsymbol {\theta}} _ {t, \ell} - \boldsymbol {\theta} ^ {*} \rangle^ {2} \\ \leq \left(\widehat {\boldsymbol {\theta}} _ {t, \ell} - \boldsymbol {\theta} ^ {*}\right) ^ {\mathsf {T}} \mathbf {V} _ {t + 1, \ell} \left(\widehat {\boldsymbol {\theta}} _ {t, \ell} - \boldsymbol {\theta} ^ {*}\right) \\ \leq \tilde {\beta} _ {t + 1, \ell} ^ {2}, \\ \end{array}
$$
where the first inequality follows from  $w_s^2 x_s x_s^\top \preceq \mathbf{V}_{t+1,\ell}$  and the second inequality is due to Lemma B.2. If we have  $2^\ell \geq 64\sqrt{\log(8t^2L/\delta)}$ , then
$$
\widetilde {\beta} _ {t + 1, \ell} ^ {2} \leq \frac {1}{8} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} + 2 (6 \cdot 2 ^ {- \ell} R \log (4 (t + 1) ^ {2} L / \delta) + 2 ^ {- \ell + 1}) ^ {2}
$$
$$
\begin{array}{l} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} \leq 4 \sum_ {s \in \Psi_ {t, \ell}} w _ {s} ^ {2} (r _ {s} - \langle \mathbf {x} _ {s}, \widehat {\boldsymbol {\theta}} _ {t, \ell} \rangle) ^ {2} + \frac {1 4}{3} R ^ {2} \log (4 t ^ {2} L / \delta) \\ + \frac {1}{2} \sum_ {s \in \Psi_ {t + 1, \ell}} w _ {s} ^ {2} \sigma_ {s} ^ {2} + 8 (6 \cdot 2 ^ {- \ell} R \log (4 (t + 1) ^ {2} L / \delta) + 2 ^ {- \ell + 1}) ^ {2}. \\ \end{array}
$$
Move  $\frac{1}{2}\sum_{s\in \Psi_{t + 1,\ell}}w_s^2\sigma_s^2$  to the other side of the inequality, we get the desired result.
Moreover, we can establish the inequality in the other direction: