SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Lemma 7. To show that  $\{M_t(\pmb {\xi})\}_{t = 1}^{\infty}$  is a non-negative super-martingale, it is sufficient to prove  $\mathbb{E}[M_{t + 1}(\pmb {\xi})\mid \mathcal{F}_t]\leq M_t(\pmb {\xi})$  for any  $t > 1$  and  $\pmb {\xi}\in \frac{1}{2}\mathcal{B}_2(Kd)$ . By the definition of  $M_{t + 1}(\pmb {\xi})$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ M _ {t + 1} (\boldsymbol {\xi}) \mid \mathcal {F} _ {t} \right] = \mathbb {E} \left[ \exp \left(\boldsymbol {\xi} ^ {\top} S _ {t + 1} - \| \boldsymbol {\xi} \| _ {\bar {H} _ {t + 1}} ^ {2}\right) \mid \mathcal {F} _ {t} \right] \\ = \mathbb {E} \left[ \exp \left(\boldsymbol {\xi} ^ {\top} \left(\boldsymbol {\varepsilon} _ {t} \otimes \mathbf {x} _ {t}\right) - \boldsymbol {\xi} ^ {\top} \left(\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi}\right) \mid \mathcal {F} _ {t} \right] M _ {t} (\boldsymbol {\xi}) \\ = \mathbb {E} \left[ \exp \left(\varepsilon_ {t} ^ {\top} \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi} - \boldsymbol {\xi} ^ {\top} \left(\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi}\right) \mid \mathcal {F} _ {t} \right] M _ {t} (\boldsymbol {\xi}). \tag {19} \\ \end{array}
$$
Let  $\pmb {\xi} = [\pmb {\xi}_1;\dots ;\pmb {\xi}_K]$  and  $\pmb {\xi}_k\in \mathbb{R}^d$  is the vector containing  $(k - 1)d + 1$  th to  $kd$  -th elements of  $\pmb{\xi}$  We can further check that
$$
\left| \boldsymbol {\varepsilon} _ {t} ^ {\top} \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi} \right| \leq \left\| \boldsymbol {\varepsilon} _ {t} \right\| _ {1} \cdot \left\| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi} \right\| _ {\infty} = \left\| \boldsymbol {\varepsilon} _ {t} \right\| _ {1} \cdot \max  _ {k \in K} \left| \boldsymbol {\xi} _ {k} ^ {\top} \mathbf {x} _ {t} \right| \leq 1,
$$
where the first inequality is due to the Hölder's inequality. The last inequality is by the condition  $\| \pmb{\varepsilon}_t\| _1\leq 2$  and  $\| \pmb {\xi}_k\| _2\leq \| \pmb {\xi}\| _2\leq 1 / 2$  for any  $k\in [K]$ . Then, we can further bound the expectation term in (19) by
$$
\begin{array}{l} \mathbb {E} \left[ \exp \left(\boldsymbol {\varepsilon} _ {t} ^ {\top} \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi} - \boldsymbol {\xi} ^ {\top} \left(\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi}\right) \mid \mathcal {F} _ {t} \right] \\ = \mathbb {E} \left[ \exp \left(\varepsilon_ {t} ^ {\top} \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi}\right) \mid \mathcal {F} _ {t} \right] \cdot \exp \left(- \boldsymbol {\xi} ^ {\top} \left(\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) \boldsymbol {\xi}\right) \\ \leq \exp (\boldsymbol {\xi} ^ {\top} (I _ {K} \otimes \mathbf {x} _ {t}) \Sigma_ {t} (I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}) \boldsymbol {\xi}) \cdot \exp (- \boldsymbol {\xi} ^ {\top} (\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}) \boldsymbol {\xi}) \\ = \exp (\boldsymbol {\xi} ^ {\top} (\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}) \boldsymbol {\xi}) \cdot \exp (- \boldsymbol {\xi} ^ {\top} (\Sigma_ {t} \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}) \boldsymbol {\xi}) = 1, \\ \end{array}
$$
where the first inequality is due to Lemma 8 and the last equality is due to mixed-product property of the Kronecker product. Therefore, we can show  $\mathbb{E}[M_{t + 1}(\pmb {\xi})\mid \mathcal{F}_t]\leq M_t(\pmb {\xi})$  and complete the proof.
Lemma 8 (Lemma 6 of [11]). Let  $\varepsilon \in \mathbb{R}^K$  be a zero-mean random vector with covariance matrix  $\Sigma$ . Then, for any vector  $\mathbf{a} \in \mathbb{R}^K$  such that  $|\boldsymbol{\varepsilon}^\top \mathbf{a}| \leq 1$ , we have  $\mathbb{E}[\exp (\boldsymbol{\varepsilon}^\top \mathbf{a})] \leq \exp (\mathbf{a}^\top \boldsymbol{\Sigma}\mathbf{a})$ .
This section provides the proof for Theorem 2.
B.2.1 Main Proof
Proof of Theorem 2. We prove the proposed algorithm can achieve  $\mathcal{O}(d\log T\sqrt{\kappa KT})$  and  $\mathcal{O}(dK\log T\sqrt{T} +\kappa d^2 K^2\log^2 T)$  bounds simultaneously. Before presenting the proofs, we introduce two matrix that measures the local curvature of the loss function and will be used in the rest part of the proof:
$$
A (\mathbf {x}, W _ {1}, W _ {2}) = \int_ {v = 0} ^ {1} \nabla \boldsymbol {\sigma} (v W _ {1} \mathbf {x} + (1 - v) W _ {2} \mathbf {x}) \mathrm {d} v; \tag {20}
$$
$$
G _ {t} \left(W _ {1}, W _ {2}\right) = \lambda I _ {K d} + \sum_ {s = 1} ^ {t - 1} A \left(\mathbf {x} _ {s}, W _ {1}, W _ {2}\right) \otimes \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top}. \tag {21}
$$
Analysis for the  $\mathcal{O}(d\log T\sqrt{\kappa KT})$  Bound. The regret can be bounded by
$$
\begin{array}{l} \operatorname {R e g} _ {T} = \sum_ {t = 1} ^ {T} \boldsymbol {\rho} ^ {\top} \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {*}\right) - \sum_ {t = 1} ^ {T} \boldsymbol {\rho} ^ {\top} \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \\ \leq \sum_ {t = 1} ^ {T} \boldsymbol {\rho} ^ {\top} \boldsymbol {\sigma} \left(W _ {t} ^ {\mathrm {O P T}} \mathbf {x} _ {t}\right) - \sum_ {t = 1} ^ {T} \boldsymbol {\rho} ^ {\top} \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \\ = \sum_ {t = 1} ^ {T} \boldsymbol {\rho} ^ {\top} A (\mathbf {x} _ {t}, W _ {*}, W _ {t} ^ {\mathrm {O P T}}) (W _ {t} ^ {\mathrm {O P T}} - W _ {*}) \mathbf {x} _ {t} \\ \end{array}
$$
$$
\begin{array}{l} \leq \sum_ {t = 1} ^ {T} \| \boldsymbol {\rho} \| _ {A (\mathbf {x} _ {t}, W _ {*}, W _ {t} ^ {\mathrm {O P T}})} \cdot \| (W _ {t} ^ {\mathrm {O P T}} - W _ {*}) \mathbf {x} _ {t} \| _ {A (\mathbf {x} _ {t}, W _ {*}, W _ {t} ^ {\mathrm {O P T}})} \\ \leq \sum_ {t = 1} ^ {T} \| \boldsymbol {\rho} \| _ {2} \cdot \left\| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \right\| _ {A \left(\mathbf {x} _ {t}, W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)} \\ \leq R \sum_ {t = 1} ^ {T} \| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \| _ {A \left(\mathbf {x} _ {t}, W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)} \\ \end{array}
$$
In the above, the term  $A(\mathbf{x}_t, W_*, W_t^{\mathrm{OPT}})$  is defined as (20) and the first inequality is a consequence of the optimistic rule (5) such that  $(\mathbf{x}_t, W_t^{\mathrm{OPT}}) = \arg \max_{\mathbf{x} \in \mathcal{X}, W \in \mathcal{C}_t(\delta)} \pmb{\rho}^\top \pmb{\sigma}(W\mathbf{x})$  and the first equality is due to the mean value theorem for the vector valued function. The second inequality is due to the Cauchy-Schwarz inequality and we can obtain the last second inequality by the fact  $\nabla \pmb{\sigma}(\mathbf{z}) \preccurlyeq I_K$  for any  $\mathbf{z} \in \mathbb{R}^K$ . The last inequality is due to Assumption 2.
Let  $G_{t}^{-\frac{1}{2}}(W_{1}, W_{2})$  be defined as (21). We can proceed to handle the gap between  $W_{t}^{\mathrm{OPT}}$  and  $W_{*}$  by
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left\| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \right\| _ {A \left(\mathbf {x} _ {t}, W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)} \\ \leq \sum_ {t = 1} ^ {T} \| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \| _ {2} \\ = \sum_ {t = 1} ^ {T} \| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \cdot \left(\overrightarrow {W} _ {t} ^ {\text {O P T}} - \overrightarrow {W} _ {*}\right) \| _ {2} \\ = \sum_ {t = 1} ^ {T} \| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \cdot G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\overrightarrow {W} _ {t} ^ {\mathrm {O P T}} - \overrightarrow {W} _ {*}\right) \| _ {2} \\ \leq \sum_ {t = 1} ^ {T} \| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \| _ {2} \cdot \| G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\overrightarrow {W} _ {t} ^ {\mathrm {O P T}} - \overrightarrow {W} _ {*}\right) \| _ {2} \\ \leq \sum_ {t = 1} ^ {T} \| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \bar {V} _ {t} ^ {- \frac {1}{2}} \| _ {2} \cdot \| G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\vec {W} _ {t} ^ {\mathrm {O P T}} - \vec {W} _ {*}\right) \| _ {2} \\ \leq \underbrace {\sqrt {\sum_ {t = 1} ^ {T} \left\| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \bar {V} _ {t} ^ {- \frac {1}{2}} \| _ {2} ^ {2} \right.}} _ {\text {t e r m (a)}} \cdot \underbrace {\sqrt {\sum_ {t = 1} ^ {T} \left\| G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\overrightarrow {W} _ {t} ^ {\mathrm {O P T}} - \overrightarrow {W} _ {*}\right) \| _ {2} ^ {2} \right.}} _ {\text {t e r m (b)}}. \tag {22} \\ \end{array}
$$
where the first inequality is due to the fact  $A(\mathbf{x}_t, W_*, W_t^{\mathrm{OPT}}) \preccurlyeq I_K$  and the second inequality is due to the Cauchy-Schwarz inequality. Since both  $W_*, W_t^{\mathrm{OPT}} \in \mathcal{W}$ , the third inequality is by the condition  $G_t(W_*, W_t^{\mathrm{OPT}}) \succcurlyeq \bar{V}_t \triangleq \lambda I_{Kd} + \frac{1}{\kappa} \sum_{s=1}^{t-1} I_K \otimes \mathbf{x}_s \mathbf{x}_s^\top$ . We can show the last inequality by using Cauchy-Schwarz inequality again.
Then, we proceed to bound term (a) and term (b) respectively.
$$
\begin{array}{l} \operatorname {t e r m} (\mathbf {a}) = \sqrt {\sum_ {t = 1} ^ {T} \| (I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}) \bar {V} _ {t} ^ {- \frac {1}{2}} \| _ {2} ^ {2}} \\ = \sqrt {\sum_ {t = 1} ^ {T} \lambda_ {\max } ((I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}) \bar {V} _ {t} ^ {- 1} (I _ {K} \otimes \mathbf {x} _ {t}))} \\ = \sqrt {\sum_ {t = 1} ^ {T} \lambda_ {\max } \left(\left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \left(I _ {K} \otimes \left(\lambda I _ {d} + \frac {1}{\kappa} \sum_ {s = 1} ^ {t - 1} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top}\right)\right) ^ {- 1} \left(I _ {K} \otimes \mathbf {x} _ {t}\right)\right)} \\ \end{array}
$$
$$
\begin{array}{l} = \sqrt {\sum_ {t = 1} ^ {T} \lambda_ {\max } \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top} \left(\lambda I _ {d} + \frac {1}{\kappa} \sum_ {s = 1} ^ {t - 1} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top}\right) ^ {- 1} \mathbf {x} _ {t}\right)} \\ = \sqrt {\sum_ {t = 1} ^ {T} \mathbf {x} _ {t} ^ {\top} \left(\lambda I _ {d} + \frac {1}{\kappa} \sum_ {s = 1} ^ {t - 1} \mathbf {x} _ {s} \mathbf {x} _ {s} ^ {\top}\right) ^ {- 1} \mathbf {x} _ {t}} \\ \leq \sqrt {\kappa d \log \left(1 + \frac {T}{\kappa \lambda d}\right)} \tag {23} \\ \end{array}
$$
In above, we denote by  $\lambda_{\mathrm{max}}$  the maximum eigenvalue of the input matrix. The second inequality is due to the property of the maximum eigenvalue such that  $\lambda_{\mathrm{max}}(ABC) = \lambda_{\mathrm{max}}(BCA)$  for matrix  $A, B$  and  $C$ . The last second equality is due to the mixed product property of the Kronecker product and  $(A \otimes B)^{-1} = A^{-1} \otimes B^{-1}$  for matrix  $A$  and  $B$ . In the last inequality, we can bound the term with the standard arguments of the elliptical potential lemma [3, Lemma 11].
It remains to bound term (b). Since Theorem 1 shows that  $W_{*} \in \mathcal{C}_{t}(\delta)$  with probability at least  $1 - \delta$  for all  $t > 1$  and the optimistic rule (5) ensures  $W_{t}^{\mathrm{OPT}} \in \mathcal{C}_{t}(\delta)$ , Lemma 9 in Appendix B.2.2 indicates that the distance between  $W_{*}$  and  $W_{t}^{\mathrm{OPT}}$  can be bounded by
$$
\left\| G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\overrightarrow {W} _ {t} ^ {\mathrm {O P T}} - \overrightarrow {W} _ {*}\right) \right\| _ {2} \leq 2 \sqrt {1 + 2 S} \beta_ {t} (\delta).
$$
Then, plugging the above displayed equation into term (b), we can bound this term by
$$
\operatorname {t e r m} (\mathfrak {b}) \leq 2 \sqrt {1 + 2 S} \sqrt {\sum_ {t = 1} ^ {T} \left(\beta_ {t} (\delta)\right) ^ {2}} \leq 2 \sqrt {(1 + 2 S) T} \beta_ {T} (\delta) = \mathcal {O} \left(\sqrt {d T K \log (T / \delta)}\right),
$$
where the last inequality holds because  $\beta_{t}(\delta)$  is a non-decreasing function with respect to  $t$ . A combination of the upper bound for term (a) and term (b) leads to
$$
\mathrm {R e g} _ {T} \leq 8 (1 + 2 S) R d \sqrt {\kappa K T \cdot \log \left(\frac {2 (1 + T / d)}{\delta}\right) \cdot \log \left(1 + \frac {T}{\kappa d}\right)} = \mathcal {O} (d \log T \sqrt {\kappa K T}).
$$
Analysis for the  $\mathcal{O}(Kd\log T\sqrt{T} +\kappa Kd^2\log^2 T)$  Bound. Inspired by the analysis for binary logistic bandit problem [9, Appendix C], we employ a different decomposition compared to the one used in the first part of the analysis
$$
\begin{array}{l} \operatorname {R e g} _ {T} = \sum_ {t = 1} ^ {T} \sum_ {k = 1} ^ {K} \rho_ {k} \left(\sigma_ {k} \left(W _ {*} \mathbf {x} _ {*}\right) - \sigma_ {k} \left(W _ {*} \mathbf {x} _ {t}\right)\right) \\ \leq \sum_ {t = 1} ^ {T} \sum_ {k = 1} ^ {K} \rho_ {k} \left(\sigma_ {k} \left(W _ {t} ^ {\mathrm {O P T}} \mathbf {x} _ {t}\right) - \sigma_ {k} \left(W _ {*} \mathbf {x} _ {t}\right)\right) \\ = \sum_ {t = 1} ^ {T} \sum_ {k = 1} ^ {K} \rho_ {k} \nabla \sigma_ {k} \left(W _ {*} \mathbf {x} _ {t}\right) ^ {\top} \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} + \sum_ {k = 1} ^ {K} \rho_ {k} \| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \| _ {\Xi_ {k, t}} ^ {2} \\ \leq \underbrace{\sum_{t = 1}^{T}\left|\sum_{k = 1}^{K}\rho_{k}\nabla\sigma_{k}(W_{*}\mathbf{x}_{t})^{\top}(W_{t}^{\mathrm{OPT}} - W_{*})\mathbf{x}_{t}\right|}_{\text{term (c)}} + \underbrace{\sum_{t = 1}^{T}\left|\sum_{k = 1}^{K}\rho_{k}\|(W_{t}^{\mathrm{OPT}} - W_{*})\mathbf{x}_{t}\|_{\Xi_{k,t}}^{2}\right|}_{\text{term (d)}}, \\ \end{array}
$$
where  $\sigma_{k}:\mathbf{x}\mapsto [\pmb {\sigma}(\mathbf{x})]_{k}$  is the  $k$  -th output of the vector-valued function  $\pmb {\sigma}(\mathbf{x})$  and  $\Xi_{k,t} = \int_{\nu = 0}^{1}(1-$ $\nu)\nabla^{2}\sigma_{k}((W_{*} + \nu (W_{t}^{\mathrm{OPT}} - W_{*}))\mathbf{x}_{t})\mathrm{d}\nu .$  In the above, the first inequality is due to the optimistic rule (5) and the last equality is due to the integral formulation of the Taylor series. Then, we proceed to handle term (c) and term (d) respectively.
As for term (c), we have
$$
\text {t e r m (c)} = \sum_ {t = 1} ^ {T} \left| \rho \nabla \sigma \left(W _ {*} \mathbf {x} _ {t}\right) ^ {\top} \left(\overrightarrow {W} _ {t} ^ {\text {O P T}} - \overrightarrow {W} _ {*}\right) \mathbf {x} _ {t} \right|
$$
$$
\begin{array}{l} = \sum_ {t = 1} ^ {T} \left| \boldsymbol {\rho} \nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) ^ {\top} \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \left(\overrightarrow {W} _ {t} ^ {\text {O P T}} - \overrightarrow {W} _ {*}\right) \right| \\ = \sum_ {t = 1} ^ {T} \left| \rho \nabla \sigma \left(W _ {*} \mathbf {x} _ {t}\right) ^ {\top} \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \cdot G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\overrightarrow {W} _ {t} ^ {\mathrm {O P T}} - \overrightarrow {W} _ {*}\right) \right| \\ \leq \sum_ {t = 1} ^ {T} \| \vec {W} _ {t} ^ {\text {O P T}} - \vec {W} _ {*} \| _ {G _ {t} \left(W _ {*}, W _ {t} ^ {\text {O P T}}\right)} \cdot \| G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\text {O P T}}\right) \left(I _ {K} \otimes \mathbf {x} _ {t}\right) \nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \boldsymbol {\rho} \| _ {2} \\ \leq 2 \sqrt {1 + 2 S} \beta_ {T} (\delta) \sum_ {t = 1} ^ {T} \| G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(I _ {K} \otimes \mathbf {x} _ {t}\right) \nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \boldsymbol {\rho} \| _ {2}, \tag {24} \\ \end{array}
$$
where the first inequality holds by Cauchy-Schwarz inequality and the last one is due to Lemma 9. Let  $\lambda_{\mathrm{max}}(\cdot)$  denote the maximum eigenvalue of the matrix. We can further bound (24) by
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left\| G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(I _ {K} \otimes \mathbf {x} _ {t}\right) \nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \boldsymbol {\rho} \right\| _ {2} \\ \leq \sum_ {t = 1} ^ {T} \| \nabla \sigma^ {\frac {1}{2}} \left(W _ {*} \mathbf {x} _ {t}\right) \boldsymbol {\rho} \| _ {2} \cdot \| G _ {t} ^ {- \frac {1}{2}} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(I _ {K} \otimes \mathbf {x} _ {t}\right) \nabla \sigma^ {\frac {1}{2}} \left(W _ {*} \mathbf {x} _ {t}\right) \| _ {2} \\ \leq R \sum_ {t = 1} ^ {T} \sqrt {\lambda_ {\max } \left(\left(\nabla \sigma^ {\frac {1}{2}} \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- 1} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right) \left(\nabla \sigma^ {\frac {1}{2}} \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t}\right)\right)} \\ = R \sum_ {t = 1} ^ {T} \sqrt {\lambda_ {\max } \left(\left(\nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- 1} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)\right)} \\ \leq R \sqrt {T} \sqrt {\sum_ {t = 1} ^ {T} \lambda_ {\max } \left(\left(\nabla \sigma \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- 1} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)\right)} \\ \leq R \sqrt {T} \sqrt {(1 + 2 S) K d \ln \left(1 + \frac {T}{2 \lambda}\right)} \tag {25} \\ \end{array}
$$
where the first inequality is due to the fact that  $\| A\mathbf{b}\| _2\leq \| A\| _2\cdot \| \mathbf{b}\|$  for a matrix  $A$  and vector  $\mathbf{b}$ . The second inequality is due to the definition of the induced norm  $\| A\| _2 = \sqrt{\lambda_{\max}(A^TA)}$  and the mixed-product property of the Kronecker production. The last equality is due to the cycle property of the maximum eigenvalue such that  $\lambda_{\mathrm{max}}(ABC) = \lambda_{\mathrm{max}}(CAB)$  for matrices  $A,B$  and  $C$  and we use the mixed-product property again. The last inequality is due to Lemma 10 in Appendix B.2.2.
Combining (24) and (25), we arrive
$$
\operatorname {t e r m} (\mathrm {c}) \leq 8 R (1 + 2 S) ^ {\frac {3}{2}} d K \sqrt {T \cdot \log \left(\frac {2 (1 + t / d)}{\delta}\right) \cdot \log \left(1 + \frac {T}{2 d}\right)} = \mathcal {O} (d K \log T \sqrt {T}). \tag {26}
$$
Then, we turn to handle term (d). Lemma 11 indicates that
$$
\Xi_ {k, t} = \int_ {\nu = 0} ^ {1} (1 - \nu) \nabla^ {2} \sigma_ {k} ((W _ {*} + \nu (W _ {t} ^ {\mathrm {O P T}} - W _ {*})) \mathbf {x} _ {t}) \mathrm {d} \nu \preccurlyeq 3 I _ {K} \int_ {\nu = 0} ^ {1} (1 - \nu) \mathrm {d} \nu \preccurlyeq 3 I _ {K}. \tag {27}
$$
As a consequence, we can bound term (d) by
$$
\begin{array}{l} \text {t e r m} (\mathrm {d}) = \sum_ {t = 1} ^ {T} \left| \sum_ {k = 1} ^ {K} \rho_ {k} \| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \| _ {\Xi_ {k, t}} ^ {2} \right| \\ \leq 3 \sum_ {t = 1} ^ {T} \left| \sum_ {k = 1} ^ {K} \rho_ {k} \| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \| _ {2} ^ {2} \right| \\ \end{array}
$$
$$
\begin{array}{l} \leq 3 R \sum_ {t = 1} ^ {T} \| \left(W _ {t} ^ {\mathrm {O P T}} - W _ {*}\right) \mathbf {x} _ {t} \| _ {2} ^ {2} \\ \leq 3 R \beta_ {T} ^ {2} (\delta) \sum_ {t = 1} ^ {T} \| \left(I _ {K} \otimes \mathbf {x} _ {t} ^ {\top}\right) \bar {V} _ {t} ^ {- \frac {1}{2}} \| _ {2} ^ {2} \\ \leq 3 R \beta_ {T} ^ {2} (\delta) \kappa d \log \left(1 + \frac {T}{\kappa \lambda d}\right) \\ \end{array}
$$
where the first inequality is due to Assumption 2 and the second inequality is due to (27). The last second inequality can be obtained by a similar argument in obtaining (22). The last inequality can be obtained by the same argument as (23).
Combining (26) and (26), we obtain
$$
\begin{array}{l} \operatorname {R e g} _ {T} \leq 8 R (1 + 2 S) ^ {\frac {3}{2}} d K \sqrt {T \cdot \log \left(\frac {2 (1 + T / d)}{\delta}\right) \cdot \log \left(1 + \frac {T}{2 d}\right)} \\ + 4 8 R (1 + S) \kappa K d ^ {2} \log \left(\frac {2 (1 + T / d)}{\delta}\right) \cdot \log \left(1 + \frac {T}{2 d}\right) \\ = \mathcal {O} (d K \log T \sqrt {T} + \kappa d ^ {2} K \log^ {2} T). \\ \end{array}
$$
B.2.2 Useful Lemma
This section provides the lemmas used in the main proof.
Lemma 9. If  $W_{*} \in \mathcal{C}_{t}(\delta)$ , then for all  $W \in \mathcal{C}_t(\delta)$ :
$$
\left\| G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W\right) \left(\overrightarrow {W} - \overrightarrow {W} _ {*}\right) \right\| _ {2} \leq 2 \sqrt {1 + 2 S} \beta_ {t} (\delta),
$$
where  $\beta_{t}(\delta) = 4\sqrt{Kd(1 + S)\log\left(2\left(1 + t / d\right) / \delta\right)} = \mathcal{O}(\sqrt{dK\log t})$  is defined in Theorem 1.
Proof of Lemma 9. Lemma 3 in [11] shows that
$$
\mathbf {g} _ {t} (\overrightarrow {W} _ {*}) - \mathbf {g} _ {t} (\overrightarrow {W}) = G _ {t} (W _ {*}, W) (\overrightarrow {W} _ {*} - \overrightarrow {W}),
$$
for any  $\overrightarrow{W} \in \mathcal{C}_t(\delta)$ . Then, we have
$$
\begin{array}{l} \left\| G _ {t} ^ {\frac {1}{2}} \left(W _ {*}, W\right) \left(\overrightarrow {W} - \overrightarrow {W} _ {*}\right) \right\| _ {2} \\ = \left\| \mathbf {g} _ {t} \left(\overrightarrow {W} _ {*}\right) - \mathbf {g} _ {t} \left(\overrightarrow {W}\right) \right\| _ {G _ {t} ^ {- 1} \left(W _ {*}, W\right)} \\ \leq \left\| \mathbf {g} _ {t} \left(\overrightarrow {W} _ {*}\right) - \mathbf {g} _ {t} \left(\overrightarrow {W} _ {t} ^ {\mathrm {M L E}}\right) \right\| _ {G _ {t} ^ {- 1} \left(W _ {*}, W\right)} + \left\| \mathbf {g} _ {t} (\overrightarrow {W}) - \mathbf {g} _ {t} \left(\overrightarrow {W} _ {t} ^ {\mathrm {M L E}}\right) \right\| _ {G _ {t} ^ {- 1} \left(W _ {*}, W\right)} \\ \leq \sqrt {1 + 2 S} \left(\| \mathbf {g} _ {t} (\overrightarrow {W} _ {*}) - \mathbf {g} _ {t} (\overrightarrow {W} _ {t} ^ {\mathrm {M L E}}) \| _ {H _ {t} ^ {- 1} (W _ {*})} + \| \mathbf {g} _ {t} (\overrightarrow {W}) - \mathbf {g} _ {t} (\overrightarrow {W} _ {t} ^ {\mathrm {M L E}}) \| _ {H _ {t} ^ {- 1} (W)}\right) \\ \leq 2 \sqrt {1 + 2 S} \beta_ {t} (\delta), \\ \end{array}
$$
where the last second inequality is due to generalized self-concordant properties of the logistic loss as shown by Lemma 13 of [11]. The last inequality is due to Theorem 1 and the condition that  $W$  is contained in the confidence set  $\mathcal{C}_t(\delta)$ .
Lemma 10. When the regularization parameter  $\lambda >2$  we have
$$
\sum_ {t = 1} ^ {T} \lambda_ {\max } \left((\nabla \sigma (W _ {*} \mathbf {x} _ {t}) \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}) G _ {t} ^ {- 1} (W _ {*}, W _ {t} ^ {\mathrm {O P T}})\right) \leq (1 + 2 S) K d \ln \left(1 + \frac {T}{2 \lambda}\right).
$$
Proof of Lemma 10. Denoting by  $\operatorname{Tr}(A)$  the trace of matrix  $A$ , We have
$$
\sum_ {t = 1} ^ {T} \lambda_ {\max } \left(\left(\nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- 1} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)\right)
$$
$$
\begin{array}{l} \leq \sum_ {t = 1} ^ {T} \operatorname {T r} \left(\left(\nabla \sigma \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) G _ {t} ^ {- 1} \left(W _ {*}, W _ {t} ^ {\mathrm {O P T}}\right)\right) \\ \leq (1 + 2 S) \sum_ {t = 1} ^ {T} \operatorname {T r} \left(\left(\nabla \boldsymbol {\sigma} \left(W _ {*} \mathbf {x} _ {t}\right) \otimes \mathbf {x} _ {t} \mathbf {x} _ {t} ^ {\top}\right) H _ {t} ^ {- 1} \left(W _ {*}\right)\right), \\ \end{array}
$$
where the last inequality is due to Lemma 4 of [11] such that  $(1 + 2S)G_{t}(W_{*},W_{t}^{\mathrm{OPT}})\succcurlyeq H_{t}(W_{*})$