SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For any reference distribution  $q \in \mathcal{P}(\mathcal{X})$  and learning rate  $\eta > 0$ , we define  $\Lambda_{q,\eta}(p,g)$  by
$$
\Lambda_ {q, \eta} (p, g) = \sup  _ {\ell \in \mathcal {L}, M \in \mathcal {M} _ {\ell}} \left\{\langle \ell , \mu (p) - \mu (q) \rangle + \operatorname {b i a s} (g, p, q, \ell) + \frac {1}{\eta} \underset {a \sim p, f \sim M _ {a}} {\mathbf {E}} [ \operatorname {s t a b} (\eta g (\cdot ; a, f), q) ] \right\}.
$$
From Lemma 1, the regret is bounded as
$$
\mathbf {E} _ {a ^ {*} \sim p ^ {*}} \left[ R _ {T} \left(a ^ {*}\right) \right] \leq \mathbf {E} \left[ \sum_ {t = 1} ^ {T} \Lambda_ {q _ {t}, \eta_ {t}} \left(p _ {t}, g _ {t}\right) + \frac {1}{\eta_ {T + 1}} D _ {\mathrm {K L}} \left(p ^ {*}, q _ {0}\right) \right]. \tag {8}
$$
In the exploration-by-optimization (EXO) approach [Lattimore and Szepesvári, 2020b, Lattimore and Gyorgy, 2021], we choose  $g_{t}$  and  $p_t$  so that the value of  $\Lambda_{q_t,\eta_t}(g_t,p_t)$  is as small as possible, i.e., we consider the following optimization problem:
$$
\text {M i n i m i z e} \quad \Lambda_ {q _ {t}, \eta_ {t}} (p, g) \quad \text {s u b j e c t t o} \quad p \in \mathcal {P} (\mathcal {X}), g \in \mathcal {G}. \tag {9}
$$
We denote the optimal value of this problem by  $\Lambda_{q_t,\eta_t}^*$ .
Remark 1. In most linear bandit algorithms based on exponential weights [Dani et al., 2008, Bubeck et al., 2012, Cesa-Bianchi and Lugosi, 2012, Hazan and Karnin, 2016, Besbes et al., 2019],  $p_t$  is defined as  $q_t$  mixed with a small component of exploration basis  $\pi_0$  and  $g_t$  is given by (4) with  $p = p_t$ . The exploration basis  $\pi_0$  here is a distribution over  $\mathcal{X}$  such that  $G(\pi_0) := \sup_{a \in \mathcal{X}} a^\top H(\pi_0)a$  is bounded. It is known that any action set  $\mathcal{X} \subseteq \mathbb{R}^d$  admits an exploration basis  $\pi_0$  such that  $G(\pi_0) \leq d$ , which is called the G-optimal design or the John's ellipsoid exploration basis. Given such a  $\pi_0$ , we set  $p_t = (1 - \gamma_t)q_t + \gamma_t\pi_0$  with some  $\gamma_t \in (0,1)$ . Under the condition of  $\eta_t = \Omega(1/d)$  and  $\gamma_t = \Theta(d\eta_t)$ , we have  $\Lambda_{q_t,\eta_t}(p_t,g_t) = O(\gamma_t + \eta_td) = O(\eta_td)$ , which implies that  $\Lambda_{q,\eta_t}^* = O(\eta_td)$  holds for any  $q \in \mathcal{P}(\mathcal{X})$ . Hence, from (8), if  $D_{\mathrm{KL}}(p^*,q_0) \leq B$ , by setting  $\eta_t = \eta = \Theta(\sqrt{B/(dT)})$ , we obtain  $\mathbf{E}_{a^* \sim p^*}[R_T(a^*)] = O(\sqrt{BdT})$ .
4.2 Sufficient condition for best-of-both-worlds regret guarantee
In this section, we show that a certain type of upper bounds on  $\Lambda_{q,\eta}^{*}$  that depend on reference distribution  $q$  will lead to best-of-both-worlds regret bounds. In the following, we set the initial distribution  $q_{0}$  to be a uniform distribution over  $\mathcal{X}$ . Denote  $\Delta_{\ell^{*}}(x) = \langle \ell^{*}, x - a^{*} \rangle$  for any  $\ell^{*} \in \mathcal{L}$ . The following lemma provides a sufficient condition for best-of-both-worlds regret guarantees:
Theorem 1. Suppose that there exist  $\alpha >0,\beta >0$  and  $\eta_0 > 0$  such that
$$
\Lambda_ {q _ {t}, \eta} ^ {*} \leq \eta \cdot \min  \left\{\alpha , \beta \cdot \Delta_ {\ell^ {*}} \left(\mu \left(q _ {t}\right)\right) \right\} \tag {10}
$$
for all  $t$  and  $\eta \leq \eta_0$ . Then, an algorithm based on the EXO approach achieves the following regret bounds simultaneously: (i) In adversarial environments, we have  $R_{T} = O\left(\sqrt{\alpha d T \log T} + \frac{d \log T}{\eta_{0}}\right)$ . (ii) In environments satisfying (2), we have  $R_{T} = O\left(\left(\beta + \frac{1}{\eta_{0}}\right) d \log T + \sqrt{C \beta d \log T}\right)$ .
In the proof of this theorem, we consider the algorithm that chooses  $(p_t,g_t)$  to be an optimal solution to (9) and that sets learning rates  $\eta_{t}$  by  $\eta_{t} = \min \left\{\eta_{0},\sqrt{\frac{d\log T}{\alpha + \sum_{s = 1}^{t - 1}\eta_{s}^{-1}\Lambda_{q_{s},\eta_{s}}^{*}}}\right\}$ . A complete proof is given in the appendix.
Remark 2. In the implementation of the algorithm, we do not necessarily need to solve the optimization problem (9) exactly. In fact, to achieve the regret upper bounds in Theorem 1, it is sufficient to compute  $p_t, g_t$ , and  $z_t > 0$  that satisfy  $\eta_t^{-1} \Lambda_{q_t, \eta_t}(p_t, g_t) \leq z_t \leq \min \{ \alpha, \beta \cdot \Delta_{\ell^*}(\mu(q_t)) \}$  for some  $\alpha > 0$  and  $\beta > 0$ , under the assumption of  $\eta_t \leq \eta_0$ . In this case, setting  $\eta_t = \min \{ \eta_0, \sqrt{d \log T} / \sqrt{\alpha + \sum_{s=1}^{t-1} z_s} \}$  will work.
4.3 Regret bounds for concrete examples
In this section, we will see how we can bound  $\Lambda_{q_t,\eta_t}^*$ . For  $q\in \mathcal{P}(\mathcal{X})$ , define  $\omega (q),\omega '(q)\geq 0$  by
$$
\omega (q) = \min  _ {p \in \mathcal {P} (\mathcal {X}): \mu (p) = \mu (q)} \left\{H (p) ^ {- 1} \bullet V (q) \right\}, \tag {11}
$$
$$
\omega^ {\prime} (q) = \min  _ {p \in \mathcal {P} (\mathcal {X}): \mu (p) = \mu (q)} \min  \{y > 0 \mid d \cdot V (q) \preceq y \cdot H (p) \}. \tag {12}
$$
Note that  $\omega(q) \leq \omega'(q)$  holds. In fact, from the definition of  $\omega'(q)$ , there exists  $p \in \mathcal{P}(\mathcal{X})$  satisfying  $\mu(p) = \mu(q)$  and  $d \cdot V(q) \preceq \omega'(q) \cdot H(p)$ . For such a distribution  $p$ , we have  $H(p)^{-1} \bullet V(q) = \operatorname{tr}(H(p)^{-1/2}V(q)H(p)^{-1/2}) \leq \operatorname{tr}(d^{-1} \cdot \omega'(q) \cdot I) = \omega'(q)$ , which implies that  $\omega(q) \leq \omega'(q)$ .
Using these, we can construct a bound  $\Lambda_{q,\eta}^{*}$  as the following lemma provides:
Lemma 2. Suppose that  $q$  is a log-concave distribution. If  $\eta \leq 1 / d$ , we then have  $\Lambda_{q,\eta}^{*} = O(\eta \omega(q))$ . If  $\eta \leq 1$ , we then have  $\Lambda_{q,\eta}^{*} = O(\eta \omega'(q))$ .
Bounds on  $\Lambda_{q,\eta}^{*}$  in this lemma can be achieved by  $p = \gamma \tilde{p} + (1 - \gamma)\pi_0$  and  $g$  defined by (4), where  $\tilde{p}$  is the minimizer on the right-hand side of (11) or (12),  $\pi_0$  is a G-optimal design for  $\mathcal{X}$ , and some mixing weight parameter  $\gamma \in (0,1)$ . The proof of this lemma is given in the appendix.
General action set We can provide an upper bound on  $\omega(q)$  by exploiting the property of log-concave distributions. A distribution  $q \in \mathcal{P}(\mathcal{X})$  is called a log-concave distribution if it has a distribution function that is proportional to  $\exp(-h(x))$  for some convex function  $h$ . Note that the reference distribution  $q_t$  is a log-concave function.
Lemma 3. Suppose that  $q$  is a log-concave distribution. Let  $\ell^{*} \in \mathcal{L}$  be an arbitrary loss vector such that  $a^{*} \in \arg \min_{a \in \mathcal{A}} \langle \ell^{*}, a \rangle$  is unique. Denote  $\Delta_{\ell^{*}, \min} = \min_{a \in \mathcal{A} \setminus \{a^{*}\}} \Delta_{\ell^{*}}(a)$ . We then have  $\omega'(q) = O(d \cdot \min \{1, \Delta_{\ell^{*}}(\mu(q)) / \Delta_{\ell^{*}, \min}\})$ .
We here provide a proof sketch for this lemma. We consider a distribution  $p \in \mathcal{P}(\mathcal{A})$  of  $a \in \mathcal{A}$  generated by the following procedure: (i) Pick  $x \in \mathcal{X}$  following  $q$ . (ii) Let  $\xi$  be an arbitrary distribution over  $\mathcal{A}$  such that  $\mu(\xi) = x$ . In other words, compute  $\xi : \mathcal{A} \to \mathbb{R}_{\geq 0}$  such that  $\sum_{a' \in \mathcal{A}} \xi(a') = 1$  and  $\sum_{a' \in \mathcal{A}} \xi(a') a' = x$ . (iii) Pick  $a \in \mathcal{A}$  following  $\xi$ . It is then clear that  $\mu(p) = \mu(q)$  holds. Further, it follows from the assumption that  $q$  is a log-concave distribution that  $(1 - p(a^{*})) u^{\top} V(p) u = \Omega(u^{\top} V(q) u)$  holds for any  $a^{*} \in \mathcal{A}$  and any  $u \in \mathbb{R}^d \setminus \{0\}$ , which implies  $d \cdot V(q) \preceq O((1 - p(a^{*})) d) \cdot V(p) \preceq O((1 - p(a^{*})) d) H(p)$ . Finally, since we have  $(1 - p(a^{*})) \leq \Delta_{\ell^{*}}(\mu(p)) / \Delta_{\ell^{*},\min}$ , the bound on  $\omega'(q)$  in Lemma 3 follows. A full proof can be found in the appendix.
From Lemmas 2 and 3, we obtain the following bounds on  $\alpha$  and  $\beta$ :
Corollary 1. For arbitrary action set  $\mathcal{A}$ , the condition (10) in Theorem 1 holds with  $\alpha = O(d)$ ,  $\beta = O(d / \Delta_{\ell^{*},\min})$ , and  $\eta_0 = O(1)$ .
Hence, there exists an algorithm that achieves  $R_{T} = O(d\sqrt{T\log T})$  in adversarial environments and  $R_{T} = O(\frac{d^{2}}{\Delta_{\ell^{*},\min}}\log T + \sqrt{\frac{Cd^{2}}{\Delta_{\ell^{*},\min}}\log T})$  in stochastically constrained adversarial environments.
Multi-armed bandits Suppose a problem setting in which the action set is the canonical bases:  $\mathcal{A} = \mathcal{A}_d\coloneqq \{e_1,e_2,\ldots ,e_d\} = \{a\in \{0,1\} ^d\mid \| a\| _1 = 1\}$ . For this action set, we have the following bound on  $\omega (q)$ :
Lemma 4. If  $\mathcal{A} = \mathcal{A}_d$ , for any log-concave distribution  $q$ ,  $\omega(q)$  defined in (11) is bounded as  $\omega(q) \leq O\left(\sum_{i=1}^{d} \mu_i(q)(1 - \mu_i(q))\right) \leq O\left(\min_{1 \leq i \leq d}\{1 - \mu_i(q)\}\right)$ .
Let  $\ell^{*} \in \mathcal{L}$  be an a loss vector such that  $e_{i^{*}} \in \arg \min_{1 \leq i \leq d} \ell_{i}^{*}$  is unique. We then have  $\Delta_{\ell^{*}}(\mu) = \sum_{i \neq i^{*}} \Delta_{\ell^{*}}(e_{i}) \mu_{i} \geq \Delta_{\ell^{*}, \min}(1 - \mu_{i^{*}})$ , which, combined with Lemma 4, implies  $\omega(q) \leq \Delta_{\ell^{*}}(\mu(q)) / \Delta_{\ell^{*}, \min}$ . From this and Lemma 2, we have the following bounds on  $\alpha$  and  $\beta$ :
Corollary 2. For the multi-armed bandit problem  $\mathcal{A} = \mathcal{A}_d$ , the condition (10) in Theorem 1 holds with  $\alpha = O(1)$ ,  $\beta = O(1 / \Delta_{\ell^{*},\min})$  and  $\eta_0 = O(1 / d)$ .
Product-space action sets Suppose that  $\mathcal{A} \subseteq \mathbb{R}^d$  is expressed as a product space of  $m$  sets, i.e.,  $\mathcal{A} = \mathcal{A}^{(1)} \times \mathcal{A}^{(2)} \times \dots \times \mathcal{A}^{(m)}$ , where  $\mathcal{A}^{(j)} \subseteq \mathbb{R}^{d_j}$  for each  $j$  and  $\sum_{j=1}^{m} d_j = d$ . Denote  $\mathcal{X}^{(j)} = \mathrm{conv}(\mathcal{A}^{(j)})$ . This setting is referred to as the multi-task bandit problem [Cesa-Bianchi and Lugosi, 2012], in which the player chooses  $a_t^{(j)} \in \mathcal{A}^{(j)}$  for each  $j$  in parallel, and then gets only a single feedback corresponding to the sum of the losses for all  $m$  actions. For such an action set, probability distribution  $q_t$  given by continuous exponential weights can be expressed as a product measure of  $m$  log-concave measures  $q_t^{(j)} \in \mathcal{P}(\mathcal{X}^{(j)})$ . In fact, if a probability density function is expressed as  $q(x) = c \exp(\langle \eta L, x \rangle)$  for some  $c > 0$ $\eta > 0$ , and  $L = (L^{(1)}, \ldots, L^{(m)}) \in \mathbb{R}^d$ , it can be decomposed as  $q(x) = c \exp(\sum_{j=1}^{m} \langle \eta L^{(j)}, x^{(j)} \rangle) = \prod_j^m c^{(j)} \exp(\langle \eta L^{(j)}, x^{(j)} \rangle) = \prod_j^m q^{(j)}(x^{(j)})$ , where  $x^{(j)} \in \mathcal{X}^{(j)}$  and  $c^{(j)} = \int_{x \in \mathcal{X}^{(j)}} \exp(\langle \eta L^{(j)}, x^{(j)} \rangle) \, \mathrm{d}x$ . This fact leads to the following bounds on  $\omega(q)$  and  $\Lambda_{q,\eta}^*$ :
Lemma 5. If  $\mathcal{A} = \mathcal{A}^{(1)} \times \dots \times \mathcal{A}^{(m)}$  and  $q \in \mathcal{P}(\mathcal{X})$  can be expressed as a product measure of  $q^{(j)} \in \mathcal{P}(\mathcal{X}^{(j)})$  for  $j = 1, \ldots, m$ , we have  $\omega(q) \leq \sum_{j=1}^{m} \omega(q^{(j)})$ .
Corollary 3. If  $\mathcal{A} = \mathcal{A}_{d_1}\times \dots \times \mathcal{A}_{d_m}$ , the condition (10) in Theorem 1 holds with  $\alpha = O(m)$ ,  $\beta = O(1 / \Delta_{\ell^{*},\min})$ , and  $\eta_0 = O(1 / d)$ . If  $\mathcal{A} = \{0,1\}^d$ , the condition (10) in Theorem 1 holds with  $\alpha = O(d)$ ,  $\beta = O(1 / \Delta_{\ell^{*},\min})$ , and  $\eta_0 = O(1 / d)$ .
4.4 Computational complexity
At this time, it is not known if there is an efficient algorithm to solve the optimization problems (9). However, it is worth noting that the optimization problem corresponding to the right-hand side of (11) is a convex optimization problem. Further, as the minimum can be achieved by  $p \in \mathcal{P}(\mathcal{A})$ , we can reduce the problem into a convex optimization problem over  $|\mathcal{A}|$ -dimensional space, which can be solved efficiently if  $|\mathcal{A}|$  is not very large.
Note that it is not always necessary to solve the optimization problem (9) exactly, as mentioned in Remark 2. For example, we can implement an algorithm achieving regret bounds of Corollary 1 given a separation oracle (or equivalently, a linear optimization oracle) for  $cA$ , without solving (9) exactly. Indeed, to achieve the regret bounds of Corollary 1, it suffices to find  $p$  and  $g$  such that  $\Lambda_{q_t,\eta_t}(p,g)$  is bounded by the RHS of (10). The construction of such  $p$  and  $g$  is provided in the proof of Lemmas 2 and 3, which can be performed using a separation oracle for  $\mathcal{A}$ . In fact, we can obtain samples from  $p$  by using the techniques for log-concave sampling (e.g., [Lovasz and Vempala, 2007]) and for computing convex combination expression (cf. Carathéodory's theorem for convex hull and [Schrijver, 1998, Corollary 14.1g]). However, the analysis of log-concave sampling and calculations of  $H(p)$  (which is required for constructing  $g$ ) including consideration of calculation errors can be highly complicated, and the computational cost can be very large, although on the order of polynomials (e.g., [Hazan and Karnin, 2016, Corollary 6.2]).
5 Mean-oriented FTRL and SCRiBLe
For linear bandit problems, we have an alternative approach that we referred to as mean-oriented FTRL, in which we compute a reference mean  $w_{t}$  given as follows:
$$
w _ {t} \in \underset {w \in \mathcal {X}} {\arg \min } \left\{\left\langle \sum_ {s = 1} ^ {t - 1} h _ {s} \left(a _ {s}, f _ {s}\right), w \right\rangle + \frac {1}{\eta_ {t}} \psi (w) \right\}, \tag {13}
$$
where  $h_s(a_s, f_s)$  is an estimator of  $\ell_s$  and  $\psi$  is a convex regularization function over  $\mathcal{X}$  such that  $\min_{x \in \mathcal{X}} \psi(x) = 0$ . Let  $D_{\psi}(x', x)$  denote the Bregman divergence associated with  $\psi$ :  $D_{\psi}(x', x) = \psi(x') - \psi(x) - \langle \nabla \psi(x), x' - x \rangle$ . This approach ensures the following regret bound:
Lemma 6. For any point  $x^{*}\in \mathcal{X}$  , we have
$$
R _ {T} (x ^ {*}) \leq {\bf E} \left[ \sum_ {t = 1} ^ {T} \left(\langle \ell_ {t}, \mu (p _ {t}) - w _ {t} \rangle + \mathrm {b i a s} (h _ {t}, p _ {t}, w _ {t}, \ell_ {t}) + \frac {\mathrm {s t a b} _ {\psi} (\eta_ {t} h _ {t} (a _ {t} , f _ {t}) , w _ {t})}{\eta_ {t}}\right) + \frac {\psi (x ^ {*})}{\eta_ {T + 1}} \right],
$$
$$
w h e r e \quad \mathrm {b i a s} (h, p, w, \ell) = \sup _ {a ^ {*} \in \mathcal {X}, M \in \mathcal {M} _ {\ell}} \left\{\mathbf {E} _ {a \sim p, f \sim M _ {a}} [ \langle \ell - h (a, f), w - a ^ {*} \rangle ] \right\},
$$
$$
\operatorname {s t a b} _ {\psi} (\hat {\ell}, w) = \sup  _ {w ^ {\prime} \in \mathcal {X}} \left\{\left\langle \hat {\ell}, w - w ^ {\prime} \right\rangle - D _ {\psi} \left(w ^ {\prime}, w\right) \right\}. \tag {14}
$$
One example of linear bandit algorithms based on mean-oriented FTRL is the SCRiBLe algorithm [Abernethy et al., 2008, 2012], which employs self-concordant barriers as regularizer functions.
We can also consider an EXO approach for mean-oriented FTRL. For any reference point  $w \in \mathcal{X}$  and learning rate  $\eta > 0$ , define
$$
\Lambda_ {w, \eta} (p, h) = \sup _ {\ell \in \mathcal {L}, M \in \mathcal {M} _ {\ell}} \left\{\langle \ell , \mu (p) - w \rangle + \mathrm {b i a s} (h, p, w, \ell) + \frac {1}{\eta} \underset {a \sim p, f \sim M _ {a}} {\mathbf {E}} [ \mathrm {s t a b} _ {\psi} (\eta h (a, f), q) ] \right\}
$$
and  $\Lambda_{w,\eta}^{*} = \inf_{p\in \mathcal{P}(\mathcal{X}),\hat{\ell}:\mathcal{X}\times \mathbb{R}\to \mathbb{R}^{d}}\Lambda_{w,\eta}(p,\hat{\ell})$
This achieves the following BOBW regret bounds, under assumptions on  $\Lambda_{w,\eta}^{*}$ :
Lemma 7. Suppose that  $\psi$  is a  $\nu$ -self-concordant barrier. Suppose that there exist  $\alpha > 0, \beta > 0$  and  $\eta_0 > 0$  such that
$$
\Lambda_ {w _ {t}, \eta} ^ {*} \leq \eta \cdot \min  \left\{\alpha , \beta \cdot \Delta_ {\ell^ {*}} \left(w _ {t}\right) \right\} \tag {15}
$$
for all  $t$  and  $\eta \leq \eta_0$ . Then an algorithm based on the EXO approach achieves the following regret bounds simultaneously: (i) In adversarial environments, we have  $R_{T} = O\left(\sqrt{\alpha\nu T\log T} +\frac{\nu\log T}{\eta_{0}}\right)$ . (ii) In environments satisfying (2), we have  $R_{T}(a^{*}) = O\left(\left(\beta +\frac{1}{\eta_{0}}\right)\nu \log T + \sqrt{C\beta\nu\log T}\right)$ .
5.1 Connection to the SCRiBLe algorithm and variants of it
Abernethy et al. [2008, 2012] have proposed a sampling scheme  $w_{t} \mapsto p_{t}$  supported on the Dikin ellipsoid, which leads to  $\Lambda_{w_t,\eta_t}(p_t,h_t) = O(\eta_t d^2)$  with an appropriately defined unbiased estimator  $h_t$ . This algorithm referred to as SCRiBLe hence achieves  $R_{T} = O(\sqrt{d^{2}\nu T\log T})$ , the adversarial regret bound in Lemma 7 with  $\alpha = O(d^{2})$ . Recent studies by Dann et al. [2023b, Theorem 3] and Ito and Takemura [2023] have shown that a modified sampling scheme based on the Dikin ellipsoid sampling achieves  $\Lambda_{w_t,\eta_t}(p_t,h_t) = O\left(\eta_t d^2 \cdot \min \{1,\Delta_{\ell^*}(w_t) / \Delta_{\ell^*,\min}\}\right)$ . Consequently, these modified algorithms achieve BOBW regret bounds in Lemma 7 with  $\alpha = O(d^{2})$  and  $\beta = O(d^{2} / \Delta_{\ell^{*},\min})$ . These results suggest that the following lemma holds:
Lemma 8 ([Dann et al., 2023b, Ito and Takemura, 2023]). If  $\psi$  is a self-concordant barrier, (15) holds with  $\alpha = O(d^{2})$ ,  $\beta = O(d^{2} / \Delta_{\ell^{*},\min})$  and  $\eta_0 = O(1 / d)$ .
5.2 Connection to EXO with continuous exponential weights
The entropic barrier is a  $\nu$ -self-concordant barrier with  $\nu \leq d$  [Chewi, 2021], for which a definition and properties are given, e.g., in [Bubeck and Eldan, 2019]. We can see that mean-oriented EXO with entropic-barrier regularization coincides with the EXO approach with exponential weights. In fact, if we compute  $w_{t}$  using (13) with  $\psi$  the entropic barrier, from the definition of the entropic barrier,  $w_{t}$  is equal to the mean of the distribution  $q_{t}$  given by the continuous exponential weights. Given this correspondence, we can see that bounds on  $\Lambda_{w_t,\eta}^*$  follow immediately from Corollary 1:
Lemma 9. If  $\psi$  is the entropic barrier for  $\mathcal{X}$ , (15) holds with  $\alpha = O(d)$ ,  $\beta = O(d / \Delta_{\ell^{*},\min})$  and  $\eta_0 = O(1)$ .
It is worth noting that these bounds are  $O(1 / d)$ -times better than the results for Lemma 8 that follow from previous studies.
6 Limitations and future work
A limitation of this work is that the regret bounds for (corrupted) stochastic environments require the assumption that the optimal arm  $a^{*} \in \arg \min_{a \in \mathcal{A}} \langle \ell^{*}, a \rangle$  is unique. While this assumption is common in analyses of best-of-both-worlds algorithms based on the self-bounding technique [Zimmert and Seldin, 2021, Dann et al., 2023b], a limited number of studies on the multi-armed bandit problem [Ito, 2021, Jin et al., 2023] have removed this uniqueness assumption by careful analyses of regret bounds for follow-the-regularized-leader methods. As these analyses rely on structures specific to the multi-armed bandit problems, extending them to other problem settings does not seem trivial. Extending this analysis technique to linear bandits and removing the uniqueness assumptions will be an important future challenge.
Another future direction is to work toward a tighter regret bound of  $O(c^{*}\log T)$  in stochastic environments while satisfying BOBW regret guarantee, where  $c^*$  is an instance-dependent quantity that characterizes the optimal asymptotic bound (see, e.g., [Lattimore and Szepesvari, 2017, Corollary 2]). As  $c^* \leq O(d / \Delta_{\mathrm{min}})$  [Lee et al., 2021, Lemma 16.] and the gap between these two quantities can be arbitrarily large [Lattimore and Szepesvari, 2017, Example 4], our bounds depending on  $1 / \Delta_{\mathrm{min}}$  can be far from tight. In the multi-armed bandit problem, a special case of linear bandits, the quantity  $c^*$  can be expressed as  $c^* = \sum_{a \in A: \Delta_{\ell^*}(a) > 0} \frac{2}{\Delta_{\ell^*}(a)}$ , and known FTRL-type BOBW algorithms with  $O(c^{*}\log T)$ -regret bounds employs the Tsallis entropy with  $\Theta(1/\sqrt{t})$ -learning rates:  $\psi_t(w) = -\sqrt{t}\sum_{i=1}^{d}\sqrt{w_i}$  [Zimmert and Seldin, 2021] or logarithmic barriers with entry-wise adaptive learning rates:  $\psi_t(w) = -\sum_{i=1}^{d}\frac{1}{\eta_{ti}}\log w_i$  [Ito, 2021, Ito et al., 2022a]. The latter would be more closely related to our results in this paper as logarithmic barriers are examples of self-concordant barriers. A major difference between the approach in this paper and the one by Ito [2021] is that the former only considers regularizer functions expressed as a scalar multiple of a fixed function, while the latter even changes the shape of regularizer functions for each round. Such a more flexible adaptive regularization may be necessary for pursuing tighter regret bounds. Another promising approach is to extend the Tsallis-entropy approach to linear bandits while the connection between the Tsallis-INF [Zimmert and Seldin, 2021] and this paper may be somewhat tenuous as the Tsallis entropy is not a self-concordant barrier.
J. Abernethy, E. Hazan, and A. Rakhlin. Competing in the dark: An efficient algorithm for bandit linear optimization. In Conference on Learning Theory, 2008.
J. D. Abernethy, E. Hazan, and A. Rakhlin. Interior-point methods for full-information and bandit online learning. IEEE Transactions on Information Theory, 58(7):4164-4175, 2012.
I. Amir, G. Azov, T. Koren, and R. Livni. Better best of both worlds bounds for bandits with switching costs. Advances in Neural Information Processing Systems, 35, 2022.
O. Besbes, Y. Gur, and A. Zeevi. Optimal exploration-exploitation in a multi-armed bandit problem with non-stationary rewards. Stochastic Systems, 9(4):319-337, 2019.
S. Bubeck and R. Eldan. The entropic barrier: Exponential families, log-concave geometry, and self-concordance. Mathematics of Operations Research, 44(1):264-276, 2019.
S. Bubeck and A. Slivkins. The best of both worlds: Stochastic and adversarial bandits. In Conference on Learning Theory, pages 42-1. PMLR, 2012.
S. Bubeck, N. Cesa-Bianchi, and S. M. Kakade. Towards minimax policies for online linear optimization with bandit feedback. In Conference on Learning Theory, pages 41-1. JMLR Workshop and Conference Proceedings, 2012.
N. Cesa-Bianchi and G. Lugosi. Combinatorial bandits. Journal of Computer and System Sciences, 78(5):1404-1422, 2012.
S. Chewi. The entropic barrier is  $n$ -self-concordant. arXiv preprint arXiv:2112.10947, 2021.
V. Dani, S. M. Kakade, and T. P. Hayes. The price of bandit information for online optimization. In Advances in Neural Information Processing Systems, pages 345-352, 2008.