SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1 BARONS: Barrier-Regularized Online Newton Step
Require:
- Parameters  $\eta, \varepsilon, \alpha > 0$ , and  $m_{\mathrm{Newton}} \geq 1$ .
- Gradient/Hessian Oracles  $\mathcal{O}_{\varepsilon}^{\mathrm{grad}} / \mathcal{O}_{\alpha}^{\mathrm{hess}}$  for self-concordant function  $\Phi$  with constant  $M_{\Phi} > 0$ .
-  $w^{\star} \in \operatorname*{argmin}_{w \in \mathcal{K}} \Phi(w)$ .
1: Set  $u_1 = w_1 = w^{\star}$ ,  $H_1 = \mathcal{O}_{\alpha}^{\mathrm{hess}}(w^{\star})$ , and  $s_0 = 0$ .
2: for  $t = 1, \ldots, T$  do
3: Play  $w_t$  and observe  $g_t \in \partial \ell_t(w_t)$ .
4: Set  $s_t = s_{t-1} + \eta g_t$ .
5: Set  $w_{t+1}^1 = w_t$ .
// Perform intermediate Newton steps to decrease the Newton decrement
6: for  $m = 1, \ldots, m_{\mathrm{Newton}} - 1$  do
7: Set  $\widehat{\nabla}_{t+1}^m = \mathcal{O}_{\varepsilon}^{\mathrm{grad}}(w_{t+1}^m)$ .
8: Set  $\widetilde{\nabla}_{t+1}^m = \widehat{\nabla}_{t+1}^m + s_t$ . //  $\widetilde{\nabla}_{t+1}^m \approx \nabla \Phi_{t+1}(w_{t+1}^m)$ 
9: Set  $w_{t+1}^{m+1} = w_{t+1}^m - H_t^{-1}\widetilde{\nabla}_{t+1}^m$ .
10: end for
11: Set  $w_{t+1} = w_{t+1}^{m+1}$ .
// Check if the landmark needs updating
12: if  $\|w_{t+1} - u_t\|_{H_t} \leq 1/(41M_\Phi)$  then
13: Set  $u_{t+1} = u_t$ . // Update landmark
14:  $H_{t+1} = H_t$  and  $H_{t+1}^{-1} = H_t^{-1}$ .
15: else
16: Set  $u_{t+1} = w_{t+1}$ . // No landmark update
17: Set  $H_{t+1} = \mathcal{O}_{\alpha}^{\mathrm{hess}}(u_{t+1})$  and compute  $H_{t+1}^{-1}$ .
18: end if
19: end for
3.3 Regret Guarantee Under Local and Euclidean Norm Bounds on the Sub-Gradients
We now instantiate the guarantee in Theorem 5 with a  $(M_{\Phi},\nu)$ -self-concordant barrier  $\Phi$  for the set  $\mathcal{K}$ , with respect to which the local norms of the sub-gradients are bounded; that is, when  $\| g_t\|_{\nabla^{-2}\Phi (w_t)}\leq b$ . We note that the regret bound in (36) has an additive  $\Phi (w)$  which may be unbounded near the boundary of  $\mathcal{K}$ . However, it is still possible to compete against comparators in int  $\mathcal{K}$  by making additional assumptions on the range of the losses [27, 32]. We discuss some of these assumptions in the sequel. For the next theorem, we will state the regret bound of BARONS relative to comparators in the restricted set:
$$
\mathcal {K} _ {c} := (1 - c) \mathcal {K} \oplus \{c w ^ {\star} \}, \tag {4}
$$
where  $\oplus$  denotes the Minkowski sum,  $w^{\star}\in \mathrm{argmin}_{w\in \mathcal{K}}\Phi (w)$ , and  $c\in (0,1)$  is a parameter.
With this, we now state a regret bound for BARONS when the sub-gradients of the losses have bounded local norms. The proof of the next theorem is in Appendix C.2.
Theorem 6 (Local norm bound). Let  $\Phi$  be an  $(M_{\Phi},\nu)$ -self-concordant barrier for  $\mathcal{K}$  and let  $c\in (0,1),b > 0$ . Further, suppose that for all  $t\in [T]$ ,  $\| g_t\|_{\nabla^{-2}\Phi (w_t)}\leq b$ , where  $(w_{t})$  are the iterates of BARONS with input parameters  $(\eta ,\varepsilon ,\alpha ,m_{\mathrm{Newton}})$  such that
$$
\eta := \sqrt {\frac {\nu \log c}{b ^ {2} T}}, \quad \varepsilon := \sqrt {\frac {\nu}{T}}, \quad \alpha := 0. 0 0 1, \quad a n d \quad m _ {\text {N e w t o n}} := \Theta \left(\log \frac {1}{\varepsilon M _ {\Phi}}\right). \tag {5}
$$
For  $T \geq 1$  large enough such that  $\eta \leq \frac{1}{1000bM_{\Phi}}$ ,  $\varepsilon \leq \frac{1}{20000M_{\Phi}}$ , the regret of BARONS is bounded as
$$
\operatorname {R e g} _ {T} ^ {\text {B A R O N S}} (w) \lesssim b \sqrt {\nu T \log c}, \quad \forall w \in \mathcal {K} _ {c}, \tag {6}
$$
where  $\mathcal{K}_c\subset \mathcal{K}$  is as in (4). Further, the computational complexity of BARONS in this case is bounded by
$$
O \left(\left(\mathcal {C} _ {\varepsilon} ^ {\mathrm {g r a d}} + d ^ {2}\right) \cdot T \cdot \log \frac {T}{\nu M _ {\Phi}} + \mathcal {C} _ {\alpha} ^ {\mathrm {h e s s}} \cdot M _ {\Phi} \sqrt {T \nu \log c}\right).
$$
Remark 1. The regret bound in Theorem 6 is stated with respect to comparators in the restricted set  $\mathcal{K}_c$  defined in (4). It is possible to extend this guarantee to all comparators in int  $\mathcal{K}$  under an additional assumption on the range of the losses. For example, if for  $w^{\star} \in \mathrm{argmin}_{w \in \mathcal{K}} \Phi(w)$ , we have
$$
\sup  _ {w \in \operatorname {i n t} \mathcal {K}, t \in [ T ]} \ell_ {t} \left(\left(1 - \frac {1}{T}\right) \cdot w + \frac {1}{T} \cdot w ^ {\star}\right) - \ell (w) \leq O \left(\frac {1}{\sqrt {T}}\right), \tag {7}
$$
then the regret guarantee in (6) can be extended to all comparators in int  $\mathcal{K}$  up to an additive  $O(\sqrt{T})$  term (see Lemma 7 in the appendix). In this case, the log  $c$  term in the computational complexity need be replaced by  $\log T$ . We note that the condition in (7) does not require a uniform bound on the losses. Instead, it only restricts the rate of growth of the losses  $(\ell_t(w))$  as  $w$  approaches the boundary of  $\mathcal{K}$ . As we show in the sequel (§4.2), (7) is satisfied for some popular losses which are not Lipschitz.
We now instantiate the guarantee in Theorem 5 when the sub-gradients are bounded in Euclidean norm (instead of local norm); that is, we assume that for all  $t \in [T]$ ,  $\| g_t \| \leq G$  for some  $G > 0$ . We note that this assumption implies (7), and we will be able to bound the regret against all comparators in int  $\mathcal{K}$  as alluded to in Remark 1. The proof of the next theorem is in Appendix C.3).
Theorem 7 (Euclidean norm bound). Let  $\Psi$  be an  $(M_{\Psi},\nu)$  self-concordant barrier for  $\mathcal{K}$  and let  $\Phi (\cdot)\coloneqq \Psi (\cdot) + \frac{\nu}{2R^2}\| \cdot \| ^2$ . Further, let  $G,R > 0$  and suppose that  $\mathcal{K}\subseteq \mathcal{B}(R)$  and for all  $t\in [T]$ ,  $\| g_t\| \leq G$  where  $g_{t}\in \partial \ell_{t}(w_{t})$  and  $(w_{t})$  are the iterates of BARONS with input parameters  $(\eta ,\varepsilon ,\alpha ,m_{\mathrm{Newton}})$  such that
$$
\eta := \frac {\nu}{R G} \sqrt {\frac {\log T + 1}{T}}, \quad \varepsilon := \sqrt {\frac {\nu}{T}}, \quad \alpha := 0. 0 0 1, \quad a n d \quad m _ {\mathrm {N e w t o n}} := \Theta \left(\log \frac {1}{\varepsilon M _ {\Psi}}\right). \tag {8}
$$
For  $T \geq 1$  large enough such that  $\eta \leq \frac{1}{1000GM_{\Psi}}$ ,  $\varepsilon \leq \frac{1}{20000M_{\Psi}}$ , the regret of BARONS is bounded as
$$
\operatorname {R e g} _ {T} ^ {\mathsf {B A R O N S}} (w) \lesssim R G \sqrt {T \log T}, \quad \forall w \in \operatorname {i n t} \mathcal {K}. \tag {9}
$$
Further, the computational complexity of BARONS in this case is bounded by
$$
O \left(\left(\mathcal {C} _ {\varepsilon} ^ {\operatorname {g r a d}} (\Psi) + d ^ {2}\right) \cdot T \cdot \log \frac {T}{\nu M _ {\Psi}} + \mathcal {C} _ {\alpha} ^ {\text {h e s s}} (\Psi) \cdot M _ {\Psi} \sqrt {T \nu \log T}\right).
$$
4 Application to Polytopes Using the Lee-Sidford Barrier
In this section, we assume that the set  $\mathcal{K}$  is a polytope in  $\mathbb{R}^d$  specified by  $m$  linear constraints:
$$
\mathcal {K} = \left\{w \in \mathbb {R} ^ {d} \mid \forall i \in [ m ], a _ {i} ^ {\top} w \geq b _ {i} ^ {\prime} \right\}, \tag {10}
$$
and we construct efficient gradient and Hessian Oracles for a self-concordant barrier for  $\mathcal{K}$ . This will then allow us to instantiate the guarantees of BARONS in Section 3 and provide explicit and state-of-the-art bounds on the regret of BARONS.
We will assume without loss of generality that  $\| a_i\| = 1$ , for all  $i\in [m]$ , and let  $A\coloneqq (a_{1},\ldots ,a_{m})^{\top}\in \mathbb{R}^{m\times d}$  denote the constraint matrix of the set  $\kappa$ . For the rest of this section, it will be convenient to define the "slack" variables  $s_{w,i} = a_i^\top w - b_i'$ , for  $i\in [m]$ . Here,  $s_{w,i}$  essentially represents the distance of  $w$  to the  $i$ th facet of the polytope  $\kappa$ . Further, we let  $S_w\coloneqq \mathrm{diag}(s_w)$  be the diagonal matrix whose  $i$ th diagonal entry is  $s_{w,i}$ .
The LS barrier. To perform Online Convex Optimization over  $\mathcal{K}$ , we pick the regularizer  $\Phi$  of BARONS to be the Lee-Sidford (LS) barrier  $\Phi^{\mathrm{LS}}$  [22] with parameter  $p > 0$ , which is defined as
$$
\Phi^ {\mathrm {L S}} (v) = \min  _ {v \in \mathbb {R} _ {> 0} ^ {m}} - \log \det  (A ^ {\top} S _ {w} V S _ {w} A) + \frac {1}{1 + p ^ {- 1}} \operatorname {t r} (V ^ {1 + 1 / p}),
$$
where  $V = \mathrm{diag}(v)$ . One way to think of the LS barrier is as a weighted log-barrier. As we will discuss in the sequel, this choice will confer computational and performance (in terms of regret) advantages over the standard log-barrier.
Self-concordance of the LS barrier. According to [8, Theorem 30], the LS barrier with the choice  $p = O(\log(m))$  is a self-concordant function with parameter  $M_{\Phi^{\mathrm{LS}}}$  satisfying
$$
M _ {\Phi^ {\mathrm {L S}}} = O \left(\log (m) ^ {2 / 5}\right) = \widetilde {O} (1),
$$
The other favorable property of this barrier is that its Newton decrement at any point  $w \in \mathcal{K}$  is of order  $\widetilde{O}(\sqrt{d})$ ; that is,
$$
\left\| \nabla \Phi^ {\mathrm {L S}} (w) \right\| _ {\nabla^ {- 2} \Phi^ {\mathrm {L S}} (w)} = \widetilde {O} (\sqrt {d}). \tag {11}
$$
Therefore,  $\Phi^{\mathrm{LS}}$  is a  $(\widetilde{O}(1), \widetilde{O}(d))$ -self-concordant barrier. For the log-barrier, the right-hand side of (11) would be  $\sqrt{m}$ .
Cost of gradient and Hessian Oracles. We consider the computational complexities of gradient and Hessian Oracles for  $\Phi^{\mathrm{LS}}$ . By [22], we have that for  $\varepsilon > 0$
$$
\mathcal {C} _ {\varepsilon} ^ {\operatorname {g r a d}} \left(\Phi^ {\mathrm {L S}}\right) \leq \widetilde {O} \left(\mathcal {C} ^ {\text {s y s}} \cdot \log \left(1 / \varepsilon\right)\right), \quad \text {a n d} \quad \mathcal {C} _ {\varepsilon} ^ {\text {h e s s}} \left(\Phi^ {\mathrm {L S}}\right) \leq \widetilde {O} \left(\mathcal {C} ^ {\text {s y s}} \sqrt {d} \cdot \log \left(1 / \varepsilon\right)\right),
$$
where  $\mathcal{C}^{\mathrm{sys}}$  is the computational cost of solving a linear system of the form  $A^{\top}\mathrm{diag}(v)Ax = y$ , for vectors  $v\in \mathbb{R}_{\geq 0}^{d}$  and  $y\in \mathbb{R}^{d}$ ; we recall that  $A = (a_{1},\ldots ,a_{m})^{\top}$  is the constraint matrix for  $\kappa$ . In the worst-case, such a linear system can be solved with cost bounded as
$$
\mathcal {C} ^ {\text {s y s}} \leq O \left(m d ^ {\omega - 1}\right), \tag {12}
$$
where  $\omega$  is the exponent of matrix multiplication, and  $m$  is the number of constraints of  $\mathcal{K}$ . However, as we show in the sequel,  $\mathcal{C}^{\mathrm{sys}}$  can be much smaller in many practical applications.
With this, we immediately obtain the following corollary for the regret and run-time of BARONS under local norm and Euclidean norm bounds on the sub-gradients.
Corollary 1 (OCO over a polytope with LS barrier). Let  $c \in (0,1), G, R, b > 0$ , and suppose  $\mathcal{K}$  is given by (10) and that  $\Phi^{\mathrm{LS}}$  is the corresponding LS barrier. Further, let  $(w_{t})$  be the iterates of BARONS, and let  $\mathcal{K}_{c}$  be the restricted version of  $\mathcal{K}$  defined in (4). Then, the following holds:
- Local norm bound: If  $\| g_t\|_{\nabla^{-2}\Phi (w_t)}\leq b,$  for all  $t\geq 1$  , and the parameters  $(\eta ,\varepsilon ,\alpha ,m_{\mathrm{Newton}})$  of BARONS are set as in Theorem 6 with  $\Phi = \Phi^{\mathrm{LS}}$  and  $(M_{\Phi},\nu) = (\widetilde{O} (1),\widetilde{O} (d))$  , then for  $T$  large enough (as specified in Theorem 6), the regret of BARONS is bounded by
$$
\operatorname {R e g} _ {T} ^ {\text {B A R O N S}} (w) \lesssim b \sqrt {d T \log c}, \quad \forall w \in \mathcal {K} _ {c}. \tag {13}
$$
- Euclidean norm bound: If  $\mathcal{K} \subseteq \mathcal{B}(R)$  and  $\| g_t\| \leq G$ , for all  $t \geq 1$ , and the parameters  $(\eta, \varepsilon, \alpha, m_{\mathrm{Newton}})$  of BARONS are set as in Theorem 7 with  $\Phi(\cdot) = \Phi^{\mathrm{LS}}(\cdot) + \frac{\nu}{2R^2}\|\cdot\|^2$  and  $(M_{\Psi}, \nu) = (\widetilde{O}(1), \widetilde{O}(d))$ , then for  $T$  large enough (as in Theorem 7) BARONS has regret bounded as
$$
\operatorname {R e g} _ {T} ^ {\text {B A R O N S}} (w) \lesssim R G \sqrt {T \log T}, \quad \forall w \in \operatorname {i n t} \mathcal {K}. \tag {14}
$$
In either case, the computational complexity is bounded by
$$
\widetilde {O} \left(\left(\mathcal {C} ^ {\text {s y s}} + d ^ {2}\right) \cdot T + \mathcal {C} ^ {\text {s y s}} \cdot d \sqrt {T}\right), \tag {15}
$$
where  $\mathcal{C}^{\mathrm{sys}}$  is the computational cost of solving a linear system of the form  $A^\top \mathrm{diag}(v)Ax = y$ , for vectors  $v \in \mathbb{R}_{\geq 0}^{d}$  and  $y \in \mathbb{R}^{d}$  (recall that  $A$  is the constraint matrix for the polytope  $\mathcal{K}$ ).
Using the log-barrier. We note that since  $\kappa$  is a polytope, we could have used the standard log-barrier
$$
\Phi^ {\log} (w) := \sum_ {i = 1} ^ {m} \log \left(b _ {i} ^ {\prime} - a _ {i} ^ {\top} w\right). \tag {16}
$$
This barrier is  $(1,m)$ -self-concordant, and so instantiating Theorem 5 with it would imply a  $\widetilde{O}(b\sqrt{m d T})$  regret bound in the case of local sub-gradient norms bounded by  $b > 0$ . Using the LS barrier replaces the  $\sqrt{m}$  term in this bound by  $\sqrt{d}$  regardless of the number of constraints—see
(13). However, this comes at a  $\mathcal{C}^{\mathrm{sys}}$  computational cost, which can be as high as  $md^{\omega - 1}$  in the worst-case (see (12)). In the case of the log-barrier, this cost would be replaced by  $md$  (essentially because  $\mathcal{C}_{\varepsilon}^{\mathrm{grad}}(\Phi^{\log}) \leq O(md)$ ). Thus, when  $m$  is of the order of  $d$ , using the log-barrier may be more computational-efficient compared to using the LS barrier. In the next corollary, we bound the regret of BARONS when  $\Phi = \Phi^{\log}$ ; this result is an immediate consequence of Theorem 7.
Corollary 2 (OCO over a polytope with the log barrier). Let  $G, b > 0$ , and suppose  $\mathcal{K}$  is given by (10) and that  $\Phi^{\log}$  is the corresponding log-barrier. Further, let  $(w_{t})$  be the iterates of BARONS. If  $\mathcal{K} \subseteq \mathcal{B}(R)$  and  $\| g_t\| \leq G$ , for all  $t \geq 1$ , and the parameters  $(\eta, \varepsilon, \alpha, m_{\mathrm{Newton}})$  of BARONS are set as in Theorem 7 with  $\Phi(\cdot) = \Phi^{\log}(\cdot) + \frac{\nu}{2R^2}\|\cdot\|^2$  and  $(M_{\Psi}, \nu) = (1, m)$ , then for  $T$  large enough (as in Theorem 7) BARONS has regret bounded as
$$
\operatorname {R e g} _ {T} ^ {\mathsf {B A R O N S}} (w) \lesssim R G \sqrt {T \log T}, \quad \forall w \in \operatorname {i n t} \mathcal {K}. \tag {17}
$$
The computational complexity is bounded by
$$
\widetilde {O} \left(\left(m d + d ^ {2}\right) \cdot T + m d ^ {\omega - 1} \sqrt {m T}\right). \tag {18}
$$
4.1 Implications for Lipschitz Losses
We now discuss implications of Corollary 1, and compare the bound of BARONS to those of existing algorithms for Lipschitz losses.
Dimension-free regret bound. We note when the Euclidean norms of the sub-gradients are bounded, BARONS achieves a dimension-free  $O(\sqrt{T})$  regret bound. In contrast, the best dimension-free regret bound<sup>4</sup> achieved by existing projection-free algorithms is of order  $O(T^{3/4})$  (see e.g. [14, 11]). We also note that existing separation/membership-based algorithms that achieve a  $\sqrt{T}$  regret; for examples those presented in [28, 11, 26], are not dimension-free. Their regret bounds are of order  $O(\kappa \sqrt{T})$ , where  $\kappa = R / r$  with  $r, R > 0$  such that  $\mathcal{B}(r) \subseteq \mathcal{K} \subseteq \mathcal{B}(R)$ . The asphericity parameter can depend on the dimension  $d$  [28], and even after a pre-conditioning step (which would involve putting the set  $\mathcal{K}$  into near-isotropic position and can cost up to  $\Omega(d^4)$  [25]),  $\kappa$  can be as large as  $d$  in the worst-case. Of course, to make a fair comparison with existing projection-free algorithms, we also need to take computational complexity into account. This is what we do next.
Computational cost. The computational cost in (15) should be compared with that of existing projection-free algorithms. For linear optimization-based projection-free algorithms, the computational cost after  $T$  rounds is typically of order  $C^{\mathrm{lin}} \cdot T$ , where  $C^{\mathrm{lin}}$  is the cost of performing linear optimization over  $\mathcal{K}$  which, for a polytope  $\mathcal{K}$ , reduces to solving a linear program. Using state-of-the-art interior point methods for solving such a linear program would cost  $C^{\mathrm{lin}} \leq \widetilde{O}(\sqrt{d} \cdot C^{\mathrm{sys}})$ ; see e.g. [22]. Thus, linear optimization-based projection-free algorithms can have a cost that is a factor  $\sqrt{d}$  worse than that of BARONS in the setting of Corollary 1. On the other hand, separation/membership-based algorithms, the computational cost scales with  $O(C^{\mathrm{sep}} \cdot T)$  after  $T$  rounds, where  $C^{\mathrm{sep}}$  is the cost of performing separation for the set  $\mathcal{K}$ . For a general polytope in  $\mathbb{R}^d$  with  $m$  constraints, we have  $C^{\mathrm{sep}} \leq O(md)$ , which may be smaller than  $C^{\mathrm{sys}}$  (the latter can be as large as  $md^{\omega-1}$  in the worse case; see (12)). Here, it may be more appropriate to compare against the computational guarantee of BARONS given in Corollary 2; by (18), we have that for  $T \geq d^{\omega-2}\sqrt{m}$ , the computational cost of BARONS in the setting of the corollary is dominated by  $(md + d^2) \cdot T$ , which is comparable to that of existing separation-based algorithms.
4.2 Implications for Non-Lipschitz Losses
Another advantage BARONS has over projection-free, and even projection-based, algorithms is that it has a regret bound that scales with a bound on the local norms of the gradients—see (13). We now showcase two online learning settings where this leads to non-trivial performance and computational improvements over existing OCO algorithms.
4The dependence in  $T$  can be improved under additional structure such as smoothness or strong-convexity of the losses.
This only concerns algorithms that use an interior point method to implement linear optimization over  $\mathcal{K}$ .
Online Portfolio Selection [5]. The portfolio selection problem is a classical online learning problem where the gradients of the losses can be unbounded. In this paragraph, we demonstrate how the guarantee of BARONS in Corollary 1 leads to a non-trivial guarantee for this setting both in terms of regret and computational complexity. In the online portfolio setting, at each round  $t$ , a learner (algorithm) chooses a distribution  $w_{t} \in \Delta_{d}$  over a fixed set of  $d$  portfolios. Then, the environment reveals a return vector  $r_{t} \in \mathbb{R}_{\geq 0}^{d}$ , and the learner suffers a loss
$$
\ell_ {t} \left(w _ {t}\right) := - \log w _ {t} ^ {\top} r _ {t}.
$$
The goal of the learner is to minimize the regret  $\mathrm{Reg}_T(w) \coloneqq \sum_{t=1}^T (\ell_t(w_t) - \ell_t(w))$  after  $T \geq 1$  rounds. For this problem, it is known that a logarithmic regret is achievable, but the specialized algorithms that achieve this have a computational complexity that scales with  $\min(d^3 T, d^2 T^2)$  [5, 27, 32, 41, 17]. On the other hand, applying the generic Online Gradient Descent or the Online Newton Step to this problem leads to regret bounds that scale with the maximum norm of the gradient (which can be unbounded). Instantiating the guarantees of BARONS in Corollary 1 with  $\Phi$  set to the standard log-barrier for the simplex $^6$ , in particular the bound in (13), to the online portfolio selection problem leads to an  $\widetilde{O}(\sqrt{dT})$  regret bound, which does not depend on the norm of the observed gradients. Furthermore, we have  $\mathcal{C}^{\mathrm{sys}} \leq O(d)$ , and so by (15) the computational complexity is essentially  $O(d^2 T)$  after  $T$  rounds. Technically, the bound in (13) is only against comparators in the restricted set  $\mathcal{K}_c$ . However, by setting  $c = 1 / T$ , it possible to extend this guarantee to all comparators in int  $\mathcal{K}$  as explained in Remark 1, since the losses in this case satisfy (6) [27, Lemma 10].
Linear prediction with the log-loss. Another classical online learning problem with unbounded gradients is that of linear prediction with the log-loss [36]. For this problem, at each round  $t$ , the learner receives a feature vector  $x_{t} \in \mathcal{X} \subseteq \mathbb{R}^{d}$ , outputs  $w_{t} \in \mathcal{W} \subseteq \mathbb{R}^{d}$ , then observes label  $y_{t} \in \mathcal{Y} := \{-1, 1\}$  and suffers loss
$$
\ell_ {t} \left(w _ {t}\right) := - \mathbb {I} \left\{y _ {t} = 1 \right\} \cdot \log \left(1 - w _ {t} ^ {\top} x _ {t}\right) - \mathbb {I} \left\{y _ {t} = 0 \right\} \cdot \log \left(1 - w _ {t} ^ {\top} x _ {t}\right).
$$
In the settings, where  $(\mathcal{X},\mathcal{W}) = (\Delta_d,\mathcal{B}_\infty (1))$  and  $(\mathcal{X},\mathcal{W}) = (\mathcal{B}_\infty (1),\Delta_d)$ , we have that  $\| \nabla \ell_t(w)\|_{\nabla^{-2}\Phi (w)}\leq O(1)$  for all  $w\in \mathrm{int}\mathcal{W}$ , where  $\Phi$  is set to the corresponding log-barrier for  $\mathcal{W}$ . Thus, instantiating Corollary 1 (in particular (13)) in this setting implies that BARONS achieves a regret bound of the form:
$$
\widetilde {O} (\sqrt {d T}), \tag {19}
$$
and has computational complexity bounded by  $\widetilde{O}(d^2 T)$ , as long as  $T \geq d$ . Again, we emphasize that the bound in (19) does not depend on the norm of the gradients, which may be unbounded.
Finally, we note that there exist a few specialized algorithms that provide sublinear regret bounds for non-lipschitz losses. This includes, for example, the Soft-Bayes algorithm [34]. However, this algorithm is specialized to the log-loss with a particular dependence on the predictions, and it is not clear, for example, what regret bound it would have in the linear prediction setting and other similar settings with non-Lipschitz losses.
${}^{6}$  Technically,we need to use a barrier for the set  $\left\{  {\widetilde{w} \in  {\mathbb{R}}_{\geq 0}^{d} \mid  \mathop{\sum }\limits_{{i \in  \left\lbrack  {d - 1}\right\rbrack  }}{\widetilde{w}}_{i} \leq  1}\right\}$  ; see e.g. [32].
ZM acknowledges support from the ONR through awards N00014-20-1-2336 and N00014-20-1-2394.