On the necessity of adaptive regularisation: Optimal anytime online learning on $\boldsymbol{\ell_p}$-balls
Authors: Emmeran Johnson
David Martínez-Rubio
Ciara Pike-Burke
Patrick Rebeschini
Venue: NeurIPS 2025 spotlight ()
Keywords: OCO
lp-balls
high-dimension
Primary area: theory
TL;DR: Our work shows that in online convex optimization over lp-balls (p>2), anytime optimality can be achieved with Follow-the-Regularized-Leader using adaptive regularization, and that for separable regularizers this adaptivity is necessary.

Abstract: We study online convex optimization on $\ell_p$-balls in $\mathbb{R}^d$ for $p > 2$. While always sub-linear, the optimal regret exhibits a shift between the high-dimensional setting ($d > T$), when the dimension $d$ is greater than the time horizon $T$ and the low-dimensional setting ($d \leq T$). We show that Follow-the-Regularised-Leader (FTRL) with time-varying regularisation which is adaptive to the dimension regime is anytime optimal for all dimension regimes. Motivated by this, we ask whether it is possible to obtain anytime optimality of FTRL with fixed non-adaptive regularisation. Our main result establishes that for separable regularisers, adaptivity in the regulariser is necessary, and that any fixed regulariser will be sub-optimal in one of the two dimension regimes. Finally, we provide lower bounds which rule out sub-linear regret bounds for the linear bandit problem in sufficiently high-dimension for all $\ell_p$-balls with $p \geq 1$.

On the necessity of adaptive regularisation: Optimal anytime online learning on lp-balls
Emmeran Johnson
David Martinez-Rubio
Department of Mathematics Imperial College London
Signal Theory and Communications Department
Carlos III University of Madrid
emmeran www.jjooosnnnnnnnnnnn 17@imperial..l..mm ac. uk
dmrubioßing uc3m. es
Ciara Pike-Burke
Patrick Rebeschini
Department of Mathematics
Department of Statistics
Imperial College London
University Oxford
Salke--buro@@mppriaal.. ac. uk
retrccct.rbbecchiii@@ttttt uk
Abstract
We study online convex optimisation on fp-balls in Rd for p > 2. While always sub-linear, the optimal regret exhibits shift between the high-dimensional set- ting (d > T), when the dimension is greater than the time horizon T and the lww-dimesiional setting (d T). We show that Follow-the -ee-RRelaiisd--eeaaaae (FTRL) with time-varying regularisation which is adaptive to the dimension regime is anytime optimal for dimension regimes. Motivated by this, we ask whether it is possible to obtain anytime optimality of FTRL with fixed non-adaptive regulari- sation. Our main result establishes that for separable regularisers, adaptivity in the regulariser is necessary, and that any fixed regulariser will be sub-optimal in one of the two dimension regimes. Finally, we provide lower bounds which rule out sub- linear regret bounds for the linear bandit problem in sufficiently high-dimension for all ly ba-balls with 2 1.
Introduction
We study Online Convex Optimisation (OCO) 16 45
a sequential game where in each round learner selects a point Jı in convex set V Id and suffers convex loss (xx)), the full loss & is then revealed to the learner before the next point #++1 is selected. The learner competes against the best fixed point in hindsight and aims to minimise its regret against this competitor:
t 1,..., T,
Rr- = Σr (xxz) m
Σ (u). Optimal performance known depend on parameters
of the problem such as the geometry of the set V and constraints on the losses & (28 23 25 42
We consider the setting where the action set V {x E Rd: ||>||p 51) is ep- in d-dimensional
space with p 2 and losses L-Lipschitz with respect to I|-IIp (ensuring R2 2LT). The study
of Ey-geometries with > has been the focus of many works in optimisation [00 28 [8] 25 as
it covers sets with varying levels of curvature, offering insights for more general spaces.
(FTRL) (and Online Mirror Descent (OMD)) family of algorithms in achieving anytime optimal regret guarantees. Anytime refers to the absence of knowledge of the time horizon T. We focus on two regimes: the high-dimensional setting where d > T and the low-dimensional setting where d < T. For ba-balls with p > the optimal regret exhibits a shift from the high-dimensional setting to the low-dimensional setting (see Table 1). If T'' is unknown, then is the dimension regime when the game begins. We show that anytime optimality can be achieved with FTRL by using adaptive
In this work, we study the behaviour of the Follow-The Beelow-The-BBee The-Theh-eeeee hiieeReguuariid--eeaaa
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

regularisation that in early high-ding dine-nsiensi rounds uses uniformly-conve regulariser of degree (see [Deiinition 2.2) and switches to a strongly-conver convex regulariser in round to = Despite achieving the anytime optimal regret through adaptive regularisation, it remains an open question whether this can be achieved through OMD or FTRL with single fixed regulariser. This would be desirable since it would provide algorithmic simplicity as well as an understanding of how to appropriately regularise ey-balls across all dimension-regimes simultaneously. Therefore, we aim to answer the following question:
Can OMD or FTRL with fixed regulariser be anytime optimal for OCO on e--balls with p > 2 ?
To answer this question, it is natural to first consider the regularisers that are optimal in one of the dimension regimes. However, we give algorihhmie-dependent lower bounds that show that these are not anytime optimal (Proposition 4.1, Proposition 4.5). More generally, we also show that any strongly-convex regulariser provably sub-optimal in the high dimensional setting (Theorem 4.2).
We then turn to our main result which provides negative answer to the above question for separable regularisers (that separate additively over dimension: vh(xr) = Ei= (r(r)) The result (Theorem 4.6) states that separable regulariser (with OMD or FTRL) cannot be anytime optimal. This also establishes that the adaptive regularisation used in our procedure to achieve anytime optimality necessary for separable regularisers. The class of separable regularisers covers wide range of regularisers including all of the form ||> for any r 2 1 which are commonly used in OMD and FTRL TO 28 45). Moreover, the result holds for any separate coordinate- wise decreasing step-sizes, showing that the widely used diagonal versions of Adagrad-style algorithms [12] are also anytime sub-optimal and emphasising the relevance of this result on practical methods. As far as we are aware, results on the failure of fixed regularization are novel in online learning. However, algorithmic specific lower bounds (like the ones we have for specific regularizers in Proppsiiion 4. and Proposition 4.5) have appeared in prior work (e.g. Theorems 3 & 4 in (37).
For any online learning problem, the learner may not know if the game will end in low or high dimensions, and designing optimal procedures in both cases is important. Our results show that achieving this not straightforward for fy-balls, highlighting that this problem should not be overlooked in online learning more broadly and pointing out that the question on universality of mirror descent started in prior works (39 is not completely answered (see discussion in infSection 4.3). We note that the Lo setting (i.e. V == Bp, (z) C B2)) is relevant in the literature, e.g., see the open problem in 177. Our work highlights the difficulty of this setting for p = q >
Finally, we consider the linear bandit problem where only the loss evaluated at x, is observed (Section 5). We would like approach this problem in the same way as the full-information problem by characterising the algorithms achieving optimal regret guarantees across all dimension regimes, However, while sub-linear regret bounds have been established in the low-dimensional setting by (25), we show that the bandit setting is fundamentally different and much more difficult since sub-linear regret bounds become unachievable when the dimension is large enough (Theorem 5.1
Table 1: Optimal regret rates for OCO on l balla (p > 2) from known results. (88) show the regret bound of /2 achieved by OMD using strongly-convex regulariser is optimal for = ((T). When > T this bound is vacuous since Rr A 2LT. For d > T, OMD with a
uniformly-come convvex regulariser of degree p guarantees a regret of OLL [40] and is optimal
34
18 17 40 These guarantees can also be achieved with FTRL using the same regularisers
36
d S T (low-dim)
> T (high-dim)
Optimal Regret Rate Regularisation (OMD or FTRL) Strongly-convex
LV
LTI-1/2
Uniformly-coveex
1.1 Contributions
We highlight our contributions for OCO on balla for p > below. Note that the case p E [1,2] is already understood across all dimension regimes and does not present shifts in the rate of regret with

---

respect to the time horizon T unlike in the > 2 case, see Remark 2.1 [2]. We focus here on FTRL but the results also hold for OMD, and we include these in Appendix H
We consider anytime bounds where the time horizon T is not known in advance and show FTRL with adaptive regularisation achieves anytime optimality (Theorem 3.I).
We establish algorithnic-specffe lower bounds for instances of FTRL that show that the fixed regu- lariser achieving optimality in low-dimensions (1|=:1 is provably sub-optimal in high-dimensions Prrppssition 4.1 and the fixed regulariser achieving optimality in high-dimensions (|<))))) is provably sub-optimal in low-dimensions (Proposition 4.5). We also provide a more general result on the sub-optimality of any prrongly-conved regulariser in high-dimensions (Theorem 4.2)
Our main result: For separable regularisers. or regularisers that are within multiplicative constant of these, we show that adaptive regularisation for OMD or FTRL is necessary to achieve anytime optimality (Theorem 4.6), ruling out the existence of a single anytime optimal separable regulariser.
we connect results from the literature to fully characterise optimality for fp-balls with p > 2 across all dimension regimes. In particular, we highlight that FTRL with strongly-convex regulariser achieves the optimal regret low-dimensions and FTRL with a uniformly-convex regulariser of degree p achieves the optimal regret in high-dimensions.
[eccionn 2
Finally, for bandit feedback where only & (2x) is revealed to the learner in each round instead of the full loss, we establish lower bounds for all convex bp-balls (p 2 1) showing any linear bandit learner suffers linear regret when the dimension is large enough (Theorem 5.1)
We also include some simulations in Appendix A which validate some of our theoretical findings.
1.2 Related works
High-dimensional Online Learning: The setting where d > T has been considered mostly for the stochastic linear bandit problem (20) 30 7 29 26], where the stochastic linear refers to the losses being fixed and linear but observed with i.i.d. noise as opposed to the harder fully adversarial nature of our setting. Beyond stochastic linear bandits, little attention has been given to the high-dimensional setting. Although the high-dimensional setting was not explicitly studied in (28], the results provided for OCO on b.-balls for p E [1. 2] fully characterise regret optimality across all dimensions (see Remark 2.1 Similarly, the results present for the nigh-dimensional case of L--balls with >> 2 in Section 2.3 follow from prior work not explicitly studying the figh-dimensionall setting [40 4
35 24) is the key ingredient regret bounds in high-dimensions. Uniformly-convex functions have been considered as regularisers for offline [10] and online optimisation [40] 4 and also as objectives [22] and losses (39]. Uniform convexity of sets [9 2 allows for interpolation of the set curvature between strong convexity and absence of curvature. In optimisation, curvature of the action set such as strong convexity typically leads to accelerated convergence rates [3 14 32) Uniformly-covee sets then allow interpolation between the faster rates of strongly-convex sets and
Uniform Convexity of functions (Dtinition 2.2, see also 46
to obtain dimension-indeeenndeee
the slower rates of sets without curvature [T 23 25 42 (see (2 for an overview). In this work, consider natural class of uniformly-come between
sets, l. balas for p > 2. These balls also interpolate (p = 2) and absence of curvature (p = oo), and in high dimensions we recover a connection between curvature and faster rates (T1-)/ which is absent in low-dimensions if we consider the dimension as fixed where the rate is O(VT) for all values of p 2 2.
1.3 Notation
We use the following notation: r, the dual of r 2 satisfying 1/r 1/r, = 1, ||01|/1
(E lxal" 1/r denotes the &, norm, ||<|||- is the dual norm of ||-]], dv denotes the i-th entry of vector x, with time index t, e denotes the i-th canonical basis vector, and (z(z) is the set of sub-gradients of a function f at x. For function f R + R. we write fz)) = (z())) (resp. (g((r)) where Ze > O E 1>0 such that for all æ > N, f(n < cg(n)
|| for 2 2, Nt,s
(resp. f(n) 2 cn)n.). We use Ô and Ω when ignore logarithmic factors.
'niiform convexity for sets and uniform convexity for functions are connected through an equivalence of uniform convexity between set and the set-induced norm 24
3

---

2 Preliminaries
In this section, we review results from prior works on OCO for Ga-balls and connect them to fully characterise the optimal rates across all dimension regimes. The action set is V = Bp { E R ||>||p - 11. the unit Ey-ball with > 2. We assume we have L-Lipschitz losses in 4p norm (i.e. 19 pplp. S L for 9r E (x)()))) ensuring the regret incurred in a single round is bounded by 2T, and the overall regret by 2LT.
We first present a general regret bound for FTRL using a uniformly-convex regulariser (Section 2.1). We focus here on FTRL because of its advantages over OMD (in unbounded domains, the regret of OMD can be linear while FTRL maintains sub-linear regret (37), although the results we discuss also hold for OMD and include these in Appendix H| We then consider these bounds with specific regularisers and provide matching lower bounds to establish the optimal regret in the low-dimensional setting (Section 2.2) and high-dimensional setting (Section 2.3). The results from this section follow from prior work and we include the missing proofs in Appendix C
Remark 2.1. We focus on by-balls for p > because the case p E [1,2] is already understood
(28)_ (28) show that when p E [1. 2). OMD with regulariser (/(x) ||=||// //(a//(a 1) and
max 1 + 1 1/log((d)), p} achieves a regret of ((VTT/(a 1)) and this is optimal for all d except if
T < 1/(a- for which sub-linear regret is not possible.
2.1 FTRL and uniformly-convex regularisation
In this section, we review the analysis of Follow-the- whw.thh--Reaarsd---aaaaeeee (FTRL) using a uniformly- convex regulariser 44] which will lead to the regret guarantees in the subsequent sections. First, we provide the definition of a uniformly-convex function from (35 (note there are also other standard equivalent definitions, see e.g. (24).
Definition 2.2 ((55) ). A differentiable function f on closed convex set V is p-miffrrmly-comee on V of degree p > 2 w.r.t. norm |1-11 if there exists μ > such that for all x, y E V.
f(y) f(x) + (Vf()), y x) + Ily x
Uniform convexity generalises strong convexity by weakening the condition of quadratic lower bound allowing functions that are locally much flatter. In particular, uniform convexity with 2 recovers strong convexity. Though FTRL usually considered with a strongly-conve analysis can be generalised to uniformly-cone which we write in a general form to ensure results in following sections directly follow from it.
regulariser, its regularisers 4. as seen in the following theorem
Theorem 2.3. Let V C Rd be convex and consider proper convex losses (f+)T= For t 2 1, let th Rd + IR be a proper, closed and differentiable u.-wuiffrmll-conee
function on V of degree re 2 2 with respect to norm I1-Ilie (we use this notation to avoid confusion with the Eg norm ll- we will denote the dual norm as [|-)) ). At time-step t = =,.... T, FTRL on linearised losses with time- varyin regularisers (zbe)= outputs the following points with gr E dle(zz) for all t,
2; = arg min (l(()) + + (ge.z)) x)}
E(gee
(I)
sEV
Then for any u E and gr E (le(x)), the points played by FTRL satisfy the following regret bound:
Σ - S (T(
T
min wh (x) +
T
(r
|1 = + (e((0)41)) - thet (1441))
ØEV
Trp.
version of this result can be found in 4 We include the proof in Appendix C. for completeness. we consider a fixed regulariser with step-size (we(()) ni- ((x) for fixed 1b) so that the condition of uniform convexity is fixed for all rounds, get the following result (proof in Appendix C.2).
Corollary 2.4. Let V c ddd be convex and consider proper convex losses (Le)E= Let 1b RRd - Rzo be a proper, closed and differentiable μ. u-uniformll--an conveex function on V of degree r 2 with respect to norm I|- Assume V is bounded and let D be such that supwev (xx) S D. Assume the losses are

---

Lipp--ipschtzz with respect to I. Consider using FTRL in 1 with regularisers (x(x) 1 m-
(x))
D4ire ///r and 1/-1 = (1 177 Then for any u E V.
/// 1/r. wir L. Dirrparr.
E - l(() S
The degree of uniform convexity in the above regret bound offers a trade-off between the dependence on the horizon and the diameter D, while leaving the Lipschitz constant unaffected. In particular, larger will shrink the dependence on the diameter at the cost of worst rate w.r.t. T. This can give better regret bounds for high-dimensional problems where the dimension dependence arises through the diameter D. is the case for bp-balls (see Section 2.3p. With 2. we recover the standard regret bound of FTRL using gtrongly-covver regulariser, Rr S 2L I DDT/P.
is also possible to obtain bounds that are Lipschitz-adaptive (do not require knowledge of (|-1)> and adapt to the sequence of sub-gradients (scale with Σ/= | instead of T1/r.) using a gradient clipping technique (see the blog-post by [1 and Section 4 (
2.2 Low-dimensional regime
In this section, we consider OCO on B, in the cow-dimensional setting where d S T. Using FTRL with strongly-convex regularisation in this setting achieves the optimal regret. Specifically, we consider the squared f2-norm (2(x1) _//==/1/3 as the regulariser. This is 1-stronglycoonvex on V with respect to I1-112 and we can apply Corollary 2.4 with r = 2. We have that T-ll-112 S L. since the losses are L-Lipschitz with respect to 11 and S 2 so ll 2 - 119elpp. S L. The diameter of B, measured by d2 is D = supxeB, 22(x) = suP 1
dd= dd2-//p. .. So FTRL, guarantees
RT S LV2dd -/PP,, which is optimal up to constants for d < T as shown by the theorem below.
Theorem 2.5. Fix d S T and let A be any algorithm for OCO on V = Bp. There exists a sequence of L-Lipschitz losses w.E.t ll-lpp such that A suffers a regret of at least /6
2/d1-2/PT.
Optimality in low-dimension was established by (28) (both upper and lower bounds). However, the lower bound present above contains better constants and a simpler analysis stemming from the "probabilistic" method instead of the reductions from estimation to testing used by (28). The proof can be found in Appendix C.3
2.3 High-dimensional regime
In this section, we consider OCO on B, in the high-dimensional setting where d > T. We saw in the previous section that the optimal regret in low-dimensions of O(Vd'-2/pp) is polynomial in the dimension. As - oc for fixed T, this polynomial dependence on the dimension cannot remain 2LT is bounded. Nevertheless, we will that sub-linear regret bounds (in T) are possible for any d T, even when d is such that Vdt- (2-2//TT > T. However, this not achievable using strongly-convex regularisation (we delay discussion of this failure to toSeectoo 4). Instead, in this section we consider uniformly-convex regularisation of degree p > that enforces less curvature, allowing points in the corners of B, to be more appropriately regularised in high dimensions. This allows us to obtain optimal regret bounds for the high-dimensional regime.
optimal since Ry I
We consider FTRL on B, with the regulariser (x) = + I|=:11B- The following proposition ensures that 6 is uniformly-com convvex of degree p with respect to ||-1/p. This is well-known result derived from Clarkson's 's inequality. We include a proof in Appendix C.5 to provide clarity on the constant of uniform convexity.
Proposition 2.6. Fixp > 2. (x( || is 2 --unforrmmlyy- of degree p w.n.t. Il-li on Bp
We now apply Corollary with r p. We have that [|| since the losses are L-Lipschitz with respect to Il The diameter of B, measured by Φp is D 1/p. So FTRL guarantees Rr < (2p.TT) 1/p. which is optimal up to constants for d > T as shown by the theorem below.
5

---

Theorem 2.7. Fix d >> T and let A be any algorithm for OCO on V = Bp. There exists a sequence of L.-Lipschitz losses W.E.. I-Ipp such that A suffers a regret of at least LTT/P-
This lower bound follows from an online-to-batch conversion of the lower bound for high-dimensional (offline) Lipschitz convex optimisation [34 18 17 or an instantiation of the lower bound by [40] (Lemma 15). We include the details of the latter in Appendix C.4
3 Anytime optimality through adaptive regularisation
In the previous section (Section 2).
that the optimal regret is achieved with ttrongly-covve regularisation in the
regularisation in the dimeenmnsio setting (d - T) and with uniformly-come high-dimensional setting (d T). To be optimal, the learner with knowledge of the dimension d and the time horizon T can evaluate whether d > T or d < T and select the appropriate regularisation based on whether the problem is high low dimensional. However, choosing the correct regulariser relies on knowing whether the problem is high (d > 7) or low (d S T) dimensional, which itself relies on knowing the horizon 7' In this section, we consider how to achieve anytime optimal regret bounds which hold without knowledge T.
We consider FTRL with regularisation that adapts to the dimension regime. Fix to 3-2pp((p 2p-(pp2(22) d) Then, in early high-dimensional rounds, the uniformly-convex op is used, until the threshold to when the low-dimensional regime reached and the regulariser switches to the strongly-convex Φ2. In both cases, the step-size used is the one in Corollary 2.4, Specifically, with or (ar = + ||= for r 2 2, we consider FTRL with regulariser at time E given by
7-- (p(r), M)-1 1(2p.t)1/2.> V21-2/1
if < lo:
(x)
1 (z)),
T--1 =
Lv21
if to.
FTRL with this sequence of regularisers is anytime optimal as shown by the following theorem.
Theorem 3.1. Let V B, (p >> 2) and consider proper convex losses (4))=1 that are L-Lipschitz with respect to I|-I|- Consider FTRL with regularisers given in (2). Then
((2x.T) 1/p. LV2Ta 2/9, >
if S to.
RT S
> to
The proof is in Appendix Djand consists of a careful application of Theorem 2.3 The time-step where the regulariser changes is handled by the specific value of the threshold to = 33 1--2p/(p-2)d) This value allows achieved using strongly-conve
recover the same dow-dimensional bound (including constants) as in /Section 2.2 regularisation from the start. For the high-dimensional setting, there
no switch in regulariser the algorithm and regret bounds are identical to those in (Secciion 2.3 In other words, being agnostic to the dimension regime comes at no cost to the regret bound. The above procedure can be used with gradient-clipping techniques discussed by 41 to obtain a Lipschitz- adaptive anytime optimal algorithm. OMD can also be used to achieve anytime optimality with similar adaptive regularisation (see Appendix Hp.
Our anytime-optimal procedure is like a restarting technique except the step-size in the later low- dimensional time-steps accounts for the earlier time-steps. This makes constants not degrade. In potentially more complicated settings requiring many switches in regularisation, not accounting for earlier time-steps in the step-size may come the cost of more than just constants. A doubling-trick approach could also be used, though at the cost of worse constants (see e.g. [Appeddxx F).
Necessity of adaptive regularisation
In the previous section, we demonstrated that adaptive regularisation achieves anytime optimal regret bounds for OCO on balballs with p 2. In this section, we show that for separable regularisers. adaptive regularisation is necessary for anytime optimality. We first show that the regularisers we have considered up to now are provably anytime sub-optimal: in Seecciionn 4. 4.1 we show that strong convexity fails in high-dimension; in Section 4.2) we show that the uniformly-cone
regulariser o, 1 || fails in cow-dimension. Then, we present the main result of this section on the failure of p
using a fixed separable regulariser Section 4.3 All the missing proofs for this section can be found in Appendix E

---

4.1 Failure of strong convexity in high-dimensions
We saw in Section 2.2 that the strongly-come convex regulariser 62(2) 211==112 achieves the optimal O(vdI-2/rT) regret guarantee in the low-dimensional setting (d < T) but this bound is sub-optimal in the high-dimensional setting (d > 'T' because of the polynomial dependence on the dimension. We show that such sub-optimal polynomial dependence on the dimension necessarily appears in the regret bound for any strongly-convex regulariser on By. This occurs since these strongly-convex regularisers can be shown to take values that scale polynomially with d for points in the corners of the bp-balls (10) Example 4.1]. The following two results establish that these sub-optimal regret bounds not loose and that strongly-come
regularisers are provably sub-optimal in high- high-dimensions. The first is a lower bound specific to FTRL with regulariser d22
Proposition 4.1. There exists sequence of linear L-Lipschitz losses (in -n-nor)) for which FTRL with regulariser (x)))) _ _=02(z)) and any sequence of decreasing π6-1 suffers regret
T 16 8
Rr 2 L. min
Td
2/p
The above proposition shows the regret of FTRL with regulariser de scales polynomially with d until it is linear in T. This demonstrates the analysis from Section from|Section 2.2 in fact tight and this algorithm sub-optimal in high-dimensions. We now state a more general result that shows that using FTRL with any strongly-comve regulariser fails if the dimension is large enough. This also establishes that strongly-convex regularisers cannot be anytime optimal (see Section 4.3).
Theorem 4.2. Consider sign invariant regulariser vb that p-strongly-com convee with respect to an arbitrary norm II-II (s.t. ||e:]| = 1 for all i) and attains its minimum value O at = 0. Consider
V = Bp with p > 2 and assume losses are L-Lipschitz in e--norm. If d 2 (4T/H)=/(p=> there exists sequence of linear L-Lipschitz losses (in l--norm) for which FTRL with regulariser 1 (x(x) n 1 (x)) and any sequence of decreasing Th-1 suffers regret Rt LT.
The above theorem is consequence of the following two lemmas (proofs in Appendix E..
Lemma 4.3. Consider a sign-invariant function va that is p-strongly-conver with respect to arbitrary norm I1-Il (s.t. ||e:]| = 1 for all i) and attaining its minimum value at x = Then
(xx 2 2 1
Lemma 4.4. Consider V = B, with > 2 and assume losses are L-Lipschitz in f--norm Let vb be
convex function satisfying for some > O and any x E Rd, vb(x) 2 2 |1:11.. If (4T/ p/(o-2) there exists a sequence of linear L-Lipschitz losses (in l--morm) for which FTRL with regulariser Th-1 (b(x)) and any sequence of decreasing fh-1 suffers regret Rr $ LT.
(x))x))
4.2 Failure of uniform convexity in low-dimension
We saw in Section 2.3 that the uniformly-conver regulariser of degree p, 6p(xx) = -=1=11 achieves optimal regret guarantees in the high-dimensional setting (d T). The next result shows its regret guarantees are provably sub-optimal in low-dimensions, where the optimal rate is O O(VTdL-2/)). Proposition 4.5. There exists sequence of linear L-Lipschitz losses (in nn-nrrm) for which FTRL with regulariser (hr())) Je-1 (x(x)) and any sequence of decreasing 7h-1 suffers regret
T T1/p. 8p
2 m-nn
8
We remark that a general result for uniformly-convex function of degree p as we had for strong convexity in Theorem 4.2 does not hold. This is because uniform convexity is condition the minimum curvature and so is not the reason for the failure of Φ, in the low-dimensional setting. of degree p and does not satisfy some
The reason for the failure is that &, is only uniformly-come stronger curvature condition. Regularisers with stronger curvature conditions such as strong convexity
A function f X C Rd + R is sign invariant if for any E {-1,1]d, f(s x) = f(x) for all æ E X where denotes coordinate- wride multiplication.
7

---

that are optimal in the -ww-dimesiional setting naturally also satisfy uniform convexity of degree p > 2 on B,. The combination the failure of strong convexity in nigh-dimensions and the necessity for strong curvature conditions in low-dimensions is the key insight for showing the necessity of adaptivity for separable regularisers the next section.
4.3 Failure of fixed separable regularisation
In this section, we study the anytime optimality of FTRL under a fixed regulariser. While the previous two sections established that the two specific regularisers we considered [Section (which are each optimal for one regime) are not able to guarantee this, it does not rule out the existence of regulariser that could. We consider the class separable regularisers defined as
d = {5b Bp + R (()) Σ g(..).FFF)
(3)
where F = {gg : IR + R2o: convex, sign-invariant, 0 arg min g()) = g() =
) 1}.
2E3
The function class F is set of I-dimensional even regularisers scaled to be in [0, 1] for [-1,1]
(e.g. x" for r 2 1). The class v covers a wide range regularisers including all of the form |>>|;; || for any 2 1. We can now state main result the failure of fixed separable regularisation. This result also holds for OMD with minor modification to the proof (see [Appendix H).
Theorem 4.6. FTRL with regulariser (h((x) = (²)((x) for de E W and any sequence of decreasing Mh-1 cannot be optimal across all dimensions. Specifically there are no constants Ch,y Q > O such that for all T, Rr S C, L,LTTi/// for all d > T and Rr S <LLVTa-- 2/p for all d S.
Proof. Let's assume there are constants Ch2 a > O such that for all T, Ry S I,LTI-I/P for all d > T and Ry S QLVTdD-2/p for all d < T and show contradiction. We begin with a lemma showing the necessity of quadratic growth of regulariser achieving optimal regret in the 1 1-dimensional case (proof in Appendix E.5).
Lemma 4.7. Consider d 1 (V B, [-1,1] 11) and tp E F. FTRL with regulariser th(())
I v(x) and arbitrary decreasing step-size Tt-1 can only guarantee R S cLv/T T for some constant
0 and all sufficiently large T for all x E [-1, xxx) 2
(11/2) x²
T00c
For d 1, we have (xx)) g(x) and under our assumption, Pr S .IVT for all T, for all
x [-,, 1() 2 al1/2) x by Lemma 4.7 Hence in the general d-dimensional setting for E Bp:
100e?
g(1/2) (xx)) Σ g(x,))
g(1/2) x 100c?
2
|
100g
t=1
We can this lower bound with Lemma 4.4 (from Section 4.1 and μ o(1/2) 50c7
This gives
that if 2 (200ef2) (p(22) then there exists a sequence of linear T.-L T-LLipschitz losses (in no-m) 0(1/2T
for which Rr 2 LT, contradicting that Rr < c. 1. for all d > T.
The above result establishes the need for regularisation adaptive to the dimension-regime for anytime optimality when using FTRL with separable regularisers from V. In particular, having seen that regularisers 1 ||==11 (strong convexity) and p ||a (uniform convexity of degree p) fail to achieve
optimal anytime regret the previous sections, we may be tempted to consider 1 ||>=||] for r E (2,p) that could trade-off the optimalities of croong-convexity in low- ow-dimension and of mniform-convestt in high--igiiin dineh--imession. However, 'Theorem 4.6 rules out this possibility. See also Proposition E. in Appendix for precise characterisation of the regret when using 1
Remark 4.8. We can slightly relax the construint of separable regularisers in Theorem 4.6 Firstly, in the definition of V (3), a different gi F can be used for each coordinate. The result then still holds but the quadratic lower bound on (xxx) stemming from Lemma 4.7 in the proof will scale
8

---

with mini gi(12)) and the dimension from which the regret becomes linear in T scales with mini 9:(1 (1/2)//(0-22) Secondly, Theorem 4.6 also holds more generally for regularisers vi which are within constants of a separable regulariser: c1²((x) < xxz) S c2 (x) for vt E V and constants C12 C2 > 0. Finally, by Theorem 4.2) the result holds for any strongly-convex regulariser. Remark 4.9. In Inpppeddxx we prove a more general version of Le 4.4 4.4 coordinate-wise step-sizes, where the FTRL update is allowed to have a different step-size Tt-1,i for each coordinate:
Tt arg gev
E'=1 9x,2 Using this version in the proof of Theorem 4.6 gives the same result for any sequence of coordinate-wiss decreasing step-sizes. This extension establishes the failure of a wider range of methods, in particular the diagonal versions of Adagrad- style algorithmss 112V.
Remark 4.10. We discuss the connection of our result to the universality of FTRL. It was shown by 339 that for a fixed r E [2, o0) and constant C >> O, for any OCO problem for which a regret upper bound of CTi-1/r can be guaranteed for all T, then for any T
e" there exists a regulariser with We present this result in more detail in Appendix A and also provide an extension to include the setting where may change according to T as in our setting. However, this latter result uses doubling trick where different regularisers are used across separate intervals. This poses a question on the universality of FTRL with fixed regularisation for more general OCO problems where the optimal rate of regret is horizon or dimension dependent. Theorem 4.6 offers a negative answer to this question for separable regularisers. However, the case of more general regularisers remains open. Note that in our setting. the non-separable regulariser from [39] is within a constant fraction of [>=||, so, Theorem 4. still applies (see Remark 4.8). To extend our result beyond separable regularisers, one possible approach is to extend satendLemma 4.7 the quadratic-growth of the regulariser beyond the 1-dimensional case.
1/r which OMD/FTRL can guarantee a regret bound of (
Remark 4.11. The generality of the failure of separable regularization beyond l, ep-ball structures is likely related to the concept of quadratically convex sets (28) for which several results are known on the possibility of getting regret that grows as vT in the low-dimensional case, while it is likely that the regret is better in the high dimensional case (when taking into account dependence on other quantities like dimension). This is an interesting direction of future research. We also note that our results hold for by-balls translated away from O, provided we consider regularisers satisfying our conditions on this translated ball (e.g. reaches its minimum in the centre of the translated Ey-ball).
4.4 Proof intuitions
Many of the results discussed in this section so far are based on the same loss construction, with the following linear losses (x) = L- x² 9t. where Sr E Bp, is defined as
(-1)(.en e1, 1 T gt v,
7
t> 2
where U E Bp. is a vector with equal entries defined as U1 d-1/p-
The above construction is motivated by the following intuition:
The gradients of the losses in the first half of the rounds cancel each other. The competitor is thus only dependent on the losses in the second half of the rounds which constant and place the competitor in the corner of the balball. This two phase construction captures a bias-variance-lic like trade-off of FTRL with fixed regularisation:
If the step-size is small, FTRL will not suffer much regret in the first half of the rounds but in the second half it will not be able to reach the corner of the ball (the competitor) sufficiently quickly and suffer large-regret (it has high "bias").
the step-size is large then FTRL will be able quickly reach the corner of the ball (the competitor) in the second half of the rounds but in the first it will suffer large regret because it moves too fast through the space and every other round it will get to close to e which will make it suffer large loss in the next round (it has high "variance").
This construction is designed so that the performance of FTRL is at its best when the step-size adequately balances the trade-off between the losses from both halves of the rounds. Analysing the resulting regret gives many of our results.
'This result was recently improved by 15 to omit any factors in T' for the case of online linear optimisation and = 2
9

---

In the case of d2 and dp. by computing the points played by FTRL we can explicitly compute the regret suffered by FIRL for all step-sizes and obtain the lower bounds in Proposition 4.1 and Proposition 4.5 respectively. For[Lemma 4.7, we exploit that in the 1 1-dimensional case the losses in the second half of the rounds are equal to the losses in the odd first half of the rounds, which enables a direct comparison between points played in the first and second half of the rounds. This comparison allows us to establish that in order to obtain /T regret, the regulariser must have quadratic growth. The full details for all the proof' are in Appendix
5 Bandit feedback
In this section, we consider OCO on Gp-balls with bandit feedback and linear losses. In the bandit feedback environment, the learner only observes the loss evaluated the point played (x²) instead of the full loss function 4, Similarly to the full information setting studied above, the optimal regret with bandit feedback also depends on the dimension regime. However, our main result in this section will show that for bandit feedback, sub-linear regret is not possible in the high dimensional regime.
The linear bandit problem has been extensively studied (see e.g. (27)). A (dd/212/22 pseudo-regre bound was established by [6] for f,-balls (p E (1, and more generally for strongly-conve action sets by (25] For 1, the same bound can be achieved via reduction to the multi- armed bandit problem (see 6)). [25] also established 0PT
pseudo-regret bounds for uniformly-com sets of degree p. which apply to bp-balls with > 2. However, since these regret guarantees are dimension-dependen they become vacuous in high-dimensions. The following result shows that with bandit feedback sub-linear regret bounds on l -balls are unachievable in high-dimensions.
Theorem 5.1. Fix T. 6 O and p 2 1. For any dimension d sufficiently large and any OCO algorithm with bandit feedback on V = B,, there exists a sequence of random linear losses (4 with sub- gradients (gt se(r) such that llgelpp. S for all rounds with probability at least 8 and LT where the expectation is with respect to the randomness of the losses. 80
E
[t]
The proof can be found in Appendiix Gaand is based on information -hoormation-theortiic arguments. We note that lower bounds in Chapter 24 of [[2]] give linear regret for high-dimensional stochastic linear bandits with p 1,2. However, as discussed in their Chapter 29, the noise in the stochastic case is outside the inner-product so these bounds do not apply for adversarial linear bandits. We also remark that dimension- dependenneee bandit lower bounds from prior work such as the one in [6] only hold in the
low-dimensional setting (e.g. in (6) their lower bound Theorem 4 only holds for T 2 (2/(1-g/2) and does not give linear regret in high dimension).
Conclusion
In this work, we studied OCO on ba-balls in IRd for p > 2, distinguishing between high-dimensional (d 7)) and low-dimensional (d < T) regimes. In high-dimensions, FTRL achieves the optimal regret of ((T)) with uniformly-convex regulariser, while in low-dimensions it achieves the optimal regret of O(T2/22t/2-1// with strongly-conves regulariser. Importantly, proved neither regulariser is optimal across both regimes. Therefore, when the dimension regime unknown, we showed that FTRL with adaptive regularisation is anytime optimal. Furthermore, established that this adaptivity is necessary to achieve anytime optimality for separable regularisers. This is a first step in answering question on the universality of FTRL with fixed regularisation for general OCO problems. However, it remains open whether there exists a fixed regulariser providing anytime optimality whether adaptivity for non-separable regularisers is necessary. Our results demonstrate that existing separable regularisers impose intrinsic limitations on FTRL. and open up an interesting avenue of research to discover more sophisticated alternatives that potentially give algorithms that are fundamentally different. The challenge in generalising our proof technique to rule out the existence of these alternative non-separable regularisers in extending Lemma 4.7 on the quadratic-growe growth of the regulariser beyond the 1 I-dimensional case. Finally, for the linear bandit problem, we ruled out the possibility for sub-lineat regret bounds in high- diggh-dmension. These results underscore the role of dimension and geometry in achieving optimal performance in OCO.
"Pseudo-regret is defined as Rr E f((x:)] - min E {((z)] where the expectation is with respect to the randomness in the learner's actions.
10

---

Acknowledgments and Disclosure of Funding
Emmeran Johnson funded by EPSRC through the Modern Statistics and Statistical Machine Learning (StatML) CDT (grant no. EP/S023151/I), David Martinez-Rubio was partially supported by a 2025 Leonardo Grant for Scientific Research and Cultural Creation from the BBVA Foundation. Patrick Rebeschini was funded by UK Research and Innovation (UKRI) under the UK government's Horizon Europe funding guarantee (grant no. EP/Y028333/I).
We would like to thank the reviewers and meta-reviewers for their time and feedback.

---

References
[1] Dominique Azé and Jean-Paul Penot. Uniformly convex and uniformly smooth convex functions. In Annales de la Faculté des sciences de Toulouse: Mathématiques, volume 4, pages 705-730,
1995.
[2] Keith Ball, Eric A. Carlen, and Elliot H. Lieb. Sharp uniform convexity and smoothness inequalities for trace norms, Inventiones Mathematicae, 115:463-482, 1994.
[3] Necdet Batir. Bounds for the gamma function. 2017. Preprint, arXiv: 1705.06167.
[4] Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, and Manish Purohit. Online learning with imperfect hints. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 822-831. PMLR, 18-18 Jul 2020. URL httphttpss//pp/ https://procceddngg/ mlr press/v119/
bhaskara20a. html
[5] Sébastien Bubeck. Introduction to online optimization. Lecture notes, 2011.
[6] Sébastien Bubeck, Michael Cohen, and Yuanzhi Li. Sparsity, variance and curvature in multi- armed bandits. In Proceedings of Algorithmic Learning Theory, Proceedings of Machine Learning Research, pages 111-127, 2018.
[7] Sunrit Chakraborty, Saptarshi Roy, and Ambuj Tewari. Thompson sampling for high- dimensional sparse linear contextual bandits. In Proceedings of the 40th International Confer- ence on Machine Learning. volume 202, pages 77794400,, 2023.
[8] James A. Clarkson. Uniformly convex spaces. Transactions of the American Mathematical Society, 40:39604111 1936.
[9] Ashok Cutkosky. Artificial constraints and hints for unbounded online learning. In Proceedings of the Thirty-Second Conference on Learning Theory, pages 874-894, 2019.
[10] Alexandre d' Aspremont, Cristóbal Guzmán, and Martin Jaggi. Optimal affine-invarian smooth minimization algorithms. SIAM Journal on Optimization, 28(3):2384-2405 2018.
[11] Ofer Dekel, Arthur Flajolet. Nika Haghtalab, and Patrick Jaillet. Online learning with hint. In Advances in Neural Information Processing Systems, volume 30, 2017.
[12] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Leurning Research, 12(61)22121-215) 201
[13] Hossein Esfandiari, Amin Karbasi, Abbas Mehrabian, and Vahab Mirrokni. Regret bounds for batched bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 7340-7348, 2021.
[14] Dan Garber and Elad Hazan. Faster rates for the frank- wolfe method over strongly-convex sets. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 541-549 2015.
[15] Khashayar Gatmiry, Jon Schneider, and Stefanie Jegelka. Computing optimal regularizers for online linear optimization. In Proceedings the 38th Conference Learning Theory, volume 291 of Proceedings of Machine Learning Research, pages 2362-2402. PMLR, 2025.
[16] Geoffrey J Gordon. Regret bounds for prediction problems. In Proceedings of the twelfth annual conference on Computational learning theory, pages 29-40, 1999.
[17] Cristóbal Guzmán. Open problem: The oracle complexity of smooth convex optimization in nonstandard settings. In Proceedings of The 28th Conference on Learning Theory, 2015.
[18] Cristóbal Guzmán and Arkadi Nemirovski. On lower complexity bounds for large-scale smooth convex optimization. Journal of Complexity, 31(1):1-14, 2015.
[19] Olof Hanner. On the uniform convexity of Lp and lp. Arkiv för Matematik, 3(3):239 244, 1956.
12

---

[20] Botao Hao. Tor Lattimore, and Mengdi Wang. figh-dimensional sparse linear bandits. In Advances Neural Information Processing Systems, volume 33, pages 10753-10763, 2020.
[21] Ruitong Huang, Tor Lattimore, András György, and Csaba Szepesvári. Following the leader and fast rates in online linear prediction: Curved constraint sets and other regularities. Journal of Machine Learning Rexearch, 18(145):1-33 2017.
[22] Anatoli Iouditski and Yuri Nesterov. Primal-dual subgradient methods for minimizing uniformly convex functions. 2014. Preprint, arXiv: 1401.1792.
[23] Thomas Kerdreux, Alexandre d' Aspremont, and Sehastian Pokutta. Projection-free optimization on uniformly convex sets. In Proceedings The 24th International Conference on Artificial Intelligence and Statistics, volume 130, pages 19-27, 2021. 2021.
[24] Thomas Kerdreux, Alexandre d' Aspremont, and Sebastian Pokutta. Local and global uniform convexity conditions. 2021. Preprint, arXiv: 2102.05134.
[25] Thomas Kerdreux, Christophe Roux, Alexandre d' Aspremont, and Sebastian Pokutta. Linear bandits on uniformly convex sets. Journal of Machine Learning Research, 22((84):1-23, 2021.
[26] Junpei Komiyama and Masaaki Imaizumi. High-dimensional contextual bandit problem without sparsity. In Advances in Neural Information Processing Systems, volume 36, pages 49416- 49427, 2023.
[27] Tor Lattimore and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020.
[28] Daniel Levy and John C Duchi. Necessary and sufficient geometries for gradient methods. In Advances in Neural Information Processing Systems, volume 32, 2019.
[29] Ke Li, Yun Yang, and Naveen N Narisetty. Regret lower bound and optimal algorithm for high-dimensional contextual linear bandit. Electronic Journal of Statistics, 15(2):5652-500 5695, 2021.
[30] Wenjie Li, Adarsh Barik, and Jean Honorio. A simple unified framework for high dimensional bandit problems. In Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 12619-12655, 2022.
[31] H. Brendan McMahan. A survey of algorithms and analysis for adaptive online learning. Journal of Machine Learning Research, 18(90): 2017.
[32] Zakaria Mhammedi. Exploiting the curvature of feasible sets for faster projection-free online learning. 2022. Preprint, arXiv: 2205.11477
[33] Marco Molinaro. Strong convexity of feasible sets in off-line and online optimization. Mathe- matics of Operations Research, 48(2):865-884, 2023.
[34] Arkadij Nemirovskij and David Yudin. Problem Complexity and Method Efficiency in Optimiza- tion. Wiley, 1983.
[35] Yurii Nesterov. Lectures convex optimization, volume 137. Springer, 2018.
[36] Francesco Orabona. 13213,
A modern introduction to online learning. 2019. Preprint, arXiv:
[37] Francesco Orabona and Dávid Pál. Scale-free online learning. Theoretical Computer Science, 716:50-69, 2018
[38] Grigoris Paouris, Petros Valettas, and Joel Zinn. Random version of dvoretzky's theorem in lpn. Stochastic Processes and their Applications, 2017.
[39] Karthik Sridharan. Learning From An Optimization Viewpoint. PhD thesis, Toyota Technological Institute, Chicago, US, 2012.
[40] Karthik Sridharan and Ambuj Tewari, Convex games banach spaces. In Proceedings of the Twenty- Twenty-Third Conference on Learning Theory, 2010.
13

---

[41] Tim van Erven. Why FTRL Is Better than Online Mirror Descent. [Online; accessed 24-Januray 2025], /twps://www... timvanerven.
vs- omd/
[42] Taira Tsuchiya and Shinji Ito. Fast rates online convex optimization by exploiting the curvature of feasible sets. 2024. Preprint, arXiv: 2402.12868.
[43] Simon Vary, David Martinez-Rubic and Patrick Rebeschini. Black-box uniform stability for non-euclidean empirical risk minimization. The 28th International Conference on Artificial Intelligence and Statisties, 2025.
[44] Andreas Winkelbauer. Moments and absolute moments of the normal distribution. 2012. Preprint, arXiv: 1209.4340.
[45] Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning, pages 928-9988888888 9389936,75 2003.
[46] Constantin Zălinescu. On uniformly convex functions. Journal of Mathematical Analysis and Applications, 95:344-374 1983.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions section in the Introduction (Section to find all the stated contributions.
clearly outline where
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The conclusion and discussion following Theorem 4.6 discuss the limitations of the generality of our main result.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: All the set of assumptions are clearly stated in the statement of our results. The proofs are either given following the result or later in the appendices, in both cases clear reference on where to find it is given.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our only experiment is in Appendix A all the details on the loss construction, implementation of the algorithm and parameters used are given.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We include small experiment in Appendix A to validate some of our the- oretical results. We include the code for this experiment in the supplementary material. No data is required or used in this experiment and all the implementation details are given. Furthermore, the experiment is deterministic (up numerical error) and poses issues with regards to replication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
guwdes/ddee//ee/ Code/ubmeeeeeaee wwwbbbmmsssiooPpoo ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Our experiment in Appendix A is very simple and does not involve many parameters: nevertheless, they all given Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: randomness.
Our experiment
Appendix A is deterministic and does not include any
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma bar than state that they have 96%
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The details given in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This work is purely theoretical and conforms to the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work purely theoretical and poses no societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work is purely theoretical and poses no such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: Our work is purely theoretical and does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.paperssit
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: Our work is purely theoretical and does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work is purely theoretical and does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work is purely theoretical and does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our work and research does not involve LLM usage as a core method develop- ment.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurd) hutt::///eerrpp.cc/oooffeeeeeee cc///oofeeeennes//225////.. for what should or should not be described.
21

---

A Experiments
In this section, we present numerical experiment in Figure to validate some of our theoretical results on the optimality and sub-optimality of fixed and adpative regularisation for FTRL. We run FTRL with different regularisers on the loss construction used in the proofs of our lower bounds from Section 4 which is described in Appendix E.1
For fixed T. observe that the regret using FTRL with Φp is constant across dimension, while the regret of FTRL with @2 increases with dimension. In particular, dh2 outperforms Φp in low- dimension while Φp outperforms (22 in high dimensions. This validates results that 9, is optimal in high dimensions (Section 2.3) but not in low-dimension (Section 4.2) and that Φ is optimal in low- dimension (Section 2.2) but not high dimensions (Section 4.1). Furthermore, the adaptive procedure from Section 3 perforrs well in both low and high dimensions. However, this experiment suggests that the theoretical threshold to = 3 3-2p/(p-2) d from Theorem 3. is perhaps overly conservative in the transient setting between low and high dimensions (at least for this loss construction) and that larger threshold lo 2d performs better here.
FTRL &2
25
FTRL 9
FTRL =-=3 A0P 2N
Regret
20
FTRL tt 2d
15
10
10
101
10a
dimension
Figure I: Comparison of FTRL with different regularisation. We fix = 40 (and 1, = 10) and vary the dimension. FTRL Φ2 refers to FTRL using the regulariser d2 || [z: from Section 2.2
with f/-1
21 FTRL op refers to FTRL using the regulariser Φ, = } ||: from Section 2.3
with 1/-1 2)=p. The final two correspond to using the procedure from Section 3 with adaptive
regularisation. The first with the threshold lo 3 3-2p/(p (2-22)) from Theorem 3.1 while the second uses the threshold lo = 2d.
The experiment was run on google colab with the default settings (including CPU) and takes around 5 minutes run. All the details of the loss construction and algorithms are provided or referenced above. The closed- foomed-form updates are provided in [pppendixx B] Note that the Bregman projections onto f. balballs are not available analytically, we use the minimize function from the scipy.optimize library to compute the projections numerically (with method= method='SI.SQP" SSPQQ'
Implementation Details:
22

---

B Closed-form update of FTRL with specific uniformly convex regulariser and related lemmas
Consider a regulariser v differentiable on RR.. Define the Bregman divergence of as Dw(r,.) (xx) ('(y) (Vv(y), y) for all z, y E Rd.
Lemma B.1. Fix r 2 2. Let (xx)) + Let = Bp Let Ir. E (lr())). The update rule of FTRL using (x) m 1 (x) as regularisers is
G.+1 = 9s
= arg min Des (z, ( signn(G aEB,
where sign, power and absolute value functions applied component-wis wise to vectors.
Proof. Given gr E @l,(z)), the update of FTRL with regulariser t/e is (see
T++1 = arg min
Σ Des a) (xx)
By Theorem 6.15 in (36 this update is equivalent
{ no 2 22. v(x)).
8t+1 arg min wERd T++1 arg min D& (x, (t..
aEB,
Now by Theorem 6.13 of [36], the first minimisation (over Idd) is equivalent to
8 = Vibb V- gs)
1=1
where vb* the Fenchel conjugate of vb.
For an arbitrary norm ||-I|, the Fenchel conjugate of f(x) = -|| is f*(x) = -| (see
Lemma 2.2 in 24). Therefore the Fenchel conjugate of (/x) is 10* (x) A | and Vis (x)
sign(z) (z) - Combining everything gives the result.
We now provide two lemmas pertaining to the Bregman projections of the FTRL update in Lemma B. for specific cases that will be of use in the proofs in Appenddix E
Lemma B.2. Consider that z & By The Bregman projection argmin =EB, D& (I., 2) with vb(x) = || of z d 1/p w. the rescaled version of w that has f--norm equal to 1.
where is vector with all entries equal to 1 and c d-1/v
Proof. We make of Lemma 5.4 in 5]: if f convex and differentiable function on B, then is minimiser of f(x) in B, if and only if Vf(x) x) for all E Bp. Consider
f(x) (v 2) - (22) Veb(2) 2).
vf(x) = Vv(x) Vx((2),
[V(()), = sign(r;)l=,1'->
23

---

Consider z = d-1/r.. w. From the lemma mentioned above, it is enough to show that vf(x)) x) 2 for all y B
V)(x) x)
- -VV((vV x)
-d-1r-1)/p c w)= x)
(c- (s -))
d
(c- - d (r- 11/ 1/p Σ y1)
==1
(c d 10) ||/||1)
(c (c-1 d dd11-1// d''1//p|eella
20,
where we used that e² d (r- 11/r > 0 and ||l/|1 S (/21///lllll S d=-1/p for all y E Bp.
Lemma B.3. Consider z c E1 where e1 is the first canonical basis vector and |c| so that
2 & Bp. The Bregman projection argmin Du(r,2) 2) with (xx) = |||||| of z is sign(c) -1.
Proof. As in the proof of Lemma B.2 is enough show that Vf(x)* (y x) 0 for all y Bp.
with x = sign(c) G1, and
f(x) Dz(a, (xx) (()) Vx(()) (x)))>)))))> 2).
fx(x) = Vv(()) - Vo(z),
[V(x=)] = sign(s:))==l''
ffxx) X) = (Vv(z) - V+b(2)) (y x)
(Vv(sign(c) e1) Vi((c (1)) (y x))
sign(c) ((d 1)e7 (x -1)
sign(c) (Ic) 1)(iin)(c) - -111 2 0,
where we used that |c| > 1 and S for all E B,.
24

---

C Proofs for Section
C.1 Proof of Theorem 2.3
We follow and extend the analysis of FTRL from 36
(Section 7) which is closely related to the
analysis in [1.. FTRL with uniformly convex regularisation orginally considered in 44] based
on the analysis in [31]. Existence and unicity the update can be handled along the same lines as Theorem 6.8 in (36 with uniform convexity.
The analysis begins with the following expression for the regret. We refer the reader to 36 for the proof.
Lemma C.I. Lemma 7.1 of [36] Denote F((x) (x) + E==i lo(x) and set It arg min Fo(x). Consider VT+1 = v/. Then, for any E we have
E -((z)- - te(u)= S vr((u) min 4 (x) - +((2))
To bound the terms Fz(z)) (1+1(1 (Z(),, we use the uniform convexity of the regularisers. In particular, we require the following result on uniformly convex functions, which is an extension of Corollary 7.7 of [36].
Lemma C.2. Let f: R2d - IR be closed, proper, sub-differentiable and p-uniformly convex of degree
r w.r.l. norm ||-1|. Let x² = arg min minzgeom! [(x). Then for all x E dom J and g E af(x), we have
--1 f(x) f(x*)((==))))))))))) < 1/- 1 (r-1)
|1
Proof. By the uniform convexity of we have
f(x*)
min =Edom
f())
{ f(xx + (g) 2 x) + -11 =|l"
min =Edom
2 f(r) + min
(g,z x) + r
-|| x1""
= f(x) EERd
(g.2) + - |=
f(x) 1 max
- 2) =1=1|1]]
sERd
f(z) 1
= f(x) 1-r, Ig
f(x) rp-/(r-1jlla ((m
where we used that the fenchel conjugate of [= is | Rearranging gives the result.
Since dh proper, closed, differentiable and pu-uniformly convex of degree r, with respect Il-l and the losses are proper and convex, Fe(x) + (xx) = (x(2)) + E la (x)) also proper, closed. lub-differentiable and pt uu-uniformly convex of degree r, with respect ||-I] Applying Lemma C.2 to F. + l,, we have with 2; arg argmin,ev F1(x:) + (z)
F(( - Fet1 (x141) + le(x))
(x:) + ((())) - (F((0e+1)) ((((42))) + (1(44)))) (e41(10442) s (x.) + (((xz)) - (Fi(xt) + +((xt)) + 2e (0((111))) - (e42(( -
T:p4 1/(rr-1j 11 (ve + ve (
25

---

where we used that 9r E a(F; + (2x, since 9: E (z) and = arg min F((z). We omit some technical details but the steps from [6] extend to our setting. Plugging the above into gives Theorem 2.3
C.2 Proof Coroolaayyy 2.4
Since &e is j--uiformly convex function on V of degree with respect to II-II, then the regulariser used 1 1 is
n--1 -uniforlly convex function on V of degree r with respect by FTRL in round t, 4/2 n--1
to |I-Il- Since ae $ M6-1- (1(()+1) - (1411 (
1 カt_
1 (x)-1) S O. By the Lipschitz
condition on the losses, have 11gell+ S T- Applying Theorem 2.3 we have
(((z)) (e(u) S ((()
Σ 11 + T. 1=1
1 11-1/1a(1 (r. 1)1/roT///
1111
T
Di
1--1
1//r
Tape--1
1////xt1//r
[=1
T-1-11 (r/ 1)2/7.7
1
1//r
r, - 1)2/r -111--1(1-1111
t/r
D4/r
D2/r
(1r- (1=-111//rolll
1
1
1/
r,(r, - 1) 1=1
t1/r
Now note that
T
dr
1
1/7
r/r
1/T
J 1/rr
1/r
00
((x))) (u(u)= S 1 D//Tu/r
(Ir
11)-1mmm
(r. - m)wr
11/1
The proof is concluded by noting that (r.- - 1) + (r.-1)1/7 in finding the optimal step-size.
//r
Lemma C.4 was helpful
C.3 Proof of Theorem 2.5
We have d < T. Let k = [T/d] 2 1. Let Y1, be i..dd Rademacher random variables for 1 S í S d. 1 S j < k, i.e. P(Y., = = = -1) = 1/2. Let éi..... Edt be the canonical basis of IR.. Define gt = LY,, 6 where t k(i 1) j (for rounds we stick to the same coordinate and draw i.i.d. Rademacher random variables). Denote the point played by A by I; and fix the loss to be (xxx) 9/th x (for / >> dk, fix (z)) = 0). The subgradient is ft. which is bounded by L in lp. -norm. The point zy depends the losses up to time t 1 but not on l, and is independent of Y,, for all t:
B((())))) = BLY, of xx]
LeVi
B(rr) =0
E
((x)) = O.
On the other hand, u = -d 1/p Σd 1 sign E/=n YosJer E Bp gives
T min E((x)= S Σ i((u)
E t=1
d
A
T d
= --dd 1/P
sign
Yi.j
(E
ΣΣva Yiges)
t=l
1=1
i=1 j=1
- 1/p
Y1
j=l
26

---

We now make use of result from 13 (proof of Lemma 7.2): fix B > 0, consider X = R, where 4 are positive integers such that i== t k and R2 are i.dd Rademacher random variables. Then E [ ] 2 k/ B/.
R;R,
In our case, with B = k and 4 1 for all i, we have that B| Yiill 2 vk/3 2 7/66 (since k [T/d] 2 T/2d for T 2 d) which gives
Td
-/p-
(xx)
min Σ BEB
(x) 20+ Ld 1/P
1dp
Ld
6d
6
The result follows by: supe,....
Rr 2 E Et (((
EBB Et= (((x)) 2 ln
Tdi-2/0
5
C.4 Proof of Theorem 2.7
have d > T. For E {I,..., let Y, i.i.d. Rademacher random variables, i.e. P(Y, =
P(Y, -1) 1/2. Let el. ed be the canonical basis of Bd At time-step t, denote the point played by A by 2, and fix the loss to (xx)) Ye x. The subgradient is Le,,, which is bounded by in lp. -nomm. The point 24 depends on the losses up to time t 1 but not on & and is independent of Y, for all t:
T
=x.) E(YLe Ee Elur) = 0 = E
(x:)) = O.
1=1
On the other hand,
T
min BEB t=1
Σ
(x)) = I. min TT
Σ Yee
B
is attained at 1/p E -i=1 Yee E Bp, giving
T (((x) -
T
T
min BEB,
-T-
Y2ee ev = 1/7 Σ Y? = 1-1p = -IT
M,1=1
The result follows by: supt,.....!y Rr > E ΣT. (xx1)) min ET= (((() Lr.
C,5 Uniform Convexity of p-OMD's regulariser
In this section, we provide the proof of Proposition 2.6 on the p-uniform convexity of degree p of (x) = ||== on B, for > 2.
Consider x, y E Bp. Following the steps in Remark 2.1 of [46. using convexity of db we have for À [0, 1/2].
I +
(0 + (1 (1-))9) 1
21
(1 122)y)
I x !
< 2AM
+ (1 -2A)((1))
2
2A
x + y
IP
(1 2A)
|
p
2
p
From Clarkson's 's inequality (equation (2.1) in
we have that
y
y
P
|
11
2
2
27

---

Using this in the above we have
(AAz + (1-
2x
|
2A
2X
y
P
+
(1 2X)
|1y1
p
2
p 2
p
2
p
|
|1v|1B
2A
X
p + (1 2)
p
p
2
P
S (zx + (1 - A))(1) 22 A)
P
(5)
p
2
This an alternative characterisation of uniform convexity, we now show (following steps in Definition 3.2 of (24)) that it is equivalent to original one (Definition 2.2 From the convexity
and differentiability of vb.
(v) + (V y) v(() + (Vb(y), by + M(= - y)] (01--9) 9)
S u + M(z -))
s Aw(x)) + (1 -))((y) - 21(1 X)
y
p
2
Rearrenging,
X(Vx(v), x - v) S ((v(x)- - -vv) 21(1 A)
y
p
2
(Vo()), x 4) S (V(x) -v()) 2(1
A)
p
p
(1) 2 (1) + (Vo()), x - v) + 2
y/
2
as A + 0. So for any x, E B we have the condition of uniform convexity with μ = 21-PP
Remark C.3. It is not possible to get the parameter of uniform convexity μ 1. Consider the 1-dimensonal case, x = 1, = 1:
+ + (Vo(z), Z) + -
y1
1 + (y -) -)--))-- P (1+ 1)P
p
P
P
122P
+ (-1 - 1)
1
2p + +122
p
This is less equal than vb(y) when
p
1 2p + u2P
<
1 - 2p + μ2 $ = μ p22-P.
p
So our constant may be loose by a factor of p but μ = 1 is not possible since -2--P < 1 as soon as
p >
In fact, we can slightly improve μ from Z to - (we present our results with - because only changes the results by a small constant and slightly avoids clutter). Here is how: In the first step of the proof, we used convexity of 1/ to obtain the following bound,
+
x + V
(22
+ (1 22)y) < 2X0b
(1-2X)((p))
2
2
However, from (5) we have that
shzz + (1 (1- S (b(()) + (1 A)b(1)
2A
x y/
(6)
P
2
and this provides a tighter bound than just using convexity:
+ y
2 ) + (1 - 2Njy) < 2A I #++, 2 + (1 - 22))
2. 2.2X
(x + v)/2 - y
1b 22
2
$ < + (1 - -x)((1)) 2A
1136
(1 + 21-p)
28

---

where we followed similar steps as in the original proof (Clarkson's 's inequality). This provides an even tighter bound than (6) and applying these tighter bounds recursively gives
1b (2221
21
+
+ (1- (11y) S Xxp(x) (1 N)b(y)
2A
-- P
1
21-p'
p
p
using that Ei= (2) 1/(1 2 Following the same steps for the remainder of the proof
gives uniform convexity of sb with μ 2 1-1
C.6 Helper lemma
Lemma C.4. Fix a. > 0, n > 1. Let f(x) = + br for > 0. Then f is minimised at
x² = (a/b(n 1) and
f(x*) al-1/182/n/ (n-1)/n n//n.
Proof. Setting the derivative of to 0 and solving gives
0 = x² =
1/n
x² + (n - 1)bx"
(n
1}b
Plugging into f gives
f(x)) =a. (0.((
(n 1261 1/n + (
a
(n-1)/n
(n 1)b
a 1 1111/(( 1) 1)1/8///
--- +b² b1-1+1/n 111 (n 1) 1/
a 111/ng22/1 (n 1)2/1
1 1 n
1
a² a1-1/up1/u(0
1)1/m
111//n///1
(a- 1)/n n
1/n
29

---

D Proofs for Section 3
D.1 Proof Theoree 3.1
I S to. then we have FTRL with fixed regulariser op and from Corollary 2.4 we have Rr S
(2p.77) 1/9- as Section 2.3. If lo. Theorem 2.3 gives
Rr S vb((1)
min thi (z)
(re 11-13 1)
l1 llsell (1
+ ( the (1e+t))
zEV
Tell
1/(rt
Φ2(u)
to
2 -
to +
-
min
(x()))
l1
p.
17-1
zEV
P+
Mh-11
mp
t=1
0p 0p(
(2(x10+1)
Σ
T
[h-1
119.11 + 2h (x141)))
1-11
my )
-1
noo
2
97-1
t=t++1
sup 6p(x)
to
1h
x2(u)
T Σ t=fo+1
2
1
17---1
P.
9T
The first two terms correspond to the regret FTRL with fixed op regularisation on lg rounds. Substituting the values of 1) and some algebra gives (see similar steps in the proof of 2.4)
supzE8, 0p(2)
- P.
2
11 lp: } S (2pptto)
7h-
The last two terms correspond to the regret of FTRL with fixed d2 regularisation over the remaining to rounds.
Lid122/0T
L/d1-2/p
M|-
11
113]
Σ
--
2
V2
2/2
Vi
t
+ Indi-2/1 1/d1-
I Lvd1-2/11 d112t
1
V2
V2
v2
LV 2/11- - L d -2/pto//2
where we used that Et=tot1 1 S p 1 dx = [2V3]] = =2(VT- vto). Combining, we have
vz
Rr < LV 2 ((2p 1/p- L/ d2-2//10//2
The proof is concluded by to 3
3-22p/
3
2p/ 1p-2) 2) guaranteeing (2p-to) 1/0. -
/dll-//to!" < 0 since
(2p.to)1/pn
3 1/p- -ー 1
00 3t
vt/2 3t vto/2 / di d2-2/pto//2
V2"
30

---

E
Proofs for Section
E.1 Loss construction for proofs
Many of the proofs in this section share the same loss construction, which we describe here. Assume that T is divisible by (use T 1, or T if not). We define the following linear losses
(x) = L x² Sr where Jr E Bp. is defined as
gt (-1) e1.
2'
2
where Bp. is a vector with equal entries defined as Vt, d-l/ (so that ||l|p 1). Note that ||l|| = d/v--//p. The cumulative loss of the competitor:
LT x 0 1
LT o'
LT d=2/p--
LT
Σ
((xx)
min E ((x)
(7)
|ul| d1/p-l/v..
1=1
The cumulative sum of sub-gradients used in the FTRL update:
if < T is even,
E1,
7 if S is odd,
Σ 95 L
(8)
1-11-1- 7)
if T
E.2 Proofs of Proposition 4.1 and Proposition 4.5
The two propositions are special cases of the following proposition. Proposition E.1. For r E [2. pl. define Φr(x) -|a=11:-- There exists a sequence of linear L- Lipschitz losses (in ---norm) for which FTRL with regulariser ve(x) o and any sequence of decreasing Th-1 suffers regret
Ry L min dr.-p.-ln..qTum
8
We now prove this proposition. The loss construction is described in Appendix H.
From Lemma B.
Tt+1 arg min Der (2, sign =EB,,
sign
T/
go
The
Define 0:-1 min(1, Using (8) the points played by FTRL on are given by
For f S It = 0
For t T/2 even: 2,, -/-i1 -1 by Lemma B.3
For T/2:
34 = min 1 In-id-1/po( 1/p (1- -
T r.
| [n-
-
2
min 1, d rr./p Th-1 -
v
by Lemma B.2 where w is a vector with equal entries equal to 1.
Fix 1 = 1- min(1, Using that 7h-1 2 nte the loss the first half of the rounds is lower bounded as
T/2
T/4
T/4
T/4
> a² -T
Σ ((x)) = Σ (Ixz) = Σ 1=1
2 12k
Σ
aai
4
kml
kml
k=/
31

---

If a 2 1, we have R 2 2
2 T and we are done. So for the rest we assume that a = 17 - 1/2.
Ar min(k" T/22 1). Note that o'v d'-2/0- = -|ee|pp- The
Let k* = 1itr//...//(r----- /nl, losses in the second half is ower-bounded as
Σ 4(x))= - 1-/2+1
Σ
T
Σ
min 1, p. (1) }
oJ U
I
|01|
T=//241
k=1
T/2-1
Σ
min 1, (116)r--1} (1
m (nn)
(
drrr./
(2 -1-- m)
1
We bound the sum with an integral as follows,
m
1- (x + 1) do
m
1
1 [1x + 1)r.
mm 1
- 5(m++
(m + 1
rx
0
rx
1=1
We get
77--1 r
( -1 1m) m)
Σ ((())) 2 -1--1./pp
(m + 1)
T=T/241
Using the cumulative loss of the competitor from
7. the regret is
T 17" TT
nf
T
Rr
P2 2 4
d -r./n.
(m + 1)r- - 2 - -1 m))
r.
11" WTT 'T dl-r./p.
11 r-
(m (m+ 1)r- + +1+ m).
4
r.
Let's consider two cases:
k* 2 T/2 - 1: m = 7/2- 1. By the definition of k-: 1rr.11--1)(fr-11 11)(1--111)) 1)
((1./p.-)/(rr-11 1)
T
1
T 2 -1)
k*
5
dv..pp. - 1)/(r.11/ -1)
1)
T
17
< 1 +
S 2'
dr./p. 11/(.-111 2
since S 1/2 and
2 (recall r S p). Using this in the regret, we get
RT 2 -d-
17" r-
1
T
r.
T
r-
2
1
"
T
r.-17
T
T
P.
d /p. 1)/(r,
2
2
+
2
1
3
r. 1 T
T
+
T.
2
2
2
T
1
3
r. 1
2 r.
2
T 2
T
4r
where we used that r. E [1,2] and that f(x) = 1 - (3/2)=-1
2 (1 - 11/)) for E [1,2].
k* < T/2 - 1: m = k- By the definition of k*:
dir./v.- (tt./.-1)/(r.-1)
dir./v. ()/(r-1)
ks d - (1)
k* S
7
3
(k* +1) S 1 1+ -11//--1)//r.-1
<
drr./p. 1)/(r. 11)
2'
32

---

since again 1/2 and d 11/////111///1111 2 1. We also have k +12 d Using this in the regret, get
7
Rr 1T T
/1-t./p. 1/--11
d
(I:* 1)" + (1 A²))
1
k* +1
17
(k* + 11
+ (1+
T+
crr./v. - (r/ 11
17" 1TT
1+ 1
k*
3 r--1
+ +(++k*
21
+ (1 + ++**)(
3 1
2
r
n" TT 'T
(1 k*)
(1--)
1
T.
d(r./p. -1)
2r1)
r. r///r.
21/r4//r.
4
where again we used that minimised over 7) using Lemma C.4
(1 since r. E [1, 2] and in the Istar step we
T di If T is not divisible 3F:
Combining both cases, we have that Rr 2 min
T-3 ders veler (T- (T-3) 2
by 4 and we use T - 1, T 2 or T - 3, we have Rr 2 min
min d (T-3)) (TT3)/1)) for T 2 6, concluding the proof.
E.3 Proof of Le 4.3
Consider a E arg min (11,,.., Ii-1, 2, Tz+1 I.)) Since v is sign-invariant, -a is also in the argmin. Consider g(2) 1b( Ti- 1, 2, rit Ta.) It is straightforward show that the strong-convexity of &e applies By convexity, we have
1 . = / 6 2 (-a)) s Go(a) + 20(0a)) g(a) min g(=),
sEB
and by strong convexity, is actually the unique minimiser. Hence
@v((x)
arg min V (..., Ii- T+11....
=ER
Ox:
22
Vu(x)TTe e; =0 for any x E Bp s.t. D; = 0. (9)
For set S and vector æ. denote z-s the vector with the coordinates in S replaced by 0. Denote Sn = {1,...,.) n). We prove the following claim by induction on n S d:
v(x) 2 (.) + 11
2
1=1
Base Caset, by strong convexity
(xx) v(x-(1)) + (V x- +
|
x- (1} 12
x
((r=(1))) + (V (1)(=111)))0 x 2
= = (1) 2
using
33

---

Inductive Step: suppose true for k. Similarly to the base case: by strong convexity,
(1 2 v0 11 (Vo(x S.11 S. -- x -- 1
|>s-ss S..
S.11 11
2
=0 + In+1 (V ent1) + 2 2
M 12 2dat1 a+1
using 9
The result follows by the inductive hypothesis:
xbxx) 2 (x s.)+
M
z/j
2 iml
2 (1 8541 ) +
11
2 n+1
+
1
2 t=1
Σ
11 n+1
2 ( ) +
x
2
1=1
When n = d, we have (xx))
IV
E.4 Proof Lemma
As discussed in Remark 4.9 we prove more general version of Lemma 4.4 for coordinate-wise step-sizes, where the FTRL update allowed have a different step-size 7/t 1,a for each coordinate:
It arg arrrrmmng min (z)(x) + >i 7/--1,
--1 E 9n.i).
We consider slight variation of the loss construction described in Appendix E.1 divisible by (use 1, 1-11, T 3 if not). We define the following linear losses (xx) = L.x" gr
Assume that T is
where 9t Bp, defined as
1)f-,, 1) es(t)s
8 2
gt
< l < 2' T t> Z
where v Bp, is vector with equal entries defined as U4 d-1/p. (so that ||I||p 1) and i(t) = arg ma (2(((-1)/2), (i.e. the coordinate of the largest step-size in the previous even round). Note that |ellp- = d2/p-1/p. The cumulative loss of the competitor:
T
L.T TTu
LT of v
LT d-2/pp
LT
4(x)=
min BEB
(()x)
(10)
2
2 ||l|pp
2 d d//11//r.
2
The cumulative sum of sub-gradients used in the FTRL update:
if < T is even,
Ei(()
T is odd,
L Σ
L.
if S
(11)
(
T
T if > 2"
2
First we consider even,
k<1 T/2. When is odd, x arg min (b(x) 0. When & is
arg min (xxx) - 1/11,)()) [
ZEB
v((r)) - 111 1,1(1) LOT Eise) S vleen)) - 111 110 Le = 1 Ln--1a00
(((x)) = J Ese 21 1111,1(1)
21 1/-1,((/2)
by def of i(l)
2 1 1T/221(T/2)
2 1/2,
34

---

when 17 := 1///2,(1/2) maxield 12/2, 2 2/L. So we have (-1 accounts for first 2 rounds not being like the rest)
2/2 Σ
(T/4-1,
if 2 2/L if < 2/
9i 21
0,
Hence if 7) 2 2,L, we have Po 2 LT/4 /4- - and the statement of the theorem holds. If 1) < 2/L, we look the second half of the rounds.
Let's now consider > T/2 and assume 1) < 2/L. Note that by definition of 1,, we have the for all 2 T/2 and for all i E [d], ne n L/ Fix 3r =t T/2 1. The FTRL update
= arg min (x) LB; x" x''nn""nn-- (n-1 101 BEB,
Let u = u//lulpp be the competitor. We can write 2y Auu+ (Ae 0) a component in the direction of u and component orthogonal to u. We have
((x)) 2 || - 2 X 12 2/D
2
Now from the FTRL update, (in the first implication, that mh- and 21,4 2 0, 12 2 0) (xx)) L8 (Mh-1 00)
<0
X2 1d 2/1 S nLBBaau
Ng 12/1 nLBBA>
21
d pdl- 12/2
LA 2 - 24I.33
48,
l((xe) = - L.. xt
2 L
Hdl 1dd111pp
udl 2/p
since 77 s 2/L. If d (4T/H)p/(0-)) have for all t T
4,
3
L
le(x)) 2 -L
ス -
pdd pdl
T
LT
T
LT
LT
LT
= RT
2
l((x))
2
(=1/2+1
If not divisible by and we use - 1, T 2 or T - 3, we have R 2 L(T-3)
2
IT for g
1
T 2 6 + T,, concluding the proof.
E.5
Proof of Lemma 4.7
Assume there exists constant > 0 such that for all T' and any sequence of losses, Rr S cLvT.
Consider T' 16c² and multiple of 4. We define the following linear losses fx(z) 9t where Se E [-,,]] 1] is defined as
(-1)). 9t L.
L, i5
t t>
z
Set f1/2-1- Recall that the FTRL update is I, = arg min wef wel -111 { 1/ (E:=) 9.) +x+((x))
With this sequence of losses, the points played by FTRL satisfy
for t < T/2 and t odd, we have E Is 0, 2 0.
35

---

fort T + 1 and t even, we have E = L so 2t = arg min re( (e[-1, (-1 1-11-
((x)}. For < - T/2 (both even), we have
-1- (xv) - Lax + (p(2)))
using the definition of zy
= - L + (xxe) + L( - -70-
< 1/1 Lox + (x(x)) + L(7)-11 7/-- 11)
using the definition of
= (Mt-1 no 11- Lay S (71-1 m/-- Lon
It S Zt
using that mp- S 7
So for all f T/2 even, we have It 2/
for t > T/2, we have 9. - T/2 1) so xt = arg mig 1
7/2 1) + (x)))
The regret can then be written as follows
T RT Ee(x)) - (-
T Σ
LT 2
LT
LT
2T/2
L
(12)
2
4
==1
t=T/2+1
from which can show the series of following statements.
We first show max25i5(/eVTT) I++ 2 ::2 not, x zg < 2
for all / S [2cvT] and from
12
LT
RT 2
+
Σ
T=/2211
t- T- (/e/++22o/T)
LT
T/2+|2evT]
T
Σ
Σ
2
2
1T/221
1=122+220/1T+1
LT
L
[2cVT] -L
[2o/T] - 2 -ToT-
2 2
L [26VT]
> cLVT.
which contradicts our initial assumption that Rr S clVT so we must have max2gi5 Fs+t 2 1.2 Note that 2c/T < T/2 ensured by 1662.
2. Next, we show that v 2 -rr
let 1* = arg max I5++ by the definition of 2T
((1/2)
(1 +1"
T L
T - 2ITH ((Iz+e)
0 - 1
2
- L 1r -1 2 (1122)
11(12)
2 (1122)
- n = TT 2 1 2 L(t 11) 2cLVT'
where in the first implication, we used that ((r 2 (1/2)) (since e is increasing on [0, 1] and rg++ 2 1/2) and Zh+t" 1.
3. From 12 we also have Rr 2 LT
4 2T/2. To achieve Ry S cLVT, we must have
TT/2
36

---

4. By the definition of 27/2 arg min min xc[-1,j] . nLa + {x)). for any E /Ac/ 1] we have
nLar!22 + ((11/2) S -nLa + (x))
= vb(x) 2 nL(l - x1/2)) 2 nb VT
2b(122) 2Cv/
(s
VT
5c (1122) 2 VT
(1112)
(TTTT)
27
5. Now fix x E [0, 1]. There exists T (multiple of 4) such that is increasing on [0, 1] and from the previous point, have
5c VT+4*
Using that e
vT
5c
(1/2)) 2(T 4
(1/2)
T
(1122)
xxx) 2 2/ +4
x²
x²
+
25c
100c2
using that T/(T+ 2 1/2 for T 8. The result is shown with μ (1122)/100c*
37

---

F Universal optimality of mirror descent for time-varying regret rates
In this section, present extension of the result of 39 briefly review the considered setting along with their result. We refer the reader to Part II Chapters 5 & 7 of [39 for the complete details. We present the results with respect OMD but they also hold for FTRL up to slightly different constants.
the universality of OMD. We first
We consider general OCO with linear losses (i.e. online linear optimisation (OLO)): The action set H. C B is convex and centrally symmetric set that is a subset of an arbitrary real vector space B. The subgradients of the linear losses belong to a set X' C B that is subset of the dual 3* of B. We focus on linear losses for simplicity but the results hold for general OCO where the subgradients of the losses in X' since the regret can be bounded by the linearised regret, see e.g. Corollary 64 in 39 or Section 2.3 in [36].
The regret is defined as
Rr 91. 91......... -0., .T) 9T))
Σ (A(g):-1) t=17
Σ (h, 9t), hEH
inf
where Ir are the subgradients defining the linear losses such that 9t E X but otherwise are arbitrary adversarial and A is learning algorithm. The minimax regret is
(HH, = inf sup RT g1.....r)) 9r). A Sr EX
We re-state the main the result on the universality of mirror descent from 39
Theorem F.1 (Theorem 71 of (39). for some constant V and some q (o), (H, S
VT1-1// for all T, then for any T >> ex-1, there exists regularizer function V and step-size т), such that the regret of the mirror descent algorithm (OMD) AMD using V against any 91. Dr E X chosen by the adversary bounded
(r I1...... 9T) < 6002 V log (T) -T))-T-1/4 T1-1/4
The result states that any regret bound with constant rate that is achievable across all time horizons can be matched by OMD to logarithmic factors. The extension that we discuss next handles the case where there may be multiple regret bounds with different constant rates that exchange ordering at different time horizon intervals.
Theorem F22 Let K > be an integer. for h: 1,... , K, there exists constants V5 0 (w.r.. T) and Qk 2 2 such that VTHH, S minnk=,..... VeTT-1//x } for all T, then for any T > e there exists a procedure AMD+ running OMD over intervals of doubling lengths such that the corresponding regret against any 91: .... Pr E X chosen by the adversary is bounded
Rr((AMD::9 9T. S (2 + /2) 6002 log)(T) mmin (V,T) 1V/T)-1/92 1/9x)
The procedure does not require knowledge Tf
This matches up to factor of 2 222 the regret bound would get by using Theorem F.1 Theorem 71 of [39] with advanced knowledge of T, and otherwise matches the minimax regret up to constant and logarithmic factors.
This procedure and result could be used to obtain a result similar to Theorem 3.1 However, the bounds are worst due to the additional logarithmic factors and much larger constants. Therefore Theorem 3. remains a valuable contribution.
We now provide the proof and procedure based on the doubling-trick.
Proof. Fix T, T, = 22, = min
j
:EN: ELSOT
T, TT We have:
B-1
2B Σ T. T
Σ 13 Σ 2 2B1 -1
TE 12 BB1 1].
Consider the following procedure AMD+. For i 0, 1,.,, B:
38

---

set k(i) arg mink
ViTT - 1/0u
to get regulariser for OMD that achieves the regret upper-bound of 6002V (()) log 2 (T.) T, 1/11/) over T, rounds. Use this the T, rounds Tj T + 1,.,, 1 T Ti When i B. just it up round T.
Use Theorem F.1 Theorem 71 of 39
Let k, = argming, (/TT1-1/9 j.
The regret is bounded as follows:
(AAAAD+:9H...... 9T) S
6002V (()) log((T.)
T::1//400 1/4ece)
1=0
s 6002 log (T")
Vi(s) T 1/anc))
since for all i, T; S T
==0
< 6002 logo (T)
Vk. T 1/40.
by definition of k(i)
1m0
B
6002 VR. logg log²(T)
(21-1/20, )
(21-1/0k 1/1/// B++1
6002 V/. log log)(T)-
21-1/9k. -1
21-1/
< 21 21-1/Ok.- -1 6002 Vv. (T') -(2B)11211110
21-1/au.
21
6002 Ve, log 2 (T) -T- T1/1// since 2 S T
21-1/ex. -1
V22 6002 Ve, log T--
<
since Qk. 2 2
V2
(2 + 2 6002 log log2(T) min V V
39

---

G Proofs of Section 5
Throughout this section, will use the notation Ry for the pseudo-regret. In fact since randomized learner is equivalent to random choice of deterministic learners, we will consider in the proofs below deterministic learners and the regret equal the pseudo-regret. In addition, since our loss constructions are oblivious to the learner's s actions, even for randomised learners, the pseudo-regret is equal the regret.
We split the proof into the case where > 4/3 is "large" and the case where p E [1,4/3] is "small" and consider separate loss constructions for each. We first highlight the intuition of the loss constructions.
For p 4/3, we take inspiration from the loss construction which 6 use prove ((dv lower bound for low-dimensional (d2 < 7) bp-ballss with > 2. The construction consists of linear Gaussian losses where the mean of each coordinate is the same distance from but the learner does not know the sign. When the dimension is large enough, the learner does not acquire enough information to determine the signs of these means in the T rounds to get sub-linear regret. This construction will not work when S 4/3 because when is close to 1, the lack of a distinct corner in the Ep Ep-ball allows any point on the boundary (including =ee) with correct signs to achieve similar loss to the competitor (a corner). The learner can therefore focus on =ej simplifying the problem to de-eensiion where sub-linear regret is achievable.
For p < 4/3, the construction consists of linear Gaussian losses where the mean vector has single non-zero positive entry, unknown to the learner. When the dimension is large enough, it does not acquire enough information to determine the non-zero coordinate of the mean in the T rounds to get sub-lineat regret. This construction will not work when > 4/3 because for p > 1 the learner can exploit the fp-ball's proximity to the hypercube by playing points with all coordinates close to 1, bypassing the need to identify the correct non-zero mean coordinate.
We present the proofs with Lipschitz constant of but they extend straightforwardly to arbitrary L >> 0.
G.1 Case p > 4/3
Theorem G.1. Fix T and 8 > 0. Consider p > 4/3 and
1
C1T
(_11o1 log
cip+ 8 CITY 8.12
d max 16T
log 8
for some universal constants C1 C1. For any OCO algorithm with bandit feedback V = Bp. there exists a sequence of random linear losses (Ce))e(T) with sub- uub--graseens (ge)ee(T) such that || l. S for all rounds with probability at least 1 - 5 and
T
E Rr
80'
where the expectation with respect to the randomness of the losses.
G.1.1 Proof
The following loss construction and analysis is inspired from the proof of Theorem 4 of (6). Their construction is designed for the low-dimensional setting in such a way that the learner has to balance exploration and exploitation rounds. We only consider the losses corresponding to exploration rounds and generalize the analysis to the high-dimensional setting.
Let = > be such that g. = l/d. Let T < ad (with a = 1/16). For a fixed & E 11, d define the losses (z( xT of where of ~ NN(E, بي Ia) (i.id.). We show that (when is sampled uniformly at the start and fixed throughout the rounds)
T
EgLE
[T
>
16'
We use Eg for the expectation with respect to & and Egg for the expectation with respect to of with fixed. We will also use Z to mean the i-th coordinate of xt-
40

---

For fixed E: [[(]]()]] = =[xx" gf] = E. x".g. So the competitor x" = arg migninee E E"x
d lRg. Let define + Σ 11=1 B[xe]. In particular one has
E 9
[T] T e ("--'")
The following lemma expresses the expected regret in terms of the expected number of rounds and coordinates for which the learner plays on the wrong side of g. The proof is in Appendix G.1.3)
Lemma G.2 (Generalization of Lemma 6 of (6).
B [TT]
Pr
Σ t=1 ==1
Σ
[a 2 o3]
And now the next lemma shows that the expected number of rounds and coordinates for which the learner plays on the wrong side of & linear in both I and d. The proof is in inpppnddix G.I.4
Lemma G.3. With T ad To. we have BelR. o3 Ex= Ei= Mar 0) 4
Combining both lemmas, we have
T
d
E[RT]
2 o E
(ar
00] 2 s.
1
T2 T
P+
4p+
4p.
16'
since p > 4/3 so p+ S Now to ensure the Lipschitz-condition with high-probability, we get an extra factor of 1 1/5 (see the next section), concluding the proof.
G.1.2 Bound Sub-gradients
Recall that p 4/3 p. S 4. Fix EE {-1,1)". 9t d d So gi d- + +
where X ~ ~N00, Ia).
From
44
we have
E [|I - E
X,P
1/D. =
((B
E
2p./2
I((p= + +11/211 1/p- S v=(22) 1/0. dt/l. s 20t/p.
V
i=1
im1
= [[llelllp.] 2 ad /n.
3.
Fix 5 >
For p. S By Theorem 1.1 in 38 for some constants C1 6 > 0, IP ( 1p. < (1 B) [1xx11.) 2 1- Ci Cxepp(cc
Assuming d 2 c log C.T have 3 v 2dd log CiT < 1 and
P P lp. S (1 6) I lp..) 1) 2 1 T
For 2 < P. S 4: By Theorem 1.1 in (38 for some constants C1, c1 > 0, PP (11l. I (1 + B) [|xx1==)) 2 C1 exp( -c (i...) e,
log 9jz) p-/2 have 3 0190370. log CT < 1 and
Assuming d
cipp
8
PP | lp. S (1 B) [1x 2 1 T
In both cases, with probability least 1 -8/T
| ll.. S (1 B) | 11 < 2E [|1X(111]] S //
| pp.. < <d Ip. + ed /p. S =4+1=. 5.
By union bound over all rounds, with probability 1 6, 11 Ilp. for all rounds t. So rescaling the losses by a factor of 5 gives aob-gradients whose é, noom is bounded by 1 with probabbiby and regret bound of:
T
E [T
80
41

---

G.1.3 Proof Lemma G.2
Let W, = i [d] : & < <0} and s = BB E
a 2 o
We have that
E of
PT| = ET. 5" (T - x²)
T
T +
Σ
Σ iEW,
Soee +
/Ta
8
of
1=1
2 8
+ +TTd
9f
ieW:
Therefore, is sufficient to show that
EPt
E
Σ
E + ETd 2
P+
iEW.
Since || llp < 1 (we use T,,,, to mean that the coordinates of xs that are not in W, are 0), by Holder's inequality we know that
Σ
ELr, = (Zr,w.)"(EEWw (EEW.) 2 -I, www. 2 -lWall/ee.
iEW;
Noting that (see (14 below)
WR ((d (.-WW/)))e.Ju S (deP.)2/e
gP. 1e--iWl
(13)
P+
we have
Σ E 2 _ - 1/pp
_ e. Σ
11 2 20} (de
E
P.
ieW,
1=1
gdr
S
E
Σ
Σ
1 2 03}
-Td=
/p.
9
Ps
91
D.
iEW
which concludes the proof.
Proof of (3): Since -//PA is concave: for all æ, E R, x//ps S y/ + - y). In particular, with 2d (d s), y d,,d, we have
ePss /d/ 1 _ 8,
e(d 8) /n- S /p. P. d)/-
(14)
P.
since eP- d = 1. Using
[WC| gives the result.
G.1.4 Proof of Lemma G.3
At round conditioned on ç, the observed feedback
||
: (e (lle=fff,,of where of
d2/p.
Denote for the law of the observed feedback to time T' conditioned on &, i.c., the law of (ff. .... f. Consider & and g' differing only in coordinate i d. By Pinsker's inequality have
dTv(Pe,Pe) S (RI...
42

---

By the chain rule for the KL divergence / operations on conditional densities:
T log Pe Pe(Ji,.,,TT) IT) Pe (J1, 11,,.19 [T)
DKL(PE Pc:) = (n..... fr)ppp
log Ps (E (F Ji) Pe (J fi)
E PiS.....ppppp
T E Bi [E log
1.... -....)))) fi) Pe (fr fr- 11l-. f1) }}
Ps Px -1..,
Now since Ty is a deterministic function of fi..., fi-1 and given 21, ft ~ NN(xx o7) under Pe we have that the inner expectation is a KL divergence between Gaussians:
Pt P Si)
(exi's - exf(()²
452x7,
Bs (c-1) log
De fr 1 f1)
207
20
of
T
T
DKL(Pe.Pe) =
E (f f1) Σ
= 2
0
oi
t=1
c2
dIv(pt.Pe:)
Epc
oi
1=1
Now, using {-: to refer to all the coordinates of except the i-th and Ex+ (resp. Ex.-) to denote that the i-th coordinate of & is +1 (resp. -1).
T
Eg Pe
1 2 07]]
Bg Bs,
Epc
1 2 0)]16-1]
1
B
Epcc+
If 111...1120 120]] + Epa..
1 Mf (-1) 2 0)]
2
T
T
Bg_1
E
If21, 2 20H| + Epsc
Σ
- If 2 > 0|1]
2
PEL+
1=1
T
T
Epsc [1/2,3 2 20}] - -1ps- [L(22,1 2 0]]]
+
E
2
2
T
1
Σ
Es_, (dvv(PE4, + drv(ps, .PS+T) )
using Pinsker's inequality
2
t=1
T
T
|dr (P. + dIv(PE_: }
Es_,
2 4
2
T
T
Σ R
T
Gil
2x
T
E
52xi.0
2
4
9t
of
9
t=1
T
T
T
e²
2212
1,i
EE
E
2
o
Summing over all possible coordinates i, we get:
1 T
Eg
(araa 2 o]1]
d
1
Σ
E5
Σ
T
E,
E² 2
2
2
\
of
of
43

---

Note that due to the concavity of the square-root,
d
2
d
IE
E
98
o
T
E
E
5.9
oi
d
T
E2
d
E 5.92
o
E²
2
E 1.
So we get
- E.E
1
ΣΣ
T d
1 1 2 03
2 d
d
E 4.
22 1
T 96
of
:=1
1=1
d
Vel+2ip.Te
d VdT
2
2 d(
L
Va
2
4
since e = (l/d)//-- and T < ad = d/16.
G.2 Case <ps4/3
Theorem G.4. Fix T and > Consider p E (1, and
d >> max (128p.T)² ( 1 (1_1 log
)
,62)
CIp.
for some universal constants C12 C₁. For any OCO algorithm with bandit feedback on V = By, there exists a sequence of random linear losses (Ce)serr with sub-gradients (ge)eetrn such that 11 p. S 1 for all rounds with probability least 1 - 8 and
T
E RT
>
16
where the expectation is with respect to the randomness of the losses.
G.2.1 Proof
Before the start of the game, draw Y ~ Unif(I,..., d) and define the losses as lx(x) = x² where
[N((0,0)) if (N(112,0*),
if - Y,
where o = (8/p.d//R.)- -1, We show that Ey E, by Rr > T/16.
44

---

Fix a E [0, 1] and define A,(a) = {t: Me.i 2 -a}. Then
T
1
Rr = s] = 2
+
Ut
Y =
2
2 -
T
Σ
yr,i =1
yt +
(g)((a)
tEA((a)
T B
Σ (-1)+
Σ (-a)ly =1]
2 2 2
(AA((a)
rEAi(a)
T 1 B
T- - AA(aa) + + =
2 2
(1 a) L [A(())))X =i]
T
= (1 a) E
Σ
[BB. 2 -a
The following lemma bounds the expected number of rounds where the learner suffers large regret (as measured by a) when Y = compared an environment where all coordinates of gr are 0-mean for all & (i.e. there is no better direction). We denote Eyo expectations with respect to this environment. The proof is in Apppeddix G.2.3]
Lemma G.5. Let of = llve 0², then
T
E
Kyns 2 -ajly = =1] Epg Σ
Kyn 2 -a]]
-
Ep
So?
From the lemma, we have
= 2 2] 1 Ppo
Eun 2 -a -T,
Σ
po
807
t=1
1=1
Taking an expectation with respect to we have:
1
E
[T]
E
RT y
=1]
d
1=1
(1
T
Hn: 2 -aYl -T,
2d
P0
Do
80
1=1
1=1
(1
d
T
Epo
Hyn 2 -ajl Σ
Σ
E3
2d
Po
So?
For the first term: fix = (d/2)-1/p and note that if Σ T( 2 -a} < d/2, then there are more than d/2 coordinates of ys whose value is less than aa, this means
d
dd 2d = 1,
which contradicts y E B, so we must have El= T( 2 -a} 2 d/2.
For the second term, using Jensen's inequality and the concavity of Vx.
d
T
Vi
| 2
1
T
E
S
Exp
S
Po
80?
80?
20
2d
45

---

Combining we have
Rr] 2
a
T
1
T
2
20
2d
Since d d2 e² 2p+1 1 - a = - (2 2 - 2 p/P = 1/2.
If d 2 (128p.7)² then d 2 2T//0² and:
T - 1
T
T T T
=
2 20
2d
2
4 2
The condition d follows from the definition of a and p. 4:
128p.T 2T d3/p,
d 2 (128p,T)² 2 (128/1)1/1-2/99) = d'-2/v. 2 128qT = 2
o²
We hence have R[Rr] 2 T/16. The following section ensures that the Lipschitz-condition is satisfied with high-probability.
G.2.2 Bound on sub-gradients
Recall that < 4/3, p. 2 4. Given Y = i, gh ~ ~N(he,, ITI)) So ge = =ox + 2e where
X ~ N0, Ia). From 44
we have
d
(.)) 1/p- 1) = ((B
T((p=
V + +11/211) 1/0.
||I
1 E
x
20
From
(Theorem 2.2), we have
P. +1 = T P. -1
(T(
2 --+1)= S
+1)
2T
P+ -
(p. 11/2
exp
P. - 1)
)
V
2 -
<
<2vTp," 2./2p, (p. 11/2
2
2
2
2
/2 [(pp + 1)/2))
1/v.
22
//
=[IIX s 2//rdd//
Bllleelpp.| s 2
2
Fix 8 > 0. By Theorem 1.1 in (38] for some constants C1 > 0.
P(1X112 < (1+ 8)B(11X1b..) 21 C1 C1
1 exp(-epe 1oxp(-c13..de Bp
P-/2 Assuming d 2 1_1 log GiT [
we have 3 c1qd5/p. log C. < 1 and
cip.
5 T
P(IIX11p. s (1 11 2 1
So with probability at least -5/T
||11|p S (1 (1+ [p.] 2E[[[X()p]]] S Avp.a'lo.
11eell.. 4/ppodd// 1
where the final inequality follows from o
&vp.d=/p.
By union bound over all rounds, with probability 1 - 6, 119ellp. S 1 for all rounds t.
46

---

G.2.3 Proof Lemma G.5
Given Y i, the observed feedback round is exactly
f : ~ Ur, un, of), where of = 119e12022
Denote p: for the law of the observed feedback up to time T given Y = i, i... the law of (fi, ft). Denote Po (use Y in notation), the law of the observed feedback up to time T when all coordinates of gi are O-mean for all (i.e. there no better direction). Under po. fl (N of). By the definition of the total-variation distance (drv),
B [[
-a] V 0]
Ve.i 2 -ajly =1] s drv(po,p.) - [T
E
Σ -a 0] R Kun 2 -ajly = :] s T d (po:
By Pinsker's inequality we have
drv p:) $ V DKL(PO. P:).
By the chain rule for the KL divergence / operations on conditional densities:
Po (f...... fr) Ps (fi f1)
DKL pi)
(B. -pp
log
log po @fflffelu.. f1)
E (fi......)
p: (fffffo- fi)
Po ((1.lf(
Bi Esseopol- log p. (f fi)
Now since y is a deterministic function of fi,... si-1 and given Ut. Se of, under po and ft ~ ( of) under pt, we have that the inner expectation is a KL divergence between Gaussians:
Jog po Po(fflfc-1... fi)
(0
22,1/2)²
E E 1)
pt (felfft-1.... fi)
20
8o?
T
v2
T
vi.
DKL(Po,PL
E (i 10 Pm
E Pa
107
80
1=1
drv(po.P)) S
E3 80
po
Combining gives the result.
G.3 Case p +
Theorem G.6. Fix T and O. Consider d max T/ 8) and p [1, + 1/ log d). For any OCO algorithm with bandit feedback on V= Bp. there exists sequence of random linear losses tem) with graddensssnts (gl)ee such that [|gtl1p for all rounds with probability at
least 1 - 8 and
T
Rr
T6'
where the expectation is with respect to the randomness of the losses.
47

---

G.3.1 Proof
We use almost the same loss construction the proof of Theorem G.4 Before the start of the game, draw Y Unif(1..... d) and define the losses as (x) x 9 where
[W((0, (N(122,0*),
if + if
where o = (xy2ex)())/logd d) The only difference with the proof of Theorem G.4 being the value of σ. We also follow the same steps until we reach for = (2:
E [
a
2
T
1
T
2
20
2d
From d 2 8, have p 2 and 2 <8 < dso a 1- (2/d))/r 2 1-2-p/p- = 1/2.
From the definition of a = (4V2 exp(1) vlggd)
T
Tlog d
TVd
T
2xxp(1)
S 2exp(11)
2exp(1))
Vi S
20
2d
d
since 2 8
This gives:
T
T
T T T
2
20
2d
2
4
We hence have [[T]] > T/16. The following section ensures that the Lipschitz-conddiioon is satisfied with high-probabillity
G.3.2 Bound on sub-gradients
Recall that p S
log so p+ 2 logd + We have
E [[1 S 2llogd d.
By the BorelI-TIS inequality,
(IX > [|[ + +18) S <exp(_B221)
So with probability 1 8/T, we have (using > T/o)
T |x - <E[|| |00| + 2 T S /2 d V 22og S 2 log d
8
1
1
- llgelb. S |l
+ + s /0d//11111)
S o exxe 2 log d
=
2
where the final equality follows from = 1:00=(4vv0 By union bound over all rounds, with probability 1- 6, 1l..llp. < 1 for all rounds
-1 and we also used that d <
48

---

H Results for Online Mirror Descent (OMD)
H.1 OMD with uniformly-convex regularisation
The results in this section are from 40 (Proposition 7). We include them for completeness.
Let : Rd R be proper, closed and differentiable y-uniformly convex function on V of degree r > 2 w.r.t. norm ||-||. The Bregman Divergence w.r.t. & is defined for all æ, y E as
D=(r, v) (x)) - (v) (V)(y), y)- -))-
Given 21 E V, at time-step / = I...,,, the following update where Sr E (xx)).
T, Online Mirror Descent (OMD) with step-size nn > outputs
Ø(41 arg min The (gt. x) + (2,
(15)
The standard regret bound of OMD stems from the following one-step regret bound lemma (e.g. see Lemma 6.9 in (36). Lemma H.1. The iterates 15 of OMD satisfy for all V.
(((2)) Eo(u) S (gt, Tu 2(41) (v( Dulu, - D
F
From Lemma [.] and the uniform convexity of vh. we can bound the regret of OMD.
Theorem H.2. The iterates (15) of OMD with decreasing step-size M1+1 S 7 (1 I t $ T) satisfy for all u E V (recall that r. is the conjugate rf i.e. 1/r 1/r, 1),
T
Du z4)
r
E((a)) - te(2) v
max 151ST
199el""
(16)
rap--
(=1
If the step-sizes are constant: T) = 17 (1 T), we have
T
Do (u,
17--11 T
((x)) - ((4) 5 Σ
|
(17)
n
r-pp--1
Proof. By the uniform convexity of v. D& (I 2 E1|20, retill". Using this in Lemma H.] along with Hölder's s inequality, we have for all u V.
(((x))) (u( < (gt, - x1-1) Do(u,x)) - Do (u,
17t 1 1|are x++1
11..11 x Dolu, - Du, (1,,))))))))))
r
11
Consider f(x) = |-]" Then the Fenchel conjugate of f is f*( = lul" (see Lemma 2.2 in 24 and from Fenchel's inequality, have zy S 1 |r|" + 3.2 lyl",, which we use in the following.
1/r llsella)
1///r 1/r || sst2ll)
7h
||aell- ||x, De+1ll
wr/r
17t
1/r
+ 1
(///- 1/r l - setall)"
wh/r 11
1/6 -1
| se
11 -1-- + F. -- --
n
'The function t can be defined on a subset X C Rdd but conditions on its behaviour on the boundary of X are then required for OMD to be well defined. For simplicity, we consider & defined on R. though the results in this section hold more generally (see Theorem 6.7 of (36) for more detail).
49

---

where we used that r r rr./ 1). Plugging this into the above inequality,
((xz)) - (((u) < The n
r.pp-. I1eell. Do(u,))) - (u Te41)
(18)
Denoting = maxI<KsT Du(uu (u, rZ). the result follows by summing over all rounds,
T
Du, (tt2))
T Σ
Σ
(((x)) E((u) Σ
n
11
17
rap
t=1
1=1
Do(u,z1)
D
T-1
D (u, xx+1) + rape--1
Σ 71
11 l|
711
7/
7-1
T
1)
+
T]++1
T
rapt"
T
1=1
1
1
+
D
+
r/ 11
1/1
1/T
m
T.p
1=1
D
+
19
Txpe--1
For constant step-size, the result follows similarly by summing 18
over giving telescoping sum,
T
Do (u( 21) - (u, IT+1)
T
Σ
((()))
(e(u)
19t
lI
T.
Do(u,z1)
+ 17
T
7
T T.100-11
which concludes the proof.
H.1.1
Regret bounds
When have T--Lipschitz losses w.r.t. for constant step-sizes becomes
||-|| and we bound (u, < D, then the regret bound
D S + TA
TL"
R
Assuming the time-horizon T is known, optimising the above bound w.r.t. 1 using Lemma C.4 gives
Rr S 1/r 1/r LDV/TA//",
(19)
for 7 = (Dp/ With we recover the standard regret bound of OMD using strongly-convex regulariser Rt S 22/2//
H.1.2 Anytime and adaptive bounds
When T' is unknown, we can use the bound in 16 and the time time-varying step-size
D1/re (r 1)1/"=p'' 1/r
L/ /rr, 1/r.T//r.
17:
tl/r.
get
Rr
///r
(20)
where Dmax is bound on maxi (w, .4). Though this can unbounded when bounded, for our purposes of ba-bal Dmaxx will only be constant away from D. The doubling trick can also be used obtain anytime bounds that depend on / instead of D (see e.g. (7)). This uses constant step-size OMD on morehooizons of doubling lengths until the unknown true T is reached.
50

---

We can also obtain bounds that adapt to the sequence of observed subgradients of the form
Diiker //rr///////////// 1/r.
T
1/r.
R1
p/rr
19 This follows the same lines as for OMD) with strongly convex regulariser (see Section 4.2.1 of [3)
by using ? Diaax (r- 1)1 uu /
H.2
OMD on b--balls
For the low-dimensional setting, consider OMD with regulariser d2(x) -/1=
We have Dmax = supx,ye,, 1 1| yl1 2d'-2/rr and using that Φ2 is L-strongly-convex with respect to
11-112, we have from 20) with 2 and 11- = +. + 8
Rr S 2LV 2d'-T/PT.
For the high-dimensional setting, consider OMD with regulariser 6p(x) Dima = S, (2, = (see below) and using that Up is uunnfoifoomlly-com of degree p with respect to |1-11p- we have from (20) with p and L p-1
1 |
We have
p p'
1-1/p
Pr S ppp.
1- 1/p. LT2-1/P
To show Dmax = 2: Fix 2, E B, and (xx)) = |laalIp. The sign, power and absolute value functions below are applied component wise to vectors.
(x( = (xx) - v(y) - (V)(v), x- v)
= | IBulI +
sign(ys) 18:10 (y: xz))
Ilull +
11r1 sign(v))) 1126 11611-- x2
p
1 (1-
||01 +
|1v1 sign(y:)) 11611 x
p
$ + [1 Σ sign(ys) 11s |u ma)
We show that the last term is bounded by by using Holder's inequality, (sign(y) |y/ .x) S |11// ||1||=1 S 1,
where in the last inequality used (recall that q p/ (p- 1))
|||yp~111, = Σ 1 (o-)))) 1/9 =
1/4 Σ lui)))
= 11111//9 < 1.
Hence suP,,ess, (T v) S Dinx-
We now show how to achieve anytime optimal bounds. Fix to (1 1/p 20/(0-2) Proposition H.3. Consider running OMD with the following regularizers
d.
(p(x)) = ||1=0 T) (p- 131/p- (2(2)) = |
ift $ to- ift> to-
Vx(x)=
Lstips
22 Lit
Assume l, convex, closed, and (x)():) not empty. Then, OMD guarantees
2p 1/1 1/p LT
ift < to. ift to
Rr <
2L 2T 2LV2T' 2
51

---

Proof. If T S to, we have just run OMD with op as regulariser over all rounds and the regret bound is the one for the high-dimensional setting above. Otherwise, from the standard bounds from the OMD analysis
D Dy (x)
Døp (u, (141))
L
RT
7/2
1/
1/
P.pp
p.-1
T
Duso Dus(u, x4)
Des (141)
I2a
T
+
+
178
2/12
1=
1=
Dp
L
D2
I
+
1T.
-1
+
Σ T
T/a
papp
p.
T/T
2/12
1=1
<
2p
/ppp
1/9a Liu
1/p-
+
D2
+
LvD2
Σ
T
1/T
2p
t=ta+1
- 2pp pr
1/p
1/p- Ltd
2/p- + LVTDD + LV D2 (VT - tt
2Lv 2T 2Td + 2p /o 1/p. Lt 2/p- 2d22po//
where we used D2 supx, Date (z. v) - 2d 2//P, Dp suP,.yEB, Doi, (1r, 2 112 =1
and Up = 2P The proof is concluded by noting that 2p 2pp/rp//" /Pp Lt L 22d 2/pto is
negative for to = (V2p ///.) 2p-(m-2)
d.
L
H.3 Failure of fixed separable regularisation for OMD
Proposition H.4. OMD with regulariser vb E To and any sequence of decreasing Ne cannot be optimal across all dimensions. Specifically there are no constants Cy. q > 0 such that for all T, RT S en LT for all d > T and RT S CLVTd 2/p for all d S T.
The proof is identical to the proof of Theorem 4.6) for FTRL with the corresponding versions of Lemma 4.4 Lemma 4.7 for OMD given below.
Lemma H.5. [Lemma 4.7 for OMD] Consider 1 B, = [-1, and ib E.. OMD with regulariser va and arbitrary decreasing step-size tpe can only guarantee Rr S cLVTT for some constant e 0 and all sufficiently large T for all 1-,, (xx) +01/2),
100c
Proof. Assume there exists a constant c > such that for all T and any sequence of losses, Ry S
cLyT. T. Consider > 16c² and a multiple of 4.
By considering the dual-version of OMD, we have that if there are no projections up to time f, the update of OMD at time t 1 can be written as
F++1
V Σ no:k
arg min (xxx Σ Th 27
(21)
TEV
where v/ is the restriction of the fenchel conjugate of s to V = B, = 1-1
We now follow the same steps as FTRL with slight modification to the loss (x(x) = x 9t where
9t E [1 is now defined as
Met1 9
L.
t 5 odd, 1 < even,
9t
L
L.
>> 2
Assume 722 is small enough s.t. 22 E int V = (-1,1) (i.e. no projection is needed). If not, we can modify the losses slightly so that 33 = 0 (if и/a large enough, if is not then set gi = 9/ = 0 and start the above losses from / 3) and then proceed similarly (i.e. if ns is still so large that = 1, then again modify the losses slightly so that v5 = 0 etc). Set v = fT /2- With this sequence of losses, the points played by OMD satisfy
52

---

fort T/2 + and t odd, we have 261 17.9. = s0
so 2, = 0.
fort & T/2+11 and even, we have E'=1 7799 = -T L so 21 = arg min min -
m7 + (x)).
For 1 < & T/2 (both even), we have
Lay + (b(v)) I -ne Lor
Lor + v1(2))
using the definition of z/
= LLL + (2r) + L - 110 )x
< m LLLY + ((xx) Lm no )x
using the definition of Fz
= (m - M10 Lav $ (n - n Laz
= It S It
using that T/e S Tp-
So for all T/2 even, we have 2₁ 2 27/2.
for t > T/2, we have E 7.95 2 E.=1 7.9s S0 S T++1
The regret can be written as follows
RT Σ ((()))
LT
LT
LT
11/2 L
Σ
T
(22)
2
4
(=T/2+1
Following similar steps the proof of Lemma 4.7 for FTRL, get
1. We first show that x 7 +2oV/
2: if not, I +222// < 1 and from (2
T/2+ [2o/T]
LT
Rr 2
L
L
Σ
T
2
(=1/2+1
t=T/2+[2ovT]
T
LT
T/2+|20VT]
T
L
Σ
1
2
2
1=7/2+1
1=T/2+ [2e/ T
LT
L
[2cVT] T
[2eVT]
2
2
2
L
[2eVT]]
2
cLVT,
which contradicts initial assumption that Ry S cLVT must have By +22VT) Note that TevT <T/2 is ensured by T > 16c2.
2. Next, we show that 17 2 #2122)
2LLVT" Until the points reach there are no projections so we can use (2T) to write the OMD update as (note that even x # the following still holds)
++120/1111
Σ+[2c/T1-1
I T + (2e2T] = arg min (x))++ +
Σ
ni9t x = arg min
(b(()) x
Σ +
7h
zEV
øEV
A T+ -1
Ti (h ((()))) )
-
i=
]+12eVTI-1
x1//22 <
Σ
The S 2c/Tn
+
ク (1/22)
2cvT
53

---

where in the second implication, we used that Φbb +[20VT) 2 20112) (since vé is in- creasing on [0, and my + +2 2 1/2), 2y + (20/T] S and h) S 1T/2 7) for all
2 T/2.
The remaining steps are identical to the proof of Lemma 4.7
Lemma H.6 (Lemma 4.4 for OMD). Consider V = B, with p > 2 and assume losses are L-Lipschitz in bp noroo. Let be a convex function satisfying for some μ > and any x E R,, v(x) 2 - If d 2 / there exists a sequence of linear I.-Lipschitz losses (in f--norm) for which OMD with regulariser (xx) and any sequence of decreasing suffers regret RT 2 LT.
Proof. We consider the loss construction described in Appendix E. with slight modification the loss (x) = I.. x² 9t where 9t E Bp. is now defined as
Me L E1.
5 T odd,
2
9t
L1,
1 1 even,
L. t
By again considering the dual-version of OMD, we have that if there are projections up to time t, the update of OMD at time + 1 can be written as
2++1 = V (-L Σ n.gi
arg min (x)x + L Σ noo 3
(23)
ZEV
1=1
1=1
where ot is the restriction of the fenchel conjugate of v to V = Bp.
First consider & < T/2. As in the proof of Lemma H.S assume 7/2 is small enough s.t. 22 E int B, (i.e. no projection needed). By (3). the steps are then the same as for FTRL since when is odd, 2, = and when t even, x, = arg minges, {v(x) - Lm x So have
T/2
(T/4, 0,
if 1r/2 2 2/L 11/2 < 2/L..
Σ
r Z
Hence if 1///2-1 2 2/L, we have Rr 2 LT/4 and the statement of the theorem holds. 11/2-1 < 2/L, we look to the second half of the rounds.
Let's now consider > T/2 and assume 7T/2-1 < 2/L. Fix 3 = 1. Until the points reaches the boundary there are no projections so we can use (2I) to write the OMD update as
Tt arg min (x) rEB,
McG.
arg min (1(x) L TT xEB,
Σ
T=T/2+1
arg min zEB,
Σ g(())
Lxnd 1/9
Σ
TT/+++
Let u = v/[lolp the competitor. Note that x, X (since the update is coordinate invariant) so only reaches the boundary once 2; still holds (this true because it is true for the last iterate before the projection and then
and for which the above equality
E Tegs is greater than for this last iterate so the argmin will give Ie = u). We have 2 0 and
(xx)) 2
1 2
- 2i 2/p -2/p
1
2
54

---

Now from the OMD update,
(1(x)) Lv Σ M) 0
pp 2/1 - xeI
Σ
T1i S 0
2
T=
T=//2+1
21.
1-1
21,3011
43t
Σ
Fli S
pel 11/2
pdl 2/pp
pdd
:=1/241
4L
= (((x)) = Lo 3t = L\ 2
pdl pdl-2/p'
since 77: S 1//2 S 2/L for all 2 T/2. If 2 (4T (4/ we have for all t T
4LBe
(x(11))
2
2/2 pdl 11/2
2
LT
T
LT LT LT
RT N
((()))
2
4
2
t=T/2+1
or T I not divisible by and we use 1, T concluding the proof.
3, we have RT 2
L(T-3)
4 2
LT for T 2
55


Reviews:
[{"review_id": "7x0rJspqg8", "reviewer": "Reviewer_SbZL", "summary": "The authors established that adaptive regularization is essential for achieving anytime optimal regret for FTRL on high-dimensional $\\ell_p$-balls ($p>2$). It demonstrates a shift in optimal regret rates between low-dimensional ($d \\leq T$, rate $\\mathcal{O}(\\sqrt{Td^{1-2/p}})$) and high-dimensional regimes ($d > T$, rate $\\mathcal{O}(T^{1-1/p})$), proving that FTRL with time-varying regularization can achieve optimality across all dimensions without prior knowledge of $T$ or the regime. Crucially, the authors show that no fixed separable regularizer (e.g., $\\|x\\|_r^r$) can be optimal in both regimes simultaneously, resolving an open problem and highlighting the necessity of adaptivity. They further establish that agents suffer unavoidable linear regret under linear bandits in sufficiently high dimensions, underscoring fundamental limitations beyond full-information settings. This paper significantly advances the understanding of geometry-aware online learning.", "questions": "- Whether the assumption that $0$ is an interior point of the action set $V$ is fundamentally necessary for this paper? Also, do the authors have some intuition about whether the $\\ell_p$-ball structure can be generalized to certain more general convex sets to still have similar negative results for separable regularizers?", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "- The regularizer in equation (2) is novel and might be of independent interest.\n- The understanding of the necessity of adaptive regularization is clear, clean, and novel.\n- This paper leaves a clear open problem on whether a non-separable static regularizer can achieve the same guarantees as the adaptive regularizer in the case of $\\ell_p$-balls.", "was_revised": true, "final_justification": "This paper is both technically highly nontrivial and significant at a message level. It brings new understanding of the non-asymptotic scaling behavior between the interplay of feature dimension and time horizon for FTPL. Therefore, my final decision is strong accept.", "rebuttal": "We thank the reviewer for their positive feedback. We address your questions below.\n\n**Q1: Whether the assumption that 0 is an interior point of the action set is fundamentally necessary for this paper?**\n\n- We could consider translating the lp-ball away from 0, then our results will hold for regularisers satisfying the conditions we need on this translated ball (e.g. reaches its minimum in the centre of the translated lp-ball). We will add a remark on this in the final version.\n\n**Q2: Do the authors have some intuition about whether the -ball structure can be generalized to certain more general convex sets to still have similar negative results for separable regularizers?**\n- This is an interesting question. What we need for Theorem 4.6 (our main result on the failure of separable regularization) to hold is \n1. that in the 1-dimensional case, the optimal regret requires quadratic growth of the regularizer.\n2. In the general d-dimensional case, quadratic growth of the regularizer leads to sub-optimal regret. \n\nPoint 1. will hold for any symmetric action set. The question then becomes whether we can show 2. for these more general convex sets, which is not clear We would also require that there is a similar shift in the optimal rate of regret between dimension-regimes, which is not necessarily obvious.\n\nHow general this phenomenon might be is likely related to the concept of quadratically convex sets [1] for which several results are known on the posibility of getting regret that grows as $\\sqrt{T}$ in the low-dimensional case, while it is likely that the regret is better in the high dimensional case (when taking into account dependence on other quantities like dimension). This is an interesting direction of future research. We will also add a remark on this in the final version.\n\n[1] Daniel Levy and John C Duchi. Necessary and sufficient geometries for gradient methods. In Advances in Neural Information Processing Systems, volume 32, 2019."}, {"review_id": "mCvrKmJ8eV", "reviewer": "Reviewer_vyFS", "summary": "The paper studies online convex optimization over $l_p$ norm ball domains with $p > 2$. It investigates the role of regularization in Follow-the-Regularized-Leader (FTRL) and Online Mirror Descent algorithms and shows that the optimal separable regularizers for low- and high-dimensional settings are not equivalent, even up to time-varying scaling factors. The authors further prove that no time-invariant separable regularizer (up to time-varying scalars) can achieve an optimal anytime regret guarantee. However, they establish the existence of a time-varying separable regularizer that does provide such a guarantee. Finally, in the bandit feedback setting, they show that for action sets of $l_p$ norm balls with $p \\ge 1$ and linear losses, linear regret is unavoidable when the dimension is sufficiently large.", "questions": "- Theorem 5.1 claims that linear regret is unavoidable in the high-dimensional setting. However, this conclusion would also follow from any dimension-dependent lower bound of the form $\\Omega(d^a T^b)$. Does the lower bound instance presented in the paper yield any nontrivial results for $l_p$ norm balls that are stronger than those known for the multi-armed bandit setting? This is particularly relevant since, for $p \\in [1, 2]$, the regret bounds are already known to be the same.\n- Have the authors considered using fixed log-barrier regularizers, which are not part of the class of separable regularizers?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper tackles the universality of FTRL algorithms with fixed regularizers, addressing a fundamental and challenging problem. It provides a negative result for the subclass of separable regularizers.\n- The analysis and proof techniques are rigorous and carefully executed.\nWeaknesses:\n- While separable regularizers are indeed widely used, they are not a fundamentally necessary class. As a result, the negative result primarily highlights the need for more sophisticated fixed regularizers rather than a limitation of FTRL itself.\n- The negative result on linear bandits appears somewhat trivial, as it essentially demonstrates the dependence of regret on the dimensionality.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive and insightful feedback. We address your comments, suggestions and questions below.\n\n# Weaknesses\n\n**While separable regularizers are indeed widely used, they are not a fundamentally necessary class. As a result, the negative result primarily highlights the need for more sophisticated fixed regularizers rather than a limitation of FTRL itself.**\n- This is correct. But in fact, we view this as one of the main messages of our work. Our results demonstrate that existing regularizers impose intrinsic limitations on FTRL and open up an interesting avenue of research to discover these more sophisticated alternatives that potentially give algorithms that are fundamentally different. Also note that our main result holds for non-separable regularizers that are a multiplicative constant away from being separable.\n\n**The negative result on linear bandits appears somewhat trivial, as it essentially demonstrates the dependence of regret on the dimensionality.**\n- Yes, it essentially demonstrates that the dependence of the low-dimensional regret on the dimensionality is maintained as the dimension increases but this is not always obvious. For example, in the full-information setting that we consider in most of our work, this is not the case. While it may be more expected that it is the case in the harder bandit feedback setting, proving it formally in the way that we do is not trivial. \n\n# Questions\n\n**Q1: Theorem 5.1 claims that linear regret is unavoidable in the high-dimensional setting. However, this conclusion would also follow from any dimension-dependent lower bound of the form $\\Omega(d^a T^b)$. Does the lower bound instance presented in the paper yield any nontrivial results for norm balls that are stronger than those known for the multi-armed bandit setting? This is particularly relevant since, for $p \\in [1,2]$, the regret bounds are already known to be the same.**\n- This conclusion would indeed also follow from any dimension-dependent lower bound of the form $\\Omega(d^a T^b)$ but usually lower bounds of this form only hold in the low-dimensional setting (e.g. in [1], their lower bound Theorem 4 only holds for $T \\geq d^{2/(1-q/2)}$ and does not give linear regret in high dimension). As far as we are aware, for $p \\in [1,2]$, the regret bounds are known to be the same as for the multi-armed bandit setting only in the low-dimensional setting, making our results non-trivial in the high-dimensional setting. We will add a remark to clarify this in the final version.\n\n**Q2: Have the authors considered using fixed log-barrier regularizers, which are not part of the class of separable regularizers?**\n- We did not consider specific non-separable log-barrier regularizers but it is unclear how we could use it since they do not satisfy uniform convexity properties that allow us to obtain regret guarantees. However, we would be really happy to receive any pointer in this direction.\n\n\n[1] Sébastien Bubeck, Michael Cohen, and Yuanzhi Li. Sparsity, variance and curvature in multi-armed bandits. In Proceedings of Algorithmic Learning Theory, Proceedings of Machine Learning Research, pages 111–127, 2018."}, {"review_id": "i8NLwoXJXP", "reviewer": "Reviewer_5h3T", "summary": "This paper investigates the fundamental question of whether Follow-the-Regularized-Leader (FTRL) algorithms can achieve optimal regret bounds across all dimensional regimes using a single fixed regularizer across time, specifically for online convex optimization on $\\ell_p$-balls with $p > 2$.\nIt is known that optimal regret rates exhibit a dimensional shift: in low-dimensional settings ($d \\leq T$), the optimal rate is $O(L\\sqrt{T d^{1-2/p}})$ achieved using strongly-convex regularizers such as $||x||^2$, while in high-dimensional settings ($d > T$), the optimal rate is $O(LT^{1-1/p})$ achieved using uniformly-convex regularizers of degree $p$. This creates a challenge for \"anytime\" algorithms that must perform optimally without knowing the time horizon $T$ in advance as the optimal regularizer depends on the time horizon. \n\nThe paper's major contributions are in two directions:\n1) Algorithm: The authors design an adaptive FTRL algorithm that achieves anytime optimality by switching regularizers at a carefully chosen threshold $t_0 = 3^{-2p/(p-2)}d$. The algorithm uses uniformly-convex regularization initially (ideal for high dimensions) and switches to strongly-convex regularization when the low-dimensional regime is reached.\n\n2) Fundamental impossibility result: The paper's central contribution proves that this adaptivity is fundamentally necessary for separable regularizers. Specifically, they show that no fixed separable regularizer (of the form $\\psi(x) = \\sum_i=1^d g(x_i)$) can simultaneously achieve optimal regret in both dimensional regimes. This impossibility result rules out regularizers like $||x||_r^r$.\n\nAdditionally, there are also algorithm-specific lower bounds for particular regularizers and extensions of the analysis to bandit feedback, showing that sub-linear regret becomes impossible in sufficiently high dimensions for linear bandits on $\\ell_p$-balls.", "questions": "- What is the main technical issue with extending the present proofs to non-separable regularizers? Is it the analog of Lemma 4.7? \n- Could you explain a bit more about the prior work in this domain? Have negative results of the flavor presented in this work appeared before in the literature? \n- Any insights on the algorithmic component to complement the impossibility results in Section 5?", "limitations": "- Yes, assumptions are clearly stated", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- Correctness: The mathematical analysis is rigorous and sound. The proofs are carefully constructed and the main impossibility result (Theorem 4.6) is established through an argument connecting quadratic growth requirements to high-dimensional failure.\n\n- Studies an interesting question: The paper addresses a fundamental and practically relevant question about the universality of FTRL in online learning algorithms. Understanding when adaptive regularization is truly necessary versus when fixed regularizers suffice has implications for algorithm design, especially in settings where the dimensional regime is unknown a priori.\n\n- The algorithm is simple and intuitive, the lower bound is quite interesting too. The adaptive algorithm switches between two well-understood regularizers at a natural threshold, making it easy to implement and understand. The impossibility result provides insight by showing that the intuitive approach of using a single \"compromise\" regularizer fundamentally cannot work, which is both (to me) surprising and illuminating.\n\nWeaknesses \n\n- Heavy technical machinery for simple switch: While the core algorithmic idea is straightforward (switch regularizers based on time), the paper employs substantial technical apparatus to analyze this simple procedure. The specific threshold calculation and analysis feel disproportionate to what is essentially an elegant but basic adaptive strategy. The paper would also benefit from more intuition behind the proofs and sketches for some of the important building blocks that are relegated to the appendix (e.g. Lemma 4.7). \n\n- Bandit section is underdeveloped: Section 5 on bandit feedback feels somewhat disconnected from the main narrative and lacks the depth of analysis given to the full-information setting. The impossibility result for high-dimensional bandits is interesting but doesn't provide corresponding positive results or adaptive algorithms for the achievable regimes, leaving this part of the story incomplete.\n\n- Assumptions on separability: The main impossibility result is restricted to separable regularizers, which, while covering many important cases, leaves open the most general version of the question. This limitation feels somewhat artificial and prevents the paper from providing a definitive answer to the broader question of FTRL universality with fixed regularizers.", "was_revised": true, "final_justification": "I keep my score as this is a good paper and recommend acceptance.", "rebuttal": "We thank the reviewer for their positive, constructive and thorough feedback. We address your comments, suggestions and questions below.\n\n# Weaknesses:\n\n**Heavy technical machinery for simple switch**\n- While the proof of Theorem 3.1 in Appendix D is tedious and may obscure from the fact that it involves mostly algebraic computations, we attempted to make clear that the result consists of arguing with a sequence of elementary arguments (line 216: “The proof is in Appendix D and consists of a careful application of Theorem 2.3.” where Theorem 2.3 is just the general regret bound for FTRL).\n\n**The paper would also benefit from more intuition behind the proofs and sketches for some of the important building blocks that are relegated to the appendix (e.g. Lemma 4.7)**\n- This is a good point - we will include intuitions for some of these results/proofs in the final version - at least in the appendix and in the main text if space allows (see also answer to reviewer QhTE).\n\n**Bandit section is underdeveloped: Section 5 on bandit feedback feels somewhat disconnected from the main narrative and lacks the depth of analysis given to the full-information setting. The impossibility result for high-dimensional bandits is interesting but doesn't provide corresponding positive results or adaptive algorithms for the achievable regimes, leaving this part of the story incomplete.**\n\n- Our main results are for the full-information setting. We included the bandit results as an addition because we thought it was relevant to provide the separation with full-information that sub-linear regret is not possible in high-dimension. However, there are indeed still many unanswered questions including the optimal rate of regret in the low-dimensional setting (there is no matching lower-bound as far as we know) that leave open many possibilities for future work.\n\n**Assumptions on separability** \n\n- Resolving what result can be achieved for general regularizer that we leave open is of great importance to better understand the limitations and strengths of FTRL (see answer to reviewer vyFS). Also note that our result holds for non-separable regularizers that are a multiplicative constant away from a separable regularizer.\n\n# Questions:\n\n**Q1: What is the main technical issue with extending the present proofs to non-separable regularizers? Is it the analog of Lemma 4.7?**\n\n- Exactly, the difficulty is in extending Lemma 4.7 on the quadratic-growth of the regulariser beyond the 1-dimensional case. In fact having a general result on the necessity for quadratic growth or some form of strong convexity to achieve optimal regret in the low-dimensional setting could be of independent interest.\n\n**Q2: Could you explain a bit more about the prior work in this domain? Have negative results of the flavor presented in this work appeared before in the literature?**\n\n- As far as we are aware, results on the failure of fixed regularization are novel in online learning. Algorithmic specific lower bounds (like the ones we have for specific regularizers - Propositions 4.1 and 4.5) have appeared in prior work (e.g. Theorems 3&4 in [1]).\n\n**Q3: Any insights on the algorithmic component to complement the impossibility results in Section 5?**\n\n- In the low-dimensional setting, the algorithm from [2] that achieves a regret bound of $\\tilde{O}(T^{1-1/p} d^{1/p})$ is based on OMD with a carefully designed barrier regularizer. This result holds for all d but becomes loose as d increases. Our lower bound says that sub-linear regret is not possible in high enough dimension. It is unclear whether there are different optimal regret regimes within the low dimensional setting which would require an adaptive algorithm, it is even unclear if the regret bound of [2] is optimal in any dimension. As mentioned in the weakness section, there is still a lot to understand on the optimal rates of regret for the bandit setting.\n\n[1] Francesco Orabona and Dávid Pál. Scale-free online learning. Theoretical Computer Science, 716:50–69, 2018.\n\n[2] Thomas Kerdreux, Christophe Roux, Alexandre d’Aspremont, and Sebastian Pokutta. Linear bandits on uniformly convex sets. Journal of Machine Learning Research, 22(284):1–23, 2021."}, {"review_id": "ldSgohkltp", "reviewer": "Reviewer_QhTE", "summary": "This paper studies online convex optimization over the $\\ell_p$ ball for $p>2$, where the loss functions are Lipschitz continuous with respect to the $\\ell_p$-norm. The case when $p\\in[1,2]$ has already been studied previously. Let $T$ denote the time horizon and $d$ the dimension of the space. Existing work has shown that the optimal regrets differ between the low-dimensional regime ($d<T$) and the high-dimensional regime ($d\\geq T$).\n\nFor lower bounds, the authors prove that\n* FTRL with $\\ell_2$-norm regularization, which is known to be optimal in low-dim, is sub-optimal in high-dim.\n* FTRL with $\\ell_p^p$-norm regularization, which is known to be optimal in high-dim, is sub-optimal in low-dim.\n* FTRL with a fixed strongly-convex or fixed separable regularizer cannot achieve optimal regret in both settings simultaneously.\n\nFor upper bounds, the authors propose an FTRL algorithm with adaptive regularization and show that it achieves the optimal regret in both settings.", "questions": "The following questions are simply out of my curiosity.\n\n1. The results in this paper show that non-adaptive separable regularizers cannot be optimal in both settings simultaneously, which applies to many existing FTRL algorithms. Are you aware of any non-separable regularizers in the literature?\n2. Could you provide any intuition or motivation behind the loss construction (Section E.1) used in the lower bound proof?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I have read the main paper and skimmed through Appendices A-E.\n\n# Strengths\n\n### Quality and Clarity\nThis paper is easy to follow. The motivation is clear, and the related work is adequately addressed. The theoretical results are sound and rigorously supported.\n\n---\n### Originality\nThis paper identifies a gap in the existing literature: Although optimal algorithms for both the high-dim ($T<d$) and the low-dim setting ($T\\geq d$) are known, there is no single algorithm that is asymptotically optimal for both settings simultaneously. This gap was not previously recognized and is rigorously established in the paper.\n\nFurthermore, the authors extend the lower bounds to non-adaptive strongly convex and non-adaptive separable regularizers. As far as I know, these lower bounds are novel and are the paper's main technical contributions.\n\n---\n### Significance\nThis paper not only closes the above gap but also raises several interesting questions, such as the universal optimality of FTRL and the benefit of non-separable regularizers. It has the potential to guide future research, and I am happy to recommend acceptance.\n\n---\n\n# Weaknesses\nNo significant weaknesses are found. Below are some minor comments:\n\n* In Line 110, should $T^{1/p}$ be $T^{1-1/p}$?\n* In Theorem 4.2, I suggest including the definition of a \"sign-invariant regularizer.\"\n* In Line 279, it is stated that \"its regret guarantees are provably dimension-independent and consequently sub-optimal in low-dimensions.\" I don't fully understand this implication. My understanding is that the rate $T^{1/p_\\star} = T^{1-1/p}$ in Proposition 4.5 is larger than the optimal rate $T^{1/2}$ for $p>2$, and therefore the uniformly convex regularizer is not optimal. Could you elaborate more on this statement?", "was_revised": true, "final_justification": "This is a good paper and there are no significant weaknesses. In their rebuttal, the authors also provide intuitions for their proofs. I am happy to recommend acceptance.", "rebuttal": "We thank the reviewer for their positive and constructive feedback. We address your comments, suggestions and questions below.\n\n# Minor comments:\n\n- In line 110, you are right it should be $T^{1-1/p}$, thank you for pointing this out!\n- We will include a definition of “sign-invariant regularizer” in the final version.\n- In Line 279, the phrasing we used was confusing. We meant the following: for $p>2$, the optimal rate in the low-dimensional setting is $O(\\sqrt{Td^{1-2/p}})$ which is dimension-dependent. Proposition 4.5 shows that the regret guarantees do not improve for smaller dimensions, so are dimension-independent and therefore sub-optimal (so “provably dimension-independent and sub-optimal” in the sense that the optimal rate is dimension-dependent). We realise this phrasing is confusing and will amend it in the final version.\n\n# Questions:\n\n**Q1: Are you aware of any non-separable regularizers in the literature?**\n\n- There is for example the squared r-norm $\\Vert x \\Vert_r^2 = (\\sum_{i=1}^d \\|x_i\\|^r)^{2/r}$ for $r\\in(1,2]$ which is not separable and has been considered in the literature (e.g. see section 6.7 in [1]). It is however strongly convex with respect to the r-norm so Theorem 4.2 guarantees that it is sub-optimal in high dimension.\n\n**Q2: Could you provide any intuition or motivation behind the loss construction (Section E.1) used in the lower bound proof?**\n- For the first half of the rounds we successively use linear losses with gradient $\\pm e_1$.\n- For the second half we use a fixed linear loss with gradient a rescaled version of a point in the corner of the $\\ell_p$-ball (so that competitor is this point in the corner which may be hard to reach). \n- The different losses in the two halves of the rounds capture some sort of bias-variance trade-off of the algorithm:\nIf the step-size is large then the algorithm will quickly reach the corner of the ball (the competitor) in the second half of the rounds but in the first it will suffer large regret because it moves too fast through the space and every other round it will get to close to $e_1$ which will make it suffer large loss in the next round (it has high “variance”).\n- If the step-size is too small, it will not suffer much regret in the first half of the rounds but in the second half it will not be able to reach the corner of the ball (the competitor) sufficiently quickly and suffer large-regret (it has high “bias”).\n- We chose this loss construction so that a large or small step-size does not benefit the algorithm: A perfectly tuned step-size reaches the correct balance between the regret suffered from both halves and corresponds to the optimal regret (both halves also correspond to the two terms that appear in the regret bounds of OMD/FTRL before tuning of the step-size). We will include this intuition in the corresponding appendix in the final version of the paper.\n\n[1] Francesco Orabona. A modern introduction to online learning. 2019. Preprint, arXiv: 1912.13213."}]

Author rebuttal:

Meta-review: (a) Summary: The paper studies the problem of achieving optimal anytime regret in online convex optimization over $l_p$-balls, for p > 2. The paper has two main contributions:
  - A positive result: showing that the FTRL algorithm with adaptive regularization achieves anytime optimality, leading to the question of whether optimality is achievable with a fixed regularizer. 
  - Accompanying negative results: anytime optimality cannot be achieved by any fixed, separable regularizer. 

Additionally, the paper extends its investigation to the bandit feedback setting, showing that for linear bandits on $l_p$-balls, linear regret is unavoidable in sufficiently high dimensions.

(b) Strengths:
  - The paper is well-motivated and studies a fundamental problem, namely the necessity of adaptive regularization for anytime optimality in OCO over lp balls.
  - The results are sounds and the analysis is rigorous. The optimal algorithm is intuitive, and the lower bound is novel.
  - The paper is well-written.

(c) Weaknesses:
  - The main lower bound applies only to separable regularizers, leaving the broader question of whether a non-separable fixed regularizer could achieve anytime optimality open.  

(d) Rationale: I recommend acceptance because it is a technically solid paper with clear motivations and interesting results. 

(e) Summary of Rebuttal and Discussion
The authors provided detailed responses to all reviewer questions, and the responses were positively received. All reviewers maintained their original positive scores. Main Points Raised by Reviewers:
  - Reviewers vyFS and 5h3T raised concerns about the limitation to separable regularizers, and the completeness of the section on bandit feedback.

How Authors Addressed the Points:
  - On separable regularizers: The authors agreed this was a limitation but they viewed it as a key message of their work, as separable regularizers are common, and the lower bound motivates the search for non-separable ones. They also clarified (in response to 5h3T) that their result already covers non-separable regularizers that are a multiplicative constant away from a separable one.
  - On the bandit section: They acknowledged to 5h3T that this section was an addition to highlight the separation from the full-information setting and that many open questions remain for future work.
Decision: Accept (spotlight)