Title: On the necessity of adaptive regularisation: Optimal anytime online learning on $\boldsymbol{\ell_p}$-balls
Venue: NeurIPS 2025
Accepted: yes
Keywords: OCO

lp-balls

high-dimension
Abstract: We study online convex optimization on $\ell_p$-balls in $\mathbb{R}^d$ for $p > 2$. While always sub-linear, the optimal regret exhibits a shift between the high-dimensional setting ($d > T$), when the dimension $d$ is greater than the time horizon $T$ and the low-dimensional setting ($d \leq T$). We show that Follow-the-Regularised-Leader (FTRL) with time-varying regularisation which is adaptive to the dimension regime is anytime optimal for all dimension regimes. Motivated by this, we ask whether it is possible to obtain anytime optimality of FTRL with fixed non-adaptive regularisation. Our main result establishes that for separable regularisers, adaptivity in the regulariser is necessary, and that any fixed regulariser will be sub-optimal in one of the two dimension regimes. Finally, we provide lower bounds which rule out sub-linear regret bounds for the linear bandit problem in sufficiently high-dimension for all $\ell_p$-balls with $p \geq 1$.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies online convex optimization over the $\\ell_p$ ball for $p>2$, where the loss functions are Lipschitz continuous with respect to the $\\ell_p$-norm. The case when $p\\in[1,2]$ has already been studied previously. Let $T$ denote the time horizon and $d$ the dimension of the space. Existing work has shown that the optimal regrets differ between the low-dimensional regime ($d<T$) and the high-dimensional regime ($d\\geq T$).\n\nFor lower bounds, the authors prove that\n* FTRL with $\\ell_2$-norm regularization, which is known to be optimal in low-dim, is sub-optimal in high-dim.\n* FTRL with $\\ell_p^p$-norm regularization, which is known to be optimal in high-dim, is sub-optimal in low-dim.\n* FTRL with a fixed strongly-convex or fixed separable regularizer cannot achieve optimal regret in both settings simultaneously.\n\nFor upper bounds, the authors propose an FTRL algorithm with adaptive regularization and show that it achieves the optimal regret in both settings.\n\nSTRENGTHS AND WEAKNESSES:\nI have read the main paper and skimmed through Appendices A-E.\n\n# Strengths\n\n### Quality and Clarity\nThis paper is easy to follow. The motivation is clear, and the related work is adequately addressed. The theoretical results are sound and rigorously supported.\n\n---\n### Originality\nThis paper identifies a gap in the existing literature: Although optimal algorithms for both the high-dim ($T<d$) and the low-dim setting ($T\\geq d$) are known, there is no single algorithm that is asymptotically optimal for both settings simultaneously. This gap was not previously recognized and is rigorously established in the paper.\n\nFurthermore, the authors extend the lower bounds to non-adaptive strongly convex and non-adaptive separable regularizers. As far as I know, these lower bounds are novel and are the paper's main technical contributions.\n\n---\n### Significance\nThis paper not only closes the above gap but also raises several interesting questions, such as the universal optimality of FTRL and the benefit of non-separable regularizers. It has the potential to guide future research, and I am happy to recommend acceptance.\n\n---\n\n# Weaknesses\nNo significant weaknesses are found. Below are some minor comments:\n\n* In Line 110, should $T^{1/p}$ be $T^{1-1/p}$?\n* In Theorem 4.2, I suggest including the definition of a \"sign-invariant regularizer.\"\n* In Line 279, it is stated that \"its regret guarantees are provably dimension-independent and consequently sub-optimal in low-dimensions.\" I don't fully understand this implication. My understanding is that the rate $T^{1/p_\\star} = T^{1-1/p}$ in Proposition 4.5 is larger than the optimal rate $T^{1/2}$ for $p>2$, and therefore the uniformly convex regularizer is not optimal. Could you elaborate more on this statement?\n\nQUESTIONS:\nThe following questions are simply out of my curiosity.\n\n1. The results in this paper show that non-adaptive separable regularizers cannot be optimal in both settings simultaneously, which applies to many existing FTRL algorithms. Are you aware of any non-separable regularizers in the literature?\n2. Could you provide any intuition or motivation behind the loss construction (Section E.1) used in the lower bound proof?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the fundamental question of whether Follow-the-Regularized-Leader (FTRL) algorithms can achieve optimal regret bounds across all dimensional regimes using a single fixed regularizer across time, specifically for online convex optimization on $\\ell_p$-balls with $p > 2$.\nIt is known that optimal regret rates exhibit a dimensional shift: in low-dimensional settings ($d \\leq T$), the optimal rate is $O(L\\sqrt{T d^{1-2/p}})$ achieved using strongly-convex regularizers such as $||x||^2$, while in high-dimensional settings ($d > T$), the optimal rate is $O(LT^{1-1/p})$ achieved using uniformly-convex regularizers of degree $p$. This creates a challenge for \"anytime\" algorithms that must perform optimally without knowing the time horizon $T$ in advance as the optimal regularizer depends on the time horizon. \n\nThe paper's major contributions are in two directions:\n1) Algorithm: The authors design an adaptive FTRL algorithm that achieves anytime optimality by switching regularizers at a carefully chosen threshold $t_0 = 3^{-2p/(p-2)}d$. The algorithm uses uniformly-convex regularization initially (ideal for high dimensions) and switches to strongly-convex regularization when the low-dimensional regime is reached.\n\n2) Fundamental impossibility result: The paper's central contribution proves that this adaptivity is fundamentally necessary for separable regularizers. Specifically, they show that no fixed separable regularizer (of the form $\\psi(x) = \\sum_i=1^d g(x_i)$) can simultaneously achieve optimal regret in both dimensional regimes. This impossibility result rules out regularizers like $||x||_r^r$.\n\nAdditionally, there are also algorithm-specific lower bounds for particular regularizers and extensions of the analysis to bandit feedback, showing that sub-linear regret becomes impossible in sufficiently high dimensions for linear bandits on $\\ell_p$-balls.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- Correctness: The mathematical analysis is rigorous and sound. The proofs are carefully constructed and the main impossibility result (Theorem 4.6) is established through an argument connecting quadratic growth requirements to high-dimensional failure.\n\n- Studies an interesting question: The paper addresses a fundamental and practically relevant question about the universality of FTRL in online learning algorithms. Understanding when adaptive regularization is truly necessary versus when fixed regularizers suffice has implications for algorithm design, especially in settings where the dimensional regime is unknown a priori.\n\n- The algorithm is simple and intuitive, the lower bound is quite interesting too. The adaptive algorithm switches between two well-understood regularizers at a natural threshold, making it easy to implement and understand. The impossibility result provides insight by showing that the intuitive approach of using a single \"compromise\" regularizer fundamentally cannot work, which is both (to me) surprising and illuminating.\n\nWeaknesses \n\n- Heavy technical machinery for simple switch: While the core algorithmic idea is straightforward (switch regularizers based on time), the paper employs substantial technical apparatus to analyze this simple procedure. The specific threshold calculation and analysis feel disproportionate to what is essentially an elegant but basic adaptive strategy. The paper would also benefit from more intuition behind the proofs and sketches for some of the important building blocks that are relegated to the appendix (e.g. Lemma 4.7). \n\n- Bandit section is underdeveloped: Section 5 on bandit feedback feels somewhat disconnected from the main narrative and lacks the depth of analysis given to the full-information setting. The impossibility result for high-dimensional bandits is interesting but doesn't provide corresponding positive results or adaptive algorithms for the achievable regimes, leaving this part of the story incomplete.\n\n- Assumptions on separability: The main impossibility result is restricted to separable regularizers, which, while covering many important cases, leaves open the most general version of the question. This limitation feels somewhat artificial and prevents the paper from providing a definitive answer to the broader question of FTRL universality with fixed regularizers.\n\nQUESTIONS:\n- What is the main technical issue with extending the present proofs to non-separable regularizers? Is it the analog of Lemma 4.7? \n- Could you explain a bit more about the prior work in this domain? Have negative results of the flavor presented in this work appeared before in the literature? \n- Any insights on the algorithmic component to complement the impossibility results in Section 5?\n\nLIMITATIONS:\n- Yes, assumptions are clearly stated", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies online convex optimization over $l_p$ norm ball domains with $p > 2$. It investigates the role of regularization in Follow-the-Regularized-Leader (FTRL) and Online Mirror Descent algorithms and shows that the optimal separable regularizers for low- and high-dimensional settings are not equivalent, even up to time-varying scaling factors. The authors further prove that no time-invariant separable regularizer (up to time-varying scalars) can achieve an optimal anytime regret guarantee. However, they establish the existence of a time-varying separable regularizer that does provide such a guarantee. Finally, in the bandit feedback setting, they show that for action sets of $l_p$ norm balls with $p \\ge 1$ and linear losses, linear regret is unavoidable when the dimension is sufficiently large.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper tackles the universality of FTRL algorithms with fixed regularizers, addressing a fundamental and challenging problem. It provides a negative result for the subclass of separable regularizers.\n- The analysis and proof techniques are rigorous and carefully executed.\nWeaknesses:\n- While separable regularizers are indeed widely used, they are not a fundamentally necessary class. As a result, the negative result primarily highlights the need for more sophisticated fixed regularizers rather than a limitation of FTRL itself.\n- The negative result on linear bandits appears somewhat trivial, as it essentially demonstrates the dependence of regret on the dimensionality.\n\nQUESTIONS:\n- Theorem 5.1 claims that linear regret is unavoidable in the high-dimensional setting. However, this conclusion would also follow from any dimension-dependent lower bound of the form $\\Omega(d^a T^b)$. Does the lower bound instance presented in the paper yield any nontrivial results for $l_p$ norm balls that are stronger than those known for the multi-armed bandit setting? This is particularly relevant since, for $p \\in [1, 2]$, the regret bounds are already known to be the same.\n- Have the authors considered using fixed log-barrier regularizers, which are not part of the class of separable regularizers?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The authors established that adaptive regularization is essential for achieving anytime optimal regret for FTRL on high-dimensional $\ell_p$-balls ($p>2$). It demonstrates a shift in optimal regret rates between low-dimensional ($d \leq T$, rate $\mathcal{O}(\sqrt{Td^{1-2/p}})$) and high-dimensional regimes ($d > T$, rate $\mathcal{O}(T^{1-1/p})$), proving that FTRL with time-varying regularization can achieve optimality across all dimensions without prior knowledge of $T$ or the regime. Crucially, the authors show that no fixed separable regularizer (e.g., $\|x\|_r^r$) can be optimal in both regimes simultaneously, resolving an open problem and highlighting the necessity of adaptivity. They further establish that agents suffer unavoidable linear regret under linear bandits in sufficiently high dimensions, underscoring fundamental limitations beyond full-information settings. This paper significantly advances the understanding of geometry-aware online learning.

STRENGTHS AND WEAKNESSES:
- The regularizer in equation (2) is novel and might be of independent interest.
- The understanding of the necessity of adaptive regularization is clear, clean, and novel.
- This paper leaves a clear open problem on whether a non-separable static regularizer can achieve the same guarantees as the adaptive regularizer in the case of $\ell_p$-balls.

QUESTIONS:
- Whether the assumption that $0$ is an interior point of the action set $V$ is fundamentally necessary for this paper? Also, do the authors have some intuition about whether the $\ell_p$-ball structure can be generalized to certain more general convex sets to still have similar negative results for separable regularizers?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-03