Title: When Lower-Order Terms Dominate: Adaptive Expert Algorithms for Heavy-Tailed Losses
Venue: NeurIPS 2025
Accepted: yes
Keywords: prediction with expert advice

adaptive algorithms

regret minimization
Abstract: We consider the problem setting of prediction with expert advice with possibly heavy-tailed losses, i.e., the only assumption on the losses is an upper bound on their second moments, denoted by $\theta$. We develop adaptive algorithms that do not require any prior knowledge about the range or the second moment of the losses. Existing adaptive algorithms have what is typically considered a lower-order term in their regret guarantees. We show that this lower-order term, which is often the maximum of the losses, can actually dominate the regret bound in our setting. Specifically, we show that even with small constant $\theta$, this lower-order term can scale as $\sqrt{KT}$, where $K$ is the number of experts and $T$ is the time horizon. We propose adaptive algorithms with improved regret bounds that avoid the dependence on such a lower-order term and guarantee $\mathcal{O}(\sqrt{\theta T\log(K)})$ regret in the worst case, and $\mathcal{O}(\theta \log(KT)/\Delta_{\min})$ regret when the losses are sampled i.i.d. from some fixed distribution, where $\Delta_{\min}$ is the difference between the mean losses of the second best expert and the best expert. Additionally, when the loss function is the squared loss, our algorithm also guarantees improved regret bounds over prior results.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of prediction with expert advice under the assumption that losses are heavy-tailed but have bounded second moments. The authors develop adaptive algorithms that improve the dependence on the number of experts from the usual $\\sqrt{K}$ to $\\sqrt{\\log K}$ in the regret bound, under appropriate assumptions. They consider both the standard linear (Hedge) setting and a quadratic loss variant, providing adaptive algorithms based on Online Mirror Descent (OMD) and Follow-The-Regularized-Leader (FTRL).\n\nSTRENGTHS AND WEAKNESSES:\nThe expert setting with heavy-tailed losses is relevant in practical scenarios involving outliers, noise, or mechanisms like local differential privacy. The paper makes an insightful observation that terms traditionally treated as lower-order can, in fact, dominate the regret when only second-moment bounds are available.\n\nFrom a technical standpoint, the paper is solid. The proposed algorithms are interesting variants of OMD and FTRL, and the analysis appears to be non-trivial. The authors also support their claims with synthetic experiments.\n\nMy main concern lies in the lack of discussion around the motivation and interpretability of the assumptions. For instance, Assumption 2.3 is introduced without much explanation. It would be helpful to include an intuitive explanation of $\\Delta_{\\min}$ and $C$, and how these constants behave under different distributions. Can $C$ grow with $T$, or is it typically constant for distributions with bounded second moments? A discussion akin to Section 4, which provided insight into why certain terms dominate, could help clarify the meaning and implications of these parameters.\n\nSimilarly, Section 2.2 introduces a quadratic loss model, but it is not clear why this setting is interesting or whether it arises naturally in known applications. Is it connected to scalar regression, or is there prior literature studying this variant? A brief contextualization would help establish its relevance.\n\nOverall, the paper is technically strong and contributes important insights to the literature on adaptive online learning under heavy-tailed losses. However, additional exposition around the assumptions and modeling choices would strengthen the presentation.\n\nQUESTIONS:\n1. Do the authors have insight into potential lower bounds for these settings? For example, do known adversarial lower bounds apply here, given that bounded adversarial losses would satisfy Assumption 2.1? If so, can the optimality of established rates be discussed?\n\n2. Are the values on $Y$-axis of the plots regret?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the experts problem when the losses are \"heavy tailed\", i.e., when the second moment of the losses are bounded above by an (unknown) constant $ \\theta $. The authors show that existing algorithms have lower-order terms that can make the regret as large $ \\sqrt{K T} $. The authors develop algorithms whose regret is at most $ O (\\sqrt{ \\theta T \\log (K)}) $. They also show improved results under more restricted settings. Their algorithms are online mirror descent and follow-the-regularized-leader, using clipped versions of instantaneous regret as the loss and using empirical variance-based entropy regularization.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The authors study an important problem, namely, achieving sublinear regret in the experts problem under heavy-tailed losses, which can occur in many real-world scenarios.\n* The core novelty of their contributions is an adaptation of the multi-scale entropy regularization of Bubeck et al. (2019): the authors use time-varying (and expert-varying as in Bubeck et al.) scales ( $ \\eta_{t,i} $ ), set the scale to a function of the observed variance and second moment of the instantaneous regret, and perform clipping per expert.\n* Section 4 provides concrete examples showing lower-order terms can be large and the authors do so by using a simple construction.\n\nWeaknesses:\n* My main concern about the paper is its clarity. While the paper is well written overall, the writing is a bit sloppy in some places.\n    * Lines 47-56 and the sentences following Theorems 2.2 and 2.4 seem to have the algorithms switched. For instance, line 47 says that Algorithm 1 achieves $ O (\\sqrt{ \\theta T \\log (K)}) $ worst-case regret. However, Line 78 (following Theorem 2.2) says this bound is achieved by Algorithm 2.\n    * The abstract says that logarithmic regret is achieved under a separation assumption and i.i.d. losses. However, the main text and the appendix do not make the assumption of i.i.d. losses to achieve logarithmic regret.\n    * The authors refer to three algorithms in the main text for their regret bound results. However, they only present one of them in the main text. It is hard to follow their discussion that references the algorithm design when they don't present those algorithms in the main text.\n    * Related to the previous point, in Section 5.2 when the authors discuss Algorithm 2, Line 283 says that \"Note that, differently from Algorithm 1, we replace the multi-scale entropic regularizer $\\psi_t $ with its Bregman divergence.\" This is not clear from the main text because Algorithm 2 is only presented in the appendix. Question to the authors: Could you elaborate on this point? I looked at the algorithm in the appendix and did not quite understand what you mean.\n* Another weakness is novelty. The core novelty of the contribution, described above, is also a bit of a weakness: the authors combine existing ideas of clipping and multi-scale entropy regularization. (The authors do make the scale data dependent by using empirical second moments.)\n\nQUESTIONS:\n1. When you discuss Algorithm 2, Line 283 says that \"Note that, differently from Algorithm 1, we replace the multi-scale entropic regularizer $\\psi_t $ with its Bregman divergence.\" This is not clear from the main text because Algorithm 2 is only presented in the appendix. Could you elaborate on this point? I looked at the algorithm in the appendix and did not quite understand what you mean.\n\n2. Is it fair to say that choosing $ \\eta_{t,i} $ as a function of the empirical variance and second moment of the instantaneous regret is the key to obtaining your result and constitutes the core novelty of your contribution? I can see that the definition of $ \\eta_{t,i} $ helps algebraically in the equation after Line 266. However, can you provide some more intuition behind the definition of $ \\eta_{t,i} $?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studied online learning with expert advice with $K$ experts and $T$ days. The simplest form of the problem is for the loss function to be bounded by $[0,1]$ so that we could obtain $O(\\sqrt{KT})$ worst-case optimal regret. This paper considered the more general setting of general loss functions with “adaptive” losses, i.e., regret bounds in the form of the instance-dependent parameters. On this front, if we know $\\max_{t,i}\\ell^{t}(i)\\leq M$, it is possible to obtain $O(M\\sqrt{T \\log K})$ regret. This paper instead asked about what will happen if we instead know the second-order moment, i.e., $\\mathbb{E}[(\\ell^{t}(i))^2]\\leq \\theta$. Here, the algorithm does not necessarily know the value of $\\theta$ in advance.\n\nThe paper started by showing an example when $\\theta=O(1)$ but algorithm that adaptive to $\\max_{t,i}\\ell^{t}(i)$ could only result in $\\Omega(\\sqrt{KT})$ regret. Then, the paper designed an algorithm that achieves $O(\\sqrt{\\theta T \\log{K}})$ regret when $\\mathbb{E}[(\\ell^{t}(i))^2]$ is bounded by $\\theta$. Additionally. For instances that satisfy a “self-bounded environment” assumption, the paper designed an “instance-sensitive” type of algorithm that achieves $O(\\theta \\log(KT)/\\Delta_{\\text{min}})$ regret. Finally, for the square loss with the expert losses bounded by $Y$ and second moment bounded by $\\sigma$, the paper achieved $O(\\log{K}\\cdot (Y^2 + \\sigma))$ regret.\n\nThe main technique of the paper focuses on removing the $M_T = \\mathbb{E}[max_{t,i}|r_t|]= \\mathbb{E}[max_{t,i}|p_t^{T} \\ell^{t} - \\ell^{t}(i)|]$. The paper has shown that almost all existing algorithms would lead to this additive term when only the guarantees on the second-order moments are provided. To avoid this term, the main idea of the algorithm appears to be the adoption of the multi-scale entropic regularizer (BDHN [JMLR’19]) and the clipping of the loss when the variance of the algorithm is high. The paper then showed that the additional cost induced by cost clipping is tolerable (Equation (3)). Due to time constraints and my experience with the literature, I did not check the proof.\n\nSTRENGTHS AND WEAKNESSES:\nIn my opinion, the paper is generally well-written, and the main conceptual message is clearly articulated: in the bandit setting, one cannot adapt to the parameter $\\theta$, while in the full-information expert setting, we could achieve $O(\\sqrt{\\theta T \\log{K}})$ regret without knowing $\\theta$ a priori. The bounds in related work were also thoroughly discussed. \n\nThe technical part of the paper was written reasonably well. I’m not very familiar with the adaptive algorithm in the expert/bandit settings, but I could nevertheless follow the argument used in section 4. The techniques become harder to follow in section 5, partly due to the fact that I’m not very familiar with the existing techniques in the literature (for adaptive regrets). \nThe proof sketch gave a nice high-level framework, but the parameters are still quite involved. \n\nOn the flip side, I think the paper could have done a better job of motivating the setting from a *practical* perspective. From the paper, it is not very clear in what application scenarios we would have bounded second-order moments. I’m OK with this for a mostly theoretical paper, but for this conference, having better practical motivations will definitely be more helpful.\n\nThe above point is also related to the significance of the results. I personally believe the separation between the expert vs. bandit case is of good conceptual significance. However, since I’m not very familiar with the literature, I’ll see what my colleagues have to say about the significance.\n\nThe paper does not contain experiments. Again, I'm personally OK with this since this is a theoretical work. However, I do want to flag this for the AC in case that is part of their decision.\n\nOverall, I will be happy to see the paper get accepted, but I cannot give a firm verdict about the significance and the practical aspect. I also did not get a time to carefully check the proof, so I can only give a weak supportive opinion.\n\n====== Post Rebuttal ======\nThe author pointed out that their paper does contain experiments, which I missed during the initial review. I looked into that section and found that although the settings are limited, experimental results have demonstrated the better performance of their algorithms in the heavy-tail settings. \n\nI've increased my confidence score to reflect this discussion. However, since I'm not very familiar with the existing techniques, I still cannot give a higher score to \"champion\" for the paper.\n\nQUESTIONS:\nI do not have additional questions. Do the authors have responses to the weaknesses I flagged?\n\nLIMITATIONS:\nN/A, this is a theoretical work about online learning, and I do not see any immediate societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper provides algorithms for prediction with expert advice with heavy tailed losses (for the conditional distribution of each instance given the history). It correctly points out a critical flaw in applying standard scale-adaptive methods to the case of conditionally heavy tails (that the random variations form sampling will dominate the regret giving $\sqrt{NT}$ bounds instead of $\sqrt{\theta T\log N}$ when the scale^2 is $\theta$). The paper provides new algorithms which depend on second moments rather than expected maxima, and hence are better order. The algorithms are variants of hedge/exponential weights, instantiated as OMD or FTRL respectively, with different learning rates per expert (given by a diagonal weight matrix). The weights are updated in a way similar to other adaptive methods, but using clipped losses to "truncate" the heavy tails.

STRENGTHS AND WEAKNESSES:
This paper is well written, clearly identifies and resolves a relevant problem, and provides practical guidance.

QUESTIONS:
-

LIMITATIONS:
-
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-07