Title: On the Universal Near Optimality of Hedge in Combinatorial Settings
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online learning

Hedge algorithm

Regret minimization

Combinatorial Games

Directed Acyclic Graphs

Dilated entropy

Online Mirror Descent
Abstract: In this paper, we study the classical Hedge algorithm in combinatorial settings. In each round, the learner selects a vector $\mathbf{x}_t$ from a set $\mathcal{X} \subseteq$ {$0,1$}$^d$, observes a full loss vector $\mathbf{y}_t \in \mathbb{R}^d$, and incurs a loss $\langle \mathbf{x}_t, \mathbf{y}_t \rangle \in [-1,1]$. This setting captures several important problems, including extensive-form games, resource allocation, $m$-sets, online multitask learning, and shortest-path problems on directed acyclic graphs (DAGs). It is well known that Hedge achieves a regret of $\mathcal{O}\big(\sqrt{T \log |\mathcal{X}|}\big)$ after $T$ rounds of interaction. In this paper, we ask whether Hedge is optimal across all combinatorial settings. To that end, we show that for any $\mathcal{X} \subseteq$ {$0,1$}$^d$, Hedge is near-optimal—specifically, up to a $\sqrt{\log d}$ factor—by establishing a lower bound of $\Omega\big(\sqrt{T \log(|\mathcal{X}|)/\log d}\big)$ that holds for any algorithm. We then identify a natural class of combinatorial sets—namely, $m$-sets with $\log d \leq m \leq \sqrt{d}$—for which this lower bound is tight, and for which Hedge is provably suboptimal by a factor of exactly $\sqrt{\log d}$. At the same time, we show that Hedge is optimal for online multitask learning, a generalization of the classical $K$-experts problem. Finally, we leverage the near-optimality of Hedge to establish the existence of a near-optimal regularizer for online shortest-path problems in DAGs—a setting that subsumes a broad range of combinatorial domains. Specifically, we show that the classical Online Mirror Descent (OMD) algorithm, when instantiated with the dilated entropy regularizer, is iterate-equivalent to Hedge, and therefore inherits its near-optimal regret guarantees for DAGs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the regret performance of HEDGE in combinatorial settings. They ask whether it’s universally optimal and conclude \"nearly\" by providing an algorithm-independent regret lower bound that match the well-known upper bound up to $\\sqrt{\\log d}$ factors.\n\nThey then provide some clarity on the nature of HEDGE's (sub-)optimality by studying two different combinatorial task instances where HEDGE is optimal and where it isn’t. \n\nFor the optimal case, they provide tighter algorithm-independent lower bounds for the online multiask learning setting, which match HEDGE's upper bound.\n\nFor the sub-optimal case, they show their original lower bound is tight for the $m$-sets task, and is enjoyed by a modified OMD algorithm. They also show that HEDGE incurs a strictly worse regret compared to the modified OMD by a $\\sqrt{\\log d}$ factor.\n\nFinally, they show how regularized OMD is iterate-equivalent to HEDGE, and therefore enjoys near-optimal regret for shortest-path problems on DAGs.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths:*\n- Compelling theoretical contributions to help clarify the nature of HEDGE’s (sub-)optimality on an important class of online learning problems\n- Organisation is very clear\n- Focus of the paper builds well, natural questions are addressed\n- FIgure nicely summarizes contributions\n\n*Weaknesses:*\n- While a gap of $\\sqrt{\\log d}$ is non-trivial, it may represent a very modest gap for many of the cited applications. The authors should add a short section discussing the significance of this gap in context of their cited motivations.\n- Although the main theorems are well stated and clear, the paper lacks a discussion on the essence of what makes $m$-sets a particularly challenging task in comparison to online multitask learning for HEDGE. The authors should add a short section providing some intuition for why one might expect HEDGE to be optimal in one but not in the other.\n- The paper mentions an adaptive adversary in the preliminaries (lines 139-140), but in the provided theorems the adversary is oblivious. The authors should address this inconsistency.\n\nQUESTIONS:\n1. In the introduction, the adversary is described as potentially reactive, in that it can choose loss vectors as a function of the history $\\mathcal{F}_t.$ However, in the main theorems, the adversary is described as oblivious, fixing the losses at the start of the trial. I may have misunderstood something, but can the authors please comment on whether they are focusing on an adaptive or oblivious adversary?\n\n2. Intuitively, why is the structure of the $m$-sets task poorly aligned with HEDGE while online multitask learning is well aligned? Currently the paper requires the reader to engage with the full proof to appreciate the differences. A discussion that helps the reader more readily appreciate the essential differences would be a valuable contribution to this work.\n\nLIMITATIONS:\nThe authors should add some additional commentary addressing the weaknesses I mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the classical Hedge algorithm for online learning in combinatorial settings. The main contribution is a series of upper and lower bounds on the regret that establish near-optimality of Hedge. In particular, the paper identifies subproblems of the combinatorial setting where Hedge is provable optimal respectively suboptimal by a factor $\\log(d)$.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper makes an important contribution to the online learning literature. Hedge is a classical algorithms but there are still open questions around its optimality in various settings. This paper focuses on the combinatorial setting, where the decision set is a subset of $\\\\{0,1\\\\}^d$. The authors first establish that any algorithm suffers a worst-case regret of $\\Omega(\\sqrt{T \\log(|X|)/ \\log d})$ ,where X is the decision set. The known upper bound of Hedge is $O(\\sqrt{T \\log(|X|)}$, hence the question is if the gap is loose or if one of the bounds can be improved. The paper subsequently establishes further lower and upper bounds for the Hedge algorithm, showing cases where Hedge is provably suboptimal, and settings where the lower bound can be improved.\n\nThe paper is very well written and easy to follow. Related work is discussed appropriately.\n\nSome minor suggestions for improvement are as follows:\n* consider include definitions of important quantities, such es worst-case and minimax regret, and iterate-equivalence. \n* In line 194, I assume you invoke an existing regret lower bound over the set I. Consider citing or stating the result again in this context.\n* Line 198: What is the expectation over here? How does the expected regret related to the original objective defined in line 142?\n\nQUESTIONS:\nAre you aware of setting where Hedge achieves the minimax lower bound $\\sqrt{ T \\log|X|/\\log(d)}$?\n\nLIMITATIONS:\nWhile the paper addresses the regret bounds for the Hedge algorithm (and minimax bounds), one would hope to have a general characterisation of the regret for any given X, e.g. by introducing a complexity measure that depends on the set X (either combinatorial or even in the general case). Another open question is to design an algorithm that is near-optimal for a larger family or all combinatorial sets.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of prediction with expert advice in a combinatorial setting (i.e., full-information setting). While most of previous works focus on characterizing regret's upper bounds of various algorithms, this paper focus on constructing hard examples for establishing regret's lower bounds and from there, analyze the optimality (or sub-optimality) of the famous HEDGE algorithm. In particular, the authors show that Hedge is near-optimality (up to a logarithm term of the dimension) in any setting, is strictly suboptimal in a special class, and optimal in another class of problem.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The paper is quite clearly written and sufficiently easy to follow. \n- As far as I know, the contribution is clear and original.\n- Significance: While the key question is quite simple, it is interesting to establish the lower bound guarantees of HEDGE as it is a well-used algorithm.\n\nWeaknesses:\n- All proofs are put in Appendices so it is a bit hard to have an immediate intuition where does the result come from. \n- The notation  Phi $\\varphi$ is used and reused repeatedly throughout various sections to refer to different choices/variants of the regularizer. It would be better if the authors distinguish these notations; for example, denote Phi_{m-set} in (4.1) and  Phi_{entropy} in 6.1.\n- A minor point: all results contain implicit constants that might involve a leading term (for example, dimension d, see questions for more details).\nThe paper limits to the full-information setting. It would be interesting to make connection to previously known results of regret's lower bounds in other setting (for example, bandit). \n\nThe implicit constants make it hard to check the proof. \n\nLacks some numerical experiments or concrete example to confirm the results. For instance, construct a toy example of a hard-case and show the numerical regret of several algorithms would be interesting.\n\nQUESTIONS:\n- Line 776 (Appendix A) assume that $T$ divides $|\\mathcal{I}|$. If this is not the case, how does Inequality in Line 787 works? In particulatr, if we partition the T rounds into $|\\mathcal{I}| - 1$ equal-length segments and 1 last segment that is smaller than that. Does this mean that there is a set {t : it = i} whose size is strictly smaller than $T$ / |\\mathcal{I}|$. As a consequence Inequality in Line 787 is a \" strictly less than\" rather than a \"less or equal\" ? \n\n- Is Lemma F.3 a restatement of Theorem 7 of (Orabona and Pál [35])? If that's the case, can you explain where does the condition c1 >= 8 comes from (or which term is it equivalent to in Orabona andPal [35])? \nMoreover, while this is a bit picky, if it is possible, I think you should follow the  template of Orabona and Pal  to obtain the explicit value of the involved constant. The reason is because such constant in Orabona and Pal contains the term log(d) in it; therefore, it is better to check and make sure that we do not misjudge the order of log(d) when writing big-oh(sqrt(Tlog(d)).\nThis is quite important as this constant in Lemma F.3 leads to constant in Lemma F.4 and hence, directly involves in Theorem 5.1.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies online learning over combinatorial decision sets $ \\\\mathcal{X} \\subset \\\\{0,1\\\\}^d $, \nwhere at each round the learner selects an action $x_t \\in \\mathcal{X}$, observes the loss vector $y_t$, and incurs loss$x_t^\\bot y_t$. The goal is to minimize cumulative regret with respect to the best fixed action in hindsight. The authors revisit the performance of the classical Hedge algorithm in the combinatorial setting. They show a general lower bound proving that Hedge is at most order $\\sqrt{\\log d}$ worst-case sub-optimal compared to its best known upper bounds.  The authors  demonstrate that this lower bound is tight for certain combinatorial structures; specifically, for m-sets where this bound is attained by an instance of mirror descent;  while Hedge  is provably suboptimal,  as proven by a lower bound on its regret for this combinatorial structure.  Finally, the authors show that Hedge is minimax optimal in other combinatorial structures such as online multitask learning and shortest-path problems in DAG.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**: The problem studied is particularly relevant in applications, given the widespread use and simplicity of Hedge in practice. \nThe authors establish a worst-case lower bound for general combinatorial structures, showing that the Hedge algorithm is minimax optimal up to a$\\sqrt{⁡\\log d}$ factor. They demonstrate the tightness of this bound for m-sets, where the optimal regret is achieved by an instance of OMD, while also showing that it is provably not attainable by Hedge for some loss sequences. The authors show matching upper and lower bounds for Hedge on DAGs and in combinatorial structure for multitask learning. These results offers valuable insight into the complexity combinatorial online learning. \n\n**Weaknesses**:\n- the lower bounds presented are worst-case in nature, which makes it difficult to determine whether the (sub)optimality arises inherently from the combinatorial structure itself or from carefully constructed adversarial instances (that could belong to other strucutures as well). While this is understandable given the adversarial setting, it limits the interpretability of the results across broader classes of structures. \n- the results are somewhat ad hoc and tailored to each specific combinatorial domain, making it nontrivial to identify a unifying complexity measure that governs the complexity of combinatorial learning\n\n**Minor comments**: \n- typo in l.232\n- $co(X)$ is used l.145 before being introduced l.182\n\nQUESTIONS:\nDoes Theorem 4.6 still hold if Hedge is run with a decreasing learning rate? In particular, could an adaptive rate improve the result, or does the lower bound remain unaffected?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIt is well known that the Hedge algorithm guarantees an external regret of O(sqrt{T log d}) in the learning with experts setting, and that this bound is tight up to constant factors. It is possible to apply Hedge to more general online learning settings -- for example, for online linear optimization when the action set is the polytope, one can run an instance of Hedge where each expert corresponds to a vertex of the polytope, getting an O(sqrt{T log V}) bound. But is this regret bound tight?\n\nThis paper studies this question for the case of *combinatorial action sets*, where the action set P of the learner is the convex hull of a subset of points in {0, 1}^d (named as it captures many combinatorial sets of interest). The authors prove the following set of results:\n\n1. First, they show that the regret bound of Hedge is within a sqrt(log d) factor of optimal -- in particular, that for any combinatorial polytope, there is a regret lower bound of Omega(sqrt{T log V / log d}). The proof of this result is a consequence of the Sauer-Shelah theorem (which bounds the number of distinct 0/1 points needed to contain a subset that projects to a hypercube).\n2. Second, they demonstrate cases where this separation is tight; specifically, they show that this separation is tight for m-sets (sets of points with Hamming weight m), by constructing a new optimal regularizer for these points (a convex combination of the quadratic and negative entropy regularizer).\n3. Thirdly, they show that Hedge is minimax optimal for all “Online Multitask Learning” problems (essentially, where the action set of the learner is the product of several simplices).\n4. Finally, they also show that Hedge is minimax optimal when P is the flow polytope in a DAG. They also show that Hedge can be interpreted as running FTRL with the dilated entropy regularizer.\n\nSTRENGTHS AND WEAKNESSES:\nI think this is a very cool set of results! Surprisingly little is known about what the optimal minimax regret algorithms are for instance of online linear optimization -- while it is well understood in very classical settings (e.g. the learner plays actions in the l_1 or l_2 norm ball and the adversary plays actions in the polar), as soon as the learner / adversary action set becomes a little bit more complex it is not clear what the optimal regret bounds or algorithms are for. This is despite this being a very fundamental primitive in many applications (e.g. game theoretic applications, online learning in combinatorial domains). This paper provides a fairly thorough answer to this question for a very important class of action sets (aside from cases like the l_2 ball, it is hard to think of natural settings in the literature this does not capture). \n\nOne possible criticism of this paper is that the proofs are fairly simple (they are generally fairly immediate consequences of the correct observations). But I think despite this, these observations were certainly not well known to other practioners in the field (I have spent some time thinking about these questions, and none of these facts were obvious to me), and so I think there is a sizable subset of the NeurIPS audience that would appreciate these observations. \n\nQuestion: Section 6 only addresses the unit flow polytope formed by the convex hull of all single paths. Do you know if this result continues to hold for general flow polytopes (either allowing for any positive flow or a specific positive flow > 1?)? My guess is that the answer is no (and that the m-set example can be embedded in the higher flow polytope of some DAG -- e.g., maybe one where the max flow is m but there are d > m edges of capacity 1 connecting two vertices), but I haven’t thought carefully about this.\n\nQUESTIONS:\nSee above -- in general, feel free to reply to any aspect of the above review.\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-07"}