Title: No-Regret Learning Under Adversarial Resource Constraints: A Spending Plan Is All You Need!
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online Learning

Online Allocation

Resource Constraints

Budget Constraints

Spending Plan
Abstract: We study online decision making problems under resource constraints, where both reward and cost functions are drawn from distributions that may change adversarially over time. We focus on two canonical settings: $(i)$ online resource allocation where rewards and costs are observed before action selection, and $(ii)$ online learning with resource constraints where they are observed after action selection, under full feedback or bandit feedback. It is well known that achieving sublinear regret in these settings is impossible when the rewards and cost distributions may change arbitrarily over time. To address this challenge, we analyze a framework in which the learner is guided by a spending plan—a sequence prescribing expected resource usage across rounds. We design general (primal-)dual methods that achieve sublinear regret with respect to baselines that follow the spending plan. Crucially, the performance of our algorithms improves when the spending plan ensures a well-balanced distribution of the budget across rounds. We additionally provide a robust variant of our methods to handle worst-case scenarios where the spending plan is highly imbalanced. To conclude, we study the regret of our algorithms when competing against benchmarks that deviate from the prescribed spending plan.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work considers online resource allocation (ORA) and online learning with resource constraints (OLRC). To achieve sublinear regret in adversarial scenarios, the authors assume that a spending plan is provided to the learner at the outset. They then establish sublinear regret guarantees with respect to baselines that follow the spending plan in expectation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nUsing the spending plan to construct baselines is the main novel contribution of this work and may have wide-ranging applications.\n\nThe paper addresses both the ORA and OLRC settings. For OLRC, the analysis covers both the full-information and bandit feedback scenarios. The results appear to be solid and rigorously derived.\n\nWeaknesses:\n\nThe work does not provide lower bounds, so it remains unclear whether the obtained results are optimal.\n\nThe readability of the paper could be enhanced; for example, adding proof sketches and technical challenges in the main text would help improve clarity.\n\nAdditionally, numerical experiments could further increase the work’s appeal by demonstrating its practical performance.\n\nQUESTIONS:\nIn lines 70–81, the authors compare their approach with [13,33]. Specifically, this work assumes that the spending plan is given at the beginning, whereas [13,33] construct the spending plan during the learning process. Which setting is more challenging? Intuitively, constructing the spending plan on the fly might be more natural.\n\nWhy do the baselines only satisfy the spending plan in expectation? Is this an inherent limitation of the proposed method?\n\nAlthough Remark 4.3 and Theorem D.3 illustrate some results for the bandit case, a corresponding formal theorem is not presented in the main paper. Could the authors clarify or include a formal theorem for the bandit setting?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the online decision-making problem under adversarial resource constraints. In this problem, a decision maker has a fixed budget for multiple resources. Requests arrive sequentially, and each request generates a reward and consumes resources based on a given allocation. Before the decision-making process, a spending plan, which specifies the spending budget for each step (or request), is provided to the decision maker. The goal of the algorithm is to minimize the regret with respect to a benchmark that follows the spending plan.\n\nThe paper considers two settings based on the timing of decisions:\n\n- Online resource allocation, where the algorithm observes the reward and resource consumption functions before making decisions.\n\n- Online learning with resource constraints, where the algorithm makes decisions first and then observes the reward and consumption functions (under full feedback or bandit feedback).\n\nThe authors develop online algorithms for both settings. In the online resource allocation setting, the algorithm achieves a dynamic regret of $\\tilde{O}(\\sqrt{T}/\\rho_{\\min})$. In the online learning with resource constraints setting, the algorithm achieves a static regret of $\\tilde{O}(\\sqrt{T}/\\rho_{\\min})$, where $\\rho_{\\min}$ is the minimum spending budget across all steps and resources.\n\nTo address scenarios where $\\rho_{\\min}$ is small, the paper proposes modifications by redefining $\\rho_{\\min}$ and the spending plan. These modifications yield regret bounds independent of $\\rho_{\\min}$, which is desirable when $\\rho_{\\min}$ is small. Additionally, the paper demonstrates robustness to sub-linear errors in the spending plan.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n- The paper addresses an important online decision-making problem with broad applications. The notion of a spending plan is a practical and reasonable addition, helping to overcome worst-case impossibility results.\n\n- The systematic algorithm design and analysis for both settings, online resource allocation and online learning with constraints, are nice.\n\n- The analysis of robustness to errors in the spending plan is a valuable and practical addition to the main results.\n\n\nWeakness\n\n\n- The algorithmic ideas are not entirely new. The online resource allocation algorithm closely resembles that of Santiago et al. (ICML 2020), with the main change being the use of a predefined spending plan instead of a uniform budget $B/T$. The online learning with constraints approach shares similarities with the bandits with knapsacks literature. The paper lacks a thorough comparison between its proposed algorithm and those existing approaches.\n\n- There are no lower bounds provided, making it unclear how tight the regret bounds are.\n\nQUESTIONS:\n- In the adversarial setting, a worst-case instance could involve that all resources are consumed in a single step, with no resource consumption in other steps. This corresponds to a spending plan where one step receives the entire budget and all others receive zero. Can the extended version of the algorithm handle this case? What happens if there are errors in the spending plan in such scenarios?\n\n- The regret bounds depend on $\\rho_{\\min}$, which is defined as the minimum budget across all steps and resources. Would it be possible to redefine $\\rho_{\\min}$ to exclude zero elements, using the minimum among all non-zero elements instead?\n\n- In practice, the full spending plan may not be available at the beginning of the decision process. Can the algorithm handle an online revelation of the spending plan? Moreover, the horizon $T$ is assumed to be known in advance, which is not always realistic. Can the algorithm be adapted for an unknown time horizon?\n\n- In the theorems in Section 5, the parameter $\\rho$ appears in many expressions, but I believe it should be $\\rho_{\\min}$. Could you clarify this?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a framework for online decision-making with resource constraints, focusing on adversarial environments where standard regret benchmarks are unachievable. The key contribution is the introduction of a spending plan benchmark: a user-specified resource consumption profile that serves as a feasible benchmark for regret analysis. The authors design primal-dual algorithms, applicable to both full and bandit feedback settings, that achieve sublinear regret relative to this spending plan. The analysis includes a meta-algorithm to handle small minimum budget values and provides robustness guarantees for small deviations from the spending plan. The work aims to unify online resource allocation and online learning with resource constraints under a common regret framework.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1) The paper introduces a general framework for online decision-making under resource constraints that applies to both adversarial and stochastic settings, including bandit and full-information feedback. \n\n2) The spending plan concept provides a structured and feasible baseline in adversarial environments, allowing the authors to derive regret guarantees that are meaningful when competing with a pre-specified resource usage profile.\n\n3) The algorithmic design is modular, leveraging black-box regret minimizers in the primal updates and providing adaptability across different feedback structures.\n\nWeaknesses:\n\n1) The paper’s novelty lies primarily in the design of the spending plan benchmark and its associated regret analysis, rather than in new algorithmic techniques.\n\n2) The choice of spending plan is external and assumed given, yet the regret guarantee depends heavily on this plan’s quality. The paper does not discuss how to design or learn a good spending plan or analyze sensitivity to suboptimal plans.\n\n3) The regret notion in the ORA setting does not reflect classical learning. Since the algorithm operates with full feedback before action selection, there is no learning of unknown reward or cost functions. The regret measure essentially tracks the loss of a dynamic optimization heuristic relative to the spending plan, rather than regret in the traditional online learning sense.\n\nQUESTIONS:\n1) In the ORA setting, since the reward and cost functions are fully known before action selection, the \"regret\" essentially measures the loss of the proposed dynamic optimization heuristic relative to a feasible baseline spending plan. There is no learning of unknown quantities involved. Could the authors clarify this point and distinguish their notion of regret from standard regret in learning settings?\n\n2) The authors claim in Remark 2.2 that sublinear regret is impossible in all settings without a spending plan. However, in the stochastic setting, standard benchmarks are achievable with sublinear regret. The authors may wish to clarify this distinction.\n\nLIMITATIONS:\nThe paper lacks a discussion on limitations and potential negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper studies the problems of online resource allocation (where rewards and costs are observed before action selection) and the online learning with resource constraints (where rewards and costs they are observed after action selection, under full feedback or bandit feedback). While it is known that a sublinear regret isn't achievable by any spending plan in these setting, the current paper considers the easier, but still realistic, setting where a budget plan is given to the algorithm. The paper provides algorithms with sublinear regret bounds in this case.

STRENGTHS AND WEAKNESSES:
Strengths:
- The paper tackles a practically relevant problem in ML.
- The algorithms presented are intuitive and simple. 
- The paper is very well-written and easy to follow. 

Weaknesses: 
- Having access to a good budget plan is not always possible, and so the methods proposed may have a narrow scope. 
- The paper does not provide any experimental validation. 
- There are a couple of places where the paper would have benefited from more detailed explanations (see my comments below).

Comments: 
- After Theorem 4.2, it would have been helpful to elaborate a little more on how the bandit setting can be reduced to the full-information setting. 
- Same comment for Section 5.1 (lines 352-358); more explanation on how to deal with small rho would have been very helpful. 
- $\rho$ in Algorithm 3 (and in Section 5.1 in general) is not defined/scoped.

QUESTIONS:
Please elaborate on how to reduce the bandit setting to the full information setting.

LIMITATIONS:
The authors did not explicitly discuss the limitations of their work in the main body.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-07