Title: Data-Dependent Regret Bounds for Constrained MABs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Data-Dependent Bounds

MAB

Hard Constraints
Abstract: This paper initiates the study of data-dependent regret bounds in constrained MAB settings. These are bounds that depend on the sequence of losses that characterize the problem instance.  Thus, in principle they can be much smaller than classical $\widetilde{\mathcal{O}}(\sqrt{T})$ regret bounds, while being equivalent to them in the worst case. Despite this, data-dependent regret bounds have been completely overlooked in constrained MABs. The goal of this paper is to answer the question: Can data-dependent regret bounds be derived in the presence of constraints? We provide an affirmative answer in constrained MABs with adversarial losses and stochastic constraints. Specifically, our main focus is on the most challenging and natural settings with hard constraints, where the learner must ensure that the constraints are always satisfied with high probability. We design an algorithm with a regret bound consisting of two data-dependent terms. The first one captures the difficulty of satisfying the constraints, while the second one encodes the complexity of learning independently of their presence. We also prove a lower bound showing that these two terms are not artifacts of our specific approach and analysis, but rather the fundamental components that inherently characterize the problem complexity. Finally, in designing our algorithm, we also derive some novel results in the related (and easier) soft constraints settings, which may be of independent interest.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the multi-armed-bandit (MAB) problem where the learner chooses an arm among K fixed choices at each round t from 1 to T. At each t the learner receives a reward associated to the chosen arm and strives to maximise the cumulated sum of rewards. The paper considers adversarial rewards, and so good strategies are randomized and actually need to choose a vector of probabilities for each arm, at each round.\n\nWhile this problem is very classical, the specificity of the paper is consider constraints. At each round there are realizations (iid over rounds) of constraints, and the probabilities chosen by the learner must satisfy the constraints, either by having a sublinear cumulated sum of violations, or by having no violation with high probability. \n\nThe paper studies the regret, defined as the difference between an optimal strategy in hindsight that satisfies the constraints, and the cumulated sum of rewards obtained by the learner. The goal is to provide bounds that are called \"data-dependent\", that is that depend on the actual adversarial sequence of rewards. These bounds are more informative than worst-case bounds. \n\nThe paper provides an algorithm called SOLB (safe OMD with log barrier) and prove an upper bound for its regret that is data-dependent and is interpreted as the sum of a safety-complexity term and a bandit-complexity term. The paper provides a lower bound as a counterpart, that also has these two terms.\n\nSTRENGTHS AND WEAKNESSES:\nSTRENGTHS The paper is well-written. Although the content is technical, the authors have made an effort to provide intuition, along the exposition. The topic is important for machine learning theory. The contribution is strong, with a main informative data-dependent regret bound, that is globally matched by a lower bound. The upper bound required to design a new algorithm. The proofs appear to have required a lot of work and appear to be theoretically interesting. \n\n\nWEAKNESSES I would prefer not to point out a specific weakness, as I do not have notable weaknesses in mind for the paper.\n\nQUESTIONS:\nRemark 2 uses a lower bound from 2016 for unconstrained MABs. I was surprised that the remark writes that, thus, an upper bound for constrained MABs is tight. Would there be a restricted class of constrained MABs that are \"far from unconstrained ones\", and for which it would be an open question to provide a lower bound?\n\nCould a discussion of concrete motivations for studying constrained MABs be added to the paper. Many readers can have practical motivations in mind for unconstrained MABs (say, for online advertisment) but I think many readers would benefit from examples of concrete constraints. \n\n\nLine 332, is it the fifth term rather than the fourth?\n\nLIMITATIONS:\nYES", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nConstrained bandits represent a compelling and practically relevant problem, motivated by numerous real-world applications. While the existing rich literature primarily focuses on horizon-dependent guarantees, this paper initiates the study of **data-dependent guarantees** in the setting of **adversarial losses** and **stochastic constraints**. The authors make the following key contributions:\n1) **COLB** : An algorithm designed for the soft constraint setting, where the learner is allowed to violate constraints. The authors provide a classical upper bound on constraint violations and a data-dependent upper bound on the regret.\n2) **SOLB** : An algorithm tailored for the hard constraint setting, where the learner must satisfy the constraints at each round with high probability. The authors prove that the algorithm does not violate the constraints and provide a data-dependent upper bound on the regret.\n3) **Lower Bound**: The paper also presents a lower bound that supports the tightness of the proposed upper bounds, reinforcing the theoretical soundness of the results.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1) The paper is well written and easy to follow. \n2) The problem addressed is interesting and opens a promising direction for research. \n3) The mathematical analysis appears to be sound.\n\n**Weaknesses:**\n1) While the question of data-dependent guarantees for constrained MABs is novel, both data-dependent regret bounds for unconstrained MABs and algorithmic design for constrained MABs have been extensively studied. This paper appears to combine several previously proposed techniques and results. In particular, the OMD with log-barrier algorithm introduced by [lee et al. 2020] is heavily utilized in the proposed algorithm, especially in the analysis techniques, which are central to the proofs of the main theorems. Additionally, the structure of the feasible sets and the combination factor idea bear strong resemblance to those presented in [Pacchiano et al. 2021] and [Amani et al. 2019] .\n2) I have an issue with the paragraph titled **Safe Decision Space** (Lines 152--160). I am unable to see how, by invoking Lemma 1, one can conclude that selecting a strategy from $S_t$ ensures satisfaction of the constraints with high probability. Specifically, the definition of $S_t$ involves a subtraction, and I believe the claim would only hold if the definition used a '+' instead of a '-' sign, or alternatively, if the constraints were of the form $g_i^\\top x \\geq \\alpha_i$. As it stands, the justification for the safety guarantee is unclear.\n3) The quantity $D_{\\psi_{t}}$ is used extensively throughout the algorithms. However, unless I missed it, it does not appear to be explicitly defined in the main text.\n4) It would strengthen the paper to include numerical simulations that empirically validate the theoretical claims. Currently, the paper lacks such experiments, which are essential for demonstrating the practical relevance and performance of the proposed algorithms.\n\nQUESTIONS:\n**Questions and remarks:**\n1) Please address the weaknesses mentioned above.\n2) In the proof of Lemma 2, particularly the first inequality following line 546, it is unclear how the bound holds uniformly for all $t$ without any additional assumption on the norm of $\\hat{g}_{t,i} - \\beta_t$. Could you please elaborate on this step?\n3) In the proof of Lemma 5, you use the inequality $\\kappa^{n_a} \\leq 5$. Could you please explain how this bound is derived?\n4) In the proof of Theorem 2, could you please clarify the inequality that appears after Line 586? It seems to me that $\\hat{l}_t$ is sparser than $l_t - \\hat{l}_t$.\n5) I suggest modifying the definition of $V_T$ by placing the '$\\max$' operator inside the summation. This adjustment would ensure that there is no compensation between rounds across different arms. Moreover, the theoretical proofs provided in the paper would remain exactly the same under this modification.\n6) Can you please share your opinion on the case where the losses (or rewards) are stochastic while the constraints are adversarial?\n\n**Minor typos:**\n1) In Line 13 of the SOLB algorithm, should it be $\\tilde{x_{t+1}}$ instead of $x_{t+1}$?\n2) In the proof of Lemma 2, the second equality following line 548 appears to reference the set $\\tilde{S}$. However, I believe it should be $S^{\\circ}$ instead, based on the context and the structure of the argument.\n\n\nI am fully open to revising my evaluation and raising my score if the issues I raised are adequately addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies data-dependent regret bounds in constrained Multi-Armed Bandit (MAB) seetings. The authors focus on the scenario of adversarial losses and stochastic hard constraints (due to the impossibility result for fully adversarial scenarios). They propose two algorithms, which are Constrained OMD with Log-Barrier (COLB) for soft constraints and the main algorithm Safe OMD with Log-Barrier (SOLB) for hard constraints. SOLB satisfies the constraints with high probability by dynamically combining and OMD-suggested strategy with a strictly feasible one. They show that the high probability regret bound has two data-dependent terms: a safety complexity term that quantifies the difficulty introduced by satisfying the constraints; and a bandit complexity term that represents the learning complexity independent of the constraints. The authors also provide a lower bound to demonstrate that these two terms are in fact fundamental and inherent characteristics of the problem and the proposed algorithm SOLB is optimal up to logarithmic terms.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1) The algorithms seem novel. COLB for soft constraints serves as a warm-up by achieving small-loss regret bounds and the primary contribution SOLB provides optimal regret for hard constraints.\n\n2) The theoretical insights on the data-dependent regret bound, especially the natural interpretation of the two independent terms safety and bandit complexity is very nice.\n\n3) The regret bound is optimal (up to logarithmic factors).\n\n## Weaknesses\n\n1) SOLB requires Slater's condition and explicit knowledge of a strictly feasible strategy and its associated costs as input, which seems limiting, especially in practical applications.\n\n2) The initial setting of the learning rate requires knowledge of the optimal cumulative loss.\n\n3) While not a major weakness, there are no empirical validation.\n\nQUESTIONS:\n1) Can you possibly relax (or maybe adaptively learn) the strictly feasible strategy and its associated costs? It would be nice to discuss this even as a future research direction.\n\n2) Can you give a more in-depth explanation for your conjecture that the $1/\\rho$ dependence should affect only the Safety Complexity term (A), and not the Bandit Complexity term (B)? What specific challenges exist in your derivations in achieving a tighter bound?\n\n3) The learning rate for COLB and SOLB requires knowledge of the optimal cumulative loss. While \"doubling trick\" is mentioned, how would this trick specifically be applied?\n\n4) What are the hypothetical practical applications where your algorithm and results would be most critical and beneficial?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors consider a constrained MAB formulation, where the goal is to provide data-dependent regret bounds. Here, losses are adversarial, but constraints are modelled as being stochastic. The authors propose an algorithm that adapts OMD in conjunction with 'safe' decision spaces.\n\nSTRENGTHS AND WEAKNESSES:\nOne natural concern is whether it is reasonable to assume adversarial losses in conjunction with stochastic constraints. While the authors do point to an impossibility result to justify the assumption, it is not clear to me whether the proposed setting captures any application reasonably.\n\nMoreover, the algorithms are assumed to have bounds on the loss of optimal action in hindsight, plus a strictly feasible strategy (Assumption 2). Again, it is perhaps true that the problem is intractable in the absence of such knowledge, but I wonder is what that  really means is that the problem is ill-posed.\n\nAside from the above reservations on the problem formulation itself, the paper seems to make useful algorithmic contributions. The decomposition of the regret into safety complexity and bandit complexity is also quite interesting.\n\nQUESTIONS:\n1. Can the authors provide an intuitive explanation of why lower confidence bounds (LCBs), rather an UCBs, of the arm constraints are used to define safe decision spaces? I am referring to the display equation after Line 155 on Page 4. Note that with high probability, the actual $g_i$ are higher than their LCB. So the following sentence that with high probability, the set $\\mathcal{S}_t$ contains actions that satisfy the constraints also feels suspicious.\n\n2. Line 11 of Algorithm 1: The function $D_{\\psi_t}(x,x_t)$ is not defined anywhere as far as I can tell. The meaning of this does not follow from the definition of $\\psi_t(x)$ that follows. A similar comment applies to Algorithm 2.\n\n3. Note that regret calculations here include expectations with respect to algorithm randomisation. However, it feels clumsy to provide a \\emph{high-probability} bound on the \\emph{average} regret. Now, I understand the \"high-probability\" bit is due to the stochastic constraint modelling, and the \"average\" bit is over the algorithm randomisation. A brief commentary on this would be useful, IMO.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-21"}