Title: Beyond $\tilde{O}(\sqrt{T})$ Constraint Violation for Online Convex Optimization with Adversarial Constraints
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online convex optimization

Regret bounds

Learning with constraints
Abstract: We study Online Convex Optimization with adversarial constraints (COCO). At each round a learner selects an action from a convex decision set and then an adversary reveals a convex cost and a convex constraint function. The goal of the learner is to select a sequence of actions to minimize both regret and the cumulative constraint violation (CCV) over a horizon of length $T$. The best-known policy for this problem achieves $O(\sqrt{T})$ regret and $\tilde{O}(\sqrt{T})$ CCV.  In this paper, we improve this by trading off regret to achieve substantially smaller CCV. This trade-off is especially important in safety-critical applications, where satisfying the safety constraints is non-negotiable. Specifically, for any bounded convex cost and constraint functions, we propose an online policy that achieves $\tilde{O}(\sqrt{dT}+ T^\beta)$ regret and $\tilde{O}(dT^{1-\beta})$ CCV, where $d$ is the dimension of the decision set and $\beta \in [0,1]$ is a tunable parameter. We begin with a special case, called the $\textsf{Constrained Expert}$ problem, where the decision set is a probability simplex and the cost and constraint functions are linear. Leveraging a new adaptive small-loss regret bound, we propose a computationally efficient policy for the $\textsf{Constrained Expert}$ problem, that attains $O(\sqrt{T\ln N}+T^{\beta})$ regret and $\tilde{O}(T^{1-\beta} \ln N)$ CCV for $N$ number of experts. The original problem is then reduced to the $\textsf{Constrained Expert}$ problem via a covering argument. Finally, with an additional $M$-smoothness assumption, we propose a computationally efficient first-order policy attaining $O(\sqrt{MT}+T^{\beta})$ regret and $\tilde{O}(MT^{1-\beta})$ CCV.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work investigates the COCO setting, that is, online convex optimization with constraints. In such a setting, the state-of-the-arts result is an algorithm that attains $\\widetilde{O}(\\sqrt{T})$ regret and violation without Slater's condition. This works improves the result by showing that $\\widetilde{O}(\\sqrt{T}+ T^{\\beta})$ regret and $\\widetilde{O}(T^{1-\\beta})$ violation is attainable. The problem is solved by focusing on the expert (MAB with full feedback) setting, where the employment of small-loss regret minimiser, Lyapunov functions and surrogate functions allows to attain the desired result. Thus, a $\\delta$-covering allows to reproduce the same results in the COCO setting (paying a cost in terms of computational complexity). Finally, the authors show that OGD with adaptive learning rate (and surrogate functions) attains similar results when the objectives (cost and constraints) functions are smooth.\n\nSTRENGTHS AND WEAKNESSES:\nThe contribution and the results are clear. The authors did a good job in presenting the core ideas behind the functioning of the algorithm and their analysis.\n\nWeaknesses:\n\n1. The paper strongly relies on the techniques of [Sinha and Vaze. 2024], that is for both Lyapunov functions and surrogate functions. Similarly, the algorithm for small-loss regret guarantees is a simple modification of existing ones [Luo, 2017]. Thus, the algorithmic novelty is limited.\n2. The result for the COCO setting is attained at the cost of paying a $T^d$ computational complexity.\n3. The technical analysis is not particularly involved. \n\nOverall, even if the paper is not breakthrough from a technical perspective, I still believe that the results are interesting and that the way the authors combine existing algorithms and techniques to attain the desired theoretical guarantees is valuable.\n\nMinor:\nI believe that the presentation of the paper can be improved, for example, by reducing the dimension of the related works table. Moreover, the checklist still has some [TODO] in it.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper revisits online convex optimization with adversarial constraints (COCO), where an adversary reveals a convex cost function and a convex constraint function at each round. This work improves upon previous best-known result $O(\\sqrt{T})$ regret and $\\tilde{O}(\\sqrt{T})$ cumulative constraint violation (CCV) by achieving a  smaller CCV by trading it off with regret. To achieve this result, this paper considers the  constrained expert problem, similar to learning with expert advice (LEA). Furthermore, the authors propose a computationally efficient policy for the constrained expert problem.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* This paper conducts an in-depth study of COCO and presents a rich set of theoretical results, including constrained expert with $N$ experts, $d$-dimensional decision set, and $M$-smooth cost and constraints. \n\nWeaknesses:\n\n* The writing of this paper needs improvement, and the formatting should be refined—for example, Table 1 exceeds the page margins significantly. \n* The main theoretical results of this paper do not significantly improve upon prior work. \n* The proposed method appears to directly adopt the existing prediction with expert advice (PEA) framework, and the technical novelty is limited.\n\nQUESTIONS:\n1. Could the authors clearly articulate the technical novelty and contributions of this work? In particular, why do the theoretical results in Table 1 represent a significant improvement over prior work? \n2. According to the preliminaries, smoothness can be leveraged to obtain small-loss bounds, i.e., problem-dependent bounds. However, in this paper, the theoretical results for smooth functions remain in terms of $T$, i.e., they are problem-independent. Therefore, it is unclear to me why smoothness does not lead to a problem-dependent bound in the context of the COCO problem.\n3. According to the description of Algorithm 1, the proposed method appears to be a direct application of the PEA framework combined with adaptive learning rates. Could the authors elaborate on the technical challenges addressed in this work?\n\nLIMITATIONS:\nCompared to existing COCO algorithms, this paper adopts the PEA framework, which requires maintaining multiple experts. As a result, the proposed algorithm incurs additional computational complexity.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies online convex optimization with adversarial constraints (COCO), where the learner’s goal is to minimize both the regret and the cumulative constraint violation (CCV). The main contribution of the paper is to show that by leveraging small-loss bounds, it is possible to achieve a more flexible trade-off between constraint violation and regret minimization. Specifically, the proposed method guarantees a regret bound of $\\mathcal{O}(\\sqrt{T} + T^{\\beta})$ with a corresponding constraint violation of $\\mathcal{O}(T^{1-\\beta})$, where $\\beta$ is a tunable parameter chosen by the learner to balance the trade-off.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* The results in this paper recover those of [Vaze and Sinha, 2025] while offering a flexible mechanism for trading off between regret and CCV.\n\n* The idea of applying small-loss bounds to the analysis of the COCO problem is novel and insightful to me.\n* the paper is well-written and structured\n\n## Weaknesses\n\n- **Comparison with related work and technical contribution:** One of the main technical contributions is the development of an algorithm for the prediction with expert advice setting with unbounded losses. However, this problem appears closely related to the Lipschitz adaptivity problem in online learning, where scale-free algorithms have been studied. For example, \\[1, Section 2] provides a scale-free second-order bound. It seems the derived second-order bound also implies the small-loss bound by simply upper bound the loss functions by $G_T$. The paper would benefit from a more detailed discussion comparing these related results and techniques.\n\n- **Tightness of the bound:** While the use of small-loss bounds in COCO allows more flexibility in trading off regret and constraint violation, it is unclear how tight the proposed bounds are. The authors claim on lines 67–68 that *“Thus the unconditional $O(\\sqrt{T})$ regret guarantee for COCO cannot be improved.”* However, this statement is not entirely clear. It appears that the learner is allowed to violate constraints, whereas the comparator must strictly satisfy them. If we permit $O(T)$ CCV, it is unclear whether $O(\\sqrt{T})$ regret truly constitutes a lower bound. \n\n[1] Lipschitz Adaptivity with Multiple Learning Rates in Online Learning. COLT 2019.\n\nQUESTIONS:\n* Could the authors provide a more detailed comparison with Lipschitz adaptivity techniques in PEA and OCO? Please refer to the first point in Weakness for more details.\n\n* The feasibility assumption seems rather strong, as it requires the comparator to strictly satisfy the constraints. Is it possible to derive regret bounds where the comparator is allowed to violate the constraints slightly, for example allowing CCV up to $\\mathcal{O}(\\sqrt{T})$?\n\nLIMITATIONS:\nYes. One limitation of the paper is the computational efficiency of the algorithm in the OCO setting. This issue is partially addressed by imposing a smoothness assumption on the loss functions in Section 5.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
In this paper, the authors revisit the COCO problem. The COCO problem is defined as follows: in each round, a learner selects an action from a convex decision set, after which an adversary reveals both a convex cost function and a convex constraint function. The learner's objective is to minimize both regret and cumulative constraint violation (CCV), where CCV quantifies the total amount by which the constraints are violated over T rounds.

STRENGTHS AND WEAKNESSES:
It is well established that in the standard Online Convex Optimization (OCO) setting, where the constraint function gt = 0 for all t, achieving regret smaller than O(√T) is fundamentally impossible—even in one-dimensional problems. As a result, the O(√T) regret guarantee for the COCO problem is unimprovable in the general case. However, whether CCV can be made substantially smaller than O(√T) under additional natural assumptions has remained an open question.

The current state-of-the-art achieves O(√T) regret and O(√T) CCV. The authors make a significant contribution by introducing a tunable trade-off between regret and CCV. Specifically, under bounded convex cost and constraint functions, they propose an online policy that achieves a regret of O(√dT + T^β) and a significantly smaller CCV of O(dT^{1−β}), where d is the dimension of the decision set and β \in [0,1] is a user-defined parameter. This approach offers a meaningful improvement, allowing practitioners to flexibly balance regret minimization and constraint satisfaction based on problem-specific priorities.

QUESTIONS:
NA

LIMITATIONS:
there is no negative social impact.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03