Title: Parameter-free Algorithms for the Stochastically Extended Adversarial Model
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online Convex Optimization

Parameter-free Algorithm
Abstract: We develop the first parameter-free algorithms for the Stochastically Extended Adversarial (SEA) model, a framework that bridges adversarial and stochastic online convex optimization. Existing approaches for the SEA model require prior knowledge of problem-specific parameters, such as the diameter of the domain $D$ and the Lipschitz constant of the loss functions $G$, which limits their practical applicability. Addressing this, we develop parameter-free methods by leveraging the Optimistic Online Newton Step (OONS) algorithm to eliminate the need for these parameters. We first establish a comparator-adaptive algorithm for the scenario with unknown domain diameter but known Lipschitz constant, achieving an expected regret bound of $\tilde{O}\big(\Vert u\Vert_2^2 + \Vert u\Vert_2(\sqrt{\sigma^2_{1:T}} + \sqrt{\Sigma^2_{1:T}})\big)$, where $u$ is the comparator vector and $\sigma^2_{1:T}$ and $\Sigma^2_{1:T}$ represent the cumulative stochastic variance and cumulative adversarial variation, respectively. We then extend this to the more general setting where both $D$ and $G$ are unknown, attaining the comparator- and Lipschitz-adaptive algorithm. Notably, the regret bound exhibits the same dependence on $\sigma^2_{1:T}$ and $\Sigma^2_{1:T}$, demonstrating the efficacy of our proposed methods even when both parameters are unknown in the SEA model.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the stochastically extended adversarial (SEA) model in the online parameter-free setting where the domain (equivalently, the norm of the comparator) is unbounded and a uniform upper bound on the Lipschitz constants of the cost functions is not known a priori. The authors use the Optimistic Online Newton Step as the base algorithm and adapt it to the parameter-free setting by running multiple copies of base learners and combining their predictions using a multiplicative weight (MsMwC) algorithm. The authors adequately survey the related literature and put their contributions into an appropriate context.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-written and easy to follow. On the flip side, as discussed in the comments below, it is not clear whether the bounds derived in this paper are tight. Please see the questions below.\n\nQUESTIONS:\n1.\tPlease make it clear in the problem statement (Section 2.1) whether the distribution $D_t$, from which the cost functions are sampled, is known to the learner before it selects the action $x_t$ for round t?\n2.\tAs a follow-up to the above question, if $D_t$ is unknown and is never “exploited” by the algorithm, then how does stochasticity help in this setting? \n3.\tIn the definition of cumulative adversarial variation (line 128), what randomness does the outer expectation refer to? The expected gradients seemed to be deterministic (c.f. Assumption 2.3). \n4.\tDo the results hold when the distributions $D_t$’s are chosen adaptively?\n5.\tNo lower bound is established for the regret bounds derived in this paper. Thu,s it is not clear whether the regret bounds are tight. \n6.\tTheorem 4.1 “improves” upon [27, Theorem 5] by moving the expectation outside of the sup over the worst-case conditional variance. However, it does so only at the expense of an extra $||u||^2$ term. The authors discuss this limitation in Remark 4.2, however this discussion is very vague and can be classified as a conjecture. Since this “improvement” is claimed as a major contribution of the paper, I would highly recommend the authors give formal proof for the matching lower bound.\n\nLIMITATIONS:\nThe paper does not have any direct negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a parameter-free stochastic online optimization algorithm based on optimistic online Newton step. The algorithm does not rely on problem parameters such as the Lipschitz constant and domain diameter. The regret bound of the algorithm is similar to existing algorithms that need the knoweldge of the problem parameters.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. Well written paper, which is easy to understand.\n2. Clear contribution: problem-paraemter-free algorithm with regret bound similar to those requiring problem parameters\n\nWeakness:\n1. A problem-parameter-free algorithm was developed in [27]. The authors tried to justify the difference of the proposed algorithm and the one in [27]. The main difference in terms of regret bound is that the one in [27] depends on $\\tilde{\\sigma}^2$ (expectation of supremum), which is generally larger than $\\sigma^2$ (supremum of expectation). Thought the former is obviously larger than the latter, I still doubt how much difference this will make. The authors are suggested to give some examples, where the regret bound in [27] is much larger (in order sense) than the one in this paper.\n\n2. In general, compared to deterministic OCO, what is the main technical challenge of the considered stochastic OCO? In determinic case, the loss functions are changing, while in the stochastic case, the distributions of the loss functions are changing. Except for a variance term for each loss function, what will be the main novelty and technical challenge of stochastic OCO compared to the deterministic one?\n\nQUESTIONS:\n1. As discussed in Weakness, The authors are suggested to give some examples, where the regret bound in [27] is much larger (in order sense) than the one in this paper.\n\n2. Justification of the technical challenge of stochastic OCO compared to deterministic OCO\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers Stochastically Extended Adversarial (SEA) model, a framework that bridges adversarial and stochastic online convex optimization, and focuses on designing parameter-free algorithms, which refers to algorithms that requires no prior information about the domain $D$ and the Lipschitz constant $G$. The authors first develop a Optimistic Online Newton Step (OONS) algorithm and establish a comparator-adaptive regret bound, and then generalize their algorithm to the parameter-free setting where $D$ and $G$ are unknown.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThe paper is well written. The OONS algorithm is novel. The authors clearly demonstrate the advantages of OONS compared to existing optimism online mirror descent algorithms. The proof is clear and easy to follow.\n\n**Weaknesses**\n\n1. The biggest issue is that the parameter-free regret bound is not worst-case optimal. When D is unknown but G is known, the algorithm’s regret is larger by a factor of $||u||_2$ compared to the setting where both D and G are known. This makes the results of this paper insufficiently strong, especially considering that in the parameter-free online learning setting, existing algorithms [3] can achieve adaptive regret with worst-case optimality. \n2. Although the authors have eliminated the dependence on D and G, their algorithm is still not fully parameter-free, as it requires knowing the value of $T$ in advance. In previous parameter-free online learning algorithms [1][2], it was typically not necessary to know $T$ in advance. \n3. ONS is a second-order method, which can be computationally inefficient in practice.\n\n\n\n\n\n[1] Cutkosky, A. (2019, June). Artificial constraints and hints for unbounded online learning. In Conference on Learning Theory (pp. 874-894). PMLR.\n\n[2] Cutkosky, A., & Orabona, F. (2018, July). Black-box reductions for parameter-free online learning in banach spaces. In Conference On Learning Theory (pp. 1493-1529). PMLR.\n\n[3] Jacobsen, A., & Cutkosky, A. (2022, June). Parameter-free mirror descent. In Conference on Learning Theory (pp. 4160-4211). PMLR.\n\nQUESTIONS:\n1. See weakness.\n2. Is the main difference between the proposed OONS and the existing OOMD [1] in this paper just in the regularization, or are there other important differences between these two algorithms?\n3. Is there any lower bound for parameter-free SEA?\n\n\n\n[1] Chen, S., Zhang, Y. J., Tu, W. W., Zhao, P., & Zhang, L. (2024). Optimistic online mirror descent for bridging stochastic and adversarial online convex optimization. Journal of Machine Learning Research, 25(178), 1-62.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper proposes the first parameter-free algorithm based on the optimistic online Newton step algorithm for the stochastically extended adversarial (SEA) model, which is a hybrid model of stochastic and adversarial frameworks for online convex optimization. The authors obtain the regret bound of the proposed algorithm for the scenario with unknown domain D but known Lipschitz constant of the loss function G, and the scenario with both G and D unknown.

STRENGTHS AND WEAKNESSES:
Strengths
1. The contributions are clear. 
2. The technical results are solid.
3. The contributions are novel since there is no parameter-free algorithm for SEA. This paper fills this gap.

Weaknesses
The dependence of the regret-bound on the norm of the comparator is not optimal. Since the authors have discussed this point in detail, I don't see this as a big concern.

QUESTIONS:
1. I understand this paper is a theoretical paper and acknowledge its theoretical contributions, but I wonder, is it possible to validate the efficiency of the algorithm by numerical experiments?
2. Could the authors discuss the computational cost under the high-dimensional settings?
3. I wonder what the order of the variance of the algorithm is or how the variance of the proposed algorithm compares with other non-parameter-free algorithms.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-05