Title: No-Regret Online Autobidding Algorithms in First-price Auctions
Venue: NeurIPS 2025
Accepted: yes
Keywords: Autobidding

No-regret Autobidding algorithm

Repeated auctions

ROI constraint
Abstract: Automated bidding to optimize online advertising with various constraints, e.g. ROI constraints and budget constraints, is widely adopted by advertisers. A key challenge lies in designing algorithms for non-truthful mechanisms with ROI constraints. While prior work has addressed truthful auctions or non-truthful auctions with weaker benchmarks, this paper provides a significant improvement: We develop online bidding algorithms for repeated first-price auctions with ROI constraints, benchmarking against the optimal randomized strategy in hindsight. In the full feedback setting, where the maximum competing bid is observed, our algorithm achieves a near-optimal $\tilde O(\sqrt{T})$ regret bound, and in the bandit feedback setting (where the bidder only observes whether the bidder wins each auction), our algorithm attains $\tilde O(T^{3/4})$ regret bound.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents online autobidding algorithms for repeated first-price auctions with ROI constraints. These algorithms compete against the optimal randomized strategy in hindsight. More specifically, the authors consider the problem where a single bidder participates in single-item first-price auctions repeated over $T$ rounds. At each time step, the learner observes the value $v_t$​ and then submits a bid $b_t$ based on this value and the history. The bidder wins the item if her submitted bid is not less than the highest competing bid  $d_t$ in that round. If she wins, she pays her submitted bid; otherwise, she pays nothing. The goal is to design an online bidding algorithm that maximizes the total realized utility, subject to a Return-On-Investment (ROI) constraint, meaning the total utility should be greater than the total payment. The authors first focus on the offline optimal strategy, which may be randomized, and show—using a convexification argument on the allocation-payment curve—that this randomized problem can be reduced to a deterministic one by replacing the original allocation function with its concave envelope. Based on this, they develop Algorithm 1 for the case where the distribution of the competition is known, and prove that it achieves a regret of order $O(T^{1/2})$. When the distribution of the competition is unknown, they consider two settings: full feedback, where the bidder observes the competing bid $d_t$ at the end of each round, and bandit feedback, where the bidder only observes a binary win/loss signal (0 or 1). For the full feedback setting, they develop an algorithm based on mirror descent that achieves a regret of $O(T^{1/2})$, and for the bandit setting, they propose an explore-then-commit strategy that achieves a regret of $O(T^{3/4})$, leaving a gap with the known lower bound for this problem $\\Omega(T^{2/3})$.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The paper is well motivated and addresses an important problem in online advertising. \n- From a technical standpoint, the author use a key convexification argument to reduce the randomized constrained optimization problem of the optimal strategy (OPT) to an equivalent deterministic optimization problem.\n- When the distribution of the bids of the competition is known, they propose an algorithm based on this convexification argument that achieves a regret of $O(T^{1/2})$. \n- When the distribution of the bids of the competition is unknown, and the bidder has a full feedback (the competing bid) at the end of the round, they propose an algorithm based on mirror descent that achieves a regret of order $O(T^{1/2})$. \n- In the more challenging bandit feedback setting—where only a binary win/loss signal is observed, they propose an algorithm based on explore-then-commit strategy that achieves a regret of order $O(T^{3/4})$, \n\nWeaknesses:\n- The paper lacks numerical simulations to empirically demonstrate the performance of the proposed algorithms.\n- Although the authors acknowledge the remaining gap in the regret bound for the bandit setting, the paper does not propose potential directions to reduce this gap.\n- The explanation of why the convexification argument preserves the optimal solution could be elaborated; in particular, the transition from optimizing under the original distribution $F$ to its concave enveloppe $F_{conv}$ needs more intuitions. \n- While the paper focuses on ROI constraints, which are important, it does not address how budget constraints—commonly present in real-world bidding environments—could be integrated into the proposed framework.\n\nQUESTIONS:\n- Could the authors provide intuitions on how the problem or solution would change if a budget constraint were introduced?\n- Are there any ideas or insights on how the regret bound in the bandit feedback setting might be improved ? \n- Can the authors discuss how the setting and results might extend to the more realistic case where multiple bidders are learning simultaneously, as is common in online advertising ? \n\nMinor comment: There appears to be a formatting typo at the end of page 9 in the expressions for $T^{1/4}$ and $T^{3/4}$, where the closing brace is missing or misplaced.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

comments: SUMMARY:
This paper studies the autobidding problem under the setting of ROI-constrained value maximizers in first-price auctions. It considers two distinct feedback settings: (1) the full-feedback setting, where the maximum competing bid is observed, and (2) the bandit-feedback setting, where the bidder only observes whether they win each auction. For each setting, the paper develops corresponding bidding algorithms with regret guarantees.

STRENGTHS AND WEAKNESSES:
Strength: 
1. The paper addresses an important problem in online advertising and provides useful regret bounds in a novel setting that avoids some of the assumptions made in prior work.  
2. The paper is well-written and well-organized.
           Weakness: 
1. The paper considers only ROI constraints and does not incorporate budget constraints, which are important in practical settings.
2. The paper lacks numerical results and real-world applications, leaving open questions about the practical feasibility and effectiveness of the proposed approach.

QUESTIONS:
1. Do the value vectors $v_t$, for $t \in [T]$, need to be i.i.d. in your setting?
2. Should the input to Algorithm 2 include the maximum competing bids revealed sequentially?
3. How does definition (6) depend on $c_l$? $d_t$ should not be known in the bandit feedback setting.

LIMITATIONS:
See  Weaknesses
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the design and analysis of autobidding algorithms for online first-price auctions. More specifically, the focus is on the problem of bidding repeatedly under a certain Return Over Investment constraint. This is motivated by the fact that these kinds of auctions and of constraints are common practice in online advertising auctions. The main challenge in this problem comes from the fact that the opposing bid distribution $F$, and the distribution of the agents' value $H$ are unknown and need to be learned online. This paper considers two types of feedback after each round: full information and bandit feedback. Before stating their algorithm, the paper provides a reduction of their problem against an arbitrary distribution $F$ to a problem against a concavified version $F$, which simplifies the problem and ensures known techniques can be better adapted. They then provide an algorithm, regret, and ROI constraint violation guarantees for this algorithm. They show their algorithm can guarantee a regret upper bounded by $\\tilde{\\mathcal{O}}\\left ( \\sqrt{T} \\right )$ under full-information feedback and $\\tilde{\\mathcal{O}}\\left ( T^{3/4} \\right )$ under bandit feedback. This result improves on the literature, not in terms of regret rates but on the class of comparator strategies against which the regret is computed (General vs Lipschitz).\n\nSTRENGTHS AND WEAKNESSES:\nThis paper presents a well-motivated and technically sound approach to autobidding in first-price auctions under ROI constraints, offering improved regret guarantees in both full-information and bandit settings, though it would benefit from lower bounds and empirical validations.\n\n# Strenght : \n\n- The paper is well written and motivated. The problem studied is interesting and the tools needed to obtain the result are non-trivial. \n\n- The paper proves new results that do not immediately follow from known ones.\n\n- The paper is technically sound and includes a helpful overview of the technical challenges, both in the introduction and in the proof sketches. \n\n# Weaknesses : \n\n- The paper doesn't provide matching lower bounds for the result stated. This makes it difficult to assess whether the rates obtained are optimal or if there is still room for improvement. \n\n- The paper lacks experiments. It would specifically be nice to have some in order to compare their result with the work of Aggarwal et al. [3], which is referred to in the paper. Their result strengthens this result in a non-straightforward manner (by extending the family of functions to which the fixed best strategy used to compute the regret belongs), as a result, it is difficult to gauge the difference made by this change. Experiments would provide a nice way to give some intuition. \n\n- The paper states that their result easily generalizes to other auction mechanisms (line 49, and footnote 4, page 4) as part of their result. It would be preferable to avoid emphasizing this claim without proof. I would be more comfortable if that was mentioned in the introduction and or conclusion (not in result / setting sections). Otherwise, you could provide (eventually in the appendix) a note describing how this would generalize to a few specific common auctions.\n\nQUESTIONS:\n1. I am wondering about the link with auctions with replenishing budget. It does seem that if we consider the setting where an algorithm for bidding is constrained by its budget but receives additional budget (say $\\rho v^t$ when the auction is won), an algorithm should be similar to the one in your setting? A setting of this kind is fully described in [1]. \n\n2. I understood that once the transformation (concavification of $F$) is done, the best strategy for each valuation $v$ is a deterministic bid. Therefore, we could define a function which maps the value observed to the best bid corresponding $b(v)$. I am wondering whether this function would be Lipschitz (or maybe only right/left Lipschitz). Do you have any counterexamples? I am asking to understand better the scale of the contribution. If the best in hindsight function $b(v)$ in the concavified version of the problem is Lipschitz, then one could use the result from previous work, right? \n(I understand that making the link between the concavified and original problem and showing how uncertainty in one is related to uncertainty in the other would remain an issue.)\n\n3. You did not provide in the paper any result on lower bounds. It seems from the theoretical results alone that it is unclear whether the tight regret bound should be $T^{2/3}$ or $T^{3/4}$. Do you have any insight? Did you run experiments for which a regret of $T^{2/3}$ could be attained?\n\n4. Is it clear that the optimal bound in the full-information feedback is $\\mathcal{O} ( \\sqrt{T} )$? Is there an existing lower bound in the literature, or could such a result be derived?\n\n\n[1] Berriaud, Damien, et al. \"To Spend or to Gain: Online Learning in Repeated Karma Auctions.\" arXiv preprint arXiv:2403.04057 (2024).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
The paper addresses the design of online bidding algorithms for repeated first-price auctions with ROI constraints. It benchmarks performance against the optimal randomized strategy in hindsight, focusing on two feedback settings: full feedback (where the highest competing bid is observed) and bandit feedback (where only win/loss outcomes are known). The goal is to minimize regret and ensure the ROI constraint is met.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The result of near-optimal algorithm for the repeated first-price auctions with ROI constraints is important.
2. Concrete theoretical analysis of the regret bounds for the proposed algorithm.

Weaknesses:
1. The algorithm design is kind of straightforward. Basically, it is built on an optimal algorithm requiring known distribution, and a subtle process dealing with estimating the unknown distribution.
2. The budget constraint is not considered. Although it is claimed that the result can be generalized to the setting with budget constraints, it seems not to be trivial.

QUESTIONS:
NA

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-04