Prior-independent Dynamic Auctions for a Value-maximizing Buyer

1 Introduction . Automatic bidding has become one of the main options for advertisers to buy advertisement opportunities in the online advertising market [ Dolan , 2020 ] . The prevalence of automatic bidding is partly driven by the fact that it significantly simplifies the interaction between the advertisers and the advertising platform . Nowadays , millions of auctions are conducted in any given day while the competitive landscape could be evolving over time and becoming vastly different across auctions . Therefore , it is difficult for the advertisers to reason about its dynamics , let alone optimizing their bids for each auction separately . Instead of soliciting for fine-grained bids for different auctions they participate in , the automatic bidding product only asks the advertisers to provide high-level objectives and constraints . With provided high-level information , the automatic bidding product bids on behalf of the advertisers to optimize the objectives while respecting the constraints . While there are many different types of automatic bidding produces , a well-adopted product bids to maximize the number of acquisitions subject to a constraint on the advertiser-specified return on investment ( ROI ) constraint , i.e. , a target ratio between the advertiser ’ s return and her advertising spend [ Facebook , 2021 , Google , 2021 ] . To bid on behalf of the advertiser to maximize the number of acquisitions , the automatic bidding products can invoke sophisticated prediction algorithms to predict the acquisition rate . These prediction algorithms have the benefit of being trained with long periods of data to have a good generalization [ Juan et al. , 2016 , McMahan et al. , 2013 , Zhou et al. , 2018 ] . In this way , the advertiser can only focus on designing their high-level objectives and specifying their constraints , while leaving the complicated bidding part to the automatic bidding products . The emergence of automatic bidding products opens up the opportunities for novel design of dynamic auction mechanisms , as classic mechanisms are heavily specialized for utility-maximizing bidders , the de facto paradigm in economic theory , which maximize the difference between their total value and total payment [ Edelman et al. , 2007 , Myerson , 1981 ] . In contrast , automated bidders behave very differently in a way such that the payment is not directly involved in their objectives ; and instead , the payment indirectly appears in the ROI constraints while the objectives aim to maximize the total value only . Therefore , classic auction design tailored for utility-maximizing bidders may not continue to perform well in an automatic bidding environment . Furthermore , recent results demonstrate that 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . for single-slot auctions , neither welfare nor revenue achieved by an incentive-compatible auction ( for utility-maximizing bidders , e.g. , second-price auctions ) at any reasonable equilibrium can be larger than 1/2 of the optimum [ Aggarwal et al. , 2019 , Deng et al. , 2021 ] . Recently , Balseiro et al . [ 2021 ] initiated the study of characterizing single-stage revenue-optimal auctions for automated bidders under various information structures . Surprisingly , they show that in a Bayesian setting in which the advertiser ’ value is private and the target ratio in the ROI constraint is public , a payment-scaled second-price auction can achieve first-best revenue , i.e. , the revenue when all information is public . For a comparison , recall that first-best revenue is in general not achievable against utility-maximizing bidders as demonstrated by the Myerson ’ s auction [ Myerson , 1981 ] . However , the assumption on perfect knowledge of the prior distribution on the advertiser ’ s values limits the application of their payment-scaled second-price auction , as the seller usually needs to learn the distribution via repeated interactions with the advertiser . 1.1 Our results . In this paper , we aim to relax such an assumption by studying the problem of designing priorindependent dynamic mechanisms against a single automated bidder . In line with the literature [ Amin et al. , 2013 ] , we consider a setting of repeated auctions with multiple stages against an impatient automated bidder whose valuations are drawn identically and independently across stages . Moreover , we assume the automated bidder bids in a way to satisfy her ex-ante ROI constraint per stage , consistent with the setup in [ Balseiro et al. , 2021 ] . In practice , the auction parameters are updated infrequently ( e.g. , once per day ) and the automated bidders participate in multiple repeated auctions in a stage , i.e. , the period between two consecutive updates . For each stage , the ROI constraints need to be satisfied on average over many auctions that the bidders participate in . Our main result is a novel dynamic mechanism achieving Õ ( T 2/3 ) regret against a strategic automated bidder , where T is the time horizon . Our dynamic mechanism adopts the explore-and-exploit scheme by first estimating the bidder ’ s value distribution . We then offer a robust version of the single-stage revenue-optimal mechanism to extract the revenue . A cornerstone of our dynamic mechanism in the exploration phase is a novel prior-independent single-stage mechanism that is incentive-compatible for automated bidders . This mechanism forces the bidder to make a trade-off between her future gain from misreporting her values and her immediate loss , leading to a Wasserstein distance bound on the magnitude of misreporting from the automated bidder — which is precisely the form of error that our exploitation mechanisms are robust against . 1.2 Related work . Optimal mechanism design is a central topic in economics and game theory with many successful real-world applications , such as combinatorial auctions for FCC spectrum auctions [ Cramton et al. , 2006 ] and generalized second-price auctions for online advertising [ Edelman et al. , 2007 ] . However , almost all of these classic works make the assumption of utility-maximizing bidders . In contrast , inspired by the development of automatic bidding products in online advertising industry , a growing body of recent literature starts to focus on automated bidders . Aggarwal et al . [ 2019 ] initiate the study of mechanism design with automated bidders and raise the question of how the mechanism designed for utility-maximizing bidders performs against automated bidders . They show that neither welfare nor revenue achieved by an incentive-compatible auction can be larger than 1/2 of the optimum . Deng et al . [ 2021 ] improve the approximation ratio to ( c+ 1 ) / ( c+ 2 ) with boosted auctions using side information that correlates with advertisers ’ values . The closest related work to ours is [ Balseiro et al. , 2021 ] , which studies single-stage revenue-optimal auctions for automated bidders under various information structures and bidding behavior . In particular , it examines the difference in auction design landscapes between utility-maximizing bidders and automated bidders and shows that first-best revenue can be achievable when either the the advertiser ’ value is public or the target ratio in the ROI constraint is public . However , their mechanisms are prior-dependent while our objective is to design prior-independent dynamic mechanisms . Our work is also related to the literature on robust online pricing against strategic agents . No-regret policies have been developed for both the non-contextual environment [ Amin et al. , 2013 , Drutsa , 2017 , 2018 , 2020 ] and the contextual environment [ Amin et al. , 2014 , Deng et al. , 2019 , 2020 , Golrezaei et al. , 2019 ] . All these results concern impatient utility-maximizing bidders and Amin et al . [ 2013 ] show that no learning algorithm can achieve sublinear revenue loss with a patient utility-maximizing bidder . In contrast , our work studies robust online pricing against a strategic automated bidder instead of a utility-maximizing bidder . 2 Preliminaries . Dynamic mechanisms . Throughout this paper , we consider dynamic mechanisms that solicit a single bid from the buyer in each stage . Roughly speaking , in each stage t , the buyer submits their bid bt , which is generally supposed to reflect their true value in stage t in some way.1 After receiving the buyer ’ s bids , the mechanism computes the fraction xt of the item allocated to the buyer and the payment pt , based on all bids that the buyer has submitted so far . Formally , let T be the time horizon , i.e. , the total number of stages . A dynamic mechanism M = { Mt } t∈ [ T ] = { ( xt , pt ) } t∈ [ T ] is a collection of allocation-payment pairs , one for each stage t. For each t , the allocation rule xt : Rt+ → [ 0 , 1 ] maps the vector of historical bids ( b1 , . . . , bt ) to the fraction of the item in stage t allocated to the buyer . Similarly , the payment rule pt : Rt+ → R+ maps historical bids to the payment from the buyer to the seller in stage t. Buyer ’ s incentives and behavior . We consider the case of automatic buyers , who are ROI constrained value maximizers . Such buyers maximize their expected value , subject to the constraint that the ratio between the value and the payment must be at least a fixed threshold in expectation . Formally , we model a value-maximizing buyer in dynamic environments in the following way . There is a value distribution D ∈ ∆ ( [ 0 , 1 ] ) which is private to the buyer . In each stage t , the buyer ’ s value vt is drawn fromD independently of everything else . For simplicity , throughout the paper , we assume that D is non-atomic , meaning that for any v ∈ [ 0 , 1 ] , Prv′∼D [ v′ = v ] = 0.2 There is a publicly known threshold τt ∈ [ 1 , ∞ ) in each stage t , which specifies the minimum ROI ratio that is acceptable to the buyer in stage t. Note that we require τt ≥ 1 because ( 1 ) conceptually , the buyer would always want their value to be at least their payment , and ( 2 ) technically , allowing τt arbitrarily close to 0 would enable the seller to extract unbounded revenue ( because the buyer is willing to pay up to vt/τt on average ) in a single stage , in which case it is impossible to achieve low regret . Moreover , in line with the literature [ Amin et al. , 2013 , Deng et al. , 2019 , Drutsa , 2017 ] ) , we assume there is a public discount factor λ ∈ ( 0 , 1 ) , which models how impatient the buyer is . Recall that a dynamic mechanism solicits a single bid bt from the buyer in each stage t. Since the mechanism decides allocation and payments based on all historical bids , the buyer may submit bids that depend on historical values in order to best respond . Thus , in general , we assume each bid bt : [ 0 , 1 ] t → R+ is a function of all historical values v1 , . . . , vt of the buyer . Fixing a dynamic mechanism M = { ( xt , pt ) } t , the buyer faces the following constrained optimization problem . max b1 , ... , bT E ( v1 , ... , vT ) ∼DT ∑ t∈ [ T ] λt · xt · vt subject to E vt∼D [ xt · vt − τt · pt | v1 , . . . , vt−1 ] ≥ 0 , ∀t ∈ [ T ] , v1 , . . . , vt−1 ∈ [ 0 , 1 ] . Here , bt = bt ( v1 , . . . , vt ) , xt = xt ( b1 , . . . , bt ) , and pt = pt ( b1 , . . . , bt ) . We assume the buyer is rational , and always chooses some optimal solution to the above optimization problem as their bidding strategy . Note that here , similar to the single-stage formulation considered in previous work [ Aggarwal et al. , 2019 , Balseiro et al. , 2021 , Deng et al. , 2021 ] , we only require ROI constraints to hold in expectation over vt ( as opposed to for all vt ) in each stage . Conceptually , this is because in practice , every stage corresponds to a period of time ( e.g. , a day ) , which normally consists of a large number of auctions happening in the same environment . Therefore , it makes more sense to set a target ROI ratio for the entire time period , rather than individual transactions which are normally quite small . Technically , in-expectation constraints allow the buyer to have richer bidding strategies , 1In the Bayesian setting ( i.e. , with a publicly known prior ) , it is without loss of generality to consider incentive-compatible mechanisms where the buyer is always incentivized to submit their true value as the bid . However , the mechanism we present , being prior-independent , is not incentive-compatible . As discussed later , we only require a rough correspondence between the bids and the true values , which , as we prove , can in fact be guaranteed by the mechanism . 2All our results still hold without this assumption . When there are mass points in D , the buyer ’ s optimal bidding strategy may be random , which complicates the presentation . and therefore , they can only make the problem of designing dynamic mechanisms harder . We also remark that one can simulate the case where the buyer submits multiple bids in each period of time , by making multiple copies of a stage with the same target ROI ratio . First-best revenue and seller ’ s goal . Our goal is to approximate the so-called first-best revenue in the long run , which is the maximum revenue possible when the buyer ’ s values are public . Formally , suppose the seller ’ s cost for producing a unit of the item in stage t is ct ∈ [ 0 , 1 ] , and the buyer ’ s value distribution and target ROI ratios are D and τt . Then , in each stage t , the maximum expected revenue OPTt ( D ) the seller can achieve is given by the following optimization formulation . max x , p E v∼D [ p ( v ) − ct · x ( v ) ] subject to E v∼D [ x ( v ) · v − τt · p ( v ) ] ≥ 0 . It is known that the first-best revenue is explicitly given by OPTt ( D ) = E v∼D [ max { v/τt − ct , 0 } ] , and that there is a single-stage incentive-compatible ( meaning that it is always in the buyer ’ s best interest to report their true value ) prior-dependent mechanism that achieves single-stage revenue OPTt ( D ) [ Balseiro et al. , 2021 ] . So overall , with access to the prior distribution D , the maximum total first-best revenue that the seller can achieve throughout the T stages is OPT ( D ) = ∑ t∈ [ T ] OPTt ( D ) . In this paper , we assume that the seller does not have access to the prior distribution D , and the goal is to achieve sublinear regret against the first-best revenue OPT ( D ) for all D with a single dynamic mechanism . Formally , we aim to find a mechanism M = { ( xt , pt ) } t∈ [ T ] , subject to the condition that for all D , there exists a bidding strategy satisfying the buyer ’ s ROI constraints,3 and moreover , sup D∈∆ ( V ) OPT ( D ) − ∑ t∈ [ T ] ( pt ( b D , M 1 , . . . , b D , M t ) − xt ( b D , M 1 , . . . , b D , M t ) · ct ) = o ( T ) , where bD , Mt = b D , M t ( v1 , . . . , vt ) is the buyer ’ s best response to mechanism M when the value distribution is D , as defined above .

Summary: This paper studies the problem of a seller who interacts repeatedly with a buyer.  In each round an item is up for sale, and the buyer draws a value from an unknown (to the seller) distribution.  The buyer is assumed to be less patient than the seller, and the seller's goal is to maximize revenue.  Prior work had established low-regret sales strategies for the seller when the buyer is a utility maximizer.  This paper considers a value-maximizing bidder, who wants to maximize value obtained subject to an interim ROI constraint.  The authors construct a dynamic mechanism with regret O(T^{2/3}), versus the best revenue attainable if the value distribution were known.  At a high level the approach is similar to the utility-maximizing case: run an explore phase that obtains low revenue but learns about the buyer's value distribution, then deploy a mechanism aimed at extracting near-optimal revenue from what was learned.  The key here is to incentivize the buyer not to misreport in the explore phase, which is done by using a strictly truthful per-round auction (and leveraging the assumption that the buyer is less patient).  For a value-maximizing buyer, what is needed for strict truthfulness is different.  So a main technical contribution of the work is to propose a new single-round auction format that is strictly single-stage incentive compatible.