Prior-independent Dynamic Auctions for a Value-maximizing Buyer

1 Introduction . Automatic bidding has become one of the main options for advertisers to buy advertisement opportunities in the online advertising market [ Dolan , 2020 ] . The prevalence of automatic bidding is partly driven by the fact that it significantly simplifies the interaction between the advertisers and the advertising platform . Nowadays , millions of auctions are conducted in any given day while the competitive landscape could be evolving over time and becoming vastly different across auctions . Therefore , it is difficult for the advertisers to reason about its dynamics , let alone optimizing their bids for each auction separately . Instead of soliciting for fine-grained bids for different auctions they participate in , the automatic bidding product only asks the advertisers to provide high-level objectives and constraints . With provided high-level information , the automatic bidding product bids on behalf of the advertisers to optimize the objectives while respecting the constraints . While there are many different types of automatic bidding produces , a well-adopted product bids to maximize the number of acquisitions subject to a constraint on the advertiser-specified return on investment ( ROI ) constraint , i.e. , a target ratio between the advertiser ’ s return and her advertising spend [ Facebook , 2021 , Google , 2021 ] . To bid on behalf of the advertiser to maximize the number of acquisitions , the automatic bidding products can invoke sophisticated prediction algorithms to predict the acquisition rate . These prediction algorithms have the benefit of being trained with long periods of data to have a good generalization [ Juan et al. , 2016 , McMahan et al. , 2013 , Zhou et al. , 2018 ] . In this way , the advertiser can only focus on designing their high-level objectives and specifying their constraints , while leaving the complicated bidding part to the automatic bidding products . The emergence of automatic bidding products opens up the opportunities for novel design of dynamic auction mechanisms , as classic mechanisms are heavily specialized for utility-maximizing bidders , the de facto paradigm in economic theory , which maximize the difference between their total value and total payment [ Edelman et al. , 2007 , Myerson , 1981 ] . In contrast , automated bidders behave very differently in a way such that the payment is not directly involved in their objectives ; and instead , the payment indirectly appears in the ROI constraints while the objectives aim to maximize the total value only . Therefore , classic auction design tailored for utility-maximizing bidders may not continue to perform well in an automatic bidding environment . Furthermore , recent results demonstrate that 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . for single-slot auctions , neither welfare nor revenue achieved by an incentive-compatible auction ( for utility-maximizing bidders , e.g. , second-price auctions ) at any reasonable equilibrium can be larger than 1/2 of the optimum [ Aggarwal et al. , 2019 , Deng et al. , 2021 ] . Recently , Balseiro et al . [ 2021 ] initiated the study of characterizing single-stage revenue-optimal auctions for automated bidders under various information structures . Surprisingly , they show that in a Bayesian setting in which the advertiser ’ value is private and the target ratio in the ROI constraint is public , a payment-scaled second-price auction can achieve first-best revenue , i.e. , the revenue when all information is public . For a comparison , recall that first-best revenue is in general not achievable against utility-maximizing bidders as demonstrated by the Myerson ’ s auction [ Myerson , 1981 ] . However , the assumption on perfect knowledge of the prior distribution on the advertiser ’ s values limits the application of their payment-scaled second-price auction , as the seller usually needs to learn the distribution via repeated interactions with the advertiser . 1.1 Our results . In this paper , we aim to relax such an assumption by studying the problem of designing priorindependent dynamic mechanisms against a single automated bidder . In line with the literature [ Amin et al. , 2013 ] , we consider a setting of repeated auctions with multiple stages against an impatient automated bidder whose valuations are drawn identically and independently across stages . Moreover , we assume the automated bidder bids in a way to satisfy her ex-ante ROI constraint per stage , consistent with the setup in [ Balseiro et al. , 2021 ] . In practice , the auction parameters are updated infrequently ( e.g. , once per day ) and the automated bidders participate in multiple repeated auctions in a stage , i.e. , the period between two consecutive updates . For each stage , the ROI constraints need to be satisfied on average over many auctions that the bidders participate in . Our main result is a novel dynamic mechanism achieving Õ ( T 2/3 ) regret against a strategic automated bidder , where T is the time horizon . Our dynamic mechanism adopts the explore-and-exploit scheme by first estimating the bidder ’ s value distribution . We then offer a robust version of the single-stage revenue-optimal mechanism to extract the revenue . A cornerstone of our dynamic mechanism in the exploration phase is a novel prior-independent single-stage mechanism that is incentive-compatible for automated bidders . This mechanism forces the bidder to make a trade-off between her future gain from misreporting her values and her immediate loss , leading to a Wasserstein distance bound on the magnitude of misreporting from the automated bidder — which is precisely the form of error that our exploitation mechanisms are robust against . 1.2 Related work . Optimal mechanism design is a central topic in economics and game theory with many successful real-world applications , such as combinatorial auctions for FCC spectrum auctions [ Cramton et al. , 2006 ] and generalized second-price auctions for online advertising [ Edelman et al. , 2007 ] . However , almost all of these classic works make the assumption of utility-maximizing bidders . In contrast , inspired by the development of automatic bidding products in online advertising industry , a growing body of recent literature starts to focus on automated bidders . Aggarwal et al . [ 2019 ] initiate the study of mechanism design with automated bidders and raise the question of how the mechanism designed for utility-maximizing bidders performs against automated bidders . They show that neither welfare nor revenue achieved by an incentive-compatible auction can be larger than 1/2 of the optimum . Deng et al . [ 2021 ] improve the approximation ratio to ( c+ 1 ) / ( c+ 2 ) with boosted auctions using side information that correlates with advertisers ’ values . The closest related work to ours is [ Balseiro et al. , 2021 ] , which studies single-stage revenue-optimal auctions for automated bidders under various information structures and bidding behavior . In particular , it examines the difference in auction design landscapes between utility-maximizing bidders and automated bidders and shows that first-best revenue can be achievable when either the the advertiser ’ value is public or the target ratio in the ROI constraint is public . However , their mechanisms are prior-dependent while our objective is to design prior-independent dynamic mechanisms . Our work is also related to the literature on robust online pricing against strategic agents . No-regret policies have been developed for both the non-contextual environment [ Amin et al. , 2013 , Drutsa , 2017 , 2018 , 2020 ] and the contextual environment [ Amin et al. , 2014 , Deng et al. , 2019 , 2020 , Golrezaei et al. , 2019 ] . All these results concern impatient utility-maximizing bidders and Amin et al . [ 2013 ] show that no learning algorithm can achieve sublinear revenue loss with a patient utility-maximizing bidder . In contrast , our work studies robust online pricing against a strategic automated bidder instead of a utility-maximizing bidder . 2 Preliminaries . Dynamic mechanisms . Throughout this paper , we consider dynamic mechanisms that solicit a single bid from the buyer in each stage . Roughly speaking , in each stage t , the buyer submits their bid bt , which is generally supposed to reflect their true value in stage t in some way.1 After receiving the buyer ’ s bids , the mechanism computes the fraction xt of the item allocated to the buyer and the payment pt , based on all bids that the buyer has submitted so far . Formally , let T be the time horizon , i.e. , the total number of stages . A dynamic mechanism M = { Mt } t∈ [ T ] = { ( xt , pt ) } t∈ [ T ] is a collection of allocation-payment pairs , one for each stage t. For each t , the allocation rule xt : Rt+ → [ 0 , 1 ] maps the vector of historical bids ( b1 , . . . , bt ) to the fraction of the item in stage t allocated to the buyer . Similarly , the payment rule pt : Rt+ → R+ maps historical bids to the payment from the buyer to the seller in stage t. Buyer ’ s incentives and behavior . We consider the case of automatic buyers , who are ROI constrained value maximizers . Such buyers maximize their expected value , subject to the constraint that the ratio between the value and the payment must be at least a fixed threshold in expectation . Formally , we model a value-maximizing buyer in dynamic environments in the following way . There is a value distribution D ∈ ∆ ( [ 0 , 1 ] ) which is private to the buyer . In each stage t , the buyer ’ s value vt is drawn fromD independently of everything else . For simplicity , throughout the paper , we assume that D is non-atomic , meaning that for any v ∈ [ 0 , 1 ] , Prv′∼D [ v′ = v ] = 0.2 There is a publicly known threshold τt ∈ [ 1 , ∞ ) in each stage t , which specifies the minimum ROI ratio that is acceptable to the buyer in stage t. Note that we require τt ≥ 1 because ( 1 ) conceptually , the buyer would always want their value to be at least their payment , and ( 2 ) technically , allowing τt arbitrarily close to 0 would enable the seller to extract unbounded revenue ( because the buyer is willing to pay up to vt/τt on average ) in a single stage , in which case it is impossible to achieve low regret . Moreover , in line with the literature [ Amin et al. , 2013 , Deng et al. , 2019 , Drutsa , 2017 ] ) , we assume there is a public discount factor λ ∈ ( 0 , 1 ) , which models how impatient the buyer is . Recall that a dynamic mechanism solicits a single bid bt from the buyer in each stage t. Since the mechanism decides allocation and payments based on all historical bids , the buyer may submit bids that depend on historical values in order to best respond . Thus , in general , we assume each bid bt : [ 0 , 1 ] t → R+ is a function of all historical values v1 , . . . , vt of the buyer . Fixing a dynamic mechanism M = { ( xt , pt ) } t , the buyer faces the following constrained optimization problem . max b1 , ... , bT E ( v1 , ... , vT ) ∼DT ∑ t∈ [ T ] λt · xt · vt subject to E vt∼D [ xt · vt − τt · pt | v1 , . . . , vt−1 ] ≥ 0 , ∀t ∈ [ T ] , v1 , . . . , vt−1 ∈ [ 0 , 1 ] . Here , bt = bt ( v1 , . . . , vt ) , xt = xt ( b1 , . . . , bt ) , and pt = pt ( b1 , . . . , bt ) . We assume the buyer is rational , and always chooses some optimal solution to the above optimization problem as their bidding strategy . Note that here , similar to the single-stage formulation considered in previous work [ Aggarwal et al. , 2019 , Balseiro et al. , 2021 , Deng et al. , 2021 ] , we only require ROI constraints to hold in expectation over vt ( as opposed to for all vt ) in each stage . Conceptually , this is because in practice , every stage corresponds to a period of time ( e.g. , a day ) , which normally consists of a large number of auctions happening in the same environment . Therefore , it makes more sense to set a target ROI ratio for the entire time period , rather than individual transactions which are normally quite small . Technically , in-expectation constraints allow the buyer to have richer bidding strategies , 1In the Bayesian setting ( i.e. , with a publicly known prior ) , it is without loss of generality to consider incentive-compatible mechanisms where the buyer is always incentivized to submit their true value as the bid . However , the mechanism we present , being prior-independent , is not incentive-compatible . As discussed later , we only require a rough correspondence between the bids and the true values , which , as we prove , can in fact be guaranteed by the mechanism . 2All our results still hold without this assumption . When there are mass points in D , the buyer ’ s optimal bidding strategy may be random , which complicates the presentation . and therefore , they can only make the problem of designing dynamic mechanisms harder . We also remark that one can simulate the case where the buyer submits multiple bids in each period of time , by making multiple copies of a stage with the same target ROI ratio . First-best revenue and seller ’ s goal . Our goal is to approximate the so-called first-best revenue in the long run , which is the maximum revenue possible when the buyer ’ s values are public . Formally , suppose the seller ’ s cost for producing a unit of the item in stage t is ct ∈ [ 0 , 1 ] , and the buyer ’ s value distribution and target ROI ratios are D and τt . Then , in each stage t , the maximum expected revenue OPTt ( D ) the seller can achieve is given by the following optimization formulation . max x , p E v∼D [ p ( v ) − ct · x ( v ) ] subject to E v∼D [ x ( v ) · v − τt · p ( v ) ] ≥ 0 . It is known that the first-best revenue is explicitly given by OPTt ( D ) = E v∼D [ max { v/τt − ct , 0 } ] , and that there is a single-stage incentive-compatible ( meaning that it is always in the buyer ’ s best interest to report their true value ) prior-dependent mechanism that achieves single-stage revenue OPTt ( D ) [ Balseiro et al. , 2021 ] . So overall , with access to the prior distribution D , the maximum total first-best revenue that the seller can achieve throughout the T stages is OPT ( D ) = ∑ t∈ [ T ] OPTt ( D ) . In this paper , we assume that the seller does not have access to the prior distribution D , and the goal is to achieve sublinear regret against the first-best revenue OPT ( D ) for all D with a single dynamic mechanism . Formally , we aim to find a mechanism M = { ( xt , pt ) } t∈ [ T ] , subject to the condition that for all D , there exists a bidding strategy satisfying the buyer ’ s ROI constraints,3 and moreover , sup D∈∆ ( V ) OPT ( D ) − ∑ t∈ [ T ] ( pt ( b D , M 1 , . . . , b D , M t ) − xt ( b D , M 1 , . . . , b D , M t ) · ct ) = o ( T ) , where bD , Mt = b D , M t ( v1 , . . . , vt ) is the buyer ’ s best response to mechanism M when the value distribution is D , as defined above .

Summary: The paper studies prior-independent dynamic auctions with a single value-maximizing buyer. In this setting, the buyer's objective is to maximize her obtained value (rather than utility) under ROI constraint (i.e., the value is at least $\tao$ times higher than the price paid). The authors show a dynamic auction mechanism that without having no prior information on the buyer’s value distribution (except from being bound in [0,1]), can extract a near-optimal revenue, i.e., sublinear regret for time horizon T. The work is under the assumption of a patient seller and a buyer has a time discount factor $\lambda$.  The algorithm is divided into three parts:  1. Exploration phase that incentivizes the buyer to report approximately truthfully. To my understanding, this is the main new technique in the work.  2. A buffer phase.  3. Exploitation phase that extracts the revenue using a robust version of the single-stage revenue-optimal mechanism for this problem.