A UCTION LEARNING AS A TWO - PLAYER GAME
A BSTRACT
Designing an incentive compatible auction that maximizes expected revenue is a central problem in Auction Design.
While theoretical approaches to the problem have hit some limits, a recent research direction initiated by Duetting et al.
(2019) consists in building neural network architectures to ﬁnd optimal auctions.
We propose two conceptual deviations from their approach which result in enhanced performance.
First, we use recent results in theoretical auction design to introduce a time-independent Lagrangian.
This not only circumvents the need for an expensive hyper-parameter search (as in prior work), but also provides a single metric to compare the performance of two auctions (absent from prior work).
Second,the optimization procedure in previous work uses an inner maximization loop to compute optimal misreports.
We amortize this process through the introduction of an additional neural network.
We demonstrate the effectiveness of our approach by learning competitive or strictly improved auctions compared to prior work.
Both results together further imply a novel formulation of Auction Design as a two-player game with stationary utility functions.
I NTRODUCTION
Efﬁciently designing truthful auctions is a core problem in Mathematical Economics.
Concrete examples include the sponsored search auctions conducted by companies as Google or auctions run on platforms as eBay.
Following seminal work of Vickrey (Vickrey, 1961) and Myerson (Myerson, 1981), auctions are typically studied in the independent private valuations model: each bidder has a valuation function over items, and their payoff depends only on the items they receive.
Moreover, the auctioneer knows aggregate information about the population that each bidder comes from, modeled as a distribution over valuation functions, but does not know precisely each bidder’s valuation (outside of any information in this Bayesian prior).
A major difﬁculty in designing auctions is that valuations are private and bidders need to be incentivized to report their valuations truthfully.
The goal of the auctioneer is to design an incentive compatible auction which maximizes expected revenue.
Auction Design has existed as a rigorous mathematical ﬁeld for several decades and yet, complete characterizations of the optimal auction only exist for a few settings.
While Myerson’s Nobel prizewinning work provides a clean characterization of the single-item optimum (Myerson, 1981), optimal multi-item auctions provably suffer from numerous formal measures of intractability (including computational intractability, high description complexity, non-monotonicity, and others)
(Daskalakis et al., 2014; Chen et al., 2014; 2015; 2018; Hart & Reny, 2015; Thanassoulis, 2004).
An orthogonal line of work instead develops deep learning architectures to ﬁnd the optimal auction.
Duetting et al.
(2019) initiated this direction by proposing RegretNet, a feed-forward architecture.
They frame the auction design problem as a constrained learning problem and lift the constraints into the objective via the augmented Lagrangian method.
Training RegretNet involves optimizing this Lagrangian-penalized objective, while simultaneously updating network parameters and the
Lagrangian multipliers themselves.
This architecture produces impressive results: recovering nearoptimal auctions in several known multi-item settings, and discovering new mechanisms when a theoretical optimum is unknown.
Yet, this approach presents several limitations.
On the conceptual front, our main insight is a connection to an exciting line of recent works (Hartline & Lucier, 2010; Hartline et al., 2011; Bei &
Huang, 2011; Daskalakis & Weinberg, 2012; Rubinstein & Weinberg, 2018; Dughmi et al., 2017;
Cai et al., 2019) on ε -truthful-to-truthful reductions.
On the technical front, we identify three areas for improvement.
First, their architecture is difﬁcult to train in practice as the objective is nonstationary.
Speciﬁcally, the Lagrangian multipliers are time-dependent and they increase following a pre-deﬁned schedule, which requires careful hyperparameter tuning (see §3.1 for experiments illustrating this).
Leveraging the aforementioned works in Auction Theory, we propose a stationary
Lagrangian objective.
Second, all prior work inevitably ﬁnds auctions which are not precisely incentive compatible, and does not provide a metric to compare, say, an auction with revenue 1 .which is 0 .
002 -truthful, or one with revenue 1 which is 0 .
001 -truthful.
We argue that our stationary
Lagrangian objective serves as a good metric (and that the second auction of our short example is “better” for our metric).
Finally, their training procedure requires an inner-loop optimization (essentially, this inner loop is the bidders trying to maximize utility in the current auction), which is itself computationally expensive.
We use amortized optimization to make this process more efﬁcient.
C ONTRIBUTIONS
This paper leverages recent work in Auction Theory to formulate the learning of revenue-optimal auctions as a two-player game.
We develop a new algorithm ALGnet (Auction Learning Game network) that produces competitive or better results compared to Duetting et al.
(2019)’s
RegretNet.
In addition to the conceptual contributions, our approach yields the following improvements (as
RegretNet
is
already learning
near
-optimal
auctions
, our improvement over RegretNet is not due to signiﬁcantly higher optimal revenues).
– Easier hyper-parameter tuning :
By constructing a time-independent loss function, we circumvent the need to search for an adequate parameter scheduling.
Our formulation also involves less hyperparameters, which makes it more robust.
– A metric to compare auctions : We propose a metric to compare the quality of two auctions which are not incentive compatible.
– More efﬁcient training : We replace the inner-loop optimization of prior work with a neural network, which makes training more efﬁcient.
– Online auctions : Since the learning formulation is time-invariant, ALGnet is able to quickly adapt in auctions where the bidders’ valuation distributions varies over time.
Such setting appears for instance in the online posted pricing problem studied in Bubeck et al.
Furthermore, these technical contributions together now imply a novel formulation of auction learning as a two-player game (not zero-sum) between an auctioneer and a misreporter.
The auctioneer is trying to design an incentive compatible auction that maximizes revenue while the misreporter is trying to identify breaches in the truthfulness of these auctions.
The paper decomposes as follows.
Section 2 introduces the standard notions of auction design.
Section 3 presents our game formulation for auction learning.
Section 4 provides a description of ALGnet and its training procedure.
Finally,
Section 5 presents numerical evidence for the effectiveness of our approach.
R ELATED WORK
Auction design and machine learning.
Machine learning and computational learning theory have been used in several ways to design auctions from samples of bidder valuations.
Machine learning has been used to analyze the sample complexity of designing optimal revenue-maximizing auctions.
This includes the framework of single-parameter settings (Morgenstern & Roughgarden, 2015; Huang et al., 2018; Hartline & Taggart, 2019; Roughgarden & Schrijvers, 2016; Gonczarowski & Nisan, 2017; Guo et al., 2019), multi-item auctions (Dughmi et al., 2014; Gonczarowski & Weinberg, 2018), combinatorial auctions (Balcan et al., 2016; Morgenstern & Roughgarden, 2016; Syrgkanis, 2017) and allocation mechanisms (Narasimhan & Parkes, 2016).
Other works have leveraged machine learning to optimize different aspects of mechanisms (Lahaie, 2011; Dütting et al., 2015).
Our approach is different as we build a deep learning architecture for auction design.
Auction design and deep learning.
While Duetting et al.
(2019) is the ﬁrst paper to design auctions through deep learning, several other paper followed-up this work.
Feng et al.
(2018) extended it to budget constrained bidders, Golowich et al.
(2018) to the facility location problem.
Tacchetti et al.
(2019) built architectures based on the Vickrey- Clarke-Groves mechanism.
Rahme et al.
(2021) used permutation-equivariant networks to design symmetric auctions.
Shen et al.
(2019) and Duetting et al.
(2019) proposed architectures that exactly satisfy incentive compatibility but are speciﬁc to single-bidder settings.
While all the previously mentioned papers consider a non-stationary objective function, we formulate a time-invariant objective that is easier to train and that makes comparisons between mechanisms possible.
A UCTION DESIGN AS A TIME - VARYING LEARNING PROBLEM
We ﬁrst review the framework of auction design and the problem of ﬁnding truthful mechanisms.
We then recall the learning problem proposed by Duetting et al.
to ﬁnd optimal auctions.
2. A UCTION DESIGN AND LINEAR PROGRAM
Auction design.
We consider an auction with n bidders and m items.
We will denote by N = { 1 , . . . , n } and M = { 1 , . . .
, m } the set of bidders and items.
Each bidder i values item j at a valuation denoted v ij .
We will focus on additive auctions.
These are auctions where the value of a set
S of items is equal to the sum of the values of the elements in that set at (cid:80) j ∈ S v ij .
Additive auctions are perhaps the most well-studied setting in multi-item auction design (Hart & Nisan, 2012; Li &
Yao, 2013; Daskalakis et al., 2014; Cai et al., 2016; Daskalakis et al., 2017).
The auctioneer does not know the exact valuation proﬁle V
= ( v ij ) i ∈ N,j ∈ M of the bidders in advance
but he does know the distribution from which they are drawn: the valuation vector of bidder i , (cid:126)v i =
( v i 1 , . . .
, v im ) is drawn from a distribution D i over R m .
We will further assume that all bidders are independent and that D 1 = · · · = D n .
As a result V is drawn from D := ⊗ ni
=1 D i =
D ⊗ n 1 .
Deﬁnition 1.
An auction is deﬁned by a randomized allocation rule g
=
( g 1 , . .
. , g n ) and a payment rule p
= ( p 1 , . .
p n ) where g i : R n × m → [0 , 1] m and p
i : R n × m → R (cid:62) 0 .
Additionally for all items j and valuation proﬁles V , the g i must satisfy (cid:80) i
[ g i ( V )]
j (cid:54) 1 .
Given a bid matrix B = ( b ij ) i ∈ N,j ∈ M , [ g i ( B )]
j is the probability that bidder i receives object j and p i ( B ) is the price bidder i has to pay to the auction.
The condition (cid:80) i
[ g i ( V )] j (cid:54) 1 allows the possibility for an item to be not allocated.
Deﬁnition 2.
The utility of bidder i is deﬁned by u i ( (cid:126)v i , B ) =
(cid:80) mj
=1 [ g i ( B )] j v ij − p i ( B ) .
Bidders seek to maximize their utility and may report bids that are different from their true valuations.
In the following, we will denote by B − i the ( n − 1)
× m bid matrix without bidder i , and by ( (cid:126)b (cid:48)
i , B − i ) the n × m bid matrix that inserts (cid:126)b (cid:48)
i into row i of B − i (for example: B :=
( (cid:126)b i , B − i ) .
We aim at auctions that incentivize bidders to bid their true valuations.
Deﬁnition 3.
An auction ( g, p ) is dominant strategy incentive compatible (DSIC) if each bidder’s utility is maximized by reporting truthfully no matter what the other bidders report.
For every bidder i, valuation (cid:126)v i ∈
D
i , bid (cid:126)b i (cid:48) ∈ D i and bids B − i ∈
D − i , u i ( (cid:126)v i , ( (cid:126)v i , B − i )) (cid:62) u i ( (cid:126)v i , ( (cid:126)b i (cid:48) , B − i )) .
Deﬁnition 4.
An auction is individually rational (IR) if for all i ∈ N, (cid:126)v i ∈ D i and B −
i ∈
D − i ,
[Equation] [Equation]
IR
[Equation]
In a DSIC auction, the bidders have the incentive to truthfully report their valuations and therefore, the revenue on valuation proﬁle V is (cid:80) ni
=1 p i ( V ) .
Optimal auction design aims at ﬁnding a DSIC and IR auction that maximizes the expected revenue rev :=
E V ∼ D [ (cid:80) ni
=1 p i ( V )] .
Since there is no known characterization of DSIC mechanisms in the multi-item setting, we resort to the relaxed notion of ex-post regret .
It measures the extent to which an auction violates DSIC.
Deﬁnition 5.
The ex-post regret for a bidder i is the maximum increase in his utility when considering all his possible bids and ﬁxing the bids of others.
For a valuation proﬁle V , it is given by r i ( V ) = max
[Equation]
V −
[Equation] [Equation]
u i ( (cid:126)v i , ( (cid:126)v i , V − i )) .
In particular, DSIC is equivalent to
[Equation]
The bid (cid:126)b (cid:48) i that achieves r i ( V ) is called the optimal misreport of bidder i for valuation proﬁle V .
Therefore, ﬁnding an optimal auction is equivalent to the following linear program:
[Equation]
i , B − i )) (cid:62) 0 , ∀ i ∈ N, (cid:126)v i ∈
D
i , B − i ∈
D − i .
(LP)
2. A UCTION DESIGN AS A LEARNING PROBLEM
As the space of auctions M may be large, we will set a parametric model.
In what follows, we consider the class of auctions ( g w , p w ) encoded by a neural network of parameter w ∈ R d .
The corresponding utility and regret function will be denoted by u wi and r wi .
Following Duetting et al.
(2019), the formulation (LP) is relaxed: the IC constraint for all V ∈ D is replaced by the expected constraint E V ∼ D [ r wi ( V )]
= 0 for all
i ∈ N.
The justiﬁcation for this relaxation can be found in Duetting et al.
By replacing expectations with empirical averages, the learning problem becomes:
[Equation]
The learning problem ( (cid:99) LP ) does not ensure (IR).
However, this constraint is usually built into the parametrization (architecture) of the model: by design, the only auction mechanism considered satisfy (IR).
Implementation details can be found in Duetting et al.
; Rahme et al.
(2021) or in Sec 4.
A UCTION LEARNING AS A TWO - PLAYER GAME
We ﬁrst present the optimization and the training procedures for ( (cid:99) LP ) proposed by Duetting et al.
We then demonstrate with numerical evidence that this approach presents two limitations: hyperparameter sensitivity and lack of interpretability.
Using the concept of ε -truthful to truthful reductions, we construct a new loss function that circumvents these two aspects.
Lastly, we resort to amortized optimization and reframe the auction learning problem as a two-player game.
3. T HE AUGMENTED L AGRANGIAN METHOD AND ITS SHORTCOMINGS
Optimization and training.
We brieﬂy review the training procedure proposed by Duetting et al.
(2019) to learn optimal auctions.
The authors apply the augmented Lagrangian method to solve the constrained problem ( (cid:99) LP ) and consider the loss:
[Equation]
where λ ∈ R n is a vector of Lagrange multipliers and ρ > 0 is a parameter controlling the weight of the quadratic penalty.
More details about the training procedure can be found in Appendix A.
Scheduling consistency problem.
The parameters λ and ρ are time-varying.
Indeed, their value changes according to a pre-deﬁned scheduling of the following form: 1) Initialize λ and ρ with respectively λ 0 and ρ 0 , 2) Update ρ every T ρ iterations : ρ t +1 ← ρ t + c, where c is a pre-deﬁned constant, 3) Update λ every T λ iterations according to λ ti ← λ ti + ρ t (cid:98) r w t i .
Therefore, this scheduling requires to set up ﬁve hyper parameters ( λ 0 , ρ 0 , c, T λ , T ρ ) .
Some of the experiments found Duetting et al.
(2019) were about learning an optimal mechanism for an n -bidder m -item auction ( n × m ) where the valuations are iid U [0 , 1] .
Different scheduling parameters were used for different values of n and m .
We report the values of the hyper parameters used for the 1 × 2 , 3 × 10 and 5 × 10 settings in Table 1(a).
A natural question is whether the choice of parameters heavily affects the performance.
We proceed to a numerical investigation of this questions by trying different schedulings (columns) for different settings (rows) and report our the results in Table 1(b).
[Table]
The auction returned by the network dramatically varies with the choice of scheduling parameters.
When applying the parameters of 1 × 2 to 5 × 10 , we obtain a revenue that is lower by 30%!
The performance of the learning algorithm strongly depends on the speciﬁc values of the hyperparameters.
Finding an adequate scheduling requires an extensive and time consuming hyperparameter search.
Lack of interpretability.
How should one compare two mechanisms with different expected revenue and regret?
Is a mechanism M 1 with revenue P 1 = 1 . 01 and an average total regret
R 1 = 0 .
02 better than a mechanism
M 2 with P 2 = 1 . 0 and R 2 = 0 .
The approach in Duetting et al.
(2019) cannot answer this question.
To see that, notice that when λ 1 = · · · = λ n = λ we can rewrite L ( w ; λ ; ρ )
= − P + λR + ρ 2 R 2 .
Which mechanism is better depends on the values of λ and ρ .
For example if ρ = 1 and λ = 0 .
1 we ﬁnd that M 1 is better, but if ρ = 1 and λ = 10 then Mis better.
Since the values of λ and ρ change with time, the Lagrangian approach in Duetting et al.
(2019) cannot provide metric to compare two mechanisms.
3. A TIME - INDEPENDENT AND INTERPRETABLE LOSS FUNCTION FOR AUCTION LEARNING
Our ﬁrst contribution consists in introducing a new loss function for auction learning that addresses the two ﬁrst limitations of Duetting et al.
(2019) mentioned in Section 3.1.
We ﬁrst motivate this loss in the one bidder case and then extend it to auctions with many bidders.
3.2. M ECHANISMS WITH ONE BIDDER
Proposition 1.
[Balcan et al.
(2005), attributed to Nisan]
Let M be an additive auction with 1 bidder and m items.
Let P and R denote the expected revenue and regret,
P = E V ∈ D [ p ( V )] and
R = E V ∈ D [ r ( V )] .
There exists a mechanism M ∗ with expected revenue P ∗ =
( √ P − √ R ) 2 and zero regret R ∗ = 0 .
A proof of this proposition can be found in Appendix C. Comparing two mechanisms is straightforward when both of them have zero-regret: the best one achieves the highest revenue.
Prop. 1 allows a natural and simple extension of this criteria for non zero-regret mechanism with one bidder: we will say that M 1 is better than M 2 if and only if M ∗ 1 is better than M ∗ 2 :
[Equation]
Using our metric, we ﬁnd that a one bidder mechanism with revenue of 1 . 00 and regret of 0 .
01 is ”better” than one with revenue 1 . 01 and regret 0 .
3.2. M ECHANISMS WITH MULTIPLE BIDDERS
Let M 1 and M 2 be two mechanisms with n bidders and m objects.
Let P i
and
R i denote their total expected revenue and regret, P i =
E V ∈ D (cid:104)(cid:80) nj =1 p j ( V ) (cid:105) and R i =
E V ∈ D (cid:104)(cid:80) nj =1 r j ( V ) (cid:105) .
We can extend our metric derived in Section 3.2.1 to the multiple bidder by the following:
[Equation]
When n = 1 we recover the criteria from Section 3.2.1 that is backed by Prop. 1.
When n > 1 , it is considered a major open problem whether the extension of Prop. 1 still holds.
Note that a multibidder variant of Prop. 1 does hold under a different solution concept termed “Bayesian Incentive
Compatible” (Rubinstein & Weinberg, 2018;
Cai et al., 2019), supporting the conjecture that Prop. indeed extends.
2 Independently of whether or not Prop. 1 holds, this reasoning implies a candidate loss function for the multi-bidder setting which we can evaluate empirically.
This way of comparing mechanisms motivates the use of loss function:
L ( P, R ) = − ( √ P − √ R ) instead of the Lagrangian from Section 3, and indeed this loss function works well in practice.
We empirically ﬁnd the loss function L m ( P, R ) =
− ( √ P − √ R ) + R further accelerates training, as it further (slightly) biases towards mechanisms with low regret.
Both of these loss function are time-independent and hyperparameter-free.
3. A MORTIZED MISREPORT OPTIMIZATION
To compute the regret
[Equation]
one has to solve the optimization problem:
[Equation]
In Duetting et al.
, this optimization problem is solved with an inner optimization loop for each valuation proﬁle.
In other words, computing the regret of each valuation proﬁle is solved separately and independently, from scratch.
If two valuation proﬁles are very close to each other, one should expect that the resulting optimization problems to have close results.
We leverage this to improve training efﬁciency.
We propose to amortize this inner loop optimization.
Instead of solving all these optimization problems independently, we will instead learn one neural network M ϕ that tries to predict the solution of all of them.
M ϕ takes as entry a valuation proﬁle and maps it to the optimal misreport:
[Equation]
(cid:48)
[Equation]
V −
[Equation] [Equation]
i ∈ N
The loss L r that M ϕ is trying to minimize follows naturally from that deﬁnition and is then given by:
L r ( ϕ, w ) = − E V ∈ D
[ (cid:80) ni
=1 u wi ( (cid:126)v i , ([ M ϕ ( V )]
i , V − i ))] .
3. A UCTION LEARNING AS A TWO - PLAYER GAME
In this section, we combine the ideas from Sections 3.2 and 3.3 to obtain a new formulation for the auction learning problem as a two-player game between an Auctioneer with parameter w and a
Misreporter with parameter ϕ .
The optimal parameters for the auction learning problem ( w ∗ , ϕ ∗ ) are a Nash Equilibrium for this game.
The Auctioneer is trying to design a truthful (IC) and rational (IR) auction that maximizes revenue.
The Misreporter is trying to maximize the bidders’ utility, for the current auction selected by
Auctioneer, w .
This is achieved by minimizing the loss function L r ( ϕ, w ) wrt to ϕ (as discussed in
Sec 3.3).
The Auctioneer in turn maximizes expected revenue, for the current misreports as chosen by
Misreporter.
This is achieved by minimizing L m ( w, ϕ )
=
− ( √ P w + √ R w,ϕ
+ R w,ϕ with respect to w (as discussed in Sec 3.2).
Here, R w,ϕ is an estimate of the total regret that auctioneer computes for the current Misreporter ϕ , R w,ϕ
=
1 L (cid:80) L (cid:96) =
1 (cid:80) i ∈ N ( u w i ( (cid:126)v i , ([ M ϕ ( V )]
i , V − i )) − u w i
(cid:126)v i , ( (cid:126)v i , V − i ))) .
This game formulation can be summarized as follows:
  Misreporter: (cid:26) loss: L r ( ϕ, w ) parameter: ϕ
Auctioneer: (cid:26) loss: L m ( w, ϕ ) parameter: w (G)
Remark 1.
The game formulation (G) reminds us of Generative Adversarial Networks (Goodfellow et al., 2014).
Contrary to GANs, it is not a zero-sum game.
A RCHITECTURE AND TRAINING PROCEDURE
We describe ALGnet, a feed-forward architecture solving for the game formulation (G) and then provide a training procedure.
ALGnet consists in two modules that are the auctioneer’s module and the misreporter’s module.
These components take as input a bid matrix B =
( b i,j ) ∈ R n × m and are trained jointly.
Their outputs are used to compute the regret and revenue of the auction.
Notation.
We use MLP ( d in , n l , h, d out ) to refer to a fully-connected neural network with input dimension d in , output dimension d out and n l hidden layers of width h and tanh activation function.
sig denotes the sigmoid activation function.
Given a matrix B =
[ (cid:126)b 1 , . .
,(cid:126)b n ] (cid:62) ∈ R n × m , we deﬁne for a ﬁxed i ∈ N , the matrix B ( i ) :=
[ (cid:126)b i ,(cid:126)b 1 , . . .
,(cid:126)b i − 1 ,(cid:126)b i +1 , . . .
,(cid:126)b n ] .
4. T HE A UCTIONEER ’ S MODULE
It is composed of an allocation network that encodes a randomized allocation g w : R nm → [0 , 1] nm and a payment network that encodes a payment rule p w : R nm → R n .
Allocation network.
It computes the allocation probabily of item j to bidder i [ g w ( B )] ij as [ g w ( B )] ij =
[ f 1 ( B )] j · [ f 2 ( B )] ij where f 1 : R n × m → [0 , 1] m and f 2 : R n × m → [0 , 1] m × n are functions computed by two feed-forward neural networks.
– [ f 1 ( B )] j is the probability that object j ∈
M is allocated and is given by [ f 1 ( B )] j = sig ( MLP ( nm, n a , h a , n )) .
– [ f 2 ( B )] ij is the probability that item j ∈ M is allocated to bidder i ∈ N conditioned on object j being allocated.
A ﬁrst MLP computes l j :=
MLP ( nm, n a , h a , m )( B ( j ) ) for all j ∈ M .
The network then concatenates all these vectors l j into a matrix L ∈ R n × m .
A softmax activation function is ﬁnally applied to L to ensure feasibility i.e. for all j ∈ M, (cid:80) i ∈ N L ij = 1 .
Payment network.
It computes the payment [ p w ( B )]
i for bidder i as [ p w ( B )] i =
˜ p
i (cid:80) mj
=1 B ij [ g w ( B )] ij , where ˜ p : R n × m → [0 , 1] n .
˜ p
i is the fraction of bidder’s i utility that she has to pay to the mechanism.
We compute ˜ p
i = sig ( MLP ( nm, n p , h p , 1))
( B ( i ) ) .
Finally, notice that by construction [ p w ( B )]
i (cid:54) (cid:80) mj
=1 B ij g w ( B ) ij which ensures that (IR) is respected.
4. T HE M ISREPORTER ’ S MODULE
The module consists in an MLP ( nm, n M , h M , m ) followed by a projection layer Proj that ensure that the output of the network is in the domain D of the valuation.
For example when the valuations are restricted to [0 , 1] , we can take Proj = sig , if they are non negative number,we can take Proj
= SoftPlus .
The optimal misreport for bidder i is then given by Proj ◦
MLP ( nm, n M , h M , m )( B ( i ) )
∈
R m .
Stacking these vectors gives us the misreport matrix M ϕ ( B ) .
4. T RAINING PROCEDURE AND OPTIMIZATION
We optimize the game (G) over the space of neural networks parameters ( w, ϕ ) .
The algorithm is easy to implement (Alg. 1).
At each time t , we sample a batch of valuation proﬁles of size B .
The algorithm performs τ updates for the Misreporter’s network (line 9) and one update on the Auctioneer’s network (line 10).
Moreover, we often reinitialize the
Misreporter’s network every T init steps in the early phases of the training ( t (cid:54) T limit ).
This step is not necessary but we found empirically that it speeds up training.
1: Input : number of agents, number of objects.
2: Parameter : γ > 0; B, T, T init , T limit , τ ∈ N .
3: Initialize misreport’s and auctioneer’s nets.
4: for t = 1 , . . .
, T do 5: if t ≡ 0 mod T init and t < T Limit then : 6:
Reinitialize Misreport Network 7:
Sample valuation batch S of size B .
8: for s = 1 , . . .
, τ do 9: ϕ s +1 ← ϕ s − γ ∇ ϕ L r ( ϕ s , w t )( S ) .
10: w t +1 ← w t − γ ∇ w L m ( w t , ϕ )( S ) .
E XPERIMENTAL R ESULTS
We show that ALGnet can recover near-optimal auctions for settings where the optimal solution is known and that it can ﬁnd new auctions for settings where analytical solution are not known.
Since
RegretNet is already capable of discovering near optimal auctions, one cannot expect ALGnet to achieve signiﬁcantly higher optimal revenue than RegretNet.
The results obtained are competitive or better than the ones obtained in Duetting et al.
(2019) while requiring much less hyperparameters (Section 3).
We also evaluate ALGnet in online auctions and compare it to RegretNet.
For each experiment, we compute the total revenue rev :=
E V ∼ D [ (cid:80) i ∈
N p wi ( V )] and average regret rgt := 1 / n E V ∼ D [ (cid:80) i ∈ N r wi ( V )] on a test set of 10 , 000 valuation proﬁles.
We run each experimenttimes with different random seeds and report the average and standard deviation of these runs.
In our comparisons we make sure that ALGnet and RegretNet have similar sizes for fairness (Appendix D).
5. A UCTIONS WITH KNOWN AND UNKNOWN OPTIMA Known settings.
We show that ALGnet is capable of recovering near optimal auction in different well-studied auctions that have an analytical solution.
These are one bidder and two items auctions where the valuations of the two items v 1 and v 2 are independent.
We consider the following settings.
(A): v 1 and v 2 are i.i.d.
from U [0 , 1] , (B): v 1 ∼ U [4 , 16] and v 2 ∼ U [4 , 7] , (C): v 1 has density f 1 ( x ) = 5 / (1 + x )
and v 2 has density f 2 ( y ) =
6 / (1 + y ) 7 .
(A) is the celebrated Manelli-Vincent auction (Manelli & Vincent, 2006); (B) is a non-i.i.d.
auction and (C) is a non-i.i.d.
heavy-tail auction and both of them are studied in Daskalakis et al.
We compare our results to the theoretical optimal auction (Table 2).
(Duetting et al.
(2019) does not evaluate RegretNet on settings (B) & (C)).
During the training process, reg decreases to 0 while rev and P ∗ converge to the optimal revenue.
For (A), we also plot rev , rgt and P ∗ as function of the number of epochs and we compare it to RegretNet (Fig. 1).
Contrary to ALGnet, we observe that RegretNet overestimates the revenue in the early stages of training at the expense of a higher regret.
As a consequence, ALGnet learns the optimal auction faster than RegretNet while being schedule-free and requiring less hyperparameters.
[Table]
[Figure]
Unknown and large-scale auctions.
We now consider settings where the optimal auction is unknown.
We look at n -bidder m -item additive settings where the valuations are sampled i.i.d from
U [0 , 1] which we will denote by n × m .
In addition to "reasonable"-scale auctions ( 1 × 10 and 2 × 2 ), we investigate large-scale auctions ( 3 × 10 and 5 × 10 ) that are much more complex.
Only deep learning methods are able to solve them efﬁciently.
Table 3 shows that ALGnet is able to discover auctions that yield comparable or better results than RegretNet.
5. O NLINE AUCTIONS
ALGnet is an online algorithm with a time-independent loss function.
We would expect it to perform well in settings where the underlying distribution of the valuations changes over time.
We consider a one bidder and two items additive auction with valuations v 1 and v 2 sampled i.i.d from
U [0 , 1 + t ] where t in increased from 0 to 1 at a steady rate.
The optimal auction at time t has revenue
[Table]
55 × (1 + t ) .
We use ALGnet and two versions of RegretNet, the original ofﬂine version (Appendix
A) and our own online version (Appendix B) and plot rev ( t ) , rgt ( t ) and P ∗ ( t ) (Fig. 1).
The ofﬂine version learns from a ﬁxed dataset of valuations sampled at t =
(i.e. with V ∼ U [0 , 1] nm ) while the online versions (as ALGnet) learns from a stream of data at each time t .
Overall, ALGnet performs better than the other methods.
It learns an optimal auction faster at the initial (especially compared to
RegretNet Online) and keep adapting to the distributional shift (contrary to vanilla RegretNet).
C ONCLUSION
We identiﬁed two inefﬁciencies in previous approaches to deep auction design and propose solutions, building upon recent trends and results from machine learning (amortization) and theoretical auction design (stationary Lagrangian).
This resulted in a novel formulation of auction learning as a twoplayer game between an Auctioneer and a Misreporter and a new architecture ALGnet.
ALGnet requires signiﬁcantly fewer hyperparameters than previous Lagrangian approaches.
We demonstrated the effectiveness of ALGnet on a variety of examples by comparing it to the theoretical optimal auction when it is known, and to RegretNet when the optimal solution is not known.
Acknowledgements.
Jad Rahme would like to thank Ryan P. Adams for helpful discussions and feedback on the manuscript.
Samy Jelassi thanks Arthur Mensch for fruitful discussions on the subject and feedback on the manuscript.
The work of Jad Rahme was funded by a Princeton SEAS
Innovation Grant.
The work of Samy Jelassi is supported by the NSF CAREER CIF 1845360.
The work of S. Matthew Weinberg was supported by NSF CCF-
A T RAINING A LGORITHM FOR R EGRET N ET
We present the training algorithm for RegretNet, more details can be found in Duetting et al.
Algorithm 2 Training Algorithm.
1: Input : Minibatches S
, S of size B 2: Parameters : γ > 0 , η > 0 , c > 0 , R ∈ N , T ∈ N , T ρ ∈ N , T λ ∈ N .
3: Initialize Parameters : ρ 0 ∈ R , w 0 ∈ R d , λ 0 ∈ R n , 4: Initialize Misreports: v (cid:48) i ( (cid:96) )
∈
D i , ∀ (cid:96) ∈
[ B ] , i ∈ N.
5: 6: for t = 0 , . . .
, T do 7:
Receive minibatch S t =
{ V (1) , . .
, V ( B ) } .
8: for r = 0 , . . .
, R do 9: ∀ (cid:96) ∈
[ B ] , i ∈ n : v (cid:48) i ( (cid:96) )
← v (cid:48) i ( (cid:96) )
+ γ ∇ v (cid:48)
i u w t
i ( v i ( (cid:96) ) ; ( v (cid:48) i ( (cid:96) ) , V ( (cid:96) ) − i ))
Get Lagrangian gradient and update w t : 12: w t +1 ← w t − η ∇ w L ( w t ; λ t ; ρ t ) .
Update ρ once in T ρ iterations: 15: if t is a multiple of T ρ then 16: ρ t +1 ← ρ t + c 17: else 18: ρ t +1 ← ρ t 19: 20:
Update Lagrange multipliers once in T λ iterations: 21: if t is a multiple of T λ then 22: λ t +1 i ← λ ti + ρ t (cid:98) r i ( w t ) , ∀ i ∈ N 23: else 24: λ
← λ
B T RAINING ALGORITHM FOR O NLINE R EGRET N ET
We present an online version of the training algorithm for RegretNet, more details can be found in Duetting et al.
This version in mentionned in the orginal paper but the algorithm is not explicitly written there.
The following code is our own adaptation of the original RegretNet algorithm for online settings.
Algorithm 3 Training Algorithm.
1: Input : Valuation’s Distribution D 2: Parameters : γ > 0 , η > 0 , c > 0 , R ∈ N , T ∈ N , T ρ ∈ N , T λ ∈ N , B ∈ N 3: Initialize Parameters : ρ 0 ∈ R , w 0 ∈ R d , λ 0 ∈ R n , 4: for t = 0 , . .
, T do 5:
Sample minibatch S t = { V (1) , . .
, V ( B ) } from distribution D .
Initialize Misreports: v (cid:48) i ( (cid:96) )
∈
D i , ∀ (cid:96) ∈
[ B ] , i ∈ N.
7: 8: for r = 0 , . . .
, R do 9: ∀ (cid:96) ∈
[ B ] , i ∈ n : v (cid:48) i ( (cid:96) )
← v (cid:48) i ( (cid:96) )
+ γ ∇ v (cid:48)
i u w t
i ( v i ( (cid:96) ) ; ( v (cid:48) i ( (cid:96) ) , V ( (cid:96) ) − i ))
Get Lagrangian gradient and update w t : 12: w t +1 ← w t − η ∇ w L ( w t ; λ t ; ρ t ) .
Update ρ once in T ρ iterations: 15: if t is a multiple of T ρ then 16: ρ t +1 ← ρ t + c 17: else 18: ρ t +1 ← ρ t 19: 20:
Update Lagrange multipliers once in T λ iterations: 21: if t is a multiple of T λ then 22: λ
[Equation]
+1 i ←
[Equation]
i
[Equation]
w t
[Equation]
, ∀ i
[Equation]
N 23: else 24: λ t +1 ← λ t
C P ROOF OF P ROP .
Lemma 1.
Let M be a one bidder m item mechanism with expected revenue P and expected regret
R , then ∀ ε > 0 , there exists a mechanism M (cid:48) with expected revenue P (cid:48) =
(1 − ε ) P − 1 − εε R and zero expected regret, R (cid:48)
= 0 .
Proof.
For every valuation vector v ∈ D , let g ( v ) and p ( v ) denote the allocation vector and price that M assigns to v .
We now consider the mechanism M (cid:48) that does the following:
• g (cid:48) ( v ) =
g ( v (cid:48) ) • p (cid:48)
( v ) =
(1 − ε ) p ( v (cid:48) )
Where v (cid:48) is given by : v (cid:48)
= argmax ˜ v ∈ D (cid:104) v , g (˜ v ) (cid:105) − (1 − ε ) p (˜ v ) .
By construction, the mechanism
M (cid:48) has zero regret, all we have to do now is bound its revenue.
If we denote by R ( v ) the regret of the proﬁle v in the mechanism M , R ( v ) = max ˜ v ∈ D (cid:104) v , g (˜ v ) − g ( v ) (cid:105) − ( p (˜ v ) − p ( v ))
we have.
[Equation]
p
[Equation]
v ) + R ( v )
Which we will write as:
[Equation]
Second, we have by construction:
[Equation]
By summing these two relations we ﬁnd :
[Equation]
Finally we get that:
[Equation]
Taking the expectation we get:
[Equation]
R
Proposition 1.
Let M be an additive auction with 1 bidders and m items.
Let P and R denote the total expected revenue and regret,
P = E V ∈ D [ p ( V )] and R = E V ∈ D [ r ( V )] .
There exists a mechanism M ∗ with expected revenue P ∗ = (cid:16) √ P − √ R (cid:17) 2 and zero regret R ∗ = 0 .
Proof.
From Lemma 1 we know that ∀ ε > 0 , we can ﬁnd a zero regret mechanism with revenue
P (cid:48)
= (1 − ε ) P − 1 − εε R .
By optimizing over ε we ﬁnd that the best mechanism is the one correspond to ε = (cid:113) RP .
The resulting optimal revenue is given by:
[Equation]
D I MPLEMENTATION AND S ETUP
We implemented ALGnet in PyTorch and all our experiments can be run on Google’s Colab plateform (with GPU).
In Alg. 1, we used batches of valuation proﬁles of size B ∈ { 500 } and set T ∈ { 160000 , 240000 } , T limit ∈ { 40000 , 60000 } , T init ∈ { 800 , 1600 } and τ ∈ { 100 } .
We used the AdamW optimizer (Loshchilov & Hutter, 2017) to train the Auctioneer’s and the
Misreporter’s networks with learning rate γ ∈ { 0 . 0005 , 0 . 001 } .
Typical values for the architecture’s parameters are n a = n p = n m ∈ [3 , 7] and h p = h n = h m ∈ { 50 , 100 , 200 } .
These networks are similar in size to the ones used for RegretNet in Duetting et al.
For each experiment, we compute the total revenue rev :=
E V ∼ D [ (cid:80) i ∈
N p wi ( V )] and average regret rgt := 1 / n E V ∼ D [ (cid:80) i ∈ N r wi ( V )] using a test set of 10 , 000 valuation proﬁles.
We run each experiment 5 times with different random seeds and report the average and standard deviation of these runs.