Optimal-er Auctions through Attention
Abstract
RegretNet is a recent breakthrough in the automated design of revenue-maximizing auctions.
It combines the flexibility of deep learning with the regret-based approach to relax the Incentive Compatibility (IC) constraint (that participants prefer to bid truthfully) in order to approximate optimal auctions.
We propose two independent improvements of RegretNet.
The first is a neural architecture denoted as RegretFormer that is based on attention layers.
The second is a loss function that requires explicit specification of an acceptable IC violation denoted as regret budget.
We investigate both modifications in an extensive experimental study that includes settings with constant and inconstant number of items and participants, as well as novel validation procedures tailored to regret-based approaches.
We find that
RegretFormer consistently outperforms RegretNet in revenue (i.e. is optimal-er ) and that our loss function both simplifies hyperparameter tuning and allows to unambiguously control the revenue-regret trade-off by selecting the regret budget.
Introduction
Several decades ago, Myerson [45] has proposed a general solution for the problem of optimal (revenue-maximizing) design of single-item auctions.
Despite the collaborative effort of the community, the results of the same generality for the multi-item auctions have not been obtained.
As an alternative to analytical solutions, automated auction design [7, 8] takes a perspective of constrained optimization.
In particular, the objective is to maximize the revenue while satisfying the Incentive
Compatibility (IC) and Individual Rationality (IR) constraints.
This framework can provide approximate solutions in settings where the optimal mechanisms are unknown, as well as hint at what the analytical solutions may look like.
The early approaches had utilized linear programming and classic machine learning.
However, after the renaissance of deep learning, combining auction design with neural networks was only a question of time.
Dütting et al.
[17] are the pioneers of this approach with their RegretNet architecture.
RegretNet is a neural network that represents a mechanism by mapping a matrix of bids that n participants report for m items into a probabilistic allocation matrix and a payment vector.
It is trained via differential optimization on a mixture of two objectives: to maximize revenue and to minimize regret, which is a relaxation of the IC constraint proposed in [15].
RegretNet recovers near-optimal revenue in the analytically solved settings and outperforms the previous state-of-the-art while having vanishing regret guarantees.
In this paper, we propose two independent improvements of RegretNet.
First, we propose a neural architecture based on attention layers.
We name it RegretFormer after the widely-known Transformers [64].
An illustration of the architecture is provided in Figure 1.
RegretFormer has several crucial properties.
On the one hand, it is by design insensitive to the order of items and participants, which is desirable for learning symmetric auctions.
Note that it can also be modified to learn asymmetric mechanisms.
On the other hand, it does not assume a predetermined constant input size, which allows one network to learn from data consisting of auctions of varying sizes, as well as to generalize to settings unseen during training.
Second, we propose an alternative objective.
Instead of optimizing arbitrary mixtures of revenue and regret, our approach maximizes revenue given an explicitly specified regret budget, which is an acceptable violation of the IC constraint.
This is technically implemented through the method of Lagrange multipliers and dual gradient decent.
Our approach allows to unambiguously balance revenue and regret by varying the regret budget.
As a bonus, we find that our loss function is significantly less sensitive to the choice of loss-related hyperparameters.
We conduct an extensive experimental study to verify the strengths of our modifications.
Specifically, we compare our and existing architectures in both symmetric and asymmetric settings with constant input size, as well as in novel settings with inconstant input size.
We find that RegretFormer consistently outperforms existing architectures in revenue given the same regret budget.
We additionally verify this result with two novel validation procedures.
The first procedure is based on estimating the regret of one network given the optimal misreports approximated by another network.
The second procedure is based on network distillation.
Regarding our loss modification, we confirm that specifying the regret budget results in the intended revenue-regret trades-off.
Background 2.1 Problem statement
Let N = { 1 , ..., n } be a set of n bidders, M = { 1 , ..., m } be a set of m items.
Each bidder i evaluates all items with valuation function v i : 2 M → R .
In our study, we only consider additive valuations, meaning that for each item j ∈ M a bidder has a valuation function v i ( j ) , and a valuation of a subset
S ⊆
M equals to the sum of valuations of the items in the subset v i ( S ) =
Σ j ∈ S v i ( j ) .
The valuation function can also be extended to evaluate probabilistic assignments: v i ( z ij ) =
z ij v i ( j ) , where z ij is the probability that participant
i is assigned item j .
The valuation function v
i is drawn independently from the distribution F i over all possible valuation functions V i .
A set of valuations of all bidders forms a valuation profile v = ( v 1 , ..., v n ) .
An auctioneer does not know the bidders’ valuations nor their distribution F = ( F 1 , ..., F n ) , but has a sample L
= ( v 1 , ..., v | L | ) of valuation profiles drawn from F .
The bidders independently report their bids b
= ( b 1 , ..., b n ) ∈ V , based on which the auction allocates the items and charges payments.
Formally, the auction is defined as a tuple ( g = ( g 1 , ..., g n ) , p = ( p 1 , ..., p n )) of probabilistic allocation rules
g i : V → [0 , 1] M and payment rules p
i : V → R ≥ .
Define the bidder’s utility as u i ( v i , b ) =
v i (
g i ( b )) − p
i ( b ) .
Define the valuation profile without the valuation v i as v − i .
Similarly, define the bids without b i as b − i , and the valuation profile of all bidders except i as V − i =
Π k ̸
= i V k .
An auction is dominant strategy incentive compatible (DSIC) if the utility of each bidder is maximized by reporting their true valuations regardless of bids of other bidders: ∀ i, v, b : u i ( v i , v ) ≥ u i ( v i , ( b i , v − i )) .
An auction is ex-post individually rational (IR) if each agent’s utility is non-negative for all possible valuations and bids: ∀ i, v, b : u i ( v i , ( b i , v − i )) ⩾ 0 .
Define the revenue of a profile as a sum of bidders’ payments Σ i p
i ( v ) .
The goal of the optimal auction design is to maximize the expected revenue subject to DSIC and IR constraints.
The problem is analytically solved for the auctions with one item in the seminal work of Myerson [45], provided the valuation distributions are known.
There are no general solutions for auctions with m > 1 .
The task of optimal auction design can be viewed as an optimization or a machine learning problem.
In this case, the expected revenue takes the place of the objective.
There is a class of auctions ( g ( w ) , p ( w ))
∈ M parameterized with w ∈ R d for some d ∈ N and a set of bidder valuation profiles L drawn as independent and identically distributed variables from F .
Then, the goal of the optimization procedure is to find an auction that minimizes the negated expected revenue − E F [Σ i ∈
N p
i ( v ; w )] while satisfying DSIC and IR constraints.
2.2 RegretNet
RegretNet was proposed by Dütting et al.
[17] as a deep learning solution for the optimal auction design.
It can be used for multi-bidder and multi-item settings, the analytical solutions for which are unknown.
RegretNet explicitly encodes the allocation and payment rules of the auction mechanism into weights of a neural network.
These rules are discovered as a result of optimization procedure.
The architecture of RegretNet consists of two networks: the allocation network and the payment network.
Both networks take flattened constant-sized bid matrix B nm as an input and process it through several fully-connected layers.
The allocation network maps a matrix of bids to categorical distributions of allocations between participants for each item:
A net ( B nm ) =
Z nm , where z ij is the probability of allocating the j -th item to the i -th bidder.
To allow for the possibility of an item remaining unallocated, the output of the final layer has the size of ( n + 1) · m , imitating an additional dummy participant with zero valuations.
Then, to obtain properly scaled allocation probabilities over participants for each item, the softmax activation is applied to the output of the final layer corresponding to each item.
Specifically, z
[Equation]
n +1 i =1 e ˜ z ij , where ˜ z ij denotes the unnormalized allocation probability (i.e. logit).
The payment network maps a matrix of bids to a vector of pay values: P net ( B nm )
=
ˆ P n , where ˆ p i is the fraction of the i -th bidder expected utility that the bidder transfers to the auctioneer.
The payment is then computed as p i
=
ˆ p
i (cid:80) m j =1 z ij b ij for i = 1 , . . . , n .
To properly scale ˆ p i ∈ [0 , 1] , a sigmoid activation is applied to the output of the final layer.
Because the payment cannot exceed the bidder’s expected utility, the mechanisms discovered by RegretNet always satisfy the IR constraint.
Given a sample of profiles L , RegretNet aims to optimize the following empirical objective:
[Equation]
where rgt i denotes the i -th bidder’s ex-post regret.
Regret is a quantifiable relaxation of the DSIC constraint first introduced in [15].
It is defined as the difference between the i -th bidder’s utilities given the optimal misreport (that provides the highest utility) and the truthful bid:
[Equation] [Equation]
To solve the constrained optimization problem 1 over the space of the network parameters w , the authors of RegretNet employ the augmented Lagrangian method.
This yields a loss function that combines revenue maximization with a penalty for violations of DSIC constraints:
[Equation]
where B denotes mini-batch, P i = 1 | B | (cid:80) l ∈ B p
i ( v l ; w ) is the average revenue from the i -th participant, (cid:101) R i =
| B | (cid:80) l ∈ B (cid:102)
rgt
i ( v ′ li , v l ; w ) is the average approximate regret of the i -th participant, and (cid:102)
rgt i ( v ′ li , v l ; w ) = (cid:0) u ( v li , ( v ′ i , v l − i ); w ) − u ( v li , v l ; w ) (cid:1) is an approximation of rgt i .
To find the approximate regret, the optimal misreports v ′ li are estimated via gradient descent in the inner optimization loop by minimizing the i -th participant negated utility as a function of their misreport:
L inner
[Equation]
The Lagrange multipliers λ i and ρ are periodically updated throughout the training according to the schedules: λ i ← λ
i + ρ (cid:101) R i and ρ ← ρ + ρ ∆ .
The starting values of λ i and ρ and the learning rate ρ ∆ are hyperparameters that control the trade-off between revenue and regret of the learned mechanism.
2.3 EquivariantNet
EquivariantNet is an alternative architecture proposed by Rahme et al.
[54] to effectively deal with symmetric auctions.
It relies on a theorem originally proven by [10] that there exists an optimal solution for a symmetric auction that is permutation-equivariant, i.e. insensitive to the order of inputs.
Like RegretNet, EquivariantNet has an allocation network and a payment network.
Both networks consist of compositions of exchangeable layers that preserve permutation-equivariance [25].
The exchangeable layer adds to each element i, j of a matrix an aggregate statistic (like sum) over the row i , the column j , and the whole matrix, weighted by learnable parameters that are shared among all matrix elements.
The precise definition is provided in the Appendix.
The authors observe that
EquiavariantNet produces competitive results but does not outperform RegretNet in revenue.
2.4 Related work
Early work that explores the automated solutions to auction design formulates the problem as linear program [7, 57, 8] or searches within specific families of DSIC auctions [38, 58].
The former approach suffers from scalability issues due to the curse of dimensionality [24], while the latter approach searches within a limited family of auctions that might not contain the optimal mechanism.
Classic machine learning has also been applied to the auction design [37, 15, 47], but these approaches are considered less flexible and general than the deep learning alternatives [17].
In recent years, multiple extensions of RegretNet have been proposed, including the extensions to the optimal auction design with budget constraints [18], fairness constraints [36], and human preferences over desirable allocations [52], as well as the problem of faculty allocation [19] and the matching problem [56].
Shen et al.
[59] and Dütting et al.
[17] propose alternatives to RegretNet that are exactly DSIC but that are only applicable to auctions with one bidder.
Curry et al.
[9] combine the approach of searching within limited families of DSIC auctions with deep learning.
Rahme et al.
[55] propose a simplified loss function for RegretNet that is easier to tune, as well as a potentially faster regret estimation procedure based on training an adversarial bidder networks.
Deep learning has been applied to other aspects of mechanism design, such as iterative combinatorial auctions [67], minimizing economic burden on bidders within the Groves family of auctions [62], optimal bidding strategies [49, 48], optimal redistribution mechanisms [41], and E-commerce advertising [39].
Several studies have combined reinforcement learning with auction design [63, 30, 50].
A plethora of research has also focused on the questions of sample complexity of designing revenue-maximizing auctions [6, 44, 13, 3, 43, 42, 46, 61, 20, 29, 26, 23, 21].
Our modifications of RegretNet
In this section, we describe our two modifications of RegretNet.
The first modification is a novel neural architecture for the optimal auction design, which we denote as RegretFormer.
The second modification is an alternative constrained objective that yields a more convenient loss function.
3.1 RegretFormer: enhancing RegretNet with attention layers
The neural architecture of RegretNet has several issues.
One issue is the sensitivity of RegretNet to the order of items and participants in the input bid matrix.
Such order should not affect the outcome of a symmetric auction [10, 53]. Even in non-symmetric auctions, permutation-equivariant solutions sometimes achieve near-optimal revenue [27, 1, 31, 32, 35].
This makes the option to wire equivariance to input order into the architecture desirable.
Another issue is that RegretNet assumes a constant number of participants and items.
This severely limits its practical applicability due to inability to generalize to unseen input sizes and learn from data with heterogenous input size.
Finally, the fully-connected layers used in RegretNet have limited expressivity and may not have the right inductive biases for auction design.
Taking these issues into account, we propose RegretFormer .
The architecture of RegretFormer is illustrated in Figure 1.
The network takes the matrix of bids B nm as the input.
First, we apply an exchangeable layer [25] to transform each bid into an initial vector of features that contains information about other bids.
Then, we apply item-wise (i.e. row-wise) and participant-wise (i.e. column-wise) self-attention layers [64] to each feature vector corresponding to each bid.
For a given bid, the outputs of the two self-attention layers are transformed into a single feature vector through a fully-connected layer.
These self-attention transformations can be applied several times.
Finally, by averaging over rows and columns we transform the output of self-attention layers into the allocation matrix Z nm and the payment vector P n .
Note that each layer shares the parameters across all bids to ensure that the network is insensitive to the order of bids and is applicable to different input sizes.
The detailed definitions and order of all layers are provided in the Appendix.
Our architecture has several advantages.
On the one hand, it leverages the expressivity of attention layers that help to achieve the state-of-the-art performance in many diverse and complex problems, e.g. [64, 12, 68].
We empirically demonstrate superiority of RegretFormer in Section 4.
On the other hand, it maintains the equivariance and the invariance to the order of items and participants.
The former forces the resulting mechanisms to be symmetric, which drastically reduces the solution search space.
The latter allows RegretFormer to learn from data with inconstant input sizes, as well as to generalize to unseen settings.
Note that RegretFormer can still learn asymmetric mechanisms by utilizing positional encoding, which we demonstrate empirically.
We note that a concurrent work by Duan et al.
[14] also combines Transformers and RegretNet.
However, their focus is on integrating contextual information of bidders and items into the framework, whereas we perform a wider and more accurate comparison of neural architectures.
3.2 Specifying regret budget
Both RegretNet and EquivariantNet optimize a mixture of two conflicting objectives (3), namely revenue maximization and regret minimization, and control their trade-off with hyperparameters like initial values and schedules of the Lagrange multipliers.
There are two issues with this approach.
The first issue is sensitivity.
These hyperparameters have to be precisely tuned for each experiment, and as Rahme et al.
[55] show, may massively degrade the performance if selected improperly.
The second issue is uninterpretability.
While increasing any of λ i , ρ, ρ ∆ tightens the regret budget of the learned mechanism, the exact effect of such changes on resulting revenue and regret is unpredictable.
Furthermore, the recipe for hyperparameters selection in new settings is unclear.
Rahme et al.
[55] propose a mixture of objectives that mitigates the first issue, but the second issue remains unresolved.
We propose an alternative perspective that mitigates both issues.
Instead of balancing the two objectives, we maximize the revenue given a maximal regret budget R max , which is pre-specified by the designer.
This corresponds to a relaxed version of the constrained objective (1):
[Equation]
Instead of the constrained objective (5), we optimize its dual by introducing the Lagrange multiplier γ (note that R max does not depend on w and thus can be dropped):
[Equation]
Critically, unlike the original RegretNet we do not hand-select the Lagrange multiplier.
Instead, γ is dynamically updated to enforce the constraint satisfaction while exhausting all available regret budget R max .
To this end, we employ dual gradient descent [4].
Specifically, we iterate between one gradient update of the network parameters w to minimize (6) and one update of γ according to:
[Equation]
where γ ∆ is the learning rate for the dual variable.
For convenience, in (7) we normalize the regret estimate by revenue.
This way, R max ∈ [0 , 1] specifies the regret budget as a percentage of revenue.
We also apply logarithms to both terms of the difference in (7) for faster convergence of γ .
Additionally, we apply a decreasing schedule to R max .
If the regret budget R max is too tight at the beginning of training, the network may fail to escape the local optima of low revenue and zero regret.
To avoid that, we initialize R max at a higher value R startmax and exponentially anneal it to the desirable budget R endmax throughout the training.
This leads the network to first find the solutions with high revenue and then tighten the regret.
This results in the following update rule for the regret budget:
[Equation]
where R multmax < 1 controls the speed of annealing of R max to R endmax .
Another example of applying dual gradient descent to enforce a constraint can be found in [51].
The proposed approach has several advantages.
On the one hand, it resolves the dichotomy of two conflicting objectives.
While the regret budget needs to be explicitly set based on the designer’s preferences, all other hyperparameters are then straightforward to tune by maximizing the revenue given the specified regret budget.
This is unlike the original objective of RegretNet (3) where the regret budget is chosen implicitly through specifying uninterpretable loss-related hyperparameters like λ i , ρ , ρ ∆ .
On the other hand, our approach is also significantly less sensitive to the loss-related hyperparameters.
We empirically demonstrate this by using the same hyperparameters related to the budget R max and its schedule in all our experiments, in contrast to the objective (3) that requires a uniquely tuned set of loss-related hyperparameters to perform well in a given setting [55].
Experiments
In this section, we empirically investigate our modifications of RegretNet.
We compare our RegretFormer with RegretNet and EquivariantNet in settings with constant number of participants and items used by Dütting et al.
[17], as well as in novel settings with inconstant input sizes that we denote as multi-settings.
All three networks are trained given the same regret budgets using our approach from
Section 3.2.
Additionally, we vary regret budgets and further investigate the differences between the architectures using novel validation procedures.
In all experiments, the valuations of all participants are additive and are independently drawn for each item from the Uniform distribution: v i ( j ) ∈ U [0 , 1] .
Training details, hyperparameters, and additional results are reported in the Appendix.
4.1 Comparison of architectures under constant input sizes
The settings only differ in the number of participants n and items m , so we denote them as n x m .
We conduct experiments in five settings: {1x2, 2x2, 2x3, 2x5, 3x10}.
The 1x2 setting is the celebrated
Manelli-Vincent auction, the analytical solution for which is provided in [40].
The optimal revenue for this auction equals 0 .
For the rest of the settings, the analytical solutions are unknown.
We compare our and existing neural architectures in the five settings given two different regret budgets R max ∈ { 10 − 3 , 10 − 4 } .
We additionally include the classic VCG [65, 5, 22] and Myerson [45] mechanisms for comparison, both of which are DSIC and ex-post IR.
The Myerson auctions are optimal for m = 1 .
We report its two extensions to multi-good settings: the ‘item-wise‘ where each item is sold separately and the ‘bundled‘ where all items are sold as a single bundle.
We report the results in Table 1.
Additionally, we report the learning curves in the Appendix.
In all settings but 1x2 and given both regret budgets, RegretFormer outperforms both RegretNet and EquivariantNet in revenue.
The performance gap is absent in the simplest setting but becomes
[Table] [Table]
especially prominent in the larger settings.
While permutation-equivariance of RegretFormer likely plays a role, it cannot fully explain the results since EquivarintNet also has this property.
We therefore attribute the superiority of RegretFormer to the expressivity of attention layers.
Additionally, in Figure 2, we provide solutions found by RegretNet and RegretFormer in the 1x setting.
Both networks find allocation probabilities that smoothly approximate the optimal solution.
In the bigger settings, EquivariantNet can also outperform RegretNet.
This result is somewhat surprising since it was not observed by the authors of EquivariantNet, but it can simply be explained by the fact that they do not test the architecture in settings that are complex enough.
Given the smallest regret budget, both baselines sometimes underperform Myerson.
While RegretFormer outperforms Myerson in revenue, its regret is still non-zero.
This highlights a potential
[Figure]
problem of the regret-based approach: when any violations of DSIC are highly undesirable, a better revenue might be achieved with classic mechanisms that are also guaranteed to be DSIC.
A potential downside of RegretFormer compared to the baselines is that it takes longer to train and requires more parameters to reach optimal performance.
We elaborate on this in the Appendix.
4.2 Does regret budget function as indented?
The core feature of our approach from Section 3.2 is that the resulting regret should be uniquely and unambiguously specified through the regret budget.
In particular, the total regret normalized by the total revenue should equal the regret budget.
To verify this quantitatively, we present the ratios of the estimated normalized total regret to the regret budget in Table 2.
A ratio being higher than 1 means that the regret budget is exceeded, and vice versa.
Below we summarize our findings.
During training, the ratios are indeed close to 1 for all three architectures, meaning that our approach functions as intended.
Additionally, we report the ratios estimated during validation, which involves more gradient descent steps to find optimal misreports than during training, making the regret estimates more precise.
Unsurprisingly, these are usually than the ratios estimated during training.
For a tighter regret approximation and a better match with the budget, one could increase the number of the inner optimization steps during training, provided that a longer training time is acceptable.
4.3 Is the performance gap genuine?
In Table 1, we have observed that RegretFormer outperforms both baselines and have attributed its success to the expressivity of attention layers.
However, there is an alternative explanation.
Recall that regret is approximated in the inner optimization (4) by repeatedly back-propagating through the whole neural network w .
It follows that regret approximation depends on the neural architecture, including the number, the types, the size, and the order of layers.
Due to absence of a regret approximation procedure that is identical for all networks, there is no guarantee that similar regret estimates between different neural architectures correspond to actual similar regret values, i.e. for different architectures w 1 , w 2 : ˜ R ( w 1 )
≈ ˜ R ( w 2 ) ̸ = ⇒ R ( w 1 )
≈ R ( w 2 ) , where R ( w ) is the expected regret and ˜ R ( w ) is its approximation.
In particular, we are concerned whether RegretFormer achieves higher revenues due to approximating the regret worse, rather than due to maximizing the revenue better.
We design two procedures to test this hypothesis.
[Table] [Table]
The first procedure is denoted as ‘cross-misreports‘.
It is based on estimating the regret of one network on the optimal misreports approximated by another network: (cid:101) R crossi ( w 1 , w 2 ) =|
B | (cid:80) l ∈ B (cid:102) rgt
i ( v ′ li ( w 2 ) , v l ; w 1 ) .
We apply this procedure to the trained networks in all five settings given R max = 10 − 3 .
If all networks approximate regret equally well, we expect the regret estimated on cross-misreports to not exceed the normally computed regret.
However, if the regret estimates of RegretFormer were higher on the misreports estimated by RegretNet or EquivariantNet, this would point towards poor regret approximation by RegretFormer.
We report the results in Table 3.
We do not find any evidence that RegretFormer underestimates the regret: in all settings, it finds misreports that produce regret either higher than or within a standard deviation from the misreports of the other two architectures.
The second procedure is based on network distillation.
The results are similar to Table 3 in that
RegretFormer does not appear to underestimate the regret, so we report them in the Appendix.
4.4 Learning asymmetric mechanisms using positional encoding
To handle asymmetric distributions, our architecture can be augmented with Positional Encoding (PE) - a common technique for transformers to incorporate information about order of elements.
To demonstrate this, we study a setting with one bidder, two items, and non-identically distributed valuations over items, where v 1 ∼ U [4 , 16] and v 2 ∼ U [4 , 7] .
The optimal mechanism is given by
Daskalakis et al.
We apply RegretNet, RegretFormer without PE, and RegretFormer with PE given R max = 10 − 3 .
As PE, we use a common technique that adds arbitrary numbers from [-1, 1] to the input (estimated as sine and cosine functions of position-dependent frequencies) proposed by
Vaswani et al.
Note that this PE has no learnable parameters.
The results are reported in the Table 4.
While RegretFormer with PE unsurprisingly performs on par with RegretNet, RegretFormer without PE learns a symmetric solution that is not much worse.
In
Figure 2, we provide illustrations of solutions found by the two versions of RegretFormer.
These show that RegretFormer with PE closely approximates the allocation probabilities of the optimal
[Table]
mechanism, whereas RegretFormer without PE finds a symmetric solution that resembles some middle ground between the optimal allocation probabilities for the two items.
4.5 Comparison of architectures in multi-settings
In practice, it may be desirable for a single network to be applicable to multiple input sizes, e.g. to save computation or due to limited data.
To test our network in such cases, we introduce multi-setting .
A multi-setting is a uniform mixture of constant-sized settings studied so far.
In our experiments, we mix the following settings: {2x3, 2x4, 2x5, 2x6, 2x7, 3x3, 3x4, 3x5, 3x6, 3x7}.
We set R max = 10 − 3 .
To adapt RegretNet to multi-settings, we fix the input size for the maximum possible number of items and participants in a multi-setting and pad unused valuations with zeros.
The results are presented in Table 5.
Comparing RegretFormer with RegretNet, our architecture consistently achieves higher revenue for the same regret budget.
While EquivariantNet achieves even higher revenue, after applying our cross-misreport validation procedure we find that EquivariantNet fails to minimize regret due to poorly approximating optimal misreports.
Full results of crossmisreport validation, as well as out-of-setting experiments, are reported in the Appendix.
Conclusion
In this study, we achieve new state-of-the-art in the application of deep learning to optimal auction design.
Our RegretFormer leverages the recent advances in deep learning to unlock the full potential of the regret-based optimization while enforcing the equivariance and the invariance to the order of participants and items.
We test the effectiveness of RegretFormer in multiple experimental settings and find that our network consistently outperforms the existing analogues.
In addition, we rethink the objective formulation of RegretNet.
The resulting loss function disentangles balancing the revenueregret trade-off and optimizing the performance, as well as reduces the burden of hyperparameter tuning.
Finally, we suggest two validation procedures for comparing regret-based approaches that may find use in future studies.
Acknowledgments
All authors .
We sincerely thank Xeniya Adayeva for creating the illustration of RegretFormer 1.
You can contact Xeniya
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See the end of Section 4.1.
(c) Did you discuss any potential negative societal impacts of your work?
[No] We do not foresee such impact.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] We include the code as a URL.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See the Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] In Table 1, we report aggregated standard deviations.
In the Appendix, we report the learning curves with the error bars.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See the Appendix.
A Layers and architecture of RegretFormer
Multi-head attention Popularized by Vaswani et al.
[64], the attention function maps a query vector and a set of key-value vector pairs to an output vector.
The procedure is typically applied to a set or a sequence of queries.
The output vector is a weighted sum of the values, and each weight reflects the compatibility of the query with the corresponding key.
While different attention mechanisms exist, the softmax attention is the most common:
[Equation]
where Q , K , and V are respectively the matrices of queries, keys, and values, and d k is the number of keys.
Self-attention is a special case of attention in which Q , K , and V are linear projections of the same inputs.
Typically, layer normalization is applied to the input before projecting [2].
Vaswani et al.
[64] also propose multi-head attention (MHA).
In this extension, H different attention heads are created, and for each attention head the input matrices are projected with head-specific weight matrices
[Equation]
[Equation] [Equation]
to calculate the inputs to attention:
[Equation]
Attention is equivariant to the order of elements in the input, which is a useful property when learning symmetric auctions since any optimal symmetric auction can be represented by a permutationequivariant function [10].
In applications where this order is important (e.g. order of words in a sentence) Positional Encoding (PE) is usually applied.
This technique augments the initial representation of the input data with the information about the order of the elements.
We demonstrate how PE can be applied to learn optimal asymmetric auctions in Section 4.4.
Exchangeable Layers
The exchangeable layer [25] is inspired by deep sets [69] and is defined as follows.
A layer is specified by the number of input channels K , the number of output channels O and five learnable parameters w 1 , w 2 , w 3 , w 4 ∈ R K
x
O and w 5 ∈ R O .
The input is a tensor B of size (K, n, m) and the output is tensor Y of size (O, n, m) .
The element (i, j) of the o -th output channel
Y ( o ) i,j is given by:
[Equation]
This layer constitutes the main building block of EquivariantNet [54] and is also used as the first layer of RegretFormer.
Architecture of RegretFormer There are several high-level differences between the architectures of RegretFormer and RegretNet.
Whereas RegretNet uses two separate networks to calculate allocations and payments, our architecture has a single shared network with both outputs.
Unlike RegretNet where the input is flattened into a vector, the input of RegretFormer is the two-dimensional matrix of bids B nm .
Furthermore, unlike RegretNet, n and m are not fixed in RegretFormer.
We now describe the architecture in detail.
Note that each described layer except the output layers is followed by a Tanh activation.
First, we apply an exchangeable layer (12) to transform each bid into an initial vector of features that already contains information about other bids.
According to our definition of the exchangeable layer, this requires adding third dummy dimension to the bid matrix B nm → B nm 1 :
[Equation]
Then, we sequentially apply several attention-based blocks.
Each block consists of two multi-head self-attention layers with residual connections, one applied item-wise and one applied participantwise.
For each layer, we accordingly reshape the input.
After applying the attention layers, we concatenate their predictions and apply the same fully-connected layer (FC) to the feature vectors of each bid (to reduce the dimensionality of the feature vectors to the initial size):
[Equation]
After applying the attention-based block N times (in our experiments, we set N to 1 or 2), we obtain the attended feature matrix L nmkN +1 .
From this matrix, we create two separate matrices by averaging over one of the dimensions: the participant feature matrix P nkN +
= 1 m (cid:80) j
( L njkN +1 ) and the item feature matrix I mkN +
= 1 n (cid:80) i
( L imkN +1 ) .
These matrices are essentially embeddings of participants and items respectively and are used to compute the allocation matrix and the payment vector.
To compute the allocation matrix, we multiply the item and the participant matrices, which gives us an n by m matrix of unscaled probabilities (logits).
Before scaling, we need to additionally consider the possibility of each item remaining unallocated.
To this end, we introduce a dummy participant n + 1 , the unscaled probability for which is estimated for each item as a negated sum of the unscaled probabilities over the real participants.
Finally, we apply the softmax function along the participants to scale the probabilities.
When summarized, the allocation matrix is obtained the following way:
[Equation]
To estimate the payment vector, we average the participant feature matrix over the feature dimension and apply the sigmoid activation to scale the output between 0 and 1:
[Equation]
Like in RegretNet, we calculate the final payments as p i
= ˆ p
i
[Equation]
m j =
z ij b ij for i = 1 , . . . , n .
B Technical details and hyperparameters
In all experiments, all networks are trained for 200000 iterations of outer optimization, each iteration corresponding to one step of the optimizer on one mini-batch.
The training dataset consists of 640000 profiles (same as in the RegretNet paper) divided into 1250 mini-batches of 512 profiles.
The validation dataset consists of 4096 profiles divided into 128 batches of 32 profiles.
The number of inner optimization steps per one outer update equals 50 during training and 1000 during validation.
The learning rate equals 0 .
001 for the outer optimization and 0 .
1 for the inner optimization.
Both use separate Adam optimizers [33].
The hyperparameters related to the neural architectures are reported in Table 6.
We report the sizes of neural networks in Table 12.
All experiments are repeated three times and the average metrics are reported.
Experiments are run on an internal cluster with V
GPUs.
The hyperparameters related to our budget-based approach are the following.
In all our experiments, we initialize γ = 1 , set γ ∆ = 0 .
5 , set R startmax = 0 . 01 , and set such R multmax that R max converges to
R endmax in two-thirds of the training time.
We set R endmax = 0 .
001 by default but additionally investigate the effect of choosing a lower budget R endmax = 0 . 0001 .
[Table]
C Additional results C.1 Learning curves
We present the learning curves of the revenue, the regret, and the penalty coefficient γ for settings {1x2, 2x2, 2x3, 2x5, 3x10} in Figures 3, 4, 5, 6, and 7, respectively.
The results of the same experiments are reported in Table 1 in the main text.
Note that the shapes of learning curves are a consequence of our regret budget schedule.
Specifically, we provide a higher regret budget at the beginning of the training so the network finds a solution with high revenue (that does not satisfy the desirable budget), and then we tighten the regret budget (which also causes the revenue to decrease).
It can also be seen that both revenue and regret start flattening at the same time as the regret penalty coefficient γ stops increasing.
This happens approximately at 2/ of the training time, in accordance with our choice of R multmax .
C.2 Network distillation
Here we elaborate on our validation procedure based on network distillation that was mentioned in
Section 4.3 of the main text.
The distillation procedure is based on training the ‘student‘ network w s to approximate the predictions of a trained ‘teacher‘ network w t .
We apply this procedure in five settings {1x2, 2x2, 2x3, 2x5, 3x10} given R max = 10 − 3 to distill RegretFormer onto RegretNet, as well as to distill RegretFormer onto EquivariantNet.
If the architecture of RegretFormer for some reason impairs its ability to approximate optimal misreports, a RegretNet or an EquivariantNet trained to closely mimic the predictions of a RegretFormer may find better misreports that produce higher regret values for the
RegretFormer.
Specifically, since the predictions of allocation and payment modules can respectively be treated as the categorical and the Bernoulli distributions, we train the student network to minimize the KL divergence from its predictions to the predictions of the teacher network.
For example, to train the allocation module, the student minimizes KL ( g ( w t ) , g ( w s ))
= 1 | B | (cid:80) l ∈ B (cid:80) i,j z ij ( v l ; w t ) · (log( z ij ( v l ; w t ))
− log( z ij ( v l ; w s ))) , and likewise for the payment module.
This approach was initially proposed in Hinton et al.
Furthermore, to satisfy DSIC on the whole support, the
KL-divergence is minimized at both the true valuations v l and the approximate optimal misreports for each participant v ′ li ( w s ) , estimated by the student network as per usual via inner optimization (4).
We report the results in Table 7.
Like in the cross-misreport validation procedure, we do not find any evidence that RegretFormer approximates regret worse than the alternative architectures.
This can be seen in Table 7 by comparing the teacher regret estimated on teacher misreports with the teacher regret estimated on student misreports: the latter is never substantially higher than the former.
As additional evidence towards the performance gap being genuine, in the distillation experiments both students achieve the same revenue as the teacher while consistently producing higher regret, up to a magnitude on the hardest 3x10 setting.
This can have two explanations.
First, the student
[Table]
networks get stuck in one of multiple local optimums, which prevents them from reaching lower regrets.
Moreover, this consistently happens both when learning from scratch (since in Table 1 both
RegretNet and EquivariantNet achieve lower revenue than RegretFormer for the same regret budgets) and when mimicking predictions of RegretFormer.
Second, the better solutions with high revenue and low regret that can be found by RegretFormer are simply absent from the space of the mechanisms that can be represented by RegretNet and EquivariantNet.
C.3 Out-of-setting generalization
In these experiments, we investigate how well the architectures generalize to unseen settings.
It’s clear that RegretNet cannot be applied out-of-domain since its layers rely on constant input size, so we compare our network with EquivariantNet.
The networks are trained in five settings {1x2, 2x2, 2x3, 2x5, 3x10} and then tested in all settings but the setting used for training.
The results are reported in Table 8.
Both networks look promising when the number of objects varies and the number of bidders remains constant.
However, generalization to the settings where the number of bidders varies is poor for both networks due to complex interactions between the participants.
Similar results were observed by Rahme et al.
Still, when the validation setting has the number of participants same as or less than the training setting, RegretFormer usually outperforms
EquivariantNet by either achieving higher revenue or lower regret.
C.4 Cross-misreport validation in multi-settings
In section 4.5 in the main text, we have mentioned performing cross-misreport validation in multisettings.
Table 10 presents full results of this experiment.
It is evident that both RegretNet and
RegretFormer approximate the optimal misreports adequately since each estimates the highest regret on own misreports.
In contrast, EquivariantNet poorly approximates misreports and underestimates regret in multi-settings since both RegretNet and RegretFormer find better misreports for this network.
C.5 Out-of-multi-setting generalization
We define two subsets of settings to respectively train and validate networks:
Train : S train ={2x3, 2x5, 2x7, 3x4, 3x6}.
Test : S test =
{2x4, 2x6, 3x3, 3x5, 3x7}.
[Table] [Table]
We compare how networks generalize to unseen settings when trained in the multi-setting regime.
To this end, we train RegretNet and RegretFormer on S train and then validate them on S test .
Because
EquivariantNet poorly underestimates regret in multi-settings (see Section 4.5 and C.4), we do not include it for comparison.
The resulting revenue and regret values are reported in Table 9.
In all out-of-domain settings RegretNet produced poor results.
On average, its regret is more than an order of magnitude larger compared to RegretFormer.
In contrast, our approach stably generalizes to all unseen settings while keeping regret low.
Remarkably, its revenue is as high and its regret is as low as in the experiments where all constant-sized settings are available during training (Table 5).
The superiority of RegretFormer over
RegretNet is especially prominent in the 3x7 setting where our network achieves a larger revenue while producing 16 times as little regret.
C.6 Training time
We report the average wall-clock training time in hours in Table 11.
It takes longer to train RegretFormer than the baselines for two reasons.
First, to perform optimally RegretFormer requires a bigger network with more parameters than baselines, especially in the bigger settings.
Please see Tablefor summarized sizes of the three neural architectures in all settings.
Second, the attention layers
[Table]
have quadratic O ( n 2 ) complexity (where n is the number of items or participants).
If training time is an issue, the attention layers can be replaced with one of multiple modifications that have O ( n log n ) or O ( n ) complexity [60, 34, 66].
[Table] [Table]
[Figure]
[Figure] [Figure]
[Figure] [Figure]