SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Learning and Collusion in Multi-unit Auctions*
Simina Brânzei  
Purdue University
Mahsa Derakhshan Northeastern University
Negin Golrezaei MIT
Yanjun Han  
New York University
In a carbon auction, licenses for CO2 emissions are allocated among multiple interested players. Inspired by this setting, we consider repeated multi-unit auctions with uniform pricing, which are widely used in practice. Our contribution is to analyze these auctions in both the offline and online settings, by designing efficient bidding algorithms with low regret and giving regret lower bounds. We also analyze the quality of the equilibria in two main variants of the auction, finding that one variant is susceptible to collusion among the bidders while the other is not.
In a multi-unit auction, a seller brings multiple identical units of a good to a group of players (buyers) interested in purchasing the items. Due to the complexity and depth of the model, the multi-unit auction has been explored in a significant volume of research [MRH89, EWK98, DN07, DLN12, FFLS12,  $\mathrm{ACP}^{+}14$ , DN15,  $\mathrm{BCI}^{+}05$ , BGN03, GML13, DHP17, BMT19] and recommended as a lens to the entire field of algorithmic mechanism design [Nis14]. The prior literature has often focused on understanding the equilibria of the auction under various solution concepts and designing revenue or welfare maximizing one-shot mechanisms.
We consider the setting where each player  $i$  has a value  $v_{i,j}$  for receiving a  $j$ -th unit in their bundle, which is reported to the auctioneer in the form of a bid  $b_{i,j}$ . After receiving the bids, the auctioneer computes a price  $p$  per unit, then sorts the bids in descending order and allocates the  $j$ -th unit to the player that submitted the  $j$ -th highest bid, charging them  $p$  for the unit. The price  $p$  is set so that all units are sold and the resulting allocation has the property that each player gets their favorite bundle given their preferences.
This allocation rule represents the well-known Walrasian mechanism in the multi-unit setting: compute the market (aka competitive or Walrasian) equilibrium with respect to the reported valuations. The template for the mechanism is: elicit the valuations from the buyers and then assign a set of prices to the goods such that when each buyer takes their favorite bundle of goods at the given prices, the market clears, i.e. all goods are sold and there is no excessive demand for any good (see, e.g., [BLNL14]). Due to the strong fairness and efficiency properties of the market equilibrium, the resulting allocation is envy-free with respect to the reported valuations. A strand of research concentrated on achieving fair pricing in auctions, which often involves setting uniform prices for identical units  $\mathrm{[GHK^{+}05]}$ .
Repeated auctions take place in many real world settings, such as when allocating licenses for carbon (CO2 emissions) [CK02], ads on online platforms [BBW15, GJM21], U.S. Treasury notes to investors or trade exchanges over the internet [MT15]. Repeated mechanisms often give rise to complex patterns of behavior. For example, the bidders may use the past transaction prices as a way of discovering the valuations of the competing bidders, or engage in collusion using bid rotation
*Authors are listed in alphabetical order. This work was done in part while the authors were visiting the Simons Institute for the Theory of Computing. Simina was supported in part by NSF CAREER grant CCF-2238372. Negin was supported in part by the Young Investigator Program (YIP) Award from the Office of Naval Research (ONR) N00014-21-1-277. 37th Conference on Neural Information Processing Systems (NeurIPS 2023).
schemes since the history of bids can serve as a communication device [Aoy03]. In fact, [CCDP21] show that in markets such as Cournot oligopolies with stochastic demand, collusion is observed even when the bidders are not explicitly trying to conspire. Instead, collusion emerges naturally when the agents use AI algorithms based on Q-learning to update their strategies.
Since repeated multi-unit auctions are used in high-stakes environments such as allocating licenses for carbon [GIL20], it is paramount to understand:  $(i)$  what features should the auction have in the first place?; and  $(ii)$  given an auction format, how should the players bid to maximize their utility?
A good bidding strategy generally guarantees the player using it will not "regret" its strategy in hindsight, regardless of the strategies of others. There has been extensive work on understanding dynamics in auctions and games under various behavioral models of the players (see, e.g., [NCKP22, RST17, DS16, MT15, HKMN11, BR11, LB10, GJM21, GJLM21, GJL23, BF19]), but due to the complexity of understanding dynamical systems many questions remain open.
In this paper, we design efficient learning algorithms that players can use for bidding, in both the offline and online settings. We also give regret lower bounds and then analyze the quality of the equilibria in two main variants of the auction, finding that one variant is susceptible to collusion among the bidders while the other is not.
1.1 Model
Consider a seller with  $K$  identical units of a good and a set of players  $[n] = \{1, \ldots, n\}$  that have quasi-linear valuations with decreasing marginal returns. Formally, let  $\mathbf{v}_i = (v_{i,1}, \ldots, v_{i,K})$  be the valuation of player  $i$ , where  $v_{i,j} \geq 0$  is the marginal value for getting a  $j$ -th unit in their bundle. The value of player  $i$  for getting  $\ell$  units is  $V_i(\ell) = \sum_{j=1}^{\ell} v_{i,j}$ . Diminishing marginal returns require that  $v_{i,1} \geq v_{i,2} \geq \ldots \geq v_{i,K} \forall i \in [n]$ . A player  $i$  is said to be hungry if  $v_{i,j} > 0 \forall j \in [K]$ .
Auction format: uniform pricing. The auctioneer announces  $K$  units of a good for sale. Each player  $i$  submits bids  $\mathbf{b}_i = (b_{i,1},\dots ,b_{i,K})$ , where  $b_{i,j}$  is player  $i$ 's bid for  $j$ -th unit. The auctioneer sorts the bids in decreasing order and computes a price  $p$ . Then for each  $j = 1,\ldots ,K$ , the auctioneer allocates the  $j$ -th unit to the player that submitted the  $j$ -th highest bid, charging them  $p$  for the unit. If multiple bids are equal to each other, ties are broken in lexicographic order of the player names.
We consider two variants of the auction:
(i) the  $K$ -th price auction, where the price per unit is set to the  $K$ -th highest bid;
(ii) the  $(K + 1)$ -st price auction, where the price is set to the  $(K + 1)$ -st highest bid.
Both of these variants implement the market equilibrium with respect to the reported valuations of the players, which are represented by their bid vectors. The  $K$ -th price format selects the maximum possible market equilibrium price, namely the highest possible price per unit at which all the goods are sold and each player purchases their favorite bundle given their utility. The  $(K + 1)$ -st format selects the minimum possible market equilibrium price.
Allocation, Price, Utility. An allocation  $\mathbf{z}$  is an assignment of units to the players such that  $z_{i} \geq 0$  is the number of units received by player  $i$  and  $\sum_{j=1}^{n} z_{j} = K$ .
Given bid profile  $\mathbf{b}$ , let  $p(\mathbf{b})$  be the price per unit and  $\mathbf{x}(\mathbf{b}) = (x_{1}(\mathbf{b}),\dots,x_{n}(\mathbf{b}))$  the allocation obtained when the auction is run with bids  $\mathbf{b}$ , where  $x_{i}(\mathbf{b})\in \{0,\ldots ,K\}$  is the number of units received by player  $i$ . The utility of player  $i$  at  $\mathbf{b}$  is:  $u_{i}(\mathbf{b}) = V_{i}(x_{i}(\mathbf{b})) - p(\mathbf{b})\cdot x_{i}(\mathbf{b})$ .
Revenue and Welfare. The revenue obtained by the auctioneer at a bid profile  $\mathbf{b}$  equals  $K\cdot p(\mathbf{b})$ . The social welfare of the bidders at  $\mathbf{b}$  is  $SW(\mathbf{b}) = \sum_{i = 1}^{n}V_{i}(x_{i}(\mathbf{b}))$
Notation. W.l.o.g., we have  $b_{i,1} \geq \ldots \geq b_{i,K}$  for each bid profile  $\mathbf{b}$  is such that and  $i \in [n]$ . Given a bid profile  $\mathbf{b} = (\mathbf{b}_1, \dots, \mathbf{b}_n)$ , let  $\mathbf{b}_{-i}$  denote the bid profile of everyone except player  $i$ . The bid profile where player  $i$  uses a bid vector  $\beta$  and the other players bid  $\mathbf{b}_{-i}$  is denoted  $(\beta, \mathbf{b}_{-i})$ .
An illustration of how the auction works is given in Example 1.
2In other words, after the players submit their bids, the auctioneer arranges these bids in descending order  $(c_{1},\dots ,c_{j},\dots ,c_{n\cdot K})$  and then allocates unit 1 to the bidder with bid  $c_{1}$ , unit 2 to the bidder with bid  $c_{2}$ , and so on.
Example 1. Let  $K = 3$  and  $n = 2$ . Suppose the valuations are  $\mathbf{v}_1 = (5,2)$  and  $\mathbf{v}_2 = (4,1)$ . If the players submit bids  $\mathbf{b}_1 = (2,1)$  and  $\mathbf{b}_2 = (3,2)$ , the bid are sorted in the order  $(b_{2,1}, b_{1,1}, b_{2,2}, b_{1,2})$ . Then the allocation is  $x_1 = 1$  and  $x_2 = 2$ .
- Under the  $K$ -th price auction, the price is set to  $p = b_{2,2} = 2$ . The utilities of the players are  $u_1(\mathbf{b}) = V_1(1) - p = 5 - 2 = 3$  and  $u_2(\mathbf{b}) = V_2(2) - 2 \cdot p = (4 + 1) - 2 \cdot 2 = 1$ .
- Under the  $(K + 1)$ -st price auction, the price is set to  $p = b_{1,2} = 1$ . The utilities of the players are  $u_{1}(\mathbf{b}) = V_{1}(1) - p = 5 - 1 = 4$  and  $u_{2}(\mathbf{b}) = V_{2}(2) - 2 \cdot p = (4 + 1) - 2 \cdot 1 = 3$ .
Repeated setting. We consider a repeated setting, where the auction is run multiple times among the same set of players, who can adjust their bids based on the outcomes from previous rounds.
Formally, in each round  $t = 1,2,\ldots ,T$  ,the next steps take place:
- The auctioneer announces  $K$  units for sale. Each player  $i \in [n]$  submits bid vector  $\mathbf{b}_i^t = (b_{i,1}^t, \ldots, b_{i,K}^t)$  privately to the auctioneer, where  $b_{i,j}^t$  is player  $i$ 's bid for a  $j$ -th unit at time  $t$ . Then the auctioneer runs the auction with bids  $\mathbf{b}^t = (\mathbf{b}_1^t, \ldots, \mathbf{b}_n^t)$  and reveals information (feedback) about the outcome to the players. We consider two feedback models for the information revealed at the end of round  $t$ :
- Full information: All the bids  $\mathbf{b}^t$  are public knowledge.
- Bandit feedback: The price  $p(\mathbf{b}^t)$  is the only public knowledge; additionally, each player  $i$  privately learns their own allocation  $x_i(\mathbf{b}^t)$ .
Thus the allocation at time  $t$  solely depends on the bids submitted by players in round  $t$ , excluding any prior rounds.
The next sections summarize our results for both the  $K$ -th and  $(K + 1)$ -st price variants.
1.2.1 Offline Setting
In the offline setting, we are given a player  $i$  with valuation  $\mathbf{v}_i$  and a history  $H_{-i} = (\mathbf{b}_{-i}^1,\dots ,\mathbf{b}_{-i}^T)$  containing the bids submitted by the other players in past auctions. Here we restrict the bidding space to a discrete domain  $\mathbb{D} = \{k\cdot \varepsilon \mid k\in \mathbb{N}\}$ , for some  $\varepsilon >0$ .
The goal is to find a fixed bid vector  $\mathbf{b}_i^* = (b_{i,1}^*,\ldots ,b_{i,K}^*)$  that maximizes player  $i$ 's cumulative utility on the data set given by the history  $H_{-i}$ , that is,  $\mathbf{b}_i^* = \arg \max_{\boldsymbol {\beta}\in \mathbb{D}^K}\sum_{t = 1}^{T}u_i(\boldsymbol {\beta},\mathbf{b}_{-i}^t)$ .
The offline problem is challenging because the decision (bid) space is exponentially large and hence naively experimenting with every possible bid vector leads to impractical algorithms.
We overcome this challenge by relying on the structural properties of the offline problem. At a high level, we carefully design a weighted directed acyclic graph (DAG) and identify a bijective map between paths from source to sink in the graph and bid profiles of player  $i$ . Since a maximum weight path in a DAG can be found in polynomial time, this yields a polynomial time algorithm for computing an optimum bid vector.
Theorem 1 (informal). Computing an optimum bid vector for one player in the offline setting is equivalent to finding a maximum-weight path in a DAG and can be solved in polynomial time.
Theorem 1 applies to both versions of the auction, with  $K$ -th and  $(K + 1)$ -st highest price.
1.2.2 Online Setting
In the online setting, the players run learning algorithms to update their bids as they participate in the auction over time. The first scenario we consider is that of full information feedback, where at the end of each round  $t$  the auctioneer makes public all the bids  $\mathbf{b}^t$  submitted in that round.
For example, consider  $K = 2$  units and  $n = 2$  players. Suppose that in round  $t = 5$ , player 1 submits the bid vector  $b_{1}^{5} = (4,2)$  and player 2 submits  $b_{2}^{5} = (5,3)$ . The auctioneer sorts these bids in decreasing order, obtaining the vector  $(5,4,3,2)$ . Then the allocation in round 5 is to give the first unit to player 1 and the second unit to player 2; the allocation disregards any events from preceding rounds.
Building on the offline analysis, we design an efficient online learning algorithm for bidding, which guarantees low regret for each player using it. The main idea is to run multiplicative weight updates, where the experts are paths in the DAG from the offline setting. Each such path corresponds to a bid profile for the learner. A challenge is that the number of paths in the graph (and so experts) is exponential. Nevertheless, we can achieve a polynomial runtime by representing the experts implicitly, using a type of weight pushing algorithm based on the method of path kernels [TW03].
Theorem 2 (Full information feedback, upper bound). For each player  $i$  and time horizon  $T$ , there is an algorithm for bidding in the repeated auction with full information feedback that runs in time  $O(T^2)$  and guarantees the player's regret is bounded by  $O(v_{i,1} \cdot \sqrt{TK^3 \log T})$ .
To the best of our knowledge, our paper is the first to connect the path kernel setting with auctions and obtain efficient algorithms for learning in auctions via this connection.
We also consider bandit feedback, which limits the amount of information the players learn about each other: at the end of each round the auctioneer announces publicly only the resulting price, and then privately tells each player their allocation. Bandit feedback could be relevant for reducing the amount of collusion among the players in repeated auction environments [HP89].
Theorem 3 (Bandit feedback, upper bound). For each player  $i$  and time horizon  $T$ , there is an algorithm for bidding in the repeated auction with bandit feedback that runs in  $O(KT + K^{-5/4} T^{7/4})$  and guarantees the player's regret is bounded by  $O(v_{i,1} \cdot \min \{\sqrt[4]{T^3 K^7 \log T}, KT\})$ .
Although our bidding policy is similar to the EXP3-type algorithm in [GLLO07], and our computational efficiency relies on the path kernel methods in [TW03], a major difference is that our edge weights are heterogeneous, i.e. the weights of different edges could be of different scales. To circumvent this issue, we need to use a different estimator for the weight under bandit feedback which in turn also changes the technical analysis.
We complement the upper bounds with the following regret lower bound, where the construction of the hard instance relies heavily on the specific structure of our auction format.
Theorem 4 (Lower bound, full information and bandit feedback). Let  $K \geq 2$ . Suppose the auction is run for  $T$  rounds. Then for each strategy of player  $i$ , under both full information and bandit feedback, there exists a bid sequence  $\{\mathbf{b}_{-i}^t\}_{t=1}^T$  by the other players such that the expected regret of player  $i$  is at least  $c \cdot v_{i,1}K\sqrt{T}$ , where  $c > 0$  is an absolute constant.
Theorems 2, 3, and 4 apply to both variants of the auction (with  $K$ -th and  $(K + 1)$ -st highest price).
1.2.3 Equilibrium Analysis
We also analyze the quality of the Nash equilibria reached in the worst case in the static version of the auction, as they naturally apply to the empirical distribution of joint actions when the players use sub-linear regret learning algorithms.
The  $(K + 1)$ -st price auction has been observed to have low or even zero revenue equilibria [Wil79, KN04,  $\mathrm{ACP}^{+}14$ , BW20]. With  $n > K$  hungry players, the next phenomena occur:
- in the  $(K + 1)$ -st price auction: every allocation  $\mathbf{z}$  that allocates all the units can be implemented in a pure Nash equilibrium  $\mathbf{b}$  with price  $\varepsilon$ , for every small enough  $\varepsilon \geq 0$ .
- in the  $K$ -th price auction: no pure Nash equilibrium with arbitrarily low price exists.
Our contribution is to show that the zero-price equilibria of the  $(K + 1)$ -st highest price auction are very stable. We do this by considering the core solution concept, which models how groups of players can coordinate.
In our setting, a strategy profile is core-stable if no group  $S$  of players can deviate by simultaneously changing their strategies such that each player in  $S$  weakly improves their utility and the improvement is strict for at least one player in  $S$ . The players outside  $S$  are assumed to have neutral reactions to the deviation, that is, they keep their strategy profiles unchanged. This is consistent with the Nash equilibrium solution concept, where only the deviating player changes their strategy.
A body of literature studied the core in auctions when the auctioneer can collude together with the bidders (see, e.g., [DM08]). However, as is standard in mechanism design, it is also meaningful to
consider the scenario where the auctioneer sets the auction format and then the bidders can strategize and potentially collude among themselves, without the auctioneer; this setting is our focus.
First, we consider the core without transfers, where a strategy profile is simply a bid profile  $\mathbf{b}$ .
Theorem 5 (Core without transfers). Consider  $K$  units and  $n > K$  hungry players. The core without transfers of the  $(K + 1)$ -st auction can be characterized as follows:
- every bid profile  $\mathbf{b}$  that is core-stable has price zero (i.e.  $p(\mathbf{b}) = 0$ );
- for every allocation  $\mathbf{z}$ , there is a core-stable bid profile  $\mathbf{b}$  with price zero that implements  $\mathbf{z}$  (i.e.  $\mathbf{x}(\mathbf{b}) = \mathbf{z}$  and  $p(\mathbf{b}) = 0$ ).
If a bid profile  $\mathbf{b}$  is core-stable, then it is also a Nash equilibrium, for the simple reason that if no group of players can deviate simultaneously from  $\mathbf{b}$ , then the group consisting of one player cannot find an improving deviation either. Thus Theorem 5 says that the equilibria with price zero are the only ones that remain stable when also allowing deviations by groups of players. Moreover, the theorem also says there are many inefficient core-stable equilibria, all with price zero.
Second, we also consider the core with transfers, where the players can make monetary transfers to each other. A strategy profile in this setting is given by a pair  $(\mathbf{b},\mathbf{t})$ , where  $\mathbf{b}$  is a bid profile and  $\mathbf{t}$  is a profile of monetary transfers, such that  $t_{i,j} \geq 0$  is the monetary transfer (payment) of player  $i$  to player  $j$  (e.g., player  $i$  could pay player  $j$  so that  $j$  keeps its bids low).
Theorem 6 (Core with transfers). Consider  $K$  units and  $n > K$  hungry players. The core with transfers of the  $(K + 1)$ -st auction can be characterized as follows.
- Let  $(\mathbf{b}, \mathbf{t})$  be an arbitrary tuple of bids and transfers that is core stable. Then the allocation  $\mathbf{x}(\mathbf{b})$  maximizes social welfare, the price is zero (i.e.  $p(\mathbf{b}) = 0$ ), and there are no transfers between the players (i.e.  $\mathbf{t} = 0$ ).
To see why Theorem 6 holds, if the price is positive at some strategy profile, the players with "highest values" can pay the other players to lower their bids. When the price reaches zero, they can start withdrawing the payments. Thus the only strategy profiles in the core with transfers are those where the players with "highest values" win and pay zero. The difference from Theorem 5 is that transfers allow the players with highest values to force getting their desired items at price zero, which they cannot enforce without transfers. Nevertheless, even without transfers, the price remains zero.
Remarks on collusion. Given that the zero price equilibria of the  $(K + 1)$ -st price auction are very stable and easy to find, one can expect they can be determined by the bidders, for example in settings with few bidders who know each other. These equilibria can be easily implemented in repeated settings once the bidders agree on their strategies.
Strikingly, these zero price equilibria have in fact been observed empirically in repeated auctions for fishing quotas in the Faroe Islands, where even inexperienced bidders were able to collude and enforce them; see details in [LMT18]. In the carbon setting, the Northeast US auction is based on the  $(K + 1)$ -st price format [GIL20] and has very low prices as well.
In contrast, the  $K$ -th price auction format does not have any Nash equilibria with price zero, let alone core-stable ones. Thus the  $K$ -th price auction format may be preferable to the  $(K + 1)$ -st price auction in settings with high stakes, such as selling rights for carbon emissions. Another remedy, also discussed in [LMT18], is to make the number of units a random variable.
Our work is related to the rich literature on combinatorial auctions (e.g., [DVV03, BN07, CSS07]) and more specifically auctions for identical goods, which are widely used in practice in various settings: Treasury Auctions [GI05, BS00], Procurement Auctions [AC05], Wholesale Electricity Markets [TSM08, FR03, FvdFH06], Carbon Auctions [GS13, SS17, GIL20]. The two most prevalent multi-unit auctions are uniform price auctions and pay-as-bid auctions that are widely studied in the literature either empirically (e.g., [BZ93, MA98, BB01, HG13]) or theoretically under the classical Bayesian setting (e.g., [AHP13, GIL20, BILL23]). In particular, there have been several studies that aim to compare uniform price auctions versus pay-as-bid auctions (e.g.,
[KCPT01, FR03, SBLS04, LMMM11,  $\mathrm{ACP}^{+}14$ ] in terms of revenue and welfare under Nash equilibria, and the possibility of collusion. Our work contributes to this literature by studying uniform price auctions the angle of designing bidding algorithms for the players.
Designing learning algorithms in auctions has attracted significant attention in recent years. Several papers study the problem of designing learning/bandit algorithms to optimize reserve prices in auctions over time (e.g., [KN14, MR16, MM16, CD17, DHL $^{+}$ 17, GJL23, GJM21, GJLM21]). There are papers that have focused on designing bandit algorithms to optimize bidding strategies in single-unit second price and first price auctions (e.g., [WPR16, BMSW18, FPS18, HZF $^{+}$ 20, HZW20, BGM $^{+}$ 22]). Prior to our work, such problems were not studied for multi-unit uniform-price auctions.
Leveraging structural information in designing bandit algorithms has been the topic of the rich literature, which includes linear reward structures [DHK08, RT10, MRT09, LS17], Lipschitz reward structures [MCP14, MLS18], convex reward distribution structure [VPG20], and combinatorial structures [AR08, UNK10, CBL12, ABL14, CWY13, CTPL15, ZLW19, KV05,  $\mathrm{NGW^{+}21}$ ]. Our work is closest to the works that aim to exploit combinatorial structures. Within this literature, several papers leverage DP-based or MDP-based structures (e.g., [TW03, GLLO07, ZN13, RM21]).
The multi-unit auction setting was studied in a large body of literature on auctions [DLN12,  $\mathrm{BCI}^{+}05$ , DN07, DL14, DN15], where the focus has been on designing truthful auctions with good approximations to some desired objective, such as the social welfare or the revenue.