SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Soumya Basu
Google Mountain View
basusoumya@google.com
Abishek Sankararaman
abishek.90@gmail.com
Double Auction enables decentralized transfer of goods between multiple buyers and sellers, thus underpinning functioning of many online marketplaces. Buyers and sellers compete in these markets through bidding, but do not often know their own valuation a-priori. As the allocation and pricing happens through bids, the profitability of participants, hence sustainability of such markets, depends crucially on learning respective valuations through repeated interactions. We initiate the study of Double Auction markets under bandit feedback on both buyers' and sellers' side. We show with confidence bound based bidding, and 'Average Pricing' there is an efficient price discovery among the participants. In particular, the regret on combined valuation of the buyers and the sellers - a.k.a. the social regret - is  $O(\log (T) / \Delta)$  in  $T$  rounds, where  $\Delta$  is the minimum price gap. Moreover, the buyers and sellers exchanging goods attain  $O(\sqrt{T})$  regret, individually. The buyers and sellers who do not benefit from exchange in turn only experience  $O(\log T / \Delta)$  regret individually in  $T$  rounds. We augment our upper bound by showing that  $\omega (\sqrt{T})$  individual regret, and  $\omega (\log T)$  social regret is unattainable in certain Double Auction markets. Our paper is the first to provide decentralized learning algorithms in a two-sided market where both sides have uncertain preference that need to be learned.
Online marketplaces, such as eBay, Craigslist, Task Rabbit, Doordash, Uber, enables allocation of resources between supply and demand side agents at a scale through market mechanisms, and dynamic pricing. In many of these markets, the valuation of the resources are often personalized across agents (both supply and demand side), and remain apriori unknown. The agents learn their own respective valuations through repeated interactions while competing in the marketplace. In turn, the learning influences the outcomes of the market mechanisms. In a recent line of research, this interplay between learning and competition in markets has been studied in multiple systems, such as bipartite matching markets [29, 30, 39, 7], centralized basic auctions [26, 22]. These works follow the 'protocol model' [2], where multiple agents follow a similar protocol/algorithm, while each agent executes her protocol using only her own observations/world-view up to the point of execution.
In this paper, we initiate the study of the decentralized Double Auction market where multiple sellers and buyers, each with their own valuation, trades an indistinguishable good. In each round, the sellers and the buyers present bids for the goods.<sup>1</sup> The auctioneer is then tasked with creating an allocation, and pricing for the goods. All sellers with bids smaller than the price set by the auctioneer sell at that price, whereas all the buyers with higher bids buy at that price. Each buyer and seller, is oblivious to all the prices including her own. Only a buyer, or a seller participating in the market observes
<sup>0</sup>Work done when AS was affiliated with UC Berkeley. All opinions are that of the authors and do not necessarily represent that of the employers
<sup>1</sup>In some literature, the bids of the sellers is called 'asks', but we use bids for both sellers and buyers.
her own valuation of the good (with added noise). Notably, our work tackles two-sided uncertainty, whereas the previous works mainly focused on one-sided uncertainty in the market.
Double auction is used in e-commerce [43] – including business-to-business and peer-to-peer markets, bandwidth allocation [24, 23], power allocation [31]. We focus on the ‘Average Mechanism’ for double auction. Average mechanism guarantees that the auctioneer and the auction participants incur no losses in the trade. It also ensures that each commodity is given to the participant that values it the highest, thus maximizing social welfare. Additionally, average mechanism can be implemented through simple transactions. These properties make average mechanism a suitable choice in large social markets, such as energy markets [32], environmental markets [35], cloud-computing markets [38, 32], bidding on wireless spectrum [18]. Our objective is to design a bandit average mechanism for double auction markets when the participants are a-prioiri unaware of their exact valuation.
Under average mechanism (detailed in Section 2.2) first an allocation is found, by maximizing  $K$  such that the  $K$  highest bidding buyers all bid higher than the  $K$  lowest bidding sellers. The price is set as the average of the  $K$ -th lowest bid among the buyers, and the  $K$  highest bid among the sellers for the chosen  $K$ . We have two-sided uncertainty in the market, as both buyers and sellers do not know their own valuation. The uncertainty in bids manifests in two ways. Firstly, each buyer needs to compete with others by bidding high enough to get allotted so that she can discover her own price. Similarly, the sellers compete by bidding lower for price discovery. The competition-driven increase in buyers' bids, and decrease in the sellers' bids may decrease the utility that a buyer or seller generates. Secondly, as the valuation needs to be estimated, the price set in each round as a function of these estimated valuations (communicated to the auctioneer in the form of bids) remains noisy. This noise in price also decreases the utility. However, when price discovery is slow the noise in price increases. Therefore, the main challenge in decentralized double auction with learning is to strike a balance between the competition-driven increase/decrease of bids, and controlling the noise in price.
Our main contributions in this paper are as follows.
1. Our paper is the first to provide decentralized learning algorithms in a two-sided market where both sides have uncertain preference that need to be learned. Unlike in the setting with one-sided uncertainty only, we identify that with two-sided uncertainty in double auction markets optimism in the face of uncertainty in learning (OFUL) from both sides causes information flow bottleneck and thus not a good strategy. We introduce the notion of domination of information flow – that increases the chance of trade and price discovery. The sellers bid the lower confidence bound (LCB), and buyers bid the upper confidence bound (UCB) of their respective valuation. By using UCB bids the buyers, and using the LCB bids the sellers decrease their reward and facilitate price discovery. Formally, with the above bids under average mechanism
- We show that the social welfare regret, i.e. the regret in the combined valuation of all the sellers and the buyers in the market is  $O(\log (T) / \Delta)$  regret in  $T$  rounds with a minimum reward gap  $\Delta$ . We also show a  $\Omega (\log (T) / \Delta)$  lower bound on social-welfare regret and thus our upper bound is order-optimal.
- For individual regret, we show that each of the sellers and the buyers that do not participate under the true valuations incur  $O(\log (T) / \Delta)$  regret, while the optimal participating buyers and sellers incur a  $O(\sqrt{T\log(T)})$  regret. Our upper bound holds for heterogeneous confidence widths, making it robust against the choices of the individual agents.
2. We complement the upper bounds by showing price discovery itself is  $\Omega(\sqrt{T})$  hard in the minimax sense. Specifically, we consider a relaxed system where  $(i)$  the price of the good is known to all, and  $(ii)$  an infinite pool of resource exists, and hence any buyer willing to pay the price gets to buy, and any seller willing to sell at the price does so. We show under this setup, for any buyer or seller, there exists a system where that agent must incur a regret of  $\Omega(\sqrt{T})$ . Similarly, we establish a  $\Omega(\log(T))$  lower bound for the social-welfare regret by showing that the centralized system can be reduced to a combinatorial semi-bandit and using the results of [14].
2 System Model
The market consists of  $N \geq 1$  buyers and  $M \geq 1$  sellers, trading a single type of item which are indistinguishable across sellers. This set of  $M + N$  market participants, repeatedly participate in the market for  $T$  rounds. Each buyer  $i \in [N]$  has valuation  $B_{i} \geq 0$ , for the item and each seller  $j \in [M]$  has valuation  $S_{i} \geq 0$ . No participant knows of their valuation apriori and learn it while repeatedly participating in the market over  $T$  rounds.
2.1 Interaction Protocol
The buyers and sellers interact through an auctioneer who implements a bilateral trade mechanism at each round. At each round  $t \geq 1$ , every buyer  $i \in [N]$  submits bids  $b_{i}(t)$  and seller  $j \in [M]$  submits asking price  $s_{j}(t)$  simultaneously. Based on the bids and asking prices in round  $t$ , the auctioneer outputs  $(i)$  subsets  $\mathcal{P}_b(t) \subseteq [N]$  and  $\mathcal{P}_s(t) \subseteq [M]$  of participating buyers and sellers with equal cardinality  $K(t) \leq \min(M, N)$ , and  $(ii)$  the trading price  $p(t)$  for the participating buyers and sellers in this round. Subsequently, every buyer  $i \in [N]$  is  $(i)$  either part of the trade at time  $t$ , in which case she gets utility  $r_{i}^{(B)}(t) := B_{i} + \nu_{b,i}(t) - p(t)$ , or  $(ii)$  is not part of the trade at time  $t$  and receives 0 utility along with a signal that she did not participate. Similarly, each seller  $j \in [M]$  is either part of the trade and receives utility  $r_{j}^{(S)}(t) := p(t) - S_{j} - \nu_{s,j}(t)$ , or is informed she is not part of the trade and receives 0 utility. The random variables  $\nu_{b,i}(t)$  and  $\nu_{s,j}(t)$  for all  $i \in [N]$ ,  $j \in [M]$  and  $t \in [T]$  are i.i.d., 0 mean, 1 sub-Gaussian random variables. $^3$
2.2 Average price mechanism
Throughout the paper, we assume the auctioneer implements the average price mechanism in every round  $t$ . Under this mechanism, at each round  $t$ , the auctioneer orders the bids by the 'natural order', i.e., sorts the buyers bids in descending order and the seller's bids in ascending order. Denote by the sorted bids from the buyer and seller as  $b_{i_1}(t) \geq \dotsb b_{i_N}(t)$  and the sorted sellers bids by  $s_{j_1}(t) \leq \dotsb s_{j_M}(t)$ . Denote by the index  $K(t)$  to be the largest index such that  $b_{i_{K(t)}}(t) \geq s_{j_{K(t)}}(t)$ . In words,  $K(t)$  is the 'break-even index' such that all buyers  $i_1, \dots, i_{K(t)}$  have placed bids offering to buy at a price strictly larger than the asking price submitted by sellers  $j_1, \dots, j_{K(t)}$ . The auctioneer then selects the participating buyers  $\mathcal{P}_b(t) = \{i_1, \dots, i_{K(t)}\}$ , and participating sellers  $\mathcal{P}_s(t) = \{j_1, \dots, j_{K(t)}\}$ . The price is set to  $p(t) := \frac{b_{i_K(t)} + s_{j_K(t)}}{2}$ , and thus the name of the mechanism is deemed as the average mechanism.
Figure 1: Average Mechanism with 6 Buyers and 5 Sellers
2.3 Regret definition
For the given bilateral trade mechanism, and true valuations  $(B_{i})_{i\in [N]}$  and  $(S_{j})_{j\in [M]}$ , denote by  $K^{*}\leq \min (M,N)$  be the number of matches and by  $p^*$  to be the price under the average mechanism when all the buyers and sellers bid their true valuations. Let  $\mathcal{P}_b^*$  to be set of the optimal participating buyers, and  $\mathcal{P}_s^*$  to be set of the optimal participating sellers. For any buyer  $i\in [N]$ , we denote by  $(B_{i} - p^{*})$  to be the true utility of the buyer. Similarly, for any seller  $j\in [N]$ , we denote by
$(p^{*} - S_{j})$  to be the true utility of seller  $j$ . From the description of the average mechanism, in the system with true valuations, all participating agents have non-negative true utilities.
Recall from the protocol description in Section 2.1 that at any time  $t$ , if buyer  $i \in [N]$  participates, then she receives a mean utility of  $(B_i - p(t))$ . For a participating seller  $j \in [M]$  her mean utility of  $(p(t) - S_j)$  in round  $t$ . If in any round  $t$ , if a buyer  $i \in [N]$  or a seller  $j \in [M]$  does not participate, then she receives a deterministic utility 0. The expected individual regret of a buyer  $i$ , namely  $R_{b,i}(T)$ ,
2Throughout, we refer to sellers 'bids' as their asking price
<sup>3</sup>We study the system with 1-sub-Gaussian to avoid clutter. Extension to general  $\sigma$ -sub-Gaussian is trivial when  $\sigma$  or an upper bound to it is known.
and a seller  $j$ , namely  $R_{s,j}(T)$ , are defined as
$$
R _ {b, i} (T) = T \left(B _ {i} - p ^ {*}\right) \mathbb {1} (i \leq K ^ {*}) - \mathbb {E} \left[ \sum_ {t: i \in \mathcal {P} _ {b} (t)} \left(B _ {i} - p (t)\right) \right],
$$
$$
R _ {s, j} (T) = T (p ^ {*} - S _ {j}) \mathbb {1} (j \leq K ^ {*}) - \mathbb {E} \Big [ \sum_ {t: j \in \mathcal {P} _ {s} (t)} (p (t) - S _ {j}) \Big ].
$$
Auctioneer has no regret as average mechanism is budget balanced, i.e. auctioneer does not gain or lose any utility during the process.
We also define the social welfare regret similar to gain from trade regret in [10]. The social welfare is defined as the total valuation of the goods after the transfer of goods from seller to buyer in each round. The expected (w.r.t reward noise) total valuation after transfer is thus defined as  $\left(\sum_{i\in \mathcal{P}_b(t)}B_i + \sum_{j\in [M]\setminus \mathcal{P}_s(t)}S_j\right)$ , while the expected total valuation under oracle average mechanism is  $\left(\sum_{i\in \mathcal{P}_b^*}B_i + \sum_{j\in [M]\setminus \mathcal{P}_s^*}S_j\right)$ . Therefore, the expected social welfare regret is defined as
$$
R _ {S W} (T) = T \left(\sum_ {i \in \mathcal {P} _ {b} ^ {*}} B _ {i} + \sum_ {j \in [ M ] \backslash \mathcal {P} _ {s} ^ {*}} S _ {j}\right) - \mathbb {E} \left[ \sum_ {t = 1} ^ {T} \left(\sum_ {i \in \mathcal {P} _ {b} (t)} B _ {i} + \sum_ {j \in [ M ] \backslash \mathcal {P} _ {s} (t)} S _ {j}\right) \right]. \tag {1}
$$
3 Decentralized Bidding for Domination of Information Flow
We consider the decentralized system where each market participant bids based on their own observation, without any additional communication. The core idea is balancing domination of information flow and over/under bidding, i.e. ensuring the number of allocation is not less than  $K^{*}$  in each round with high probability, and the bids converge to each agent's true valuation.
Each seller  $j \in [M]$ , with  $n_{s,j}(t)$  participation upto round  $t$ , at time  $t + 1$  bids the lower confidence bound (scaled by  $\alpha_{s,j}$ ),  $\mathrm{LCB}(\alpha_{s,j})$  in short, of its own valuation of the item. Each buyer  $i \in [N]$ , with  $n_{s,j}(t)$  participation upto round  $t$ , at time  $t + 1$ , bids the upper confidence bound (scaled by  $\alpha_{b,i}$ ),  $\mathrm{UCB}(\alpha_{b,i})$  in short, of its own valuation of the item. The bids are specified in Equation 2.
$$
s _ {j} (t + 1) = \hat {s} _ {j} (t) - \sqrt {\frac {\alpha_ {s , j} \log (t)}{n _ {s , j} (t)}}, \quad b _ {j} (t + 1) = \hat {b} _ {i} (t) + \sqrt {\frac {\alpha_ {b , i} \log (t)}{n _ {b , i} (t)}} \tag {2}
$$
Here,  $\hat{s}_j(t) = \frac{1}{n_{s,j}(t)}\sum_{t'\leq t:j\in \mathcal{P}_s(t')}Y_{s,j}(t')$ , and  $\hat{b}_i(t) = \frac{1}{n_{b,i}(t)}\sum_{t'\leq t:i\in \mathcal{P}_b(t')}Y_{b,i}(t')$  are the observed empirical valuation of the item up to time  $t$  by seller  $j$ , and buyer  $i$ , respectively.
Our buyers and sellers follow the protocol model (which is ubiquitous in bandit learning for markets) and agree on UCB and LCB based bids, respectively. They are heterogeneous as they may use different  $\alpha_{b,i}$  and  $\alpha_{s,j}$  scaling parameters. The only restriction (as seen in Section 4) is  $\min \{\alpha_{b,i},\alpha_{s,j}\} \geq 4$  which they agree on as part of the protocol.
Key Insights: We now contrast our algorithm design from standard multi-armed-bandit (MAB) problems. In a typical MAB problems, including other multi-agent settings, algorithm is designed based on optimism in the face of uncertainty in learning (OFUL) principle [1]. The UCB-type indices under OFUL arises as optimistic estimate of the rewards of arms/actions. However, such optimism used from both sides, i.e. both buyer and seller using UCB indices, may lead to a standstill. The bids of the buyers with constant probability can remain below the sellers' bids. Instead, we emphasize information flow through trade. In our algorithm buyers' UCB bids and sellers' LCB bids ensure that the system increases the chance for buyer and seller to participate in each round as compared to using their true valuations. There is domination of information flow, and consequently they discover their own valuation in the market.
However, too aggressive over or under bidding can disrupt the price setting process. In particular, if even one non-participating buyer is bidding high enough (due to UCB bids) to exceed a non-participating seller's price, firstly she participates and accrues regret. More importantly, the participating sets deviate, resulting in a deviation of the price of the good from  $p^*$ . Thus resulting in regret for all participating agents as well. Similar problems arise if one or more non-participating
seller predicts lower. On the other hand, too low aggression is also harmful as the price discovery may not happen resulting in deviation of participating set, deviation of price, and high regret. In the next section, we show the aggression remains within desired range, i.e. the regret of the agents remain low, even with heterogeneous UCBs and LCBs.
4 Regret Upper Bound
In this section, we derive the regret upper bound for all the buyers and sellers in the system. Without loss of generality, let us assume that the buyers are sorted in decreasing order of valuation  $B_{1} > B_{2} > \ldots > B_{N}$ . The sellers are sorted in increasing order of valuation  $S_{1} < S_{2} < \ldots < S_{N}$ . The buyers  $i = 1, \ldots, K^{*}$  are the optimal participating buyers, and, similarly, the sellers  $j = 1, \ldots, K^{*}$  are the optimal participating sellers. Let us define  $\alpha_{\min} := \min \{\alpha_{b,i}, \alpha_{s,j}\}$ , and  $\alpha_{\max} := \max \{\alpha_{b,i}, \alpha_{s,j}\}$ . We define the minimum distance of an agent's true valuation from the true price  $p^{*}$  as  $\Delta = \min_{i, j \in [N], j \in [M]} \{|p^{*} - S_{j}|, |B_{i} - p^{*}|\}$ . As an item is sold only if the matched buyer bids strictly greater than the matched seller's bid we have  $\Delta > 0$ .
Our first main result, Theorem 1, proves a  $O(\log (T) / \Delta)$  upper bound for the social welfare regret.
Theorem 1. The expected social welfare regret of the Average mechanism with buyers bidding  $UCB(\alpha_b)$ , and sellers bidding  $LCB(\alpha_s)$  of their estimated valuation, for  $\alpha_{\min} > 2$ , is bounded as:
$$
\begin{array}{l} R_{SW}(T)\leq \sum_{i\leq K^{*}}\sum_{i^{\prime} > K^{*}}\frac{(\sqrt{\alpha_{\max}} + 2)^{2}}{(B_{i} - B_{i^{\prime}})}\log (T) + \sum_{j\leq K^{*}}\sum_{j^{\prime} > K^{*}}\frac{(\sqrt{\alpha_{\max}} + 2)^{2}}{(S_{j^{\prime}} - S_{j})}\log (T) \\ + \sum_ {j ^ {\prime} > K ^ {*}} \sum_ {i ^ {\prime} > K ^ {*}} \frac {(\sqrt {\alpha_ {\operatorname* {m a x}}} + 2) ^ {2}}{(S _ {j ^ {\prime}} - B _ {i ^ {\prime}})} \log (T) + M N b _ {m a x} \zeta (\alpha_ {\min } / 2), \\ \end{array}
$$
where  $\zeta (x)$  is the Riemann-zeta function which is finite for  $x > 1$
The following theorem provides the individual regret bounds for all sellers and buyers in  $T$  rounds.
Theorem 2. The expected regret of the Average mechanism with buyers bidding  $UCB(\alpha_b)$ , and sellers bidding  $LCB(\alpha_s)$  of their estimated valuation, for  $\alpha_{\mathrm{min}} > 2$ , is bounded as:
- for a participating buyer  $i \in [K^{*}]$  as  $R_{b,i}(T) \leq (2 + \sqrt{\alpha_{\max}})\sqrt{T\log(T)} + C_{b',i}\log (T)$ ,
- for a participating seller  $j \in  \left\lbrack  {K}^{ * }\right\rbrack$  as  ${R}_{s,j}\left( T\right)  \leq  \left( {2 + \sqrt{{\alpha }_{\max }}}\right) \sqrt{T\log \left( T\right) } + {C}_{{s}^{\prime },{j}}\log \left( T\right)$  ,
- for a non-participating buyer  $i \geq (K^{*} + 1)$  as  $R_{b,i}(T) \leq \frac{\sqrt{(M - K^{*} + 1)(2 + \sqrt{\alpha_{\max}})^{2}}}{(B_{K^{*}} - B_{i})} \log(T)$ ,
- for a non-participating seller  $j \geq (K^{*} + 1)$  as  $R_{s,j}(T) \leq \frac{\sqrt{(N - K^{*} + 1)(2 + \sqrt{\alpha_{\max}})^{2}}}{(S_{j} - S_{K^{*}})} \log(T)$ .
Here  $C_{b',i}$  and  $C_{s',j}$  are  $O\left(\frac{(M - K^{*} + 1)(N - K^{*} + 1)}{\Delta}\right)$  constants (see Theorem 16 in Appendix A).
We can summarize our main results (with lower bounds taken from Section 5) as
Regret	Social	Participant	Non-participant
Buyer	Seller
Upper	O(MN/Δ log(T))	O((√T + MN/Δ) log(T))	O(√M/Δ log(T))	O(√N/Δ log(T))
Lower	Ω(M+N/Δ log(T))	Ω(√T)*	0	0
Table 1: Regret bounds derived in this paper. The participant regret lower bound is instance independent indicated by  $(\ast)$ . Rest are instance dependent. Lower bound of 0 indicates absence of non-trivial bound.
Several comments on our main results are in order.
Social and Individual Regret: The social regret as well as the individual regret of the optimal non-participating buyers and sellers under the average mechanism with confidence bound bids grow as  $O(\log (T))$ . However, the individual regret of the optimal participating agents grows as  $O(\sqrt{T\log(T)})$ . The social regret, is determined by how many times a participant buyers fails to participate, and non-participants end up participating. Also, the non-participants incur individual
regret through participation. The effect of price setting is not present in both the cases, and counting the number of bad events lead to  $O(\log (T))$  regret. For the participant agents the error in price-setting dominates their respective individual regret of  $O(\sqrt{T\log(T)})$ .
Individually rational for  $\Delta = \omega (\sqrt{\log(T) / T})$ : We observe that the non-participating buyers and sellers, are only having individual regret  $O(\log (T) / \Delta)$ . And the participating buyers and sellers are having individual regret  $O(\sqrt{T\log(T)})$ . This is reassuring as this does not discourage buyers and sellers from participation for a large range of system a-priori. Indeed, for the last participating buyer and seller the utility is  $(B_K^* -S_K^*) / 2$  which is close to  $\Delta$ . Hence, as long as  $\Delta T = \omega (\max \{\log (T) / \Delta ,\sqrt{T\log(T)}\})$  or  $\Delta = \omega (\sqrt{\log(T) / T})$  a non-participating buyer or seller prefers entering the market then discovering her price and getting out, as compared to not participating in the beginning. Also, participating buyer or seller is guaranteed return through participation.
Incentives and Deviations: We now discuss the incentives of individual users closely following the notion of symmetric equilibrium in double auction [42]. In this setting, a myopic agent who knows her true valuation, and greedily maximizes her reward in each round assuming all the non-strategic agents use confidence-based bidding. For average mechanism only the price-setting agents (i.e. the  $K^{*}$ -th buyer and the  $K^{*}$ -th seller) have incentive to deviate from their true valuation to increase their single-round reward. The non-price-setting agents are truthful. When the  $K^{*}$ -th buyer deviates then each participating buyer has an average per round surplus of  $(B_{K^{*}} - \max(S_{K^{*}}, B_{K^{*}+1})) / 2$ , and each participating seller has the same deficit. On the contrary, when the  $K^{*}$ -th seller deviates each participating seller has average per round surplus of  $(S_{K^{*}} - \min(B_{K^{*}}, S_{K^{*}+1})) / 2$  surplus, and each participating buyer has the same average deficit in each round. See Appendix C for more discussions.
Scaling of Regret with  $N,M,K^{*},\Delta$