SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yigit Efe Erginbas<sup>1</sup>
Thomas Courtade<sup>1</sup>
Soham Phade2
<sup>1</sup>University of California, Berkeley
$^{2}$ Wayve Technologies
Online pricing has been the focus of extensive research in recent years, particularly in the context of selling an item to sequentially arriving users. However, what if a provider wants to maximize revenue by selling multiple items to multiple users in each round? This presents a complex problem, as the provider must intelligently offer the items to those users who value them the most without exceeding their highest acceptable prices. In this study, we tackle this challenge by designing online algorithms that can efficiently offer and price items while learning user valuations from accept/reject feedback. We focus on three user valuation models (fixed valuations, random experiences, and random valuations) and provide algorithms with nearly-optimal revenue regret guarantees. In particular, for any market setting with  $N$  users,  $M$  items, and load  $L$  (which roughly corresponds to the maximum number of simultaneous allocations possible), our algorithms achieve regret of order  $O(NM\log \log (LT))$  under fixed valuations model,  $\tilde{O} (\sqrt{NMLT})$  under random experiences model and  $\tilde{O} (\sqrt{NMLT})$  under random valuations model in  $T$  rounds.
The ability to design algorithms that can achieve the optimal sale of goods to multiple users having time-varying valuations for each of the goods is both timely and relevant, given the explosion in the use of data in large-scale systems. This problem is commonly encountered in various contexts, such as in the e-commerce (Amazon, eBay), ride-share (Uber, Lyft), airline, and hotel (Airbnb, Booking.com) industries. Since the provider's goal of maximizing revenue can only be achieved through a delicate balance of considering prior transactions and adjusting offers and prices, it presents a unique opportunity to advance our understanding of dynamic pricing.
We presently consider the problem of designing algorithms that aim to optimize the sale of multiple goods to multiple users having time-varying valuations over the course of repeated rounds. At each round, the provider offers each item to a user at a chosen price, and users decide whether or not to buy, by comparing the offered price to their private valuation for the good. The provider may decide on offers and prices based on outcomes of prior transactions, but each individual user accepts or rejects their offer based only on their valuation for the current round. The provider's goal is to maximize the revenue accumulated in multiple rounds by judiciously selecting the offers and associated prices.
The provider, who is endowed with multiple items at each round, repeatedly offers these items to multiple users at well-chosen prices. In response, the provider obtains feedback regarding the acceptance or rejection decisions of the users and receives revenue for the accepted items. The provider's goal is to maximize the accumulated revenue over time by making offers that respect the endowment constraints and user demands. In the process of identifying the best way of offering the items to target users, the provider encounters challenges regarding two separate aspects of the problem; such challenges are addressed in our work. As depicted in Figure 1, the first challenge is to learn user preferences from interactive feedback; the second is to find offers and prices that will result in maximal revenue.
The challenge of learning arises from the fact that the provider does not have knowledge of the user valuations ahead of time and hence has to learn them while continually taking action. On the other hand, the challenge of offering and pricing stems from both of the facts that (a) available items are scarce and (b) user demands are limited. Therefore, to achieve better outcomes, the provider needs to make careful decisions on which items to offer to which user within the limits of these constraints. However, since the valuations of the users are unknown, the offering and pricing decisions involve a trade-off between learning individual customer preferences in order to increase long-term revenues and earning short-term revenues by leveraging the information acquired so far.
In this study, we focus on algorithms that offer each item to only one user during each round. This assumption eliminates the risk of multiple users requesting the same item and removes the need for the provider to decide who should receive it. Relaxing this assumption could possibly offer more flexibility and revenue potential during the earlier phases while learning the val
valuations. However, it does not result in a loss in the maximum achievable revenue under known valuations, and hence our algorithms can still achieve no-regret guarantees. Additionally, offering a limited number of items simplifies the process for users by reducing the number of choices they need to consider. As the provider gains insights into user valuations over time, the offers become more tailored and relevant, enabling users to focus on evaluating items that the provider believes will be of the highest interest to them. This saves time and reduces the effort required to make a decision.
Figure 1: The provider's goal is to maximize the revenue obtained from sales over multiple rounds. The provider decides on offers and prices while interactively learning the user valuations from the accept/reject feedback. At each round, only a subset of items may be available for sale and only a subset of users may be active.
To the best of our knowledge, we are the first to address the problem of dynamic pricing for the sale of multiple items to multiple users with unknown valuations. Our contributions are as follows.
- We consider three user valuation models: fixed valuations, random experiences, and random valuations. While the fixed valuation and random valuation models are standard models explored extensively in prior work on dynamic pricing (Bubeck et al., 2019), we also propose and analyze the random experiences model as a more realistic representation of user behavior.
- We introduce a problem-dependent load parameter  $L$  that roughly corresponds to the maximum number of simultaneous allocations possible (see Def. 2). We uncover its crucial role in characterizing problem classes for which we can establish matching upper and lower regret bounds.
- We design regret-optimal (up to smaller order terms) algorithms for each setting. For any market setting with  $N$  users,  $M$  items, and maximum load  $L$ , our algorithms achieve regret  $O(NM \log \log (LT))$  under fixed valuations model,  $\widetilde{O}(\sqrt{NMLT})$  under random experiences model and  $\widetilde{O}(\sqrt{NMLT})$  under random valuations model in  $T$  rounds.
- All proposed algorithms have computational complexity  $\mathcal{O}(NML)$  per round in the worst case. The algorithms for fixed valuations and random experiences models have space complexity  $\mathcal{O}(NM)$  while the algorithm for random valuations has space complexity  $\mathcal{O}(NM(LT)^{1/4})$ .
Bandits for dynamic posted-pricing. The problem of dynamic pricing has been typically modeled as a variant of the multi-armed bandit problem starting with Rothschild (1974). In their seminal work, Kleinberg and Leighton (2003) developed a more general and widely-appreciated framework to maximize revenue from selling copies of a single good to sequentially arriving users. In the following years, there has been a growing body of work on multi-period single-product dynamic
pricing problems under different user valuation models including non-parametric models (Besbes and Zeevi, 2009; Keskin and Zeevi, 2014; Cesa-Bianchi et al., 2019; Bubeck et al., 2019), contextual (feature-based) models (Paes Leme and Schneider, 2018; Cohen et al., 2020; Xu and Wang, 2021), and other parametric models (Araman and Caldentey, 2009; Broder and Rusmevichientong, 2012; Harrison et al., 2012; Chen and Farias, 2013; Besbes and Zeevi, 2015; Ferreira et al., 2018). However, all of these works address the posted price problem for selling a single item in each round. Our contribution stands out by considering the combinatorial aspect of the allocation problem faced in simultaneously selling multiple items, a factor that was not taken into account in prior literature.
Combinatorial multi-armed bandits. The semi-bandits framework of Audibert et al. (2011) and the combinatorial multi-armed bandits frameworks of Chen et al. (2013) and Kveton et al. (2015) model problems where a player selects a combination of arms in each round and observes random rewards from the played arms. Therefore, the selection of the offers in our setting shows parallelism with these frameworks. However, their algorithmic solutions cannot be directly applied to our problem because the feedback and reward mechanisms in dynamic pricing are crucially different than the models considered in this literature. It is mainly because the feedback (acceptance/rejection) and reward (revenue) are not only affected by the offers but also by the accompanying prices.
Bandits in matching markets. One recent line of related literature in computational economics studies algorithms for learning socially-optimal matching in two-sided markets (Liu et al., 2020; Johari et al., 2017; Jagadeesan et al., 2021). These frameworks can be used to model the problem of allocating multiple items to multiple users with unknown valuations with the goal of maximizing social welfare. However, these works only consider scenarios where all the users accept their matchings (i.e. offers) without being affected by prices and send the provider real-valued random feedback representing the welfare they achieve from this matching. In their recent work, Erginbas et al. (2023) also analyze a similar problem of optimal and stable allocations and further allow users to accept or reject their recommendations based on the prices. However, their framework also requires random feedback to be sent to the provider regarding their valuation for each accepted matching, whereas our problem setting limits the provider to only observe acceptance or rejection decisions.
Learning in repeated auctions. The learning literature on auctions considers both offline (Morgenstern and Roughgarden, 2016; Cai and Daskalakis, 2017) and online approaches (Bar-Yossef et al., 2002; Lavi and Nisan, 2000) to maximize the provider's revenue from selling multiple items to multiple users. Nonetheless, the implementation of these auctions often presents significant difficulties due to the inherent complexities in obtaining precise valuations from human participants for all the items. As a result, there is a growing interest in designing mechanisms that are accessible, simple to use, and can easily elicit the valuations of the buyers while maximizing the revenue of the provider. In this direction, our approach in this study follows the main premise of posted-price mechanisms (Feldman et al., 2015; Einav et al., 2018), in which the provider sets a fixed price for each item, and buyers decide whether or not to purchase the item at that price.
Notation: We use bold font for vectors  $\mathbf{x}$  and matrices  $\mathbf{X}$ , and calligraphic font  $\mathcal{X}$  for sets. For a vector  $\mathbf{x}$ , we denote its  $i$ -th entry by  $x_{i}$  and for a matrix  $\mathbf{X}$ , we denote its  $(i,j)$ -th entry by  $x_{ij}$ . For any positive integer  $n$ , we use  $[n]$  to denote the set  $\{1,2,\ldots,n\}$ . For real numbers  $a$  and  $b$ , we use  $a \wedge b$  to denote their minimum and  $a \vee b$  to denote their maximum.
Suppose the market consists of a set of users  $\mathcal{N}$  of size  $N$  and a set of items  $\mathcal{I}$  of size  $M$ . At each round  $t \in [T]$  over some fixed time horizon  $T$ , a provider is endowed with a subset of the items denoted by  $\mathcal{E}^t \subseteq \mathcal{I}$  and tries to sell these items to users with the goal of obtaining revenue. We assume that endowed items  $\mathcal{E}^t$  are only available for sale at time interval  $t$ . That is, the items that are not sold at a round cannot be stored to be sold in future rounds.
Each user  $u \in \mathcal{N}$  has an unknown and possibly time-varying valuation  $v_{ui}^{t}$  for each item  $i$  at round  $t$ . In this work, we consider that each user  $u$  has a time-varying request for at most  $d_u^t$  items and has an additive utility model over items. Thus, the provider decides on a price vector  $p^t \in \mathbb{R}_+^M$  and offers each user  $u \in \mathcal{N}$  a subset of the available items denoted by  $S_{u}^{t} \subseteq \mathcal{E}^{t}$  of size at most  $|\mathcal{S}_u^t| \leq d_u^t$ . Then, users decide to accept a subset of their offered items  $A_{u}^{t} \subseteq S_{u}^{t}$  based on their valuations and the prices of the items in order to maximize their surplus given by  $\sum_{i \in A_u^t} (v_{ui}^t - p_i^t)$ .
<sup>1</sup>Our framework readily extends to scenarios where the provider is endowed with multiple copies of each type of item. However, for simplicity in our analysis and presentation, we do not consider this generalization.
Due to the additive utility assumption, when user  $u$  is offered  $S_{u}^{t}$ , it accepts all items  $i \in S_{u}^{t}$  that give positive surplus (i.e.  $v_{ui}^{t} \geq p_{i}^{t}$ ) while rejecting all other items. Hence, the set of accepted items can be written as
$$
\mathcal {A} _ {u} ^ {t} = \left\{i \in \mathcal {S} _ {u} ^ {t}: v _ {u i} ^ {t} \geq p _ {i} ^ {t} \right\}.
$$
For future reference, we also denote the collections of offered and accepted items at time  $t$  by  $\mathcal{S}^t = \{\mathcal{S}_u^t | u \in \mathcal{N}\}$  and  $\mathcal{A}^t = \{\mathcal{A}_u^t | u \in \mathcal{N}\}$ , respectively. To eliminate the possibility of an item getting accepted by multiple users, we consider posted-price offer mechanisms that offer each item to at most one user. Therefore, the offered sets of items are disjoint, i.e.  $S_u^t \cap S_{u'}^t = \emptyset$  for  $u \neq u'$ .
Whenever a user accepts the offer of item  $i$  at round  $t$ , this sale generates  $p_i^t$  revenue for the provider. Therefore, the cumulative revenue obtained over  $T$  rounds equals to
$$
\sum_ {t = 1} ^ {T} \sum_ {u \in \mathcal {N}} \sum_ {i \in \mathcal {A} _ {u} ^ {t}} p _ {i} ^ {t} = \sum_ {t = 1} ^ {T} \sum_ {u \in \mathcal {N}} \sum_ {i \in \mathcal {S} _ {u} ^ {t}} p _ {i} ^ {t} \mathbb {1} \left\{v _ {u i} ^ {t} \geq p _ {i} ^ {t} \right\}. \tag {1}
$$
After the provider decides on the price vector  $\pmb{p}^t$  and the offers  $\mathcal{S}^t = \{\mathcal{S}_u^t | u \in \mathcal{N}\}$ , the users report their set of accepted items  $\mathcal{A}^t = \{\mathcal{A}_u^t | u \in \mathcal{N}\}$ . We denote by  $H_t$  the history  $\{\mathcal{S}^\tau, \pmb{p}^\tau, \mathcal{A}^\tau\}_{\tau=1}^{t-1}$  of observations available to the provider when choosing the next set of offers  $\mathcal{S}^t$  along with the next price vector  $\pmb{p}^t$ . The provider employs a policy  $\pi = \{\pi^t | t \in \mathbb{N}\}$ , which is a sequence of functions, each mapping the history  $H_t$  to an action  $(\mathcal{S}^t, \pmb{p}^t)$ .
The task of the provider is to repeatedly offer the items to users and choose the prices such that it can achieve maximal revenue. To evaluate policies in achieving this objective, we define regret metrics that measure the gap between the expected revenue of policy  $\pi$  and an optimal algorithm.
Definition 1. For a policy  $\pi$ , its revenue regret in  $T$  rounds is defined as
$$
\mathcal {R} (T, \boldsymbol {\pi}) = \mathrm {O P T} - \sum_ {t = 1} ^ {T} \sum_ {u \in \mathcal {N}} \sum_ {i \in \mathcal {S} _ {u} ^ {t}} p _ {i} ^ {t} \mathbb {1} \left\{v _ {u i} ^ {t} \geq p _ {i} ^ {t} \right\}, \tag {2}
$$
where OPT denotes the revenue of the optimal algorithm which will be defined separately for each different valuation model in Section 4.
We can also represent the offers  $\mathcal{S}^t = \{\mathcal{S}_u^t | u \in \mathcal{N}\}$  using binary variables  $x_{ui}^t = \mathbb{1}\{i \in \mathcal{S}_u^t\}$  that indicate whether item  $i$  is a member of each of the sets  $\mathcal{S}_u^t$ . With this definition, each variable  $x_{ui}^t$  is equal to 1 if user  $u$  is offered item  $i$  at time  $t$  and 0 otherwise. We collect these variables into a matrix  $\mathbf{X}^t \in \{0,1\}^{N \times M}$  called the offer matrix. Due to endowment and demand constraints, the offer matrix  $\mathbf{X}^t$  at each time  $t$  needs to belong to the set
$$
\mathcal {X} ^ {t} = \left\{\boldsymbol {X} \in \{0, 1 \} ^ {N \times M}: \sum_ {i \in \mathcal {I}} x _ {u i} ^ {t} \leq d _ {u} ^ {t}, \forall u \in \mathcal {N} \text {a n d} \sum_ {u \in \mathcal {N}} x _ {u i} \leq e _ {i} ^ {t}, \forall i \in \mathcal {I} \right\}, \tag {3}
$$
where each endowment quantity  $e_i^t = \mathbb{1}\{i \in \mathcal{E}^t\}$  is equal to 1 when item  $i$  is available to be offered at time  $t$ , and 0 otherwise. Using this notation, we can write the cumulative revenue as
$$
\sum_ {t = 1} ^ {T} \sum_ {u \in \mathcal {N}} \sum_ {i \in \mathcal {N}} x _ {u i} ^ {t} p _ {i} ^ {t} \mathbb {1} \left\{v _ {u i} ^ {t} \geq p _ {i} ^ {t} \right\}. \tag {4}
$$
Lastly, we define the maximum load parameter which refers to the maximum amount of simultaneous demand and supply in the market. Formally,
Definition 2. For a problem with endowment sequence  $(\mathcal{E}^t)_{1\leq t\leq T}$  and demand sequences  $(d_u^t)_{1\leq t\leq T}$  of all users  $u\in \mathcal{N}$ , the maximum load is defined as
$$
L = \max  _ {1 \leq t \leq T} \left\{D _ {t} \wedge E _ {t} \right\}, \tag {5}
$$
where  $D_{t} \coloneqq \sum_{u \in \mathcal{N}} d_{u}^{t}$  and  $E_{t} \coloneqq |\mathcal{E}^{t}|$  are the total demand and endowment at time  $t$ , respectively.
Note that the maximum load parameter  $L$  is an upper bound for the number of offers that can be made at any round  $t$ . This parameter is central to our analysis since the problem becomes more complex as the maximum load increases.
2In this definition, we assume that all tie-breaks are resolved in favor of the provider.
3.1 Summary of results
Our goal in this work is to provide insights into how to strategize multi-round posted-price offers when the provider does not have prior knowledge of user valuations. In particular, we consider the problem under three different valuation models as described below.
1. Fixed valuations: The valuations of users do not change over time. Formally, there exist values  $v_{ui}$  such that  $v_{ui}^{t} = v_{ui}$  for all  $t \in [T]$ .
2. Random experiences: The valuations of users are given as their average historical experience where each experience is independently drawn from distributions specific to each user and each item. Formally, we consider the experience of user  $u$  with item  $i$  to be given as the random variable  $z_{ui}^{t}$  independently drawn from some distribution with cumulative distribution function  $F_{ui}$  for all rounds at which user  $u$  has accepted item  $i$ . Then, each valuation  $v_{ui}^{t}$  is given as the average of the past experiences, i.e., the average of values  $\{z_{ui}^{\tau} | \tau < t, i \in \mathcal{A}_{u}^{t}\}$ .
3. Random valuations: The valuations of users at different rounds are independently drawn from distributions specific to each user and each item. Formally, there exist cumulative distribution functions  $F_{ui}$  for all  $(u,i) \in \mathcal{N} \times \mathcal{I}$  such that each  $v_{ui}^{t}$  is independently drawn from a distribution with cdf  $F_{ui}$ .
For each of the models described above, we derive upper and lower bounds for revenue regret, matching up to logarithmic factors. We summarize our results in Table 1.
Table 1: Upper and lower bounds for revenue regret under different valuation models.
Model	Upper Bound	Lower Bound
Fixed Valuations (Section 4.1)	O(NM log log(LT))	Ω(NM log log(LT/NM))
Random Experiences (Section 4.2)	\( \widetilde{O}\left( \sqrt{NMLT} \right) \)	Ω(√NMLT)
Random Valuations (Section 4.3)	\( \widetilde{O}\left( \sqrt{NMLT} \right) \)	Ω(√NMLT)
Remark. The frameworks of fixed valuations and random valuations can also be used to model settings where each interacting user is associated with a type that determines their valuations. In this case, the set  $\mathcal{N}$  corresponds to the set of all user types and each demand parameter  $d_u^t$  represents the total demand of users of type  $u$  in round  $t$ . Under the fixed valuations model, all users of type  $u$  have valuation  $v_{ui}^{t} = v_{ui}$  for item  $i$  at all rounds  $t$ . Under the random valuations model, each user of type  $u$  has a random valuation with distribution  $F_{ui}$  for item  $i$  independently for each user at each time  $t$ . Since at most one user receives any item  $i$  in any round, it is sufficient to consider a single random valuation  $v_{ui}^{t}$  for each type  $u$  and item  $i$  at time  $t$ .
We provide algorithms for achieving sub-linear revenue regret under different user valuation models described in Section 3.1. While the strategies for the selection of offers are similar under different models, we use different pricing strategies for different models as depicted in Figure 2.
(a) Fixed valuations
(b) Random experiences
Figure 2: The selection of prices for different valuation models. (a) If the offer is accepted (rejected), we set the new value of  $a_{ui}$  ( $b_{ui}$ ) as the offered price  $p_i^t$ . (b) If  $n_{ui} = 2^j$  for some  $j \in \mathbb{N}$  and the offer is accepted (rejected), we set the new value of  $a_{ui}$  ( $b_{ui}$ ) as a number smaller (larger) than  $p_i^t$ . (c) After offering an item at price  $k / K$ , we update the  $k^{\text{th}}$  confidence interval based on whether the offer is accepted or rejected.
(c) Random valuations
Algorithm 1 Offerings with incremental search prices
$a_{ui}\gets 0,b_{ui}\gets 1$  and  $\beta_{ui}\gets 0.5$  for all  $u\in \mathcal{N},i\in \mathcal{I}$    
for  $t = 1,2,\ldots ,T$  do Calculate  $\pmb{X}^{t}$  by solving problem (8) using  $b_{ui}$  values.   
for  $(u,i)\in \{(u,i):x_{ui}^t = 1\}$  do ▷ offer item i to user u  $p_i^t\gets a_{ui} + \beta_{ui}\mathbb{1}\{b_{ui} - a_{ui}\geq \frac{1}{LT}\}$  ▷ set the price Offer item i to user u at price  $p_i^t$  and observe  $\mathbb{1}\{v_{ui}^t\geq p_i^t\}$  if  $v_{ui}^t\geq p_i^t$  then  $a_{ui}\gets a_{ui}\lor p_i^t$  else  $b_{ui}\gets b_{ui}\land p_i^t$  ▷ update the interval if  $b_{ui} - a_{ui}\leq \beta_{ui}$  then  $\beta_{ui}\gets \beta_{ui}^{2}$  end for
4.1 Revenue maximization under fixed valuations
In this section, we focus on the scenario where the provider makes posted-price offers to users whose valuations are fixed over time. We formalize this condition in the following assumption.
Assumption 1. The valuation of any user  $u \in \mathcal{N}$  for any item  $i \in \mathcal{I}$  is given by  $v_{ui}^{t} = v_{ui}$  for some  $v_{ui} \in [0,1]$  at all rounds  $t \in [T]$ .
In the case of fixed valuations, we define the optimum strategy as the one that maximizes the revenue under complete information on user valuations. Therefore, the optimum offers  $\mathbf{X}_{*}^{t}$  and prices  $p_{*}^{t}$  for all rounds  $t$  are given by maximizing the cumulative revenue as given in (4). That is, we define the optimum objective value as
$$
\mathrm {O P T} = \sum_ {t = 1} ^ {T} \left[ \max  _ {\boldsymbol {X} ^ {t} \in \mathcal {X} ^ {t}} \max  _ {\boldsymbol {p} ^ {t} \in \mathbb {R} _ {+} ^ {M}} \sum_ {u \in \mathcal {N}} \sum_ {i \in \mathcal {I}} x _ {u i} ^ {t} p _ {i} ^ {t} \mathbb {1} \left\{v _ {u i} \geq p _ {i} ^ {t} \right\} \right], \tag {6}
$$
and let  $\pmb{X}_{*}^{t}$  and  $\pmb{p}_{*}^{t}$  be the solutions that maximize term  $t$ . Note that for any allocation  $\pmb{X}^{t} \in \mathcal{X}^{t}$ , we have  $\sum_{u \in \mathcal{N}} x_{ui}^{t} \leq e_{i}^{t} \leq 1$ . Therefore, whenever an item  $i$  is offered to a user  $u$ , the optimum price of item  $i$  is equal to the user's valuation  $v_{ui}$ . In other words, the optimum price for item  $i$  is given by  $p_{i}^{t} = \sum_{u \in \mathcal{N}} x_{ui}^{t} v_{ui}$ . Based on this observation, the revenue-maximizing offer can be found as