Title: Online Pricing for Multi-User Multi-Item Markets
Venue: NeurIPS 2023
Accepted: yes
Keywords: revenue

price

offer

online
Abstract: Online pricing has been the focus of extensive research in recent years, particularly in the context of selling an item to sequentially arriving users. However, what if a provider wants to maximize revenue by selling multiple items to multiple users in each round? This presents a complex problem, as the provider must intelligently offer the items to those users who value them the most without exceeding their highest acceptable prices. In this study, we tackle this challenge by designing online algorithms that can efficiently offer and price items while learning user valuations from accept/reject feedback. We focus on three user valuation models (fixed valuations, random experiences, and random valuations) and provide algorithms with nearly-optimal revenue regret guarantees. In particular, for any market setting with $N$ users, $M$ items, and load $L$ (which roughly corresponds to the maximum number of simultaneous allocations possible), our algorithms achieve regret of order $O(NM\log\log(LT))$ under fixed valuations model, $\widetilde{O}(\sqrt{NMLT})$ under random experiences model and $\widetilde{O}(\sqrt{NMLT})$ under random valuations model in $T$ rounds.

Online Pricing for Multi-User Multi-Item Markets

Abstract

Online pricing has been the focus of extensive research in recent years, particularly in the context of selling an item to sequentially arriving users. However, what if a provider wants to maximize revenue by selling multiple items to multiple users in each round? This presents a complex problem, as the provider must intelligently offer the items to those users who value them the most without exceeding their highest acceptable prices. In this study, we tackle this challenge by designing online algorithms that can efficiently offer and price items while learning user valuations from accept/reject feedback. We focus on three user valuation models (fixed valuations, random experiences, and random valuations) and provide algorithms with nearly-optimal revenue regret guarantees. In particular, for any market setting with N users, M items, and load L (which roughly corresponds to the maximum number of simultaneous allocations possible), our algorithms achieve regret of order O ( NM log log( LT )) under fixed valuations model, ˜ O ( √ NMLT ) under random experiences model and ˜ O ( √ NMLT ) under random valuations model in T rounds.

1 Introduction

The ability to design algorithms that can achieve the optimal sale of goods to multiple users having time-varying valuations for each of the goods is both timely and relevant, given the explosion in the use of data in large-scale systems. This problem is commonly encountered in various contexts, such as in the e-commerce (Amazon, eBay), ride-share (Uber, Lyft), airline, and hotel (Airbnb, Booking.com) industries. Since the provider's goal of maximizing revenue can only be achieved through a delicate balance of considering prior transactions and adjusting offers and prices, it presents a unique opportunity to advance our understanding of dynamic pricing.

We presently consider the problem of designing algorithms that aim to optimize the sale of multiple goods to multiple users having time-varying valuations over the course of repeated rounds. At each round, the provider offers each item to a user at a chosen price, and users decide whether or not to buy, by comparing the offered price to their private valuation for the good. The provider may decide on offers and prices based on outcomes of prior transactions, but each individual user accepts or rejects their offer based only on their valuation for the current round. The provider's goal is to maximize the revenue accumulated in multiple rounds by judiciously selecting the offers and associated prices.

The provider, who is endowed with multiple items at each round, repeatedly offers these items to multiple users at well-chosen prices. In response, the provider obtains feedback regarding the acceptance or rejection decisions of the users and receives revenue for the accepted items. The provider's goal is to maximize the accumulated revenue over time by making offers that respect the endowment constraints and user demands. In the process of identifying the best way of offering the items to target users, the provider encounters challenges regarding two separate aspects of the problem; such challenges are addressed in our work. As depicted in Figure 1, the first challenge is to learn user preferences from interactive feedback; the second is to find offers and prices that will result in maximal revenue.

The challenge of learning arises from the fact that the provider does not have knowledge of the user valuations ahead of time and hence has to learn them while continually taking action. On the other hand, the challenge of offering and pricing stems from both of the facts that (a) available items are scarce and (b) user demands are limited. Therefore, to achieve better outcomes, the provider needs to make careful decisions on which items to offer to which user within the limits of these constraints. However, since the valuations of the users are unknown, the offering and pricing decisions involve a trade-off between learning individual customer preferences in order to increase long-term revenues and earning short-term revenues by leveraging the information acquired so far.

In this study, we focus on algorithms that offer each item to only one user during each round. This assumption eliminates the risk of multiple users requesting the same item and removes the need for the provider to decide who should receive it. Relaxing this assumption could possibly offer more flexibility and revenue potential during the earlier phases while learning the val-

user

val uat i ons

O ff er s   &

Paym ent   &

Feedback

Provi der

U sers

It em s

GS

Gs

uations. However, it does not result in a loss in the maximum achievable revenue under known valuations, and hence our algorithms can still achieve no-regret guarantees. Additionally, offering a limited number of items simplifies the process for users by reducing the number of choices they need to consider. As the provider gains insights into user valuations over time, the offers become more tailored and relevant, enabling users to focus on evaluating items that the provider believes will be of the highest interest to them. This saves time and reduces the effort required to make a decision.

1.1 Our contributions

To the best of our knowledge, we are the first to address the problem of dynamic pricing for the sale of multiple items to multiple users with unknown valuations. Our contributions are as follows.

We consider three user valuation models: fixed valuations, random experiences, and random valuations. While the fixed valuation and random valuation models are standard models explored extensively in prior work on dynamic pricing (Bubeck et al., 2019), we also propose and analyze the random experiences model as a more realistic representation of user behavior.

We introduce a problem-dependent load parameter L that roughly corresponds to the maximum number of simultaneous allocations possible (see Def. 2). We uncover its crucial role in characterizing problem classes for which we can establish matching upper and lower regret bounds.

We design regret-optimal (up to smaller order terms) algorithms for each setting. For any market setting with N users, M items, and maximum load L , our algorithms achieve regret O ( NM log log( LT )) under fixed valuations model, ˜ O ( √ NMLT ) under random experiences model and ˜ O ( √ NMLT ) under random valuations model in T rounds.

All proposed algorithms have computational complexity O ( NML ) per round in the worst case. The algorithms for fixed valuations and random experiences models have space complexity O ( NM ) while the algorithm for random valuations has space complexity O ( NM ( LT ) 1 / 4 ) .

2 Related work

Bandits for dynamic posted-pricing. The problem of dynamic pricing has been typically modeled as a variant of the multi-armed bandit problem starting with Rothschild (1974). In their seminal work, Kleinberg and Leighton (2003) developed a more general and widely-appreciated framework to maximize revenue from selling copies of a single good to sequentially arriving users. In the following years, there has been a growing body of work on multi-period single-product dynamic

pricing problems under different user valuation models including non-parametric models (Besbes and Zeevi, 2009; Keskin and Zeevi, 2014; Cesa-Bianchi et al., 2019; Bubeck et al., 2019), contextual (feature-based) models (Paes Leme and Schneider, 2018; Cohen et al., 2020; Xu and Wang, 2021), and other parametric models (Araman and Caldentey, 2009; Broder and Rusmevichientong, 2012; Harrison et al., 2012; Chen and Farias, 2013; Besbes and Zeevi, 2015; Ferreira et al., 2018). However, all of these works address the posted price problem for selling a single item in each round. Our contribution stands out by considering the combinatorial aspect of the allocation problem faced in simultaneously selling multiple items , a factor that was not taken into account in prior literature.

Combinatorial multi-armed bandits. The semi-bandits framework of Audibert et al. (2011) and the combinatorial multi-armed bandits frameworks of Chen et al. (2013) and Kveton et al. (2015) model problems where a player selects a combination of arms in each round and observes random rewards from the played arms. Therefore, the selection of the offers in our setting shows parallelism with these frameworks. However, their algorithmic solutions cannot be directly applied to our problem because the feedback and reward mechanisms in dynamic pricing are crucially different than the models considered in this literature. It is mainly because the feedback (acceptance/rejection) and reward (revenue) are not only affected by the offers but also by the accompanying prices.

Bandits in matching markets. One recent line of related literature in computational economics studies algorithms for learning socially-optimal matching in two-sided markets (Liu et al., 2020; Johari et al., 2017; Jagadeesan et al., 2021). These frameworks can be used to model the problem of allocating multiple items to multiple users with unknown valuations with the goal of maximizing social welfare. However, these works only consider scenarios where all the users accept their matchings (i.e. offers) without being affected by prices and send the provider real-valued random feedback representing the welfare they achieve from this matching. In their recent work, Erginbas et al. (2023) also analyze a similar problem of optimal and stable allocations and further allow users to accept or reject their recommendations based on the prices. However, their framework also requires random feedback to be sent to the provider regarding their valuation for each accepted matching, whereas our problem setting limits the provider to only observe acceptance or rejection decisions.

Learning in repeated auctions. The learning literature on auctions considers both offline (Morgenstern and Roughgarden, 2016; Cai and Daskalakis, 2017) and online approaches (Bar-Yossef et al., 2002; Lavi and Nisan, 2000) to maximize the provider's revenue from selling multiple items to multiple users. Nonetheless, the implementation of these auctions often presents significant difficulties due to the inherent complexities in obtaining precise valuations from human participants for all the items. As a result, there is a growing interest in designing mechanisms that are accessible, simple to use, and can easily elicit the valuations of the buyers while maximizing the revenue of the provider. In this direction, our approach in this study follows the main premise of posted-price mechanisms (Feldman et al., 2015; Einav et al., 2018), in which the provider sets a fixed price for each item, and buyers decide whether or not to purchase the item at that price.

3 Problem setting

Notation: We use bold font for vectors x and matrices X , and calligraphic font X for sets. For a vector x , we denote its i -th entry by x i and for a matrix X , we denote its ( i, j ) -th entry by x ij . For any positive integer n , we use [ n ] to denote the set { 1 , 2 , . . . , n } . For real numbers a and b , we use a ∧ b to denote their minimum and a ∨ b to denote their maximum.

Suppose the market consists of a set of users N of size N and a set of items I of size M . At each round t ∈ [ T ] over some fixed time horizon T , a provider is endowed with a subset of the items denoted by E t ⊆ I and tries to sell these items to users with the goal of obtaining revenue. 1 We assume that endowed items E t are only available for sale at time interval t . That is, the items that are not sold at a round cannot be stored to be sold in future rounds.

Each user u ∈ N has an unknown and possibly time-varying valuation v t ui for each item i at round t . In this work, we consider that each user u has a time-varying request for at most d t u items and has an additive utility model over items. Thus, the provider decides on a price vector p t ∈ R M + and offers each user u ∈ N a subset of the available items denoted by S t u ⊆ E t of size at most |S t u | ≤ d t u . Then, users decide to accept a subset of their offered items A t u ⊆ S t u based on their valuations and the prices of the items in order to maximize their surplus given by ∑ i ∈A t u ( v t ui -p t i ) .

Due to the additive utility assumption, when user u is offered S t u , it accepts all items i ∈ S t u that give positive surplus (i.e. v t ui ≥ p t i ) while rejecting all other items. 2 Hence, the set of accepted items can be written as

̸

For future reference, we also denote the collections of offered and accepted items at time t by S t = {S t u | u ∈ N} and A t = {A t u | u ∈ N} , respectively. To eliminate the possibility of an item getting accepted by multiple users, we consider posted-price offer mechanisms that offer each item to at most one user. Therefore, the offered sets of items are disjoint, i.e. S t u ∩ S t u ′ = ∅ for u = u ′ .

Whenever a user accepts the offer of item i at round t , this sale generates p t i revenue for the provider. Therefore, the cumulative revenue obtained over T rounds equals to

After the provider decides on the price vector p t and the offers S t = {S t u | u ∈ N} , the users report their set of accepted items A t = {A t u | u ∈ N} . We denote by H t the history { S τ , p τ , A τ } t -1 τ =1 of observations available to the provider when choosing the next set of offers S t along with the next price vector p t . The provider employs a policy π = { π t | t ∈ N } , which is a sequence of functions, each mapping the history H t to an action ( S t , p t ) .

The task of the provider is to repeatedly offer the items to users and choose the prices such that it can achieve maximal revenue. To evaluate policies in achieving this objective, we define regret metrics that measure the gap between the expected revenue of policy π and an optimal algorithm.

Definition 1. For a policy π , its revenue regret in T rounds is defined as

where OPT denotes the revenue of the optimal algorithm which will be defined separately for each different valuation model in Section 4.

We can also represent the offers S t = {S t u | u ∈ N} using binary variables x t ui = 1 { i ∈ S t u } that indicate whether item i is a member of each of the sets S t u . With this definition, each variable x t ui is equal to 1 if user u is offered item i at time t and 0 otherwise. We collect these variables into a matrix X t ∈ { 0 , 1 } N × M called the offer matrix. Due to endowment and demand constraints, the offer matrix X t at each time t needs to belong to the set

where each endowment quantity e t i = 1 { i ∈ E t } is equal to 1 when item i is available to be offered at time t , and 0 otherwise. Using this notation, we can write the cumulative revenue as

Lastly, we define the maximum load parameter which refers to the maximum amount of simultaneous demand and supply in the market. Formally,

Definition 2. For a problem with endowment sequence ( E t ) 1 ≤ t ≤ T and demand sequences ( d t u ) 1 ≤ t ≤ T of all users u ∈ N , the maximum load is defined as

where D t := ∑ u ∈N d t u and E t := |E t | are the total demand and endowment at time t , respectively.

Note that the maximum load parameter L is an upper bound for the number of offers that can be made at any round t . This parameter is central to our analysis since the problem becomes more complex as the maximum load increases.

3.1 Summary of results

Our goal in this work is to provide insights into how to strategize multi-round posted-price offers when the provider does not have prior knowledge of user valuations. In particular, we consider the problem under three different valuation models as described below.

Fixed valuations: The valuations of users do not change over time. Formally, there exist values v ui such that v t ui = v ui for all t ∈ [ T ] .

Random experiences: The valuations of users are given as their average historical experience where each experience is independently drawn from distributions specific to each user and each item. Formally, we consider the experience of user u with item i to be given as the random variable z t ui independently drawn from some distribution with cumulative distribution function F ui for all rounds at which user u has accepted item i . Then, each valuation v t ui is given as the average of the past experiences, i.e., the average of values { z τ ui | τ < t, i ∈ A t u } .

Random valuations: The valuations of users at different rounds are independently drawn from distributions specific to each user and each item. Formally, there exist cumulative distribution functions F ui for all ( u, i ) ∈ N × I such that each v t ui is independently drawn from a distribution with cdf F ui .

For each of the models described above, we derive upper and lower bounds for revenue regret, matching up to logarithmic factors. We summarize our results in Table 1.

Remark. The frameworks of fixed valuations and random valuations can also be used to model settings where each interacting user is associated with a type that determines their valuations. In this case, the set N corresponds to the set of all user types and each demand parameter d t u represents the total demand of users of type u in round t . Under the fixed valuations model, all users of type u have valuation v t ui = v ui for item i at all rounds t . Under the random valuations model, each user of type u has a random valuation with distribution F ui for item i independently for each user at each time t . Since at most one user receives any item i in any round, it is sufficient to consider a single random valuation v t ui for each type u and item i at time t .

4 Methodology

We provide algorithms for achieving sub-linear revenue regret under different user valuation models described in Section 3.1. While the strategies for the selection of offers are similar under different models, we use different pricing strategies for different models as depicted in Figure 2.

. . .

. . .

. .

. . .

(a ) F i

x e d  v a l ua t

i o n s

(b) R a n d o m  e x pe r

i e n c e s

(c) R a n d o m  v a l ua t

i o n s

i f r e j

e c t

e d

i f r e j

e c t

e d

i f r e j

e c t

e d

i f a c c e pt e d

i f a c c e pt e d

i f a c c e pt e d

...

...

9log(NMT)

mui

yui(p)

aui

p

bui

aui

bui

2

aui

bui

βui

else

p

t

for somejeIN

if

nui=2j

K

1

K

2

K

k

1

2

4.1 Revenue maximization under fixed valuations

In this section, we focus on the scenario where the provider makes posted-price offers to users whose valuations are fixed over time. We formalize this condition in the following assumption.

Assumption 1. The valuation of any user u ∈ N for any item i ∈ I is given by v t ui = v ui for some v ui ∈ [0 , 1] at all rounds t ∈ [ T ] .

In the case of fixed valuations, we define the optimum strategy as the one that maximizes the revenue under complete information on user valuations. Therefore, the optimum offers X t ∗ and prices p t ∗ for all rounds t are given by maximizing the cumulative revenue as given in (4). That is, we define the optimum objective value as

and let X t ∗ and p t ∗ be the solutions that maximize term t . Note that for any allocation X t ∈ X t , we have ∑ u ∈N x t ui ≤ e t i ≤ 1 . Therefore, whenever an item i is offered to a user u , the optimum price of item i is equal to the user's valuation v ui . In other words, the optimum price for item i is given by p t i = ∑ u ∈N x t ui v ui . Based on this observation, the revenue-maximizing offer can be found as

We note that the integer program in (7) can be written as an instance of maximum weight bipartite matching. Then, using a variant of the Hungarian algorithm for unbalanced bipartite graphs (Ramshaw and Tarjan, 2012), this problem can be solved in space O ( NM ) and time O ( NML ) in the worst case. (See Appendix A for details.)

Since there is no randomness in user responses, every response from a user u about item i gives the provider complete information about a lower or upper bound on v ui , depending on whether the user response was to accept or reject the offered price for the item i . For this reason, the algorithm operates by keeping track of intervals [ a ui , b ui ] that contain v ui value for each u ∈ N , i ∈ I at all rounds. At each time t , the algorithm chooses which items to offer to each user using these intervals and then determines a price for each offered item. The selection of the allocations is done according to the OFU principle (Dani et al., 2008; Abbasi-Yadkori, 2011) in order to ensure low regret. In particular, the offers are chosen by replacing each v ui in problem (7) with b t ui to obtain

Having decided on the offers, the next step is to decide on the price of the offered items. Similar to the challenge we encountered in selecting the offers, the selection of prices should also serve two different goals that are in tension with each other. On one hand, learning new information about users' valuations requires us to set prices to values from ( a ui , b ui ) . On the other hand, to ensure the offers are accepted and generate revenue, we would need to select prices lower than or equal to a ui . The crucial property that enables us to obtain O ( NM log log( T )) regret is that the function p → v ui -p 1 { p ≤ v ui } is asymmetric and decreases more slowly on the left than on the right as discussed in Kleinberg and Leighton (2003). Based on this observation, we design a pricing algorithm

that operates by offering item i to user u at prices increasing with increments of β ui . When the width of the interval [ a ui , b ui ] becomes smaller than the precision parameter β ui , we set the new precision parameter as β 2 ui and continue exploration with this smaller step size. However, as we show in the proof of Theorem 1, continuing to explore indefinitely would result in a regret linear in T . To avoid this issue, the algorithm should stop exploration after achieving a certain level of precision and offer item i to user u at the maximum price that is certainly acceptable, namely a ui .

We provide a summary of this algorithm in Algorithm 1 and provide an upper bound for its regret in Theorem 1. As the lower bound provided in Theorem 2 shows, the regret of this algorithm is order-optimal up to smaller terms.

Theorem 1 (Upper bound for Fixed valuations) . Assuming fixed valuations in a market with N users, M items, and maximum load L ; the regret of Algorithm 1 in T rounds satisfies

Theorem 2 (Lower bound for Fixed valuations) . If N , M , and L ≤ M are given parameters and π is any randomized policy; there exist randomly generated market instances of N users with fixed valuations for M items and maximum load L such that the expected regret of π in T ≥ 4 NM L rounds is Ω( NM log log( LT/NM )) .

4.2 Revenue maximization under random experiences

This section focuses on the case where the valuations of the users are given as the average of their past experience. We assume that each user obtains independent experiences about the items that they accept at every round. Then, the users form their valuation at each time as the average of their experiences so far. Formally, we make the following assumption.

Assumption 2. Whenever a user u ∈ N accepts an item i ∈ I at round t ∈ [ T ] , it obtains an independent random experience z t ui with unknown cdf F ui over [0 , 1] with mean v ui . Then, the valuation of any user u ∈ N for any item i ∈ I at round t ∈ [ T ] is given as

where T t ui = { τ < t : i ∈ A t u } is the set of rounds before round t at which user u has accepted and experienced item i , and m t ui := |T t ui | denotes the size of this set.

Under the random experiences model, we define the optimum strategy as the one that maximizes the revenue with complete information on possible user experiences z t ui , and hence user valuations v t ui at all times. Therefore, the optimum revenue that can be obtained in T rounds is given by

which is a random variable due to the randomness in z t ui and hence the randomness in v t ui . Next, we note that for any fixed sequence of offers { X t | t ∈ [ T ] } , the objective is maximized at prices that satisfy p t i = ∑ u ∈N x t ui v ui at all t . Therefore, we can write the optimum value for the objective as

Note that in problem (11), we need to globally maximize over X t for all rounds t ∈ [ T ] because the values of v t ui depend on selections of X t at previous rounds. In order to deal with this dependency between the selections of X t , in Lemma 9, we show that OPT is not likely to be much larger than the sum of the mean valuations of the best offers at all rounds.

Next, we describe our algorithm in Algorithm 2. As the users accept and experience the items, their valuations converge to the mean of their experience distribution. Therefore, before extracting the mean valuation information from the users, the provider must ensure that users accept and sufficiently experience the items. Then, to extract this information, the algorithm occasionally asks for higher prices while taking the risk of rejected offers. However, since there are at most log( T ) such learning rounds per user-item pair, it does not cause any significant loss in revenue.

Based on these observations, we establish an upper bound for our algorithm's regret in Theorem 3. Furthermore, as indicated by the regret lower bound presented in Theorem 4, our algorithm's regret is order-optimal up to smaller terms.

Theorem 3 (Upper bound for random experiences) . In any market of N users satisfying the random experience model given in Assumption 2 for M items and maximum load L , with probability 1 -2 δ , the revenue regret of Algorithm 2 satisfies

Proof.

Theorem 4 (Lower bound for random experiences) . If N , M , and L ≤ M are given parameters and π is any randomized policy; there exist randomly generated market instances of N users satisfying the random experience model given in Assumption 2 for M items and maximum load L such that the expected regret of π in T ≥ N rounds is Ω( √ NMLT ) .

Proof.

4.3 Revenue maximization under random valuations

In this section, we consider the case where the valuations of the users are given as independent random variables drawn from distributions specific to each user and item. Formally,

Assumption 3. The valuation of any user u ∈ N for any item i ∈ I at round t ∈ [ T ] is given as an independent random variable v t ui with unknown cdf F ui over [0 , 1] .

Given foreknowledge of the distribution of valuations, but not of the individual valuations at different rounds, it is easy to see what the optimal pricing strategy would be. The expected revenue obtained from offering item i to user u at price p is given as ψ ui ( p ) = p (1 -F ui ( p )) , which we call to be the revenue function. Since valuations are independent over time and their distribution is known, the individual responses provide no useful information about future realizations of the valuations. Therefore, the best price for offering item i to user u is given by

Thus, letting ψ ∗ ui = ψ ui ( p ∗ ui ) denote the maximum expected revenue that can be obtained by offering item i to user u , an optimum policy that knows the distribution of valuations can obtain revenue

However, without the knowledge of the distributions, we are required to learn the expected revenue (i.e. rewards) at different prices via exploration. As previously done in the literature on pricing under random valuation models (Kleinberg and Leighton, 2003), we apply techniques from the literature on the multi-armed bandit problem to develop an algorithm with low regret guarantees. To do so, we quantize the set of possible prices by limiting the provider to strategies that only offer prices belonging to the set { 1 /K, 2 /K,... 1 } for suitably chosen K . This brings us into a setting where each offer of item i to a user u at price k/K yields a revenue which is a random variable taking values in [0 , 1] , whose distribution depends on ( u, i, k ) , but the rewards for a given action are i.i.d. across the rounds. Therefore, offering item i to user u and k th price level can be represented as pulling an arm that generates revenue with expectation ψ uik = ψ ui ( k/K ) . In total, the expected revenue of any offering and pricing is

where x t uik are binary variables that denote whether user u is offered item i at k th price level at round t . Due to endowment and demand constraints, these variables at time t must satisfy conditions (1) ∑ i ∈I ∑ K k =1 x uik ≤ d t u , (2) ∑ u ∈N ∑ K k =1 x uik ≤ e t i , and (3) ∑ K k =1 x uik ≤ 1 for all u ∈ N , i ∈ I .

In the literature on combinatorial multi-armed bandits, the standard regret bounds for UCB-based algorithms have an inverse dependency on the gap between the rewards of optimal and suboptimal arms (Kveton et al., 2015). To use similar techniques in proving regret bounds for our algorithm, we make the following hypothesis on the distributions of valuations, which translates directly into bounds on price sub-optimality gaps max k ψ uik -ψ uik .

Assumption 4. The revenue function ψ ui ( p ) has a unique global maximum at p ∗ ui ∈ (0 , 1) , and ψ ′′ ui ( p ∗ ui ) is defined and strictly negative.

Next, we show that our algorithm can attain the regret upper bound stated in Theorem 5. Moreover, as illustrated in Theorem 6, our algorithm's regret is order-optimal up to smaller terms.

Theorem 5 (Upper bound for random valuations) . In any market of N users satisfying the random valuation model given in Assumptions 3 and 4 for M items and maximum load L , with probability 1 -δ , the revenue regret of Algorithm 3 satisfies

Theorem 6 (Lower bound for random valuations) . If N , M , and L ≤ M are given parameters and π is any randomized policy; there exist randomly generated market instances of N users satisfying the random valuation model given in Assumptions 3 and 4 for M items and maximum load L such that the expected regret of π is Ω( √ NMLT ) in T rounds.

5 Numerical experiments

In this section, we demonstrate the efficacy of our proposed algorithms through a numerical study. We provide our results in Figures 3 and 4. At each round t ∈ [ T ] , the provider is endowed with each item i ∈ I (i.e. i ∈ E t ) independently with probability 0 . 5 . On the other hand, each user u ∈ N has a random demand d t u with uniform probability over { 0 , 1 , 2 } . For the case of fixed valuations, we choose each v ui independently from Beta(2 , 2) . For other two models, we set each F ui as the cdf of Beta( α ui , β ui ) where α ui and β ui are uniformly and independently chosen over [1 , 5] .

250

500

750

1000

Iteration

10

20

30

40

Instantaneous Regret

Fixed Valuations

750

1500

2250

3000

Iteration

10

20

30

40

Instantaneous Regret

Random Experiences

1250

2500

3750

5000

Iteration

10

20

30

40

Instantaneous Regret

Random Valuations

2 7

2 9

2 11

2 13

Time Horizon

(

T

)

1

2

Regret

1e4

Fixed Valuations

1000

2000

3000

Time Horizon

(

T

)

2

4

6

Regret

1e4

Random Experiences

1000

2000

3000

Time Horizon

(

T

)

2.5

5.0

7.5

Regret

1e4

Random Valuations

6 Conclusion

Our study presents a comprehensive solution to maximizing expected revenue in a repeated interaction setting, where a provider seeks to sell multiple items to multiple users. By focusing on different valuation models, we design online learning algorithms that can infer user valuations and offer items to those who value them most to ensure approximately optimal revenue regret. The results of this study have important implications for online marketplaces and can help providers optimize pricing strategies and maximize revenue in a dynamic and uncertain environment.

References

Abbasi-Yadkori, Y. (2011). Improved algorithms for linear stochastic bandits. Proceedings of Twenty-Fifth Conference on Neural Information Processing Systems (NeurIPS) , pages 2312-2320.

Araman, V. F. and Caldentey, R. (2009). Dynamic pricing for nonperishable products with demand learning. Operations Research , 57(5):1169-1188.

Audibert, J.-Y., Bubeck, S., and Lugosi, G. (2011). Minimax policies for combinatorial prediction games. In Proceedings of the Twenty-Fourth Annual Conference on Learning Theory (COLT) , volume 19, pages 107-132.

Bar-Yossef, Z., Hildrum, K., and Wu, F. (2002). Incentive-compatible online auctions for digital goods. In Proceedings of the Thirteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 964-970.

Besbes, O. and Zeevi, A. (2009). Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Operations Research , 57(6):1407-1420.

Besbes, O. and Zeevi, A. (2015). On the surprising sufficiency of linear models for dynamic pricing with demand learning. Management Science , 61(4):723-739.

Broder, J. and Rusmevichientong, P. (2012). Dynamic pricing under a general parametric choice model. Operations Research , 60(4):965-980.

Bubeck, S., Devanur, N. R., Huang, Z., and Niazadeh, R. (2019). Multi-scale online learning: Theory and applications to online auctions and pricing. Journal of Machine Learning Research , 20(62):1-37.

Cai, Y. and Daskalakis, C. (2017). Learning multi-item auctions with (or without) samples. In Proceedings of the Fifty-Eighth Annual IEEE Symposium on Foundations of Computer Science (FOCS) , pages 516-527.

Cesa-Bianchi, N., Cesari, T., and Perchet, V. (2019). Dynamic pricing with finitely many unknown valuations. In Proceedings of the Thirtieth International Conference on Algorithmic Learning Theory , pages 247-273.

Chen, W., Wang, Y., and Yuan, Y. (2013). Combinatorial multi-armed bandit: General framework and applications. In Proceedings of the Thirtieth International Conference on Machine Learning (ICML) , volume 28, pages 151-159.

Chen, Y. and Farias, V. F. (2013). Simple policies for dynamic pricing with imperfect forecasts. Operations Research , 61(3):612-624.

Cohen, M. C., Lobel, I., and Paes Leme, R. (2020). Feature-based dynamic pricing. Management Science , 66(11):4921-4943.

Dani, V., Hayes, T., and Kakade, S. M. (2008). Stochastic linear optimization under bandit feedback. Proceedings of the Twenty-First Annual Conference on Learning Theory (COLT) , pages 355-366.

Einav, L., Farronato, C., Levin, J., and Sundaresan, N. (2018). Auctions versus posted prices in online markets. Journal of Political Economy , 126(1):178-215.

Erginbas, Y. E., Phade, S., and Ramchandran, K. (2023). Interactive learning with pricing for optimal and stable allocations in markets. In Proceedings of the Twenty-Sixth International Conference on Artificial Intelligence and Statistics (AISTATS) , pages 9773-9806.

Feldman, M., Gravin, N., and Lucier, B. (2015). Combinatorial auctions via posted prices. In Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 123-135.

Ferreira, K. J., Simchi-Levi, D., and Wang, H. (2018). Online network revenue management using thompson sampling. Operations Research , 66(6):1586-1602.

Harrison, J. M., Keskin, N. B., and Zeevi, A. (2012). Bayesian dynamic pricing policies: Learning and earning under a binary prior distribution. Management Science , 58(3):570-586.

Jagadeesan, M., Wei, A., Wang, Y., Jordan, M. I., and Steinhardt, J. (2021). Learning equilibria in matching markets from bandit feedback. Proceedings of Thity-Fifth Conference on Neural Information Processing Systems (NeurIPS) , pages 3323-3335.

Johari, R., Kamble, V., and Kanoria, Y. (2017). Matching while learning. In Proceedings of the ACM Conference on Economics and Computation , pages 119-119.

Keskin, N. B. and Zeevi, A. (2014). Dynamic pricing with an unknown demand model: Asymptotically optimal semi-myopic policies. Operations Research , 62:1142-1167.

Kleinberg, R. and Leighton, T. (2003). The value of knowing a demand curve: bounds on regret for online posted-price auctions. In Proceedings of the Forty-Fourth Annual IEEE Symposium on Foundations of Computer Science (FOCS) , pages 594-605.

Kveton, B., Wen, Z., Ashkan, A., and Szepesvari, C. (2015). Tight Regret Bounds for Stochastic Combinatorial Semi-Bandits. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics (AISTATS) , volume 38, pages 535-543.

Lavi, R. and Nisan, N. (2000). Competitive analysis of incentive compatible on-line auctions. In Proceedings of the Second ACM Conference on Electronic Commerce .

Liu, L. T., Mania, H., and Jordan, M. I. (2020). Competing bandits in matching markets. In Proceedings of the Twenty-Third International Conference on Artificial Intelligence and Statistics (AISTATS) , pages 1618-1628.

Morgenstern, J. and Roughgarden, T. (2016). Learning simple auctions. In Proceedings of the 29th Annual Conference on Learning Theory (COLT) , pages 1298-1318.

Paes Leme, R. and Schneider, J. (2018). Contextual search via intrinsic volumes. In Proceedings of the Fifty-Ninth Annual IEEE Symposium on Foundations of Computer Science (FOCS) , pages 268-282.

Ramshaw, L. and Tarjan, R. E. (2012). On minimum-cost assignments in unbalanced bipartite graphs. HP Labs, Palo Alto, CA, USA, Tech. Rep. HPL-2012-40R1 , 20.

Rothschild, M. (1974). A two-armed bandit theory of market pricing. Journal of Economic Theory , 9(2):185-202.

Xu, J. and Wang, Y.-X. (2021). Logarithmic regret in feature-based dynamic pricing. In Proceedings of Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS) , volume 34, pages 13898-13910.

A Algorithmic Details for Solving the Integer Program

The integer linear program in (7) can be written as an instance of maximum weight bipartite matching. In this graph, we represent each user u with d t u left nodes and we represent each endowed item i with a right node. Then, we construct the complete weighted bipartite graph where the weight of an edge between a node for user u and a node for item i is given as v ui . In total, this graph has D t left vertices, E t right vertices, and D t E t weighted edges where D t and E t correspond to total demand and endowment at time t (as given in Definition 2).

Then, using a variant of the Hungarian algorithm for unbalanced bipartite graphs (Ramshaw and Tarjan, 2012), this problem can be solved in space O ( D t E t ) and time O ( D t E t min { D t , E t } ) .

Since we can upper bound D t ≤ N , E t ≤ M , and min { D t , E t } ≤ L , we conclude that the algorithm runs in space O ( NM ) and time O ( NML ) in the worst case.

B Proofs for Revenue Maximization with Fixed Valuations

B.1 Proof of Theorem 1

Letting OPT t denote the optimum revenue at each time t and R t denote the regret at round t ,

Then, we sum over all 1 ≤ t ≤ T to have

where T ui = { t : 1 ≤ t ≤ T, x t ui = 1 } denotes the time indices where ( u, i ) is offered. Now, we let

be an upper bound for the total regret incurred from offering the pair ( u, i ) such that we have R ( T, π ) ≤ ∑ u ∈N ∑ i ∈I R ui . Now, let k th learning epoch for pair ( u, i ) correspond to the time indices in which β t ui = (1 / 2) k and the pair ( u, i ) is offered. That is,

Note that we have b t ui -a t ui ≤ β t ui = (1 / 2) k for each t ∈ T k ui . During the learning phase, each epoch T k ui ends either when the offer ( u, i ) is rejected or the offer ( u, i ) is accepted 2 k -1 times in a row. Therefore, in epoch k , there are at most one rejection and 2 k -1 acceptances. As a result, the regret incurred by offers ( u, i ) during each T k ui is upper bounded as

Note that a learning epoch T k ui can last for anywhere between 1 to 2 k rounds in total. Therefore, if we were to continue exploration indefinitely, the number of learning epochs could be as large as T in the worst case. Consequently, continuing to explore indefinitely would result in linear regret. To avoid this issue, the algorithm should stop exploration after achieving a certain level of precision and offer item i to user u at the maximum price that is certainly acceptable, namely a ui . We let this precision level be ϵ and make the algorithm choose price a ui when b t ui -a t ui ≤ ϵ . Based on this stopping rule, K ui = ⌈ log log(1 /ϵ ) ⌉ becomes the last epoch for ( u, i ) and therefore we run at most K ui -1 learning epochs before the last epoch.

Since the offers are always accepted in this last epoch, we incur at most ϵ regret at each round of it. Consequently, we have

Since X t includes at most L offers, we also have ∑ u ∈N ∑ i ∈I |T ui | ≤ LT . Thus,

Letting ϵ = 1 / ( LT ) , we obtain

B.2 Proof of Theorem 2

Consider a market with users N = [ N ] where the demands of users are given as

so that only one user has nonzero demand in each round. Next, let M ′ be the smallest prime number larger than or equal to M . If N = M ′ , let M ′ be the next smallest prime number. By the prime number theorem, we always have M ′ ≤ 4 M , and N and M ′ are always co-prime. Then, consider a set of items I = [ M ′ ] such that the valuation of all users for the first M items are uniformly and independently chosen over [0 , 1] while the last M ′ -M items are artificial items that have zero valuation for all users. Next, assume that the set of available items E t at each time t is given according to

such that at most L of the items are available at each round, and hence the maximum load parameter in this market is equal to L . By construction, the optimum offering pattern at each time is to offer all available items to a single user that has non-zero demand. Hence, the problem of the provider reduces to only learning the price at which it should offer each item. There are NM actual user-item pairs and each item i ∈ [ M ] is offered to user u ∈ [ N ] for at least ⌊ LT/NM ′ ⌋ ≥ ⌊ LT/ 4 NM ⌋ ≥ 1 rounds. In the literature on pricing optimization, each pricing problem is known to have Ω(log log( T o )) regret in T o rounds (Kleinberg and Leighton, 2003). Therefore, any policy must have Ω( NM log log( LT/NM )) regret in total.

C Proofs for Revenue Maximization with Random Experiences

C.1 Preliminary Lemmas

Lemma 7. With probability 1 -δ ,

for all ( u, i ) ∈ N × I and for all t ∈ N .

Proof. Let H t -1 be the σ -algebra generated by ( H t , X t , p t ) and let H 0 = σ ( ∅ , Ω) . Fix some ( u, i ) and define ϵ t ui := z t ui -v ui for all t ∈ N . By previous assumptions, E [ ϵ t ui |H t -1 ] = 0 and E [exp( λϵ t ui ) |H t -1 ] ≤ exp ( λ 2 / 2 ) for all t ∈ N .

Therefore, the conditional mean and conditional cumulant generating function satisfy

Applying Lemma 14, we have, for all x ≥ 0 and λ ≥ 0 ,

Therefore,

Noting that v t ui = 1 m t ui ∑ τ ∈T t ui z τ ui , we have ∑ τ ∈T t ui [( z τ ui -v ui ) 2 -( z τ ui -v t ui ) 2 ] ≥ 0 for all t . Then, choosing λ = 1 4 and x = log 1 δ gives

which implies

By applying a union bound over all ( u, i ) pairs, we have

Lemma 8. With probability 1 -δ , for all ( u, i ) ∈ N × I and for all t ∈ N .

Proof. From Lemma 7, we have | v t ui -v ui | ≤ √ 8 log( NM/δ ) /m t ui for all ( u, i ) ∈ N × I and for all t ∈ [ T ] with probability 1 -δ . Assume that this condition holds true.

We prove the statement of the lemma by induction. Assume v ui ∈ [ a t ui , b t ui ] as the inductive hypothesis. Then, if user u accepts item i at price p t i , then we have v t ui ≥ p t i . Therefore, p t i -v ui ≤ v t ui -v ui ≤ √ 8 log( NM/δ ) /m t ui . So, a t +1 ui = a t ui ∨ ( p t i -√ 8 log( NM/δ ) /m t ui ) ≤ v ui . Similarly, we can also show b t +1 ui = b t ui ∧ ( p t i + √ 8 log( NM/δ ) /m t ui ) ≥ v ui . Therefore, v t +1 ui ∈ [ a t +1 ui , b t +1 ui ] .

Lemma 9. With probability 1 -δ , the value of OPT satisfies

t

ui

t

ui

v

ui

∈

[

a

, b

]

Proof. Let us define

Then, using the definition of OPT and ˜ OPT together with Lemma 7, we have

Recall that n t ui counts the number of times user u is offered item i and m t ui counts the number of corresponding acceptances. Since an item i can be rejected only when n t ui = 2 j for some j ∈ N , we can write

where we upper bound the first term inside the parentheses using ∑ n k =1 1 √ k ≤ ∫ n 0 d x √ x = 2 √ n and we upper bound the second term by noting that it can have at most log T terms. Also note that the total number of offers over all time intervals is upper bounded by LT , and therefore ∑ u ∈N ∑ i ∈I m T ui ≤ LT . Then, using the Cauchy-Schwartz inequality, we can write

Putting all together, we conclude the proof of the lemma.

C.2 Proof of Theorem 3

By Lemma 8, the confidence bounds include the mean, i.e., v ui ∈ [ a t ui , b t ui ] with probability 1 -δ . Assume that this condition holds.

Then, the offered items are accepted whenever there does not exist j ∈ N such that n t ui = 2 j . So, we have that m t ui ≥ n t ui / 2 because n t ui counts the number of times item i is offered to user u , and m t ui counts the number of times item i is accepted by user u .

Furthermore, we recall that intervals are only updated at time step t if x t ui = 1 and n t ui = 2 j for some j ∈ N . Therefore, we can show an upper bound for the width of the intervals [ a t ui , b t ui ] . Define the width g t ui = b t ui -a t ui . When the update happens, the interval's size g t ui becomes at most g t -1 ui / 2 + √ 8 log( NM/δ ) /m t ui ≤ g t -1 ui / 2 + √ 16 log( NM/δ ) /n t ui . Therefore,

where K = log 2 n t ui is the number of times the interval is updated before time t . Then,

Now, let L t ui = { x t ui = 1 , ∃ j ∈ N , n t ui = 2 j } denote the learning event for pair ( u, i ) at round t . Also note that x t ui = 1 at some t , then p t i = 1 {¬ L t ui } a t ui + 1 { L t ui } ( a t ui + b t ui ) / 2 . Next, we recall the definition of ˜ OPT in (19) and let

with entries y t ui . Then, the regret with respect to ˜ OPT is given as the difference

Recalling that the confidence bounds include the mean, i.e., v ui ∈ [ a t ui , b t ui ] ,

Since X t is selected according to

we can upper bound the first term as

On the other hand, the second term is upper bounded as

Therefore, we can upper bound ˜ R ( T, π ) as

Then, we can show

where the proof of the last step is similar to the proof of Lemma 9. Therefore, with probability 1 -δ ,

Then, we use Lemma 9 which holds with probability 1 -δ . Therefore, by applying a union bound, we can show that

with probability 1 -2 δ .

C.3 Proof of Theorem 4

Recall that our observations are limited to accept-reject signals given by indicator variables 1 { v t ui ≥ p i } for all ( u, i ) such that x t ui = 1 . Furthermore, the valuations are given by

where z t ui are i.i.d. random variables with mean v ui and T ui ( t ) = { τ < t : x t ui = 1 } . Hence, the signal 1 { v t ui ≥ p i } is a function of z t ui in T ui ( t ) . As a result, the problem of learning revenuemaximizing offers by observing z t ui for all ( u, i ) such that x t ui = 1 is no harder than that of learning revenue-maximizing offers by observing 1 { v t ui ≥ p i } signals for all ( u, i ) such that x t ui = 1 .

On top of observing z t ui , if we further assume that an oracle sets the prices p i = v t ui whenever x t ui = 1 (such that all the offers are accepted by achieving revenue v t ui ), the corresponding problem becomes an online linear optimization problem with semi-bandit feedback. In this problem, at each round t , the algorithm chooses an offering X t from the set of feasible offers X t , observes z t ui for all ( u, i ) such that x t ui = 1 , and aims to maximize the total revenue (note that the revenue at time t is equal to the sum of valuations for all ( u, i ) such that x t ui = 1 ). Since this problem is also no harder than our original problem, any lower bound for this problem also applies to our setting.

Let N = [ N ] and I = [ M ] be the set of users and items respectively. Then, consider a market setting where the demands of users are given as

so that only one user has nonzero demand in each round and each user is active for Θ( T/N ) rounds. Let E t = I for all rounds and note that the maximum load in this market is L as needed. Consequently, we have N independent sub-problems where each sub-problem corresponds to offering the best L items in I to a single user. In the literature on online linear optimization with semi-bandit feedback, this problem is known as the L-sets problem and it is known that for any algorithm there exists an instance such that expected regret Ω( √ LMT o ) in T o rounds Kleinberg and Leighton (2003). Therefore, for any algorithm, each of N sub-problems has Ω( √ LMT/N ) regret which corresponds to Ω( √ NMLT ) regret in total for the whole problem.

D Proofs for Revenue Maximization with Random Valuations

D.1 Preliminary Lemmas

Lemma 10. For any u ∈ N and i ∈ I , there exists constants C 1 and C 2 such that

for all p ∈ [0 , 1] .

Proof. Since ψ ′′ ui ( p ∗ ui ) is strictly negative, there exists constants A 1 , A 2 , ϵ > 0 such that A 1 ( p -p ∗ ui ) 2 < ψ ui ( p ∗ ui ) -ψ ui ( p ) < A 2 ( p -p ∗ ui ) 2 for all p ∈ ( p ∗ ui -ϵ, p ∗ ui + ϵ ) . Since the set S = { p ∈ [0 , 1] : | p -p ∗ ui | ≥ ϵ } is compact and ψ ui ( p ∗ ui ) -ψ ui ( p ) > 0 for all p ∈ S , there exists constants B 1 , B 2 such that B 1 ( p -p ∗ ui ) 2 < ψ ui ( p ∗ ui ) -ψ ui ( p ) < B 2 ( p -p ∗ ui ) 2 for all p ∈ S . Hence, if we set C 1 = A 1 ∧ B 1 and C 2 = A 2 ∨ B 2 , we obtain the statement of the lemma.

Lemma 11. For any u ∈ N and i ∈ I , define ∆ uik = max k ψ uik -ψ uik . If ˜ ∆ ui 0 ≤ ˜ ∆ ui 1 ≤ · · · ≤ ˜ ∆ ui ( K -1) are the elements of the set { ∆ ui 1 , . . . , ∆ uiK } sorted in ascending order, then ˜ ∆ uik ≥ c 1 ( k/ 2 K ) 2 .

Proof. Applying Lemma 10 and using the definition of ∆ uik , we can show that ∆ uik ≥ C 1 ( p ∗ ui -k/K ) 2 . Then, lower bound on ˜ ∆ uik follows upon observing that at most j elements of the set { 1 /K, 2 /K,... , 1 } lie within a distance j/ 2 K of p ∗ ui .

Lemma 12. For any u ∈ N and i ∈ I , we have ψ ∗ ui -max k ψ uik ≤ C 2 /K 2 .

Proof. Applying Lemma 10 and noting that at least one of the prices { 1 /K, 2 /K,... , 1 } lies within a distance 1 /K of p ∗ ui , we obtain the statement of the lemma.

D.2 Proof of Theorem 5

Let π ∗ be the algorithm that chooses the optimum choice of offers and prices. Let π ∗ K be the algorithm that chooses the optimum choice of offers, but chooses the prices as the best price from the set { 1 /K, 2 /K,... , 1 } . Let π denote our algorithm described in Algorithm 3. Furthermore, let ρ ( · ) represent the expected revenue obtained by an algorithm. Then, the regret of policy π satisfies

We begin with upper bounding ρ ( π ∗ K ) -ρ ( π ) . Recall that we can represent the expected revenue of any offering and pricing (that is constrained to set the set { 1 /K, 2 /K,... , 1 } ) as

where x t uik are binary variables that denote whether user u is offered item i at k th price level. Due to the endowment, demand constraints, and the constraint that requires the prices to be from the set { 1 /K, 2 /K,... , 1 } , the x t uik variables at each time t need to belong to the set

Similar to the proof of Lemma 7, we have | ̂ ψ t uik -ψ uik | ≤ √ 8 log( NMK/δ ) /n t uik for all u ∈ N , i ∈ I and t ∈ [ T ] . For ease of notation, let us define B := √ 8 log( NMK/δ ) and w t uik = √ 1 /n t uik . Assuming that this condition holds, we can show that

Hence, with probability 1 -δ , we have

At each time step t , consider the list consisting of w t uik for all ( u, i, k ) ∈ O t := { ( u, i, k ) : x t uik = 1 } . Let's now consider the overall list consisting of the concatenation of all of these lists over all rounds. Let's order this list in decreasing order to obtain a list ˜ w = ( ˜ w 1 , ˜ w 2 , . . . , ˜ w J ) where J = ∑ T t =1 |O t | ≤ LT . Using this notation, we have

First, for any ( u, i, k ) pair, the list ˜ w can contain at most 1 + 1 /c 2 elements that are associated with ( u, i, k ) and larger than or equal to c . Secondly, note that x t uik = 1 only if B/w t uik ≥ max k ψ uik -ψ uik = ∆ uik . As a result, size of { t : x t uik = 1 } is at most B 2 / ∆ 2 uik . Therefore, the number of times a ( u, i, k ) pair can appear in the list ˜ w is also upper bounded by B 2 / ∆ 2 uik . Therefore, summing over all k ∈ [ K ] , we can upper bound the number of elements that are associated

with ( u, i ) and larger than or equal to c by

where the second step uses Lemma 11. Thus, the total number of times that any confidence set can have size at least ˜ w j is upper bounded by O ( NM ( 1 + 1 ˜ w 2 j ) + NMBK 2 √ 1 + 1 ˜ w 2 j ) . Using this result, we can write ˜ w j = O ( min { 1 , 1 √ ( j/BNMK 2 ) 2 -1 + 1 √ j/NM -1 }) . Hence,

Therefore,

Next, we bound ρ ( π ∗ ) -ρ ( π ∗ K ) . Note that for each ( u, i ) , at least one of the numbers { 1 /K, 2 /K,... , 1 } lies within 1 /K of p ∗ ( u, i ) . Then, by Lemma 12, ψ ∗ ui -max k ψ uik ≤ C 2 /K 2 for some absolute constant C 2 . Therefore, the gap ρ ( π ∗ ) -ρ ( π ∗ K ) is upper bounded as

Combining equations (24) and (25), we have

Lastly, we choose K 4 = Θ ( LT NM log( LT ) ) to obtain

where the last step uses 0 ≤ K 4 ≤ T and T ≥ 1 .

D.3 Proof of Theorem 6

Consider the same market setting (demands and endowments) in Appendix Section B.2, but assume that the valuations are i.i.d. random instead of being fixed. By construction, the optimum offering pattern at each time is to offer all available items to a single user that has non-zero demand. Hence, the problem of the provider reduces to only learning the price at which it should offer each item. There are NM actual user-item pairs and each item i ∈ [ M ] is offered to user u ∈ [ N ] for Θ( LT/NM ′ ) rounds. In the literature on pricing optimization with i.i.d. valuations, each pricing problem is known to have Ω( √ T o ) regret in T o rounds (Kleinberg and Leighton, 2003). Therefore, any policy must have Ω( √ NMLT ) regret in total.

E Martingale Exponential Inequalities

Consider a sequence of random variables ( Z n ) n ∈ N adapted to the filtration ( H n ) n ∈ N . Assume E [exp( λZ i )] is finite for all λ . Define the conditional mean µ i = E [ Z i |H i -1 ] , and define the conditional cumulant generating function of the centered random variable [ Z i -µ i ] by ψ i ( λ ) := log E [exp( λ [ Z i -µ i ]) |H i -1 ] . Then, for a process ( M n ( λ )) n ∈ N defined as

we can prove the following properties.

Lemma 13. ( M n ( λ )) n ∈ N is a martingale with respect to the filtration ( H n ) n ∈ N , and E [ M n ( λ )] = 1 for any λ .

Proof. By definition, we have

Then, for any n ≥ 2 ,

since M n -1 ( λ ) is a measurable function of the filtration H n -1 .

Lemma 14. For all x ≥ 0 and λ ≥ 0 ,

Proof. For any λ , ( M n ( λ )) n ∈ N is a martingale with respect to ( H n ) n ∈ N and E [ M n ( λ )] = 1 by Lemma 13. For arbitrary x ≥ 0 , define τ x = inf { n ≥ 0 | M n ( λ ) ≥ x } and note that τ x is a stopping time corresponding to the first time M n crosses the boundary x . Since τ is a stopping time with respect to ( H n ) n ∈ N , we have E [ M τ x ∧ n ( λ )] = 1 . Then, by Markov's inequality

Noting that the event { M τ x ∧ n ( λ ) ≥ x } = ⋃ n k =1 { M k ( λ ) ≥ x } , we have

Taking the limit as n →∞ , and applying monotone convergence theorem shows that

Consequently, we can write

Then, by definition of M k ( λ ) , we conclude

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes online pricing and allocation strategies for multi-user multi-markets model under three different valuation models.\n\nThe main contribution is to extend dynamic pricing strategies from the literature to the case of more than one item and more than one user.\n\nThe setting goes as follows:\n- At every iteration, each user $u$ presents a maximum demand for items $d_u^t$, that is observed.\n- the algorithm allocates items and prices.\n- a user accepts the items for which his valuation is higher than the offered price, accepting at most $d_u^t$ items.\n\nThe study the fixed valuation model in which the valuation of users for each item is fixed and deterministic, the random experience model, where a user's valuation for an item is the mean of his past experiences, and the random valuation model where the valuations are drawn from fixed but unknown distributions.\n\nPresented algorithms construct confidence intervals for the user-item valuations. At every iteration, the items are allocated according to the so-called optimism principle, meaning the optimal allocation for the upper bounds on the user-item valuations is chosen.\n\nThe price set for the item depends on the valuation model.\n\nSTRENGTHS:\nThe paper is clear and the algorithms well presented for the most part. The study of larger markets than one item/one user certainly is interesting and relevant for applications.\n\nWEAKNESSES:\nMy main criticism concerns the way the paper is written. A lot of emphasis is put on the pricing strategies. However, at least two of the method for pricing, for the fixed valuation model and the random valuation model, already exist in the literature (reference 3) and this is not discussed in the paper.\nMore precisely, the methods of geometrically decreasing intervals and discretization of the price interval are exactly the methods used in ref 3.\n\nThere are some unclarities in the proofs, as listed below.\n\n1. The proof of Theorem 1 is not clear at all. Parameter $\\epsilon$ is set to $1/LT$ without explanation. This can only be done because exploration stops when the interval for the valuation becomes small enough. But although it appears in the pseudo-code of the algorithm, it is not evoked in the main text nor in the proof. However, this feature of stopping exploration after some time is crucial in avoiding linear regret. This is clear for a reader familiar with ref 3, but I do not believe it would be otherwise.\n\n2. In Appendix C, $y_ui$ is used without introduction. In figure 2, notations are also used before they are introduced.\n\n3. Line 494, the lists are defined too informally.\n\nQUESTIONS:\nOverall, the paper has interesting elements, but a good discussion on the dependence of the problem in the parameter L, M and N is missing, as this is the main contribution of the paper. \n\nPrecise references to where previously existing methods were used are missing. Adding this would help the reader understand the contribution of the paper better.\n\nOn a side note, the assumption that $d_u^t$ is observed seems rather strong, if the authors have a good argument as to why it is necessary (or not), it would be interesting to add.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the online pricing problem where there are several users and items. Each user has a (private) valuation for each item and a (public) upper bound for the number of items desired, both of which could vary at different rounds. The provider has an available item set at each round, and the goal is to distribute them to the users such that the total revenue is maximized. The paper considers three settings for the uses' valuation functions: (1) fixed valuations, (2) random experiences, and (3) random valuations. For each setting, the corresponding algorithm is proposed and analyzed, along the theoretical lower bound. For the first two settings, the basic algorithmic idea is trying to learn an accurate valuation interval for each item-user pair, and then computing the assignment and the prices according to the estimated intervals; while for the last setting, the authors quantize the set of possible prices and then borrow the idea from the multi-armed bandit problem to give an algorithm. Finally, the proposed algorithms are evaluated empirically.\n\nSTRENGTHS:\n- The paper makes theoretical contributions. The multi-user multi-item online pricing problem introduced makes sense. The authors consider different valuation settings of this problem, and obtain small gaps between the upper bounds and the lower bounds. The analysis is non-trival.\n\n- The paper is well-written. For each algorithm, the authors always provide the basic algorithmic intuition, which can help the reader understand the algorithm.\n\nWEAKNESSES:\n- The discussion of the algorithms' running time is missing. Further, in the current statement, the algorithms need to solve linear programs at each round, but essentially, this can be formulated into a bipartite matching instance and thus, the classical matching algorithm can be applied and help to improve the running time.\n\nQUESTIONS:\nMinor comments:\n\n- It might be better to point out that $\\lor$ and $\\land$ represent \"taking max\" and \"taking min\" respectively in the paper.\n\n- In the statement of algorithm 3, $n_{uik}$ is initialized by 0, but in the first iteration, we compute the term $\\sqrt{8\\log (NMKT) / n_{uik}}$.\n\nLIMITATIONS:\nI didn't see any negative societal impact of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

comments: SUMMARY:
The paper studies the problem of dynamic pricing in which multiple items are offered to multiple users at each round. Authors propose a novel algorithm for maximizing revenue under three user valuation models: fixed valuations,  random experiences and random valuations. Authors provide theoretical guarantees and provide simulation results supporting the theoretical claims.

STRENGTHS:
**Strengths**

1.) The paper is well written and easy to follow. Authors motivate the problem formulation well.

2.) Authors provide a detailed related work section and contributions well positioned in the relevant literature.

3.) The theoretical results appear to be sound and the paper also provide novel algorithmic contributions.

WEAKNESSES:
**Weaknesses**

1.) The main weakness of the paper is lack of comparison with existing methods. Authors do not provide a discussion comparing their theoretical or simulation results with existing work.

2.) Including a proof sketch in the main paper will improve the quality of the paper.

3.) Including additional simulation results will improve the paper.

QUESTIONS:
1.) Authors assume that items that are not sold in one round can not be stored to be sold in future rounds. What are technical challenges associated with relaxing this assumption?

2.) Can this results be extended to the case where users only accept an item if its price is lower than the valuation by some

LIMITATIONS:
The paper does not discuss the limitations of the method and potential societal impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes algorithms for optimizing the sale of multiple goods to multiple users, taking into account their time-varying valuations throughout repeated rounds. These algorithms efficiently learn from users' accept or reject feedback and utilize this information to make optimal offers and prices based on three user valuations: fixed valuation, random experience, and random valuation. To evaluate the effectiveness of the proposed algorithms, the paper provides revenue regret guarantees. Notably, the significant contribution of this paper lies in the introduction of the concept of user's random experience as a crucial valuation to consider in dynamic pricing algorithms.\n\nSTRENGTHS:\nThe paper addresses the problem of revenue maximization in a dynamic market with time-varying valuations. Specifically, the paper introduces the concept of \"random experience\", which is a novel and unexplored idea in prior literature.\nThe paper offers a rigorous theoretical analysis of the proposed algorithms and assesses their effectiveness and efficiency in practice by evaluating nearly-optimal revenue regret guarantees for each algorithm.\nThe paper exhibits a well-organized structure, presenting the problem formulation, proposed algorithms, theoretical analysis, and experimental results in a clear and coherent manner.\n\nWEAKNESSES:\nSome statements are general and lack support or explicit example, such as in line 17, which states, “The ability to design algorithms that can achieve the optimal sale of goods to multiple users having time-varying valuations for each of the goods is both timely and relevant, given the explosion in the use of data in large-scale systems.” How does the author come to this statement? I would suggest briefly citing more literature to support the argument.\nFigure 1 is unclear, and it is challenging to understand how the provider learns user valuation from the current depiction. Additionally, the meaning of the gray user representative is not apparent. To improve the figure's clarity, I recommend providing a more detailed description.\nThe paper may lack comparisons with relevant literature. The author mentions that fixed valuation and random valuation have been used as standard models for dynamic pricing in previous research. Therefore, I would suggest that the author could further elaborate on and compare this paper’s results with previous work. In comparison with previous works, the author can emphasize the advantages and differences of their proposed method, such as differences in experimental design or interpretation of results. This will help clarify the innovation and contribution of the paper.\n\nQUESTIONS:\nIn line 56, why are the algorithms focused on offering each item to only one user during each round? The framework of \"multi-user multi-item\" proposed in the paper suggests that interactions happen between multiple users and multiple items over repeated rounds, indicating a dynamic market setting. While the constraint of offering one item to one user during each round simplifies the problem, it may not fully reflect the dynamic market. Therefore, I would like to see that the algorithms can be focused on more items to one user, one item to more users, or more items to more users, which aligns better with the multi-user multi-item market framework proposed in the paper. \nIn Section 5, why are two different datasets used for numerical experiments? Figure 3 is based on 20 sets of experiments with N=150 users and M=100 items, while Figure 4 is based on 5 sets of experiments with N=30 users and M=20 items.\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of online pricing in multi-user multi-item markets. The main objective is to maximize revenue by selling multiple items to multiple users in each round. The paper proposes algorithms that efficiently offer and price items while learning user valuations from accept/reject feedback. It considers three user valuation models and provides algorithms with nearly-optimal revenue regret guarantees. Additionally, the paper introduces a problem-dependent load parameter and designs regret-optimal algorithms for different market settings. Overall, the paper makes contributions in terms of user valuation models, load parameter analysis, and algorithm design, being the first to address this problem.\n\nSTRENGTHS:\nThis paper introduces a novel problem of online pricing in multi-user multi-item markets and addresses the specific challenge of maximizing revenue by selling multiple items to multiple users in each round. The quality of the paper is commendable, with a comprehensive analysis of different user valuation models, well-explained methodology, and rigorously analyzed algorithms. The paper is also clearly written and well-structured, making it easy for readers to follow the flow of ideas. Overall, the paper demonstrates originality in problem formulation, high-quality research methodology, clear presentation of ideas, and significant contributions to the field.\n\nWEAKNESSES:\n1.\tThe paper makes a simplifying assumption that each item is offered to only one user during each round. However, this assumption may not capture the realistic settings in which multiple users may have overlapping preferences for the same item and providers may have incentives to offer the same item to multiple users to increase their revenue.\n2.\tThe paper does not investigate the scalability of the proposed algorithms. It would be important to evaluate how the algorithms scale with respect to the number of users, items, and rounds. This evaluation would reveal the computational complexity and resource requirements of the algorithms in large-scale multi-user multi-item markets.\n3.\tThe paper does not discuss practical considerations that may arise in real-world implementations, such as privacy concerns, fairness considerations, or strategic user behavior. It would be beneficial to address these practical considerations and discuss how the proposed algorithms can cope with such challenges or potential extensions to handle them.\n\nQUESTIONS:\nPlease refer to weaknesses 1, 2, and 3\n\nLIMITATIONS:\nThe paper presents several limitations that could be addressed in future work. Firstly, the assumption that each item is offered to only one user during each round may not reflect realistic settings where multiple users may have overlapping preferences for the same item. Secondly, the paper does not address practical considerations such as privacy concerns, fairness considerations, or strategic user behavior. Lastly, the exploration of different market settings is limited, and it would be interesting to analyze the performance of the proposed algorithms in diverse market scenarios. Addressing these limitations would provide a more comprehensive understanding of the applicability and robustness of the algorithms.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-24"}