Title: Learning to Price Homogeneous Data
Venue: NeurIPS 2024
Accepted: yes
Keywords: Learning to price

Revenue maximization

Data marketplaces
Abstract: We study a data pricing problem, where a seller has access to $N$ homogeneous data points (e.g. drawn i.i.d. from some distribution).
There are $m$ types of buyers in the market, where buyers of the same type $i$ have the same valuation curve $v_i:[N]\rightarrow [0,1]$, where $v_i(n)$ is the value for having $n$ data points.
*A priori*, the seller is unaware of the
distribution of buyers, but can repeat the market for $T$ rounds so as to learn the revenue-optimal pricing curve $p:[N] \rightarrow [0, 1]$.
To solve this online learning problem,
we first develop novel discretization schemes to approximate any pricing curve.
When compared to prior work,
the size of our discretization schemes scales gracefully with the approximation parameter, which translates to better regret in online learning.
Under assumptions like smoothness and diminishing returns which are satisfied by data, the discretization size can be reduced further.
We then turn to the online learning problem, 
both in the stochastic and adversarial settings.
On each round, the seller chooses an *anonymous* pricing curve $p_t$.
A new buyer appears and may choose to purchase some amount of data.
She then reveals her type *only if* she makes a purchase.
Our online algorithms build on classical algorithms such as UCB and FTPL, but require novel ideas to account for the asymmetric nature of this feedback and to deal with the vastness of the space of pricing curves.
Using the improved discretization schemes previously developed, we are able to achieve 
$\widetilde{O}(m\sqrt{T})$ regret in the stochastic setting and $\widetilde{\mathcal{O}}(m^{3/2}\sqrt{T})$ regret in the adversarial setting.

Learning to Price Homogeneous Data

Abstract

We study a data pricing problem, where a seller has access to N homogeneous data points (e.g. drawn i.i.d. from some distribution). There are m types of buyers in the market, where buyers of the same type i have the same valuation curve v i : [ N ] → [0 , 1] , where v i ( n ) is the value for having n data points. A priori , the seller is unaware of the distribution of buyers, but can repeat the market for T rounds so as to learn the revenue-optimal pricing curve p : [ N ] → [0 , 1] . To solve this online learning problem, we first develop novel discretization schemes to approximate any pricing curve. When compared to prior work, the size of our discretization schemes scales gracefully with the approximation parameter, which translates to better regret in online learning. Under assumptions like smoothness and diminishing returns which are satisfied by data, the discretization size can be reduced further. We then turn to the online learning problem, both in the stochastic and adversarial settings. On each round, the seller chooses an anonymous pricing curve p t . A new buyer appears and may choose to purchase some amount of data. She then reveals her type only if she makes a purchase. Our online algorithms build on classical algorithms such as UCB and FTPL, but require novel ideas to account for the asymmetric nature of this feedback and to deal with the vastness of the space of pricing curves. Our algorithms achieve ˜ O ( m √ T ) regret in the stochastic setting and ˜ O ( m 3 / 2 √ T ) regret in the adversarial setting.

1 Introduction

Due to the rise in popularity of machine learning, there is an increased demand for data. However, not all users of data have the wherewithal to collect data on their own, and have to rely on data marketplaces to acquire the data they need. For example, a materials data platform (e.g. [18]), may have collected vast amounts of data from various proprietary sources. Materials scientists in smaller organizations and academia, who do not have large experimental apparatuses, may wish to purchase this data to aid in their research. Similarly, small businesses may wish to purchase customer data for advertising and product recommendations [4, 5], while small technology companies may wish to purchase data about cloud operations to optimize their computing infrastructure [2, 3].

Model. Motivated by the emergence of such data marketplaces, we study the following online data pricing problem. A seller has access to N homogeneous data points, (e.g. drawn i.i.d. from some distribution). He wishes to sell the data to a sequence of distinct buyers over T rounds, and intends to achieve large revenue. There are m types of buyers in the data marketplace, with all buyers in type i having the same valuation curve v i : [ N ] → [0 , 1] for the data, where v i ( n ) represents the buyer's value for having n points. As data is homogeneous, we can treat an agent's value as a function of the amount of data n (we will illustrate this in the sequel). Valuation curves are monotone non-decreasing, as more data is better. At each round t , the seller chooses a price curve p t : [ N ] → [0 , 1] , where p t ( n ) is the price for purchasing n data points. Then, a buyer with type i t arrives and purchases an amount of data that maximizes her utility (value minus price), provided that she can achieve non-negative utility. A buyer will reveal her type to the seller only if she makes a purchase, and only after she

makes the purchase. The seller has knowledge of valuation curves of the m types, but does not know the distribution q over types (stochastic setting), or the buyer sequence (adversarial setting). Moreover, he cannot practice non-anonymous (discriminatory) pricing, as he needs to choose the pricing curve p t without knowledge of the buyer's type on that round.

While there is extensive research on revenue-optimal pricing and learning to price , data marketplaces merit special attention, both due to their recent emergence and the unique characteristics of data. Typically the number of data N (number of goods) is very large, but data usually satisfies additional properties such as smoothness (an agent's value does not increase significantly with a small amount of additional data) and diminishing returns (additional data is more valuable when a buyer has less data). To illustrate further, note that two steps are essential to develop an effective online learning solution for data pricing. (1) First, we need to solve the planning problem , i.e. find a revenue-optimal pricing curve when the type distribution q is known. (2) Second, when q is unknown, we need to combine the algorithm in step (1) with estimates for q to maximize long-term revenue.

Methods in the existing literature fall short in both steps. (1) When the type distribution q is known, the data pricing problem resembles an ordered item pricing problem, which is known to be NP -hard [13, 25]. Hence, prior work has aimed at approximating the optimal pricing curves via discretization schemes. Unfortunately, existing discretization schemes have poor, often exponential, dependence on the approximation parameter ϵ . However, achieving sublinear regret in online learning requires choosing ϵ that vanishes with longer time horizons, i.e. ϵ → 0 as T →∞ . Therefore, directly using existing discretization schemes in an online setting leads to poor statistical and computational properties of the associated online algorithm. This requires us to leverage the above properties of data to design discretization schemes with better dependence on ϵ . (2) While there is prior work on learning optimal prices [22, 27, 33], these techniques either fall short of addressing the complexities in our setting, or fail to account for the properties of data, and hence do not scale gracefully when the amount of data N is very large. Moreover, in our online learning setup, the seller faces a trade-off between setting high prices to maximize instantaneous revenue versus setting low prices so as to guarantee a purchase, which results in the buyer revealing their type, which in turn can be helpful in future rounds. Prior work has studied this asymmetric feedback model only in single-item markets which is significantly simpler, and only in the stochastic setting [23, 47].

1.1 Summary of our contributions

Our contributions in this work are threefold: (1) First, in §3, we develop discretization schemes for revenue-optimal data pricing under a variety of assumptions, which we will use later in our online learning schemes. (2) In §4, we study learning a revenue-optimal price in a stochastic setting, where the customer types on each round are drawn from a fixed but unknown distribution q . (3) Finally, in §5, we study online learning when the buyer types are chosen by an oblivious adversary.

1. Discretization (approximation) schemes for revenue-optimal data pricing. Assuming only monotonicity, we show that there is a discretization of size ˜ O (( N/ϵ ) m ) which is an O ( ϵ ) additive approximation to any pricing curve. When compared to prior work [14, 25], our discretization scheme has smaller dependence on ϵ -1 when the number of types m is small (see Table 1). This will be useful, both statistically and computationally, when we study the online setting, as we need to choose ϵ → 0 as T →∞ to achieve sublinear regret. This is still quite large in real-world data marketplaces, where N may be very large. Hence, we also study two other assumptions. First, when valuations are smooth, satisfying an L -Lipschitz-like condition, we construct a discretization of size ˜ O ( ( L/ϵ 2 ) m ) , which has no dependence on N . Next, under a diminishing returns condition, we construct a discretization of size O ( J m ϵ -3 m log m ( N ) ) , which only has polylogarithmic dependence on N .

Key insights. We first show that when there are only m types, for any price function p : [ N ] → [0 , 1] , there exists an ' m -step' price function p ′ whose revenue is at least as much as that of p on any type distribution q . An m -step function is non-decreasing and changes values at most m times, allowing us to focus on this restricted class and thereby reduce the search space when m ≪ N . We then consider discretizations of the data space [ N ] and valuations [0 , 1] which allow us to obtain an O ( ϵ ) -approximation to any pricing curve, and then apply this insight to construct our discretizations. Finally, we show that with monotonicity and diminishing returns, similarly accurate approximations are attainable with substantially coarser discretizations.

2. Learning to price in the stochastic setting. Next, we turn to the online learning problem described in the beginning in a stochastic setting. On each round, our algorithm computes an

upper confidence bound (UCB) [8, 38] on the revenue for each price curve in the discretization previously developed; we then choose the price curve with the highest UCB. As summarized in Table 2, this algorithm achieves a ˜ O ( m √ T ) bound on the regret for any discretization scheme, including those from prior work. In the stochastic setting, the key advantage of our discretization schemes is computational.

Key insights. Both the design and the anlaysis of an algorithm is challenging in this setting due to two reasons: (i) the large size of the discretization and (i) the asymmetric nature of feedback. First, naively maintaining UCBs for each price leads to large confidence intervals, and hence large regret as the size of the discretization is large; instead, we construct confidence intervals on estimates of the type distribution, and translate them to UCBs for the revenue. Second, the asymmetric nature of the feedback places us between bandit and full-information settings. Treating this like a bandit setting would lead to poor, exponential dependence on m in the regret. However, we are unable to treat this as full information since the type distribution is revealed only if there is a purchase. Handling this asymmetry requires a delicate construction of the UCB.

3. Learning to price in the adversarial setting. We study learning in an adversarial setting where the types on each round may be chosen adversarially. Table 2 shows the regret and time complexity of our method when paired with various discretization schemes. In the adversarial setting, our discretization schemes offer both computational and statistical advantages compared to prior work.

Key insights. Our algorithm builds on the Follow-the-Perturbed-Leader (FTPL) [31], originally designed for full-information settings and not directly applicable here. To handle asymmetric feedback, we use the information we have about the valuation curves to keep track of which customers would not have made a purchase given a price curve. If a purchase is made and we observe feedback, we use the usual FTPL update, but if not, we reward each pricing curve with the sum of revenue of all types that would not purchase in that current round.

1.2 Related work

Dynamic pricing . The online posted-price mechanism, also known as dynamic pricing, is a central research area in algorithmic market design [19, 33]. In the most classical setting [33], the seller sets a price for an item in each round, and a buyer purchases the item only if their valuation exceeds the posted price. While several extensions of this setting have been explored for both parametric [12, 20, 28, 29, 32, 46] and non-parametric [11, 17, 39, 40, 44] demands, most focus on single-parameter demands, i.e., selling a single item to buyers. Our data pricing problem is multi-parameter, as demands are parameterized by multiple outcomes, i.e. the number of data points.

Bayesian unit-demand pricing problem . Formally, our data pricing problem is a variant of the Bayesian Unit-demand Pricing Problem (BUPP) [13]. BUPP addresses the problem of (offline) revenue maximization over a known distribution of unit-demand buyers, meaning they want to buy at

most one item from the inventory. In BUPP, a seller has N distinct items to sell to a unit-demand buyer whose valuations are v = ( v 1 , . . . , v N ) , where v i is the value of the i th item. Given prices p i , i ∈ [ N ] , the unit-demand buyer purchases a single item i ∈ [ N ] that maximizes their utility: v i -p i . Assuming the valuation profile v follows a known distribution D , the goal of BUPP is to find the best prices { p i } i ∈ [ N ] that maximize the seller's expected revenue.

Our data pricing problem is a variant of BUPP in two ways: (1) We study the sequential setting where type distributions are unknown , while valuation profiles for each type are known, and (2) We assume monotonic values v 1 ≤ · · · ≤ v N , which is natural in data pricing. Unfortunately, BUPP is a computationally intractable problem, as is ours. BUPP is known to be NP -hard even when D is a product distribution [16]. Moreover, even assuming that values are monotonic (i.e., v 1 ≤ · · · ≤ v N ), the problem remains (strongly) NP -hard [14]. Therefore, we aim to provide a reasonably efficient no-regret algorithm for our problem, especially when the number of types m is a fixed constant.

The previous works most relevant to our paper are Hartline and Koltun [25] and Chawla et al. [14], which study offline revenue maximization for unit-demand buyers. Buyers in our problem are also unitdemand, as each amount of data points can be seen as an individual item. Revenue maximization for unit-demand buyers is known to be computationally intractable [24], even with ordered (monotonic) buyer values [14], leading these works to focus on approximation algorithms. Hartline and Koltun [25] proposed an approximation algorithm with near-linear runtime in the number of buyers, given a fixed number of items. Chawla et al. [14] introduced a polynomial-time approximation scheme (PTAS) for unit-demand buyers with monotonic values. In this work, we extend the framework to the online setting with partial feedback, which has more practical implications.

In addition, Balcan and Beyhaghi [10] provide new guarantees for learning revenue-maximizing menus of lotteries and two-part tariffs, demonstrating that their discretization technique yields efficient solutions for specific pricing models. Similar discretization methods could be investigated in future work to potentially improve our approach in more complex data pricing scenarios.

Market design for data-sharing . In recent years, there has been a plethora of work devoted to algorithmic market design for data sharing [6, 7, 30, 43]. These works provide ingenious solutions to challenges unique to the data market, such as free replicability and the difficulty of valuation due to the combinatorial nature of data. Except for Agarwal et al. [6], the above-cited solutions are inherently offline or single-shot . While we focus on a simplified yet relevant setting where data comes from a single source, resulting in monotonic valuations, in this work, we tackle the problem in a sequential, dynamic setting, which has practical importance. In contrast to our approach, Agarwal et al. [6] considered the price to be a constant (i.e., a scalar rather than a price vector) to address the inherent computational intractability of multi-dimensional pricing. Instead, we maintain the price

as a vector (i.e., a price function) but focus on cases where the valuation function satisfies natural properties such as monotonicity, smoothness, and diminishing returns.

2 Problem setting, assumptions, and challenges

Aseller has N homogeneous data points. There are m types of buyers who wish to purchase this data. A buyer of type i ∈ [ m ] has a valuation curve v i : [ N ] → [0 , 1] , where v i ( n ) is her value for n data points. We will assume v i ( n ) is non-decreasing as more data is valuable, and further that v i (0) = 0 .

Example 1. To motivate this model, consider a seller with N ordered data points { x 1 , . . . , x N } , drawn i.i.d. from a distribution D . If a buyer purchases n points, she receives the first n points, X n = { x 1 , . . . , x n } . Her ex-post value ˜ v i ( X n ) may represent the accuracy of her ML model trained with X n . However, as the buyer has not seen the data before the purchase, she does not know which specific points she will receive, and hence her ( ex-ante ) value v i ( n ) = E X n [ ˜ v i ( X n )] is the expected model accuracy when n i.i.d points are drawn from D . The different types could be buyers who use the data for different tasks or models. For instance, with ImageNet's [21], N ≈ 1.4 million data points, different types of buyers could perform different learning tasks such as object detection, identification, and segmentation, and/or train different models such as AlexNet [36], ResNet [26], and GoogLeNet [42]. Both empirically and theoretically, for many learning tasks, v i ( n ) is non-decreasing, and satisfies additional characteristics such as smoothness and/or diminishing returns.

Pricing curves, buyer utility, and buyer purchase model. Let p : [ N ] → [0 , 1] be a pricing curve chosen by the seller. Let P ∆ = { p : [ N ] → [0 , 1] : p (0) = 0 } denote the set of all pricing curves. If a buyer purchases n points, her utility is u i ( n ) = v i ( n ) -p ( n ) . If a buyer can achieve non-negative utility, i.e. v i ( n ) ≥ p ( n ) for some n ∈ [ N ] , she will purchase an amount of data to maximize her utility. To fully specify the buyer's purchase model, we will assume that when there are multiple n which maximizes her utility, she will choose the largest such n . Formally, for a given pricing curve p , a buyer of type i will purchase n i,p points where,

Optimal revenue. It follows that the revenue from a buyer of type is p ( n i,p ) . Let q = ( q 1 , . . . , q m ) be the distribution of the buyers. Under this distribution q , the expected revenue rev( p ) for a price curve p , the optimal price p OPT , and the optimal revenue OPT as follows:

We have omitted the dependence on q in rev , p OPT , and OPT . There is no closed-form solution to finding the optimal pricing curve, even when q is known. Therefore, in §3, we explore discretization methods to approximate p OPT , which will then be used in §4 and §5 to develop online learning algorithms. Unfortunately, the size of this discretization can be very large in N and m without further assumptions. Therefore, we also consider two additional commonly satisfied conditions by data.

Our first such assumption states that buyer valuation curves satisfy a Lipschitz-like smoothness condition with Lipschitz constant L/N . We use L/N instead of L since the number of data has a range [0 , N ] , while the valuations only have a range [0 , 1] . This condition states that a buyer's valuation does not change significantly if she only purchases a few additional points.

Assumption 1 (Smoothness, S ) . For all n, n ′ ∈ [ N ] , we have v i ( n + n ′ ) -v i ( n ) ≤ L N n ′ .

Our second condition is based on the fact that data typically exhibits diminishing returns [34, 35]. This means that an additional data point is more valuable when there is less data, i.e. v i ( n +1) -v i ( n ) is decreasing with n . We will in fact make a stronger assumption, and justify it below.

Assumption 2 (Diminishing returns, D ) . There exists some J > 0 such that, for all types i ∈ [ m ] , and for all n ∈ [ N ] , we have v i ( n +1) -v i ( n ) ≤ J n .

Assumption 2 quantifies the rate of decrease of diminishing returns. Following Example 1, the valuation (accuracy) curves for many learning problems take the form v i ( n ) = α -βn -γ ; for

instance, for binary classification in a VC class H , α may be the best accuracy in H , β ∈ O ( √ d H ) where d H is the VC dimension, and γ = 1 / 2 [41]; similarly, for nonparametric regression of a twice differentiable function, α and β are constants while γ = 2 / 5 [45]. In such cases, Assumption 2 is satisfied with J = βγ . Note that neither assumption subsumes the other: a non-concave Lipschitz function will not satisfy Assumption 2, while a suitable L for a function which satisfies Assumption 2 may need to be very large for Assumption 1 to hold for small n .

2.1 Learning to price in online settings

In this work, we will also study how a seller may learn to maximize revenue. In our learning problem, the seller is aware of the valuation curves { v i } i of each type, but does not know the distribution of types (stochastic setting) or there may be no such distribution (adversarial setting).

Setup. The seller repeats the data market for T rounds. At the beginning of each round, he chooses some price curve p t ∈ P . After the seller has chosen p t , a new buyer of type i t ∈ [ m ] appears and purchases n t = n i t ,p t amount of data (see (1)). The buyer is aware of her own valuation curve. If she makes a purchase, that is if n t > 0 , she pays p t ( n t ) to the seller and reveals her type i t . Otherwise, the buyer will make no payment and not reveal her type.

We have assumed that a priori , the seller is aware of the buyer valuation curves { v i } i ∈ [ m ] , and that buyers are aware of their own valuation curves. In Example 1, a seller can profile how different machine learning models perform with different amounts of data and publish them ahead of time. The buyers can also gauge their value from these curves, even though they do not have access to the data. Next, we have also assumed that buyers will reveal their type after the purchase. In modern machine learning as a service platforms [1, 4, 18], buyers directly run their jobs in the seller's computing platform, so the seller can observe the buyers job type directly. Even if this is not the case, sellers can elicit this information via questionnaires and reviews from customers who have made a purchase [23].

Challenges. Despite these assumptions, the learning problem remains challenging for two main reasons. First, the space of price curves is vast: discretizing the valuations in [0 , 1] into K bins, still leaves O ( K N ) possible price curves, which is both statistically and computationally intractable, especially for large N . Second, in addition to the exploration-exploitation trade-off usually encountered in sequential decision-making, the seller faces a tension between high instantaneous revenue and information acquisition: setting high prices can yield high immediate revenue if a purchase occurs, but it also increases the risk of no purchase, resulting in no revenue and crucially no feedback about the buyer type which could help him in future rounds. This trade-off was recently studied for single-item markets in a stochastic setting [23, 47], but is more complex in our multi-item problem. Moreover, to our knowledge, no existing work addresses this asymmetric feedback model in an adversarial setting, even for single-item markets. Next, we describe the buyer arrival model and define the regret for the learning problem in both stochastic and adversarial settings.

Stochastic setting. Here, there is some fixed but unknown distribution of types q . On each round, a buyer of type i t ∼ q is drawn independently. The optimal expected revenue OPT under type distribution q is as defined in (2). The regret R T is as defined below. We wish to design algorithms which have small expected regret E [ R T ] , where the expectation accounts for both the sampling of types i t ∼ q and any randomness in the algorithm. We have,

Adversarial setting. Here, the types on each round { i t } T t =1 are chosen arbitrarily, possibly by an oblivious adversary, ahead of time. The type on round t is revealed to the seller only at the end of the round, and only if there is a purchase. In the adversarial setting, we define our regret R T with respect to the single best price in P in hindsight. We wish to design algorithms with small expected regret E [ R T ] , where the expectation is with respect to any randomness in the algorithm. We have,

Given: Approximation parameter ϵ > 0 . Let W be discretization of the valuation space [0 , 1] defined as follows,

Set P to be the class of all ' m -step' functions mapping [ N ] to W .

3 Efficient discretization of price curves with small errors

We first study the revenue maximization problem in the offline setting, where the seller knows both the valuation curves v i , i ∈ [ m ] , and the type distribution q . Our goal is to design a discretization so as to achieve revenue within a gap of O ( ϵ ) from OPT . Before discussing our discretization algorithms, we first show that the optimal pricing curve is 'simple' when there are at most m types.

Lemma 3.1. Assume there are m types with non-decreasing value curves { v i } i ∈ [ m ] . For any nondecreasing price curve p , there exists an ' m -step' price curve ¯ p that yields expected revenue at least that of p with respect to any distribution over the m types. Here, m -step refers to non-decreasing functions f : [ N ] → [0 , 1] where f ( n +1) -f ( n ) > 0 in at most m points (i.e., at most m jumps).

Lemma 3.1, proven in Appendix A.1, will be an important tool in all three discretization algorithms of this section. It will allow us to reduce the space of pricing curves as we only need to focus on m -step price curves. Next, we present our first discretization procedure in Algorithm 1, which only assumes the monotonicity of the valuation curves.

Discretization scheme under monotonic valuations. Our discretization proecdure, outlined in Algorithm 1, adapts the method in Hartline and Koltun [25] using Lemma 3.1. For this, we will first construct a discretization W of the valuation space as follows. Let Z i = ϵ (1 + ϵ ) i , i = 0 , 1 , . . . , ⌈ log 1+ ϵ 1 ϵ ⌉ be the powers of (1 + ϵ ) on price space [ ϵ, 1] . For each i , we let W i be a uniform discretization of the interval [ Z i -1 , Z i +1 ) uniformly with gap Z i -1 · ϵ m . Finally, let W be the union of all such W i . According to Lemma 3.1, every price function in P has the same revenue as an m -step function. We set P to be all choices of non-decreasing m -step functions that take value in W . We have the following theorem about Algorithm 1 which we prove in Appendix A.2.

Theorem 3.1. Consider the discretization P as constructed in Algorithm 1. For any type distribution, there exists p ∈ P such that rev( p ) ≥ OPT - O ( ϵ ) . Moreover, we have |P| ≤ ( e ( N -1) m ) m ( e ⌈ (2 + ϵ ) ⌉ ⌈ log 1+ ϵ 1 ϵ ⌉) m ∈ ˜ O ( ( N ϵ ) m ) .

Discretization scheme for smooth monotonic valuations. Due to space constraints, we present our algorithm, under Assumption 1 in Appendix A.4. We have the following theorem about Algorithm 5.

Theorem 3.2. Consider the discretization P as constructed in Algorithm 5. Under Assumption 1, for any type distribution, there exists p ∈ P such that rev( p ) ≥ OPT - O ( ϵ ) . Moreover, |P| ∈ O ( log m 1+ ϵ (1 /ϵ ) · ( L/ϵ ) m ) ∈ ˜ O ( ( L ϵ 2 ) m ) .

Discretization scheme for monotone valuations under diminishing returns. Finally, we study discretization schemes under the diminishing returns condition. Our procedure, outlined in Algorithm 2 proceeds as follows. We use the same discretization W of the valuation space from Algorithm 1. Next, we will discretize the dataspace [ N ] . To exploit the structure in the diminishing returns n Y = 2 Jm (1 + ϵ 2 ) i condition, we will need to do so more densely when is small. For this, let i ϵ 2 , i = 0 , . . . , ⌈ log 1+ ϵ 2 Nϵ 2 2 Jm ⌉ be the powers of (1 + ϵ 2 ) on data space [ 2 Jm ϵ 2 , N ] . For each i , the set Q i further partitions the interval [ Y i , Y i +1 ) uniformly with gap Y i · ϵ 2 2 Jm . For n smaller than 2 Jm ϵ 2 , we do not discretize it as the valuations may change rapidly when n is small. Let N D be the union of

Algorithm 2 Price discretization scheme monotonic valuations under diminishing returns

Given: Diminishing returns constant J , approximation parameter ϵ . Let W ∆ = ⋃ ⌈ log 1+ ϵ 1 ϵ ⌉ i =2 W i , were W i s are the same as in Algorithm 1. Let N D be discretization of the interval [0 , N ] defined as follows,

The discretization price set P is the class of all ' m -step' price curves on function space N D → W .

{ 1 , 2 , . . . , ⌊ 2 Jm ϵ 2 ⌋} and all the set Q i . Therefore, N D has a size of at most 2 Jm ϵ 2 +2 Jm ⌈ log 1+ ϵ 2 Nϵ 2 2 Jm ⌉ . We have the following theorem about Algorithm 2 which we prove in Appendix A.5.

Theorem 3.3. Consider the discretization P as constructed in Algorithm 2. Under Assumption 2, for any type distribution, there exists p ∈ P such that rev( p ) ≥ OPT -O ( ϵ ) . Moreover,

Proof outline . By Lemma 3.1, we may assume the optimal price curve p ⋆ = { ( n ⋆ i , p ⋆ i ) } m i =1 is an m -step function, where p ⋆ i denote the value of p on step i . We generate an m -step price curve p = { ( n i , p i ) } m i =1 on space N D → W such that n i is obtained by rounding down n ⋆ i to the closest value in N D , and p i ≥ p ⋆ i / (1 + ϵ ) . We then show that if a buyer purchases at step i under price p ⋆ , she will not purchase at step j < i under new price p . Therefore, the revenue from this buyer is at least p i ≥ p ⋆ i / (1 + ϵ ) = p ⋆ i -O ( ϵ ) , which ensures that rev( p ) ≥ OPT -O ( ϵ ) .

4 Online learning in the stochastic setting

We now study the online learning problem outlined in §2.1 in the stochastic setting. Our Algorithm, outlined in Algorithm 3 is based on the classical upper confidence bound (UCB) algorithm for stochastic bandits [8, 38]. It takes a discretization P of the pricing curves as input, and on each round chooses a p t ∈ P which has the largest UCB on the revenue.

The key challenge lies in constructing an UCB. As P is large, naively constructing UCB over prices in P will lead to a √ |P| T log T upper bound, leading to poor, exponential dependence on m . This is the bound if we only observe the reward for the prices that are actually pulled, but do not observe the types after purchase. Therefore, naively applying UCB is like bandit feedback. On the other extreme, had we been in an alternative setting where we observe the type regardless of purchase, this is like a full information feedback because once observe the type, we know the revenue for all prices. Then

UCB gives us √ log( |P| ) T log T upper bound. We are in an intermediate regime between bandit feedback and full information: The challenge in constructing the UCB arises because we only observe types upon purchase. As the key unknown is the type distribution, we maintain UCBs for it and translate them to UCBs for the revenue. In particular, our UCB depends on how many times a buyer could have purchased at a given round, which is a random quantity depending on the algorithm itself.

Construction of UCB. We will now show how to construct the upper confidence bound ̂ rev t at the end of round t , which will be used in computing p t +1 . For τ ≤ t , let S τ , defined below in (5), be the set of types who would have purchased in round τ at price p τ had they appeared in that round. Then, for any type i ∈ [ m ] , we define T i,t to be the number of times that type i appears in set S τ for

Algorithm 3 Online data pricing in the stochastic setting.

τ ∈ { 1 , . . . , t } . That is, T i,t measures the number of times a buyer of type i would have purchased during the first t rounds. We have,

Note that as we use the 0 price function on round 1, i.e. p 1 ( · ) = 0 , we have T i,t > 0 for all t > 1 . Next, we estimate q i via the fraction of times that type i has appeared in the past t rounds, provided that i ∈ S τ for τ ∈ { 1 , . . . , t } . We have defined this quantity, q i,t below in (6). Via a standard application of Hoeffding's inequality, we can show that ∣ ∣ q i -q i,t ∣ ∣ ≤ √ (log T ) /T i,t with high probability. Using this, we can construct an upper confidence bound ̂ q i,t as follows,

We now translate the UCBs on q to the UCBs on the revenue. Recall from (1) that a buyer of type i will purchase n i,p points at price p and the revenue from this buyer will be p ( n i,p ) . Note that as the seller has access to the valuation curves, he can compute n i,p for any i and price curve p . Since rev( p ) = E i ∼ q [ p ( n i,p )] , we have the following natural UCB for rev( p ) on round t :

This completes the description of our construction. The following theorem bounds the regret for Algorithm 3 when paired with any of the discretization schemes in §3. While the computational complexity of our method depends on |P| , there is no dependence on the regret because of the above construction of the UCB. The proof is given in Appendix C.

Theorem 4.1. Suppose in Algorithm 3 we use a discretization P which is a O (1 / √ T ) additive approximation to any price curve. Then, the regret of Algorithm 3 satisfies E [ R T ] ∈ ˜ O ( m √ T ) .

Proof challenges. When bounding the regret, we first observe that the subsets S ⊂ [ m ] induces a partitioning of the price curves, where p belongs to the partition of S , if all types in S would make a purchase at price p , and all types in S c would not make a purchase at price p . With this insight, we can view the action of a seller as not just choosing a price curve, but also choosing a set S t ⊂ [ n ] . That is, S t can be viewed as a super-arm in a combinatorial semi-bandit problem [37].

5 Online learning in the adversarial setting

We now study the adversarial setting. Similar to the stochastic setting, our algorithm will use a discretization of the price curves from §3. We will control regret by bounding both the discretization error and the algorithm's regret relative to the best pricing curve in the discretization.

Before proceeding, let us first contextualize our feedback model against prior work. If the buyers do not reveal their types, this becomes an adversarial bandit problem with |P| arms (pricing curves) [33]. Using an algorithm such as EXP-3 [9] results in large ˜ O ( T 1 / 2 |P| 1 / 2 ) regret, which is not ideal due to |P| 's exponential dependence in m . Conversely, if buyers reveal their types regardless of purchase,

Algorithm 4 Online data pricing in the adversarial setting.

this is equivalent to full information feedback, where algorithms such as Hedge or Follow-theperturbed-leader (FTPL) [31] yield O ( T 1 / 2 log 1 / 2 |P| ) regret, translating to ˜ O (( mT ) 1 / 2 ) with our discretization schemes in §3. In our intermediate regime, where feedback is only revealed upon purchase, we aim for a middle ground. We show our algorithm, outlined in Algorithm 4, achieves ˜ O ( m 3 / 2 T 1 / 2 ) regret, which is worse than full information, but still depends polynomially on m .

Our algorithm takes a discretization P and a perturbation parameter θ as input. First, it samples a random perturbation θ p from an exponential distribution with pdf θe -θx for each pricing curve p in P . It maintains rewards { r t ( p ) } t,p for each round t and price curve p . On each round, it chooses the price curve that maximizes the perturbed cumulative reward ∑ t τ =1 r τ ( p ) + θ p .

This scheme is similar to FTPL, but the key difference is in how we design the rewards { r t ( p ) } t,p . To describe this, let S t , defined exactly as in (5), be the set of agents who would have purchased in round t at price p t . At the end of the round, if there was a purchase, for all prices p ∈ P , we set the reward to be r t ( p ) = p ( n i t ,p ) , i.e. the payment we would have received from the buyer at that round, had the price been p (see (1)). If there was no purchase, we know that i t / ∈ S t , in which case we set r t ( p ) = ∑ i ∈ S c t p ( n i,p ) . In this case, r t ( p ) is an upper bound on p ( n i t ,p ) , and this upper bound is tight around prices similar to the chosen price p t ; in fact, r t ( p t ) = 0 if there was no purchase. Intuitively, r t ( p ) deals with the uncertainty of not knowing the type on round t by providing a large reward (as we are taking the sum) to prices that could have resulted in a purchase, which encourages exploration of such prices in future rounds. This intuition will help us bound the regret.

Theorem 5.1 provides a bound on the regret for Algorithm 4. Its proof is given in Appendix B. Combining this with the size of P under the various assumptions in §3, we obtain ˜ O ( m 3 / 2 √ T ) regret.

Theorem 5.1. Suppose in Algorithm 4 we use a discretization P which is a O (1 / √ T ) additive approximation to any price curve. Let R T be as defined in (4) . Then, for Algorithm 4, we have E [ R T ] ∈ O ( m 2 θT + θ -1 ( 1 + log ∣ ∣ P ∣ ∣ )) . Setting θ = √ 1+log | P | m 2 T , we have E [ R T ] ∈ O ( m √ T log ∣ ∣ P ∣ ∣ ) .

6 Conclusion and Discussion

We designed revenue-optimal learning algorithms for pricing data. First, we leveraged properties like smoothness and diminishing returns to create novel discretization schemes for approximating any pricing curve. These schemes were then used in our learning algorithms to improve their statistical and computational properties. Our algorithms build on classical methods like UCB and FTPL but required significant adaptations to handle the vast space of pricing curves and the asymmetric feedback. An interesting future direction would be to relax the assumption that the seller knows the valuation curves v i .

Computational complexity. Our algorithm is designed to achieve polynomial computational complexity with respect to the number of data points when the number of types is fixed, making it suitable for practical data pricing scenarios where the type count is typically small or bounded. While the overall computational cost grows exponentially with the number of types due to the problem's strong NP-hardness (see [14]), this design choice ensures computational feasibility in settings with large datasets and a limited number of types.

References

AWS Forecast. https://aws.amazon.com/forecast/ , . Accessed: 2024-05-12.

AWSData Hub. https://aws.amazon.com/blogs/big-data/tag/datahub/ , . Accessed: 2024-05-11.

Azure Data Share. https://azure.microsoft.com/en-us/products/data-share . Accessed: 2024-05-10.

Delta Sharing. https://docs.databricks.com/en/data-sharing/index.html . Accessed: 2024-05-11.

Ads Data Hub. https://developers.google.com/ads-data-hub/guides/intro . Accessed: 2022-05-10.

A. Agarwal, M. Dahleh, and T. Sarkar. A marketplace for data: An algorithmic solution. In Proceedings of the 2019 ACM Conference on Economics and Computation , pages 701-726, 2019.

A. Agarwal, M. Dahleh, T. Horel, and M. Rui. Towards data auctions with externalities. arXiv preprint arXiv:2003.08345 , 2020.

P. Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research , 3(Nov):397-422, 2002.

P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. The nonstochastic multiarmed bandit problem. SIAM journal on computing , 32(1):48-77, 2002.

M. F. Balcan and H. Beyhaghi. New guarantees for learning revenue maximizing menus of lotteries and two-part tariffs. Transactions on Machine Learning Research , 2024.

O. Besbes and A. Zeevi. Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Operations Research , 57(6):1407-1420, 2009.

O. Besbes and A. Zeevi. On the (surprising) sufficiency of linear models for dynamic pricing with demand learning. Management Science , 61(4):723-739, 2015.

S. Chawla, J. D. Hartline, and R. Kleinberg. Algorithmic pricing via virtual valuations. In Proceedings of the 8th ACM Conference on Electronic Commerce , pages 243-251, 2007.

S. Chawla, R. Rezvan, Y. Teng, and C. Tzamos. Pricing ordered items. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing , pages 722-735, 2022.

W. Chen, W. Hu, F. Li, J. Li, Y. Liu, and P. Lu. Combinatorial multi-armed bandit with general reward functions. Advances in Neural Information Processing Systems , 29, 2016.

X. Chen, I. Diakonikolas, D. Paparas, X. Sun, and M. Yannakakis. The complexity of optimal multidimensional pricing. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms , pages 1319-1328. SIAM, 2014.

W. C. Cheung, D. Simchi-Levi, and H. Wang. Dynamic pricing and demand learning with limited price experimentation. Operations Research , 65(6):1722-1731, 2017.

Citrine Informatics. Citrine Informatics - Accelerating Materials Innovation. URL: https: //citrine.io/ , 2024. Accessed: March 9, 2024.

A. V. Den Boer. Dynamic pricing and learning: Historical origins, current research, and new directions. Surveys in Operations Research and Management Science , 20(1):1-18, 2015.

A. V. den Boer and B. Zwart. Simultaneously learning and optimizing using controlled variance pricing. Management Science , 60(3):770-783, 2014.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

M. Dudík, N. Haghtalab, H. Luo, R. E. Schapire, V. Syrgkanis, and J. W. Vaughan. Oracleefficient online learning and auction design. Journal of the ACM (JACM) , 67(5):1-57, 2020.

W. Guo, N. Haghtalab, K. Kandasamy, and E. Vitercik. Leveraging reviews: Learning to price with buyer and seller uncertainty. In Proceedings of the 24th ACM Conference on Economics and Computation , pages 816-816, 2023.

V. Guruswami, J. D. Hartline, A. R. Karlin, D. Kempe, C. Kenyon, and F. McSherry. On profit-maximizing envy-free pricing. In SODA , volume 5, pages 1164-1173, 2005.

J. D. Hartline and V. Koltun. Near-optimal pricing in near-linear time. In Proceedings of the 9th International Conference on Algorithms and Data Structures , WADS'05, page 422-431, Berlin, Heidelberg, 2005. Springer-Verlag. ISBN 3540281010. doi: 10.1007/11534273_37. URL https://doi.org/10.1007/11534273_37 .

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770778, 2016.

M. Jagadeesan, A. Wei, Y. Wang, M. Jordan, and J. Steinhardt. Learning equilibria in matching markets from bandit feedback. Advances in Neural Information Processing Systems , 34:33233335, 2021.

A. Javanmard. Perishability of data: dynamic pricing under varying-coefficient models. The Journal of Machine Learning Research , 18(1):1714-1744, 2017.

A. Javanmard and H. Nazerzadeh. Dynamic pricing in high-dimensions. The Journal of Machine Learning Research , 20(1):315-363, 2019.

R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. Gürel, B. Li, C. Zhang, D. Song, and C. J. Spanos. Towards efficient data valuation based on the Shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 1167-1176. PMLR, 2019.

A. Kalai and S. Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences , 71(3):291-307, 2005.

N. B. Keskin and A. Zeevi. Dynamic pricing with an unknown demand model: Asymptotically optimal semi-myopic policies. Operations Research , 62(5):1142-1167, 2014.

R. Kleinberg and T. Leighton. The value of knowing a demand curve: Bounds on regret for online posted-price auctions. In 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings. , pages 594-605. IEEE, 2003.

A. Krause and C. Guestrin. Submodularity and its applications in optimized information gathering. ACM Transactions on Intelligent Systems and Technology (TIST) , 2(4):1-20, 2011.

A. Krause, H. B. McMahan, C. Guestrin, and A. Gupta. Robust submodular observation selection. Journal of Machine Learning Research , 9(12), 2008.

A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems , 25, 2012.

B. Kveton, Z. Wen, A. Ashkan, and C. Szepesvari. Tight regret bounds for stochastic combinatorial semi-bandits. In Artificial Intelligence and Statistics , pages 535-543. PMLR, 2015.

T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in applied mathematics , 6(1):4-22, 1985.

K. Misra, E. M. Schwartz, and J. Abernethy. Dynamic online pricing with incomplete information using multiarmed bandit experiments. Marketing Science , 38(2):226-252, 2019.

G. Perakis and D. Singhvi. Dynamic pricing with unknown nonparametric demand and limited price changes. Operations Research , 2023.

S. Shalev-Shwartz and S. Ben-David. Understanding machine learning: From theory to algorithms . Cambridge university press, 2014.

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1-9, 2015.

T. Wang, J. Rausch, C. Zhang, R. Jia, and D. Song. A principled approach to data valuation for federated learning. Federated Learning: Privacy and Incentive , pages 153-167, 2020.

Y. Wang, B. Chen, and D. Simchi-Levi. Multimodal dynamic pricing. Management Science , 67 (10):6136-6152, 2021.

L. Wasserman. All of nonparametric statistics . Springer Science & Business Media, 2006.

J. Xu and Y.-X. Wang. Logarithmic regret in feature-based dynamic pricing. Advances in Neural Information Processing Systems , 34:13898-13910, 2021.

H. Zhao and W. Chen. Stochastic one-sided full-information bandit. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 150-166. Springer, 2019.

A Omitted Details from Section 3

A.1 Proof of Lemma 3.1

Lemma 3.1. Assume there are m types with non-decreasing value curves { v i } i ∈ [ m ] . For any nondecreasing price curve p , there exists an ' m -step' price curve ¯ p that yields expected revenue at least that of p with respect to any distribution over the m types. Here, m -step refers to non-decreasing functions f : [ N ] → [0 , 1] where f ( n +1) -f ( n ) > 0 in at most m points (i.e., at most m jumps).

Proof of Lemma 3.1. Fix a price curve p . Let n i,p be the amount of data type i purchase at price curve p , that is

For { n i,p } i ∈ [ m ] , let π : [ m ] → [ m ] be a permutation such that n π (1) ,p ≤ n π (2) ,p ≤ · · · ≤ n π ( m ) ,p . Let n ( i ) ∆ = n π ( i ) ,p . Then, define a function ¯ p : [ N ] → [0 , 1] as follows,

so that ¯ p has at most m steps. Then, ¯ p has following properties,

We next prove that for any i ∈ [ m ] , after changing the price function from p to ¯ p , the type i buyer either purchases at ( n i,p , p ( n i,p )) or at ( N,p ( n ( m ) )) .

For any type i and any amount of data n ≤ n ( m ) , there exists k such that n ( k -1) < n ≤ n ( k ) (let n (0) = 0 ), we then have

As shown in the above, type i still prefers purchasing n i,p data over all n ≤ n ( m ) under price ¯ p .

For n ∈ { n ( m ) +1 , . . . , N } , by the monotonicity of value curves, we have

Therefore, for any i ∈ [ m ] , type i either purchases at ( n i,p , p ( n i,p )) , or purchases at ( N, ¯ p ( N )) = ( N,p ( n ( m ) )) under price ¯ p . No matter in which case, type i contributes no less revenue under ¯ p than p . It then follows that, for any type distribution q ,

A.2 Proof of Theorem 3.1

In this subsection, we prove Theorem 3.1 by decomposing it into three technical lemmas (Lemma A.1, A.2 and A.3). In Lemma A.1 and A.2, we prove the approximation guarantee of our discretization scheme and, in Lemma A.3 we provide an upper bound on the size of the discretization.

Lemma A.1. For any type distribution, there exists a pricing function ˜ p : [ N ] → [ ϵ, 1] such that

Proof of Lemma A.1. Consider the optimal pricing function p ⋆ : [ N ] → [0 , 1] , i.e., OPT = rev( p ⋆ ) . Consider price curve ˜ p : [ N ] → [ ϵ, 1] where ˜ p ( n ) = max( ϵ, p ⋆ ( n )) .

Let J ∆ = { n ∈ [ N ] : ˜ p ( n ) = p ⋆ ( n ) } be the set of data quantities whose price under ˜ p are the same as those under p . Any buyer type who would have purchased n ∈ J amount of data under p ⋆ will purchase the same amount of data under ˜ p . On the other hand, for buyer types who would have purchased n / ∈ J amount of data under p ⋆ , since ˜ p ( n ) = ϵ > p ⋆ ( n ) for n / ∈ J , the expected revenue contribution from such buyers under p ⋆ is at most ϵ , hence no matter they purchase or not under ˜ p , we have rev( ˜ p ) ≥ OPT -ϵ .

Lemma A.2. For any ˜ p ∈ [ ϵ, 1] N there exists p ′ ∈ P such that rev( p ′ ) ≥ rev( ˜ p ) / (1 + ϵ ) , for any type distribution q .

Proof of Lemma A.2. For m buyer types, by Lemma 3.1, there exists a non-decreasing step function ¯ p ∈ [ ϵ, 1] N with at most m steps, whose expected revenue is at least rev( ˜ p ) . Assume ¯ p has k steps, k ≤ m . To simplify the notation, for 1 ≤ j ≤ k , let ¯ p j denote the price ¯ p on j th step. That is,

Where i 1 , . . . , i k -1 ∈ [ N ] are discontinuities in ¯ p .

Recall the definitions of Z and W as stated in Algorithm 1,

Let i k = N and for each j ∈ [ k ] , let Z i j be the price obtained by rounding ¯ p j down to the nearest value in Z . By constructions of Z and W above, W i j is a partition of interval ( Z i j -1 , Z i j +1 ) . Let w j be the price obtained by rounding ¯ p j down to the nearest value in W i j . Set d j ∆ = ϵ m · Z i j -1 and consider k -step function p defined by whose price at j th step (denoted p j ) is w j -( j -1) d j ∈ W i j , that is

By the tie-breaking rule and the monotonicity of valuation curves, buyers only purchase among 0 , i 1 , i 1 , . . . , i k number of data under p and ¯ p .

Subclaim. Then, p and ¯ p satisfies the following

with respect to any type distribution.

Proof of the Subclaim. We prove the above subclaim with two steps.

Step 1 : No buyer who prefers to purchase i j data under ¯ p would prefer i j ′ data for some j ′ < j under p (i.e., one with a less price). This is because, when going from price ¯ p to p , the increase in the buyer's utility for i j data is ¯ p j -p j , which is higher than the increase ¯ p j ′ -p j ′ for i j ′ data. Formally, this can be seen as follows: For any j ′ < j we have,

as ¯ p j ≥ w j and p j = w j -( j -1) d j . Moreover,

The inequality ¯ p j ′ < w j ′ + d j ′ holds because w j ′ is the result of rounding down ¯ p j to the nearest value in W i j .

By constructions of sets Z and W , we have d j ≥ d j ′ which implies ( j -1) d j ≥ j ′ d j ′ . Then, by combining the above inequalities, we obtain

Consider a buyer with value curve v who prefers to purchase at i j under price ¯ p , then it must be

Then, by combining (10) and (11), we have

therefore the buyer would not purchase at i j ′ < i j under p .

Step 2 : Next, we claim that p j ≥ ¯ p j / (1 + ϵ ) for all step j ∈ [ k ] . Since Z i j is obtained by rounding ¯ p j down to the nearest value in Z , we have

By (9) and the above, we have

where the first inequality is by (9), the second is by (12), and the third is because m ≤ j .

Then, it follows that

So far we have proved p j ≥ ¯ p j / (1 + ϵ ) and no type wants to change their preference to a smaller amount of data under p . If one type purchase at ¯ p i under ¯ p and p k under p for k ≥ i , then p k ≥ p i ≥ ¯ p i / (1 + ϵ ) . Therefore, we have

Since the construction of price p is not relevant to type distribution, the above holds for any type distribution q , which proves the subclaim.

Note that p constructed in the above subclaim is not necessarily non-decreasing as a larger amount of data surfers more price deduction when going from ¯ p to p . In this case, we can directly construct a non-decreasing price curve p ′ ∈ P from p such that

Let S ∆ = { i ∈ [ k ] : ∃ j < i, s.t. p j > p i } . If S is empty, this implies that p is non-decreasing, hence setting p ′ = p . If S is not empty, we define p ′ as follows: Let p ′ be a k -step function with the same jump points i 1 , . . . , i k as p . Let p ′ i be the value of p ′ on i th step. Then, for i / ∈ S , let p ′ i = p i ; and for i ∈ S , let p ′ i = max j / ∈ S,j<i p j . By construction, p ′ is non-decreasing. Moreover, p ′ = p on set S c and p ′ > p on set S .

Next, we claim that ¯ p j -p ′ j is non-decreasing for all j ∈ [ k ] . Both (¯ p j -p j ) j ∈ [ k ] and ¯ p are non-decreasing with respect to j by the previous results. Hence,

Therefore, any type that prefers to purchase at j th step under ¯ p would not prefer purchasing at any step j ′ < j under p ′ , and since p ′ j ≥ p j ≥ ¯ p j / (1 + ϵ ) , we have

Lemma A.3. When n > m , ∣ ∣ P ∣ ∣ ≤ ( eN m ) m ( e ⌈ (2 + ϵ ) ⌉ ⌈ log 1+ ϵ 1 ϵ ⌉) m .

Proof of Lemma A.3. For any integer i ≤ m , the number of non-decreasing i -step price function is ( N -1 i )( | W | i ) , hence we have

In the last inequality, we use the fact that | W | ≤ ⌈ (2 + ϵ ) m ⌉ ⌈ log 1+ ϵ 1 ϵ ⌉ .

Finally, Theorem 3.1 follows directly from the above lemmas.

Theorem 3.1. Consider the discretization P as constructed in Algorithm 1. For any type distribution, there exists p ∈ P such that rev( p ) ≥ OPT - O ( ϵ ) . Moreover, we have |P| ≤ ( e ( N -1) m ) m ( e ⌈ (2 + ϵ ) ⌉ ⌈ log 1+ ϵ 1 ϵ ⌉) m ∈ ˜ O ( ( N ϵ ) m ) .

Proof of Theorem 3.1. Combining Lemma A.1 and Lemma A.2 together, we conclude that there exists price curve p ′ ∈ P such that

The size of P follows from Lemma A.3.

A.3 Price discretization scheme for smooth monotonic valuations

We study discretization schemes to approximate monotone valuations under the smoothness condition in Assumption 1. Our procedure is outlined in Algorithm 5. The discretization W of the valuation space follows Algorithm 1. Additionally, we uniformly split the data space into multiples of ⌊ ϵN mL ⌋ , denoting them as the set N S . We then set the discretization P to be the class of all ' m -step' price curves on the function space N S → W . The following theorem, proven in Appendix A.4, outlines the main properties of this discretization scheme: the size of the discretization has no dependence on the number of data N .

Algorithm 5 Price discretization scheme for smooth monotonic valuations

Given: Smoothness constant L , approximation parameter ϵ > 0 . Let W be discretization of the valuation space [0 , 1]

given in Algorithm 1.

Let N S be the following discretization of the interval [0 , N ] ,

Set P to be the class of all ' m -step' functions mapping N S → W .

A.4 Proof of Theorem 3.2

Theorem 3.2. Consider the discretization P as constructed in Algorithm 5. Under Assumption 1, for any type distribution, there exists p ∈ P such that rev( p ) ≥ OPT - O ( ϵ ) . Moreover, |P| ∈ O ( log m 1+ ϵ (1 /ϵ ) · ( L/ϵ ) m ) ∈ ˜ O ( ( L ϵ 2 ) m ) .

Proof of Theorem 3.2. By Lemma 3.1, there is a revenue optimal price curve p ⋆ : [ N ] → [0 , 1] which is a k -step function, for some k ∈ [ m ] . Where p ⋆ can be compactly represented as the following set of tuples:

where n ⋆ 1 , . . . , n ⋆ k denote the locations of jumps and p ⋆ i denote the value of p ⋆ on step i ∈ [ k ] (i.e. p ⋆ ( n ) = p ⋆ i for n ∈ ( n ⋆ i -1 , n ⋆ i ] ).

Let ¯ ϵ := ϵ m . Next, we generate a price p ′ using Algorithm 6, which ensures that the price curve p generated in the following step (13) is non-decreasing. We demonstrate that in each round of Algorithm 6, we incur a revenue loss of at most ¯ ϵ . If p ′ i > p ′ i -1 +¯ ϵ , everything remains the same and thus does not affect the expected revenue. If not, we combine the price of step i with step i -1 , let p ′ j ∆ = p ′ j -( p ′ i -p ′ i -1 ) for j = i, . . . , k . During this process, buyers either make purchases at the same step, or switch to purchase at a higher step. Note that p ′ i -p ′ i -1 < ¯ ϵ , so the revenue loss of each type is at most ¯ ϵ . This implies that the revenue loss in each round is at most ¯ ϵ . As there are k rounds, we lose expected revenue of at most m ¯ ϵ . We conclude that rev( p ′ ) is within a gap of ϵ from OPT , i.e., rev( p ′ ) ≥ OPT -ϵ .

Algorithm 6 Auxiliary Price Adjustment

After combining some steps in Algorithm 6, Assume that p ′ is a ¯ k -step function ( ¯ k ≤ k ) represented by

Then, we define a new price curve p ∈ P as follows: let δ := ⌊ ¯ ϵN L ⌋ , then p is a ¯ k -step function represented by

First, we show that no buyer who purchases at step i under p ′ would purchase at step j < i under p . Let the buyer's valuation be v . First, we prove that the buyer's utility is non-negative at n i :

Then, we prove that the buyer's utility at n i is larger than that of n j for j < i , therefore, the buyer would not prefer buying at step j < i under price p .

Finally, fix the type distribution ( q 1 , . . . , q m ) , then we have

where

where n i is given by

Hence, rev( p ) is within a gap of 2 ϵ from OPT .

We then apply Theorem 3.1 to price p . Therefore, it is enough to consider price functions from the set N S ∆ = { kδ : k = 1 , . . . , ⌈ N δ ⌉} ⊆ [ N ] to W to approximate the revenue within O ( ϵ ) gap. Moreover, this discretization is of the size ⌈ N δ ⌉ | W | ∈ O ( ( log 1+ ϵ ( 1 ϵ )) m ( L ϵ ) m ) as ⌈ N δ ⌉ ∈ O ( Lm ϵ ) .

A.5 Proof of Theorem 3.3

Theorem 3.3. Consider the discretization P as constructed in Algorithm 2. Under Assumption 2, for any type distribution, there exists p ∈ P such that rev( p ) ≥ OPT -O ( ϵ ) . Moreover,

Proof of Theorem 3.3. For each i = 0 , 1 , . . . , ⌈ log 1+ ϵ 2 ( Nϵ 2 2 Jm )⌉ , let Y i ∆ = ⌊ 2 Jm ϵ 2 (1 + ϵ 2 ) i ⌋ , and Q i be the set {⌊ Y i + Y i ϵ 2 2 Jm k ⌋ : k = 1 , . . . , ⌊ 2 Jm ⌋ } , i.e., Q i splits the interval [ Y i , Y i +1 ] equally into 2 mJ parts.

The union of Q i s and the set { 1 , 2 , . . . , ⌊ 2 Jm ϵ 2 ⌋} form a set of grids on [0 , N ] , denoted by N D . There are at most 2 Jm ϵ 2 +2 Jm log 1+ ϵ 2 ( Nϵ 2 2 Jm ) grids in total.

By Lemma 3.1, there is a revenue optimal price curve p ⋆ : [ N ] → [0 , 1] which is a k -step function, for some k ∈ [ m ] . Where p ⋆ can be compactly represented as the following set of tuples:

where n ⋆ 1 , . . . , n ⋆ k denote the locations of jumps and p ⋆ i denote the value of p ⋆ on step i ∈ [ k ] (i.e. p ⋆ ( n ) = p ⋆ i for n ∈ ( n ⋆ i -1 , n ⋆ i ] ).

Then, define a new k -step price curve p via

n i ← round down n ⋆ i to the closest grid in N D .

Then we define p i below. If p ⋆ i < ϵ (1 + ϵ ) , let p i = ϵ (1 + ϵ ) ; otherwise, let Z n ⋆ i be the price obtained by rounding p ⋆ i down to the nearest value in Z . By constructions of Z and W above, W n ⋆ i is a partition of interval ( Z n ⋆ i -1 , Z n ⋆ i +1 ) . Let w i be the price obtained by rounding p ⋆ i down to the nearest value in W n ⋆ i . Set d i ∆ = ϵ m · Z n ⋆ i -1 . Then define p i ∆ = w i -i · d i ∈ W n ⋆ i .

First, we prove for i satisfying p ⋆ i > ϵ (1 + ϵ ) , if a buyer purchases at n i under price p ⋆ , she will not purchase at n j , j < i under new price p . We prove this property separately when n i ≤ 2 Jm ϵ 2 and n i > 2 Jm ϵ 2 .

(i) When n i > 2 Jm ϵ 2 .

The buyer's utility at n i under price p is,

Let δ i ∆ = v ( n ⋆ i ) -v ( n i ) . Then δ i is upper bounded by,

where the third inequality is due to Lemma A.4.

By the construction of p , we have

Therefore, by (14), v ( n i ) -p i ≥ v ( n ⋆ i ) -p ⋆ i ≥ 0 , buyer's utility at n i under price p is non-negative.

Next, we claim that v ( n i ) -p i -( v ( n j ) -p j ) ≥ 0 . To prove this, for any j < i , let δ j ∆ = v ( n ⋆ j ) -v ( n j ) , then we have

Where v ( n ⋆ i ) -p ⋆ i -( v ( n ⋆ j ) -p ⋆ j ) ≥ 0 because the buyer prefers n ⋆ i over n ⋆ j under price p ⋆ . Recall that we have δ j ≥ 0 , then we bound δ i -δ j as follows,

By the construction of p i , we have,

Therefore, combining (17) and (18) together, we have

We conclude that under price p , the buyer prefers n i over n j , for any j < i .

(ii) When n i ≤ 2 Jm ϵ 2 .

In this case, n i = n ⋆ i , and for any j < i , we still have n j = n ⋆ j . First, we prove the buyer's utility at n ′ i under p is non-negative:

Then, we show that the buyer prefers n i over n j under p :

where the first inequality is due to (18), and the second is because the buyer prefers n ⋆ i over n ⋆ j under p ⋆ .

So far we have completed the proof that for i satisfying p ⋆ i > ϵ (1 + ϵ ) , if a buyer purchases at n i under price p ⋆ , she will not purchase at n j , j < i under new price p .

Then, similar to Step 2 in the proof of Lemma A.2, we have p ≥ p ⋆ 1+ ϵ pointwise. We then conclude the proof by observing

Lemma A.4. When n i > 2 Jm ϵ 2 , we have n ⋆ j -n i ≤ n i · ϵ 2 2 Jm +1 .

Proof of Lemma A.4. By the construction of discretization set, n i must have the following form,

Since n ′ j is obtained by rounding down n j to the nearest grid in N D , n j satisfies the following inequality,

Therefore, we have

Where in the last inequality, since Y i ′ is an integer, and we have

B Proof of Theorem 5.1

Theorem 5.1. Suppose in Algorithm 4 we use a discretization P which is a O (1 / √ T ) additive approximation to any price curve. Let R T be as defined in (4) . Then, for Algorithm 4, we have E [ R T ] ∈ O ( m 2 θT + θ -1 ( 1 + log ∣ ∣ P ∣ ∣ )) . Setting θ = √ 1+log | P | m 2 T , we have E [ R T ] ∈ O ( m √ T log ∣ ∣ P ∣ ∣ ) .

Proof of Theorem 5.1. Recall that the regret R T for the adversarial setting is

We decompose R T into two regrets. The first term is the sacrifice of revenue on discretization. The second term is the algorithm regret when competing against the optimal price within the discretization set P .

According to Theorem 3.1, our discretization scheme approaches optimal revenue within a gap of 2 ϵ 1+ ϵ :

Therefore, the first term can be bounded by 2 ϵT .

According to Theorem B.1, the second term discretization regret is upper bounded by

Combining (20) and (21) together, we have,

Plug in the size of discretization set in Section 3, we have,

Theorem B.1. The discretization regret R T defined in (19) has upper bound O ( m √ T log ∣ ∣ P ∣ ∣ ) .

Proof of Theorem B.1. We first claim that r t ( p t ) = r ( i t , p t ) all t . If the buyer make a purchase at round t , r t ( p t ) = r ( i t , p t ) holds by definition. But if the buyer does not purchase at a price p t on round t , r ( i t , p t ) = 0 . Since S c t contains all the types that would not make a purchase at p t , we have r ( i, p t ) = 0 , ∀ i ∈ S c t , and

Therefore, r t ( p t ) = r ( i t , p t ) holds for every round t ∈ [ T ] . Denote p ⋆ as,

Then, we decompose the regret as follows,

We bound three terms in (22) separately.

The first term. For any price p and any round t , we have r t ( p ) ≥ r ( i t , p ) by definition. Hence,

The second term. Since p ⋆ = argmax p ∈P ∑ T t =1 r ( i t , p ) . We apply Lemma B.1 to p ⋆ ,

Note that both θ p 1 and θ p ⋆ are drawn i.i.d. from exponential distribution,

We have

The third term. Note that for any price p ∈ P and any round t , r t ( p ) ≤ m . Therefore we have,

̸

The price curve on round t is p t , then by the price updation rule,

For all p ′ ∈ P , let c t -1 ,p ′ denote

then p t = p ′ is equivalent to

̸

̸

which is equivalent to, then p t +1 = p t .

Proof of the Subclaim. If (27) holds for all p ∈ P ,

Hence,

Subclaim. If θ p t also satisfies the following condition (27),

Therefore, (27) is a sufficient condition for p t +1 = p t . We then bound the probability of p t +1 = p t by computing the probability of (27) happening.

̸

Therefore, P ( p t = p t +1 ) ≤ mθ . Hence, the third term can be bounded as

Set θ = √ log | P | m 2 T . Combining the upper bounds for three terms (23), (24) and (28) together, we have

Plugging in the size of the discretization set (Theorem 3.1), we have,

Proof of Lemma B.1. We prove this by induction. For T = 0 , the inequality θ p 1 ≥ θ p holds by definition p 1 = argmax p ∈P θ p . Assume that the inequality holds for some T . Then for any p ∈ P ,

Where the first inequality is by the induction hypothesis, and the second inequality is by

By the induction, the inequality (29) holds for any T ≥ 0 .

C Proof of Theorem 4.1

In this section, we prove, Theorem 4.1, our regret upper bound of Algorithm 3. We prove the theorem by first decomposing the regret into two parts: Regret with respect to the best price in a discretized set (called 'discretization regret') and the residual error due to discretization. The residual error is controlled by the approximation guarantees developed in Section 3. Then, the key lemma in this appendix is Lemma C.1 which controls the discretization. We prove Lemma C.1 using a technique adapted from Chen et al. [15].

Theorem 4.1. Suppose in Algorithm 3 we use a discretization P which is a O (1 / √ T ) additive approximation to any price curve. Then, the regret of Algorithm 3 satisfies E [ R T ] ∈ ˜ O ( m √ T ) .

Proof of Theorem 4.1. For the sake of simplicity, we define r ( i, p ) as the revenue under type i and price p , i.e, r ( i, p ) ∆ = p ( n i,p ) . Therefore, on every round, we have r ( i t , p t ) = p t ( n i t ,p t ) . Recall that the regret R T is

We decompose R T into two parts. The first term is the sacrifice of revenue on discretization. The second term is the algorithm regret when competing against the optimal price within the discretization set P .

According to Theorem 3.1, our discretization scheme approaches OPT within a gap of 2 ϵ 1+ ϵ ,

Therefore, the first term can be bounded as,

By Lemma C.1, the second term, discretization regret, is upper bounded by

Combining (31) and (32) together, we have,

Lemma C.1. The discretization regret R T defined in (30) is at most ˜ O ( m √ T ) .

Proof of Lemma C.1. The discretization regret R T

We can further decompose E [ R T ] into ∑ T t =1 E [ δ p t · ■ ( A t )] and ∑ T t =1 E [ δ p t · ■ ( A c t )] . Where for any round t , we define the good event A t as follows,

Define q i,t ∆ = ∑ t τ =1 ■ ( i ∈ S τ ,i τ = i ) T i,t = ∑ t s =1 ■ ( i ∈ S τ ) · ■ ( i τ = i ) ∑ t τ =1 ■ ( i ∈ S τ ) . Note that ■ ( i τ = i ) is a random variable that follows Bernoulli distribution Ber ( q i ) , and one can only observe ■ ( i τ = i ) when i ∈ S τ , let x i,j denote the mean value of first j i.i.d. observations of ( i s = i ) . Then, we have

Where in the first inequality, the event { ∣ ∣ q i,t -q i ∣ ∣ > √ log T T i,t , T i,t = j } indicates { | x i,j -q i | > √ log T j } , and the second inequality follows from Hoeffding's inequality. We then bound the second term in (33)

Define event H t ∆ = { 0 < δ p t < 2 ∑ i ∈ S t √ log T T i,t -1 } . By Lemma C.3, we know that

It remains to prove the upper bound for ∑ T t =1 E [ δ p t ■ ( A T )] . For t ∈ { 1 , . . . , T } and k ∈ Z + , let and

Then, we define an event

which means 'In the t -th round, at least β k m types in S t has been observed at most m k,t times'. Then, by Lemma C.5, we have

For i ∈ [ m ] , k ∈ Z + , t ∈ [ T ] , define an event

Then by the definitions of G k,t and G i,k,t we have

Therefore,

For any price function p , define δ p ∆ = rev( p ⋆ ) -rev( p ) . If δ p > 0 , we call it a 'bad' price. Let E B ∆ = { i ∈ [ m ] : type i would make a purchase at least one bad price } .

For each type i ∈ E B , suppose i is contained in N i bad prices p B i, 1 , p B i, 2 , . . . , p B i,N i . Let δ i,l ∆ = δ p B i,l ( l ∈ [ N i ]) . Without loss of generality, we assume δ i, 1 ≥ δ i, 2 ≥ · · · ≥ δ i,N i . Let δ i, min ∆ = δ i,N i . For convenience, we also define δ i, 0 = + ∞ , i.e., α k ( 2 m δ i, 0 ) 2 = 0 . Then, we have

where the last inequality is due to Lemma C.4. Finally, for each i ∈ E B we have

It follows that

So far, the distribution-dependent regret bound is proven. To prove the distribution-independent bound, we decompose ∑ T t =1 ( H t ) · δ p t into two parts:

where ϵ > 0 is a constant to be determined. The second term can be bounded in the same way as in the proof of the distribution-dependent regret bound, except that we only consider the case

δ p t > ϵ . (For each type i ∈ E B , suppose i is contained in N i bad prices p B i, 1 , p B i, 2 , . . . , p B i,N i . Let δ i,l ∆ = δ p B i,l ( l ∈ [ N i ]) satisfies δ i, 1 ≥ δ i, 2 ≥ . . . ≥ δ i,N i ≥ ϵ . Also let δ i, min ∆ = δ i,N i .) Thus, we can replace (34) by

It follows that

Finally, letting ϵ = √ 2136 m 2 log T T , we get

Lemma C.2. Under good event A t , for any price function p , let S p denote the set of types who would purchase at price p , then we have

Proof of Lemma C.2. When A t happens,

for all i ∈ [ m ] .

Therefore, we have

and

The last inequality is by r ( i, p ) ≤ 1 .

Lemma C.3. For each t ∈ [ T ] , under good event A t -1 , the following inequality holds,

Proof of Lemma C.3. When A t -1 happens, by Lemma C.2,

It then follows that,

Since p t = argmax p ∈P ̂ rev t -1 ( p ) , we have

Lemma C.4 (Theorem 4 of Kveton et al. [37]) . We can choose { α k } k ≥ 0 and { β k } k ≥ 0 , which satisfy the following properties: { α k } k ≥ 0 and { β k } k ≥ 0 are positive and

such that lim k →∞ α k = lim k →∞ β k = 0 . Moreover,

Lemma C.5. On round t , if event H t happens, then at least one event G k,t , k ∈ Z + happens, where

and m k,t = α k ( m δ p t ) 2 log T when δ p t > 0 and + ∞ otherwise.

Proof of Lemma C.5. Assume that H t happens and that none of G 1 ,t , G 2 ,t , . . . happens. Then | A k,t | < β k m for all k ∈ Z + . Let A 0 ,t = S t and ¯ A k,t = S t \ A k,t for k ∈ Z + ∪ { 0 } . Thus ¯ A k -1 ,t ⊆ ¯ A k,t for all k ∈ Z + . Note that lim k →∞ m k,t = 0 . Thus there exists N ∈ Z + such that ¯ A k,t = S t for all k ≥ N , and then we have S t = ⋃ ∞ k =1 ( ¯ A k,t \ ¯ A k -1 ,t ) . Finally, note that for all i ∈ ¯ A k,t , we have T i,t -1 > m k,t . Therefore

Under event H t , we have

where the last inequality is due to Lemma C.4. We reach a contradiction here, hence the lemma follows.

D Miscellaneous

D.1 Notations

The following table contains the notations used in this paper.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We fully included paper's contributions and scope in the appendix. See §1.1 for the summary of our contributions.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See §6 for future works on relaxing one of key assumptions of the paper.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provided the full set of assumptions. Moreover, we provided the full proofs of each Lemma and Theorem in this paper both in main text and Appendices.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: This paper does not include experiments.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: This paper does not include experiments requiring code.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: This paper does not include experiments.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: This paper does not include experiments.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: This paper does not include experiments.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research conforms, in every respect, with the NeurIPS Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our research is theoretical and have no societal impact in a foreseeable future.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper poses no such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: This paper does not use existing assets.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies an online learning problem for pricing homogeneous data, where the seller needs to offer a price function/curve based on the data size and learn the arrival probabilities of different customer types to maximize cumulative revenue. The paper first analyzes the structure of the optimal price function when there is a finite number of customer types with known arrival probabilities. It introduces new discretization schemes to achieve better dependence on the approximation error compared to existing methods. When the arrival probabilities are unknown (i.e., in the online learning setting), the paper develops algorithms for both stochastic and adversarial settings with theoretically bounded regret.\n\nSTRENGTHS:\nThe paper is well-written and introduces an interesting online learning problem in data pricing. It explores the unique structure of the optimal pricing curve in this problem and provides algorithms to solve this task under both stochastic and adversarial settings with theoretical regret bounds. The paper also discusses its technical contributions in deriving theoretical results.\n\nWEAKNESSES:\n(1) Computational Complexity: The provided algorithms (Algorithm 3 and 4) have computational complexity depending on the horizon $T$, which can be large in practice.\n\n(2) Lack of Numerical Validations: Since the problem is motivated by a practical pricing scenario, numerical experiments could help validate the efficiency of the algorithms, especially given the potentially large computational complexity.\n\nQUESTIONS:\n1. In Theorems 4.1 and 5.1, can  the discretization additive approximation $O(\\frac{1}{\\sqrt{T}})$ be replaced by $O(\\frac{1}{\\sqrt{t}})$ while maintaining the same order of regret (i.e., the discretization level changes across the periods)? The intuition is that when the estimations of $q$'s  are inaccurate, we may only need an inaccurate discretization of the price curve, potentially reducing computational complexity.\n2. How can the Upper Confidence Bound (UCB) on $q$'s facilitate exploration in learning $q$'s (i.e., more explorations)? My understanding is that to learn $q$'s (i.e., collecting samples of $q$'s), we need to encourage purchases by setting a price curve lower than the estimated optimal one. However, the UCB on $q$'s does not obviously encourage lower prices.\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMotivated by the emergence of data marketplaces, this paper studies an online data pricing problem involving $ N $ homogeneous data points and $ m $ types of buyers in the market. Specifically, it assumes that each type of buyer has a specific value function $ v_i: [N] \\rightarrow [0, 1] $. While the sellers know all the value curves $ v_i $, they do not know the distribution of buyers. The sellers need to choose a pricing curve $ p \\in \\mathcal{P}: [N] \\rightarrow [0, 1] $ at each time period. To address this online pricing problem, the authors develop novel discretization schemes to approximate any pricing curve. To minimize the discretization size, they propose assumptions such as smoothness and diminishing returns. To solve the online learning problem, they build on classical algorithms like UCB (Upper Confidence Bound) and FTPL (Follow-The-Perturbed-Leader), providing corresponding regret bounds. These advancements contribute to more effective and efficient online data pricing strategies.\n\nSTRENGTHS:\n1. The topic related to marketplaces is very interesting. This paper is well-written and easy to follow.\n\n2. The authors propose three types of price discretization schemes under different assumptions: monotonic valuations, smooth monotonic valuations, and monotonic valuations under diminishing returns. These schemes, based on Algorithm 1, significantly reduce the discretization size. While Algorithm 1 is an existing technique, the proposed price discretization schemes are non-trivial and add substantial value.\n\n3. The proof provided is rigorous, and the theoretical results demonstrate sublinear regret with respect to $T$.\n\nWEAKNESSES:\n1. I think Sections 3 and 4-5 are independent. Section 3 primarily introduces the price discretization schemes, while Sections 4-5 discuss online learning algorithms for both settings. In my opinion, the main contribution of your paper lies in Section 3, as the techniques in Sections 4-5 seem to be standard.\n\n2. You claim that your analysis when constructing the UCB in this way is non-trivial since the types are observed only if they make a purchase. However, this seems common. Could you elaborate on the additional difficulties for the UCB algorithms in your case? Additionally, in line 266, maintaining UCBs for the type distribution appears to be a standard approach, and I don't see this as a significant contribution.\n\n3. Section 5 is a little confusing. Why is it necessary to add perturbation?  Maybe you should give me some intuition. Moreover, in line 320, why is \\( r_t(p) \\) an upper bound on \\( p(n_{i_t,p}) \\)?\n\nQUESTIONS:\n1. While I understand that Algorithm 1 is for price discretization, and the valuation space is discretized, it remains unclear how the price set \\( \\bar{\\mathcal{P}} \\) is determined. Could you provide more details on this process? \n2. Although you have provided the regret bound, additional discussion on its tightness would be valuable. Can you elaborate on the tightness of the regret bound and compare it with existing results? \n3. Sections 4-5 discuss the online learning algorithms. Could you highlight the specific difficulties you encountered in these sections? What new techniques did you develop? What are your unique contributions? Clarifying these points will help distinguish your work from standard techniques.\n\nLIMITATIONS:\nAs mentioned above, the studied topic is very interesting. However, I am eager to see more detailed statements regarding the unique contributions, particularly in Sections 4-5.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers a data pricing problem in which a seller has $N$ homogeneous data points they wish to sell access to.  The seller sets a price curve $p(n)$ which specifies the price a buyer must pay for access to $n$ data points for each $n \\in [N]$.  Upon arrival, a buyer sees the price curve, and chooses an amount $n$ by maximizing their utility $u(n) = v(n) - p(n)$, where $v(n)$ is their valuation curve.  The buyer leaves without making a purchase if their utility is negative for all $n$.  It is assumed that there are $m$ buyer types, where all buyers of type $i$ have the same valuation curve $v_i$, and that there is a distribution $q$ over buyer types in which the arriving type is sampled i.i.d. from $q$ in each step.  The objective is then to find a price curve which maximizes the expected revenue over this distribution.  This paper focuses on online learning settings in which the distribution is unknown and must be learned or the arrival sequence is arbitrary and we want to compare to the best fixed price curve in hindsight, where now the goal is to minimize the regret.\n\nDue to the nature of this problem (a relation to revenue maximization with unit-demand buyers), computing the exact optimal is intractable, so approximations are considered.  This is done by discretizing the space of valuation curves.  The main result of this paper are new discretization schemes for this problem which can accommodate various further assumptions such as smoothness or diminishing returns in the valuation curves.  These are then applied to develop algorithms with regret $\\tilde{O}(m \\sqrt{T})$ and $\\tilde{O}(m^{3/2} \\sqrt{T})$ in the stochastic and adversarial settings, respectively.\n\n-----------------------\n\nEdit: following the rebuttal some of my questions have been answered and my score has been raised.\n\nSTRENGTHS:\nOverall, this is a well-written paper which introduces a very interesting and relevant problem and gives a clean description of the proposed solutions.  While the techniques for online learning are based on fairly standard ideas (UCB and FTPL), they still require some small tricks that are particular to the new setting in this paper.\n\nWEAKNESSES:\nThere are a few concerns:\n - Lack of tightness - it is not clear to what extent these results are tight as no lower bounds are given.\n - The finite type assumption - assuming both that the number of types is small (so that the complexity bounds are reasonable) and that all the types are known up front is somewhat limiting.  Anything to address either of these comments is a significant improvement.\n - Once given the discretization scheme, the online learning methods utilize fairly standard techniques.  It would be interesting if there are other approaches to this problem.\n - A lack of an experimental evaluation which might help to provide further evidence of the practicality of the proposed methods.\n\nAs a minor comment, the organization in the appendix is poor as it does not follow the order in the main paper (appendix C gives proofs for section 5, while appendix D gives proofs for section 4).  This leads to a confusion about notation, since the definition that $r(i, p ) = p(n_{i,p})$ isn't introduced until appendix D, but is used in appendix C.\n\nQUESTIONS:\n- Can anything be done to remove the assumption that the types are known up front?\n - Can any tight lower bounds be given for the stochastic and adversarial settings (i.e., in terms of both $m$ and T$)?\n\nMinor comments:\n - In Eq. (22) in the appendix, it has $f_t(p^*)$ instead of $r_t(p^*)$.\n\nLIMITATIONS:\nThe authors have made assumptions clear.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}

comments: SUMMARY:
This paper addresses the problem of exploration in pricing strategy. It proposes an algorithm and proves that it has lower regret compared to previous algorithms.

STRENGTHS:
The setting is interesting, and the results in this paper seem to show improvement over previous algorithms.

WEAKNESSES:
1. One of the contributions of this paper is a better discretization scheme. An introduction to the insights that allowed you to design this improved discretization method would make the contribution clearer.

2. The construction of the confidence interval has room for improvement. The UCB in this paper is |q_i-q_{i,t}| \leq |log T/T_{i,t}|. Such an interval is typewise. I believe that using a joint confidence interval (for example, \sum T_{i,t} |q_i-q_{i,t}| \leq log T or similar form )or a similar form) could improve the dependency of the regret on $m$.

3. It is unclear whether Line 6 of Algorithm 3 is computationally efficient.

QUESTIONS:
See the 'Weakness' part

LIMITATIONS:
See the 'Weakness' part
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-22