Context-Based Dynamic Pricing with Partially Linear Demand Model
Anonymous Author(s)
AffiliationAddress email
Abstract
In today’s data-rich environment, context-based dynamic pricing has gained muchattention.
To model the demand as a function of price and context, the existing literature either adopts a parametric model or a non-parametric model.
The former iseasier to implement but may suffer from model mis-specification, whereas the latteris more robust but does not leverage many structural properties of the underlyingproblem.
This paper combines these two approaches by studying the context-baseddynamic pricing with online learning, where the unknown expected demand admitsa semi-parametric partially linear structure.
Specifically, we consider two demandmodels, whose expected demand at price p and context x ∈ R d is given by bp
+ g ( x )and f ( p ) + a ⊤ x respectively.
We assume that g ( x ) is β -Hölder continuous in thefirst model, and f ( p ) is k th-order smooth with an additional parameter δ in the second model.
For both models, we design an efficient online learning algorithm withprovable regret upper bounds, and establish matching lower bounds.
This enablesus to characterize the statistical complexity for the two learning models, whoseoptimal regret rates are (cid:101) Θ( √ T
∨ T d d +2 β ) and (cid:101) Θ( √ T ∨ ( δT k +1 )
1 2 k +1 ) respectively.
The numerical results demonstrate that our learning algorithms are more effectivethan benchmark algorithms, and also reveal the effects of parameters d , β and δ onthe algorithm’s empirical regret, which are consistent with our theoretical findings.
Introduction
The recent success of online retailing has provided a data-rich environment for sellers to take betteradvantage of contextual information to dynamically improve their pricing strategies.
Examplesof such contextual information include products’ characteristics, seasonality, festival information,economic indicators etc.
A fundamental question stemming from the presence of contexts is how toselect a predictive model to capture the relationship among demand, price and context.
In the current literature, there are two modeling approaches.
The first approach is to adopt a parametricmodel, among which the linear model, i.e., demand =
b × price + a × context, is arguably the mostfundamental one.
The parametric assumption brings great convenience to practical implementation,and inspires many efficient learning algorithms.
However, it may also lead to a serious issue of modelmis-specification, as discussed in [23].
The second approach is to employ a fully non-parametricmodel, i.e., demand = d ( price, context ) for some unknown function d ( · ) without any parametricassumption.
The non-parametric model can be quite robust to different application contexts.
However,it does not leverage many structural properties of the underlying problem that may have been detectedfrom huge amounts of historical data, resulting in much complexity for learning the function d ( · ) .
Motivated by the idea of combining these two approaches, in this paper, we study context-baseddynamic pricing with online learning where the unknown expected demand admits a semi-parametricpartially linear structure.
Such a structure preserves the advantages of simple interpretability fromlinear models and great flexibility from non-parametric models.
Thus, it has also attracted significantattention in many fields, especially when there exist high-dimensional contexts, such as statistics [13],econometrics [33], and biomedicines [34].
Formally, consider a seller selling products over a finite selling horizon of T periods.
At the beginningof each period t , the seller observes an independently and identically distributed (i.i.d.)
contextvector x t ∈ [0 , 1] ∈ R d encoding the product’s characteristics and other exogenous information inperiod t .
The seller then chooses a price p t from the feasible price range [ p, p ] and customers observethe posted price p t .
The random demand D t is then generated according to the following function:
D t ( p t ) = d ( p t , x t ) + ε t , where d ( p t , x t ) is an unknown function and { ε t } t ≥ 1 are i.i.d.
sub-Gaussianrandom variables (r.v.’s) with zero mean and variance proxy σ 2 , i.e., E [ e λε t ] ≤ e λ 2 σ 2for any λ ∈ R .
We consider two models for d ( p t , x t ) :
Dynamic Pricing with Linear Pricing Effect (DPLPE) : d ( p t , x t ) =
bp t + g ( x t ) for unknownprice elasticity b ∈ R and β -Hölder continuous function g : [0 , 1] d → R with smoothness parameter0 < β ≤ 1 (see Assumption 1);
Dynamic Pricing with Linear Contextual Effect (DPLCE) : d ( p t , x t ) =
f ( p t )
+ a ⊤ x t for unknownparameter a ∈ R d and k th-order smooth function f : [ p, p ] → R with degree k ≥ 1 and parameterδ > 0 that captures how tightly f can be approximated by
a polynomial function (see Assumption 2).
The seller observes demand D t and collects revenue p t D t .
The expected revenue at price p andcontext x t is denoted by r ( p, x t ) =
pd ( p, x t ) for any p ∈
[ p, p ] .
An admissible pricing policy πis defined as a sequence of functions { π t ( · ) : t ≥ 1 } , where each π t ( · ) maps the historical information ( x 1 , p 1 , D 1 , . .
, x t − 1 , p t − 1 , D t − 1 , x t ) observed up to period t and possibly some externalrandomness to a feasible price in [ p, p ] .
A clairvoyant optimal policy is assumed to know functiond ( · ) and therefore always chooses the optimal price p ∗ t := arg max p ∈ [ p,p ] r ( p, x t ) in each periodt .
The performance of a pricing policy π is measured by regret R π ( T ) , defined as the differencebetween the T -period expected revenues generated by the clairvoyant optimal policy and by thepricing policy π .
That is, R π ( T ) =
(cid:80) Tt =1 E
[ r ( p ∗ t , x t ) − r
( p t , x t )] , where the expectation is takenover all the randomness associated with the environment and the algorithm.
1.1 Main results
Our main contributions lie in systematically characterizing the statistical complexity of the contextbased dynamic pricing with partially linear demand models DPLPE and DPLCE.
Throughout thepaper, we use notations O ( · ) , Ω( · ) and Θ( · ) to hide constant factors, and use (cid:101) O ( · ) , (cid:101) Ω( · ) and (cid:101)
Θ( · ) tohide both constant and logarithmic factors.
Statistical complexity for DPLPE.
For the first model DPLPE, we design an online learningalgorithm that integrates the ideas of random shock pricing in [23] and contextual space binning andapproximation.
We prove that our algorithm achieves the theoretical regret upper bound (cid:101) O ( √ T ∨
T d d +2 β ) .
We also establish the information-theoretic lower bound Ω( √ T ∨ T d d +2 β ) for any admissiblepolicy, which matches the regret upper bound up to logarithmic factors.
The key novelty in our lowerbound analysis is the construction of a family of high-dimensional β -Hölder continuous functionsthat are difficult to distinguish from each other.
To the best of our knowledge, this is also the firstmodel that generalizes the commonly used Lipschitz continuous assumption to Hölder continuity incontext-based dynamic pricing, with tight characterizations on the regret upper and lower bounds.
Statistical complexity for DPLCE.
For the second model DPLCE, we devise a different learningalgorithm based on the biased linear contextual bandit borrowed from [30] and our novel idea ofbeing more optimistic to chase the context-dependent optimal price.
We prove that our algorithmachieves the regret upper bound (cid:101) O ( √ T ∨ ( δT k +1 ) 1 2 k +1 ) , and also establish a matching lower boundΩ( √ T ∨ ( δT k +1 )
1 2 k +1 ) .
Since δ is a parameter that depends on the specific problem instance f ( · ) , we refer to the regret bounds we derive as the instance-dependent bounds.
To our knowledge, this isthe first tight instance-dependent regret bound in context-based dynamic pricing with online learning.
To achieve this, in the regret lower bound analysis, we need to construct a family of k th-order smoothfunctions that are adapted to parameters k and δ , which is also new to dynamic pricing literature.
1.2 Related work
Non-contextual dynamic pricing.
There is a vast literature on dynamic pricing with online demandlearning without context.
These works can be classified into two categories: parametric models (see,e.g., [7], [18]) and non-parametric models (see, e.g., [4], [5], [31],[30], [21]).
We refer the reader to[12] for a comprehensive review on this stream of literature.
Among these studies, [30] is the mostrelevant to our paper.
[30] study dynamic pricing without context, and assume the expected demandis k th-order smooth in price for some integer k .
They design a learning algorithm by applying the
Optimism-in-the-Face-of-Uncertainty (OFU) principle borrowed from bandits, and prove the optimalregret is (cid:101)
Θ( T k +1 2 k +1 ) .
The idea of algorithmic design in our DPLCE model is inspired by [30] but thereare two major differences.
First, the optimal price is stationary in [30], but can change over time inour setting.
Therefore, the idea of multi-armed bandit protocol proposed in their paper for learningthe stationary optimal price does not work.
To overcome this difficulty, we propose a novel ideaof being more optimistic over OFU in price selection to chase the context-dependent optimal price.
Second, our notion of k th-order smoothness is more general than theirs by allowing k not necessarilyto be an integer and δ not necessarily to be a constant.
The general values of k and δ require a morecareful selection of the parameters used by the algorithm, and a more sophisticated construction forthe hard instances in the lower bound analysis.
Context-based dynamic pricing.
There is a growing body of literature on context-based dynamicpricing (e.g., [25], [16], [20], [3], [22]).
We refer to [3] for a recent review and briefly discuss theworks related to this paper.
[25] consider the demand model a ⊤ x + bp in an incumbent-price setting,i.e., the expected demand for some given price is exactly known, and show that the greedy algorithmachieves the regret O (log T ) .
[23] study the same demand model as our DPLPE, and prove thatthe regret of a random price shock (RPS) algorithm compared with the clairvoyant who knows thebest linear approximation to g ( x ) is O ( √ T ) .
We borrow their RPS idea to design the learningalgorithm for our DPLPE model, but analyze the regret of our algorithm relative to the optimal policythat knows the true demand model.
[8] consider a non-parametric demand model in context-baseddynamic pricing.
For the Lipschitz continuous and locally concave revenue function, they prove thatthe optimal regret is (cid:101)
Θ( T d +2 d +4 ) .
This regret rate is strictly higher than (cid:101) Θ( √ T ∨ T dd +2 ) in our DPLPEmodel with β = 1 due to the separated effects of the price and context on demand we assume.
There is also a stream of works formulating demand using a binary choice model (see, e.g., [16], [19],[10], [28], [32], [14]).
Adopting the different aggregate-demand modeling approach, our paper isdifferent from these works in both the algorithm development and regret analysis.
Bandits with contextual information.
The most studied model in contextual bandit is the linearmodel (see, e.g., [2], [11], [27], [9], [1]), where the expected reward is a linear combination ofcontexts.
The algorithms developed in these works are mostly built upon the celebrated idea ofthe OFU principle.
In our DPLCE model, we borrow the OFU idea to design a learning algorithm.
There are also a substantial amount of literature considering non-parametric reward feedback under
Hölder continuous assumption (see, e.g., [26], [29], [24], [15]).
Among these studies, [29] assuminga continuous action space is the most relevant to this work.
For the general Lipschitz reward function,[29] proves that the optimal regret is (cid:101)
Θ( T d + dp +1 d + dp +2 ) , where d and d p are the dimensions for the contextspace and action space respectively.
Letting d p = 1 leads to the optimal regret rate (cid:101) Θ( T d +2 d +3 ) for adynamic pricing problem.
This rate is strictly higher than the optimal regret (cid:101) Θ( √ T ∨ T dd +2 ) in our
DPLPE model with β = 1 , and than (cid:101) Θ( T 23 ) in our DPLCE with k = 1 and δ = Θ(1) .
Table 1 provides a summary for the main results of this work and the mostly related literature.
[Table]
Dynamic Pricing with Linear Pricing Effect (DPLPE)
In this section [Table] we study the semi-parametric demand model with linear pricing effect [Table]
i
[Table]
e., d ( p t , x t ) =
bp t + g ( x t ) .
We assume that the unknown price elasticity b belongs to some known interval[ b, b ] ⊆
( −∞ , 0) [Table] and the optimal price p ∗ t = − g ( x t ) 2 b ∈ [ p, p ] for any b ∈ [ b, b ] and x t ∈ [0 , 1] d .
Wenow introduce our assumption on function g ( · ) .
Assumption 1 The function g : R d → R is β -Hölder continuous for some parameter 0 < β ≤ 1 ,denoted by g ∈ G ( β, d ) , meaning that there exists a constant L > 0 , such that for any x 1 , x 2 ∈ [0 , 1] d ,| g ( x 1 ) − g ( x 2 ) | ≤ L ∥ x 1 − x 2 ∥ β .
When β = 1 , G ( β, d ) is the class of Lipschitz continuous functions, which is usually required indynamic pricing literature (see, e.g., [8]).
We consider a more general class of Hölder continuousfunctions with arbitrary 0 < β ≤ 1 .
We first propose Algorithm 1 for our DPLPE model.
Our algorithm combines two main ideas: (i)context space binning and approximation; and (ii) pricing with random shock.
Context space binning and approximation.
To learn function g ( · ) , the β -Hölder continuousassumption guarantees that it cannot change dramatically in a local area.
This inspires a natural ideaof dividing the context space [0 , 1] d into different small bins and using a constant to approximate g ( · )in each bin.
Specifically, we divide each coordinate of [0 , 1] d into M equally sized segments and get
M d small bins.
We then estimate E
[ g ( x ) | x ∈ M j ] (and g ( x t ) for any x t ∈ M j ) using the sampleaverage approximation (SAA) from the residuals { d s − p s ˆ b t : 1 ≤ s ≤ t − 1 , x s ∈ M j } , where ˆ b t isan estimate for b to be discussed in the next paragraph.
It’s worth noting that there exists an importanttrade-off for the choice of M .
If M is large, the amount of data collected for each bin will be limited,discouraging the success of learning g ( · ) .
If M is small, we may incur a larger approximation error ofg ( · ) by using a constant conditional expectation in each bin, leading to a suboptimal pricing strategyand a greater revenue loss.
The specific choice of M is given in Theorem 1.
Pricing with random shock.
The idea of pricing with random shock is adopted from [23].
Ineach period t , the algorithm first computes a greedy price p gt by plugging in the estimates forg ( x t ) and b , and then charges a price p t by adding a random shock ∆ t , which takes the value ofδ t or − δ t with equal probability, to the greedy price p gt .
Following [23], instead of regressing d tagainst the true price p t , we estimate b by regressing d t against the price shock ∆ t .
Noting that
[Equation]
Since ∆ s is independent ofbp
gs + g ( x s ) + ε s for each 1 ≤ s ≤ t and has zero mean, (cid:80) ts =1 ∆ s d s (cid:80) ts =1 ∆ 2 s is an unbiased estimate for b .
This also explains the necessity of regressing d t against the random shock ∆ t .
The following theorem shows the regret upper bound of Algorithm 1, whose proof is in Appendix A.
Theorem 1 Let Algorithm 1 run with M = ⌈ T 1 d +2 β ⌉ .
Then the regret of Algorithm 1 is
[Equation]
The bound in Theorem 1 consists of two terms (cid:101) O ( √ T ) and (cid:101)
O ( T d d +2 β ) .
The first term (cid:101) O ( √ T ) arisesfrom the complexity of learning b .
For the simplest linear demand model without context, the squaredestimation error of b incurred by an asymptotically optimal policy is in the order of t − 1 / 2 (see[18]).
Therefore, a cumulative regret O ( √ T ) is in general unavoidable.
The second term (cid:101) O
( T d d +2 β )captures the complexity for learning the non-parametric function g ( · ) .
This bound is monotonicallyincreasing in dimension d , which agrees with the intuition that a higher dimension of the contextspace leads to a more challenging task of learning the function g ( · ) .
In addition, this bound decreasesin β .
This is consistent with the fact that if g ( · ) is smoother (i.e., β is larger), it can be more accuratelyapproximated by a constant in local bins.
We also note that only when d = 1 and β > 12 is (cid:101) O ( √ T )larger than (cid:101)
O
( T d d +2 β ) .
Otherwise, the regret upper bound is always (cid:101)
O
( T d d +2 β ) .
We then establish the regret lower bound.
We denote the regret of policy π as R πg,b, P , D ( T ) when thedemand function is bp + g ( x ) + ε , and the distributions of x and ε are P and D respectively.
We use
E ( σ ) to denote the class of sub-Gaussian distributions with variance proxy σ 2 .
Theorem 2 For any G ( β, d ) , 0 < b ≤ b and σ ≥ 0 , there exists a constant K 1 > 0 independent of
T , such that for any admissible policy π ,
[Equation]
Theorem 2 shows that the regret upper bound achieved by Algorithm 1 in Theorem 1 is unimprovablein terms of the dependency on the learning horizon T .
The first lower bound Ω( √ T ) can be directlyobtained from the existing literature, e.g., [17].
To prove the second lower bound Ω( T d d +2 β ) , weconstruct a series of Hölder continuous functions in [0 , 1] d that are difficult to distinguish from eachother.
We then apply the Bretagnolle–Huber inequality (see [6]) and KL divergence arguments tobound the regret of any algorithm from below.
See our construction and detailed proof in Appendix B.
We next compare our result with two relevant papers [23] and [8].
In [23], the same demand modelbp + g ( x ) is considered, but the performance of their learning algorithm is benchmarked with theoptimal policy for the “best” linear demand function.
In this case, they show that the optimal regretis Θ( √ T ) .
By contrast, our regret notion is defined against the optimal policy for the true demandfunction, and thus, our optimal regret has an extra term (cid:101) Θ( T d d +2 β ) capturing the complexity oflearning the function g ( · ) .
In [8], the authors prove the optimal regret (cid:101)
Θ( T d +2 d +4 ) for general Lipschitzcontinuous demand functions (in both context and price) assuming local concavity of the revenuefunction.
When β = 1 , the demand functions we consider here are a special case of theirs.
For thiscase, our optimal regret becomes (cid:101) Θ( √ T ∨
T dd +2 ) , which is strictly lower than theirs.
This reductionof the optimal regret benefits from the separated effects of the price and context assumed in ourmodel.
In terms of the algorithmic design, the algorithm in [8] divides both the price space andcontext space into bins, and treats the learning problem in each small bin as independent ones.
In oursetting, we leverage all the historical data to estimate the price sensitivity and achieve informationsharing among different bins in the context space.
Dynamic Pricing with Linear Contextual Effect (DPLCE)
In this section, we study the semi-parametric demand model with linear contextual effect, i.e.,d ( p t , x t ) =
f ( p t )
+ a ⊤ x t .
We assume || a || ≤ a for some constant a > 0 .
We now introduce ourassumption on function f ( · ) .
Assumption 2 The function f : [ p, p ] → R is k th-order smooth with degree k ≥ 1 and parameter δ ,denoted by f ∈ F k ([ p, p ]; δ ) , meaning that f is b ( k ) -times differentiable on [ p, p ] and(cid:12)(cid:12)(cid:12)
[Equation]
where b ( k ) :
= sup { i ∈ N : i < k } is the maximal integer that is strictly less than k .
Eq.
(1) requires that the b ( k ) th-order derivative of f ( · ) is ( k − b ( k ))
-Hölder continuous.
Thisassumption also implies that all the i th-order derivatives are bounded for i ≤ b ( k ) (i.e., C 0 :
=sup 0 ≤ i ≤ b ( k ) sup p ∈
[ p,p ] | f ( i ) ( p ) | < ∞ ).
When k is integer, i.e., b ( k ) =
k − 1 , Eq.
(1) becomes the
Lipschitz-continuous condition and Assumption 2 reduces to Assumption 1 in [30].
Different from[30] that sees δ as a constant, we allow the order of δ to depend on T , e.g., δ
=
O
( T − λ ) for someλ ≥ 0 .
As will be seen later, δ shows how tightly f ( p ) can be approximated by a polynomial function,and will have impacts on both the regret upper bound and lower bound.
We propose Algorithm 2 for DPLCE.
The key ideas are three-fold: (i) local polynomial approximation;(ii) biased linear contextual bandit and OFU; and (iii) optimism over OFU for price selection.
Local polynomial approximation.
The idea of local approximation for smooth non-parametricfunctions through polynomial functions is borrowed from [30].
Specifically, we partition the priceinterval [ p, p ] into N segments of equal size, denoted as I 1 , · · · , I N .
For each price segment I j :=[ a j , b j ) , we use a polynomial function of degree b ( k ) :
P I j ( p ) :
= (cid:80) b ( k )
i =
f ( i ) ( a j ) i !
( p − a j ) i , to locallyapproximate the true function f ( p ) .
To learn the the coefficients of P I j ( p ) , we apply the ridgeregression method.
By our assumption that f b ( k ) ( p ) is ( k − b ( k ))
-Hölder continuous, it can beverified from Taylor expansion that | f
( p ) − P
I
j
( p )
| ≤ | ( p − a j ) b ( k ) b ( k )!
| · | f ( b ( k ))
( a ′ j ) − f ( b ( k ))
( a j ) | ≤δ · ( p − p ) k b ( k )!
· N k =
O ( δN k ) , where a ′ j ∈ [ a j , p ] .
Therefore, δ reflects how tightly f can be approximatedby a local polynomial function.
Moreover, the number of price segments N needs to be carefullyselected to balance the accuracy of local polynomial approximation and the learning efficiency foreach price segment.
The specific choice of N is given in Theorem 3.
Biased linear contextual bandit and OFU.
To suggest a good price for each segment, we borrowthe biased linear contextual bandit from [30]. Specifically, for each t ≥ 1 and p t ∈ I j , we canrewrite D t ( p t ) as follows: D t ( p t ) = P I j ( p t ) + a ⊤ x t + β t = θ ⊤ j φ ( p t ) + a ⊤ x t + β t , where 1 Input: time horizon T , price range [ p, p ] , degree k , parameter C 0 , context dimension d , upper bound ¯ a , number of price segments N , error control terms ∆ , noise variance proxy σ 2 Initialization: 3 Partition p, p ] into N segments of equal length, denoted as I i for i = 1 , 2 , · · · , N ; 4 Initialize for all i ∈
[ N ] : D i
∅ .
5 Main Steps: 6 for t = 1 , 2 , · · · , T do
Observe x t ; for i = 1 , 2 , · · · , N do (ˆ θ t,i , ˆ a t,i , γ t,i , V t,i ) =
RLC ( b ( k ) , d, I i , D i , ∆ , 1 /T 2 , C 0 , σ, ¯ a ) ;
[Equation]
[Equation]
[Equation]
; end for
Select i t = arg max i ≤ N − 1 ˆ r t,i and charge p t = ˆ p t,i t ;
Observe realized demand d t ;
Update D i t ← D i t ∪ { ( x t , p t , d t ) } ; 16 end for
1 Input: polynomial
, probability , smoothness parameter C , noise variance proxy σ , upper bound ¯ a 2 Compute λ
= ( u − l ) 2 k − 1 ( u − l ) 2 − 1 + d ; 3 Compute γ = σ (cid:114) ( d + k ) ln (cid:16)
d + k + |D| d + k (cid:17) − 2 ln ϵ + λ 12 ( C 2 k + a 2 ) 12 + ∆ (cid:112) |D| ; 4 Compute V = λI ( k + d ) × ( k + d ) + (cid:80) ( x,p,d ) ∈D ϕ
( p, x ) ϕ ( p, x ) ⊤ , where ϕ ( p, x ) =
( φ ( p ) ⊤ , x ⊤ ) ⊤ and φ ( p ) =
(1 , ( p − l ) , · · · , ( p − l ) k − 1 ) ⊤ ; 5 Compute Ridge estimate (ˆ θ, ˆ a ) =
arg min (cid:80) ( x,p,d ) ∈D ( d − ⟨ θ, φ ( p ) ⟩ − ⟨ a, x ⟩ ) 2 + λ ( ∥ θ ∥ 22 + ∥ a ∥ 22 ) ; 6 Output: ˆ θ, ˆ a, γ, V β t := f ( p t ) − P I j ( p t ) + ε t , φ ( p t ) :
= (1 , ( p t − a j ) , · · · , ( p t − a j ) b ( k ) ) and θ j ∈ R b ( k
)+1 whosei th element is f ( i − 1) ( a j ) ( i − 1)!
for 1 ≤ i ≤ b ( k ) + 1 .
This model is similar to the linear contextual banditin the literature (see, e.g., [1], [9]), where the OFU idea that selects actions in an optimistic wayis commonly adopted.
The key difference is that β t contains a biased term that is not mean-zeroand depends on the pricing decision.
[30] address this issue by adding an additional term ∆ whenimplementing the OFU principle to compute an optimistic price and optimistic revenue.
We adaptthis idea to incorporate the effect of contexts in our setting.
Optimism over OFU for price selection.
In the absence of context, the optimal price for the dynamicpricing problem considered by [30] is fixed .
To learn this fixed optimal price, [30] propose the ideaof embedding the biased linear contextual bandit into a multi-armed bandit protocol that treats eachprice segment as an arm.
The segment to which the fixed optimal price belongs is regraded as the“best” arm.
By contrast, in our context-based setting, the optimal price depends on the random contextrevealed in each period and changes over time.
Thus, our “best” arm in each period also changes,and the idea of multi-armed bandit protocol in [30] does not work.
To overcome this challenge, wepropose a new idea of optimism over OFU, which proceeds as follows.
We first implement the OFUprinciple to each price segment I i and compute an optimistic price ˆ p t,i and optimistic revenue ˆ r t,i .
Then we choose the most optimistic price from the N candidate prices ˆ p t, 1 , . . .
, ˆ p t,N that achievesthe highest optimistic revenue.
The following theorem presents the regret upper bound for Algorithm 2, whose proof is in Appendix C.
Theorem 3 Let Algorithm 2 run with N = ⌈ δ 2 2 k +1 T 1
k +
⌉ + 1 and ∆ =
δ ( p − p ) k /N k .
Then theregret of Algorithm 2 is
[Equation]
First, when δ is a constant and k is an integer, the regret upper bound becomes (cid:101) O ( dT k +1 2 k +1 ) , recovering
Theorem 1 in [30] for the setting without contexts (i.e., d = 1 and x is a constant).
As k becomeslarger while keeping δ a constant, the upper bound decreases according to (cid:101) O ( dT k +1 2 k +1 ) .
In the extremecase of k = ∞ , the upper bound becomes (cid:101) O
( d √ T ) , which coincides with the optimal regret rate forthe linear model d ( p, x ) =
bp + a ⊤ x
(see, e.g., [3]) 1 .
Second, when δ is not a constant whose ordercan shrink as T increases, Theorem 3 shows an instance-dependent bound which further differentiatesour result from that in [30].
We can see that when δ = Ω( T − 12 ) , the regret bound is (cid:101) O
( δ 1 2 k +1 T k +1 2 k +1 ) ,and when δ decreases to O ( T − 12 ) , the regret bound is always (cid:101) O ( √ T ) .
This shows that the magnitudeof δ greatly affects the regret upper bound, and the smaller δ is, the lower the regret bound will be.
This is because when δ becomes smaller, the approximation error using a local polynomial function
O ( δN k ) becomes smaller, leading to more accurate estimates for f ( · ) .
Since the polynomial functionis differentiable infinitely many times, its regret is (cid:101) O ( √ T ) from Section 3.4.1 in [30].
Therefore, asδ decreases, the regret upper bound decays to (cid:101) O ( √ T ) , which is attained when δ
= O ( T − 12 ) .
We then establish the regret lower bound.
Similar to Theorem 1, we denote the regret of policy π by
R πf,a, P , D ( T, k, δ ) when the demand function is f ( p ) + a ⊤ x + ε , where f ∈ F k ([ p, p ; δ ]) , and thedistributions of x and ε are P and D respectively.
Theorem 4 For any F k ([ p, p ; δ ) , ¯ a ≥ 0 , and σ ≥ 0 , there exists a constant K 2 > 0 independent of
T and δ , such that for any admissible policy π ,sup
[Equation]
Theorem 4 shows that the regret upper bound achieved by Algorithm 2 in Theorem 3 is unimprovablein terms of its dependency on the learning horizon T and parameter δ .
Note that the upper bound in
Theorem 3 grows linearly in d , but the lower bound in Theorem 4 is independent of d .
This is becausein the lower bound analysis, we simply consider a = 0 .
We leave the problem of analyzing the morecomplicated dimension-dependent lower bound as future research.
The first lower bound Ω( √ T ) in
Eq.
(2) directly follows from the existing literature (e.g., [17]).
The proof of the second lower boundΩ(( δT k +1 ) 1 2 k +1 ) relies on constructing a series of k th-order smooth functions with parameter δ andapplying Pinsker’s inequality to bound the total variation of two probability measures through their
KL divergence.
See our construction and detailed proof in Appendix D.
Numerical Study
In this section, we conduct a numerical study to test the empirical performances of our algorithms.
We measure the performance of a learning algorithm π by the relative regret defined as follows:
[Equation]
=
[Equation]
[Equation]
× 100% .
For both models, we compare our algorithms with the lineargreedy algorithm that estimates the demand function by a linear function and myopically selects theoptimal price that maximizes the proxy revenue in each period t .
For our DPLPE model, we alsocompare our algorithm with the random price shock (RPS) algorithm proposed by [23].
For eachinstance, we repeat our experiments for 50 independent runs, and approximate the relative regret bythe empirical relative regret averaged over 50 runs.
Numerical results for DPLPE model.
For our DPLPE model with demand function d ( p, x t ) =
bp + g ( x t ) , we set b = − 5 , ε ∼ N (0 , 5) , [ b, b ] =
[ − 8 , − 3] , [ p, p ] =
[1 , 19] , x t ∼ Uniform ([0 , 1] d ) ,and consider the following form of function g ( x ) :
[Equation]
otherwise , where D ( x, ∂ [0 , 1] d ) denotes the Euclidean distance between x and the boundary of [0 , 1] d .
In
Figure 1(a), we compare the empirical relative regret of our ADPLP with those of RPS algorithm
andthe linear greedy algorithm.
The figure shows that our algorithm performs much better than bothalgorithms.
In Figure 1(b), we investigate the effect of d on the empirical relative regret of ADPLP byvarying d in the set { 1 , 3 , 10 } while keeping β = 1 .
As shown in the figure, as d decreases, the regretdecreases, which is consistent with our theoretical results.
In Figure 1(c), we test the effect of β onthe empirical relative regret of ADPLP by varying β in the set { 0 . 2 , 0 . 4 , 0 . 6 , 1 } while keeping d = 1 .
When β is relatively small, a slight increment of β , e.g., β = 0 .
2 to β = 0 . 4 , will lead to significantspeed-up of the convergence.
The results of β = 0 .
6 and β = 1 are very close, which indicates thatwhen β is relatively large, increasing β may only bring limited improvement.
[Figure]
[Figure]
Numerical results for DPLCE model.
For our DPLCE model, we consider the following demand:
[Equation] +
[Equation]
d · x t + ε t , where 1 d denotes (1 , 1 , · · · , 1) ∈ R d , x t is uniformlydistributed on [0 , 1] d independently and ε t ∼ N (0 , 0 .
It can be verified that k = 2 . 5 in thiscase.
In Figure 2(a), we fix d = 2 , δ = 3 .
75 and [ p, p ] =
[2 . 6 , 3 . 8] , and compare the empirical relativeregret of our ADPLC with that of the linear greedy algorithm.
The linear greedy algorithm has almosta constant empirical relative regret, showing that it fails to identify the optimal solution.
By contrast,the empirical relative regret of ADPLC decreases as t increases.
We also test the effect of δ on theempirical relative regret of ADPLC by varying β ∈ { 3 . 75 , 2 . 5 , 1 } .
As shown in Figure 2(b), as δbecomes smaller, the empirical relative regret decreases, which is consistent with our theoretic results.
[Figure]
l
[Figure]
Conclusion
In this paper, we study the context-based dynamic pricing with online learning, where the unknownexpected demand admits a partially linear semi-parametric structure.
For both models DPLPEand DPLCE, we develop an efficient learning algorithm and characterize the optimal regret byproving matching upper and lower bounds.
We note that our algorithms need to have the preliminaryknowledge about the smoothness parameters β in DPLPE model and k and δ in DPLCE model.
One important direction for future research is to explore whether parameter-free algorithms can bedeveloped and still achieve the optimal regret rate.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]Please do not modify the questions and only use the
provided macros for your answers
Note that theChecklist section does not count towards the page limit. In your
paper,
please delete
this
instructions block and only keep the Checklist section heading above along with the questions/answers below.1.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] See the last paragraph of Section3 and Section 5.(c)
Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sections 2and 3.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Appendix.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Section4.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A](d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]4.
If you are using existing assets (e.g.,
code, data, models) or curating releasing new assets.
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? N/A] b) Did
you describe any potential participant risks with links to Institutional Review
Board
(IRB) approvals, if applicable
N/A]
(c) Did you include the estimated
hourly wage paid
to
participants
and the
total
amount
spent
on participant compensation?
[N/A]