Logarithmic Regret in Feature-based Dynamic Pricing

( min { 1 λ2min log T , √ T } ) and O ( T 2/3 ) respectively , with λmin being the smallest eigenvalue of E [ xxT ] that could be arbitrarily close to 0 . We also prove an Ω ( √ T ) information-theoretic lower bound for a slightly more general setting , which demonstrates that “ knowing-the-demand-curve ” leads to an exponential improvement in feature-based dynamic pricing . 1 Introduction . The problem of pricing — to find a high-and-acceptable price — has been studied since Cournot [ 1897 ] . In order to locate the optimal price that maximizes the revenue , a firm may adjust their prices of products frequently , which inspires the dynamic pricing problem . Existing works [ Kleinberg and Leighton , 2003 , Broder and Rusmevichientong , 2012 , Chen and Farias , 2013 , Besbes and Zeevi , 2015 ] primarily focus on pricing a single product , which usually will not work well in another setting when thousands of new products are being listed every day with no prior experience in selling them . Therefore , we seek methods that approach an acceptable-and-profitable price with only observations on this single product and some historical selling records of other products . In this work , we consider a “ feature-based dynamic pricing ” problem , which was studied by Amin et al . [ 2014 ] , Cohen et al . [ 2020 ] , Javanmard and Nazerzadeh [ 2019 ] . In this problem setting , a sales session ( product , customer and other environmental variables ) is described by a feature vector , and the customer ’ s expected valuation is modeled as a linear function of this feature vector . Feature-based dynamic pricing . For t = 1 , 2 , ... , T : 1 . A feature vector xt ∈ Rd is revealed that describes a sales session ( product , customer and context ) . 2 . The customer valuates the product as wt = x⊤t θ∗ +Nt . 3 . The seller proposes a price vt > 0 concurrently ( according to xt and historical sales records ) . 4 . The transaction is successful if vt ≤ wt , i.e. , the seller gets a reward ( payment ) of rt = vt · 1 ( vt ≤ wt ) . Here T is unknown to the seller ( and thus can go to infinity ) , xt ’ s can be either stochastic ( e.g. , each sales session is drawn i.i.d . ) or adversarial ( e.g. , the sessions arrive in a strategic sequence ) , θ∗ ∈ Rd is a fixed parameter for all time periods , Nt is a zero-mean noise , and 1t = 1 ( vt ≤ wt ) is an indicator that equals 1 if vt ≤ wt and 0 otherwise . In this online-fashioned setting , we only see 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and sell one product at each time . Also , the feedback is Boolean Censored , which means we can only observe 1t instead of knowing wt directly . The best pricing policy for this problem is the one that maximizes the expected reward , and the regret of a pricing policy is accordingly defined as the difference of expected rewards between this selected policy and the best policy . Summary of Results . Our contributions are threefold . 1 . When xt ’ s are independently and identically distributed ( i.i.d . ) from an unknown distribution , we propose an “ Epoch-based Max-Likelihood Pricing ( EMLP ) ” algorithm that guarantees a regret bound at O ( d log T ) . The design of EMLP is similar to that of the RMLP algorithm in Javanmard and Nazerzadeh [ 2019 ] , but our new analysis improves their regret bound at O ( √ T ) when E [ xx⊤ ] is near singular . 2 . When xt ’ s are adversarial , we propose an “ Online-Newton-Step Pricing ( ONSP ) ” algorithm that achieves O ( d log T ) regret on constant-level noises for the first time , which exponentially improves the best existing result of O ( T 2/3 ) [ Cohen et al. , 2020 ] .1 3 . Our methods that achieve logarithmic regret require knowing the exact distribution of Nt in advance , as is also assumed in Javanmard and Nazerzadeh [ 2019 ] . We prove an Ω ( √ T ) lower bound on the regret if Nt ∼ N ( 0 , σ2 ) where σ is unknown , even with θ∗ given and xt fixed for all t. The O ( log T ) regret of EMLP and ONSP meets the information-theoretical lower bound [ Theorem 5 , Javanmard and Nazerzadeh , 2019 ] . In fact , the bound is optimal even when wt is revealed to the learner [ Mourtada , 2019 ] . From the perspective of characterizing the hardness of dynamic pricing problems , we generalize the classical results on “ The Value of Knowing a Demand Curve ” [ Kleinberg and Leighton , 2003 ] by further dividing the random-valuation class with an exponential separation of : ( 1 ) O ( log T ) regret for knowing the demand curve exactly ( even with adversarial features ) , and ( 2 ) Ω ( √ T ) regret for almost knowing the demand curves ( up to a one-parameter parametric family ) . 2 Related Works . In this section , we discuss our results relative to existing works on feature-based dynamic pricing , and highlight the connections and differences to the related settings of contextual bandits and contextual search ( for a broader discussion , see Appendix A ) . Feature-based Dynamic Pricing . There is a growing body of work on dynamic pricing with linear features [ Amin et al. , 2014 , Qiang and Bayati , 2016 , Cohen et al. , 2020 , Javanmard and Nazerzadeh , 2019 ] . Table 1 summarizes the differences in the settings and results2 . Among these work , our paper directly builds upon [ Cohen et al. , 2020 ] and [ Javanmard and Nazerzadeh , 2019 ] , as we share the same setting of online feature vectors , linear and noisy valuations and Boolean-censored feedback . Relative to the results in [ Javanmard and Nazerzadeh , 2019 ] , we obtain O ( d log T ) regret under weaker assumptions on the sequence of input features — in both distribution-free stochastic feature 1Previous works [ Cohen et al. , 2020 , Krishnamurthy et al. , 2021 ] did achieve polylog regrets , but only for negligible noise with σ = O ( 1 T log T ) . 2We only concern the dependence on T since there are various different assumptions on d. setting and the adversarial feature setting . It is to be noted that [ Javanmard and Nazerzadeh , 2019 ] also covers the sparse high-dimensional setting , and handles a slightly broader class of demand curves . Relative to [ Cohen et al. , 2020 ] , in which the adversarial feature-based dynamic pricing was first studied , our algorithm ONSP enjoys the optimal O ( d log T ) regret when the noise-level is a constant . In comparison , Cohen et al . [ 2020 ] reduces the problem to contextual bandits and applies the ( computationally inefficient ) “ EXP-4 ” algorithm [ Auer et al. , 2002 ] to achieve a Õ ( T 2/3 ) regret . The “ bisection ” style-algorithm in both Cohen et al . [ 2020 ] and Krishnamurthy et al . [ 2021 ] could achieve Õ ( poly ( d ) poly log ( T ) ) regrets but requires a small-variance subgaussian noise satisfying σ = O ( 1T log T ) . Lower Bounds . Most existing works focus on the lower regret bounds of non-feature-based models . Kleinberg and Leighton [ 2003 ] divides the problem setting as fixed , random , and adversarial valuations , and then proves each a Θ ( log log T ) , Θ ( √ T ) , and Θ ( T 2/3 ) regret , respectively . Broder and Rusmevichientong [ 2012 ] further proves a Θ ( √ T ) regret in general parametric valuation models . In this work , we generalize the methods of Broder and Rusmevichientong [ 2012 ] to our feature-based setting and further narrow it down to a linear-feature Gaussian-noisy model . As a complement to Kleinberg and Leighton [ 2003 ] , we further separate the exponential regret gap between : ( 1 ) O ( log T ) of the hardest ( adversarial feature ) totally-parametric model , and ( 2 ) Ω ( √ T ) of the simplest ( fixed known expectation ) unknown-σ Gaussian model . Contextual Bandits . For readers familiar with the online learning literature , our problem can be reduced to a contextual bandits problem [ Langford and Zhang , 2007 , Agarwal et al. , 2014 ] by discretizing the prices . But this reduction only results in O ( T 2/3 ) regret , as it does not capture the special structure of the feedback : an accepted price indicates the acceptance of all lower prices , and vise versa . Moreover , when comparing to linear bandits [ Chu et al. , 2011 ] , it is the valuation instead of the expected reward that we assume to be linear . Contextual Search . Feature-based dynamic pricing is also related to the contextual search problem [ Lobel et al. , 2018 , Leme and Schneider , 2018 , Liu et al. , 2021 , Krishnamurthy et al. , 2021 ] , which often involves learning from Boolean feedbacks , sometimes with a “ pricing loss ” and “ noisy ” feedback . These shared jargons make this problem appearing very similar to our problem . However , except for the noiseless cases [ Lobel et al. , 2018 , Leme and Schneider , 2018 ] , contextual search algorithms , even with “ pricing losses ” and “ Noisy Boolean feedback ” [ e.g. , Liu et al. , 2021 ] , do not imply meaningful regret bounds in our problem setup due to several subtle but important differences in the problem settings . Specifically , the noisy-boolean feedback model of [ Liu et al. , 2021 ] is about randomly toggling the “ purchase decision ” determined by the noiseless valuation x⊤θ∗ with probability 0.5 − ϵ . This is incompatible to our problem setting where the purchasing decision is determined by a noisy valuation x⊤θ∗ + Noise . Ultimately , in the setting of [ Liu et al. , 2021 ] , the optimal policy alway plays x⊤θ∗ , but our problem is harder in that we need to exploit the noise and the optimal price could be very different from x⊤θ∗ . 3 Krishnamurthy et al . [ 2021 ] also discussed this issue explicitly and considered the more natural noisy Boolean feedback model studied in this paper . Their result , similar to Cohen et al . [ 2020 ] , only achieves a logarithmic regret when the noise on the valuation is vanishing in an Õ ( 1/T ) rate . 3 Problem Setup . Symbols and Notations . Now we introduce the mathematical symbols and notations involved in the following pages . The game consists of T rounds . xt ∈ Rd , vt ∈ R+ and Nt ∈ R denote the feature vector , the proposed price and the noise respectively at round t = 1 , 2 , ... , T .4 We denote the product ut : = x ⊤ t θ ∗ as an expected valuation . At each round , we receive a payoff ( reward ) rt = vt · 1t , where the binary variable 1t indicates whether the price is accepted or not , i.e. , 1t = 1 ( vt ≤ wt ) . As we may estimate θ∗ in our algorithms , we denote θ̂t as an estimator of θ∗ , which we will formally define in the algorithms . Furthermore , we denote some functions that are related to noise distribution : F ( ω ) and f ( ω ) denote the cumulative distribution function ( CDF ) and probability density function ( PDF ) sequentially . We know that F ′ ( ω ) = f ( ω ) if we assume differentiability . To concisely denote all data observed up to round τ ( i.e. , feature , price and payoff of all past rounds ) , we define 3As an explicit example , suppose the valuation x⊤θ∗ = 0 , then the optimal price must be > 0 in order to avoid zero return . 4In an epoch-design situation , a subscript ( k , t ) indicates round t of epoch k. hist ( τ ) = { ( xt , vt,1t ) for t = 1 , 2 , ... , τ } . hist ( τ ) represents the transcript of all observed random variables before round ( τ + 1 ) . We define lt ( θ ) : = −1t · log ( 1− F ( vt − x⊤t θ ) ) − ( 1− 1t ) log ( F ( vt − x⊤t θ ) ) ( 1 ) as a negative log-likelihood function at round t. Also , we define an expected log-likelihood function Lt ( θ ) : Lt ( θ ) : = ENt [ lt ( θ ) |xt ] ( 2 ) Notice that we will later define an L̂k ( θ ) which is , however , not an expectation . Definitions of Key Quantities . We firstly define an expected reward function g ( v , u ) . g ( v , u ) : = E [ rt|vt = v , x⊤t θ∗ = u ] = v · P [ v ≤ x⊤t θ∗ +Nt ] = v · ( 1− F ( v − u ) ) . ( 3 ) This indicates that if the expected valuation is u and the proposed price is v , then the ( conditionally ) expected reward is g ( v , u ) . Now we formally define the regret of a policy ( algorithm ) A as is promised in Section 1 . Definition 1 ( Regret ) . Let A : Rd × ( Rd , R , { 0 , 1 } ) t−1 → R be a policy of pricing , i.e . A ( xt , hist ( t− 1 ) ) = vt . The regret of A is defined as follows . RegA = T∑ t=1 max v g ( v , x⊤t θ ∗ ) − g ( A ( xt , hist ( t− 1 ) ) , x⊤t θ∗ ) . ( 4 ) Here hist ( t− 1 ) is the historical records until ( t− 1 ) th round . Summary of Assumptions . We specify the problem settings by proposing three assumptions . Assumption 1 ( Known , bounded , strictly log-concave distribution ) . The noise Nt is independently and identically sampled from a distribution whose CDF is F . Assume that F ∈ C2 is strictly increasing and that F and ( 1− F ) are strictly log-concave . Also assume that f and f ′ are bounded , and denote Bf : = supω∈R f ( ω ) , Bf ′ : = supω∈R |f ′ ( ω ) | as two constants . Assumption 2 ( Bounded convex parameter space ) . The true parameter θ∗ ∈ H , where H ⊆ { θ : ||θ||2 ≤ B1 } is a bounded convex set and B1 is a constant . Assume H is known to us ( but θ∗ is not ) . Assumption 3 ( Bounded feature space ) . Assume xt ∈ D ⊆ { x : ||x||2 ≤ B2 } , ∀t = 1 , 2 , . . . , T . Also , 0 ≤ x⊤θ ≤ B , ∀x ∈ D , ∀θ ∈ H , where B = B1 ·B2 is a constant . Assumption 2 and 3 are mild as we can choose B1 and B2 large enough . In Section 4.1 , we may add further complement to Assumption 3 to form a stochastic setting . Assumption 1 is stronger since we might not know the exact CDF in practice , but it is still acceptable from an informationtheoretic perspective . There are at least three reasons that lead to this assumption : Primarily , this is necessary if we hope to achieve an O ( log ( T ) ) regret . We will prove in Section 5.3 that an Ω ( √ T ) is unavoidable if we can not know one parameter exactly . Moreover , the pioneering work of Javanmard and Nazerzadeh [ 2019 ] also assumes a known noise distribution with log-concave CDF , and many common distributions are actually strictly log-concave , such as Gaussian and logistic.5 Besides , although we did not present a method to precisely estimate σ in this work , it is a reasonable algorithm to replace with a plug-in estimator estimated using historical offline data . As we have shown , not knowing σ requires O ( √ T ) regret in general , but the lower bound does not rule out the plug-in approach achieving a smaller regret for interesting subclasses of problems in practice . Finally , we state a lemma and define an argmax function helpful for our algorithm design . Lemma 2 ( Uniqueness ) . For any u ≥ 0 , there exists a unique v∗ ≥ 0 such that g ( v∗ , u ) = maxv∈R g ( v , u ) . Thus , we can define a greedily pricing function that maximizes the expected reward : J ( u ) : = argmax v g ( v , u ) ( 5 ) Please see the proof of Lemma 2 in Appendix B.1 .

Summary: This paper studies the problem of feature-based dynamic pricing and offers strong new regret bounds. The headline result is the construction of Online Newton Step Pricing and the proof that it obtains regret $O(d \log T)$ even with adversarial features. This had been proven before only by assuming either (1) small noise or (2) iid features plus a minimum-eigenvalue assumption.