Logarithmic Regret in Feature-based Dynamic Pricing

( min { 1 λ2min log T , √ T } ) and O ( T 2/3 ) respectively , with λmin being the smallest eigenvalue of E [ xxT ] that could be arbitrarily close to 0 . We also prove an Ω ( √ T ) information-theoretic lower bound for a slightly more general setting , which demonstrates that “ knowing-the-demand-curve ” leads to an exponential improvement in feature-based dynamic pricing . 1 Introduction . The problem of pricing — to find a high-and-acceptable price — has been studied since Cournot [ 1897 ] . In order to locate the optimal price that maximizes the revenue , a firm may adjust their prices of products frequently , which inspires the dynamic pricing problem . Existing works [ Kleinberg and Leighton , 2003 , Broder and Rusmevichientong , 2012 , Chen and Farias , 2013 , Besbes and Zeevi , 2015 ] primarily focus on pricing a single product , which usually will not work well in another setting when thousands of new products are being listed every day with no prior experience in selling them . Therefore , we seek methods that approach an acceptable-and-profitable price with only observations on this single product and some historical selling records of other products . In this work , we consider a “ feature-based dynamic pricing ” problem , which was studied by Amin et al . [ 2014 ] , Cohen et al . [ 2020 ] , Javanmard and Nazerzadeh [ 2019 ] . In this problem setting , a sales session ( product , customer and other environmental variables ) is described by a feature vector , and the customer ’ s expected valuation is modeled as a linear function of this feature vector . Feature-based dynamic pricing . For t = 1 , 2 , ... , T : 1 . A feature vector xt ∈ Rd is revealed that describes a sales session ( product , customer and context ) . 2 . The customer valuates the product as wt = x⊤t θ∗ +Nt . 3 . The seller proposes a price vt > 0 concurrently ( according to xt and historical sales records ) . 4 . The transaction is successful if vt ≤ wt , i.e. , the seller gets a reward ( payment ) of rt = vt · 1 ( vt ≤ wt ) . Here T is unknown to the seller ( and thus can go to infinity ) , xt ’ s can be either stochastic ( e.g. , each sales session is drawn i.i.d . ) or adversarial ( e.g. , the sessions arrive in a strategic sequence ) , θ∗ ∈ Rd is a fixed parameter for all time periods , Nt is a zero-mean noise , and 1t = 1 ( vt ≤ wt ) is an indicator that equals 1 if vt ≤ wt and 0 otherwise . In this online-fashioned setting , we only see 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and sell one product at each time . Also , the feedback is Boolean Censored , which means we can only observe 1t instead of knowing wt directly . The best pricing policy for this problem is the one that maximizes the expected reward , and the regret of a pricing policy is accordingly defined as the difference of expected rewards between this selected policy and the best policy . Summary of Results . Our contributions are threefold . 1 . When xt ’ s are independently and identically distributed ( i.i.d . ) from an unknown distribution , we propose an “ Epoch-based Max-Likelihood Pricing ( EMLP ) ” algorithm that guarantees a regret bound at O ( d log T ) . The design of EMLP is similar to that of the RMLP algorithm in Javanmard and Nazerzadeh [ 2019 ] , but our new analysis improves their regret bound at O ( √ T ) when E [ xx⊤ ] is near singular . 2 . When xt ’ s are adversarial , we propose an “ Online-Newton-Step Pricing ( ONSP ) ” algorithm that achieves O ( d log T ) regret on constant-level noises for the first time , which exponentially improves the best existing result of O ( T 2/3 ) [ Cohen et al. , 2020 ] .1 3 . Our methods that achieve logarithmic regret require knowing the exact distribution of Nt in advance , as is also assumed in Javanmard and Nazerzadeh [ 2019 ] . We prove an Ω ( √ T ) lower bound on the regret if Nt ∼ N ( 0 , σ2 ) where σ is unknown , even with θ∗ given and xt fixed for all t. The O ( log T ) regret of EMLP and ONSP meets the information-theoretical lower bound [ Theorem 5 , Javanmard and Nazerzadeh , 2019 ] . In fact , the bound is optimal even when wt is revealed to the learner [ Mourtada , 2019 ] . From the perspective of characterizing the hardness of dynamic pricing problems , we generalize the classical results on “ The Value of Knowing a Demand Curve ” [ Kleinberg and Leighton , 2003 ] by further dividing the random-valuation class with an exponential separation of : ( 1 ) O ( log T ) regret for knowing the demand curve exactly ( even with adversarial features ) , and ( 2 ) Ω ( √ T ) regret for almost knowing the demand curves ( up to a one-parameter parametric family ) . 2 Related Works . In this section , we discuss our results relative to existing works on feature-based dynamic pricing , and highlight the connections and differences to the related settings of contextual bandits and contextual search ( for a broader discussion , see Appendix A ) . Feature-based Dynamic Pricing . There is a growing body of work on dynamic pricing with linear features [ Amin et al. , 2014 , Qiang and Bayati , 2016 , Cohen et al. , 2020 , Javanmard and Nazerzadeh , 2019 ] . Table 1 summarizes the differences in the settings and results2 . Among these work , our paper directly builds upon [ Cohen et al. , 2020 ] and [ Javanmard and Nazerzadeh , 2019 ] , as we share the same setting of online feature vectors , linear and noisy valuations and Boolean-censored feedback . Relative to the results in [ Javanmard and Nazerzadeh , 2019 ] , we obtain O ( d log T ) regret under weaker assumptions on the sequence of input features — in both distribution-free stochastic feature 1Previous works [ Cohen et al. , 2020 , Krishnamurthy et al. , 2021 ] did achieve polylog regrets , but only for negligible noise with σ = O ( 1 T log T ) . 2We only concern the dependence on T since there are various different assumptions on d. setting and the adversarial feature setting . It is to be noted that [ Javanmard and Nazerzadeh , 2019 ] also covers the sparse high-dimensional setting , and handles a slightly broader class of demand curves . Relative to [ Cohen et al. , 2020 ] , in which the adversarial feature-based dynamic pricing was first studied , our algorithm ONSP enjoys the optimal O ( d log T ) regret when the noise-level is a constant . In comparison , Cohen et al . [ 2020 ] reduces the problem to contextual bandits and applies the ( computationally inefficient ) “ EXP-4 ” algorithm [ Auer et al. , 2002 ] to achieve a Õ ( T 2/3 ) regret . The “ bisection ” style-algorithm in both Cohen et al . [ 2020 ] and Krishnamurthy et al . [ 2021 ] could achieve Õ ( poly ( d ) poly log ( T ) ) regrets but requires a small-variance subgaussian noise satisfying σ = O ( 1T log T ) . Lower Bounds . Most existing works focus on the lower regret bounds of non-feature-based models . Kleinberg and Leighton [ 2003 ] divides the problem setting as fixed , random , and adversarial valuations , and then proves each a Θ ( log log T ) , Θ ( √ T ) , and Θ ( T 2/3 ) regret , respectively . Broder and Rusmevichientong [ 2012 ] further proves a Θ ( √ T ) regret in general parametric valuation models . In this work , we generalize the methods of Broder and Rusmevichientong [ 2012 ] to our feature-based setting and further narrow it down to a linear-feature Gaussian-noisy model . As a complement to Kleinberg and Leighton [ 2003 ] , we further separate the exponential regret gap between : ( 1 ) O ( log T ) of the hardest ( adversarial feature ) totally-parametric model , and ( 2 ) Ω ( √ T ) of the simplest ( fixed known expectation ) unknown-σ Gaussian model . Contextual Bandits . For readers familiar with the online learning literature , our problem can be reduced to a contextual bandits problem [ Langford and Zhang , 2007 , Agarwal et al. , 2014 ] by discretizing the prices . But this reduction only results in O ( T 2/3 ) regret , as it does not capture the special structure of the feedback : an accepted price indicates the acceptance of all lower prices , and vise versa . Moreover , when comparing to linear bandits [ Chu et al. , 2011 ] , it is the valuation instead of the expected reward that we assume to be linear . Contextual Search . Feature-based dynamic pricing is also related to the contextual search problem [ Lobel et al. , 2018 , Leme and Schneider , 2018 , Liu et al. , 2021 , Krishnamurthy et al. , 2021 ] , which often involves learning from Boolean feedbacks , sometimes with a “ pricing loss ” and “ noisy ” feedback . These shared jargons make this problem appearing very similar to our problem . However , except for the noiseless cases [ Lobel et al. , 2018 , Leme and Schneider , 2018 ] , contextual search algorithms , even with “ pricing losses ” and “ Noisy Boolean feedback ” [ e.g. , Liu et al. , 2021 ] , do not imply meaningful regret bounds in our problem setup due to several subtle but important differences in the problem settings . Specifically , the noisy-boolean feedback model of [ Liu et al. , 2021 ] is about randomly toggling the “ purchase decision ” determined by the noiseless valuation x⊤θ∗ with probability 0.5 − ϵ . This is incompatible to our problem setting where the purchasing decision is determined by a noisy valuation x⊤θ∗ + Noise . Ultimately , in the setting of [ Liu et al. , 2021 ] , the optimal policy alway plays x⊤θ∗ , but our problem is harder in that we need to exploit the noise and the optimal price could be very different from x⊤θ∗ . 3 Krishnamurthy et al . [ 2021 ] also discussed this issue explicitly and considered the more natural noisy Boolean feedback model studied in this paper . Their result , similar to Cohen et al . [ 2020 ] , only achieves a logarithmic regret when the noise on the valuation is vanishing in an Õ ( 1/T ) rate . 3 Problem Setup . Symbols and Notations . Now we introduce the mathematical symbols and notations involved in the following pages . The game consists of T rounds . xt ∈ Rd , vt ∈ R+ and Nt ∈ R denote the feature vector , the proposed price and the noise respectively at round t = 1 , 2 , ... , T .4 We denote the product ut : = x ⊤ t θ ∗ as an expected valuation . At each round , we receive a payoff ( reward ) rt = vt · 1t , where the binary variable 1t indicates whether the price is accepted or not , i.e. , 1t = 1 ( vt ≤ wt ) . As we may estimate θ∗ in our algorithms , we denote θ̂t as an estimator of θ∗ , which we will formally define in the algorithms . Furthermore , we denote some functions that are related to noise distribution : F ( ω ) and f ( ω ) denote the cumulative distribution function ( CDF ) and probability density function ( PDF ) sequentially . We know that F ′ ( ω ) = f ( ω ) if we assume differentiability . To concisely denote all data observed up to round τ ( i.e. , feature , price and payoff of all past rounds ) , we define 3As an explicit example , suppose the valuation x⊤θ∗ = 0 , then the optimal price must be > 0 in order to avoid zero return . 4In an epoch-design situation , a subscript ( k , t ) indicates round t of epoch k. hist ( τ ) = { ( xt , vt,1t ) for t = 1 , 2 , ... , τ } . hist ( τ ) represents the transcript of all observed random variables before round ( τ + 1 ) . We define lt ( θ ) : = −1t · log ( 1− F ( vt − x⊤t θ ) ) − ( 1− 1t ) log ( F ( vt − x⊤t θ ) ) ( 1 ) as a negative log-likelihood function at round t. Also , we define an expected log-likelihood function Lt ( θ ) : Lt ( θ ) : = ENt [ lt ( θ ) |xt ] ( 2 ) Notice that we will later define an L̂k ( θ ) which is , however , not an expectation . Definitions of Key Quantities . We firstly define an expected reward function g ( v , u ) . g ( v , u ) : = E [ rt|vt = v , x⊤t θ∗ = u ] = v · P [ v ≤ x⊤t θ∗ +Nt ] = v · ( 1− F ( v − u ) ) . ( 3 ) This indicates that if the expected valuation is u and the proposed price is v , then the ( conditionally ) expected reward is g ( v , u ) . Now we formally define the regret of a policy ( algorithm ) A as is promised in Section 1 . Definition 1 ( Regret ) . Let A : Rd × ( Rd , R , { 0 , 1 } ) t−1 → R be a policy of pricing , i.e . A ( xt , hist ( t− 1 ) ) = vt . The regret of A is defined as follows . RegA = T∑ t=1 max v g ( v , x⊤t θ ∗ ) − g ( A ( xt , hist ( t− 1 ) ) , x⊤t θ∗ ) . ( 4 ) Here hist ( t− 1 ) is the historical records until ( t− 1 ) th round . Summary of Assumptions . We specify the problem settings by proposing three assumptions . Assumption 1 ( Known , bounded , strictly log-concave distribution ) . The noise Nt is independently and identically sampled from a distribution whose CDF is F . Assume that F ∈ C2 is strictly increasing and that F and ( 1− F ) are strictly log-concave . Also assume that f and f ′ are bounded , and denote Bf : = supω∈R f ( ω ) , Bf ′ : = supω∈R |f ′ ( ω ) | as two constants . Assumption 2 ( Bounded convex parameter space ) . The true parameter θ∗ ∈ H , where H ⊆ { θ : ||θ||2 ≤ B1 } is a bounded convex set and B1 is a constant . Assume H is known to us ( but θ∗ is not ) . Assumption 3 ( Bounded feature space ) . Assume xt ∈ D ⊆ { x : ||x||2 ≤ B2 } , ∀t = 1 , 2 , . . . , T . Also , 0 ≤ x⊤θ ≤ B , ∀x ∈ D , ∀θ ∈ H , where B = B1 ·B2 is a constant . Assumption 2 and 3 are mild as we can choose B1 and B2 large enough . In Section 4.1 , we may add further complement to Assumption 3 to form a stochastic setting . Assumption 1 is stronger since we might not know the exact CDF in practice , but it is still acceptable from an informationtheoretic perspective . There are at least three reasons that lead to this assumption : Primarily , this is necessary if we hope to achieve an O ( log ( T ) ) regret . We will prove in Section 5.3 that an Ω ( √ T ) is unavoidable if we can not know one parameter exactly . Moreover , the pioneering work of Javanmard and Nazerzadeh [ 2019 ] also assumes a known noise distribution with log-concave CDF , and many common distributions are actually strictly log-concave , such as Gaussian and logistic.5 Besides , although we did not present a method to precisely estimate σ in this work , it is a reasonable algorithm to replace with a plug-in estimator estimated using historical offline data . As we have shown , not knowing σ requires O ( √ T ) regret in general , but the lower bound does not rule out the plug-in approach achieving a smaller regret for interesting subclasses of problems in practice . Finally , we state a lemma and define an argmax function helpful for our algorithm design . Lemma 2 ( Uniqueness ) . For any u ≥ 0 , there exists a unique v∗ ≥ 0 such that g ( v∗ , u ) = maxv∈R g ( v , u ) . Thus , we can define a greedily pricing function that maximizes the expected reward : J ( u ) : = argmax v g ( v , u ) ( 5 ) Please see the proof of Lemma 2 in Appendix B.1 .

Summary: (Disclaimer: I have previously reviewed this paper for ICML 2021).  This paper develops algorithms for the problem of contextual dynamic pricing. This is an online learning problem that takes place over T rounds. There is a customer with a hidden value vector theta in R^d. Every round, the learner is given as context a feature vector x in R^d (both x and v are suitably bounded), representing the features of the current item. The customer is willing to pay up to v = <x, theta> + a Gaussian noise term for the item. The learner must set a price p for the item this round; if p > v the customer does not buy the item (and the learner receives 0 utility), otherwise the customer buys the item at price p and the learner receives p utility. The goal of the learner is to maximize their total learner (minimizing their regret compared to the optimal dynamic policy).  This paper presents two algorithms for two different settings. Their first algorithm (EMLP) applies when the feature vectors x_t are iid drawn from an unknown distribution. Their second algorithm (ONSP) applies when the feature vectors x_t are chosen adversarially. In both cases, they obtain O(d*log T) regret. Both algorithms require that the noise is Gaussian with a known variance sigma^2 (The authors show an O(sqrt(T)) lower bound if sigma is unknown). (In fact, this known variance sigma influences the constant in the O(d*log T) regret; see comments below).   Roughly, both algorithms work by maintaining an estimate of theta, and submitting the expected-utility-maximizing price p (assuming the theta estimate is accurate) per round. The two algorithms differ in how they maintain their estimate of theta. EMLP divides the rounds into log T geometrically-sized epochs, and uses a MLE to recompute the value of theta at the end of each epoch. ONSP instead runs an online Newton step method each round to obtain an estimate of theta.  The authors additionally present some empirical simulations of their algorithms (on synthetic data), showing that they outperform EXP4.