Original: Contextual Dynamic Pricing with Unknown Noise: Explore-then-UCB Strategy and Improved Regrets
Revised: Contextual Dynamic Pricing with Unknown Noise: Explore-then-UCB Strategy and Improved Regrets

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Dynamic pricing is a fast-moving research area in machine learning and operationsmanagement.
Revised: Dynamic pricing is a fast-moving research area in machine learning and operationsmanagement.

Original: A lot of work has been done for this problem with known noise.
Revised: A lot of work has been done for this problem with known noise.

Original: In this paper, we consider a contextual dynamic pricing problem under a linearcustomer valuation model with an unknown market noise distribution F .
Revised: In this paper, we consider a contextual dynamic pricing problem under a linearcustomer valuation model with an unknown market noise distribution F .

Original: Thisproblem is very challenging due to the difﬁculty in balancing three tangled tasks ofrevenue-maximization, estimating the linear valuation parameter θ 0 , and learningthe nonparametric F .
Revised: Thisproblem is very challenging due to the difﬁculty in balancing three tangled tasks ofrevenue-maximization, estimating the linear valuation parameter θ 0 , and learningthe nonparametric F .

Original: To address this issue, we develop a novel Explore-then-UCB(ExUCB) strategy that includes an exploration for θ 0 -learning and a followed UCB procedure of joint revenue-maximization and F -learning.
Revised: To address this issue, we develop a novel Explore-then-UCB(ExUCB) strategy that includes an exploration for θ 0 -learning and a followed UCBprocedure of joint revenue-maximization and F -learning.

Original: Under Lipschitzand concavity assumptions on F , ExUCB is the ﬁrst approach to achieve the˜ O ( T 2 / 3 ) regret rate.
Revised: Under Lipschitz and2nd-order smoothness assumptions on F , ExUCB is the ﬁrst approach to achievethe ˜ O ( T 2 / 3 ) regret rate.

Original: Under the Lipschitz assumption only, ExUCB can match thebest existing regret of ˜ O
Revised: Under the Lipschitz assumption only, ExUCB matches thebest existing regret of ˜ O

Original: ( T 3 / 4 ).
Revised: ( T 3 / 4 )

Revised: and is computationally more efﬁcient.

Original: Furthermore, for regret lower bounds under thenonparametric F , not much work has been done beyond only assuming Lipschitz.
Revised: Furthermore,for regret lower bounds under the nonparametric F , not much work has been donebeyond only assuming Lipschitz.

Original: To ﬁll this gap, we provide the ﬁrst ˜Ω( T 3 / 5 ) lower bound under Lipschitz andconcavity assumptions.
Revised: To ﬁll this gap, we provide the ﬁrst ˜Ω( T 3 / 5 )lower bound under Lipschitz and 2nd-order smoothness assumptions.

Original:  Introduction
Revised:  Introduction

Original: Dynamic pricing is a process of continuously adjusting the prices by learning from the customers’feedback.
Revised: Dynamic pricing is a process of continuously adjusting the prices by learning from the customers’feedback.

Original: The feedback usually depends on the pricing action.
Revised: The feedback usually depends on the pricing action.

Original: To maximize the overall revenuesin a sales horizon, a pricing policy should well balance between learning the customers’ demands(exploration) and setting revenue-maximizing prices based on the current knowledge (exploitation).
Revised: To maximize the overall revenuesin a sales horizon, a pricing policy should well balance between learning the customers’ demands(exploration) and setting revenue-maximizing prices based on the current knowledge (exploitation).

Original: There is a rich literature on this information-regret tradeoff under different settings [9, 30, 18, 14, 22].
Revised: There is a rich literature on this information-regret tradeoff under different settings [7, 27, 16, 12, 19].

Original: In this paper we consider an important setting in dynamic pricing where some contextual informationis available in eachselling time period.
Revised: In this paper we consider an important setting in dynamic pricing where some contextual informationis available in each time period.

Original: It is interesting and challenging to improve the revenuesby well exploiting the sales-relevant contextual information such as product features and marketenvironments.
Revised: It is interesting and challenging to improve the revenues by wellexploiting the sales-relevant contextual information such as product features and market environments.

Original: To model the inﬂuence of the contextual information, we adopt a binary feedback model whichincorporates a comparison between the customer’s valuation and the seller’s price [4, 28, 47, 23, 35].
Revised: To model the inﬂuence of the contextual information, we adopt a binary feedback model whichincorporates a comparison between the customer’s valuation and the seller’s price [3, 25, 47, 20, 35].

Original: Speciﬁcally, in the selling time period t with the associated context x t ∈ R d 0 , the customer’svaluation v t of the product is assumed to be linear with repect to x t , together with some noise z t ,i.e., v t = v ( x t ) =
Revised: Speciﬁcally, in the selling time period t with the associated context x t ∈ R d 0 , the customer’svaluation v t of the product is assumed to be linear with repect to x t , together with some noise z t ,i.e., v t = v ( x t ) =

Original: x (cid:62) t θ 

Revised: x (cid:62) t θ

Original: + z t .
Revised: + z t .

Original: Here the noises { z t } t ∈ [ T ] across the time horizon [ T ] = { 1 , . . .
Revised: Here the noises { z t } t ∈ [ T ] across the time horizon [ T ] = { 1 , . . .

Original: , T }are independent and identically distributed (i.i.d.) from a Cumulative Distribution Function (CDF)
Revised: , T }are independent and identically distributed (i.i.d.) from a Cumulative Distribution Function (CDF)

Original: F .
Revised: F .

Original: Then under the seller’s price p t , a binary feedback y t = 1 { v t ≥ p t } representing the customer’spurchasing decision is observed by the seller.
Revised: Then under the seller’s price p t , a binary feedback y t = 1 { v t ≥ p t } representing the customer’spurchasing decision is observed by the seller.

Original: Namely, the purchase at time period t happens if andonly if the customer’s valuation v t is greater than or equal to the seller’s price p t .
Revised: Namely, the purchase at time period t happens if andonly if the customer’s valuation v t is greater than or equal to the seller’s price p t .

Original: The seller thencollects the current feedback y t which can help the pricing decision at the next time period.
Revised: The seller thencollects the current feedback y t which can help the pricing decision at the next time period.

Original: Note
Revised: Notethat the binary feedback structure y t = 1 { v t ≥ p t } is critical to this online pricing problem’s bandit

Original: [Table]
Revised: [Table]

Original: that the binary feedback structure y t = 1 { v t ≥ p t } is critical to this online pricing problem’s bandit 36 nature.
Revised: nature.

Original: We cannot observe the full information v t at each time period.
Revised: We cannot observe the full information v t at each time period.

Original: Instead, we can only observethe partial information
Revised: Instead, we can only observethe partial information

Original: y t
Revised: y t

Original: = 1 { v t ≥ p t } which varies with different pricing action p t .
Revised: = 1 { v t ≥ p t } which varies with different pricing action p t .

Original: At each time period t , the expected reward of any price p is E (cid:0) p 1 { v t ≥ p } (cid:1) .
Revised: At each time period t , the expected reward of any price p is E (cid:0) p 1 { v t ≥ p } (cid:1) .

Original: Thus, the optimal pricingdepends on the distribution of the valuation v t = v ( x t ) =
Revised: Thus, the optimal pricingdepends on the distribution of the valuation v t = v ( x t ) =

Original: x (cid:62) t θ 

Revised: x (cid:62) t θ

Original: + z t .
Revised: + z t .

Original: We assume both unknownparameter θ 0 and unknown noise CDF F at the beginning of the horizon.
Revised: We assume both unknownparameter θ 0 and unknown noise CDF F at the beginning of the horizon.

Original: Therefore, the sellerneeds to gradually learn both θ 0 and F for better pricing.
Revised: Therefore, the sellerneeds to gradually learn both θ 0 and F for better pricing.

Original: In this paper, we investigate two differentsmoothness assumptions for the noise CDF F .
Revised: In this paper, we investigate two differentsmoothness assumptions for the noise CDF F .

Original: The ﬁrst one assumes Lipschitz continuity andconcavity, while the second only assumes Lipschitz.
Revised: The ﬁrst one assumes Lipschitz continuity and2nd-order smoothness, while the second only assumes Lipschitz.

Original: To tackle these two sub-problems under different noise smoothness assumptions, we propose twosub-policies uniﬁed under a single pricing policy framework, namely Explore then Upper Conﬁdence
Revised: To tackle these two sub-problems under different noise smoothness assumptions, we propose twosub-policies uniﬁed under a single pricing policy framework, namely Explore then Upper Conﬁdence

Original: Bound (ExUCB).
Revised: Bound (ExUCB).

Original: In particular, our proposed ExUCB ﬁrst estimates
Revised: In particular, our proposed ExUCB ﬁrst estimates

Original: θ 0 through some random pricingexploration phase, and then uses this estimate to drive a UCB procedure that well balances revenuemaximizing and F -learning.
Revised: θ 0 through some random pricingexploration phase, and then uses this estimate to drive a Upper Conﬁdence Bound (UCB) procedurethat well balances revenue-maximizing and F -learning.

Original: The same pricing problem has been investigated in the literature [35, 23, 48] under a variety ofsmoothness assumptions on  noise.
Revised: The same pricing problem has been investigated in the literature [35, 20, 48] under a variety ofsmoothness assumptions on the noise.

Original: The existing regret bounds are presented in Table 1 along with ournew regret results in this paper.
Revised: The existing regret bounds are presented in Table 1 along withour new regret results in this paper.

Original: In summary, our theoretical contributions are threefold.1.
Revised: In summary, our theoretical contributions are threefold.1.

Original: Under the Lipschitz and concavity smoothness assumptions, our proposed ExUCB policyis the ﬁrst procedure to achieve ˜ O
Revised: Under the Lipschitz and 2nd-order smoothness assumptions, our proposed ExUCB policyis the ﬁrst procedure to achieve ˜ O

Original: ( T 2 / 3 ) regret rate.
Revised: ( T 2 / 3 ) regret rate.

Original: We prove valid θ 0 estimation accuracyand quantify its inﬂuence on the regret of the UCB procedure driven by the θ 0 -estimate.
Revised: We prove valid θ 0 estimation accuracyand quantify its inﬂuence on the regret of the UCB procedure driven by the θ 0 -estimate.

Original: Thishelps us determine an optimal balance between random pricing exploration and the followed
Revised: Thishelps us determine an optimal balance between random pricing exploration and the followed

Original: UCB phase.
Revised: UCB phase.

Original: Our ˜ O ( T 2 / 3 ) regret improves the existing regret bound of ˜ O
Revised: Our ˜ O ( T 2 / 3 ) regret improves the existing regret bound of ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) )
Revised: ( T 2 / 3 ∨ (1 − α ) )

Original: in[35] which has an indeterministic α .
Revised: in[35] which has an indeterministic α .

Original: It also improves the ˜ O
Revised: It also improves the ˜ O

Original: ( T 5 / 7 ) regret in [23] whichassumes a stronger smoothness assumption of twice continuously differentiable F .2.
Revised: ( T 5 / 7 ) regret in [20] whichassumes a stronger smoothness assumption of twice continuously differentiable F .2.

Original: Under the Lipschitz and concavity smoothness assumptions, we obtain the lower bound of˜Ω( T 3 / 5 ) by constructing the instances such that any policy cannot perform well on all ofthem.
Revised: Under the Lipschitz and 2nd-order smoothness assumptions, we obtain the lower boundof ˜Ω( T 3 / 5 ) by constructing the instances such that any policy cannot perform well onall of them.

Original: To our limited knowledge, this is the ﬁrst lower bound result under such smoothnessassumptions.
Revised: To our limited knowledge, this is the ﬁrst lower bound result under suchsmoothness assumptions.

Original: To construct instances under the Lipschitz and concavity assumptions, the corestep is to build “bump towers” by piling up an inﬁnite series of “quadratically shrinking” basic bump functions.
Revised: To construct instances that satisfy this smoothness assumption,the core step is to build “bump towers” by piling up an inﬁnite series of “quadraticallyshrinking” basic bump functions.

Original: Note that the ˜Ω( T 2 / 3 ) lower bound in [48] only applies to the Lipschitzassumption since their constructed instances do not satisfy the concavity assumption.3.
Revised: Note that the ˜Ω( T 2 / 3 ) lower bound in [48] only appliesto the Lipschitz assumption since their constructed instances do not satisfy the 2nd-ordersmoothness assumption.3.

Original: Under the Lipschitzsmoothness assumption, ExUCB can match the best existing upperbound of ˜ O ( T 3 / 4 ) in [48], which shows the adaptivity of ExUCB to different levels ofsmoothness assumptions on the noise.
Revised: Under the Lipschitz assumption, ExUCB can match the best existing upper bound of˜ O ( T 3 / 4 ) in [48] and is computationally more efﬁcient. This shows the adaptivity of ExUCBto different noise smoothness levels.

Original: Our improved regret over existing results in [35, 23] demonstrates the methodological novelty ofthe proposed Explore-then-UCB strategy.
Revised: Our improved regret over existing results in [35, 20] demonstrates the methodological novelty ofthe proposed Explore-then-UCB strategy.

Original: In [35], the authors implemented the UCB idea but didnot apply random pricing explorations.
Revised: In [35], the authors implemented the UCB idea but didnot apply random pricing explorations.

Original: Thus they can only use adaptive data to estimate θ 0 , whichresults in an indeterministic α in their regret of ˜ O
Revised: Thus they can only use adaptive data to estimate θ 0 , whichresults in an indeterministic α in their regret of ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) .
Revised: ( T 2 / 3 ∨ (1 − α ) ) .

Original: In contrast, by using a randomexploration phase, ExUCB achieves the exact regret of ˜ O
Revised: In contrast, by using a randomexploration phase, ExUCB achieves the exact regret of ˜ O

Original: ( T 2 / 3 ) .
Revised: ( T 2 / 3 ) .

Original: In [23], the authors proposed an
Revised: In [20], the authors proposed an

Original: Explore-then-Commit type of policy that ﬁrst estimates θ 0 and F in an exploration phase and thencommits to these estimates for pricing in the exploitation phase.
Revised: Explore-then-Commit type of policy that ﬁrst estimates θ 0 and F in an exploration phase and thencommits to these estimates for pricing in the exploitation phase.

Original: In comparison, ExUCB imposes a UCB procedure to adaptively balance between revenue-maximizing and F -learning.
Revised: In comparison, ExUCB imposes a UCB procedure to adaptively balance between revenue-maximizing and F -learning.

Original: The regretadvantage of ExUCB indicates the importance of our UCB procedure after the exploration.
Revised: The regretadvantage of ExUCB indicates the importance of our UCB procedure after the exploration.

Original:  Related Works
Revised:  Related Works

Original: In this section, we discuss how our work relates to the literature of dynamic pricing and bandit.
Revised: In this section, we discuss how our work relates to the literature of dynamic pricing, bandits andcontextual search..

Original: Non-Contextual Dynamic Pricing.
Revised: Non-Contextual Dynamic Pricing.

Original: Extensive investigations have been conducted on dynamicpricing problems without contextual information [9, 11, 46, 10, 16].
Revised: Extensive investigations have been conducted on dynamicpricing problems without contextual information [7, 9, 46, 8, 14].

Original: Due to the bandit nature ofonline pricing problems, the Upper Conﬁdence Bound (UCB) idea [5, 1] has  been successfullyapplied in several different non-contextual settings [32, 37, 44].
Revised: For m -th smooth demand functions,[44] applied the UCB idea with local-bin approximations to achieve an ˜ O ( T ( m +1) / (2 m +1) ) regret,and proved a matching lower bound. The UCB approach has also been adopted in [29, 37].

Original: However, these methods are not ableto utilize the potential contextual information for better pricing.
Revised: However,these methods are not able to utilize the potential contextual information for better pricing.

Original: Contextual Dynamic Pricing.
Revised: Contextual Dynamic Pricing.

Original: There are signiﬁcant interests among researchers in contextualdynamic pricing [41, 27, 36, 38, 8, 43, 7, 21, 45, 29, 15, 17].
Revised: There are signiﬁcant interests among researchers in contextualdynamic pricing [41, 24, 36, 38, 6, 43, 5, 18, 45, 26, 13, 15].

Original: Withfully nonparametric expectedrevenue functions, [15] designed the pricing policy with ˜ O
Revised: With nonparametric revenue functions,[13] designed a policy with ˜ O

Original: ( T ( d 0 +2) /
Revised: ( T ( d 0 +2) /

Original: ( d 0 +4) ) regret based on theadaptive binning idea from nonparametric contextual bandit [40].
Revised: ( d 0 +4) ) regret based on the adaptive binning idea fromnonparametric contextual bandit [40].

Original:  In [43], the authors adopted alog-linear valuation model with a very special property that the linear structure and noise distributionhave separable inﬂuences on optimal prices. They proposed a pricing algorithm with ˜ O
Revised: Note that their proved Ω( T 3 / 5 ) regret lower bound for theone-dimensional case does not apply to our setting since their constructed revenue functions arebeyond our revenue function class. In [43], the authors adopted a log-linear valuation model andproposed a pricing algorithm with ˜ O

Original: ( T 1 / 2 )regret that took advantage of this property.
Revised: ( T 1 / 2 ) regret.

Original: Some recent literature [28, 25, 26, 47, 23,  35, 48]adopted the same linear valuation model[-v t = v ( x t ) = x (cid:62) t θ 
 + z t-] as in this work.
Revised: Some recent literature [25, 22, 23, 47, 20, 35, 48]adopted the same linear valuation model[-v t = v ( x t ) = x (cid:62) t θ 
 + z t-] as in this work.

Original: By assuming aknown noise distribution, [28, 47] designed algorithms with O (log T ) regret.
Revised: By assuming a known noise distribution,[25, 47] designed algorithms with O (log T ) regret.

Original: For noise in a knownparametric family, [28] proposed apricing policy with O ( T 1 / 2 ) regret.
Revised: For noise in a known parametric family, [25]proposed a policy with O ( T 1 / 2 ) regret.

Original: In [26], the authors assumeda known ambiguity set that contains the noise distribution and proposed an algorithm with a ˜ O
Revised: In [23], the authors assumed a known ambiguity set thatcontains the noise distribution and proposed an algorithm with a ˜ O

Original: ( T 2 / 3 )regret with respect to a robust benchmark corresponding to the ambiguity set.
Revised: ( T 2 / 3 ) regret with respect to arobust benchmark.

Original: With unknown noisedistribution, the ambiguity set would be extremely large and the robust benchmark can be far fromthe benchmark corresponding to the true optimal policy.
Revised: With unknown noise distribution, the ambiguity set would be extremely large andthe robust benchmark can be far from the true optimal one.

Original: In [25], the authors considered unknownnoise witha “full information” feedback structure that the seller can observe all valuations (bids) andproposed an algorithm with ˜ O
Revised: In [22], the authors considered unknownnoise with “full information” feedbacks and proposed an algorithm with ˜ O

Original: ( T 1 / 2 ) regret.
Revised: ( T 1 / 2 ) regret.

Original: The most related works to ours are [23, 35, 48], which considered binary cencored feedback andunknown noise distributions under different smoothness assumptions.
Revised: The most related works to ours are [20, 35, 48], which considered binary cencored feedback andunknown noise distributions under different smoothness assumptions.

Original: In [23], the authors proposedan Explore-then-Commitpricing policy with an ˜ O
Revised: In [20], the authors proposedan Explore-then-Commit policy with an ˜ O

Original: ( T 2 m +1 4 m − 1 ) regret under m ( ≥ 2) times continuouslydifferentiable F .
Revised: ( T 2 m +1 4 m − 1 ) regret under m ( ≥ 2) times continuously differentiable F .

Original: Their twice continuously differentiable assumption ( m = 2 ) would imply the Lipschitz and concavity assumptions.
Revised: The m = 2 case would imply our Lipschitz and 2nd-order smoothness assumptions.

Original: Thusour proposed ExUCBalgorithm can achieve a lowerregret of ˜ O
Revised: Thus ExUCB can achieve a lower ˜ O

Original: ( T 2 / 3 )  than the ˜ O
Revised: ( T 2 / 3 ) regret than the ˜ O

Original: ( T 5 / 7 ) regret in [23] even under weaker smoothness assumptions.
Revised: ( T 5 / 7 ) rate in [20] even under weaker smoothness assumptions.

Original: In[35], the authors proved an ˜ O
Revised: In [35], the authors proved an ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) regret.
Revised: ( T 2 / 3 ∨ (1 − α ) ) regret.

Original: The value of α depends on the convergencerate of theirproposed linear classiﬁcation θ 0 -estimation method and is indeterministic because theestimation is based on adaptive data.
Revised: The value of α depends onthe convergence rate of their θ 0 estimates. However, α is indeterministic and no rigorous justiﬁcationshas been made.

Original: In comparison, our proposed policy achieves an exact regret of˜ O ( T 2 / 3 ) .
Revised: In comparison, our ExUCB policy achieves an exact regret of ˜ O ( T 2 / 3 ) .

Original: In [48], the authors developed an adaptive pricing policy that achieved an ˜ O ( T 3 / 4 ) regretfor adversarial contexts and arbitrary bounded noise distributions.
Revised: In [48],the authors developed an adaptive pricing policy that achieved an ˜ O ( T 3 / 4 ) regret for adversarialcontexts and arbitrary bounded noise distributions.

Original: In comparison, our ExUCB policymatches the ˜ O
Revised: Our ExUCB policy matches the ˜ O

Original: ( T 3 / 4 ) regret under a Lipschitz F and can improve the regret to ˜ O
Revised: ( T 3 / 4 ) regretunder a Lipschitz F and can improve the regret to ˜ O

Original: ( T 2 / 3 ) with anadditional concavity assumption.
Revised: ( T 2 / 3 ) with an additional 2nd-order smoothnessassumption.

Revised: In addition, since the EXP4-based policy in [48] requires exponential computations w.r.t.the covariate dimension d 0 , ExUCB is computationally more efﬁcient with a time complexity that ispolynomial with d 0 .

Original: Bandit Algorithms.
Revised: Bandit Algorithms.

Original: Bandit-type feedback structure isvery natural in dynamic pricing problems[32, 15, 48].
Revised: Bandit-type feedback structure is natural in dynamic pricing [29, 13, 48].

Original: The bandit literature provides a signiﬁcant variety of methods to resolve the explorationexploitation tradeoff that arises with the bandit feedback [12, 33].
Revised: Thebandit literature provides a signiﬁcant variety of methods to resolve the exploration-exploitationtradeoff that arises with the bandit feedback [10, 31].

Original: A key tool we used in this paperis the perturbed linear bandit  [35].
Revised: A key tool we used in this paper is the perturbedlinear bandit (PLB) [35].

Original: It is related to the misspeciﬁed linear bandits [34, 39, 24] andnon-stationary linear bandits [19, 42, 49].
Revised: It is related to the misspeciﬁed linear bandits [32, 39, 21] and non-stationarylinear bandits [17, 42, 49].

Original: In addition,because of the continuous nature of the pricingactions, dynamic pricing is closely related to continuum-armed bandit [2, 31, 6, 13].
Revised: In addition, dynamic pricing is closely related to continuum-armed bandit[2, 28, 4, 11].

Original: The lower boundwe proved borrows the “needle in haystack” idea that is widely applied in the continuum-armedbandit literature [31, 48]. 
Revised: The lower bound we proved borrows the “needle in haystack” idea that is widelyapplied in continuum-armed bandits [28, 48]. Contextual Search.

Revised: Contextual pricing with binary feedback can be formulated into the contextualsearch problem [34, 33, 30].

Revised: However, different noises are considered other than our stochasticvaluation ones.

Revised: [34] is noiseless and only small-variance noises are handled in [30]; A “ﬂipping”noise on customers’ decisions are investigated in [33].

Original:  Preliminaries
Revised:  Preliminaries

Original: Problem Setting The sales time horizon is [ T ] = { 1 , . ..
Revised: Problem Setting. The sales time horizon is [ T ] = { 1 , . .

Original:  , T } with the initial time period t = 1 .
Revised: . , T } with the initial time period t = 1 .

Original: We present the online pricing procedure as follows.
Revised: We present the online pricing procedure as follows.

Original: At time period t , the seller observes a context x t ∈ R d 0 .
Revised: At time period t , the seller observes a context x t ∈ R d 0 .

Original: The customer valuates the product at v t = x (cid:62) t θ 0 + z t , where z t i.i.d.
Revised: The customer valuates the product at v t = x (cid:62) t θ 0 + z t , where z t i.i.d.

Original: ∼ F .
Revised: ∼ F .

Original: The seller sets a price p t based on x t and the past sales data { ( x s , p s , y s ) } s ≤ t − 1 .
Revised: The seller sets a price p t based on x t and the past sales data { ( x s , p s , y s ) } s ≤ t − 1 .

Original: The seller observes the binary feedback y t
Revised: The seller observes the binary feedback y t

Original: = 1 { v t ≥ p t } and collects the revenue p t y t .
Revised: = 1 { v t ≥ p t } and collects the revenue p t y t .

Original: Let t = t
Revised: Let t = t

Original: + 1 and go back to Step 1.
Revised: + 1 and go back to Step 1.

Original: Regret Deﬁnition Given the context x t , the probability of a purchase is 1 − F ( p t − x (cid:62) t θ 0 ) andthus the expected reward of setting the price p t is p t (cid:0) 1 − F ( p t − x (cid:62) t θ 0 ) (cid:1) .
Revised: Regret Deﬁnition. Given the context x t , the probability of a purchase is 1 − F ( p t − x (cid:62) t θ 0 ) andthus the expected reward of setting the price p t is p t (cid:0) 1 − F ( p t − x (cid:62) t θ 0 ) (cid:1) .

Original: Deﬁne the optimalprice given the context x as p ∗ ( x ) =
Revised: Deﬁne the optimalprice given the context x as p ∗ ( x ) =

Original: arg max p ≥ 0 p (cid:0) 1 − F ( p − x (cid:62) θ 0 ) (cid:1) .
Revised: arg max p ≥ 0 p (cid:0) 1 − F ( p − x (cid:62) θ 0 ) (cid:1) .

Original: Then the regret r t at timeperiod t is deﬁned as the expected revenue loss with respect to the optimal price p ∗ t = p ∗ ( x t ) , i.e.,r t = p 
Revised: Then the regret r t at timeperiod t is deﬁned as the expected revenue loss with respect to the optimal price p ∗ t = p ∗ ( x t ) , i.e.,r t = p ∗ t (cid:0) 1 − F ( p ∗ t − x (cid:62) t θ 0 ) (cid:1) − p t (cid:0) 1 − F ( p t − x (cid:62) t θ 0 ) (cid:1) .

Original: [Equation]{+Deﬁnition 
+}
Revised: {+Deﬁnition 
+}

Original: (cid:0)
Revised: The cumulative regret across the horizon is deﬁned as R

Original: [Equation]
Revised: [Equation]

Original: ( p
Revised: =

Original: [Equation]
Revised: [Equation]

Original: (cid:62)
Revised: (cid:88) t = r

Original: [Equation]
Revised: [Equation]

Original: (cid:1)
Revised: = T (cid:88) t = (cid:104) p

Original: [Equation]
Revised: [Equation]

Original: (cid:0)
Revised: (cid:0)

Original: [Equation]
Revised: [Equation]

Original:  ( p[Equation] −
Revised: − F ( p

Original: [Equation]
Revised: [Equation]

Original:  (cid:62)[-[Equation] (cid:1) . Deﬁnition 
-]
Revised: − x (cid:62)[-[Equation] (cid:1) . Deﬁnition 
-]

Original: The cumulative regret across the horizon is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: The expected cumulative regret E ( R T ) is obtained from taking expectation over the potential randomness of the data and the pricing policy.
Revised: The expected cumulative regret E ( R T ) is obtained from taking expectation over the potential randomness of the data and the pricing policy.

Original: Our goal is to minimize E ( R T ) by dynamically settingthe price p t under unknown θ 0 and F .
Revised: Our goal is to minimize E ( R T ) by dynamically settingthe price p t under unknown θ 0 and F .

Original: Perturbed Linear Bandit Deﬁnition We use the Perturbed Linear Bandit (PLB)

Original: [-[35] formulationto tackle the balance between F -learning and revenue-maximizing. We introduce its deﬁnition here. Deﬁnition 
 The rewards Z t , parameters ξ t and action sets A t form a perturbed linear bandit with a perturbation constant C p , if-]
Revised: [-[35] formulationto tackle the balance between F -learning and revenue-maximizing. We introduce its deﬁnition here. Deﬁnition 
 The rewards Z t , parameters ξ t and action sets A t form a perturbed linear bandit with a perturbation constant C p , if-]Technical Assumptions.

Original: Z t = (cid:104) ξ t , A t (cid:105) + η t for any selected action A t ∈

Original: A t , and any two parameters are close to each other, i.e.,|| ξ s − ξ t || ∞ ≤ C p , ∀ s, t ∈ N + . Here η t is σ -sub-Gaussian conditional on the ﬁltration F t − 1 = σ ( ξ 1 , A 1 , Z 1 , . . . , ξ t , A t ) . The “all-close-to-each-other” condition on the linear parameters ξ t ’s implies the existence of a“central” parameter ξ ∗ such that || ξ t − ξ ∗ || ∞ ≤ C p 2 for any t .
Revised: We now present our main assumptions.

Original: Thus the reward structure at time tregulated by ξ t can be viewed as a perturbation from that regulated by ξ ∗ . The linear bandit [1, 20, 3]is a zero-perturbation PLB with ξ t = ξ ∗ for any t . 3.1 Technical Assumptions We summarize our main assumptions as follows.
Revised: Assumptions 1 – 2 are standardin dynamic pricing [25, 13, 23, 20, 35, 48].

Original: Assumption 1 (Bounded contexts and parameter)
Revised: Assumption 1 (Bounded contexts and parameter)

Original: The covariates x t are bounded as || x t || ∞ ≤ 1 .
Revised: The covariates x t are bounded as || x t || ∞ ≤ 1 .

Original: The (cid:96) 1 norm ||
Revised: The (cid:96) 1 norm ||

Original: θ
Revised: θ

Original: || 1 of θ 0 is bounded by a constant W .
Revised: || 1 of θ 0 is bounded by a constant W .

Original: Assumption 2 (i.i.d. contexts)
Revised: Assumption 2 (i.i.d. contexts)

Original: The covariates x t i.i.d.
Revised: The covariates x t i.i.d.

Original: ∼ P x with the support X and the matrix Σ = 152 E (cid:0) (1 , x (cid:62) t ) (cid:62) (1 , x (cid:62) t ) (cid:1) satisﬁes that Σ − c
Revised: ∼ P x with the support X and the matrix Σ = 146 E (cid:0) (1 , x (cid:62) t ) (cid:62) (1 , x (cid:62) t ) (cid:1) satisﬁes that Σ − c

Original: I is positive-deﬁnite for some positive constant c 0 .
Revised: I is positive-deﬁnite for some positive constant c 0 .

Original: Assumption 3 (Bounded noise mean)
Revised: Assumption 3 (Bounded valuations)

Original: The absolute value of the noise expectation | E ( z ) | is boundedby a  constant U , where z ∼ F . 
Revised: The customers’ valuations v t ∈ [0 , B ] for a known constant B . Assumption 3 assumes a known upper bound for the customers’ valuations [23, 20], which isreasonable for real-life products.

Original:  Assumption 4 (Bounded valuations) All realized valuations v t ∈ [0 , B ] for a known constant B .
Revised: Note that Assumption 3 indicates a known upper bound p max = Bfor the optimal prices.

Original: Assumption 5 (Bounded optimal prices)
Revised: In addition, Assumptions 1 and 3 imply an upper bound U = W + Bfor the noise absolute value. In the following, we further introduce two smoothness assumptions.

Original: Any optimal price p ∗ ( x ) for x ∈ X satisﬁes p ∗ ( x ) ∈(0 , p max ) for a known constant p max . 
Revised: The Lipschitz condition in Assumption 4 is basic and was considered in [35, 48]. Assumption 5assumes the 2nd-order smoothness of the expected revenue functions around the optimal prices andhas also been imposed in [13, 35]. It is satisﬁed with bounded second derivatives of F [35] butﬁts for a broader class of F . Both Assumptions 4 – 5 are satisﬁed by m ( ≥ 2) times continuouslydifferentiablility of F as considered in [20].

Original: Assumption 6 (Lipschitz Continuity)
Revised: Assumption 4 (Lipschitz Continuity)

Original: The noise CDF F is Lipschitz continuous with a constant L ,i.e., | F ( x ) − F ( y ) | ≤ L | x − y | , ∀ x, y ∈ R .
Revised: The noise CDF F is Lipschitz continuous with a constant L ,i.e., | F ( x ) − F ( y ) | ≤ L | x − y | , ∀ x, y ∈ R .

Original: Assumption 7 (Concavity) Deﬁne the general expected revenue functions associated with the noisedistribution F as f q ( p ) =
Revised: Assumption 5 (2nd-order Smoothness) Deﬁne the general expected revenue functions associatedwith the noise distribution F as f q ( p ) =

Original: p (1 − F ( p − q )) .
Revised: p (1 − F ( p − q )) .

Original: There exists a positive constant C such that for anyx ∈ X and q = x (cid:62) θ 0 , we have f q ( p ∗ ( x ))
Revised: There exists a positive constant C suchthat for any x ∈ X and q = x (cid:62) θ 0 , we have f q ( p ∗ ( x ))

Original: − f q ( p ) ≤ C ( p ∗ ( x ) − p ) 2 , ∀ p ∈ [0 , p max ] .
Revised: − f q ( p ) ≤ C ( p ∗ ( x ) − p ) 2 , ∀ p ∈ [0 , p max ] .

Original: Assumptions 1 – 2 are standard assumptions in dynamic pricing literature [28, 15, 26, 23, 35, 48].

Original: Assumption 3 is mild and weaker than the bounded noise assumption in [26, 23, 48].

Original: Assumption 4assumes bounded valuations [26, 23] while Assumption 5 assumes bounded optimal prices [15, 35].

Original: As mentioned in [15, 26, 23, 35], for real-life products, it is reasonable to know the upper boundsof customer valuations and optimal prices.

Original: The Lipschitz condition in 6 has also been consideredin [35, 48] and is a basic smoothness assumption.

Original: Assumption 7 assumes local concavity of theexpected revenue functions around the optimal prices and has also been imposed in [15, 35].

Original: It issatisﬁed with bounded second derivatives of F [35] but ﬁts for a much broader class of F that isnot everywhere differentiable.

Original: Both Assumptions 6 – 7 are satisﬁed by m ( ≥ 2) times continuouslydifferentiablility of F as considered in [23].

Original: In this paper, we investigate two smoothness levels on the unknown noise distribution F . Case (A) Lipschitz and concavity, i
Revised: In this paper, we investigate two smoothness levels on the unknown noise distribution F , i.e., Case(A) :

Original: e., both Assumptions 6 – 7 hold.

Original:  Case (B) Lipschitz-only, i e., only Assumption 6 holds.
Revised: Lipschitz and 2nd-order smoothness; Case (B) : Lipschitz-only.

Original: For these two scenarios, we design respective algorithms that are uniﬁed under a single ExUCBpolicy framework.
Revised: For these two scenarios, wedesign respective algorithms that are uniﬁed under a single ExUCB policy framework.

Original:  Algorithm
Revised:  Algorithm

Original: We ﬁrst propose the general ExUCB policy in Algorithm 1.
Revised: We ﬁrst propose the general ExUCB policy in Algorithm 1.

Original: Without the knowledge of the horizonlength T , we utilize the doubling trick [33] to cut the horizon into episodes.
Revised: Without the knowledge of the horizonlength T , we utilize the doubling trick [31] to cut the horizon into episodes.

Original: Each episode consistsof an exploration phase and a followed UCB phase.
Revised: Each episode consistsof an exploration phase and a followed UCB phase.

Original: Denote the ﬁrst episode length as α 1 and thenumber of episodes as n ( T, α 1 ) .
Revised: Denote the ﬁrst episode length as α 1 and thenumber of episodes as n ( T, α 1 ) .

Original: The schematic of ExUCB is displayed in Figure 1.
Revised: The schematic of ExUCB is displayed in Figure 1.

Original:  1: Input: (at time 0) p max , B, α 1 , C 1 , C 2 , β, γ, λ 2: Input: (arrives over time) covariates { x t } t ∈
Revised: Algorithm 1: Input: (at time 0) p max , B, α 1 , C 1 , C 2 , β, γ, λ 2: Input: (arrives over time) covariates { x t } t ∈

Original: [ T ] 3:
Revised: [ T ] 3:

Original: For episodes k = 1 , 2 , . . . , n (= n ( T, α 1 )) , do 4:
Revised: For episodes k = 1 , 2 , . . . , n (= n ( T, α 1 )) , do 4:

Original: Set the (projected) length of the k -th episode as (cid:96) k
Revised: Set the (projected) length of the k -th episode as (cid:96) k

Original: = 2 k − 1 α 1 .
Revised: = 2 k − 1 α 1 .

Original: 5: (Exploration Phase) 6:
Revised: 5: (Exploration Phase) 6:

Original: For time t ∈ E k := { (cid:80) k − 1 i =1 (cid:96) i + 1 , . . .
Revised: For time t ∈ E k := { (cid:80) k − 1 i =1 (cid:96) i + 1 , . . .

Original: , (cid:80) k − 1 i =1 (cid:96) i + (cid:100) C 1 (cid:96) βk (cid:101)} , do 7:
Revised: , (cid:80) k − 1 i =1 (cid:96) i + (cid:100) C 1 (cid:96) βk (cid:101)} , do 7:

Original: Set a price p t uniformly randomly from (0 , B ) .
Revised: Set a price p t uniformly randomly from (0 , B ) .

Original: Receive a binary response y t .
Revised: Receive a binary response y t .

Original: Calculate the θ 0 -estimate ˆ θ k by (ˆ µ k , ˆ θ k ) =
Revised: Calculate the θ 0 -estimate ˆ θ k by (ˆ µ k , ˆ θ k ) =

Original: arg min µ,θ |E k | (cid:88) t ∈E k (cid:0) By t − (1 , x (cid:62) t )( µ, θ (cid:62) ) (cid:62) (cid:1) 2 .
Revised: arg min µ,θ |E k | (cid:88) t ∈E k (cid:0) By t − (1 , x (cid:62) t )( µ, θ (cid:62) ) (cid:62) (cid:1) 2 .

Original: 10: (UCB Phase) 11:
Revised: 10: (UCB Phase) 11:

Original: For time t ∈
Revised: For time t ∈

Original: U k :=
Revised: U k :=

Original: { (cid:80) k − 1 i =1 (cid:96) i + (cid:100) C 1 (cid:96) β k (cid:101) + 1 , . . .
Revised: { (cid:80) k − 1 i =1 (cid:96) i + (cid:100) C 1 (cid:96) βk (cid:101) + 1 , . . .

Original: , (cid:80) k − 1 i =1 (cid:96) i + (cid:96) k } , do 12:
Revised: , (cid:80) k − 1 i =1 (cid:96) i + (cid:96) k } , do 12:

Original: Apply the Inner UCB Algorithm on the coming sequential covariates { x t } t ∈U k with the θ 0 -estimate ˆ θ k , the discretization number d k = (cid:100) C 2 ( (cid:96) k − (cid:100) C 1 (cid:96) βk (cid:101) ) γ (cid:101) , the optimal price bound p max , the projected length ( (cid:96) k − (cid:100) C 1 (cid:96) βk (cid:101) ) , and the regularization parameter λ .
Revised: Apply the Inner UCB Algorithm on the coming sequential covariates { x t } t ∈U k with the θ 0 -estimate ˆ θ k , the discretization number d k = (cid:100) C 2 ( (cid:96) k − (cid:100) C 1 (cid:96) βk (cid:101) ) γ (cid:101) , the optimal price bound p max , the projected length ( (cid:96) k − (cid:100) C 1 (cid:96) βk (cid:101) ) , and the regularization parameter λ .

Original: In the exploration phase, we conduct random pricing and ﬁnally obtain a θ 0 estimate.
Revised: In the exploration phase, we conduct random pricing and ﬁnally obtain a θ 0 estimate.

Original: In the followed
Revised: In the followed

Original: UCB phase, we implement a UCB procedure that uses the θ 0 estimate to drive the balance between
Revised: UCB phase, we implement a UCB procedure that uses the θ 0 estimate to drive the balance between

Original: F -learning and revenue-maximizing.
Revised: F -learning and revenue-maximizing.

Original: We discuss the two components in the following.
Revised: We discuss the two components in the following.

Original: Estimation of θ 0 .
Revised: Estimation of θ 0 .

Original: At the beginning of each episode, we impose a random pricing exploration phaseto generate data for θ 0 estimation.
Revised: At the beginning of each episode, we impose a random pricing exploration phaseto generate data for θ 0 estimation.

Original: The exploration phase length (cid:100) C 1 (cid:96) βk (cid:101) is set as some β order of theepisode length (cid:96) k .
Revised: The exploration phase length (cid:100) C 1 (cid:96) βk (cid:101) is set as some β order of theepisode length (cid:96) k .

Original: By uniformly random pricing in the bounded valuation range, there arises a linearregression structure [26, 23] involving the signal of θ 0 .
Revised: By uniformly random pricing in the bounded valuation range, there arises a linearregression structure [23, 20] involving the signal of θ 0 .

Original: Speciﬁcally, we have
Revised: Speciﬁcally, we have

Original: [Equation]
Revised: [Equation]

Original: Thus we are able to provide guarantees for ˆ θ k using classical analysis techniques on linear regression.
Revised: Thus we are able to provide guarantees for ˆ θ k using classical analysis techniques on linear regression.

Original: In contrast, [35] applied a linear classiﬁcation method on the adaptive data of the previous episode toobtain a θ 0 -estimate, which lacks theoretical guarantees for the estimation accuracy.ˆ θ k -driven UCB Procedure.
Revised: In contrast, [35] applied a linear classiﬁcation method on the adaptive data of the previous episode toobtain a θ 0 -estimate, which lacks theoretical guarantees for the estimation accuracy.ˆ θ k -driven UCB Procedure.

Original: In the UCB phase of episode k , we use the obtained linear regressionestimate ˆ θ k to guide a balance between F -learning and revenue-maximizing.
Revised: In the UCB phase of episode k , we use the obtained linear regressionestimate ˆ θ k to drive a UCB procedure that balances between F -learning and revenue-maximizing.

Original: Different from [35],our UCB procedure does not start at the beginning of each episode.
Revised: Different from [35], our UCB procedure does not start at the beginning of each episode.

Original: In addition, we offer a differentsource of θ 0 estimate that is independent of previous episodes and allow more general discretizationstuned by a parameter γ .F -learning interval G (ˆ θ ) = [ −|| ˆ θ || 1 , p max + ||
Revised: In addition,we offer a different source of θ 0 estimate that is independent of previous episodes and allow moregeneral discretizations tuned by a parameter γ .

Original: ˆ θ || 1 ] into d same-length sub-intervals with their midpoints denoted as m 1 , . . .

Original: , m d .

Original: The Inner UCB Algorithm is explicitly presented as Algorithm 2 above.
Revised: The Inner UCB Algorithm is explicitly presented as Algorithm 2.

Original: In summary, the knowledgeof F is continuously updated and the prices are set accordingly.
Revised: In summary, the knowledge of F is continuously updated and the prices are set accordingly.

Original: Since any potential optimal pricep ∈ (0 , p max ) , we only need to explore those F -values on the potential range of p − x (cid:62) t θ 0 , i.e.,[ −|| θ 0 || 1 , p max + || θ 

Revised: Since any potential optimal price p ∈ (0 , p max ) , we only need to explore those F -values on the potential range of p − x (cid:62) t θ 0 , i.e [ −|| θ 0 || 1 , p max + || θ

Original: || 1 ] .
Revised: || 1 ] .

Original: Thus we ﬁrst restrict our F -learning attention to the interval G (ˆ θ )
Revised: Thus we ﬁrst restrict our F -learning attention to the interval G (ˆ θ )

Original: =[ −|| ˆ θ || 1 , p max
Revised: =[ −|| ˆ θ || 1 , p max

Original: + ||
Revised: + ||

Original: ˆ θ || 1 ] .
Revised: ˆ θ || 1 ] .

Original: Then we implement a key discretization procedure of G ( ˆ θ )  and further focusthe learning of F on the midpoints { m j } j ∈ [ d ] of the discretized sub-intervals.
Revised: Now we borrow the discretization approach in [35]. Speciﬁcally, we discretize G (ˆ θ ) into d sub-intervals and further focus the learning of F on their d midpoints { m j } j ∈ [ d ] .

Original: Now at each time period t , we only consider those prices p t that would require some value of{ 1 − F ( m j ) } j ∈
Revised: Notethat we use d 0 to refer to the covariate dimensionality, and use d, d k to refer to the discretizationnumbers.

Original: [ d ] to calculate its expected revenue.
Revised: This discretization idea would enable a mutual reinforcement procedure of the discretizedprice selection and the knowledge accumulation of F on these d discretized points. They canrepeatedly enhance each other in a closed loop.

Original: We construct the candidate price set S t ={ m j + x (cid:62) t ˆ θ | j ∈ [ d ] , m j + x (cid:62) t ˆ θ ∈ (0 , p max ) } .
Revised: To do so, at each time period t , we construct thecandidate price set S t = { m j + x (cid:62) t ˆ θ | j ∈ [ d ] , m j + x (cid:62) t ˆ θ ∈ (0 , p max ) } .

Original: Then for any candidate price p t = m j + x (cid:62) t θ 0 ∈ S t , itsassociated purchasing probability (1 − F ( p t − x (cid:62) t θ 0 )) would be close to (1 − F ( m j )) , on which wehave accumulated much knowledge  from  the past data.
Revised: Then for any candidateprice p t = m j + x (cid:62) t θ 0 ∈ S t , its associated purchasing probability (1 − F ( p t − x (cid:62) t θ 0 )) would beclose to (1 − F ( m j )) . Therefore, the knowledge accumulation of F on { m j } j ∈ [ d ] helps with theexpected revenue evaluation of the candidate prices and thus the price selection from S t . On the otherhand, the selection of any candidate price p t = m j + x (cid:62) t θ 0 ∈ S t invokes a binary outcome with aprobability close to 1 − F ( m j ) , thus helping with the knowledge accumulation of F on m j .

Original: Denote D t − 1 ,j := { s : 1 ≤ s ≤ t − 1 , j s = j }as all past time periods s
Revised: Denote D t − 1 ,j := { s : 1 ≤ s ≤ t − 1 , j s = j } as all past time periods s

Original: such that p s − x (cid:62) t ˆ θ = m j .
Revised: such that p s − x (cid:62) t ˆ θ = m j .

Original: Then we construct the UCB of 1 − F ( m j ) as[Equation] m [Equation]
Revised: Then weconstruct the UCB of 1 − F ( m j ) as

Original: for D t −

Original: [Equation]
Revised: [Equation]

Original: (cid:54) = ∅

Original: [Equation]

Original: + ∞ [Equation] for D t − 1 ,j = ∅ . Note that for s ∈ D t − 1 ,j , y s ∼ Ber (1 − F ( p s − x (cid:62) s θ 0 ))
Revised: Note that for s ∈ D t − 1 ,j , y s ∼ Ber (1 − F ( p s − x (cid:62) s θ 0 ))

Original: ≈
Revised: ≈

Original: Ber (1 − F ( m j )) .
Revised: Ber (1 − F ( m j )) .

Original: Therefore, the ﬁrst termin the right-hand side of (1) for D t − 1 ,j (cid:54) = ∅ is a regularized weighted average of those y s and thus anestimate of 1 − F ( m j ) .
Revised: Therefore, the ﬁrst termin the right-hand side of (1) for D t − 1 ,j (cid:54) = ∅ is a regularized weighted average of those y s and thus anestimate of 1 − F ( m j ) .

Original: The second term is an associated conﬁdence interval length.
Revised: The second term is an associated conﬁdence interval length.

Original: Based on these upper conﬁdence bounds for 1 − F ( m j ) , we select the price p t = m j + x (cid:62) t ˆ θ from the candidate set
Revised: Based on theseupper conﬁdence bounds for 1 − F ( m j ) , we select the price p t = m j + x (cid:62) t ˆ θ from the candidate set

Original: S t that maximizes the optimism expected revenue ( m j + x (cid:62) t ˆ θ )
Revised: S t that maximizes the optimism expected revenue ( m j + x (cid:62) t ˆ θ )

Original: UCB t (1 − F ( m j )) .
Revised: UCB t (1 − F ( m j )) .

Original: It remains to specify the choice of β t in Equation (1).
Revised: It remains to specify the choice of β t in Equation (1).

Original: We will discuss its choice by the followingperturbed linear bandit  formulation of the pricing problem in  Inner UCB Algorithm.
Revised: We will discuss its choice by the followingperturbed linear bandit [35] formulation of the pricing problem in the Inner UCB Algorithm.

Original: Perturbed Linear Bandit Formulation With the restriction on the candidate sets S t , we are ableto formulate the pricing problem guided by ˆ θ as a perturbed linear bandit.
Revised: Perturbed Linear Bandit Formulation With the restriction on the candidate sets S t , we are ableto formulate the pricing problem guided by ˆ θ as a perturbed linear bandit.

Revised: The PLB is an extension ofthe linear bandit

Original: By specifying the linearparameter ξ t = (1 − F ( m 1 + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ) , . . .
Revised: and we present its formal deﬁnition in Appendix C. The expected reward of the PLBtakes the form of (cid:104) ξ t , A t (cid:105) at each time period t , where A t is the action vector. Different from thelinear bandit, the linear parameters ξ t in the PLB can vary across time periods and are perturbationsfrom a central linear parameter ξ ∗ . In our pricing problem, by specifying the linear parameter ξ t = (1 − F ( m 1 + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ) , . . .

Original: , 1 − F ( m d + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ))
Revised: , 1 − F ( m h + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ))

Original: (cid:62) ∈ R d and mappingeach candidate price p t = m j + x (cid:62) t
Revised: (cid:62) ∈ R d and mapping each candidate price p t = m j + x (cid:62) t

Original: ˆ θ to an action vector
Revised: ˆ θ to an action vector

Original: A t with a single non-zero j -th element p t ,the expected revenue of setting price p t can be rewritten as
Revised: A t with a single non-zero j -th element p t , the expected revenue of setting price p t can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: In addition, all linear parameters ξ t are around the central parameter ξ ∗ =
Revised: In addition, all linear parameters ξ t are around the central parameter ξ ∗ =

Original: (1 − F ( m 1 ) , . . .
Revised: (1 − F ( m 1 ) , . . .

Original: , 1 −
Revised: , 1 −

Original: F ( m d )) (cid:62) and thus close to each other with a perturbation constant  2 L || ˆ θ − θ 

Revised: F ( m h )) (cid:62) and thus close to each other with a perturbation constant C p = 2 L || ˆ θ − θ

Original: || 1 .
Revised: || 1 .

Revised: Namely, we have || ξ s − ξ t || ∞ ≤ C p , ∀ s, t ∈ N + .

Original: This implies alower (cid:96) 1 estimation error of ˆ θ would lead to less perturbations and probably incur less regret.
Revised: This implies a lower (cid:96) 1 estimation error of ˆ θ would lead to lessperturbations and probably incur less regret.

Original: Under the PLB formulation, the Inner UCB Algorithm is indeed equivalent to a modiﬁed version ofthe LinUCB algorithm [1, 33, 35].
Revised: Under the PLB formulation, the Inner UCB Algorithm is indeed equivalent to a modiﬁed version ofthe LinUCB algorithm [1, 31, 35].

Original: Thus we specify β t in (1) as β t = β ∗ t = p 2max (1 ∨ ( 1 p max √ λd +
Revised: Thus we specify β t in (1) as β t = β ∗ t = p 2max (1 ∨ ( 1 p max √ λd +

Original: [Equation]
Revised: [Equation]

Original:  Regret Analysis
Revised:  Regret Analysis

Original: In this section, we analyze the regret of our proposed Explore-then-UCB algorithm.
Revised: In this section, we analyze the regret of our proposed Explore-then-UCB algorithm.

Original: For both Case(A) and Case (B) , we specify the parameters β and γ in Algorithm 1 appropriately and prove therespective upper bounds.
Revised: For both Case(A) and Case (B) , we specify the parameters β and γ in Algorithm 1 appropriately and prove therespective upper bounds.

Original: Furthermore, we prove the ﬁrst regret lower bound for Case (A) .
Revised: Furthermore, we prove the ﬁrst regret lower bound for Case (A) .

Original: 5.1 Upper Bounds
Revised: 5.1 Upper Bounds

Original: For upper bounds, we ﬁrst analyze a single episode in Algorithm 1 and then extend it to the entirehorizon.
Revised: For upper bounds, we ﬁrst analyze a single episode in Algorithm 1 and then extend it to the entirehorizon.

Original: Firstly, we provide the (cid:96) 1 estimation error of our θ 0 estimation procedure.
Revised: Firstly, we provide the (cid:96) 1 estimation error of our θ 0 estimation procedure.

Original: Lemma 1 Under Assumptions 1 – 4, there exists positive constants ˜ c 1 , ˜ c 2 , ˜ c 3 such that for anyepisode k with the exploration phase length n k ≥ ˜ c
Revised: Lemma 1 Under Assumptions 1 – 3, there exists positive constants ˜ c 1 , ˜ c 2 , ˜ c 3 such that for anyepisode k with the exploration phase length n k ≥ ˜ c

Original: 3 ( d 0 + 1) 3 , we have with probability at least1 − 2 n k − ˜ c
Revised: 3 ( d 0 + 1) 3 , we have with probability at least1 − 2 n k − ˜ c

Original: 1 e −˜
Revised: 1 e −

Original: [Equation]
Revised: [Equation]

Revised: d

Original: [Equation]
Revised: [Equation]

Original: d 0+1)2 n k that
Revised: + [Equation] )2 n k that

Original: [Equation]
Revised: [Equation]

Original: Therefore, with an exploration phase length n k that scales as (cid:96) β k , we obtain an estimate ˆ θ k with a highprobability (cid:96) 1 -error of ˜ O
Revised: Therefore, with an exploration phase length n k that scales as (cid:96) βk , we obtain an estimate ˆ θ k with a highprobability (cid:96) 1 -error of ˜ O

Original: ( (cid:96) − β/ 2 k ) .
Revised: ( (cid:96) − β/ 2 k ) .

Original: As this ˆ θ k will guide the UCB procedure, we need to investigatehow the estimation error propagates into the UCB phase regret.
Revised: As this ˆ θ k will guide the UCB procedure, we need to investigatehow the estimation error propagates into the UCB phase regret.

Original: A lower error rate is expected to causeless regret.
Revised: A lower error rate is expected to causeless regret.

Original: On the other hand, a lower error rate requires a larger β and costs a longer explorationphase with more regret.
Revised: On the other hand, a lower error rate requires a larger β and costs a longer explorationphase with more regret.

Original: Therefore, besides the “inner” balance between revenue-maximizing and
Revised: Therefore, besides the “inner” balance between revenue-maximizing and

Original: F -learning in the UCB phase, there is an “outer” balance between the exploration phase regret andthe UCB phase regret, both associated with θ 0 -learning.
Revised: F -learning in the UCB phase, there is an “outer” balance between the exploration phase regret andthe UCB phase regret, both associated with θ 0 -learning.

Original: This outer balance is regulated by the valueof β , which should be set differently under Case (A) and Case (B) to achieve the optimal balance.
Revised: This outer balance is regulated by the valueof β , which should be set differently under Case (A) and Case (B) to achieve the optimal balance.

Original: Secondly, we analyze the regret for the Inner UCB Algorithm.
Revised: Secondly, we analyze the regret for the Inner UCB Algorithm.

Original: By deﬁning the discrete best prices ˜
Revised: By deﬁning the discrete best prices ˜

Original: p ∗ t := arg max p ∈S t p (1 − F ( p − x (cid:62) t θ 0 )) in S t , the time t regret r t can be decomposed as ˜ p
Revised: p ∗ t := arg max p ∈S t p (1 − F ( p − x (cid:62) t θ 0 )) in S t , the time t regret r t can be decomposed as ˜ p

Original: [Equation]
Revised: [Equation]

Original: t

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: F
Revised: p

Original: [Equation]
Revised: [Equation]

Original: p
Revised: x

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: x
Revised: − p t (

Original: [Equation]
Revised: [Equation]

Original: t
Revised: F ( p

Original: [Equation]
Revised: [Equation]

Revised: x

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: We refer to R T 0 , 1 = (cid:80) T 0 t =1 r t, 1 and R T 0 , 2 = (cid:80) T 0 t =1 r t, 2 as the discrete-part and continuous-partregrets.
Revised: We refer to R T 0 , 1 = (cid:80) T 0 t =1 r t, 1 and R T 0 , 2 = (cid:80) T 0 t =1 r t, 2 as the discrete-part and continuous-partregrets.

Original: By the PLB formulation of the pricing problem, there is a correspondence between candidateprices and PLB actions.
Revised: By the PLB formulation of the pricing problem, there is a correspondence between candidateprices and PLB actions.

Original: Thus the discrete-part regret is equivalent to the PLB regret.
Revised: Thus the discrete-part regret is equivalent to the PLB regret.

Original: To bound the [Table] discrete-part regret of the Inner UCB Algorithm, we prove the following Proposition 1 based on
Revised: To bound thediscrete-part regret of the Inner UCB Algorithm, we prove the following Proposition 1 based on

Original: Theorem 1 in [35].
Revised: Theorem 1 in [35].

Original: Proposition 1 Under Assumptions 1 and 6, there exists positive constants C (cid:48) 1 , C [Equation]
Revised: Proposition 1 Under Assumptions 1 and 4, there exists positive constants C (cid:48) 1 , C (cid:48) 2 and C (cid:48)

Original: 2 such that withprobability at least 1 − 1 T 0 , the Inner UCB Algorithm yields a discrete-part regret
Revised: 3 such thatwith probability at least 1 − 1 T 0 , the Inner UCB Algorithm yields a discrete-part regret

Original: [Equation]
Revised: [Equation]

Original: In Proposition 1, the ﬁrst ˜ O ( d √ T 0 ) component is typical in linear bandit and attributes to the lack ofknowledge for F and the central parameter ξ ∗ in the PLB formulation.
Revised: In Proposition 1, the ﬁrst ˜ O ( d √ T 0 ) component is typical in linear bandit and attributes to the lack ofknowledge for F and the central parameter ξ ∗ in the PLB formulation.

Original: The second L || ˆ θ − θ 

Revised: The second L || ˆ θ − θ

Original: || 1 T [Equation] component is proportional to the PLB perturbation constant 2 L
Revised: || 1 T 0component is proportional to the PLB perturbation constant 2 L

Original: || ˆ θ − θ 

Revised: || ˆ θ − θ

Original: || .
Revised: || .

Original: It demonstrates howestimation errors inﬂuence the regret upper bounds, which matches with the intuition that a betterestimation would incur a lower regret in the UCB phase. 
Revised: It demonstrates howestimation errors inﬂuence the regret upper bounds, which matches with the intuition that a betterestimation would incur a lower regret in the UCB phase. [Table]

Original: The discretization number d plays a critical role in balancing the discrete-part and continuous-partregret.
Revised: The discretization number d plays a critical role in balancing the discrete-part and continuous-partregret.

Original: A larger d leads to a higher discrete-part regret as indicated by Proposition 1, which is intuitivesince more discretizations yield more candidate prices and hence a more challenging search process.
Revised: A larger d leads to a higher discrete-part regret as indicated by Proposition 1, which is intuitivesince more discretizations yield more candidate prices and hence a more challenging search process.

Original: On the other hand, a larger d and a denser discretization would make the discrete best price “closer”to the overall best price and decrease the continuous-part regret.
Revised: On the other hand, a larger h and a denser discretization would make the discrete best price “closer”to the overall best price and decrease the continuous-part regret.

Original: Speciﬁcally, under Case (A) and
Revised: Speciﬁcally, under Case (A) and

Original: Case (B) , we can bound the continuous-part regret with the order O
Revised: Case (B) , we can bound the continuous-part regret with the order O

Original: ( T 0 d 2 ) and O ( T 0 d ) respectively.
Revised: ( T 0 d 2 ) and O ( T 0 d ) respectively.

Original: Namely, we could achieve a lower rate with the extra concavity Assumption 7.
Revised: Namely, we could achieve a lower rate with the extra 2nd-order smoothness Assumption 5.

Original: Since we choose thediscretization number d k in episode k to scale as ( u k ) γ where u k denotes the UCB phase length, γregulates the balance between the discrete-part and continuous-part regret in the UCB phase andshould be set differently for optimal balances in Case (A) and Case (B) .
Revised: Since wechoose the discretization number d k in episode k to scale as ( u k ) γ where u k denotes the UCB phaselength, γ regulates the balance between the discrete-part and continuous-part regret in the UCB phaseand should be set differently for optimal balances in Case (A) and Case (B) .

Original: Now we are ready to present the two regret upper bounds for Case (A) and Case (B) .
Revised: Now we are ready to present the two regret upper bounds for Case (A) and Case (B) .

Original: Theorem 1 Under Assumptions 1 – 7, by choosing β = 23 and γ = 16 in Algorithm 1, the expectedregret satisﬁes E ( R T ) =
Revised: Theorem 1 Under Assumptions 1 – 5, by choosing β = 23 and γ = 16 in Algorithm 1, the expectedregret satisﬁes E ( R T ) =

Revised: ˜ O ( d 20 T 2 / 3 ) =

Original: ˜ O ( T 2 / 3 ) .
Revised: ˜ O ( T 2 / 3 ) .

Original: Theorem 2 Under Assumptions 1 – 6, by choosing β = 34 and γ = 14 in Algorithm 1, the expectedregret satisﬁes E ( R T ) = 
Revised: Theorem 2 Under Assumptions 1 – 4, by choosing β = 34 and γ = 14 in Algorithm 1, the expectedregret satisﬁes E ( R T ) = ˜ O ( d 0 T 3 / 4 ) =

Original: ˜ O ( T 3 / 4 ) .
Revised: ˜ O ( T 3 / 4 ) .

Original: We illustrate the regret components’ orders in Table 2 for a single episode with a generic length T 0 .
Revised: We illustrate the regret components’ orders in Table 2 for a single episode with a generic length T 0 .

Original: Table 2 explains the different choices of β and γ in the two cases to minimize the overallregret rates.
Revised: Table 2 explains the different choices of β and γ in the two cases to minimize the overall regret rates.

Original: Itclearly demonstrates that β balances between the exploration phase regret and thediscrete-part regret due to θ 0 estimation error; while γ balances between the continuous-part regretand the discrete-part regret due to F -learning.
Revised: It demonstrates that β balances between the exploration phase regret and the discrete-part regret dueto θ 0 estimation error; while γ balances between the continuous-part regret and the discrete-part regretdue to F -learning.

Original: As shown in Theorem 1, through optimal balance, ExUCB improves the existing regret of ˜ O
Revised: As shown in Theorem 1, through optimal balance, ExUCB improves the existingregret of ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) in [35] and ˜ O ( T 5 / 7 ) in [23] to ˜ O
Revised: ( T 2 / 3 ∨ (1 − α ) ) in [35] and ˜ O ( T 5 / 7 ) in [20] to ˜ O

Original: ( T 2 / 3 )under Lipschitz and concavity assumptions.
Revised: ( T 2 / 3 ) under Lipschitz and 2nd-ordersmoothness assumptions.

Original: In addition, an optimal choice of β and γ in Theorem 2helps ExUCB match the best existing regret of ˜ O
Revised: In addition, an optimal choice of β and γ in Theorem 2 helps ExUCBmatch the best existing regret of ˜ O

Original: ( T 3 / 4 ) [48] under the Lipschitz-only assumption.
Revised: ( T 3 / 4 ) [48] under the Lipschitz-only assumption.

Original: 5.2 Lower Bound
Revised: 5.2 Lower Bound

Original: We next prove a regret lower bound of ˜Ω( T 3 / 5 ) for Case (A) .
Revised: We next prove a regret lower bound of ˜Ω( T 3 / 5 ) for Case (A) .

Original: To our limited knowledge, this is theﬁrst lower bound result under the Lipschitz and concavity assumptions.
Revised: To our limited knowledge, this is theﬁrst lower bound result under the Lipschitz and 2nd-order smoothness assumptions.

Original: Note that there is a gapbetween our proved upper and lower bounds for Case (A) .
Revised: Note that thereis a gap between our proved upper and lower bounds for Case (A) .

Original: Indeed, such a gap also happens in otherworks [23, 48] on the unknown noise pricing problems.
Revised: Indeed, such a gap also happensin other works [20, 48] on the unknown noise pricing problems.

Original: This may be due to the inherent difﬁcultiesof these problems, e.g., in learning both θ 0 and F .
Revised: This may be due to the inherentdifﬁculties of these problems, e.g., in learning both θ 0 and F .

Original: Theorem 3 For any δ > 0 , no policy can achieve an O ( T 3 / 5 − δ ) regret for the dynamic pricingproblem under Assumptions 1 – 7.
Revised: Theorem 3 For any δ > 0 , no policy can achieve an O ( T 3 / 5 − δ ) regret for the dynamic pricingproblem under Assumptions 1 – 5.

Original: Remark 

Revised: Remark

Original: In [48], the authors proved an ˜Ω( T 2 / 3 ) lower bound with constructed instances thatcan ﬁt into Case (B) but does not apply to Case (A) .
Revised: In [48], the authors proved an ˜Ω( T 2 / 3 ) lower bound with constructed instances thatcan ﬁt into Case (B) but does not apply to Case (A) .

Original: Our proved lower bound rate is lower sinceour instances need to satisfy more assumptions and lie in a more benign class.
Revised: Our proved lower bound rate is lower sinceour instances need to satisfy more assumptions and lie in a more benign class.

Original: Another similarlower bound of Ω( T 3 / 5 ) is proved in [15] for 1 -dimensional nonparametric contextual expectedrevenue functions.
Revised: Another similarlower bound of Ω( T 3 / 5 ) is proved in [13] for 1 -dimensional nonparametric contextual expectedrevenue functions.

Original: However, their instances are constructed in a local-bin fashion with respect tothe covariate and thus are outside our revenue function class.
Revised: However, their instances are constructed in a local-bin fashion with respect tothe covariate and thus are outside our revenue function class.

Original: Therefore, their lower bound does notapply to our setting.
Revised: Therefore, their lower bound does notapply to our setting.

Original: Proof Sketch of Theorem 3.
Revised: Proof Sketch of Theorem 3.

Original: We follow similar ideas in [31, 48] to construct the instances.
Revised: We follow similar ideas in [28, 48] to construct the instances.

Original: Firstlywe set θ 0 = 0 and ignore the difﬁculty from contexts.
Revised: Firstlywe set θ 0 = 0 and relieve the difﬁculty from contexts.

Original: Namely, the instances we construct are indeednon-contextual.

Original: Secondly, we construct a series of “bump towers” andthen transform them to validexpected revenue functions in the form of p (1 − F ( p )) , while still preserving the intended properties.
Revised: Secondly, we construct a series of “bump towers” and transform them to valid expected revenue functions in the form of p (1 − F ( p )) , whilestill preserving the intended properties.

Original: To construct the series of “bump towers”, we construct  an inﬁnite number of inﬁnitely-nested intervalseries [0 , 1] =
Revised: We construct each bump tower from an inﬁnitely-nestedinterval series [0 , 1] =

Original: [ a 0 , b 0 ] ⊃ [ a 1 , b 1 ] ⊃ · · · ⊃ [ a k , b k ] ⊃ . . . .
Revised: [ a 0 , b 0 ] ⊃ [ a 1 , b 1 ] ⊃ · · · ⊃ [ a k , b k ] ⊃ . . . .

Original: Speciﬁcally, we divide [ a k , b k ] with lengthw k = 3 − k !
Revised: Speciﬁcally, we divide [ a k , b k ]with length w k = 3 − k !

Original: into three same-length sub-intervals and further divide the middle sub-interval into w k w k +1candidate intervals.
Revised: into three same-length sub-intervals and further divide the middle one intow k w k +1 candidate intervals.

Original: Then each of these candidate intervals forms one version of [ a k +1 , b k +1 ] .
Revised: Then each of the candidate intervals forms one case of [ a k +1 , b k +1 ] .

Original: Foreach of the inﬁnitely-nested interval series, we add up bump functions on all these nested intervals.
Revised: Foreach of these inﬁnitely-nested interval series, we add up bump functions on the nested intervals toform the bump tower.

Original: Different from [48], to construct the expected revenue functions that satisfy the concavity assumption,we adopt a different basic bump function and develop a “quadratically-shrinking” adding pattern ofthe bump functions.
Revised: Different from [48], to construct the expected revenue functions that satisfythe 2nd-order smoothness assumption, we adopt a different basic bump function and develop a“quadratically-shrinking” adding pattern.

Original: With these constructions, we prove that any policy will miss the peaks of somerevenue function instances often enough, thus accumulating an inevitable amount of the regret.
Revised: Finally, we prove that any policy will miss the peaks ofsome revenue function instances often enough, thus accumulating an inevitable amount of the regret.

Original:  Numerical Experiments
Revised:  Numerical Experiments

Original: We conduct numerical experiments to support our theoretical regret bounds of ExUCB under both
Revised: We conduct numerical experiments to support our theoretical regret bounds of ExUCB under both

Original: Case (A) and Case (B) .
Revised: Case (A) and Case (B) .

Original: We consider a total horizon length T =
Revised: We consider a total horizon length T =

Original: (cid:80) 10 i =1 2 

Revised: (cid:80) 10 i =1 2

Original: + i that is divided into 10episodes with the ﬁrst episode length
Revised: + i that is divided into 10episodes with the ﬁrst episode length

Original: α 1 = 2 9 .
Revised: α 1 = 2 9 .

Original: For both cases, we specify the linear parameterθ 0 = 30 and sample the i.i.d.
Revised: For both cases, we specify the linear parameterθ 0 = 30 and sample the i.i.d.

Original: covariates as x t ∼ Unif (1 / 2 , 1) .
Revised: covariates as x t ∼ Unif (1 / 2 , 1) .

Original: For Case (A) , the noise distributionis set as a Uniform mixture 34 Unif ( − 15 , 0) + 14 Unif (0 , 15) ; while for Case (B) , we adopt another
Revised: For Case (A) , the noise distributionis set as a Uniform mixture 34 Unif ( − 15 , 0) + 14 Unif (0 , 15) ; while for Case (B) , we adopt another

Original: Uniform mixture 14 Unif ( − 15 , 0) + 34 Unif (0 , 15) .
Revised: Uniform mixture 14 Unif ( − 15 , 0) + 34 Unif (0 , 15) .

Original: It can be veriﬁed that the ﬁrst distribution satisﬁesthe Lipschitz and concavity assumptions while the second one only satisﬁes the Lipschitz assumption.
Revised: It can be veriﬁed that the ﬁrst distribution satisﬁesthe Lipschitz and 2nd-order smoothness assumptions while the second one only satisﬁes the Lipschitzassumption.

Original: We apply ExUCB with different choices of β, γ speciﬁed in Theorems 1 – 2 to these two instances.
Revised: We apply ExUCB with different choices of β, γ speciﬁed in Theorems 1 – 2 to these twoinstances.

Original: Forboth cases, we set the constants in Algorithm 1 as p max = 50 , B = 50 , C 1 = 1 , C 2 = 20 , λ = 0 .
Revised: For both cases, we set the constants in Algorithm 1 as p max = 50 , B = 50 , C 1 = 1 , C 2 =20 , λ = 0 .

Original: With 100 replications, we plot the log-log scale of average accumulative regrets versus the timeperiods in Figure 2 along with the 95% conﬁdence intervals.
Revised: With 100 replications, we plot the log-log scale of average accumulative regrets versusthe time periods in Figure 2 along with the 95% conﬁdence intervals.

Original: The linear ﬁts extract a slope of 0 .
Revised: The linear ﬁts extract a slopeof 0 .

Original: 670for Case (A) and a slope of 0 .
Revised: 670 for Case (A) and a slope of 0 .

Original: 724 for Case (B) , which indicates that our proved regrets of ˜ O ( T 2 / 3 )for Case (A) and ˜ O
Revised: 724 for Case (B) , which indicates that our proved regrets of˜ O ( T 2 / 3 ) for Case (A) and ˜ O

Original: ( T 3 / 4 ) for Case (B) are sharp.
Revised: ( T 3 / 4 ) for Case (B) are sharp.

Original: [Figure]
Revised: [Figure]

Revised:  Conclusion

Revised: In this paper, we introduce a novel Explore-then-UCB strategy to tackle the contextual dynamicpricing problem with unknown linear valuation and unknown nonparametric noise distribution F .

Original: Checklist
Revised: Under Lipschitz and 2nd-order smoothness assumptions on F , ExUCB policy improves the bestknown regret upper bounds to ˜ O ( T 2 / 3 ) . Under the Lipschitz-only assumption, ExUCB matches thebest existing regret of ˜ O ( T 3 / 4 ) with better computational efﬁciency.

Revised: In addition, we prove a ﬁrst˜ O ( T 3 / 5 ) lower bound for our considered contextual dynamic pricing problem under the Lipschitzand 2nd-order smoothness assumptions.

Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?

Original: [Yes] We summarized our main contributions at the end of
Revised: [Yes] We summarized our main contributions at the end of

Original: Section 1.
Revised: Section 1.

Original: To support these results, we rigorously speciﬁed the problem settings,
Revised: To support these results, we rigorously speciﬁed the problem settings

Original:  pre-
Revised: , pre-

Original: sented the algorithms and theoretical analysis, and conducted numerical experiments.
Revised: sented the algorithms and theoretical analysis, and conducted numerical experiments.

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] In Section 5.2, we discussed the
Revised: [Yes] In Section 5.2, we discussed the

Original: gap between our proved upper and lower bounds.
Revised: gap between our proved upper and lower bounds.

Original: It is still an open problem to further
Revised: It is still an open problem to further

Original: close the gaps.
Revised: close the gaps.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] InAppendix C, we discussed the issue of pricing discriminations.
Revised: [Yes] InAppendix D, we discussed the issue of pricing discriminations.

Original: However, this issue
Revised: However, this issue

Original: naturally exists and our algorithms would not worsen the cases.
Revised: naturally exists and our algorithms would not worsen the cases.

Original: In fact, our proposed
Revised: In fact, our proposed

Original: algorithms may help identify the right prices for both buyers and sellers, and thus
Revised: algorithms may help identify the right prices for both buyers and sellers, and thus

Original: improve the market efﬁciency.
Revised: improve the market efﬁciency.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes] Our numerical experiments only used simulated data.
Revised: [Yes] Our numerical experiments only used simulated data.

Original: The simulated data
Revised: The simulated data

Original: has no relations with natural sciences or human subjects.
Revised: has no relations with natural sciences or human subjects.

Original: The numerical simulation is
Revised: The numerical simulation is

Original: only for validating our theoretical results.
Revised: only for validating our theoretical results.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] We stated a
Revised: [Yes] We stated a

Original: full set of seven assumptions in Section 3 “Technical Assumptions”.
Revised: full set of seven assumptions in Section 3 “Technical Assumptions”.

Original: In each of ourlemma, proposition and theorems, we stated the exact assumptions it requires among these seven assumptions.(b)
Revised: In each of ourlemma, proposition and theorems, we stated the exact assumptions it requires amongthese seven assumptions.

Original:  Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] In Appendix A, wecompletely proved all theoretical results we claim.3.
Revised: [Yes] In Appendix A, we completely proved all theoretical results we claim.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] We included
Revised: [Yes] We included

Original: all codes in the supplementary material.
Revised: all codes in the supplementary material.

Original: The codes are accompanied with instantexplanations.
Revised: The codes are accompanied with instant explanations.

Original: We also provided a “Readme” document with instructions on running the
Revised: We also provided a “Readme” document with instructions on running the

Original: codes to reproduce our results.
Revised: codes to reproduce our results.

Original: There is no input data for our codes as we simulated the
Revised: There is no input data for our codes as we simulated the

Original: data in our codes.
Revised: data in our codes.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] As we considered online learning problems, we did not pre-train
Revised: [Yes] As we considered online learning problems, we did not pre-train

Original: the algorithms.
Revised: the algorithms.

Original: However, we did specify the hyperparameters and the way they were
Revised: However, we did specify the hyperparameters and the way they were

Original: chosen in Section 6.
Revised: chosen in Section 6.

Original: (c) Did you report error bars (e g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] We report all error bars with 0.95 coverage by using the
Revised: [Yes] We report all error bars with 0.95 coverage by using the

Original: 0.025 and 0.975 quantiles of the 100 replication results.
Revised: 0.025 and 0.975 quantiles of the 100 replication results.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] We ran all numerical experiments
Revised: [Yes] We ran all numerical experiments

Original: on a laptop.
Revised: on a laptop.

Original: We reported the time consumed for each replication in our codes.
Revised: We reported the time consumed for each replication in our codes.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: We did not use
Revised: We did not use

Original: existing assets.
Revised: existing assets.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: We did not use crowdsourcing or conduct research with human
Revised: We did not use crowdsourcing or conduct research with human

Original: subjects.
Revised: subjects.

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A
Revised: [N/A

Original: c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation

Original: [N/A
Revised: N /A