Original: Contextual Dynamic Pricing with Unknown Noise: Explore-then-UCB Strategy and Improved Regrets
Revised: Contextual Dynamic Pricing with Unknown Noise: Explore-then-UCB Strategy and Improved Regrets

Original: Abstract
Revised: Abstract

Original: Dynamic pricing is a fast-moving research area in machine learning and operations management.
Revised: Dynamic pricing is a fast-moving research area in machine learning and operations management.

Original: A lot of work has been done for this problem with known noise.
Revised: A lot of work has been done for this problem with known noise.

Original: In this paper, we consider a contextual dynamic pricing problem under a linear customer valuation model with an unknown market noise distribution F .
Revised: In this paper, we consider a contextual dynamic pricing problem under a linear customer valuation model with an unknown market noise distribution F .

Original: This problem is very challenging due to the difﬁculty in balancing three tangled tasks of revenue-maximization, estimating the linear valuation parameter θ 0 , and learning the nonparametric F .
Revised: This problem is very challenging due to the difﬁculty in balancing three tangled tasks of revenue-maximization, estimating the linear valuation parameter θ 0 , and learning the nonparametric F .

Original: To address this issue, we develop a novel Explore-then-UCB (ExUCB) strategy that includes an exploration for θ 0 -learning and a followed UCB procedure of joint revenue-maximization and F -learning.
Revised: To address this issue, we develop a novel Explore-then-UCB (ExUCB) strategy that includes an exploration for θ 0 -learning and a followed UCB procedure of joint revenue-maximization and F -learning.

Original: Under Lipschitz and 2nd-order smoothness assumptions on F , ExUCB is the ﬁrst approach to achieve the ˜ O ( T 2 / 3 ) regret rate.
Revised: Under Lipschitz and 2nd-order smoothness assumptions on F , ExUCB is the ﬁrst approach to achieve the ˜ O ( T 2 / 3 ) regret rate.

Original: Under the Lipschitz assumption only, ExUCB matches the best existing regret of ˜ O
Revised: Under the Lipschitz assumption only, ExUCB matches the best existing regret of ˜ O

Original: ( T 3 / 4 )
Revised: ( T 3 / 4 )

Original: and is computationally more efﬁcient.
Revised: and is computationally more efﬁcient.

Original: Furthermore, for regret lower bounds under the nonparametric F , not much work has been done beyond only assuming Lipschitz.
Revised: Furthermore, for regret lower bounds under the nonparametric F , not much work has been done beyond only assuming Lipschitz.

Original: To ﬁll this gap, we provide the ﬁrst ˜Ω( T 3 / 5 ) lower bound under Lipschitz and 2nd-order smoothness assumptions.
Revised: To ﬁll this gap, we provide the ﬁrst ˜Ω( T 3 / 5 ) lower bound under Lipschitz and 2nd-order smoothness assumptions.

Original:  Introduction
Revised:  Introduction

Original: Dynamic pricing is a process of continuously adjusting the prices by learning from the customers’ feedback.
Revised: Dynamic pricing is a process of continuously adjusting the prices by learning from the customers’ feedback.

Original: The feedback usually depends on the pricing action.
Revised: The feedback usually depends on the pricing action.

Original: To maximize the overall revenues in a sales horizon, a pricing policy should well balance between learning the customers’ demands (exploration) and setting revenue-maximizing prices based on the current knowledge (exploitation).
Revised: To maximize the overall revenues in a sales horizon, a pricing policy should well balance between learning the customers’ demands (exploration) and setting revenue-maximizing prices based on the current knowledge (exploitation).

Original: There is a rich literature on this information-regret tradeoff under different settings [7, 27, 16, 12, 19].
Revised: There is a rich literature on this information-regret tradeoff under different settings [7, 27, 16, 12, 19].

Original: In this paper, we consider an important setting in dynamic pricing where some contextual information is available in each time period.
Revised: In this paper, we consider an important setting in dynamic pricing where some contextual information is available in each time period.

Original: It is interesting and challenging to improve the revenues by well exploiting the sales-relevant contextual information such as product features and market environments.
Revised: It is interesting and challenging to improve the revenues by well exploiting the sales-relevant contextual information such as product features and market environments.

Original: To model the inﬂuence of the contextual information, we adopt a binary feedback model which incorporates a comparison between the customer’s valuation and the seller’s price [3, 25, 47, 20, 35].
Revised: To model the inﬂuence of the contextual information, we adopt a binary feedback model which incorporates a comparison between the customer’s valuation and the seller’s price [3, 25, 47, 35, 20].

Original: [Table]
Revised: [Table]

Original: Speciﬁcally, in the selling time period t with the associated context x t ∈ R d 0 , the customer’s valuation v t of the product is assumed to be linear with repect to x t , together with some noise z t , i.e., v t = v ( x t ) =
Revised: Speciﬁcally, in the selling time period t with the associated context x t ∈ R d 0 , the customer’s valuation v t of the product is assumed to be linear with repect to x t , together with some noise z t , i.e., v t = v ( x t ) =

Original: x (cid:62) t θ 

Revised: x (cid:62) t θ

Original: + z t .
Revised: + z t .

Original: Here the noises { z t } t ∈ [ T ] across the time horizon [ T ] = { 1 , . . .
Revised: Here the noises { z t } t ∈ [ T ] across the time horizon [ T ] = { 1 , . . .

Original: , T } are independent and identically distributed (i.i.d.) from a Cumulative Distribution Function (CDF)
Revised: , T } are independent and identically distributed (i.i.d.) from a Cumulative Distribution Function (CDF)

Original: F .
Revised: F .

Original: Then under the seller’s price p t , a binary feedback y t = 1 { v t ≥ p t } representing the customer’s purchasing decision is observed by the seller.
Revised: Then under the seller’s price p t , a binary feedback y t = 1 { v t ≥ p t } representing the customer’s purchasing decision is observed by the seller.

Original: Namely, the purchase at time period t happens if and only if the customer’s valuation v t is greater than or equal to the seller’s price p t .
Revised: Namely, the purchase at time period t happens if and only if the customer’s valuation v t is greater than or equal to the seller’s price p t .

Original: The seller then collects the current feedback y t which can help the pricing decision at the next time period.
Revised: The seller then collects the current feedback y t which can help the pricing decision at the next time period.

Original: Note that the binary feedback structure y t = 1 { v t ≥ p t } is critical to this online pricing problem’s bandit nature.
Revised: Note that the binary feedback structure y t = 1 { v t ≥ p t } is critical to this online pricing problem’s bandit nature.

Original: We cannot observe the full information v t at each time period.
Revised: We cannot observe the full information v t at each time period.

Original: Instead, we can only observe the partial information y t
Revised: Instead, we can only observe the partial information y t

Original: = 1 { v t ≥ p t } which varies with different pricing action p t .
Revised: = 1 { v t ≥ p t } which varies with different pricing action p t .

Original: At each time period t , the expected reward of any price p is E (cid:0) p 1 { v t ≥ p } (cid:1) .
Revised: At each time period t , the expected reward of any price p is E (cid:0) p 1 { v t ≥ p } (cid:1) .

Original: Thus, the optimal pricing depends on the distribution of the valuation v t = v ( x t ) =
Revised: Thus, the optimal pricing depends on the distribution of the valuation v t = v ( x t ) =

Original: x (cid:62) t θ 

Revised: x (cid:62) t θ

Original: + z t .
Revised: + z t .

Original: We assume both unknown parameter θ 0 and unknown noise CDF F at the beginning of the horizon.
Revised: We assume both unknown parameter θ 0 and unknown noise CDF F at the beginning of the horizon.

Original: Therefore, the seller needs to gradually learn both θ 0 and F for better pricing.
Revised: Therefore, the seller needs to gradually learn both θ 0 and F for better pricing.

Original: In this paper, we investigate two different smoothness assumptions for the noise CDF F .
Revised: In this paper, we investigate two different smoothness assumptions for the noise CDF F .

Original: The ﬁrst one assumes Lipschitz continuity and 2nd-order smoothness, while the second only assumes Lipschitz.
Revised: The ﬁrst one assumes Lipschitz continuity and 2nd-order smoothness, while the second only assumes Lipschitz.

Original: To tackle these two sub-problems under different noise smoothness assumptions, we propose two sub-policies uniﬁed under a single pricing policy framework, namely Explore then Upper Conﬁdence
Revised: To tackle these two sub-problems under different noise smoothness assumptions, we propose two sub-policies uniﬁed under a single pricing policy framework, namely Explore then Upper Conﬁdence

Original: Bound (ExUCB).
Revised: Bound (ExUCB).

Original: In particular, our proposed ExUCB ﬁrst estimates
Revised: In particular, our proposed ExUCB ﬁrst estimates

Original: θ 0 through some random pricing exploration phase, and then uses this estimate to drive a Upper Conﬁdence Bound (UCB) procedure that well balances revenue-maximizing and F -learning.
Revised: θ 0 through some random pricing exploration phase, and then uses this estimate to drive a Upper Conﬁdence Bound (UCB) procedure that well balances revenue-maximizing and F -learning.

Original: The same pricing problem has been investigated in the literature [35, 20, 48] under a variety of smoothness assumptions on the noise.
Revised: The same pricing problem has been investigated in the literature [35, 20, 48] under a variety of smoothness assumptions on the noise.

Original: The existing regret bounds are presented in Table 1 along with our new regret results in this paper.
Revised: The existing regret bounds are presented in Table 1 along with our new regret results in this paper.

Original: In summary, our theoretical contributions are threefold.
Revised: In summary, our theoretical contributions are threefold.

Original: Under the Lipschitz and 2nd-order smoothness assumptions, our proposed ExUCB policy is the ﬁrst procedure to achieve ˜ O
Revised: Under the Lipschitz and 2nd-order smoothness assumptions, our proposed ExUCB policy is the ﬁrst procedure to achieve ˜ O

Original: ( T 2 / 3 ) regret rate.
Revised: ( T 2 / 3 ) regret rate.

Original: We prove valid θ 0 estimation accuracy and quantify its inﬂuence on the regret of the UCB procedure driven by the θ 0 -estimate.
Revised: We prove valid θ 0 estimation accuracy and quantify its inﬂuence on the regret of the UCB procedure driven by the θ 0 -estimate.

Original: This helps us determine an optimal balance between random pricing exploration and the followed UCB phase.
Revised: This helps us determine an optimal balance between random pricing exploration and the followed UCB phase.

Original: Our ˜ O ( T 2 / 3 ) regret improves the existing regret bound of ˜ O
Revised: Our ˜ O ( T 2 / 3 ) regret improves the existing regret bound of ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) in [35] which has an indeterministic α .
Revised: ( T 2 / 3 ∨ (1 − α ) ) in [35] which has an indeterministic α .

Original: It also improves the ˜ O
Revised: It also improves the ˜ O

Original: ( T 5 / 7 ) regret in [20] which assumes a stronger smoothness assumption of twice continuously differentiable F . 2.
Revised: ( T 5 / 7 ) regret in [20] which assumes a stronger smoothness assumption of twice continuously differentiable F . 2.

Original: Under the Lipschitz and 2nd-order smoothness assumptions, we obtain the lower bound of ˜Ω( T 3 / 5 ) by constructing the instances such that any policy cannot perform well on all of them.
Revised: Under the Lipschitz and 2nd-order smoothness assumptions, we obtain the lower bound of ˜Ω( T 3 / 5 ) by constructing the instances such that any policy cannot perform well on all of them.

Original: To our limited knowledge, this is the ﬁrst lower bound result under such smoothness assumptions.
Revised: To our limited knowledge, this is the ﬁrst lower bound result under such smoothness assumptions.

Original: To construct instances that satisfy this smoothness assumption, the core step is to build “bump towers” by piling up an inﬁnite series of “quadratically shrinking” basic bump functions.
Revised: To construct instances that satisfy this smoothness assumption, the core step is to build “bump towers” by piling up an inﬁnite series of “quadratically shrinking” basic bump functions.

Original: Note that the ˜Ω( T 2 / 3 ) lower bound in [48] only applies to the Lipschitz assumption since their constructed instances do not satisfy the 2nd-order smoothness assumption.
Revised: Note that the ˜Ω( T 2 / 3 ) lower bound in [48] only applies to the Lipschitz assumption since their constructed instances do not satisfy the 2nd-order smoothness assumption.

Original: Under the Lipschitz assumption, ExUCB can match the best existing upper bound of ˜ O
Revised: Under the Lipschitz assumption, ExUCB can match the best existing upper bound of ˜ O

Original: ( T 3 / 4 ) in [48] and is computationally more efﬁcient.
Revised: ( T 3 / 4 ) in [48] and is computationally more efﬁcient.

Original: This shows the adaptivity of ExUCB to different noise smoothness levels.
Revised: This shows the adaptivity of ExUCB to different noise smoothness levels.

Original: Our improved regret over existing results in [35, 20] demonstrates the methodological novelty of the proposed Explore-then-UCB strategy.
Revised: Our improved regret over existing results in [35, 20] demonstrates the methodological novelty of the proposed Explore-then-UCB strategy.

Original: In [35], the authors implemented the UCB idea but did not apply random pricing explorations.
Revised: In [35], the authors implemented the UCB idea but did not apply random pricing explorations.

Original: Thus they can only use adaptive data to estimate θ 0 , which results in an indeterministic α in their regret of ˜ O
Revised: Thus they can only use adaptive data to estimate θ 0 , which results in an indeterministic α in their regret of ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) .
Revised: ( T 2 / 3 ∨ (1 − α ) ) .

Original: In contrast, by using a random exploration phase, ExUCB achieves the exact regret of ˜ O
Revised: In contrast, by using a random exploration phase, ExUCB achieves the exact regret of ˜ O

Original: ( T 2 / 3 ) .
Revised: ( T 2 / 3 ) .

Original: In [20], the authors proposed an
Revised: In [20], the authors proposed an

Original: Explore-then-Commit type of policy that ﬁrst estimates θ 0 and F in an exploration phase and then commits to these estimates for pricing in the exploitation phase.
Revised: Explore-then-Commit type of policy that ﬁrst estimates θ 0 and F in an exploration phase and then commits to these estimates for pricing in the exploitation phase.

Original: In comparison, ExUCB imposes a UCB procedure to adaptively balance between revenue-maximizing and F -learning.
Revised: In comparison, ExUCB imposes a UCB procedure to adaptively balance between revenue-maximizing and F -learning.

Original: The regret advantage of ExUCB indicates the importance of our UCB procedure after the exploration.
Revised: The regret advantage of ExUCB indicates the importance of our UCB procedure after the exploration.

Original:  Related Works
Revised:  Related Works

Original: In this section, we discuss how our work relates to the literature of dynamic pricing, bandits and contextual search..
Revised: In this section, we discuss how our work relates to the literature of dynamic pricing, bandits and contextual search..

Original: Non-Contextual Dynamic Pricing.
Revised: Non-Contextual Dynamic Pricing.

Original: Extensive investigations have been conducted on dynamic pricing problems without contextual information [7, 9, 46, 8, 14].
Revised: Extensive investigations have been conducted on dynamic pricing problems without contextual information [7, 9, 46, 8, 14].

Original: For m -th smooth demand functions, [44] applied the UCB idea with local-bin approximations to achieve an ˜ O ( T ( m +1) / (2 m +1) ) regret, and proved a matching lower bound.
Revised: For m -th smooth demand functions, [44] applied the UCB idea with local-bin approximations to achieve an ˜ O ( T ( m +1) / (2 m +1) ) regret, and proved a matching lower bound.

Original: The UCB approach has also been adopted in [29, 37].
Revised: The UCB approach has also been adopted in [29, 37].

Original: However, these methods are not able to utilize the potential contextual information for better pricing.
Revised: However, these methods are not able to utilize the potential contextual information for better pricing.

Original: Contextual Dynamic Pricing.
Revised: Contextual Dynamic Pricing.

Original: There are signiﬁcant interests among researchers in contextual dynamic pricing [41, 24, 36, 38, 6, 43, 5, 18, 45, 26, 13, 15].
Revised: There are signiﬁcant interests among researchers in contextual dynamic pricing [41, 24, 36, 38, 6, 43, 5, 18, 45, 26, 13, 15].

Original: With nonparametric revenue functions, [13] designed a policy with ˜ O
Revised: With nonparametric revenue functions, [13] designed a policy with ˜ O

Original: ( T ( d 0 +2) /
Revised: ( T ( d 0 +2) /

Original: ( d 0 +4) ) regret based on the adaptive binning idea from nonparametric contextual bandit [40].
Revised: ( d 0 +4) ) regret based on the adaptive binning idea from nonparametric contextual bandit [40].

Original: Note that their proved Ω( T 3 / 5 ) regret lower bound for the one-dimensional case does not apply to our setting since their constructed revenue functions are beyond our revenue function class.
Revised: Note that their proved Ω( T 3 / 5 ) regret lower bound for the one-dimensional case does not apply to our setting since their constructed revenue functions are beyond our revenue function class.

Original: In [43], the authors adopted a log-linear valuation model and proposed a pricing algorithm with ˜ O
Revised: In [43], the authors adopted a log-linear valuation model and proposed a pricing algorithm with ˜ O

Original: ( T 1 / 2 ) regret.
Revised: ( T 1 / 2 ) regret.

Original: Some recent literature [25, 22, 23, 47,20, 35,  48] adopted the same linear valuation model as in this work.
Revised: Some recent literature [25, 22, 23, 47, 35, 20, 48] adopted the same linear valuation model as in this work.

Original: By assuming a known noise distribution, [25, 47] designed algorithms with O (log T ) regret.
Revised: By assuming a known noise distribution, [25, 47] designed algorithms with O (log T ) regret.

Original: For noise in a known parametric family, [25] proposed a policy with O ( T 1 / 2 ) regret.
Revised: For noise in a known parametric family, [25] proposed a policy with O ( T 1 / 2 ) regret.

Original: In [23], the authors assumed a known ambiguity set that contains the noise distribution and proposed an algorithm with a ˜ O
Revised: In [23], the authors assumed a known ambiguity set that contains the noise distribution and proposed an algorithm with a ˜ O

Original: ( T 2 / 3 ) regret with respect to a robust benchmark.
Revised: ( T 2 / 3 ) regret with respect to a robust benchmark.

Original: With unknown noise distribution, the ambiguity set would be extremely large and the robust benchmark can be far from the true optimal one.
Revised: With unknown noise distribution, the ambiguity set would be extremely large and the robust benchmark can be far from the true optimal one.

Original: In [22], the authors considered unknown noise with “full information” feedbacks and proposed an algorithm with ˜ O
Revised: In [22], the authors considered unknown noise with “full information” feedbacks and proposed an algorithm with ˜ O

Original: ( T 1 / 2 ) regret.
Revised: ( T 1 / 2 ) regret.

Original: The most related works to ours are [20, 35, 48], which considered binary cencored feedback and unknown noise distributions under different smoothness assumptions.
Revised: The most related works to ours are [35, 20, 48], which considered binary cencored feedback and unknown noise distributions under different smoothness assumptions.

Original: In [20], the authors proposed an Explore-then-Commit policy with an ˜ O
Revised: In [20], the authors proposed an Explore-then-Commit policy with an ˜ O

Original: ( T 2 m +1 4 m − 1 ) regret under m ( ≥ 2) times continuously differentiable F .
Revised: ( T 2 m +1 4 m − 1 ) regret under m ( ≥ 2) times continuously differentiable F .

Original: The m = 2 case would imply our Lipschitz and 2nd-order smoothness assumptions.
Revised: The m = 2 case would imply our Lipschitz and 2nd-order smoothness assumptions.

Original: Thus
Revised: Thus

Original: ExUCB can achieve a lower ˜ O
Revised: ExUCB can achieve a lower ˜ O

Original: ( T 2 / 3 ) regret than the ˜ O
Revised: ( T 2 / 3 ) regret than the ˜ O

Original: ( T 5 / 7 ) rate in [20] even under weaker smoothness assumptions.
Revised: ( T 5 / 7 ) rate in [20] even under weaker smoothness assumptions.

Original: In [35], the authors proved an ˜ O
Revised: In [35], the authors proved an ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) regret.
Revised: ( T 2 / 3 ∨ (1 − α ) ) regret.

Original: The value of α depends on the convergence rate of their θ 0 estimates.
Revised: The value of α depends on the convergence rate of their θ 0 estimates.

Original: However, α is indeterministic and no rigorous justiﬁcations has been made.
Revised: However, α is indeterministic and no rigorous justiﬁcations has been made.

Original: In comparison, our ExUCB policy achieves an exact regret of ˜ O
Revised: In comparison, our ExUCB policy achieves an exact regret of ˜ O

Original: ( T 2 / 3 ) .
Revised: ( T 2 / 3 ) .

Original: In [48], the authors developed an adaptive pricing policy that achieved an ˜ O ( T 3 / 4 ) regret for adversarial contexts and arbitrary bounded noise distributions.
Revised: In [48], the authors developed an adaptive pricing policy that achieved an ˜ O ( T 3 / 4 ) regret for adversarial contexts and arbitrary bounded noise distributions.

Original: Our ExUCB policy matches the ˜ O
Revised: Our ExUCB policy matches the ˜ O

Original: ( T 3 / 4 ) regret under a Lipschitz F and can improve the regret to ˜ O
Revised: ( T 3 / 4 ) regret under a Lipschitz F and can improve the regret to ˜ O

Original: ( T 2 / 3 ) with an additional 2nd-order smoothness assumption.
Revised: ( T 2 / 3 ) with an additional 2nd-order smoothness assumption.

Original: In addition, since the EXP4-based policy in [48] requires exponential computations w.r.t.
Revised: In addition, since the EXP4-based policy in [48] requires exponential computations w.r.t.

Original: the covariate dimension d 0 , ExUCB is computationally more efﬁcient with a time complexity that is polynomial with d 0 .
Revised: the covariate dimension d 0 , ExUCB is computationally more efﬁcient with a time complexity that is polynomial with d 0 .

Original: Bandit Algorithms.
Revised: Bandit Algorithms.

Original: Bandit-type feedback structure is natural in dynamic pricing [29, 13, 48].
Revised: Bandit-type feedback structure is natural in dynamic pricing [29, 13, 48].

Original: The bandit literature provides a signiﬁcant variety of methods to resolve the exploration-exploitation tradeoff that arises with the bandit feedback [10, 31].
Revised: The bandit literature provides a signiﬁcant variety of methods to resolve the exploration-exploitation tradeoff that arises with the bandit feedback [10, 31].

Original: A key tool we used in this paper is the perturbed linear bandit (PLB) [35].
Revised: A key tool we used in this paper is the perturbed linear bandit (PLB) [35].

Original: It is related to the misspeciﬁed linear bandits [32, 39, 21] and non-stationary linear bandits [17, 42, 49].
Revised: It is related to the misspeciﬁed linear bandits [32, 39, 21] and non-stationary linear bandits [17, 42, 49].

Original: In addition, dynamic pricing is closely related to continuum-armed bandit
Revised: In addition, dynamic pricing is closely related to continuum-armed bandit

Original: The lower bound we proved borrows the “needle in haystack” idea that is widely applied in continuum-armed bandits [28, 48].
Revised: The lower bound we proved borrows the “needle in haystack” idea that is widely applied in continuum-armed bandits [28, 48].

Original: Contextual Search.
Revised: Contextual Search.

Original: Contextual pricing with binary feedback can be formulated into the contextual search problem [34, 33, 30].
Revised: Contextual pricing with binary feedback can be formulated into the contextual search problem [34, 33, 30].

Original: However, different noises are considered other than our stochastic valuation ones.
Revised: However, different noises are considered other than our stochastic valuation ones.

Original: [34] is noiseless and only small-variance noises are handled in [30]; A “ﬂipping” noise on customers’ decisions are investigated in [33].
Revised: [34] is noiseless and only small-variance noises are handled in [30]; A “ﬂipping” noise on customers’ decisions are investigated in [33].

Original:  Preliminaries
Revised:  Preliminaries

Original: Problem Setting.
Revised: Problem Setting.

Original: The sales time horizon is [ T ] = { 1 , . .
Revised: The sales time horizon is [ T ] = { 1 , . .

Original: . , T } with the initial time period t = 1 .
Revised: . , T } with the initial time period t = 1 .

Original: We present the online pricing procedure as follows.
Revised: We present the online pricing procedure as follows.

Original: At time period t , the seller observes a context x t ∈ R d 0 . 2.
Revised: At time period t , the seller observes a context x t ∈ R d 0 . 2.

Original: The customer valuates the product at v t = x (cid:62) t θ 0 + z t , where z t i.i.d.
Revised: The customer valuates the product at v t = x (cid:62) t θ 0 + z t , where z t i.i.d.

Original: ∼ F . 3.
Revised: ∼ F . 3.

Original: The seller sets a price p t based on x t and the past sales data { ( x s , p s , y s ) } s ≤ t − 1 . 4.
Revised: The seller sets a price p t based on x t and the past sales data { ( x s , p s , y s ) } s ≤ t − 1 . 4.

Original: The seller observes the binary feedback y t
Revised: The seller observes the binary feedback y t

Original: = 1 { v t ≥ p t } and collects the revenue p t y t .
Revised: = 1 { v t ≥ p t } and collects the revenue p t y t .

Original: Let t = t
Revised: Let t = t

Original: + 1 and go back to Step 1.
Revised: + 1 and go back to Step 1.

Original: Regret Deﬁnition.
Revised: Regret Deﬁnition.

Original: Given the context x t , the probability of a purchase is 1 − F ( p t − x (cid:62) t θ 0 ) and thus the expected reward of setting the price p t is p t (cid:0) 1 − F ( p t − x (cid:62) t θ 0 ) (cid:1) .
Revised: Given the context x t , the probability of a purchase is 1 − F ( p t − x (cid:62) t θ 0 ) and thus the expected reward of setting the price p t is p t (cid:0) 1 − F ( p t − x (cid:62) t θ 0 ) (cid:1) .

Original: Deﬁne the optimal price given the context x as p ∗ ( x ) =
Revised: Deﬁne the optimal price given the context x as p ∗ ( x ) =

Original: arg max p ≥ 0 p (cid:0) 1 − F ( p − x (cid:62) θ 0 ) (cid:1) .
Revised: arg max p ≥ 0 p (cid:0) 1 − F ( p − x (cid:62) θ 0 ) (cid:1) .

Original: Then the regret r t at time period t is deﬁned as the expected revenue loss with respect to the optimal price p ∗ t = p ∗ ( x t ) , i.e., r t = p ∗ t (cid:0) 1 − F ( p ∗ t − x (cid:62)
Revised: Then the regret r t at time period t is deﬁned as the expected revenue loss with respect to the optimal price p ∗ t = p ∗ ( x t ) , i.e., r t = p ∗ t (cid:0) 1 − F ( p ∗ t − x (cid:62)

Original: [Equation]
Revised: [Equation]

Original: Deﬁnition 

Revised: Deﬁnition

Original: The cumulative regret across the horizon is deﬁned as
Revised: The cumulative regret across the horizon is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: The expected cumulative regret E ( R T ) is obtained from taking expectation over the potential randomness of the data and the pricing policy.
Revised: The expected cumulative regret E ( R T ) is obtained from taking expectation over the potential randomness of the data and the pricing policy.

Original: Our goal is to minimize E ( R T ) by dynamically setting the price p t under unknown θ 0 and F .
Revised: Our goal is to minimize E ( R T ) by dynamically setting the price p t under unknown θ 0 and F .

Original: Technical Assumptions.
Revised: Technical Assumptions.

Original: We now present our main assumptions.
Revised: We now present our main assumptions.

Original: Assumptions 1 – 2 are standard in dynamic pricing [25, 13, 23,20, 35,  48].
Revised: Assumptions 1 – 2 are standard in dynamic pricing [25, 13, 23, 35, 20, 48].

Original: Assumption 1 (Bounded contexts and parameter)
Revised: Assumption 1 (Bounded contexts and parameter)

Original: The covariates x t are bounded as || x t || ∞ ≤ 1 .
Revised: The covariates x t are bounded as || x t || ∞ ≤ 1 .

Original: The (cid:96) 1 norm ||
Revised: The (cid:96) 1 norm ||

Original: θ
Revised: θ

Original: || 1 of θ 0 is bounded by a known constant W .
Revised: || 1 of θ 0 is bounded by a known constant W .

Original: Assumption 2 (i.i.d. contexts)
Revised: Assumption 2 (i.i.d. contexts)

Original: The covariates x t i.i.d.
Revised: The covariates x t i.i.d.

Original: ∼ P x with the support X and the matrix Σ = E (cid:0) (1 , x (cid:62) t ) (cid:62) (1 , x (cid:62) t ) (cid:1) satisﬁes that Σ − c
Revised: ∼ P x with the support X and the matrix Σ = E (cid:0) (1 , x (cid:62) t ) (cid:62) (1 , x (cid:62) t ) (cid:1) satisﬁes that Σ − c

Original: I is positive-deﬁnite for some positive constant c 0 .
Revised: I is positive-deﬁnite for some positive constant c 0 .

Original: Assumption 3 (Bounded valuations)
Revised: Assumption 3 (Bounded valuations)

Original: The customers’ valuations v t ∈ [0 , B ] for a known constant B .
Revised: The customers’ valuations v t ∈ [0 , B ] for a known constant B .

Original: Assumption 3 assumes a known upper bound for the customers’ valuations [23, 20], which is reasonable for real-life products.
Revised: Assumption 3 assumes a known upper bound for the customers’ valuations [23, 20], which is reasonable for real-life products.

Original: Note that Assumption 3 indicates a known upper bound p max
Revised: Note that Assumption 3 indicates a known upper bound p max

Original: =
Revised: =

Original: B for the optimal prices.
Revised: B for the optimal prices.

Original: In addition, Assumptions 1 and 3 imply an upper bound U
Revised: In addition, Assumptions 1 and 3 imply an upper bound U

Original: = W + B for the noise absolute value.
Revised: = W + B for the noise absolute value.

Original: In the following, we further introduce two smoothness assumptions.
Revised: In the following, we further introduce two smoothness assumptions.

Original: The Lipschitz condition in Assumption 4 is basic and was considered in [35, 48].
Revised: The Lipschitz condition in Assumption 4 is basic and was considered in [35, 48].

Original: Assumptionassumes the 2nd-order smoothness of the expected revenue functions around the optimal prices and has also been imposed in [13, 35].
Revised: Assumptionassumes the 2nd-order smoothness of the expected revenue functions around the optimal prices and has also been imposed in [13, 35].

Original: It is satisﬁed with bounded second derivatives of F [35] but ﬁts for a broader class of F .
Revised: It is satisﬁed with bounded second derivatives of F [35] but ﬁts for a broader class of F .

Original: Both Assumptions 4 – 5 are satisﬁed by m ( ≥ 2) times continuously differentiablility of F as considered in [20].
Revised: Both Assumptions 4 – 5 are satisﬁed by m ( ≥ 2) times continuously differentiablility of F as considered in [20].

Original: Assumption 4 (Lipschitz Continuity)
Revised: Assumption 4 (Lipschitz Continuity)

Original: The noise CDF F is Lipschitz continuous with a constant L , i.e., | F ( x ) − F ( y ) | ≤ L | x − y | , ∀ x, y ∈ R .
Revised: The noise CDF F is Lipschitz continuous with a constant L , i.e., | F ( x ) − F ( y ) | ≤ L | x − y | , ∀ x, y ∈ R .

Original: Assumption 5 (2nd-order Smoothness) Deﬁne the general expected revenue functions associated with the noise distribution F as f q ( p ) =
Revised: Assumption 5 (2nd-order Smoothness) Deﬁne the general expected revenue functions associated with the noise distribution F as f q ( p ) =

Original: p (1 − F ( p − q )) .
Revised: p (1 − F ( p − q )) .

Original: There exists a positive constant C such that for any x ∈ X and q = x (cid:62) θ 0 , we have f q ( p ∗ ( x ))
Revised: There exists a positive constant C such that for any x ∈ X and q = x (cid:62) θ 0 , we have f q ( p ∗ ( x ))

Original: − f q ( p ) ≤ C ( p ∗ ( x ) − p ) 2 , ∀ p ∈ [0 , p max ] .
Revised: − f q ( p ) ≤ C ( p ∗ ( x ) − p ) 2 , ∀ p ∈ [0 , p max ] .

Original: In this paper, we investigate two smoothness levels on the unknown noise distribution F , i.e., Case (A) : Lipschitz and 2nd-order smoothness; Case (B) : Lipschitz-only.
Revised: In this paper, we investigate two smoothness levels on the unknown noise distribution F , i.e., Case (A) : Lipschitz and 2nd-order smoothness; Case (B) : Lipschitz-only.

Original: For these two scenarios, we design respective algorithms that are uniﬁed under a single ExUCB policy framework.
Revised: For these two scenarios, we design respective algorithms that are uniﬁed under a single ExUCB policy framework.

Original:  Algorithm
Revised:  Algorithm

Original: We ﬁrst propose the general ExUCB policy in Algorithm 1.
Revised: We ﬁrst propose the general ExUCB policy in Algorithm 1.

Original: Without the knowledge of the horizon length T , we utilize the doubling trick [31] to cut the horizon into episodes.
Revised: Without the knowledge of the horizon length T , we utilize the doubling trick [31] to cut the horizon into episodes.

Original: Each episode consists of an exploration phase and a followed UCB phase.
Revised: Each episode consists of an exploration phase and a followed UCB phase.

Original: Denote the ﬁrst episode length as α 1 and the number of episodes as n ( T, α 1 ) .
Revised: Denote the ﬁrst episode length as α 1 and the number of episodes as n ( T, α 1 ) .

Original: The schematic of ExUCB is displayed in Figure 1.
Revised: The schematic of ExUCB is displayed in Figure 1.

Original: Set a price p t uniformly randomly from (0 , B ) .
Revised: Set a price p t uniformly randomly from (0 , B ) .

Original: Receive a binary response y t .
Revised: Receive a binary response y t .

Original: Calculate the θ 0 -estimate ˆ θ k by (ˆ µ k , ˆ θ k ) =
Revised: Calculate the θ 0 -estimate ˆ θ k by (ˆ µ k , ˆ θ k ) =

Original: arg min µ,θ |E k | (cid:88) t ∈E k (cid:0) By t − (1 , x (cid:62) t )( µ, θ (cid:62) ) (cid:62) (cid:1) 2 .
Revised: arg min µ,θ |E k | (cid:88) t ∈E k (cid:0) By t − (1 , x (cid:62) t )( µ, θ (cid:62) ) (cid:62) (cid:1) 2 .

Original: 10: (UCB Phase) 11:
Revised: 10: (UCB Phase) 11:

Original: For time t ∈
Revised: For time t ∈

Original: U k :=
Revised: U k :=

Original: { (cid:80) k − 1 i =1 (cid:96) i + (cid:100) C 1 (cid:96) βk (cid:101) + 1 , . . .
Revised: { (cid:80) k − 1 i =1 (cid:96) i + (cid:100) C 1 (cid:96) βk (cid:101) + 1 , . . .

Original: , (cid:80) k − 1 i =1 (cid:96) i + (cid:96) k } , do 12:
Revised: , (cid:80) k − 1 i =1 (cid:96) i + (cid:96) k } , do 12:

Original: Apply the Inner UCB Algorithm on the coming sequential covariates { x t } t ∈U k with the θ 0 -estimate ˆ θ k , the discretization number d k = (cid:100) C 2 ( (cid:96) k − (cid:100) C 1 (cid:96) βk (cid:101) ) γ (cid:101) , the optimal price bound p max , the projected length ( (cid:96) k − (cid:100) C 1 (cid:96) β k (cid:101) ) , and the regularization parameter λ .
Revised: Apply the Inner UCB Algorithm on the coming sequential covariates { x t } t ∈U k with the θ 0 -estimate ˆ θ k , the discretization number d k = (cid:100) C 2 ( (cid:96) k − (cid:100) C 1 (cid:96) βk (cid:101) ) γ (cid:101) , the optimal price bound p max , the projected length ( (cid:96) k − (cid:100) C 1 (cid:96) β k (cid:101) ) , and the regularization parameter λ .

Original: In the exploration phase, we conduct random pricing and ﬁnally obtain a θ 0 estimate.
Revised: In the exploration phase, we conduct random pricing and ﬁnally obtain a θ 0 estimate.

Original: In the followed
Revised: In the followed

Original: UCB phase, we implement a UCB procedure that uses the θ 0 estimate to drive the balance between
Revised: UCB phase, we implement a UCB procedure that uses the θ 0 estimate to drive the balance between

Original: F -learning and revenue-maximizing.
Revised: F -learning and revenue-maximizing.

Original: We discuss the two components in the following.
Revised: We discuss the two components in the following.

Original: Estimation of θ 0 .
Revised: Estimation of θ 0 .

Original: At the beginning of each episode, we impose a random pricing exploration phase to generate data for θ 0 estimation.
Revised: At the beginning of each episode, we impose a random pricing exploration phase to generate data for θ 0 estimation.

Original: The exploration phase length (cid:100) C 1 (cid:96) βk (cid:101) is set as some β order of the episode length (cid:96) k .
Revised: The exploration phase length (cid:100) C 1 (cid:96) βk (cid:101) is set as some β order of the episode length (cid:96) k .

Original: By uniformly random pricing in (0 , B ) , there arises a linear regression structure [23, 20] involving the signal of θ 0 .
Revised: By uniformly random pricing in (0 , B ) , there arises a linear regression structure [23, 20] involving the signal of θ 0 .

Original: Speciﬁcally, we have
Revised: Speciﬁcally, we have

Original: [Equation]
Revised: [Equation]

Original: Thus we are able to provide guarantees for ˆ θ k using classical analysis techniques on linear regression.
Revised: Thus we are able to provide guarantees for ˆ θ k using classical analysis techniques on linear regression.

Original: In contrast, [35] applied a linear classiﬁcation method on the adaptive data of the previous episode to obtain a θ 0 -estimate, which lacks theoretical guarantees for the estimation accuracy.
Revised: In contrast, [35] applied a linear classiﬁcation method on the adaptive data of the previous episode to obtain a θ 0 -estimate, which lacks theoretical guarantees for the estimation accuracy.

Original: ˆ θ k -driven UCB Procedure.
Revised: ˆ θ k -driven UCB Procedure.

Original: In the UCB phase of episode k , we use the obtained linear regression estimate ˆ θ k to drive a UCB procedure that balances between F -learning and revenue-maximizing.
Revised: In the UCB phase of episode k , we use the obtained linear regression estimate ˆ θ k to drive a UCB procedure that balances between F -learning and revenue-maximizing.

Original: Different from [35], our UCB procedure does not start at the beginning of each episode.
Revised: Different from [35], our UCB procedure does not start at the beginning of each episode.

Original: In addition, we offer a different source of θ 0 estimate that is independent of previous episodes and allow more general discretizations tuned by a parameter γ .
Revised: In addition, we offer a different source of θ 0 estimate that is independent of previous episodes and allow more general discretizations tuned by a parameter γ .

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: The Inner UCB Algorithm is explicitly presented as Algorithm 2.
Revised: The Inner UCB Algorithm is explicitly presented as Algorithm 2.

Original: In summary, the knowledge of
Revised: In summary, the knowledge of

Original: F is continuously updated and the prices are set accordingly.
Revised: F is continuously updated and the prices are set accordingly.

Original: Since any potential optimal price p ∈ (0 , p max ) , we only need to explore those F -values on the potential range of p − x (cid:62) t θ 0 , i.e., [ −|| θ 0 || 1 , p max + || θ 

Revised: Since any potential optimal price p ∈ (0 , p max ) , we only need to explore those F -values on the potential range of p − x (cid:62) t θ 0 , i.e., [ −|| θ 0 || 1 , p max + || θ

Original: || 1 ] .
Revised: || 1 ] .

Original: Thus we ﬁrst restrict our F -learning attention to the interval G (ˆ θ ) =
Revised: Thus we ﬁrst restrict our F -learning attention to the interval G (ˆ θ ) =

Original: [ −|| ˆ θ || 1 , p max +
Revised: [ −|| ˆ θ || 1 , p max +

Original: ||
Revised: ||

Original: ˆ θ || 1 ] .
Revised: ˆ θ || 1 ] .

Original: Now we borrow the discretization approach in [35].
Revised: Now we borrow the discretization approach in [35].

Original: Speciﬁcally, we discretize
Revised: Speciﬁcally, we discretize

Original: G (ˆ θ ) into d sub-intervals and further focus the learning of F on their d midpoints { m j } j ∈ [ d ] .
Revised: G (ˆ θ ) into d sub-intervals and further focus the learning of F on their d midpoints { m j } j ∈ [ d ] .

Original: Note that we use d 0 to refer to the covariate dimensionality, and use d, d k to refer to the discretization numbers.
Revised: Note that we use d 0 to refer to the covariate dimensionality, and use d, d k to refer to the discretization numbers.

Original: This discretization idea would enable a mutual reinforcement procedure of the discretized price selection and the knowledge accumulation of F on these d discretized points.
Revised: This discretization idea would enable a mutual reinforcement procedure of the discretized price selection and the knowledge accumulation of F on these d discretized points.

Original: They can repeatedly enhance each other in a closed loop.
Revised: They can repeatedly enhance each other in a closed loop.

Original: To do so, at each time period t , we construct the candidate price set S t = { m j + x (cid:62) t
Revised: To do so, at each time period t , we construct the candidate price set S t = { m j + x (cid:62) t

Original: ˆ θ | j ∈ [ d ] , m j + x (cid:62) t ˆ θ ∈ (0 , p max ) } .
Revised: ˆ θ | j ∈ [ d ] , m j + x (cid:62) t ˆ θ ∈ (0 , p max ) } .

Original: Then for any candidate price p t = m j + x (cid:62) t θ 0 ∈ S t , its associated purchasing probability (1 − F ( p t − x (cid:62) t θ 0 )) would be close to (1 − F ( m j )) .
Revised: Then for any candidate price p t = m j + x (cid:62) t θ 0 ∈ S t , its associated purchasing probability (1 − F ( p t − x (cid:62) t θ 0 )) would be close to (1 − F ( m j )) .

Original: Therefore, the knowledge accumulation of F on { m j } j ∈ [ d ] helps with the expected revenue evaluation of the candidate prices and thus the price selection from S t .
Revised: Therefore, the knowledge accumulation of F on { m j } j ∈ [ d ] helps with the expected revenue evaluation of the candidate prices and thus the price selection from S t .

Original: On the other hand, the selection of any candidate price p t = m j + x (cid:62) t θ 0 ∈ S t invokes a binary outcome with a probability close to 1 − F ( m j ) , thus helping with the knowledge accumulation of F on m j .
Revised: On the other hand, the selection of any candidate price p t = m j + x (cid:62) t θ 0 ∈ S t invokes a binary outcome with a probability close to 1 − F ( m j ) , thus helping with the knowledge accumulation of F on m j .

Original: Denote
Revised: Denote

Original: D t − 1 ,j := { s : 1 ≤ s ≤ t − 1 , j s = j } as all past time periods s
Revised: D t − 1 ,j := { s : 1 ≤ s ≤ t − 1 , j s = j } as all past time periods s

Original: such that p s − x (cid:62) t ˆ θ = m j .
Revised: such that p s − x (cid:62) t ˆ θ = m j .

Original: Then we construct the UCB of 1 − F ( m j ) as
Revised: Then we construct the UCB of 1 − F ( m j ) as

Original: [Equation]
Revised: [Equation]

Original:  
Revised:  

Original: [Equation]
Revised: [Equation]

Original: s ∈D
Revised: s ∈D

Original: [Equation]
Revised: [Equation]

Original: (cid:114)
Revised: (cid:114)

Original: [Equation]
Revised: [Equation]

Original: λ + (cid:80) s ∈D t − 1 ,j p
Revised: λ + (cid:80) s ∈D t − 1 ,j p

Original: [Equation]
Revised: [Equation]

Original: s , for D t − 1 ,j (cid:54) = ∅ ; + ∞ , for D t − 1 ,j = ∅ .
Revised: s , for D t − 1 ,j (cid:54) = ∅ ; + ∞ , for D t − 1 ,j = ∅ .

Original: Note that for s ∈ D t − 1 ,j , y s ∼ Ber (1 − F ( p s − x (cid:62) s θ 0 ))
Revised: Note that for s ∈ D t − 1 ,j , y s ∼ Ber (1 − F ( p s − x (cid:62) s θ 0 ))

Original: ≈
Revised: ≈

Original: Ber (1 − F ( m j )) .
Revised: Ber (1 − F ( m j )) .

Original: Therefore, the ﬁrst term in the right-hand side of (1) for D t − 1 ,j (cid:54) = ∅ is a regularized weighted average of those y s and thus an estimate of 1 − F ( m j ) .
Revised: Therefore, the ﬁrst term in the right-hand side of (1) for D t − 1 ,j (cid:54) = ∅ is a regularized weighted average of those y s and thus an estimate of 1 − F ( m j ) .

Original: The second term is an associated conﬁdence interval length.
Revised: The second term is an associated conﬁdence interval length.

Original: Based on these upper conﬁdence bounds for 1 − F ( m j ) , we select the price p t = m j + x (cid:62) t ˆ θ from the candidate set
Revised: Based on these upper conﬁdence bounds for 1 − F ( m j ) , we select the price p t = m j + x (cid:62) t ˆ θ from the candidate set

Original: S t that maximizes the optimism expected revenue ( m j + x (cid:62) t ˆ θ )
Revised: S t that maximizes the optimism expected revenue ( m j + x (cid:62) t ˆ θ )

Original: UCB t (1 − F ( m j )) .
Revised: UCB t (1 − F ( m j )) .

Original: It remains to specify the choice of β t in Equation (1).
Revised: It remains to specify the choice of β t in Equation (1).

Original: We will discuss its choice by the following perturbed linear bandit [35] formulation of the pricing problem in the Inner UCB Algorithm.
Revised: We will discuss its choice by the following perturbed linear bandit [35] formulation of the pricing problem in the Inner UCB Algorithm.

Original: Perturbed Linear Bandit Formulation With the restriction on the candidate sets S t , we are able to formulate the pricing problem guided by ˆ θ as a perturbed linear bandit.
Revised: Perturbed Linear Bandit Formulation With the restriction on the candidate sets S t , we are able to formulate the pricing problem guided by ˆ θ as a perturbed linear bandit.

Original: The PLB is an extension of the linear bandit and we present its formal deﬁnition in Appendix C.
Revised: The PLB is an extension of the linear bandit and we present its formal deﬁnition in Appendix C.

Original: The expected reward of the PLB takes the form of (cid:104) ξ t , A t (cid:105) at each time period t , where A t is the action vector.
Revised: The expected reward of the PLB takes the form of (cid:104) ξ t , A t (cid:105) at each time period t , where A t is the action vector.

Original: Different from the linear bandit, the linear parameters ξ t in the PLB can vary across time periods and are perturbations from a central linear parameter ξ ∗ .
Revised: Different from the linear bandit, the linear parameters ξ t in the PLB can vary across time periods and are perturbations from a central linear parameter ξ ∗ .

Original: In our pricing problem, by specifying the linear parameter ξ t = (1 − F ( m 1 + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ) , . . .
Revised: In our pricing problem, by specifying the linear parameter ξ t = (1 − F ( m 1 + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ) , . . .

Original: , 1 −
Revised: , 1 −

Original: F ( m d + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ))
Revised: F ( m d + x (cid:62) t ˆ θ − x (cid:62) t θ 0 ))

Original: (cid:62) ∈ R d and mapping each candidate price p t = m j + x (cid:62) t
Revised: (cid:62) ∈ R d and mapping each candidate price p t = m j + x (cid:62) t

Original: ˆ θ to an action vector
Revised: ˆ θ to an action vector

Original: A t with a single non-zero j -th element p t , the expected revenue of setting price p t can be rewritten as
Revised: A t with a single non-zero j -th element p t , the expected revenue of setting price p t can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: In addition, all linear parameters ξ t are around the central parameter ξ ∗ =
Revised: In addition, all linear parameters ξ t are around the central parameter ξ ∗ =

Original: (1 − F ( m 1 ) , . . .
Revised: (1 − F ( m 1 ) , . . .

Original: , 1 −
Revised: , 1 −

Original: F ( m d )) (cid:62) and thus close to each other with a perturbation constant C p = 2 L || ˆ θ − θ 

Revised: F ( m d )) (cid:62) and thus close to each other with a perturbation constant C p = 2 L || ˆ θ − θ

Original: || 1 .
Revised: || 1 .

Original: Namely, we have || ξ s − ξ t || ∞ ≤ C p , ∀ s, t ∈ N + .
Revised: Namely, we have || ξ s − ξ t || ∞ ≤ C p , ∀ s, t ∈ N + .

Original: This implies a lower (cid:96) 1 estimation error of ˆ θ would lead to less perturbations and probably incur less regret.
Revised: This implies a lower (cid:96) 1 estimation error of ˆ θ would lead to less perturbations and probably incur less regret.

Original: Under the PLB formulation, the Inner UCB Algorithm is indeed equivalent to a modiﬁed version of the LinUCB algorithm [1, 31, 35].
Revised: Under the PLB formulation, the Inner UCB Algorithm is indeed equivalent to a modiﬁed version of the LinUCB algorithm [1, 31, 35].

Original: Thus we specify β t in (1) as
Revised: Thus we specify β t in (1) as

Original: [Equation]
Revised: [Equation]

Original:  Regret Analysis
Revised:  Regret Analysis

Original: In this section, we analyze the regret of our proposed Explore-then-UCB algorithm.
Revised: In this section, we analyze the regret of our proposed Explore-then-UCB algorithm.

Original: For both Case (A) and Case (B) , we specify the parameters β and γ in Algorithm 1 appropriately and prove the respective upper bounds.
Revised: For both Case (A) and Case (B) , we specify the parameters β and γ in Algorithm 1 appropriately and prove the respective upper bounds.

Original: Furthermore, we prove the ﬁrst regret lower bound for Case (A) .
Revised: Furthermore, we prove the ﬁrst regret lower bound for Case (A) .

Original: 5.1 Upper Bounds
Revised: 5.1 Upper Bounds

Original: For upper bounds, we ﬁrst analyze a single episode in Algorithm 1 and then extend it to the entire horizon.
Revised: For upper bounds, we ﬁrst analyze a single episode in Algorithm 1 and then extend it to the entire horizon.

Original: Firstly, we provide the (cid:96) 1 estimation error of our θ 0 estimation procedure.
Revised: Firstly, we provide the (cid:96) 1 estimation error of our θ 0 estimation procedure.

Original: Lemma 1 Under Assumptions 1 – 3, there exists positive constants ˜ c 1 , ˜ c 2 , ˜ c 3 such that for any episode k with the exploration phase length n k ≥ ˜ c
Revised: Lemma 1 Under Assumptions 1 – 3, there exists positive constants ˜ c 1 , ˜ c 2 , ˜ c 3 such that for any episode k with the exploration phase length n k ≥ ˜ c

Original: 3 ( d 0 + 1) 3 , we have with probability at least 1 − 2 n k − ˜ c
Revised: 3 ( d 0 + 1) 3 , we have with probability at least 1 − 2 n k − ˜ c

Original: 1 e −
Revised: 1 e −

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: k that
Revised: k that

Original: [Equation]
Revised: [Equation]

Original: Therefore, with an exploration phase length n k that scales as (cid:96) βk , we obtain an estimate ˆ θ k with a high probability (cid:96) 1 -error of ˜ O
Revised: Therefore, with an exploration phase length n k that scales as (cid:96) βk , we obtain an estimate ˆ θ k with a high probability (cid:96) 1 -error of ˜ O

Original: ( (cid:96) − β/ 2 k ) .
Revised: ( (cid:96) − β/ 2 k ) .

Original: As this ˆ θ k will guide the UCB procedure, we need to investigate how the estimation error propagates into the UCB phase regret.
Revised: As this ˆ θ k will guide the UCB procedure, we need to investigate how the estimation error propagates into the UCB phase regret.

Original: A lower error rate is expected to cause less regret.
Revised: A lower error rate is expected to cause less regret.

Original: On the other hand, a lower error rate requires a larger β and costs a longer exploration phase with more regret.
Revised: On the other hand, a lower error rate requires a larger β and costs a longer exploration phase with more regret.

Original: Therefore, besides the “inner” balance between revenue-maximizing and
Revised: Therefore, besides the “inner” balance between revenue-maximizing and

Original: F -learning in the UCB phase, there is an “outer” balance between the exploration phase regret and the UCB phase regret, both associated with θ 0 -learning.
Revised: F -learning in the UCB phase, there is an “outer” balance between the exploration phase regret and the UCB phase regret, both associated with θ 0 -learning.

Original: This outer balance is regulated by the value of β , which should be set differently under Case (A) and Case (B) to achieve the optimal balance.
Revised: This outer balance is regulated by the value of β , which should be set differently under Case (A) and Case (B) to achieve the optimal balance.

Original: Secondly, we analyze the regret for the Inner UCB Algorithm.
Revised: Secondly, we analyze the regret for the Inner UCB Algorithm.

Original: By deﬁning the discrete best prices ˜
Revised: By deﬁning the discrete best prices ˜

Original: p ∗ t := arg max p ∈S t p (1 − F ( p − x (cid:62) t θ 0 )) in S t , the time t regret r t can be decomposed as ˜ p ∗ t (
Revised: p ∗ t := arg max p ∈S t p (1 − F ( p − x (cid:62) t θ 0 )) in S t , the time t regret r t can be decomposed as ˜ p ∗ t (

Original: [Equation]
Revised: [Equation]

Original: F (˜ p ∗ t − x (cid:62) t θ 0 ))
Revised: F (˜ p ∗ t − x (cid:62) t θ 0 ))

Original: − p t (
Revised: − p t (

Original: [Equation]
Revised: [Equation]

Original: − F ( p t − x (cid:62) t θ 0 ))
Revised: − F ( p t − x (cid:62) t θ 0 ))

Original: [Equation]
Revised: [Equation]

Original: t (1 − F ( p ∗ t − x (cid:62) t θ 0 ))
Revised: t (1 − F ( p ∗ t − x (cid:62) t θ 0 ))

Original: − ˜ p ∗ t (1 − F (˜ p ∗ t − x (cid:62) t θ 0 ))
Revised: − ˜ p ∗ t (1 − F (˜ p ∗ t − x (cid:62) t θ 0 ))

Original: (cid:124) (cid:123)(cid:122) (cid:125) r t,.
Revised: (cid:124) (cid:123)(cid:122) (cid:125) r t,.

Original: We refer to R T 0 , 1 = (cid:80) T 0 t =1 r t, 1 and R T 0 , 2 = (cid:80) T 0 t =1 r t, 2 as the discrete-part and continuous-part regrets.
Revised: We refer to R T 0 , 1 = (cid:80) T 0 t =1 r t, 1 and R T 0 , 2 = (cid:80) T 0 t =1 r t, 2 as the discrete-part and continuous-part regrets.

Original: By the PLB formulation of the pricing problem, there is a correspondence between candidate prices and PLB actions.
Revised: By the PLB formulation of the pricing problem, there is a correspondence between candidate prices and PLB actions.

Original: Thus the discrete-part regret is equivalent to the PLB regret.
Revised: Thus the discrete-part regret is equivalent to the PLB regret.

Original: To bound the discrete-part regret of the Inner UCB Algorithm, we prove the following Proposition 1 based on
Revised: To bound the discrete-part regret of the Inner UCB Algorithm, we prove the following Proposition 1 based on

Original: Theorem 1 in [35].
Revised: Theorem 1 in [35].

Original: Proposition 1 Under Assumptions 1 and 4, there exists positive constants C (cid:48) 1 , C (cid:48) 2 and C (cid:48)
Revised: Proposition 1 Under Assumptions 1 and 4, there exists positive constants C (cid:48) 1 , C (cid:48) 2 and C (cid:48)

Original: 3 such that with probability at least 1 − 1 T 0 , the Inner UCB Algorithm yields a discrete-part regret
Revised: 3 such that with probability at least 1 − 1 T 0 , the Inner UCB Algorithm yields a discrete-part regret

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: In Proposition 1, the ﬁrst ˜ O ( d √ T 0 ) component is typical in linear bandit and attributes to the lack of knowledge for F and the central parameter ξ ∗ in the PLB formulation.
Revised: In Proposition 1, the ﬁrst ˜ O ( d √ T 0 ) component is typical in linear bandit and attributes to the lack of knowledge for F and the central parameter ξ ∗ in the PLB formulation.

Original: The second L || ˆ θ − θ
Revised: The second L || ˆ θ − θ

Original: [Table]
Revised: [Table]

Original: || 1 Tcomponent is proportional to the PLB perturbation constant 2 L
Revised: || 1 Tcomponent is proportional to the PLB perturbation constant 2 L

Original: || ˆ θ − θ 

Revised: || ˆ θ − θ

Original: || .
Revised: || .

Original: It demonstrates how estimation errors inﬂuence the regret upper bounds, which matches with the intuition that a better estimation would incur a lower regret in the UCB phase.
Revised: It demonstrates how estimation errors inﬂuence the regret upper bounds, which matches with the intuition that a better estimation would incur a lower regret in the UCB phase.

Original: The discretization number d plays a critical role in balancing the discrete-part and continuous-part regret.
Revised: The discretization number d plays a critical role in balancing the discrete-part and continuous-part regret.

Original: A larger d leads to a higher discrete-part regret as indicated by Proposition 1, which is intuitive since more discretizations yield more candidate prices and hence a more challenging search process.
Revised: A larger d leads to a higher discrete-part regret as indicated by Proposition 1, which is intuitive since more discretizations yield more candidate prices and hence a more challenging search process.

Original: On the other hand, a larger h and a denser discretization would make the discrete best price “closer” to the overall best price and decrease the continuous-part regret.
Revised: On the other hand, a larger h and a denser discretization would make the discrete best price “closer” to the overall best price and decrease the continuous-part regret.

Original: Speciﬁcally, under Case (A) and
Revised: Speciﬁcally, under Case (A) and

Original: Case (B) , we can bound the continuous-part regret with the order O
Revised: Case (B) , we can bound the continuous-part regret with the order O

Original: ( T 0 d 2 ) and O ( T 0 d ) respectively.
Revised: ( T 0 d 2 ) and O ( T 0 d ) respectively.

Original: Namely, we could achieve a lower rate with the extra 2nd-order smoothness Assumption 5.
Revised: Namely, we could achieve a lower rate with the extra 2nd-order smoothness Assumption 5.

Original: Since we choose the discretization number d k in episode k to scale as ( u k ) γ where u k denotes the UCB phase length, γ regulates the balance between the discrete-part and continuous-part regret in the UCB phase and should be set differently for optimal balances in Case (A) and Case (B) .
Revised: Since we choose the discretization number d k in episode k to scale as ( u k ) γ where u k denotes the UCB phase length, γ regulates the balance between the discrete-part and continuous-part regret in the UCB phase and should be set differently for optimal balances in Case (A) and Case (B) .

Original: Now we are ready to present the two regret upper bounds for Case (A) and Case (B) .
Revised: Now we are ready to present the two regret upper bounds for Case (A) and Case (B) .

Original: Theorem 1 Under Assumptions 1 – 5, by choosing β = 23 and γ = 16 in Algorithm 1, the expected regret satisﬁes E ( R T ) =
Revised: Theorem 1 Under Assumptions 1 – 5, by choosing β = 23 and γ = 16 in Algorithm 1, the expected regret satisﬁes E ( R T ) =

Original: ˜ O ( d 20 T 2 / 3 ) =
Revised: ˜ O ( d 20 T 2 / 3 ) =

Original: ˜ O ( T 2 / 3 ) .
Revised: ˜ O ( T 2 / 3 ) .

Original: Theorem 2 Under Assumptions 1 – 4, by choosing β = 34 and γ = 14 in Algorithm 1, the expected regret satisﬁes E ( R T ) =
Revised: Theorem 2 Under Assumptions 1 – 4, by choosing β = 34 and γ = 14 in Algorithm 1, the expected regret satisﬁes E ( R T ) =

Original: ˜ O
Revised: ˜ O

Original: ( d 0 T 3 / 4 ) =
Revised: ( d 0 T 3 / 4 ) =

Original: ˜ O ( T 3 / 4 ) .
Revised: ˜ O ( T 3 / 4 ) .

Original: We illustrate the regret components’ orders in Table 2 for a single episode with a generic length T 0 .
Revised: We illustrate the regret components’ orders in Table 2 for a single episode with a generic length T 0 .

Original: Table 2 explains the different choices of β and γ in the two cases to minimize the overall regret rates.
Revised: Table 2 explains the different choices of β and γ in the two cases to minimize the overall regret rates.

Original: It demonstrates that β balances between the exploration phase regret and the discrete-part regret due to θ 0 estimation error; while γ balances between the continuous-part regret and the discrete-part regret due to F -learning.
Revised: It demonstrates that β balances between the exploration phase regret and the discrete-part regret due to θ 0 estimation error; while γ balances between the continuous-part regret and the discrete-part regret due to F -learning.

Original: As shown in Theorem 1, through optimal balance, ExUCB improves the existing regret of ˜ O
Revised: As shown in Theorem 1, through optimal balance, ExUCB improves the existing regret of ˜ O

Original: ( T 2 / 3 ∨ (1 − α ) ) in [35] and ˜ O ( T 5 / 7 ) in [20] to ˜ O
Revised: ( T 2 / 3 ∨ (1 − α ) ) in [35] and ˜ O ( T 5 / 7 ) in [20] to ˜ O

Original: ( T 2 / 3 ) under Lipschitz and 2nd-order smoothness assumptions.
Revised: ( T 2 / 3 ) under Lipschitz and 2nd-order smoothness assumptions.

Original: In addition, an optimal choice of β and γ in Theorem 2 helps ExUCB match the best existing regret of ˜ O
Revised: In addition, an optimal choice of β and γ in Theorem 2 helps ExUCB match the best existing regret of ˜ O

Original: ( T 3 / 4 ) [48] under the Lipschitz-only assumption.
Revised: ( T 3 / 4 ) [48] under the Lipschitz-only assumption.

Original: 5.2 Lower Bound
Revised: 5.2 Lower Bound

Original: We next prove a regret lower bound of ˜Ω( T 3 / 5 ) for Case (A) .
Revised: We next prove a regret lower bound of ˜Ω( T 3 / 5 ) for Case (A) .

Original: To our limited knowledge, this is the ﬁrst lower bound result under the Lipschitz and 2nd-order smoothness assumptions.
Revised: To our limited knowledge, this is the ﬁrst lower bound result under the Lipschitz and 2nd-order smoothness assumptions.

Original: Note that there is a gap between our proved upper and lower bounds for Case (A) .
Revised: Note that there is a gap between our proved upper and lower bounds for Case (A) .

Original: Indeed, such a gap also happens in other works [20, 48] on the unknown noise pricing problems.
Revised: Indeed, such a gap also happens in other works [20, 48] on the unknown noise pricing problems.

Original: This may be due to the inherent difﬁculties of these problems, e.g., in learning both θ 0 and F .
Revised: This may be due to the inherent difﬁculties of these problems, e.g., in learning both θ 0 and F .

Original: Theorem 3 For any δ > 0 , no policy can achieve an O ( T 3 / 5 − δ ) regret for the dynamic pricing problem under Assumptions 1 – 5.
Revised: Theorem 3 For any δ > 0 , no policy can achieve an O ( T 3 / 5 − δ ) regret for the dynamic pricing problem under Assumptions 1 – 5.

Original: Remark 

Revised: Remark

Original: In [48], the authors proved an ˜Ω( T 2 / 3 ) lower bound with constructed instances that can ﬁt into Case (B) but does not apply to Case (A) .
Revised: In [48], the authors proved an ˜Ω( T 2 / 3 ) lower bound with constructed instances that can ﬁt into Case (B) but does not apply to Case (A) .

Original: Our proved lower bound rate is lower since our instances need to satisfy more assumptions and lie in a more benign class.
Revised: Our proved lower bound rate is lower since our instances need to satisfy more assumptions and lie in a more benign class.

Original: Another similar lower bound of Ω( T 3 / 5 ) is proved in [13] for 1 -dimensional nonparametric contextual expected revenue functions.
Revised: Another similar lower bound of Ω( T 3 / 5 ) is proved in [13] for 1 -dimensional nonparametric contextual expected revenue functions.

Original: However, their instances are constructed in a local-bin fashion with respect to the covariate and thus are outside our revenue function class.
Revised: However, their instances are constructed in a local-bin fashion with respect to the covariate and thus are outside our revenue function class.

Original: Therefore, their lower bound does not apply to our setting.
Revised: Therefore, their lower bound does not apply to our setting.

Original: Proof Sketch of Theorem 3.
Revised: Proof Sketch of Theorem 3.

Original: We follow similar ideas in [28, 48] to construct the instances.
Revised: We follow similar ideas in [28, 48] to construct the instances.

Original: Firstly we set θ 0 = 0 and relieve the difﬁculty from contexts.
Revised: Firstly we set θ 0 = 0 and relieve the difﬁculty from contexts.

Original: Secondly, we construct a series of “bump towers” and transform them to valid expected revenue functions in the form of p (1 − F ( p )) , while still preserving the intended properties.
Revised: Secondly, we construct a series of “bump towers” and transform them to valid expected revenue functions in the form of p (1 − F ( p )) , while still preserving the intended properties.

Original: We construct each bump tower from an inﬁnitely-nested interval series [0 , 1] =
Revised: We construct each bump tower from an inﬁnitely-nested interval series [0 , 1] =

Original: [ a 0 , b 0 ] ⊃ [ a 1 , b 1 ] ⊃ · · · ⊃ [ a k , b k ] ⊃ . . . .
Revised: [ a 0 , b 0 ] ⊃ [ a 1 , b 1 ] ⊃ · · · ⊃ [ a k , b k ] ⊃ . . . .

Original: Speciﬁcally, we divide [ a k , b k ] with length w k = 3 − k !
Revised: Speciﬁcally, we divide [ a k , b k ] with length w k = 3 − k !

Original: into three same-length sub-intervals and further divide the middle one into w k w k +1 candidate intervals.
Revised: into three same-length sub-intervals and further divide the middle one into w k w k +1 candidate intervals.

Original: Then each of the candidate intervals forms one case of [ a k +1 , b k +1 ] .
Revised: Then each of the candidate intervals forms one case of [ a k +1 , b k +1 ] .

Original: For each of these inﬁnitely-nested interval series, we add up bump functions on the nested intervals to form the bump tower.
Revised: For each of these inﬁnitely-nested interval series, we add up bump functions on the nested intervals to form the bump tower.

Original: Different from [48], to construct the expected revenue functions that satisfy the 2nd-order smoothness assumption, we adopt a different basic bump function and develop a “quadratically-shrinking” adding pattern.
Revised: Different from [48], to construct the expected revenue functions that satisfy the 2nd-order smoothness assumption, we adopt a different basic bump function and develop a “quadratically-shrinking” adding pattern.

Original: Finally, we prove that any policy will miss the peaks of some revenue function instances often enough, thus accumulating an inevitable amount of the regret.
Revised: Finally, we prove that any policy will miss the peaks of some revenue function instances often enough, thus accumulating an inevitable amount of the regret.

Original:  Numerical Experiments
Revised:  Numerical Experiments

Original: We conduct numerical experiments to support our theoretical regret bounds of ExUCB under both
Revised: We conduct numerical experiments to support our theoretical regret bounds of ExUCB under both

Original: Case (A) and Case (B) .
Revised: Case (A) and Case (B) .

Original: We consider a total horizon length T =
Revised: We consider a total horizon length T =

Original: (cid:80) 10 i =1 2 

Revised: (cid:80) 10 i =1 2

Original: + i that is divided intoepisodes with the ﬁrst episode length α 1 = 2 9 .
Revised: + i that is divided intoepisodes with the ﬁrst episode length α 1 = 2 9 .

Original: For both cases, we specify the linear parameter θ 0 = 30 and sample the i.i.d.
Revised: For both cases, we specify the linear parameter θ 0 = 30 and sample the i.i.d.

Original: covariates as x t ∼ Unif (1 / 2 , 1) .
Revised: covariates as x t ∼ Unif (1 / 2 , 1) .

Original: For Case (A) , the noise distribution is set as a Uniform mixture 34 Unif ( − 15 , 0) + 14 Unif (0 , 15) ; while for Case (B) , we adopt another
Revised: For Case (A) , the noise distribution is set as a Uniform mixture 34 Unif ( − 15 , 0) + 14 Unif (0 , 15) ; while for Case (B) , we adopt another

Original: Uniform mixture 14 Unif ( − 15 , 0) + 34 Unif (0 , 15) .
Revised: Uniform mixture 14 Unif ( − 15 , 0) + 34 Unif (0 , 15) .

Original: It can be veriﬁed that the ﬁrst distribution satisﬁes the Lipschitz and 2nd-order smoothness assumptions while the second one only satisﬁes the Lipschitz assumption.
Revised: It can be veriﬁed that the ﬁrst distribution satisﬁes the Lipschitz and 2nd-order smoothness assumptions while the second one only satisﬁes the Lipschitz assumption.

Original: We apply ExUCB with different choices of β, γ speciﬁed in Theorems 1 – 2 to these two instances.
Revised: We apply ExUCB with different choices of β, γ speciﬁed in Theorems 1 – 2 to these two instances.

Original: For both cases, we set the constants in Algorithm 1 as p max = 50 , B = 50 , C 1 = 1 , C 2 = 20 , λ = 0 .
Revised: For both cases, we set the constants in Algorithm 1 as p max = 50 , B = 50 , C 1 = 1 , C 2 = 20 , λ = 0 .

Original: With 100 replications, we plot the log-log scale of average accumulative regrets versus the time periods in Figure 2 along with the 95% conﬁdence intervals.
Revised: With 100 replications, we plot the log-log scale of average accumulative regrets versus the time periods in Figure 2 along with the 95% conﬁdence intervals.

Original: The linear ﬁts extract a slope of 0 .
Revised: The linear ﬁts extract a slope of 0 .

Original: 670 for Case (A) and a slope of 0 .
Revised: 670 for Case (A) and a slope of 0 .

Original: 724 for Case (B) , which indicates that our proved regrets of ˜
Revised: 724 for Case (B) , which indicates that our proved regrets of ˜

Original: O ( T 2 / 3 ) for Case (A) and ˜ O
Revised: O ( T 2 / 3 ) for Case (A) and ˜ O

Original: ( T 3 / 4 ) for Case (B) are sharp.
Revised: ( T 3 / 4 ) for Case (B) are sharp.

Original: [Figure]
Revised: [Figure]

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we introduce a novel Explore-then-UCB strategy to tackle the contextual dynamic pricing problem with unknown linear valuation and unknown nonparametric noise distribution F .
Revised: In this paper, we introduce a novel Explore-then-UCB strategy to tackle the contextual dynamic pricing problem with unknown linear valuation and unknown nonparametric noise distribution F .

Original: Under Lipschitz and 2nd-order smoothness assumptions on F , ExUCB policy improves the bestknown regret upper bounds to ˜ O
Revised: Under Lipschitz and 2nd-order smoothness assumptions on F , ExUCB policy improves the bestknown regret upper bounds to ˜ O

Original: ( T 2 / 3 ) .
Revised: ( T 2 / 3 ) .

Original: Under the Lipschitz-only assumption, ExUCB matches the best existing regret of ˜ O
Revised: Under the Lipschitz-only assumption, ExUCB matches the best existing regret of ˜ O

Original: ( T 3 / 4 ) with better computational efﬁciency.
Revised: ( T 3 / 4 ) with better computational efﬁciency.

Original: In addition, we prove a ﬁrst ˜Ω( T 3 / 5 ) lower bound for our considered contextual dynamic pricing problem under the Lipschitz and 2nd-order smoothness assumptions.
Revised: In addition, we prove a ﬁrst ˜Ω( T 3 / 5 ) lower bound for our considered contextual dynamic pricing problem under the Lipschitz and 2nd-order smoothness assumptions.

Original:  Acknowledgments
Revised:  Acknowledgments

Original: The authors would like to thank the helpful and constructive comments from the reviewers which led to a much improved presentation of this paper.
Revised: The authors would like to thank the helpful and constructive comments from the reviewers which led to a much improved presentation of this paper.

Original: Will Wei Sun and Yufeng Liu acknowledge support from the National Science Foundation (Award SES 2217440).
Revised: Will Wei Sun and Yufeng Liu acknowledge support from the National Science Foundation (Award SES 2217440).

Original: Any opinions, ﬁndings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reﬂect the views of National Science Foundation
Revised: Any opinions, ﬁndings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reﬂect the views of National Science Foundation