Title: Dynamic Pricing and Learning with Bayesian Persuasion
Venue: NeurIPS 2023
Accepted: yes
Keywords: dynamic pricing

information design

regret minimization
Abstract: We consider a novel dynamic pricing and learning setting where in addition to setting prices of products in sequential rounds, the seller also ex-ante commits to ‘advertising schemes’. That is, in the beginning of each round the seller can decide what kind of signal they will provide to the buyer about the product’s quality upon realization. Using the popular Bayesian persuasion framework to model the effect of these signals on the buyers’ valuation and purchase responses, we formulate the problem of finding an optimal design of the advertising scheme along with a pricing scheme that maximizes the seller’s expected revenue. Without any apriori knowledge of the buyers’ demand function, our goal is to design an online algorithm that can use past purchase responses to adaptively learn the optimal pricing and advertising strategy. We study the regret of the algorithm when compared to the optimal clairvoyant price and advertising
scheme. 

Our main result is a computationally efficient online algorithm that achieves an $O(T^{2/3}(m \log T )^{1/3})$ regret bound when the valuation function is linear in the product quality. Here $m$ is the cardinality of the discrete product quality domain and $T$ is the time horizon. This result requires some natural monotonicity and Lipschitz assumptions on the valuation function, but no Lipschitz or smoothness assumption on the buyers’ demand function. For constant $m$, our result matches the regret lower bound for dynamic pricing within logarithmic factors, which is a special case of our problem. We also obtain several improved results for the widely considered special case of additive valuations, including an $\tilde{O}(T^{2/3})$ regret bound independent of $m$ when $m\le T^{1/3}$.

Dynamic Pricing and Learning with Bayesian Persuasion

Abstract

We consider a novel dynamic pricing and learning setting where in addition to setting prices of products in sequential rounds, the seller also ex-ante commits to 'advertising schemes'. That is, in the beginning of each round the seller can decide what kind of signal they will provide to the buyer about the product's quality upon realization. Using the popular Bayesian persuasion framework to model the effect of these signals on the buyers' valuation and purchase responses, we formulate the problem of finding an optimal design of the advertising scheme along with a pricing scheme that maximizes the seller's expected revenue. Without any apriori knowledge of the buyers' demand function, our goal is to design an online algorithm that can use past purchase responses to adaptively learn the optimal pricing and advertising strategy. We study the regret of the algorithm when compared to the optimal clairvoyant price and advertising scheme.

Our main result is a computationally efficient online algorithm that achieves an O ( T 2 / 3 ( m log T ) 1 / 3 ) regret bound when the valuation function is linear in the product quality. Here m is the cardinality of the discrete product quality domain and T is the time horizon. This result requires some natural monotonicity and Lipschitz assumptions on the valuation function, but no Lipschitz or smoothness assumption on the buyers' demand function. For constant m , our result matches the regret lower bound for dynamic pricing within logarithmic factors, which is a special case of our problem. We also obtain several improved results for the widely considered special case of additive valuations, including an ˜ O ( T 2 / 3 ) regret bound independent of m when m ≤ T 1 / 3 . 1

1 Introduction

Dynamic pricing is a key strategy in revenue management that allows sellers to anticipate and influence demand in order to maximize revenue and/or utility. When the customer valuation and demand response for a product is apriori unknown, price variation can also be used to observe and learn the demand function in order to adaptively optimize price and revenue over time. This learning and optimization problem has been a focus of much recent literature that uses exploration-exploitation and multi-armed bandit techniques with dynamic pricing algorithms (e.g., see [43, 16, 41, 8]).

In practice, there is another important tool available to sellers in the form of advertising , using which the sellers can inform and shape customers' valuations of a product. It has been theoretically [53, 54] and empirically [55, 49] shown that advertisements can serve as a credible signal of the quality or characteristics of the advertised product. Sellers can use advertising to provide partial information about a product in order to better position the product in the market and potentially increase customers' chances of purchasing the product. For example, as a common strategy to drive

subscriptions, online newspapers may use a 'teaser' that selectively includes previews of some news articles that are likely to entice readers to subscribe for access to the full story; in the online used-car market, the dealer can advertise the used car by emphasizing different aspects of the car, such as fuel efficiency/mileage/unique features, or selectively disclose history-report information from reputable third parties, catering to specific buyer interests; a film distributor may advertise the movie by selectively showing footage from the film.

However, advertising must be carefully designed to achieve the desired gains. At one glance upselling or inflating the product quality by selectively disclosing only favorable information might appear as a profitable advertising strategy. But such strategies carry the disadvantage of not being very effective in modifying customer beliefs as customers may not trust that the provided information accurately reflects the product's true quality. Also, the design of the advertising strategy needs to interact with the design of the pricing strategy and account for the demand function. For example, to sell highly-priced products or under heavy competition/low demand, the customer may need to be convinced of a good match through more information and thorough insights about the product characteristics. On the other hand, in markets with high demand or for very low-priced goods, the seller may get away with revealing very little information. An extreme example of this phenomenon is the concept of mystery deal boxes sold by some retailers like Amazon/Woot, where the customers are not even made aware of the exact contents of the low-cost box that they are purchasing. 2

In this paper, we use a Bayesian persuasion framework [39] to model the effect of an advertising strategy on customers' beliefs about product quality and consequently their purchase decisions. Our novel formulation combines the Bayesian persuasion model with dynamic pricing and learning in order to quantify the tradeoffs between the design of the pricing and advertising strategies and their combined impact on the revenue outcomes. Without any apriori knowledge of the demand function, our goal is to design an online algorithm that can use past customer responses to adaptively learn a joint pricing and advertising strategy that maximizes the seller's revenue.

Bayesian persuasion is a popular framework for information design with several different settings considered in the literature [39, 30, 38, 10]. We consider a Bayesian persuasion model where the sender (seller) ex-ante commits to an information policy (advertising strategy) that prescribes the distribution of signals the sender will provide to the receiver (buyer) on observing the true state of the world (product quality). The receiver, on observing the sender's signal, uses Bayes' rule to form a posterior on the state of the world. The receiver's action (purchase decision) then optimizes their expected utility under this posterior.

Problem formulation. Specifically, our dynamic pricing and advertising problem is formulated as follows. There are T sequential and discrete rounds. In each round, a fresh product is offered by the seller, with a public prior distribution λ on the product quality ω ∈ Ω ⊆ [0 , 1] . At the beginning of each round t , before observing the realized quality of the t th product, the seller commits to a price p t ∈ [0 , U ] and an advertising strategy φ t , where φ t ( σ | ω ) prescribes the distribution of signal σ ∈ Σ where Σ is an arbitrary signaling space given the realized product quality ω ∈ Ω .

A buyer arrives in each round t with private type θ t generated i.i.d. from a distribution with CDF F ( · ) and support 3 Θ = [0 , 1] . The CDF F ( · ) (or equivalently the demand function D ( · ) /defines 1 -F ( · ) ) is fixed but unknown to the seller. For a buyer of type θ ∈ Θ , the valuation of a product with product quality ω is given by function v ( θ , ω ) .

The t th product quality ω t ∼ λ is then realized and observed by the seller. The buyer cannot observe the realized product quality, but only a signal σ t ∼ φ t ( ·| ω t ) provided by the seller. The buyer uses this signal along with the prior λ to formulate a Bayesian posterior distribution on the product quality µ t ( ω | σ t ) ∝ φ t ( σ t | ω ) · λ ( ω ) . The buyer then purchases the product if and only if the expected valuation under this posterior E ω ∼ µ t ( ·| σ t ) [ v ( θ t , ω )] is greater than or equal to the price p t . We denote the buyer decision at time t by a t ∈ { 0 , 1 } with a t = 1 denoting purchase.

Remark 1.1. We consider the setting where buyers use only the prior distribution and signal in the current round, and not the signals in the past rounds, to make their decisions. This is motivated by

the fact that at each round, the buyer is facing a fresh product, whose quality is drawn independently across time. Thus, previous signals do not carry information about the current product. Fresh products with independent qualities are common in many real-world applications such as secondhand markets, mystery boxes sold by Amazon/Woot, etc. This similar problem structure has also been studied in the online/sequential Bayesian persuasion literature with repeated interactions of sender and receivers, for example, see [61, 24, 23, 33, 58, 15].

Asummary of the game timeline is as follows: at t ∈ [ T ] , (1) the seller commits to a price p t ∈ [0 , U ] and an advertising strategy φ t ; (2) a buyer t with private type θ t ∼ F arrives; (3) a product with quality ω t ∼ λ is realized; the seller sends signal σ t ∼ φ t ( ·| ω t ) to the buyer; (4) the buyer formulates Bayesian posterior µ t ( ω | σ t ) , and the buyer purchases the product (denoted by a t = 1 ) to generate revenue p t if and only if her expected value exceeds the price, i.e., E ω ∼ µ t ( ·| σ t ) [ v ( θ t , ω )] ≥ p t .

Our goal is to design an online learning algorithm that sequentially chooses the price and advertising strategy p t , φ t in each round t based on the buyers' responses in the previous rounds, in order to optimize total expected revenue over a time horizon T without apriori knowledge of the distribution F . Let Rev ( p t , φ t ) /defines E [ p t · a t ; p t , φ t ] denote the expected revenue at time t under the price p t and advertising strategy φ t . Here the expectation is over realizations of customer type θ t ∼ F , product quality ω t ∼ λ and advertising signal σ t ∼ φ t ( ·| ω t ) . Note that since product quality and types are i.i.d. across time, for any given p t = p, φ t = φ , the expected per-round revenue Rev ( p, φ ) does not depend on time. Thus a static price and advertising strategy maximizes total expected revenue over the time horizon T . Therefore, we can measure the performance of an algorithm in terms of regret that compares the total expected revenue of the algorithm to that of the best static pricing and advertising strategy. We define

Our contributions. In this work, we present a computationally efficient online pricing and advertis- ing algorithm that achieves an O ( T 2 / 3 ( m log T ) 1 / 3 ) bound on the regret in time T , where m = | Ω | is the cardinality of the (discrete) product quality space. Importantly, we achieve this result without any assumptions like Lipschitz or smoothness on the demand function D ( · ) = 1 -F ( · ) . However, our results require certain assumptions on the valuation function. Following the literature, we make the common assumption that the function v ( θ , ω ) is linear in the product quality ω . Furthermore, we assume the following monotonicity and Lipschitz properties on the valuation function.

Assumption 1. Buyer's valuation function v ( · , · ) satisfies:

1a Fix any buyer type θ , function v ( θ , ω ) is non-decreasing w.r.t. quality ω .

1b Fix any quality ω , function v ( θ , ω ) is increasing and 1 -Lipschitz 4 w.r.t. type θ .

Such assumptions are in fact common in literature and natural in many economic situations where the valuation of a product increases with the product quality and buyer's type (paying ability/need). Existing literature on Bayesian persuasion/dynamic pricing often makes even stronger assumptions about the receiver's utility function. For example, both the additive functions v ( θ , ω ) = θ + ω [cf. 35, 45, 27] and the multiplicative functions v ( θ , ω ) = θω + θ [cf. 22, 48], are linear in ω and satisfy Assumption 1. Our main result is then summarized as follows:

Theorem 1.1. For any type CDF F , given a valuation function v ( θ , ω ) that is linear in product quality ω and satisfies Assumption 1, Algorithm 1 with parameter ε = Θ (( m log T / T ) 1 / 3 ) has an expected regret of O ( T 2 / 3 ( m log T ) 1 / 3 ) . Here, m is the cardinality of the discrete quality space Ω .

Furthermore, we obtain several improved results for the widely considered special case of additive valuations, i.e., for v ( θ , ω ) = θ + ω . See Appendix B for the formal statements and analysis.

(Theorem B.1) Consider discrete sets Ω that are 'equally-spaced', e.g., Ω = { 0 , 1 } or Ω = [ m ] . Given such a product quality space Ω and additive valuation function, we show that the regret of Algorithm 1 is bounded by O ( T 2 / 3 (log T ) 1 / 3 ) when m ≤ ( T / log T ) 1 / 3 , and by O ( √ mT log T ) for larger m .

(Theorem B.2) For any arbitrary (discrete or continuous) product quality space Ω , given additive valuation functions, we have a slightly modified algorithm (see Algorithm 3 in Appendix C.2) with an expected regret of O ( T 3 / 4 (log T ) 1 / 4 ) independent of m .

When the valuation function v ( · , ω ) is L -Lipschitz w.r.t. type θ , our regret bound in Theorem 1.1 would become O ( T 2 / 3 ( Lm log T ) 1 / 3 ) , and the regret bound for arbitrary space Ω would become O ( T 3 / 4 ( L log T ) 1 / 4 ) .

We might compare our results to the best regret bounds available for the well-studied dynamic pricing and learning problem with unlimited supply [43, 8], which is a special case of our problem if the product quality is deterministic, i.e., m = 1 , and the advertising scheme reveals no information and thus has no impact on the buyer's purchase decision. For the dynamic pricing problem a lower bound of Ω ( T 2 / 3 ) on regret is known [43]. Therefore the dependence on T in our results cannot be improved. In fact, our result matches this lower bound in the case of binary or constant size quality space, which are common settings in information design literature [39, 18, 5, 57, 33, 12].

High-level descriptions of the proposed algorithm and challenges. Our problem can be viewed as a very high dimensional combinatorial multi-armed bandit problem, where each arm is a pair of a price and a feasible advertising strategy: the set of feasible advertising strategies being the set of all possible conditional distributions { φ ( ·| ω ) , ω ∈ Ω } over signal space Σ . As a first step towards obtaining a more tractable setting, we present an equivalent reformulation of the problem which uses the observation that advertising affects the buyer's decision only via the posterior distribution over quality. By the linearity of valuation function v ( · , · ) over product quality ω , seller's choice of advertising strategy in every round can be further simplified to selecting a distribution over posterior means that is subject to a feasibility constraint.

From here, the seller's decision space now becomes two-dimensional (a price and a distribution of posterior means). Viewing seller's expected revenue as an unknown (nonlinear) function over this two-dimensional decision space, one may consider applying algorithms in contextual bandits or Lipschitz bandits to get sublinear regrets, e.g., ˜ O ( T 3 / 4 poly ( m )) regret for two-dimensional decision space if there exists a Lipschitz property of reward function relative to seller's decision space. However, it is unclear whether one can establish such Lipschitz property given that we do not assume Lipschitzness or smoothness on demand function and we have complex constraints on the feasibility of advertising space. Instead, in our algorithm we use a 'model-based approach': we use buyers' purchase responses to explore the demand function over the (discretized) type space and jointly learn the optimal advertising and pricing. To explore the demand function over the one-dimensional (discretized) type space, we propose a novel discretization scheme such that it enables near-optimal price and advertising strategy even without Lipschitzness or smoothness assumption and with the complex feasibility constraint on the advertising space. These treatments lead us to the optimal ˜ O ( T 2 / 3 m 1 / 3 ) regret.

Related work. Our work is related to several streams of research. Below we briefly review the some of these connections.

In our setting, the seller can utilize her information advantage to design an advertisement to signal the product quality to the buyer. There is a long line of research in the literature, from both empirical and theoretical perspective, dedicated to study how to use advertisement as a signal to steer buyers' evaluations of advertised goods [53, 54, 42, 52, 37, 55, 40]. In our problem, we follow the literature in information design, a.k.a., Bayesian persuasion [39] [also see the recent surveys by 30, 38, 10], where the seller can commit to an information policy that can strategically disclose product information to the buyer so that to influence buyer's belief about the product quality. Similar formulation for advertising has also appeared in [19, 32, 3, 34, 14, 13, 28]. Our work differs from these works in several ways. First, the seller's offline problem in our setting is a joint pricing and advertising problem. Second, we focus on an online setting where the seller has no apriori knowledge of the demand function and has to use past buyers' purchase responses to adaptively learn optimal pricing and advertising strategy.

Our problem shares similarity to the problem on sale of information in economics and computer science literature [7, 9, 11, 26, 20, 48, 12, 59, 47, 25]. In particular, similar to the problem on sale of information, the seller, in our setting, also commits to design an information structure to reveal information about the realized state to the decision maker (i.e., buyer); and the buyer, in our setting,

then makes the payment based on the declared information structure, not for specific realizations of the seller's informative signals [12, 20, 48]. However, different from these works, the seller in our setting is selling a product with some inherent value and not just information. The valuation of the product can be shaped by providing information. This gives new interesting tradeoffs between information and revenue in our problem that are absent in the settings where only information is being sold. Moreover, in most literature of selling information, the buyers' type distribution is usually assumed to be known. We consider a more practical data-driven setting where the underlying buyers' type distribution (demand function) is apriori unknown to the seller and needs to be learnt from observations.

Facing unknown buyer's preference (i.e., buyer's private type), the seller's dynamic advertising problem also relates to the growing line of work in information design on relaxing one fundamental assumption in the canonical Bayesian persuasion model - the sender perfectly knows receiver's preference. The present paper joins the recent increased interests on using online learning approach to study the regret minimization when the sender repeatedly interacts with receivers [23, 24, 61, 33] without knowing receivers' preferences. Moreover, our work also conceptually relates to research on Bayesian exploration in multi-armed bandit [46, 51, 50, 36] which also studies an online setting where one player can utilize her information advantage to persuade another player to take the desired action. Our work departs from this line of work in terms of both the setting and the application domain. Particularly, the above works typically consider an online setting on how to learn optimal signaling scheme whereas in our setting the optimal policy is a joint pricing and signaling (advertising) scheme.

When there is no uncertainty in the product quality, the seller's problem in our setting reduces to a standard dynamic pricing and learning problem with unknown non-parametric demand function [43, 16, 41, 8]. However, given any non-trivial product quality space and prior distributions, in our problem, in addition to a price, the seller needs to choose a non-trivial advertising strategy in order to maximize revenue. This makes the seller's decision space high dimensional and (as we discuss in the next section) introduces significant complexities and difficulties so that the typical techniques (like uniform or adaptive discretization) used in pricing and continuous/combinatorial multi-armed bandit literature cannot be directly applied.

2 Algorithm Design

In this section, we present our main algorithm for the dynamic pricing and advertising problem. In subsection 2.1, we present an equivalent reformulation for tractable advertising strategies, then in subsection 2.2, we discuss many important challenges even after this simplification, and finally, in subsection 2.3 we present our algorithm.

2.1 An equivalent reformulation for tractable advertising strategies

Recall that in every round, the buyer t sees the offered price p t and advertising strategy φ t that specifies the distributions over signals φ t ( ·| ω ) ∈ ∆ Σ that the seller will send for each possible value ω of the realized product quality. After the product quality ω t is realized, the buyer t sees a signal σ t ∼ φ t ( ·| ω t ) from the seller's declared advertising strategy. The buyer uses this signal along with the prior λ to form a Bayesian posterior µ t ( ·| σ t ) ∈ ∆ Ω on the product quality. The Bayesian rational buyer then takes the action a t ∈ { 0 , 1 } , based on expected utility maximization. In particular, we have a t = 1 if E ω ∼ µ t ( ·| σ t ) [ v ( θ t , ω )] ≥ p t and a t = 0 otherwise.

From the decision formula above, it is clear that the choice of advertising strategy affects the buyer t 's decision only through the realized posterior µ t ( ·| σ t ) . Consequently, the seller's choice of advertising strategy in time t can be reduced to selecting a distribution over posteriors µ t . Seller's choice can in fact be further simplified in the case where the buyer's valuation function is linear in the product quality ω since in that case we have

where q t is the realized posterior mean q t /defines E ω ∼ µ t ( ·| σ t ) [ ω ] . Here q t ∈ [0 , 1] since ω ∈ Ω ⊆ [0 , 1] . Therefore the buyer purchases ( a t = 1 ) if and only if v ( θ t , q t ) ≥ p t . As a result, we can reduce the seller's advertising in round t to the choice of distribution (pdf) ρ t ( · ) ∈ ∆ [0 , 1] over posterior means.

However, the choice of ρ t must be restricted to only feasible distributions of posterior means, that is, all possible distributions over posterior means that can be induced by any advertising scheme given the prior λ . It is well known that the distribution over posterior means ρ is feasible if and only if it is the mean-preserving contraction of the prior [17, 6]. This condition can be equivalently written in terms of a Bayes-consistency condition [39] on the conditional means ρ ( ·| ω ) , ω ∈ Ω . For simplicity of exposition, we consider discrete quality space Ω = { ¯ ω 1 , . . . , ¯ ω m } ⊆ [0 , 1] , where 0 = ¯ ω 1 < ¯ ω 2 < . . . < ¯ ω m = 1 and m = | Ω | is the cardinality of the quality space. Then, a distribution ρ over posterior means is feasible if one can construct a set of conditional distributions ( ρ i ) i ∈ [ m ] satisfying the following Bayes-consistency condition, and vice versa [39]:

Throughout this paper, we use the collection of distributions ( ρ i ) i ∈ [ m ] satisfying (BC) condition as a convenient way to construct feasible distributions over posterior means: ρ ( q ) = ∑ i ρ i ( q ) λ i .

With the above observations, we can without loss of generality assume that seller's advertising strategy is to directly choose a distribution ρ t over the posterior means that satisfies (BC), without considering the design of the underlying signaling scheme { φ ( σ | ω ) , Σ } .

We summarize the new equivalent game timeline as follows: at t ∈ [ T ] , (1) the seller commits to a price p t ∈ [0 , U ] and an advertising strategy ρ t = ( ρ i,t ) i ∈ [ m ] satisfying (BC); (2) a buyer t with private type θ t ∼ F arrives; (3) a product with quality ω t ∼ λ is realized; a posterior mean q t ∼ ρ t is realized; (4) the buyer observes the posterior mean q t ; the buyer purchases the product ( a t = 1 ) to generate revenue p t if only if v ( θ t , q t ) ≥ p t . Note that the seller knows the form of the buyer's valuation function v . Moreover, the seller observes the realized product quality ω t , the realized posterior mean q t , and the buyer's purchase decision a t , but does not know type CDF F (i.e., the demand function D ) and the realized buyer type θ t .

Revenue and regret. Given the new formulation, we can also rewrite the revenue and regret in terms of the choices of ρ t , t = 1 , . . . , T . We define the following function κ ( p, q ) , which we refer to as the critical type for a given price p and posterior mean q .

Definition 2.1 (Critical type) . For any p ∈ [0 , U ] and q ∈ [0 , 1] , define function κ ( · , · ) as κ ( p, q ) /defines min { θ ∈ Θ : v ( θ , q ) ≥ p } .

Now under Assumption 1b , due to the monotonicity of the valuation function in buyer's type, we have that given any p, q, θ , 1 [ v ( θ , q ) ≥ p ] = 1 [ θ ≥ κ ( p, q )] . Therefore, the buyer t will purchase the product if and only if θ t ≥ κ ( p t , q t ) .

Therefore, given the price, advertising p t = p, ρ t = ρ and prior distribution λ , the expected revenue in any round t is given by 5

Let the seller's online policy offer price p t and advertising ρ t at time t , where p t , ρ t can depend on the history of observations/events up to time t . Then expected regret defined in Section 1 can be equivalently written as

Here, the expectation is taken with respect to any randomness in the algorithm's choice of p t , ρ t ; and p ∗ , ρ ∗ = ( ρ ∗ i ) i ∈ [ m ] are defined as the best price and advertising strategy for a given F (and κ ( · , · ) which is determined by F ). Given the expression for Rev ( p, ρ ) derived above, these can be characterized by the following optimization program

2.2 Algorithm design: challenges and ideas

Challenge: high-dimensional continuous decision space. In the last section, we obtained a considerable simplification of the problem by reducing the seller's advertising strategy in every round t to a distribution ρ t ∈ ∆ [0 , 1] over posterior means satisfying the (BC) condition. However, the decision space (a.k.a space of arms) still remains high dimensional and therefore a naive application of (uniform or adaptive) discretization-based bandit techniques, e.g. from Lipschitz bandit literature [44, 56], would not achieve the desired results.

Algorithm design idea: exploring over one-dimesnional type space. Our algorithm uses a 'model-based approach' instead, where we use buyer purchase responses to develop (upper confidence bound) estimates of the demand model D ( θ ) = 1 -F ( θ ) on the points of a discretized type space S ⊆ Θ . Wethen use these upper confidence bounds to construct a piecewise-constant demand function that is an upper confidence bound (UCB) for the demand function D . Then, in each round we solve for the optimal price and advertising strategy by solving an optimization problem similar to P OPT , but with the UCB demand function.

Challenge: efficient discretization of type space. The challenge then is to design a discretization scheme for the type space such that we have a) efficient learning, i.e., the discretized space can be efficiently explored to accurately estimate the demand function on those points, and b) Lipschitz property, i.e., the optimal revenue with the UCB estimate of demand function is close to the true optimal revenue as long as the estimation error on the discretized space is small.

There are two main difficulties in achieving this: (1) Lack of any smoothness/Lipschitz assumption on CDF F . (2) Sensitivity of the Bayes-consistency condition (BC). To see these difficulties, recall that given a price p t and realized posterior mean q t , the t th buyer's purchase decision is given by a t = 1 [ θ t ≥ κ ( p t , q t )] ; thus the seller can obtain demand function estimate at point κ ( p t , q t ) . Without any smoothness or Lipschitz property of demand function, estimates of demand function cannot be extrapolated accurately to other points. This means that in our revenue optimization problem (estimated version of P OPT ), we need to find a price and advertising strategy that we can only use estimates of demand function on a discretized type space, say S ⊆ Θ . However, if we restrict to a discretized type space S , then the support of posterior mean distributions (a.k.a advertising strategy) must be restricted to the points q such that the corresponding critical types κ ( p, q ) are in the set S .

Unfortunately, the (BC) condition makes the set of feasible advertising strategies very sensitive to their support. In particular, if we use uniform-grid based discretization (which is commonly used in previous dynamic pricing literature such as [43, 8]), it is easy to construct examples of prior distribution and valuation function such that there are no or very few feasible advertising strategies with the corresponding restricted support.

Example 2.2. Consider additive valuation function, i.e., v ( θ , ω ) = θ + ω , and thus κ ( p, q ) = p -q . Consider quality space Ω = { ¯ ω i } i ∈ [3] . Given a small ε , consider a uniform-grid based discretization for the type space S , i.e., S = { 0 , ε , 2 ε , . . . , 1 } . If we also use a price p that is from uniform-grid based discretized price space, i.e., p = k ε for some k ∈ N + , then to ensure κ ( p, q ) ∈ S , the support of advertising strategy (i.e., the distribution of the posterior means) must also be restricted within the set S . However, if the prior distribution λ has negligible probabilities on qualities ¯ ω 1 , ¯ ω 3 , and quality ¯ ω 2 / ∈ S , then we cannot construct any posterior distribution with the mean in the set S . Therefore, there does not exist any feasible advertising strategy.

Note that this difficulty cannot be fixed by simply modifying the discretized type space to S ∪ Ω , because even then we would need to construct an advertising such that κ ( p, q ) = p -q ∈ S ∪ Ω in the grid for all p . That would need that the support of the strategy (i.e., posterior means q ) is restricted to be in { k /epsilon1 -¯ ω i } k ∈ N + ,i ∈ [3] ; such posterior means again may not be achieved here.

Algorithm design idea: novel discretization scheme. Our algorithm employs a carefully-designed quality-and-price-dependent discretization scheme. The above example shows that we cannot uniformly discretize price and type using ε -grids, as we may not have any feasible advertising strategy under such discretization. And furthermore, it also shows that this difficulty cannot be fixed by simply adding the m points in quality space to the discretized type space S . Instead, in our discretization scheme, we first uniformly discretize the price space to an ε -grid P . Then to construct a discretized

type space S , in addition to the points on an ε -grid over [0 , 1] , we also include points { κ ( p, ω ) } for every price p ∈ P and quality ω ∈ Ω . This gives us a discretized type space S of size mU/ ε . We claim that our construction ensures that there exist near-optimal price and advertising strategy with support in the discretized type space S . The proof of this claim requires a careful rounding argument, which forms one of the main technical ingredients for our regret analysis in Section 3.

2.3 Details of the proposed algorithm

Our dynamic pricing and advertising algorithm jointly discretizes the price space and type space using the following quality-and-price-dependent discretiztion scheme: given parameter ε , we define the following set:

At time t , we restrict the price and advertising strategies ( p t , ρ t ) to the set of ( p, ρ = ( ρ i ) i ∈ [ m ] ) such that p ∈ P , and given price p , each conditional distribution ρ i has restricted support Q p defined as Q p /defines { q : κ ( p, q ) ∈ S} . Given the price and advertising p t , ρ t , let the realized posterior mean at time t be q t ∼ ρ t , and let the corresponding critical type be x t /defines κ ( p t , q t ) ∈ [0 , 1] . Then, note that the above restrictions on price and advertising strategies guarantee that x t ∈ S .

Next, to compute the offered price and advertising strategy in round t , we optimize an upper confidence bound on the revenue function that we develop using upper confidence bounds D UCB ( x ) , x ∈ S of the demand function computed as follows. For every type x ∈ S , let N t ( x ) denote the set of time rounds before time t that the induced critical type is exactly x , and let N t ( x ) be the number of such time rounds. That is, N t ( x ) /defines { τ < t : κ ( p τ , q τ ) = x } , N t ( x ) /defines |N t ( x ) | , x ∈ S .

Recall that buyer's purchase decision follows a τ = 1 [ θ τ ≥ κ ( p τ , q τ )] . We estimate the demand function at x as: ¯ D t ( x ) /defines ∑ τ ∈ N t ( x ) a τ N t ( x ) . We can now define the following UCB index:

Then, for any pair of discretized price p ∈ P and advertising strategy with discretized support for that price ρ = ( ρ i ∈ ∆ Q p , i ∈ [ m ] } , we define the following seller's revenue estimates:

Above is well-defined since by definition κ ( p, q ) ∈ S for each such ( p, q ) ∈ P × Q p . Finally, we let p t , ρ t be the optimal solution to the following optimization problem:

We summarize our algorithm as Algorithm 1. The main computational bottleneck of Algorithm 1 is to solve the high-dimensional program P UCB t at each time t ≥ |S| + 1 . As we illustrate in Proposition 2.1, there exists an efficient method to optimally solve this program. The proof of this result utilizes the monotoncity of the function D UCB t .

Proposition 2.1 (Adopted from [4, 21]) . Let ε be the discretization parameter for the set P defined in (2) . There exists a polynomial time (in |S| U / ε ) algorithm that can solve the program P UCB t .

Proof. Since function D UCB t is monotone with discontinuities at the points in the set S , when we fix a price p ∈ P , the function D UCB t ( κ ( p, · )) is also monotone with discontinuities at the points in Q p = { q : κ ( p, q ) = x } x ∈ S . Given a prior λ , optimizing a monotone function with discontinuities over all feasible advertising strategies induced from the prior λ subject to the constraint where the support of advertising strategies must be in the set Q p has been studied in [4, 21]. It has been shown that there

Algorithm 1: Algorithm for Dynamic Pricing and Advertising with Demand Learning. 1 Input: Discretization parameter ε . 2 For the first |S| rounds, for each x ∈ S , offer a price p with no information advertising s.t. κ ( p, E ω ∼ λ [ ω ]) = x . // No information advertising provides completely uninformative signal - the distribution φ ( ·| ω ) of signals does not depend on the realized quality ω 3 for each round t = |S| +1 , |S| +2 , . . . , T do 4 For all x ∈ S , compute D UCB t ( x ) as defined in (3). 5 Offer the price p t and an advertising ρ t computed as an optimal solution to program P UCB t . /* p t , ρ t satisfies κ ( p t , q ) ∈ S for every q ∈ supp ( ρ t ) . */ 6 Observe realized posterior mean q t ∼ ρ t and buyer's purchase decision a t ∈ { 0 , 1 } . 7 Update { N t +1 ( x ) , N t +1 ( x ) , ¯ D t +1 ( x ) } x ∈ S .

exists a polynomial (w.r.t. the number of discontinuities) algorithm based on convex programming that can find an optimal advertising strategy (see Proposition 2 in 4). Thus, an exhaustively search over the discretized price space P can lead to an optimal solution to the program P UCB t .

We conclude this section with the following remark on the extension to unbounded type support.

Remark 2.3. Our algorithm and analysis can be extended to the case with unbounded type support (e.g., Θ = [0 , ∞ ) ). In particular, since the price is bounded by [0 , U ] , and the quality is bounded within [0 , 1] , by the monotoncity of the valuation function, we know the critical types κ ( p, q ) induced by any possible p ∈ [0 , U ] and q ∈ [0 , 1] is bounded within [ κ (0 , 1) , κ ( U, 0)] . Thus, an instance with unbounded type support is equivalent to an instance with bounded type support [ κ (0 , 1) , κ ( U, 0)] .

3 Regret Analysis: Proof Overview of Theorem 1.1

In this section, we present our main regret bound for Algorithm 1, as stated in Theorem 1.1. Specifically, we show that for any type CDF F , given a valuation function v ( θ , ω ) that is linear in product quality ω and satisfies Assumption 1, our algorithm (Algorithm 1 with parameter ε = Θ (( m log T / T ) 1 / 3 ) ), has an expected regret of O ( T 2 / 3 ( m log T ) 1 / 3 ) . Importantly, for this result we do not assume any smoothness or Lipschitz properties of distribution F .

For this result, we consider arbitrary but discrete quality space Ω of cardinality m . Later in Appendix B, we show improved regret bounds for the case of additive valuations and equally-spaced quality space (see Theorem B.1), and also extend to arbitrary large and continuous quality spaces (see Theorem B.2).

Proof outline. Recall that in every round t , Algorithm 1 sets the price p t and advertising strategy ρ t as an optimal solution of program P UCB t that approximates the benchmark P OPT in two ways. Firstly, it restricts the price and support of advertising strategy to be in a discretized space P × {Q p , p ∈ P} . Secondly, it approximates the true demand function with an upper bound D UCB t . Our proof consists of two main steps that bound the errors due to each of the above approximations. Due to the space limit, all missing proofs in this section are deferred to Appendix A.

Step 1: bounding the discretization error using a rounding argument (see Appendix A.1). To separate the discretization error from the error due to demand function estimation, we consider an intermediate optimization problem ˜ P (in Appendix A.1) obtained on replacing the UCB demand function D UCB t with the true demand function D (while keeping the discretized space for p, ρ ). Let ˜ p ∗ , ˜ ρ ∗ be an optimal solution of program ˜ P . We show that the revenue Rev ( ˜ p ∗ , ˜ ρ ∗ ) is sufficiently close (within 2 ε ) to the optimal revenue Rev ( p ∗ , ρ ∗ ) . This bound is obtained using a careful rounding argument: we show that the optimal price p ∗ and the optimal advertising ρ ∗ can be rounded to a new price p † and a new advertising ρ † that satisfy

(i) feasibility ( Lemma A.4 ): p † ∈ P , supp ( ρ † ) ⊆ supp ( Q p † ) ; and

(ii) revenue guarantee ( Lemma A.5 ): Rev ( p † , ρ † ) ≥ Rev ( p ∗ , ρ ∗ ) -2 /epsilon1 .

Step 2: bounding estimation error and establishing optimism (see Appendix A.3). Next, we show that the UCB estimates of the demand function D UCB t ( x ) , x ∈ S converge to the true demand function D with high probability, along with concentration bounds on the gap between the true and estimated function ( Lemma A.7 ). This allows us to show that

(i) Revenue optimism ( Lemma A.8 ): we show that the algorithm's revenue estimates are (almost) optimistic, i.e., Rev UCB ( p , ρ ) Rev ( p ∗ , ρ ∗ ) Rev ( p ∗ , ρ ∗ ) 2 ε .

(ii) Revenue approximation ( Lemma A.9 ): we show that the optimistic revenue estimates are close to the true revenue in round t , with the gap between the two being inversely proportional to the number of observations, in particular, Rev UCB t ( p t , ρ t ) -Rev ( p t , ρ t ) ≤ 5 p t E q ρ t log T/N t ( κ ( p t , q )) .

Finally, in Appendix A.7 we put it all together to bound the regret as stated in Theorem 1.1. We first use the above observations to show that regret over each round can be roughly bounded as 2 /epsilon1 T +5 p t E q ∼ ρ t [ √ log T/N t ( κ ( p t , q )) ] . Then, using the constraint ∑ x ∈ S N T ( x ) ≤ T , we show that in the worst case, total regret over time T is bounded by O ( T /epsilon1 + √ |S| T log T ) . The theorem is then obtained by substituting |S| = O ( m / /epsilon1 ) and optimizing ε .

t t t ≥ ˜ ˜ ≥ -∼ [ √ ]

4 Conclusions and Future Direction

In this work, we use a foundational information design framework, Bayesian persuasion, to model the effect of an advertising strategy on customers' beliefs about product quality and consequently their purchase decisions. Our formulation combines the Bayesian persuasion model with dynamic pricing and learning to quantify the tradeoffs between the design of the pricing and advertising strategies and their combined impact on the revenue outcomes. Without any apriori knowledge of the demand function, we show that there exists an efficient online policy that has the regret O ( T 2 / 3 ( m log T ) 1 / 3 ) for a finite state space with cardinality m . This result implies that when the number of the states m is a constant, there is almost no additional learning regret for the seller to additionally learn the optimal advertising compared to the dynamic pricing (without advertising) for non-parametric demand learning problem. There are interesting future directions from this work. First, in our current formulation, buyers' valuation is linear w.r.t the product states. It would be interesting to explore whether our results could be generalized to more general valuation function. Secondly, in our setting, a buyer with i.i.d private type arrives at each time round, and leaves the market no matter what the purchase decision is. However, in practice, buyers may strategize their purchase time for a more favorable price [2, 29, 60]. How to model the advertising effect with strategic buyers and achieve efficient demand learning is another interesting future direction.

References

Yasin Abbasi-Yadkori, David Pal, and Csaba Szepesvari. Online-to-confidence-set conversions and application to sparse stochastic bandits. In Artificial Intelligence and Statistics , pages 1-9. PMLR, 2012.

Kareem Amin, Afshin Rostamizadeh, and Umar Syed. Learning prices for repeated auctions with strategic buyers. Advances in Neural Information Processing Systems , 26, 2013.

Itai Arieli and Yakov Babichenko. Private bayesian persuasion. Journal of Economic Theory , 182:185-217, 2019.

Itai Arieli, Yakov Babichenko, Rann Smorodinsky, and Takuro Yamashita. Optimal persuasion via bi-pooling. In Proceedings of the 21st ACM Conference on Economics and Computation , pages 641-641, 2020.

Pak Hung Au and Keiichi Kawai. Competitive information disclosure by multiple senders. Games and Economic Behavior , 119:56-78, 2020.

Robert J Aumann, Michael Maschler, and Richard E Stearns. Repeated games with incomplete information . MIT press, 1995.

Moshe Babaioff, Robert Kleinberg, and Renato Paes Leme. Optimal mechanisms for selling information. In Proceedings of the 13th ACM Conference on Electronic Commerce , pages 92-109, 2012.

Moshe Babaioff, Shaddin Dughmi, Robert Kleinberg, and Aleksandrs Slivkins. Dynamic pricing with limited supply. 3(1), 2015.

Dirk Bergemann and Alessandro Bonatti. Selling cookies. American Economic Journal: Microeconomics , 7(3):259-94, 2015.

Dirk Bergemann and Stephen Morris. Information design: A unified perspective. Journal of Economic Literature , 57(1):44-95, 2019.

Dirk Bergemann, Alessandro Bonatti, and Alex Smolin. The design and price of information. American economic review , 108(1):1-48, 2018.

Dirk Bergemann, Yang Cai, Grigoris Velegkas, and Mingfei Zhao. Is selling complete information (approximately) optimal? arXiv preprint arXiv:2202.09013 , 2022.

Dirk Bergemann, Tibor Heumann, and Stephen Morris. Screening with persuasion. arXiv preprint arXiv:2212.03360 , 2022.

Dirk Bergemann, Tibor Heumann, Stephen Morris, Constantine Sorokin, and Eyal Winter. Optimal information disclosure in auctions. American Economic Review: Insights , 2022.

Martino Bernasconi, Matteo Castiglioni, Andrea Celli, Alberto Marchesi, Francesco Trovò, and Nicola Gatti. Optimal rates and efficient algorithms for online bayesian persuasion. In International Conference on Machine Learning , pages 2164-2183. PMLR, 2023.

Omar Besbes and Assaf Zeevi. Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Operations Research , 57(6):1407-1420, 2009.

David A Blackwell and Meyer A Girshick. Theory of games and statistical decisions . Courier Corporation, 1979.

Alessandro Bonatti, Munther Dahleh, Thibaut Horel, and Amir Nouripour. Selling information in competitive environments. arXiv preprint arXiv:2202.08780 , 2022.

Peter Bro Miltersen and Or Sheffet. Send mixed signals: earn more, work less. In Proceedings of the 13th ACM Conference on Electronic Commerce , pages 234-247, 2012.

Yang Cai and Grigoris Velegkas. How to sell information optimally: An algorithmic study. In Proceedings of the12th Innovations in Theoretical Computer Science Conference , volume 185, 2021.

Ozan Candogan. Persuasion in networks: Public signals and k-cores. In Proceedings of the 2019 ACM Conference on Economics and Computation , pages 133-134, 2019.

Ozan Candogan and Philipp Strack. Optimal disclosure of information to a privately informed receiver. In Proceedings of the 22nd ACM Conference on Economics and Computation , pages 263-263, 2021.

Matteo Castiglioni, Andrea Celli, Alberto Marchesi, and Nicola Gatti. Online bayesian persuasion. Advances in Neural Information Processing Systems , 33, 2020.

Matteo Castiglioni, Alberto Marchesi, Andrea Celli, and Nicola Gatti. Multi-receiver online bayesian persuasion. In International Conference on Machine Learning , pages 1314-1323. PMLR, 2021.

Yanlin Chen and Jun Zhang. Signalling by bayesian persuasion and pricing strategy. The Economic Journal , 130(628):976-1007, 2020.

Yiling Chen, Haifeng Xu, and Shuran Zheng. Selling information through consulting. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 2412-2431. SIAM, 2020.

Davide Crapis, Bar Ifrach, Costis Maglaras, and Marco Scarsini. Monopoly pricing in the presence of social learning. Management Science , 63(11):3586-3608, 2017.

Bolin Ding, Yiding Feng, Chien-Ju Ho, Wei Tang, and Haifeng Xu. Competitive information design for pandora's box. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 353-381. SIAM, 2023.

Alexey Drutsa. Optimal non-parametric learning in repeated contextual auctions with strategic buyer. In International Conference on Machine Learning , pages 2668-2677. PMLR, 2020.

Shaddin Dughmi. Algorithmic information structure design: a survey. ACM SIGecom Exchanges , 15(2):2-24, 2017.

Adam N Elmachtoub and Michael L Hamilton. The power of opaque products in pricing. Management Science , 67(8):4686-4702, 2021.

Yuval Emek, Michal Feldman, Iftah Gamzu, Renato PaesLeme, and Moshe Tennenholtz. Signaling schemes for revenue maximization. ACMTransactions on Economics and Computation (TEAC) , 2(2):1-19, 2014.

Yiding Feng, Wei Tang, and Haifeng Xu. Online bayesian recommendation with no regret. In Proceedings of the 23rd ACM Conference on Economics and Computation , pages 818-819, 2022.

Ilwoo Hwang, Kyungmin Kim, and Raphael Boleslavsky. Competitive advertising and pricing. 2019.

Bar Ifrach, Costis Maglaras, Marco Scarsini, and Anna Zseleva. Bayesian social learning from consumer reviews. Operations Research , 67(5):1209-1221, 2019.

Nicole Immorlica, Jieming Mao, Aleksandrs Slivkins, and Zhiwei Steven Wu. Incentivizing exploration with selective data disclosure. In Proceedings of the 21st ACM Conference on Economics and Computation , pages 647-648, 2020.

Kenneth L Judd and Michael H Riordan. Price and quality in a new product monopoly. The Review of Economic Studies , 61(4):773-789, 1994.

Emir Kamenica. Bayesian persuasion and information design. Annual Review of Economics , 11:249-272, 2019.

Emir Kamenica and Matthew Gentzkow. Bayesian persuasion. American Economic Review , 101(6):2590-2615, 2011.

Kei Kawai, Ken Onishi, and Kosuke Uetake. Signaling in online credit markets. Journal of Political Economy , 130(6):000-000, 2022.

N Bora Keskin and Assaf Zeevi. Dynamic pricing with an unknown demand model: Asymptotically optimal semi-myopic policies. Operations research , 62(5):1142-1167, 2014.

Richard E Kihlstrom and Michael H Riordan. Advertising as a signal. Journal of Political Economy , 92(3):427-450, 1984.

Robert Kleinberg and Tom Leighton. The value of knowing a demand curve: Bounds on regret for online posted-price auctions. In 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings. , pages 594-605. IEEE, 2003.

Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Multi-armed bandits in metric spaces. In Proceedings of the fortieth annual ACM symposium on Theory of computing , pages 681-690, 2008.

Anton Kolotilin, Tymofiy Mylovanov, Andriy Zapechelnyuk, and Ming Li. Persuasion of a privately informed receiver. Econometrica , 85(6):1949-1964, 2017.

Ilan Kremer, Yishay Mansour, and Motty Perry. Implementing the 'wisdom of the crowd'. Journal of Political Economy , 122(5):988-1012, 2014.

Yingkai Li. Selling data to an agent with endogenous information. In Proceedings of the 23rd ACM Conference on Economics and Computation , pages 664-665, 2022.

Shuze Liu, Weiran Shen, and Haifeng Xu. Optimal pricing of information. In Proceedings of the 22nd ACM Conference on Economics and Computation , pages 693-693, 2021.

Puneet Manchanda, Jean-Pierre Dubé, Khim Yong Goh, and Pradeep K Chintagunta. The effect of banner advertising on internet purchasing. Journal of Marketing Research , 43(1): 98-108, 2006.

Yishay Mansour, Aleksandrs Slivkins, and Vasilis Syrgkanis. Bayesian incentive-compatible bandit exploration. In Proceedings of the Sixteenth ACM Conference on Economics and Computation , pages 565-582, 2015.

Yishay Mansour, Alex Slivkins, Vasilis Syrgkanis, and Zhiwei Steven Wu. Bayesian exploration: Incentivizing exploration in bayesian games. Operations Research , 70(2):1105-1127, 2022.

Paul Milgrom and John Roberts. Price and advertising signals of product quality. Journal of political economy , 94(4):796-821, 1986.

Phillip Nelson. Information and consumer behavior. Journal of political economy , 78(2): 311-329, 1970.

Phillip Nelson. Advertising as information. Journal of political economy , 82(4):729-754, 1974.

Navdeep S Sahni and Harikesh S Nair. Does advertising serve as a signal? evidence from a field experiment in mobile search. The Review of Economic Studies , 87(3):1529-1564, 2020.

Aleksandrs Slivkins. Contextual bandits with similarity information. In Proceedings of the 24th annual Conference On Learning Theory , pages 679-702. JMLR Workshop and Conference Proceedings, 2011.

Wei Tang and Chien-Ju Ho. On the bayesian rational assumption in information design. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , volume 9, pages 120-130, 2021.

Jibang Wu, Zixuan Zhang, Zhe Feng, Zhaoran Wang, Zhuoran Yang, Michael I Jordan, and Haifeng Xu. Sequential information design: Markov persuasion process and its efficient reinforcement learning. In Proceedings of the 23th ACM Conference on Economics and Computation , 2022.

Shuran Zheng and Yiling Chen. Optimal advertising for information products. In Proceedings of the 22nd ACM Conference on Economics and Computation , pages 888-906, 2021.

Anton Zhiyanov and Alexey Drutsa. Bisection-based pricing for repeated contextual auctions against strategic buyer. In International Conference on Machine Learning , pages 1146911480. PMLR, 2020.

You Zu, Krishnamurthy Iyer, and Haifeng Xu. Learning to persuade on the fly: Robustness against ignorance. In Proceedings of the 22nd ACM Conference on Economics and Computation , pages 927-928, 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies a dynamic pricing model augmented by advertisement campaigns. The problem is framed in a Bayesian persuasion framework, where the authors focus on the online learning version of the model in which the agent is not aware of buyer the demand function from which its types are drawn.\n\nSTRENGTHS:\nThe paper studies an interesting model of dynamic pricing, which leans on the Bayesian persuasion framework. Due to the continuity/size of the action space, standard techniques cannot be applied and the results presented seems technically interesting.\nMoreover the paper is mostly clear and easy to follow and does a good job at motivating the model.\n\nWEAKNESSES:\nI found the description of the discretisation (line 308) a bit hard to follow. A little bit more intuitive discussion on this would be appreciated  for this part.\nSee the question on the lower bound in the Question section of the review\nIf convinced on this point I’m willing to revise my score\n\nQUESTIONS:\n1) would revelation like principle hold in this context? Could the space of signals be made binary as the action set or not? A brief discussion on this would be interesting for the BP readers.\n2) The types in this problem are stochastic. What if they would be drawn by an adversary? Most of the online BP literature studies the adversarial problem.\n3) the authors cite a T^{2/3} lower bound for the dynamic pricing problem. I guess that the your extended model reduces to the standard DP model when the utilities are zero sum and thus the sender commits to uniformly random signalling schemes and thus the posterior would be equal to the prior. Am I correct? If so, I would like to be convinced that zero sum utilities between seller and buyer can be achieved in your model. Moreover more recent UB for the online BP problem with adversarial types have been found, and guarantee a sqrt(T) regret. The work I’m referencing is [Bernasconi, Martino, et al. \"Optimal Rates and Efficient Algorithms for Online Bayesian Persuasion.\" (2023).] which I think should be discussed.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This work focuses on advertising and selling markets under the Bayesian persuasion framework. On the one hand, this paper models the effect of the advertisement strategy on customers’ beliefs and purchase decisions. On the other hand, the authors also address the problem of online learning the optimal advertisement strategy via learning the buyers’ type distribution and show $O(T^{2/3} m^{1/3})$ regret for discrete quality space and $O(T^{3/4})$ regret for arbitrary quality space.

STRENGTHS:
The problem and model discussed in this paper are intriguing, and the authors devise a novel quality-and-price-dependent discretization scheme for proving the learning result. Notably, the learning process only necessitates observations of the buyers' purchase decisions, eliminating the need for explicit knowledge of their individual types. This work may be of interest to the community.

WEAKNESSES:
Is the value function required to be linear in $\omega$? I notice that using $q$ for simplification relies on this linearity assumption. Additionally, in order to determine the critical type, the seller would typically need prior knowledge of the buyers' value function, which may not always be feasible. Is it possible to relax this requirement within this framework?

QUESTIONS:
1. Are the regret bounds of $O(T^{2/3})$ and $O(T^{3/4})$ tight for discrete and continuous quality spaces, respectively?
2. Is there a deeper understanding of the optimization program P_opt? Can it provide insights into the phenomenon where revealing more or less information about the product can sometimes lead to increased revenue?

LIMITATIONS:
No concerns here.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider an online single product dynamic pricing product, where a buyer's purchase decision is not only influenced by the offered price from the selling, but also depends on the advertising strategy used by the seller. The authors formalized the latter via the model of Bayesian persuasion framework, which is a well establish model in the economic literature (to the best of my knowledge). In the only problem, the seller is uncertain about the type distribution of a customer, but the seller knows the probability distribution on the quality of a product (which is a random variable). The authors' main results is the design and analysis of a UCB algorithm, where the authors crucially use a mild assumption on the customer's utility to reduce the size of decision variables on the advertising strategy. The regret  bound derived is nearly tight (module a factor of $m^{1/3}$, where $m$ is the number of possible quality. )\n\nSTRENGTHS:\n- The setting and the algorithm design, analysis seem novel, and this is the first piece of work on online revenue mgmt with Bayesian persuasion to my knowledge. \n\n- Despite the complexity of the set up and the algorithm, the authors provide enough intuitions for readers to understand the content, and I feel that the proposed work could inspire further interest in other pricing problems. \n\n- The whole algorithm framework is well crafted such that the tightness on $T$ is achieved.\n\nWEAKNESSES:\n- If I understand the model set up correctly (please also see Question (1) in \"Questions\"), the seller seems to need to know $\\mu_t$ that relates to the time t customer's perception on a product's quality given the seller's advertising strategy. This assumption could appear a bit strong, since in most online pricing setting the learning agent should be uncertain about a customer's purchase characteristics.\n\n- I find the modeling assumption on the product quality being random a little strange. Putting myself as a seller who can observe the quality of a product before an action, it could seem natural for myself to sell products of high quality first (let say if I see a low quality product, then I will just forgo it, and repeat until I can find a high quality product for the customer) in order to yield a higher revenue? If the seller could not observe the quality of a product, I do find modeling the unseen quality as a random variable reasonable.\n\n- There is a looseness in $m$, but I regard it as a minor weakness given the complexity of the problem.\n\n- There is a lack of numerical results, which could yield some immediate intuitions.\n\nQUESTIONS:\n(1) Can the authors explicit confirm if the seller possesses the knowledge on $\\mu_t$, the posterior distribution of time $t$ customer's perceived quality conditional on an advertisement strategy? Just to double confirm my understanding, in the reduction from selecting $\\sigma_t$ to the selection of $\\rho_t$ which is a uni-variate probability distribution, the reduction involves considering the mean of a mixture of different $\\mu_t$ distribution. Thus, in order to arrive with the desired $\\rho_t$, we need to have the correct mixture, which involves knowing $\\mu_t$. Can I please check if my understanding above is correct?\n\n(2) Can the author improve the dependence on $T$ if we have a discrete set of (say $K$) prices?\n\n(3) If my understanding in Question (1) is correct, could there be a way to extend the framework to the case when $\\mu_t$ is stationary across time but not known to the seller, or is there a $\\Omega(T)$ regret lower bound due to the insufficiency in the information in the outcome?\n\nLIMITATIONS:\nThere is no negative societal impact, this is a theoretical work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper extends the traditional dynamic pricing model (e.g. Kleinberg and Leighton) to also include product quality. In each period, we get both a random type of the buyer as well as a random quality of the product. The seller can employ an advertising strategy, which is a mapping from product quality to a signal, which must be decided in each step before the signal is realized. To be more precise, in each period:\n* buyer type is drawn $\\theta_t \\sim F$\n* seller decides on a signaling scheme mapping item quality $\\omega$ to a signal $\\sigma$\n* seller also decides on a price $p_t$ for the product\n* item quality $\\omega \\sim \\lambda$ is realized. Signal $\\sigma$ is computed and sent to the buyer:\n* buyer decides to purchase if $\\mathbb{E}[\\theta, \\omega) \\mid \\sigma] \\geq p_t$\n\n\nThe seller knows all parameters of the model except $F$;. The goal of the seller is to learn $F$ and compete against the algorithm that knows $F$ and chooses the prices and advertising strategies optimally in each period. Naive MAB approaches are too high dimensional, so the authors use the following clever idea: define the threshold type $\\kappa(p_t, \\rho_t)$ which is the threshold specifying above which types they buy. Now, If we choose the signaling scheme carefully we can observe whether $\\theta$ was above or below $\\kappa(p_t, \\rho_t)$. with that, we can build an estimate of $F$. The question now becomes discretizing the prices and advertising strategies appropriately. Once the estimate is built, we use the UCB estimates to solve a program and allocate according to that.\n\nThe algorithm has $O(T^{2/3})$ regret due to two effects: (i) price discretization; (ii) explore and exploit.\n\nSTRENGTHS:\n* Interesting model mixing persuasion and pricing. This is a nice and natural extension of the pricing model.\n\n* There is a clever idea: instead of discretizing in the space of signals, we discretize prices and estimate $F$ directly.\n\n* The results match the lower bound of Kleinberg-Leighton. On that, the authors should probably discuss more why the Kleinberg-Leighton lower bound applies to this problem. I imagine their setting corresponds to known quality, so we don’t need to signal anything. I recommend saying this explicitly.\n\nWEAKNESSES:\n* In terms of motivation, it is unclear that we change the advertising so fast. Typically it takes a few iterations with the signaling scheme for a buyer to learn the mapping from signals to actions. Now that the buyer is seeing a new one in each period, I don't see how this would translate to a practical situation. \n\n\n* Mention \"linear\" in Assumption 1a to make it clear. From reading assumption 1, I was under the impression that we were just assuming $v$ to be monotone on $\\omega$. Btw: I think you mean \"affine\" instead of linear. The function $v(\\theta, \\omega) = \\theta + \\omega$ is not linear (in the usual definition of linear)\n\n* I think Explore-then-Exploit are underwhelming in terms of ML algorithms. You can always learn the parameters in an explore phase and just optimize. There are non-trivial many aspects of the error analysis, but in terms of algorithmic techniques, I found it to be limited.\n\n* The paper gets too long to get to the actual algorithm with many discussions that could be done much faster. For example, teh entire discussion in Section 2.1 is the Splitting Lemma of Aumann-Maschler.\n\nQUESTIONS:\nSee my comments in the last box.\n\nLIMITATIONS:\nNo concerns.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors study a novel setup of repeated posted-price auctions, where a single seller deals with a single myopic buyer (or, equiv, iid new buyer in each round). In each round, a new good is offered and has characteristics that affect buyer valuation (so, goods are not equal between rounds). The novelty of the setup consists in introducing a new interaction step, where the seller reveals characteristics of each good through some signals that might improve perception of the buyer. The buyer is aware of this step, which is referred to as advertising (due to analogy with cases in practice).\nSo, in each round the seller submits two inputs (price, advertising signal), while the buyer reacts by a decision: accept or reject the price for this good. \n\nThe authors clearly argue why this setup differs from earlier considered ones (either price-only, or advertising-only). They provide a novel algorithm (which is announced in advance. before the game) and prove that this algorithm has regret upper bound which is slightly above the lower bound by the factor of log T (for the setup without advertising).\n\nSTRENGTHS:\n- novel setup in the area of repeated auctions with plenty opportunity for future work\n- new algorithm with theoretical guarantees \n- easy understandable basic ideas behind the proofs\n\nWEAKNESSES:\n- presentation (a lot of space for improvement)\n- some questions to setup and setting\n- lack of analysis and discussion of proposed algorithm (computational complexity, limitations in practice, ...)\n(see “Questions” for details)\n\nQUESTIONS:\nA. Setting questions and other matter stuff:\n\n1. Line 70 about definition of Omega: Any extentions to multidimentional \\Omega?\n\n2. Footnote 3: “ This is motivated by the fact that at each round, the buyer is facing a fresh product, whose quality is drawn independently across time.” I would add that this is not just because of it. But, also because, in each round, we consider a fresh buyer (myopic to the other rounds). This setting might be changed even in the case of different products. For instance, see settings with a single buyer that makes strategic decisions to optimize his/her utility over all rounds in the cases of different products: [Zhiyanov2020, Drutsa2020] (by the way, these works are extensions of the work [39] Kleinberg et al 2003 you are citing).\nSo, I would suggest upgrading your footnote by adding these thoughts and probably indicate some future work about injecting the strategic buyer in your setting.\n\n3. Lines 96-97 “and a static price and advertising strategy maximizes total expected revenue over the time horizon T” Hard to parse this sentence. It sounds like, static (p, \\phi) are the best solutions over all possible ({p_t} , {\\phi_t}). The authors seem to state that these are the best among static p_t, \\phi_t. If yes, please reformulate the sentence to be clearer\n\n4. Formula after Line 99: \nOver what is expectation taken right after the \\Sum sign? It looks like all randomness is under the hood of Rev()..\nOr, ... do we expect mixed (non-pure) strategies of the Seller?\n\n5. Line 102 “of the (discrete) product quality space.” What will be when \\Omega is not discrete? E.g., continuous?\n\n6. Footnote 4 “ arbitrary Lipschitz constant can be treated similarly.” It would be nice to understand how this constant participates in the upper bound .. E.g. in workds [Mao2018, Drutsa2020], it plays some notable role and shows severe dependance on the degree of Lipschitz assumption. Is it true in your case?\n\n7. Algorithm 1. Line 5. “ computed as an optimal solution to program P^{UCB}.” What are the costs for finding optimal solution in P_t^{UCB}? Overall, it would be nice to clearly address complexity of the algorithm in a separate paragraph in the text with discussions. \n\n\nB. Presentation/wording issues:\n\n1. Lines 14-17. It is written “the valuation function is linear in the product quality” and “This result requires some natural monotonicity and Lipschitz assumptions on the valuation function…”. It is unclear: linear valuation is Lipschitz by definition... Or result is about non-parametric valuation.. ?\nLater, in the main text, I found that valuation depends on 2 parameters, where properties on linearity and Lipschitz are on different parameters. I suggest rewriting Line 17 to make it clearer from abstract. \n\n2. Structure of the paper. I suggest to improve structure of the work. At least\n- By splitting Inroduction in separate Sections: Intro, Problem formulation (statement)\n- By adding conclusion\n\n3. Line 70 “with a public prior distribution \\lambda on.. ” Hard to parse. Only in Line 78, I've got that lambda is ex-ante for the whole game play (before round 1). \n\n4. Lines 81-82: “if the expected valuation” Why is this the reasonable way to decide? I mean this way to transform quality distribution to a number for comparison with price? Maybe, Median, Moda, Quantiles are better? \nIs it because Buyer maximizes some utility? If yes, what is the utility in this case?\nLater I found the answer in Lines 215-217. However, I'd suggest improving the text in Page2 by adding the info that the setting is about Bayesian rational buyer\n\n5. Line 287: “we can only use estimates of demand function on a discretized type space, say S” Hard to read. Please, simplify. \n\n6. Algorithm 1. Line 2 “any no”. I do not understand \"any no\". \n\n7. Lines 353-358 (and many other earlier): Text here repeats earlier stated (several times) assumptions and reference of results to Appendices. \nIt is better to save space and show more details on proofs, limitations, computational complexity, and conclusions...\n\n\nC. Minor issues:\n- Lines 19-20 “the regret lower bound for dynamic pricing within logarithmic factors, which is a special case of our problem.” Wording: hard to parse what is the special case: factor or pricing. Please, rewrite\n- Line 167: “the these” -> “these”?\n- No line counts between Lines 217-218\n- Line 286: “to solve to find” -> “to find”?\n\n\n\nLITERATURE: \n[Mao2018] Mao,  Leme, and Schneider “Contextual pricing for lipschitz buyers” NeurIPS 2018.\n[Zhiyanov2020] Zhiyanov and Drutsa \"Bisection-based pricing for repeated contextual auctions against strategic buyer.\" ICML  2020.\n[Drutsa2020] Drutsa \"Optimal non-parametric learning in repeated contextual auctions with strategic buyer.\" ICML 2020.\n\n\n\n\n========\n\nAFTER REBUTTAL\nI thank the authors for answering several questions from my list. I hope that the answers will be reflected and fully addressed in the new revision of the work - including those questions that remain without comments from the authors.\n\nLIMITATIONS:\nNo. I would suggest adding a paragraph about computational complexity analysis.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-18"}