SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Shipra Agrawal
Columbia University sa3305@columbia.edu
Yiding Feng
University of Chicago yidingfeng@uchicago.edu
Wei Tang  
Columbia University  
wt2359@columbia.edu
We consider a novel dynamic pricing and learning setting where in addition to setting prices of products in sequential rounds, the seller also ex-ante commits to 'advertising schemes'. That is, in the beginning of each round the seller can decide what kind of signal they will provide to the buyer about the product's quality upon realization. Using the popular Bayesian persuasion framework to model the effect of these signals on the buyers' valuation and purchase responses, we formulate the problem of finding an optimal design of the advertising scheme along with a pricing scheme that maximizes the seller's expected revenue. Without any apriori knowledge of the buyers' demand function, our goal is to design an online algorithm that can use past purchase responses to adaptively learn the optimal pricing and advertising strategy. We study the regret of the algorithm when compared to the optimal clairvoyant price and advertising scheme.
Our main result is a computationally efficient online algorithm that achieves an  $O(T^{2/3}(m \log T)^{1/3})$  regret bound when the valuation function is linear in the product quality. Here  $m$  is the cardinality of the discrete product quality domain and  $T$  is the time horizon. This result requires some natural monotonicity and Lipschitz assumptions on the valuation function, but no Lipschitz or smoothness assumption on the buyers' demand function. For constant  $m$ , our result matches the regret lower bound for dynamic pricing within logarithmic factors, which is a special case of our problem. We also obtain several improved results for the widely considered special case of additive valuations, including an  $\tilde{O}\left(T^{2/3}\right)$  regret bound independent of  $m$  when  $m \leq T^{1/3}$ .
Dynamic pricing is a key strategy in revenue management that allows sellers to anticipate and influence demand in order to maximize revenue and/or utility. When the customer valuation and demand response for a product is apriori unknown, price variation can also be used to observe and learn the demand function in order to adaptively optimize price and revenue over time. This learning and optimization problem has been a focus of much recent literature that uses exploration-exploitation and multi-armed bandit techniques with dynamic pricing algorithms (e.g., see [43, 16, 41, 8]).
In practice, there is another important tool available to sellers in the form of advertising, using which the sellers can inform and shape customers' valuations of a product. It has been theoretically [53, 54] and empirically [55, 49] shown that advertisements can serve as a credible signal of the quality or characteristics of the advertised product. Sellers can use advertising to provide partial information about a product in order to better position the product in the market and potentially increase customers' chances of purchasing the product. For example, as a common strategy to drive
A full version of this work is at https://arxiv.org/abs/2304.14385
subscriptions, online newspapers may use a "teaser" that selectively includes previews of some news articles that are likely to entice readers to subscribe for access to the full story; in the online used-car market, the dealer can advertise the used car by emphasizing different aspects of the car, such as fuel efficiency/mileage/unique features, or selectively disclose history-report information from reputable third parties, catering to specific buyer interests; a film distributor may advertise the movie by selectively showing footage from the film.
However, advertising must be carefully designed to achieve the desired gains. At one glance up-selling or inflating the product quality by selectively disclosing only favorable information might appear as a profitable advertising strategy. But such strategies carry the disadvantage of not being very effective in modifying customer beliefs as customers may not trust that the provided information accurately reflects the product's true quality. Also, the design of the advertising strategy needs to interact with the design of the pricing strategy and account for the demand function. For example, to sell highly-priced products or under heavy competition/low demand, the customer may need to be convinced of a good match through more information and thorough insights about the product characteristics. On the other hand, in markets with high demand or for very low-priced goods, the seller may get away with revealing very little information. An extreme example of this phenomenon is the concept of mystery deal boxes sold by some retailers like Amazon/Woot, where the customers are not even made aware of the exact contents of the low-cost box that they are purchasing.[2]
In this paper, we use a Bayesian persuasion framework [39] to model the effect of an advertising strategy on customers' beliefs about product quality and consequently their purchase decisions. Our novel formulation combines the Bayesian persuasion model with dynamic pricing and learning in order to quantify the tradeoffs between the design of the pricing and advertising strategies and their combined impact on the revenue outcomes. Without any apriori knowledge of the demand function, our goal is to design an online algorithm that can use past customer responses to adaptively learn a joint pricing and advertising strategy that maximizes the seller's revenue.
Bayesian persuasion is a popular framework for information design with several different settings considered in the literature [39, 30, 38, 10]. We consider a Bayesian persuasion model where the sender (seller) ex-ante commits to an information policy (advertising strategy) that prescribes the distribution of signals the sender will provide to the receiver (buyer) on observing the true state of the world (product quality). The receiver, on observing the sender's signal, uses Bayes' rule to form a posteriori on the state of the world. The receiver's action (purchase decision) then optimizes their expected utility under this posterior.
Problem formulation. Specifically, our dynamic pricing and advertising problem is formulated as follows. There are  $T$  sequential and discrete rounds. In each round, a fresh product is offered by the seller, with a public prior distribution  $\lambda$  on the product quality  $\omega \in \Omega \subseteq [0,1]$ . At the beginning of each round  $t$ , before observing the realized quality of the  $t^{th}$  product, the seller commits to a price  $p_t \in [0,U]$  and an advertising strategy  $\phi_t$ , where  $\phi_t(\sigma|\omega)$  prescribes the distribution of signal  $\sigma \in \Sigma$  where  $\Sigma$  is an arbitrary signaling space given the realized product quality  $\omega \in \Omega$ .
A buyer arrives in each round  $t$  with private type  $\theta_t$  generated i.i.d. from a distribution with CDF  $F(\cdot)$  and support  $\Theta = [0,1]$ . The CDF  $F(\cdot)$  (or equivalently the demand function  $D(\cdot) \triangleq 1 - F(\cdot)$ ) is fixed but unknown to the seller. For a buyer of type  $\theta \in \Theta$ , the valuation of a product with product quality  $\omega$  is given by function  $v(\theta, \omega)$ .
The  $t^{th}$  product quality  $\omega_{t} \sim \lambda$  is then realized and observed by the seller. The buyer cannot observe the realized product quality, but only a signal  $\sigma_{t} \sim \phi_{t}(\cdot|\omega_{t})$  provided by the seller. The buyer uses this signal along with the prior  $\lambda$  to formulate a Bayesian posterior distribution on the product quality  $\mu_{t}(\omega|\sigma_{t}) \propto \phi_{t}(\sigma_{t}|\omega) \cdot \lambda(\omega)$ . The buyer then purchases the product if and only if the expected valuation under this posterior  $\mathbb{E}_{\omega \sim \mu_{t}(\cdot|\sigma_{t})}[v(\theta_{t},\omega)]$  is greater than or equal to the price  $p_{t}$ . We denote the buyer decision at time  $t$  by  $a_{t} \in \{0,1\}$  with  $a_{t} = 1$  denoting purchase.
Remark 1.1. We consider the setting where buyers use only the prior distribution and signal in the current round, and not the signals in the past rounds, to make their decisions. This is motivated by
2For example, when selling the opaque products, the precise product features or characteristics are hidden from the customers [31].
3Our results can be generalized to the setting where  $\Theta$  is any compact interval  $[\underline{\theta},\bar{\theta} ]\subseteq \mathbb{R}^{+}$ , or unbounded (see Remark 2.3).
the fact that at each round, the buyer is facing a fresh product, whose quality is drawn independently across time. Thus, previous signals do not carry information about the current product. Fresh products with independent qualities are common in many real-world applications such as secondhand markets, mystery boxes sold by Amazon/Woot, etc. This similar problem structure has also been studied in the online/sequential Bayesian persuasion literature with repeated interactions of sender and receivers, for example, see [61 24 23 33 58 15].
A summary of the game timeline is as follows: at  $t \in [T]$ , (1) the seller commits to a price  $p_t \in [0, U]$  and an advertising strategy  $\phi_t$ ; (2) a buyer  $t$  with private type  $\theta_t \sim F$  arrives; (3) a product with quality  $\omega_t \sim \lambda$  is realized; the seller sends signal  $\sigma_t \sim \phi_t(\cdot | \omega_t)$  to the buyer; (4) the buyer formulates Bayesian posterior  $\mu_t(\omega | \sigma_t)$ , and the buyer purchases the product (denoted by  $a_t = 1$ ) to generate revenue  $p_t$  if and only if her expected value exceeds the price, i.e.,  $\mathbb{E}_{\omega \sim \mu_t(\cdot | \sigma_t)}[v(\theta_t, \omega)] \geq p_t$ .
Our goal is to design an online learning algorithm that sequentially chooses the price and advertising strategy  $p_t$ ,  $\phi_t$  in each round  $t$  based on the buyers' responses in the previous rounds, in order to optimize total expected revenue over a time horizon  $T$  without apriori knowledge of the distribution  $F$ . Let  $\mathrm{Rev}(p_t, \phi_t) \triangleq \mathbb{E}[p_t \cdot a_t; p_t, \phi_t]$  denote the expected revenue at time  $t$  under the price  $p_t$  and advertising strategy  $\phi_t$ . Here the expectation is over realizations of customer type  $\theta_t \sim F$ , product quality  $\omega_t \sim \lambda$  and advertising signal  $\sigma_t \sim \phi_t(\cdot | \omega_t)$ . Note that since product quality and types are i.i.d. across time, for any given  $p_t = p, \phi_t = \phi$ , the expected per-round revenue  $\mathrm{Rev}(p, \phi)$  does not depend on time. Thus a static price and advertising strategy maximizes total expected revenue over the time horizon  $T$ . Therefore, we can measure the performance of an algorithm in terms of regret that compares the total expected revenue of the algorithm to that of the best static pricing and advertising strategy. We define
$$
\operatorname {R e g r e t} (T) \triangleq T \max  _ {p, \phi} \operatorname {R e v} (p, \phi) - \sum_ {t = 1} ^ {T} \operatorname {R e v} \left(p _ {t}, \phi_ {t}\right).
$$
Our contributions. In this work, we present a computationally efficient online pricing and advertis-
ing algorithm that achieves an  $O\left(T^{2/3}(m \log T)^{1/3}\right)$  bound on the regret in time  $T$ , where  $m = |\Omega|$  is the cardinality of the (discrete) product quality space. Importantly, we achieve this result without any assumptions like Lipschitz or smoothness on the demand function  $D(\cdot) = 1 - F(\cdot)$ . However, our results require certain assumptions on the valuation function. Following the literature, we make the common assumption that the function  $v(\theta, \omega)$  is linear in the product quality  $\omega$ . Furthermore, we assume the following monotonicity and Lipschitz properties on the valuation function.
Assumption 1. Buyer's valuation function  $v(\cdot, \cdot)$  satisfies:
1a Fix any buyer type  $\theta$ , function  $v(\theta, \omega)$  is non-decreasing w.r.t. quality  $\omega$ .
1b Fix any quality  $\omega$ , function  $v(\theta, \omega)$  is increasing and 1-Lipschitz w.r.t. type  $\theta$ .
Such assumptions are in fact common in literature and natural in many economic situations where the valuation of a product increases with the product quality and buyer's type (paying ability/need). Existing literature on Bayesian persuasion/dynamic pricing often makes even stronger assumptions about the receiver's utility function. For example, both the additive functions  $v(\theta, \omega) = \theta + \omega$  [cf. 35, 45, 27] and the multiplicative functions  $v(\theta, \omega) = \theta \omega + \theta$  [cf. 22, 48], are linear in  $\omega$  and satisfy Assumption 1. Our main result is then summarized as follows:
Theorem 1.1. For any type CDF  $F$ , given a valuation function  $v(\theta, \omega)$  that is linear in product quality  $\omega$  and satisfies Assumption  $\boxed{I}$  Algorithm  $\boxed{I}$  with parameter  $\varepsilon = \Theta((m \log T / T)^{1/3})$  has an expected regret of  $O\left(T^{2/3}(m \log T)^{1/3}\right)$ . Here,  $m$  is the cardinality of the discrete quality space  $\Omega$ .
Furthermore, we obtain several improved results for the widely considered special case of additive valuations, i.e., for  $v(\theta, \omega) = \theta + \omega$ . See Appendix B for the formal statements and analysis.
1. (Theorem B.1) Consider discrete sets  $\Omega$  that are 'equally-spaced', e.g.,  $\Omega = \{0, 1\}$  or  $\Omega = [m]$ . Given such a product quality space  $\Omega$  and additive valuation function, we show that the regret of Algorithm 1 is bounded by  $O(T^{2/3} (\log T)^{1/3})$  when  $m \leq (T / \log T)^{1/3}$ , and by  $O(\sqrt{mT \log T})$  for larger  $m$ .
Here 1 Lipschitz constant is for exposition simplicity, an arbitrary Lipschitz constant  $L$  can be treated similarly.
2. (Theorem B.2) For any arbitrary (discrete or continuous) product quality space  $\Omega$ , given additive valuation functions, we have a slightly modified algorithm (see Algorithm 3 in Appendix C.2) with an expected regret of  $O(T^{3/4} (\log T)^{1/4})$  independent of  $m$ .
When the valuation function  $v(\cdot, \omega)$  is  $L$ -Lipschitz w.r.t. type  $\theta$ , our regret bound in Theorem 1.1 would become  $O(T^{2/3}(Lm\log T)^{1/3})$ , and the regret bound for arbitrary space  $\Omega$  would become  $O(T^{3/4}(L\log T)^{1/4})$ .
We might compare our results to the best regret bounds available for the well-studied dynamic pricing and learning problem with unlimited supply [43, 8], which is a special case of our problem if the product quality is deterministic, i.e.,  $m = 1$ , and the advertising scheme reveals no information and thus has no impact on the buyer's purchase decision. For the dynamic pricing problem a lower bound of  $\Omega (T^{2 / 3})$  on regret is known [43]. Therefore the dependence on  $T$  in our results cannot be improved. In fact, our result matches this lower bound in the case of binary or constant size quality space, which are common settings in information design literature [39, 18, 5, 57, 33, 12].
High-level descriptions of the proposed algorithm and challenges. Our problem can be viewed as a very high dimensional combinatorial multi-armed bandit problem, where each arm is a pair of a price and a feasible advertising strategy: the set of feasible advertising strategies being the set of all possible conditional distributions  $\{\phi (\cdot |\omega),\omega \in \Omega \}$  over signal space  $\Sigma$ . As a first step towards obtaining a more tractable setting, we present an equivalent reformulation of the problem which uses the observation that advertising affects the buyer's decision only via the posterior distribution over quality. By the linearity of valuation function  $v(\cdot ,\cdot)$  over product quality  $\omega$ , seller's choice of advertising strategy in every round can be further simplified to selecting a distribution over posterior means that is subject to a feasibility constraint.
From here, the seller's decision space now becomes two-dimensional (a price and a distribution of posterior means). Viewing seller's expected revenue as an unknown (nonlinear) function over this two-dimensional decision space, one may consider applying algorithms in contextual bandits or Lipschitz bandits to get sublinear regrets, e.g.,  $\tilde{O}(T^{3/4}\mathrm{poly}(m))$  regret for two-dimensional decision space if there exists a Lipschitz property of reward function relative to seller's decision space. However, it is unclear whether one can establish such Lipschitz property given that we do not assume Lipschitzness or smoothness on demand function and we have complex constraints on the feasibility of advertising space. Instead, in our algorithm we use a 'model-based approach': we use buyers' purchase responses to explore the demand function over the (discretized) type space and jointly learn the optimal advertising and pricing. To explore the demand function over the one-dimensional (discretized) type space, we propose a novel discretization scheme such that it enables near-optimal price and advertising strategy even without Lipschitzness or smoothness assumption and with the complex feasibility constraint on the advertising space. These treatments lead us to the optimal  $\tilde{O}(T^{2/3}m^{1/3})$  regret.
Related work. Our work is related to several streams of research. Below we briefly review the some of these connections.
In our setting, the seller can utilize her information advantage to design an advertisement to signal the product quality to the buyer. There is a long line of research in the literature, from both empirical and theoretical perspective, dedicated to study how to use advertisement as a signal to steer buyers' evaluations of advertised goods [53, 54, 42, 52, 37, 55, 40]. In our problem, we follow the literature in information design, a.k.a., Bayesian persuasion [39] [also see the recent surveys by [30, 38, 10], where the seller can commit to an information policy that can strategically disclose product information to the buyer so that to influence buyer's belief about the product quality. Similar formulation for advertising has also appeared in [19, 32, 34, 14, 13, 28]. Our work differs from these works in several ways. First, the seller's offline problem in our setting is a joint pricing and advertising problem. Second, we focus on an online setting where the seller has no apriori knowledge of the demand function and has to use past buyers' purchase responses to adaptively learn optimal pricing and advertising strategy.
Our problem shares similarity to the problem on sale of information in economics and computer science literature [7, 9, 11, 26, 20, 48, 12, 59, 47, 25]. In particular, similar to the problem on sale of information, the seller, in our setting, also commits to design an information structure to reveal information about the realized state to the decision maker (i.e., buyer); and the buyer, in our setting,
then makes the payment based on the declared information structure, not for specific realizations of the seller's informative signals [12, 20, 48]. However, different from these works, the seller in our setting is selling a product with some inherent value and not just information. The valuation of the product can be shaped by providing information. This gives new interesting tradeoffs between information and revenue in our problem that are absent in the settings where only information is being sold. Moreover, in most literature of selling information, the buyers' type distribution is usually assumed to be known. We consider a more practical data-driven setting where the underlying buyers' type distribution (demand function) is apriori unknown to the seller and needs to be learnt from observations.
Facing unknown buyer's preference (i.e., buyer's private type), the seller's dynamic advertising problem also relates to the growing line of work in information design on relaxing one fundamental assumption in the canonical Bayesian persuasion model – the sender perfectly knows receiver's preference. The present paper joins the recent increased interests on using online learning approach to study the regret minimization when the sender repeatedly interacts with receivers [23, 24, 61, 33] without knowing receivers' preferences. Moreover, our work also conceptually relates to research on Bayesian exploration in multi-armed bandit [46, 51, 50, 36] which also studies an online setting where one player can utilize her information advantage to persuade another player to take the desired action. Our work departs from this line of work in terms of both the setting and the application domain. Particularly, the above works typically consider an online setting on how to learn optimal signaling scheme whereas in our setting the optimal policy is a joint pricing and signaling (advertising) scheme.
When there is no uncertainty in the product quality, the seller's problem in our setting reduces to a standard dynamic pricing and learning problem with unknown non-parametric demand function [43, 16, 41, 8]. However, given any non-trivial product quality space and prior distributions, in our problem, in addition to a price, the seller needs to choose a non-trivial advertising strategy in order to maximize revenue. This makes the seller's decision space high dimensional and (as we discuss in the next section) introduces significant complexities and difficulties so that the typical techniques (like uniform or adaptive discretization) used in pricing and continuous/combinatorial multi-armed bandit literature cannot be directly applied.
2 Algorithm Design
In this section, we present our main algorithm for the dynamic pricing and advertising problem. In subsection 2.1, we present an equivalent reformulation for tractable advertising strategies, then in subsection 2.2, we discuss many important challenges even after this simplification, and finally, in subsection 2.3, we present our algorithm.
2.1 An equivalent reformulation for tractable advertising strategies
Recall that in every round, the buyer  $t$  sees the offered price  $p_t$  and advertising strategy  $\phi_t$  that specifies the distributions over signals  $\phi_t(\cdot|\omega) \in \Delta^\Sigma$  that the seller will send for each possible value  $\omega$  of the realized product quality. After the product quality  $\omega_t$  is realized, the buyer  $t$  sees a signal  $\sigma_t \sim \phi_t(\cdot|\omega_t)$  from the seller's declared advertising strategy. The buyer uses this signal along with the prior  $\lambda$  to form a Bayesian posterior  $\mu_t(\cdot|\sigma_t) \in \Delta^\Omega$  on the product quality. The Bayesian rational buyer then takes the action  $a_t \in \{0,1\}$ , based on expected utility maximization. In particular, we have  $a_t = 1$  if  $\mathbb{E}_{\omega \sim \mu_t(\cdot|\sigma_t)}[v(\theta_t,\omega)] \geq p_t$  and  $a_t = 0$  otherwise.
From the decision formula above, it is clear that the choice of advertising strategy affects the buyer  $t$ 's decision only through the realized posterior  $\mu_t(\cdot|\sigma_t)$ . Consequently, the seller's choice of advertising strategy in time  $t$  can be reduced to selecting a distribution over posteriors  $\mu_t$ . Seller's choice can in fact be further simplified in the case where the buyer's valuation function is linear in the product quality  $\omega$  since in that case we have
$$
\mathbb {E} _ {\omega \sim \mu_ {t} (\cdot | \sigma_ {t})} [ v (\theta_ {t}, \omega) ] = v (\theta_ {t}, \mathbb {E} _ {\omega \sim \mu_ {t} (\cdot | \sigma_ {t})} [ \omega ]) = v (\theta_ {t}, q _ {t})
$$
where  $q_{t}$  is the realized posterior mean  $q_{t} \triangleq \mathbb{E}_{\omega \sim \mu_{t}(\cdot|\sigma_{t})}[\omega]$ . Here  $q_{t} \in [0,1]$  since  $\omega \in \Omega \subseteq [0,1]$ . Therefore the buyer purchases  $(a_{t} = 1)$  if and only if  $v(\theta_{t},q_{t}) \geq p_{t}$ . As a result, we can reduce the seller's advertising in round  $t$  to the choice of distribution (pdf)  $\rho_{t}(\cdot) \in \Delta^{[0,1]}$  over posterior means.