SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Mengzi Amy Guo
mengzi_guo@berkeley.edu
Donghao Ying
donghaoy@berkeley.edu
Zuo-Jun Max Shen
maxshen@berkeley.edu
This work is dedicated to the algorithm design in a competitive framework, with the primary goal of learning a stable equilibrium. We consider the dynamic price competition between two firms operating within an opaque marketplace, where each firm lacks information about its competitor. The demand follows the multinomial logit (MNL) choice model, which depends on the consumers' observed price and their reference price, and consecutive periods in the repeated games are connected by reference price updates. We use the notion of stationary Nash equilibrium (SNE), defined as the fixed point of the equilibrium pricing policy for the single-period game, to simultaneously capture the long-run market equilibrium and stability. We propose the online projected gradient ascent algorithm (OPGA), where the firms adjust prices using the first-order derivatives of their log-revenues that can be obtained from the market feedback mechanism. Despite the absence of typical properties required for the convergence of online games, such as strong monotonicity and variational stability, we demonstrate that under diminishing step-sizes, the price and reference price paths generated by OPGA converge to the unique SNE, thereby achieving the no-regret learning and a stable market. Moreover, with appropriate step-sizes, we prove that this convergence exhibits a rate of  $\mathcal{O}(1/t)$ .
The memory-based reference effect is a well-studied strategic consumer behavior in marketing and economics literature, which refers to the phenomenon that consumers shape their price expectations (known as reference prices) based on the past encounters and then use them to judge the current price (see [1] for a review). A substantial body of empirical research has shown that the current demand is significantly influenced by the historical prices through reference effects (see, e.g., [2, 3, 4, 5]). Driven by the ubiquitous evidence, many studies have investigated various pricing strategies in the presence of reference effects [6, 7, 8, 9]. However, the aforementioned works have all focused on the case with a monopolistic seller, leaving the understanding of how the reference effect functions in a competition relatively limited compared to its practical importance. This issue becomes even more pronounced with the surge of e-commerce, as the increased availability of information incentivizes consumers to make comparisons among different retailers.
Moreover, we notice that in competitive markets, the pricing problem with reference effects is further complicated by the lack of transparency, where firms are cautious about revealing confidential information to their rivals. Although the rise of digital markets greatly accelerates data transmission and promotes the information transparency, which is generally considered beneficial because of the improvement in marketplace efficiency [10], many firms remain hesitant to fully embrace such transparency for fear of losing their informational advantages. This concern is well-founded in the literature, for example the work [11] demonstrates that dealers with lower transparency levels generate higher profits than their more transparent counterparts, and [12] highlights that in reverse auctions, transparency typically enables buyers to drive the price down to the firm's marginal cost. Hence, as recommended by [13], companies should focus on the strategic and selective disclosure of information to enhance their competitive edge, rather than pursuing complete transparency.
Inspired by these real-world practices, in this article, we study the duopoly competition with reference effects in an opaque market, where each firm has access to its own information but does not possess any knowledge about its competitor, including their price, reference price, and demand. The consumer demand follows the multinomial logit (MNL) choice model, which naturally reflects the cross-product effects among substitutes. Furthermore, given the intertemporal characteristic of the memory-based reference effect, we consider the game in a dynamic framework, i.e., the firms engage in repeated competitions with consecutive periods linked by reference price updates. In this setting, it is natural to question whether the firms can achieve some notion of stable equilibrium by employing common online learning algorithms to sequentially set their prices. A vast majority of the literature on online games with incomplete information targets the problem of finding no-regret algorithms that can direct agents toward a Nash equilibrium, a stable state at which the agents have no incentive to revise their actions (see, e.g., [14, 15, 16]). Yet, the Nash equilibrium alone is inadequate to determine the stable state for our problem of interest, given the evolution of reference prices. For instance, even if an equilibrium price is reached in one period, the reference price update makes it highly probable that the firms will deviate from this equilibrium in subsequent periods. As a result, to jointly capture the market equilibrium and stability, we consider the concept of stationary Nash equilibrium (SNE), defined as the fixed point of the equilibrium pricing policy for the single-period game. Attaining this long-term market stability is especially appealing for firms in a competitive environment, as a stable market fosters favorable conditions for implementing efficient planning strategies and facilitating upstream operations in supply chain management [17]. In contrast, fluctuating demands necessitate more carefully crafted strategies for effective logistics management [18, 19, 20].
The concept of SNE has also been investigated in [21] and [22]. Specifically, [21] examines the long-run market behavior in a duopoly competition with linear demand and a common reference price for both products. However, compared to the MNL demand in our work, linear demand models generally fall short in addressing interdependence among multiple products [4, 23, 24]. On the other hand, [22] adopts the MNL demand with product-specific reference price formulation; yet, their analysis of long-term market dynamics relies on complete information and is inapplicable to an opaque market. In contrast, our work accommodates both the partial information setting and MNL choice model. We summarize our main contributions below:
1. Formulation. We introduce a duopoly competition framework with an opaque market setup that takes into account both cross-period and cross-product effects through consumer reference effects and the MNL choice model. We use the notion of stationary Nash equilibrium (SNE) to simultaneously depict the equilibrium price and market stability.
2. Algorithm and convergence. We propose a no-regret algorithm, namely the Online Projected Gradient Ascent (OPGA), where each firm adjusts its posted price using the first-order derivative of its log-revenue. When the firms execute OPGA with diminishing step-sizes, we show that their prices and reference prices converge to the unique SNE, leading to the long-run market equilibrium and stability. Furthermore, when the step-sizes decrease appropriately in the order of  $\Theta(1/t)$ , we demonstrate that the prices and reference prices converge at a rate of  $\mathcal{O}(1/t)$ .
3. Analysis. We propose a novel analysis for the convergence of OPGA by exploiting characteristic properties of the MNL demand model. General convergence results for online games typically require restrictive assumptions such as strong monotonicity [25, 14, 16] or variational stability [26, 15, 27, 28], as well as the convexity of the loss function [29, 30]. However, our problem lacks these favorable properties, rendering the existing techniques inapplicable. Additionally, compared to standard games where the underlying environment is static, the reference price evolution in this work further perplexes the convergence analysis.
4. Managerial insights. Our study illuminates a common issue in practice, where the firms are willing to cooperate but are reluctant to divulge their information to others. The OPGA algorithm can help address this issue by guiding the firms to achieve the SNE as if they had perfect information, while still preserving their privacy.
2 Related literature
Our work on the dynamic competition with reference effects in an opaque market is related to the several streams of literature.
Modeling of reference effect. The concept of reference effects can be traced back to the adaptation-level theory proposed by [31], which states that consumers evaluate prices against the level they have adapted to. Extensive research has been dedicated to the formulation of reference effects in the marketing literature, where two mainstream models emerge: memory-based reference price and stimulus-based reference price [4]. The memory-based reference model, also known as the internal reference price, leverages historical prices to form the benchmark (see, e.g., [3, 32, 5]). On the contrary, the stimulus-based reference model, or the external reference price, asserts that the price judgment is established at the moment of purchase utilizing current external information such as the prices of substitutable products, rather than drawing on past memories (see, e.g., [33, 2]). According to the comparative analysis by [4], among different reference models, the memory-based model that relies on a product's own historical prices offers the best fit and strongest predictive power in multi-product settings. Hence, our paper adopts this type of reference model, referred to as the brand-specific past prices formulation in [4].
Dynamic pricing in monopolist market with reference effect. The research on reference effects has recently garnered increasing attention in the field of operations research, particularly in monopolist pricing problems. As memory-based reference effect models give rise to the intertemporal nature, the price optimization problem is usually formulated as a dynamic program. Similar to our work, their objectives typically involve determining the long-run market stability under the optimal or heuristic pricing strategies. The classic studies by [7] and [6] show that in either discrete- or continuous-time framework, the optimal pricing policy converges and leads to market stabilization under both loss-neutral and loss-averse reference effects. More recent research on reference effects primarily concentrates on the piecewise linear demand in single-product contexts and delves into more comprehensive characterizations of myopic and optimal pricing policies (see, e.g., [34, 8]). Deviating from the linear demand, [9] and [24] employ the logit demand and analyze the long-term market behaviors under the optimal pricing policy, where the former emphasizes on consumer heterogeneity and the later innovates in the multi-product setting.
While the common assumption in the aforementioned studies is that the firm knows the demand function, another line of research tackles the problem under uncertain demand, where they couple monopolistic dynamic pricing with reference effects and online demand learning [35, 36]. Although these works include an online learning component, our paper distinguishes itself from them in two aspects. Firstly, the uncertainty that needs to be learned is situated in different areas. The works by [35] and [36] assume that the seller recognizes the structure of the demand function (i.e., linear demand) but requires to estimate the model's responsiveness parameters. By contrast, in our competitive framework, the firms are aware of their own demands but lack knowledge about their rivals that should be learned. Secondly, the objectives of [35] and [36] are to design algorithms to boost the total revenues from a monopolist perspective, whereas our algorithm aims to guide firms to reach Nash equilibrium and market stability concurrently.
Price competition with reference effects. Our work is pertinent to the studies on price competition with reference effects [37, 38, 21, 39, 22]. In particular, [38] is concerned with single-period price competitions under various reference price formulations. The paper [37] expands the scope to a dynamic competition with symmetric reference effects and linear demand, though their theoretical analysis on unique subgame perfect Nash equilibrium is confined to a two-period horizon. The more recent work [39] further extends the game to the multi-stage setting, where they obtain the Markov perfect equilibrium for consumers who are either loss-averse or loss-neutral. Nonetheless, unlike the discrete-time framework in [37] and this paper, [39] employs the continuous-time framework, which significantly differs from its discrete-time counterpart in terms of analytical techniques. The recent article [22] also studies the long-run market behavior and bears similarity to our work in model formulations. However, a crucial difference exists: [22] assumes a transparent market setting,
whereas we consider the more realistic scenario where the market is opaque and firms cannot access information of their competitors.
More closely related to our work, the work [21] also looks into the long-run market stability of the duopoly price competition in an opaque marketplace. However, there are two notable differences between our paper and theirs. The key distinction lies in the selection of demand function. We favor the logit demand over the linear demand used in [21], as the logit demand exhibits superior performance in the presence of reference effects [23, 9]. However, the logit model imposes challenges for convergence result since a crucial part of the analysis in [21] hinges on the demand linearity [21, Lemma 9.1], which is not satisfied by the logit demand. Second, [21] assumes a uniform reference price for both products, whereas we consider the product-specific reference price, which has the best empirical performance as illustrated in [4]. These adaptations, even though beneficial to the expressiveness and flexibility of the model, render the convergence analysis in [21] not generalizable to our setting.
General convergence results for online games. Our paper is closely related to the study of online games, where a typical research question is whether online learning algorithms can achieve the Nash equilibrium for multiple agents who aim to minimize their local loss functions. In this section, we review a few recent works in this field. For games with continuous actions, [14] and [40] show that the online mirror descent converges to the Nash equilibrium in strongly monotone games. The work [16] further relaxes the strong monotonicity assumption and examines the last-iterate convergence for games with unconstrained action sets that satisfy the so-called "cocoercive" condition. In addition, [26] and [15] establish the convergence of the dual averaging method under a more general condition called global variational stability, which encompasses the cocoercive condition as a subcase. More recently, [41, 27, 28] demonstrate that extra-gradient approaches, such as the optimistic gradient method, can achieve faster convergence to the Nash equilibrium in monotone and variationally stable games. We point out that in the cited literature above, either the assumption itself implies the convexity of the local loss function such as the strong monotonicity or the convergence to the Nash equilibrium additionally requires the loss function to be convex. By contrast, in our problem, the revenue function of each firm is not concave in its price and does not satisfy either of the properties listed above. Additionally, incorporating the reference effect further complicates the analysis, as the standard notion of Nash equilibrium is insufficient to characterize convergence due to the dynamic nature of reference price.
3.1 MNL demand model with reference effects
We study a duopoly price competition with reference price effects, where two firms each offer a substitutable product, labeled as  $H$  and  $L$ , respectively. Both firms set prices simultaneously in each period throughout an infinite-time horizon. To accommodate the interaction between the two products, we employ a multinomial logit (MNL) model, which inherently captures such cross-product effects. The consumers' utility at period  $t$ , which depends on the posted price  $p_i^t$  and reference price  $r_i^t$ , is defined as:
$$
U _ {i} \left(p _ {i} ^ {t}, r _ {i} ^ {t}\right) = u _ {i} \left(p _ {i} ^ {t}, r _ {i} ^ {t}\right) + \epsilon_ {i} ^ {t} = a _ {i} - b _ {i} \cdot p _ {i} ^ {t} + c _ {i} \cdot \left(r _ {i} ^ {t} - p _ {i} ^ {t}\right) + \epsilon_ {i} ^ {t}, \quad \forall i \in \{H, L \}, \tag {1}
$$
where  $u_{i}(p_{i}^{t},r_{i}^{t})$  is the deterministic component, and  $(a_{i},b_{i},c_{i})$  are the given parameters. In addition, the notation  $\epsilon_i^t$  denotes the random fluctuation following the i.i.d. standard Gumbel distribution. According to the random utility maximization theory [42], the demand/market share at period  $t$  with the posted price  $\mathbf{p}^t = (p_H^t,p_L^t)$  and reference price  $\mathbf{r}^t = (r_H^t,r_L^t)$  for product  $i\in \{H,L\}$  is given by
$$
d _ {i} \left(\mathbf {p} ^ {t}, \mathbf {r} ^ {t}\right) = d _ {i} \left(\left(p _ {i} ^ {t}, p _ {- i} ^ {t}\right), \left(r _ {i} ^ {t}, r _ {- i} ^ {t}\right)\right) = \frac {\exp \left(u _ {i} \left(p _ {i} ^ {t} , r _ {i} ^ {t}\right)\right)}{1 + \exp \left(u _ {i} \left(p _ {i} ^ {t} , r _ {i} ^ {t}\right)\right) + \exp \left(u _ {- i} \left(p _ {- i} ^ {t} , r _ {- i} ^ {t}\right)\right)}, \tag {2}
$$
where the subscript  $-i$  denotes the other product besides product  $i$ . Consequently, the expected revenue for each firm/product at period  $t$  can be expressed as
$$
\Pi_ {i} \left(\mathbf {p} ^ {t}, \mathbf {r} ^ {t}\right) = \Pi_ {i} \left(\left(p _ {i} ^ {t}, p _ {- i} ^ {t}\right), \left(r _ {i} ^ {t}, r _ {- i} ^ {t}\right)\right) = p _ {i} ^ {t} \cdot d _ {i} \left(\mathbf {p} ^ {t}, \mathbf {r} ^ {t}\right), \quad \forall i \in \{H, L \}. \tag {3}
$$
The interpretation of parameters  $(a_{i},b_{i},c_{i})$  in Eq. (1) is as follows. For product  $i\in \{H,L\}$ ,  $a_{i}$  refers to product's intrinsic value,  $b_{i}$  represents consumers' responsiveness to price, also known as price
sensitivity, and  $c_{i}$  corresponds to the reference price sensitivity. When the offered price exceeds the internal reference price  $(r_i^t < p_i^t)$ , consumers perceive it as a loss or surcharge, whereas the price below this reference price  $(r_i^t > p_i^t)$  is regarded as a gain or discount. These sensitivity parameters are assumed to be positive, i.e.,  $b_{i}, c_{i} > 0$  for  $i \in \{H, L\}$ , which aligns with consumer behaviors towards substitutable products and has been widely adopted in similar pricing problems (see, e.g., [34, 8, 21, 24, 22]). Precisely, this assumption guarantees that an increase in  $p_i^t$  would result in a lower consumer utility for product  $i$ , which in turn reduces its demand  $d_{i}(\mathbf{p}^{t}, \mathbf{r}^{t})$  and increases the demand of the competing product  $d_{-i}(\mathbf{p}^t, \mathbf{r}^t)$ . Conversely, a rise in the reference price  $r_i^t$  increases the utility for product  $i$ , consequently influencing the consumer demands in the opposite direction.
We stipulate the feasible range for price and reference price to be  $\mathcal{P} = [\underline{p},\overline{p} ]$  , where  $\underline{p},\overline{p} >0$  denote the price lower bound and upper bound, respectively. This boundedness of prices is in accordance with real-world price floors or price ceilings, whose validity is further reinforced by its frequent use in the literature on price optimization with reference effects (see, e.g., [34, 8, 21]).
We formulate the reference price using the brand-specific past prices (PASTBRSP) model proposed by [4], which posits that the reference price is product-specific and memory-based. This model is preferred over other reference price models evaluated in [4], as it exhibits superior performance in terms of fit and prediction. In particular, the reference price for product  $i$  is constructed by applying exponential smoothing to its own historical prices, where the memory parameter  $\alpha \in [0,1]$  governs the rate at which the reference price evolves. Starting with an initial reference price  $r_0$ , the reference price update for each product at period  $t$  can be described as
$$
r _ {i} ^ {t + 1} = \alpha \cdot r _ {i} ^ {t} + (1 - \alpha) \cdot p _ {i} ^ {t}, \quad \forall i \in \{H, L \}, \quad t \geq 0. \tag {4}
$$
The exponential smoothing technique is among the most prevalent and empirically substantiated reference price update mechanism in the existing literature (see, e.g., [7, 1, 6, 34, 8]). We remark that the theories established in this work are readily generalizable to the scenario of time-varying memory parameters  $\alpha$ , and we present the static setting only for the sake of brevity.
3.2 Opaque market setup
In this study, we consider the partial information setting where the firms possess no knowledge of their competitors, their own reference prices, as well as the reference price update scheme. Each firm  $i$  is only aware of its own sensitivity parameters  $b_{i}, c_{i}$ , and its previously posted price. Under this configuration, the firms cannot directly compute their demands from the expression in Eq. (2). However, it is legitimate to assume that each firm can access its own last-period demands through market feedback, i.e., determining the demand as the received revenue divided by the posted price. We point out that, in this non-transparent and non-cooperative market, the presence of feedback mechanisms is crucial for price optimization.
3.3 Market equilibrium and stability
The goal of our paper is to find a simple and intuitive pricing mechanism for firms so that the market equilibrium and stability can be achieved in the long-run while protecting firms' privacy. Before introducing the equilibrium notion considered in this work, we first define the equilibrium pricing policy denoted by  $\mathbf{p}^{\star}(\mathbf{r}) = (p_H^{\star}(\mathbf{r}), p_L^{\star}(\mathbf{r}))$ , which is a function that maps reference price to price and achieves the pure strategy Nash equilibrium in the single-period game. Mathematically, the equilibrium pricing policy satisfies that
$$
p _ {i} ^ {\star} (\mathbf {r}) = \underset {p _ {i} \in \mathcal {P}} {\arg \max } p _ {i} \cdot d _ {i} \left(\left(p _ {i}, p _ {- i} ^ {\star} (\mathbf {r})\right), \mathbf {r}\right), \quad \forall i \in \{H, L \}. \tag {5}
$$
Next, we formally introduce the concept of stationary Nash equilibrium, which is utilized to jointly characterize the market equilibrium and stability.
Definition 3.1 (Stationary Nash equilibrium) A point  $\mathbf{p}^{\star \star}$  is considered a stationary Nash equilibrium (SNE) if  $\mathbf{p}^{\star}(\mathbf{p}^{\star \star}) = \mathbf{p}^{\star \star}$ , i.e., the equilibrium price is equal to its reference price.
The notion of SNE has also been studied in [21, 22]. From Eq. (5) and Definition 3.1, we observe that an SNE possesses the following two properties:
- Equilibrium. The revenue function for each firm  $i \in \{H, L\}$  satisfies  $\Pi_i((p_i, p_{-i}^{\star \star}), \mathbf{p}^{\star \star}) \leq \Pi_i(\mathbf{p}^{\star \star}, \mathbf{p}^{\star \star})$  for all  $p_i \in \mathcal{P}$ , i.e., when the reference price and firm  $-i$ 's price are equal to the SNE price, the best-response price for firm  $i$  is the SNE price  $p_i^{\star \star}$ .
- Stability. If the price and the reference price attain the SNE at some period  $t$ , the reference price remains unchanged in the following period, i.e.,  $\mathbf{p}^t = \mathbf{r}^t = \mathbf{p}^{\star \star}$  implies that  $\mathbf{r}^{t + 1} = \mathbf{p}^{\star \star}$ .
As a result, when the market reaches the SNE, the firms have no incentive to deviate, and the market remains stable in subsequent competitions. The following proposition states the uniqueness of the SNE and characterizes the boundedness of the SNE.