SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proposition 3.1 There exists a unique stationary Nash equilibrium, denoted by  $\mathbf{p}^{\star \star} = (p_H^{\star \star}, p_L^{\star \star})$ . In addition, it holds that
$$
\frac {1}{b _ {i} + c _ {i}} <   p _ {i} ^ {\star \star} <   \frac {1}{b _ {i} + c _ {i}} + \frac {1}{b _ {i}} W \left(\frac {b _ {i}}{b _ {i} + c _ {i}} \exp \left(a _ {i} - \frac {b _ {i}}{b _ {i} + c _ {i}}\right)\right), \quad \forall i \in \{H, L \}, \tag {6}
$$
where  $W(\cdot)$  is the Lambert  $W$  function (see definition in Eq. (22)).
Without loss of generality, we assume that the feasible price range  $\mathcal{P}^2 = [\underline{p},\overline{p} ]^2$  is sufficiently large to contain the unique SNE, i.e.,  $\mathbf{p}^{\star \star}\in [\underline{p},\overline{p} ]^2$ . Proposition 3.1 provides a quantitative characterization for this assumption: it suffices to choose the price lower bound  $\underline{p}$  to be any real number between  $(0,\min_{i\in \{H,L\}}\{1 / (b_i + c_i)\})$ , and the price upper bound  $\overline{p}$  can be any value such that
$$
\bar {p} \geq \max  _ {i \in \{H, L \}} \left\{\frac {1}{b _ {i} + c _ {i}} + \frac {1}{b _ {i}} W \left(\frac {b _ {i}}{b _ {i} + c _ {i}} \exp \left(a _ {i} - \frac {b _ {i}}{b _ {i} + c _ {i}}\right)\right) \right\}. \tag {7}
$$
This assumption is mild as the bound in Eq. (7) is independent of both the price and reference price, and it does not grow exponentially fast with respect to any parameters. Hence, there is no need for  $\overline{p}$  to be excessively large. For example, when  $a_{H} = a_{L} = 10$  and  $b_{H} = b_{L} = c_{H} = c_{L} = 1$ , Eq. (7) becomes  $\overline{p} \geq 7.3785$ . We refer the reader to Appendix B for further discussions on the structure and computation of the equilibrium pricing policy and SNE, as well as the proof of Proposition 3.1.
4 No-regret learning: Online Projected Gradient Ascent
In this section, we examine the long-term dynamics of the price and reference price paths to determine if the market stabilizes over time. As studied in [22], under perfect information, the firms operating with full rationality will follow the equilibrium pricing policy in each period, while those functioning with bounded rationality will adhere to the best-response pricing policy throughout the planning horizon. In both situations, the market would stabilize in the long run. However, with partial information, the firms are incapable of computing either the equilibrium or the best-response policies due to the unavailability of reference prices and their competitor's price. Thus, one viable strategy to boost firms' revenues is to dynamically modify prices in response to market feedback.
In light of the success of gradient-based algorithms in the online learning literature (see, e.g., [43, 44, 45]), we propose the Online Projected Gradient Ascent (OPGA) method, as outlined in Algorithm 1. Specifically, in each period, both firms update their current prices using the first-order derivatives of their log-revenues with the same learning rate (see Eq. (9)). It is noteworthy that the difference between the derivatives of the log-revenue and the standard revenue is a scaling factor equal to the revenue itself, i.e.,
$$
\frac {\partial \log (\Pi_ {i} (\mathbf {p} , \mathbf {r}))}{\partial p _ {i}} = \frac {1}{\Pi_ {i} (\mathbf {p} , \mathbf {r})} \cdot \frac {\partial (\Pi_ {i} (\mathbf {p} , \mathbf {r}))}{\partial p _ {i}}, \quad \forall i \in \{H, L \}. \tag {8}
$$
Therefore, the price update in Algorithm 1 can be equivalently viewed as an adaptively regularized gradient ascent using the standard revenue function, where the regularizer at period  $t$  is  $\left(1 / \Pi_{H}(\mathbf{p}^{t},\mathbf{r}^{t}),1 / \Pi_{L}(\mathbf{p}^{t},\mathbf{r}^{t})\right)$ .
We highlight that by leveraging the structure of MNL model, each firm  $i$  can obtain the derivative of its log-revenue  $D_{i}^{t}$  in Eq. (9) through the market feedback mechanism, i.e., last-period demand. The firms do not need to know their own reference price when executing the OPGA algorithm, and the reference price update in Line 6 is automatically performed by the market. In fact, computing the
Algorithm 1 Online Projected Gradient Ascent (OPGA)
1: Input: Initial reference price  $\mathbf{r}^0 = (r_H^0, r_L^0)$ , initial price  $\mathbf{p}^0 = (p_H^0, p_L^0)$ , and step-sizes  $\{\eta^t\}_{t \geq 0}$ .
2: for  $t = 0,1,2,\ldots$  do
3: for  $i\in \{H,L\}$  do
4: Compute derivative  $D_{i}^{t}$  from price and demand of firm  $i$  at period  $t$ :
$$
D _ {i} ^ {t} \leftarrow \frac {\partial \log (\Pi_ {i} (\mathbf {p} ^ {t} , \mathbf {r} ^ {t}))}{\partial p _ {i}} = \frac {1}{p _ {i} ^ {t}} + (b _ {i} + c _ {i}) \cdot d _ {i} (\mathbf {p} ^ {t}, \mathbf {r} ^ {t}) - (b _ {i} + c _ {i}). \tag {9}
$$
5: Update posted price:  $p_i^{t+1} \gets \operatorname{Proj}_{\mathcal{P}}(p_i^t + \eta^t D_i^t)$ .
6: Reference price update:  $\mathbf{r}^{t + 1}\gets \alpha \mathbf{r}^t +(1 - \alpha)\mathbf{p}^t$
derivative  $D_{i}$  is identical to querying the first-order oracle, which is a common assumption in the optimization literature [46]. Further, this derivative can also be acquired through a minor perturbation of the posted price, even when the firms lack access to historical prices and market feedback, making the algorithm applicable in a variety of scenarios.
There are two potential ways to analyze Algorithm 1 using well-established theories. Below, we briefly introduce these methods and the underlying challenges, while referring readers to Appendix A for a more detailed discussion.
- First, by adding two virtual firms to represent the reference price, the game can be converted into a standard four-player online game, effectively eliminating the evolution of the underlying state (reference price). However, the challenge in analyzing this four-player game arises from the fact that, while real firms have the flexibility to dynamically adjust their step-sizes, the learning rate for virtual firms is fixed to the constant  $(1 - \alpha)$ , where  $\alpha$  is the memory parameter for reference price updates. This disparity hinders the direct application of the existing results from multi-agent online learning literature, as these results typically require the step-sizes of all agents either diminish at comparable rates or remain as a small enough constant [47, 48, 14, 15].
- The second approach involves translating the OPGA algorithm into a discrete nonlinear system by treating  $(\mathbf{p}^{t + 1},\mathbf{r}^{t + 1})$  as a vector-valued function of  $(\mathbf{p}^t,\mathbf{r}^t)$ , i.e.,  $(\mathbf{p}^{t + 1},\mathbf{r}^{t + 1}) = \mathbf{f}(\mathbf{p}^t,\mathbf{r}^t)$  for some function  $\mathbf{f}(\cdot)$ . In this context, analyzing the convergence of Algorithm 1 is equivalent to examining the stability of the fixed point of  $\mathbf{f}(\cdot)$ , which is related to the spectral radius of the Jacobian matrix  $\nabla \mathbf{f}(\mathbf{p}^{\star \star},\mathbf{p}^{\star \star})$  [49, 50]. However, the SNE lacks a closed-form expression, making it difficult to calculate the eigenvalues of  $\nabla \mathbf{f}(\mathbf{p}^{\star \star},\mathbf{p}^{\star \star})$ . In addition, the function  $\mathbf{f}(\cdot)$  is non-smooth due to the presence of the projection operator and can become non-stationary when the firms adopt time-varying step-sizes, such as diminishing step-sizes. Moreover, typical results in dynamical systems only guarantee local convergence [51], i.e., the asymptotic stability of the fixed point, whereas our goal is to establish the global convergence of both the price and reference price.
In the following section, we show that the OPGA algorithm with diminishing step-sizes converges to the unique SNE by exploiting characteristic properties of our model. This convergence result indicates that the OPGA algorithm provably achieves the no-regret learning, i.e., in the long run, the algorithm performs at least as well as the best fixed action in hindsight. However, it is essential to note that the reverse is not necessarily true: being no-regret does not guarantee the convergence at all, let alone the convergence to an equilibrium (see, e.g., [52, 53]). In fact, beyond the finite games, the agents may exhibit entirely unpredictable and chaotic behaviors under a no-regret policy [54].
5 Convergence results
In this section, we investigate the convergence properties of the OPGA algorithm. First, in Theorem 5.1, we establish the global convergence of the price path and reference price to the unique SNE under diminishing step-sizes. Subsequently, in Theorem 5.2, we further show that this convergence exhibits a rate of  $\mathcal{O}(1 / t)$ , provided that the step-sizes are selected appropriately. The primary proofs for this section can be found in Appendices C and D, while supporting lemmas are located in Appendix F.
Theorem 5.1 (Global convergence) Let the step-sizes  $\{\eta^t\}_{t\geq 0}$  be a non-increasing sequence such that  $\lim_{t\to \infty}\eta^t = 0$  and  $\sum_{t = 0}^{\infty}\eta^{t} = \infty$  hold. Then, the price paths and reference price paths generated by Algorithm 1 converge to the unique stationary Nash equilibrium.
Theorem 5.1 demonstrates the global convergence of the OPGA algorithm to the SNE, thus ensuring the market equilibrium and stability in the long-run. Compared to [22] that establishes the convergence to SNE under the perfect information setting, Theorem 5.1 ensures that such convergence can also be achieved in an opaque market where the firms are reluctant to share the information with their competitors. The only condition we need for the convergence is diminishing step-sizes such that  $\lim_{t\to \infty}\eta^t = 0$  and  $\sum_{t = 0}^{\infty}\eta^{t} = \infty$ . This assumption is widely adopted in the study of online games (see, e.g., [15, 14, 40]). Since the firms will likely become more familiar with their competitors through repeated competitions, it is reasonable for the firms to gradually become more conservative in adjusting their prices and decrease their learning rates.
As discussed in Sections 2 and 4, the existing methods in the online game and dynamical system literatures are not applicable to our problem due to the absence of standard structural properties and the presence of the underlying dynamic state, i.e., reference price. Consequently, we develop a novel analysis to prove Theorem 5.1, leveraging the characteristic properties of the MNL demand model. We provide a proof sketch below, with the complete proof deferred to Appendix C. Our proof consists of two primary parts:
- In Part 1 (Appendix C.1), we show that the price path  $\{\mathbf{p}^t\}_{t\geq 0}$  would enter the neighborhood  $N_{\epsilon}^{1} := \{\mathbf{p} \in \mathcal{P}^{2} \mid \varepsilon(\mathbf{p}) < \epsilon\}$  infinitely many times for any  $\epsilon > 0$ , where  $\varepsilon(\cdot)$  is a weighted  $\ell_1$  distance function defined as  $\varepsilon(\mathbf{p}) := |p_H^{\star\star} - p_H| / (b_H + c_H) + |p_L^{\star\star} - p_L| / (b_L + c_L)$ . To prove Part 1, we divide  $\mathcal{P}^2$  into four quadrants with  $\mathbf{p}^{\star\star}$  as the origin. Then, employing a contradiction-based argument, we suppose that the price path only visits  $N_{\epsilon}^{1}$  finitely many times. Yet, we show that this forces the price path to oscillate between adjacent quadrants and ultimately converge to the SNE, which violates the initial assumption.
- In Part 2 (Appendix C.2), we show that when the price path  $\{\mathbf{p}^t\}_{t\geq 0}$  enters the  $\ell_2$ -neighborhood  $N_{\epsilon}^{2} \coloneqq \{\mathbf{p} \in \mathcal{P}^{2} \mid \| \mathbf{p} - \mathbf{p}^{\star \star}\|_{2} < \epsilon\}$  for some sufficiently small  $\epsilon > 0$  and with small enough step-sizes, the price path will remain in  $N_{\epsilon}^{2}$  in subsequent periods. The proof of Part 2 relies on a local property of the MNL demand model around the SNE, with which we demonstrate that the price update provides adequate ascent to ensure the price path stays within the target neighborhood.
Owing to the equivalence between different distance functions in Euclidean spaces [55], these two parts jointly imply the convergence of the price path to the SNE. Since the reference price is equal to an exponential smoothing of the historical prices, the convergence of the reference price path follows that of the price path.
It is worth noting that [21, Theorem 5.1] also employs a two-part proof and shows the asymptotic convergence of online mirror descent [56, 57] to the SNE in the linear demand setting. However, their analysis heavily depends on the linear structure of the demand, which ensures that a property similar to the variational stability is globally satisfied [21, Lemma 9.1]. In contrast, such properties no longer hold for the MNL demand model considered in this paper. Therefore, we come up with two distinct distance functions for the two parts of the proof, respectively. Moreover, the proof by [21] relies on an assumption concerning the relationship between responsiveness parameters, whereas our convergence analysis only requires the minimum assumption that the responsiveness parameters are positive (otherwise, the model becomes counter-intuitive for substitutable products).
Theorem 5.2 (Convergence rate) When both firms adopt Algorithm 1, there exists a sequence of step-sizes  $\{\eta^t\}_{t\geq 0}$  with  $\eta^t = \Theta (1 / t)$  and constants  $d_{p},d_{r} > 0$  such that
$$
\left\| \mathbf {p} ^ {\star \star} - \mathbf {p} ^ {t} \right\| _ {2} ^ {2} \leq \frac {d _ {p}}{t}, \quad \left\| \mathbf {p} ^ {\star \star} - \mathbf {r} ^ {t} \right\| _ {2} ^ {2} \leq \frac {d _ {r}}{t}, \quad \forall t \geq 1. \tag {10}
$$
Theorem 5.2 improves the result of Theorem 5.1 by demonstrating the non-asymptotic convergence rate for the price and reference price. Although non-concavity usually anticipates slower convergence, our rate of  $\mathcal{O}(1 / t)$  matches [21, Theorem 5.2] that assumes linear demand and exhibits concavity.
The proof of Theorem 5.2 is primarily based on the local convergence rate when the price vector remains in a specific neighborhood  $N_{\epsilon_0}^2$  of the SNE after some period  $T_{\epsilon_0}$ . As the choice  $\eta^t = \Theta(1/t)$
ensures that  $\lim_{t\to \infty}\eta^t = 0$  and  $\sum_{t = 0}^{\infty}\eta^{t} = \infty$ , the existence of such a period  $T_{\epsilon_0}$  is guaranteed by Theorem 5.1. Utilizing an inductive argument, we first show that the difference between price and reference price decreases at a faster rate of  $\mathcal{O}(1 / t^2)$ , i.e.,  $\left\| \mathbf{r}^t -\mathbf{p}^t\right\| _2^2 = \mathcal{O}\left(1 / t^2\right),\forall t\geq 1$  (see Eq. (67)). Then, by exploiting a local property around the SNE (Lemma F.3), we use another induction to establish the convergence rate of the price path after it enters the neighborhood  $N_{\epsilon_0}^2$ . In the meanwhile, the convergence rate of the reference price path can be determined through a triangular inequality:  $\left\| \mathbf{p}^{\star \star} - \mathbf{r}^t\right\| _2^2 = \left\| \mathbf{p}^{\star \star} - \mathbf{p}^t +\mathbf{p}^t -\mathbf{r}^t\right\| _2^2\leq 2\left\| \mathbf{p}^{\star \star} - \mathbf{p}^t\right\| _2^2 +2\left\| \mathbf{p}^t -\mathbf{r}^t\right\| _2^2$ . Finally, due to the boundedness of the feasible price range  $\mathcal{P}^2$ , we can obtain the global convergence rate for all  $t\geq 1$  by choosing sufficiently large constants  $d_p$  and  $d_r$ .
Remark 5.3 (Constant step-sizes) The convergence analysis presented in this work can be readily extended to accommodate constant step-sizes. Particularly, our theoretical framework supports the selection of  $\eta^t \equiv \mathcal{O}(\epsilon_0)$ , where  $\epsilon_0$  refers to the size of the neighborhood utilized in Theorem 5.2. When  $\eta^t \equiv \mathcal{O}(\epsilon_0)$ , an approach akin to Part 1 can be employed to demonstrate that the price path enters the neighborhood  $N_{\epsilon}^{1}$  infinitely many times for any  $\epsilon \geq \mathcal{O}(\epsilon_0)$ . Subsequently, by exploiting the local property in a manner similar to Part 2, we can establish that once the price path enters the neighborhood  $N_{\epsilon}^{2}$  for some  $\epsilon = \mathcal{O}(\epsilon_0)$ , it will remain within that neighborhood.
(a) Convergence with  $\eta^t = 3 / t$
(b) Cyclic pattern with  $\eta^t = 1$
(c) OPGA vs. equilibrium policy.
Figure 1: Price and reference price paths for Examples 1, 2, and 3, where the parameters are  $(a_H, b_H, c_H) = (8.70, 2.00, 0.82)$ ,  $(a_L, b_L, c_L) = (4.30, 1.20, 0.32)$ ,  $(r_H^0, r_L^0) = (0.10, 2.95)$ ,  $(p_H^0, p_L^0) = (4.85, 4.86)$ , and  $\alpha = 0.90$ .
We perform two numerical experiments, differentiated solely by their sequences of step-sizes, to highlight the significance of step-sizes in achieving the convergence. In particular, Example 1 (see Figure 1a) corroborates Theorem 5.1 by demonstrating that the price and reference price trajectories converge to the unique SNE when we choose diminishing step-sizes that fulfill the criteria specified in Theorem 5.1. By comparison, the over-large constant step-sizes employed in Example 2 (see Figure 1b) fails to ensure convergence, leading to cyclic patterns in the long run. Moreover, in Example 3 (see Figure 1c), we compare the OPGA algorithm and the repeated application of the equilibrium pricing policy in Eq. (5) by plotting the reference price trajectories produced by both approaches. Figure 1c conveys that the two algorithms reach the SNE at a comparable rate, which indicates that the OPGA algorithm allows firms to attain the market equilibrium and stability as though they operate under perfect information, while preventing excessive information disclosure.
6 Extensions
In previous sections, we assume that each firm  $i$  possesses accurate information regarding its sensitivity parameters  $(b_{i},c_{i})$  as well as its realized market share  $d_i^t$ . This is equivalent to having access to an exact first-order oracle. However, a more practical scenario to consider is one where the firm can only obtain a rough approximation for its market share and needs to estimate the sensitivities from historical data. This would bring extra noise to the computation of the first-order derivative  $D_{i}^{t}$  in Eq. (9). In this section, we first elaborate on the feasibility of estimating the market share and sensitivities from realistic data. Then, we discuss the impact of an inexact first-order oracle on the convergence.
We consider the approximation of market share and the calibration of sensitivities under both uncensored and censored data cases. With uncensored data, both purchase and no-purchase data are
available, a scenario analogous to firms selling substitute goods on third-party online retail platforms like Amazon. The platform can track the non-purchase statistics by monitoring consumers who visited its website but did not purchase any product. Consequently, this facilitates a direct estimation of the total market size and, when complemented with sales quantities, allows for a precise computation of the firm's market share. Additionally, sensitivity parameters can be efficiently calibrated through the classical Maximum Likelihood Estimation (MLE) method. Due to the concavity of the log-likelihood function for the MNL model with respect to its parameters, many numerical algorithms (e.g., Newton-Raphson, BHHH-2 [58], and the steepest ascent) are guaranteed to improve at each iteration and achieve fast convergence [59, 60].
In many cases, the traditional transaction data only captures the realized demand, with non-purchases typically unrecorded. This scenario is known as the censored data case. In this case, the sensitives and the total market size of the MNL model can be estimated via the generalized expectation-maximization (GEM) gradient method proposed by [59], which is an iterative algorithm that is inspired by the expectation-maximization (EM) approach. The capability to approximate the sensitivity parameters, the market share, and thereby the gradient  $D_{i}^{t}$  enhances the credibility and practicability of the OPGA algorithm, suggesting its great potential for broader applications in retailing.
In the presence of approximation errors, firms cannot precisely compute the derivative  $D_{i}^{t}$ . Hence, the convergence results in Section 5 are not directly applicable. Indeed, if the errors are disruptive enough, one can anticipate Algorithm 1 to possibly not show any convergent behavior. However, if the errors are uniformly bounded by some small number  $\delta$ , the following theorem demonstrates that both the price and reference price paths converge to a  $\mathcal{O}(\delta)$ -neighborhood of the unique SNE.
Theorem 6.1 (Inexact first-order oracle) Suppose both firms only have access to an inexact first-order oracle such that  $|D_i^t - \partial \log(\Pi_i(\mathbf{p}^t, \mathbf{r}^t)) / \partial p_i| \leq \delta$ ,  $\forall i \in \{H, L\}$  and  $\forall t \geq 0$ . Let the step-sizes  $\{\eta^t\}_{t \geq 0}$  be a non-increasing sequence such that  $\lim_{t \to \infty} \eta^t = 0$  and  $\sum_{t=0}^{\infty} \eta^t = \infty$  hold. Then, the price paths reference price paths generated by Algorithm 1 converge to a neighborhood with radius  $\mathcal{O}(\delta)$  of the unique SNE.
We remark that the inexact first-order oracle studied in our setting is different from the stochastic gradient, which generally assumes a zero-mean noise with finite variance. In that case, it is possible to derive the convergence to a limiting point in expectation or with high probability. In contrast, the noise in  $D_{i}^{t}$  is a kind of approximation error without any distributional properties. Therefore, with the step-sizes defined in Theorem 6.1, we expect the price and reference price paths to converge to the neighborhood of the SNE, but continue to fluctuate around the neighborhood without admitting a limiting point.
The proof of Theorem 6.1 is based upon Theorem 5.1. Specifically, we demonstrate that the inexact gradient still guides the price path toward the SNE if the magnitude of the true gradient dominates the noise. Conversely, if noise levels are comparable with the true gradient, we show that the price path is already close to the SNE. The formal proof of Theorem 6.1 can be found in Appendix E
This article considers the price competition with reference effects in an opaque market, where we formulate the problem as an online game with an underlying dynamic state, known as reference price. We accomplish the goal of simultaneously capturing the market equilibrium and stability via proposing the no-regret algorithm named OPGA and providing the theoretical guarantee for its global convergence to the SNE. In addition, under appropriate step-sizes, we show that the OPGA algorithm has the convergence rate of  $\mathcal{O}(1 / t)$ .
In this paper, we focus on the symmetric reference effects, where consumers exhibit equal sensitivity to the gains and losses. As empirical studies suggest that consumers may display asymmetric reactions [61, 32, 62], one possible extension involves accounting for asymmetric reference effects displayed by loss-averse and gain-seeking consumers. Additionally, while our study considers the duopoly competition between two firms, it would be valuable to explore the more general game involving  $n$  players. Lastly, our research is based on a deterministic setting and pure strategy Nash equilibrium. Another intriguing direction for future work is to investigate mixed-strategy learning [63] in dynamic competition problems.
We extend our gratitude to the anonymous reviewers for their insightful comments. This research was particularly supported by NSF China Grant 71991462.