Title: No-Regret Learning in Dynamic Competition with Reference Effects Under Logit Demand
Venue: NeurIPS 2023
Accepted: yes
Keywords: no-regret learning

price competition

reference effect

last-iterate convergence
Abstract: This work is dedicated to the algorithm design in a competitive framework, with the primary goal of learning a stable equilibrium. We consider the dynamic price competition between two firms operating within an opaque marketplace, where each firm lacks information about its competitor. The demand follows the multinomial logit (MNL) choice model, which depends on the consumers' observed price and their reference price, and consecutive periods in the repeated games are connected by reference price updates. We use the notion of stationary Nash equilibrium (SNE), defined as the fixed point of the equilibrium pricing policy for the single-period game, to simultaneously capture the long-run market equilibrium and stability. We propose the online projected gradient ascent algorithm (OPGA), where the firms adjust prices using the first-order derivatives of their log-revenues that can be obtained from the market feedback mechanism. Despite the absence of typical properties required for the convergence of online games, such as strong monotonicity and variational stability, we demonstrate that under diminishing step-sizes, the price and reference price paths generated by OPGA converge to the unique SNE, thereby achieving the no-regret learning and a stable market. Moreover, with appropriate step-sizes, we prove that this convergence exhibits a rate of $\mathcal{O}(1/t)$.

No-Regret Learning in Dynamic Competition with Reference Effects Under Logit Demand

Abstract

This work is dedicated to the algorithm design in a competitive framework, with the primary goal of learning a stable equilibrium. We consider the dynamic price competition between two firms operating within an opaque marketplace, where each firm lacks information about its competitor. The demand follows the multinomial logit (MNL) choice model, which depends on the consumers' observed price and their reference price, and consecutive periods in the repeated games are connected by reference price updates. We use the notion of stationary Nash equilibrium (SNE), defined as the fixed point of the equilibrium pricing policy for the single-period game, to simultaneously capture the long-run market equilibrium and stability. We propose the online projected gradient ascent algorithm (OPGA), where the firms adjust prices using the first-order derivatives of their log-revenues that can be obtained from the market feedback mechanism. Despite the absence of typical properties required for the convergence of online games, such as strong monotonicity and variational stability, we demonstrate that under diminishing step-sizes, the price and reference price paths generated by OPGA converge to the unique SNE, thereby achieving the no-regret learning and a stable market. Moreover, with appropriate step-sizes, we prove that this convergence exhibits a rate of O (1 /t ) .

1 Introduction

The memory-based reference effect is a well-studied strategic consumer behavior in marketing and economics literature, which refers to the phenomenon that consumers shape their price expectations (known as reference prices) based on the past encounters and then use them to judge the current price (see [1] for a review). A substantial body of empirical research has shown that the current demand is significantly influenced by the historical prices through reference effects (see, e.g., [2, 3, 4, 5]). Driven by the ubiquitous evidence, many studies have investigated various pricing strategies in the presence of reference effects [6, 7, 8, 9]. However, the aforementioned works have all focused on the case with a monopolistic seller, leaving the understanding of how the reference effect functions in a competition relatively limited compared to its practical importance. This issue becomes even more pronounced with the surge of e-commerce, as the increased availability of information incentivizes consumers to make comparisons among different retailers.

Moreover, we notice that in competitive markets, the pricing problem with reference effects is further complicated by the lack of transparency, where firms are cautious about revealing confidential information to their rivals. Although the rise of digital markets greatly accelerates data transmission and promotes the information transparency, which is generally considered beneficial because of the improvement in marketplace efficiency [10], many firms remain hesitant to fully embrace such transparency for fear of losing their informational advantages. This concern is well-founded in the literature, for example the work [11] demonstrates that dealers with lower transparency levels generate higher profits than their more transparent counterparts, and [12] highlights that in reverse auctions, transparency typically enables buyers to drive the price down to the firm's marginal cost. Hence, as recommended by [13], companies should focus on the strategic and selective disclosure of information to enhance their competitive edge, rather than pursuing complete transparency.

Inspired by these real-world practices, in this article, we study the duopoly competition with reference effects in an opaque market , where each firm has access to its own information but does not possess any knowledge about its competitor, including their price, reference price, and demand. The consumer demand follows the multinomial logit (MNL) choice model, which naturally reflects the cross-product effects among substitutes. Furthermore, given the intertemporal characteristic of the memory-based reference effect, we consider the game in a dynamic framework, i.e., the firms engage in repeated competitions with consecutive periods linked by reference price updates. In this setting, it is natural to question whether the firms can achieve some notion of stable equilibrium by employing common online learning algorithms to sequentially set their prices. A vast majority of the literature on online games with incomplete information targets the problem of finding no-regret algorithms that can direct agents toward a Nash equilibrium, a stable state at which the agents have no incentive to revise their actions (see, e.g., [14, 15, 16]). Yet, the Nash equilibrium alone is inadequate to determine the stable state for our problem of interest, given the evolution of reference prices. For instance, even if an equilibrium price is reached in one period, the reference price update makes it highly probable that the firms will deviate from this equilibrium in subsequent periods. As a result, to jointly capture the market equilibrium and stability, we consider the concept of stationary Nash equilibrium (SNE), defined as the fixed point of the equilibrium pricing policy for the single-period game. Attaining this long-term market stability is especially appealing for firms in a competitive environment, as a stable market fosters favorable conditions for implementing efficient planning strategies and facilitating upstream operations in supply chain management [17]. In contrast, fluctuating demands necessitate more carefully crafted strategies for effective logistics management [18, 19, 20].

The concept of SNE has also been investigated in [21] and [22]. Specifically, [21] examines the long-run market behavior in a duopoly competition with linear demand and a common reference price for both products. However, compared to the MNL demand in our work, linear demand models generally fall short in addressing interdependence among multiple products [4, 23, 24]. On the other hand, [22] adopts the MNL demand with product-specific reference price formulation; yet, their analysis of long-term market dynamics relies on complete information and is inapplicable to an opaque market. In contrast, our work accommodates both the partial information setting and MNL choice model. We summarize our main contributions below:

Formulation. We introduce a duopoly competition framework with an opaque market setup that takes into account both cross-period and cross-product effects through consumer reference effects and the MNL choice model. We use the notion of stationary Nash equilibrium (SNE) to simultaneously depict the equilibrium price and market stability.

Algorithm and convergence. We propose a no-regret algorithm, namely the Online Projected Gradient Ascent (OPGA), where each firm adjusts its posted price using the first-order derivative of its log-revenue. When the firms execute OPGA with diminishing step-sizes, we show that their prices and reference prices converge to the unique SNE, leading to the long-run market equilibrium and stability. Furthermore, when the step-sizes decrease appropriately in the order of Θ(1 /t ) , we demonstrate that the prices and reference prices converge at a rate of O (1 /t ) .

Analysis. We propose a novel analysis for the convergence of OPGA by exploiting characteristic properties of the MNL demand model. General convergence results for online games typically require restrictive assumptions such as strong monotonicity [25, 14, 16] or variational stability [26, 15, 27, 28], as well as the convexity of the loss function [29, 30]. However, our problem lacks these favorable properties, rendering the existing techniques inapplicable. Additionally, compared to standard games where the underlying environment is static, the reference price evolution in this work further perplexes the convergence analysis.

Managerial insights. Our study illuminates a common issue in practice, where the firms are willing to cooperate but are reluctant to divulge their information to others. The OPGA algorithm can help address this issue by guiding the firms to achieve the SNE as if they had perfect information, while still preserving their privacy.

2 Related literature

Our work on the dynamic competition with reference effects in an opaque market is related to the several streams of literature.

Modeling of reference effect. The concept of reference effects can be traced back to the adaptationlevel theory proposed by [31], which states that consumers evaluate prices against the level they have adapted to. Extensive research has been dedicated to the formulation of reference effects in the marketing literature, where two mainstream models emerge: memory-based reference price and stimulus-based reference price [4]. The memory-based reference model, also known as the internal reference price, leverages historical prices to form the benchmark (see, e.g., [3, 32, 5]). On the contrary, the stimulus-based reference model, or the external reference price, asserts that the price judgment is established at the moment of purchase utilizing current external information such as the prices of substitutable products, rather than drawing on past memories (see, e.g., [33, 2]). According to the comparative analysis by [4], among different reference models, the memory-based model that relies on a product's own historical prices offers the best fit and strongest predictive power in multi-product settings. Hence, our paper adopts this type of reference model, referred to as the brand-specific past prices formulation in [4].

Dynamic pricing in monopolist market with reference effect. The research on reference effects has recently garnered increasing attention in the field of operations research, particularly in monopolist pricing problems. As memory-based reference effect models give rise to the intertemporal nature, the price optimization problem is usually formulated as a dynamic program. Similar to our work, their objectives typically involve determining the long-run market stability under the optimal or heuristic pricing strategies. The classic studies by [7] and [6] show that in either discrete- or continuous-time framework, the optimal pricing policy converges and leads to market stabilization under both loss-neutral and loss-averse reference effects. More recent research on reference effects primarily concentrates on the piecewise linear demand in single-product contexts and delves into more comprehensive characterizations of myopic and optimal pricing policies (see, e.g., [34, 8]). Deviating from the linear demand, [9] and [24] employ the logit demand and analyze the long-term market behaviors under the optimal pricing policy, where the former emphasizes on consumer heterogeneity and the later innovates in the multi-product setting.

While the common assumption in the aforementioned studies is that the firm knows the demand function, another line of research tackles the problem under uncertain demand, where they couple monopolistic dynamic pricing with reference effects and online demand learning [35, 36]. Although these works include an online learning component, our paper distinguishes itself from them in two aspects. Firstly, the uncertainty that needs to be learned is situated in different areas. The works by [35] and [36] assume that the seller recognizes the structure of the demand function (i.e., linear demand) but requires to estimate the model's responsiveness parameters. By contrast, in our competitive framework, the firms are aware of their own demands but lack knowledge about their rivals that should be learned. Secondly, the objectives of [35] and [36] are to design algorithms to boost the total revenues from a monopolist perspective, whereas our algorithm aims to guide firms to reach Nash equilibrium and market stability concurrently.

Price competition with reference effects. Our work is pertinent to the studies on price competition with reference effects [37, 38, 21, 39, 22]. In particular, [38] is concerned with single-period price competitions under various reference price formulations. The paper [37] expands the scope to a dynamic competition with symmetric reference effects and linear demand, though their theoretical analysis on unique subgame perfect Nash equilibrium is confined to a two-period horizon. The more recent work [39] further extends the game to the multi-stage setting, where they obtain the Markov perfect equilibrium for consumers who are either loss-averse or loss-neutral. Nonetheless, unlike the discrete-time framework in [37] and this paper, [39] employs the continuous-time framework, which significantly differs from its discrete-time counterpart in terms of analytical techniques. The recent article [22] also studies the long-run market behavior and bears similarity to our work in model formulations. However, a crucial difference exists: [22] assumes a transparent market setting,

whereas we consider the more realistic scenario where the market is opaque and firms cannot access information of their competitors.

More closely related to our work, the work [21] also looks into the long-run market stability of the duopoly price competition in an opaque marketplace. However, there are two notable differences between our paper and theirs. The key distinction lies in the selection of demand function. We favor the logit demand over the linear demand used in [21], as the logit demand exhibits superior performance in the presence of reference effects [23, 9]. However, the logit model imposes challenges for convergence result since a crucial part of the analysis in [21] hinges on the demand linearity [21, Lemma 9.1], which is not satisfied by the logit demand. Second, [21] assumes a uniform reference price for both products, whereas we consider the product-specific reference price, which has the best empirical performance as illustrated in [4]. These adaptations, even though beneficial to the expressiveness and flexibility of the model, render the convergence analysis in [21] not generalizable to our setting.

General convergence results for online games. Our paper is closely related to the study of online games, where a typical research question is whether online learning algorithms can achieve the Nash equilibrium for multiple agents who aim to minimize their local loss functions. In this section, we review a few recent works in this field. For games with continuous actions, [14] and [40] show that the online mirror descent converges to the Nash equilibrium in strongly monotone games. The work [16] further relaxes the strong monotonicity assumption and examines the last-iterate convergence for games with unconstrained action sets that satisfy the so-called 'cocoercive' condition. In addition, [26] and [15] establish the convergence of the dual averaging method under a more general condition called global variational stability, which encompasses the cocoercive condition as a subcase. More recently, [41, 27, 28] demonstrate that extra-gradient approaches, such as the optimistic gradient method, can achieve faster convergence to the Nash equilibrium in monotone and variationally stable games. We point out that in the cited literature above, either the assumption itself implies the convexity of the local loss function such as the strong monotonicity or the convergence to the Nash equilibrium additionally requires the loss function to be convex. By contrast, in our problem, the revenue function of each firm is not concave in its price and does not satisfy either of the properties listed above. Additionally, incorporating the reference effect further complicates the analysis, as the standard notion of Nash equilibrium is insufficient to characterize convergence due to the dynamic nature of reference price.

3 Problem formulation

3.1 MNLdemand model with reference effects

We study a duopoly price competition with reference price effects, where two firms each offer a substitutable product, labeled as H and L , respectively. Both firms set prices simultaneously in each period throughout an infinite-time horizon. To accommodate the interaction between the two products, we employ a multinomial logit (MNL) model, which inherently captures such cross-product effects. The consumers' utility at period t , which depends on the posted price p t i and reference price r t i , is defined as:

where u i ( p t i , r t i ) is the deterministic component, and ( a i , b i , c i ) are the given parameters. In addition, the notation ϵ t i denotes the random fluctuation following the i.i.d. standard Gumbel distribution. According to the random utility maximization theory [42], the demand/market share at period t with the posted price p t = ( p t H , p t L ) and reference price r t = ( r t H , r t L ) for product i ∈ { H,L } is given by

where the subscript -i denotes the other product besides product i . Consequently, the expected revenue for each firm/product at period t can be expressed as

The interpretation of parameters ( a i , b i , c i ) in Eq. (1) is as follows. For product i ∈ { H,L } , a i refers to product's intrinsic value, b i represents consumers' responsiveness to price, also known as price

sensitivity, and c i corresponds to the reference price sensitivity. When the offered price exceeds the internal reference price ( r t i < p t i ) , consumers perceive it as a loss or surcharge, whereas the price below this reference price ( r t i > p t i ) is regarded as a gain or discount. These sensitivity parameters are assumed to be positive, i.e., b i , c i > 0 for i ∈ { H,L } , which aligns with consumer behaviors towards substitutable products and has been widely adopted in similar pricing problems (see, e.g., [34, 8, 21, 24, 22]). Precisely, this assumption guarantees that an increase in p t i would result in a lower consumer utility for product i , which in turn reduces its demand d i ( p t , r t ) and increases the demand of the competing product d -i ( p t , r t ) . Conversely, a rise in the reference price r t i increases the utility for product i , consequently influencing the consumer demands in the opposite direction.

We stipulate the feasible range for price and reference price to be P = [ p, p ] , where p, p > 0 denote the price lower bound and upper bound, respectively. This boundedness of prices is in accordance with real-world price floors or price ceilings, whose validity is further reinforced by its frequent use in the literature on price optimization with reference effects (see, e.g., [34, 8, 21]).

We formulate the reference price using the brand-specific past prices ( PASTBRSP ) model proposed by [4], which posits that the reference price is product-specific and memory-based. This model is preferred over other reference price models evaluated in [4], as it exhibits superior performance in terms of fit and prediction. In particular, the reference price for product i is constructed by applying exponential smoothing to its own historical prices, where the memory parameter α ∈ [0 , 1] governs the rate at which the reference price evolves. Starting with an initial reference price r 0 , the reference price update for each product at period t can be described as

The exponential smoothing technique is among the most prevalent and empirically substantiated reference price update mechanism in the existing literature (see, e.g., [7, 1, 6, 34, 8]). We remark that the theories established in this work are readily generalizable to the scenario of time-varying memory parameters α , and we present the static setting only for the sake of brevity.

3.2 Opaque market setup

In this study, we consider the partial information setting where the firms possess no knowledge of their competitors, their own reference prices, as well as the reference price update scheme. Each firm i is only aware of its own sensitivity parameters b i , c i , and its previously posted price. Under this configuration, the firms cannot directly compute their demands from the expression in Eq. (2). However, it is legitimate to assume that each firm can access its own last-period demands through market feedback, i.e., determining the demand as the received revenue divided by the posted price. We point out that, in this non-transparent and non-cooperative market, the presence of feedback mechanisms is crucial for price optimization.

3.3 Market equilibrium and stability

The goal of our paper is to find a simple and intuitive pricing mechanism for firms so that the market equilibrium and stability can be achieved in the long-run while protecting firms' privacy. Before introducing the equilibrium notion considered in this work, we first define the equilibrium pricing policy denoted by p ⋆ ( r ) = ( p ⋆ H ( r ) , p ⋆ L ( r ) ) , which is a function that maps reference price to price and achieves the pure strategy Nash equilibrium in the single-period game. Mathematically, the equilibrium pricing policy satisfies that

Next, we formally introduce the concept of stationary Nash equilibrium, which is utilized to jointly characterize the market equilibrium and stability.

Definition 3.1 (Stationary Nash equilibrium) A point p ⋆⋆ is considered a stationary Nash equilibrium (SNE) if p ⋆ ( p ⋆⋆ ) = p ⋆⋆ , i.e., the equilibrium price is equal to its reference price.

The notion of SNE has also been studied in [21, 22]. From Eq. (5) and Definition 3.1, we observe that an SNE possesses the following two properties:

Equilibrium. The revenue function for each firm i ∈ { H,L } satisfies Π i ( ( p i , p ⋆⋆ -i ) , p ⋆⋆ ) ≤ Π i ( p ⋆⋆ , p ⋆⋆ ) for all p i ∈ P , i.e., when the reference price and firm -i 's price are equal to the SNE price, the best-response price for firm i is the SNE price p ⋆⋆ i .

Stability. If the price and the reference price attain the SNE at some period t , the reference price remains unchanged in the following period, i.e., p t = r t = p ⋆⋆ implies that r t +1 = p ⋆⋆ .

As a result, when the market reaches the SNE, the firms have no incentive to deviate, and the market remains stable in subsequent competitions. The following proposition states the uniqueness of the SNE and characterizes the boundedness of the SNE.

Proposition 3.1 There exists a unique stationary Nash equilibrium, denoted by p ⋆⋆ = ( p ⋆⋆ H , p ⋆⋆ L ) . In addition, it holds that

where W ( · ) is the Lambert W function (see definition in Eq. (22) ).

Without loss of generality, we assume that the feasible price range P 2 = [ p, p ] 2 is sufficiently large to contain the unique SNE, i.e., p ⋆⋆ ∈ [ p, p ] 2 . Proposition 3.1 provides a quantitative characterization for this assumption: it suffices to choose the price lower bound p to be any real number between ( 0 , min i ∈{ H,L } { 1 / ( b i + c i ) } ] , and the price upper bound p can be any value such that

This assumption is mild as the bound in Eq. (7) is independent of both the price and reference price, and it does not grow exponentially fast with respect to any parameters. Hence, there is no need for p to be excessively large. For example, when a H = a L = 10 and b H = b L = c H = c L = 1 , Eq. (7) becomes p ≥ 7 . 3785 . We refer the reader to Appendix B for further discussions on the structure and computation of the equilibrium pricing policy and SNE, as well as the proof of Proposition 3.1.

4 No-regret learning: Online Projected Gradient Ascent

In this section, we examine the long-term dynamics of the the price and reference price paths to determine if the market stabilizes over time. As studied in [22], under perfect information, the firms operating with full rationality will follow the equilibrium pricing policy in each period, while those functioning with bounded rationality will adhere to the best-response pricing policy throughout the planning horizon. In both situations, the market would stabilize in the long run. However, with partial information, the firms are incapable of computing either the equilibrium or the best-response policies due to the unavailability of reference prices and their competitor's price. Thus, one viable strategy to boost firms' revenues is to dynamically modify prices in response to market feedback.

In light of the success of gradient-based algorithms in the online learning literature (see, e.g., [43, 44, 45]), we propose the Online Projected Gradient Ascent (OPGA) method, as outlined in Algorithm 1. Specifically, in each period, both firms update their current prices using the first-order derivatives of their log-revenues with the same learning rate (see Eq. (9)). It is noteworthy that the difference between the derivatives of the log-revenue and the standard revenue is a scaling factor equal to the revenue itself, i.e.,

Therefore, the price update in Algorithm 1 can be equivalently viewed as an adaptively regularized gradient ascent using the standard revenue function, where the regularizer at period t is ( 1 / Π H ( p t , r t ) , 1 / Π L ( p t , r t ) ) .

We highlight that by leveraging the structure of MNL model, each firm i can obtain the derivative of its log-revenue D t i in Eq. (9) through the market feedback mechanism, i.e., last-period demand. The firms do not need to know their own reference price when executing the OPGA algorithm, and the reference price update in Line 6 is automatically performed by the market. In fact, computing the

Algorithm 1 Online Projected Gradient Ascent (OPGA)

1: Input: Initial reference price r 0 = ( r 0 H , r 0 L ) , initial price p 0 = ( p 0 H , p 0 L ) , and step-sizes { η t } t ≥ 0 .

2: for t = 0 , 1 , 2 , . . . do

3: for i ∈ { H,L } do

4: Compute derivative D t i from price and demand of firm i at period t :

5: Update posted price: p t +1 i ← Proj P ( p t i + η t D t i ) .

6: Reference price update: r t +1 ← α r t +(1 -α ) p t .

derivative D i is identical to querying the first-order oracle, which is a common assumption in the optimization literature [46]. Further, this derivative can also be acquired through a minor perturbation of the posted price, even when the firms lack access to historical prices and market feedback, making the algorithm applicable in a variety of scenarios.

There are two potential ways to analyze Algorithm 1 using well-established theories. Below, we briefly introduce these methods and the underlying challenges, while referring readers to Appendix A for a more detailed discussion.

First, by adding two virtual firms to represent the reference price, the game can be converted into a standard four-player online game, effectively eliminating the evolution of the underlying state (reference price). However, the challenge in analyzing this four-player game arises from the fact that, while real firms have the flexibility to dynamically adjust their step-sizes, the learning rate for virtual firms is fixed to the constant (1 -α ) , where α is the memory parameter for reference price updates. This disparity hinders the direct application of the existing results from multi-agent online learning literature, as these results typically require the step-sizes of all agents either diminish at comparable rates or remain as a small enough constant [47, 48, 14, 15].

The second approach involves translating the OPGA algorithm into a discrete nonlinear system by treating ( p t +1 , r t +1 ) as a vector-valued function of ( p t , r t ) , i.e., ( p t +1 , r t +1 ) = f ( p t , r t ) for some function f ( · ) . In this context, analyzing the convergence of Algorithm 1 is equivalent to examining the stability of the fixed point of f ( · ) , which is related to the spectral radius of the Jacobian matrix ∇ f ( p ⋆⋆ , p ⋆⋆ ) [49, 50]. However, the SNE lacks a closed-form expression, making it difficult to calculate the eigenvalues of ∇ f ( p ⋆⋆ , p ⋆⋆ ) . In addition, the function f ( · ) is non-smooth due to the presence of the projection operator and can become non-stationary when the firms adopt time-varying step-sizes, such as diminishing step-sizes. Moreover, typical results in dynamical systems only guarantee local convergence [51], i.e., the asymptotic stability of the fixed point, whereas our goal is to establish the global convergence of both the price and reference price.

In the following section, we show that the OPGA algorithm with diminishing step-sizes converges to the unique SNE by exploiting characteristic properties of our model. This convergence result indicates that the OPGA algorithm provably achieves the no-regret learning, i.e., in the long run, the algorithm performs at least as well as the best fixed action in hindsight. However, it is essential to note that the reverse is not necessarily true: being no-regret does not guarantee the convergence at all, let alone the convergence to an equilibrium (see, e.g., [52, 53]). In fact, beyond the finite games, the agents may exhibit entirely unpredictable and chaotic behaviors under a no-regret policy [54].

5 Convergence results

In this section, we investigate the convergence properties of the OPGA algorithm. First, in Theorem 5.1, we establish the global convergence of the price path and reference price to the unique SNE under diminishing step-sizes. Subsequently, in Theorem 5.2, we further show that this convergence exhibits a rate of O (1 /t ) , provided that the step-sizes are selected appropriately. The primary proofs for this section can be found in Appendices C and D, while supporting lemmas are located in Appendix F.

Theorem 5.1 (Global convergence) Let the step-sizes { η t } t ≥ 0 be a non-increasing sequence such that lim t →∞ η t = 0 and ∑ ∞ t =0 η t = ∞ hold. Then, the price paths and reference price paths generated by Algorithm 1 converge to the unique stationary Nash equilibrium.

Theorem 5.1 demonstrates the global convergence of the OPGA algorithm to the SNE, thus ensuring the market equilibrium and stability in the long-run. Compared to [22] that establishes the convergence to SNE under the perfect information setting, Theorem 5.1 ensures that such convergence can also be achieved in an opaque market where the firms are reluctant to share the information with their competitors. The only condition we need for the convergence is diminishing step-sizes such that lim t →∞ η t = 0 and ∑ ∞ t =0 η t = ∞ . This assumption is widely adopted in the study of online games (see, e.g., [15, 14, 40]). Since the firms will likely become more familiar with their competitors through repeated competitions, it is reasonable for the firms to gradually become more conservative in adjusting their prices and decrease their learning rates.

As discussed in Sections 2 and 4, the existing methods in the online game and dynamical system literatures are not applicable to our problem due to the absence of standard structural properties and the presence of the underlying dynamic state, i.e., reference price. Consequently, we develop a novel analysis to prove Theorem 5.1, leveraging the characteristic properties of the MNL demand model. We provide a proof sketch below, with the complete proof deferred to Appendix C. Our proof consists of two primary parts:

In Part 1 (Appendix C.1), we show that the price path { p t } t ≥ 0 would enter the neighborhood N 1 ϵ := { p ∈ P 2 | ε ( p ) < ϵ } infinitely many times for any ϵ > 0 , where ε ( · ) is a weighted ℓ 1 distance function defined as ε ( p ) := | p ⋆⋆ H -p H | / ( b H + c H ) + | p ⋆⋆ L -p L | / ( b L + c L ) . To prove Part 1, we divide P 2 into four quadrants with p ⋆⋆ as the origin. Then, employing a contradictionbased argument, we suppose that the price path only visits N 1 ϵ finitely many times. Yet, we show that this forces the price path to oscillate between adjacent quadrants and ultimately converge to the SNE, which violates the initial assumption.

In Part 2 (Appendix C.2), we show that when the price path { p t } t ≥ 0 enters the ℓ 2 -neighborhood N 2 ϵ := { p ∈ P 2 | ∥ p -p ⋆⋆ ∥ 2 < ϵ } for some sufficiently small ϵ > 0 and with small enough step-sizes, the price path will remain in N 2 ϵ in subsequent periods. The proof of Part 2 relies on a local property of the MNL demand model around the SNE, with which we demonstrate that the price update provides adequate ascent to ensure the price path stays within the target neighborhood.

Owing to the equivalence between different distance functions in Euclidean spaces [55], these two parts jointly imply the convergence of the price path to the SNE. Since the reference price is equal to an exponential smoothing of the historical prices, the convergence of the reference price path follows that of the price path.

It is worth noting that [21, Theorem 5.1] also employs a two-part proof and shows the asymptotic convergence of online mirror descent [56, 57] to the SNE in the linear demand setting. However, their analysis heavily depends on the linear structure of the demand, which ensures that a property similar to the variational stability is globally satisfied [21, Lemma 9.1]. In contrast, such properties no longer hold for the MNL demand model considered in this paper. Therefore, we come up with two distinct distance functions for the two parts of the proof, respectively. Moreover, the proof by [21] relies on an assumption concerning the relationship between responsiveness parameters, whereas our convergence analysis only requires the minimum assumption that the responsiveness parameters are positive (otherwise, the model becomes counter-intuitive for substitutable products).

Theorem 5.2 (Convergence rate) When both firms adopt Algorithm 1, there exists a sequence of step-sizes { η t } t ≥ 0 with η t = Θ(1 /t ) and constants d p , d r > 0 such that

Theorem 5.2 improves the result of Theorem 5.1 by demonstrating the non-asymptotic convergence rate for the price and reference price. Although non-concavity usually anticipates slower convergence, our rate of O (1 /t ) matches [21, Theorem 5.2] that assumes linear demand and exhibits concavity.

The proof of Theorem 5.2 is primarily based on the local convergence rate when the price vector remains in a specific neighborhood N 2 ϵ 0 of the SNE after some period T ϵ 0 . As the choice η t = Θ(1 /t )

ensures that lim t →∞ η t = 0 and ∑ ∞ t =0 η t = ∞ , the existence of such a period T ϵ 0 is guaranteed by Theorem 5.1. Utilizing an inductive argument, we first show that the difference between price and reference price decreases at a faster rate of O (1 /t 2 ) , i.e., ∥ ∥ r t -p t ∥ ∥ 2 2 = O ( 1 /t 2 ) , ∀ t ≥ 1 (see Eq. (67)). Then, by exploiting a local property around the SNE (Lemma F.3), we use another induction to establish the convergence rate of the price path after it enters the neighborhood N 2 ϵ 0 . In the meanwhile, the convergence rate of the reference price path can be determined through a triangular inequality: ∥ ∥ p ⋆⋆ -r t ∥ ∥ 2 2 = ∥ ∥ p ⋆⋆ -p t + p t -r t ∥ ∥ 2 2 ≤ 2 ∥ ∥ p ⋆⋆ -p t ∥ ∥ 2 2 + 2 ∥ ∥ p t -r t ∥ ∥ 2 2 . Finally, due to the boundedness of the feasible price range P 2 , we can obtain the global convergence rate for all t ≥ 1 by choosing sufficiently large constants d p and d r .

(a) Convergence with

η

t

= 3

/t

.

(b) Cyclic pattern with

η

t

= 1

.

(c) OPGA vs. equilibrium policy.

5

PH

PL

rH

rL

P*

Pt*

2.50

2.25

PH

PL

rH

rL

P*

P*

3.0

四

L

=

OPGA:TH

OPGA: rL

中

H :3N

NE: rL

Pt*

P*

2.5

四

2.00

四

2.0

Price

3

Price

1.75

Price

1.50

1.5

2

1.25

1.0

1.00

0.5

0

0.75

111111?

0.0

0

10

20

30

40

50

6

450

460

470

480

490

5C

0

10

20

30

40

50

60

Time

Time

Time

We perform two numerical experiments, differentiated solely by their sequences of step-sizes, to highlight the significance of step-sizes in achieving the convergence. In particular, Example 1 (see Figure 1a) corroborates Theorem 5.1 by demonstrating that the price and reference price trajectories converge to the unique SNE when we choose diminishing step-sizes that fulfill the criteria specified in Theorem 5.1. By comparison, the over-large constant step-sizes employed in Example 2 (see Figure 1b) fails to ensure convergence, leading to cyclic patterns in the long run. Moreover, in Example 3 (see Figure 1c), we compare the OPGA algorithm and the repeated application of the equilibrium pricing policy in Eq. (5) by plotting the reference price trajectories produced by both approaches. Figure 1c conveys that the two algorithms reach the SNE at a comparable rate, which indicates that the OPGA algorithm allows firms to attain the market equilibrium and stability as though they operate under perfect information, while preventing excessive information disclosure.

6 Extensions

In previous sections, we assume that each firm i possesses accurate information regarding its sensitivity parameters ( b i , c i ) as well as its realized market share d t i . This is equivalent to having access to an exact first-order oracle. However, a more practical scenario to consider is one where the firm can only obtain a rough approximation for its market share and needs to estimate the sensitivities from historical data. This would bring extra noise to the computation of the first-order derivative D t i in Eq. (9). In this section, we first elaborate on the feasibility of estimating the market share and sensitivities from realistic data. Then, we discuss the impact of an inexact first-order oracle on the convergence.

We consider the approximation of market share and the calibration of sensitivities under both uncensored and censored data cases. With uncensored data, both purchase and no-purchase data are

available, a scenario analogous to firms selling substitute goods on third-party online retail platforms like Amazon. The platform can track the non-purchase statistics by monitoring consumers who visited its website but did not purchase any product. Consequently, this facilitates a direct estimation of the total market size and, when complemented with sales quantities, allows for a precise computation of the firm's market share. Additionally, sensitivity parameters can be efficiently calibrated through the classical Maximum Likelihood Estimation (MLE) method. Due to the concavity of the log-likelihood function for the MNL model with respect to its parameters, many numerical algorithms (e.g., NewtonRaphson, BHHH-2 [58], and the steepest ascent) are guaranteed to improve at each iteration and achieve fast convergence [59, 60].

In many cases, the traditional transaction data only captures the realized demand, with non-purchases typically unrecorded. This scenario is known as the censored data case. In this case, the sensitives and the total market size of the MNL model can be estimated via the generalized expectation-maximization (GEM) gradient method proposed by [59], which is an iterative algorithm that is inspired by the expectation-maximization (EM) approach. The capability to approximate the sensitivity parameters, the market share, and thereby the gradient D t i enhances the credibility and practicability of the OPGA algorithm, suggesting its great potential for broader applications in retailing.

In the presence of approximation errors, firms cannot precisely compute the derivative D t i . Hence, the convergence results in Section 5 are not directly applicable. Indeed, if the errors are disruptive enough, one can anticipate Algorithm 1 to possibly not show any convergent behavior. However, if the errors are uniformly bounded by some small number δ , the following theorem demonstrates that both the price and reference price paths converge to a O ( δ ) -neighborhood of the unique SNE.

Theorem 6.1 (Inexact first-order oracle) Suppose both firms only have access to an inexact firstorder oracle such that | D t i -∂ log(Π i ( p t , r t )) /∂p i | ≤ δ, ∀ i ∈ { H,L } and ∀ t ≥ 0 . Let the step-sizes { η t } t ≥ 0 be a non-increasing sequence such that lim t →∞ η t = 0 and ∑ ∞ t =0 η t = ∞ hold. Then, the price paths reference price paths generated by Algorithm 1 converge to a neighborhood with radius O ( δ ) of the unique SNE.

We remark that the inexact first-order oracle studied in our setting is different from the stochastic gradient, which generally assumes a zero-mean noise with finite variance. In that case, it is possible to derive the convergence to a limiting point in expectation or with high probability. In contrast, the noise in D t i is a kind of approximation error without any distributional properties. Therefore, with the step-sizes defined in Theorem 6.1, we expect the price and reference price paths to converge to the neighborhood of the SNE, but continue to fluctuate around the neighborhood without admitting a limiting point.

The proof of Theorem 6.1 is based upon Theorem 5.1. Specifically, we demonstrate that the inexact gradient still guides the price path toward the SNE if the magnitude of the true gradient dominates the noise. Conversely, if noise levels are comparable with the true gradient, we show that the price path is already close to the SNE. The formal proof of Theorem 6.1 can be found in Appendix E

7 Conclusion and future work

This article considers the price competition with reference effects in an opaque market, where we formulate the problem as an online game with an underlying dynamic state, known as reference price. We accomplish the goal of simultaneously capturing the market equilibrium and stability via proposing the no-regret algorithm named OPGA and providing the theoretical guarantee for its global convergence to the SNE. In addition, under appropriate step-sizes, we show that the OPGA algorithm has the convergence rate of O (1 /t ) .

In this paper, we focus on the symmetric reference effects, where consumers exhibit equal sensitivity to the gains and losses. As empirical studies suggest that consumers may display asymmetric reactions [61, 32, 62], one possible extension involves accounting for asymmetric reference effects displayed by loss-averse and gain-seeking consumers. Additionally, while our study considers the duopoly competition between two firms, it would be valuable to explore the more general game involving n players. Lastly, our research is based on a deterministic setting and pure strategy Nash equilibrium. Another intriguing direction for future work is to investigate mixed-strategy learning [63] in dynamic competition problems.

References

Tridib Mazumdar, Sevilimedu P Raj, and Indrajit Sinha. Reference price research: Review and propositions. Journal of Marketing , 69(4):84-102, 2005.

Bruce GS Hardie, Eric J Johnson, and Peter S Fader. Modeling loss aversion and reference dependence effects on brand choice. Marketing Science , 12(4):378-394, 1993.

James M Lattin and Randolph E Bucklin. Reference effects of price and promotion on brand choice behavior. Journal of Marketing Research , 26(3):299-310, 1989.

Richard A Briesch, Lakshman Krishnamurthi, Tridib Mazumdar, and Sevilimedu P Raj. A comparative analysis of reference price models. Journal of Consumer Research , 24(2):202-214, 1997.

Gurumurthy Kalyanaram and Russell S Winer. Empirical generalizations from reference price research. Marketing Science , 14(3_supplement):G161-G169, 1995.

Ioana Popescu and Yaozhong Wu. Dynamic pricing strategies with reference effects. Operations Research , 55(3):413-429, 2007.

Gadi Fibich, Arieh Gavious, and Oded Lowengart. Explicit solutions of optimization models and differential games with nonsmooth (asymmetric) reference-price effects. Operations Research , 51(5):721-734, 2003.

Xin Chen, Peng Hu, and Zhenyu Hu. Efficient algorithms for the dynamic pricing problem with reference price effect. Management Science , 63(12):4389-4408, 2017.

Hansheng Jiang, Junyu Cao, and Zuo-Jun Max Shen. Intertemporal pricing via nonparametric estimation: Integrating reference effects and consumer heterogeneity. Manufacturing & Service Operations Management , Article in advance, 2022.

Don Tapscott and David Ticoll. The naked corporation: How the age of transparency will revolutionize business . Simon and Schuster, 2003.

Robert Bloomfield and Maureen O'Hara. Can transparent markets survive? Journal of Financial Economics , 55(3):425-459, 2000.

Tim Wilson. B2b sellers fight back on pricing. Internet Week , 841:1-2, 2000.

Nelson Granados and Alok Gupta. Transparency strategy: Competing with information in a digital world. MIS quarterly , pages 637-641, 2013.

Mario Bravo, David Leslie, and Panayotis Mertikopoulos. Bandit learning in concave n-person games. Advances in Neural Information Processing Systems , 31, 2018.

Panayotis Mertikopoulos and Zhengyuan Zhou. Learning in games with continuous action sets and unknown payoff functions. Mathematical Programming , 173:465-507, 2019.

Tianyi Lin, Zhengyuan Zhou, Panayotis Mertikopoulos, and Michael Jordan. Finite-time lastiterate convergence for multi-agent learning in games. In International Conference on Machine Learning , pages 6161-6171. PMLR, 2020.

Richard E Caves and Michael E Porter. Market structure, oligopoly, and stability of market shares. The Journal of Industrial Economics , pages 289-313, 1978.

Jing-Sheng Song and Paul Zipkin. Inventory control in a fluctuating demand environment. Operations Research , 41(2):351-370, 1993.

Yossi Aviv and Awi Federgruen. Capacitated multi-item inventory systems with random and seasonally fluctuating demands: implications for postponement strategies. Management Science , 47(4):512-531, 2001.

James T Treharne and Charles R Sox. Adaptive inventory control for nonstationary demand and partial information. Management Science , 48(5):607-624, 2002.

Negin Golrezaei, Patrick Jaillet, and Jason Cheuk Nam Liang. No-regret learning in price competitions under consumer reference effects. Advances in Neural Information Processing Systems , 33:21416-21427, 2020.

Mengzi Amy Guo and Zuo-Jun Max Shen. Duopoly price competition with reference effects under logit demand. Available at SSRN 4389003 , 2023.

Ruxian Wang. When prospect theory meets consumer choice models: Assortment and pricing management with reference prices. Manufacturing & Service Operations Management , 20(3):583-600, 2018.

Mengzi Amy Guo, Hansheng Jiang, and Zuo-Jun Max Shen. Multi-product dynamic pricing with reference effects under logit demand. Available at SSRN 4189049 , 2022.

Tatiana Tatarenko and Maryam Kamgarpour. Learning generalized nash equilibria in a class of convex games. IEEE Transactions on Automatic Control , 64(4):1426-1439, 2018.

Panayotis Mertikopoulos and Mathias Staudigl. Convergence to nash equilibrium in continuous games with noisy first-order feedback. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC) , pages 5609-5614. IEEE, 2017.

Yu-Guan Hsieh, Kimon Antonakopoulos, and Panayotis Mertikopoulos. Adaptive learning in continuous games: Optimal regret bounds and convergence to nash equilibrium. In Conference on Learning Theory , pages 2388-2422. PMLR, 2021.

Yu-Guan Hsieh, Kimon Antonakopoulos, Volkan Cevher, and Panayotis Mertikopoulos. Noregret learning in games with noisy feedback: Faster rates and adaptivity via learning rate separation. Advances in Neural Information Processing Systems , 35:6544-6556, 2022.

Jacob Abernethy, Peter L Bartlett, Alexander Rakhlin, and Ambuj Tewari. Optimal strategies and minimax lower bounds for online convex games. COLT '08: Proceedings of the 21st Annual Conference on Learning Theory , pages 415-423, 2008.

Geoffrey J Gordon, Amy Greenwald, and Casey Marks. No-regret learning in convex games. In Proceedings of the 25th international conference on Machine learning , pages 360-367, 2008.

Harry Helson. Adaptation-level theory: An experimental and systematic approach to behavior. Harper and Row: New York, 1964.

Gurumurthy Kalyanaram and John DC Little. An empirical analysis of latitude of price acceptance in consumer package goods. Journal of Consumer Research , 21(3):408-418, 1994.

John G Lynch Jr and Thomas K Srull. Memory and attentional factors in consumer choice: Concepts and research methods. Journal of Consumer Research , 9(1):18-37, 1982.

Zhenyu Hu, Xin Chen, and Peng Hu. Dynamic pricing with gain-seeking reference price effects. Operations Research , 64(1):150-157, 2016.

Arnoud V den Boer and N Bora Keskin. Dynamic pricing with demand learning and reference effects. Management Science , 68(10):7112-7130, 2022.

Sheng Ji, Yi Yang, and Cong Shi. Online learning and pricing for multiple products with reference price effects. Available at SSRN 4349904 , 2023.

Brian Coulter and Srini Krishnamoorthy. Pricing strategies with reference effects in competitive industries. International transactions in operational Research , 21(2):263-274, 2014.

Awi Federgruen and Lijian Lu. Price competition based on relative prices. Columbia Business School Research Paper , (13-9), 2016.

Luca Colombo and Paola Labrecciosa. Dynamic oligopoly pricing with reference-price effects. European Journal of Operational Research , 288(3):1006-1016, 2021.

Tianyi Lin, Zhengyuan Zhou, Wenjia Ba, and Jiawei Zhang. Optimal no-regret learning in strongly monotone games with bandit feedback. arXiv preprint arXiv:2112.02856 , 2021.

Noah Golowich, Sarath Pattathil, and Constantinos Daskalakis. Tight last-iterate convergence rates for no-regret learning in multi-player games. Advances in neural information processing systems , 33:20766-20778, 2020.

Daniel McFadden. Conditional logit analysis of qualitative choice behavior. Frontiers in Econometrics , pages 105-142, 1974.

Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03) , pages 928-936, 2003.

Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning , 69(2-3):169-192, 2007.

Jacob Abernethy, Peter L Bartlett, and Elad Hazan. Blackwell approachability and no-regret learning are equivalent. In Proceedings of the 24th Annual Conference on Learning Theory , pages 27-46. JMLR Workshop and Conference Proceedings, 2011.

Yurii Nesterov. Introductory lectures on convex optimization: A basic course , volume 87. Springer Science & Business Media, 2003.

Anna Nagurney and Ding Zhang. Projected dynamical systems and variational inequalities with applications , volume 2. Springer Science & Business Media, 1995.

Gesualdo Scutari, Daniel P Palomar, Francisco Facchinei, and Jong-Shi Pang. Convex optimization, game theory, and variational inequality theory. IEEE Signal Processing Magazine , 27(3):35-49, 2010.

David K Arrowsmith, Colin M Place, CH Place, et al. An introduction to dynamical systems . Cambridge university press, 1990.

Peter J Olver. Nonlinear systems. Lecture Notes, University of Minnesota , 2015.

H.K. Khalil. Nonlinear Systems . Pearson Education. Prentice Hall, 2002.

Panayotis Mertikopoulos, Christos Papadimitriou, and Georgios Piliouras. Cycles in adversarial regularized learning. SODA '18: Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 2703-2717, 2018.

Panayotis Mertikopoulos, Bruno Lecouat, Houssam Zenati, Chuan-Sheng Foo, Vijay Chandrasekhar, and Georgios Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. In ICLR 2019-7th International Conference on Learning Representations , pages 1-23, 2019.

Gerasimos Palaiopanos, Ioannis Panageas, and Georgios Piliouras. Multiplicative weights update with constant step-size in congestion games: Convergence, limit cycles and chaos. Advances in Neural Information Processing Systems , 30, 2017.

W. Rudin. Real and Complex Analysis . Mathematics series. McGraw-Hill, 1987.

Zhengyuan Zhou, Panayotis Mertikopoulos, Aris L Moustakas, Nicholas Bambos, and Peter Glynn. Mirror descent learning in continuous games. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC) , pages 5776-5783. IEEE, 2017.

Steven CH Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. Online learning: A comprehensive survey. Neurocomputing , 459:249-289, 2021.

Ernst R Berndt, Bronwyn H Hall, Robert E Hall, and Jerry A Hausman. Estimation and inference in nonlinear structural models. In Annals of Economic and Social Measurement, Volume 3, Number 4 , pages 653-665. NBER, 1974.

Ruxian Wang and Zizhuo Wang. Consumer choice models with endogenous network effects. Management Science , 63(11):3944-3960, 2017.

Kenneth E Train. Discrete choice methods with simulation . Cambridge university press, 2009.

MUKalwani, CK Yim, HJ Rinne, and Y Sugita. 0a price expectations model of consumer brand choice. V Journal of Marketing Research , 27, 1990.

Robert Slonim and Ellen Garbarino. Similarities and differences between stockpiling and reference effects. Managerial and Decision Economics , 30(6):351-371, 2009.

Steven Perkins, Panayotis Mertikopoulos, and David S Leslie. Mixed-strategy learning with continuous action sets. IEEE Transactions on Automatic Control , 62(1):379-384, 2015.

Eric Mazumdar, Lillian J Ratliff, and S Shankar Sastry. On gradient-based learning in continuous games. SIAM Journal on Mathematics of Data Science , 2(1):103-131, 2020.

Eric W Weisstein. Lambert w-function. https://mathworld. wolfram. com/ , 2002.

Appendix A Discussion about alternative methods

As we briefly mentioned in Section 3, our problem can also be translated into a four-player online game or a dynamical system. In this section, we will discuss these alternative methods and explain why existing tools from the literature of online game and dynamical system cannot be applied.

A.1 Four-player online game formulation

By viewing the underlying state variables ( r H , r L ) as virtual players that undergo deterministic transitions, we are able to convert our problem into a general four-player game. Specifically, we can construct revenue functions R i ( p i , r i ) for i ∈ { H,L } and a common sequence of step-sizes { η t r } t ≥ 0 for the two virtual players such that when firms H , L and virtual players implement OPGA using the corresponding step-sizes, the resulting price path { p t } t ≥ 0 and reference price path { r t } t ≥ 0 recover those generated by Algorithm 1. The functions R i ( · , · ) , ∀ i ∈ { H,L } and step-sizes { η t r } t ≥ 0 are specified as follows

Then, given the initial reference price r 0 and initial price p 0 , the four players respectively update their states through the online projected gradient ascent specified as follows, for t ≥ 0 :

It can be easily seen that the pure strategy Nash equilibrium of this four-player static game is equivalent to the SNE of the original two-player dynamic game, as defined in Definition 3.1. However, even after converting to the static game, no general convergence results are readily applicable in this setting. One obstacle on the convergence analysis comes from the absence of critical properties, such as monotonicity of the static game [14, 16] or variational stability at its Nash equilibrium [26, 15].

Meanwhile, anther obstacle stems from the asynchronous updates for real firms (price players) and virtual firms (reference price players). While the real firms have the flexibility in adopting timevarying step-sizes, the updates for the two virtual firms in Eq. (12) stick to the constant step-size of (1 -α ) . As a result, this rigidity of virtual players perplexes the analysis, given that the typical convergence results of online games call for the step-sizes of multiple players to have the same pattern (all diminishing or constant step-sizes) [47, 48, 14, 15].

A.2 Dynamical system formulation

The study of the limiting behavior of a competitive gradient-based learning algorithm is related to dynamical system theories [64]. In fact, the update of Algorithm 1 can be viewed as a nonlinear dynamical system. Assume a constant step-size is employed, i.e., η t ≡ η , ∀ t ≥ 0 . Then, Lines 5 and 6 in Algorithm 1 are equivalent to the dynamical system

where f ( · ) is a vector-valued function defined as

Under the assumption that p ⋆⋆ ∈ P 2 , it is evident that p ⋆⋆ is the unique fixed point of the system in Eq. (13). Generally, fixed points can be categorized into three classes:

asymptotically stable , when all nearby solutions converge to it,

stable , when all nearby solutions remain in close proximity,

unstable , when almost all nearby solutions diverge away from the fixed point.

Hence, if we can demonstrate the asymptotic stability of p ⋆⋆ , we can at least prove the local convergence of the price and reference price.

Standard dynamical systems theory [49, 50] states that p ⋆⋆ is asymptotically stable if the spectral radius of the Jacobian matrix ∇ f ( p ⋆⋆ , p ⋆⋆ ) is strictly less than one. However, computing the spectral radius is not straightforward. The primary challenge stems from the fact that, while the entries of ∇ f ( p ⋆⋆ , p ⋆⋆ ) contain p ⋆⋆ and d i ( p ⋆⋆ , p ⋆⋆ ) , there is no closed-form expression for p ⋆⋆ .

Apart from the above issue, it is worth noting that the function f ( · ) is not globally smooth due to the presence of the projection operator. Furthermore, the function f ( · ) also depends on the step-size η . When firms adopt time-varying step-sizes, the dynamical system in Eq. (13) becomes non-stationary, i.e., ( p t +1 , r t +1 ) = f t ( p t , r t ) . Although the sequence of functions f t t ≥ 0 shares the same fixed point, verifying the convergence (stability) of the system requires examining the spectral radius of ∇ f t ( p ⋆⋆ , p ⋆⋆ ) for all t ≥ 0 .

Most importantly, even if asymptotic stability holds, it can only guarantee local convergence of Algorithm 1. Our goal, however, is to prove global convergence, such that both the price and reference price converge to the SNE for arbitrary initializations.

Appendix B More details about SNE

B.1 Connection between equilibrium pricing policy and SNE

First, we further elaborate on the connection between equilibrium pricing policy and SNE. The work [22] investigates the properties of equilibrium pricing policy under the full information setting. They demonstrate that the policy p ⋆ ( r ) is a well-defined function since it outputs a unique equilibrium price for every given reference price r . Further, for any given reference price r , the equilibrium price ( p ⋆ H ( r ) , p ⋆ L ( r ) ) is the unique solution to the following system of first-order equations [22, Eq. (B.10)]:

The intertemporal aspect of the memory-based reference effect prompts researchers to examine the long-term consequence of repeatedly applying the equilibrium pricing policy in conjunction with reference price updates. Theorem 4 in [22] confirms that in the complete information setting, the price paths and reference price paths generated by the equilibrium pricing policy will converge to the SNE in the long run. Moreover, based on Eq. (15) for the equilibrium pricing policy, the SNE p ⋆⋆ is

the unique solution of the following system of equations:

It can be seen from Eq. (16) that the value of SNE depends only on the model parameters.

B.2 Proof of Proposition 3.1

Proof. Firstly, the uniqueness of SNE has been established in the discussion in Appendix B.1. Below, we show the boundedness of the SNE. By performing a transformation on Eq. (16), we obtain the following inequality for p ⋆⋆ :

To make the above inequality valid, it immediately follows that

Now, we derive the upper bound for p ⋆⋆ i from the second inequality in Eq. (18). Since the quantity on the right-hand side of Eq. (18) is monotone decreasing in p ⋆⋆ i , the value of p ⋆⋆ i must be upper-bounded by the unique solution to the following equation with respect to p i :

Define x := -b i / ( b i + c i ) + b i · p i . Then, one can easily verify that the above equation can be converted into which implies that

where W ( · ) is known as the Lambert W function [65]. For any value y ≥ 0 , W ( y ) is defined as the unique real solution to the equation

Hence, we have

Together with the lower bound provided in Eq. (18), this completest the proof. □

Appendix C Proof of Theorem 5.1

Proof. In this section, we present the proof of Theorem 5.1. By Lemma F.1, when the step-sizes in Algorithm 1 are non-increasing and lim t →∞ η t = 0 , the difference between reference price and price converges to zero as t goes to infinity, i.e., lim t →∞ ( r t -p t ) = 0 . Thus, we are left to verify that the price path { p t } t ≥ 0 converges to the unique SNE, denoted by p ⋆⋆ (see Definition 3.1). Recall that the uniqueness of the SNE is established in Proposition 3.1.

Consider the following weighted ℓ 1 -distance between a point p and the SNE p ⋆⋆ :

In a nutshell, our proof contains the following two parts:

In Part 1 (Appendix C.1), we show that the price path { p t } t ≥ 0 would enter the ℓ 1 -neighborhood N 1 ϵ 0 := { p ∈ P 2 | ε ( p ) < ϵ 0 } infinitely many times for any ϵ 0 > 0 .

In Part 2 (Appendix C.2), we show that when the price path { p t } t ≥ 0 enters the ℓ 2 -neighborhood N 2 ϵ 0 := { p ∈ P 2 | ∥ p -p ⋆⋆ ∥ 2 < ϵ 0 } for some sufficiently small ϵ 0 > 0 with small enough step-sizes, the price path will stay in N 2 ϵ 0 during subsequent periods.

Due to the diminishing step-sizes and the equivalence between ℓ 1 and ℓ 2 norms, Part 1 guarantees that the price path would enter any ℓ 2 -neighborhood N 2 ϵ 0 of p ⋆⋆ with arbitrarily small step-sizes. Together with Part 2 , this proves that for any sufficiently small ϵ 0 > 0 , there exists some T ϵ 0 > 0 such that ∥ p t -p ⋆⋆ ∥ 2 ≤ ϵ 0 for every t ≥ T ϵ 0 , which implies the convergence to the SNE.

C.1 Proof of Part 1

We argue by contradiction. Suppose there eixsts ϵ 0 > 0 such that { p } t ≥ 0 only visits N 1 ϵ 0 finitely many times. This is equivalent to say that ∃ T ϵ 0 such that { p t } t ≥ 0 never visits N 1 ϵ 0 if t ≥ T ϵ 0 .

Firstly, let G i ( p , r ) be the scaled partial derivative of the log-revenue, defined as

For the ease of notation, we denote P i := { p/ ( b i + c i ) | p ∈ P} as the scaled price range. Then, the price update in Line 5 of Algorithm 1 is equivalent to

Let sign( · ) be the sign function defined as

Then, an essential observation from Eq. (26) is that: if sign( p ⋆⋆ i -p t i ) · G i ( p t , r t ) > 0 , we have that sign( p t +1 i -p t i ) = sign ( G i ( p t , r t ) ) = sign( p ⋆⋆ i -p t i ) , i.e., the update from p t i to p t +1 i is toward the direction of the SNE price p ⋆⋆ i . Conversely, if sign( p ⋆⋆ i -p t i ) · G i ( p t , r t ) < 0 , the update from p t i to p t +1 i is deviating from p ⋆⋆ i .

We separate the whole feasible price range into four quadrants with p ⋆⋆ being the origin:

Below, we first show that when t is sufficiently large, prices in consecutive periods cannot always stay within a same region. We prove it by a contradiction argument. Suppose that there exists some period T ϵ 0 1 > T ϵ 0 , after which the price path { p t } t ≥ T ϵ 0 1 stays within a same region, i.e.,

where we use the property of the projection operator in (∆ 1 ) . The equality (∆ 2 ) follows since sign ( p ⋆⋆ i -p t +1 i ) = sign ( p ⋆⋆ i -p t i ) for i ∈ { H,L } , and the projection does not change the sign, then we have

The inequality (∆ 3 ) results from Lemma F.5. In particular, since ∥ ∥ ∇ r G i ( p , r ) ∥ ∥ 2 ≤ ℓ r , where ℓ r = (1 / 4) √ c 2 H + c 2 L , it follows that

∣ ∣ G i ( p t , r t ) -G i ( p t , p t ) ∣ ∣ ≤ ∥ ∥ ∇ r G i ( p , r ) ∥ ∥ 2 · ∥ ∥ r t -p t ∥ ∥ 2 ≤ ℓ r ∥ ∥ r t -p t ∥ ∥ 2 , ∀ i ∈ { H,L } . (30) In step (∆ 4 ) , we introduce the function G ( p ) , which is defined as

Finally, the last step (∆ 5 ) is due to Lemma F.2, which states that ∃ M ϵ 0 > 0 such that G ( p ) ≥ M ϵ 0 for every p ∈ P 2 with ε ( p ) > ϵ 0 . Since for t ≥ T ϵ 0 1 > T ϵ 0 , we have ε ( p t ) ≥ ϵ 0 by the premise, it follows that G ( p t ) ≥ M ϵ 0 .

By Lemma F.1, the difference { r t -p t } t ≥ 0 converges to 0 as t approaches to infinity . As a result, we can always find T ϵ 0 2 > 0 to ensure ∥ r t -p t ∥ 2 ≤ M ϵ 0 / (4 ℓ r ) for all t ≥ T ϵ 0 2 . Then, the last line in Eq. (29) can be further upper-bounded as

By a telescoping sum from any period T -1 ≥ ̂ T ϵ 0 down to period ̂ T ϵ 0 , we have that

Since the step-sizes satisfy ∑ ∞ t =0 η t = ∞ , we have that lim T →∞ ( M ϵ 0 / 2) ∑ T -1 t = ̂ T ϵ 0 η t = ∞ , which further implies the contradiction that lim T →∞ ε ( p T ) →-∞ . Thus, the prices in consecutive period will not always stay in the same region, i.e., the price path { p t } t ≥ 0 keeps oscillating between different regions N j for j ∈ { 1 , 2 , 3 , 4 } .

̸

Next, we prove that ∃ T ϵ 0 3 ≥ T ϵ 0 such that the price path { p t } t ≥ T ϵ 0 3 only oscillates between adjacent quadrants. Arguing by contradiction, we assume η T ϵ 0 3 < ϵ 0 / (2 M G ) , where M G is the upper bound of ∣ ∣ G i ( p , r ) ∣ ∣ defined in Eq. (100) in Lemma F.5. If p t ∈ N 1 (or N 2 ) and p t +1 ∈ N 3 (or N 4 ), i.e., sign ( p ⋆⋆ i -p t i ) = sign ( p ⋆⋆ i -p t +1 i ) for both i ∈ { H,L } , then it automatically follows from the update rule (see Eq. (26)) that

which again contradicts the premise that { p t } t ≥ T ϵ 0 never visits N 1 ϵ 0 . Thus, when t ≥ T ϵ 0 3 , the price path { p t } t ≥ T ϵ 0 3 only oscillates between adjacent quadrants.

Because of the following two established facts: ( i ) when t ≥ max { T ϵ 0 2 , T ϵ 0 3 } , the price path will not remain in the same quadrant but can only oscillate between adjacent quadrants, and ( ii ) the step-sizes { η t } t ≥ 0 decrease to 0 as t →∞ , we conclude that the price path { p } t ≥ 0 must visit the set

infinitely many times for any ϵ 1 > 0 . Intuitively, set ̂ N ϵ 1 can be understood as the boundary regions between adjacent quadrants. We define T ϵ 1 as the set of 'jumping periods' from ( N 1 ∪ N 3 ) ∩ ̂ N ϵ 1 to ( N 2 ∪ N 4 ) ∩ ̂ N ϵ 1 after T ϵ 0 , i.e.,

Then, the above fact ( i ) implies that |T ϵ 1 | = ∞ for any ϵ 1 > 0 .

The key idea of the following proof is that if { p t } t ≥ 0 visits ̂ N ϵ 1 with some sufficiently small ϵ 1 and a relatively smaller step-size, it will stay in ̂ N ϵ 1 and converge to N 1 ϵ 0 concurrently. This results in a contradiction with our initial assumption that { p t } t ≥ 0 never visits N 1 ϵ 0 when t ≥ T ϵ 0 .

Let T ϵ 1 1 ∈ T ϵ 1 be a jumping period with ϵ 1 ≪ ϵ 0 . Without loss of generality, suppose that ∣ ∣ p ⋆⋆ H -p T ϵ 1 1 H ∣ ∣ / ( b H + c H ) < ϵ 1 . Then, since p T ϵ 1 1 ̸∈ N 1 ϵ 0 , we accordingly have that ∣ ∣ p ⋆⋆ L -p T ϵ 1 1 L ∣ ∣ / ( b L + c L ) >

ϵ 0 -ϵ 1 . By the update rule of price in Eq. (26), it holds that

We note that, the last step above is due to the premise that the step-size is sufficiently small: since ∣ ∣ p ⋆⋆ L -p T ϵ 1 1 L ∣ ∣ / ( b L + c L ) > ϵ 0 -ϵ 1 , the equality sign ( p ⋆⋆ L -p T ϵ 1 1 +1 L ) = sign ( p ⋆⋆ L -p T ϵ 1 1 L ) holds if η T ϵ 1 1 < ( ϵ 0 -ϵ 1 ) /M G . Next, we show that the second term on the right-hand side of Eq. (37) is upper-bounded away from zero:

where (∆ 1 ) holds because of the mean value theorem and the fact that ∥∇ r G L ( p , r ) ∥ 2 ≤ ℓ r for p , r ∈ P 2 by Lemma F.5, inequality (∆ 2 ) applies the mean value theorem again to the demand function. In (∆ 3 ) , we use the assumption that ∣ ∣ p ⋆⋆ H -p T ϵ 1 1 H ∣ ∣ < ( b H + c H ) ϵ 1 and apply a similar

argument as Eq. (101) to derive that

Since ε ( ( p ⋆⋆ H , p T ϵ 1 1 L ) ) > ϵ 0 -ϵ 1 , by Lemma F.2, there exists a constant M ϵ 0 -ϵ 1 > 0 , such that G ( ( p ⋆⋆ H , p T ϵ 1 1 L ) ) ≥ M ϵ 0 -ϵ 1 . Meanwhile, we can choose ϵ 1 sufficiently smaller than ϵ 0 to ensure that (1 / 2) M ϵ 0 -ϵ 1 -(1 / 4) b H ( b H + c H ) ϵ 1 > 0 . Furthermore, recall that T ϵ 1 1 can be arbitrarily chosen from T ϵ 1 , where |T ϵ 1 | = ∞ . Thus, we can always find a sufficiently large T ϵ 1 1 ∈ T ϵ 1 to guarantee that ℓ r ∥ ∥ r T ϵ 1 1 -p T ϵ 1 1 ∥ ∥ 2 ≤ (1 / 2) M ϵ 0 -ϵ 1 -(1 / 4) b H ( b H + c H ) ϵ 1 by Lemma F.1. Back to Eq. (38), it follows that

By substituting Eq. (40) into Eq. (37), we further derive that

This indicates that the price update from p T ϵ 1 1 L to p T ϵ 1 1 +1 L is towards the SNE price p ⋆⋆ L when ∣ ∣ p ⋆⋆ H -p T ϵ 1 1 H ∣ ∣ < ( b H + c H ) ϵ 1 .

Now, we use a strong induction to show that ∣ ∣ p ⋆⋆ H -p t H ∣ ∣ / ( b H + c H ) < ϵ 1 holds for all t ≥ T ϵ 1 1 provided that ϵ 1 is sufficiently small and T ϵ 1 1 is sufficiently large. Suppose there exists T ϵ 1 2 ≥ T ϵ 1 1 such that ∣ ∣ p ⋆⋆ H -p t H ∣ ∣ / ( b H + c H ) < ϵ 1 holds for all t = T ϵ 1 1 , T ϵ 1 1 +1 , . . . , T ϵ 1 2 , we aim to show that ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 +1 H ∣ ∣ / ( b H + c H ) < ϵ 1 . Following the same derivation from Eq. (37) to Eq. (41), it holds that

By telescoping the above inequality from T ϵ 1 2 -1 down to T ϵ 1 1 , we have that

Additionally, we note that ∣ ∣ p ⋆⋆ H -p t H ∣ ∣ / ( b H + c H ) < ϵ 1 and p t ̸∈ N 1 ϵ 0 together imply that ∣ ∣ p ⋆⋆ L -p t L ∣ ∣ / ( b L + c L ) > ( ϵ 0 -ϵ 1 ) for all t = T ϵ 1 1 , T ϵ 1 1 + 1 , . . . , T ϵ 1 2 . As the step-size is sufficiently small, we conclude that the price path { p t L } T ϵ 1 2 t = T ϵ 1 1 lies on the same side of the SNE price p ⋆⋆ L , and thus sign ( p ⋆⋆ L -p T ϵ 1 2 L ) = sign ( p ⋆⋆ L -p T ϵ 1 1 L ) . Below, we discuss two cases based on the value of ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 H ∣ ∣ :

Case 1: ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 H ∣ ∣ / ( b H + c H ) < ϵ 1 -η T ϵ 1 1 M G .

Following a similar derivation as Eq.(37), we have that

where (∆ 1 ) is from Lemma F.5 that ∣ ∣ G H ( p , r ) ∣ ∣ ≤ M G for p , r ∈ P 2 , and (∆ 2 ) is because the sequence of step-sizes is non-increasing, hence η T ϵ 1 1 ≥ η T ϵ 1 2 .

It suffices to show that ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 +1 H ∣ ∣ ≤ ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 H ∣ ∣ . According to the observation below Eq. (27), this is equivalent to showing that

We further discuss two scenarios depending on whether p T ϵ 1 2 ∈ N 1 ∪ N 3 or p T ϵ 1 2 ∈ N 2 ∪ N 4 .

Suppose that p T ϵ 1 2 ∈ N 1 ∪ N 3 . According to the definition of set T ϵ 1 in Eq. (36), we have that p T ϵ 1 1 ∈ N 1 ∪ N 3 since T ϵ 1 1 ∈ T ϵ 1 is a jumping period. Together with the established fact that sign ( p ⋆⋆ L -p T ϵ 1 2 L ) = sign ( p ⋆⋆ L -p T ϵ 1 1 L ) (see the argument below Eq. (43)), we conclude that p T ϵ 1 2 and p T ϵ 1 1 are in the same quadrant, and sign ( p ⋆⋆ H -p T ϵ 1 2 H ) = sign ( p ⋆⋆ H -p T ϵ 1 1 H ) . Then, we can write that

where we apply Lemma F.4 in (∆) by utilizing the relation ∣ ∣ p ⋆⋆ L -p T ϵ 1 2 L ∣ ∣ ≤ ∣ ∣ p ⋆⋆ L -p T ϵ 1 1 L ∣ ∣ proved in Eq. (43). Next, as T ϵ 1 1 is a jumping period, it follows from the price update rule in Eq. (26) that ∣ ∣ p ⋆⋆ H -p T ϵ 1 1 H ∣ ∣ / ( b H + c H ) ≤ ∣ ∣ η T ϵ 1 1 G H ( p T ϵ 1 1 , p T ϵ 1 1 ))∣ ∣ ≤ η T ϵ 1 1 M G . Thus, when the chosen jumping period T ϵ 1 1 is sufficiently large so that the step-size η T ϵ 1 1 is considerably smaller than ϵ 1 , we further derive that

With Eq. (47), we use Lemma F.4 again to upper-bound the last term on the right-hand side of Eq. (46):

where (∆ 1 ) holds since the difference in the previous step is non-negative by Lemma F.4. Furthermore, it is straightforward to observe that the terms diff 1 and diff 2 have the same sign, which results in the final inequality (∆ 2 ) . We substitute Eq. (48) back into Eq. (46):

In inequality (∆ 1 ) , we use the fact that sign ( p ⋆⋆ H -p T ϵ 1 1 H ) · G H ( p T ϵ 1 1 , r T ϵ 1 1 ) > 0 since T ϵ 1 1 is a jumping period in T ϵ 1 . Then, inequality (∆ 2 ) is implied by the properties that sign ( p ⋆⋆ H -p T ϵ 1 2 H ) = sign ( p ⋆⋆ H -p T ϵ 1 1 H ) , ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 H ∣ ∣ / ( b H + c H ) ≥ ϵ 1 -η T ϵ 1 1 M G , and ∣ ∣ p ⋆⋆ H -p T ϵ 1 1 H ∣ ∣ / ( b H + c H ) ≤ η T ϵ 1 1 M G . Since both the step-size η t and the difference ∥ r t -p t ∥ 2 decrease to 0 as t → ∞ , by choosing a sufficiently large jumping period T ϵ 1 1 from T ϵ 1 , the right-hand side of Eq. (46) is guaranteed to be non-negative, i.e., sign ( p ⋆⋆ H -p T ϵ 1 2 H ) · G H ( p T ϵ 1 2 , r T ϵ 1 2 ) ≥ 0 . This completes the proof of scenario 1.

2. Suppose p T ϵ 1 2 ∈ N 2 ∪ N 4 . We deduce that

The inequality (∆ 1 ) follows from Lemma F.4, where sign ( p ⋆⋆ H -p H ) · G H ( p , p ) is deceasing when | p ⋆⋆ L -p L | decreases . The equality (∆ 2 ) is due to the definition of G ( p ) in Eq. (31) and the fact that sign( p ⋆⋆ L -p ⋆⋆ L ) = 0 . Finally, in the last line (∆ 3 ) , we leverage the initial assumption of Case 2 , which implies that ε ( ( p T ϵ 1 2 H , p ⋆⋆ L ) ) = ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 H ∣ ∣ / ( b H + c H ) ∈ [ ϵ 1 -η T ϵ 1 1 M G , ϵ 1 ) . Then, by Lemma F.2, there must exist M ̂ ϵ 1 > 0 with ̂ ϵ 1 := ϵ 1 -η T ϵ 1 2 M G such that G ( ( p T ϵ 1 2 H , p ⋆⋆ L ) ) ≥ M ̂ ϵ 1 . As |T ϵ 1 | = ∞ , we can pick a sufficiently large T ϵ 1 1 ∈ T ϵ 1 to guarantee that ℓ r ∥ ∥ r t -p t ∥ ∥ 2 ≤ M ̂ ϵ 1 for t ≥ T ϵ 1 1 by Lemma F.1. Consequently, we obtain the desired conclusion that sign ( p ⋆⋆ H -p T ϵ 1 2 H ) · G H ( p T ϵ 1 2 , r T ϵ 1 2 ) > 0 when p T ϵ 1 2 ∈ N 2 ∪ N 4 , which completes the proof of scenario 2.

Combining Case 1 and Case 2 , we have shown that ∣ ∣ p ⋆⋆ H -p T ϵ 1 2 +1 H ∣ ∣ / ( b H + c H ) < ϵ 1 , which completes the strong induction, i.e., ∣ ∣ p ⋆⋆ H -p t H ∣ ∣ / ( b H + c H ) < ϵ 1 for all t ≥ T ϵ 1 1 . Under the assumption that { p t } t ≥ 0 never visits N 1 ϵ 0 if t ≥ T ϵ 0 , we accordingly have that ∣ ∣ p ⋆⋆ L -p t L ∣ ∣ / ( b L + c L ) > ( ϵ 0 -ϵ 1 ) for all t ≥ T ϵ 1 1 . Therefore, following the derivations from Eq. (37) to Eq. (41), we deduce that Eq. (42) holds for all t ≥ T ϵ 1 1 . Using a telescoping sum, it holds for any period T > T ϵ 1 1 that

Since M ϵ 0 -ϵ 1 > 0 is a constant and lim T →∞ ∑ T -1 t = T ϵ 1 1 η t = ∞ , we arrive at the contradiction that ∣ ∣ p ⋆⋆ L -p T L ∣ ∣ / ( b L + c L ) →-∞ as T →∞ . Consequently, our initial assumption is incorrect and the price path { p t } t ≥ 0 would visit the ℓ 1 -neighborhood N 1 ϵ 0 infinitely many times for any ϵ 0 > 0 , which completes the proof of Part 1 .

C.2 Proof of Part 2

In particular, we show that when p t ∈ N 2 ϵ 0 for some sufficiently large t , then it also holds p t +1 ∈ N 2 ϵ 0 . The value of ϵ 0 will be specified later in the proof.

By the update rule of Algorithm 1, it holds that

where we use | G i ( p , r ) | ≤ M G and the mean value theorem in the last inequality (see Lemma F.5). Let H ( p ) be a function defined as

Then, by summing Eq. (52) over both products i ∈ { H,L } , we have that

By Lemma F.3, there exist γ > 0 and a open set U γ ∋ p ⋆⋆ such that H ( p ) ≥ γ ·∥ p -p ⋆⋆ ∥ 2 2 , ∀ p ∈ U γ . Consider ϵ 0 > 0 such that the ℓ 2 -neighborhood N 2 ϵ 0 = { p ∈ P 2 | ∥ p -p ⋆⋆ ∥ 2 < ϵ 0 } ⊂ U γ . Furthermore, let T γ be some period such that

The existence of such a number T γ follows from the fact that lim t →∞ η t = 0 and lim t →∞ ∥ r t -p t ∥ 2 = 0 (see Lemma F.1). Below, we discuss two cases depending on the location of p t in N 2 ϵ 0 .

Since H ( p ) ≥ 0 , ∀ p ∈ U γ by Lemma F.3, it follows from Eq. (54) and Eq. (55) that

where inequality (∆) is due to ∥ r t -p t ∥ 2 ≤ √ 2( p -p ) . Eq. (56) implies that p t +1 ∈ N 2 ϵ 0 .

By Lemma F.3, we have that H ( p t ) ≥ γ ∥ ∥ p ⋆⋆ -p t ∥ ∥ 2 2 ≥ γ ( ϵ 0 ) 2 / 4 . Thus, again by Eq. (54) and Eq. (55), we have that

which implies p t +1 ∈ N 2 ϵ 0 . Therefore, we conclude by induction that the price path will stay in the ℓ 2 -neighborhood N 2 ϵ 0 . This completes the proof of Theorem 5.1. □

Appendix D Proof of Theorem 5.2

Proof. Recall the function H ( p ) defined in Eq. (53). By Lemma F.3, there exist γ > 0 and a open set U γ ∋ p ⋆⋆ such that H ( p ) ≥ γ · ∥ p -p ⋆⋆ ∥ 2 2 , ∀ p ∈ U γ . Consider ϵ 0 > 0 such that the ℓ 2 -neighborhood N 2 ϵ 0 = { p ∈ P 2 | ∥ p -p ⋆⋆ ∥ 2 < ϵ 0 } ⊂ U γ . Below, we first show that the price path { p t } t ≥ 0 enjoys the sublinear convergence rate in N 2 ϵ 0 when t is greater than some constant T ϵ 0 . Then, we will show that this convergence rate also holds for any t ≤ T ϵ 0 .

By Part 2 in the proof of Theorem 5.1, there exists T ϵ 0 > 0 such that p t ∈ N 2 ϵ 0 for every t ≥ T ϵ 0 . Following a similar argument as Eq. (52) and Eq. (54), we have that

In step (∆ 1 ) , ℓ r is the Lipschitz constant defined in Eq. (100) and the constant C 1 is defined in Eq. (54). In step (∆ 2 ) , we utilize Lemma F.3 and the following inequality

where we define ˆ k := √ 2 max i ∈{ H,L } { b i + c i } . Step (∆ 3 ) in Eq. (58) follows from the inequality of arithmetic and geometric means, i.e., 2 xy ≤ Ax 2 +(1 /A ) y 2 for any constant A > 0 . The value of constant k in (∆ 4 ) is given by k := ( ℓ r ˆ k ) 2 /γ = 2 ( ℓ r max i ∈{ H,L } { b i + c i } ) 2 /γ .

To upper-bound the right-hand side of Eq. (58), we first focus on the term ∥ ∥ r t -p t ∥ ∥ 2 2 and inductively show that √

where λ := (1+ α 2 ) / 2 < 1 . We use the notation D t := ( D t H , D t L ) = ( ( b H + c H ) G H ( p t , r t ) , ( b L + c L ) G L ( p t , r t ) ) , where we recall that D t i is the partial derivative specified in Eq. (9) and function G i ( · , · ) is defined in Eq. (25). Then, the term ∥ ∥ r t -p t ∥ ∥ 2 2 can be upper-bounded as follows

where (∆ 1 ) holds due to the Cauchy-Schwarz inequality and the property of the projection operator (see Line 5 in Algorithm 1). Step (∆ 2 ) is derived from the inequality of arithmetic and geometric means. Lastly, step (∆ 3 ) applies the upper bound on function G i ( · , · ) in Lemma F.5 and the definition of C 1 in Eq. (54). For the simplicity of notation, we denote the coefficient of ( η t -1 ) 2 in the last line as λ 0 .

Let the step-size be η t = d η / ( t +1) for t ≥ 0 , where d η is some constant that will be determined later. Suppose that there exists a constant d rp such that

for some t ≥ T λ +1 . Then, together with Eq. (61), we have that

where the inequality (∆) results from the choice of T λ . Hence, the induction follows if 0 . 5( λ + 1) d rp + λ 0 ( d η ) 2 ≤ d rp , which is further equivalent to

Lastly, the base case of the induction requires that ∥ ∥ r T λ -p T λ ∥ ∥ 2 2 ≤ d rp ( T λ ) 2 . Therefore, by Eq. (64) and the definition of feasible price range P = [ p, p ] , it suffices to choose

where the constants λ 0 and T λ are respectively defined in Eq. (61)and Eq. (60). Note that, under this choice of constant d rp , it also holds that

Together with Eq. (60), we derive that

For every t ≥ T ϵ 0 , we can further upper-bound the right-hand side of Eq. (58) by exploiting the upper bound of ∥ ∥ r t -p t ∥ ∥ 2 2 in Eq. (67) and the choice of η t = d η / ( t +1) :

Now, we inductively show that

Suppose there exists a constant d p such that for a fixed period t ≥ T ϵ 0

To establish the induction, it suffices for the following inequality to hold

which is further equivalent to

To satisfy the base case of the induction, we can select d p such that d p ≥ T ϵ 0 · ∥ ∥ p ⋆⋆ -p T ϵ 0 ∥ ∥ 2 2 . In summary, one possible set of constants ( d η , d rp , d p ) that satisfies all the requirements can be

where the constants λ 0 , T λ , and C 3 are respectively defined in Eq. (61), Eq. (60), and Eq. (68). Now, for any period 1 ≤ t < T ϵ 0 , it follows that

Together with Eq. (69), this proves the convergence rate for the price path, i.e.,

Finally, the convergence rate of the reference price path can be deduced from the following triangular inequality:

where (∆) follows from Eq. (67) and Eq. (75). Therefore, it suffices to choose d r := 2 d p +2 d rp , and this completes the proof.

□

Appendix E Proof of Theorem 6.1

Proof. Recall the definition of G i ( p , r ) from Eq. (25). We can write the inexact derivative as D t i = ( b i + c i ) G i ( p t , r t ) + n t i , where n t i represents the error satisfying | n t i | < δ , ∀ i ∈ { H,L } and ∀ t ≥ 0 .

We also recall the two-part proof for Theorem 5.1: in Part 1, we show that the price path and reference price paths converge towards the SNE and visit the neighborhood N 1 ϵ infinitely many times, for any ϵ > 0 . In Part 2, we establish that, given ϵ is below a certain threshold ϵ 0 , the price vector would remain in the neighborhood N 2 ϵ after entering it with a sufficiently small step-size. Since the relative scale of δ and ϵ 0 is unsure, our proof below is mainly based on Part 1.

In Part 1, the original proof assuming the exact gradient oracle employs a contradiction-based argument. Suppose the price path does not converge to the SNE, the proof consists of the following major steps (different from Appendix C, we use ϵ in lieu of ϵ 0 to avoid confusion in the following proof):

In Eq. (29), it is demonstrated that the price path steadily approaches the SNE if it stays in the same quadrant defined in Eq. (28). The main technique we use is G ( p ) > M ϵ when ε ( p ) > ϵ . The definition of G ( p ) and the validation for the technique are stated in Lemma F.2.

In Eq. (34), we show that the price path only oscillates between adjacent quadrants provided the step-sizes are sufficiently small.

From Eqs (37) to (41), we prove that even when the price path does not stay in the same quadrant, it will still converge toward the SNE if it is at the boundary regions between two quadrants. The pivotal inequality employed here is again G (( p ⋆⋆ H , p T ϵ 1 1 L )) ≥ M ϵ -ϵ 1 , provided ε (( p ⋆⋆ H , p T ϵ 1 1 L )) > ϵ -ϵ 1 (see the end of Eqs. (38) and (40)).

Finally, in Eq. (44) and subsequent equations, we provide supplementary justification for the above bullet that the price path remains adjacent to the boundaries given that the step-size is small. A crucial consideration here is opting for a much smaller ϵ 1 relative to ϵ when defining the boundary region. In addition, the inequality G (( p T ϵ 1 2 H , p ⋆⋆ L )) ≥ M ̂ ϵ 1 is also utilized in Eq. (50).

To summarize, the key to the proof is Lemma F.2., i.e., G ( p ) > M ϵ > 0 as long as ε ( p ) > ϵ . By definition, G ( p t ) approximately characterizes the difference ε ( p t ) -ε ( p t +1 ) as seen in Eq. (29). As a result, the effect of the lower bound M ϵ is that, if other terms can be upper-bounded by M ϵ , we can show that the updated price is closer to the SNE. For example, in the right-hand side of Eq. (29), once the term 2 ℓ r ∥ r t -p t ∥ 2 is below M ϵ , it suggests that the price vector is heading towards the SNE.

With inexact first-order oracles, the immediate consequence is that there will always be an error term accompanying G ( p ) . For instance, given D t i = ( b i + c i ) G i ( p t , r t ) + n t i , we observe that Eq. (29) evolves to

This observation is consistent across the proof. Thus, if the error n t i is substantially smaller than M ϵ , the proof of Theorem 5.1 remains valid, implying that the price vector converges towards the SNE. The subtlety arises when the size of n t i is comparable with M ϵ . Under such circumstances, the analysis in both Eq. (29) and Eqs. (37) to (41) becomes invalid, as the errors are substantial enough to negate any assurance that the price path strictly approaches the SNE.

However, if the error n t i is similar in magnitude to G ( p ) , we can show that the price vector p is already close to the SNE p ⋆⋆ . More precisely, since the errors are bounded by δ , it is equivalent to show that G ( p ) = O ( δ ) also implies ∥ p -p ⋆⋆ ∥ 2 = O ( δ ) . This can be proved by a refined version of Lemma F.2.

Lemma E.1 (Refined Lemma F.2) Let G ( p ) be the function defined in Eq. (86) . Then, it holds that

i.e., G ( p ) is lower-bounded by a weighted ℓ 1 distance between p and p ⋆⋆ . By the equivalence of norms in the Euclidean space, there exists some constant C such that G ( p ) ≥ C · ∥ p -p ⋆⋆ ∥ 2 .

Proof. Similar to the proof of the original Lemma F.2, we separately consider the four possible scenarios where p belongs to one of the quadrant defined in Eq. (28).

Suppose p H > p ⋆⋆ H and p L ≥ p ⋆⋆ L , i.e., p ∈ N 1 . Since G i ( p ⋆⋆ ) = 0 , we have that

By definition of the non-purchase probability, we observe that d 0 ( p , p ) -d 0 ( p ⋆⋆ , p ⋆⋆ ) > 0 . Hence, the above equation implies that

where we use the fact that p i ∈ P = [ p, p ] and the presumption p ∈ N 1 .

Suppose p H ≤ p ⋆⋆ H and p L > p ⋆⋆ L , i.e., p ∈ N 2 . Again, using the fact that G i ( p ⋆⋆ ) = 0 , we derive that

Since p ∈ N 2 , we have that d H ( p , p ) -d H ( p ⋆⋆ , p ⋆⋆ ) > 0 and d L ( p ⋆⋆ , p ⋆⋆ ) -d L ( p , p ) > 0 . Hence, similar to the first case, it follows that

The same conclusion can be drawn when p ∈ N 3 ∪ N 4 , and the proof is intrinsically the same. Hence, we conclude the proof of this lemma. □

Finally, due to Lemma E.1, we have that ∥ p -p ⋆⋆ ∥ 2 ≤ 1 /C · G ( p ) . Therefore, when G ( p ) = O ( δ ) , we also have ∥ p -p ⋆⋆ ∥ 2 = O ( δ ) , i.e., the price vector is already in a O ( δ ) neighborhood of the SNE. This completes the proof of Theorem 6.1. □

Appendix F Supporting lemmas

Lemma F.1 (Convergence of price to reference price) Let { p t } t ≥ 0 and { r t } t ≥ 0 be the price path and reference path generated by Algorithm 1 with non-increasing step-sizes { η t } t ≥ 0 such that lim t →∞ η t = 0 . Then, their difference { r t -p t } t ≥ 0 converges to 0 as t goes to infinity.

Proof. First, we recall that D t i = ( b i + c i ) · G i ( p t , r t ) , where G i ( p , r ) is the scaled partial derivative defined in (25). Thus, it follows from Lemma F.5 that | D t i | ≤ ( b i + c i ) M G . Since { η t } t ≥ 0 is a non-increasing sequence with lim t →∞ η t = 0 , for any constant η > 0 , there exists T η ∈ N such that | η t D t i | ≤ η for every t ≥ T η and for every i ∈ { H,L } . Therefore, it holds that

where the first inequality is due to the property of the projection operator. Then, by the reference price update, we have for t ≥ T η and for i ∈ { H,L } that

where the last line follows from the upper bound in Eq. (83). Applying Eq. (84) recursively from t to T η , we further derive that

Since η can be arbitrarily close to 0 , we have that | r t i -p t i | → 0 as t → ∞ , which completes the proof of the convergence. □

where G i ( p , r ) is the scaled partial derivative introduced in Eq. (25) . Then, it always holds that G ( p ) > 0 , ∀ p ∈ P 2 \{ p ⋆⋆ } , where p ⋆⋆ = ( p ⋆⋆ H , p ⋆⋆ L ) denotes the unique SNE, and the function sign( · ) is defined in Eq. (27) .

Furthermore, for every ϵ > 0 , there exists M ϵ > 0 such that G ( p ) ≥ M ϵ if ε ( p ) ≥ ϵ , where ε ( p ) is the weighted ℓ 1 -distance function defined in Eq. (24) .

Proof. Firstly, by the first-order condition at the SNE (see Eq. (16)), we have G H ( p ⋆⋆ , p ⋆⋆ ) = G L ( p ⋆⋆ , p ⋆⋆ ) = 0 , which also implies G ( p ⋆⋆ ) = 0 . Then, we recall the definition of four regions N 1 , N 2 , N 3 , and N 4 in Eq. (28). We show that G ( p ) > 0 when p belongs to either one of the four regions.

⋆⋆ ⋆⋆

When p ∈ N , i.e., p > p and p ≥ p , the function G ( p ) becomes

where d 0 ( p , r ) = 1 -d H ( p , r ) -d L ( p , r ) denotes the no purchase probability. We observe from Eq. (87) that G ( p ) is strictly increasing in p H and p L . Together with the fact that p H and p L are lowered bounded by p ⋆⋆ H and p ⋆⋆ L , respectively, and G ( p ⋆⋆ ) = 0 , we verify that G ( p ) > 0 when p ∈ N 1 . With the similar approach, we show that when p ∈ N 3 , i.e., p H < p ⋆⋆ H and p L ≤ p ⋆⋆ L , it also follows that G ( p ) > 0 .

When p ∈ N 2 , i.e., p H ≤ p ⋆⋆ H and p L > p ⋆⋆ L , the function G ( p ) becomes

By Eq. (88), we notice that G ( p ) under region N 2 is strictly decreasing in p H and strictly increasing in p L . Meanwhile, since p H ≤ p ⋆⋆ H , p L > p ⋆⋆ L , and G ( p ⋆⋆ ) = 0 , it implies that G ( p ) > 0 when p ∈ N 2 . Moreover, by similar reasoning, we show that when p ∈ N 4 , i.e., p H ≥ p ⋆⋆ H and p L < p ⋆⋆ L , the inequality G ( p ) > 0 also holds.

Finally, we are left to establish the existence of M ϵ . It suffices to show that

for every ϵ 1 > ϵ 2 > 0 . Suppose p ϵ 1 := arg min ε ( p )= ϵ 1 G ( p ) . Define p ϵ 2 := ( ϵ 2 /ϵ 1 ) p ϵ 1 +(1 -ϵ 2 /ϵ 1 ) p ⋆⋆ , which satisfies that ε ( p ϵ 2 ) = ϵ 2 . Then, we have that

where (∆ 1 ) follows from substituting p ϵ 2 i in sign( p ⋆⋆ i -p ϵ 2 i ) with p ϵ 2 i = ( ϵ 2 /ϵ 1 ) p ϵ 1 i +(1 -ϵ 2 /ϵ 1 ) p ⋆⋆ i . To see why (∆ 2 ) holds, recall that we have shown in Eq. (87) and Eq. (88) that when two prices are in the same region (see four regions defined in Eq. (28)), the price closer to p ⋆⋆ in terms of the metric ε ( · ) has a greater value of G ( p ) . Since p ϵ 1 and p ϵ 2 are from the same region and ϵ 1 > ϵ 2 , we conclude that min ε ( p )= ϵ 1 G ( p ) = G ( p ϵ 1 ) > G ( p ϵ 2 ) ≥ min ε ( p )= ϵ 2 G ( p ) . □

Lemma F.3 Define function H ( p ) as follows

Then, there exist γ > 0 and a open set U ∋ p such that

Proof. According to the partial derivatives in Eq. (98a) and Eq. (98b) from Lemma F.5, we have

Then, to compute the gradient ∇H ( p ) = [ ∂ H ( p ) /∂p H , ∂ H ( p ) /∂p L ] , we utilize partial derivatives of G i ( p , p ) in Eq. (93) and obtain the partial derivatives of H ( p ) for i ∈ { H,L } :

From the definition of G i ( · , · ) in Eq. (31) and the system equations of first-order condition in Eq. (16), we have G i ( p ⋆⋆ , p ⋆⋆ ) = 1 / [ ( b i + c i ) p ⋆⋆ i ] +( d ⋆⋆ i -1) = 0 , where d ⋆⋆ i := d i ( p ⋆⋆ , p ⋆⋆ ) denotes the market share of product i at the SNE. Thereby, it follows that ∇H ( p ⋆⋆ ) = 0 .

Next, the Hessian matrix ∇ 2 H ( p ) evaluated at p ⋆⋆ can be computed as

Note that the last equality results again from substituting in the identity G i ( p ⋆⋆ , p ⋆⋆ ) = 1 / [ ( b i + c i ) · p ⋆⋆ i ] +( d ⋆⋆ i -1) = 0 .

The diagonal entries of ∇ 2 H ( p ⋆⋆ ) are clearly positive. For i ∈ { H,L } , define k i := b i / ( b i + c i ) ∈ [0 , 1] . Then, the determinant of ∇ 2 H ( p ⋆⋆ ) can be computed as follows:

(95) where inequalities respectively result from the following facts (∆ 1 ) : k H + k L ≤ 2 ; (∆ 2 ) : 1 -d ⋆⋆ i > d ⋆⋆ -i for i ∈ { H,L } ; (∆ 3 ) : 1 -k i ≤ 1 for i ∈ { H,L } ; (∆ 4 ) : d ⋆⋆ H + d ⋆⋆ L < 1 . Thus, we conclude that ∇ 2 H ( p ⋆⋆ ) is positive definite.

By the continuity of ∇ 2 H ( p ) , there exists some constant γ > 0 and a open set U γ ∋ p ⋆⋆ such that ∇ 2 H ( p ) ⪰ 2 γI 2 , ∀ p ∈ U γ , where I 2 is the 2 × 2 identity matrix. Using the second-order Taylor expansion at p ⋆⋆ , for all p ∈ U γ , there exists ˜ p ∈ U γ such that

where the second equality arises from that H ( p ⋆⋆ ) = 0 and ∇H ( p ⋆⋆ ) = 0 .

□

where G i ( p , r ) is the scaled partial derivative defined in Eq. (25) . Then, ̂ G i ( p ) is always increasing as | p ⋆⋆ i -p i | increases, and

when p ∈ N 1 ∪ N 3 (see the definition in Eq. (28) ), ̂ G i ( p ) is decreasing as | p ⋆⋆ -i -p -i | increases;

when p ∈ N 2 ∪ N 4 , ̂ G i ( p ) is increasing as | p ⋆⋆ -i -p -i | increases.

Proof. Without loss of generality, consider the case that product i = H and product -i = L .

Apparently, we have sign ( p ⋆⋆ H -p H ) ≤ 0 in N 1 ∪ N 4 , and sign ( p ⋆⋆ H -p H ) ≥ 0 in N 2 ∪ N 3 by definitions (see Eq. (28)). On the other hand, by Eq. (98b) in Lemma F.5, it holds that ∂G H ( p , p ) /∂p H < 0 and ∂G H ( p , p ) /∂p L > 0 , ∀ p ∈ P 2 . Thus,

in N 1 ∪ N 4 , G H ( p , p ) is decreasing as | p ⋆⋆ H -p H | increases; in N 2 ∪ N 3 , G H ( p , p ) is increasing as | p ⋆⋆ H -p H | increases;

in N 1 ∪ N 2 , G H ( p , p ) is increasing as | p ⋆⋆ L -p L | increases; conversely, G H ( p , p ) is decreasing as | p ⋆⋆ L -p L | increases in N 3 ∪ N 4 .

The final results directly follows by combining the above pieces together. □

Lemma F.5 Let G i ( p , r ) be the scaled partial derivative defined in Eq. (25) , then partial derivatives of G i ( p , r ) with respect to p and r are given as

Meanwhile, G i ( p , r ) and its gradient are bounded as follows

where the upper bound constant M G and the Lipschitz constant ℓ r are defined as

Proof. We first verify the partial derivatives from Eq. (98a) to Eq. (98d):

Then, the partial derivatives with respect to r , as shown in Eq. (98c) and Eq. (98d), can be similarly computed.

In the next part, we show that G i ( p , r ) is bounded for p , r ∈ P 2 :

Hence, it follows that ∣ ∣ G i ( p , r ) ∣ ∣ for every i ∈ { H,L } is upper bounded by

Finally, we demonstrate that ∥∇ r G i ( p , r ) ∥ is also bounded ∀ p , r ∈ P 2 and ∀ i ∈ { H,L } . From Eq. (98c) and Eq. (98d), we have

where the inequality follows from the fact that x · y ≤ 1 / 4 for any two numbers such that x, y > 0 and x + y ≤ 1 . Thus, it follows that ∥ ∥ ∇ r G i ( p , r ) ∥ ∥ 2 ≤ (1 / 4) √ c 2 H + c 2 L := l r , ∀ p , r ∈ P 2 and ∀ i ∈ { H,L } . □

Limitaions

This is a theoretical work that concerns with algorithm design in a competitive market, with the primary goal of learning a stable equilibrium. The consumer demand follows the multinomial logit model, and we assume that their price/reference price sensitivities are positive, i.e., b i , c i > 0 . The feasible price range P = [ p, p ] is assumed to contain the unique SNE, with the lower bound p > 0 . We have justified both assumptions in the main body of the paper (see the discussion below Eq. (3) and Proposition 3.1).

The convergence results in the paper assume the firms take diminishing step-sizes { η t } t ≥ 0 with ∑ ∞ t =0 η t = ∞ , which is common in the literature of online games (see, e.g., [15, 14, 40]). We also discuss the extension to constant step-sizes in Remark 5.3.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies gradient descent dynamics in duopoly competitions with reference effects and logit demand. \nConvergence results are proven to show that Online Projected Gradient Ascent (OPGA) with decreasing step size converges to a stationary Nash equilibrium. This is a novel result requiring new analysis because the considered game is neither convex nor strongly monotone or variationally stable. The setup and results extend the ones of (Golrezaei et al., 2020) which however considered linear demands and a uniform reference price.\n\nSTRENGTHS:\n- The paper is well written and the results are sound and original\n- The authors did a good job of connecting the considered games with the existing literature and explaining why existing results do not apply\n- The proven theorems are novel and the analysis seems somewhat original.\n\nWEAKNESSES:\n- The analysis is very specific to the considered duopoly games. Although they are widely studied in the Marketing and Management literature, they may not be of high interest to the machine learning community. That said, online learning and convergence to NE is a relevant topic within NeurIPS.\n- The authors repeatedly refer to no-regret (even in the title). However, the notion of regret is never defined and -- moreover -- bounds on the regret are not provided. I agree that last-iterate convergence to NE is a stronger guarantee, but it is not clear to me that this implies sublinear regret.\n\nQUESTIONS:\nHow realistic is it for the competing firms to compute exact gradients? Which set of information is required to do so at each round?\nAlso, as pointed out by the authors, one-point gradient estimates could be obtained. Does the current analysis allow draw conclusions on the resulting convergence results?\n\nLIMITATIONS:\nLimitations are discussed in combination with future directions.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author consider a multi-period pricing competition problem between two firms, dubbed H and L. Each of these firms sells 1 product. At each time step, the demands on each product is governed by an MNL model, which is paramaterized by not only the firms' offered prices in the current time period, but also the current reference prices of the firms. These reference prices are dependent on the past prices. In the game theoretic setting, each of the firm is to set prices under the uncertainty on the other firms strategy. The authors' main result is that, when each of the firms applies the standard online gradient descent algorithm, they both converge to a stationary Nash equilibrium, and the rate of convergence is also provided.\n\nSTRENGTHS:\nThe pricing problem combines both the game theoretic element in a duopoly pricing setting, as well as the reference price setting, which are both well known in the literature. The provided algorithm is sensible and reasonable, and these results are supported with numerical experiments. The analysis in the gradient descent algorithm seems interesting, since it seems to deviate from the traditional Online Gradient Descent works that require convexity.\n\nWEAKNESSES:\n- The model assumption on the demand model seems quite strong, since it requires the random noise to follow a particular probability distribution (Gumbel), in order for the result to hold. While there is another linearity assumption on the utility, I find it a milder assumption than the Gumbel assumption. While I understand that the Gumbel distribution, which gives rise to the MNL model, is a common assumption in assortment optimization, I find that it could be a limiting assumption in the pricing setting. Crucially, if the noise distribution is changed, it is not quite clear of the OPGA still converges to a stationary Nash equilibrium.\n\n- While Theorem 5.2 is a more refined form than Theorem 5.1 in that the former provides a finite time performance guarantee, I am not clear on what is the hidden parameter in the $\\Theta(\\cdot)$ notation for the learning rate $\\eta_t$. While the OPGA can be implemented with any sequence of learning rates, it is desirable to have a set of default learning rates that come with an explicit performance guarantee.\n\nQUESTIONS:\n(1) Can the authors provide a discussion on how easy or how difficult it is to generalize from the Gumbel distribution noise to a more general class of noise (for example, 1-subGassuian)?\n\n(2) As shown in Figure 1a, b, the choice of learning rate is rather important. Can the authors demonstrate a plot with the setting of learning rate $\\eta^t = \\Theta(1/t)$ in Theorem 5.2? I think it will be more informative than setting $\\eta^t = 1/\\sqrt{t}$ in Figure 1a. In addition, can the authors provide explicit expressions for the coefficients $d_r, d_p$ in Theorem 5.2, on how they depend on the MNL assumption and the model parameters?\n\nLIMITATIONS:\nThis is a theoretical work, there is minimal negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors' objective is to develop an algorithm to aid the firms in converging to a stationary Nash equilibrium (SNE). In pursuit of this, the authors have:\n\n* Proposed an online projected gradient ascent (OPGA) algorithm.\n* Proven the global convergence of the OPGA to SNE within the given problem setting.\n* Established the convergence rate of the proposed algorithm to SNE.\n\nSTRENGTHS:\nI commend the authors for their clear and engaging writing. The background, related literature, concepts, algorithms, and more, are all explained lucidly. The proposed algorithm, and the results derived from it, are noteworthy contributions to the field. The technical addition to extend the algorithm's convergence beyond linear demand and convex loss function is particularly valuable.\n\nWEAKNESSES:\nI noticed certain assumptions and explanations that might require further substantiation. Please refer to the \"questions section\" for a detailed account. I believe addressing these points could add considerable depth and credibility to the paper's findings and arguments.\n\nQUESTIONS:\nDespite the significant merits of the work, I do have a few queries. If the authors could address these, I would be more than pleased to highly rate the paper:\n\n* My first question might be rather straightforward. Could you please elaborate on why online mirror descent, proposed to solve a similar problem, does not apply in your problem setting? If online mirror descent does apply, what advantages does your OPGA algorithm offer?\n* As for the assumptions listed above, I'm somewhat puzzled:\n    - The last assumption posits that each firm can access its own demand from the last period. However, I would argue that this \"demand\" differs from the one defined by\n$$d_i(\\bm{p}^t, \\bm{r}^t) = \\frac{\\exp(u_i(p_i^t, r_i^t))}{1+\\exp(u_i(p_i^t, r_i^t)) + \\exp(u_{-i}(p_{-i}^t, r_{-i}^t))}.$$ \nwhich, to the best of my understanding, represents the market share of firm $i$ at period $t$. While the \"demand\" that firm $i$ can access using the proposed method is the quantity of product sold by firm $i$, such as 1000 electronic devices. Aren't these two different concepts? I may be overlooking something, but it would be helpful if you could clarify the potential connection. It would also be instructive if you could provide an example explaining the use of $d_i(\\bm{p}^t, \\bm{r}^t)$ in the numerical example.\n    - It is assumed that $b_i$ and $c_i$ are known to firm $i$. However, I believe these parameters can only be learned from data. Therefore, what is the role of the learning process in the convergence to SNE? Do we handle them as a two-stage problem, or can we combine the two learning processes?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the problem of dynamic pricing in a competitive environment, where there are two revenue-maximizing competing firms selling substitutable products to customers and each of them has no access to the information of its competitor. In addition, customer’s utility follows a linear function with current price and reference prices, while the demand function is captured by a MNL model. The authors propose a simple online projected gradient ascent algorithm to update the price based on the historical prices and observed demands. The main contribution of this paper is that the authors provide the convergence guarantee of this algorithm to a unique stable Nash equilibrium and shows the convergence rate when choosing step size optimally.\n\nSTRENGTHS:\n1. This paper analyzes a very interesting problem --- dynamic pricing in the presences of competition across firms and reference prices of customers and provides a very succinct formulation to abstract this problem.\n\n2. The paper is well-written and I really enjoy reading this paper even though I don't have much background on MNL model.\n\n3. The results are promising that the authors show that a simple online projected gradient ascent algorithm works well in practice and it can converge to the unique stable NE. \n\n4. The paper is technically solid and the proof is nontrivial to me, however, I have no expertise to determine the novelty of the proof shown in the paper compared with the related work [21].\n\nWEAKNESSES:\nIn general, I find the paper interesting, technically solid and shows a very promising result. Though I still have some questions needed to be clarified.\n\n1. I find the uniqueness of the stable NE in this paper very important and crucial to the convergence results. Can authors discuss whether this property can be generalized? In addition, with this property, can we replace online projected gradient ascent algorithm by other online optimization algorithm to achieve similar convergence rate?\n\n2. If I understand correctly, the authors need $d_i(p^t, r^t)$ can be directly observed, right? If so, please clarify it more explicitly in the algorithm.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}