SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yoav Kolumbus
Cornell University  
yoav.kolumbus@cornell.edu
Menahem Levy
The Hebrew University of Jerusalem menahem.levy@mail.huji.ac.il
Noam Nisan
The Hebrew University of Jerusalem noam@cs.huji.ac.il
We study Proportional Response Dynamics (PRD) in linear Fisher markets, where participants act asynchronously. We model this scenario as a sequential process in which at each step, an adversary selects a subset of the players to update their bids, subject to liveness constraints. We show that if every bidder individually applies the PRD update rule whenever they are included in the group of bidders selected by the adversary, then, in the generic case, the entire dynamic converges to a competitive equilibrium of the market. Our proof technique reveals additional properties of linear Fisher markets, such as the uniqueness of the market equilibrium for generic parameters and the convergence of associated no swap regret dynamics and best response dynamics under certain conditions.
A central notion in the study of markets is the equilibrium: a state of affairs where no single party wishes to unilaterally deviate from it. The main benefit of focusing on the notion of equilibria lies in what it ignores: how the market can reach an equilibrium (if at all). This latter question is obviously of much interest as well, especially when considering computational aspects, $^{1}$  and a significant amount of research has been dedicated to the study of "market dynamics" and their possible convergence to an equilibrium. Almost all works that study market dynamics consider synchronous dynamics.
Synchronous Dynamics: At each time step  $t$ , all participants simultaneously update their behavior based on the state at time  $t - 1$ .
Such synchronization is clearly difficult to achieve in real markets, and so one might naturally wonder to what extent full synchrony is needed or whether convergence of market dynamics occurs even asynchronously. There are various possible levels of generality of asynchrony to consider. The simplest model considers a sequential scenario in which, at every time step  $t$ , an adversary chooses a single participant, and only this participant updates their behavior based on the state at time  $t - 1$ . The adversary is constrained to adhere to some liveness condition, such as scheduling every participant infinitely often or at least once every  $T$  steps. In the most general model [59], the adversary may also introduce message delays, leading players to respond to dated information. In this paper, we focus on an intermediate level of permissible asynchrony where updates may occur in an arbitrary asynchronous manner, but message delays are always shorter than the granularity of activation. In our proofs, the adversary's goal is to select the schedule of strategy updates by the players in a way that prevents the dynamic from converging to a market equilibrium.
<sup>1</sup>As we know that finding an equilibrium may be computationally intractable in general.
Activation Asynchrony:2 Every time step  $t$ , an arbitrary subset of participants is chosen by the adversary and all of these participants update their behavior based on the state at time  $t - 1$ . The adversary must adhere to the liveness condition where for every participant some set that includes them must be chosen at least once every  $T$  consecutive steps.
The market dynamics that we study in this paper are linear Fisher markets [9] with proportional response dynamics (PRD), a model that has received much previous attention [5, 10, 19, 74] and for which synchronous convergence to equilibrium is known. While there are a few asynchronous convergence results known for other dynamics, specifically for tatonnement dynamics [18, 23], there are no such results known for proportional response dynamics, and achieving such results has been mentioned as an open problem in [19, 74].
Fisher Markets with Linear Utilities: There are  $n$  players and  $m$  goods. Each player  $i$  has a budget  $B_{i}$  and each good  $j$  has, w.l.o.g., a total quantity of one. Buyer  $i$ 's utility from getting an allocation  $x_{i} = (x_{i1},\dots,x_{im})$  is given by  $u_{i}(x_{i}) = \sum_{j}a_{ij}x_{ij}$ , where the parameters  $a_{ij}\geq 0$  are part of the definition of the market. A market equilibrium is an allocation  $X = (x_{ij})$  (where  $0\leq x_{ij}\leq 1$ ) and a pricing  $p = (p_j)$  with the following properties. (1) Market clearing: for every good  $j$  it holds that  $\sum_{i}x_{ij} = 1$ ; (2) budget feasibility: for every player  $i$  it holds that  $\sum_{j}x_{ij}p_{j}\leq B_{i}$ ; and (3) utility maximization: for every player  $i$  and every alternative allocation  $y = (y_{1},\dots,y_{m})$  with  $\sum_{j}y_{j}p_{j}\leq B_{i}$  we have that  $u_{i}(x_{i})\geq u_{i}(y)$ .
Proportional Response Dynamics: At each time step  $t$ , each player  $i$  will make a bid  $b_{ij}^{t} \geq 0$  for every good  $j$ , where  $\sum_{j} b_{ij}^{t} = B_{i}$ . In the first step, the bid is arbitrary. Once bids for time  $t$  are announced, we calculate  $p_{j}^{t} = \sum_{i} b_{ij}^{t}$  and allocate the goods proportionally:  $x_{ij}^{t} = b_{ij}^{t} / p_{j}^{t}$ , providing each player  $i$  with utility  $u_{i}^{t} = \sum_{j} a_{ij} x_{ij}^{t}$ . At this point, player  $i$  updates its bids for the next step by bidding on each good proportionally to the utility obtained from that good:  $b_{ij}^{t+1} = B_{i} \cdot a_{ij} x_{ij}^{t} / u_{i}^{t}$ .
From the perspective of the player, proportional response updates can be thought of as a simple parameter-free online learning heuristic, with some similarity to regret-matching [43] in its proportional updates, but one that considers the utilities directly, rather than the more sophisticated regret vector loss.
It is not difficult to see that a fixed point of this proportional response dynamic is indeed an equilibrium of the Fisher market. Significantly, it was shown in [5] that this dynamic does converge, in the synchronous model, to an equilibrium. As mentioned, the question of asynchronous convergence was left open. We provide the first analysis of proportional response dynamics in the asynchronous setting, and provide a positive answer to this open question in our "intermediate" level of asynchrony.
Theorem 1. For linear Fisher markets, proportional response dynamics with adversarial activation asynchrony, where each player is activated at least once every  $T$  steps, approach the set of market equilibrium bid profiles. The prices in the dynamics converge to the unique equilibrium prices.
The dynamics approaching the set of equilibria means that the distance between the bids at time  $t$  and the set of market equilibrium bid profiles converges to zero as  $t \to \infty$ . Additionally, in Section 7, we show that for generic parameters (i.e., except for measure zero of possible  $(a_{ij})$ 's), the market equilibrium bid profile is unique, and thus Theorem 1 implies convergence of the bids to a point in the strategy space. We do not know whether the genericity condition is required for asynchronous convergence of the bids to a point, and we leave this as a minor open problem. We did not analyze the rate of convergence to equilibrium; we leave such analysis as a second open problem. Our main open problem, however, is the generalization to full asynchrony with message delays.
Open Problem: Does such convergence occur also in the full asynchronous model where the adversary may introduce arbitrary message delays?
Our techniques rely on considering an associated game obtained by using "modified" utility functions for each of the players:  $\tilde{u}_i(b) = \sum_j b_{ij} \ln(a_{ij}) + \sum_j p_j (1 - \ln(p_j))$ . We show that a competitive market equilibrium (with the original utility functions) corresponds to a Nash equilibrium in the associated game. These modified utility functions are an adaptation to an individual utility of a
2In [59] this was termed "simultaneous."
<sup>3</sup>It is worthwhile to emphasize, though, that a competitive market equilibrium is not a Nash equilibrium in the original market, since the players are price takers rather than fully rational. See Section 3.
function  $\Phi (b) = \sum_{ij}b_{ij}\ln (a_{ij}) + \sum_{j}p_{j}(1 - \ln (p_{j}))$  that was proposed in [65] as an objective for a convex program for equilibrium computation. This function was first linked with proportional response dynamics in [5] where it was proven that synchronous proportional response dynamics act as mirror descent on this function. We show that  $\Phi$  serves as a potential in our associated game.
Theorem 2. The following three sets of bid profiles are identical: (1) the set of pure strategy Nash equilibria of the associated game, (2) the set of market equilibria of the Fisher market, and (3) the maximizing set of the potential function  $\Phi$ .
The technical core of our proof is to show that not only does a synchronized proportional response step by all the players increase the potential function  $\Phi$  but, in fact, every proportional response step by any subset of the players increases this potential function.
The point of view of market equilibria as Nash equilibria of the associated game offers several other advantages, e.g., suggesting several other dynamics that are related to proportional response that can be of interest. For example, we show that letting players best-respond in the associated game corresponds to the limit of a sequence of proportional response steps by a single player, but can be implemented as a single step of such a best-response, which can be computed efficiently by the players and may converge faster to the market equilibrium. Another possibility is using some (internal) regret-minimization dynamics (for the associated game), which would also converge to equilibrium in the generic case since, applying [56], it is the unique correlated Equilibrium as well.
The structure of the rest of the paper is as follows. In Section 2, we provide further formal details and notations that will be useful for our analysis. In Section 3, we present the associated game and its relation to the competitive equilibria of the market. In Section 4, we study best response dynamics in the associated game and their relation to PRD. In Section 5, we present a key lemma regarding the potential function of the associated game under bid updates by subsets of the players. Then, in Section 6 we complete our proof of convergence for asynchronous PRD. In Section 7, we show the uniqueness of the market equilibrium for generic markets. In Section 8, we provide simulation results that compare the convergence of proportional response dynamics with best response dynamics in the associated game in terms of the actual economic parameters in the market, namely, social welfare and the convergence of the bid profiles. Finally, in Section 9, we conclude and discuss the limitations of our technique and open questions. All proofs in this paper are deferred to the appendix.
1.1 Further Related Work
Proportional response dynamics (PRD) were originally studied in the context of bandwidth allocation in file-sharing systems, where it was proven to converge to equilibrium, albeit only for a restrictive setting [71]. Since then, PRD has been studied in a variety of other contexts, including Fisher markets, linear exchange economies, and production markets. See [10] for further references. In Fisher markets, synchronous PRD has been shown to converge to market equilibrium for Constant Elasticity of Substitution (CES) utilities in the substitutes regime [74]. For the linear Fisher market setting, synchronous PRD was explained in [5] as mirror descent on a convex program, previously discovered while developing an algorithm to compute the market equilibrium [65], and later proven to be equivalent to the famous Eisenberg-Gale program [22]. By advancing the approach of citebirnbaum2011distributed, synchronous PRD with mild modifications was proven to converge to a market equilibrium for CES utilities in the complements regime as well [19]. In linear exchange economies, synchronous PRD has been shown to converge to equilibrium in the space of utilities and allocations while prices do not converge and may cycle, whereas for a damped version of PRD, also the prices converge [11]. In production markets, synchronous PRD has been shown to increase both growth and inequalities in the market [13]. PRD has also been proven to converge with quasi-linear utilities [38], and to remain close to market equilibrium for markets with parameters that vary over time [20]. All the above works consider simultaneous updates by all the players, and the question of analyzing asynchronous dynamics and whether they converge was raised by several authors as an open problem [19, 74].
Asynchronous dynamics in markets have been the subject of study in several recent works. However, these works examine different models and dynamics than ours, and to our knowledge, our work presents the first analysis of asynchronous proportional response bidding dynamics. In [23], it is
4 Notice that  $\Phi$  is not the sum of the  $\tilde{u}_i$ 's, as the second term appears only once.
shown that tatonnement dynamics under the activation asynchrony model converge to equilibrium, with results for settings both with and without carryover effects between time units. A later work, [18], showed that tatonnement price dynamics converge to a market equilibrium under a model of sequential activation where in every step a single agent is activated, and where additionally, the information available to the activated seller about the current demand may be inaccurate within some interval that depends on the range of past demands. A different approach taken in [29] assumes that each seller has a set of rules affected by the other players' actions and governing its price updates; it is shown that the dynamics in which sellers update the prices based on such rules converge to a unique equilibrium of prices in the activation asynchrony model.
Online learning with delayed feedback has been studied for a single agent in [61], which established regret bounds that are sub-linear in the total delay. A different approach, presented in [52], extends online gradient descent by estimating gradients when the reward of an action is not yet available due to delays. When considering multiple agents, [75] extended the results of [61]. They demonstrated that in continuous games with particular stability property, if agents experience equal delays in each round, online mirror descent converges to the set of Nash equilibria and a modification of the algorithm converges even when delays are not equal.
Classic results regarding the computation of competitive equilibria in markets mostly consider centralized computation and vary from combinatorial approaches using flow networks [26-28, 45], interior point [72], and ellipsoid [21, 44] methods, and many more [3, 25, 26, 30, 39, 40, 63]. Eisenberg and Gale devised a convex program which captures competitive equilibria of the Fisher model as its solution [31]. Notable also is the tatonnement model of price convergence in markets dated back to Walras [70] and studied extensively from Arrow [1] and in later works.
More broadly, in the game theoretic literature, our study is related to a long line of work on learning in games, starting from seminal works in the 1950s [6, 14, 42, 62], and continuing to be an active field of theoretical research [24, 35, 49, 60, 66], also covering a wide range of classic economic settings including competition in markets [7, 54], bilateral trade [16, 32], and auctions [2, 34, 48], as well as applications such as blockchain fee markets [50, 51, 57] and strategic queuing systems [36, 37]. For a broad introduction to the field of learning in games, see [17, 43]. The vast majority of this literature studies repeated games under the synchronous dynamics model. Notable examples of analyses of games with asynchronous dynamics are [46], which study best response dynamics with sequential activation, and [59], which explore best response dynamics in a full asynchrony setting which includes also information delays, and show that in a class of games called max-solvable, convergence of best response dynamics is guaranteed. Our analysis of best response dynamics in Section 4 takes a different route, and does not conclude whether the associated game that we study is max-solvable or not; such an analysis seems to require new ideas.
Our work is also related to a large literature on asynchronous distributed algorithms. We refer to a survey on this literature [41]. The liveness constraint that we consider in the dynamics<sup>5</sup> is related to those, e.g., in [4, 67]. Recent works that are conceptually more closely related are [15, 69, 73], which propose asynchronous distributed algorithms for computing Nash equilibria in network games. Notably, [15] propose an algorithm that converges to an equilibrium in a large class of games in asynchronous settings with information delays. Their approach, however, does not capture proportional response dynamics and does not apply to our case of linear Fisher markets.
2 Model and Preliminaries
The Fisher market: We consider the classic Fisher model of a networked market in which there is a set of buyers  $\mathcal{B}$  and a set of divisible goods  $\mathcal{G}$ . We denote the number of buyers and number of goods as  $n = |\mathcal{B}|$ ,  $m = |\mathcal{G}|$ , respectively, and index buyers with  $i$  and goods with  $j$ . Buyers are assigned budgets  $B_{i} \in \mathbb{R}^{+}$  and have some value<sup>6</sup>  $a_{ij} \geq 0$  for each good  $j$ . Buyers' valuations are normalized such that  $\sum_{j} a_{ij} = 1$ . It is convenient to write the budgets as a vector  $B = (B_{i})$  and the valuations as a matrix  $A_{n \times m} = (a_{ij})$ , such that  $A, B$  are the parameters defining the market. We denote the allocation of goods to buyers as a matrix  $X = (x_{ij})$  where  $x_{ij} \geq 0$  is the (fractional) amount of good
5 Intuitively, if one allows some of the parameters in the dynamic not to update, these parameters become irrelevant, as they will remain frozen, and thus one cannot hope to see any convergence of the entire system.
For the ease of exposition, our proofs use w.l.o.g.  $a_{ij} > 0$ . This is since in all cases where  $a_{ij} = 0$  might have any implication on the proof, such as  $\ln(a_{ij})$ , these expressions are multiplied by zero in our dynamics.
$j$  that buyer  $i$  obtained. We assume w.l.o.g. (by proper normalization) that there is a unit quantity of each good. The price of good  $j$  (which depends on the players' actions in the market, as explained below) is denoted by  $p_j \geq 0$  and prices are listed as a vector  $p = (p_j)$ . Buyers have a linear utility function  $u_i(x_i) = \sum_j a_{ij} x_{ij}$  with the budget constraint  $\sum_j x_{ij} p_j \leq B_i$ . We assume w.l.o.g. that the economy is normalized, i.e.,  $\sum_i B_i = \sum_j p_j = 1$ .
Market equilibrium: The competitive equilibrium (or "market equilibrium") is defined in terms of allocations and prices as follows.
Definition 1. (Market Equilibrium): A pair of allocations and prices  $(X^{*},p^{*})$  is said to be market equilibrium if the following properties hold:
1. Market clearing:  $\forall j, \sum_{i} x_{ij}^{*} = 1,$
2. Budget feasibility:  $\forall i, \sum_{j} x_{ij}^{*} p_{j}^{*} \leq B_{i},$
3. Utility maximization:  $\forall i, x_{i}^{*} \in \arg \max_{x_{i}} u_{i}(x_{i})$ .
In words, under equilibrium prices all the goods are allocated, all budgets are used, and no player has an incentive to change their bids given that the prices remain fixed. Notice that this notion of equilibrium is different from a Nash equilibrium of the game where the buyers select their bids strategically, since in the former case, players do not consider the direct effect of possible deviation in their bids on the prices. We discuss this further in Section 3. For linear Fisher markets, it is well established that competitive equilibrium utilities  $u^{*}$  and prices  $p^{*}$  are unique, equilibrium allocations are known to form a convex set, and the following conditions are satisfied.
$$
\forall i, j \quad \frac {a _ {i j}}{p _ {j} ^ {*}} \leq \frac {u _ {i} ^ {*}}{B _ {i}} \quad \text {a n d} \quad x _ {i j} > 0 \Longrightarrow \frac {a _ {i j}}{p _ {j} ^ {*}} = \frac {u _ {i} ^ {*}}{B _ {i}}.
$$
This is a detailed characterization of the equilibrium allocation: every buyer gets a bundle of goods in which all goods maximize the value per unit of money. The quantity  $a_{ij} / p_j^*$  is informally known as "bang-per-buck" (ch. 5 & 6 in [58]), the marginal profit from adding a small investment in good  $j$ .
Market equilibrium bids are also known to maximize the Nash social welfare function (see [31])  $\mathrm{NSW}(X) = \prod_{i\in \mathcal{B}}u_i(x_i)^{B_i}$  and to be Pareto efficient, i.e., no buyer can improve their utility without making anyone else worse off (as stated in the first welfare theorem).
The trading post mechanism and the market game (Shapley-Shubik): First described in [64] and studied under different names [33, 47], the trading post mechanism is an allocation and pricing mechanism which attempts to capture how a price is modified by demand. Buyers place bids on goods, where buyer  $i$  places bid  $b_{ij}$  on good  $j$ . Then, the mechanism computes the good's price as the total amount spent on that good and allocates the good proportionally to the bids, i.e., for bids  $b$ :
$$
p _ {j} = \sum_ {i = 1} ^ {n} b _ {i j} \qquad x _ {i j} = \left\{ \begin{array}{l l} \frac {b _ {i j}}{p _ {j}} & b _ {i j} > 0 \\ 0 & \text {o t h e r w i s e .} \end{array} \right.
$$
Note that the trading post mechanism guarantees market clearing for every bid profile  $b$  in which all goods have at least one buyer who is interested in buying. The feasible bid set of a buyer under the budget constraint is  $S_{i} = \{b_{i}\in \mathbb{R}^{m}|\forall j b_{ij}\geq 0\sum_{j}b_{ij} = B_{i}\}$ , i.e., a scaled simplex. Denote  $S = \prod_{i\in \mathcal{B}}S_{i}$  and  $S_{-i} = \prod_{k\in \mathcal{B}\setminus \{i\}}S_k$ . Considering the buyers as strategic, one can define the market game as  $G = \{\mathcal{B},(S_i)_{i\in \mathcal{B}},(u_i)_{i\in \mathcal{B}}\}$  where the utility functions can be written explicitly as  $u_{i}(b) = u_{i}(x_{i}(b)) = \sum_{j = 1}^{m}\frac{a_{ij}b_{ij}}{p_j}$ . We sometimes use the notation  $u_{i}(b_{i},b_{-i})$ , where  $b_{i}$  is the bid vector of player  $i$  and  $b_{-i}$  denotes the bids of the other players.
Potential function and Nash equilibrium: For completeness, we add the following definitions. Potential function: A function  $\Phi$  is an exact potential function[53] if  $\forall i\in \mathcal{B},\forall b_{-i}\in S_{-i}$  and  $\forall b_i,b_i'\in S_i$  we have that  $\Phi (b_i',b_{-i}) - \Phi (b_i,b_{-i}) = u_i(b_i',b_{-i}) - u_i(b_i,b_{-i})$ , with  $u_{i}$  being  $i$ 's utility function in the game. Best response:  $b_{i}^{*}$  is a best response to  $b_{-i}$  if  $\forall b_i\in S_i\quad u_i(b_i^*,b_{-i})\geq u_i(b_i,b_{-i})$ . That is, no other response of  $i$  can yield a higher utility. Nash equilibrium:  $b^{*}$  is Nash equilibrium if  $\forall i b_{i}^{*}$  is a best response to  $b_{-i}^{*}$  (no player is incentivized to change their strategy).
Proportional response dynamics: As explained in the introduction, the proportional response dynamic is specified by an initial bid profile  $b^{0}$ , with  $b_{ij}^{0} > 0$  whenever  $a_{ij} > 0$ , and the following
update rule for every player that is activated by the adversary:  $b_{ij}^{t + 1} = \frac{a_{ij}x_{ij}^t}{u_i(x_i^t)} B_i$ . See Section 5 for further details on activation of subsets of the players.
3 The Associated Game
As mentioned above, the Fisher market can be naturally thought of as a game in which every one of the  $n$  players aims to optimize their individual utility  $u_{i}(b_{i}, b_{-i})$  (see Section 2 for the formal definition). However, it is known that the set of Nash equilibria of this game does not coincide with the set of market equilibria [33, 64], and so a solution to this game (if indeed the players reach a Nash equilibrium) is economically inefficient [12].