SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A natural question that arises is whether there is some other objective for an individual player that when maximized by all the players, yields the market equilibrium. We answer positively to this question and show that there is a family of utility functions such that in the "associated games" with these utilities for the players, the set of Nash equilibria is identical to the set of market equilibria of the original game (for further details, see also Appendix A).
However, the fact that a Nash equilibrium of an associated game is a market equilibrium still does not guarantee that the players' dynamics will indeed reach this equilibrium. A key element in our proof technique is that we identify, among this family of associated games, a single game, defined by the "associated utility"  $\tilde{u}_i(b) = \sum_j b_{ij} \ln(a_{ij}) + \sum_j p_j (1 - \ln(p_j))$ , which admits an exact potential. We then use a relation which we show between this game and the proportional response update rule to prove the convergence of our dynamics (Theorem 1).
Definition 2. (The Associated Game): Let  $G$  be a market game. Define the associated utility of a player  $i$  as  $\tilde{u}_i(b) = \sum_j b_{ij} \ln(a_{ij}) + \sum_j p_j (1 - \ln(p_j))$ . The associated game  $\tilde{G}$  is the game with the associated utilities for the players and the same parameters as in  $G$ .
Theorem 3. For every Fisher market, the associated game  $\tilde{G}$  admits an exact potential function that is given by $^7$ $\Phi (b) = \sum_{ij}b_{ij}\ln (a_{ij}) + \sum_jp_j(1 - \ln (p_j))$
$\tilde{G}$  is constructed such that the function  $\Phi$  is its potential. Note that although having similar structure,  $\tilde{u}_i$  and  $\Phi$  differ via summation on  $i$  only in the first term ( $\Phi$  is not the sum of the players' utilities).
Once having the potential function defined, the proof is straightforward: the derivatives of the utilities  $\tilde{u}_i$  and the potential  $\Phi$  with respect to  $b_i$  are equal for all  $i$ . Theorem 2, formally restated below, connects between the associated game, the market equilibria and the potential.
Theorem. (Restatement of Theorem 2). The following three sets of bid profiles are equal. (1) The set of pure-strategy Nash equilibria of the associated game:  $NE(\tilde{G}) = \{b^{*} \mid \forall b \in S \quad \tilde{u}_{i}(b^{*}) \geq \tilde{u}_{i}(b)\}$ ; (2) the set of market equilibrium bid profiles of the Fisher market:  $\{b^{*} \mid (x(b^{*}), p(b^{*})) \text{ satisfy Def. } 1\}$ ; and (3) the maximizing set of the potential from Theorem 3:  $\arg \max_{b \in S} \Phi(b)$ .
The proof uses a different associated game  $G'$  that has simpler structure than  $\tilde{G}$ , but does not have an exact potential, and shows that: (i) Nash equilibria of  $G'$  are the market equilibria; (ii) all the best responses of players  $i$  to bid profiles  $b_{-i}$  in  $G'$  are the same as those in  $\tilde{G}$ ; and (iii) every equilibrium of  $\tilde{G}$  maximizes the potential  $\Phi$  (immediate by the definition of potential).
4 Best Response Dynamics
In this section we explore another property of the associated game: we show that if instead of using the proportional response update rule, each player myopically plays their best response to the last bid profile with respect to their associated utility, then the entire asynchronous sequence of bids converges to a market equilibrium, as stated in the following theorem. We then show that there is a close relation between best response and proportional response dynamics.
<sup>7</sup>Since we discuss the players' associated utilities, we consider maximization of this potential. Of course, if the reader feels more comfortable with minimizing the potential, one can think of the negative function.
Theorem 4. For generic linear Fisher markets in a sequential asynchrony model where in every step a single player is activated, best response dynamics converge to the Market Equilibrium. For non-generic markets the prices are guaranteed to converge to the equilibrium prices.
The idea of the proof is to show that the best-response functions are single valued  $(\forall i, b_{-i} : \tilde{u}_i(\cdot, b_{-i})$  has a unique maximizer) and continuous (using the structure of best-response bids). Together with the existence of the potential function  $\Phi$  it holds that the analysis of [46] applies for these dynamics with the associated utilities<sup>8</sup> and thus convergence is guaranteed.
One of the appealing points about proportional response dynamics is their simplicity — in each update, a player observes the obtained utilities and can easily compute the next set of bids. We show that also the best response of a player can be computed efficiently by reducing the calculation to a search over a small part of the subsets of all goods which can be solved by a simple iterative process.
Proposition 1. For every player  $i$  and any fixed bid profile  $b_{-i}$  for the other players, the best response of  $i$  is unique and can be computed in polynomial time.
Roughly, best responses are characterized uniquely by a one-dimensional variable  $c^*$ . For every subset of goods  $s$  we define a variable  $c_s$  and prove that  $c^*$  is the maximum amongst all  $c_s$ . So finding  $c^*$  is equivalent to searching a specific subset with maximal  $c_s$ . The optimal subset of goods admits a certain property that allows to narrow down the search domain from all subsets to only  $m$  subsets.
The relation between the best response and proportional response updates can intuitively be thought of as follows. While in PRD players split their budget between all the goods according the utility that each good yields, and so gradually shift more budget to the more profitable subset of goods, best response bids of player  $i$  with respect to  $\tilde{u}_i$  can be understood as spending the entire budget on a subset of goods which, after bidding so (considering the effect of bids on prices), will jointly have the maximum bang-per-buck (in our notation  $a_{ij} / p_j$ ) amongst all subsets of goods, given the bids  $b_{-i}^t$  of the other players. Those bids can be regarded as "water-filling" bids as they level the bang-per-buck amongst all goods purchased by player  $i$  (for a further discussion see the appendix).
It turns out that there is a clear formal connection between the best response of a player in the associated game and the proportional response update rule in the true game: the best response bids are the limit point of an infinite sequence of proportional response updates by the same player when the bids of the others are held fixed, as expressed in the following proposition.
Proposition 2. Fix any player  $i$  and fix any bid profile  $b_{-i}$  for the other players. Let  $b_i^* = \arg\max_{b_i \in S_i} \tilde{u}_i(b_i, b_{-i})$  and let  $(b_i^t)_{t=1}^\infty$  be a sequence of consecutive proportional response steps applied by player  $i$ , where  $b_{-i}$  is held fixed at all times  $t$ . Then  $\lim_{t \to \infty} b_i^t = b_i^*$ .
5 Simultaneous Play by Subsets of Agents
In this section, we shift our focus back to proportional response dynamics under the activation asynchrony model in which the adversary can choose in every step any subset of players to update their bids. Towards proving that proportional response dynamics converges to a market equilibrium in this setting, we utilize the associated game and potential function presented in Section 3 to show that any activated subset of players performing a PRD step will increase the potential. Formally, let  $v \subseteq \mathcal{B}$  be a subset of players activated by the adversary and let  $f_v(b)$  be a function that applies proportional response to members of  $v$  and acts as the identity function for all the other players. The update for time  $t + 1$  when the adversary activates a subset of the players  $v^t \subseteq \mathcal{B}$  is therefore:
$$
b _ {i j} ^ {t + 1} = (f _ {v ^ {t}} (b ^ {t})) _ {i j} = \left\{ \begin{array}{l l} \frac {a _ {i j} x _ {i j} ^ {t}}{u _ {i} ^ {t}} B _ {i} & \quad \text {i f i \in v ^ {t}} \\ b _ {i j} ^ {t} & \quad \text {o t h e r w i s e}. \end{array} \right.
$$
Lemma 1. For all  $v \subseteq \mathcal{B}$  and for all  $b \in S$  it holds that  $\Phi(f_v(b)) > \Phi(b)$ , unless  $f_v(b) = b$ .
The proof shows that for any subset of players  $v^t$ , a PRD step  $b^{t + 1}$  is the solution to some maximization problem of a function  $g^{t}(b)$  different from  $\Phi$ , such that  $\Phi (b^{t + 1}) > g^{t}(b^{t + 1})\geq g^{t}(b^{t}) = \Phi (b^{t})$ .
Note that while the best-reply function with respect to the standard utility function is formally undefined at zero, the associated utility and its best-reply function are well defined and continuous at zero.
Notable to mention is the sequential case where all subsets are singletons, i.e., for all  $t$ ,  $v^t = \{i^t\}$  for some  $i^t \in \mathcal{B}$ . In that case, the above result yields that the best-response bids can be expressed as the solution to an optimization problem over the bids  $b$  on a function that is monotone in the KL divergence between the prices induced by  $b$  and the current prices, whereas PRD is the solution to an optimization problem on a similar function, but one that depends on the KL divergence between the bids  $b$  and the current bids. Thus, sequential PRD can be regarded as a relaxation of best response; on the one hand, it is somewhat simpler to compute a step, and on the other hand, it takes more steps to reach the best response (see Proposition 2 and the simulations in Section 8).
6 Convergence of Asynchronous Proportional Response Dynamics
With the results from the previous sections under our belts (namely, the associated game, Theorems 2, 3 about its potential and equilibria, and Lemma 1 about updates by several players simultaneously), we are now ready to complete the proof of Theorem 1 on the convergence of asynchronous proportional response dynamics. We explain here the idea of the proof. The full proof is in the appendix.
Proof idea of Theorem 1: Our starting point is that we now know that Proportional Response Dynamics (PRD) steps by subsets of players increase the potential. Therefore, the bids should somehow converge to reach the maximum potential, which is obtained only at the set of market equilibria. Technically, since the sequence of bids  $b^{t}$  is bounded, it must have condensation points. The proof then proceeds by way of contradiction. If the sequence does not converge to the set of equilibrium bid profiles,  $ME = \{b^{*} \mid b^{*}$  is a market equilibrium bid profile\}, then there is some subsequence that converges to a bid profile  $b^{**}$  outside of this set, which by Theorem 2, must achieve a lower potential than any  $b^{*} \in ME$  (since it is not a market equilibrium, and recall that only market equilibria maximize the potential function).
From this point, the main idea is to show that if the dynamic preserves a "livness" property where the maximum time interval between consecutive updates of a player is bounded by some constant  $T$ , then the dynamic must reach a point where the bids are sufficiently close to  $b^{**}$  such that there must be some future update by some subset of the players under which the potential increases to more than  $\Phi(b^{**})$ , contradicting the existence of condensation points other than market equilibria (note that the sequence of potential values  $\Phi(b^t)$  is increasing in  $t$ ). To show this, the proof requires several additional arguments on the continuity of compositions of PRD update functions that arise under adversarial scheduling, and the impact of such compositions on the potential function. The full proof is in the appendix.
7 Generic Markets
Here we show that in the generic case, linear Fisher markets have a unique equilibrium bid profile. While it is well known that in linear Fisher markets equilibrium prices and utilities are unique, and the equilibrium bids and allocations form convex sets (see section 2), we show that multiplicity of equilibrium bid profiles can result only from a special degeneracy in the market parameters that has measure zero in the parameter space. In other words, if the market parameters are not carefully tailored to satisfy a particular equality (formally described below), or, equivalently, if the parameters are slightly perturbed, the market will have a unique equilibrium. Similar property was known for linear exchange markets [8] and we present a simple and concise proof for the Fisher model.
Definition 3. A Fisher market is called generic if the non-zero valuations of the buyers  $(a_{ij})$  do not admit any multiplicative equality. That is, for any distinct and non empty  $K$ ,  $K' \subseteq \mathcal{B} \times \mathcal{G}$  it holds that  $\prod_{(i,j) \in K} a_{ij} \neq \prod_{(i',j') \in K'} a_{i'j'}$ .
Theorem 5. Every generic linear fisher market has a unique market equilibrium bid profile  $b^{*}$ .
Before discussing the proof of Theorem 5, we have the following corollaries.
Corollary 1. For generic linear Fisher markets, proportional response dynamics with adversarial activation asynchrony, where each player is activated at least once every  $T$  steps, converge to the unique market equilibrium.
The main theorem, Theorem 1, suggests that proportional response dynamics with adversarial activation asynchrony converges to the set of market equilibria. Since the previous theorem guarantees
that a generic market has a unique market equilibrium bid profile, it is clear the dynamics converge to that bid profile.
Corollary 2. In generic linear Fisher markets, no-swap regret dynamics in the associated game converge to the market equilibrium.
This follows from [56], who showed that in games with convex strategy sets and a continuously differentiable potential function  $\Phi$  (as in our case), the set of correlated equilibria consists of mixtures of elements in  $\arg \max_{b}\Phi$ . Theorem 2 yields that  $\arg \max_{b}\Phi = b^{*}$ , and so there is a unique correlated equilibrium, which is the market equilibrium, and we have that no-swap regret guarantees convergence to it.
To prove Theorem 5, we use a representation of the bids in the market as a bipartite graph of players and goods  $\Gamma(b) = \{V, E\}$  with  $V = \mathcal{B} \cup \mathcal{G}$  and  $E = \{(i, j) | b_{ij} > 0\}$ . The proof shows that if a market has more than one equilibrium bid profile, then there has to be an equilibrium  $b$  with  $\Gamma(b)$  containing a cycle, whereas the following lemma forbids this for generic markets.
Lemma 2. If  $b^{*}$  are equilibrium bids in a generic linear Fisher market, then  $\Gamma(b^{*})$  has no cycles.
A key observation for proving this lemma is that at a market equilibrium, for a particular buyer  $i$ , the quantity  $a_{ij} / p_j^*$  is constant amongst goods purchased, and so it is possible to trace a cycle and have all the  $p_j^*$  cancel out and obtain an equation contradicting the genericity condition.
An observation that arises from Lemma 2 is that when the number of buyers in the market is of the same order of magnitude as the number of goods or larger, then, in equilibrium, most buyers will only buy a small number of goods. Since there are no cycles in  $\Gamma(b^{*})$  and there are  $n + m$  vertices, there are at most  $n + m - 1$  edges. Thus, with  $n$  buyers, the average degree of a buyer is  $1 + \frac{m - 1}{n}$ .
8 Simulations
Next, we look at simulations of the dynamics that we study and compare the convergence of proportional response dynamics to best response dynamics in the associated game, as discussed in Section 4. The metrics we focus on here for every dynamic are the Nash social welfare [55], which, as mentioned in Section 2, is maximized at the market equilibrium, and the Euclidean distance between the bids at time  $t$  and the equilibrium bids. Additionally, we look at the progression over time of the value of the potential  $\Phi(b^t)$  (for the definition, see Section 3).
Figure 1 presents simulations of an ensemble of markets, each with ten buyers and ten goods. The parameters in each market (defined in the matrices  $A$  and  $B$ ) are uniformly sampled, ensuring that the genericity condition (defined in Section 2) holds with probability one. These parameters are also normalized, as explained in Section 2. For each market, the parameters remain fixed throughout the dynamics. The initial condition in all simulation runs is the uniform distribution of bids over items, and the schedule is sequential, such that a single player updates their bids in each time step.
Figure 1a (main plot) show our metrics for PRD averaged over a sample of 300 such simulations. The insets show the plots of a random sample of 50 individual simulations (without averaging) over a longer time period. Figure 1b show similar plots for best response dynamics.
As could be expected based on our analysis in Section 4, best response dynamics converge faster than PRD, as can be seen in the different time scales on the horizontal axes. A closer look at the individual bid dynamics depicted in the insets reveals a qualitative difference between the two types of dynamics: in PRD, the bids in each dynamic smoothly approach the equilibrium profile, whereas best response bid dynamics are more irregular. Additionally, the collection of curves for the individual simulations shows that under uniformly distributed market parameters, both dynamics exhibit variance in convergence times, with a skewed distribution. In most markets, the dynamics converge quickly, but there is a distribution tail of slower-converging dynamics.
9 Conclusion
We have shown that proportional response bid dynamics converge to a market equilibrium in a setting where the schedule of bid updates can be chosen adversarially, allowing for sequential or
(a) Proportional Response
(b) Best Response
Figure 1: Proportional response and best response dynamics. The main figures show the progression of the average metrics over time and the insets show a collection of individual dynamics over a longer time period.
simultaneous updates for any subset of players. We proposed a novel approach to address this problem by identifying a family of associated games related to proportional response dynamics, showing their relation to the competitive equilibria of the market, and leveraging these relations to prove convergence of the dynamics. En route, we showed that other types of dynamics, such as myopic best response and no swap regret, also converge in the associated game. Additionally, we note that our result on the uniqueness of market equilibria in the generic case (e.g., if the market parameters have some element of randomness) may also be of interest for future research in the Fisher market setting.
One main open question that we did not analyze is whether proportional response dynamics converge under the full asynchrony model, which includes information delays. The analysis of this model raises several complications, as it creates further coupling between past and current bid profiles. We conjecture that if information delays are bounded, then convergence also occurs in this model. However, it is not clear whether our approach could be extended to argue that proportional response updates by subsets of players with respect to delayed information increase the potential in our associated game, or whether proving convergence in this setting will require new methods. One limitation of our analysis is that we provide a guarantee that under any bid update by any subset of players chosen by an adversary, the potential function of the associated game increases, but our technique does not specify by how much the potential increases in every step, and therefore, we do not provide speed of convergence results. Such analysis seems to require new techniques, and we see this as an interesting open problem for future work.
This work was supported by the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation Programme (grant agreement no. 740282) and by a grant from the Israeli Science Foundation (ISF number 505/23). During part of this project, Yoav Kolumbus has been affiliated with the Hebrew University of Jerusalem.
[57] Noam Nisan. Serial monopoly on blockchains draft—comments welcome. 2023.
[58] Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V Vazirani. Algorithmic game theory. Cambridge university press, 2007.
[59] Noam Nisan, Michael Schapira, and Aviv Zohar. Asynchronous best-reply dynamics. In Internet and Network Economics: 4th International Workshop, WINE 2008, Shanghai, China, December 17-20, 2008. Proceedings 4, pages 531-538. Springer, 2008.
[60] Christos Papadimitriou and Georgios Piliouras. From nash equilibria to chain recurrent sets: An algorithmic solution concept for game theory. Entropy, 20(10):782, 2018.
[61] Kent Quanrud and Daniel Khashabi. Online learning with adversarial delays. Advances in neural information processing systems, 28, 2015.
[62] Julia Robinson. An iterative method of solving a game. Annals of mathematics, pages 296-301, 1951.
[63] Herbert E Scarf. The computation of equilibrium prices: an exposition. Handbook of mathematical economics, 2:1007-1061, 1982.
[64] Lloyd Shapley and Martin Shubik. Trade using one commodity as a means of payment. Journal of political economy, 85(5):937-968, 1977.
[65] Vadim I Shmyrev. An algorithm for finding equilibrium in the linear exchange model with fixed budgets. Journal of Applied and Industrial Mathematics, 3(4):505, 2009.
[66] Vasilis Syrgkanis, Alekh Agarwal, Haipeng Luo, and Robert E Schapire. Fast convergence of regularized learning in games. Advances in Neural Information Processing Systems, 28:2989-2997, 2015.
[67] John Tsitsiklis, Dimitri Bertsekas, and Michael Athans. Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE transactions on automatic control, 31(9):803-812, 1986.
[68] Mark Voorneveld. Best-response potential games. Economics letters, 66(3):289-295, 2000.
[69] Mohamed Wahbi and Kenneth N Brown. A distributed asynchronous solver for nash equilibria in hypergraphical games. In Frontiers in Artificial Intelligence and Applications, pages 1291-1299, 2016.
[70] Léon Walras. Éléments d'économie politique pure: ou, Théorie de la richesse sociale. F. Rouge, 1900.
[71] Fang Wu and Li Zhang. Proportional response dynamics leads to market equilibrium. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing, pages 354-363, 2007.
[72] Yinyu Ye. A path to the arrow-debreu competitive market equilibrium. Mathematical Programming, 111(1-2):315-348, 2008.
[73] Peng Yi and Lacra Pavel. Asynchronous distributed algorithms for seeking generalized nash equilibria under full and partial-decision information. IEEE transactions on cybernetics, 50(6):2514-2526, 2019.
[74] Li Zhang. Proportional response dynamics in the fisher market. Theoretical Computer Science, 412(24):2691-2698, 2011.
[75] Zhengyuan Zhou, Panayotis Mertikopoulos, Nicholas Bambos, Peter W Glynn, and Claire Tomlin. Countering feedback delays in multi-agent learning. Advances in Neural Information Processing Systems, 30, 2017.
In the following sections we provide the proofs for the results presented in the main text as well as further technical details and discussion.
In the following, we use  $\nabla_{b_i}f$  to denote the gradient of a function  $f$  with respect to the bids  $b_i$  of player  $i$  only. We use  $\partial_{b_{ij}}f$  to denote the partial derivative with respect to the bid of player  $i$  on good  $j$ , and  $\partial_{b_{ij}}^2 f$  to denote the second derivative. We denote by  $\theta_{i} = \sum_{k\neq i}b_{i}$  as the 'pre-prices,' which represent the prices excluding the bids  $b_i$  (and so, for every player  $i$  and every bid profile,  $p = \theta_{i} + b_{i}$ ). In some of the proofs, we use the abbreviated notation  $(f)^{+} = \max(f,0)$ . All other notations are as defined in the main text.
Appendix A The Associated Game
Proof. (Theorem 3): A sufficient condition for  $\Phi$  being an exact potential[53] is
$$
\forall i \quad \nabla_ {b _ {i}} \Phi (b _ {i}, b _ {- i}) = \nabla_ {b _ {i}} \tilde {u} _ {i} (b _ {i}, b _ {i}).
$$
And indeed, in our case we have: