SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Large games The study of non-cooperative games with many players (i.e., large games) has been a classical topic in economic theory [96, 41, 68, 95, 83, 38, 50], most recently revived in the context of mean-field games (e.g., [64, 47, 74, 51, 16, 82, 100, 80, 29, 65]). Indeed, many traditional motivating scenarios in algorithmic game theory, including markets and Internet routing, often feature a large number of players in practice. In particular, it has emerged that, under certain conditions, equilibria
<sup>3</sup>We clarify that Theorem 4.1 could have also been stated as follows: CGD outputs an approximate CCE with social welfare attaining the right-hand side of (3); this is evident from the proof in Appendix A.9.
in large games exhibit certain remarkable robustness and stability properties; see, for example, the recent survey of Gradwohl and Kalai [49], as well as the older treatment of Kalai [56] on the subject. Furthermore, mechanism design in large games, along with privacy guarantees, is explored in the work of Kearns et al. [58] (see also [57, 59]).
Efficiency in large games Of particular importance to our work, and specifically the precondition of Theorem 3.1, is the line of work uncovering the by now well-documented phenomenon in economics that large games exhibit, under certain relatively mild assumptions, fully efficient equilibria. Our framework additionally requires that the efficiency of equilibria can be derived via a smoothness argument, in the sense of Roughgarden [90]; we stress again that efficiency alone is of little use when it comes to equilibrium computation (Proposition 3.7). Fortunately, smoothness has emerged as the canonical paradigm for bounding the price of anarchy (e.g., see the survey of Roughgarden et al. [93]), albeit with some notable exceptions [40, 55]. In particular, Feldman et al. [39] quantify the price of anarchy in large games via the smoothness framework. They show that in a general combinatorial domain with simultaneous uniform-price auctions, it holds that  $\rho_{n} \rightarrow 1$  with a rate of  $1 / \sqrt{n}$  as long as there is probabilistic demand, meaning that every buyer abstains from the auction with a constant probability. Several other papers have studied the price of anarchy in large games [63, 23, 24, 26, 25, 17]. In particular, we highlight the work of Cole and Tao [23] which, as Feldman et al. [39], relies on a smoothness argument to establish full efficiency in the limit with a rate of  $1 / \sqrt{n}$  in a Walrasian auction, while asymptotic full efficiency is also shown for Fisher markets under the gross substitutes condition. Further, Carmona et al. [17] provide sufficient conditions under which equilibria are fully efficient in a class of mean-field games; understanding thus whether our framework has new implications in such games is an interesting direction for the future. We finally point out that many other papers have focused on learning in auctions and markets; see [21, 43, 104, 8, 9, 32], and references therein.
6 Conclusions and future work
In conclusion, we have furnished a new sufficient condition under which a family of no-regret learning algorithms, including optimistic gradient descent (OGD), approaches (weak) Nash equilibria. Our criterion has a natural economic interpretation, being intricately connected with Roughgarden's smoothness framework, and captures other well-studied conditions such as the Minty property. We have also shown that clairvoyant gradient descent attains an improved welfare bound compared to that predicted by the smoothness framework, while ensuring at the same time fast convergence to the set of CCE.
There are many promising directions for future work. First, we have seen that under the condition  $\rho = 1$  there exists an algorithm that computes an  $\epsilon$ -NE in time poly  $(1 / \epsilon)$ , leading to a pseudo polynomial-time algorithm (under natural game representations); is there an algorithm that instead runs in time poly  $(\log(1 / \epsilon))$ ?
Convergence to Nash equilibria via computational hardness? Another promising approach for showing convergence to Nash equilibria is by harnessing computational hardness results for the underlying welfare maximization problem. To be specific, we consider the following condition.
Condition 6.1. Consider a multi-player  $(\lambda, \mu)$ -smooth game  $\mathcal{G}$  with  $\rho_{\mathcal{C}} \coloneqq \frac{\lambda}{1 + \mu}$  from a class of games  $\mathcal{C}$  with the polynomial expectation property [79]. For any  $\mathcal{G} \in \mathcal{C}$ , computing a joint strategy profile  $\pmb{x} \in \prod_{i=1}^{n} \mathcal{X}_i$  such that  $\mathrm{SW}(\pmb{x}) \geq \rho_{\mathcal{C}} \cdot \mathrm{OPT}_{\mathcal{G}} + 1 / \mathrm{poly}(\mathcal{G})$  is NP-hard, for any  $\mathrm{poly}(\mathcal{G})$ .
Indeed, smoothness often circumscribes the welfare of polynomial algorithms, such as combinatorial auctions under XOS valuations—in fact, unconditionally under polynomial communication; see [34, Theorem 1.4] and [102, Appendix A.7]. Now, the role of Condition 6.1 is that (unless  $\mathsf{P} = \mathsf{NP}$ ) a polynomially-bounded algorithm such as OGD—which is efficiently implementable (for games with a polynomial number of actions) under the polynomial expectation property—will have the property that  $\frac{1}{T}\sum_{i=1}^{n}\operatorname{Reg}_i^{(T)}\geq -1/poly(\mathcal{G})$ , for any  $\mathcal{G}\in \mathcal{C}$  and poly( $\mathcal{G}$ ), which in turn leads to the following.
Theorem 6.2. Consider a class  $\mathcal{C}$  satisfying Condition 6.1. For any  $\mathcal{G} \in \mathcal{C}$  and  $\epsilon = 1 / \mathrm{poly}(\mathcal{G})$ , there is a polynomial-time algorithm for computing an  $\epsilon$ -Nash equilibrium, unless  $\mathsf{P} = \mathsf{NP}$ .
By virtue of Corollary 4.3, the same conclusion applies even under the weaker condition that computing a CCE with welfare improving over the smoothness bound is hard; this is related to the hardness result of Barman and Ligett [6], discussed in the full version of this paper.
We are grateful to anonymous reviewers and the area chair at NeurIPS for valuable feedback. We also thank Brendan Lucier for helpful pointers to the literature. This material is based on work supported by the Vannevar Bush Faculty Fellowship ONR N00014-23-1-2876, National Science Foundation grants RI-2312342 and RI-1901403, ARO award W911NF2210266, and NIH award A240108S001.
[1] I. Anagnostides, I. Panageas, G. Farina, and T. Sandholm. On last-iterate convergence beyond zero-sum games. In International Conference on Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pages 536–581. PMLR, 2022.
[2] I. Arieli and Y. Babichenko. Simple approximate equilibria in games with many players. In ACM Conference on Economics and Computation (EC), pages 681-691. ACM, 2017.
[3] Y. Azrieli and E. Shmaya. Lipschitz games. Mathematics of Operation Research, 38(2): 350-357, 2013.
[4] Y. Babichenko and A. Rubinstein. Communication complexity of approximate nash equilibria. Games and Economic Behavior, 134:376-398, 2022.
[5] Y. Babichenko, C. H. Papadimitriou, and A. Rubinstein. Can almost everybody be almost happy? In Conference on Innovations in Theoretical Computer Science (ITCS), pages 1-9. ACM, 2016.
[6] S. Barman and K. Ligett. Finding any nontrivial coarse correlated equilibrium is hard. In ACM Conference on Economics and Computation (EC), pages 815-816. ACM, 2015.
[7] M. Bowling, N. Burch, M. Johanson, and O. Tammelin. Heads-up limit hold'em poker is solved. Science, 347(6218), Jan. 2015.
[8] S. Brânzei. Exchange markets: proportional response dynamics and beyond. SIGecom Exch., 19(2):37-45, 2021.
[9] S. Brânzei, N. R. Devanur, and Y. Rabani. Proportional dynamics in exchange economies. In ACM Conference on Economics and Computation (EC), pages 180–201. ACM, 2021.
[10] N. Brown and T. Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418-424, 2018.
[11] A. Böhm. Solving nonconvex-nonconcave min-max problems exhibiting weak minty solutions, 2022.
[12] Y. Cai and W. Zheng. Accelerated single-call methods for constrained min-max optimization. CoRR, abs/2210.03096, 2022.
[13] Y. Cai, O. Candogan, C. Daskalakis, and C. Papadimitriou. Zero-sum polymatrix games: A generalization of minmax. Mathematics of Operations Research, 41(2):648-655, 2016.
[14] Y. Cai, A. Oikonomou, and W. Zheng. Accelerated algorithms for monotone inclusions and constrained nonconvex-nonconcave min-max optimization. CoRR, abs/2206.05248, 2022.
[15] Y. Cai, A. Oikonomou, and W. Zheng. Finite-time last-iterate convergence for learning in multi-player games. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
[16] L. Campi and M. Fischer. Correlated equilibria and mean field games: A simple model. Mathematics of Operations Research, 47(3):2240-2259, 2022.
[17] R. Carmona, C. V. Graves, and Z. Tan. Price of anarchy for mean field games. *ESAIM: Proceedings and Surveys*, 65:349–383, 2019.
[92] T. Roughgarden and F. Schoppmann. Local smoothness and the price of anarchy in splittable congestion games. Journal of Economic Theory, 156:317-342, 2015.
[93] T. Roughgarden, V. Syrgkanis, and E. Tardos. The price of anarchy in auctions. Journal of Artificial Intelligence Research, 59:59–101, 2017.
[94] A. Rubinstein. Inapproximability of nash equilibrium. SIAM Journal on Computing, 47(3): 917-959, 2018.
[95] A. Rustichini, M. Satterthwaite, and S. Williams. Convergence to efficiency in a simple market with incomplete information. *Econometrica*, 62:1041–1063, 1994.
[96] D. Schmeidler. Equilibrium points of nonatomic games. Journal of statistical Physics, 7: 295-300, 1973.
[97] P. G. Sessa, I. Bogunovic, A. Krause, and M. Kamgarpour. Contextual games: Multi-agent learning with side information. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
[98] S. Shalev-Shwartz. Online learning and online convex optimization. Foundations and Trends® in Machine Learning, 4(2), 2012. ISSN 1935-8237.
[99] L. S. Shapley. Some topics in two-person games. In M. Drescher, L. S. Shapley, and A. W. Tucker, editors, Advances in Game Theory. Princeton University Press, 1964.
[100] S. G. Subramanian, M. E. Taylor, M. Crowley, and P. Poupart. Decentralized mean field games. In AAAI Conference on Artificial Intelligence (AAAI), pages 9439-9447. AAAI Press, 2022.
[101] V. Syrgkanis. Bayesian games and the smoothness framework. CoRR, abs/1203.5155, 2012.
[102] V. Syrgkanis and É. Tardos. Composable and efficient mechanisms. In Symposium on Theory of Computing (STOC), pages 211–220. ACM, 2013.
[103] V. Syrgkanis, A. Agarwal, H. Luo, and R. E. Schapire. Fast convergence of regularized learning in games. In Advances in Neural Information Processing Systems, pages 2989-2997, 2015.
[104] Y. Tao. Market Efficiency and Dynamics. PhD thesis, New York University, 2020.
[105] E. Vlatakis-Gkaragkounis, L. Flokas, T. Lianeas, P. Mertikopoulos, and G. Piliouras. No-regret learning and mixed nash equilibria: They do not mix. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020.
In this section, we provide the proofs and a number of results omitted from the main body.
We commence with the proof of Theorem 3.1. Below, we give a more detailed version of the statement provided earlier in the main body. We first state an auxiliary lemma which will be useful for the proof, and can be extracted from earlier work [1]
Lemma A.1. Suppose that player  $i \in [[n]]$  updates its strategy using OGD with learning rate  $\eta > 0$ . Then, for any time  $t \in \mathbb{N}$ ,
$$
\operatorname {B R G A P} _ {i} \left(\boldsymbol {x} ^ {(t)}\right) \leq \left(\frac {D _ {\mathcal {X} _ {i}}}{\eta} + \| \boldsymbol {u} _ {i} ^ {(t)} \| _ {2}\right) \left(\| \boldsymbol {x} _ {i} ^ {(t)} - \hat {\boldsymbol {x}} _ {i} ^ {(t)} \| _ {2} + \| \boldsymbol {x} _ {i} ^ {(t)} - \hat {\boldsymbol {x}} _ {i} ^ {(t + 1)} \| _ {2}\right).
$$
In the sequel, we denote by  $B_{i}\in \mathbb{R}_{>0}$  any number such that  $\| \pmb {u}_i(\pmb{x}_{-i})\| _2\leq B_i$  , for any  $\pmb {x}\in \prod_{i = 1}^{n}\mathcal{X}_i$  In the asymptotic notation below, we make the standard assumption that the parameters  $B_{i}$  and  $D_{\mathcal{X}_i}$  do not depend on the number of players  $n$
Theorem A.2 (Precise version of Theorem 3.1). Consider an  $n$ -player  $(\lambda, \mu)$ -smooth game  $\mathcal{G}_n$  such that the game operator  $F_n$  is  $L_n$ -Lipschitz continuous and  $\lambda \geq (1 - \epsilon_n)(1 + \mu)$  ( $\rho_n \geq 1 - \epsilon_n$ ). Suppose further that all players follow OG with learning rate  $\eta_n = \frac{1}{4L_n}$  and any initialization  $(\hat{\boldsymbol{x}}_1^{(1)}, \dots, \hat{\boldsymbol{x}}_n^{(1)}) \in \prod_{i=1}^n \mathcal{X}_i$ . If  $\epsilon_n > 0$ , then after  $T := \frac{D_{\mathcal{X}}^2}{2\eta_n\epsilon_n(1 + \mu)\mathrm{OPT}_{\mathcal{G}_n}}$  iterations there is a time  $t^\star \in [[T]]$  such that
$$
\begin{array}{l} \sum_ {i = 1} ^ {n} \left(\mathrm {B R G A P} _ {i} \left(\boldsymbol {x} ^ {(t ^ {\star})}\right)\right) ^ {2} \leq 3 2 \left(\frac {\max  _ {1 \leq i \leq n} D _ {\mathcal {X} _ {i}} ^ {2}}{(\eta_ {n}) ^ {2}} + \max  _ {1 \leq i \leq n} B _ {i} ^ {2}\right) \eta_ {n} \epsilon_ {n} (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}} (4) \\ = O _ {n} \left(L _ {n} \operatorname {O P T} _ {\mathcal {G} _ {n}} \epsilon_ {n}\right). (5) \\ \end{array}
$$
In particular, for  $\delta \in \left\{\frac{1}{n}, \frac{2}{n}, \ldots, 1\right\}$ ,  $t^\star$  constitutes a
$$
\left(\frac {1}{\sqrt {\delta}} O _ {n} \left(\sqrt {\frac {L _ {n} \mathrm {O P T} _ {\mathcal {G} _ {n}} \epsilon_ {n}}{n}}\right), \delta\right) - w e a k N a s h e q u i l i b r i u m.
$$
On the other hand, if  $\epsilon_{n} = 0$ , then for any  $T \in \mathbb{N}$  there is a time  $t^{\star} \in [[T]]$  such that
$$
\sum_ {i = 1} ^ {n} \left(\mathrm {B R G A P} _ {i} (\pmb {x} ^ {(t ^ {\star})})\right) ^ {2} \leq \frac {8}{T} \left(\frac {\max _ {1 \leq i \leq n} D _ {\mathcal {X} _ {i}} ^ {2}}{(\eta_ {n}) ^ {2}} + \max _ {1 \leq i \leq n} B _ {i} ^ {2}\right) D _ {\mathcal {X}} ^ {2}.
$$
Before we proceed with the proof, we note that the underlying assumption  $\mu = O_{n}(1)$  in the asymptotic notation above is consistent with known smoothness bounds in large games [39]; we also refer to Remark A.10 for an important point regarding the range of the smoothness parameters.
Proof of Theorem A.2. We will first translate the assumed property  $\rho_{n} \geq 1 - \epsilon_{n}$  to a lower bound on the sum of the players' regrets. In particular, we have that the sum of the players' regrets  $\sum_{i=1}^{n} \mathsf{Reg}_{i}^{(T)}$  for any  $T \in \mathbb{N}$  can be expressed as
$$
\sum_ {i = 1} ^ {n} \max  _ {\boldsymbol {x} _ {i} ^ {*} \in \mathcal {X} _ {i}} \left\{\sum_ {t = 1} ^ {T} u _ {i} \left(\boldsymbol {x} _ {i} ^ {\star}, \boldsymbol {x} _ {- i} ^ {(t)}\right) \right\} - \sum_ {t = 1} ^ {T} \operatorname {S W} \left(\boldsymbol {x} ^ {(t)}\right) \geq \lambda \operatorname {O P T} _ {\mathcal {G} _ {n}} T - (1 + \mu) \sum_ {t = 1} ^ {T} \operatorname {S W} \left(\boldsymbol {x} ^ {(t)}\right), \tag {6}
$$
by Definition 2.1, where  $(\lambda ,\mu)$  are the assumed smoothness parameter of  $\mathcal{G}_n$ . Now, using the assumption that  $\rho_{n}\geq 1 - \epsilon_{n}$ , it follows from (6) that  $\sum_{i = 1}^{n}\mathrm{Reg}_{i}^{(T)}$  is in turn lower bounded by
$$
(1 - \epsilon_ {n}) (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}} T - (1 + \mu) \sum_ {t = 1} ^ {T} \mathrm {S W} \left(\boldsymbol {x} ^ {(t)}\right) \geq - \epsilon_ {n} (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}} T, \tag {7}
$$
where we used the fact that  $\mathrm{OPT}_{\mathcal{G}_n} \geq \mathrm{SW}(\boldsymbol{x}^{(t)})$  (by definition of the optimal welfare). We next appropriately upper bound the sum of the players' regrets  $\sum_{i=1}^{n} \mathrm{Reg}_i^{(T)}$ . Using a slight refinement of the RVU bound [103, 87], it follows that the regret  $\mathrm{Reg}_i^{(T)}$  can be upper bounded by
$$
\frac {1}{2 \eta_ {n}} D _ {\mathcal {X} _ {i}} ^ {2} + \eta_ {n} \sum_ {t = 1} ^ {T} \| \boldsymbol {u} _ {i} ^ {(t)} - \boldsymbol {m} _ {i} ^ {(t)} \| _ {2} ^ {2} - \frac {1}{2 \eta_ {n}} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} _ {i} ^ {(t)} - \hat {\boldsymbol {x}} _ {i} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {x} _ {i} ^ {(t)} - \hat {\boldsymbol {x}} _ {i} ^ {(t + 1)} \| _ {2} ^ {2}\right),
$$
which in turn implies that the sum of the regrets  $\sum_{i=1}^{n} \operatorname{Reg}_i^{(T)}$  is upper bounded by
$$
\frac {1}{2 \eta_ {n}} \sum_ {i = 1} ^ {n} D _ {\mathcal {X} _ {i}} ^ {2} + \eta_ {n} \sum_ {t = 1} ^ {T} \| \boldsymbol {u} ^ {(t)} - \boldsymbol {m} ^ {(t)} \| _ {2} ^ {2} - \frac {1}{2 \eta_ {n}} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} ^ {2}\right), \tag {8}
$$
where we have defined  $\pmb{u}^{(t)}\coloneqq (\pmb{u}_1^{(t)},\dots,\pmb{u}_n^{(t)})$  and  $\pmb{m}^{(t)}\coloneqq (\pmb{m}_1^{(t)},\dots,\pmb{m}_n^{(t)})$ . Moreover, by the  $L_{n}$ -Lipschitz continuity of the game operator  $F_{\mathcal{G}_n}$ , it follows that  $\| \pmb{u}^{(t)} - \pmb{m}^{(t)}\| _2 = \| F_{\mathcal{G}_n}(\pmb{x}^{(t)}) - F_{\mathcal{G}_n}(\pmb{x}^{(t - 1)})\| _2\leq L_n\| \pmb{x}^{(t)} - \pmb{x}^{(t - 1)}\| _2$  for any  $t\in [[T]]$ , where we also used the fact that  $\pmb{m}^{(t)} = \pmb{u}^{(t - 1)}$  and  $\pmb{x}^{(0)}\coloneqq \hat{\pmb{x}}^{(1)}$ . Combining with (8), it follows that for any  $\eta_n\leq \frac{1}{4L_n}$
$$
\sum_ {i = 1} ^ {n} \operatorname {R e g} _ {i} ^ {T} \leq \frac {1}{2 \eta_ {n}} \sum_ {i = 1} ^ {n} D _ {\mathcal {X} _ {i}} ^ {2} - \frac {1}{4 \eta_ {n}} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} ^ {2}\right). \tag {9}
$$
As a result, combining (7) and (9) we have that
$$
\sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \leq 2 \sum_ {i = 1} ^ {n} D _ {\mathcal {X} _ {i}} ^ {2} + 4 \eta_ {n} \epsilon_ {n} (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}} T. \tag {10}
$$
Next, applying Lemma A.1 yields that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {n} \left(\mathbf {B R G A P} _ {i} (\boldsymbol {x} ^ {(t)})\right) ^ {2} \leq 4 \left(\frac {\max _ {i} D _ {\chi_ {i}} ^ {2}}{\eta_ {n} ^ {2}} + \max _ {i} B _ {i} ^ {2}\right) \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| ^ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| ^ {2}\right) \\ \leq 4 \left(\frac {\operatorname* {m a x} _ {i} D _ {\mathcal {X} _ {i}} ^ {2}}{\eta_ {n} ^ {2}} + \max _ {i} B _ {i} ^ {2}\right) \left(2 D _ {\mathcal {X}} ^ {2} + 4 \eta_ {n} \epsilon_ {n} (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}} T\right), \tag {11} \\ \end{array}
$$
where the first bound uses the inequality  $(\alpha + \beta)^2 \leq 2\alpha^2 + 2\beta^2$ , while the second bound follows from (10) by noting that  $D_{\mathcal{X}}^2 = \sum_{i=1}^{n} D_{\mathcal{X}_i}^2$ . As a result, we conclude that there exists  $t^\star \in [[T]]$ , namely  $t^\star \coloneqq \arg \min_{1 \leq t \leq T} \sum_{i=1}^{n} \left(\mathrm{BRGAP}_i(\boldsymbol{x}^{(t)})\right)^2$ , such that
$$
\sum_ {i = 1} ^ {n} \left(\mathrm {B R G A P} _ {i} \left(\boldsymbol {x} ^ {(t ^ {\star})}\right)\right) ^ {2} \leq 4 \left(\frac {\max  _ {1 \leq i \leq n} D _ {\mathcal {X} _ {i}} ^ {2}}{\eta_ {n} ^ {2}} + \max  _ {1 \leq i \leq n} B _ {i} ^ {2}\right) \left(\frac {2}{T} D _ {\mathcal {X}} ^ {2} + 4 \eta_ {n} \epsilon_ {n} (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}}\right). \tag {12}
$$
Now, if  $\epsilon_{n} \neq 0$ , taking  $T := \frac{D_{\mathcal{X}}^{2}}{2\eta_{n}\epsilon_{n}(1 + \mu)\mathrm{OPT}_{\mathcal{G}_{n}}} = \frac{2L_{n}^{2}D_{\mathcal{X}}^{2}}{(1 + \mu)\gamma_{n}}$  implies (4). The claimed approximation guarantee in terms of weak Nash equilibria (per Definition 2.2) in (2) can be derived from (4) as follows. We consider a parameter  $\delta \in (0,1)$  so that  $\delta n = \lfloor \delta n \rfloor$ . Let also  $\epsilon$  be the minimum among the  $\delta n$  largest numbers from  $\mathrm{BRGAP}_1(\pmb{x}^{(t^\star)}), \ldots, \mathrm{BRGAP}_n(\pmb{x}^{(t^\star)})$ . Then,  $\sum_{i=1}^{n} \left( \mathrm{BRGAP}_i(\pmb{x}^{(t^\star)}) \right)^2 \geq \delta \epsilon^2 n$ , and by (12) it follows that
$$
\epsilon \leq \frac {1}{\sqrt {\delta}} \sqrt {3 2 \left(\frac {\max _ {1 \leq i \leq n} D _ {\mathcal {X} _ {i}} ^ {2}}{\eta_ {n} ^ {2}} + \max _ {1 \leq i \leq n} B _ {i} ^ {2}\right) \frac {\eta_ {n} \epsilon_ {n} (1 + \mu) \mathrm {O P T} _ {\mathcal {G} _ {n}}}{n}}.
$$
This implies (2) since  $\pmb{x}^{(t^{\star})}$  is by definition an  $(\epsilon, \delta)$ -weak Nash equilibrium. Finally, if  $\epsilon_{n} = 0$  the claimed bound follows directly from (12).
We note that the asymptotic notation in (2) and (5) applies in the regime  $L_{n}\geq \Omega_{n}(1)$ ; this can be enforced by simply taking  $L_n^\prime \coloneqq \max \{1,L_n\}$ .
Remark A.3. One can also introduce a variant of weak Nash equilibria which is instead parameterized by the average over the players' best response gaps  $\frac{1}{n}\sum_{i=1}^{n}\mathbf{BRGAP}_i(\cdot)$ . Inequality (4) implies that the average best response gap of  $\pmb{x}^{(t^*)}$  can be bounded by  $O_n\left(\sqrt{\frac{L_n\mathrm{OPT}_{\mathcal{G}_n}\epsilon_n}{n}}\right)$ .
Remark A.4. While Theorem A.2 bounds the (weak) Nash equilibrium gap of a single iterate of the dynamics, (11) implies that at least a  $1 - \gamma$  fraction of the iterates of the dynamics constitutes a  $\left(\frac{1}{\sqrt{\delta\gamma}} O_n\left(\sqrt{\frac{L_n\mathrm{OPT}_{\mathcal{G}_n}\epsilon_n}{n}}\right),\delta\right)$ -weak Nash equilibrium. So, by selecting a time index  $t^{\star} \in [[T]]$  uniformly at random we obtain the desired guarantee with high probability, incurring only a small degradation in the solution quality.
In particular, if  $\epsilon_{n}$  approaches to 0 with a sufficiently fast rate, Theorem A.2 also implies convergence to the more standard notion of Nash equilibrium (i.e., Definition 2.2 with  $\delta \coloneqq 0$ ), as we state below. In particular, the following corollary can be derived directly from (4).
Corollary A.5. In the setting of Theorem A.2, if it additionally holds that  $L_{n}\epsilon_{n}\mathrm{OPT}_{\mathcal{G}_{n}} \leq o_{n}(1)$ , OGD yields an  $o_{n}(1)$ -Nash equilibrium after a sufficiently large number of iterations.
On the Lipschitz constant Next, we make some remarks regarding the dependence of the Lipschitz constant  $L_{n}$  on the number of players in the context of general normal-form games. We first note that the Lipschitz constant  $L_{n}$  of the underlying game operator can always be bounded as  $O_{n}(n)$ .
Lemma A.6 (Lipschitz constant in normal-form games). For any  $n$ -player normal-form game  $\mathcal{G}$  with utilities bounded in  $[-1, 1]$ , the Lipschitz constant  $L_{n}$  of the game operator satisfies  $L_{n} \leq n \max_{1 \leq i \leq n} |\mathcal{A}_{i}|$ .
Proof. For any player  $i \in [[n]]$  and any joint strategies  $\pmb{x}, \pmb{x}' \in \prod_{i=1}^{n} \Delta(\mathcal{A}_i)$ , we have