SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Computational tractability and social welfare (aka. efficiency) of equilibria are two fundamental but in general orthogonal considerations in algorithmic game theory. Nevertheless, we show that when (approximate) full efficiency can be guaranteed via a smoothness argument à la Roughgarden, Nash equilibria are approachable under a family of no-regret learning algorithms, thereby enabling fast and decentralized computation. We leverage this connection to obtain new convergence results in large games—wherein the number of players  $n \gg 1$ —under the well-documented property of full efficiency via smoothness in the limit. Surprisingly, our framework unifies equilibrium computation in disparate classes of problems including games with vanishing strategic sensitivity and two-player zero-sum games, illuminating en route an immediate but overlooked equivalence between smoothness and a well-studied condition in the optimization literature known as the Minty property. Finally, we establish that a family of no-regret dynamics attains a welfare bound that improves over the smoothness framework while at the same time guaranteeing convergence to the set of coarse correlated equilibria. We show this by employing the clairvoyant mirror descent algorithm recently introduced by Piliouras et al.
The Nash equilibrium (NE) [76] formalizes the notion of a stable outcome in a multiagent strategic interaction, and has arguably served as the most influential solution concept in the development of game theory. Indeed, algorithms designed to approximate Nash equilibria in two-player zero-sum games have recently resolved major challenges in AI [7, 10, 81]. Its prescriptive power, however, has been severely undermined in multi-player general-sum games by intrinsic computational barriers [30, 20, 94, 35, 28, 44], limitations which also manifest in the inability of natural learning algorithms to converge [73, 105, 60, 70]. Another well-established but orthogonal critique is the equilibrium selection problem [52]: a general-sum game may have multiple Nash equilibria with widely different welfare. As a result, a modern research agenda in computational game theory has been to identify and characterize natural classes of games that circumvent those fundamental limitations.
In this paper, we uncover a new natural class of games for which the aforementioned caveats of Nash equilibria can be effectively addressed. In particular, our investigation originates from a natural question: when are efficient—in terms of social welfare—Nash equilibria easy to compute? The answer to this question is, at first glance, unsatisfactory: even if Nash equilibria are fully efficient, computational hardness still persists given that constant-sum multi-player games are hard. Indeed, efficiency and computational tractability are, in general, two orthogonal considerations. We show,
however, an interesting twist, an unexpected interplay between efficiency—when viewed from a specific lens—and the behavior of a family of no-regret learning algorithms.
To elucidate the alluded connection that drives much of our results, we first have to recall that the canonical paradigm for establishing the efficiency of equilibria is Roughgarden's celebrated smoothness framework [90] (exposed thoroughly in Section 2). In this context, we observe that if full efficiency of equilibria can be guaranteed via a smoothness argument (with bounded parameters), then Nash equilibria are approachable under a family of no-regret learning algorithms. In other words, full efficiency via smoothness implies computational tractability of NE. This is surprising in that Roughgarden's smoothness framework was developed primarily in order to automatically extend price of anarchy (PoA) bounds to more permissive equilibrium concepts, such as coarse correlated equilibria (CCE); tractability of NE appears at first glance entirely unrelated. In fact, while applying the smoothness framework to (multi-player) constant-sum games appears to make little sense, given that PoA considerations are trivial in such games (all outcomes attain the same social welfare), it turns out that a certain regime of smoothness in (multi-player) constant-sum games is equivalent to a well-known condition in the optimization literature called the Minty property [36] (Observation 3.4).
The condition described above already captures—somewhat unexpectedly—well-studied settings, such as games that admit a minimax theorem [13], but we have found that the most fertile and novel ground to apply this theory revolves around large games, that is, games with a large number of players  $n \gg 1$ . The reason we focus on large games is a well-documented economic phenomenon: equilibria in large games approach—under natural conditions—full efficiency as  $n \to +\infty$ , a property often established via smoothness [39, 23]—a crucial ingredient in our framework. (One rough intuition for this is that in large games each player's influence on the outcome becomes negligible, making optimal behavior easier to characterize [39].) To state our first main result, we denote by  $(\lambda, \mu)$  a pair of bounded smoothness parameters of a game  $\mathcal{G}$ ; the ratio  $\rho(\lambda, \mu) \coloneqq \frac{\lambda}{1 + \mu}$  circumscribes the (in)efficiency of equilibria of  $\mathcal{G}$ , in that all equilibria will attain at least a  $\rho$  fraction of the optimal welfare.
Theorem 1.1 (Informal). Consider a sequence of  $n$ -player  $(\lambda_n, \mu_n)$ -smooth games  $(\mathcal{G}_n)_{n \geq 1}$  such that  $\rho_n := \frac{\lambda_n}{1 + \mu_n} \to 1$  with a sufficiently fast rate. Then, there are decentralized and computationally efficient no-regret dynamics approaching an  $(o_n(1), o_n(1))$ -weak Nash equilibrium.
A few remarks are in order. First, in Theorems 3.1 and A.2 we give a precise non-asymptotic characterization that quantifies the number of iterations as well as the approximation error. We also recall that in an  $(o_n(1), o_n(1))$ -weak Nash equilibrium almost all players are almost best responding (Definition 2.2); this is a well-studied relaxation for which hardness results carry over in general [4, 2, 5, 94]. A limitation of a weak Nash equilibrium is that it can prescribe a strategy profile in which a large numbers of players—albeit of vanishing fraction—can significantly benefit from deviating; this limitation is inherent in a certain regime of Theorem 1.1. Depending on the rate with which  $\rho_n$  approaches full efficiency, Theorem 1.1 can also imply convergence to the usual notion of Nash equilibrium—wherein all players are almost best responding (Corollary A.5). More generally, for a broad class of games that includes graphical games with bounded degree, polymatrix games, and games exhibiting small strategic sensitivity, the conclusion of Theorem 1.1 applies without imposing any restrictions on the rate of convergence of  $\rho_n$ ; those are the most commonly studied classes of games when  $n \gg 1$ .
There are ample compelling aspects in connecting the convergence of no-regret learning algorithms with Roughgarden's smoothness framework. First, there has been a considerable interest in understanding smoothness, insights that can now be inherited to a seemingly entirely different but equally fundamental problem. For example, smoothness naturally extends to Bayesian games [101, 91, 53] (see also [97] for the related class of contextual games), a property that we leverage in Theorem 3.8 to expand our scope to mechanisms of incomplete information. Additional extensions based on local smoothness [92, 78] and the refined primal-dual framework of Nadav and Roughgarden [75] are also described in Appendices A.3 and A.4. Finally, our criterion has a clear and natural economic interpretation, and is part of an ongoing research effort to identify tractable classes of variational inequalities (VIs) beyond the Minty property [33].
<sup>1</sup>We use the terminology large games here—in accordance with much of the economics literature—to refer to games with a large number of players; it should not be confused with another common usage referring to games with a "large" action space.
As a concrete example, our framework subsumes games wherein each player's effect on the outcome vanishes as  $n \to \infty$ ; this captures, for example, simple voting settings [57], as well as general auction design problems where establishing that property turns out to be highly non-trivial and quite delicate [39]. More concretely, for games with vanishing sensitivity  $\epsilon_{n} \rightarrow 0$ , in that unilateral deviations can only affect a player's utility by an additive  $\epsilon_{n}$  (Definition 3.5), we show that the conclusion of Theorem 1.1 applies as long as  $\epsilon_{n} \leq o_{n}(1 / \sqrt{n})$  (Theorem 3.6). At the same time, the condition that  $\rho_{n} \rightarrow 1$  goes much deeper than games with vanishing sensitivity, and, as we explained earlier, surprisingly applies to two-player zero-sum games as well. We find it conceptually appealing that a unifying framework can establish tractability of Nash equilibria in two seemingly disparate classes of problems such as two-player zero-sum games and games with vanishing strategic sensitivity.
Remaining on smooth games, but relaxing the assumption  $\rho \approx 1$ , we next study conditions under which the efficiency guaranteed by the smoothness framework can be improved, while at the same time ensuring the no-regret property, thereby implying convergence to the set of CCE. The smoothness bound is known to be applicable to any outcome of no-regret dynamics, but here we are instead interested in more refined guarantees when specific learning algorithms are in place. Building on a recent result [1], we show in Theorem 4.1 that the clairvoyant variant of gradient descent, introduced by Piliouras et al. [85], enjoys an improved welfare bound and ensures fast convergence to the set of CCE for the average correlated distribution of play. Crucially, compared to an earlier result [1], the clairvoyant algorithm manages to satisfy an appealing notion of per-player incentive compatibility, in the form of convergence to CCE. In other words, improving the welfare predicted by smoothness is not at odds with incentive compatibility (Corollary 4.3).
Notation We let  $\mathbb{N} = \{1,2,\ldots\}$  be the set of natural numbers. For  $n \in \mathbb{N}$ , we denote by  $\llbracket n\rrbracket := \{1,2,\dots,n\}$ . For a vector  $\pmb{x} \in \mathbb{R}^d$ , we denote by  $\| \pmb{x} \|_2$  its (Euclidean)  $\ell_2$  norm. For a convex, compact and nonempty set  $\mathcal{X}$ , we let  $\mathcal{P}_{\mathcal{X}}(\cdot)$  represent the Euclidean projection operator with respect to  $\mathcal{X}$ . We let  $D_{\mathcal{X}}$  be the  $\ell_2$ -diameter of  $\mathcal{X}$ . To simplify the exposition, we often use the  $O(\cdot)$  notation in the main body to suppress the dependence on certain parameters; we also write  $O_n(\cdot)$  to indicate the asymptotic growth solely as a function of  $n$ , so as to lighten the exposition.
Multilinear games We consider  $n$ -player multilinear games. In a multilinear game  $\mathcal{G}$  each player  $i \in [[n]]$  has a convex, compact and nonempty set of feasible strategies  $\mathcal{X}_i \subseteq \mathbb{R}^{d_i}$ , for some dimension  $d_i \in \mathbb{N}$ . Under a joint strategy profile  $\pmb{x} = (x_1, \dots, x_n) \in \prod_{i=1}^n \mathcal{X}_i =: \mathcal{X}$ , there is a continuous utility function  $u_i : (\pmb{x}_1, \dots, \pmb{x}_n) \mapsto \mathbb{R}$  such that  $u_i(\pmb{x}) = \langle \pmb{x}_i, \pmb{u}_i(\pmb{x}_{-i}) \rangle$ , for some function  $\pmb{u}_i : \pmb{x}_{-i} \mapsto \mathbb{R}^{d_i}$ ; here, we used for convenience the standard notation  $\pmb{x}_{-i} := (\pmb{x}_1, \dots, \pmb{x}_{i-1}, \pmb{x}_{i+1}, \dots, \pmb{x}_n)$ . This setup readily captures normal- as well as extensive-form games.
Specifically, in a normal-form game every player  $i \in [[n]]$  selects as strategy a probability distribution  $\pmb{x}_i \in \Delta(\mathcal{A}_i)$  over a finite set of available actions  $\mathcal{A}_i$ . There is an arbitrary utility function  $u_i: \mathcal{A} := \prod_{i=1}^n \mathcal{A}_i \to [-1,1]$  that maps a joint action profile to a utility for Player  $i$ ; we will be making the standard assumption that the range of each player's utilities is bounded by an absolute constant, which is in particular independent of the number of players  $n$ . In this setting, the mixed extension of the utility function indeed satisfies the multilinearity condition imposed above: for  $\pmb{x} \in \prod_{i=1}^n \Delta(\mathcal{A}_i)$ ,  $u_i(\pmb{x}) := \mathbb{E}_{\pmb{a} \sim \pmb{x}}[u_i(\pmb{a})] = \langle \pmb{x}_i, \nabla_{\pmb{x}_i} u_i(\pmb{x}) \rangle$ , where we overloaded the notation  $u_i(\cdot)$ .
Returning to the general setting of multilinear games, we let  $F_{\mathcal{G}}: \prod_{i=1}^{n} \mathcal{X}_{i} \to \prod_{i=1}^{n} \mathcal{X}_{i}$  denote the underlying operator of the game  $\mathcal{G}$ , defined as  $F_{\mathcal{G}}: (\pmb{x}_1, \dots, \pmb{x}_n) \mapsto (\pmb{u}_1(\pmb{x}_{-1}), \dots, \pmb{u}_n(\pmb{x}_{-n}))$ , where function  $\pmb{u}_i$  was introduced earlier. For notational simplicity, we will often omit the subscript  $\mathcal{G}$  when it is clear from the context. We will say that  $F$  is  $L$ -Lipschitz continuous (w.r.t. the  $\ell_2$  norm) if for any  $\pmb{x}, \pmb{x}' \in \prod_{i=1}^{n} \mathcal{X}_i$  it holds that  $\| F(\pmb{x}) - F(\pmb{x}') \|_2 \leq L \| \pmb{x} - \pmb{x}' \|_2$ .
Welfare and the price of anarchy For a joint strategy profile  $\pmb{x} \in \prod_{i=1}^{n} \mathcal{X}_i$ , we define the social welfare attained under  $\pmb{x}$  as  $\mathrm{SW}(\pmb{x}) := \sum_{i=1}^{n} u_i(\pmb{x})$ . The maximum possible social welfare attainable in a game  $\mathcal{G}$  will be denoted by  $\mathrm{OPT}_{\mathcal{G}}$ . Without any essential loss of generality, it will be assumed that  $\mathrm{OPT}_{\mathcal{G}} > 0$ . We say that the game is constant-sum if  $\mathrm{SW}(\pmb{x}) = V \in \mathbb{R}_{>0}$  for any  $\pmb{x} \in \prod_{i=1}^{n} \mathcal{X}_i$ . The price of anarchy (PoA) of a game  $\mathcal{G}$  quantifies the loss in efficiency incurred on
account of strategic players [61]. Formally, if  $\mathbb{N}\mathbb{E}_{\mathcal{G}}$  is the nonempty set of (mixed) Nash equilibria of  $\mathcal{G}$  (Definition 2.2) [88], we define  $\mathrm{PoA}_{\mathcal{G}} := \sup_{\boldsymbol{x} \in \mathbb{N}\mathbb{E}_{\mathcal{G}}} \left\{ \frac{\mathrm{SW}(\boldsymbol{x})}{\mathrm{OPT}_{\mathcal{G}}} \right\}$ .
Smooth games We are now ready to recall the seminal notion of a smooth game,2 conceived in the pioneering work of Roughgarden [90] as a technique to (lower) bound the price of anarchy.
Definition 2.1 (Smooth game [90]). An  $n$ -player game  $\mathcal{G}$  is called  $(\lambda, \mu)$ -smooth, where  $\lambda > 0$  and  $\mu > -1$ , if there exists  $\pmb{x}^{\star} \in \prod_{i=1}^{n} \mathcal{X}_i$  with  $\mathrm{SW}(\pmb{x}^{\star}) = \mathrm{OPT}_{\mathcal{G}}$  such that for every  $\pmb{x} \in \prod_{i=1}^{n} \mathcal{X}_i$ ,
$$
\sum_ {i = 1} ^ {n} u _ {i} \left(\boldsymbol {x} _ {i} ^ {\star}, \boldsymbol {x} _ {- i}\right) \geq \lambda \operatorname {O P T} _ {\mathcal {G}} - \mu \mathrm {S W} (\boldsymbol {x}). \tag {1}
$$
(In the definition above, and throughout this paper, we slightly abuse notation by parsing  $u_{i}(\pmb{x}_{i}^{\star}, \pmb{x}_{-i})$  as  $u_{i}(\pmb{x}_{1}, \dots, \pmb{x}_{i - 1}, \pmb{x}_{i}^{\star}, \pmb{x}_{i + 1}, \dots, \pmb{x}_{n})$ .) Roughgarden [90] observed that in a  $(\lambda, \mu)$ -smooth game, in the sense of Definition 2.1, every Nash equilibrium attains at least a  $\rho(\lambda, \mu) := \frac{\lambda}{1 + \mu}$  fraction of the optimal social welfare  $\mathrm{OPT}_{\mathcal{G}}$ . Importantly, this efficiency guarantee immediately carries over to outcomes of no-regret learning algorithms as well. The robust price of anarchy  $(\mathsf{rPoA}_{\mathcal{G}})$  is the best (i.e., largest) price of anarchy bound provable via a smoothness argument, and can be defined as the solution to the linear program induced by the smoothness constraints given in (1) (see (24) in Appendix A.5). One delicate point here is that the value of  $\mathsf{rPoA}_{\mathcal{G}}$  could be associated with unbounded smoothness parameters, which is a pathological and rather trivial manifestation of smoothness (Remark A.10 elaborates on this point). To be clear, when we say a game  $\mathcal{G}$  attains a certain value  $\rho_{\mathcal{G}}$ , we mean that there exists a finite pair of legitimate smoothness parameters  $(\lambda, \mu)$  such that  $\rho_{\mathcal{G}} = \frac{\lambda}{1 + \mu}$ . With this convention, it might be the case that  $\rho_{\mathcal{G}}(\lambda, \mu) \neq \mathsf{rPoA}_{\mathcal{G}}$  under any finite pair of smoothness parameters  $(\lambda, \mu)$  (Remark A.10). It is also easy to see that  $\mathsf{PoA}_{\mathcal{G}} \geq \mathsf{rPoA}_{\mathcal{G}}$ .
Nash equilibrium We next recall the concept of a weak Nash equilibrium, a natural generalization of the standard notion which is meaningful in multi-player games.
Definition 2.2 (Weak Nash equilibrium [5]). Let  $\delta \in [0,1)$  and  $\epsilon \in \mathbb{R}_{\geq 0}$ . A joint strategy profile  $\pmb{x} \in \prod_{i=1}^{n} \mathcal{X}_i$  is an  $(\epsilon, \delta)$ -weak Nash equilibrium if at least a  $1 - \delta$  fraction of the players are  $\epsilon$ -best responding. An  $(\epsilon, 0)$ -weak Nash equilibrium will be simply referred to as  $\epsilon$ -Nash equilibrium.
In the definition above, we clarify that a player  $i \in [[n]]$  is said to be  $\epsilon$ -best responding if  $\mathrm{BRGAP}_i(\pmb{x}_1, \ldots, \pmb{x}_n) := \max_{\pmb{x}_i' \in \mathcal{X}_i} \langle \pmb{x}_i', \pmb{u}_i(\pmb{x}_{-i}) \rangle - \langle \pmb{x}_i, \pmb{u}_i(\pmb{x}_{-i}) \rangle \leq \epsilon$ . We also define the Nash equilibrium gap as  $\mathrm{NEGAP}(\pmb{x}) := \max_{1 \leq i \leq n} \mathrm{BRGAP}_i(\pmb{x})$ . It has been shown that hardness results in multi-player general-sum games persist under the weak Nash equilibrium concept introduced above, even when  $\epsilon$  and  $\delta$  are absolute constants bounded away from 0 [4, 94].
Regret We are operating in the usual online learning setting. At every time  $t \in \mathbb{N}$  a player  $i \in [[n]]$  selects a strategy  $\pmb{x}_i^{(t)} \in \mathcal{X}_i$ , and then receives as feedback the linear utility function  $\pmb{x}_i \mapsto \langle \pmb{x}_i, \pmb{u}_i^{(t)} \rangle$ , where we overload notation so that  $\pmb{u}_i^{(t)} := \pmb{u}_i(\pmb{x}_{-i}^{(t)})$ . The regret of player  $i \in [[n]]$  under a time horizon  $T \in \mathbb{N}$  is defined as
$$
\operatorname {R e g} _ {i} ^ {(T)} := \max  _ {\boldsymbol {x} _ {i} ^ {*} \in \mathcal {X} _ {i}} \left\{\sum_ {t = 1} ^ {T} u _ {i} (\boldsymbol {x} _ {i} ^ {\star}, \boldsymbol {x} _ {- i} ^ {(t)}) \right\} - \sum_ {t = 1} ^ {T} u _ {i} (\boldsymbol {x} ^ {(t)}).
$$
Optimistic gradient descent By now, there are many online algorithms known to guarantee sublinear regret,  $\mathsf{Reg}_i^{(T)} = o(T)$ , even if the observed utilities are selected adversarially [18, 98]. The main no-regret learning algorithm we consider in this paper is optimistic gradient descent (henceforth OGD) [86, 22], which is known to yield improved regret guarantees in the setting of learning in games [103]. For each player  $i\in [[n]]$ , OGD is defined through the following update rule for  $t\in \mathbb{N}$ .
$$
\boldsymbol {x} _ {i} ^ {(t)} := \mathcal {P} _ {\mathcal {X} _ {i}} \left(\hat {\boldsymbol {x}} _ {i} ^ {(t)} + \eta \boldsymbol {m} _ {i} ^ {(t)}\right),
$$
$$
\hat {\boldsymbol {x}} _ {i} ^ {(t + 1)} := \mathcal {P} _ {\mathcal {X} _ {i}} \left(\hat {\boldsymbol {x}} _ {i} ^ {(t)} + \eta \boldsymbol {u} _ {i} ^ {(t)}\right). \tag {OGD}
$$
${}^{2}$  Smoothness in the sense of Definition 2.1 should not be confused with the unrelated notion of smoothness in the optimization nomenclature.
Here,  $\eta > 0$  is the learning rate;  $\pmb{m}_i^{(t)} \in \mathbb{R}^{d_i}$  is the prediction vector; and  $\hat{\pmb{x}}_i^{(1)} \in \mathcal{X}_i$  is the initialization. It is assumed that the strategy set  $\mathcal{X}_i$  is such that the Euclidean projection  $\mathcal{P}_{\mathcal{X}_i}(\cdot)$  can be implemented efficiently. We will let  $\pmb{m}_i^{(t)} := \pmb{u}_i^{(t-1)}$  for any  $t \in \mathbb{N}$ , where  $\pmb{u}_i^{(0)} := \pmb{u}_i(\hat{\pmb{x}}_{-i}^{(1)})$ .
3 Convergence to Nash equilibria via smoothness
In this section, we study the convergence of optimistic gradient descent (OGD) in large games, that is to say, in the regime  $n \gg 1$ . In our first main result, stated below as Theorem 3.1, we show that when  $\lim_{n \to +\infty} \rho_n = 1$  with a sufficiently fast rate, not only are Nash equilibria approaching the optimal social welfare, but they can also be computed efficiently in a decentralized fashion via OGD.
In the sequel, when considering a sequence of games  $(\mathcal{G}_n)_{n\geq 1}$ , with each game  $\mathcal{G}_n$  being parameterized by the number of players  $n$ , we will use a subscript with variable  $n$  to index the  $n$ th game in the sequence; that notation will also be used to refer to the other underlying parameters of the game that depend on the number of players.
Theorem 3.1. Consider an  $n$ -player  $(\lambda, \mu)$ -smooth game  $\mathcal{G}_n$  such that the game operator  $F_n$  is  $L_n$ -Lipschitz continuous and  $\lambda \geq (1 - \epsilon_n)(1 + \mu)$ , with  $\epsilon_n \neq 0$ . Suppose further that all players follow OGD with learning rate  $\eta_n = 1/(4L_n)$ . Then, for  $\delta \in (0,1)$  and a sufficiently large number of iterations  $T = O_n(nL_n^2/\gamma_n)$ , where  $\gamma_n \coloneqq L_n\mathrm{OPT}_{\mathcal{G}_n}\epsilon_n$ , there is time  $t^\star \in [[T]]$  such that  $\pmb{x}^{(t^\star)} \coloneqq (\pmb{x}_1^{(t^\star)}, \dots, \pmb{x}_n^{(t^\star)})$  is a
$$
\left(\frac {1}{\sqrt {\delta}} O _ {n} \left(\sqrt {\frac {\gamma_ {n}}{n}}\right), \delta\right) - \text {w e a k N a s h e q u i l i b r i u m}. \tag {2}
$$
In particular, if  $\gamma_{n} = O_{n}(n^{1 - \alpha})$ , for  $\alpha \in (0,1)$ ,  $\mathsf{OGD}$  yields an  $(O_{n}(n^{-\frac{\alpha}{3}}), O_{n}(n^{-\frac{\alpha}{3}}))$ -weak NE.
We remark that if it further holds that  $\gamma_{n} = o_{n}(1)$ , the above theorem establishes convergence to the standard notion of Nash equilibrium—wherein all players are (almost) best responding (Corollary A.5). Parameter  $\gamma_{n}$ , which is proportional to the error term  $\epsilon_{n}$ , controls both the number of iterations and the approximation guarantee in (2); we refer to Theorem A.2 (in Appendix A) for a more precise non-asymptotic characterization, which bounds the players' cumulative best response gap as a function of the growth of  $\gamma_{n}$ . We also note that Theorem 3.1 can be strengthened so that (2) holds for most iterates of OGD (say  $99\%$ ), not just a single one (Remark A.4).