Title: On the Interplay between Social Welfare and Tractability of Equilibria
Venue: NeurIPS 2023
Accepted: yes
Keywords: learning in games

optimistic gradient descent

Nash equilibrium

price of anarchy

smooth games

social welfare
Abstract: Computational tractability and social welfare (aka. efficiency) of equilibria are two fundamental but in general orthogonal considerations in algorithmic game theory. Nevertheless, we show that when (approximate) full efficiency can be guaranteed via a smoothness argument a la Roughgarden, Nash equilibria are approachable under a family of no-regret learning algorithms, thereby enabling fast and decentralized computation. We leverage this connection to obtain new convergence results in large games---wherein the number of players $n \gg 1$---under the well-documented property of full efficiency via smoothness in the limit. Surprisingly, our framework unifies equilibrium computation in disparate classes of problems including games with vanishing strategic sensitivity and two-player zero-sum games, illuminating en route an immediate but overlooked equivalence between smoothness and a well-studied condition in the optimization literature known as the Minty property. Finally, we establish that a family of no-regret dynamics attains a welfare bound that improves over the smoothness framework while at the same time guaranteeing convergence to the set of coarse correlated equilibria. We show this by employing the clairvoyant mirror descent algortihm recently introduced by Piliouras et al.

On the Interplay between Social Welfare and Tractability of Equilibria

Abstract

Computational tractability and social welfare (aka. efficiency) of equilibria are two fundamental but in general orthogonal considerations in algorithmic game theory. Nevertheless, we show that when (approximate) full efficiency can be guaranteed via a smoothness argument à la Roughgarden, Nash equilibria are approachable under a family of no-regret learning algorithms, thereby enabling fast and decentralized computation. We leverage this connection to obtain new convergence results in large games -wherein the number of players n ≫ 1 -under the well-documented property of full efficiency via smoothness in the limit. Surprisingly, our framework unifies equilibrium computation in disparate classes of problems including games with vanishing strategic sensitivity and two-player zero-sum games, illuminating en route an immediate but overlooked equivalence between smoothness and a well-studied condition in the optimization literature known as the Minty property . Finally, we establish that a family of no-regret dynamics attains a welfare bound that improves over the smoothness framework while at the same time guaranteeing convergence to the set of coarse correlated equilibria. We show this by employing the clairvoyant mirror descent algortihm recently introduced by Piliouras et al.

1 Introduction

The Nash equilibrium (NE) [76] formalizes the notion of a stable outcome in a multiagent strategic interaction, and has arguably served as the most influential solution concept in the development of game theory. Indeed, algorithms designed to approximate Nash equilibria in two-player zerosum games have recently resolved major challenges in AI [7, 10, 81]. Its prescriptive power, however, has been severely undermined in multi-player general-sum games by intrinsic computational barriers [30, 20, 94, 35, 28, 44], limitations which also manifest in the inability of natural learning algorithms to converge [73, 105, 60, 70]. Another well-established but orthogonal critique is the equilibrium selection problem [52]: a general-sum game may have multiple Nash equilibria with widely different welfare. As a result, a modern research agenda in computational game theory has been to identify and characterize natural classes of games that circumvent those fundamental limitations.

In this paper, we uncover a new natural class of games for which the aforementioned caveats of Nash equilibria can be effectively addressed. In particular, our investigation originates from a natural question: when are efficient -in terms of social welfare-Nash equilibria easy to compute? The answer to this question is, at first glance, unsatisfactory: even if Nash equilibria are fully efficient, computational hardness still persists given that constant-sum multi-player games are hard. Indeed, efficiency and computational tractability are, in general, two orthogonal considerations. We show,

however, an interesting twist, an unexpected interplay between efficiency-when viewed from a specific lens-and the behavior of a family of no-regret learning algorithms.

1.1 Our results

To elucidate the alluded connection that drives much of our results, we first have to recall that the canonical paradigm for establishing the efficiency of equilibria is Roughgarden's celebrated smoothness framework [90] (exposed thoroughly in Section 2). In this context, we observe that if full efficiency of equilibria can be guaranteed via a smoothness argument (with bounded parameters), then Nash equilibria are approachable under a family of no-regret learning algorithms. In other words, full efficiency via smoothness implies computational tractability of NE. This is surprising in that Roughgarden's smoothness framework was developed primarily in order to automatically extend price of anarchy (PoA) bounds to more permissive equilibrium concepts, such as coarse correlated equilibria (CCE) ; tractability of NE appears at first glance entirely unrelated. In fact, while applying the smoothness framework to (multi-player) constant-sum games appears to make little sense, given that PoA considerations are trivial in such games (all outcomes attain the same social welfare), it turns out that a certain regime of smoothness in (multi-player) constant-sum games is equivalent to a well-known condition in the optimization literature called the Minty property [36] (Observation 3.4).

The condition described above already captures-somewhat unexpectedly-well-studied settings, such as games that admit a minimax theorem [13], but we have found that the most fertile and novel ground to apply this theory revolves around large games , that is, games with a large number of players n ≫ 1 . 1 The reason we focus on large games is a well-documented economic phenomenon: equilibria in large games approach-under natural conditions-full efficiency as n → + ∞ , a property often established via smoothness [39, 23]-a crucial ingredient in our framework. (One rough intuition for this is that in large games each player's influence on the outcome becomes negligible, making optimal behavior easier to characterize [39].) To state our first main result, we denote by ( λ, µ ) a pair of bounded smoothness parameters of a game G ; the ratio ρ ( λ, µ ) := λ 1+ µ circumscribes the (in)efficiency of equilibria of G , in that all equilibria will attain at least a ρ fraction of the optimal welfare.

Theorem 1.1 (Informal) . Consider a sequence of n -player ( λ n , µ n ) -smooth games ( G n ) n ≥ 1 such that ρ n := λ n 1+ µ n → 1 with a sufficiently fast rate. Then, there are decentralized and computationally efficient no-regret dynamics approaching an ( o n (1) , o n (1)) -weak Nash equilibrium.

Afew remarks are in order. First, in Theorems 3.1 and A.2 we give a precise non-asymptotic characterization that quantifies the number of iterations as well as the approximation error. We also recall that in an ( o n (1) , o n (1)) -weak Nash equilibrium almost all players are almost best responding (Definition 2.2); this is a well-studied relaxation for which hardness results carry over in general [4, 2, 5, 94]. Alimitation of a weak Nash equilibrium is that it can prescribe a strategy profile in which a large numbers of players-albeit of vanishing fraction-can significantly benefit from deviating; this limitation is inherent in a certain regime of Theorem 1.1. Depending on the rate with which ρ n approaches full efficiency, Theorem 1.1 can also imply convergence to the usual notion of Nash equilibrium-wherein all players are almost best responding (Corollary A.5). More generally, for a broad class of games that includes graphical games with bounded degree, polymatrix games, and games exhibiting small strategic sensitivity , the conclusion of Theorem 1.1 applies without imposing any restrictions on the rate of convergence of ρ n ; those are the most commonly studied classes of games when n ≫ 1 .

There are ample compelling aspects in connecting the convergence of no-regret learning algorithms with Roughgarden's smoothness framework. First, there has been a considerable interest in understanding smoothness, insights that can now be inherited to a seemingly entirely different but equally fundamental problem. For example, smoothness naturally extends to Bayesian games [101, 91, 53] (see also [97] for the related class of contextual games), a property that we leverage in Theorem 3.8 to expand our scope to mechanisms of incomplete information. Additional extensions based on local smoothness [92, 78] and the refined primal-dual framework of Nadav and Roughgarden [75] are also described in Appendices A.3 and A.4. Finally, our criterion has a clear and natural economic interpretation, and is part of an ongoing research effort to identify tractable classes of variational inequalities (VIs) beyond the Minty property [33].

As a concrete example, our framework subsumes games wherein each player's effect on the outcome vanishes as n → ∞ ; this captures, for example, simple voting settings [57], as well as general auction design problems where establishing that property turns out to be highly non-trivial and quite delicate [39]. More concretely, for games with vanishing sensitivity ϵ n → 0 , in that unilateral deviations can only affect a player's utility by an additive ϵ n (Definition 3.5), we show that the conclusion of Theorem 1.1 applies as long as ϵ n ≤ o n (1 / √ n ) (Theorem 3.6). At the same time, the condition that ρ n → 1 goes much deeper than games with vanishing sensitivity, and, as we explained earlier, surprisingly applies to two-player zero-sum games as well. We find it conceptually appealing that a unifying framework can establish tractability of Nash equilibria in two seemingly disparate classes of problems such as two-player zero-sum games and games with vanishing strategic sensitivity.

Remaining on smooth games, but relaxing the assumption ρ ≈ 1 , we next study conditions under which the efficiency guaranteed by the smoothness framework can be improved, while at the same time ensuring the no-regret property, thereby implying convergence to the set of CCE. The smoothness bound is known to be applicable to any outcome of no-regret dynamics, but here we are instead interested in more refined guarantees when specific learning algorithms are in place. Building on a recent result [1], we show in Theorem 4.1 that the clairvoyant variant of gradient descent, introduced by Piliouras et al. [85], enjoys an improved welfare bound and ensures fast convergence to the set of CCE for the average correlated distribution of play. Crucially, compared to an earlier result [1], the clairvoyant algorithm manages to satisfy an appealing notion of per-player incentive compatibility, in the form of convergence to CCE. In other words, improving the welfare predicted by smoothness is not at odds with incentive compatibility (Corollary 4.3).

2 Background

Notation We let N = { 1 , 2 , . . . } be the set of natural numbers. For n ∈ N , we denote by J n K := { 1 , 2 , . . . , n } . For a vector x ∈ R d , we denote by ∥ x ∥ 2 its (Euclidean) ℓ 2 norm. For a convex, compact and nonempty set X , we let P X ( · ) represent the Euclidean projection operator with respect to X . We let D X be the ℓ 2 -diameter of X . To simplify the exposition, we often use the O ( · ) notation in the main body to suppress the dependence on certain parameters; we also write O n ( · ) to indicate the asymptotic growth solely as a function of n , so as to lighten the exposition.

Multilinear games We consider n -player multilinear games. In a multilinear game G each player i ∈ J n K has a convex, compact and nonempty set of feasible strategies X i ⊆ R d i , for some dimension d i ∈ N . Under a joint strategy profile x = ( x 1 , . . . , x n ) ∈ ∏ n i =1 X i =: X , there is a continuous utility function u i : ( x 1 , . . . , x n ) ↦→ R such that u i ( x ) = ⟨ x i , u i ( x -i ) ⟩ , for some function u i : x -i ↦→ R d i ; here, we used for convenience the standard notation x -i := ( x 1 , . . . , x i -1 , x i +1 , . . . , x n ) . This setup readily captures normalas well as extensive-form games.

Specifically, in a normal-form game every player i ∈ J n K selects as strategy a probability distribution x i ∈ ∆( A i ) over a finite set of available actions A i . There is an arbitrary utility function u i : A := ∏ n i =1 A i → [ -1 , 1] that maps a joint action profile to a utility for Player i ; we will be making the standard assumption that the range of each player's utilities is bounded by an absolute constant, which is in particular independent of the number of players n . In this setting, the mixed extension of the utility function indeed satisfies the multilinearity condition imposed above: for x ∈ ∏ n i =1 ∆( A i ) , u i ( x ) := E a ∼ x [ u i ( a )] = ⟨ x i , ∇ x i u i ( x ) ⟩ , where we overloaded the notation u i ( · ) .

Returning to the general setting of multilinear games, we let F G : ∏ n i =1 X i → ∏ n i =1 X i denote the underlying operator of the game G , defined as F G : ( x 1 , . . . , x n ) ↦→ ( u 1 ( x -1 ) , . . . , u n ( x -n )) , where function u i was introduced earlier. For notational simplicity, we will often omit the subscript G when it is clear from the context. We will say that F is L -Lipschitz continuous (w.r.t. the ℓ 2 norm) if for any x , x ′ ∈ ∏ n i =1 X i it holds that ∥ F ( x ) -F ( x ′ ) ∥ 2 ≤ L ∥ x -x ′ ∥ 2 .

Welfare and the price of anarchy For a joint strategy profile x ∈ ∏ n i =1 X i , we define the social welfare attained under x as SW ( x ) := ∑ n i =1 u i ( x ) . The maximum possible social welfare attainable in a game G will be denoted by OPT G . Without any essential loss of generality, it will be assumed that OPT G > 0 . We say that the game is constant-sum if SW ( x ) = V ∈ R > 0 for any x ∈ ∏ n i =1 X i . The price of anarchy (PoA) of a game G quantifies the loss in efficiency incurred on

account of strategic players [61]. Formally, if NE G is the nonempty set of (mixed) Nash equilibria of G (Definition 2.2) [88], we define PoA G := sup x ∈ NE G { SW ( x ) OPT G } .

Smooth games We are now ready to recall the seminal notion of a smooth game , 2 conceived in the pioneering work of Roughgarden [90] as a technique to (lower) bound the price of anarchy.

Definition 2.1 (Smooth game [90]) . An n -player game G is called ( λ, µ ) -smooth , where λ > 0 and µ > -1 , if there exists x ⋆ ∈ ∏ n i =1 X i with SW ( x ⋆ ) = OPT G such that for every x ∈ ∏ n i =1 X i ,

̸

(In the definition above, and throughout this paper, we slightly abuse notation by parsing u i ( x ⋆ i , x -i ) as u i ( x 1 , . . . , x i -1 , x ⋆ i , x i +1 , . . . , x n ) .) Roughgarden [90] observed that in a ( λ, µ ) -smooth game, in the sense of Definition 2.1, every Nash equilibrium attains at least a ρ ( λ, µ ) := λ 1+ µ fraction of the optimal social welfare OPT G . Importantly, this efficiency guarantee immediately carries over to outcomes of no-regret learning algorithms as well. The robust price of anarchy ( rPoA G ) is the best ( i.e. , largest) price of anarchy bound provable via a smoothness argument, and can be defined as the solution to the linear program induced by the smoothness constraints given in (1) (see (24) in Appendix A.5). One delicate point here is that the value of rPoA G could be associated with unbounded smoothness parameters, which is a pathological and rather trivial manifestation of smoothness (Remark A.10 elaborates on this point). To be clear, when we say a game G attains a certain value ρ G , we mean that there exists a finite pair of legitimate smoothness parameters ( λ, µ ) such that ρ G = λ 1+ µ . With this convention, it might be the case that ρ G ( λ, µ ) = rPoA G under any finite pair of smoothness parameters ( λ, µ ) (Remark A.10). It is also easy to see that PoA G ≥ rPoA G .

Nash equilibrium We next recall the concept of a weak Nash equilibrium, a natural generalization of the standard notion which is meaningful in multi-player games.

Definition 2.2 (Weak Nash equilibrium [5]) . Let δ ∈ [0 , 1) and ϵ ∈ R ≥ 0 . A joint strategy profile x ∈ ∏ n i =1 X i is an ( ϵ, δ ) -weak Nash equilibrium if at least a 1 -δ fraction of the players are ϵ -best responding. An ( ϵ, 0) -weak Nash equilibrium will be simply referred to as ϵ -Nash equilibrium .

In the definition above, we clarify that a player i ∈ J n K is said to be ϵ -best responding if BRGAP i ( x 1 , . . . , x n ) := max x ′ i ∈X i ⟨ x ′ i , u i ( x -i ) ⟩ - ⟨ x i , u i ( x -i ) ⟩ ≤ ϵ . We also define the Nash equilibrium gap as NEGAP ( x ) := max 1 ≤ i ≤ n BRGAP i ( x ) . It has been shown that hardness results in multi-player general-sum games persist under the weak Nash equilibrium concept introduced above, even when ϵ and δ are absolute constants bounded away from 0 [4, 94].

Regret We are operating in the usual online learning setting. At every time t ∈ N a player i ∈ J n K selects a strategy x ( t ) i ∈ X i , and then receives as feedback the linear utility function x i ↦→⟨ x i , u ( t ) i ⟩ , where we overload notation so that u ( t ) i := u i ( x ( t ) -i ) . The regret of player i ∈ J n K under a time horizon T ∈ N is defined as

Optimistic gradient descent By now, there are many online algorithms known to guarantee sublinear regret, Reg ( T ) i = o ( T ) , even if the observed utilities are selected adversarially [18, 98]. The main no-regret learning algorithm we consider in this paper is optimistic gradient descent (henceforth OGD ) [86, 22], which is known to yield improved regret guarantees in the setting of learning in games [103]. For each player i ∈ J n K , OGD is defined through the following update rule for t ∈ N .

Here, η > 0 is the learning rate ; m ( t ) i ∈ R d i is the prediction vector; and ˆ x (1) i ∈ X i is the initialization . It is assumed that the strategy set X i is such that the Euclidean projection P X i ( · ) can be implemented efficiently. We will let m ( t ) i := u ( t -1) i for any t ∈ N , where u (0) i := u i (ˆ x (1) -i ) .

3 Convergence to Nash equilibria via smoothness

In this section, we study the convergence of optimistic gradient descent ( OGD ) in large games, that is to say, in the regime n ≫ 1 . In our first main result, stated below as Theorem 3.1, we show that when lim n → + ∞ ρ n = 1 with a sufficiently fast rate, not only are Nash equilibria approaching the optimal social welfare, but they can also be computed efficiently in a decentralized fashion via OGD .

In the sequel, when considering a sequence of games ( G n ) n ≥ 1 , with each game G n being parameterized by the number of players n , we will use a subscript with variable n to index the n th game in the sequence; that notation will also be used to refer to the other underlying parameters of the game that depend on the number of players.

̸

Theorem 3.1. Consider an n -player ( λ, µ ) -smooth game G n such that the game operator F n is L n -Lipschitz continuous and λ ≥ (1 -ϵ n )(1 + µ ) , with ϵ n = 0 . Suppose further that all players follow OGD with learning rate η n = 1 / (4 L n ) . Then, for δ ∈ (0 , 1) and a sufficiently large number of iterations T = O n ( nL 2 n /γ n ) , where γ n := L n OPT G n ϵ n , there is time t ⋆ ∈ J T K such that x ( t ⋆ ) := ( x ( t ⋆ ) 1 , . . . , x ( t ⋆ ) n ) is a

In particular, if γ n = O n ( n 1 -α ) , for α ∈ (0 , 1) , OGD yields an ( O n ( n -α 3 ) , O n ( n -α 3 )) -weak NE.

We remark that if it further holds that γ n = o n (1) , the above theorem establishes convergence to the standard notion of Nash equilibrium-wherein all players are (almost) best responding (Corollary A.5). Parameter γ n , which is proportional to the error term ϵ n , controls both the number of iterations and the approximation guarantee in (2); we refer to Theorem A.2 (in Appendix A) for a more precise non-asymptotic characterization, which bounds the players' cumulative best response gap as a function of the growth of γ n . We also note that Theorem 3.1 can be strengthened so that (2) holds for most iterates of OGD (say 99%), not just a single one (Remark A.4).

Now, to be more concrete regarding the preconditions of Theorem 3.1, we first observe that the growth of the Lipschitz constant L n depends on the normalization assumptions as well as the structure of the underlying game G n . In particular, let us assume-as is standard-that the range of the utility functions is independent of n , in which case OPT G n = O n ( n ) . We then show that L n = O n (1) in each of the following cases: graphical games with bounded degree (Lemma A.7), games with O n (1 /n ) strategic sensitivity per Definition 3.5 (Lemma A.8), and polymatrix (general-sum) games even with unbounded neighborhoods (Lemma A.9); the first two of the aforementioned classes are the most commonly studied classes in the literature under the regime n ≫ 1 . For all of those classes, applying Theorem 3.1 yields an ( o n (1) , o n (1)) -weak Nash equilibrium for any ϵ n ≤ o n (1) . More generally, the Lipschitz constant L n can grow with n (Lemma A.6), in which case ϵ n must vanish with a faster rate for the conclusion of Theorem 3.1 to kick in. One important limitation of Theorem 3.1 is that, at least in a certain regime, it prescribes a strategy in which many players-albeit a vanishing fraction-may have a profitable deviation (in accordance with Definition 2.2); this is admittedly an inherent feature of our framework.

Remark 3.2 . In a decentralized environment, one question that arises from Theorem 3.1 concerns the identification of a time index t ⋆ ∈ J T K that satisfies (2), which can be viewed as the stopping condition of the algorithm. We suggest two possible approaches. First, if we accept that players have access to a common source of randomness, then all players can sample the same index t ⋆ uniformly at random from the set J T K . As we point out in Remark A.4, this suffices to provide a guarantee with high probability with only a small degradation in the solution quality. The second approach, which does not rest any having a common source of randomness, involves a coordinator who can communicate with the players but possesses no information whatsoever about the underlying game. The coordinator sets a target solution quality parameterized by ( ϵ, δ ) (per Definition 2.2), and after each iteration t elicits from each player i ∈ J n K a single bit, encoding whether ✶ { BRGAP i ( x ( t ) ) > ϵ } . (We note that each player can indeed determine its best response gap with only its local information-namely,

the utility feedback.) The coordinator can then evaluate whether the fraction of the players with at most an ϵ best response gap matches the desired accuracy. While the second approach makes for a less decentralized protocol, the communication overhead described above is arguably very limited.

The key precondition of Theorem 3.1 pertains the behavior of the smoothness parameters, to be discussed next after we first sketch the proof of Theorem 3.1, which extends a recent technique [1].

Proof sketch of Theorem 3.1 The proof is based on the fact that the sum of the players' regrets ∑ n i =1 Reg ( T ) i cannot be too negative , which in turn follows from the assumption that ρ n ≥ 1 -ϵ n . The argument then proceeds by bounding the players' cumulative best response gap across the T iterations ∑ T t =1 ∑ n i =1 ( BRGAP i ( x ( t ) ) ) 2 as a function of γ n and the time horizon T ∈ N , ultimately leading to the conclusion of Theorem 3.1.

Connection with the Minty property While we have stated Theorem 3.1 in the regime n ≫ 1 , under the premise that ρ n is sufficiently close to 1 (as a function of n ), its conclusion is in fact interesting beyond that regime. Indeed, an important observation is that any two-player constant-sum game G := ( A , B ) , with ⟨ x 1 , A x 2 ⟩ + ⟨ x 1 , B x 2 ⟩ = V for any ( x 1 , x 2 ) ∈ X 1 ×X 2 , satisfies ρ G = 1 . This is indeed a consequence of Von Neumann's minimax theorem: ∃ ( x ⋆ 1 , x ⋆ 2 ) ∈ X 1 ×X 2 such that u 1 ( x ⋆ 1 , x 2 ) + u 2 ( x ⋆ 2 , x 1 ) -V = ⟨ x ⋆ 1 , A x 2 ⟩ - ⟨ x 1 , A x ⋆ 2 ⟩ ≥ 0 for any ( x 1 , x 2 ) ∈ X 1 ×X 2 , in turn implying that u 1 ( x ⋆ 1 , x 2 ) + u 2 ( x ⋆ 2 , x 1 ) ≥ (1 + µ ) OPT G -µ SW ( x ) = V ; that is, any two-player constant-sum game is (1+ µ, µ ) -smooth, thereby making the conclusion of Theorem 3.1 readily applicable (by taking ρ ≥ 1 -ϵ 2 for any ϵ > 0 ; see the explicit statement of Theorem A.2). This captures and unifies earlier iteration complexity bounds under OGD and the extra-gradient method [15, 48, 46].

Proposition 3.3. Any two-player constant-sum game G is (1 + µ, µ ) -smooth for µ > -1 , implying that ρ G = 1 . Thus, O (1 /ϵ 2 ) iterations of OGD suffice to obtain an ϵ -Nash equilibrium, for any ϵ > 0 .

More broadly, there is a surprisingly overlooked but immediate connection between smooth games (per Definition 2.1) and the Minty property , a well-known condition in the literature on variational inequalities (VIs) [36, 66, 71]. More precisely, the Minty property postulates the existence of a strategy profile x ⋆ ∈ ∏ n i =1 X i such that ⟨ x ⋆ -x , F ( x ) ⟩ ≥ 0 for any x ∈ ∏ n i =1 X i , where we recall that F is the operator of the game. (We caution that the last inequality is typically stated with the opposite sign since the operator F is defined oppositely.) The following connection is thus immediate from the fact that ⟨ x ⋆ , F ( x ) ⟩ = ∑ n i =1 ⟨ x ⋆ i , u i ( x -i ) ⟩ = ∑ n i =1 u i ( x ⋆ i , x -i ) and ⟨ x , F ( x ) ⟩ = ∑ n i =1 ⟨ x i , u i ( x -i ) ⟩ = SW ( x ) (by multilinearity).

Observation 3.4. For any (multi-player) constant-sum game G , the Minty property is equivalent to G being (1 + µ, µ ) -smooth for some µ > -1 .

Indeed, the Minty property implies that ∑ n i =1 u i ( x ⋆ i , x -i ) ≥ V = (1 + µ ) OPT G -µ SW ( x ) , and the converse direction is also immediate. In fact, for (multi-player) zero-sum games, the Minty property is equivalent to G satisfying (1) under some pair ( λ, µ ) ∈ R 2 . We stress that even if ρ G = 1 , traditional no-regret learning algorithms such as online mirror descent do not generally enjoy iterate convergence to Nash equilibria [70], which stands in stark contrast to the behavior of OGD (Proposition 3.3). In light of Observation 3.4, Theorem 3.1 should also be viewed as part of an ongoing effort to establish sufficient conditions of tractability that are more permissive than the Minty property ( e.g. , [33, 12, 14, 84, 11]). The criterion we furnish herein, based on the smoothness framework, has the important benefit of enjoying a natural economic interpretation, as well as having being extensively studied in the literature. Indeed, we will leverage insights from prior work to obtain several interesting extensions in the remainder of this section.

Games with vanishing sensitivity Returning to the regime n ≫ 1 , why should we expect ρ n → 1 ? Indeed, if anything large games are more general than games with a small number of players since one can always incorporate 'dummy' players into the game. Yet, the point is that large games oftentimes exhibit a structure that leads to more efficient outcomes. For example, one immediate implication of our framework relates to games with a vanishing (strategic) sensitivity (see, e.g. , [57, 3, 72, 31, 45]). There are various ways of defining sensitivity; here, we adopt the following standard definition.

Definition 3.5. The strategic sensitivity ϵ ∈ R > 0 of an n -player game in normal form is defined as

In words, a unilateral deviation can only impact a player's utility by an additive ϵ . Now, as long as the sensitivity decays fast enough, a proof analogous to that of Theorem 3.1 implies the following.

Theorem 3.6. Consider an n -player game G n with sensitivity ϵ n ∈ R > 0 . Then, T = O n ( n ) iterations of OGD suffice to obtain a ( 1 √ δ O n ( ϵ n √ n ) , δ ) -weak Nash equilibrium, for δ ∈ (0 , 1) .

In particular, Theorem 3.6 yields an ( o n (1) , o n (1)) -weak Nash equilibrium as long as ϵ n = o n ( √ n ) . Further, in the canonical regime where ϵ n = O n (1 /n ) , Theorem 3.6 circumscribes the best response gap for all but a constant number of players (by taking δ = O n (1 /n ) ). There are many natural settings where we should expect results such as Theorem 3.6 to be applicable [57, 58, 69]. We find it conceptually compelling that our framework can provide in a unifying way equilibrium guarantees for two seemingly disparate classes of games, namely two-player zero-sum games and games with vanishing strategic sensitivity.

In a similar vein, Feldman et al. [39] showed that ρ n → 1 with a rate of 1 / √ n in a general auction design problem (see also [23]) under the relatively mild assumption that each player participates in the market with some constant probability (aka. probabilistic demand), thereby bypassing known barriers regarding the inefficiency of equilibria in general combinatorial domains. Their proof is based on the fact that each bidder's impact on the prices-under a simultaneous uniform-price auction formatbecomes asymptotically negligible, in the spirit of Definition 3.5 introduced above. The difficulty that arises in that setting is that the natural representation of the utility functions violates our multilinearity assumption (postulated in Section 2). Instead, one would have to resort to some form of discretization before attempting to apply Theorem 3.1, which could be computationally prohibitive; the other prerequisite is that L n = o n ( √ n ) , for which our approach in Lemma A.8 in conjunction with the insights of Feldman et al. [39] could be useful. Understanding whether our techniques can be applied in the combinatorial auction setting of Feldman et al. [39] is left as a challenging open question.

Efficiency of equilibria does not suffice for tractability A natural question arising from Theorem 3.1 is whether a similar statement applies under the assumption that PoA → 1 , that is, assuming that all Nash equilibria of G are (approximately) fully efficient. This is clearly a weaker assumption, but it is unfortunately not sufficient to yield any non-trivial guarantees even in normal-form games:

Proposition 3.7. Even under the promise that PoA G = 1 , computing a (1 / poly ( G )) -Nash equilibrium in normal-form games in polynomial time is impossible when n ≥ 3 , unless PPAD ⊆ P .

This stands in contrast to the class of (1 + µ, µ ) -smooth games, where a fully polynomial-time approximation scheme ( FPTAS ) is implied by Theorem 3.1-assuming access to a utility and a projection oracle, both of which are available in, for example, most succinct normal-form games. Proposition 3.7 is a straightforward consequence of the fact that Nash equilibria are hard to compute even in constant-sum 3 -player games [20]. Furthermore, in Example A.11 we identify a specific 3 -player game in which OGD fails (unconditionally) to converge to ϵ -Nash equilibria for a constant value of ϵ > 0 , even though PoA = 1 . Our example is based on a variant of Shapley's game [99].

Another notable advantage of smoothness as a criterion of convergence is that, at least when the game is represented explicitly, it is easy to compute; this is in contrast to PoA , whose identification even in two-player games is NP -hard (Proposition A.12).

Extensions It turns out that smoothness per Definition 2.1 can be further sharpened using a primaldual framework [75]. Such refined guarantees can also be translated into our setting (in the context of Theorem 3.1), as we elaborate on in Appendix A.3. The upshot is that the modification of Nadav and Roughgarden [75] necessitates analyzing the weighted sum of the players' regrets ∑ n i =1 z i Reg ( T ) i under a dual set of variables { z i } n i =1 . Another interesting extension worth noting relies on the local smoothness framework [92, 78], as we explain in Appendix A.4; the key observation is that local smoothness per Nguyen [78] can be associated with a linearized notion of regret, at which point the analysis of Theorem 3.1 readily carries over. Finally, we also expand our scope to Bayesian mechanisms, as we expound in the upcoming subsection.

Before we proceed, it is important to point out that, unsurprisingly, smoothness is not merely enough to guarantee convergence of OGD ; see Example A.13. It is instead crucial to additionally ensure that ρ ≈ 1 in order to obtain interesting guarantees for the behavior of algorithms such as OGD .

3.1 Bayesian mechanisms

Next, we leverage the connection between smoothness and convergence to Nash equilibria to extend the scope of our framework to Bayesian mechanisms. In particular, analogously to Definition 2.1, Syrgkanis and Tardos [102] have introduced the notion of a smooth mechanism (Definition A.14); detailed background on Bayesian mechanisms and smoothness in that realm is provided in Appendix A.7. In this context, we leverage a reduction of Hartline et al. [53] from an incomplete-information to a complete-information mechanism to arrive at the following theorem. (The standard notion of a BayesNash equilibrium is analogous to Definition 2.2, and is recalled in Definition A.15 of Appendix A.7.) Theorem 3.8. Consider a Bayesian mechanism M such that ρ M = 1 . Then, for any ϵ > 0 , T = O (1 /ϵ 2 ) iterations of OGD suffice to obtain an ϵ -Bayes-Nash equilibrium of M .

In particular, OGD above is executed on the so-called agent-form representation of M (Appendix A.7). Analogously to Theorem 3.1, the above theorem can also be extended in the large n ≫ 1 under the assumption that ρ n → 1 . It is worth noting that Theorem 3.1 already can be applied to certain games of incomplete information (such as imperfect-information extensive-form games), but Theorem 3.8 additionally makes a connection with the literature on smoothness in mechanism design, which facilitates characterizing the smoothness parameters.

4 Improved welfare for no-regret dynamics

Roughgarden's seminal work [90] established that no-regret learning algorithms always attain asymptotically at least rPoA fraction of the optimal social welfare (on average). This guarantee is satisfactory for many classes of games where rPoA is close to 1 (emphatically those studied earlier in Section 3), but smoothness is certainly not a universal phenomenon: there are simple games in which the smoothness framework only provides vacuous guarantees; one such example is Shapley's game, discussed in Appendix A.10. As a result, one important question arising is whether it is possible to improve the efficiency bound predicted by smoothness when specific learning algorithms are in place, while at the same time still guaranteeing convergence to the set of coarse correlated equilibria (CCE) . We stress that optimizing over the set of CCE is typically NP -hard in succinct games [79, 6], making this question interesting also from a complexity-theoretic standpoint.

In this section, we show that it is indeed possible to obtain improved efficiency bounds under a generic condition, while at the same time guaranteeing the no-regret property for each player. A key ingredient in our improvement is the use of clairvoyant mirror descent, an algorithm recently introduced by Piliouras et al. [85]. More precisely, we will instantiate that algorithm with (squared) Euclidean regularization, which can be defined as follows. Let Π x i ( u i ) := arg max x ′ i ∈X i { ⟨ x ′ i , u i ⟩ -1 2 ∥ x i -x ′ i ∥ 2 2 } be the induced prox operator , where x i ∈ X i and u i ∈ R d i . Clairvoyant gradient descent (henceforth CGD ) at time t ∈ N outputs x ( t ) = Π x ( t -1) ( ηF ( w ( t ) )) := (Π x ( t -1) 1 ( η u 1 ( w ( t ) -1 )) , . . . , Π x ( t -1) n ( η u n ( w ( t ) -n ))) , where w ( t ) is any ϵ ( t ) -approximate fixed point of the map ∏ n i =1 X i ∋ w ↦→ Π x ( t -1) ( ηF ( w )) , and x (0) ∈ X is an arbitrary initialization. It turns out that for η < 1 /L , this map is a contraction [37, 85, 19], thereby making approximate fixed points easy to compute. Furthermore, there is also an uncoupled implementation of CGD [85], making the algorithm compelling from a decentralized standpoint as well, but we will not dwell on this issue here. We are now ready to state the main result of this section.

Theorem 4.1. Suppose that all players are updating their strategies using CGD with ϵ ( t ) ≤ min i D X i t 2 and learning rate η = 1 2 L in a ( λ, µ ) -smooth game G , where L is the Lipschtz-continuity parameter of F . Then, for any ϵ 0 > 0 and T ≥ 64 L 2 D 4 X ϵ 2 0 iterations,

the average correlated distribution of play is a 4 LD 2 X T -CCE;

This is the first result that establishes simultaneously these properties under a computationally efficient algorithm, improving a recent work [1] (see also [42, 67] for related results) that failed to

guarantee convergence to CCE. In particular, that earlier work was analyzing OGD , and as it turns out, under a time-invariant learning rate η it is not even known whether OGD ensures sublinear per-player regret, let alone constant (as in Corollary 4.3). The basic ingredient to this improvement is a new property of CGD , which we explain below. Before we sketch the proof, we note that Item 2 above can be readily strengthened so that the improvement holds for the average welfare of most of the strategies, not just a single one (Remark A.22).

Proof sketch of Theorem 4.1 The key step in the proof is showing (in Corollary A.21) that CGD satisfies the remarkable per-player regret bound Reg ( T ) i ≤ α -γ ∑ T t =1 ( BRGAP i ( x ( t ) ) ) 2 , where α > 0 depends on the approximation error of the fixed points of CGD -and can be made time-invariant with only an O (log T ) per-iteration overhead-and γ > 0 . To do this, we crucially rely on a certain property of the Euclidean regularizer (Lemma A.20), which we use in conjunction with the analysis of Farina et al. [37] who extended the original argument of Piliouras et al. [85] beyond entropic regularization.

It is worth noting that the above per-player regret bound (Corollary A.21) implies that a player with nonnegative regret will be almost always approximately best responding, a rather singular occurrence in the context of learning in games; this has interesting implications and goes well-beyond what is currently known for OGD . In particular, it is an open question whether Theorem 4.1 holds under OGD .

Next, we shall describe a concrete implication of Theorem 4.1 under a generic condition. To do so, let us denote by PoA ϵ G the price of anarchy in G with respect to the worst-case ϵ -Nash equilibrium (so that PoA 0 G ≡ PoA G ).

Condition 4.2. Consider a game G and some game-dependent parameter C = C ( G ) > 0 . There exists an ϵ 0 > 0 such that PoA ϵ 0 G > rPoA G + ϵ 2 0 C .

Naturally, it is always the case that PoA G ≥ rPoA G . Further, rPoA G is in general strictly smaller since it measures the worst-case welfare over a larger set than PoA G (even broader than outcomes of noregret learning); Figure 1 in Appendix A.8 further corroborates this premise in a sequence of random normal-form games. Now assuming that PoA G > rPoA G , Condition 4.2 is met if lim ϵ → 0 PoA ϵ G = PoA G , a mild continuity condition (see, for example, the discussion by Roughgarden [89]).

Corollary 4.3. Consider a ( λ, µ ) -smooth game G that satisfies Condition 4.2 under some ϵ 0 > 0 . Then, CGD after T ≥ 64 L 2 D 4 X ϵ 2 0 iterations and η = 1 2 L satisfies the following:

the average correlated distribution of play is an O ( 1 T ) -CCE;

there is a time t ⋆ ∈ J T K and C ′ ( G ) > 0 such that SW ( x ( t ⋆ ) ) ≥ ( ρ G ( λ, µ )+ ϵ 2 0 C ′ ( G )) OPT G .

A fundamental question that arises from Theorem 4.1 is whether there exists a computationally efficient algorithm that determines a CCE with social welfare at least a PoA fraction of the optimal welfare. 3 In games where PoA = rPoA this is clearly possible; in contrast, while Theorem 4.1 improves over the smoothness bound, it does not always guarantee welfare up to PoA . This is a central question in light of the intractability of Nash equilibria [30, 20], which has indeed served as a primary critique to the literature quantifying the price of anarchy of Nash equilibria [90].

Another promising avenue to improving the welfare predicted by the smoothness framework revolves around eliminating certain strategy profiles by arguing that they are reached with negligible probability. For example, in Appendix A.10 we identify an example where iteratively eliminating strictly dominated actions can improve the predictive power of the smoothness framework.

5 Further related work

Large games The study of non-cooperative games with many players ( i.e. , large games) has been a classical topic in economic theory [96, 41, 68, 95, 83, 38, 50], most recently revived in the context of mean-field games ( e.g. , [64, 47, 74, 51, 16, 82, 100, 80, 29, 65]). Indeed, many traditional motivating scenarios in algorithmic game theory, including markets and Internet routing, often feature a large number of players in practice. In particular, it has emerged that, under certain conditions, equilibria

in large games exhibit certain remarkable robustness and stability properties; see, for example, the recent survey of Gradwohl and Kalai [49], as well as the older treatment of Kalai [56] on the subject. Furthermore, mechanism design in large games, along with privacy guarantees, is explored in the work of Kearns et al. [58] (see also [57, 59]).

Efficiency in large games Of particular importance to our work, and specifically the precondition of Theorem 3.1, is the line of work uncovering the by now well-documented phenomenon in economics that large games exhibit, under certain relatively mild assumptions, fully efficient equilibria. Our framework additionally requires that the efficiency of equilibria can be derived via a smoothness argument, in the sense of Roughgarden [90]; we stress again that efficiency alone is of little use when it comes to equilibrium computation (Proposition 3.7). Fortunately, smoothness has emerged as the canonical paradigm for bounding the price of anarchy ( e.g. , see the survey of Roughgarden et al. [93]), albeit with some notable exceptions [40, 55]. In particular, Feldman et al. [39] quantify the price of anarchy in large games via the smoothness framework. They show that in a general combinatorial domain with simultaneous uniform-price auctions, it holds that ρ n → 1 with a rate of 1 / √ n as long as there is probabilistic demand , meaning that every buyer abstains from the auction with a constant probability. Several other papers have studied the price of anarchy in large games [63, 23, 24, 26, 25, 17]. In particular, we highlight the work of Cole and Tao [23] which, as Feldman et al. [39], relies on a smoothness argument to establish full efficiency in the limit with a rate of 1 / √ n in a Walrasian auction, while asymptotic full efficiency is also shown for Fisher markets under the gross substitutes condition. Further, Carmona et al. [17] provide sufficient conditions under which equilibria are fully efficient in a class of mean-field games; understanding thus whether our framework has new implications in such games is an interesting direction for the future. We finally point out that many other papers have focused on learning in auctions and markets; see [21, 43, 104, 8, 9, 32], and references therein.

6 Conclusions and future work

In conclusion, we have furnished a new sufficient condition under which a family of no-regret learning algorithms, including optimistic gradient descent ( OGD ), approaches (weak) Nash equilibria. Our criterion has a natural economic interpretation, being intricately connected with Roughgarden's smoothness framework, and captures other well-studied conditions such as the Minty property. We have also shown that clairvoyant gradient descent attains an improved welfare bound compared to that predicted by the smoothness framework, while ensuring at the same time fast convergence to the set of CCE.

There are many promising directions for future work. First, we have seen that under the condition ρ = 1 there exists an algorithm that computes an ϵ -NE in time poly (1 /ϵ ) , leading to a pseudo polynomial-time algorithm (under natural game representations); is there an algorithm that instead runs in time poly (log(1 /ϵ )) ?

Convergence to Nash equilibria via computational hardness? Another promising approach for showing convergence to Nash equilibria is by harnessing computational hardness results for the underlying welfare maximization problem. To be specific, we consider the following condition.

Condition 6.1. Consider a multi-player ( λ, µ ) -smooth game G with ρ C := λ 1+ µ from a class of games C with the polynomial expectation property [79]. For any G ∈ C , computing a joint strategy profile x ∈ ∏ n i =1 X i such that SW ( x ) ≥ ρ C · OPT G +1 / poly ( G ) is NP -hard, for any poly ( G ) .

Indeed, smoothness often circumscribes the welfare of polynomial algorithms, such as combinatorial auctions under XOS valuations-in fact, unconditionally under polynomial communication; see [34, Theorem 1.4] and [102, Appendix A.7]. Now, the role of Condition 6.1 is that (unless P = NP ) a polynomially-bounded algorithm such as OGD -which is efficiently implementable (for games with a polynomial number of actions) under the polynomial expectation property-will have the property that 1 T ∑ n i =1 Reg ( T ) i ≥ -1 / poly ( G ) , for any G ∈ C and poly ( G ) , which in turn leads to the following. Theorem 6.2. Consider a class C satisfying Condition 6.1. For any G ∈ C and ϵ = 1 / poly ( G ) , there is a polynomial-time algorithm for computing an ϵ -Nash equilibrium, unless P = NP .

By virtue of Corollary 4.3, the same conclusion applies even under the weaker condition that computing a CCE with welfare improving over the smoothness bound is hard; this is related to the hardness result of Barman and Ligett [6], discussed in the full version of this paper.

Acknowledgments and Disclosure of Funding

We are grateful to anonymous reviewers and the area chair at NeurIPS for valuable feedback. We also thank Brendan Lucier for helpful pointers to the literature. This material is based on work supported by the Vannevar Bush Faculty Fellowship ONR N00014-23-1-2876, National Science Foundation grants RI-2312342 and RI-1901403, ARO award W911NF2210266, and NIH award A240108S001.

References

I. Anagnostides, I. Panageas, G. Farina, and T. Sandholm. On last-iterate convergence beyond zero-sum games. In International Conference on Machine Learning (ICML) , volume 162 of Proceedings of Machine Learning Research , pages 536-581. PMLR, 2022.

I. Arieli and Y. Babichenko. Simple approximate equilibria in games with many players. In ACM Conference on Economics and Computation (EC) , pages 681-691. ACM, 2017.

Y. Azrieli and E. Shmaya. Lipschitz games. Mathematics of Operation Research , 38(2): 350-357, 2013.

Y. Babichenko and A. Rubinstein. Communication complexity of approximate nash equilibria. Games and Economic Behavior , 134:376-398, 2022.

Y. Babichenko, C. H. Papadimitriou, and A. Rubinstein. Can almost everybody be almost happy? In Conference on Innovations in Theoretical Computer Science (ITCS) , pages 1-9. ACM, 2016.

S. Barman and K. Ligett. Finding any nontrivial coarse correlated equilibrium is hard. In ACM Conference on Economics and Computation (EC) , pages 815-816. ACM, 2015.

M. Bowling, N. Burch, M. Johanson, and O. Tammelin. Heads-up limit hold'em poker is solved. Science , 347(6218), Jan. 2015.

S. Brânzei. Exchange markets: proportional response dynamics and beyond. SIGecom Exch. , 19(2):37-45, 2021.

S. Brânzei, N. R. Devanur, and Y. Rabani. Proportional dynamics in exchange economies. In ACM Conference on Economics and Computation (EC) , pages 180-201. ACM, 2021.

N. Brown and T. Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science , 359(6374):418-424, 2018.

A. Böhm. Solving nonconvex-nonconcave min-max problems exhibiting weak minty solutions, 2022.

Y. Cai and W. Zheng. Accelerated single-call methods for constrained min-max optimization. CoRR , abs/2210.03096, 2022.

Y. Cai, O. Candogan, C. Daskalakis, and C. Papadimitriou. Zero-sum polymatrix games: A generalization of minmax. Mathematics of Operations Research , 41(2):648-655, 2016.

Y. Cai, A. Oikonomou, and W. Zheng. Accelerated algorithms for monotone inclusions and constrained nonconvex-nonconcave min-max optimization. CoRR , abs/2206.05248, 2022.

Y. Cai, A. Oikonomou, and W. Zheng. Finite-time last-iterate convergence for learning in multi-player games. In Conference on Neural Information Processing Systems (NeurIPS) , 2022.

L. Campi and M. Fischer. Correlated equilibria and mean field games: A simple model. Mathematics of Operations Research , 47(3):2240-2259, 2022.

R. Carmona, C. V. Graves, and Z. Tan. Price of anarchy for mean field games. ESAIM: Proceedings and Surveys , 65:349-383, 2019.

N. Cesa-Bianchi and G. Lugosi. Prediction, learning, and games . Cambridge University Press, 2006.

V. Cevher, G. Piliouras, R. Sim, and S. Skoulakis. Min-max optimization made simple: Approximating the proximal point method via contraction maps. In Symposium on Simplicity in Algorithms (SOSA) , pages 192-206. SIAM, 2023.

X. Chen, X. Deng, and S.-H. Teng. Settling the complexity of computing two-player Nash equilibria. Journal of the ACM , 2009.

Y. K. Cheung, S. Leonardos, and G. Piliouras. Learning in markets: Greed leads to chaos but following the price is right. In International Joint Conference on Artificial Intelligence (IJCAI) , pages 111-117. ijcai.org, 2021.

C.-K. Chiang, T. Yang, C.-J. Lee, M. Mahdavi, C.-J. Lu, R. Jin, and S. Zhu. Online optimization with gradual variations. In Conference on Learning Theory (COLT) , pages 6-1, 2012.

R. Cole and Y. Tao. Large market games with near optimal efficiency. In ACM Conference on Economics and Computation (EC) , pages 791-808. ACM, 2016.

R. Colini-Baldeschi, R. Cominetti, P. Mertikopoulos, and M. Scarsini. The asymptotic behavior of the price of anarchy. In International Workshop On Internet And Network Economics (WINE) , pages 133-145. Springer, 2017.

R. Colini-Baldeschi, R. Cominetti, and M. Scarsini. Price of anarchy for highly congested routing games in parallel networks. Theory of Computing Systems , 63:90-113, 2019.

R. Colini-Baldeschi, R. Cominetti, P. Mertikopoulos, and M. Scarsini. When is selfish routing bad? the price of anarchy in light and heavy traffic. Operations Research , 68(2):411-434, 2020.

V. Conitzer and T. Sandholm. Complexity of constructing solutions in the core based on synergies among coalitions. Artificial Intelligence , 170(6-7):607-619, 2006. Earlier version in IJCAI-03.

V. Conitzer and T. Sandholm. New complexity results about Nash equilibria. Games and Economic Behavior , 63(2):621-641, 2008. Early version in IJCAI-03.

K. Cui and H. Koeppl. Learning graphon mean field games and approximate nash equilibria. In International Conference on Learning Representations (ICLR) . OpenReview.net, 2022.

C. Daskalakis, P. Goldberg, and C. Papadimitriou. The complexity of computing a Nash equilibrium. In Symposium on Theory of Computing (STOC) , 2006.

J. Deb and E. Kalai. Stability in large bayesian games with heterogeneous players. Journal of Economic Theory , 157:1041-1055, 2015.

X. Deng, X. Hu, T. Lin, and W. Zheng. Nash convergence of mean-based learning algorithms in first price auctions. In WWW'22: The ACM Web Conference , pages 141-150. ACM, 2022.

J. Diakonikolas, C. Daskalakis, and M. I. Jordan. Efficient methods for structured nonconvexnonconcave min-max optimization. In International Conference on Artificial Intelligence and Statistics (AISTATS) , volume 130 of Proceedings of Machine Learning Research , pages 2746-2754. PMLR, 2021.

S. Dobzinski, N. Nisan, and M. Schapira. Approximation algorithms for combinatorial auctions with complement-free bidders. Mathematics of Operation Research , 35(1):1-13, 2010.

K. Etessami and M. Yannakakis. On the complexity of Nash equilibria and other fixed points (extended abstract). In Annual Symposium on Foundations of Computer Science (FOCS) , pages 113-123, 2007.

F. Facchinei and J.-S. Pang. Finite-dimensional variational inequalities and complementarity problems . Springer, 2003.

G. Farina, C. Kroer, C. Lee, and H. Luo. Clairvoyant regret minimization: Equivalence with nemirovski's conceptual prox method and extension to general convex games. CoRR , abs/2208.14891, 2022.

T. Feddersen and W. Pesendorfer. Voting behavior and information aggregation in elections with private information. Econometrica , 65(5):1029-1058, 1997.

M. Feldman, N. Immorlica, B. Lucier, T. Roughgarden, and V. Syrgkanis. The price of anarchy in large games. In Symposium on Theory of Computing (STOC) , pages 963-976. ACM, 2016.

M. Feldman, H. Fu, N. Gravin, and B. Lucier. Simultaneous auctions without complements are (almost) efficient. Games and Economic Behavior , 123:327-341, 2020.

D. Fudenberg and D. K. Levine. Open-loop and closed-loop equilibria in dynamic games with many players. Journal of Economic Theory , 44(1):1-18, 1988.

J. Gaitonde, Y. Li, B. Light, B. Lucier, and A. Slivkins. Budget pacing in repeated auctions: Regret and efficiency without convergence. In Innovations in Theoretical Computer Science Conference (ITCS) , volume 251 of LIPIcs , pages 52:1-52:1. Schloss Dagstuhl - LeibnizZentrum für Informatik, 2023.

Y. Gao. New Optimization Models and Methods for Classical, Infinite-Dimensional, and Online Fisher Markets . Columbia University, 2022.

I. Gilboa and E. Zemel. Nash and correlated equilibria: Some complexity considerations. Games and Economic Behavior , 1:80-93, 1989.

P. W. Goldberg and M. J. Katzman. Lower bounds for the query complexity of equilibria in lipschitz games. Theoretical Comput. Sci. , 962, 2023.

N. Golowich, S. Pattathil, and C. Daskalakis. Tight last-iterate convergence rates for no-regret learning in multi-player games. In Conference on Neural Information Processing Systems (NeurIPS) , 2020.

D. A. Gomes and J. Saúde. Mean field games models-a brief survey. Dynamic Games and Applications , 4:110-154, 2014.

E. Gorbunov, N. Loizou, and G. Gidel. Extragradient method: O (1 /K ) last-iterate convergence for monotone variational inequalities and connections with cocoercivity. In International Conference on Artificial Intelligence and Statistics (AISTATS) , volume 151 of Proceedings of Machine Learning Research , pages 366-402. PMLR, 2022.

R. Gradwohl and E. Kalai. Large games: Robustness and stability. Annual Review of Economics , 13:39-56, 2021.

R. Gradwohl and O. Reingold. Fault tolerance in large games. Games and Economic Behavior , 86:438-457, 2014.

Y. Guan, M. Afshari, and P. Tsiotras. Zero-sum games between mean-field teams: A common information and reachability based analysis. CoRR , abs/2303.12243, 2023.

J. C. Harsanyi, R. Selten, et al. A general theory of equilibrium selection in games. MIT Press Books , 1, 1988.

J. D. Hartline, V. Syrgkanis, and É. Tardos. No-regret learning in bayesian games. In Conference on Neural Information Processing Systems (NeurIPS) , pages 3061-3069, 2015.

W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association , 58(301):13-30, 1963.

Y. Jin and P. Lu. First price auction is 1 -1 /e 2 efficient. In Annual Symposium on Foundations of Computer Science (FOCS) , pages 179-187. IEEE, 2022.

E. Kalai. Large robust games. Econometrica , 72(6):1631-1665, 2004.

M. J. Kearns and Y. Mansour. Efficient nash computation in large population games with bounded influence. In Conference on Uncertainty in Artificial Intelligence (UAI) , pages 259-266. Morgan Kaufmann, 2002.

M. J. Kearns, M. M. Pai, A. Roth, and J. R. Ullman. Mechanism design in large games: incentives and privacy. In Innovations in Theoretical Computer Science (ITCS) , pages 403410. ACM, 2014.

M. J. Kearns, M. M. Pai, R. M. Rogers, A. Roth, and J. R. Ullman. Robust mediators in large games. CoRR , abs/1512.02698, 2015.

D. Kim, M. Riemer, M. Liu, J. N. Foerster, M. Everett, C. Sun, G. Tesauro, and J. P. How. Influencing long-term behavior in multiagent reinforcement learning. CoRR , abs/2203.03535, 2022.

E. Koutsoupias and C. Papadimitriou. Worst-case equilibria. In Symposium on Theoretical Aspects in Computer Science (STACS) , 1999.

J. Kulkarni and V. S. Mirrokni. Robust price of anarchy bounds via LP and fenchel duality. In Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 1030-1049. SIAM, 2015.

D. Lacker and K. Ramanan. Rare nash equilibria and the price of anarchy in large static games. Mathematics of Operations Research , 44(2):400-422, 2019.

J.-M. Lasry and P.-L. Lions. Mean field games. Japanese journal of mathematics , 2(1): 229-260, 2007.

M. Laurière, S. Perrin, S. Girgin, P. Muller, A. Jain, T. Cabannes, G. Piliouras, J. Pérolat, R. Elie, O. Pietquin, and M. Geist. Scalable deep reinforcement learning algorithms for mean field games. In International Conference on Machine Learning (ICML) , volume 162 of Proceedings of Machine Learning Research , pages 12078-12095. PMLR, 2022.

T. Lin, Z. Zhou, P. Mertikopoulos, and M. I. Jordan. Finite-time last-iterate convergence for multi-agent learning in games. In International Conference on Machine Learning (ICML) , volume 119 of Proceedings of Machine Learning Research , pages 6161-6171. PMLR, 2020.

B. Lucier, S. Pattathil, A. Slivkins, and M. Zhang. Autobidders with budget and ROI constraints: Efficiency, regret, and pacing dynamics. CoRR , abs/2301.13306, 2023.

G. J. Mailath and A. Postlewaite. Asymmetric information bargaining problems with many agents. The Review of Economic Studies , 57(3):351-367, 1990.

F. McSherry and K. Talwar. Mechanism design via differential privacy. In Annual Symposium on Foundations of Computer Science (FOCS) , pages 94-103. IEEE Computer Society, 2007.

P. Mertikopoulos, C. H. Papadimitriou, and G. Piliouras. Cycles in adversarial regularized learning. In Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 2703-2717. SIAM, 2018.

P. Mertikopoulos, B. Lecouat, H. Zenati, C. Foo, V. Chandrasekhar, and G. Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. In International Conference on Learning Representations (ICLR) . OpenReview.net, 2019.

P. Milgrom and R. Weber. Distributional strategies for games with incomplete infromation. Mathematics of Operations Research , 10:619-632, 1985.

J. Milionis, C. H. Papadimitriou, G. Piliouras, and K. Spendlove. Nash, conley, and computation: Impossibility and incompleteness in game dynamics. CoRR , abs/2203.14129, 2022.

P. Muller, M. Rowland, R. Elie, G. Piliouras, J. Pérolat, M. Laurière, R. Marinier, O. Pietquin, and K. Tuyls. Learning equilibria in mean-field games: Introducing mean-field PSRO. In Autonomous Agents and Multi-Agent Systems , pages 926-934. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2022.

U. Nadav and T. Roughgarden. The limits of smoothness: A primal-dual framework for price of anarchy bounds. In International Workshop On Internet And Network Economics (WINE) , volume 6484 of Lecture Notes in Computer Science , pages 319-326. Springer, 2010.

J. Nash. Equilibrium points in n-person games. National Academy of Sciences , 36:48-49, 1950.

A. Nemirovski. Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization , 15(1), 2004.

K. T. Nguyen. Game efficiency through linear programming duality. In A. Blum, editor, Innovations in Theoretical Computer Science Conference (ITCS) , volume 124 of LIPIcs , pages 66:1-66:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2019.

C. H. Papadimitriou and T. Roughgarden. Computing correlated equilibria in multi-player games. Journal of the ACM , 55(3):14:1-14:29, 2008.

J. Pérolat, S. Perrin, R. Elie, M. Laurière, G. Piliouras, M. Geist, K. Tuyls, and O. Pietquin. Scaling mean field games by online mirror descent. In Autonomous Agents and Multi-Agent Systems , pages 1028-1037. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2022.

J. Perolat, B. D. Vylder, D. Hennes, E. Tarassov, F. Strub, V. de Boer, P. Muller, J. T. Connor, N. Burch, T. Anthony, S. McAleer, R. Elie, S. H. Cen, Z. Wang, A. Gruslys, A. Malysheva, M. Khan, S. Ozair, F. Timbers, T. Pohlen, T. Eccles, M. Rowland, M. Lanctot, J.-B. Lespiau, B. Piot, S. Omidshafiei, E. Lockhart, L. Sifre, N. Beauguerlange, R. Munos, D. Silver, S. Singh, D. Hassabis, and K. Tuyls. Mastering the game of stratego with model-free multiagent reinforcement learning. Science , 378(6623):990-996, 2022.

S. Perrin, M. Laurière, J. Pérolat, R. Élie, M. Geist, and O. Pietquin. Generalization in mean field games by learning master policies. In AAAI Conference on Artificial Intelligence (AAAI) , pages 9413-9421. AAAI Press, 2022.

W. Pesendorfer and J. M. Swinkels. The loser's curse and information aggregation in common value auctions. Econometrica , 65(6):1247-1281, 1997.

T. Pethick, P. Latafat, P. Patrinos, O. Fercoq, and V . Cevher. Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems. CoRR , abs/2302.09831, 2023.

G. Piliouras, R. Sim, and S. Skoulakis. Beyond time-average convergence: Near-optimal uncoupled online learning via clairvoyant multiplicative weights update. In Conference on Neural Information Processing Systems (NeurIPS) , 2022.

A. Rakhlin and K. Sridharan. Online learning with predictable sequences. In Conference on Learning Theory , pages 993-1019, 2013.

S. Rakhlin and K. Sridharan. Optimization, learning, and games with predictable sequences. In Advances in Neural Information Processing Systems , pages 3066-3074, 2013.

J. B. Rosen. Existence and uniqueness of equilibrium points for concave n-person games. Econometrica , 33(3):520-534, 1965.

T. Roughgarden. Barriers to near-optimal equilibria. In Annual Symposium on Foundations of Computer Science (FOCS) , pages 71-80. IEEE Computer Society, 2014.

T. Roughgarden. Intrinsic robustness of the price of anarchy. Journal of the ACM , 62(5): 32:1-32:42, 2015.

T. Roughgarden. The price of anarchy in games of incomplete information. ACM Trans. Economics and Comput. , 3(1):6:1-6:20, 2015.

T. Roughgarden and F. Schoppmann. Local smoothness and the price of anarchy in splittable congestion games. Journal of Economic Theory , 156:317-342, 2015.

T. Roughgarden, V. Syrgkanis, and É. Tardos. The price of anarchy in auctions. Journal of Artificial Intelligence Research , 59:59-101, 2017.

A. Rubinstein. Inapproximability of nash equilibrium. SIAM Journal on Computing , 47(3): 917-959, 2018.

A. Rustichini, M. Satterthwaite, and S. Williams. Convergence to efficiency in a simple market with incomplete information. Econometrica , 62:1041-1063, 1994.

D. Schmeidler. Equilibrium points of nonatomic games. Journal of statistical Physics , 7: 295-300, 1973.

P. G. Sessa, I. Bogunovic, A. Krause, and M. Kamgarpour. Contextual games: Multi-agent learning with side information. In Conference on Neural Information Processing Systems (NeurIPS) , 2020.

S. Shalev-Shwartz. Online learning and online convex optimization. Foundations and Trends® in Machine Learning , 4(2), 2012. ISSN 1935-8237.

L. S. Shapley. Some topics in two-person games. In M. Drescher, L. S. Shapley, and A. W. Tucker, editors, Advances in Game Theory . Princeton University Press, 1964.

S. G. Subramanian, M. E. Taylor, M. Crowley, and P. Poupart. Decentralized mean field games. In AAAI Conference on Artificial Intelligence (AAAI) , pages 9439-9447. AAAI Press, 2022.

V. Syrgkanis. Bayesian games and the smoothness framework. CoRR , abs/1203.5155, 2012.

V. Syrgkanis and É. Tardos. Composable and efficient mechanisms. In Symposium on Theory of Computing (STOC) , pages 211-220. ACM, 2013.

V. Syrgkanis, A. Agarwal, H. Luo, and R. E. Schapire. Fast convergence of regularized learning in games. In Advances in Neural Information Processing Systems , pages 2989-2997, 2015.

Y. Tao. Market Efficiency and Dynamics . PhD thesis, New York University, 2020.

E. Vlatakis-Gkaragkounis, L. Flokas, T. Lianeas, P. Mertikopoulos, and G. Piliouras. No-regret learning and mixed nash equilibria: They do not mix. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 , 2020.

A Omitted proofs

In this section, we provide the proofs and a number of results omitted from the main body.

A.1 Proof of Theorem 3.1

We commence with the proof of Theorem 3.1. Below, we give a more detailed version of the statement provided earlier in the main body. We first state an auxiliary lemma which will be useful for the proof, and can be extracted from earlier work [1]

Lemma A.1. Suppose that player i ∈ J n K updates its strategy using OGD with learning rate η > 0 . Then, for any time t ∈ N ,

In the sequel, we denote by B i ∈ R > 0 any number such that ∥ u i ( x -i ) ∥ 2 ≤ B i , for any x ∈ ∏ n i =1 X i . In the asymptotic notation below, we make the standard assumption that the parameters B i and D X i do not depend on the number of players n .

Theorem A.2 (Precise version of Theorem 3.1) . Consider an n -player ( λ, µ ) -smooth game G n such that the game operator F n is L n -Lipschitz continuous and λ ≥ (1 -ϵ n )(1 + µ ) ( ρ n ≥ 1 -ϵ n ). Suppose further that all players follow OGD with learning rate η n = 1 4 L n and any initialization (ˆ x (1) 1 , . . . , ˆ x (1) n ) ∈ ∏ n i =1 X i . If ϵ n > 0 , then after T := D 2 X 2 η n ϵ n (1+ µ ) OPT G n iterations there is a time t ⋆ ∈ J T K such that

In particular, for δ ∈ { 1 n , 2 n , . . . , 1 } , t ⋆ constitutes a

On the other hand, if ϵ n = 0 , then for any T ∈ N there is a time t ⋆ ∈ J T K such that

Before we proceed with the proof, we note that the underlying assumption µ = O n (1) in the asymptotic notation above is consistent with known smoothness bounds in large games [39]; we also refer to Remark A.10 for an important point regarding the range of the smoothness parameters.

Proof of Theorem A.2. We will first translate the assumed property ρ n ≥ 1 -ϵ n to a lower bound on the sum of the players' regrets. In particular, we have that the sum of the players' regrets ∑ n i =1 Reg ( T ) i for any T ∈ N can be expressed as

by Definition 2.1, where ( λ, µ ) are the assumed smoothness parameter of G n . Now, using the assumption that ρ n ≥ 1 -ϵ n , it follows from (6) that ∑ n i =1 Reg ( T ) i is in turn lower bounded by

where we used the fact that OPT G n ≥ SW ( x ( t ) ) (by definition of the optimal welfare). We next appropriately upper bound the sum of the players' regrets ∑ n i =1 Reg ( T ) i . Using a slight refinement of the RVU bound [103, 87], it follows that the regret Reg ( T ) i can be upper bounded by

which in turn implies that the sum of the regrets ∑ n i =1 Reg ( T ) i is upper bounded by

where we have defined u ( t ) := ( u ( t ) 1 , . . . , u ( t ) n ) and m ( t ) := ( m ( t ) 1 , . . . , m ( t ) n ) . Moreover, by the L n -Lipschitz continuity of the game operator F G n , it follows that ∥ u ( t ) -m ( t ) ∥ 2 = ∥ F G n ( x ( t ) ) -F G n ( x ( t -1) ) ∥ 2 ≤ L n ∥ x ( t ) -x ( t -1) ∥ 2 for any t ∈ J T K , where we also used the fact that m ( t ) = u ( t -1) and x (0) := ˆ x (1) . Combining with (8), it follows that for any η n ≤ 1 4 L n ,

As a result, combining (7) and (9) we have that

Next, applying Lemma A.1 yields that

where the first bound uses the inequality ( α + β ) 2 ≤ 2 α 2 +2 β 2 , while the second bound follows from (10) by noting that D 2 X = ∑ n i =1 D 2 X i . As a result, we conclude that there exists t ⋆ ∈ J T K , namely t ⋆ := arg min 1 ≤ t ≤ T ∑ n i =1 ( BRGAP i ( x ( t ) ) ) 2 , such that

̸

Now, if ϵ n = 0 , taking T := D 2 X 2 η n ϵ n (1+ µ ) OPT G n = 2 L 2 n D 2 X (1+ µ ) γ n implies (4). The claimed approximation guarantee in terms of weak Nash equilibria (per Definition 2.2) in (2) can be derived from (4) as follows. We consider a parameter δ ∈ (0 , 1) so that δn = ⌊ δn ⌋ . Let also ϵ be the minimum among the δn largest numbers from BRGAP 1 ( x ( t ⋆ ) ) , . . . , BRGAP n ( x ( t ⋆ ) ) . Then, ∑ n i =1 ( BRGAP i ( x ( t ⋆ ) ) ) 2 ≥ δϵ 2 n , and by (12) it follows that

This implies (2) since x ( t ⋆ ) is by definition an ( ϵ, δ ) -weak Nash equilibrium. Finally, if ϵ n = 0 the claimed bound follows directly from (12).

We note that the asymptotic notation in (2) and (5) applies in the regime L n ≥ Ω n (1) ; this can be enforced by simply taking L ′ n := max { 1 , L n } .

Remark A.3 . One can also introduce a variant of weak Nash equilibria which is instead parameterized by the average over the players' best response gaps 1 n ∑ n i =1 BRGAP i ( · ) . Inequality (4) implies that the average best response gap of x ( t ⋆ ) can be bounded by O n (√ L n OPT G n ϵ n n ) .

Remark A.4 . While Theorem A.2 bounds the (weak) Nash equilibrium gap of a single iterate of the dynamics, (11) implies that at least a 1 -γ fraction of the iterates of the dynamics constitutes a ( 1 √ δγ O n (√ L n OPT G n ϵ n n ) , δ ) -weak Nash equilibrium. So, by selecting a time index t ⋆ ∈ J T K uniformly at random we obtain the desired guarantee with high probability, incurring only a small degradation in the solution quality.

In particular, if ϵ n approaches to 0 with a sufficiently fast rate, Theorem A.2 also implies convergence to the more standard notion of Nash equilibrium ( i.e. , Definition 2.2 with δ := 0 ), as we state below. In particular, the following corollary can be derived directly from (4).

Corollary A.5. In the setting of Theorem A.2, if it additionally holds that L n ϵ n OPT G n ≤ o n (1) , OGD yields an o n (1) -Nash equilibrium after a sufficiently large number of iterations.

On the Lipschitz constant Next, we make some remarks regarding the dependence of the Lipschitz constant L n on the number of players in the context of general normal-form games. We first note that the Lipschitz constant L n of the underlying game operator can always be bounded as O n ( n ) .

Lemma A.6 (Lipschitz constant in normal-form games) . For any n -player normal-form game G with utilities bounded in [ -1 , 1] , the Lipschitz constant L n of the game operator satisfies L n ≤ n max 1 ≤ i ≤ n |A i | .

Proof. For any player i ∈ J n K and any joint strategies x , x ′ ∈ ∏ n i =1 ∆( A i ) , we have

̸

̸

̸

̸

̸

̸

̸

where (13) uses the equivalence between the ℓ 2 and the ℓ ∞ norm; (14) follows from the definition of the (expected) utility: u i ( a i , x -i ) := E a -i ∼ x -i [ u i ( a )] = ∑ a -i ∈A -i u i ( a ) ∏ i ′ = i x i ′ [ a i ′ ] , for any a i ∈ A i ; (15) uses the triangle inequality along with the assumption that | u i ( a ) | ≤ 1 ; and (16) follows from the well-known fact that the total variation distance between two product distributions can be upper bounded by the sum of the total variation distance of each individual component [54], as well as the equivalence between the ℓ 1 and the ℓ 2 norm. As a result, continuing from (16), we have

̸

̸

where the last inequality used that, by Jensen's inequality, ( ∑ i ′ = i ∥ x i ′ -x ′ i ′ ∥ 2 ) 2 ≤ ( n -1) ∑ i ′ = i ∥ x i ′ -x ′ i ′ ∥ 2 2 ≤ n ∥ x -x ′ ∥ 2 2 . This concludes the proof.

̸

Graphical games As a byproduct of the proof above, we next point out an important refinement of Lemma A.6 concerning graphical games . In particular, here we assume that the utility of each player i ∈ J n K only depends on the actions of players belonging to its neighborhood N i ⊆ J n K \ { i } .

We further assume that |N i | ≤ ∆ for any player i ∈ J n K , where ∆ ∈ N will be referred to as the degree of the graphical game. To conclude the definition, we also posit that each player i ∈ J n K can only affect the utilities of at most ∆ other players: | i ′ ∈ n : i ∈ N i ′ | ≤ ∆ .

J K Lemma A.7 (Lipschitz constant in graphical games) . For any n -player graphical game with degree ∆ ∈ N and utilities bounded in [ -1 , 1] , the Lipschitz constant L n of the game operator satisfies L n ≤ ∆max 1 ≤ i ≤ n |A i | .

Proof. For any player i ∈ J n K and any joint strategies x , x ′ ∈ ∏ n i =1 ∆( A i ) , we have

where the derivation above is similar to that in Lemma A.6. As a result,

Games with vanishing sensitivity We next focus on a different subclass of normal-form games; namely, games with small sensitivity (per Definition 3.5). Taking a step back, in graphical games every player can only be impacted by (and have an impact to) a small number of other players. Instead, here we consider games where a player's utility can be impacted by all other players, but only by a small amount.

Lemma A.8 (Lipschitz constant in games with vanishing sensitivity) . For any n -player normalform game with sensitivity ϵ n ∈ R > 0 , the Lipschitz constant L n of the game operator satisfies L n ≤ ϵ n n max 1 ≤ i ≤ n |A i | .

Proof. Let i ∈ J n K . For x 1 , x ′ 1 ∈ ∆( A 1 ) and a -1 ∈ A -1 (restricting on Player 1 here is without any loss, and only made for the sake of simplicity in the notation), it follows that

for some a ′ 1 ∈ A 1 , where we used that ( x 1 [ a ′ 1 ] -x ′ 1 [ a ′ 1 ]) = ∑ a 1 ∈A 1 \{ a ′ 1 } ( x ′ 1 [ a 1 ] -x 1 [ a 1 ]) since x 1 , x ′ 1 ∈ ∆( A 1 ) . Continuing from (17), we have

where ϵ n is the sensitivity of the game. Similar reasoning yields that | u i ( x ) -u i ( x ′ 1 , x -1 ) | ≤ ϵ n ∥ x 1 -x ′ 1 ∥ 1 , for any x -1 ∈ ∏ n i =2 ∆( A i ) . As a result, we have

̸

̸

By symmetry, we have shown that ∥ u i ( x -i ) -u i ( x ′ -i ) ∥ ∞ ≤ ϵ n ∑ i ′ = i ∥ x i ′ -x ′ i ′ ∥ 1 , and the rest of the argument is identical to that of Lemma A.6.

Polymatrix games A careful examination of the proof of Lemma A.8 reveals that its conclusion in fact applies under a more relaxed condition compared to what imposed by Definition 3.5; namely, we can define

̸

In words, when considering the utility of a player i ∈ J n K , we only bound deviations by players besides i . It is easy to see that Lemma A.8 in fact applies even if ϵ ∈ R > 0 is defined as in (18). This observation enables capturing other interesting classes of games under the premise that L n = O n (1) , such as polymatrix games . Specifically, a polymatrix game is defined with respect to an underlying directed graph G = ( J n K , E ) , so that each node of G is uniquely associated with the corresponding player. For every edge ( i, i ′ ) ∈ E there is a matrix A i,i ′ ∈ R A i ×A i ′ so that the utility of Player i is defined as

where N i := { i ′ ∈ J n K : ( i, i ′ ) ∈ E } . Unlike the class of graphical games we saw earlier, here we do not restrict the size of the neighborhoods. For this reason, we have normalized each player's utility by a 1 /n factor in (19), for otherwise the utilities are not guaranteed to be bounded (independent of the number of players n ). It is then easy to see that polymatrix games are subject to Lemma A.8 under ϵ defined in (18), which here satisfies ϵ = O (1 /n ) . Below, we provide a simpler and sharper argument compared to Lemma A.8.

Lemma A.9. For any n -player polymatrix game, the Lipschitz constant L n of the game operator satisfies L n ≤ max ( i,i ′ ) ∈ E ∥ A i,i ′ ∥ 2 , where ∥ · ∥ 2 here denotes the spectral norm.

Proof. By definition of the utility functions in (19), we have that for any player i ∈ J n K and x , x ′ ∈ ∏ n i =1 ∆( A i ) ,

and the claim follows.

A.2 Games with vanishing sensitivity

We next establish an important implication of our framework concerning the class of games with vanishing strategic sensitivity (per Definition 3.5); the statement of the theorem is recalled below.

Theorem 3.6. Consider an n -player game G n with sensitivity ϵ n ∈ R > 0 . Then, T = O n ( n ) iterations of OGD suffice to obtain a ( 1 √ δ O n ( ϵ n √ n ) , δ ) -weak Nash equilibrium, for δ ∈ (0 , 1) .

Proof. Let i ∈ J n K . Following the proof of Lemma A.8, the definition of sensitivity implies that | u i ( x ′ i , x -i ) -u i ( x ) | ≤ ϵ n ∥ x ′ i -x i ∥ 1 ≤ 2 ϵ n , for any x ∈ ∏ n i =1 ∆( A i ) and x ′ i ∈ ∆( A i ) . The proof now follows that of Theorem A.2. In particular, we can lower bound the sum of the players' regrets as follows.

As a result, following the proof of Theorem A.2, we conclude that for learning rate η n := 1 4 L n there exists a time t ⋆ ∈ J T K such that

Thus, setting

where the asymptotic notation above applies in the regime L n ≥ Ω n (1) . The statement thus follows.

We note that the conclusion of Theorem 3.6 also applies under the weaker notion of sensitivity defined in (18), if we additionally assume that there exists x ⋆ ∈ ∏ n i =1 ∆( A i ) such that for any x ∈ ∏ n i =1 ∆( A i ) ,

Condition (20) can be met even in games with large sensitivity (Observation 3.4), which is why we chose to state Theorem 3.6 under a stronger but more interpretable condition.

Next, we point out two interesting extensions of our approach based on developments following the original smoothness framework of Roughgarden [90].

A.3 Refined smoothness

Throughout this paper we have relied on the original smoothness framework [90] to derive much of our results. Nevertheless, it is worth noting that there are certain extensions documented in the Nadav and Roughgarden [75] noted that the original smoothness framework is in fact able to provide efficiency bounds for a set of equilibria even more permissive than CCE, which they refer to as average CCE with respect to a welfare-maximizing strategy x ⋆ ∈ ∏ n ∆( A i ) (ACCE ∗ ). In particular, the latter relaxation only requires a guarantee for the average regret over the players, n ∑ i =1 Reg i ( x ⋆ i ) while CCE instead requires a guarantee for the maximum regret max 1 ≤ i ≤ n Reg ( T ) i . Based on this observation, Nadav and Roughgarden [75] developed a primal-dual framework in order to provide refined guarantees (beyond the original smoothness framework) for CCE ∗ , which boils down to solving the following linear program.

literature that can make our predictions sharper. To be more precise, focusing on normal-form games, i =1 1 n ( T ) ,

Here, it is assumed that the underlying game is in normal form, with ∏ n i =1 A i being the set of joint action profiles, and a ⋆ ∈ ∏ n i =1 A i a welfare-maximizing joint action. (The formulation of Nadav and Roughgarden [75] is based on cost-minimization games, but of course their LP can be cast directly for payoff-maximization games in the form of (21).) The above LP is a simple transformation of the fractional-linear program corresponding to optimizing ρ := λ 1+ µ subject to the smoothness constraints of Definition 2.1, but with the additional flexibility of optimizing over a vector z ∈ R n ≥ 0 . In particular, when restricting z 1 = z 2 = · · · = z n , this exactly recovers the LP for computing the robust price of anarchy-given in (24). Nevertheless, Nadav and Roughgarden [75] pointed out that the additional flexibility of (21) can have an arbitrarily large impact on the predicted efficiency [75, Remark 1].

The sharper definition of smoothness introduced by Nadav and Roughgarden [75] can be leveraged in our framework as follows. We can define a generalized robust price of anarchy as the solution of the LP (21). If this quantity approaches to 1 with a sufficiently fast rate (in terms of the number of players), we can extend Theorem A.2 by analyzing the weighted sum of the players' regrets ∑ n i =1 z i Reg ( T ) i . It is easy to see that the argument of Theorem 3.1 readily carries over under the condition that the ratio z i /z i ′ is bounded for any i, i ′ ∈ J n K , as well as the assumption that each z i is bounded away from 0 . In contrast, if there exists a pair i, i ′ ∈ J n K for which the ratio z i /z i ′ is unbounded, our current techniques do not appear to be applicable; this is related to the well-known difficulty of deriving so-called RVU bounds for the maximum of the players' regret, instead of their sum [103]. In other words, our current techniques can sharpen Theorem 3.1 by considering solutions of (21) under the additional (linear) constraint that the variables { z i } n i =1 have bounded pairwise ratio. Given that the more general framework of Nadav and Roughgarden [75] leads to improved bounds, we expect that this modification should have applications in our setting as well. In particular, we point out that the 2 -player example of Nadav and Roughgarden [75] that separates ACCE ∗ from CCE ∗ indeed satisfies z 1 /z 2 ≈ 2 . 3 [75, Proposition 2], so the separation manifests itself even when the pairwise ratio is bounded by an absolute constant.

We finally refer to the works of Nguyen [78] and Kulkarni and Mirrokni [62] for a different primaldual take on smoothness.

A.4 Local smoothness

Another interesting extension of our techniques can be obtained using the framework of local smoothness , first introduced by Roughgarden and Schoppmann [92] and recently refined by Nguyen [78] in the context of splittable congestion games. In what follows, we follow the treatment of Nguyen

[78] as it fits our framework. In this context, Nguyen [78] introduced the notion of a ( λ, µ ) -dualsmooth differentiable utility function u : R ≥ 0 → R ≥ 0 with the property that for every vectors z = ( z 1 , . . . , z n ) ∈ R n ≥ 0 and z ′ = ( z ′ 1 , . . . , z ′ n ) ∈ R n ≥ 0 ,

where z := ∑ n i =1 z i and z ′ := ∑ n i =1 z ′ i . We also note that u ′ above denotes the derivative of u . (Again, (22) has been translated to utility-maximization games compared to the original formulation of Nguyen [78].) Furthermore, a splittable congestion game is called ( λ, µ ) -dual-smooth [78] if for every resource e ∈ E the utility function u e : R ≥ 0 → R ≥ 0 is ( λ, µ ) -dual-smooth in the sense of (22). The importance of Nguyen's extension is that it can be applied to coarse correlated equilibria, as opposed to the original definition of Roughgarden and Schoppmann [92] that was applicable to correlated equilibria; this distinction is incidentally important for our framework.

We will connect this concept of local smoothness with the linearization of the players' regrets. In particular, the utility of a player i ∈ J n K under a joint strategy x is defined as u i ( x ) := ∑ e ∈ E x i [ e ] u e ( ∑ n i =1 x i [ e ]) . Then,

As a result, for any t ∈ N ,

with the understanding that x [ e ] := ∑ n i =1 x i [ e ] and x ⋆ [ e ] := ∑ n i =1 x ⋆ i [ e ] for any e ∈ E . Now let us define Reg ( T ) L ,i ( x ⋆ i ) := ∑ T t =1 ⟨ x ⋆ i -x ( t ) i , ∇ x i u i ( x ) ⟩ . Combining the last displayed equality with local smoothness (22), we get that

Consequently, if we define a local smoothness bound, ρ L := λ 1+ µ with ( λ, µ ) being subject to (22), Theorem 3.1 can be readily extended in the regime ρ L → 1 based on (23).

A.5 Considerations based on PoA

It is natural to ask if the conclusion of Theorem 3.1 can be relaxed to PoA → 1 . Here, we point out that such an assumption does not suffice to obtain interesting guarantees. In particular, we next note Proposition 3.7, stated earlier in the main body, which is an immediate byproduct of the hardness of computing Nash equilibria in constant-sum games.

Proposition 3.7. Even under the promise that PoA G = 1 , computing a (1 / poly ( G )) -Nash equilibrium in normal-form games in polynomial time is impossible when n ≥ 3 , unless PPAD ⊆ P .

Proof. Chen et al. [20] showed that computing a (1 / poly (max i |A i | )) -Nash equilibrium in a generalsum two-player game in normal form is PPAD -hard. As a result, the same applies in constant-sum 3 -player games by suitably incorporating an additional player who has no strategic impact on the game. Further, in any constant-sum game G it clearly holds that PoA G = 1 , concluding the proof.

Remark A.10 . The positive result established in Theorem A.2 requires that ρ ≈ 1 under a bounded pair of smoothness parameters ( λ, µ ) . One may wonder whether similar conclusions apply even

when the smoothness parameters are unbounded. In particular, the robust price of anarchy ( rPoA ) can be defined as the solution to the following linear program.

̸

Above a ⋆ = ( a ⋆ 1 , . . . , a ⋆ n ) ∈ ∏ n i =1 A i is a welfare-maximizing action profile, which can be assumed to be unique for the purposes of our discussion here. In this context, can we extend the conclusion of Theorem A.2 under the assumption that rPoA G n → 1 ? In general, that is not possible. Indeed, in any constant-sum game one can make the LP (24) feasible by taking z = 0 and ρ = 1 ; that is, rPoA G n = 1 for any constant-sum game G . Thus, assuming merely that rPoA G → 1 is not enough to obtain interesting guarantees for computing Nash equilibria (in accordance with Proposition 3.7). In other words, our underlying assumption that the smoothness parameters are bounded is necessary. To further explain this discrepancy, we note that a constant-sum game is generally not (1 + µ, µ ) -smooth (for a finite µ > -1 ), for otherwise any constant-sum game would satisfy the Minty property (Proposition 3.3), which would in turn contradict Proposition 3.7. The case where z = 0 in any optimal solution of (24) essentially corresponds to a pathological manifestation of smoothness in which the underlying parameters are unbounded. We stress again that throughout this paper, when we say that a game is ( λ, µ ) -smooth we, of course, posit that those smoothness parameters are finite. Further, when we say that ρ G = 1 in a game G , we accept that there are finite smoothness parameters associated with ρ G . With this convention, we reiterate that there are games in which rPoA G = ρ G .

Next, we provide a concrete example based on Shapley's game [99] in which OGD (unconditionally) fails to converge to an ϵ -Nash equilibrium, for a constant ϵ > 0 .

Example A.11 . We consider a 3 -player game G defined as follows. We let

Then, we set u 1 ( x 1 , x 2 , x 3 ) := x ⊤ 1 A x 2 , u 2 ( x 1 , x 2 , x 3 ) := x ⊤ 1 B x 2 , and u 3 ( x 1 , x 2 , x 3 ) := 3 -x ⊤ 1 A x 2 -x ⊤ 1 B x 2 . Thus, for any joint strategy ( x 1 , x 2 , x 3 ) it holds that SW ( x 1 , x 2 , x 3 ) = 3 , implying that PoA G = 1 . Further, G is in normal form. Now, through a numerical simulation we draw the following conclusion: Although PoA G = 1 , for T ≫ 1 OGD with learning rate η := 0 . 01 and initialization (ˆ x (1) 1 , ˆ x (1) 2 , · ) := ((0 . 5 , 0 . 25 , 0 . 25) , (0 . 25 , 0 . 5 , 0 . 25) , · ) satisfies NEGAP ( x ( t ) ) ≥ 0 . 1875 for any t ∈ J T K , where ( x ( t ) ) t ≥ 1 is the sequence of iterates produced by OGD . We note that here we do not consider the initialization from the uniform distribution simply because that happens to be the unique Nash equilibrium in Shapley's game; the conclusion above readily applies for any initialization by suitably modifying the underlying game. We also note that the specific value for the learning rate specified above is used for concreteness, and the conclusion is not tied to that specific value.

We next note another separation between the bound predicted by smoothness and PoA , which gives an additional reason why the smoothness framework is more suited as a criterion for determining tractability. Below, we make for simplicity the assumption that the game has a unique welfaremaximizing action profile, which can always be enforced by incorporating an arbitrarily small noise in the players' utilities.

Proposition A.12. Determining whether a game is ( λ, µ ) -smooth can be done in polynomial time in explicitly represented normal-form games. In contrast, even in two-player games, determining PoA is NP -hard.

Proof. First of all, the welfare-maximizing action profile a ⋆ ∈ ∏ n i =1 A i can be trivially computed in polynomial time (in the size of the input) since the game is explicitly represented. For a legitimate ( λ, µ ) pair, determining whether the game is ( λ, µ ) -smooth can be phrased as a feasibility linear program, with a number of constraints equal to the number of possible joint action profiles, each corresponding to a separate constraint in (1); namely, ∑ n i =1 u i ( a ⋆ i , a -i ) ≥ λ OPT -µ ∑ n i =1 u i ( a ) . As a result, the number of constraints is polynomial in the description of the game (since it is assumed that the game is explicitly represented). Furthermore, one can optimize over the smoothness parameters by considering the LP (of polynomial size) given in (24), which determines the robust price

of anarchy ( rPoA ). In accordance with Theorem A.2 (see Remark A.10), one can also incorporate the constraint z ≥ 1 / poly ( G ) . Regarding the second claim, hardness of determining PoA even in two-player games follows directly from the reduction of Conitzer and Sandholm [27, Theorem 1]. In particular, an algorithm computing PoA would enable determining the satisfiability of a SAT formula.

One important question is whether smoothness can be identified in polynomial time even in succinctly represented games [79]. Indeed, the obvious algorithm for identifying smoothness described above requires a number of constraints that scales exponentially with the number of players, which is especially problematic in the regime of large games we focus on in Section 3.

We need to clarify, however, that knowing the smoothness parameters is certainly not a prerequisite for applying our approach. In detail, one can first of all apply Theorem 3.1 in classes of games where there are available analytical bounds for ρ n as a function of n , obviating the need to determine whether ρ n is close to 1 . Besides this point, and more importantly, one can always execute the algorithm prescribed by Theorem 3.1-which does not require any knowledge regarding the smoothness parameters-and then efficiently evaluate the solution quality (per Definition 2.2). If the desired accuracy has been reached, then this is precisely the initial goal; otherwise, we can safely assume that the preconditons of Theorem 3.1 are not met.

A.6 Smoothness does not suffice for convergence

Next, we provide for completeness an example of a smooth game where OGD fails to converge to Nash equilibria. This shows that, as expected, it is not merely enough to know that ρ = 0 to obtain interesting guarantees.

̸

Example A.13 . This example is based on a bimatrix game in normal form described with the payoff matrices

We first claim that this bimatrix game G satisfies ρ G ≥ 0 . 125 . Indeed, we first see that the welfaremaximizing profile of (26) reads ( x ⋆ 1 , x ⋆ 2 ) = ((0 , 0 , 1 , 0) , (1 , 0 , 0 , 0)) . We also claim that for any pair of actions a 1 ∈ A 1 and a 2 ∈ A 2 it holds that

where OPT G = 1 . 6 . As a result, it follows that for any x 1 ∈ ∆( A 1 ) and x 2 ∈ ∆( A 2 ) it holds that E a 1 ∼ x 1 ,a 2 ∼ x 2 [ ∑ 2 i =1 u i ( x ⋆ i , a -i )] ≥ 0 . 125 · OPT G , in turn implying that ∑ 2 i =1 u i ( x ⋆ i , x -i ) ≥ 0 . 125 · OPT G . This means that G is (0 . 125 , 0) -smooth. Furthermore, through a numerical simulation we draw the following conclusion: for T ≫ 1 , OGD with learning rate η := 0 . 01 satisfies NEGAP ( x ( t ) 1 , x ( t ) 2 ) ≥ 0 . 046 for any t ∈ J T K , where ( x ( t ) 1 , x ( t ) 2 ) t ≥ 1 is the sequence of iterates produced by OGD . Once again, it is worth noting that this conclusion is not tied to the specific choice of learning rate we chose above, which is only used for concreteness.

A.7 Bayesian mechanisms

In this subsection, we consider the standard independent private value model of Bayesian mechanisms. In particular, each player i ∈ J n K has a type v i drawn from a distribution F i over a finite set of types V i ; without any loss, we may assume that F i is the uniform distribution over V i . It is further assumed that players' types are pairwise independent. After each player i ∈ J n K draws a type v i ∼ F i , i selects an action a i ( v i ) ∈ A i , which is a function of its type v i . Now consider a fixed profile of types v ∈ V := V 1 ×V 2 . . . V n . The (expected) utility of Player i under a joint strategy profile x ( v ) is denoted by u i ( x ; v i ) := E a ∼ x [ u i ( a ; v i )] . There is also a principal agent who does not take an action in the game, and whose utility under x is given by R ( x ) := E a ∼ x [ R ( a )] . Accordingly, the social welfare is defined as SW ( x , v ) := ∑ n i =1 u i ( x ; v i ) + R ( x ) , while OPT M ( v ) represents the optimal social welfare of mechanism M as a function of the joint type v ∈ V . We will make the assumption that the utility functions assign solely nonnegative values.

Smooth mechanisms Analogously to the notion of a smooth game (Definition 2.1), Syrgkanis and Tardos [102] introduced the notion of a smooth mechanism, formally recalled below.

Definition A.14 (Smooth mechanism [102]) . A mechanism M is ( λ, µ ) -smooth, where λ, µ ≥ 0 , if there exists an strategy profile x ⋆ ( v ) ∈ ∏ n i =1 ∆( A i ) , for every type profile v ∈ V , such that for any action profile a ∈ A and type profile v = ( v 1 , . . . , v n ) ∈ V ,

As it turns out, many important mechanisms satisfy the above definition under various parameters ( λ, µ ) [102]. For a ( λ, µ ) -smooth mechanism M , we define ρ M := λ max { 1 ,µ } .

Definition A.15. Let x i : V i → ∆( A i ) be the strategy of each player i ∈ J n K . A joint strategy profile x is a Bayes-Nash equilibrium (BNE) if for any player i ∈ J n K , any type v i ∈ V i and deviation a ′ i ∈ A i ,

This definition coincides with the standard notion of Nash equilibrium we saw in Definition 2.2 when each distribution F i is a point mass. We also note that an ϵ -BNE incorporates an ϵ ≥ 0 additive slackness in (28).

Population interpretation of Bayesian games In the agent-form representation of a Bayesian game [53], it is assumed that there are n finite subpopulations of players, each corresponding to a player i ∈ J n K . Each player belonging to population i corresponds to a type v i ∈ V i , which is distinct in each population and across populations. In this induced population game, nature first draws one player from each population, and then each player v i selects an action a i ( v i ) ; the game is then played under those selected actions. In symbols, the utility of Player v i from population i reads u AG i,v i ( a ) := E v ∼F [ u i ( a ( v ); v i ) ✶ { v i } ] , (29) where by { v i } above we denote the event that type v i is selected by nature among the population corresponding to Player i . The importance of the population interpretation of the Bayesian game is that it induces a mechanism of complete information, which will be denoted by M AG = M AG ( M ) . The following characterization highlights an important connection between M AG and M .

Theorem A.16 ([53]) . If a mechanism M is ( λ, µ ) -smooth, then the complete information mechanism M AG = M AG ( M ) is also ( λ, µ ) -smooth.

We now proceed with the proof of Theorem 3.8. To keep the exposition self-contained, we will not make explicit use of Theorem A.16.

Theorem 3.8. Consider a Bayesian mechanism M such that ρ M = 1 . Then, for any ϵ > 0 , T = O (1 /ϵ 2 ) iterations of OGD suffice to obtain an ϵ -Bayes-Nash equilibrium of M .

Proof. Under the assumption that ρ M = 1 , it follows that there exists a pair ( λ, µ ) ∈ R 2 ≥ 0 such that M is ( λ, µ ) -smooth with λ = max { 1 , µ } . Further, by definition it holds that OPT M ( v ) ≥ ∑ n i =1 u i ( a ; v i ) + R ( a ) , for any action profile a ∈ A . Combining with (27), we have that there exists a strategy profile x ⋆ ( v ) ∈ ∏ n i =1 ∆( A i ) such that for every type profile v ∈ V and action profile a ∈ A ,

since λ = max { 1 , µ } and utilities are nonnegative. As a result, for any x ( v ) ∈ ∏ n i =1 ∆( A i ) ,

Further, by definition of the agent-form utilities (29) and the law of total expectation, (30) can be equivalently cast as

where we used the notation x i,v i := x i ( v i ) . This implies that the sum of the players' regrets in the agent-form representation is nonnegative. The proof of Theorem 3.8 then follows from the correspondence between Nash equilibria in the agent-form representation of M and Bayes-Nash equilibria in the original incomplete-information mechanism M .

A.8 PoA vs robust PoA

̸

In this subsection, we provide some additional justification for the condition rPoA = PoA required in Corollary 4.3. In particular, we conduct experiments on a set of random normal-form games. Some illustrative results for 10 random games are demonstrated in Figure 1. Overall, we observe that not only rPoA = PoA , but in fact the gap between the two quantities is typically substantial. This discrepancy is expected since, as we have already stressed, rPoA quantifies the worst-case welfare over a (typically) much broader set of equilibria.

1

2

3

4

5

6

7

8

9

10

Game

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Fraction of optimal welfare

PoA

rPoA

A.9 Proof of Theorem 4.1

In this subsection, we provide the proof of Theorem 4.1. For completeness, we also include the proofs of certain results known from prior work [37, 85, 77].

To this end, we first recall that the prox operator associated with the (squared) Euclidean regularizer 1 2 ∥ · ∥ 2 2 is defined as

under some utility vector u i ∈ R d i . Accordingly, we let Π x ( u ) := (Π x 1 ( u 1 ) , . . . , Π x n ( u n )) , where u := ( u 1 , . . . , u n ) . With this definition in mind, we note the following property of the prox operator. Lemma A.17 ([37, 77]) . The prox operator Π x ( · ) is 1 -Lipschitz continuous with respect to ∥ · ∥ 2 for any x ∈ ∏ n i =1 X i .

Proof. Let i ∈ J n K . For any u i , u ′ i ∈ R d i and x i ∈ X i ,

where we used the fact that the projection operator is non-expansive with respect to ∥ · ∥ 2 , and that

This implies that ∥ Π x ( u ) -Π x ( u ′ ) ∥ 2 ≤ ∥ u -u ′ ∥ , where u = ( u 1 , . . . , u n ) and u ′ = ( u ′ 1 , . . . , u ′ n ) .

̸

Using this lemma, a key observation for the analysis of clairvoyant mirror descent is the following contraction property [85, 37].

Proposition A.18 ([85, 37]) . Suppose that F is L -Lipschitz continuous. For any x ′ ∈ ∏ n i =1 X i the function

is ( ηL ) -Lipschitz continuous. As a result, function (32) is a contraction mapping as long as η < 1 L .

This follows directly from Lemma A.17: ∥ Π x ′ ( ηF ( w )) -Π x ′ ( ηF ( w ′ )) ∥ 2 ≤ η ∥ F ( w ) -F ( w ′ ) ∥ 2 ≤ ηL ∥ w -w ′ ∥ 2 .

Proposition A.18 reassures us that fixed points of the contraction mapping (32) not only exist, but ϵ -approximate fixed points can also be computed in a time proportional to log(1 /ϵ ) [85]. In this context, if x (0) ∈ ∏ n i =1 X i is an arbitrary point, we consider the update rule defined for t ∈ N via

where w ( t ) ∈ ∏ n i =1 X i is any point such that ∥ w ( t ) -Π x ( t -1) ( ηF ( w ( t ) )) ∥ 2 ≤ ϵ ( t ) . It is important to note that this sequence ( x ( t ) ) t ≥ 1 is not uniquely defined, but with a slight abuse we refer to any sequence satisfying (33) as clairvoyant gradient descent ( CGD ) . CGD satisfies the following remarkable regret bound.

Theorem A.19 ([85, 37]) . For any T ∈ N , the regret of player i ∈ J n K under CGD satisfies

where L i ∈ R > 0 is the Lipschitz constant of u i with respect to ∥ · ∥ 2 .

Proof. Given that x ( t ) i := Π x ( t -1) i ( η u i ( w ( t ) -i )) , the first-order optimality condition implies that for any x ⋆ i ∈ X i and t ∈ J T K ,

We further have that

thereby implying through a telescopic summation that

Combining with (35),

The claim thus follows from the fact that

Unlike prior work, here we will make crucial use of the negative term in (34). The important lemma below will enable us to express the regret bound (34) in terms of i 's best response gap; it is analogous to Lemma A.1 we stated earlier for OGD .

Lemma A.20. Fix any t ∈ N and let x ( t ) = Π x ( t -1) ( ηF ( w ( t ) )) , where w ( t ) ∈ ∏ n i =1 X i is any point such that ∥ w ( t ) -Π x ( t -1) ( ηF ( w ( t ) )) ∥ 2 ≤ ϵ ( t ) . Then,

Proof. By the first-order optimality condition, it follows that for any x ⋆ i ∈ X i ,

in turn implying that

Furthermore,

Combining (36) and (37) yields that

concluding the proof.

Corollary A.21. For any T ∈ N , the regret of player i ∈ n under CGD can be bounded as

In particular, if η := 1 2 L and ϵ ( t ) ≤ D X i t 2 for any t ∈ J T K ,

Proof. By Lemma A.20, it follows that for any t ∈ T ,

Summing over all t ∈ J T K and combining with Theorem A.19 implies the statement since L i D X i ∑ T t =1 ϵ ( t ) ≤ L i D 2 X i ∑ T t =1 1 t 2 ≤ 2 LD 2 X i .

We are now ready to prove Theorem 4.1, which is recalled below.

Theorem 4.1. Suppose that all players are updating their strategies using CGD with ϵ ( t ) ≤ min i D X i t 2 and learning rate η = 1 2 L in a ( λ, µ ) -smooth game G , where L is the Lipschtz-continuity parameter of F . Then, for any ϵ 0 > 0 and T ≥ 64 L 2 D 4 X ϵ 2 0 iterations,

the average correlated distribution of play is a 4 LD 2 X T -CCE;

there is a time t ⋆ ∈ T such that

Proof. First, by Corollary A.21,

where we used the fact that ∑ T t =1 ( ϵ ( t ) ) 2 ≤ D 2 X ∑ T t =1 1 t 4 ≤ 2 D 2 X and η = 1 2 L . As a result, Item 1 follows directly from (38) by invoking the well-known fact that the CCE gap is bounded by max 1 ≤ i ≤ n Reg ( T ) i .

For Item 2, let us fix any ϵ ≥ ϵ 0 . Suppose that at every time t ∈ J T K it holds that BRGAP i ( x ( t ) ) > ϵ for some player i ∈ J n K . Then, by (38) we conclude that

since T ≥ 64 L 2 D 4 X ϵ 2 0 ≥ 64 L 2 D 4 X ϵ 2 . As a result, by ( λ, µ ) -smoothness, it follows that

in turn implying that

As a result, there is a time t ⋆ ∈ J T K such that SW ( x ( t ⋆ ) ) ≥ ρ G ( λ, µ ) · OPT G + ϵ 2 16( µ +1) LD 2 X . In the contrary case, if there is a time t ⋆ ∈ J T K such that BRGAP i ( x ( t ⋆ ) ) ≤ ϵ for any player i ∈ J n K , it follows that SW ( x ( t ⋆ ) ) ≥ PoA ϵ G · OPT G (by definition). This concludes the proof.

In particular, we note that Corollary 4.3 stated earlier in the main body is an immediate consequence of Theorem 4.1 under Condition 4.2 since rPoA G ≥ ρ G . It is also worth noting that for the special case of normal-form games, one can state Theorem 4.1 so that the first term in the right-hand side of (3) reads ρ G ( λ, µ ) · OPT G + ϵ 2 32( µ +1) L ; thus, for a broad class of games (see Lemmas A.7 to A.9), the improvement over the smoothness bound is an absolute constant when ϵ and µ are also bounded by absolute constants. Relatedly, we should note that Theorem 6.2 applies in the regime where µ is polynomially bounded (we are not aware of any application of smoothness where this is not the case). Remark A.22 . It is direct to see that Item 2 of Theorem 4.1 can be refined so that there is a set S ⊆ J T K , with | S | ≥ (1 -γ ) T , so that

Remark A.23 . Theorem 4.1 can also be refined using the primal-dual framework of Nadav and Roughgarden [75] discussed in Appendix A.3, with the caveat that the variables { z i } n i =1 need to

̸

again have a bounded pairwise ratio. It is interesting to note, however, that using CGD enables making non-trivial conclusions even when a subset of the variables { z i } n i =1 are zero in an optimal solution. In particular, when the quantity ∑ n i =1 z i Reg ( T ) i is nonnegative, we can readily draw the non-trivial conclusion that all players in the set { i ∈ J n K : z i = 0 } are eventually best responding (by virtue of Corollary A.21). It is not at all clear if such conclusions apply to OGD as well. In other words, using CGD one can essentially replace PoA ϵ G in Theorem 4.1 by the price of anarchy with respect to a solution concept in which a particular subset of players are best responding. Shedding light to this type of guarantee necessitates understanding the implications of having some of the variables { z i } n i =1 being zero in an optimal solution of the LP (21). Relatedly, even if we constraint z 1 = z 2 = · · · = z n = z , can we characterize the games for which z ≈ 0 ? We explained earlier in Remark A.10 that such pathologies (typically) occur in constant-sum games, for which questions concerning social welfare are trivial.

A.10 Beyond smoothness

In this subsection, we discuss an example in which the welfare predicted by the smoothness framework is far from the welfare obtained by learning algorithms such as OGD . We then suggest a natural direction that enables obtaining sharper predictions; we have already seen refined guarantees beyond the standard smoothness framework in Appendices A.3 and A.4, but here we explore a different direction.

Our example is again based on Shapley's game, a bimatrix game in normal form defined with the matrices

Claim A.24. Let G be the bimatrix game defined in (39) . Then, rPoA G = 0 .

Proof. Let ( x ⋆ 1 , x ⋆ 2 ) := ((1 , 0 , 0) , (0 , 1 , 0)) be a welfare-maximizing joint action. By symmetry, the following argument will apply to any welfare-maximizing joint action. Consider ( x 1 , x 2 ) = ((0 , 1 , 0) , (0 , 1 , 0)) . Then, we have that u 1 ( x ⋆ 1 , x 2 ) + u 2 ( x ⋆ 2 , x 1 ) = ( x ⋆ 1 ) ⊤ A x 2 + x ⊤ 1 B x ⋆ 2 = 0 + 0 . As a result, the smoothness constraint corresponding to ( x 1 , x 2 ) necessitates that 0 ≥ λ , which is incompatible with the constraint that λ > 0 (Definition 2.1).

In spite of the fact that rPoA G = 0 , we see in Figure 2 (left) that OGD approaches close to the optimal social welfare 1 . This can be explained by the fact that certain joint actions-in particular, those in the diagonal-are played with small probability under OGD (right-side of Figure 2). This motivates introducing a more refined notion of smoothness in which the smoothness constraints are only enforced on the joint actions that are visited with a non-negligible probability.

0

500

1000

1500

2000

Iteration

0

.

7

0

.

8

0

.

9

Average social welfare

0

500

1000

1500

2000

Iteration

0

.

0

0

.

1

0

.

2

0

.

3

Probability of bad state

One class of games in which one can easily make such refinements concerns games solvable under iterated elimination of strictly dominated actions. As a concrete example, let

Observation A.25. Let G be the game defined in (40) . Then, rPoA G = 1 2 . Furthermore, if G ′ is the game 4 resulting from iterative removal of strictly dominated actions, then rPoA G ′ = 1 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers a specific type of smooth game first introduced in [Roughgarden 2015]. The main result of this paper is that when the game has robust PoA lower bounded by 1-$\\epsilon$ (defined by the concept of smoothness of the game) and all the players apply optimistic gradient descent algorithm (OGD), the best iterate over the learning process will achieve $(\\sqrt{\\epsilon}/\\sqrt{\\delta n}, \\delta)$ weak Nash-equilibrium, meaning that there are $\\delta$ fraction of the players having low NE gap. The authors also show that under certain condition of the game, CGD, a variant of OGD, performs better social welfare guarantees.\n\nSTRENGTHS:\n- The paper considers a certain type of game with good rPoA guarantee and shows that classic OGD algorithm performs good results with provable guarantees.\n- The general idea of the proof is intuitive and clear to me. The analysis also looks correct to me in general.\n\nWEAKNESSES:\n- One issue is the motivation of the study and the novelty of the proposed analysis. Specifically, it is less clear why the class of game with $rPoA\\geq 1-\\epsilon$ is an interesting class of games. It seems to me that this is more about a technical assumption that makes the proof work. More concretely, with respect to the first result showing the convergence to Nash, the analysis follows the recent line of works about last-iterate convergence and constant individual regret bound in general-sum games using OGD/OFTRL, which show that lower bounded sum-of-regret leads to stability of the dynamic, which furthre leads to low NE gap guarantees. The rPoA condition basically guarantees that the sum-of-regret is lower bounded according to the smoothness condition of the game.\n- Another related issue is the trackability of rPoA. As mentioned mentioned by the authors, while rPoA can be calculated within polynomial time if the game is explicitly represented, meaning that the game representation already takes exponential space with respect to the number of players, which may not be the case when the utility function for each player is define by certain concrete functions. Therefore, deciding whether rPoA is lower bounded or not is also computationally intractable when the number of players is large.\n- The description of Observation 3.3 does not look very clear. Specifically, the equivalence should define a certain choice of $(\\lambda, \\mu)$ in the definition of smooth game and I do not find a proof for this observation.\n- The notations used in this paper are not very clear. \n    - In section 3, many notations are with superscript $n$, which would be better if they are replaced by $*^{(n)}$ in order to distinguish with the exponent notation. Specifically, in line 194, $rPoA\\geq 1-\\epsilon^n$ is very misleading.\n    - In line 302, should $rPoA_G^{\\epsilon}$ be replaced by $PoA_G^{\\epsilon}$, since $rPoA$ is not defined with respect to NE.\n    - In line 324, the guarantee should be $rPoA_G\\cdot OPT$. Specifically, in line below line 926, the RHS should also include $OPT$.\n\nQUESTIONS:\n- Can the authors explain more on the class of game with $rPoA\\geq 1-\\epsilon$? Is there a certain class of previously studied games or real applications that satisfy this condition?\n- The obtained results for good NE gap strategy is with respect to the best-iterate. I wonder whether the results may also extend to the average-iterate case, which may lead to less computational complexity to derive the output strategy?\n\nLIMITATIONS:\nSee weakness and questions for details.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the convergence properties of no-regret dynamics in $(\\lambda,\\mu)$-smooth games. More precisely the authors show that for any $(\\lambda,\\mu)$-smooth game at which the bound of $\\frac{\\lambda}{1+\\mu}$ converges to $1$ as the number of agents grows, the resulting Online Gradient Descent dynamics converges in polynomial time to an $(\\epsilon,\\delta)$-weak Nash Equilibrium. In an $(\\epsilon,\\delta)$-weak NE, $(1-\\delta)$-fraction of the agents cannot increase their payoff by more that $\\epsilon$ by deviating to another mixed strategy. The latter implies that if $\\frac{\\lambda}{1-\\mu} = 1$ then the resulting OGD dynamics converges to an $\\epsilon$-MNE in $O(1/\\epsilon^2)$ steps. The latter reveals an interesting phase transition on the computational complexity of equilibrium computation of general multi-player games with PoA = 1 and the smaller class of $(\\lambda,\\mu)$-smooth games with $\\frac{\\lambda}{1-\\mu} = 1$ (this naturally implies PoA = 1). \n\nMotivated by the work of Roughgarden et al. establishing that no-regret dynamics in $(\\lambda,\\mu)$-smooth games guarantees $\\frac{\\lambda}{1-\\mu}$ fraction of the optimal payoff, the authors show that the latter result can be improved for a specific class of $(\\lambda,\\mu)$-smooth satisfying an additional condition (Condition 4). More precisely, the authors show that if all agents adopt Clairvoyant Gradient Descent then the time-average joint probability distribution converges to a CCE while there exists an iteration at which the respective mixed strategy profile attains strictly more than $\\frac{\\lambda}{1+\\mu}$ fraction of the optimal payoff.\n\nSTRENGTHS:\nI believe that the convergence results to an $(\\epsilon,\\delta)$-weak NE in $(\\lambda,\\mu)$-smooth games is a solid contribution. I also find surprising the phase transition on the computational complexity of equilibrium computation of games with PoA = 1 (that are PPAD hard) and the respective complexity of $(\\lambda,\\mu)$-smooth games with $\\frac{\\lambda}{1+\\mu} = 1$. Also I find the second presented result, improving upon the $\\frac{\\lambda}{1+\\mu}$ fraction of the optimal social welfare, interesting. I overall believe that this is a good paper aligned with the interest of the online learning/game theory audience of NeuRIPS.\n\nWEAKNESSES:\nIn my opinion the only weakness of the paper concerns the comparison of the second result with the respective previous result of Roughgarden et al. As far as I understand, Roughgarden et al. show that the time-average payoff attained by any no-regret dynamics (in an $(\\lambda,\\mu)$-smooth game) is at least $\\frac{\\lambda}{1+ \\mu}$ fraction of the optimal social welfare. However Theorem~4.2 guarantees social welfare strictly more than $\\frac{\\lambda}{1+ \\mu}$ fraction the optimal one only at a specific iteration. \n\nThat being said I still consider the result interesting. However in case I am not missing something, an additionally discussion is needed so as to fairly compare the two results.\n\nQUESTIONS:\nYour second results refers to the $best-iterate$ while the result of Roughgarden et al. refets to the time-average payoff. Could you elaborate more on how the two results compare?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses the convergence and welfare of learning algorithms in smooth games. The authors show that when approximate full efficiency can be guaranteed via a smoothness argument, Nash equilibria are approachable under a family of no-regret learning algorithms, thereby guaranteeing fast and decentralized computation. They also leverage this connection to obtain new convergence results in large games, as well as extensions to Bayesian mechanisms. The paper unifies recent works in two-player zero-sum games, illuminating an equivalence between smoothness and a well-studied condition in the optimization literature known as the Minty property. Finally, the authors establish that a family of no-regret learning dynamics outperforms the welfare predicted by the smoothness framework under a generic condition, while at the same time guaranteeing convergence to the set of coarse correlated equilibria.\n\nSTRENGTHS:\nThe strength of this paper lies in its contribution to the understanding of the convergence and welfare of learning algorithms in smooth games. The authors provide new insights into the relationship between efficiency and the behavior of a family of no-regret learning algorithms, and they also unify recent works in two-player zero-sum games.\n\nWEAKNESSES:\nThe paper does have quite a few notations, but it's understandable. I've got a few questions, which might also point out some areas in the paper that could be improved. I've put these questions and potential weaknesses together in the question section for easy reference. But, I think this is an overall well-written paper.\n\nQUESTIONS:\n1. Is there any counterexample that if rPOG does not converge to 1, then the decentralized algorithm (OGD) provided in this paper does not converge to Nash Equilibrium? or If Condition 4.1 does not hold, then still thm 4.2.1 hold?\n2. What is the indication of Condition 4.1? Are there some examples that condition 4.1 holds?\n\nLIMITATIONS:\nThe authors adequately addressed the limitation and potential negative societal impact on their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the connections between the efficiency of Nash equilibria, as measured for example by social welfare, and tractability of computing these equilibria through efficient no regret learning algorithms. The authors provide the key insight that the smoothness framework introduced by Roughgarden, that is typically used to analyze price of anarchy, can be also leveraged to analyze the efficient convergence of optimistic gradient descent (OGD) to Nash equilibria. They use this insight to prove that if as the number of players increases, the robust price of anarchy goes to 1 without increasing the games Lipschitz constant too quickly, then OGD converges to weak Nash equilibria. This finding unifies previous findings on zero sum games and is connected to existing work on the Minty property. Even beyond OGD, the authors study the connections between convergence to CCE and social welfare for clairvoyant GD (CGD). They show for the first time that convergence to CCE and outperforming restricted price of anarchy can be combined through an efficient algorithm.\n\nSTRENGTHS:\nOverall I believe that the paper makes a very strong and original contribution. To the best of my knowledge, connections between efficiency of equilibria and tractability have not received a lot of attention. This is evidenced by the fact that relatively simple observations like 3.3 are commonly overlooked. In addition to the novelty of this works topic, the technical results are very extensive and non trivial.\n\nWEAKNESSES:\nThe paragraph from 179 to 190 could benefit from expansion. In the common setting where we analyze a fixed game, one can always normalize the utilities to get a Lipschitz constant of 1. I have a sense that such a normalization would affect the value of $\\epsilon$ in the general case thus we cannot always get $\\gamma$ to be small for free. I am not sure though. Adding more intuition would help.\n\nQUESTIONS:\nSee above for a question/suggestion.\n\nLIMITATIONS:\nNo limitations applicable.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

comments: SUMMARY:
This work studies the connection between efficiency and computation tractability of equilibrium in smooth games. Its major finding is that optimistic mirror descent reaches a weak Nash equilibrium for large games satisfying the property that asymptotically smoothness can guarantee efficiency of equilibria. This result establishes the connection between smoothness arguments and the approachability of no-regret learning algorithms to Nash equilibria.

STRENGTHS:
This work draws new insights on an interesting angle connecting two fundamental subjects in algorithmic game theory. Robust price of anarchy characterizes the fraction of optimal social welfare that smoothness argument can guarantee for every Nash equilibria. It turns out from this paper that robust PoA $\to 1$ also ensures the convergence of no-regret dynamics, which is non-trivial. The results also recover existing theories of the covergence of OMD in two-player zero-sum games.

WEAKNESSES:
The rates (e.g. $n^{-\alpha/3}$) are dependent on the number of players instead of the number of iterations, which is different from what we commonly see/expect. I am not sure how useful the rates are, for example, in large games, still a $n^{2/3}$ number of players may yield bad responses.

QUESTIONS:
What are some natural examples where the conditions in Thm 3.1 are satisfied and non-trivial rates are obtained?

Is there direct generalization to mean-field games?

LIMITATIONS:
The authors adequately addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06