SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[100] Brian Hu Zhang and Tuomas Sandholm. Polynomial-time optimal equilibria with a mediator in extensive-form games. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
[101] Brian Hu Zhang, Gabriele Farina, Andrea Celli, and Tuomas Sandholm. Optimal correlated equilibria in general-sum extensive-form games: Fixed-parameter algorithms, hardness, and two-sided column-generation. In ACM Conference on Economics and Computation (EC), pages 1119-1120. ACM, 2022.
[102] Brian Hu Zhang, Gabriele Farina, Ioannis Anagnostides, Federico Cacciamani, Stephen Marcus McAleer, Andreas Alexander Haupt, Andrea Celli, Nicola Gatti, Vincent Conitzer, and Tuomas Sandholm. Steering no-regret learners to optimal equilibria. CoRR, abs/2306.05221, 2023.
[103] Brian Hu Zhang, Gabriele Farina, and Tuomas Sandholm. Team belief DAG: generalizing the sequence form to team games for fast computation of correlated team max-min equilibria via regret minimization. In International Conference on Machine Learning (ICML), volume 202 of Proceedings of Machine Learning Research, pages 40996-41018. PMLR, 2023.
[104] Hugh Zhang. A simple adaptive procedure converging to forgiving correlated equilibria. CoRR, abs/2207.06548, 2022.
[105] Stephan Zheng, Alexander Trott, Sunil Srinivasa, David C. Parkes, and Richard Socher. The ai economist: Taxation policy design via two-level deep multiagent reinforcement learning. Science Advances, 8(18):eabk2607, 2022.
[106] Martin Zinkevich, Michael Bowling, Michael Johanson, and Carmelo Piccione. Regret minimization in games with incomplete information. In Conference on Neural Information Processing Systems (NeurIPS), 2007.
A Illustrative examples of mediator-augmented games
In this section, we further clarify the framework of mediator-augmented games we operate in through a couple of examples. We begin by noting that the family of solution concepts for extensive-form games captured by this framework includes, but is not limited to, the following:
- normal-form coarse correlated equilibrium* [5, 79],
- extensive-form coarse correlated equilibrium* [31],
- extensive-form correlated equilibrium* [98],
- certification (under the nested range condition [46, 37]) [37, 100],
- communication equilibrium [81, 36],
- mechanism design for sequential settings, and
- information design/Bayesian persuasion for sequential settings [58].
We refer the interested reader to Zhang and Sandholm [100, Appendix G] for additional interesting concepts not mentioned above.
Example A.1 (Single-item auction). Consider the single-good monopolist problem studied by Myerson [80]. Each player  $i \in [[n]]$  has a valuation  $v_{i} \in V_{i}$ . Agent valuations may be correlated, and distributed according to  $\mathcal{F} \in \Delta(V)$ , where  $V := V_{1} \times V_{2} \times \dots \times V_{n}$ . The mechanism selects a (potentially random) payment  $p$  and a winner  $i^{*} \in [[n]]$ . The agents' utilities are quasilinear:  $u_{i}(i^{*}, p; v_{i}) = v_{i} - p$  if  $i^{*} = i$  and 0 otherwise. The seller wishes to maximize expected payment from the agents. This has the following timeline.
1. The mechanism commits to a (potentially randomized) mapping  $\phi : \pmb{v} = (v_{1},\dots,v_{n}) \mapsto (i^{*},p)$ .
2. Nature samples valuations  $\pmb{v} = (v_{1},\dots ,v_{n})\sim \mathcal{F}$
3. Each player  $i \in [[n]]$  privately observes her valuation  $v_i$ , and then decides what valuation  $v_i'$  to report to the mediator.
4. The winner and payment are selected according to  $\phi (\pmb {v}^{\prime})$
5. Player  $i^{*}$  gets utility  $v_{i^{*}} - p$ , while all other players get 0. The mediator obtains utility  $u_{0} = p$ .
In this extensive-form game, the primitives from our paper are:
- a (pure) mediator strategy  $\pmb{\mu} \in \Xi$  is a mapping from valuation reports  $\pmb{v}' = (v_1', \dots, v_n')$  to outcomes  $(i^*, p)$ —that is, mediator strategies are mechanisms, and mixed strategies are randomized mechanisms;
- a (pure) player strategy  $\pmb{x}_i \in X_i$  for each player  $i \in [[n]]$  is a mapping from  $V_i$  to  $V_i$  indicating what valuation Player  $i$  reports as a function of its valuation  $v_i \in V_i$ ;
- the direct (in mechanism design language, truthful) strategy  $\pmb{d}_i$  for each player  $i$  is the identity map from  $V_i$  to  $V_i$ . (Hence, in particular,  $\pmb{d}_i \in X_i$  is a strategy of Player  $i$ , so it makes sense, for example, to call  $(\pmb{\mu}, \pmb{d}) = (\pmb{\mu}, \pmb{d}_1, \dots, \pmb{d}_n)$  a strategy profile.)
In particular, if profile  $(\pmb{\mu}, \pmb{d}_1, \dots, \pmb{d}_n)$  is such that each player  $i$  is playing a best response, then  $\pmb{\mu}$  is a truthful mechanism.
The conversion in Proposition 3.2 creates a zero-sum extensive-form game  $\hat{\Gamma}$ , whose equilibria for the mediator (for sufficiently large  $\lambda$ ) are precisely the revenue-maximizing mechanisms.  $\hat{\Gamma}$  has the following timeline:
*For notions of correlated equilibrium in extensive-form games, the mediator must have imperfect recall, and therefore the representation of the mediator's decision space  $\Xi$  may not be polynomial. This is unavoidable, since the problem of computing an optimal equilibrium under these notions is NP-hard in general [98]. In this paper, we will largely ignore these concerns and assume that the representation of the mixed strategy set  $\Xi$  is part of the input.
1. Nature picks, with equal probability, whether there is a deviator. If nature picks that there is a deviator, nature also selects which player  $i \in [[n]]$  is represented by the deviator.
2. Nature samples valuations  $\pmb{v} = (v_{1},\dots ,v_{n})\sim \mathcal{F}$
3. If nature selected that there is a deviator, the deviator observes  $i$  and its valuation  $v_{i} \in V_{i}$ , and selects a deviation  $v_{i}' \in V_{i}$ .
4. The mediator observes  $(v_{i}^{\prime},\pmb{v}_{-i})$  (i.e., all other players are assumed to have reported honestly) and selects a winner  $i^{*}$  and payment  $p$ , as before.
5. There are now two cases. If nature selected at the root that there was to be a deviator, the utility for the mediator is  $-2\lambda nu_i(i^*, p; v_i)$ . If nature selected at the root that there was to be no deviator, the utility for the mediator is  $2p + 2\lambda \sum_{i=1}^{n} u_i(i^*, p; v_i) = 2p + 2\lambda (v_{i^*} - p)$ .
As our second example, we show how the problem of computing a social-welfare-maximizing correlated equilibrium (CE) in a normal-form game can be captured using mediator-augmented games. Example A.2 (Social welfare-optimal correlated equilibria in normal-form games). Let  $A_{i}$  be the action set for each player  $i\in \llbracket n\rrbracket$  in the game, and let utility functions  $u_{i}:A\to \mathbb{R}$ , where  $A\coloneqq A_1\times A_2\times \dots \times A_n$ . The social welfare is the function  $u_{0}:A\to \mathbb{R}$  given by  $u_{0}(\pmb {a})\coloneqq \sum_{i = 1}^{n}u_{i}(\pmb {a})$ . In the traditional formulation, a CE is a correlated distribution  $\pmb{\mu}$  over  $A$ . The elements  $(a_{1},\ldots ,a_{n})$  sampled from  $\pmb{\mu}$  can be thought of as profiles of action recommendations for the players such that no player has any incentive to not follow the recommendation (obedience). This has the following well-known timeline.
1. At the beginning of the game, the mediator player chooses a profile of recommendations  $\mathbf{a} = (a_{1},\dots ,a_{n})\in A$
2. Each player observes its recommendation  $a_{i}$  and chooses an action  $a_{i}'$ .
3. Each player gets utility  $u_{i}(\pmb{a}^{\prime})$ , and the mediator gets utility  $u_{0}(\pmb{a}^{\prime}) = \sum_{i=1}^{n} u_{i}(\pmb{a}^{\prime})$ .
In this game:
- mixed strategies  $\pmb{\mu}$  for the mediator are distributions over  $A$ , that is, they are correlated profiles;
- a (pure) strategy for player  $i \in [[n]]$  is a mapping from  $A_i$  to  $A_i$ , encoding the action player  $i \in [[n]]$  will take upon receiving each recommendation;
- the direct strategy  $d_{i}$  is again the identity map (i.e., each player selects as action what the mediator recommended to him/her).
In particular, if profile  $(\pmb{\mu}, \pmb{d}_1, \dots, \pmb{d}_n)$  is such that each player  $i$  is playing a best response, then  $\pmb{\mu}$  is a CE.
Proposition 3.2 yields the following zero-sum game whose mediator equilibrium strategies (for sufficiently large  $\lambda$ ) are precisely the welfare-optimal equilibria:
1. Nature picks, with equal probability, whether there is a deviator. If nature picks that there is a deviator, nature also selects which player  $i \in [[n]]$  is represented by the deviator.
2. The mediator picks a pure strategy profile  $\pmb{a} = (a_{1},\dots ,a_{n})\in A$
3. If there is a deviator, the deviator observes  $i \in [[n]]$  and the recommendation  $a_i$  and picks an action  $a_i'$ .
4. There are now two cases. If nature selected at the root that there was to be a deviator, the utility for the mediator is  $-2\lambda nu_i(a_i', \mathbf{a}_{-i})$ . If nature selected at the root that there was to be no deviator, the utility for the mediator is  $2u_0(\mathbf{a}) + 2\lambda \sum_{i=1}^{n} u_i(\mathbf{a}) = 2(1 + \lambda)u_0(\mathbf{a})$ .
B Further related work
In this section, we provide additional related work omitted from the main body.
We first elaborate further on prior work regarding the complexity of computing equilibria in games. Much attention has been focused on the complexity of computing just any one Nash equilibrium. This has been motivated in part by the idea that if even this is hard to compute, then this casts doubt on the concept of Nash equilibrium as a whole [23]; but the interest also stemmed from the fact that the complexity of the problem was open for a long time [82], and ended up being complete for an exotic complexity class [23, 17], whereas computing a Nash equilibrium that reaches a certain objective value is "simply" NP-complete [41, 21]. None of this, however, justifies settling for just any one equilibrium in practice. Moreover, for correlated equilibria and related concepts, the complexity considerations are different. While one (extensive-form) correlated equilibrium can be computed in polynomial time even for multi-player succinct games [83, 56, 54] (under the polynomial expectation property), computing one that maximizes some objective function is typically NP-hard [83, 98]. To make matters worse, even finding one that is strictly better—in terms of social welfare—than the worst one is also computationally intractable [9]. Of course, our results do not contradict those lower bounds. For example, in multi-player normal-form games the strategy space of the mediator has an exponential description, thereby rendering all our algorithms exponential in the number of players. We stress again that while there exist algorithms that avoid this exponential dependence, they are not guaranteed to compute an optimal equilibrium, which is the main focus of this paper.
Moreover, in our formulation the mediator has the power to commit to a strategy. As such, our results also relate to the literature on learning and computing Stackelberg equilibria [8, 35, 66, 84, 20], as well as the work of Camara et al. [15] which casts mechanism design as a repeated interaction between a principal and an agent. Stackelberg equilibria in extensive-form games are, however, hard to find in general [65]. Our Stackelberg game has a much nicer form than general Stackelberg games—in particular, we know in advance what the equilibrium strategies will be for the followers (namely, the direct strategies, ). This observation is what allows the reduction to zero-sum games, sidestepping the need to use Stackleberg-specific technology or solvers and resulting in efficient algorithms.
In an independent and concurrent work, Fujii [40] provided independent learning dynamics converging to the set of communication equilibria in Bayesian games, but unlike our algorithm there are no guarantees for finding an optimal one. Also in independent and concurrent work, Ivanov et al. [55] develop similar Lagrangian-based dynamics for the equilibrium notion that, in the language of this paper and Zhang and Sandholm [100], is coarse full-certification equilibrium. Differing from ours, their paper does not present any theoretical guarantees (instead focusing on practical results).
C Proof of Proposition 3.1
In this section, we provide the proof of Proposition 3.1, the statement of which is recalled below.
Proof. Let  $v$  be the optimal value of (G). The Lagrangian of (G) is
$$
\max_{\boldsymbol {\mu}\in \Xi}\min_{\substack{\lambda_{i}\in \mathbb{R}_{\geq 0},\\ \boldsymbol{x}_{i}\in X_{i}:i\in [[n]]}}\quad \boldsymbol{c}^{\top}\boldsymbol {\mu} - \sum_{i = 1}^{n}\lambda_{i}\boldsymbol{\mu}^{\top}\mathbf{A}_{i}\boldsymbol{x}_{i}.
$$
Now, making the change of variables  $\bar{x}_i\coloneqq \lambda_i x_i$  , the above problem is equivalent to
$$
\max  _ {\boldsymbol {\mu} \in \Xi} \min  _ {\bar {x} _ {i} \in \bar {X} _ {i}: i \in [ n ]} \quad \boldsymbol {c} ^ {\top} \boldsymbol {\mu} - \sum_ {i = 1} ^ {n} \boldsymbol {\mu} ^ {\top} \mathbf {A} _ {i} \bar {\boldsymbol {x}} _ {i}. \tag {1}
$$
where  $\bar{X}_i$  is the conic hull of  $X_i$ :  $\bar{X}_i \coloneqq \{\lambda_i \pmb{x}_i : \pmb{x}_i \in X_i\}$ . Note that, when  $X_i$  is a polytope of the form  $X_i \coloneqq \{\mathbf{F}_i \pmb{x}_i = \pmb{f}_i, \pmb{x}_i \geq 0\}$ , its conic hull can be expressed as  $\bar{X}_i = \{\mathbf{F}_i \pmb{x}_i = \lambda_i \pmb{f}_i, \pmb{x}_i \geq 0, \lambda_i \geq 0\}$ . Thus, (1) is a bilinear saddle-point problem, where  $\Xi$  is compact and convex and  $\bar{X}_i$  is
convex. Thus, Sion's minimax theorem [92] applies, and we have that the value of (1) is equal to the value of the problem
$$
\min  _ {\bar {\boldsymbol {x}} _ {i} \in \bar {X} _ {i}: i \in [ n ]} \max  _ {\boldsymbol {\mu} \in \Xi} \quad \boldsymbol {c} ^ {\top} \boldsymbol {\mu} - \sum_ {i = 1} ^ {n} \boldsymbol {\mu} ^ {\top} \mathbf {A} _ {i} \bar {\boldsymbol {x}} _ {i}. \tag {2}
$$
Since this is a linear program with a finite value, its optimum value must be achieved by some  $\bar{\pmb{x}}\coloneqq (\bar{\pmb{x}}_1,\dots ,\bar{\pmb{x}}_n)\coloneqq (\lambda_1\pmb {x}_1,\dots ,\lambda_n\pmb {x}_n)$ . Let  $\lambda^{*}\coloneqq \max_{i}\lambda_{i}$ . Using the fact that  $\pmb {\mu}^{\top}\mathbf{A}_{i}\pmb{d}_{i} = 0$  for all  $\pmb{\mu}$ , the profile
$$
\bar {\boldsymbol {x}} ^ {\prime} := \left(\lambda^ {*} \boldsymbol {x} _ {1} ^ {\prime}, \dots , \lambda^ {*} \boldsymbol {x} _ {n} ^ {\prime}\right) \quad \text {w h e r e} \quad \boldsymbol {x} _ {i} ^ {\prime} = \boldsymbol {d} _ {i} + \frac {\lambda_ {i}}{\lambda^ {*}} (\boldsymbol {x} _ {i} - \boldsymbol {d} _ {i})
$$
is also an optimal solution in (2). Therefore, for any  $\lambda \geq \lambda^{*}$ ,  $\pmb{x}^{\prime} := (\pmb{x}_{1}^{\prime},\dots,\pmb{x}_{n}^{\prime})$  is an optimal solution for the minimizer in (L1) that achieves the value of (G), so (G) and (L1) have the same value.
Now take  $\lambda >\lambda^{*}$ , and suppose for contradiction that (L1) admits some optimal  $\pmb {\mu}\in \Xi$  that is not optimal in (G). Then, either  $c^{\top}\pmb {\mu} < v$ , or  $\pmb{\mu}$  violates some constraint  $\max_{\pmb {x}_i}\pmb{\mu}^\top \mathbf{A}_i\pmb{x}_i\leq 0$ . The first case is impossible because then setting  $\pmb {x}_i = \pmb {d}_i$  for all  $i$  yields value less than  $v$  in (L1). In the second case, since we know that (L1) and (G) have the same value when  $\lambda = \lambda^{*}$ , we have
$$
\boldsymbol {c} ^ {\top} \boldsymbol {\mu} - \lambda \max  _ {\boldsymbol {x} \in X} \sum_ {i = 1} ^ {n} \boldsymbol {\mu} ^ {\top} \mathbf {A} _ {i} \boldsymbol {x} _ {i} <   \boldsymbol {c} ^ {\top} \boldsymbol {\mu} - \lambda^ {*} \max  _ {\boldsymbol {x} \in X} \sum_ {i = 1} ^ {n} \boldsymbol {\mu} ^ {\top} \mathbf {A} _ {i} \boldsymbol {x} _ {i} \leq v.
$$
D Fast computation of optimal equilibria via regret minimization
In this section, we focus on the computational aspects of solving the induced saddle-point problem (L1) using regret minimization techniques. In particular, this section serves to elaborate on our results presented earlier in Section 3.1. All of the omitted proofs are deferred to Appendix E for the sake of exposition.
As we explained in Section 3.1, the first challenge that arises in the solution of (L1) is that the domain of Player min is unbounded—the Lagrange multiplier is allowed to take any nonnegative value. Nevertheless, we show in the theorem below that it suffices to set the Lagrange multiplier to a fixed value (that may depend on the time horizon); we reiterate that appropriately setting that value will allow us to trade off between the equilibrium gap and the optimality gap. Before we proceed, we remark that the problem of Player min in (L1) can be decomposed into the subproblems faced by each player separately, so that the regret of Player min can be cast as the sum of the players' regrets (see Corollary E.2); this justifies the notation  $\sum_{i=1}^{n} \operatorname{Reg}_{X_i}^T$  used for the regret of Player min below.
Theorem D.1. Suppose that Player max in the saddle-point problem (L1) incurs regret  $\mathrm{Reg}_{\Xi}^{T}$  and Player min incurs regret  $\sum_{i=1}^{n} \mathrm{Reg}_{X_i}^{T}$  after  $T \in \mathbb{N}$  repetitions, for a fixed  $\lambda = \lambda(T) > 0$ . Then, the average mediator strategy  $\Xi \ni \bar{\mu} \coloneqq \frac{1}{T} \sum_{t=1}^{T} \boldsymbol{\mu}^{(t)}$  satisfies the following:
1. For any strategy  $\pmb{\mu}^{*}\in \Xi$  such that  $\max_{i\in \llbracket n\rrbracket}\max_{\pmb{x}_i^*\in X_i}(\pmb{\mu}^*)^\top \mathbf{A}_i\pmb{x}_i^*\leq 0,$
$$
\boldsymbol {c} ^ {\top} \bar {\boldsymbol {\mu}} \geq \boldsymbol {c} ^ {\top} \boldsymbol {\mu} ^ {*} - \frac {1}{T} \left(\operatorname {R e g} _ {\Xi} ^ {T} + \sum_ {i = 1} ^ {n} \operatorname {R e g} _ {X _ {i}} ^ {T}\right);
$$
2. The equilibrium gap of  $\bar{\mu}$  decays with a rate of  $\lambda^{-1}$ :
$$
\max  _ {i \in [   [ n ]   ]} \max  _ {\boldsymbol {x} _ {i} ^ {*} \in X _ {i}} \bar {\boldsymbol {\mu}} ^ {\top} \mathbf {A} _ {i} \boldsymbol {x} _ {i} ^ {*} \leq \frac {\max  _ {\boldsymbol {\mu} , \boldsymbol {\mu} ^ {\prime} \in \Xi} \boldsymbol {c} ^ {\top} (\boldsymbol {\mu} - \boldsymbol {\mu} ^ {\prime})}{\lambda} + \frac {1}{\lambda T} \left(\operatorname {R e g} _ {\Xi} ^ {T} + \sum_ {i = 1} ^ {n} \operatorname {R e g} _ {X _ {i}} ^ {T}\right).
$$
As a result, if we can simultaneously guarantee that  $\lambda (T)\to +\infty$  and  $\frac{1}{T}\left(\mathrm{Reg}_{\Xi}^{T} + \sum_{i = 1}^{n}\mathrm{Reg}_{X_{i}}^{T}\right)\rightarrow 0$ , as  $T\to +\infty$ , Theorem D.1 shows that both the optimality gap (Item 1) and the equilibrium gap
9This holds by taking a dual of the inner minimization.
(Item 2) converge to 0. We show that this is indeed possible in the sequel (Corollaries 3.3 and 3.4), obtaining favorable rates of convergence as well.
It is important to stress that while there exists a bounded critical Lagrange multiplier for our problem (Proposition 3.1), thereby obviating the need for truncating its value, such a bound is not necessarily polynomial. For example, halving the players' utilities while maintaining the utility of the mediator would require doubling the magnitude of the critical Lagrange multiplier.
Next, we combine Theorem D.1 with suitable regret minimization algorithms in order to guarantee fast convergence to optimal equilibria. Let us first focus on the side of Player min in (L1), which, as pointed out earlier, can be decomposed into subproblems corresponding to each player separately (Corollary E.2). Minimizing regret over the sequence-form polytope can be performed efficiently with a variety of techniques, which can be classified into two basic approaches. The first one is based on the standard online mirror descent algorithm (see, e.g., [91]), endowed with appropriate distance generating functions (DGFs) [32]. The alternative approach is based on regret decomposition, in the style of CFR [106, 29]. In particular, given that the players' observed utilities have range  $O(\lambda)$ , the regret of each player under suitable learning algorithms will grow as  $O(\lambda \sqrt{T})$  (see Proposition E.1). Furthermore, efficiently minimizing regret from the side of the mediator depends on the equilibrium concept at hand. For NFCCE, EFCCE and EFCE, the imperfect recall of the mediator [101] induces a computationally hard problem [19], which nevertheless admits fixed-parameter tractable algorithms [103] (Proposition E.3). In contrast, for communication and certification equilibria the perfect recall of the mediator enables efficient computation for any extensive-form game. As a result, selecting a bound of  $\lambda \coloneqq T^{1/4}$  on the Lagrange multiplier, so as to optimally trade off Items 1 and 2 of Theorem D.1, leads to the following conclusion.
Furthermore, we leverage the technique of optimism, pioneered by Chiang et al. [18], Rakhlin and Sridharan [87], Syrgkanis et al. [94] in the context of learning in games, in order to obtain faster rates. In particular, using optimistic mirror descent we can guarantee that the sum of the agents' regrets in the saddle-point problem (L1) will now grow as  $O(\lambda)$  (Proposition E.4), instead of the previous bound  $O(\lambda \sqrt{T})$  obtained using vanilla mirror descent. Thus, letting  $\lambda = T^{1/2}$  leads to the following improved rate of convergence.
We reiterate that while this rate is slower than the (near)  $T^{-1}$  rates known for converging to some of those equilibria [24, 34, 85, 3], Corollaries 3.3 and 3.4 additionally guarantee convergence to optimal equilibria; improving the  $T^{-1/2}$  rate of Corollary 3.4 is an interesting direction for the future.
Last-iterate convergence The results we have stated thus far apply for the average strategy of the mediator—a typical feature of traditional guarantees in the no-regret framework. In contrast, there is a recent line of work that endeavors to recover last-iterate guarantees as well [22, 45, 1, 14, 6, 99, 64, 43, 67, 44]. Yet, despite many efforts, the known last-iterate guarantees of no-regret learning algorithms apply only for restricted classes of games, such as two-player zero-sum games. There is an inherent reason for the limited scope of those results: last-iterate convergence is inherently tied to Nash equilibria, which in turn are hard to compute in general games [23, 17]—let alone computing an optimal one [41, 21]. Indeed, any given joint strategy profile of the players induces a product distribution, so iterate convergence requires—essentially by definition—at the very least computing an approximate Nash equilibrium.
Proposition D.2 (Informal). Any independent learning dynamics (without a mediator) require superpolynomial time to guarantee  $\varepsilon$ -last-iterate convergence, for a sufficiently small  $\varepsilon = O(m^{-c})$ , even for two-player  $m$ -action normal-form games, unless  $\mathsf{PPAD} \subseteq \mathsf{P}$ .
There are also unconditional exponential communication-complexity lower bounds for uncoupled methods [7, 53, 89, 48], as well as other pertinent impossibility results [50, 74] that document the inherent persistence of limit cycles in general-sum games. In contrast, an important advantage of our mediator-augmented formulation is that we can guarantee last-iterate convergence to optimal equilibria in general games. Indeed, this follows readily from our reduction to two-player zero-sum games, for which the known bound of  $O(\lambda / \sqrt{T})$  for the iterate gap of (online) optimistic gradient descent can be employed (see Appendix E).
As such, our mediator-augmented learning paradigm bypasses the hardness of Proposition D.2 since last-iterate convergence is no longer tied to convergence to Nash equilibria.
E Omitted proofs from Section D
In this section, we provide the omitted proofs from Appendix D, which concerns the solution of the saddle-point problem described in (L1) using regret minimization. Appendix E.1 then presents a slightly different approach for solving (L1) using regret minimization over conic hulls, which is used in our experiments.