SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 4.6. Let  $G$  be a CSP game. If  $G$  is  $(0, \gamma)$ -subgame stable, then for any player  $i \in N$  and  $CCE \mu$  of  $G$ , we have  $\mathrm{Vul}_i(s^\mu, S_{-i}) \leq |E_i| \gamma$ .
Theorem 4.6 tells us that using self-play to compute a marginal strategy  $s^{\mu}$  on constant-sum polymatrix games will have low vulnerability against worst-case opponents if  $\gamma$  is low. Thus, these are a set of multiplayer games where self-play is an effective training procedure.
Proof idea. Since  $G$  is a CSP game,  $s^{\mu}$  is a Nash equilibrium. Since  $G$  is  $(0, \gamma)$ -subgame stable,  $(s_i^\mu, s_j^\mu)$  is a  $\gamma$ -Nash equilibrium in each subgame, which bounds the vulnerability in each subgame. This is because
$$
\min  _ {s - i \in S _ {- i}} u _ {i} \left(s _ {i} ^ {\mu}, s _ {- i}\right) = \sum_ {(i, j) \in E _ {i}} \min  _ {s _ {j} \in S _ {j}} u _ {i j} \left(s _ {i} ^ {\mu}, s _ {j}\right)
$$
since players  $j \neq i$  can minimize  $i$ 's utility without coordinating, as  $G$  is a polymatrix game.
We give an algorithm for finding the minimum value of  $\gamma$  such that a CSP game is  $(0, \gamma)$ -subgame stable in Appendix D.1.
Figure 2: (a) Offense-Defense, a simple CSP game. We only show payoffs for the row player, column player payoffs are zero minus the row player's payoffs. (b) Bad Card: a game that is not overall CSP, but the subset of strategies learnable by self-play are. At the terminals, we show the dealers utility first, followed by players 0, 1 and 2, respectively.
4.3 Approximate Constant-Sum Polymatrix Games
Most games are not factorizable into CSP games. However, we can take any game  $G$  and project it into the space of CSP games.
Definition 4.7 ( $\delta$ -constant sum polymatrix). A game  $G$  is  $\delta$ -constant sum polymatrix ( $\delta$ -CSP) if there exists a CSP game  $\check{G}$  with global utility function  $\check{u}$  such that  $\forall i \in N, \rho \in \mathrm{P}, |u_i(\rho) - \check{u}_i(\rho)| \leq \delta$ . Given  $G$ , we denote the set of such CSP games as  $\mathrm{CSP}_{\delta}(G)$ .
Proposition 4.8. In a  $\delta$ -CSP game  $G$  the following hold.
1. Any  $CCE$  of  $G$  is a  $2\delta$ - $CCE$  of any  $\tilde{G} \in \mathrm{CSP}_{\delta}(G)$ .
2. The marginal strategy profile of any CCE of  $G$  is a 2nδ-Nash equilibrium of any  $\check{G} \in \mathrm{CSP}_{\delta}(G)$ .
3. The marginal strategy profile of any CCE of  $G$  is a  $2(n + 1)\delta$ -Nash equilibrium of  $G$ .
From (3) we have that the removal of the mediator impacts players utilities by a bounded amount in  $\delta$ -CSP games. We give a linear program in Appendix D.2 that will find the minimum  $\delta$  such that  $G$  is  $\delta$ -CSP and returns a CSP game  $\tilde{G} \in \mathrm{CSP}_{\delta}(G)$ .
Combining  $\delta$ -CSP with  $(\epsilon, \gamma)$ -subgame stability lets us bound vulnerability in any game.
Theorem 4.9. If  $G$  is  $\delta$ -CSP and  $\exists \check{G} \in \mathrm{CSP}_{\delta}(G)$  that is  $(2n\delta, \gamma)$ -subgame stable and  $\mu$  is a CCE of  $G$ , then
$$
\operatorname {V u l} _ {i} \left(s ^ {\mu}, S _ {- i}\right) \leq \left| E _ {i} \right| \gamma + 2 \delta \leq (n - 1) \gamma + 2 \delta .
$$
Theorem 4.9 shows that games which are close to the space of subgame stable CSP (SS-CSP) games are cases where the marginal strategies learned through self-play have bounded worst-case performance. This makes them suitable for any no-external-regret learning algorithm.
5 Vulnerability Against Other Self-Taught Agents
Theorem 4.9 bounds vulnerability in worst-case scenarios, where  $-i$  play any strategy profile to minimize  $i$ 's utility. In reality, however, each player  $j \in -i$  has their own interests and would only play a strategy that is reasonable under these own interests. In particular, what if each agent were also determining their own strategy via self-play in a separate training instance. How much utility can  $i$  guarantee themselves in this setup?
While no-external-regret learning algorithms converge to the set of CCE, other assumptions can be made with additional information about the type of regret being minimized. For example, no-external-regret learning algorithms will play strictly dominated strategies with vanishing probability and CFR will play dominated actions with vanishing probability (Gibson, 2014). These refinements can tighten our bounds, since the part of the game that no-regret learning algorithms converge to might be closer to a CSP game than the game overall.
Consider the game shown in Figure 2b, called "Bad Card". The game starts with each player except the dealer putting  $\beta /2$  into the pot. A dealer player  $d$  who always receives utility 0 regardless of the strategies of the other players—then selects a player from  $\{0,1,2\}$  to receive a "bad card", while the other two players receive a "good card". The player who receives the bad card has an option to fold, after which the game ends and all players receive their ante back. Otherwise if this player calls, the other two players can either fold or call. The pot of  $\beta$  is divided among the players with good cards who call. If one player with a good card calls, they win the pot of  $\beta$ . If both good card players call then they split the pot. If both players with good cards fold, then the player with the bad card wins the pot.
As we shall soon show, Bad Card does not have a CSP decomposition—in fact it does not have any polymatrix decomposition. Since Bad Card is an extensive-form game without chance, each pure strategy profile leads to a single terminal history. Let  $\mathrm{P}(z)$  be the set of pure strategy profiles that play to a terminal  $z$ . In order for Bad Card to be polymatrix, we would need to find subgame utility functions such that  $\forall \rho \in \mathrm{P}$ ,  $u_0(\rho) = u_{0,d}(\rho_0,\rho_d) + u_{0,1}(\rho_0,\rho_1) + u_{0,2}(\rho_0,\rho_2)$ . Equivalently, we
could write  $\forall z\in Z,\rho \in \mathrm{P}(z),u_0(z) = u_{0,d}(\rho_0,\rho_d) + u_{0,1}(\rho_0,\rho_1) + u_{0,2}(\rho_0,\rho_2)$  where  $Z$  is the set of terminals. A subset of these constraints results in an infeasible system of equations.
Consider the terminals in the subtree shown in Figure 2b:  $z^1 = (0, c, c, c)$ ,  $z^2 = (0, c, c, f)$ ,  $z^3 = (0, c, f, c)$  and  $z^4 = (0, c, f, f)$ . Let  $\rho_i^c$  be any pure strategy that plays  $c$  in this subtree and  $\rho_i^f$  be any strategy that plays  $f$  in this subtree for player  $i$ . In order for Bad Card to decompose into a polymatrix game we would need to solve the following infeasible system of linear equations:
$$
u _ {0} (z ^ {1}) = u _ {0, d} (\rho_ {0} ^ {c}, 0) + u _ {0, 1} (\rho_ {0} ^ {c}, \rho_ {1} ^ {c}) + u _ {0, 2} (\rho_ {0} ^ {c}, \rho_ {2} ^ {c}) = - \beta
$$
$$
u _ {0} (z ^ {2}) = u _ {0, d} (\rho_ {0} ^ {c}, 0) + u _ {0, 1} (\rho_ {0} ^ {c}, \rho_ {1} ^ {c}) + u _ {0, 2} (\rho_ {0} ^ {c}, \rho_ {1} ^ {f}) = - \beta
$$
$$
u _ {0} (z ^ {3}) = u _ {0, d} (\rho_ {0} ^ {c}, 0) + u _ {0, 1} (\rho_ {0} ^ {c}, \rho_ {1} ^ {f}) + u _ {0, 2} (\rho_ {0} ^ {c}, \rho_ {2} ^ {c}) = - \beta
$$
$$
u _ {0} (z ^ {4}) = u _ {0, d} (\rho_ {0} ^ {c}, 0) + u _ {0, 1} (\rho_ {0} ^ {c}, \rho_ {1} ^ {f}) + u _ {0, 2} (\rho_ {0} ^ {c}, \rho_ {2} ^ {f}) = \beta
$$
Thus, Bad Card is not a CSP game, although it is a  $\beta$ -CSP game. However, if we prune out dominated actions (namely, those in which a player folds after receiving a good card), the resulting game is indeed a 0-CSP game.
Let  $\mathcal{M}(\mathcal{A})$  be the set of mediated equilibria than an algorithm  $\mathcal{A}$  converges to in self-play. For example, if  $\mathcal{A}$  is a no-external-regret algorithm,  $\mathcal{M}(\mathcal{A})$  is the set of CCE without strictly dominated strategies in their support. Let  $S(\mathcal{A})\doteq \{s^{\mu}\mid (\mu ,(\Phi_i)_{i\in N})\in \mathcal{M}(\mathcal{A})\}$  be the set of marginal strategy profiles of  $\mathcal{M}(\mathcal{A})$ , and let  $S_{i}(\mathcal{A})\doteq \{s_{i}\mid s\in S(\mathcal{A})\}$  be the set of  $i$ 's marginal strategies from  $S(\mathcal{A})$ .
Now, consider if each player  $i$  learns with their own self-play algorithm  $\mathcal{A}_i$ . Let  $\mathcal{A}_N \doteq (\mathcal{A}_1, \ldots, \mathcal{A}_n)$  be the profile of learning algorithms, then let  $S^{\times}(\mathcal{A}_N) \doteq \times_{i \in N} S_i(\mathcal{A}_i)$  and  $S_{-i}^{\times}(\mathcal{A}_N) \doteq \times_{j \in -i} S_j(\mathcal{A}_j)$ . Summarizing, if each player learns with a no- $\Phi_i$ -regret learning algorithm  $\mathcal{A}_i$ , they will converge to the set of  $\mathcal{M}(\mathcal{A}_i)$  equilibria. The set of marginal strategies from this set of equilibria is  $S_i(\mathcal{A}_i)$  and the set of marginal strategy profiles is  $S(\mathcal{A}_i)$ . If each player plays a (potentially) different learning algorithm,  $S^{\times}(\mathcal{A}_N)$  is the set of possible joint match-ups if each player plays a marginal strategy from their own algorithm's set of equilibria and  $S_{-i}^{\times}(\mathcal{A}_N)$  is the set of profiles for  $-i$ .
Definition 5.1. We say a game  $G$  is  $\delta$ -CSP in the neighborhood of  $S' \subseteq S$  if there exists a CSP game  $\tilde{G}$  such that  $\forall s \in S'$  we have  $|u_i(s) - \check{u}_i(s)| \leq \delta$ . We denote the set of such CSP games as  $\mathrm{CSP}_{\delta}(G, S')$ .
Definition 5.2. We say a polymatrix game  $G$  is  $\gamma$ -subgame stable in the neighborhood of  $S'$  if  $\forall s \in S', \forall (i,j) \in E$  we have that  $(s_i, s_j)$  is a  $\gamma$ -Nash of  $G_{ij}$ .
These definitions allow us to prove the following generalization of Theorem 4.9.
Theorem 5.3. For any  $i \in N$ , if  $G$  is  $\delta$ -CSP in the neighborhood of  $S^{\times}(\mathcal{A}_N)$  and  $\exists \check{G} \in \mathrm{CSP}_{\delta}(G, S^{\times}(\mathcal{A}_N))$  that is  $\gamma$ -subgame stable in the neighborhood of  $S(\mathcal{A}_i)$ , then for any  $s \in S(\mathcal{A}_i)$
$$
\operatorname {V u l} _ {i} \left(s, S _ {- i} ^ {\times} (\mathcal {A} _ {N})\right) \leq | E _ {i} | \gamma + 2 \delta \leq (n - 1) \gamma + 2 \delta .
$$
An implication of Theorem 5.3 is that if agents use self-play to compute a marginal strategy from some mediated equilibrium and there is an SS-CSP game that is close to the original game for these strategies, then this is sufficient to bound vulnerability against strategies learned in self-play.
6 Computing an SS-CSP Decomposition in a Neighborhood
How might one determine if a game is well-approximated by an SS-CSP game? In addition to the algorithms presented in Appendix D, we give an algorithm, SGDecompose, that finds an SS-CSP decomposition for a game in a given neighborhood of strategy profiles. Pseudocode is given in Algorithm 1. SGDecompose could be used to test whether a game is well-approximated by an SS-CSP game before potentially analytically showing this property holds. We will use this algorithm in the following section to decompose Leduc poker.
As input, SGDecompose receives a neighborhood of strategies  $S'$  and the set of match-ups between strategies in  $S'$ , given by  $S^{\times} \doteq \bigvee_{i \in N} S_i'$ . The idea is to compute a CSP game  $\tilde{G}$  that minimizes
a loss function with two components: how close  $\check{G}$  is to  $G$  in the neighborhood of  $S^{\times}$  and how subgame stable  $\check{G}$  is for the neighborhood of  $S^{\prime}$ . First,  $\mathcal{L}^{\delta}$  is the error between the utility functions of  $G$  and  $\check{G}$  ( $u$  and  $\check{u}$ , respectively); it is a proxy for  $\delta$  in  $\delta$ -CSP. The loss for a single strategy profile  $s$  is
$$
\mathcal {L} ^ {\delta} (s; \check {u}, u) \doteq \sum_ {i \in N} | \check {u} _ {i} (s) - u _ {i} (s) |.
$$
The other component of the overall loss function,  $\mathcal{L}^{\gamma}$ , measures the subgame stability. First, we define  $\mathcal{L}_{ij}^{\gamma}$ , which only applies to a single subgame. Let  $s_{ij} = (s_i,s_j)$  be a profile for a subgame and  $s_{ij}^{*} = (s_{i}^{*},s_{j}^{*})$  is a profile of deviations for that subgame. The  $\mathcal{L}_{ij}^{\gamma}$  loss for this subgame is
$$
\mathcal {L} _ {i j} ^ {\gamma} \left(s _ {i j}, s _ {i j} ^ {*}; \check {u}\right) \doteq \max  \left(\check {u} _ {i j} \left(s _ {i} ^ {*}, s _ {j}\right) - \check {u} _ {i j} \left(s _ {i j}\right), 0\right) + \max  \left(\check {u} _ {j i} \left(s _ {i}, s _ {j} ^ {*}\right) - \check {u} _ {j i} \left(s _ {i j}\right), 0\right).
$$
Then, given a strategy profile  $s$  and deviation profile  $s^*$  for all players  $N$ , we have
$$
\mathcal {L} ^ {\gamma} \left(s, s ^ {*}; \check {u}\right) \doteq \sum_ {(i, j) \in E} \mathcal {L} _ {i j} ^ {\gamma} \left(s _ {i j}, s _ {i j} ^ {*}; \check {u}\right).
$$
SGDecompose repeats over a number of epochs  $T$ . At the start of an epoch, we compute a best-response (for example, via sequence-form linear programming in extensive-form games) to each strategy  $s_i'$  in  $S'$  in each subgame; the full process is shown in Algorithm 4 in the appendix. After computing these best-responses for the current utility function of  $\check{G}$ , SGDecompose fits  $\check{u}$  to be nearly CSP in the neighborhood of  $S^{\times}$  and subgame stable in the neighborhood of  $S'$ . Since  $S^{\times}$  is exponentially larger than  $S'$ , we partition it into batches, then use batch gradient descent.<sup>3</sup>
We use the following batch loss function, which computes the average values of  $\mathcal{L}^{\delta}$  and  $\mathcal{L}^{\gamma}$  over the batch then weights the losses with  $\lambda$ . Let  $S^{b}$  denote a batch of strategy profiles from  $S^{\times}$  with size  $B$
$$
\mathcal {L} \left(S ^ {b}, S ^ {\prime}, S ^ {*}; \check {u}, u\right) \doteq \frac {\lambda}{B} \sum_ {s \in S ^ {b}} \mathcal {L} ^ {\delta} (s; \check {u}, u) + \frac {(1 - \lambda)}{\left| S ^ {\prime} \right|} \sum_ {s \in S ^ {\prime}} \sum_ {s ^ {*} \in S ^ {*}} \mathcal {L} ^ {\gamma} (s, s ^ {*}; \check {u}).
$$
We use this loss function to update  $\check{u}$ , which is guaranteed to be a valid utility function for a CSP game via its representation, see Appendix F for details. In Appendix F, we give the procedure in terms of a more efficient representation of polymatrix decompositions for extensive-form games, which we call poly-EFGs; which we describe in Appendix E.
Algorithm 1 SGDecompose
Input:  $G, S'$ , hyperparameters  $\eta, T, \lambda, B$   
Initialize  $\tilde{u}$  to all 0  
 $S^{\times} \leftarrow \times_{i \in N} \hat{S}_i$   
for  $t \in 1 \dots T$  do  
\[
\begin{aligned}
& S^{*} \leftarrow \text{getBRs}(\check{G}, S') \\ & \mathcal{B} \leftarrow \text{partition of } S^{\times} \text{ into batches of size } B \\ & \text{for } S^{b} \in \mathcal{B} \text{ do} \\ & g \leftarrow \nabla_{\tilde{u}} \mathcal{L}(S^{b}, S', S^{*}; \check{u}, u) \\ & \check{u} \leftarrow \check{u} - \eta \cdot \frac{g}{\|g\|_{2}} \{ \text{update } \check{u} \text{ using normalized gradient; this helps with stability} \} \\ & \text{end for} \\ & \text{end for} \\ & \{\text{Lastly, output } \delta \text{ and } \gamma\} \\ & \delta \leftarrow \max_{s \in S^{\times}} |u_{i}(s) - \check{u}_{i}(s)| \\ & \gamma \leftarrow \max_{s \in S'} \max_{i \neq j \in N \times N} (\check{u}_{ij}(BR_{ij}(s_{j}), s_{j}) - \check{u}_{ij}(s_{i}, s_{j})) \\ & \text{return } \check{u}, \gamma, \delta
\end{aligned}
\]
Approaches using regret-minimization in self-play have been shown to outperform expert human players in some multiplayer games, the most notable example being multiplayer no-limit Texas hold 'em (Brown & Sandholm, 2019), despite no formal guarantees.
One could also partition  $S'$  into batches if it were too large.
Conjecture 7.1. Self-play with regret minimization performs well in multiplayer Texas hold 'em because "good" players (whether professional players or strategies learned by self-play) play in a part of the games' strategy space that is close to an SS-CSP game (i.e. low values of  $\gamma, \delta$ ).
While multiplayer no-limit Texas hold 'em is too large to directly check the properties developed in this work, we use a smaller poker game, called Leduc poker (Southey et al., 2012), to suggest why regret-minimization "works" in multiplayer Texas hold 'em. Leduc poker was originally developed for two players but was extended to a 3-player variant by Abou Risk & Szafron (2010); we use the 3-player variant here. The game has 8 cards, two rounds of betting, one private and one public card.
We first use a self-play algorithm to learn strategies, then use SGDecompose to see if this part of Leduc Poker is close to an SS-CSP game. We give a summary of results here, please refer to Appendix G for full details. We use  $\mathrm{CFR + }$  (Tammelin, 2014; Tammelin et al., 2015) as a self-play algorithm to compute a set of approximate marginal strategies.  $^4\mathrm{CFR + }$  was chosen because of its improved efficiency over CFR.  $\mathrm{CFR + }$  is a deterministic algorithm, so we use different random initializations of  $\mathrm{CFR + }$ 's initial strategy in order to generate a set of CCE. We will use 30 runs of  $\mathrm{CFR + }$  as input to SGDecompose; and have 30 runs of SGDecompose (i.e. we trained  $\mathrm{CFR + }$  900 times in total in self-play). This will give us a value of  $\delta$  and  $\gamma$  for each run.
We found that Leduc poker was well-approximated by an SS-CSP game in the neighborhood of strategies learned by  $\mathrm{CFR + }$ . In particular, across runs, Leduc poker was on average (with standard errors)  $\delta = 0.009\pm 0.00046$  -CSP and  $\gamma = 0.004\pm 0.00016$  subgame stable in the neighborhood of  $\mathrm{CFR + }$ -learned strategies. How well do these values bound vulnerability with respect to other CFR-learned strategies? For each of the runs, we computed the vulnerability with respect to the strategies of that run, by evaluating each strategy against each other and taking the maximum vulnerability. We compare these values to the upper bounds implies by the values of  $\delta$  and  $\gamma$  for each run and Theorem 5.3. We found the computed values of  $\delta$  and  $\gamma$  do a good job of upper bounding the vulnerability. Across the runs, the bounds are at minimum 1.89 times the vulnerability, at maximum 3.05 times the vulnerability and on average 2.51 times as large, with a standard error of 0.049.
We repeated these experiments with a toy hanabi game—where strategies learned in self-play are highly vulnerable—which we found to have much higher values of  $\delta$  and  $\gamma$ ; details are in Appendix H.
It was previously believed that CFR does not compute an  $\epsilon$ -Nash equilibrium on 3-player Leduc for any reasonable value of  $\epsilon$ . Abou Risk & Szafron (2010) found CFR computed a 0.130-Nash equilibrium. We found that  $\mathrm{CFR + }$  always computed an approximate Nash equilibrium with  $\epsilon \leq 0.013$ . Appendix G.2 shows that CFR also computes an approximate Nash equilibrium with  $\epsilon \leq 0.017$ .
Self-play has been incredibly successful in producing strategies that perform well against new opponents in two-player constant-sum games. Despite a lack of theoretical guarantees, self-play seems to also produce good strategies in some multiplayer games (Brown & Sandholm, 2019). We identify a structural property of multiplayer, general-sum game that allows us to establish guarantees on the performance of strategies learned via self-play against new opponents. We show that any game can be projected into the space of constant-sum polymatrix games, and if there exists a game with this set with high subgame stability (low  $\gamma$ ), strategies learned through self-play have bounded loss of performance against new opponents.
We conjecture that Texas hold 'em is one such game. We investigate this claim on Leduc poker, and find that  $\mathrm{CFR + }$  plays strategies from a part of the strategy space in Leduc poker that is wellapproximated by a subgame stable constant-sum polymatrix game. This work lays the groundwork for guarantees for self-play in multiplayer games. However, there is room for algorithmic improvement and efficiency gains for checking these properties in very large extensive-form games.
4We use the OpenSpiel implementation (Lanctot et al., 2019).
Computation for this work was provided by the Digital Research Alliance of Canada. Revan MacQueen was supported by Alberta Innovates and NSERC during completion of this work. This work was funded in part by an NSERC Discovery Grant. James R. Wright holds a Canada CIFAR AI Chair through the Alberta Machine Intelligence Institute. Thank you to Dustin Morrill, Michael Bowling and Nathan Sturtevant for helpful conversations and feedback, and the anonymous reviewers for valuable comments.
Abou Risk, N. and Szafron, D. Using counterfactual regret minimization to create competitive multiplayer poker agents. International Conference on Autonomous Agents and Multiagent Systems, 2010.
Aumann, R. J. Subjectivity and correlation in randomized strategies. Journal of Mathematical Economics, 1(1):67-96, 1974.
Bakhtin, A., Wu, D., Lerer, A., and Brown, N. No-press diplomacy from scratch. Advances in Neural Information Processing Systems, 34:18063-18074, 2021.
Bakhtin, A., Wu, D. J., Lerer, A., Gray, J., Jacob, A. P., Farina, G., Miller, A. H., and Brown, N. Mastering the game of no-press diplomacy via human-regularized reinforcement learning and planning. arXiv preprint arXiv:2210.05492, 2022.
Bard, N., Foerster, J. N., Chandar, S., Burch, N., Lanctot, M., Song, H. F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., Dunning, I., Mourad, S., Larochelle, H., Bellemare, M. G., and Bowling, M. The hanabi challenge: A new frontier for ai research. Artificial Intelligence, 2020.
Bergman, L. and Fokin, I. On separable non-cooperative zero-sum games. Optimization, 44(1): 69-84, 1998.
Brown, N. and Sandholm, T. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418-424, 2018.
Brown, N. and Sandholm, T. Superhuman AI for multiplayer poker. Science, 365(6456):885-890, 2019.
Cai, Y. and Daskalakis, C. On minmax theorems for multiplayer games. ACM-SIAM Symposium on Discrete algorithms, 2011.
Cai, Y., Candogan, O., Daskalakis, C., and Papadimitriou, C. Zero-sum polymatrix games: A generalization of minmax. Mathematics of Operations Research, 41(2):648-655, 2016.
Celli, A., Marchesi, A., Farina, G., and Gatti, N. No-regret learning dynamics for extensive-form correlated equilibrium. Neural Information Processing Systems, 2020.
Chen, Y., Deng, X., Li, C., Mguni, D., Wang, J., Yan, X., and Yang, Y. On the convergence of fictitious play: A decomposition approach. International Joint Conferences on Artificial Intelligence, 2022.
Cheung, Y. K. and Tao, Y. Chaos of learning beyond zero-sum and coordination via game decompositions. arXiv preprint arXiv:2008.00540, 2020.
Daskalakis, C. and Papadimitriou, C. H. Three-player games are hard. *Electron. Colloquium Comput. Complex.*, 2005.
Daskalakis, C., Goldberg, P. W., and Papadimitriou, C. H. The complexity of computing a nash equilibrium. SIAM Journal on Computing, 39(1):195-259, 2009.
Farina, G., Ling, C. K., Fang, F., and Sandholm, T. Correlation in extensive-form games: Saddle-point formulation and benchmarks. Neural Information Processing Systems, 2019.
Farina, G., Bianchi, T., and Sandholm, T. Coarse correlation in extensive-form games. AAAI conference on Artificial Intelligence, 2020.