SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Robert J Aumann. Subjectivity and correlation in randomized strategies. Journal of mathematical Economics, 1(1):67-96, 1974.
[2] Nicola Basilico, Stefano Coniglio, Nicola Gatti, and Alberto Marchesi. Bilevel programming approaches to the computation of optimistic and pessimistic single-leader-multi-follower equilibria. In SEA, volume 75, pages 1-14, 2017.
[3] Nicola Basilico, Stefano Coniglio, Nicola Gatti, and Alberto Marchesi. Bilevel programming methods for computing single-leader-multi-follower equilibria in normal-form and polymatrix games. EURO Journal on Computational Optimization, 8(1):3-31, 2020.
[4] Kimmo Berg and Tuomas Sandholm. Exclusion method for finding Nash equilibrium in multiplayer games. In AAAI, pages 383-389, 2017.
[5] Noam Brown and Tuomas Sandholm. Safe and nested subgame solving for imperfect-information games. In NeurIPS, 2017.
[6] Noam Brown and Tuomas Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418-424, 2018.
[7] Yang Cai and Constantinos Daskalakis. On minmax theorems for multiplayer games. In SODA, pages 217-234, 2011.
[8] Xi Chen and Xiaotie Deng. 3-Nash is PPAD-complete. In Electronic Colloquium on Computational Complexity, volume 134, pages 2-29, 2005.
[9] Vincent Conitzer and Tuomas Sandholm. Complexity results about Nash equilibria. In *IJCAI*, pages 765-771, 2003.
[10] Hélène Fargier, Paul Jourdan, and Régis Sabbadin. A path-following polynomial equations systems approach for computing Nash equilibria. In AAMAS, pages 418-426, 2022.
[11] Gabriele Farina, Andrea Celli, Nicola Gatti, and Tuomas Sandholm. Connecting optimal ex-ante collusion in teams to extensive-form correlation: Faster algorithms and positive complexity results. In ICML, pages 3164–3173. PMLR, 2021.
[12] Jiarui Gan, Edith Elkind, Sarit Kraus, and Michael Wooldridge. Mechanism design for defense coordination in security games. In AAMAS, pages 402-410, 2020.
[13] Sam Ganzfried. Fast complete algorithm for multiplayer Nash equilibrium. arXiv preprint arXiv:2002.04734, 2021.
[14] Itzhak Gilboa and Eitan Zemel. Nash and correlated equilibria: Some complexity considerations. Games and Economic Behavior, 1(1):80-93, 1989.
[15] Ambros M Gleixner, Timo Berthold, Benjamin Müller, and Stefan Weltge. Three enhancements for optimization-based bound tightening. Journal of Global Optimization, 67(4):731-757, 2017.
[16] Srihari Govindan and Robert Wilson. A global Newton method to compute Nash equilibria. Journal of Economic Theory, 110(1):65-86, 2003.
[17] Srihari Govindan and Robert Wilson. Computing Nash equilibria by iterated polymatrix approximation. Journal of Economic Dynamics and Control, 28(7):1229-1241, 2004.
A Illustration of CRM
In four-player games,
$$
\overline {{\mathcal {N}}} = \{\{1, 2, 3 \}, \{1, 2, 4 \}, \{1, 3, 4 \}, \{2, 3, 4 \}, \{1, 2 \}, \{1, 3 \}, \{2, 3 \}, \{1, 4 \}, \{3, 4 \}, \{2, 4 \} \}.
$$
By Algorithm 1 in CRM, we have:
$$
\underline {{\mathcal {N}}} = \{\{1, 2, 3 \}, \{4, 2, 3 \}, \{1, 4, 3 \}, \{1, 2, 4 \}, \{1, 2 \}, \{4, 2 \}, \{1, 4 \} \},
$$
i.e.,  $\underline{\mathcal{N}} = \{-4, -1, -2, -3, \{1, 2\}, \{2, 4\}, \{1, 4\}\}$ . To obtain this set, we first have a full binary tree  $T_{-4}$  with the set of internal nodes  $\mathcal{N}_{T_{-4}} = \{\{1, 2, 3\}, \{1, 2\}\}$ . Then, for each element in  $\mathcal{N}_{T_{-4}}$ , we replace 1 with 4 to obtain  $\mathcal{N}_{T_{-1}} = \{\{4, 2, 3\}, \{4, 2\}\}$ ; replace 2 with 4 to obtain  $\mathcal{N}_{T_{-2}} = \{\{1, 4, 3\}, \{1, 4\}\}$ ; and replace 3 with 4 to obtain  $\mathcal{N}_{T_{-3}} = \{\{1, 2, 4\}, \{1, 2\}\}$ . Then  $\underline{\mathcal{N}} = \mathcal{N}_{T_{-1}} \cup \mathcal{N}_{T_{-2}} \cup \mathcal{N}_{T_{-3}} \cup \mathcal{N}_{T_{-4}}$ .
In three-player games, Algorithm 1 cannot reduce the number of internal nodes because each element in  $\{-i\mid i\in \{1,2,3\} \}$  includes only two players. Then  $\overline{\mathcal{N}} = \underline{\mathcal{N}} = \{\{1,2\} ,\{1,3\} ,\{2,3\} \}$ , which means that  $\overline{\mathcal{N}}$  and  $\underline{\mathcal{N}}$  will result in the same set of bilinear terms in three-player games.
To be simplified, we show how to formulate the program according to Algorithm 2 (i.e., CRM) in a three-player game  $G = (N, A, u)$  with  $N = \{1, 2, 3\}$ ,  $A_i = \{a_i, a_i'\}$  first. In three-player games,  $-1 = \{2, 3\}$ , and  $\underline{\mathcal{N}} = \{-1, -2, -3\} = \{\{2, 3\}, \{1, 3\}, \{1, 2\}\}$ .
Note that  $A_{-1} = A_{2,3} = A_2 \times A_3 = \{(a_2, a_3), (a_2, a_3'), (a_2', a_3), (a_2', a_3')\}$  with  $N' = -1 = \{2, 3\}, N'_l = \{2\}, N'_r = \{3\}$ . We show the constraints related to player 1 here: We first show player 1's constraints in the Nash equilibria space based on Eqs.(1b)-(1e):
$$
x _ {1} \left(a _ {1}\right) + x _ {1} \left(a _ {1} ^ {\prime}\right) = 1 (\text {b a s e d o n E q s . (1 b)})
$$
$$
1 - b _ {a _ {1}} \geq x _ {1} \left(a _ {1}\right), 1 - b _ {a _ {1} ^ {\prime}} \geq x _ {1} \left(a _ {1} ^ {\prime}\right) (\text {b a s e d o n E q . (1 c)})
$$
$$
u _ {1} (x) \geq u _ {1} \left(a _ {1}, x _ {- 1}\right), u _ {1} (x) - u _ {1} \left(a _ {1}, x _ {- 1}\right) \leq b _ {a _ {1}} \left(U _ {\max } - U _ {\min }\right) (\text {b a s e d o n E q s . (1 d)}) \text {a n d (1 e)}
$$
$$
u _ {1} (x) \geq u _ {1} \left(a _ {1} ^ {\prime}, x _ {- 1}\right), u _ {1} (x) - u _ {1} \left(a _ {1} ^ {\prime}, x _ {- 1}\right) \leq b _ {a _ {1} ^ {\prime}} \left(U _ {\max } - U _ {\min }\right) (\text {b a s e d o n E q s . (1 d)} \text {a n d (1 e)})
$$
The above constraints include player 1's expected utility variables  $u_{1}(a_{1},x_{-1})$  and  $u_{1}(a_{1}^{\prime},x_{-1})$ , which are represented by the following constraints based on Eq.(4):
$$
\begin{array}{l} u _ {1} \left(a _ {1}, x _ {- 1}\right) = u _ {1} \left(a _ {1}, a _ {2}, a _ {3}\right) x _ {- 1} \left(a _ {2}, a _ {3}\right) + u _ {1} \left(a _ {1}, a _ {2}, a _ {3} ^ {\prime}\right) x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) \\ + u _ {1} \left(a _ {1}, a _ {2} ^ {\prime}, a _ {3}\right) x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}\right) + u _ {1} \left(a _ {1}, a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) (\text {b a s e d o n E q . (4)}) \\ \end{array}
$$
$$
\begin{array}{l} u _ {1} \left(a _ {1} ^ {\prime}, x _ {- 1}\right) = u _ {1} \left(a _ {1} ^ {\prime}, a _ {2}, a _ {3}\right) x _ {- 1} \left(a _ {2}, a _ {3}\right) + u _ {1} \left(a _ {1} ^ {\prime}, a _ {2}, a _ {3} ^ {\prime}\right) x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) \\ + u _ {1} \left(a _ {1} ^ {\prime}, a _ {2} ^ {\prime}, a _ {3}\right) x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}\right) + u _ {1} \left(a _ {1} ^ {\prime}, a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) (\text {b a s e d o n E q . (4)}), \\ \end{array}
$$
where the correlation plan  $x_{-1}$  of  $-1$  over  $A_{-1}$ , based on Eq.(2), is defined by:
$$
x _ {- 1} \left(a _ {2}, a _ {3}\right) + x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}\right) + x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) = 1 (\text {t h e p l a n o f - 1 b a s e d o n E q . (2)}).
$$
In the above correlation plan  $x_{-1}, x_{-1}(a_2, a_3), x_{-1}(a_2, a_3')$ ,  $x_{-1}(a_2', a_3)$ ,  $x_{-1}(a_2', a_3')$  represent the following four bilinear terms (constraints):
$$
\begin{array}{l} x _ {- 1} \left(a _ {2}, a _ {3}\right) = x _ {2} \left(a _ {2}\right) x _ {3} \left(a _ {3}\right) (\text {t h e b i l i n e a r c o n s t r a i n t b a s e d o n E q . (3 a)}) w i t h \\ a _ {- 1} = \left(a _ {2}, a _ {3}\right) = \left(a _ {N _ {l} ^ {\prime}}, a _ {N _ {r} ^ {\prime}}\right), a _ {N _ {l} ^ {\prime}} = \left(a _ {2}\right), a _ {N _ {r} ^ {\prime}} = \left(a _ {3}\right) \\ \end{array}
$$
$$
\begin{array}{l} x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) = x _ {2} \left(a _ {2}\right) x _ {3} \left(a _ {3} ^ {\prime}\right) (\text {t h e b i l i n e a r c o n s t r a i n t b a s e d o n E q . (3 a)}) w i t h \\ a _ {- 1} = \left(a _ {2}, a _ {3} ^ {\prime}\right) = \left(a _ {N _ {l} ^ {\prime}}, a _ {N _ {r} ^ {\prime}}\right), a _ {N _ {l} ^ {\prime}} = \left(a _ {2}\right), a _ {N _ {r} ^ {\prime}} = \left(a _ {3} ^ {\prime}\right) \\ \end{array}
$$
$$
\begin{array}{l} x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}\right) = x _ {2} \left(a _ {2} ^ {\prime}\right) x _ {3} \left(a _ {3}\right) (\text {t h e b i l i n e a r c o n s t r a i n t b a s e d o n E q . (3 a)}) w i t h \\ a _ {- 1} = (a _ {2} ^ {\prime}, a _ {3}) = (a _ {N _ {l} ^ {\prime}}, a _ {N _ {r} ^ {\prime}}), a _ {N _ {l} ^ {\prime}} = (a _ {2} ^ {\prime}), a _ {N _ {r} ^ {\prime}} = (a _ {3}) \\ \end{array}
$$
$$
x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) = x _ {2} \left(a _ {2} ^ {\prime}\right) x _ {3} \left(a _ {3} ^ {\prime}\right) (\text {t h e b i l i n e a r c o n s t r a i n t b a s e d o n E q . (3 a)}) w i t h
$$
$$
a _ {- 1} = (a _ {2} ^ {\prime}, a _ {3} ^ {\prime}) = (a _ {N _ {l} ^ {\prime}}, a _ {N _ {r} ^ {\prime}}), a _ {N _ {l} ^ {\prime}} = (a _ {2} ^ {\prime}), a _ {N _ {r} ^ {\prime}} = (a _ {3} ^ {\prime}),
$$
where, based on the binary division in  $\underline{\mathcal{N}}$ , any joint action  $a_{N'} \in A_{N'}$  can be divided into two sub-joint actions  $a_{N'_l} \in A_{N'_l}$  and  $a_{N'_r} \in A_{N'_r}$  such that  $a_{N'} = (a_{N'_l}, a_{N'_r})$ . Note that  $x_2$  and  $x_3$  are special correlation plans with that 2 is set to be equivalent to  $\{2\}$  and 3 is set to be equivalent to  $\{3\}$  for simplification. Correlation plans  $x_{-1}$  and  $x_2$  (or  $x_3$ ) have the following relation:
$$
\begin{array}{l} x _ {- 1} \left(a _ {2}, a _ {3}\right) + x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) = x _ {2} \left(a _ {2}\right) (\text {t h e r e l a t i o n o f c o r r e l a t i o n p l a n s} x _ {- 1} \text {a n d} x _ {2} \text {b a s e d o n E q . (1 1)}), \\ \forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2}, a _ {3}\right), \left(a _ {2}, a _ {3} ^ {\prime}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} (2) = a _ {2}, i. e., a _ {2} i s p l a y e r 2 ^ {\prime} s a c t i o n i n a _ {N ^ {\prime}} \\ \end{array}
$$
$$
\begin{array}{l} x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) = x _ {2} \left(a _ {2} ^ {\prime}\right) (\text {t h e r e l a t i o n o f c o r r e l a t i o n p l a n s} x _ {- 1} \text {a n d} x _ {2} \text {b a s e d o n E q . (1 1)} \\ \forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2} ^ {\prime}, a _ {3}\right), \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} (2) = a _ {2} ^ {\prime}, i. e., a _ {2} ^ {\prime} \text {i s p l a y e r 2 ’ s a c t i o n i n} a _ {N ^ {\prime}} \\ \end{array}
$$
$$
\begin{array}{l} x _ {- 1} \left(a _ {2}, a _ {3}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}\right) = x _ {3} \left(a _ {3}\right) (\text {t h e r e l a t i o n o f c o r r e l a t i o n p l a n s} x _ {- 1} \text {a n d} x _ {3} \text {b a s e d o n E q . (1 1)} \\ \forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2}, a _ {3}\right), \left(a _ {2} ^ {\prime}, a _ {3}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} (3) = a _ {3}, \text {i . e ,} a _ {3} \text {i s p l a y e r 3 ’ s a c t i o n i n} a _ {N ^ {\prime}} \\ \end{array}
$$
$$
\begin{array}{l} x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) = x _ {3} \left(a _ {3} ^ {\prime}\right) (\text {t h e r e l a t i o n o f c o r r e l a t i o n p l a n s} x _ {- 1} \text {a n d} x _ {3} \text {b a s e d o n E q . (1 1)}), \\ \forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2}, a _ {3} ^ {\prime}\right), \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} (3) = a _ {3} ^ {\prime}, i. e., a _ {3} ^ {\prime} \text {i s p l a y e r 3 ’ s a c t i o n i n} a _ {N ^ {\prime}} \\ \end{array}
$$
Constraints related to other players are created similarly. As shown in Algorithm 2 (i.e., CRM), after creating all of these constraints in the bilinear program for the Nash equilibria space, we can solve the program to optimize an objective function by using a global optimization solver, e.g., Gurobi.
In three-player games,  $x_{-1}(a_{-1})$  just represents a bilinear term, and then a chain of bilinear constraints (equalities) to transform a multilinear term into bilinear terms is not explicit. In four-player games, a chain of bilinear constraints (equalities) to transform a multilinear term into bilinear terms is more explicit. For example, in a game  $G = (N, A, u)$  with  $N = \{1, 2, 3, 4\}$ ,  $A_i = \{a_i, a_i'\}$ , and  $\underline{\mathcal{N}} = \{-4, -1, -2, -3, \{1, 2\}, \{2, 4\}, \{1, 4\}\}$ , based on Eq.(3a), we have:
$$
x _ {- 1} \left(a _ {2}, a _ {3}, a _ {4}\right) = x _ {2, 4} \left(a _ {2}, a _ {4}\right) x _ {3} \left(a _ {3}\right), x _ {2, 4} \left(a _ {2}, a _ {4}\right) = x _ {2} \left(a _ {2}\right) x _ {4} \left(a _ {4}\right) (\text {a c h a i n o f b i l i n e a r c o n s t r a i n t s}),
$$
where  $N' = -1 = \{2,3,4\}$ ,  $N'_l = \{2,4\}$  and  $N'_r = \{3\}$  based on  $\underline{\mathcal{N}}$ . That is,  $a_{-1} = (a_2,a_3,a_4) = (a_{N'_l},a_{N'_r})$  with  $a_{N'_l} = (a_2,a_4)$ ,  $a_{N'_r} = (a_3)$  (i.e., joint action  $a_{-1}$  is divided into two sub-joint actions  $a_{N'_l}$  and  $a_{N'_r}$  such that  $a_{-1} = (a_{N'_l},a_{N'_r})$ ) based on  $\underline{\mathcal{N}}$ , and then we can have the above chain of bilinear constraints for it. Each joint action in  $A_{-1} = A_2\times A_3\times A_4 = A_{N'_l}\times A_{N'_r} = \{(a_2,a_3,a_4),(a_2,a_3,a_4),(a_2,a_3,a_4),(a_2',a_3,a_4),(a_2',a_3,a_4),(a_2',a_3,a_4)\}$  is divided into two disjoint sets similarly, and then we can have a chain of bilinear constraints for it.
In three-player games, each element in  $\{-i\mid i\in \{1,2,3\} \}$  includes only two players, and then the resulting program does not include the constraints for the relation of correlation plans based on Eq.(12). To show the constraints based on Eq.(12), we consider four-player games. The following constraints are player 1's constraints based on Eq.(12) for solving a game  $G = (N,A,u)$  with  $N = \{1,2,3,4\}$ ,  $A_{i} = \{a_{i},a_{i}^{\prime}\}$ :
$$
x _ {- 1} \left(a _ {2}, a _ {3}, a _ {4}\right) + x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}, a _ {4}\right) = x _ {2, 4} \left(a _ {2}, a _ {4}\right) (\text {b a s e d o n E q . (1 2)}) \text {w i t h} a _ {N _ {l} ^ {\prime}} = \left(a _ {2}, a _ {4}\right),
$$
$$
\forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2}, a _ {3}, a _ {4}\right), \left(a _ {2}, a _ {3} ^ {\prime}, a _ {4}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} \left(N _ {l} ^ {\prime}\right) = \left(a _ {2}, a _ {4}\right),
$$
i.e.,  $(a_{2},a_{4})$  is  $N_l^{\prime}$ 's sub-joint action in  $a_{N^{\prime}}$
$$
x _ {- 1} \left(a _ {2}, a _ {3}, a _ {4} ^ {\prime}\right) + x _ {- 1} \left(a _ {2}, a _ {3} ^ {\prime}, a _ {4} ^ {\prime}\right) = x _ {2, 4} \left(a _ {2}, a _ {4} ^ {\prime}\right) (\text {b a s e d o n E q . (1 2)}) \text {w i t h} a _ {N _ {l} ^ {\prime}} = \left(a _ {2}, a _ {4} ^ {\prime}\right),
$$
$$
\forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2}, a _ {3}, a _ {4} ^ {\prime}\right), \left(a _ {2}, a _ {3} ^ {\prime}, a _ {4} ^ {\prime}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} \left(N _ {l} ^ {\prime}\right) = \left(a _ {2}, a _ {4} ^ {\prime}\right),
$$
i.e.,  $(a_{2},a_{4})$  is  $N_l^{\prime}$  's sub-joint action in  $a_{N'}$
$$
x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}, a _ {4}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}, a _ {4}\right) = x _ {2, 4} \left(a _ {2} ^ {\prime}, a _ {4}\right) (\text {b a s e d o n E q . (1 2)}) \text {w i t h} a _ {N _ {l} ^ {\prime}} = \left(a _ {2} ^ {\prime}, a _ {4}\right),
$$
$$
\forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2} ^ {\prime}, a _ {3}, a _ {4}\right), \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}, a _ {4}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} \left(N _ {l} ^ {\prime}\right) = \left(a _ {2} ^ {\prime}, a _ {4}\right),
$$
$$
x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}, a _ {4} ^ {\prime}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}, a _ {4} ^ {\prime}\right) = x _ {2, 4} \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right) (\text {b a s e d o n E q . (1 2)}) \text {w i t h} a _ {N _ {i} ^ {\prime}} = \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right),
$$
$$
\forall a _ {N ^ {\prime}} \in \left\{\left(a _ {2} ^ {\prime}, a _ {3}, a _ {4} ^ {\prime}\right), \left(a _ {2} ^ {\prime}, a _ {3} ^ {\prime}, a _ {4} ^ {\prime}\right) \right\} \subseteq A _ {- 1}, a _ {N ^ {\prime}} \left(N _ {l} ^ {\prime}\right) = \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right),
$$
i.e.,  $(a_2', a_4')$  is  $N_l''$ 's sub-joint action in  $a_{N'}$ ,
where  $N' = -1 = \{2,3,4\}$ ,  $N_{l}' = \{2,4\}$  and  $N_{r}' = \{3\}$  based on  $\mathcal{N}$ . The we can divide the joint actions in  $A_{-1}$ , e.g.,  $a_{-1} = (a_2,a_3,a_4) = (a_{N_l'},a_{N_r'})$  with  $a_{N_l'} = (a_2,a_4)$ ,  $a_{N_r'} = (a_3)$ . For elements in  $\{N_l',N_r'\}$ , we only consider constraints based on Eq.(12) for  $N_{l}^{\prime}$  because  $|N_{l}^{\prime}| = 2 > 1$
and  $|N_r'| = 1$ . We could use Eq.(11) to generate constraints for  $N_r' = \{3\}$  for the relation between  $x_{-1}$  and  $x_3$ , e.g., for  $a_3 \in A_3$ ,
$$
x _ {- 1} \left(a _ {2}, a _ {3}, a _ {4}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}, a _ {4}\right) + x _ {- 1} \left(a _ {2}, a _ {3}, a _ {4} ^ {\prime}\right) + x _ {- 1} \left(a _ {2} ^ {\prime}, a _ {3}, a _ {4} ^ {\prime}\right) = x _ {3} \left(a _ {3}\right),
$$
where, for each  $a_{N'} \in \{(a_2, a_3, a_4), (a_2', a_3, a_4), (a_2, a_3, a_4'), (a_2', a_3, a_4')\} \subseteq A_{-1}, a_{N'}(3) = a_3$ , i.e.,  $a_3$  is player 3's action in  $a_{N'}$ .
Other constraints for four-player games are created similarly to the above creation of constraints for three-player games.
Theorem 1. The feasible solution space of mixed strategies (i.e.,  $x_{i}(a_{i})$  for each  $i \in N$ ,  $a_{i} \in A_{i}$ ) in Eqs.(1b)-(1g), (3), and (4) is the space of NEs.
Proof. Eq.(1) describing the space of NEs and Eqs.(1b)-(1g), (3), and (4) both include Eqs.(1b)-(1g), which describe the condition of NEs. Then we only need to show that Eq.(4) is equivalent to Eq.(1a). That is, we need to show  $\prod_{j\in -i}x_j(a_{-i}(j)) = x_{-i}(a_{-i})$  for each  $i\in N$  and  $a_{-i}\in A_{-i}$ . To achieve this result, we show  $\prod_{j\in N'}x_j(a_{N'}(j)) = x_{N'}(a_{N'})$  for each  $N'\in \mathcal{N}$  and  $a_{N'}\in A_{N'}$ . We show that this statement holds by induction. For any  $N'\in \mathcal{N}$  with  $|N'| = 2$ , we obviously have  $\prod_{j\in N'}x_j(a_{N'}(j)) = x_{N'}(a_{N'})$  for each  $a_{N'}\in A_{N'}$  by Eq.(3). Suppose, for any  $N'\in \mathcal{N}$  with  $|N'| = k\geq 2$ , the statement holds. Now for any  $N'\in \mathcal{N}$  with  $|N'| = k + 1$  and its two children  $N_l'$  and  $N_r'$ , we have: for any  $a_{N'}\in A_{N'}$ ,
$$
\begin{array}{l} x _ {N ^ {\prime}} (a _ {N ^ {\prime}}) = x _ {N _ {l} ^ {\prime}} (a _ {N _ {l} ^ {\prime}}) x _ {N _ {r} ^ {\prime}} (a _ {N _ {r} ^ {\prime}}) \\ = \prod_ {i \in N _ {l} ^ {\prime}} x _ {i} \left(a _ {N _ {l} ^ {\prime}} (i)\right) \prod_ {j \in N _ {r} ^ {\prime}} x _ {j} \left(a _ {N _ {r} ^ {\prime}} (j)\right) \\ = \prod_ {j \in N ^ {\prime}} x _ {j} (a _ {N ^ {\prime}} (j)), \\ \end{array}
$$
where the second “ $=$ ” is based on the assumption that, for any  $N' \in \mathcal{N}$  with  $|N'| = k \geq 2$ , the statement holds. Then  $\prod_{j \in N'} x_j(a_{N'}(j)) = x_{N'}(a_{N'})$  for each  $N' \in \mathcal{N}$  and  $a_{N'} \in A_{N'}$ . Therefore, the theorem holds.
Proof. (i) We first show that  $\mathcal{T} \subset \mathcal{R}$ . Given any  $x_{\{i,j\}}(a_i, a_j) = x_i(a_i)x_j(a_j)$ , in  $\mathcal{T}$ , we have  $x_{\{i,j\}}(a_i, a_j) \leq \min \{x_i(a_i), x_j(a_j)\}$  according to Eq.(11). Suppose  $x_i(a_i) + x_j(a_j) - 1 > x_{\{i,j\}}(a_i, a_j)$ . According to Eqs.(11) and (1b), we have the following contradiction:
$$
\begin{array}{l} x _ {i} (a _ {i}) \\ = \sum_ {a _ {j} ^ {\prime} \in A _ {j}} x _ {\{i, j \}} \left(a _ {i}, a _ {j} ^ {\prime}\right) \\ <   x _ {i} (a _ {i}) + x _ {j} (a _ {j}) - 1 + \sum_ {a _ {j} ^ {\prime} \in A _ {j}, a _ {j} ^ {\prime} \neq a _ {j}} x _ {\{i, j \}} (a _ {i}, a _ {j} ^ {\prime}) \\ \leq x _ {i} \left(a _ {i}\right) - 1 + \sum_ {a _ {j} \in A _ {j}} x _ {j} \left(a _ {j}\right) \\ = x _ {i} \left(a _ {i}\right), \\ \end{array}
$$
where the first “ $=$ ” is according to Eq.(11), “ $\leq$ ” is according to the assumption, “ $\leq$ ” is according to Eq.(11), and the last “ $=$ ” is according to Eq.(1b). This contradiction implies that  $x_{i}(a_{i}) + x_{j}(a_{j}) - 1 \leq x_{\{i,j\}}(a_{i}, a_{j})$ , i.e.,  $\max \{0, x_{i}(a_{i}) + x_{j}(a_{j}) - 1\} \leq x_{\{i,j\}}(a_{i}, a_{j})$ . Similarly, for any  $x_{N'}(a_{N'}) = x_{N'_l}(a_{N'_l})x_{N'_r}(a_{N'_r})$  with two children  $N'_l$  and  $N'_r$  of  $N'$ , in  $\mathcal{T}$ , we have:
$$
\begin{array}{l} \max  \{x _ {N _ {l} ^ {\prime}} (a _ {N _ {l} ^ {\prime}}) + x _ {N _ {r} ^ {\prime}} (a _ {N _ {r} ^ {\prime}}) - 1, 0 \} \\ \leq x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) \\ \leq \min \{x _ {N _ {l} ^ {\prime}} (a _ {N _ {l} ^ {\prime}}), x _ {N _ {r} ^ {\prime}} (a _ {N _ {r} ^ {\prime}}) \}. \\ \end{array}
$$
Therefore,  $\mathcal{T}\subseteq \mathcal{R}$
Given any  $x_{\{i,j\}}(a_1,a_2) = x_1(a_1)x_2(a_2)$  and  $x_{\{i,j\}}(a_1',a_2) = x_1(a_1')x_2(a_2)$ , by Eq.(6), we can have a feasible solution such that:
$$
x _ {\{i, j \}} \left(a _ {1}, a _ {2}\right) = \min  \left\{x _ {1} \left(a _ {1}\right), x _ {2} \left(a _ {2}\right) \right\}
$$
$$
x _ {\{i, j \}} \left(a _ {1} ^ {\prime}, a _ {2}\right) = \min  \left\{x _ {1} \left(a _ {1} ^ {\prime}\right), x _ {2} \left(a _ {2}\right) \right\}.
$$
Then  $x_{\{i,j\}}(a_1,a_2) + x_{\{i,j\}}(a_1',a_2) > x_2(a_2)$  when  $0 < x_{2}(a_{2}) < \min \{x_{1}(a_{1}),x_{1}(a_{1}^{\prime})\} < 1$  However, in  $\mathcal{T}$
$$
x _ {\{i, j \}} (a _ {1}, a _ {2}) + x _ {\{i, j \}} \left(a _ {1} ^ {\prime}, a _ {2}\right) \leq x _ {2} (a _ {2}).
$$
Then  $\mathcal{R} \nsubseteq \mathcal{T}$ . Therefore,  $\mathcal{T} \subset \mathcal{R}$ , i.e.,  $\mathcal{T}$  is strictly smaller than  $\mathcal{R}$ .
(ii) Now we show that  $\mathcal{M} \subset \mathcal{T}$ . In  $\mathcal{M}$ , for each  $a_{N'} \in A_{N'}$ ,  $N' \in \mathcal{N}$  with two children  $N'_l$  and  $N'_r$  of  $N'$ , there is a bilinear constraint  $x_{N'}(a_{N'}) = x_{N'_l}(a_{N'_l})x_{N'_r}(a_{N'_r})$  based on  $a_{N'} = (a_{N'_l}, a_{N'_r})$ , where  $x_{N'}(a_{N'}) = x_i(a_i)$  for  $N' = \{i\}$ . We first show  $\mathcal{M} \subseteq \mathcal{T}$  for Eq.(2) by induction. For any  $N' = (i, j) \in \mathcal{N}$ , by  $\mathcal{M}$ , we have:
$$
\sum_ {(a _ {i}, a _ {j}) \in A _ {\{i, j \}}} x _ {\{i, j \}} (a _ {i}, a _ {j}) = \sum_ {a _ {i} \in A _ {i}} x _ {i} (a _ {i}) \sum_ {a _ {j} \in A _ {j}} x _ {j} (a _ {j}) = 1.
$$
Suppose, for any  $N^{\prime}\in \mathcal{N}$  with  $|N^{\prime}| = k$  and  $k\geq 2$
$$
\sum_ {a _ {N ^ {\prime}} \in A _ {N ^ {\prime}}} x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) = 1.
$$
Now for any  $N' \in \mathcal{N}$  with  $|N'| = k + 1$ , with two children  $N_{l}'$  and  $N_{r}'$  of  $N'$ , by  $\mathcal{M}$  and the assumption of  $N_{k}$ , we have:
$$
\begin{array}{l} \sum_ {a _ {N ^ {\prime}} \in A _ {N ^ {\prime}}} x _ {N ^ {\prime}} (a _ {N ^ {\prime}}) = \sum_ {a _ {N _ {l} ^ {\prime}} \in A _ {N _ {l} ^ {\prime}}} x _ {N _ {l} ^ {\prime}} (a _ {N _ {l} ^ {\prime}}) \sum_ {a _ {N _ {r} ^ {\prime}} \in A _ {N _ {r} ^ {\prime}}} x _ {N _ {r} ^ {\prime}} (a _ {N _ {r} ^ {\prime}}) \\ = 1. \\ \end{array}
$$
Therefore, Eq.(2) is implied by  $\mathcal{M}$ . Similarly, for each  $N' \subset N$ , we have  $\sum_{a_{N'} \in A_{N'}} x_{N'}(a_{N'}) = 1$ . For any  $N' = (i,j) \in \mathcal{N}$ , by  $\mathcal{M}$ , we have: for any  $a_i \in A_i$ ,