SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Youzhi Zhang
Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science & Innovation Chinese Academy of Sciences youzhi.zhang@cair-cas.org.hk
Bo An
School of Computer Science and Engineering Nanyang Technological University Singapore boan@ntu.edu.sg
V. S. Subrahmanian
Evanston, USA
vss@northwestern.edu
Designing efficient algorithms to compute a Nash Equilibrium (NE) in multiplayer games is still an open challenge. In this paper, we focus on computing an NE that optimizes a given objective function. For example, when there is a team of players independently playing against an adversary in a game (e.g., several groups in a forest trying to interdict illegal loggers in green security games), these team members may need to find an NE minimizing the adversary's utility. Finding an optimal NE in multiplayer games can be formulated as a mixed-integer bilinear program by introducing auxiliary variables to represent bilinear terms, leading to a huge number of bilinear terms, making it hard to solve. To overcome this challenge, we first propose a general framework for this formulation based on a set of correlation plans. We then develop a novel algorithm called CRM based on this framework, which uses Correlation plans with their Relations to restrict the feasible solution space after the convex relaxation of bilinear terms while Minimizing the number of correlation plans to reduce the number of bilinear terms. We show that our techniques can significantly reduce the time complexity, and CRM can be several orders of magnitude faster than the state-of-the-art baseline.
One of the important problems in artificial intelligence is the design of algorithms for agents to make decisions in interactive environments [33]. To this day, many results have been achieved in two-player non-cooperative environments, for example, security games [39], the game of Go [38], and poker games [6]. One of the most important solution concepts behind these results is the well-known Nash Equilibrium (NE) [30]. Indeed, there are many efficient algorithms, e.g., algorithms based on linear programs [42, 43, 37, 47, 48] or counterfactual regret minimization [56, 5], to compute Nash equilibria (NEs) in two-player zero-sum games. However, there are fewer results on efficient algorithms for NEs with theoretical guarantees in multiplayer games (see the discussion in [4]), and most of these results are for games with particular structures (e.g., polymatrix games [7, 10]). The main reason is that finding NEs in multiplayer games is hard — it is PPAD-complete even for zero-sum three-player games [8]. Designing efficient algorithms to compute NEs in multiplayer games is thus still an open challenge.
In this paper, we focus on computing an optimal NE that optimizes a specific objective over the space of NEs. In the real world, we may need to optimize our objective over the space of NEs [34]. Possible objectives [9] could be maximizing social welfare (the sum of the players' expected utilities), maximizing the expected utilities of one player or several players, maximizing the minimum utility among players, minimizing the support sizes of the NE strategies, and so on. In addition, when there is a team of players in a game, team members need to consider finding an equilibrium that optimizes some objective [45, 12]. For example, in green security games where several heterogeneous groups (e.g., local police, the Madagascar National Parks, NGOs, and community volunteers) try to protect forests from illegal logging [26], the groups involved may need to find an NE that minimizes the adversary's utility<sup>1</sup>.
Unfortunately, the problems mentioned above are NP-hard [14, 9]. In two-player games, finding an optimal NE can be formulated as a mixed-integer linear program [34]. In this formulation, finding an optimal solution means optimizing an objective over the space of NEs, and this space is modeled as the feasible solution space of the mixed-integer linear program. We can directly extend this two-player formulation to find an optimal NE in multiplayer games by representing the space of NEs as the feasible solution space of a mixed-integer bilinear program transformed from a multilinear program by using auxiliary variables to represent bilinear terms. Then finding an optimal NE requires solving a non-convex program. Unfortunately, such a formulation is not efficient because there are exponentially many bilinear terms in the program. There are other approaches (e.g., [4]) that guarantee finding an NE in multiplayer games. However, these approaches need to enumerate all NEs to find an optimal NE, which is very inefficient [34] (see our experimental results) because there can be exponentially many NEs [44].
To tackle this challenge, we first propose a general framework for transforming a multilinear program for computing optimal NEs into a bilinear program based on a set of correlation plans, where each correlation plan (i.e., a probability distribution over joint actions) corresponds to a set of auxiliary variables representing a set of bilinear terms. We then develop a novel algorithm called CRM based on this framework, which uses Correlation plans with their Relations to strictly reduce the feasible solution space after the convex relaxation of bilinear terms while Minimizing the number of correlation plans to reduce the number of bilinear terms. We show that our techniques can significantly reduce the time complexity, and CRM can be several orders of magnitude faster than the state-of-the-art baseline. To our best knowledge, CRM is the first algorithm to use a minimum set of correlation plans to reformulate the program for computing optimal NEs in multiplayer games.
Consider a normal-form game $^2$ $G = (N, A, u)$  [37]. We denote the set of players as  $N = \{1, \dots, n\}$ ; the set of all players' joint actions is  $A = \times_{i \in N} A_i$ , where  $A_i$  is the finite set of player  $i$ 's pure strategies (actions) with  $a_i \in A_i$ ; and the set of all players' payoff functions is  $u = (u_1, \dots, u_n)$ , where  $u_i : A \to \mathbb{R}$  is player  $i$ 's payoff function. Let  $U_{max} = \max_{i \in N} \max_{a \in A} u_i(a)$ , and  $U_{min} = \min_{i \in N} \min_{a \in A} u_i(a)$ . In addition, the set of (joint) mixed strategy profiles  $X = \times_{i \in N} X_i$ , where  $X_i = \Delta(A_i)$  (i.e., the set of probability distributions over  $A_i$ ) is the set of mixed strategies of player  $i$ , and  $x_i(a_i)$  is the probability that any action  $a_i \in A_i$  is played. Let  $-i$  be the set of all players excluding player  $i$ , i.e.,  $-i = N \setminus \{i\}$ , and  $A_{-i}$  be  $\times_{j \in N \setminus \{i\}} A_j$ . Generally, given  $N' \subseteq N$ ,  $a_{N'} \in A_{N'} = \times_{i \in N'} A_i$ ,  $a_{N'}(i)$  is the action of player  $i \in N'$  in the joint action  $a_{N'}$ . For example, if  $a_{N'} = (a_1, a_3, a_5)$  with  $N' = \{1, 3, 5\}$ ,  $a_{N'}(3) = a_3$ . If  $N' = N$ , we ignore the subscript, i.e.,  $a = a_N$  and  $A = A_N$ . For each  $x \in X$ , player  $i$ 's expected payoff is:
$$
u _ {i} (x) = \sum_ {a \in A} u _ {i} (a) \prod_ {j \in N} x _ {j} (a (j)),
$$
and, if player  $i$  plays  $a_{i}$ :
$$
u _ {i} \left(a _ {i}, x _ {- i}\right) = \sum_ {a _ {- i} \in A _ {- i}} u _ {i} \left(a _ {i}, a _ {- i}\right) \prod_ {j \in - i} x _ {j} \left(a _ {- i} (j)\right).
$$
<sup>1</sup>Here, if all team members play strategies according to an NE minimizing the adversary's utility, the adversary cannot deviate from the equilibrium strategy to obtain a higher utility.
2Our methods mostly apply to normal-form games including green security games mentioned in Section 1. Extensive-form games can be first converted to normal-form games to be solved, and exploiting their game structure is the future work.
In this paper, we consider multiplayer games, i.e.,  $n > 2$ .
A Nash Equilibrium (NE, and NEs for Nash Equilibria) [30] is a stable strategy profile in which no player has an incentive to change her strategy given other players' strategies and always exists. Formally, a strategy profile  $x^{*}$  is an NE if, for each player  $i$ ,  $x_{i}^{*}$  is a best response to  $x_{-i}^{*}$ , i.e.,  $u_{i}(x_{i}^{*}, x_{-i}^{*}) \geq u_{i}(x_{i}, x_{-i}^{*}), \forall x_{i} \in X_{i}$ , which is equivalent to  $u_{i}(x_{i}^{*}, x_{-i}^{*}) \geq u_{i}(a_{i}, x_{-i}^{*}), \forall a_{i} \in A_{i}$ .
With the above condition of NEs, we could use a multilinear program to represent the space of NEs, but it will involve the product of strategies in  $u_{i}(x)$ , whose degree is  $n$  and is higher than the product of strategies in  $u_{i}(a_{i}, x_{-i})$ . To reduce the degree of the program representing the space of NEs from  $n$  to  $n - 1$  (i.e., only the product of strategies in  $u_{i}(a_{i}, x_{-i})$  is required), in two-player games, the previous work [34] exploited the following NE's property, which can be used in multiplayer games as well. For each strategy profile  $x \in X$ , the regret of an action  $a_{i}$  is the difference in player  $i$ 's expected utility between playing  $x_{i}$  in  $x$  and playing  $a_{i}$ , i.e.,  $u_{i}(x) - u_{i}(a_{i}, x_{-i})$ . Obviously, a strategy profile  $x \in X$  is an NE if and only if every action either has the regret 0, or is played with the probability 0 in  $x$ . Then the space of NEs of a game can be formulated as the feasible solution space of a mixed-integer program by using a binary variable  $b_{a_{i}}$  to represent that any action  $a_{i}$  either has the regret 0, or is played with the probability 0:
$$
u _ {i} \left(a _ {i}, x _ {- i}\right) = \sum_ {a _ {- i} \in A _ {- i}} u _ {i} \left(a _ {i}, a _ {- i}\right) \prod_ {j \in - i} x _ {j} \left(a _ {- i} (j)\right) \quad \forall i, a _ {i} \in A _ {i} \tag {1a}
$$
$$
\sum_ {a _ {i} \in A _ {i}} x _ {i} \left(a _ {i}\right) = 1 \quad \forall i \in N \tag {1b}
$$
$$
1 - b _ {a _ {i}} \geq x _ {i} (a _ {i}) \quad \forall i \in N, a _ {i} \in A _ {i} \tag {1c}
$$
$$
u _ {i} (x) \geq u _ {i} \left(a _ {i}, x _ {- i}\right) \quad \forall i \in N, a _ {i} \in A _ {i} \tag {1d}
$$
$$
u _ {i} (x) - u _ {i} \left(a _ {i}, x _ {- i}\right) \leq b _ {a _ {i}} \left(U _ {\max } - U _ {\min }\right) \quad \forall i \in N, a _ {i} \in A _ {i}, \tag {1e}
$$
$$
u _ {i} \left(a _ {i}, x _ {- i}\right) \in \left[ U _ {\min }, U _ {\max }\right], u _ {i} (x) \in \left[ U _ {\min }, U _ {\max } \right] \forall i \in N, a _ {i} \in A _ {i}, \tag {1f}
$$
$$
x _ {i} \left(a _ {i}\right) \in [ 0, 1 ], b _ {a _ {i}} \in \{0, 1 \}, \quad \forall i \in N, a _ {i} \in A _ {i}, \tag {1g}
$$
where we use the notations of utility functions  $u_{i}(x)$  and  $u_{i}(a_{i}, x_{-i})$  to represent the corresponding variables in the program. Eq.(1c) ensures that binary variable  $b_{a_i}$  is set to 0 when  $x_{i}(a_{i}) > 0$  and can be set to 1 only when  $x_{i}(a_{i}) = 0$ ; and Eq.(1e) ensures that the regret of action  $a_i$  equals 0 (i.e.,  $u_{i}(x) = u_{i}(a_{i}, x_{-i})$ ), unless  $b_{a_i} = 1$  where the constraint  $u_{i}(x) - u_{i}(a_{i}, x_{-i}) \leq (U_{max} - U_{min})$  always holds.
An optimal NE is an NE optimizing an objective function  $g(x)$  over the space of NEs, where  $g(x)$  is a linear objective function and could be maximizing social welfare, maximizing the expected utilities of one player or several players, maximizing the minimum utility among players, minimizing the support sizes of the NE strategies, and so on. Unfortunately, finding an optimal NE optimizing the above objectives is NP-hard [9].
3 Computing Optimal Nash Equilibria
The problem of finding an optimal NE in multiplayer games requires optimizing an objective over the space of NEs. This space is represented by Eq.(1), which involves nonlinear terms in Eq.(1a) to represent the strategies of players in  $-i$ , which is bilinear when  $n = 3$  and is multilinear when  $n \geq 4$ . The multilinear program is usually transformed into a bilinear program to make the program solvable using global optimization solvers, e.g., Gurobi [19]. Here, we propose a general framework for this transformation based on a set of correlation plans for any binary collection of subsets of players, where each set in this collection is divided into two disjoint sets, and each correlation plan corresponds to a set of auxiliary variables representing a set of bilinear terms. However, there are two challenges for solving this bilinear program: 1) this bilinear program usually involves a large number of bilinear terms, and 2) an important step used by state-of-the-art algorithms to solve such bilinear programs is to use convex relaxation to replace each bilinear term in the program [15, 18], which significantly enlarges the feasible solution space. To overcome these challenges, we develop a novel algorithm called CRM that uses Correlation plans with their Relations to strictly reduce the feasible
<sup>3</sup>If  $g(x)$  is nonlinear, we can use a variable (i.e., a linear function)  $v$  as the new objective with the constraint such that  $v = g(x)$
solution space after the convex relaxation while Minimizing the number of correlation plans to reduce the number of bilinear terms. Section 3.4 shows that our techniques can significantly reduce the time complexity. The procedure of CRM is shown in Algorithm 2, which is illustrated in Appendix A.
3.1 A General Transformation Framework
A correlation plan is a probability distribution over the joint action space of a subset of players, and we focus on correlation plans for certain special collections of subsets of players, which can be used to transform a multilinear program for computing optimal NEs into a bilinear program.
Definition 1. A collection  $\mathcal{N}$  of subsets of players is a binary collection if:
1.  $\{-i\mid i\in N\} \subseteq \mathcal{N}$
2. for each  $N^{\prime}\in \mathcal{N}$ $N^{\prime}\subset N$  with  $|N^{\prime}|\geq 2$  and
3. for each  $N' \in \mathcal{N}$ , there are two disjoint children  $N'_l$  and  $N'_r$  in  $\{\{i\} \mid i \in N\} \cup \mathcal{N}$  such that  $N'_l \cap N'_r = \emptyset$  and  $N' = N'_l \cup N'_r$ , i.e.,  $N'$  is divided into two disjoint sets.
Let  $N_{l}^{\prime}$  and  $N_{r}^{\prime}$  be the left child and the right child of  $N^{\prime} \in \mathcal{N}$ , respectively. For each  $N^{\prime}$  in any binary collection  $\mathcal{N}$ , a correlation plan of  $N^{\prime}$  is a probability distribution  $x_{N^{\prime}}$  over  $A_{N^{\prime}}$ : given  $x_{N^{\prime}}(a_{N^{\prime}}) \in [0,1]$  ( $\forall a_{N^{\prime}} \in A_{N^{\prime}}, N^{\prime} \in \mathcal{N}$ ),
$$
\sum_ {a _ {N ^ {\prime}} \in A _ {N ^ {\prime}}} x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) = 1 \quad \forall N ^ {\prime} \in \mathcal {N}. \tag {2}
$$
For simplification, let  $i$  be equivalent to  $\{i\}$  for each  $i \in N$ . That is,  $x_{i}$  is a special correlation plan  $x_{\{i\}}$  (i.e.,  $x_{i} = x_{\{i\}}$ ),  $a_{i} \in A_{i}$  is a special joint action  $a_{\{i\}} \in A_{\{i\}}$  (i.e.,  $a_{i} = a_{\{i\}}$ ). Each element  $N'$  in a binary collection  $\mathcal{N}$  has the binary division, i.e., it is divided into two disjoint sets  $N_{l}'$  and  $N_{r}'$ . Based on this binary division, any joint action  $a_{N'} \in A_{N'}$  can be divided into two sub-joint actions  $a_{N_{l}'} \in A_{N_{l}'}$  and  $a_{N_{r}'} \in A_{N_{r}'}$  such that  $a_{N'} = (a_{N_{l}'}', a_{N_{r}'}')$ . Then we can use this binary division to ensure that  $\prod_{j \in N'} x_{j}(a_{N'}(j)) = x_{N'}(a_{N'})$  for the correlation plan  $x_{N'}$ , as shown in Example 1.
Example 1.  $\{\{1,2,3\}, \{1,2,4\}, \{1,3,4\}, \{2,3,4\}, \{1,2\}, \{1,3\}, \{2,3\}, \{1,4\}, \{3,4\}, \{2,4\}\}$  is a binary collection for a four-player game. For  $N' = \{1,2,3\}$  in this collection with  $N'_l = \{1,2\}$  (having two children  $\{1\}$  and  $\{2\}$ ) and  $N'_r = \{3\}$ , we have  $a_{N'} = (a_1, a_2, a_3) = (a_{\{1,2\}}, a_3) \in A_{N'}$  and  $a_{\{1,2\}} = (a_1, a_2) \in A_{\{1,2\}}$ . Then we can have a chain of bilinear constraints (equalities):  $x_{N'}(a_{N'}) = x_{\{1,2\}}(a_{\{1,2\}}) x_3(a_3)$  and  $x_{\{1,2\}}(a_{\{1,2\}}) = x_1(a_1) x_2(a_2)$ , which guarantees that  $x_{N'}(a_{N'}) = x_1(a_1) x_2(a_2) x_3(a_3)$ . In other words, we use  $x_{\{1,2\}}(a_{\{1,2\}})$  and  $x_{N'}(a_{N'})$  as the auxiliary variables to represent bilinear terms  $x_1(a_1) x_2(a_2)$  and  $x_{\{1,2\}}(a_{\{1,2\}}) x_3(a_3)$ , respectively.
This property of correlation plans of a binary collection  $\mathcal{N}$  can be used to transform the multilinear Program (1) into a bilinear program. First, we use the binary division of each element  $N^{\prime}$  in  $\mathcal{N}$  to connect correlation plans, i.e., for any  $N^{\prime} \in \mathcal{N}$  with its children  $N_{l}^{\prime}$  and  $N_{r}^{\prime}$ :
$$
x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) = x _ {N _ {l} ^ {\prime}} \left(a _ {N _ {l} ^ {\prime}}\right) x _ {N _ {r} ^ {\prime}} \left(a _ {N _ {r} ^ {\prime}}\right) \quad \forall a _ {N ^ {\prime}} = \left(a _ {N _ {l} ^ {\prime}}, a _ {N _ {r} ^ {\prime}}\right) \in A _ {N ^ {\prime}} \tag {3a}
$$
$$
x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) \in [ 0, 1 ] \quad \forall a _ {N ^ {\prime}} \in A _ {N ^ {\prime}}. \tag {3b}
$$
Second, we replace  $\prod_{j\in -i}x_j(a_{-i}(j))$  in Eq.(1a) with  $x_{-i}(a_{-i})$
$$
u _ {i} \left(a _ {i}, x _ {- i}\right) = \sum_ {a _ {- i} \in A _ {- i}} u _ {i} \left(a _ {i}, a _ {- i}\right) x _ {- i} \left(a _ {- i}\right) \quad \forall i \in N, a _ {i} \in A _ {i}. \tag {4}
$$
In the above transformation, each correlation plan corresponds to a set of auxiliary variables (e.g.,  $x_{N'}(a_{N'})$ ) representing a set of bilinear terms (e.g.,  $x_{N'_l}(a_{N'_l})x_{N'_r}(a_{N'_r})$ ). Eq.(3) guarantees that  $\prod_{j\in N'}x_j(a_{N'}(j)) = x_{N'}(a_{N'})$ , and then the feasible solution space of Eqs.(1b)-(1g), (3), and (4) represents the space of NEs.
Theorem 1. The feasible solution space of mixed strategies (i.e.,  $x_{i}(a_{i})$  for each  $i \in N$ ,  $a_{i} \in A_{i}$ ) in Eqs.(1b)-(1g), (3), and (4) is the space of NEs. (Proofs are in Appendix B.)
We can then compute an optimal NE by solving the following mixed-integer bilinear program according to any binary collection  $\mathcal{N}$ :
$$
\max  _ {x} g (x) \tag {5a}
$$
$$
\text {s . t .} \quad \text {E q s .} (1 b) - (1 g), (3), (4). \tag {5b}
$$
It is straightforward to solve Program (5) by using the vanilla binary collection  $\overline{\mathcal{N}}$  that includes all non-singleton proper subsets of  $N$ , i.e.,  $\overline{\mathcal{N}} = \{N' \mid N' \subset N, |N'| \geq 2\}$ , where, for each  $N' \in \overline{\mathcal{N}}$ ,  $N'_l$  is  $N' \setminus \{j\}$  and  $N'_r$  is  $\{j = \max_{i \in N'} i\}$ . Example 1 provides an example of  $\overline{\mathcal{N}}$ .
3.2 Exploit Correlation Plans with Their Relations
In this section, we use correlation plans with their relations to restrict the feasible solution space after the convex relaxation. The common convex relaxation technique [27, 35, 18] before searching for the optimal solution is: each bilinear term  $x_{N'}(a_{N'}) = y_1y_2$  with  $y_1, y_2 \in [0,1]$  is represented by the following constraints including four linear constraints:
$$
\max  \left\{0, y _ {1} + y _ {2} - 1 \right\} \leq x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) \leq \min  \left\{y _ {1}, y _ {2} \right\}, \tag {6}
$$
which significantly enlarges the feasible solution space. We now show the motivation to use correlation plans with their relations to reduce this feasible solution space.
Example 2. Given  $N' = \{2,4\} \subset N$  with two actions for each player (i.e.,  $A_i = \{a_i,a_i'\}$ ) in a game  $G$ , by Eq.(6), bilinear terms (e.g.,  $x_{N'}(a_2,a_4) = x_2(a_2)x_4(a_4)$ ) are relaxed according to Eq.(6), e.g.,  $\max \{0,x_2(a_2) + x_4(a_4) - 1\} \leq x_{N'}(a_2,a_4) \leq \min \{x_2(a_2),x_4(a_4)\}$ . With additional constraints by Eq.(1b) (e.g.,  $x_4(a_4) + x_4(a_4') = 1$ ), the following assignment could be a feasible solution:
$$
\begin{array}{l} x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2}, a _ {4} ^ {\prime}\right) = x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right) \\ \begin{array}{l} x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) \\ = x _ {2} \left(a _ {2}\right) = x _ {2} \left(a _ {2} ^ {\prime}\right) = x _ {4} \left(a _ {4}\right) = x _ {4} \left(a _ {4} ^ {\prime}\right) = 0. 5. \end{array} \tag {7} \\ \end{array}
$$
Obviously, in Eq.(7),  $x_{N'}(a_2, a_4)$  is not equal to  $x_2(a_2)x_4(a_4)$ . In fact, based on Eq.(2), we have:
$$
x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) + x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) + x _ {N ^ {\prime}} \left(a _ {2}, a _ {4} ^ {\prime}\right) + x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right) = 1, \tag {8}
$$
which will make the solution in Eq.(7) infeasible. Moreover, the following assignment is a feasible solution after the relaxation and satisfies Eq.(8):
$$
\begin{array}{l} x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) = x _ {N ^ {\prime}} \left(a _ {2}, a _ {4} ^ {\prime}\right) = 0. 2, \\ \begin{array}{l} w _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) = w _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) = w _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) = 0. 2, \\ x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right) = 0. 4, x _ {2} \left(a _ {2}\right) = x _ {4} \left(a _ {4}\right) = 0. 5. \end{array} \tag {9} \\ \end{array}
$$
However, in Eq.(9),  $x_{N'}(a_2, a_4)$  is still not equal to  $x_2(a_2)x_4(a_4)$ . Actually, we have:
$$
\begin{array}{l} x _ {N ^ {\prime}} \left(a _ {2}, a _ {4}\right) + x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4}\right) = x _ {4} \left(a _ {4}\right), \\ \begin{array}{l} w _ {N} \left(a _ {2}, a _ {4}\right) + w _ {N} \left(a _ {2}, a _ {4}\right) = w _ {4} \left(a _ {4}\right), \\ x _ {N ^ {\prime}} \left(a _ {2}, a _ {4} ^ {\prime}\right) + x _ {N ^ {\prime}} \left(a _ {2} ^ {\prime}, a _ {4} ^ {\prime}\right) = x _ {4} \left(a _ {4} ^ {\prime}\right), \end{array} \tag {10} \\ \end{array}
$$
which will make the solution in Eq.(9) infeasible.
This above example shows that we can use the definition of a correlation plan (i.e., Eq.(2)) and the relation of correlation plans to reduce the feasible solution space after the relaxation.
Each element  $N'$  in any binary collection  $\mathcal{N}$  is defined by  $N_{l}'$  and  $N_{r}'$ , which actually defines relations between correlation plans for elements in  $\{\{i\} \mid i \in N\} \cup \mathcal{N}$ . In Example 2, Eq.(10) actually represents a relation between the correlation plan  $x_{N'}$  and the mixed strategy  $x_{4}$  (i.e., the special correlation plan  $x_{\{4\}}$ ). Formally, for any  $N' \in \mathcal{N}$  and  $i \in N'$ :
$$
\sum_ {a _ {N ^ {\prime}} \in A _ {N ^ {\prime}}, a _ {N ^ {\prime}} (i) = a _ {i}} x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) = x _ {i} \left(a _ {i}\right) \quad \forall a _ {i} \in A _ {i}, \tag {11}
$$
where  $a_{N'}(i)$  is the action of player  $i \in N'$  in the joint action  $a_{N'}$ . Similarly, let  $a_{N'}(N'')$  be the sub-joint action of player  $N'' \subset N'$  in the joint action  $a_{N'}$ . Then, for any  $N' \in \mathcal{N}$  with its children  $N'_l$  and  $N'_r$ , and  $N'' \in \{N'_l, N'_r\}$ :
$$
\sum_ {a _ {N ^ {\prime}} \in A _ {N ^ {\prime}}, a _ {N ^ {\prime}} (N ^ {\prime \prime}) = a _ {N ^ {\prime \prime}}} x _ {N ^ {\prime}} \left(a _ {N ^ {\prime}}\right) = x _ {N ^ {\prime \prime}} \left(a _ {N ^ {\prime \prime}}\right) \quad \forall a _ {N ^ {\prime \prime}} \in A _ {N ^ {\prime \prime}} \tag {12}
$$
Algorithm 1 Generate  $\mathcal{N}$