SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 6: Ablation study. No time limit for CRM, CR, and CM in games (8, 2). We can see that each component of our approach significantly boosts its performance. Note that  $\overline{\mathcal{N}}$  (in CR, C, and MIBP) and  $\underline{\mathcal{N}}$  (in CRM, CM, and M) result in the same bilinear terms in three-player games because each element in  $\{-i\mid i\in \{1,2,3\}\}$  includes only two elements such that Algorithm 1 cannot reduce the number of internal nodes to reduce the number of bilinear terms, where CR and CRM (or C and CM, or MIBP and M) have the same performance. For one case in the game (9, 2), which CRM and CM cannot solve within 1000s, after removing the time limit for it, CRM solves it by using 1198s, but CM solves it by using 12751s. The unit of the runtime is second.
Runtime ± 95% Confidence Interval (Percentage of Games not Solved)
Game	CRM	CR	CM	C	M	MIBP
(9,2)	658±128 (50%)	988 ± 23 (97%)	782±113 (63%)	1000±0 (100%)	1000 ± 0 (100%)	1000 ± 0 (100%)
(8,2)	166±97	2334± 1742	278± 207	763 ± 120 (60%)	1000 ± 0 (100%)	1000 ± 0 (100%)
(7,2)	25 ± 17	89 ± 51	36 ± 28	408 ± 157 (30%)	488 ± 111 (10%)	429 ± 131 (20%)
(3,15)	167± 86 (3%)	167 ± 86 (3%)	317± 137 (17%)	317 ± 137 (17%)	558 ± 150 (40%)	558 ± 150 (40%)
(3,17)	231±122 (10%)	231 ±122 (10%)	326± 134 (20%)	326 ± 134 (20%)	784 ± 102 (53%)	784 ± 102 (53%)
Bidirectional LEG	1.6±1	5.4 ± 4	2.2 ± 2	86 ± 3	991 ± 18 (97%)	972 ± 54 (97%)
Collaboration	1 ± 0.2	2 ± 0.2	1 ± 0.1	2 ± 0.4	867 ± 122 (87%)	967 ± 63 (97%)
Covariant r = 0.5	5 ± 6	12 ± 10	5 ± 6	18 ± 18	1000 ± 0 (100%)	1000 ± 0 (100%)
PolyMatrix	26 ± 44	33 ± 25	36 ± 64 (3%)	17 ± 21	267 ± 97 (10%)	194 ± 74 (3%)
Random LEG	2 ± 1	6 ± 5	2.5 ± 2	5 ± 5	1000 ± 0 (100%)	1000 ± 0 (100%)
Random graphical	0.1 ± 0.1	0.4 ± 0.1	0.2 ± 0.1	0.6 ± 0.4	814 ± 134(80%)	803 ± 140(83%)
Uniform LEG	2.2 ± 1	5 ± 4	2.5 ± 2	5 ± 5	999 ± 2 (97%)	1000 ± 0 (100%)
to efficiently evaluate the algorithms, we set a time limit of 1000 seconds for each case unless stated otherwise.
Metric: We use the runtime and the percentage of games that are not solved within the time limit to measure the performance of our approach. In addition to the average runtime, we show a  $95\%$  confidence interval. CRM and MIBP guarantee finding an optimal NE. For algorithms that can enumerate all NEs, we can choose an optimal NE from the output for all NEs. We then compare ENUMPOLY to our approach only in the runtime. For algorithms that only guarantee to converge to an NE, we may need to use them to enumerate all NEs. However, it is unclear whether algorithms like EXCLUSION can enumerate all NEs. Therefore, we compare EXCLUSION to our approach in the runtime and the utility gap. The utility gap is the relative distance between the optimal objective value  $(g^{*})$  in our problem (returned by CRM) and the objective value  $(g_{0})$  in the Nash equilibrium returned by EXCLUSION, i.e.,  $|g^{*} - g_{0}| / |g_{0}| \times 100\%$ . In some cases, a solution is returned even if it has not reached the given accuracy within the time limit, which is still used as a solution of EXCLUSION. A larger gap means that we will lose more while using EXCLUSION.
G Game Size
Table 3 shows the number of integer variables, bilinear terms, and correlation plans for the games we used in experiments. Note that the used GAMUT games have six players and three actions. Also, note that we do not reduce the number of bilinear terms and correlation plans in games with only three players.
H Details of Experimental Results
The details of experimental results are in Tables 4, 5, and 6.
Specially, CR is solving Program (13) based on  $\overline{\mathcal{N}}$ , and CRM is solving Program (13) based on  $\underline{\mathcal{N}}$ . We discussed the difference between  $\overline{\mathcal{N}}$  and  $\underline{\mathcal{N}}$  in Section 3.3. Basically,  $\overline{\mathcal{N}}$  and  $\underline{\mathcal{N}}$  are the same in
Table 7: Results on more Gambit algorithms for random games. The format for each result is: Average Runtime  $\pm 95\%$  Confidence Interval (Percentage of Games not Solved within the Time Limit). Note that, these Gambit algorithms only achieve some NE if the game is solved, which may not be optimal. Even so, these algorithms fail to solve many games, and even run significantly slower than our CRM (see Table 4) in many games.
Runtime ± 95% Confidence Interval (Percentage of Games not Solved)
(n,m)	GNM	IPA	LIAP	SIMPDIV	LOGIT
(3,2)	0.03 ± 0.02	567 ± 177 (57%)	0.06 ± 0.02 (77%)	0.07 ± 0.06	0.04 ± 0.02 (100%)
(5,2)	0.04 ± 0.01 (3%)	867 ± 122 (87%)	0.45 ± 0.04 (100%)	1000 ± 0 (100%)	0.02 ± 0 (100%)
(7,2)	333 ± 169 (53%)	400 ± 175 (37%)	6 ± 0.4 (100%)	79.4 ± 78.6	0.05 ± 0.05 (100%)
(3,3)	0.03 ± 0 (3%)	933 ± 89 (93%)	0.16 ± 0.02 (100%)	300 ± 163 (30%)	0.04 ± 0.02 (100%)
(4,3)	0.17 ± 0.11 (3%)	500 ± 179 (50%)	0.76 ± 0.08 (100%)	500 ± 179 (50%)	0.02 ± 0 (100%)
(5,3)	0.15 ± 0.03 (30%)	773 ± 158 (73%)	6.3 ± 0.3 (100%)	900 ± 107 (90%)	0.02 ± 0.01 (100%)
(4,2)	0.03 ± 0.01 (3%)	667 ± 169 (63%)	0.13 ± 0.02 (97%)	733 ± 158 (73%)	0.02 ± 0 (100%)
(4,3)	0.17 ± 0.11 (3%)	500 ± 179 (50%)	0.76 ± 0.08 (100%)	500 ± 179 (50%)	0.02 ± 0 (100%)
(4,4)	800 ± 143 (80%)	367 ± 172 (33%)	4.9 ± 0.4 (100%)	867 ± 121 (87%)	0.07 ± 0.07 (100%)
(4,5)	0.33 ± 0.07 (37%)	833 ± 133 (83%)	16 ± 1 (100%)	900 ± 107 (90%)	0.04 ± 0.03 (100%)
(3,5)	600 ± 175 (63%)	767 ± 151 (77%)	1.5 ± 0.1 (100%)	933 ± 89 (93%)	0.06 ± 0.09 (100%)
(3,8)	0.76 ± 0.26 (17%)	506 ± 177 (50%)	11 ± 0.7 (100%)	867 ± 122 (87%)	0.02 ± 0 (100%)
(3,10)	334 ± 168 (53%)	767 ± 153 (77%)	37 ± 3 (100%)	867 ± 121 (87%)	0.02 ± 0 (100%)
(3,13)	3.8 ± 1.1 (47%)	1000 ± 0 (100%)	132 ± 11 (100%)	805 ± 140 (80%)	0.03 ± 0 (100%)
Table 8: Results on more Gambit algorithms for six-player three-action GAMUT games. The format for each result is: Average Runtime ± 95% Confidence Interval (Percentage of Games not Solved within the Time Limit). Note that, these Gambit algorithms only achieve some NE if the game is solved, which may not be optimal. Even so, these algorithms fail to solve many games, and even run significantly slower than our CRM (see Table 5) in many games.
Runtime ± 95% Confidence Interval (Percentage of Games not Solved)
Game	GNM	IPA	LIAP	SIMPDIV	LOGIT
Bidirectional LEG	167 ± 133 (63%)	10 ± 17	8 ± 0.6 (100%)	667 ± 169 (67%)	0.03 ± 0 (100%)
Collaboration	34 ± 64 (3%)	0.03 ± 0	24 ± 2 (100%)	0.03 ± 0	0.03 ± 0 (100%)
Covariant r = 0.5	100 ± 107 (17%)	0.04 ± 0	21 ± 2 (100%)	367 ± 172 (37%)	0.04 ± 0.03 (100%)
PolyMatrix	0.13 ± 0.03 (7%)	8.3 ± 8	9 ± 0.8 (100%)	24 ± 27	0.05 ± 0.05 (100%)
Random LEG	0.19 ± 0.04 (47%)	0.04 ± 0	7.5 ± 0.6 (100%)	777 ± 146 (77%)	0.03 ± 0 (100%)
Random graphical	0.05 ± 0 (3%)	1000 ± 0 (100%)	9 ± 1 (100%)	0.05 ± 0.03	0.04 ± 0.02 (100%)
Uniform LEG	0.16 ± 0.04 (47%)	0.04 ± 0	7.3 ± 0.6 (100%)	776 ± 146 (77%)	0.02 ± 0 (100%)
3-player games, so CR and CRM have the same performance in 3-player games shown in Table 6. When the number of players increases, the size of  $\overline{\mathcal{N}}$  is larger and larger than  $\underline{\mathcal{N}}$ , and then CRM's advantage over CR is more significant. This statement is verified by our result: game (7, 2): CRM with  $25 \pm 17$  and CR with  $89 \pm 51$ ; and game (8, 2): CRM with  $156 \pm 83$  ( $3\%$ ) and CR with  $612 \pm 129$  ( $33\%$ ) (see Table 2). However, from the game (7, 2) to the game (8, 2), the trend that CRM's advantage over CR is more significant is not very clear based on the above data due to the time limit in the game (8, 2). To show this trend clearly, we remove the time limit for CRM and CR in the game (8, 2) and obtain: CRM with  $166 \pm 97$  and CR with  $2334 \pm 1742$ , which is shown in Table 6. Then from the game (7, 2), where CRM is about 3 times faster than CR, to the game (8, 2), where CRM is about 13 times faster than CR, we can clearly see the trend that CRM's advantage over CR is more significant when the number of players increases. The difference between  $\overline{\mathcal{N}}$  and  $\underline{\mathcal{N}}$  also explains that CR is slower than CM in the game (8, 2).
I Results on More Gambit Algorithms
Results in Tables 7 and 8 show that Gambit algorithms, i.e., GNM, IPA, LIAP, SIMPDIV, and LOGIT, cannot guarantee finding an NE, i.e., fail to solve many games, and even run significantly slower than our CRM (see Tables 4 and 5) in many games:
- GNM fails to solve many large games: GNM stops without output in some of these cases, and cannot stop within the time limit in other cases. GNM runs significantly slower than our CRM in many games, e.g., random games (7, 2), (4, 4), (3, 5), (3, 10), and GAMUT games Bidirectional LEG, Collaboration, Covariant.
- IPA fails to solve many large games: IPA cannot stop within the time limit in most of these games. IPA runs significantly slower than our CRM in most random games, and the GAMUT game Random graphical.
- LIAP can only solve several games in small random games (3, 2) and (4, 2), and fails to solve all of the other games: LIAP stops without output in these games. Even so, LIAP runs significantly slower than our CRM in most games.
- SIMPDIV fails to solve many large games: SIMPDIV cannot stop within the time limit in almost all of these games. SIMPDIV runs significantly slower than our CRM in most games.
- LOGIT fails to solve all of these games: LOGIT stops without output in all of these games. LOGIT may not work properly in this latest GAMBIT version. Even so, LOGIT runs significantly slower than our CRM in the random game (3, 2).