SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Liar's Dice is another standard benchmark in the EFG-solving community [29]. In the version that we test on, each player rolls an unbiased six-sided die, and they take turns either calling higher bids or challenging the other player. A bid consists of a combination of a value  $v$  between one and six, and a number of dice between one and two,  $n$ , representing the number of dice between the two players that has  $v$  pips showing. A higher bid involves either increasing  $n$  holding  $v$  fixed, increasing  $v$  holding  $n$  fixed, or both. When a player is challenged (or the highest possible bid of "two dice each showing six pips" is called), the dice are revealed, and whoever is correct wins 1 (either the challenger if the bid is not true, or the player who last called a bid, if the bid is true), and the other player receives a utility of -1.
E.3 Leduc (13 ranks)
Leduc is yet another standard benchmark in the EFG-solving community [40] and is a simplified version of Texas Hold'em. In the version we test on, there are 13 unique cards (ranks), and there are 2 copies of each rank (half the size of a standard 52 card deck). There are two rounds of betting that take place, and before the first round each player places an ante of 1 into the pot, and is dealt a single pocket (private) card. In addition, two cards are placed face down, and these are community cards that will be used to form hands. The two hands that can be formed with the community cards are pair, and highest card.
During the first round of betting, player 1 acts first. There is a max of two raises allowed in each round of betting. Each player can either check, raise, or fold. If a player folds, the other player immediately wins the pots and the game terminates. If a player checks, the other player has an opportunity to raise if they have not already previously checked or raised, and if they previously checked, the game moves on to the next round. If a player raises, the other player has an opportunity to raise if they have not already previously raised. The game then moves on the second round, during which one of the community cards is placed face up, and then similar betting dynamics as the first round take place. After the second round terminates, there is a showdown, and whoever can form the better hand (higher ranked pair, or highest card) with the community cards takes the pot.
E.4 Battleship
This is an instantiation of the classic board game, Battleship, in which players take turns shooting at the opposing player's ships. Before the game begins, the players place two ships of length 2 and value 4, on a grid of size 2 by 3. The ships need to be placed in a way so that the ships take up exactly four spaces within the grid (they do not overlap with each other, and are contained entirely in the grid). Each player gets three shots, and players take turns firing at cells of their opponent's grid. A ship is sunk when the two cells it has been placed on have been shot at. At the end of the game, the utility for a player is the difference between the cumulative value of the opponent's sunk ships and the cumulative value of the player's sunk ships.
F Block Construction Strategies
As discussed in the main paper, the postorder block construction strategy can be viewed as traversing the decision points of the treeplex in postorder, treating decision points with the same parent sequence as the same node, and then greedily putting decision points in the same block until we reach a decision point that has a child decision point in the current block (at which point we start a new block). We make this postorder traversal and greedy block construction explicit in Algorithm 3.
In Algorithm 4 we provide pseudocode for constructing blocks using the children block construction strategy. As discussed in the main paper, the children block construction strategy corresponds to placing decision points with the same parent decision point (same decision point at which their parent sequences start at) in the same block. In our implementation, instead of doing a bottom-up traversal, we do a top down implementation, and at the end, reverse the order of the blocks (this allows us to respect the treeplex ordering).
In both Algorithm 3 and Algorithm 4,  $\varnothing$  represents the empty sequence.
Algorithm 3 Postorder Block Construction
1: procedure POSTORDERHELPER(j,a)  
2: accumulator = []  
3: for  $j' \in \mathcal{C}_{j,a}$  do  
4: for  $a' \in A_{j'}$  do  
5: accumulator.insert(postorder(j', a'))  
6: for  $j' \in \mathcal{C}_{j,a}$  do  
7: accumulator.insert(j')  
8: Return: accumulator  
9: procedure POSTORDERBLOCKS(J)  
10: ordered = POSTORDERHELPER()  
11: blocks = []  
12: current_block = []  
13: for  $j \in$  ordered do  
14: if  $\exists j' \in$  current_block.s.t.  $j'$  is a child decision point of  $j$  then  
15: blocks.insert(current_block)  
16: current_block = [j]  
17: else  
18: current_block.insert(j)  
19: return: blocks
Algorithm 4 Children Block Construction
1: procedure CHILDRENBLOCKS(J)  
2: blocks = []  
3: explore = C0  
4: for j ∈ explore do  
5: current_block = []  
6: for a ∈ Aj do  
7: for j' ∈ Cj,a do  
8: current_block.insert(j')  
9: explore.insert(j')  
10: return: blocks.reverse()
We can now illustrate each of the block construction strategies on the treeplex for player 1 in Kuhn that was presented in Appendix B. If we use single block, then we have  $\mathcal{J}_{\chi}^{(1)} = \mathcal{J}_{\chi} = \{0,1,2,3,4,5,6\}$ . If we use infosets, then we have  $\mathcal{J}_{\chi}^{(i)} = \{7 - i\}$  for  $i \in \{1,2,3,4,5,6,7\}$  (we have to subtract in order to label the infosets in a manner that respects the treeplex ordering). If we use children, then we have  $\mathcal{J}_{\chi}^{(1)} = \{4\}, \mathcal{J}_{\chi}^{(2)} = \{5\}, \mathcal{J}_{\chi}^{(3)} = \{6\}, \mathcal{J}_{\chi}^{(4)} = \{1,2,3\}$ , and  $\mathcal{J}_{\chi}^{(5)} = \{0\}$ . If we use postorder, then we have  $\mathcal{J}_{\chi}^{(1)} = \{4,5,6\}, \mathcal{J}_{\chi}^{(2)} = \{1,2,3\}$ , and  $\mathcal{J}_{\chi}^{(3)} = \{0\}$ .
Note that in the implementation of our algorithm, it is not actually important that the number of blocks for both players are the same; if one player has more blocks than the other, for iterations of our algorithm that correspond to block numbers that do not exist for the other player, we just do not do anything for the other player. Nevertheless, the output of the algorithm does not change if we combine all the blocks for the player with more blocks after the minimum number of blocks between the two players is exceeded, into one block. For example, if player 1 has  $s_1$  blocks, and player 2 has  $s_2$  blocks, with  $s_1 < s_2$ , we can actually combine blocks  $s_1 + 1, \ldots, s_2$  all into the same block for player 2, and this would not change the execution of the algorithm. This is what we do in our implementation.
Additionally, given a choice of a partition of decision points into blocks, there may exist many permutations of decision points within the blocks which satisfy the treeplex ordering of the decision points. Unless the game that is being tested upon possesses some structure which leads to a single canonical ordering of the decision points (which respects the treeplex ordering), an arbitrary decision needs to be made regarding what order is used.
G Experiments
G.1 Additional Experimental Details
Block Construction Strategy Comparison. In this section, we provide additional plots (Figures 6 to 14) comparing different block construction strategies for our algorithm, for specific choices of regularizer and averaging scheme. Note that for the games for which there is a benefit to using blocks (Liar's Dice and Battleship), the benefit is generally apparent across different regularizers and averaging schemes. Furthermore, when there is not a benefit for a particular regularizer and averaging scheme, there is no significant cost either (using blocks does not lead to worse performance).
Block Construction Strategy Comparison with Restarts. We repeat a similar analysis as above (comparing the block construction strategies holding a regularizer and averaging scheme fixed) but this time with the adaptive restarting heuristic applied to our algorithm: the plots can be seen in (Figures 15 to 23). As mentioned in Section 4, we prepend the algorithm name with "r" to denote the restarted variant of the algorithm in the plots.
As discussed in the main body, the trend of the benefit of using blocks being more pronounced with restarting (for games for which blocks are beneficial) holds generally even when holding the regularizer and averaging scheme fixed. This can be seen by comparing each of the restarted block construction strategy comparison plots with the corresponding non-restarted block construction strategy comparison plot.
Figure 6: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated entropy regularizer and uniform averaging.
Figure 7: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilatable global entropy regularizer and uniform averaging.
Figure 8: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated  $\ell^2$  regularizer and uniform averaging.
Figure 9: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated entropy regularizer and linear averaging.
Figure 10: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilatable global entropy regularizer and linear averaging.
Figure 11: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated  $\ell^2$  regularizer and linear averaging.
Figure 12: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated entropy regularizer and quadratic averaging.
Figure 13: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilatable global entropy regularizer and quadratic averaging.
Figure 14: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated  $\ell^2$  regularizer and quadratic averaging.
Figure 15: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated entropy regularizer and uniform averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 16: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilatable global entropy regularizer and uniform averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 17: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated  $\ell^2$  regularizer and uniform averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 18: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated entropy regularizer and linear averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 19: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilatable global entropy regularizer and linear averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 20: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated  $\ell^2$  regularizer and linear averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 21: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated entropy regularizer and quadratic averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 22: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilatable global entropy regularizer and quadratic averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Figure 23: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA with different block construction strategies when using the dilated  $\ell^2$  regularizer and quadratic averaging as well as restarting. We take the best duality gap seen so far so that the plot is monotonic.
Regularizer Comparison. In this section (Figures 24 to 26) we compare the performance of ECyclicPDA and MP instantiated with different regularizers for each averaging scheme, against the performance of  $\mathrm{CFR}^+$  and  $\mathrm{PCFR}^+$ .
It is apparent from these plots, that our algorithm generally outperforms MP, holding the averaging scheme and regularizer fixed. This can be seen by examining the corresponding figure for a choice of averaging scheme, and noting that for any given regularizer, the corresponding MP line is generally above the corresponding ECyclicPDA line.
Regularizer Comparisons with Restarts. We repeat a similar analysis in this section (Figures 27 to 29), instead now comparing the performance of ECyclicPDA and MP instantiated with different regularizers for each averaging scheme, against the performance of  $\mathrm{CFR}^+$  and  $\mathrm{PCFR}^+$ , when all methods are restarted. The trend noted above of our method generally beating MP, even holding the regularizer and averaging scheme fixed, still holds even when restarting.
Figure 24: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA, MP,  $\mathrm{CFR}^+$ ,  $\mathrm{PCFR}^+$ , using a uniform averaging scheme for ECyclicPDA and MP.
Figure 25: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA, MP,  $\mathrm{CFR}^+$ ,  $\mathrm{PCFR}^+$ , using a linear averaging scheme for ECyclicPDA and MP.
Figure 26: Duality gap as a function of the number of full (or equivalent) gradient computations for ECyclicPDA, MP,  $\mathrm{CFR}^+$ ,  $\mathrm{PCFR}^+$ , using a quadratic averaging scheme for ECyclicPDA and MP.
Figure 27: Duality gap as a function of the number of full (or equivalent) gradient computations for when restarting is applied to ECyclicPDA, MP,  $\mathrm{CFR}^+$ ,  $\mathrm{PCFR}^+$ , using a uniform averaging scheme for ECyclicPDA and MP. We take the best duality gap seen so far so that the plot is monotonic.
Figure 28: Duality gap as a function of the number of full (or equivalent) gradient computations for when restarting is applied to ECyclicPDA, MP,  $\mathrm{CFR}^{+}$ ,  $\mathrm{PCFR}^{+}$ , using a linear averaging scheme for ECyclicPDA and MP. We take the best duality gap seen so far so that the plot is monotonic.
Figure 29: Duality gap as a function of the number of full (or equivalent) gradient computations for when restarting is applied to ECyclicPDA, MP,  $\mathrm{CFR}^+$ ,  $\mathrm{PCFR}^+$ , using a quadratic averaging scheme for ECyclicPDA and MP. We take the best duality gap seen so far so that the plot is monotonic.
rECyclicPDA dilated entropy	rECyclicPDA dilated ξ2	rMP dilatable global entropy	rCFR+
rECyclicPDA dilatable global entropy	rMP dilated entropy	rMP dilated ξ2	rPCFR+
Additional wall-clock time experiments In Table 3, we show the wall-clock time required to reach a duality gap of  $10^{-4}$ . It is clear that our algorithm is competitive with MP: MP and its restarted variant time out on Leduc, and MP times out on Battleship (while our algorithm does not even take close to 30 seconds). Furthermore, we are outperforming  $\mathrm{CFR}^+$  and its restarted variant in Battleship.
Table 3: The wall clock time in seconds required for ECyclicPDA, MP,  $\mathrm{CFR}^+$ , and  $\mathrm{PCFR}^+$ , and their restarted variants, denoted using an "r" at the front of the algorithm name, to reach a duality gap of  $10^{-4}$ . We let each algorithm run for at most 30 seconds; a value of 30.000 means the algorithm could not reach the target gap in 30 seconds. The duality gap is computed every 100 iterations.
Name	Goofspiel (4 ranks)	Liar's Dice	Leduc (13 ranks)	Battleship
ECyclicPDA	8.797	6.148	9.343	16.818
rECyclicPDA	6.688	3.519	10.724	4.914
MP	3.183	3.930	30.000	30.000
rMP	2.691	2.360	30.000	8.374
CFR+	5.869	0.111	0.236	6.001
rCFR+	5.866	0.057	0.161	6.042
PCFR+	0.244	0.067	0.225	0.736
rPCFR+	0.208	0.077	0.197	0.736