Original: T HE P OWER OF R EGULARIZATION IN S OLVING E XTENSIVE -F ORM G AMES
Revised: T HE P OWER OF R EGULARIZATION IN S OLVING E XTENSIVE -F ORM G AMES

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In this paper, we investigate the power of regularization , a common technique in reinforcement learning and optimization, in solving extensive-form games (EFGs).
Revised: In this paper, we investigate the power of regularization , a common technique in reinforcement learning and optimization, in solving extensive-form games (EFGs).

Original: We propose a series of new algorithms based on regularizing the payoff functions of the game, and establish a set of convergence results that strictly improve over the existing ones, with either weaker assumptions or stronger convergence guarantees.
Revised: We propose a series of new algorithms based on regularizing the payoff functions of the game, and establish a set of convergence results that strictly improve over the existing ones, with either weaker assumptions or stronger convergence guarantees.

Original: In particular, we first show that dilated optimistic mirror descent (DOMD), an efficient variant of OMD for solving EFGs, with adaptive regularization can achieve a fast (cid:101) O (1 /T ) last-iterate convergence in terms of duality gap without the uniqueness assumption ofthe Nash equilibrium (NE). Moreover, regularized dilated optimistic multiplicative weights update ( Reg-DOMWU ), an instance of Reg-DOMD , further enjoys the (cid:101) O (1 /T ) last-iterate convergence rate of thedistance to the set of NE.
Revised: In particular, we first show that dilated optimistic mirror descent (DOMD), an efficient variant of OMD for solving EFGs, with adaptive regularization can achieve a fast (cid:101) O (1 /T ) last-iterate convergence in terms of duality gap and distance to the set of Nash equilibrium (NE) without uniqueness assumption of the NE.

Original: Second, we show that regularized counterfactual regret minimization ( Reg-CFR ), with a variant of optimistic mirror descent algorithm as regret-minimizer, can achieve O (1 /T 1 / 4 ) best-iterate, and O (1 /T 3 / 4 ) average-iterate convergence rate for finding NE in EFGs.
Revised: Second, we show that regularized counterfactual regret minimization ( Reg-CFR ), with a variant of optimistic mirror descent algorithm as regret-minimizer, can achieve O (1 /T 1 / 4 ) best-iterate, and O (1 /T 3 / 4 ) averageiterate convergence rate for finding NE in EFGs.

Original: Finally, we show that Reg-CFR can achieve asymptotic last-iterate convergence, and optimal O (1 /T ) average-iterate convergence rate, for finding the NE of perturbed EFGs, which is useful for finding approximate extensive-form perfect equilibria (EFPE).
Revised: Finally, we show that Reg-CFR can achieve asymptotic last-iterate convergence, and optimal O (1 /T ) averageiterate convergence rate, for finding the NE of perturbed EFGs, which is useful for finding approximate extensive-form perfect equilibria (EFPE).

Original: To the best of our knowledge, they constitute the first last-iterate convergence results for CFR-type algorithms, while matching the state-of-the-art average-iterate convergence rate in finding NE for non-perturbed EFGs.
Revised: To the best of our knowledge, they constitute the first last-iterate convergence results for CFRtype algorithms, while matching the state-of-the-art average-iterate convergence rate in finding NE for non-perturbed EFGs

Original: We also provide numerical results to corroborate the advantages of our algorithms.
Revised: We also provide numerical results to corroborate the advantages of our algorithms.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Extensive-form games (EFGs) are widely used in modeling sequential decision-making of multiple agents with imperfect information.
Revised: Extensive-form games (EFGs) are widely used in modeling sequential decision-making of multiple agents with imperfect information.

Original: Many popular real-world multi-agent learning problems can be modeled as EFGs, including Poker (Brown and Sandholm, 2018; 2019b), Scotland Yard (Schmid et al., 2021), Bridge (Tian et al., 2020), cloud computing (Kakkad et al., 2019), and auctions (Shubik, 1971), etc.
Revised: Many popular real-world multi-agent learning problems can be modeled as EFGs, including Poker (Brown and Sandholm, 2018; 2019b), Scotland Yard (Schmid et al., 2021), Bridge (Tian et al., 2020), cloud computing (Kakkad et al., 2019), and auctions (Shubik, 1971), etc.

Original: Despite the recent success of many of these applications, efficiently solving large-scale
Revised: Despite the recent success of many of these applications, efficiently solving large-scale

Original: EFGs is still challenging.
Revised: EFGs is still challenging.

Original: Solving EFGs typically refers to as finding a Nash equilibrium (NE) of the game, especially in the two-player zero-sum setting.
Revised: Solving EFGs typically refers to as finding a Nash equilibrium (NE) of the game, especially in the two-player zero-sum setting.

Original: In the past decades, the most popular methods in solving EFGs are arguably regret-minimization based methods, such as counterfactual regret minimization (CFR) (Zinkevich et al., 2007) and its variants (Tammelin et al., 2015; Brown and Sandholm, 2019a).
Revised: In the past decades, the most popular methods in solving EFGs are arguably regret-minimization based methods, such as counterfactual regret minimization (CFR) (Zinkevich et al., 2007) and its variants (Tammelin et al., 2015; Brown and Sandholm, 2019a).

Original: By controlling the regret of each player, the average of strategies constitute an approximated NE in two-player zero-sum games, which is called average-iterate convergence (Zinkevich et al., 2007;
Revised: By controlling the regret of each player, the average of strategies constitute an approximated NE in two-player zero-sum games, which is called average-iterate convergence (Zinkevich et al., 2007;

Original: Tammelin et al., 2015; Farina et al., 2019a).
Revised: Tammelin et al., 2015; Farina et al., 2019a).

Original: However, averaging the strategies can be undesirable, which not only incurs more computation (Bowling et al., 2015) (additional memory and computation for the average strategy), but also introduces additional representation and optimization errors when function approximation is used.
Revised: However, averaging the strategies can be undesirable, which not only incurs more computation (Bowling et al., 2015) (additional memory and computation for the average strategy), but also intro- duces additional representation and optimization errors when function approximation is used.

Original: For example, when using neural networks to parameterize the strategies, the averaged strategy may not be able to be represented properly and the optimization object can be highly non-convex.
Revised: For example, when using neural networks to parameterize the strategies, the averaged strategy may not be able to be represented properly and the optimization object can be highly non-convex.

Original: Therefore, it is imperative to understand if (approximated) NE can be efficiently solved without average, which motivates the study of last-iterate convergence .
Revised: Therefore, it is imperative to understand if (approximated) NE can be efficiently solved without average, which motivates the study of last-iterate convergence .

Original: In fact, the popular CFR-type algorithms mentioned above only enjoy average-iterate convergence guarantees so far (Zinkevich et al., 2007; Tammelin et al., 2015; Farina et al., 2019a), and it is unclear if such a last-iterate convergence is achievable for this type of algorithms.
Revised: In fact, the popular CFR-type algorithms mentioned above only enjoy average-iterate convergence guarantees so far (Zinkevich et al., 2007; Tammelin et al., 2015; Farina et al., 2019a), and it is unclear if such a last-iterate convergence is achievable for this type of algorithms.

Original: The recent advances of Optimistic Mirror Descent (Rakhlin and Sridharan, 2013; Mertikopoulos et al., 2019; Wei et al., 2021; Cai et al., 2022) shed lights on how to achieve last-iterate convergence for solving normal-form games (NFGs), a strict sub-class of EFGs.
Revised: The recent advances of Optimistic Mirror Descent (Rakhlin and Sridharan, 2013; Mertikopoulos et al., 2019; Wei et al., 2021; Cai et al., 2022) shed lights on how to achieve last-iterate convergence for solving normal-form games (NFGs), a strict sub-class of EFGs.

Original: The last-iterate convergence in EFGs has not received attention until recently (Bowling et al., 2015; Farina et al., 2019c; Lee et al., 2021).
Revised: The last-iterate convergence in EFGs has not received attention until recently (Bowling et al., 2015; Farina et al., 2019c; Lee et al., 2021).

Original: Specifically, Bowling et al.
Revised: Specifically, Bowling et al.

Original: (2015) provided some empirical evidence of last-iterate convergence for CFR-type algorithms, while Farina et al.
Revised: (2015) provided some empirical evidence of last-iterate convergence for CFR-type algorithms, while Farina et al.

Original: (2019c) empirically proved that OMD enjoyed last-iterate convergence in EFGs.
Revised: (2019c) empirically proved that OMD enjoyed last-iterate convergence in EFGs.

Original: Lee et al.
Revised: Lee et al.

Original: (2021) proposed an OMD variant with the first last-iterate convergence guarantees in EFGs, but the solution itself might have room for improvement: To make the update computationally efficient, the mirror map needs to be generated through a dilated operation (see §2 for more details); and for this case, the analysis in Lee et al.
Revised: (2021) proposed an OMD variant with the first last-iterate convergence guarantees in EFGs, but the solution itself might have room for improvement: To make the update computationally efficient, the mirror map needs to be generated through a dilated operation (see §2 for more details); and for this case, the analysis in Lee et al.

Original: (2021) requires the NE to be unique.
Revised: (2021) requires the NE to be unique.

Original: In particular, an important and arguably most well-studied instance of OMD for no-regret learning over simplex, i.e., the optimistic multiplicative weights update (OMWU)
Revised: In particular, an important and arguably most well-studied instance of OMD for no-regret learning over simplex, i.e., the optimistic multiplicative weights update (OMWU)

Original: (Daskalakis and Panageas, 2019;
Revised: (Daskalakis and Panageas, 2019;

Original: Wei et al., 2021), cannot be shown to have explicit lastiterate convergence rate so far , without such a uniqueness condition, even for normal-form games.
Revised: Wei et al., 2021), cannot be shown to have explicit lastiterate convergence rate so far , without such a uniqueness condition, even for normal-form games.

Original: Anagnostides et al.
Revised: Anagnostides et al.

Original: (2022) can only guarantee an asymptotic  convergence rate without uniqueness assumption.
Revised: (2022) can only guarantee an asymptotic last-iterate convergence rate without uniqueness assumption 1 .

Original: Indeed, it is left as an open question in (Wei et al., 2021) if the uniqueness condition is necessary for OMWU to converge with an explicit rate for this strict sub-class of EFGs, when constant stepsize is used.
Revised: Indeed, it is left as an open question in (Wei et al., 2021) if the uniqueness condition is necessary for OMWU to converge with an explicit rate for this strict sub-class of EFGs, when constant stepsize is used.

Original: In this paper, we remove the uniqueness condition, while establishing the last-iterate convergence for policy extragradient algorithms with multiplicative updates (we will refer to it as OMWU-type algorithms hereafter) in EFGs.
Revised: In this paper, we remove the uniqueness condition, while establishing the last-iterate convergence for Dilated Optimistic Mirror Descent ( DOMD ) type methods.

Original: The solution relies on exploiting the power of the regularization techniques in EFGs.
Revised: The solution relies on exploiting the power of the regularization techniques in EFGs.

Original: Our last-iterate convergence guarantee is not only for the convergence of duality gap , a common metric used in the literature, but also for the actual iterate, i.e., the convergence of the distance to the set of NE.
Revised: Our last-iterate convergence guarantee is not only for the convergence of duality gap , a common metric used in the literature, but also for the actual iterate, i.e., the convergence of the distance to the set of NE.

Original: This matches the bona fide last-iterate convergence studied in the literature, e.g., Daskalakis and Panageas (2019); Wei et al.
Revised: This matches the bona fide last-iterate convergence studied in the literature, e.g., Daskalakis and Panageas (2019); Wei et al.

Original: (2021), and such a kind of last-iterate guarantee is unknown when the mirror map is either dilated or entropy-based.
Revised: (2021), and such a kind of last-iterate guarantee is unknown when the mirror map is either dilated or entropybased.

Original: More importantly, the techniques we develop can also be applied to CFR, resulting in the first last-iterate convergence guarantee for CFR-type algorithms.
Revised: More importantly, the techniques we develop can also be applied to CFR, resulting in the first last-iterate convergence guarantee for CFR-type algorithms.

Original: We detail our contributions as follows.
Revised: We detail our contributions as follows.

Original: Contributions.
Revised: Contributions.

Original: Our contributions are mainly four-fold: (i)
Revised: Our contributions are mainly four-fold: (i)

Original: We develop a new type of dilated OMD algorithms, an efficient variant of OMD that exploits the structure of EFGs, with adaptive regularization ( Reg-DOMD ), and prove an explicit convergence rate of the duality gap, without the uniqueness assumption of the NE.
Revised: We develop a new type of dilated OMD algorithms, an efficient variant of OMD that exploits the structure of EFGs, with adaptive regularization ( Reg-DOMD ), and prove an explicit convergence rate of the duality gap, without the uniqueness assumption of the NE.

Original: (ii) We further establish a last-iterate convergence rate for dilated optimistic multiplicative weights update to the NE of EFGs (beyond the duality gap as in Cen et al.
Revised: (ii) We further establish a last-iterate convergence rate for dilated optimistic multiplicative weights update to the NE of EFGs (beyond the duality gap as in Cen et al.

Original: (2021b), for the NFG setting), when constant stepsize is used.
Revised: (2021b), for the NFG setting), when constant stepsize is used.

Original: This also moves one step further towards solving the open question for the NFG setting, about whether the uniqueness assumption can be removed to prove last-iterate convergence of the authentic OMWU algorithms with constant stepsizes (Daskalakis and Panageas, 2019; Wei et al., 2021).
Revised: This also moves one step further towards solving the open question for the NFG setting, about whether the uniqueness assumption can be removed to prove last-iterate convergence of the authentic OMWU algorithms with constant stepsizes (Daskalakis and Panageas, 2019; Wei et al., 2021).

Original: (iii)
Revised: (iii)

Original: For CFR-type algorithms, using the regularization technique, we establish the first best-iterate convergence rate of O (1 /T 1 / 4 ) for finding the NE of non-perturbed EFGs, and last-iterate asymptotic convergence for finding the NE of perturbed EFGs in terms of duality gap, which is useful for finding approximate extensive-form perfect equilibrium (EFPE) (Selten, 1975).
Revised: For CFR-type algorithms, using the regularization technique, we establish the first best-iterate convergence rate of O (1 /T 1 / 4 ) for finding the NE of non-perturbed EFGs, and last-iterate asymptotic convergence for finding the NE of perturbed EFGs in terms of duality gap, which is useful for finding approximate extensive-form perfect equilibrium (EFPE) (Selten, 1975).

Original: (iv) As a by-product of our analysis, we also provide a faster and optimal rate of O (1 /T ) average-iterate convergence guarantee in finding NE of perturbed EFGs (see formal definition in §4.1), while also matching the state-of-the-art guarantees for CFR-type algorithms in finding NE for the non-perturbed EFGs in terms of duality gap (Farina et al., 2019a).
Revised: (iv) As a by-product of our analysis, we also provide a faster and optimal rate of O (1 /T ) average-iterate convergence guarantee in finding NE of perturbed EFGs (see formal definition in §4.1), while also matching the state-of-the-art guarantees for CFR-type algorithms in finding NE for the non-perturbed EFGs in terms of duality gap (Farina et al., 2019a).

Original: Technical challenges.
Revised: Technical challenges.

Original: We emphasize the technical challenges we address as follows.
Revised: We emphasize the technical challenges we address as follows.

Original: First, by adding regularization to the original problem, Reg-DOMD will converge to the NE of the regularized  problem, instead of that of the original one.
Revised: First, by adding regularization to the original problem, Reg-DOMD will converge to the NE of the regularized [Table] [Table] problem, instead of that of the original one.

Original: By controlling the regularization, one can readily obtain  convergence guarantee of the regularized algorithm in terms of duality gap, as in Cen et al.
Revised: By controlling the regularization, one can readily obtain a convergence guarantee of the regularized algorithm in terms of duality gap, as in Cen et al.

Original: (2021b)
Revised: (2021b)

Original: for NFGs.
Revised: for NFGs.

Original: However, it is highly non-trivial to connect back to the iterate convergence.
Revised: However, it is highly non-trivial to connect back to the iterate convergence.

Original: We achieve so by proving a novel relation between the distance to the NE set and the duality gap (see §3.2 for a detailed discussion and Lemma D.5 for the complete proof).[Table]
Revised: We achieve so by proving a relationship between the distance to the NE set and the duality gap (See Lemma D. for the complete proof).

Original: [Table] Second, it is challenging to obtain last-iterate  guarantees in CFR-type algorithms, since the regret minimizer in each information set cannot synchronize with the other regret minimizers, and they operate independently.
Revised: Second, it is challenging to obtain last-iterate convergence guarantees in CFR-type algorithms, since the regret minimizer in each information set cannot synchronize with the other regret minimizers, and they operate independently.

Original: Hence, unlike OMD, the individual iterates (not the average) obtained from these independent regret-minimizers may reach an NE of the game asynchronously.
Revised: Hence, unlike OMD, the individual iterates (not the average) obtained from these independent regret-minimizers may reach an NE of the game asynchronously.

Original: For the same reason, it is also challenging to extend the fast-rate no-regret learning results from NFGs (over simplex) to EFGs under the CFR framework.
Revised: For the same reason, it is also challenging to extend the fast-rate no-regret learning results from NFGs (over simplex) to EFGs under the CFR framework.

Original: We provide a detailed related work discussion to Appendix A.
Revised: We provide a detailed related work discussion in Appendix A.

Original: Here we provide Table 1 and Tableto compare our work with the literature.
Revised: Here we provide Table 1 and Tableto compare our work with the literature.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Notation.
Revised: Notation.

Original: We use x i to denote the i -th coordinate of vector x and ∥ x ∥ p to denote its p -norm.
Revised: We use x i to denote the i -th coordinate of vector x and ∥ x ∥ p to denote its p -norm.

Original: By default, we use ∥ x ∥ to denote the 2-norm ∥ x ∥ 2 .
Revised: By default, we use ∥ x ∥ to denote the 2-norm ∥ x ∥ 2 .

Original: We use ∆ m to denote the m − 1 dimension probability simplex { x ∈ [0 , 1] m : (cid:80) mi
Revised: We use ∆ m to denote the m − 1 dimension probability simplex { x ∈ [0 , 1] m : (cid:80) mi

Original: =1 x i = 1 } , and we sometimes omit the subscript m when it is clear from the context.
Revised: =1 x i = 1 } , and we sometimes omit the subscript m when it is clear from the context.

Original: For any convex and differentiable function ψ , its associated Bregman divergence is defined as D ψ ( u , v ) :
Revised: For any convex and differentiable function ψ , its associated Bregman divergence is defined as D ψ ( u , v ) :

Original: = ψ ( u ) − ψ ( v ) − ⟨∇ ψ ( v ) , u − v ⟩ .
Revised: = ψ ( u ) − ψ ( v ) − ⟨∇ ψ ( v ) , u − v ⟩ .

Original: Finally, we use (cid:81) C ( u ) to denote the projection of u to a convex set C with respect to Euclidean distance.
Revised: Finally, we use (cid:81) C ( u ) to denote the projection of u to a convex set C with respect to Euclidean distance.

Original: Bilinear optimization problem.
Revised: Bilinear optimization problem.

Original: Strategies in two-player zero-sum extensive-form games with perfect recall can be interpreted in sequence-form (Von Stengel, 1996).
Revised: Strategies in two-player zero-sum extensive-form games with perfect recall can be interpreted in sequence-form (Von Stengel, 1996).

Original: Thus, finding the Nash equilibrium reduces to solving a bilinear saddle-point problem,
Revised: Thus, finding the Nash equilibrium reduces to solving a bilinear saddle-point problem,

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where X ⊂ R MT , Y ⊂ R NT are the decision sets for min/max players called treeplexes (to be defined next).
Revised: where X ⊂ R MT , Y ⊂ R NT are the decision sets for min/max players called treeplexes (to be defined next).

Original: In sequence-form representation, x i denotes the probability of reaching node i in the treeplex when only counting the uncertainty incurred by the min-player, and y i can be interpreted similarly.
Revised: In sequence-form representation, x i denotes the probability of reaching node i in the treeplex when only counting the uncertainty incurred by the min-player, and y i can be interpreted similarly.

Original: The matrix A ∈ [ − 1 , 1] M × N , where A i,j denotes the payoff of the max-player when the min-player reaches i and max-player reaches j .
Revised: The matrix A ∈ [ − 1 , 1] M × N , where A i,j denotes the payoff of the max-player when the min-player reaches i and max-player reaches j .

Original: Nash equilibria are just the solutions to Eq (2.1).
Revised: Nash equilibria are just the solutions to Eq (2.1).

Original: We define
Revised: We define

Original: Z ∗ =
Revised: Z ∗ =

Original: X ∗ ×
Revised: X ∗ ×

Original: Y ∗ to denote the set of NE, which is always convex for two player zero-sum game.
Revised: Y ∗ to denote the set of NE, which is always convex for two player zero-sum game.

Original: For convenience, we use P :=
Revised: For convenience, we use P :=

Original: M + N to denote the dimension of problem (2.1), and concatenate the sequence form for both players by defining z :=
Revised: M + N to denote the dimension of problem (2.1), and concatenate the sequence form for both players by defining z :=

Original: ( x , y ) ∈
Revised: ( x , y ) ∈

Original: Z :=
Revised: Z :=

Original: X × Y and the gradient of the bilinear form (2.1) by defining F ( z ) :
Revised: X × Y and the gradient of the bilinear form (2.1) by defining F ( z ) :

Original: = ( Ay , − A ⊤ x ) .
Revised: = ( Ay , − A ⊤ x ) .

Original: By re-normalizing A , we can assume ∥ F ( z ) ∥ ∞ ≤ 1 without loss of generality.
Revised: By re-normalizing A , we can assume ∥ F ( z ) ∥ ∞ ≤ 1 without loss of generality.

Original: Treeplex and dilated regularizer.
Revised: Treeplex and dilated regularizer.

Original: The structure of a sequence-form is enforced implicitly by the treeplexes , which we define formally here:
Revised: The structure of a sequence-form is enforced implicitly by the treeplexes , which we define formally here:

Original: Definition 2.1 ( Hoda et al.
Revised: Definition 2.1 ( Hoda et al.

Original: Treeplex is recursively defined as follows:
Revised: Treeplex is recursively defined as follows:

Original: Each probability simplex is a treeplex.
Revised: Each probability simplex is a treeplex.

Original: The Cartesian product of multiple treeplexes is a treeplex.
Revised: The Cartesian product of multiple treeplexes is a treeplex.

Original: The branching of two treeplexes is a treeplex, where for integers m, n > 0 , the branching of two treeplexes Z 1 ⊂ R mT , Z 2 ⊂ R nT on index i ∈ { 1 , 2 , ..., m } is defined as
Revised: The branching of two treeplexes is a treeplex, where for integers m, n > 0 , the branching of two treeplexes Z 1 ⊂ R mT , Z 2 ⊂ R nT on index i ∈ { 1 , 2 , ..., m } is defined as

Original: [Equation]
Revised: [Equation]

Original: See an illustration of treeplex in Figure 1 of Appendix A.
Revised: See an illustration of treeplex in Figure 1 of Appendix A.

Original: The simplexes in the treeplex specify the decision points for both players, which are also called information sets in the EFG literature (Zinkevich et al., 2007; Tammelin et al., 2015; Farina et al., 2019a).
Revised: The simplexes in the treeplex specify the decision points for both players, which are also called information sets in the EFG literature (Zinkevich et al., 2007; Tammelin et al., 2015; Farina et al., 2019a).

Original: The collection of information sets in treeplex Z is denoted as H Z .
Revised: The collection of information sets in treeplex Z is denoted as H Z .

Original: For any h ∈ H Z , we use Ω h to denote the indices in Z belonging to decision point h and h ( i ) to denote the information set that index i belongs to.
Revised: For any h ∈ H Z , we use Ω h to denote the indices in Z belonging to decision point h and h ( i ) to denote the information set that index i belongs to.

Original: That is, h ( i ) =
Revised: That is, h ( i ) =

Original: h
Revised: h

Original: if and only if i ∈ Ω h .
Revised: if and only if i ∈ Ω h .

Original: We use σ ( h ) to denote the index of the parent variable of h and H i
Revised: We use σ ( h ) to denote the index of the parent variable of h and H i

Original: = { h ∈ H Z : σ ( h ) = i } .
Revised: = { h ∈ H Z : σ ( h ) = i } .

Original: For a simplex Z , the parent of the only information set h ∈ H Z does not exist and we use σ ( h ) = 0 to denote it.
Revised: For a simplex Z , the parent of the only information set h ∈ H Z does not exist and we use σ ( h ) = 0 to denote it.

Original: And when applying Cartesian product on multiple treeplexes, it will not change the parent of any information set.
Revised: And when applying Cartesian product on multiple treeplexes, it will not change the parent of any information set.

Original: When we branch two treeplexes, that is Z 1 i Z 2 , then the parent of all information set h ∈
Revised: When we branch two treeplexes, that is Z 1 i Z 2 , then the parent of all information set h ∈

Original: H Z 2 with σ ( h ) = 0 will be updated to σ ( h ) =
Revised: H Z 2 with σ ( h ) = 0 will be updated to σ ( h ) =

Original: i .
Revised: i .

Original: For convenience, we use z h to denote the slice of z with indices in Ω h .
Revised: For convenience, we use z h to denote the slice of z with indices in Ω h .

Original: Let C Ω := max h ∈H Z | Ω h | denote the maximum number of indices in each individual information set.
Revised: Let C Ω := max h ∈H Z | Ω h | denote the maximum number of indices in each individual information set.

Original: For convenience, we define vector q ∈ R P with q i :=
Revised: For convenience, we define vector q ∈ R P with q i :=

Original: z i /z
Revised: z i /z

Original: σ ( h ( i )) for any i .
Revised: σ ( h ( i )) for any i .

Original: In the EFG terminology, q h ∈ R | Ω h | , the slice of q in information set h , is the probability distribution of actions in information set h .
Revised: In the EFG terminology, q h ∈ R | Ω h | , the slice of q in information set h , is the probability distribution of actions in information set h .

Original: The treeplex structure motivates a natural dilation operation to generate regularizers that leads to efficient computation in EFGs (Hoda et al., 2010).
Revised: The treeplex structure motivates a natural dilation operation to generate regularizers that leads to efficient computation in EFGs (Hoda et al., 2010).

Original: For any strongly-convex base regularizer ψ ∆ defined on a simplex, the dilated regularizer is defined by
Revised: For any strongly-convex base regularizer ψ ∆ defined on a simplex, the dilated regularizer is defined by

Original: [Equation]
Revised: [Equation]

Original: where z σ ( h ) is the probability of reaching the parent variable of information set h .
Revised: where z σ ( h ) is the probability of reaching the parent variable of information set h .

Original: And α h is the h th element of vector α ∈ R |H
Revised: And α h is the h th element of vector α ∈ R |H

Original: Z | + which is some hyper-parameter set according to ψ ∆ to guarantee that ψ Z is 1-strongly convex with respect to 2-norm (Hoda et al., 2010; Kroer et al., 2020), i.e., D ψ Z ( z 1 , z 2 ) ≥ 12 ∥
Revised: Z | + which is some hyper-parameter set according to ψ ∆ to guarantee that ψ Z is 1-strongly convex with respect to 2-norm (Hoda et al., 2010; Kroer et al., 2020), i.e., D ψ Z ( z 1 , z 2 ) ≥ 12 ∥

Original: z 1 − z 2 ∥ 2 .
Revised: z 1 − z 2 ∥ 2 .

Original: Two common base regularizers are the negative entropy ψ ∆Entropy ( p ) =
Revised: Two common base regularizers are the negative entropy ψ ∆Entropy ( p ) =

Original: (cid:80) i p i log p
Revised: (cid:80) i p i log p

Original: i and the Euclidean norm ψ ∆Euclidean ( p ) =
Revised: i and the Euclidean norm ψ ∆Euclidean ( p ) =

Original: (cid:80) i p 2 i , where p ∈ ∆ is a probability distribution.
Revised: (cid:80) i p 2 i , where p ∈ ∆ is a probability distribution.

Original: Finding NE and regret minimization.
Revised: Finding NE and regret minimization.

Original: Given a strategy z in sequence form, there are two criteria to evaluate the performance:
Revised: Given a strategy z in sequence form, there are two criteria to evaluate the performance:

Original: • the Euclidean distance to the set of NE ∥ (cid:81)
Revised: • the Euclidean distance to the set of NE ∥ (cid:81)

Original: Z ∗ ( z ) − z ∥ , • the duality gap max (cid:98) z ∈Z F ( z ) ⊤ ( z − (cid:98) z ) .
Revised: Z ∗ ( z ) − z ∥ , • the duality gap max (cid:98) z ∈Z F ( z ) ⊤ ( z − (cid:98) z ) .

Original: When one or both of the above quantities are close to zero, we find an approximate NE.
Revised: When one or both of the above quantities are close to zero, we find an approximate NE.

Original: A common approach to minimize duality gap is by regret minimization , where we define the (external) regret of the min-player as
Revised: A common approach to minimize duality gap is by regret minimization , where we define the (external) regret of the min-player as

Original: [Equation]
Revised: [Equation]

Original: where l t is the loss function at iteration t and x t is the output of the regret minimizer at iteration t .
Revised: where l t is the loss function at iteration t and x t is the output of the regret minimizer at iteration t .

Original: Regret of the max-player can be defined similarly.
Revised: Regret of the max-player can be defined similarly.

Original: When regret is growing sublinearly with respect with T , the average regret is converging to zero (hence the name no-regret).
Revised: When regret is growing sublinearly with respect to T , the average regret is converging to zero (hence the name no-regret).

Original: The following Nash folklore theorem(Kozuno et al., 2021, Appendix A) implies  the average strategy will converge to NE.
Revised: The following Nash folklore theorem implies that the average strategy will converge to NE.

Original: Lemma 2.2.
Revised: Lemma 2.2.

Original: For a bilinear zero-sum game where l X t ( x t ) = − l Y t ( y t ) =
Revised: For a bilinear zero-sum game where l X t ( x t ) = − l Y t ( y t ) =

Original: x
Revised: x

Original: ⊤ t
Revised: ⊤ t

Original: Ay t , the duality gap of the average strategy ( 1 T (cid:80)
Revised: Ay t , the duality gap of the average strategy ( 1 T (cid:80)

Original: Tt =1 x t , 1 T (cid:80)
Revised: Tt =1 x t , 1 T (cid:80)

Original: Tt =1 y t ) is bounded by ( R X T + R Y T ) /T .
Revised: Tt =1 y t ) is bounded by ( R X T + R Y T ) /T .

Original:  R EGULARIZED D ILATED O PTIMISTIC M IRROR D ESCENT ( R EG -DOMD ) 3. S OLVING A REGULARIZED PROBLEM
Revised:  R EGULARIZED D ILATED O PTIMISTIC M IRROR D ESCENT ( R EG -DOMD ) 3. S OLVING A REGULARIZED PROBLEM

Original: To obtain  faster convergence rate of OMD algorithms, we will solve the NE of the regularized problem below (and thus strongly convex-concave) as an intermediate step.
Revised: To obtain a faster convergence rate for OMD algorithms, we will solve the NE of the regularized problem below (and thus strongly convex-concave) as an intermediate step.

Original: In the literature (McKelvey and Palfrey, 1995), the solution to the regularized problem is called the quantal-response equilibrium (QRE), when the regularizer ψ Z is entropy:
Revised: In the literature (McKelvey and Palfrey, 1995), the solution to the regularized problem is called the quantal-response equilibrium (QRE), when the regularizer ψ Z is entropy:

Original: [Equation]
Revised: [Equation]

Original:  where τ ∈ (0 , 1] is the weight of regularization and ψ Z is a strongly-convex regularizer.
Revised: (3.1) where τ ∈ (0 , 1] is the weight of regularization and ψ Z is a strongly-convex regularizer.

Original: Thanks to the strong convexity of ψ Z , Eq (3.1) has a unique NE, denoted by z ∗ τ .
Revised: Thanks to the strong convexity of ψ Z , Eq (3.1) has a unique NE, denoted by z ∗ τ .

Original: For t = 1 , 2 , ... , the update rule of optimistic mirror descent for the regularized problem (3.1), which we refer to as Reg-DOMD , can be written as
Revised: For t = 1 , 2 , ... , the update rule of optimistic mirror descent for the regularized problem (3.1), which we refer to as Reg-DOMD , can be written as

Original: [Equation]
Revised: [Equation]

Original:  where we initialize z [-
-] = (cid:98) z 1 as uniform strategy, that is, z 0 ,h z 0 ,σ ( h ) is uniform distribution in ∆ | Ω h | , and η > 0 is the stepsize.
Revised: (3.2) where we set z [-
-]0 = (cid:98) z 1 as uniform strategy, i.e., z 0 ,h z 0 ,σ ( h ) is uniform distribution in ∆ | Ω h | , and η >is the stepsize.

Original: The Dilated Optimistic Mirror Descent (DOMD)
Revised: The Dilated Optimistic Mirror Descent (DOMD)

Original: (Lee et al., 2021) now becomes a special case when τ = 0 .
Revised: (Lee et al., 2021) now becomes a special case when τ = 0 .

Original: We call the update rule (3.2)
Revised: We call the update rule (3.2)

Original: Regularized Dilated Optimistic 
Revised: Regularized Dilated Optimistic Multiplicative

Original: Multiplicative Weights Update ( Reg-DOMWU ) when the base regularizer ψ ∆ is negative entropy, and Regularized Dilated Optimistic Gradient Descent Ascent ( Reg-DOGDA ) when ψ ∆ is Euclidean norm.
Revised: Weights Update ( Reg-DOMWU ) when the base regularizer ψ ∆ is negative entropy, and Regularized Dilated Optimistic Gradient Descent Ascent ( Reg-DOGDA ) when ψ ∆ is Euclidean norm.

Original: As desired, (cid:98)
Revised: As desired, (cid:98)

Original: z t +1 converges to z ∗ τ at a linear rate for any fixed τ .
Revised: z t +1 converges to z ∗ τ at a linear rate for any fixed τ .

Original: Theorem 3.1.
Revised: Theorem 3.1.

Original: With η ≤ 1 8 P , τ ≤ 1 and ψ Z being a 1-strongly convex function with respect to the 2-norm, Reg-DOMD guarantees that D ψ Z ( z ∗ τ , (cid:98) z t +1 ) ≤ (1 − ητ )
Revised: With η ≤ 18 P , τ ≤ 1 and ψ Z being a 1-strongly convex function with respect to the 2-norm, Reg-DOMD guarantees that D ψ Z ( z ∗ τ , (cid:98) z t +1 ) ≤ (1 − ητ )

Original: t D ψ Z ( z ∗ τ , (cid:98) z 1 ) for any t ≥ 1 when we initialize z 0 = (cid:98) z 1 .
Revised: t D ψ Z ( z ∗ τ , (cid:98) z 1 ) for any t ≥ 1 when we initialize z 0 = (cid:98) z 1 .

Original: The results in Theorem 3.1 are for general dilated regularizers, and apply to the regularized version of two representative algorithms, Reg-DOMWU and Reg-DOGDA , as studied in Lee et al.
Revised: The results in Theorem 3.1 are for general dilated regularizers, and apply to the regularized version of two representative algorithms, Reg-DOMWU and Reg-DOGDA , as studied in Lee et al.

Original: The detailed proof is postponed to Appendix C. We sketch the proof below.
Revised: The detailed proof is postponed to Appendix C. We sketch the proof below.

Original: Proof Sketch of Theorem 3.1.
Revised: Proof sketch of Theorem 3.1.

Original: When ψ Z is a 1-strongly convex function with respect to 2-norm and η ≤ 18 P , then for any z ∈ Z and t ≥ 1 , we have
Revised: When ψ Z is a 1-strongly convex function with respect to 2-norm and η ≤ 18 P , then for any z ∈ Z and t ≥ 1 , we have

Original: [Equation]
Revised: [Equation]

Original: which isan adaptedversion of standard result from  OMD analysis (Rakhlin and Sridharan, 2013) for the regularized problem.
Revised: which is adapted from the standard OMD analysis (Rakhlin and Sridharan, 2013), but for the regularized problem.

Original: See Lemma C.2 for the proof.
Revised: See Lemma C.2 for the proof.

Original: Taking z
Revised: Taking z

Original: = z ∗ τ in Eq (3.3), we have
Revised: = z ∗ τ in Eq (3.3), we have

Original: [Equation]
Revised: [Equation]

Original: where ( i ) follows by definition of z ∗ τ .
Revised: where ( i ) follows by definition of z ∗ τ .

Original: Letting Θ t [-+
-]
Revised: Letting Θ t [-+
-]+

Original: = D ψ Z ( z ∗ τ , (cid:98) z t +1 ) + D ψ Z ( (cid:98)
Revised: [Equation]

Original: z t
Revised: = D

Original: [-+
-]
Revised: [-+
-][Equation]

Original: z t  ) , inequality (3.4) can be written as
Revised: z t +1 , z t ) , inequality (3.4) can be written as

Original: [Equation]
Revised: [Equation]

Original: where the second inequality comes from ητ ≤ η ≤ 78 .
Revised: where the second inequality comes from ητ ≤ η ≤ 78 .

Original: This justifies the linear convergence.
Revised: This justifies the linear convergence.

Original: In the existing work Wei et al.
Revised: In the existing work Wei et al.

Original: ; Lee et al.
Revised: ; Lee et al.

Original: (2021) without regularization, i.e., when τ = 0 , the above argument cannot guarantee the linearly shrinking property of Θ t .
Revised: (2021) without regularization, i.e., when τ = 0 , the above argument cannot guarantee the linearly shrinking property of Θ t .

Original: With the unique NE assumption, one can prove some “slope” in the original bilinear objective which implies an explicit convergence rate (Lee et al., 2021, Lemma 15).
Revised: With the unique NE assumption, one can prove some “slope” in the original bilinear objective which implies an explicit convergence rate (Lee et al., 2021, Lemma 15).

Original: It was unclear if such an assumption can be removed.
Revised: It was unclear if such an assumption can be removed.

Original: Here, the regularization technique enables us to avoid such an assumption.
Revised: Here, the regularization technique enables us to avoid such an assumption.

Original: See a more detailed and technical discussion below Lemma D.4.
Revised: See a more detailed and technical discussion below Lemma D.5.

Original: 3. F ROM THE REGULARIZED PROBLEM TO THE ORIGINAL PROBLEM
Revised: 3. F ROM THE REGULARIZED PROBLEM TO THE ORIGINAL PROBLEM

Original: Intuitively, if the weight of regularization τ is sufficiently small, NE for the regularized problem should be close to the NE of the original problem (2.1).
Revised: Intuitively, if the weight of regularization τ is sufficiently small, NE for the regularized problem should be close to the NE of the original problem (2.1).

Original: In the following we formalize this intuition and show how Theorem 3.1 implies a last-iterate guarantee.
Revised: In the following we formalize this intuition and show how Theorem 3.1 implies a last-iterate guarantee.

Original: We shrink the weight of regularization τ as follows: First initialize τ = τ 0 for some hyper-parameter τ 0 at the beginning and run Reg-DOMD in episodes.
Revised: We shrink the weight of regularization τ as follows: First initialize τ = τ 0 for some hyper-parameter τ 0 at the beginning and run Reg-DOMD in episodes.

Original: In each episode, we update the parameters z t and (cid:98) z t +1 for (cid:101)
Revised: In each episode, we update the parameters z t and (cid:98) z t +1 for (cid:101)

Original: Θ(1 /τ ) iterations so that the duality gap of (cid:98) z t will be lower than O ( τ ) according to
Revised: Θ(1 /τ ) iterations so that the duality gap of (cid:98) z t will be lower than O ( τ ) according to

Original: Lemma D.1 and Theorem 3.1.
Revised: Lemma D.1 and Theorem 3.1.

Original: Then, we will shrink τ by one half and start the next episode from scratch.
Revised: Then, we will shrink τ by one half and start the next episode from scratch.

Original: Notice that although τ is changing, the stepsize η keeps fixed/constant, which differs from
Revised: Notice that although τ is changing, the stepsize η keeps fixed/constant, which differs from

Original: Hsieh et al.
Revised: Hsieh et al.

Original: (2021), where the stepsize is adaptive .
Revised: (2021), where the stepsize is adaptive .

Original: Theorem 3.2.
Revised: Theorem 3.2.

Original: With the shrinking algorithm described above, the duality gap satisfiesmax z ∈Z F ( (cid:98) z t +1 )
Revised: With the shrinking algorithm described above, the duality gap satisfies

Original:  ⊤ ( (cid:98) z t +1 − z ) ≤ (cid:101) O ( 1 t ) for t = 1 , 2 , ..., T .
Revised: [Equation] F ( (cid:98) z t +1 ) ⊤ ( (cid:98) z t +1 − z ) ≤ (cid:101) O ( 1 t ) for t = 1 , 2 , ..., T .

Original: When ψ ∆ is entropy, which leads to the Reg-DOMWU algorithm, wefurther have an iterate convergence rate of ∥ (cid:98)
Revised: Moreover, we have an iterate convergence rate of ∥ (cid:98)

Original: z t +1 − (cid:81)
Revised: z t +1 − (cid:81)

Original: Z ∗ ( (cid:98) z t +1 )
Revised: Z ∗ ( (cid:98) z t +1 )

Original: ∥ ≤ (cid:101) O ( 1 t ) .
Revised: ∥ ≤ (cid:101) O ( 1 t ) .

Original: In practice, we use an adaptive weight-shrinking rule proposed in Appendix A, which is motivated by Yang et al.
Revised: In practice, we use an adaptive weight-shrinking rule proposed in Appendix A, which is motivated by Yang et al.

Original: Note thatthe first part of Theorem 3.2 applies for both Reg-DOMWU and Reg-DOGDA .
Revised: Note that Theorem 3.2 applies for both Reg-DOMWU and Reg-DOGDA .

Original: To the best of our knowledge, this is the first result to obtainduality gap convergence rate  in EFGs without the unique NE assumption, when the mirror map is generated through a dilated operation (Lee et al., 2021).It is also the first result to obtain last-iterate convergence for OMWU-type algorithms in terms of the distance to the NE set , for both NFGs and EFGs without the unique NE assumption.
Revised: To the best of our knowledge, this is the first result to obtain convergence rate for duality gap and the distance to the NE set in EFGs without the unique NE assumption, when the mirror map is generated through a dilated operation (Lee et al., 2021).

Original: Technical overview.
Revised: Technical overview.

Original: We briefly sketch the intuition behind the proof and defer the full details to
Revised: We briefly sketch the intuition behind the proof and defer the full details to

Original: Appendix D. To prove the duality gap guarantee, first notice that in the regularized problem, (cid:98) z t has a small duality gap thanks to the last-iterate guarantee in Theorem 3.1.
Revised: Appendix D. To prove the duality gap guarantee, first notice that in the regularized problem, (cid:98) z t has a small duality gap thanks to the last-iterate guarantee in Theorem 3.1.

Original: So we only need to argue that the duality gap of z ∗ τ in the original problem is also small, which turns out to be O ( τ ) .
Revised: So we only need to argue that the duality gap of z ∗ τ in the original problem is also small, which turns out to be O ( τ ) .

Original: However, this argument does not imply a small distance to the NE set, because the distance between z ∗ τ and z ∗ is unknown.
Revised: However, this argument does not imply a small distance to the NE set, because the distance between z ∗ τ and z ∗ is unknown.

Original: Instead, we prove that the lower-bound of the “slope” of the duality gap is strictly positive in the region that the iterates of Reg-DOMWU could reach .
Revised: Instead, we need the result that the lower-bound of the “slope” of the duality gap is strictly positive, i.e., for any z , we have max z ′ ∈Z F ( z ) ⊤ ( z − z ′ ) ≥ c ∥

Original: That is, for any (cid:98) z t produced by Reg-DOMWU , we have max z ∈Z F ((cid:98) zt +1 ) ⊤ ((cid:98) zt +1 − z  ) ≥ c
Revised: z − (cid:81) Z ∗ ( z ) ∥ for some constant c > 0 . Moreover, compared to existing “slope” results (Gilpin et al., 2008; Wei et al., 2021), we provide a stronger one when the regularizer is entropy since we prove that max z ′ F ( z ) ⊤ ( z − z ′ ) ≥ c

Original: ∥
Revised: ∥

Original: (cid:98) zt +1 − (cid:81) Z ∗ ((cid:98) zt +1 ) ∥ for some constant c > 0 .
Revised: z − (cid:81) Z ∗ ( z ) ∥ when z ′ is restricted to a subset of Z (see Lemma D.6).

Original: Proving that c > 0 is highly nontrivial even in NFGs, and is one of our contributions of the paper.

Original: See Lemma D.5 for more details.

Original: Due to the regularization, our dependence on the EFG size P is quite mild.
Revised: Due to the regularization, our dependence on the EFG size P is quite mild.

Original: There’s only a P ∥ α ∥ ∞ dependence on the EFG size for the duality gap convergence result ( ∥ α ∥ ∞ is usually O ( P 2 ) regarding the specific type of dilation (Hoda et al., 2010; Kroer et al., 2020; Farina et al., 2021)), which can be referred to Appendix D. The convergence rate of the distance to the NE set of the original problem depends on the slope c , which also depends on the reward matrix.
Revised: There’s only a P ∥ α ∥ ∞ dependence on the EFG size for the duality gap convergence result ( ∥ α ∥ ∞ is usually O ( P 2 ) regarding the specific type of dilation (Hoda et al., 2010; Kroer et al., 2020; Farina et al., 2021)), which can be found in Appendix D. The convergence rate of the distance to the NE set of the original problem depends on the slope c , which also depends on the reward matrix.

Original:  R EGULARIZED C OUNTERFACTUAL R EGRET M INIMIZATION ( Reg-CFR )
Revised:  R EGULARIZED C OUNTERFACTUAL R EGRET M INIMIZATION ( Reg-CFR )

Original: Counterfactual regret minimization is the most widely used solution framework in EFGs in the past decades, and has achieved many successes including defeating the professional human player in
Revised: Counterfactual regret minimization is the most widely used solution framework in EFGs in the past decades, and has achieved many successes including defeating the professional human player in

Original: Texas Hold’em (Brown and Sandholm, 2018; 2019b).
Revised: Texas Hold’em (Brown and Sandholm, 2018; 2019b).

Original: Through the framework, the (global) regret of the EFG in (2.4) can be minimized by minimizing the local regret in each information set separately.
Revised: Through the framework, the (global) regret of the EFG in (2.4) can be minimized by minimizing the local regret in each information set separately.

Original: To describe the regret decomposition framework in its full generality, we first introduce some additional notation.
Revised: To describe the regret decomposition framework in its full generality, we first introduce some additional notation.

Original: W h ( z ) is the value at the treeplex rooted at information set h of the player h belongs to when both players play according to z .
Revised: W h ( z ) is the value at the treeplex rooted at information set h of the player h belongs to when both players play according to z .

Original: For any h ∈ H X , W h ( z ) can be recursively defined as
Revised: For any h ∈ H X , W h ( z ) can be recursively defined as

Original: [Equation]
Revised: [Equation]

Original: where q i =
Revised: where q i =

Original: z i /z
Revised: z i /z

Original: σ ( h ( i ))
Revised: σ ( h ( i ))

Original: ∈ ∆ | Ω h ( i ) | is the (conditional-form) strategy on information set h ( i ) (it lies in a simplex due to the definition of treeplex in Definition 2.1) and α h is the hyper-parameter defined in Eq (2.3).
Revised: ∈ ∆ | Ω h ( i ) | is the (conditional-form) strategy on information set h ( i ) (it lies in a simplex due to the definition of treeplex in Definition 2.1) and α h is the hyper-parameter defined in Eq (2.3).

Original: For h ∈ H Y ,
Revised: For h ∈ H Y ,

Original: W h
Revised: W h

Original: ( z ) can be defined similarly.
Revised: ( z ) can be defined similarly.

Original: The local loss l ht ( q h ): ∆ | Ω h | → R at any information set h ∈ H Z can be defined by 
Revised: The local loss l ht ( q h ): ∆ | Ω h | → R at any information set h ∈ H Z can be defined by l ht ( q h )

Original: [Equation]
Revised: [Equation]

Original: where V h ( z) :
Revised: = [Equation] z

Original: = (cid:0) ( Ay ) i + (cid:88)
Revised: [Equation] [Equation] + τα

Original: h ′ ∈H i W h ′ ( z ) (cid:1) i ∈
Revised: [Equation] [Equation]

Original: Ω h .
Revised: [Equation]

Original: Notice that W h ( z ) is a scalar while V h ( z ) is a vector.
Revised: Notice that W h ( z ) is a scalar while V h ( z ) is a vector.

Original: Furthermore, the two quantities can be related to each other by W h ( z )
Revised: Furthermore, the two quantities can be related to each other by W h ( z )

Original: =
Revised: =

Original: (cid:68) z h z

Original: [Equation]

Original: [Equation]

Original: , V

Original: [Equation] ( z [Equation] (cid:69) + τα h ψ ∆ ( z h z σ ( h 
Revised: (cid:68) z h z σ ( h ) , V h ( z ) (cid:69) + τα h ψ ∆ ( z h z σ ( h ) ) .

Original: [Equation]

Original: The local difference at information set h is just G hT ( q h ) :
Revised: The local difference at information set h is just G hT ( q h ) :

Original: = (cid:80) Tt =
Revised: = (cid:80) Tt =

Original: 1 l ht ( q t,h ) − (cid:80)
Revised: 1 l ht ( q t,h ) − (cid:80)

Original: Tt =
Revised: Tt =

Original: 1 l ht ( q h ) and the local regret R hT := max (cid:98) q h ∈ ∆ | Ω h | G hT ( (cid:98) q h ) .
Revised: 1 l ht ( q h ) and the local regret R hT := max (cid:98) q h ∈ ∆ | Ω h | G hT ( (cid:98) q h ) .

Original: The following decomposition implies that the global regret can be controlled by the sum of local regrets:
Revised: The following decomposition implies that the global regret can be controlled by the sum of local regrets:

Original: Lemma 4.1 (Laminar regret decomposition (Farina et al., 2019b)) .
Revised: Lemma 4.1 (Laminar regret decomposition (Farina et al., 2019b)) .

Original: For any z 1 , z 2 , ..., z T , z ∈ Z and τ ≥ 0 , we have
Revised: For any z 1 , z 2 , ..., z T , z ∈ Z and τ ≥ 0 , we have

Original: [Equation]
Revised: [Equation]

Original: where R Z T is the sum of the regret of min-player and max-player defined in Eq (2.4) instantiated with l t ( z ) =
Revised: where R Z T is the sum of the regret of min-player and max-player defined in Eq (2.4) instantiated with l t ( z ) =

Original: ⟨ F ( z t ) , z ⟩
Revised: ⟨ F ( z t ) , z ⟩

Original: + τψ Z ( z ) .
Revised: + τψ Z ( z ) .

Original: The proof is postponed to Appendix E. Hence, by minimizing
Revised: The proof is postponed to Appendix E. Hence, by minimizing

Original: R h T at each information set h ∈ H Z , R Z T will also be minimized.
Revised: R h T at each information set h ∈ H Z , R Z T will also be minimized.

Original: By Lemma 2.2, the average strategy will converge to NE when τ = 0 .
Revised: By Lemma 2.2, the average strategy will converge to NE when τ = 0 .

Original: In fact, when τ > 0 , the average strategy will converge to the corresponding NE of the regularized problem z ∗ τ according to a stronger version of Lemma 2.
Revised: In fact, when τ > 0 , the average strategy will converge to the corresponding NE of the regularized problem z ∗ τ according to a stronger version of Lemma 2.

Original: (Theorem 3 ; Farina et al., 2019b).
Revised: (Theorem 3 ; Farina et al., 2019b).

Original: For completeness, we provide the formal version as Lemma F.3.
Revised: For completeness, we provide the formal version as Lemma F.3.

Original: To describe our main results in full generality, we introduce the notion of perturbed EFGs before diving into the algorithm and analysis.
Revised: To describe our main results in full generality, we introduce the notion of perturbed EFGs before diving into the algorithm and analysis.

Original: 4. P ERTURBED EXTENSIVE - FORM GAME AND EXTENSIVE - FORM PERFECT N ASH EQUILIBRIUM
Revised: 4. P ERTURBED EXTENSIVE - FORM GAME AND EXTENSIVE - FORM PERFECT N ASH EQUILIBRIUM

Original: Although NE specifies a natural notion of optimality in EFGs, an NE strategy is not necessarily behaving reasonably in information sets that it will not reach almost surely.
Revised: Although NE specifies a natural notion of optimality in EFGs, an NE strategy is not necessarily behaving reasonably in information sets that it will not reach almost surely.

Original: To avoid this issue, a stronger and refined notion of equilibirum, extensive-form perfect equilibria , has been proposed in
Revised: To avoid this issue, a stronger and refined notion of equilibirum, extensive-form perfect equilibria , has been proposed in

Original: Selten (1975), which takes every information set into consideration by perturbing the EFG to force the players to reach every information set.
Revised: Selten (1975), which takes every information set into consideration by perturbing the EFG to force the players to reach every information set.

Original: We formally introduce the definitions below.
Revised: We formally introduce the definitions below.

Original: Definition 4.2.
Revised: Definition 4.2.

Original: For any γ ≥ 0 , a γ -perturbed EFG is an EFG with a γ -perturbed treeplex
Revised: For any γ ≥ 0 , a γ -perturbed EFG is an EFG with a γ -perturbed treeplex

Original: Z γ :=
Revised: Z γ :=

Original: X γ × Y γ which restricts that q i =
Revised: X γ × Y γ which restricts that q i =

Original: z i z σ ( h ( i )) ≥ γ for any z ∈ Z γ and index i .
Revised: z i z σ ( h ( i )) ≥ γ for any z ∈ Z γ and index i .

Original: An extensive-form perfect equilibrium is a limit point of { z γ, ∗ } γ → 0 where z γ, ∗ is the NE of the γ -perturbed EFG.
Revised: An extensive-form perfect equilibrium is a limit point of { z γ, ∗ } γ → 0 where z γ, ∗ is the NE of the γ -perturbed EFG.

Original: The simplest instance of γ -perturbed treeplex is a γ -perturbed probability simplex ∆ γ where all entries have a probability larger than γ .
Revised: The simplest instance of γ -perturbed treeplex is a γ -perturbed probability simplex ∆ γ where all entries have a probability larger than γ .

Original: Since the standard EFG is just a perturbed EFG with γ = 0 , we will only describe our results in γ -perturbed EFG to keep the argument unified and general, and only translate our result to the γ = 0 case when necessary.
Revised: Since the standard EFG is just a perturbed EFG with γ = 0 , we will only describe our results in γ -perturbed EFG to keep the argument unified and general, and only translate our result to the γ = 0 case when necessary.

Original: Correspondingly, we use z γ, ∗ τ to denote the Nash equilibrium of the regularized game in Eq (3.1)
Revised: Correspondingly, we use z γ, ∗ τ to denote the Nash equilibrium of the regularized game in Eq (3.1)

Original: when ( x , y ) ∈
Revised: when ( x , y ) ∈

Original: Z γ .
Revised: Z γ .

Original: When γ > 0 , z γ, ∗ is empirically used as an approximation to the EFPE (Kroer et al., 2017; Farina et al., 2017).
Revised: When γ > 0 , z γ, ∗ is empirically used as an approximation to the EFPE (Kroer et al., 2017; Farina et al., 2017).

Original: We prove that z γ, ∗ could been seen as an approximation of EFPE in terms of duality gap (See Lemma F.4 for more details about this approximation, which might be of independent interest).
Revised: We prove that z γ, ∗ could been seen as an approximation of EFPE in terms of duality gap (See Lemma F.4 for more details about this approximation, which might be of independent interest).

Original: 4. M AIN RESULT
Revised: 4. M AIN RESULT

Original: Given the regret decomposition in Lemma 4.1, we instantiate the regret minimizer in each information set by the regularized version of the Dual Stabilized Optimistic Mirror Descent algorithm (Hsieh et al., 2021), i.e., Reg-DS-OptMD .
Revised: Given the regret decomposition in Lemma 4.1, we instantiate the regret minimizer in each information set by the regularized version of the Dual Stabilized Optimistic Mirror Descent algorithm (Hsieh et al., 2021), i.e., Reg-DS-OptMD .

Original: The DS-OptMD algorithm in (Hsieh et al., 2021) achieves constant regret in two player zero-sum NFGs, which to the best of our knowledge, is the state-of-the-art result that achieves this desired property.
Revised: The DS-OptMD algorithm in (Hsieh et al., 2021) achieves constant regret in two player zero-sum NFGs, which to the best of our knowledge, is the state-of-the-art result that achieves this desired property.

Original: Hence, we develop our local regret minimizer based on this algorithm.
Revised: Hence, we develop our local regret minimizer based on this algorithm.

Original: For any information set h ∈ H Z and t = 1 , 2 , ..., T , the full update rule of our proposed algorithm, Regularized Counterfactual Regret Minimization ( Reg-CFR ), follows
Revised: For any information set h ∈ H Z and t = 1 , 2 , ..., T , the full update rule of our proposed algorithm, Regularized Counterfactual Regret Minimization ( Reg-CFR ), follows

Original: [Equation]
Revised: [Equation]

Original: where the adaptive stepsize is defined by λ ht := (cid:113) κ + (cid:80) t − 1 s =1 δ hs and κ ≥ 1 is a hyper-parameter.
Revised: where the adaptive stepsize is defined by λ ht := (cid:113) κ + (cid:80) t − 1 s =1 δ hs and κ ≥ 1 is a hyper-parameter.

Original: δ hs :=
Revised: δ hs :=

Original: ∥ V h ( z s + 1 2 ) − V h ( z s − 1 2 ) ∥ 2 is the variation of value function and q t,h
Revised: ∥ V h ( z s + 12 ) − V h ( z s − 12 ) ∥ 2 is the variation of value function and q t,h

Original: = z t,h z t,σ ( h ) .
Revised: = z t,h z t,σ ( h ) .

Original: Again, for any h ∈ H Z , q 0 ,h = q 12 ,h are intialized as uniform distribution in ∆ | Ω h | .
Revised: Again, for any h ∈ H Z , q 0 ,h = q 12 ,h are intialized as uniform distribution in ∆ | Ω h | .

Original: With the adaptive stepsize in Reg-DS-OptMD , we no longer need to tune the stepsize for each individual information set.
Revised: With the adaptive stepsize in Reg-DS-OptMD , we no longer need to tune the stepsize for each individual information set.

Original: Reg-CFR enjoys a desirable last-iterate convergence guarantee of the actual iterate as follows:
Revised: Reg-CFR enjoys a desirable last-iterate convergence guarantee of the actual iterate as follows:

Original: Theorem 4.3.
Revised: Theorem 4.3.

Original: Consider the case when τ > 0 .
Revised: Consider the case when τ > 0 .

Original: In γ -perturbed EFGs, if we use Euclidean norm as the regularizer ψ ∆ in Reg-CFR , then (cid:80) Tt
Revised: In γ -perturbed EFGs, if we use Euclidean norm as the regularizer ψ ∆ in Reg-CFR , then (cid:80) Tt

Original: =1 D ψ Z ( z γ, ∗ τ , z t ) ≤ C γ τ , where C γ is some positive variable depending on γ .
Revised: =1 D ψ Z ( z γ, ∗ τ , z t ) ≤ C γ τ , where C γ is some positive variable depending on γ .

Original: As a result,
Revised: As a result,

Original: • When γ > 0 and τ ≤ 1 2 ∥ α ∥ ∞ , C γ is a constant which implies asymptotic last-iterate convergence to z γ, ∗ τ in terms of Bregman distance.
Revised: • When γ > 0 and τ ≤ 1 2 ∥ α ∥ ∞ , C γ is a constant which implies asymptotic last-iterate convergence to z γ, ∗ τ in terms of Bregman distance.

Original: • When γ = 0 , C γ ≤ O ( T 1 / 4 ) , implying a O ( T − 3 / 4 ) best-iterate convergence rate to z ∗ τ in terms of Bregman distance.
Revised: • When γ = 0 , C γ ≤ O ( T 1 / 4 ) , implying a O ( T − 3 / 4 ) best-iterate convergence rate to z ∗ τ in terms of Bregman distance.

Original: To the best of our knowledge, under the regret decomposition framework, although some CFR-type algorithms, like CFR+ (Tammelin et al., 2015), have been empirically observed to have last-iterate convergence (Bowling et al., 2015), there is no theoretical justifications for them in the literature yet.
Revised: To the best of our knowledge, under the regret decomposition framework, although some CFR-type algorithms, like CFR+ (Tammelin et al., 2015), have been empirically observed to have last-iterate convergence (Bowling et al., 2015), there is no theoretical justifications for them in the literature yet.

Original: Our results appear to be the first to establish the provable best- and last-iterate convergence results under the regret decomposition framework of CFR.
Revised: Our results appear to be the first to establish the provable best- and last-iterate convergence results under the regret decomposition framework of CFR.

Original: Even in terms of empirical performance, our algorithm Reg-CFR can achieve faster last-iterate convergence rate comparing to previous ones.
Revised: Even in terms of empirical performance, our algorithm Reg-CFR can achieve faster last-iterate convergence rate comparing to previous ones.

Original: More interestingly, by applying regularization to CFR (Zinkevich et al., 2007) and CFR+ (Tammelin et al., 2015), we empirically show that regularization can improve the last-iterate performance.
Revised: More interestingly, by applying regularization to CFR (Zinkevich et al., 2007) and CFR+ (Tammelin et al., 2015), we empirically show that regularization can improve the last-iterate performance.

Original: We will discuss them in Appendix B.
Revised: We will discuss them in Appendix B.

Original: Significance of last-iterate convergence for CFR.
Revised: Significance of last-iterate convergence for CFR.

Original: We believe that Theorem 4.3 paves the way for more tractable CFR-type algorithms with function approximation in large-scale EFGs like Texas
Revised: We believe that Theorem 4.3 paves the way for more tractable CFR-type algorithms with function approximation in large-scale EFGs like Texas

Original: Hold’em.
Revised: Hold’em.

Original: Previously, although Brown and Sandholm (2018; 2019b) achieved super-human level performance in Texas Hold’em, they utilized domain-specific abstraction technique (Ganzfried and
Revised: Previously, although Brown and Sandholm (2018; 2019b) achieved super-human level performance in Texas Hold’em, they utilized domain-specific abstraction techniques (Ganzfried and

Original: Sandholm, 2014; Brown et al., 2015), which will merge the similar nodes in Texas Hold’em into one to make the total number of nodes tractable.
Revised: Sandholm, 2014; Brown et al., 2015), which will merge the similar nodes in Texas Hold’em into one to make the total number of nodes tractable.

Original: However, the existing abstraction methods are highly restricted to the poker games.
Revised: However, the existing abstraction methods are highly restricted to the poker games.

Original: Therefore, it is crucial to design algorithms with function approximation to do such abstraction in an end-to-end manner.
Revised: Therefore, it is crucial to design algorithms with function approximation to do such abstraction in an end-to-end manner.

Original: Currently, the average-iterate convergence of CFR is an obstacle to  function approximation.
Revised: Currently, the average-iterate convergence of CFR is an obstacle to using function approximation.

Original: In the seminal work Deep-CFR (Brown et al., 2019), the authors trained an additional network to maintain the average policy, which caused additional approximation error.
Revised: In the seminal work Deep-CFR (Brown et al., 2019), the authors trained an additional network to maintain the average policy, which caused additional approximation errors.

Original: In the following work (Steinberger, 2019; Steinberger et al., 2020), to get the average policy, they stored  networks at every iteration on disk and sample one randomly to follow.
Revised: In the subsequent work (Steinberger, 2019; Steinberger et al., 2020), to get the average policy, they stored the networks at every iteration on disk and sampled one randomly to follow.

Original: Though sampling successfully eliminates  additional approximation error, given that it takes at least 10 5 iterations to converge in large poker games, storing all networks on disk is not tractable for large games like Texas Hold’em.
Revised: Though sampling successfully eliminates the additional approximation error, given that it takes at least 10 5 iterations to converge in large poker games, storing all networks on disk is not tractable for large games like Texas Hold’em.

Original: With Theorem 4.3, we can easily run CFR with function approximation since we only need to take the best model during learning due to the best-iterate guarantee 1 .
Revised: With Theorem 4.3, we can easily run CFR with function approximation since we only need to take the best model during learning due to the best-iterate guarantee 2 .

Original: A direct consequence of the theorem above is the following corollary.
Revised: A direct consequence of the theorem above is the following corollary.

Original: Corollary 4.4.
Revised: Corollary 4.4.

Original: For any desired duality gap ϵ , we can set τ =
Revised: For any desired duality gap ϵ , we can set τ =

Original: Θ( ϵ ) .
Revised: Θ( ϵ ) .

Original: The best-iterate convergence to the NE z ∗ when γ = 0 would be O
Revised: The best-iterate convergence to the NE z ∗ when γ = 0 would be O

Original: ( T − 1 / 4 ) .
Revised: ( T − 1 / 4 ) .

Original: We will still have asymptotic last-iterate convergencewhen γ > 0 to z ∗ ,γ , the NE of the γ -perturbed EFG, both in terms of duality gap.
Revised: When γ > 0 , we will still have asymptotic last-iterate convergence to z ∗ ,γ , the NE of the γ -perturbed EFG, both in terms of duality gap.

Original: Remark 4.5 (Technical challenges in showing best-iterate convergence for CFR-type algorithms) .
Revised: Remark 4.5 (Technical challenges in showing best-iterate convergence for CFR-type algorithms) .

Original: Although OMD achieves last-iterate convergence (Daskalakis and Panageas, 2019; Wei et al., 2021) and fast average-iterate convergence (Rakhlin and Sridharan, 2013; Syrgkanis et al., 2015), applying
Revised: Although OMD achieves last-iterate convergence (Daskalakis and Panageas, 2019; Wei et al., 2021) and fast average-iterate convergence (Rakhlin and Sridharan, 2013; Syrgkanis et al., 2015), applying

Original: OMD as local regret minimizer in the CFR framework does not enjoy those results since the loss function for the regret minimizer depends on the global strategy in the treeplex which is not totally controlled by the local regret minimizer as in the NFGs.
Revised: OMD as local regret minimizer in the CFR framework does not enjoy those results since the loss function for the regret minimizer depends on the global strategy in the treeplex which is not totally controlled by the local regret minimizer as in the NFGs.

Original: Therefore, the local regret minimizer could be seen as deployed in a changing environment where the previous results do not apply.
Revised: Therefore, the local regret minimizer could be seen as deployed in a changing environment where the previous results do not apply.

Original: Moreover, as a by-product, we find that the average strategy produced by Reg-CFR is also superior comparing to the previous variants of CFR algorithms to our best knowledge.
Revised: Moreover, as a by-product, we find that the average strategy produced by Reg-CFR is also superior comparing to the previous variants of CFR algorithms to our best knowledge.

Original: Notice that when picking τ = 0 , the algorithm will converge to the NE z ∗ ,γ of the γ -perturbed EFG.
Revised: Notice that when picking τ = 0 , the algorithm will converge to the NE z ∗ ,γ of the γ -perturbed EFG.

Original: Theorem 4.6.
Revised: Theorem 4.6.

Original: Consider the case when τ ≥ 0 and the regularizer is Euclidean norm.
Revised: Consider the case when τ ≥ 0 and the regularizer is Euclidean norm.

Original: In γ -perturbed
Revised: In γ -perturbed

Original: EFGs with γ > 0 and τ ≤ 1 2 ∥ α ∥ ∞ , the average strategy output by Reg-CFR converges to z γ, ∗ τ with convergence rate O (1 /T ) , which is the optimal rate.
Revised: EFGs with γ > 0 and τ ≤ 1 2 ∥ α ∥ ∞ , the average strategy output by Reg-CFR converges to z γ, ∗ τ with convergence rate O (1 /T ) , which is the optimal rate.

Original: In the original EFG with γ = 0 , the average strategy output by Reg-CFR converges to z ∗ τ with convergence rate O (1 /T 3 / 4 ) .
Revised: In the original EFG with γ = 0 , the average strategy output by Reg-CFR converges to z ∗ τ with convergence rate O (1 /T 3 / 4 ) .

Original: To the best of our knowledge, Reg-CFR is the first CFR-type algorithm that achieves the theoretically optimal average-iterate convergence rate O (1 /T )
Revised: To the best of our knowledge, Reg-CFR is the first CFR-type algorithm that achieves the theoretically optimal average-iterate convergence rate O (1 /T )

Original: when γ > 0 (for both τ > 0 and τ = 0 ).
Revised: when γ > 0 (for both τ > 0 and τ = 0 ).

Original: Furthermore, it maintains the current state-of-the-art average-iterate O (1 /T 3 / 4 ) convergence rate established by Farina et al. (2019a) in the original EFG where γ = 0 . 
Revised: Furthermore, it maintains the current state-of-the-art average-iterate O (1 /T 3 / 4 ) convergence rate established by Farina et al. (2019a) in the original EFG where γ = 0 . A CKNOWLEDGEMENT

Revised: T.Y. was supported by NSF CCF-2112665 (TILOS AI Research Institute).

Original: For the conclusion and discussion, please refer to Appendix G.
Revised: A.O and K.Z. were supported by MIT-DSTA grant 031017-00016. K.Z. also acknowledges support from Simons-Berkeley Research Fellowship. The authors also thank Suvrit Sra for the valuable discussions

Original: “The Power of Regularization in Solving 
Revised: “The Power of Regularization in Solving Extensive-Form Games” A O MITTED D ETAILS

Original: Extensive -Form Games” A O MITTED D ETAILS

Original: Here we present some details omitted in the maintext.
Revised: Here we present some details omitted in the maintext.

Original: A. R ELATED WORK
Revised: A. R ELATED WORK

Original: Regularization.
Revised: Regularization.

Original: In reinforcement learning, regularization has been widely used to accelerate convergence and encourage exploration (Tuyls et al., 2003; Geist et al., 2019; Cen et al., 2021a; Mei et al., 2020).
Revised: In reinforcement learning, regularization has been widely used to accelerate convergence and encourage exploration (Tuyls et al., 2003; Geist et al., 2019; Cen et al., 2021a; Mei et al., 2020).

Original: In game theory, regularization can be used to turn the bilinear objective in normalform games into a strongly-convex-strongly-concave one (Hofbauer and Hopkins, 2005; Cen et al., 2021b).
Revised: In game theory, regularization can be used to turn the bilinear objective in normalform games into a strongly-convex-strongly-concave one (Hofbauer and Hopkins, 2005; Cen et al., 2021b).

Original: However, Hofbauer and Hopkins (2005) only gave asymptotic convergence to the NE of the regularized game under the best-response dynamics and Cen et al.
Revised: However, Hofbauer and Hopkins (2005) only gave asymptotic convergence to the NE of the regularized game under the best-response dynamics and Cen et al.

Original: (2021b) only provided convergence of OMWU to the original NE in terms of duality gap.
Revised: (2021b) only provided convergence of OMWU to the original NE in terms of duality gap.

Original: Similar ideas could be dated back to the smoothing techniques led by Nesterov (2003).
Revised: Similar ideas could be dated back to the smoothing techniques led by Nesterov (2003).

Original: This way,  linear convergence rate to the saddle point of the new objective can be guaranteed.
Revised: This way, the linear convergence rate to the saddle point of the new objective can be guaranteed.

Original: Letting the regularization  small, the solution to the regularized problem can be close to the NE of the original problem, in terms of duality gap (Cen et al., 2021b).
Revised: Letting the regularization be small, the solution to the regularized problem can be close to the NE of the original problem, in terms of duality gap (Cen et al., 2021b).

Original: In contrast, we aim to show the convergence in terms of not only  duality gap, but also the distance to the NE set (of the original problem) , and for the more complicated setting of EFGs.
Revised: In contrast, we aim to show the convergence in terms of not only the duality gap, but also the distance to the NE set (of the original problem) , and for the more complicated setting of EFGs.

Original: The idea of using regularization in learning in games has also been explored recently in various different settings (Perolat et al., 2021;
Revised: The idea of using regularization in learning in games has also been explored recently in various different settings (Perolat et al., 2021;

Original: Leonardos et al., 2021).
Revised: Leonardos et al., 2021).

Original: Specifically, Perolat et al.
Revised: Specifically, Perolat et al.

Original:  Leonardos et al.
Revised: ; Leonardos et al.

Original: (2021) study continuous-time dynamics and establish convergence to NE, either only gave rate to the NE of the regularized game, or only guaranteed asymptotic convergence to the NE of the original game, using techniques based on Lyapunov arguments.
Revised: (2021) study continuous-time dynamics and establish convergence to NE, either only gave rate to the NE of the regularized game, or only guaranteed asymptotic convergence to the NE of the original game, using techniques based on Lyapunov arguments.

Original: Instead, our focus was on discrete-time optimistic mirror-descent algorithms with constant stepsizes, with convergence rates for both duality gap and iterate-distance.
Revised: Instead, our focus was on discrete-time optimistic mirror-descent algorithms with constant stepsizes, with convergence rates for both duality gap and iterate-distance.

Original: Finally, we note that the framework ofCFR (for solving
Revised: Finally, we note that the framework of

Original:  EFGs) was not investigated in these recent works.
Revised: CFR (for solving EFGs) was not investigated in these recent works.

Original: Last-iterate convergence.
Revised: Last-iterate convergence.

Original: Finding the NE in EFGs could be formulated as finding the saddle point of a bilinear objective function.
Revised: Finding the NE in EFGs could be formulated as finding the saddle point of a bilinear objective function.

Original: While mirror descent diverges in simple cases (in terms of the last-iterate) (Mertikopoulos et al., 2018; Bailey and Piliouras, 2018), its optimistic version receives great success in finding the saddle point, enabling both faster and last-iterate convergence guarantees (Rakhlin and Sridharan, 2013;
Revised: While mirror descent diverges in simple cases (in terms of the last-iterate) (Mertikopoulos et al., 2018; Bailey and Piliouras, 2018), its optimistic version receives great success in finding the saddle point, enabling both faster and last-iterate convergence guarantees (Rakhlin and Sridharan, 2013;

Original: Mertikopoulos et al., 2019; Lei et al., 2021; Daskalakis et al., 2018; Mokhtari et al., 2020).
Revised: Mertikopoulos et al., 2019; Lei et al., 2021; Daskalakis et al., 2018; Mokhtari et al., 2020).

Original: However, these previous works either only consider the case without constraints (which do not apply to the NFG/EFG setting), or provide only asymptotic convergence without explicit rate.
Revised: However, these previous works either only consider the case without constraints (which do not apply to the NFG/EFG setting), or provide only asymptotic convergence without explicit rate.

Original: Recently, with the unique NE assumption, Daskalakis and Panageas (2019) gives an asymptotic last-iterate convergence result for OMWU in NFGs.
Revised: Recently, with the unique NE assumption, Daskalakis and Panageas (2019) gives an asymptotic last-iterate convergence result for OMWU in NFGs.

Original: Wei et al.
Revised: Wei et al.

Original: (2021) further improves the result by showing that both OMWU and OGDA converge to the NE with a global sublinear convergence rate O (1 /T ) and a local linear convergence rate in NFGs.
Revised: (2021) further improves the result by showing that both OMWU and OGDA converge to the NE with a global sublinear convergence rate O (1 /T ) and a local linear convergence rate in NFGs.

Original: Among them, OMWU requires the unique NE assumption.
Revised: Among them, OMWU requires the unique NE assumption.

Original: Very recently, Cai et al.
Revised: Very recently, Cai et al.

Original: (2022) provides a tight last-iterate convergence for OGDA.
Revised: (2022) provides a tight last-iterate convergence for OGDA.

Original: Finally, Lee et al.
Revised: Finally, Lee et al.

Original: (2021) extends the result of OMWU from NFGs in Wei et al.
Revised: (2021) extends the result of OMWU from NFGs in Wei et al.

Original: (2021) to EFGs, and still requires the unique NE assumption.
Revised: (2021) to EFGs, and still requires the unique NE assumption.

Original: Concurrent to our submission, we are aware of Piliouras et al.
Revised: Concurrent to our submission, we are aware of Piliouras et al.

Original: (2022), which studies network zero-sum EFGs with last-iterate convergence rate guarantees, also without the unique NE assumption.
Revised: (2022), which studies network zero-sum EFGs with last-iterate convergence rate guarantees, also without the unique NE assumption.

Original: However, the regularizer therein for the OMD update rule is neither dilated nor entropy-based, which makes the algorithm less scalable than the one we study, with dilated and entropy-based regularizer, see Lee et al.
Revised: However, the regularizer therein for the OMD update rule is neither dilated nor entropy-based, which makes the algorithm less scalable than the one we study, with dilated and entropy-based regularizer, see Lee et al.

Original: (2021) for a related discussion.
Revised: (2021) for a related discussion.

Original: Counterfactual regret minimization (CFR).
Revised: regret minimization (CFR).

Original: CFR-type algorithms are based on the idea that the regret in an EFG could be decomposed into the local regret of each information set.
Revised: CFR-type algorithms are based on the idea that the regret in an EFG could be decomposed into the local regret of each information set.

Original: By minimizing the local regret, the global regret will be minimized and the algorithms will achieve average-iterate convergence thereby.
Revised: By minimizing the local regret, the global regret will be minimized and the algorithms will achieve average-iterate convergence thereby.

Original: Recent work Farina et al. (2019a) utilizes the progress in the aforementioned optimistic methods, and achieves a faster average-iterate convergence rate of O (1 /T 3 / 4 ) in EFGs.
Revised: Recent work Farina et al. (2019a) utilizes the progress in the aforementioned optimistic methods, and achieves a faster average-iterate convergence rate of O (1 /T 3 / 4 ) in EFGs.

Original: However, since CFR-type methods rely on the regret decomposition that breaks the structure of the strategy, up to now no CFR (and variant) algorithms are able to inherit the optimal rate optimistic algorithms have enjoyed in NFGs, to the best of our knowledge.
Revised: However, since CFR-type methods rely on the regret decomposition that breaks the structure of the strategy, up to now no CFR (and variant) algorithms are able to inherit the optimal rate optimistic algorithms have enjoyed in NFGs, to the best of our knowledge.

Original: Also, due to the decomposition, although Bowling et al.
Revised: Also, due to the decomposition, although Bowling et al.

Original: (2015) has found that the last iterate of CFR+ (Tammelin et al., 2015), a variant of CFR, converges empirically, no CFR-type algorithm have the last-iterate convergence guarantee theoretically.
Revised: (2015) has found that the last iterate of CFR+ (Tammelin et al., 2015), a variant of CFR, converges empirically, no CFR-type algorithm have the last-iterate convergence guarantee theoretically.

Original: Extensive-form perfect equilibrium and perturbed EFGs.
Revised: Extensive-form perfect equilibrium and perturbed EFGs.

Original: Nash equilibrium in EFGs does not have any guarantee at the places with zero probability to reach when all players follow the NE.
Revised: Nash equilibrium in EFGs does not have any guarantee at the places with zero probability to reach when all players follow the NE.

Original: Therefore, in reality when players make an error that leads to an impossible state in the NE, still following the NE may be suboptimal.
Revised: Therefore, in reality when players make an error that leads to an impossible state in the NE, still following the NE may be suboptimal.

Original: The concept of extensive-form perfect equilibria has thus been proposed to resolve the issue (Selten, 1975).
Revised: The concept of extensive-form perfect equilibria has thus been proposed to resolve the issue (Selten, 1975).

Original: To find the EFPEs, Miltersen and Sørensen (2010); Farina and Gatti (2017) formulate the problem as a linear programming (LP), which is not tractable for large EFGs.
Revised: To find the EFPEs, Miltersen and Sørensen (2010); Farina and Gatti (2017) formulate the problem as a linear programming (LP), which is not tractable for large EFGs.

Original: Kroer et al.
Revised: Kroer et al.

Original: (2017) and Farina et al.
Revised: (2017) and Farina et al.

Original: (2017) extend the first-order method (Nesterov, 2005) and CFR to the perturbed extensive-form game (Selten, 1975) (which can be used for finding approximate EFPEs), where players have a small probability choosing to act randomly at every information set.
Revised: (2017) extend the first-order method (Nesterov, 2005) and CFR to the perturbed extensive-form game (Selten, 1975) (which can be used for finding approximate EFPEs), where players have a small probability choosing to act randomly at every information set.

Original: Both of the results do not have last-iterate convergence.
Revised: Both of the results do not have last-iterate convergence.

Original: A. A GRAPHICAL ILLUSTRATION OF TREEPLEX
Revised: A. A GRAPHICAL ILLUSTRATION OF TREEPLEX

Original: For better understanding of the structure of treeplex, we show the treeplex of the player who moves first in Kuhn Poker in Figure 1.
Revised: For better understanding of the structure of treeplex, we show the treeplex of the player who moves first in Kuhn Poker in Figure 1.

Original: [Figure]
Revised: [Figure]

Original: Branching: h 1 1 h 2 .
Revised: Branching: h 1 1 h 2 .

Original: • Cartesian Product: Cartesian product of 3 similar treeplexes under Jack, Queen & King individually.
Revised: • Cartesian Product: Cartesian product of 3 similar treeplexes under Jack, Queen & King individually.

Original: And the whole game tree of Kuhn Poker is shown in Figure
Revised: And the whole game tree of Kuhn Poker is shown in Figure

Original: [Figure]
Revised: [Figure]

Original: A. P SEUDOCODE OF THE ADAPTIVE WEIGHT - SHRINKING ALGORITHM
Revised: P SEUDOCODE OF THE ADAPTIVE WEIGHT - SHRINKING ALGORITHM

Original: Here’s the practical version of adaptively shrinking τ framework mentioned in §3.2.
Revised: Here’s the practical version of adaptively shrinking τ framework mentioned in §3.2.

Original: 1: τ ← τ2: δ 
Revised: τ ← τ2: δ τ 0 ← max

Original: [Equation]
Revised: [Equation]

Original: ←
Revised: F

Original: [Equation]
Revised: [Equation]

Original: 3: z 0 , (cid:98)z 1 ← Uniform Strategy 4: for t = 1 , 2 , ... do 5: z t , (cid:98) 
Revised: z [Equation] (cid:98) do 5: z t , (cid:98) +

Original: z t +1 ← Reg-DOMD (z t −1 , (cid:98)z t ) 6: if max z ′ F ( (cid:98)
Revised: Reg DOMD ( t − (cid:98) ) 6: if max z ′ F ( (cid:98)

Original: z t ) ⊤ ( z t − z ′ )
Revised: z t ) ⊤ ( z t − z ′ )

Original: + τψ Z ( (cid:98) z t ) − τψ Z ( z ′ ) ≤
Revised: + τψ Z ( (cid:98) z t ) − τψ Z ( z ′ ) ≤

Original: δ τ 4 then 7: τ ← [Equation] 8: δ τ ← max z [Equation]
Revised: δ τ 4 then 7: τ ← τ8:

Original:  F [Equation] (cid:98)z t ) ⊤
Revised: δ τ ← max z ′ F ( (cid:98)

Original: [Equation] z t − z [Equation]
Revised: z t ) ⊤ ( z t − z ′ )

Original: + τψ Z ( (cid:98) z t [Equation]{+) − τψ Z ( z ′ ) 
+}
Revised: + τψ Z ( (cid:98) z t {+) − τψ Z ( z ′ ) 
+}

Original: − τψ Z ( z ′ ) 9: z t ← (cid:98) z t +1 10: end if 11: end for
Revised: z t ← (cid:98) z t +1 10: end if 11: end for

Original: Notice that this framework can also be applied to Reg-CFR by simply changing Reg-DOMD to
Revised: Notice that this framework can also be applied to Reg-CFR by simply changing Reg-DOMD to

Original: Reg-CFR .
Revised: Reg-CFR .

Original: A. E XPERIMENT ENVIRONMENTS
Revised: A. E XPERIMENT ENVIRONMENTS

Original: Kuhn Poker (Kuhn, 1950).
Revised: Kuhn Poker (Kuhn, 1950).

Original: In Kuhn Poker, there are two players and three cards, Jack , Queen and King .
Revised: In Kuhn Poker, there are two players and three cards, Jack , Queen and King .

Original: And at the beginning, each player should place 1 chip into the pot
Revised: And at the beginning, each player should place 1 chip into the pot

Original: and then 1 private card will be dealt to each player.
Revised: and then 1 private card will be dealt to each player.

Original: And each player can call, raise or fold in each round.
Revised: And each player can call, raise or fold in each round.

Original: If a player call, then she should ensure that each player contributes equally to the pot.
Revised: If a player call, then she should ensure that each player contributes equally to the pot.

Original: If a player raise, she should put 1 more chip in the pot than the other.
Revised: If a player raise, she should put 1 more chip in the pot than the other.

Original: If a player fold, then the other player takes all the chips in the pot.
Revised: If a player fold, then the other player takes all the chips in the pot.

Original: There will be at most 1 raise in the game.
Revised: There will be at most 1 raise in the game.

Original: And a betting round ends when both players call or one of them fold.
Revised: And a betting round ends when both players call or one of them fold.

Original: After the game ends and nobody folds, the two players reveal their private cards and the one with higher rank takes all the chips in the pot.
Revised: After the game ends and nobody folds, the two players reveal their private cards and the one with higher rank takes all the chips in the pot.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Leduc Poker (Southey et al., 2005) .
Revised: Leduc Poker (Southey et al., 2005) .

Original: Leduc Poker is similar to Kuhn Poker.
Revised: Leduc Poker is similar to Kuhn Poker.

Original: It has 6 cards, three ranks ( { J, Q, K } ) with two suits ( { a, b } ) each.
Revised: It has 6 cards, three ranks ( { J, Q, K } ) with two suits ( { a, b } ) each.

Original: There are two betting rounds in Leduc Poker, each round admits two raises.
Revised: There are two betting rounds in Leduc Poker, each round admits two raises.

Original: The player who raises should place 1 more chip in the first round andchips in the second.
Revised: The player who raises should place 1 more chip in the first round andchips in the second.

Original: If the game ends and nobody folds, then the players reveal their private cards.
Revised: If the game ends and nobody folds, then the players reveal their private cards.

Original: The one who has the same private card as the public card wins.
Revised: The one who has the same private card as the public card wins.

Original: If nobody has the same private card as the public card, then the one with higher rank wins.
Revised: If nobody has the same private card as the public card, then the one with higher rank wins.

Original: Otherwise the game draws and the two players share the pot equally.
Revised: Otherwise the game draws and the two players share the pot equally.

Original: B E XPERIMENT RESULTS
Revised: B E XPERIMENT RESULTS

Original: Beyond sharp theoretical guarantees, regularized algorithms in EFG also have superior performance in practice, which we showcase in this section through numerical experiments in Kuhn Poker (Kuhn, 1950) and Leduc Poker (Southey et al., 2005).
Revised: Beyond sharp theoretical guarantees, regularized algorithms in EFG also have superior performance in practice, which we showcase in this section through numerical experiments in Kuhn Poker (Kuhn, 1950) and Leduc Poker (Southey et al., 2005).

Original: The details of the experiment setup are illustrated in
Revised: The details of the experiment setup are illustrated in

Original: Appendix A.
Revised: Appendix A.

Original: The results are shown in Figure 3 for the last-iterate convergence in duality gap.
Revised: The results are shown in Figure 3 for the last-iterate convergence in duality gap.

Original: We used grid search to find the best parameters for each algorithm.
Revised: We used grid search to find the best parameters for each algorithm.

Original: The algorithms Reg-DOMWU , Reg-DOGDA ,
Revised: The algorithms Reg-DOMWU , Reg-DOGDA ,

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: Reg-CFR all apply the adaptive weight-shrinking framework proposed as Algorithm 1 in Appendix
Revised: Reg-CFR all apply the adaptive weight-shrinking framework proposed as Algorithm 1 in Appendix

Original: A.
Revised: A.

Original: As shown in Figure 4, we show the regret upper-bound max (cid:98) z ∈Z (cid:80) h ∈H Z (cid:98)
Revised: As shown in Figure 4, we show the regret upper-bound max (cid:98) z ∈Z (cid:80) h ∈H Z (cid:98)

Original: z σ ( h ) R hT .
Revised: z σ ( h ) R hT .

Original: We can see that Reg-CFR has constant regret even in a non-perturbed EFG in practice.
Revised: We can see that Reg-CFR has constant regret even in a non-perturbed EFG in practice.

Original: Moreover, we further empirically show that regularization is also helpful for CFR and CFR+.
Revised: Moreover, we further empirically show that regularization is also helpful for CFR and CFR+.

Original: That is, with RM and RM+ as local regret minimizers, adding regularization still helps the algorithm enjoy last-iterate convergence.
Revised: That is, with RM and RM+ as local regret minimizers, adding regularization still helps the algorithm enjoy last-iterate convergence.

Original: See Figure 5 for the details.
Revised: See Figure 5 for the details.

Original: To minimize the regret of a convex but non-linear loss function l t ( x t ) , we feed ⟨∇ l t ( x t ) , x t ⟩ into RM and RM+ as the loss function.
Revised: To minimize the regret of a convex but non-linear loss function l t ( x t ) , we feed ⟨∇ l t ( x t ) , x t ⟩ into RM and RM+ as the loss function.

Original: See (Farina et al., 2019d, §2.1) for more details.
Revised: See (Farina et al., 2019d, §2.1) for more details.

Original: Moreover, the average-iterate convergence rate of this regularized version is still competitive with the original version.
Revised: Moreover, the average-iterate convergence rate of this regularized version is still competitive with the original version.

Original: See Figure 6 for details.
Revised: See Figure 6 for details.

Original: Figure 7 illustrates the duality gap of average iterate.
Revised: Figure 7 illustrates the duality gap of average iterate.

Original: We can see that Reg-CFR is faster than CFR in both environments and has a comparable performance with CFR+ in smaller environments like
Revised: We can see that Reg-CFR is faster than CFR in both environments and has a comparable performance with CFR+ in smaller environments like

Original: Kuhn Poker.
Revised: Kuhn Poker.

Original: Figure 8 illustrates the maximum cumulative regret across all information sets, conditioned on reaching that information set.
Revised: Figure 8 illustrates the maximum cumulative regret across all information sets, conditioned on reaching that information set.

Original: This is also used as metric in Farina et al.
Revised: This is also used as metric in Farina et al.

Original: ; Kroer et al.
Revised: ; Kroer et al.

Original: This metric can be used to measure the ”closeness” to EFPEs.
Revised: This metric can be used to measure the ”closeness” to EFPEs.

Original: We can see that with γ > 0 , Reg-CFR significantly outperforms CFR and CFR+ in finding EFPEs.
Revised: We can see that with γ > 0 , Reg-CFR significantly outperforms CFR and CFR+ in finding EFPEs.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: C P ROOF OF T HEOREM 3. 
Revised: C P ROOF OF T HEOREM 3.

Original: Lemma C.1.
Revised: Lemma C.1.

Original: For any τ ≤ 1 and z ∈ Z , the NE of the regularized problem Eq (3.1) satisfies that
Revised: For any τ ≤ 1 and z ∈ Z , the NE of the regularized problem Eq (3.1) satisfies that

Original: [Equation]
Revised: [Equation]

Original: Lemma C.2.
Revised: Lemma C.2.

Original: Consider the update rule in Eq (3.2).
Revised: Consider the update rule in Eq (3.2).

Original: When ψ Z satisfies Eq (C.6) with p = 2 and η ≤ 18 P , then for any z ∈ Z and t ≥
Revised: When ψ Z satisfies Eq (C.6) with p = 2 and η ≤ 18 P , then for any z ∈ Z and t ≥

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Proof of Theorem
Revised: Proof of Theorem

Original: [Equation]
Revised: [Equation]

Original: Taking z
Revised: Taking z

Original: = z ∗ τ in Lemma C.
Revised: = z ∗ τ in Lemma C.

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where ( i ) is by Lemma C.1.
Revised: where ( i ) is by Lemma C.1.

Original: Letting Θ t +

Revised: Letting Θ t +

Original: = D ψ Z ( z ∗ τ , (cid:98) z t +1 )
Revised: = D ψ Z ( z ∗ τ , (cid:98) z t +1 )

Original: + D ψ Z ( (cid:98)
Revised: + D ψ Z ( (cid:98)

Original: z t
Revised: z t

Original: +

Revised: +

Original: z t ) , inequality (C.2) can be written as
Revised: z t ) , inequality (C.2) can be written as

Original: [Equation]
Revised: [Equation]

Original: where the second inequality comes from ητ ≤ η ≤ 78 .
Revised: where the second inequality comes from ητ ≤ η ≤ 78 .

Original: As a result,
Revised: As a result,

Original: [Equation]
Revised: [Equation]

Original: where the last equation is satisfied when we initialize z
Revised: where the last equation is satisfied when we initialize z

Original: 0 = (cid:98) z 1 .
Revised: 0 = (cid:98) z 1 .

Original: Lemma C.3.
Revised: Lemma C.3.

Original: F ( z ) is P -Lipschitz for any z ∈ Z .
Revised: F ( z ) is P -Lipschitz for any z ∈ Z .

Original: That is, for any z , z ′ ∈ Z , we have
Revised: That is, for any z , z ′ ∈ Z , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: Lemma C.4.
Revised: Lemma C.4.

Original: Let C be a convex set and u 1 = argmin (cid:98) 
Revised: Let C be a convex set and u 1 = argmin (cid:98) u

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: u ) } where ψ C is a strongly-convex function in C .
Revised: u ) } where ψ C is a strongly-convex function in C .

Original: Then for any u 2 ∈ C , τ ∈ [0 , 1] , η > 0 , 
Revised: Then for any u 2 ∈ C , τ ∈ [0 , 1] , η > 0 , [Equation] [Equation] D ψ C ( u

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Plug in the definition of Bregman divergence D ψ C ( u 1 , u ) =
Revised: Plug in the definition of Bregman divergence D ψ C ( u 1 , u ) =

Original: ψ C ( u 1 ) − ψ C [Equation] u ) − ⟨∇ ψ C ( u ) , u 1 − u ⟩ , the right-hand side of it is equal to,
Revised: ψ C ( u 1 ) − ψ C ( u ) − ⟨∇ ψ C ( u ) , u 1 − u ⟩ , the right-hand side of it is equal to,

Original: [Equation]
Revised: [Equation]

Original: where ( i ) is by the first order optimality of u 1 , i.e.,
Revised: where ( i ) is by the first order optimality of u 1 , i.e.,

Original: [Equation]
Revised: [Equation]

Original: Lemma C.5.
Revised: Lemma C.5.

Original: Suppose that ψ C is a 1-strongly convex function with respect to p -norm in C such that
Revised: Suppose that ψ C is a 1-strongly convex function with respect to p -norm in C such that

Original: [Equation]
Revised: [Equation]

Original:  for some p ≥ 1 , and u , u 1 , u 2 are members of a convex set C such that,
Revised: C [Equation] [Equation] for some p ≥ 1 , and u , u 1 , u 2 are members of a convex set C such that,

Original: [Equation]
Revised: [Equation]

Original: Then we have,
Revised: Then we have,

Original: [Equation]
Revised: [Equation]

Original: where q ≥ 1 and 1 p + 1 q = 1 .
Revised: where q ≥ 1 and 1 p + 1 q = 1 .

Original: Proof.
Revised: Proof.

Original: By the first-order optimality of u 1 , u 2 , we have
Revised: By the first-order optimality of u 1 , u 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Summing up and rearranging the terms,
Revised: Summing up and rearranging the terms,

Original: [Equation]
Revised: [Equation]

Original: To bound the right-hand side of inequality (C.10), By the lower bound of Bregman divergence (C.6), we have
Revised: To bound the right-hand side of inequality (C.10), By the lower bound of Bregman divergence (C.6), we have

Original: [Equation]
Revised: [Equation]

Original: Summing them up we have
Revised: Summing them up we have

Original: [Equation]
Revised: [Equation]

Original: Combining with inequality (C.10)
Revised: Combining with inequality (C.10)

Original: [Equation]
Revised: [Equation]

Original: Finally, by H ¨ older’s inequality
Revised: Finally, by H ¨ older’s inequality

Original: [Equation]
Revised: [Equation]

Original: and as a result ∥ u 1 − u 2 ∥ p ≤ ∥ g 1 − g 

Revised: and as a result ∥ u 1 − u 2 ∥ p ≤ ∥ g 1 − g

Original: ∥ q as claimed.
Revised: ∥ q as claimed.

Original: Proof of Lemma C.

Revised: Proof of Lemma C.

Original: By definition of NE, we have
Revised: By definition of NE, we have

Original: [Equation]
Revised: [Equation]

Original: x ⊤
Revised: x ⊤

Original: Ay ∗ τ
Revised: Ay ∗ τ

Original: + τψ Z ( x ) (cid:17) − τ ( ψ Z ( x ) + ψ Z ( y ))
Revised: + τψ Z ( x ) (cid:17) − τ ( ψ Z ( x ) + ψ Z ( y ))

Original: ≥ − x ∗⊤ τ
Revised: ≥ − x ∗⊤ τ

Original: Ay ∗ τ
Revised: Ay ∗ τ

Original: + τψ Z ( y ∗ τ )
Revised: + τψ Z ( y ∗ τ )

Original: + x ∗⊤ τ
Revised: + x ∗⊤ τ

Original: Ay ∗ τ + τψ Z ( x ∗ τ ) − τ ( ψ Z ( x ) + ψ Z ( y ))
Revised: Ay ∗ τ + τψ Z ( x ∗ τ ) − τ ( ψ Z ( x ) + ψ Z ( y ))

Original: = τψ Z ( z ∗ τ ) − τψ Z ( z ) .
Revised: = τψ Z ( z ∗ τ ) − τψ Z ( z ) .

Original: Proof of Lemma C.2.
Revised: Proof of Lemma C.2.

Original: Plug u = (cid:98) z t , u 1 = (cid:98) z t +1 , u 2 =
Revised: Plug u = (cid:98) z t , u 1 = (cid:98) z t +1 , u 2 =

Original: z , g = F ( z t ) , ψ C = ψ
Revised: z , g = F ( z t ) , ψ C = ψ

Original: Z into Lemma C.4, ητψ Z ( (cid:98)
Revised: Z into Lemma C.4, ητψ Z ( (cid:98)

Original: z t +1 ) − ητψ Z ( z )+ η ⟨ (cid:98) z t +1 − z , F ( z t ) ⟩ ≤ (1 − ητ )
Revised: z t +1 ) − ητψ Z ( z )+ η ⟨ (cid:98) z t +1 − z , F ( z t ) ⟩ ≤ (1 − ητ )

Original: D ψ Z ( z , (cid:98) z t ) − D ψ Z
Revised: D ψ Z ( z , (cid:98) z t ) − D ψ Z

Original: ( z , (cid:98) z t +1 ) − (1 − ητ )
Revised: ( z , (cid:98) z t +1 ) − (1 − ητ )

Original: D ψ Z ( (cid:98) z t +1 , (cid:98) z t ) .
Revised: D ψ Z ( (cid:98) z t +1 , (cid:98) z t ) .

Original: Plug u = (cid:98) z t , u 1 = z t , u 2 =
Revised: Plug u = (cid:98) z t , u 1 = z t , u 2 =

Original: z t +1 , g = F ( z t − 1 ) and ψ C = ψ
Revised: z t +1 , g = F ( z t − 1 ) and ψ C = ψ

Original: Z into Lemma C.4, ητψ Z ( z t ) − ητψ Z ( (cid:98) z t +1 )
Revised: Z into Lemma C.4, ητψ Z ( z t ) − ητψ Z ( (cid:98) z t +1 )

Original: + η ⟨ z t − (cid:98) z t +1 , F ( z t − 1 ) ⟩ ≤ (1 − ητ )
Revised: + η ⟨ z t − (cid:98) z t +1 , F ( z t − 1 ) ⟩ ≤ (1 − ητ )

Original: D ψ Z ( (cid:98) z t +1 , (cid:98) z t ) − D ψ Z ( (cid:98)
Revised: D ψ Z ( (cid:98) z t +1 , (cid:98) z t ) − D ψ Z ( (cid:98)

Original: z t +1 , z t ) − (1 − ητ )
Revised: z t +1 , z t ) − (1 − ητ )

Original: D ψ Z ( z t , (cid:98) z t ) .
Revised: D ψ Z ( z t , (cid:98) z t ) .

Original: Summing them up and adding ⟨ F ( z t ) − F ( z t − 1 ) , z t − (cid:98) z t +1 ⟩ to both sides, we have 
Revised: Summing them up and adding ⟨ F ( z t ) − F ( z t − 1 ) , z t − (cid:98) z t +1 ⟩ to both sides, we have [Table] [Equation] [Table]

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Revised: ⟩ ≤

Original: [Equation]
Revised: [Equation]

Original: 1 − ητ )
Revised: 1 − ητ )

Original: D ψ Z ( z
Revised: D ψ Z ( z

Original: [Equation]
Revised: [Equation]

Revised: (cid:98) z t ) − D

Revised: [Equation]

Revised: Z ( z

Revised: [Equation]

Revised: (cid:98)

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: [Equation]

Original: (cid:98)
Revised: (cid:98)

Original: [Equation]
Revised: [Equation]

Original: t ) − D ψ
Revised: − (

Original: [Equation]
Revised: [Equation]

Original: ( z
Revised: [Equation] (cid:98)

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: − (1 − ητ )

Original: D ψ Z ( z t 
Revised: ( z t −

Original: [Equation]
Revised: [Equation]

Original: (cid:98) z t) [Equation] ( z t ) − F ( z t −
Revised: z t

Original: [Equation]
Revised: [Equation]

Original: z t − (cid:98) z t
Revised: (cid:98) z t

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: It remains to bound the last term
Revised: It remains to bound the last term

Original: [Equation]
Revised: [Equation]

Original: which is
Revised: which is

Original: [Equation]
Revised: [Equation]

Original: where ( i ) is by H¨older’s inequality
Revised: where ( i ) is by H¨older’s inequality

Original: [Equation]
Revised: [Equation]

Original: ( ii ) is by Lemma C.5 with p = q = 2 , ( iii ) is by Lemma C.3, and ( iv ) is by η ≤ 18 P .
Revised: ( ii ) is by Lemma C.5 with p = q = 2 , ( iii ) is by Lemma C.3, and ( iv ) is by η ≤ 18 P .

Original: The proof of the claim is completed by putting everything together.
Revised: The proof of the claim is completed by putting everything together.

Original: D P ROOF OF T HEOREM 3. 
Revised: D P ROOF OF T HEOREM 3.

Original: Firstly, we will prove that the approximate NE of the regularized problem is close to the NE of the original problem in terms of duality gap.
Revised: Firstly, we will prove that the approximate NE of the regularized problem is close to the NE of the original problem in terms of duality gap.

Original: Lemma D.1.
Revised: Lemma D.1.

Original: For any τ > 0 and z ∈ Z , we have
Revised: For any τ > 0 and z ∈ Z , we have

Original: [Equation]
Revised: [Equation]

Original: where C B is the upper-bound of the regularizer ψ Z .
Revised: where C B is the upper-bound of the regularizer ψ Z .

Original: Proof.
Revised: Proof.

Original: max
Revised: max

Original: [Equation]
Revised: [Equation]

Original: where ( i ) is because of the definition of z ∗ τ and ∥ F ( z ) ∥ ∞ ≤ 1 for any z ∈ Z .
Revised: where ( i ) is because of the definition of z ∗ τ and ∥ F ( z ) ∥ ∞ ≤ 1 for any z ∈ Z .

Original: ( ii ) is by ∥ x − x ∗ τ ∥ 1 ≤ √
Revised: ( ii ) is by ∥ x − x ∗ τ ∥ 1 ≤ √

Original: P ∥ x − x ∗ τ ∥ , ∥ y − y ∗ τ ∥ 1 ≤ √ P ∥ y − y ∗ τ ∥ and a + b ≤ 2 √ a 2 + b 2 .
Revised: P ∥ x − x ∗ τ ∥ , ∥ y − y ∗ τ ∥ 1 ≤ √ P ∥ y − y ∗ τ ∥ and a + b ≤ 2 √ a 2 + b 2 .

Original: C B is the upper-bound of the regularizer ψ Z .
Revised: C B is the upper-bound of the regularizer ψ Z .

Original: It would be P ∥ α ∥ ∞ log C Ω for entropy regularizer and P ∥ α ∥ ∞ C Ω for Euclidean regularizer, where C Ω = max h ∈H Z | Ω h | .
Revised: It would be P ∥ α ∥ ∞ log C Ω for entropy regularizer and P ∥ α ∥ ∞ C Ω for Euclidean regularizer, where C Ω = max h ∈H Z | Ω h | .

Original: A direct consequence of the lemma is that for any ϵ > 0 , we can set τ = ϵ 4 C B , then after
Revised: A direct consequence of the lemma is that for any ϵ > 0 , we can set τ = ϵ 4 C B , then after

Original: [Equation]
Revised: [Equation]

Original: iterations, (cid:98) z t produced by
Revised: iterations, (cid:98) z t produced by

Original: Reg-DOMD will satisfies that
Revised: Reg-DOMD will satisfies that

Original: [Equation]
Revised: [Equation]

Original: by Theorem 3.1.
Revised: by Theorem 3.1.

Original: Proof of Theorem 3.2.
Revised: Proof of Theorem 3.2.

Original: Sublinear convergence rate of duality gap.
Revised: Sublinear convergence rate of duality gap.

Original: For any ϵ , the number of iterations that the duality gap reach ϵ is no larger than
Revised: For any ϵ , the number of iterations that the duality gap reach ϵ is no larger than

Original: [Equation]
Revised: [Equation]

Original: by the discussion above.
Revised: by the discussion above.

Original: Therefore, while duality gap reaching ϵ = ϵ 0 2 K , the number of iterations performed so far is no larger than
Revised: Therefore, while duality gap reaching ϵ = ϵ 0 2 K , the number of iterations performed so far is no larger than

Original: [Equation]
Revised: [Equation]

Original: Iterate convergence.
Revised: Iterate convergence.

Original: For the second claim, we will prove that the “slope” of the sequence-form bilinear game is strictly positive in the region that Reg-DOMWU could reach.
Revised: From the proof of Theorem 5 in Wei et al. (2021), we have the following lemma. Lemma D.2 (Proved in Theorem 5 of Wei et al. (2021)) . Consider a bilinear zero-sum game.

Original: The detail is postponed to the sections below. For any (cid:98) z t = ( (cid:98) x t , (cid:98) y t ) produced by Reg-DOMWU , we know that (cid:98) x t ∈ F x and (cid:98) y t ∈ F y .
Revised: Let ρ := min x ∈X max y ∈Y x ⊤ Ay be the game value.

Original: Therefore, by Lemma D.5, we have 
Revised: When X , Y are polytopes, we have max

Original: [Equation]
Revised: [Equation]

Original: where the second equation comes from the definition of V ∗ (  (cid:81) X ∗ ((cid:98) x t )) .
Revised: y − (cid:81) Y ∗ ( y ) (cid:13)(cid:13) ) for some constant c > 0 where (cid:81) X ∗ ( x ) ( (cid:81)

Original: Similarly, ρ −
Revised: Y ∗ ( y ) ) is the projection of x ( y ) to the NE set X ∗ ( Y ∗ ) of min-player (max-player). Then, since the treeplex is a polytope by definition, we have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality comes from √ a + b ≤ √ a + √ b .
Revised: where the last inequality comes from √ a + b ≤ √ a + √ b .

Original: Therefore, ∥ (cid:98) z t − (cid:81) Z ∗ ( (cid:98) z t )
Revised: Therefore, ∥ (cid:98) z t − (cid:81) Z ∗ ( (cid:98) z t )

Original: ∥ ≤
Revised: ∥ ≤

Original: [Equation]
Revised: [Equation]

Revised: Notice that comparing to the results in Gilpin et al.

Revised: ; Wei et al.

Revised: (2021), our slope result (Lemma

Revised: D.6) is based on different techniques.

Revised: In Lemma D.6, we prove that max (cid:98)

Revised: y ∈

Revised: V ∗ ( (cid:81) X∗

Revised: ( x )) x ⊤ A (cid:98) y − ρ ≥ c x ∥ x − (cid:81) X ∗ ( x ) ∥ where V ∗ ( (cid:81) X ∗ ( x )) ⊆

Revised: Y when x ∈ F x and F x ⊆ X contains all possible iterates generated by DOMWU .

Revised: That is, our result is stronger than the existing results, when the algorithm is DOMWU 3 .

Revised: Moreover, our result can be viewed as an extension of (Lee et al., 2021, Lemma 14) to the non-unique NE cases.

Revised: Given that (Lee et al., 2021, Lemma 14) plays an critical role in proving the last-iterate convergence with unique NE assumption, Lemma D.6 may be useful when proving last-iterate convergence in EFGs without unique NE assumption and regularization.

Original: D. C OMPLEMENTARY SLACKNESS
Revised: D. C OMPLEMENTARY SLACKNESS

Original: This part of discussion is similar to the one in Lee et al.
Revised: This part of discussion is similar to the one in Lee et al.

Original: From Definition 2.1, we have ∀ h ∈ H Y , (cid:88) 
Revised: From Definition 2.1, we have ∀ [Equation] (cid:88) [Equation]

Original: i ∈
Revised: ∈

Original: Ω h
Revised: [Equation] y [Equation] = y

Original:  y i{+
+}
Revised: [Equation] [Equation] y {+
+}

Original: =
Revised: =

Original: y σ ( h ) , y 0 =(D.7) which can be written compactly as E Y y =
Revised: [Equation] (D.6) which can be written compactly as E Y y =

Original: e Y where E Y ∈ R ( |H Y | +1)
Revised: e Y where E Y ∈ R ( |H Y | +1)

Original: × N and e Y =
Revised: × N and e Y =

Original: (1 , 0 , 0 , ..., 0) ∈ R |H
Revised: (1 , 0 , 0 , ..., 0) ∈ R |H

Original: Y | +1 .
Revised: Y | +1 .

Original: Except the first row of E Y where there’s 1 on index 0 and 0 otherwise, all other rows have 1 on index σ ( h ) and − 1 on all
Revised: Except the first row of E Y where there’s 1 on index 0 and 0 otherwise, all other rows have 1 on index σ ( h ) and − 1 on all

Original: i ∈ Ω h .
Revised: i ∈ Ω h .

Original: Therefore, for any fixed x , the objective of y can be written as
Revised: Therefore, for any fixed x , the objective of y can be written as

Original: [Equation]
Revised: [Equation]

Original: (D.8) whose dual problem is
Revised: whose dual problem is

Original: [Equation]
Revised: [Equation]

Original: where e ⊤Y g = g 0 since e Y = (1 , 0 , 0 , ..., 0)
Revised: where e ⊤Y g = g 0 since e Y = (1 , 0 , 0 , ..., 0)

Original: Remind that the primal formulation of the original problem is
Revised: Remind that the primal formulation of the original problem is

Original: [Equation]
Revised: [Equation]

Original: Therefore, every solution y ∗ of the original problem would be a solution of the following problem.
Revised: Therefore, every solution y ∗ of the original problem would be a solution of the following problem.

Original: [Equation]
Revised: [Equation]

Original: (D.11)

Original: The dual of this one is
Revised: The dual of this one is

Original: [Equation]
Revised: [Equation]

Original: D.12) Note that X ∗ , Y ∗ are the optimal solution of Eq (D.11) and Eq (D.12).
Revised: Note that X ∗ , Y ∗ are the optimal solution of Eq (D.10) and Eq (D.11).

Original: By complementary slackness, for any optimal solution pair ( x ∗ , g ∗ ) , ( y ∗ , f ∗ ) , we have slackness variables w ∗ ∈ R M , s ∗ ∈ R N
Revised: By complementary slackness, for any optimal solution pair ( x ∗ , g ∗ ) , ( y ∗ , f ∗ ) , we have slackness variables w ∗ ∈ R M , s ∗ ∈ R N

Original: so that
Revised: so that

Original: [Equation]
Revised: [Equation]

Original: where ⊙ denotes the element-wise product.
Revised: where ⊙ denotes the element-wise product.

Original: As a direct consequence, we have the following lemma.
Revised: As a direct consequence, we have the following lemma.

Original: Lemma D.2.
Revised: Lemma D.3.

Original: For any optimal solution pair ( x ∗ , g ∗ ) , ( y ∗ , f ∗ ) of Eq (D.11) and Eq (D.12), we have
Revised: For any optimal solution pair ( x ∗ , g ∗ ) , ( y ∗ , f ∗ ) of Eq (D.10) and Eq (D.11), we have

Original: [Equation]
Revised: [Equation]

Original: where supp( x ) denotes the support set of vector x and supp( C )
Revised: where supp( x ) denotes the support set of vector x and supp( C )

Original: = (cid:83) x ∈C supp( x ) denotes the support set of a convex set C .
Revised: = (cid:83) x ∈C supp( x ) denotes the support set of a convex set C .

Original: Proof.
Revised: Proof.

Original: Since ( E ⊤X f ) i
Revised: Since ( E ⊤X f ) i

Original: = f ∗ h ( i ) − (cid:80)
Revised: = f ∗ h ( i ) − (cid:80)

Original: h ∈H i f ∗ h by definition of E , from Eq (D.13), we have
Revised: h ∈H i f ∗ h by definition of E , from Eq (D.12), we have

Original: [Equation]
Revised: [Equation]

Revised: D.14)

Original: For any i where there’s x ∗ ∈
Revised: For any i where there’s x ∗ ∈

Original: X ∗ and x ∗ i > 0 , from x ∗ ⊙ w ∗ = 0 , we have w ∗ i = 0 .
Revised: X ∗ and x ∗ i > 0 , from x ∗ ⊙ w ∗ = 0 , we have w ∗ i = 0 .

Original: Thus, the above inequality takes the equality.
Revised: Thus, the above inequality takes the equality.

Original: So the first two lines of Lemma D.2 are proved.
Revised: So the first two lines of Lemma D.3 are proved.

Original: Similarly, we can prove the last two lines.
Revised: Similarly, we can prove the last two lines.

Original: We further introduce the following definitions.
Revised: We further introduce the following definitions.

Original: Definition D.3.
Revised: Definition D.4.

Original: [Equation]
Revised: [Equation]

Original:  where C ( S ) denotes the minimum convex set covering all points in S .
Revised: D [Equation] [Equation] where C ( S ) denotes the minimum convex set covering all points in S .

Original: A fact from the definition is that ∀ y ∈ V ∗ ( x ∗ ) , x ∗⊤ Ay = ρ and ∀ x ∈ V ∗ ( y ∗ ) ,
Revised: A fact from the definition is that ∀ y ∈ V ∗ ( x ∗ ) , x ∗⊤ Ay = ρ and ∀ x ∈ V ∗ ( y ∗ ) ,

Original: x ⊤
Revised: x ⊤

Original: Ay ∗ = ρ .
Revised: Ay ∗ = ρ .

Original: Lemma D.4.
Revised: Lemma D.5.

Original: V ∗ ( x ∗ ) , V ∗ ( y ∗ ) are not empty for any x ∗ ∈
Revised: V ∗ ( x ∗ ) , V ∗ ( y ∗ ) are not empty for any x ∗ ∈

Original: X ∗ , y ∗ ∈ Y ∗ .
Revised: X ∗ , y ∗ ∈ Y ∗ .

Original: Proof.
Revised: Proof.

Original: For any x ∈ X , y ∗ ∈ Y ∗ , f ∗ so that supp( x ) ⊆ supp( X ∗ ) and ( f ∗ , y ∗ ) is a pair of optimal solution of Eq (D.12), we have
Revised: For any x ∈ X , y ∗ ∈ Y ∗ , f ∗ so that supp( x ) ⊆ supp( X ∗ ) and ( f ∗ , y ∗ ) is a pair of optimal solution of Eq (D.11), we have

Original: [Equation]
Revised: [Equation]

Original: where the second equality is because supp( x ) ⊆ supp( X ∗ ) and Lemma D.2.
Revised: where the second equality is because supp( x ) ⊆ supp( X ∗ ) and Lemma D.3.

Original: The fourth equality comes from the fact that (cid:80) i ∈
Revised: The fourth equality comes from the fact that (cid:80) i ∈

Original: Ω
Revised: Ω

Original: h x i = x σ ( h ) .
Revised: h x i = x σ ( h ) .

Original: Therefore, V ∗ ( y ∗ ) is not empty for any y ∗ ∈ Y ∗ .
Revised: Therefore, V ∗ ( y ∗ ) is not empty for any y ∗ ∈ Y ∗ .

Original: Similarly, V ∗ ( x ∗ ) is not empty for any x ∗ ∈ X ∗ .
Revised: Similarly, V ∗ ( x ∗ ) is not empty for any x ∗ ∈ X ∗ .

Original: When assuming unique NE as in Lee et al.
Revised: When assuming unique NE as in Lee et al.

Original: (2021), the second line and the fourth line in Lemma D. will be strictly larger than and strictly less than by strict complementary slackness.
Revised: (2021), the second line and the fourth line in Lemma D. will be strictly larger than and strictly less than by strict complementary slackness.

Original: The discussion in Lemma D.4 turns out to be if and only if supp( x ) ⊆ supp( X ∗ ) , we have x
Revised: The discussion in Lemma D.5 turns out to be if and only if supp( x ) ⊆ supp( X ∗ ) , we have x

Original: ⊤ Ay ∗ =
Revised: ⊤ Ay ∗ =

Original: ρ which strengthen our conclusion here.
Revised: ρ which strengthen our conclusion here.

Original: D. C ONNECTION BETWEEN DUALITY GAP AND ITERATE DISTANCE
Revised: D. C ONNECTION BETWEEN DUALITY GAP AND ITERATE DISTANCE

Original: Lemma D.5.
Revised: Lemma D.6.

Original: The constants c x , c y defined below satisfy that c x , c y > 0 .
Revised: The constants c x , c y defined below satisfy that c x , c y > 0 .

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: and ϵ dil is some game dependent constant defined in Lemma D.6.
Revised: and ϵ dil is some game dependent constant defined in Lemma D.7.

Original: Proof.
Revised: Proof.

Original: Define the set X ′
Revised: Define the set X ′

Original: =
Revised: =

Original: { x | x ∈ X , ∥ x − (cid:81) X ∗ ( x ) ∥ ≥ ϵ dil } .
Revised: { x | x ∈ X , ∥ x − (cid:81) X ∗ ( x ) ∥ ≥ ϵ dil } .

Original: In the following, we will show that we only need to consider x ∈ X ′ instead of F x \ X ∗ .
Revised: In the following, we will show that we only need to consider x ∈ X ′ instead of F x \ X ∗ .

Original: Formally we will prove that for any x ∈ F x \ X ∗ , we have x ′ ∈ X ′
Revised: Formally we will prove that for any x ∈ F x \ X ∗ , we have x ′ ∈ X ′

Original: so that
Revised: so that

Original: [Equation]
Revised: [Equation]

Original: The claim trivially holds if x ∈ X ′ .
Revised: The claim trivially holds if x ∈ X ′ .

Original: Otherwise, let x ′ =
Revised: Otherwise, let x ′ =

Original: (cid:81) X ∗ ( x ) + ϵ dil ∥ x − (cid:81) X∗( x
Revised: (cid:81) X ∗ ( x ) + ϵ dil ∥ x − (cid:81) X∗

Original: [Equation]

Original:  ∥ ( x − (cid:81)X ∗ ( x ))
Revised: ( x ) ∥ ( x − (cid:81)

Original: [Equation]
Revised: X ∗ ( x )) .

Original: For any element that x i ≥ (cid:81) X ∗ ( x ) i ≥ 0 , we know that x
Revised: For any element that x i ≥ (cid:81) X ∗ ( x ) i ≥ 0 , we know that x

Original: ′ i ≥ 0 .
Revised: ′ i ≥ 0 .

Original: For elements that (cid:81) X ∗ ( x ) i > x i ≥ 0 , we can ensure that i ∈ supp( X ∗ ) , which means that (cid:81)
Revised: For elements that (cid:81) X ∗ ( x ) i > x i ≥ 0 , we can ensure that i ∈ supp( X ∗ ) , which means that (cid:81)

Original: X ∗ ( x ) i > x i ≥ ϵ dil since x ∈ F x \ X ∗ .
Revised: X ∗ ( x ) i > x i ≥ ϵ dil since x ∈ F x \ X ∗ .

Original: Therefore, we have x ′ i ≥ (cid:81)
Revised: Therefore, we have x ′ i ≥ (cid:81)

Original: X ∗ ( x ) i − | x i − (cid:81)
Revised: X ∗ ( x ) i − | x i − (cid:81)

Original: X ∗ ( x ) i | · ϵ dil ∥ x [Equation] (cid:81) X∗
Revised: X ∗ ( x ) i | · ϵ dil ∥ x − (cid:81) X∗

Original: ( x ) ∥ ≥ (cid:81) X ∗ ( x )
Revised: ( x ) ∥ ≥ (cid:81) X ∗ ( x )

Original: i − ϵ dil ≥ 0 .
Revised: i − ϵ dil ≥ 0 .

Original: Also, for any h ∈ H X ,
Revised: Also, for any h ∈ H X ,

Original: [Equation]
Revised: [Equation]

Original: (D.21)

Original: Therefore, x ′ ∈ X
Revised: Therefore, x ′ ∈ X

Original: and we can conclude that x ′ ∈ X ′ since (cid:81) X ∗ ( x ) =
Revised: and we can conclude that x ′ ∈ X ′ since (cid:81) X ∗ ( x ) =

Original: (cid:81) X ∗ ( x ′ ) .
Revised: (cid:81) X ∗ ( x ′ ) .

Original: Moreover, since x ′ − (cid:81) X ∗ ( x ) and x − (cid:81) X ∗ ( x ) are parallel and (cid:81)
Revised: Moreover, since x ′ − (cid:81) X ∗ ( x ) and x − (cid:81) X ∗ ( x ) are parallel and (cid:81)

Original: X ∗ ( x ) =
Revised: X ∗ ( x ) =

Original: (cid:81) X ∗ ( x ′ ) , we can conclude that Eq (D.20) is satisfied.
Revised: (cid:81) X ∗ ( x ′ ) , we can conclude that Eq (D.19) is satisfied.

Original: Because X ′ is closed, we can define
Revised: Because X ′ is closed, we can define

Original: [Equation]
Revised: [Equation]

Original: with the inequality that c x ≥ c ′ x and c y ≥ c ′ y by the discussion above.
Revised: with the inequality that c x ≥ c ′ x and c y ≥ c ′ y by the discussion above.

Original: Then, we will prove that c
Revised: Then, we will prove that c

Original: ′ x , c ′ y > 0 .
Revised: ′ x , c ′ y > 0 .

Original: Firstly, we will prove that c ′ y ≥ 0 .
Revised: Firstly, we will prove that c ′ y ≥ 0 .

Original: If c ′ y < 0 , then it says that there’s some y so that
Revised: If c ′ y < 0 , then it says that there’s some y so that

Original: [Equation]
Revised: [Equation]

Original: which implies that for any x ∗ ∈ X ∗ ,
Revised: which implies that for any x ∗ ∈ X ∗ ,

Original: x ∗⊤ Ay > ρ .
Revised: x ∗⊤ Ay > ρ .

Original: And it contradicts with the definition of X ∗ .
Revised: And it contradicts with the definition of X ∗ .

Original: If c ′ y = 0 , then for some y ̸∈ Y ∗ ,
Revised: If c ′ y = 0 , then for some y ̸∈ Y ∗ ,

Original: [Equation]
Revised: [Equation]

Original: Let PS X denote all pure strategies of x .
Revised: Let PS X denote all pure strategies of x .

Original: If PS ( y ∗ ) = PS X , then V ∗ ( (cid:81)
Revised: If PS ( y ∗ ) = PS X , then V ∗ ( (cid:81)

Original: Y ∗ ( y ))
Revised: Y ∗ ( y ))

Original: =
Revised: =

Original: X .
Revised: X .

Original: Eq (D.24) implies that min x ∈X x ⊤ Ay
Revised: Eq (D.23) implies that min x ∈X x ⊤ Ay

Original: = ρ
Revised: = ρ

Original: so that y ∈ Y ∗ .
Revised: so that y ∈ Y ∗ .

Original: But this contradicts with the definition that y ̸∈ Y ∗ .
Revised: But this contradicts with the definition that y ̸∈ Y ∗ .

Original: If PS ( y ∗ ) ̸ = PS X , we define
Revised: If PS ( y ∗ ) ̸ = PS X , we define

Original: [Equation]
Revised: [Equation]

Original: And we can prove that ξ ( y ∗ ) ∈ (0 , 2 M ] .
Revised: And we can prove that ξ ( y ∗ ) ∈ (0 , 2 M ] .

Original: The lower bound is directly from Lemma D.2 and the upperbound is from the assumption on A that ∀ y ∈ Y , ∥
Revised: The lower bound is directly from Lemma D.3 and the upperbound is from the assumption on A that ∀ y ∈ Y , ∥

Original: Ay ∥ ∞ ≤ 1 .
Revised: Ay ∥ ∞ ≤ 1 .

Original: Let y ′
Revised: Let y ′

Original: = (cid:81)
Revised: = (cid:81)

Original: Y [Equation] + 
Revised: Y ∗ ( y ) + ξ ( (cid:81) Y∗ ( y ))

Original: [Equation] ( y − (cid:81)
Revised: 2 N · M ( y − (cid:81)

Original: Y ∗ ( y ))
Revised: Y ∗ ( y ))

Original: ∈ Y .
Revised: ∈ Y .

Original: For any pure strategy x ∈ PS X \ PS ( y ∗ ) , we have
Revised: For any pure strategy x ∈ PS X \ PS ( y ∗ ) , we have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality comes from the definition of ξ ( (cid:81) Y ∗ ( y )) in Eq (D.25).
Revised: where the last inequality comes from the definition of ξ ( (cid:81) Y ∗ ( y )) in Eq (D.24).

Original: For any pure strategy x ∈ PS ( y ∗ ) , we have
Revised: For any pure strategy x ∈ PS ( y ∗ ) , we have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Therefore, min x ∈X x ⊤ Ay ′ ≥ ρ since any x ∈ X is a linear combination of pure strategies.
Revised: Therefore, min x ∈X x ⊤ Ay ′ ≥ ρ since any x ∈ X is a linear combination of pure strategies.

Original: And it implies that y ′
Revised: And it implies that y ′

Original: ̸∈ Y ∗ is also a maximin point, contradicting with the definition of Y ∗ .
Revised: ̸∈ Y ∗ is also a maximin point, contradicting with the definition of Y ∗ .

Original: So, c ′ y > 0 and so does c ′ x .
Revised: So, c ′ y > 0 and so does c ′ x .

Original: And further we have that c x , c y > 0 .
Revised: And further we have that c x , c y > 0 .

Original: Lemma D.6.
Revised: Lemma D.7.

Original: For any t = 1 , 2 , ... , and i ∈ supp( Z ∗ ) , and η ≤ 18 P , Reg-DOMWU ensures that (cid:98) z t,i ≥ ϵ dil where ϵ dil is some game-dependent constant.
Revised: For any t = 1 , 2 , ... , and i ∈ supp( Z ∗ ) , and η ≤ 18 P , Reg-DOMWU ensures that (cid:98) z t,i ≥ ϵ dil where ϵ dil is some game-dependent constant.

Original: Proof.
Revised: Proof.

Original: By Lemma C.2, Reg-DOMD satisfies
Revised: By Lemma C.2, Reg-DOMD satisfies

Original: [Equation]
Revised: [Equation]

Original: Pick z
Revised: Pick z

Original: = z ∗ such that supp( z ∗ )
Revised: = z ∗ such that supp( z ∗ )

Original: =
Revised: =

Original: supp( Z ∗ ) (note that such a z ∗ ∈ Z ∗ must exist since Z ∗ is convex).
Revised: supp( Z ∗ ) (note that such a z ∗ ∈ Z ∗ must exist since Z ∗ is convex).

Original: Then, we have
Revised: Then, we have

Original: [Equation]
Revised: [Equation]

Original: where the first inequality comes from F ( z t )
Revised: where the first inequality comes from F ( z t )

Original: ⊤ ( z t − z ∗ ) =
Revised: ⊤ ( z t − z ∗ ) =

Original: x
Revised: x

Original: ⊤ t
Revised: ⊤ t

Original: Ay ∗ − x ∗⊤
Revised: Ay ∗ − x ∗⊤

Original: Ay t ≥ 0 by definition of
Revised: Ay t ≥ 0 by definition of

Original: NE.
Revised: NE.

Original: And it further implies that
Revised: And it further implies that

Original: [Equation]
Revised: [Equation]

Original:  when ητ ≤ η ≤ 38 .
Revised: (D.29) when ητ ≤ η ≤ 38 .

Original: When τ = 0 , we have
Revised: When τ = 0 , we have

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: And when τ > 0 , we have
Revised: And when τ > 0 , we have

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Therefore, for any i ∈ supp( Z ∗ ) = supp( z ∗ ) ,z ∗ i log
Revised: Therefore, for any i ∈ supp( Z ∗ ) = supp( z ∗ ) ,

Original: [Equation]
Revised: [Equation]

Revised: i log

Original: [Equation]
Revised: [Equation]

Original: where the last inequality comes from the fact that (cid:98) z 1 is initialized as a uniform strategy.
Revised: where the last inequality comes from the fact that (cid:98) z 1 is initialized as a uniform strategy

Revised: [Equation]

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: for any i ∈ supp( Z ∗ ) .
Revised: for any i ∈ supp( Z ∗ ) .

Original: And we further have
Revised: And we further have

Original: [Equation]
Revised: [Equation]

Original: completing the proof.
Revised: completing the proof.

Original: E P ROOF OF L EMMA 4. 
Revised: E P ROOF OF L EMMA 4.

Original: Our regret decomposition framework follows the laminar regret decomposition (Farina et al., 2019b), which is a more general case of the original counterfactual regret minimization (Zinkevich et al., 2007).
Revised: Our regret decomposition framework follows the laminar regret decomposition (Farina et al., 2019b), which is a more general case of the original counterfactual regret minimization (Zinkevich et al., 2007).

Original: The second part of Lemma 4.1, the boundedness of regret, also appears in (Farina et al., 2019b, Theorem 2).
Revised: The second part of Lemma 4.1, the boundedness of regret, also appears in (Farina et al., 2019b, Theorem 2).

Original: But here we use Lemma E.1 to prove it which is more concise.
Revised: But here we use Lemma E.1 to prove it which is more concise.

Original: Lemma E.1 (First part of Lemma 4.1) .
Revised: Lemma E.1 (First part of Lemma 4.1) .

Original: The difference satisfies that G Z T ( z )
Revised: The difference satisfies that G Z T ( z )

Original: = (cid:80) h ∈H Z z σ ( h ) G hT ( z ) for any z ∈ Z γ and γ ≥ 0 .
Revised: = (cid:80) h ∈H Z z σ ( h ) G hT ( z ) for any z ∈ Z γ and γ ≥ 0 .

Original: Proof.
Revised: Proof.

Original: We define the scalar subtree value S ht ( z ) recursively,
Revised: We define the scalar subtree value S ht ( z ) recursively,

Original: [Equation]
Revised: [Equation]

Original: For terminal nodes, H i will be empty set and thus Sht
Revised: For terminal nodes, H i will be empty set and thus S

Original: ( z ) = (cid:80) i ∈

Original: Ω

Original: h q i

Original: ( Ay t ) i + τα h ψ ∆ ( q h ) .

Original: By definition, for any z ∈ Z γ , we have

Original: G Z T ( z

Original: [Equation]
Revised: [Equation]

Original: T (cid:88) t

Original: [Equation]

Original: ( ⟨ F ( z t ) [Equation] z t ⟩ + τψ Z ( z t )) − T (cid:88) t
Revised: h ) .

Original: = ( ⟨ F ( z t ) , z ⟩ + τψ Z ( z )) = T (cid:88) t = (cid:88) h ∈H S [Equation]
Revised: By definition, for any z ∈ Z γ , we have

Original: [Equation]

Original: t )

Original: [Equation]
Revised: [Equation]

Original: T (cid:88) t = (cid:88) h ∈H S ht ( z ) = (cid:88) h ∈H(cid:16) T (cid:88){+where H 
+}
Revised: {+where H 
+}

Original: [Equation] ) where H 0 = { h : h ∈ H Z , σ ( h ) = 0 } is the set of information set at the root of treeplex.
Revised: = { h : h ∈ H Z , σ ( h ) = 0 } is the set of information set at the root of treeplex.

Original: Note that
Revised: Note that

Original: Z γ =
Revised: Z γ =

Original: Z γh 1 × Z γh 2 × ...
Revised: Z γh 1 × Z γh 2 × ...

Original: × Z h m where H 0 = { h 1 , h 2 , ..., h m } .
Revised: × Z h m where H 0 = { h 1 , h 2 , ..., h m } .

Original: Then, the inequality in the second line is simply by expanding the definition of S
Revised: Then, the inequality in the second line is simply by expanding the definition of S

Original: ht
Revised: ht

Original: ( z ) from the recursive manner.
Revised: ( z ) from the recursive manner.

Original: We further define G hT, sub := (cid:80) Tt =
Revised: We further define G hT, sub := (cid:80) Tt =

Original: 1 S ht ( z t ) − (cid:80) Tt =1 S ht
Revised: 1 S ht ( z t ) − (cid:80) Tt =1 S ht

Original: ( z ) .
Revised: ( z ) .

Original: Then,
Revised: Then,

Original: G hT, sub ( z ) =
Revised: G hT, sub ( z ) =

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: S

Original: [Equation]
Revised: [Equation]

Original: ( z t

Original: [Equation]

Original: −

Original: [Equation]

Original: (cid:88) t =

Original: [Equation]
Revised: [Equation]

Original: z

Original: [Equation]

Original: =
Revised: =

Original: T (cid:88) t =

Original: [Equation]
Revised: [Equation]

Original: t

Original: [Equation]

Original: [Equation]

Original: =
Revised: =

Original: T (cid:88) t =

Original: S ht

Original: ( z

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (cid:16)
Revised: (cid:16)

Original: T (cid:88) t

Original: = (cid:0) (cid:88) i ∈

Original: Ω

Original: h q i (cid:0)

Original: ( Ay

Original: [Equation]
Revised: [Equation]

Original:  i+ (cid:88)
Revised: h q i (cid:88)

Original: h ′ ∈H i S h ′
Revised: h ′ ∈H i −

Original: [Equation]

Original: ( z t )

Original: [Equation]

Original: +

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: − (cid:16) (cid:88) i ∈

Original: Ω h q i (cid:88)

Original: h ′ ∈H i − G h ′

Original: [Equation]

Original: [Equation] ) (cid:17) =G [Equation]
Revised: G h ′ sub ( z ) (cid:17) =

Original: [Equation]

Original:  (cid:88) i [Equation] h q
Revised: G hT ( q h ) + (cid:88) i ∈

Original: [Equation] (cid:88)
Revised: Ω h q i (cid:88)

Original: h [Equation] ∈H i
Revised: h ′ ∈H i

Original: G h ′ sub ( z ) (E.3) where ( i ) comes from (cid:80)
Revised: G h ′ sub ( z ) (E.3) where ( i ) comes from (cid:80)

Original: Tt =1 S h
Revised: Tt =1 S h

Original: ′ t ( z ) =
Revised: ′ t ( z ) =

Original: (cid:80) Tt =
Revised: (cid:80) Tt =

Original: 1 S h ′ t ( z t ) − G h
Revised: 1 S h ′ t ( z t ) − G h

Original: sub
Revised: sub

Original: ( z ) .
Revised: ( z ) .

Original: By applying it recursively, we will get for any z ∈ Z γ ,
Revised: By applying it recursively, we will get for any z ∈ Z γ ,

Original: [Equation]
Revised: G Z T ( z ) = (cid:88)

Original:  (E.4) which completes the proof.
Revised: h ∈H Z z σ ( h ) G hT ( q h ) , (E.4) which completes the proof.

Original: Lemma
Revised: Lemma

Original: E.
Revised: E.

Original: (Second part of
Revised: (Second part of

Original: Lemma 4.1) .
Revised: Lemma 4.1) .

Original: The regret satisfies that
Revised: The regret satisfies that

Original: RZ T ≤ max (cid:98) [Equation] ∈Z
Revised: R

Original: [Equation]
Revised: [Equation]

Original: (cid:80) h ∈H Z (cid:98) z σ ( h ) R hT for any γ ≥ 0 .
Revised: hT for any γ ≥ 0 .

Original: Proof.
Revised: Proof.

Original: By Lemma E.1, we have
Revised: By Lemma E.1, we have

Original: [Equation]
Revised: [Equation]

Original: which completes the proof.
Revised: which completes the proof.

Original: F P ROOF OF T HEOREM 4.3 AND T HEOREM 4. F. P ROOF OF L EMMA F. 
Revised: F P ROOF OF T HEOREM 4.3 AND T HEOREM 4. F. P ROOF OF L EMMA F.

Original: Lemma F.1.
Revised: Lemma F.1.

Original: For any information set h ∈ H Z , q h ∈ ∆ γ | Ω h | and τ ≤1 2 ∥ α ∥ ∞ , Reg-CFR guarantees G [Equation] +
Revised: For any information set h ∈ H Z , q h ∈ ∆ γ | Ω h | and τ ≤

Original: [Equation]
Revised: [Equation]

Original: t = (cid:16) ∥

Original: [Equation] [Equation] [Equation] λ ht − λ
Revised: -CFR guarantees

Original: [Equation]
Revised: [Equation]

Original: 8 ∥ q

Original: [Equation]

Original: [Equation]

Original: q t [Equation] [Equation] ∥
Revised: F.

Original: [Equation]
Revised: [Equation]

Original: (cid:17) .

Original: (F.1)

Original: Proof.
Revised: Proof.

Original: By Lemma 
Revised: By Lemma F.6,

Original: [Equation]
Revised: [Equation]

Original: By the strong convexity of ψ ∆ ,
Revised: By the strong convexity of ψ ∆ ,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Also,
Revised: Also,

Original: [Equation]
Revised: [Equation]

Original: where the second inequality is by Young’s inequality
Revised: where the second inequality is by Young’s inequality

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore, with τα h ≤ 12 ≤ λ ht − 1 2 ,

Original: [Equation]
Revised: [Equation]

Original: with τα h ≤ 12 ≤ λ ht − 1 2 ,
Revised: t [Equation] [Equation] +

Original: [Equation]
Revised: [Equation]

Revised: +

Revised: [Equation]

Revised: t =

Revised: [Equation]

Original: which completes the proof.
Revised: which completes the proof.

Original: For simplicity, we use constant M h as the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .
Revised: For simplicity, we use constant M h as the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .

Original: D ψ ∆ ( q h , q 1 ,h ) is upper-bounded since q 1 ,h is initialized as uniform distribution in ∆ | Ω h | .
Revised: D ψ ∆ ( q h , q 1 ,h ) is upper-bounded since q 1 ,h is initialized as uniform distribution in ∆ | Ω h | .

Original: F. P ROOF OF T HEOREM 4. 
Revised: F. P ROOF OF T HEOREM 4.

Original: By Lemma E.1, we have
Revised: By Lemma E.1, we have

Original: [Equation]
Revised: [Equation]

Original: where the first inequality is by definition of z γ, ∗ τ .
Revised: where the first inequality is by definition of z γ, ∗ τ .

Original: Now by Lemma F.1 taking q h = q γ, ∗ τ,h = z γ, ∗ τ,h z γ,∗ τ,σ ( h ) , 0 ≤ (cid:88)
Revised: Now by Lemma F.1 taking q h = q γ, ∗ τ,h = z γ, [Equation] τ,h z γ,

Original: [Equation]
Revised: [Equation]

Original:  where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .
Revised: τ, [Equation] [Equation] where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .

Original: D ψ ∆ ( q h , q 1 ,h ) is upper-bounded since q 1 ,h is initialized as uniform distribution in ∆ | Ω h | .
Revised: D ψ ∆ ( q h , q 1 ,h ) is upper-bounded since q 1 ,h is initialized as uniform distribution in ∆ | Ω h | .

Original: By rearranging the terms [Equation] we have
Revised: By rearranging the terms, we have

Original: [Equation]
Revised: [Equation]

Original: where ( i ) is by the expanded form of the (dilated) Bregman divergence D ψ Z (see Lemma F.8 for a detailed proof) and the constant C γ is defined by
Revised: where ( i ) is by the expanded form of the (dilated) Bregman divergence D ψ Z (see Lemma F.8 for a detailed proof) and the constant C γ is defined by

Original: [Equation]
Revised: [Equation]

Original: Non-perturbed EFG best-iterate convergence.
Revised: Non-perturbed EFG best-iterate convergence.

Original: To bound the quantity λ hT +1 M h+
Revised: To bound the quantity λ hT +1 M h

Original: [Equation]
Revised: [Equation]

Original: λ
Revised: λ

Original: [Equation]
Revised: [Equation]

Original: in C γ (other parts of C γ have been already bounded by constant), we introduce the following Lemma, whose proof is postponed to F.5.
Revised: in C γ (other parts of C γ have been already bounded by constant), we introduce the following Lemma, whose proof is postponed to F.5.

Original: Lemma F.2.
Revised: Lemma F.2.

Original: Consider update-rule in Eq (4.2).
Revised: Consider update-rule in Eq (4.2).

Original: For any h ∈ H Z , by taking κ = T 12 , Reg-CFR satisfies that
Revised: For any h ∈ H Z , by taking κ = T 12 , Reg-CFR satisfies that

Original: [Equation]
Revised: [Equation]

Original: where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .
Revised: where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .

Original: By Lemma F.2, we know that
Revised: By Lemma F.2, we know that

Original: C γ ≤ O ( T 14 ) , which is
Revised: C γ ≤ O ( T 14 ) , which is

Original: [Equation]
Revised: [Equation]

Original: Therefore, there exists t ′ ∈ { 2 , 3 , ..., T } ,
Revised: Therefore, there exists t ′ ∈ { 2 , 3 , ..., T } ,

Original: [Equation]
Revised: [Equation]

Original: So, z t ′ converges to z ∗ τ with convergence rate O ( T − 34 ) .
Revised: So, z t ′ converges to z ∗ τ with convergence rate O ( T − 3 4 ) .

Original: Perturbed EFG asymptotic last-iterate convergence.
Revised: Perturbed EFG asymptotic last-iterate convergence.

Original: From the form of constant C γ Eq (F.7) and λ ht − 1 ≥ κ ≥ 1 , we have
Revised: From the form of constant C γ Eq (F.7) and λ ht − 1 ≥ κ ≥ 1 , we have

Original: [Equation]
Revised: [Equation]

Original: where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .
Revised: where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .

Original: We will prove that C γ ≤ O (1) when γ > 0 .
Revised: We will prove that C γ ≤ O (1) when γ > 0 .

Original: By the Lipschitz property of V h ( z ) (see Lemma F. for a full proof), we have
Revised: By the Lipschitz property of V h ( z ) (see Lemma F. for a full proof), we have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is because
Revised: where the last inequality is because

Original: [Equation]
Revised: [Equation]

Original:  ≥ γ for any i by definition of γ -perturbed EFG so that z γ, ∗ τ,i ≥ γ P .
Revised: ) ≥ γ for any i by definition of γ -perturbed EFG so that [Equation] γ P .

Original: Since z γ, ∗ τ,σ ( h ) ≤ 1 , (cid:88)
Revised: Since z γ, ∗ τ,σ ( h ) ≤ 1 , (cid:88)

Original: [Equation]
Revised: [Equation]

Original: for any h ∈ H Z .
Revised: for any h ∈ H Z .

Original: Plugging inequality (F.13) to equation (F.11), we have
Revised: Plugging inequality (F.13) to equation (F.11), we have

Original: [Equation]
Revised: [Equation]

Original: As a result, it remains to bound the following two quantities in Eq (F.15) and Eq (F.16) separately by some constant:
Revised: As a result [Equation] it remains to bound the following two quantities in Eq (F.15) and Eq (F.16) separately by some constant:

Original: [Equation]
Revised: [Equation]

Original: where we use ι := γ P 16 P 2 L 22 for convenience.
Revised: where we use ι := γ P 16 P 2 L 22 for convenience.

Original: For Eq (F.15), since λ hT +1 = (cid:113) κ+ (cid:80)
Revised: For Eq (F.15) [Equation] since λ hT +1 = (cid:113) κ

Original:  Tt =

Revised: + (cid:80) Tt =

Original: δ ht where δ ht =
Revised: δ ht where δ ht =

Original: ∥
Revised: ∥

Original: V h ( z t + 12 ) − V h ( z t − 12 ) ∥ 2 , we can get
Revised: V h ( z t + 12 ) − V h ( z t − 12 ) ∥ 2 , we can get

Original: M h (cid:118)(cid:117)(cid:117)(cid:116) 
Revised: M h (cid:118)(cid:117)(cid:117)(cid:116) κ

Original: [Equation]
Revised: + T (cid:88) t = δ ht − ι T (cid:88) t = δ ht ≤ M h (cid:113) κ + δ h 1 + M h (cid:118)(cid:117)(cid:117)(cid:116)

Revised: T (cid:88) t =

Revised: δ ht − ι

Revised: T (cid:88) t = δ ht

Revised: = f h

Revised: ( (cid:118)(cid:117)(cid:117)(cid:116) T (cid:88) t

Original:  )  where the second inequality comes from √ a + b ≤ √ a + √ b and f is a quadratic function with negative coefficient on the quadratic term.
Revised: = δ ht ) (F.17) where the second inequality comes from √ a + b ≤ √ a + √ b and f is a quadratic function with negative coefficient on the quadratic term.

Original: Therefore, it is upper-bounded by a constant.
Revised: Therefore, it is upper-bounded by a constant.

Original: As for Eq (F.16), we discuss the two possible cases separately.
Revised: As for Eq (F.16), we discuss the two possible cases separately.

Original: When lim t →∞ λ ht < + ∞ , then (cid:80) ∞ t =1 ∥ V h ( z t + 12 ) − V h ( z t − 12 ) ∥ 2 < + ∞ from definition of λ ht so that Eq (F.16) is bounded by a constant.
Revised: When lim t →∞ λ ht < + ∞ , then (cid:80) ∞ t =1 ∥ V h ( z t + 12 ) − V h ( z t − 12 ) ∥ 2 < + ∞ from definition of λ ht so that Eq (F.16) is bounded by a constant.

Original: When lim t →∞ λ ht = + ∞ , then we must have t ′ =
Revised: When lim t →∞ λ ht = + ∞ , then we must have t ′ =

Original: min t { t : 1 /λ ht ≤ ι } .
Revised: min t { t : 1 /λ ht ≤ ι } .

Original: Therefore, Eq (F.16) is bounded by (cid:80) t ′ t =1 ∥ V h
Revised: Therefore, Eq (F.16) is bounded by (cid:80) t ′ t =1 ∥ V h

Original: ( z t + 12 )
Revised: ( z t + 12 )

Original: − V h
Revised: − V h

Original: ( z t − 12 ) ∥ 2 < + ∞ .
Revised: z t [Equation] 12 ) ∥ 2 < + ∞ .

Original: Therefore,
Revised: Therefore,

Original: T (cid:88)

Original: [Equation]
Revised: [Equation]

Original: (F.18) so that z t converges asymptotically to z γ, ∗ τ .
Revised: 18) so that z t converges asymptotically to z γ, ∗ τ .

Original: Proof of Corollary 4.4.
Revised: Proof of Corollary 4.4.

Original: By Lemma D.1, we know that when τ = ϵ 4 C B , we will getmax
Revised: By Lemma D.1, we know that when τ = [Equation] [Equation] B , we will get

Original: [Equation]
Revised: [Equation]

Original: (F.19)

Original: Using Theorem 4.3, the proof is done.
Revised: Using Theorem 4.3, the proof is done.

Original: F. P ROOF OF T HEOREM 4. 
Revised: F. P ROOF OF T HEOREM 4.

Original: We first state a stronger version of the folklore theorem here (Theorem 3 ; Farina et al., 2019b), to provide gurantees for average iterate below.
Revised: We first state a stronger version of the folklore theorem here (Theorem 3 ; Farina et al., 2019b), to provide gurantees for average iterate below.

Original: Lemma F.3.
Revised: Lemma F.3.

Original: For a EFG where l X t ( x t ) = x ⊤
Revised: For a EFG where l X t ( x t ) = x ⊤

Original: Ay t + τψ Z ( x ) , l Y t ( y ) =
Revised: Ay t + τψ Z ( x ) , l Y t ( y ) =

Original: − x ⊤ t
Revised: − x ⊤ t

Original: Ay + τψ Z ( y ) , the saddle point residual max (cid:98) z ∈Z F ( z ) ⊤ ( z − (cid:98) z ) + τψ ( z ) − τψ ( (cid:98) z ) of the average strategy ( 1 T(cid:80) Tt =1 x t , 1 T (cid:80)
Revised: Ay + τψ Z ( y ) , the saddle point residual max (cid:98) z ∈Z F ( z ) ⊤ ( z − (cid:98) z ) + τψ ( z ) − τψ ( (cid:98) z ) of the average strategy ( 1 T

Revised: [Equation]

Original: Tt =1  y t ) is bounded by R X + R Y T .
Revised: Tt =1 x t , 1 T (cid:80) Tt =1 y t ) is bounded by R X + R Y T .

Original: Non-perturbed EFG average-iterate convergence.
Revised: Non-perturbed EFG average-iterate convergence.

Original: From Lemma E.2 and Lemma F.1, by taking (cid:98) z = argmax z ∈Z (cid:80)h
Revised: From Lemma E.2 and Lemma F.1, by taking (cid:98) z = argmax z ∈Z (cid:80)

Original: [Equation]

Original:  Z z σ ( h ) R hT , we haveR Z
Revised: h ∈H Z z σ ( h ) R hT , we have

Original: [Equation]
Revised: [Equation]

Original: ≤ [Equation] where the last inequality is by Lemma F.2 and constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .
Revised: (F.20) where the last inequality is by Lemma F.2 and constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .

Original: Therefore, by Lemma F.3, the average iterate enjoys O ( T − 34 ) convergence rate in terms of duality gap.
Revised: Therefore, by Lemma F.3, the average iterate enjoys O ( T − 34 ) convergence rate in terms of duality gap.

Original: Perturbed EFG average-iterate convergence.
Revised: Perturbed EFG average-iterate convergence.

Original: By taking q h = [Equation]
Revised: By taking q h = (cid:98)

Original: z h (cid:98) z σ ( h ) where (cid:98) z = argmax 
Revised: z h (cid:98) z σ ( h ) where (cid:98) z = argmax z ∈Z γ

Original: [Equation]
Revised: [Equation]

Original: ∈Z

Original: [Equation]

Original:  Z z [Equation] ( h ) R h T , from Lemma F.1, we have 
Revised: h ∈H Z z σ ( h ) R hT , from Lemma F.1, we have R

Original: [Equation]
Revised: [Equation]

Original: where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .
Revised: where constant M h is the maximum value of D ψ ∆ ( q h , q 1 ,h ) in information set h .

Original: Follow the same analysis in F.2, we will get R Z T ≤ O (1) which means the duality gap converges with convergence rate O ( 1 T ) by Lemma F.3.
Revised: Follow the same analysis in F.2, we will get R Z T ≤ O (1) which means the duality gap converges with convergence rate O ( 1 T ) by Lemma F.3.

Original: F. A PPROXIMATE EXTENSIVE - FORM PERFECT EQUILIBRIA
Revised: F. A PPROXIMATE EXTENSIVE - FORM PERFECT EQUILIBRIA

Original: To illustrate why the NE of Z γ for some fixed γ > 0 is a good approximation to the extensive-form perfect equilibria, we propose the following lemma.
Revised: To illustrate why the NE of Z γ for some fixed γ > 0 is a good approximation to the extensive-form perfect equilibria, we propose the following lemma.

Original: Lemma F.4.
Revised: Lemma F.4.

Original: The approximate NE z γ of a γ -perturbed EFG is an approximation of the EFPE in terms of duality gap.
Revised: The approximate NE z γ of a γ -perturbed EFG is an approximation of the EFPE in terms of duality gap.

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: (F.22) where Z 0 is an infinitely small perturbed treeplex whose NE is exactly EFPE.
Revised: where Z 0 is an infinitely small perturbed treeplex whose NE is exactly EFPE.

Original: Proof.
Revised: Proof.

Original: For any z ∈ Z 0 , we can define z ′ ∈
Revised: For any z ∈ Z 0 , we can define z ′ ∈

Original: Z γ as
Revised: Z γ as

Original: [Equation]
Revised: [Equation]

Original: (F.23)

Original: Then, we will use induction to prove that ∥ z − z ′ ∥ ∞ ≤ γP .
Revised: Then, we will use induction to prove that ∥ z − z ′ ∥ ∞ ≤ γP .

Original: Note that we will use anc ( i ) :
Revised: Note that we will use anc ( i ) :

Original: = { i, σ ( h ( i )) , σ ( h ( σ ( h ( i )))) , ..., i ′ } where σ ( h ( i ′ ))
Revised: = { i, σ ( h ( i )) , σ ( h ( σ ( h ( i )))) , ..., i ′ } where σ ( h ( i ′ ))

Original: = 

Revised: =

Original: to denote the set of ancestors of index i in the treeplex.
Revised: to denote the set of ancestors of index i in the treeplex.

Original: Firstly, for index i which satisfies that σ ( h ( i ))
Revised: Firstly, for index i which satisfies that σ ( h ( i ))

Original: = 0 , we have| (cid:89)
Revised: = 0 , we have

Original: [Equation]
Revised: [Equation]

Revised: = | − γ | Ω h ( i ) | q i + γ |

Original: [Equation]
Revised: [Equation]

Original: Then, assume that we already prove that | (cid:81)
Revised: γ | [Equation]

Original: j
Revised: h ( i

Original: [Equation]
Revised: [Equation]

Original:  anc ( σ ( h ( i ))) (cid:16) (1 − γ | Ω h ( j ) | ) q j + γ (cid:17) − (cid:81) j ∈ anc ( σ ( h ( i )))
Revised: | . (F.24) Then, assume that we already prove that | (cid:81) j ∈ anc ( σ ( h ( i ))) (cid:16) (1 − γ | Ω h ( j ) | ) q j + γ (cid:17) − (cid:81) j ∈ anc ( σ ( h ( i )))

Original: q j | ≤ γC σ ( h ( i )) for an index
Revised: q j | ≤ γC σ ( h ( i )) for an index

Original: i where C σ ( h ( i ))
Revised: i where C σ ( h ( i ))

Original: = (cid:80) j ∈ anc ( σ ( h ( i )))
Revised: = (cid:80) j ∈ anc ( σ ( h ( i )))

Original: | Ω h ( j ) | , then (cid:89)
Revised: | Ω h ( j ) | , then (cid:89)

Original: [Equation]
Revised: [Equation]

Original: and similarly, we have the upperbound γ (1 + C σ ( h ( i )) ) .
Revised: and similarly, we have the upperbound γ (1 + C σ ( h ( i )) ) .

Original: Therefore, we have ∥ z − z ′
Revised: Therefore, we have ∥ z − z ′

Original: ∥ ∞ ≤ γP .
Revised: ∥ ∞ ≤ γP .

Original: Therefore, for y = argmax (cid:98) y ∈Y 0 x γ ⊤ A (cid:98) y where z γ = ( x γ , y γ ) is an approximate NE in a γperturbed EFG, we have
Revised: Therefore, for y = argmax (cid:98) y ∈Y 0 x γ ⊤ A (cid:98) y where z γ = ( x γ , y γ ) is an approximate NE in a γperturbed EFG, we have

Original: [Equation]
Revised: [Equation]

Original: which implies that 
Revised: which implies that max

Original: [Equation]
Revised: [Equation]

Original: where the last inequality comes from ∥ F ( z ) ∥ ∞ ≤ 1 for any z ∈ Z .
Revised: where the last inequality comes from ∥ F ( z ) ∥ ∞ ≤ 1 for any z ∈ Z .

Original: F. P ROPERTIES OF Reg-DS-OptMD (4.2)
Revised: F. P ROPERTIES OF Reg-DS-OptMD (4.2)

Original: We first prove some standard results in DS-OptMD (Hsieh et al., 2021) when adding regularization [Equation]
Revised: We first prove some standard results in DS-OptMD (Hsieh et al., 2021) when adding regularization.

Original: Lemma F.5.
Revised: Lemma F.5.

Original: For any convex set C and u 0 , u ∈ C [Equation] consider the update rule 
Revised: For any convex set C and u 0 , u ∈ C , consider the update rule u 1 = argmin

Original: [Equation]
Revised: [Equation]

Original: where ψ C is a strongly convex function in C .
Revised: where ψ C is a strongly convex function in C .

Original: Then for any u 2 ∈ C ,
Revised: Then for any u 2 ∈ C ,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Proof.
Revised: Proof.

Original: Since u 1 = argmin
Revised: Since u 1 = argmin

Original: [Equation]
Revised: [Equation]

Original: by first-order optimality condition,
Revised: by first-order optimality condition,

Original: [Equation]
Revised: [Equation]

Original: Notice that
Revised: Notice that

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Sum them up,
Revised: Sum them up,

Original: [Equation]
Revised: [Equation]

Original: where the last equation comes from Eq (F.28)
Revised: where the last equation comes from Eq (F.28)

Original: [Equation]
Revised: [Equation]

Original: Lemma F.6.
Revised: Lemma F.6.

Original: Consider the update rule Eq (4.2).
Revised: Consider the update rule Eq (4.2).

Original: For any information set h ∈ H Z , q h ∈ ∆ γ | Ω h | and t = 1 , 2 , ..., T , we have
Revised: For any information set h ∈ H Z , q h ∈ ∆ γ | Ω h | and t = 1 , 2 , ..., T , we have

Original: [Equation]
Revised: [Equation]

Revised: + 12 ,h − q h (cid:69) ≤ ( λ ht − τα h ) D ψ ∆ ( q h , q t,h )

Revised: − λ ht +1 D ψ ∆ ( q h , q t +1 ,h )

Revised: +

Revised: ( λ ht +1 − λ ht ) D ψ ∆ ( q h , q 1 ,h )

Revised: + (cid:68) V h ( z t + 12 ) − V h ( z t − 12 ) , q t + 12 ,h − q t +1 ,h (cid:69) − λ ht D ψ ∆ ( q t +1 ,h , q t + 12 ,h ) − ( λ h t − τα h ) D ψ ∆ ( q t + 12 ,h , q t,h ) .

Revised: (F.32)

Original: Proof.
Revised: Proof.

Original: Plug u 1 = q t + 1 2 ,h , u 2 = q t +1 ,h , g = V h
Revised: Plug u 1 = q [Equation] + [Equation]

Original: ( z t − 12 ) , ψ C = α h ψ ∆ into Lemma C.4,
Revised: [Equation] into Lemma C

Original: [Equation]
Revised: [Equation]

Original: F

Original: [Equation]
Revised: [Equation]

Original: Plug u 1 = 
Revised: Plug u 1 = q t +

Original: [Equation]
Revised: [Equation]

Original: t
Revised: u

Original: [Equation]
Revised: [Equation]

Revised: q

Original: [Equation]
Revised: [Equation]

Original: h , u
Revised: , g = V

Original: [Equation]
Revised: [Equation]

Original: = q h , g = V h
Revised: z t [Equation] [Equation]

Original: ( z t + 12 ) , λ 
Revised: λ [Equation]

Original: 1 = λ 
Revised: λ [Equation]

Original: ht , λ
Revised: λ

Original: 2 =
Revised: 2 =

Original: λ ht +1 , ψ C =
Revised: λ ht + [Equation]

Original:  α h ψ ∆ into Lemma F.5,
Revised: ψ C = α h ψ ∆ into Lemma F

Original: [Equation]
Revised: [Equation]

Original: t +

Original: (F.34)
Revised: [Equation]

Original: By summing Eq (F.33) and Eq (F.34) up, then adding (cid:68) Vh ( z t + 12 ) − V h ( z
Revised: By summing Eq (F.33) and Eq (F.34) up, then adding (cid:68) V

Original: [Equation]
Revised: [Equation]

Original: 12 ) , q [Equation] + [Equation]
Revised: −

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: on both sides,
Revised: on both sides,

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: By the two lemmas above
Revised: By the two lemmas above

Original: [Equation]
Revised: [Equation]

Original: we can prove that the update of Reg-DS-OptMD (4.2) is stable.
Revised: we can prove that the update of Reg-DS-OptMD (4.2) is stable.

Original: Lemma F.7 (Stability of Reg-DS-OptMD ) .
Revised: Lemma F.7 (Stability of Reg-DS-OptMD ) .

Original: For any t = 1 , 2 , ... , when ψ ∆ is Euclidean norm,
Revised: For any t = 1 , 2 , ... , when ψ ∆ is Euclidean norm,

Original: Reg-CFR satisfies that
Revised: Reg-CFR satisfies that

Original: [Equation]
Revised: [Equation]

Original: for some constant C 1 .
Revised: for some constant C 1 .

Original: Proof.
Revised: Proof.

Original: Consider the update rule Eq (4.2), by first-order optimality [Equation] for any h ∈ H Z , we have
Revised: Consider the update rule Eq (4.2), by first-order optimality, for any h ∈ H Z , we have

Original: [Equation]
Revised: [Equation]

Original: Add them up,
Revised: Add them up,

Original: [Equation]
Revised: [Equation]

Original: Since ψ ∆ 1-strong convex with respect to 2-norm, we have
Revised: Since ψ ∆ 1-strong convex with respect to 2-norm [Equation] we have

Original: [Equation]
Revised: [Equation]

Original: Add them up then we will get,
Revised: Add them up then we will get,

Original: [Equation]
Revised: [Equation]

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: ≤∥
Revised: ≤∥

Original: [Equation]
Revised: [Equation]

Original: And by definition,
Revised: And by definition,

Original: [Equation]
Revised: [Equation]

Original: so that
Revised: so that

Original: [Equation]
Revised: [Equation]

Original: which implies that
Revised: which implies that

Original: [Equation]
Revised: [Equation]

Original: since ∇ ψ ∆ is bounded by constant when ψ ∆ is Euclidean norm.
Revised: since ∇ ψ ∆ is bounded by constant when ψ ∆ is Euclidean norm.

Original: And ∥ V h ( z t − 12 ) − V h ( z t − 32 ) ∥ is also bounded by constant since both the regularizer and ∥ F ( z ) ∥ ∞ are bounded.
Revised: And ∥ V h ( z t − 12 ) − V h ( z t − 32 ) ∥ is also bounded by constant since both the regularizer and ∥ F ( z ) ∥ ∞ are bounded.

Original: At the same time, directly from update rule Eq (4.2),
Revised: At the same time, directly from update rule Eq (4.2),

Original: [Equation]
Revised: [Equation]

Original: which implies that
Revised: which implies that

Original: [Equation]
Revised: [Equation]

Original: Hence, we have
Revised: Hence, we have

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma F.2.
Revised: Proof of Lemma F

Revised: [Equation]

Original: By Lemma F.10,
Revised: By Lemma F.10,

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is by and Lemma F.7 and
Revised: where the last inequality is by and Lemma F.7 and

Original: [Equation]
Revised: [Equation]

Original: Then, by letting κ = T [Equation] we have
Revised: Then, by letting κ = T 12 , we have

Original: [Equation]
Revised: [Equation]

Original: which completes the proof
Revised: which completes the proof.

Original: [Equation]

Original: F. A UXILIARY LEMMAS FOR Reg-CFR
Revised: F. A UXILIARY LEMMAS FOR Reg-CFR

Original: In this section, we prove some auxiliary lemmas for Reg-CFR .
Revised: In this section, we prove some auxiliary lemmas for Reg-CFR .

Original: We begin with the expanding form of the Bregman divergence generated by the dilated Euclidean norm.
Revised: We begin with the expanding form of the Bregman divergence generated by the dilated Euclidean norm.

Original: Lemma F.8.
Revised: Lemma F.8.

Original: When ψ ∆ ( q ) = 12 (cid:80) i q 2 i , we have 
Revised: When ψ ∆ ( q ) = 12 (cid:80) i q 2 i , we have D

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Firstly [Equation] we can write ψ Z in the form
Revised: Firstly, we can write ψ Z in the form

Original: [Equation]
Revised: [Equation]

Original: then ∂ψ Z ( z ) ∂z i will be ∂ψ
Revised: then ∂ψ Z ( z ) ∂z i will be ∂ψ

Original: [Equation]
Revised: [Equation]

Original: z

Original: [Equation]
Revised: [Equation]

Original: where q i =
Revised: where q i =

Original: z i z σ ( h ( 
Revised: z i z σ ( h ( i

Original: [Equation]
Revised: [Equation]

Original: And by the definition of Bregman divergence
Revised: And by the definition of Bregman divergence

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: F.49)

Original: Notice that (cid:88)
Revised: Notice that (cid:88)

Revised: i

Original: [Equation]
Revised: [Equation]

Original: Similarly, we will get (cid:80) i ∈
Revised: Similarly, we will get (cid:80) i ∈

Original: Ω h z 2 ,i ( q 2 ,i − 2 q 2 ,
Revised: Ω h z 2 ,i ( q 2 ,i − 2 q 2 ,

Original: i + (cid:80)
Revised: i + (cid:80)

Original: k ∈
Revised: k ∈

Original: Ω h
Revised: Ω h

Original: ( i ) q 22 ,k )
Revised: ( i ) q 22 ,k )

Original: = 0 .
Revised: = 0 .

Original: Therefore, D ψ Z ( z 1 , z2 ) =
Revised: Therefore, D ψ Z ( z 1 , z

Original: (cid:80) h ∈H

Original: [Equation]
Revised: [Equation]

Original: 2 z 1 ,σ ( h ) ∥
Revised: =

Original: z 1 ,h z 1 ,σ ( h ) − z 2 ,h z 2 ,σ ( h ) ∥ 2 .
Revised: [Equation]

Original: Lipschitz continuity of V h ( z ) Here we will show that V h ( z ) is Lipschitz continuous with respect to q .
Revised: Lipschitz continuity of V h ( z ) Here we will show that V h ( z ) is Lipschitz continuous with respect to q .

Original: We first show that z is in a Lipschitz continuous manner with respect to q .
Revised: We first show that z is in a Lipschitz continuous manner with respect to q .

Original: Lemma F.9.
Revised: Lemma F.9.

Original: In γ − perturbed treeplex Z γ with γ ≥ 0 , for any z , z ′ ∈
Revised: In γ − perturbed treeplex Z γ with γ ≥ 0 , for any z , z ′ ∈

Original: Z γ , we have
Revised: Z γ , we have

Original: [Equation]
Revised: [Equation]

Original: for some game-dependent constant L 1 .
Revised: for some game-dependent constant L 1 .

Original: Proof.
Revised: Proof.

Original: We first consider the base case when Z γ is a γ -perturbed simplex, where Eq (F.51) is satisfied with L 1 = 1 since q = z .
Revised: We first consider the base case when Z γ is a γ -perturbed simplex, where Eq (F.51) is satisfied with L 1 = 1 since q = z .

Original: We consider the two basic operator, Cartesian product and branching, in the definition of treeplex (see Definition 2.1 for details).
Revised: We consider the two basic operator, Cartesian product and branching, in the definition of treeplex (see Definition 2.1 for details).

Original: We want to prove that both of them keep smoothness, that is, Eq (F.51) remains satisfied after applying the operation to multiple treeplexes where Eq (F.51) is satisfied.
Revised: We want to prove that both of them keep smoothness, that is, Eq (F.51) remains satisfied after applying the operation to multiple treeplexes where Eq (F.51) is satisfied.

Original: Firstly, for Cartesian product , if Eq (F.51) is satisfied for Z γ 1 , Z γ 2 , ..., Z γm , then for any z
Revised: Firstly, for Cartesian product , if Eq (F.51) is satisfied for Z γ 1 , Z γ 2 , ..., Z γm , then for any z

Original: = ( z 1 , z 2 , ..., z m ) , z ′ =
Revised: = ( z 1 , z 2 , ..., z m ) , z ′ =

Original: ( z ′ 1 , z ′ 2 , ..., z ′ m
Revised: ( z ′ 1 , z ′ 2 , ..., z ′ m

Original: ) ∈
Revised: ) ∈

Original: Z γ =
Revised: Z γ =

Original: Z γ 1 × Z γ 2 × ...
Revised: Z γ 1 × Z γ 2 × ...

Original: × Z γm , we have
Revised: × Z γm , we have

Original: [Equation]
Revised: [Equation]

Original: Notice that we abuse the notation H Z i and H Z here to denote the set of all information sets in Z γi and Z γ .
Revised: Notice that we abuse the notation H Z i and H Z here to denote the set of all information sets in Z γi and Z γ .

Original: To be convenient, let define the branching of m γ − perturbed treeplexes Z γ 1 , Z γ 2 ,..., Z γm and a γperturbed simplex ∆ γm be Z γ
Revised: To be convenient, let define the branching of m γ − perturbed treeplexes Z γ [Equation] , Z [Equation]

Revised: 2 , ..., Z γm and a γperturbed simplex ∆ γm be Z γ =

Original: = { ( p , p 1 z 1 , p 2 z 2 , ..., p m z m ) : p ∈ ∆ γm , z i ∈
Revised: { ( p , p 1 z 1 , p 2 z 2 , ..., p m z m ) : p ∈ ∆ γm , z i ∈

Original: Z γi } .
Revised: Z γi } .

Original: It’s easy to see that it is equivalent to using the original branching operator for m times in a bottom-up manner.
Revised: It’s easy to see that it is equivalent to using the original branching operator for m times in a bottom-up manner.

Original: Suppose for any z i , z ′ i
Revised: Suppose for any z i , z ′ i

Original: ∈
Revised: ∈

Original: Z γi ,∥
Revised: Z γi ,

Original: z i − z i ∥ ≤ L i (cid:80) h ∈H Z i ∥ q i,h − q ′
Revised: [Equation]

Original: i,h ∥ . For z = ( p , p 1 z 1 , p 2 z 2 + ..., p m z m ) , z ′ = ( p ′ , p ′ 1 z ′ 1 , p ′ 2 z 
Revised: z i

Original: [Equation]
Revised: [Equation]

Original: 2 , ..., p
Revised: z

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: And we have
Revised: And we have

Original: [Equation]
Revised: [Equation]

Original: where the fourth inequality is by Z γi ⊂ R |Z γi | .
Revised: where the fourth inequality is by Z γi ⊂ R |Z γ i | .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: where the third line is by the inductive assumption and the fourth line is by definition of H Z and q .
Revised: where the third line is by the inductive assumption and the fourth line is by definition of H Z and q .

Original: Therefore, recursively applying Eq (F.52) and Eq (F.55), we will have for
Revised: Therefore, recursively applying Eq (F.52) and Eq (F.55), we will have for

Original: any z , z ′ ∈ Z γ
Revised: any z , z ′ ∈ Z γ

Original: [Equation]
Revised: [Equation]

Revised: (F.

Revised: [Equation]

Original: where we take L 1 = P + 1 .
Revised: where we take L 1 = P + 1 .

Original: Finally, since both operator keeps the smoothness, by induction, we know that for any treeplex Eq (F.51) is satisfied.
Revised: Finally, since both operator keeps the smoothness, by induction, we know that for any treeplex Eq (F.51) is satisfied.

Original: Now we can prove that V h ( z ) is Lipschitz continuous with respect to q .
Revised: Now we can prove that V h ( z ) is Lipschitz continuous with respect to q .

Original: Lemma F.10.
Revised: Lemma F.10.

Original: When ψ ∆ is L p -Lipschitz continuous, that is,
Revised: When ψ ∆ is L p -Lipschitz continuous, that is,

Original: | ψ ∆ ( x ) − ψ ∆ ( x ′ )
Revised: | ψ ∆ ( x ) − ψ ∆ ( x ′ )

Original: | ≤ L p ∥ x − x ′ ∥ for any x , x ′ ∈ ∆ γ , and | ψ ∆ ( x ) | is upper-bounded by a constant C ∆ B for any x ∈ ∆ γ , then for any z ,
Revised: | ≤ L p ∥ x − x ′ ∥ for any x , x ′ ∈ ∆ γ , and | ψ ∆ ( x ) | is upper-bounded by a constant C ∆ B for any x ∈ ∆ γ , then for any z ,

Original: z
Revised: z

Original: ′ ∈ Z γ and h ∈ H , we have
Revised: ′ ∈ Z γ and h ∈ H , we have

Original: [Equation]
Revised: [Equation]

Original: where L 2 is a game-dependent constant.
Revised: where L 2 is a game-dependent constant.

Original: Proof.
Revised: Proof.

Original: Here we consider h ∈ H X , and h ∈ H Y can be addressed similarly.
Revised: Here we consider h ∈ H X , and h ∈ H Y can be addressed similarly.

Original: [Equation]
Revised: [Equation]

Original: where the second inequality is because each entry of A is in [ − 1 , 1] and the last inequality is by ∥ y − y ′
Revised: where the second inequality is because each entry of A is in [ − 1 , 1] and the last inequality is by [Equation] ≤ [Equation]

Revised: +

Original: ∥ 1 ≤ √ P ∥ y − y ′ ∥.
Revised: [Equation] 1 + τ [Equation] ∥

Revised: [Equation]

Revised: ∥

Revised: [Equation]

Revised: q

Original: [Equation]
Revised: [Equation]

Revised: q

Revised: [Equation]

Revised: ∥

Revised: [Equation]

Revised: [Equation]

Original: Here ( i ) is by H ¨ older’s inequality.
Revised: Here ( i ) is by H¨older’s inequality.

Original: ( ii ) is by ∥ q h ∥ 1 = 1 and ∥
Revised: ( ii ) is by ∥ q h ∥ 1 = 1 and ∥

Original: V h ( z ) ∥ ≤ ∥
Revised: V h ( z ) ∥ ≤ ∥

Original: Ay ∥ 1 + Pτ ∥ α ∥ ∞ C ∆ B ≤
Revised: Ay ∥ 1 + Pτ ∥ α ∥ ∞ C ∆ B ≤

Original: P (1 + τ ∥ α ∥ ∞ C ∆ B ) .
Revised: P (1 + τ ∥ α ∥ ∞ C ∆ B ) .

Original: By recursively applying this inequality, we have| W [Equation] z [Equation]
Revised: By recursively applying this inequality, we have

Original: − W

Original: [Equation]

Original: z

Original: [Equation]
Revised: [Equation]

Original: Notice that
Revised: Notice that

Original: [Equation]
Revised: [Equation]

Original: where the first inequality is because A ∈ [ − 1 , 1] M × N and the third inequality is by Lemma F.9.
Revised: where the first inequality is because A ∈ [ − 1 , 1] M × N and the third inequality is by Lemma F.9.

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: 62) where L 3 = P 2 L 1 + P (1 + τ ∥ α ∥ ∞ C∆ B )
Revised: ) where L 3 = P 2 L 1 + P (1 + τ ∥ [Equation] C

Revised: [Equation]

Original: + τ ∥ α ∥ ∞
Revised: + τ ∥ α ∥ ∞

Original: L p .
Revised: L p .

Original: And back to Eq (F.58), we have
Revised: And back to Eq (F.58), we have

Original: [Equation]
Revised: [Equation]

Original: where the second inequality comes from Lemma F.9.
Revised: where the second inequality comes from Lemma F.9.

Original: G C ONCLUSIONS AND F UTURE W ORK
Revised: G C ONCLUSIONS AND F UTURE W ORK

Original: In this paper, we investigate the regularization technique, a widely used one in reinforcement learning and optimization, in solving EFGs.
Revised: In this paper, we investigate the regularization technique, a widely used one in reinforcement learning and optimization, in solving EFGs.

Original: Firstly, we prove that Reg-DOMD can achieve the first result of last-iterate convergence rate to  NE without the unique NE assumption, for dilated OMD algorithms withBregman distance associated with entropy, and with constant stepsizes, in terms of both duality gap and the distance to the set of NE.
Revised: Firstly, we prove that Reg-DOMD can achieve the first result of last-iterate convergence rate to the NE without the unique NE assumption, for dilated OMD-type algorithms with constant stepsizes, in terms of both duality gap and the distance to the set of NE.

Original: An interesting future direction is whether we can extend the iterate-convergence result to other Bregman distance such as Euclidean distance.

Original: We further prove that by solving the regularized problem, CFR with Reg-DS-OptMD as regret minimizer, which we called Reg-CFR , can achieve best-iterate convergence result in finding NEs and asymptotic last-iterate convergence in finding approximate extensive-form perfect equilibria.
Revised: We further prove that by solving the regularized problem, CFR with Reg-DS-OptMD as regret minimizer, which we called Reg-CFR , can achieve best-iterate convergence result in finding NEs and asymptotic last-iterate convergence in finding approximate extensive-form perfect equilibria.

Original: These results constitute the first last-iterate convergence results for CFR-type algorithms.
Revised: These results constitute the first last-iterate convergence results for CFR-type algorithms.

Original: Furthermore, we have shown empirically that for CFR and CFR+, solving the regularized problem can achieve better last-iterate performance, further demonstrating the power of regularization in solving EFGs.
Revised: Furthermore, we have shown empirically that for CFR and CFR+, solving the regularized problem can achieve better last-iterate performance, further demonstrating the power of regularization in solving EFGs.

Original: We leave it for future work to study its explicit convergence rate.
Revised: We leave it for future work to study its explicit convergence rate.