Original: F
Revised: F

Original: ASTER N O -R EGRET L EARNING D YNAMICS FOR
Revised: ASTER N O -R EGRET L EARNING D YNAMICS FOR

Original: E XTENSIVE -F ORM C ORRELATED E QUILIBRIUM
Revised: E XTENSIVE -F ORM C ORRELATED E QUILIBRIUM

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: A recent emerging trend in the literature on learning in games has been concerned with providing accelerated learning dynamics for correlated and coarse correlated equilibria in normal-form games.
Revised: A recent emerging trend in the literature on learning in games has been concerned with providing accelerated learning dynamics for correlated and coarse correlated equilibria in normal-form games.

Original: Much less is known about the signiﬁcantly more challenging setting of extensive-form games, which can capture sequential and simultaneous moves, as well as imperfect information.
Revised: Much less is known about the signiﬁcantly more challenging setting of extensive-form games, which can capture sequential and simultaneous moves, as well as imperfect information.

Original: In this paper, we develop faster no-regret learning dynamics for extensive-form correlated equilibrium (EFCE) in multiplayer general-sum imperfect-information extensive-form games.
Revised: In this paper, we develop faster no-regret learning dynamics for extensive-form correlated equilibrium (EFCE) in multiplayer general-sum imperfect-information extensive-form games.

Original: When all agents play T repetitions of the game according to the accelerated dynamics, the correlated distribution of play is an O ( T  3 / 4 ) -approximate EFCE.
Revised: When all agents play T repetitions of the game according to the accelerated dynamics, the correlated distribution of play is an O ( T  3 / 4 ) -approximate EFCE.

Original: This signiﬁcantly improves over the best prior rate of O ( T  1 / 2 ) .
Revised: This signiﬁcantly improves over the best prior rate of O ( T  1 / 2 ) .

Original: One of our conceptual contributions is to connect predictive (that is, optimistic) regret minimization with the framework of  -regret.
Revised: One of our conceptual contributions is to connect predictive (that is, optimistic) regret minimization with the framework of  -regret.

Original: One of our main technical contributions is to characterize the stability of certain ﬁxed point strategies through a reﬁned perturbation analysis of a structured Markov chain, which may be of independent interest.
Revised: One of our main technical contributions is to characterize the stability of certain ﬁxed point strategies through a reﬁned perturbation analysis of a structured Markov chain, which may be of independent interest.

Original: Finally, experiments on standard benchmarks corroborate our ﬁndings.
Revised: Finally, experiments on standard benchmarks corroborate our ﬁndings.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Game-theoretic solution concepts describe how agents should rationally act in games.
Revised: Game-theoretic solution concepts describe how agents should rationally act in games.

Original: Over the last two decades there has been tremendous progress in imperfect-information game solving and algorithms based on game-theoretic solution concepts have become the state of the art.
Revised: Over the last two decades there has been tremendous progress in imperfect-information game solving and algorithms based on game-theoretic solution concepts have become the state of the art.

Original: Prominent milestones of this were an optimal strategy for Rhode Island hold’em poker (Gilpin & Sandholm, 2007), a near-optimal strategy for limit Texas hold’em
Revised: Prominent milestones of this were an optimal strategy for Rhode Island hold’em poker (Gilpin & Sandholm, 2007), a near-optimal strategy for limit Texas hold’em

Original: (Bowling et al., 2015), and a superhuman strategy for no-limit Texas hold’em (Brown & Sandholm, 2017).
Revised: (Bowling et al., 2015), and a superhuman strategy for no-limit Texas hold’em (Brown & Sandholm, 2017).

Original: In particular, these advances rely on algorithms that approximate Nash equilibria ( NE ) of two-player zero-sum extensive-form games ( EFGs ).
Revised: In particular, these advances rely on algorithms that approximate Nash equilibria ( NE ) of two-player zero-sum extensive-form games ( EFGs ).

Original: EFGs are a broad class of games that capture sequential and simultaneous interaction, and imperfect information.
Revised: EFGs are a broad class of games that capture sequential and simultaneous interaction, and imperfect information.

Original: For two-player zero-sum EFGs, it is by now well-understood how to compute a Nash equilibrium at scale: in theory this can be achieved using accelerated uncoupled noregret learning dynamics, for example by having each player use an optimistic regret minimizer and leveraging suitable distance-generating functions (Hoda et al., 2010; Kroer et al., 2020; Farina et al., 2021c) for the EFG decision space.
Revised: For two-player zero-sum EFGs, it is by now well-understood how to compute a Nash equilibrium at scale: in theory this can be achieved using accelerated uncoupled noregret learning dynamics, for example by having each player use an optimistic regret minimizer and leveraging suitable distance-generating functions (Hoda et al., 2010; Kroer et al., 2020; Farina et al., 2021c) for the EFG decision space.

Original: Such a setup converges to an equilibrium at a rate of O ( T  1 ) .
Revised: Such a setup converges to an equilibrium at a rate of O ( T  1 ) .

Original: In practice, modern variants of the counterfactual regret minimization ( CFR ) framework typically lead to better performance, although the worst-case convergence rate is O ( T  1 / 2 ) (Zinkevich et al., 2007).
Revised: In practice, modern variants of the counterfactual regret minimization ( CFR ) framework typically lead to better performance, although the worst-case convergence rate is O ( T  1 / 2 ) (Zinkevich et al., 2007).

Original: CFR is also an uncoupled no-regret learning dynamic.
Revised: CFR is also an uncoupled no-regret learning dynamic.

Original: However, many real-world applications are not two-player zero-sum games, but instead have general-sum utilities and often more than two players.
Revised: However, many real-world applications are not two-player zero-sum games, but instead have general-sum utilities and often more than two players.

Original: In such settings, Nash equilibrium suffers from several drawbacks when used as a prescriptive tool.
Revised: In such settings, Nash equilibrium suffers from several drawbacks when used as a prescriptive tool.

Original: First, there can be multiple equilibria, and an equilibrium strategy may perform very poorly when played against the “wrong” equilibrium strategies of the other player(s).
Revised: First, there can be multiple equilibria, and an equilibrium strategy may perform very poorly when played against the “wrong” equilibrium strategies of the other player(s).

Original: Thus, the players effectively would need to communicate in order to ﬁnd an equilibrium, or hope to converge to it via some sort of learning dynamics.
Revised: Thus, the players effectively would need to communicate in order to ﬁnd an equilibrium, or hope to converge to it via some sort of learning dynamics.

Original: Second, ﬁnding a Nash equilibrium is computationally hard both in theory (Daskalakis et al., 2006; Etessami &
Revised: Second, ﬁnding a Nash equilibrium is computationally hard both in theory (Daskalakis et al., 2006; Etessami &

Original: Yannakakis, 2007) and in practice (Berg & Sandholm, 2017).
Revised: Yannakakis, 2007) and in practice (Berg & Sandholm, 2017).

Original: This effectively squashes any hope of developing efﬁcient learning dynamics that converge to general-sum Nash equilibria.
Revised: This effectively squashes any hope of developing efﬁcient learning dynamics that converge to general-sum Nash equilibria.

Original: A competing notion of rationality proposed by Aumann (1974) is that of correlated equilibrium ( CE ), typically modeled via a trusted mediator who privately recommends actions to the players.
Revised: A competing notion of rationality proposed by Aumann (1974) is that of correlated equilibrium ( CE ), typically modeled via a trusted mediator who privately recommends actions to the players.

Original: Unlike NE, it is known that the latter can be computed in polynomial time and, perhaps even more importantly, it can be attained through uncoupled learning dynamics, where the players only need to reason about their own observed utilities.
Revised: Unlike NE, it is known that the latter can be computed in polynomial time and, perhaps even more importantly, it can be attained through uncoupled learning dynamics, where the players only need to reason about their own observed utilities.

Original: This overcomes the often unreasonable presumption that players have knowledge about the other players’ utilities.
Revised: This overcomes the often unreasonable presumption that players have knowledge about the other players’ utilities.

Original: At the same time, uncoupled learning algorithms have proven to be a remarkably scalable approach for computing equilibria in large-scale games, as described above.
Revised: At the same time, uncoupled learning algorithms have proven to be a remarkably scalable approach for computing equilibria in large-scale games, as described above.

Original: The basic CE notion is deﬁned for normal-form games, and there it has long been known that uncoupled no-regret learning dynamics can converge to CE or the coarse correlated equilibrium ( CCE ) variant at a rate of O ( T  1 / 2 )
Revised: The basic CE notion is deﬁned for normal-form games, and there it has long been known that uncoupled no-regret learning dynamics can converge to CE or the coarse correlated equilibrium ( CCE ) variant at a rate of O ( T  1 / 2 )

Original: (Hart & Mas-Colell, 2000; Celli et al., 2019).
Revised: (Hart & Mas-Colell, 2000; Celli et al., 2019).

Original: More recently, it was shown that accelerated uncoupled no-regret learning dynamics can compute CCE and CE at a rate of O ( T  3 / 4 ) (Syrgkanis et al., 2015; Chen & Peng, 2020).
Revised: More recently, it was shown that accelerated uncoupled no-regret learning dynamics can compute CCE and CE at a rate of O ( T  3 / 4 ) (Syrgkanis et al., 2015; Chen & Peng, 2020).

Original: In the context of EFGs, the idea of correlation is much more intricate, and there are several notions of correlated equilibrium, based on when the mediator gives recommendations and how the mediator reacts to players who disregard the advice.
Revised: In the context of EFGs, the idea of correlation is much more intricate, and there are several notions of correlated equilibrium, based on when the mediator gives recommendations and how the mediator reacts to players who disregard the advice.

Original: One of the most compelling notions for EFGs is the extensive-form correlated equilibrium (henceforth EFCE) (von Stengel & Forges, 2008) for extensive-form games with perfect recall .
Revised: One of the most compelling notions for EFGs is the extensive-form correlated equilibrium (henceforth EFCE) (von Stengel & Forges, 2008) for extensive-form games with perfect recall .

Original: In an EFCE, the mediator forms recommendations for each of the possible decision points an agent may encounter in the game, and recommended actions are gradually revealed to players as they reach new information sets; thus, the mediator must take into account the evolution of the players’ beliefs throughout the game.

Original: Because of the sequential nature, the presence of private information in the game, and the gradual revelation of recommendations, the constraints associated with EFCE are signiﬁcantly more complex than for normal-form games.
Revised: Because of the sequential nature, the presence of private information in the game, and the gradual revelation of recommendations, the constraints associated with EFCE are signiﬁcantly more complex than for normal-form games.

Original: For these reasons, the question of whether uncoupled learning dynamics can converge to an EFCE was only very recently resolved by Celli et al.
Revised: For these reasons, the question of whether uncoupled learning dynamics can converge to an EFCE was only very recently resolved by Celli et al.

Original: Very recent follow-up work  established an explicit rate of convergence of O ( T  1 / 2 ) (Farina et al., 2021a).
Revised: Moreover, in a follow-up work they also established an explicit rate of convergence of O ( T  1 / 2 ) (Farina et al., 2021a).

Original: Our paper is concerned with the following fundamental question: Can one develop faster uncoupled no-regret learning dynamics for 
Revised: Our paper is concerned with the following fundamental question: Can one develop faster uncoupled no-regret learning dynamics for EFCE ?

Original: EFCE ?
Revised: Contributions.

Original: We answer this  in the positive:
Revised: Our primary contribution is to answer this question in the positive:

Original: Theorem 1.1.
Revised: Theorem 1.1.

Original: On any ﬁnite perfect-recall general-sum multiplayer extensive-form game, the uncoupled no-regret learning dynamics described in this paper lead to a correlated distribution of play that is an O ( T  3 / 4 ) -approximate EFCE , where the O ( · ) notation suppresses game-speciﬁc parameters polynomial in the size of the game.
Revised: On any ﬁnite perfect-recall general-sum multiplayer extensive-form game, the uncoupled no-regret learning dynamics described in this paper lead to a correlated distribution of play that is an O ( T  3 / 4 ) -approximate EFCE , where the O ( · ) notation suppresses game-speciﬁc parameters polynomial in the size of the game.

Original: We achieve this result using the framework of predictive (also known as optimistic ) regret minimization (Chiang et al., 2012; Rakhlin & Sridharan, 2013b).
Revised: We achieve this result using the framework of predictive (also known as optimistic ) regret minimization (Chiang et al., 2012; Rakhlin & Sridharan, 2013b).

Original: One of our conceptual contributions is to connect this line of work with the framework of  - regret minimization of Greenwald & Jafari (2003); Gordon et al.
Revised: One of our conceptual contributions is to connect this line of work with the framework of  - regret minimization of Greenwald & Jafari (2003); Gordon et al.

Original: (2008), by providing a general template for stable-predictive  -regret minimization.
Revised: (2008), by providing a general template for stable-predictive  -regret minimization.

Original: The importance of  -regret is that it leads to substantially more powerful notions of hindsight rationality, beyond the usual external regret. These include the powerful notion of swap regret (Blum & Mansour, 2007).
Revised: The importance of  -regret is that it leads to substantially more powerful notions of hindsight rationality, beyond the usual external regret (Gordon et al., 2008), including the powerful notion of swap regret (Blum & Mansour, 2007).

Original: One of the primary insights behind the result of Farina et al.
Revised: Moreover, one of the primary insights behind the result of Farina et al.

Original: (2021a) is to cast convergence to an EFCE as a  -regret minimization problem.
Revised: (2021a) is to cast convergence to an EFCE as a  -regret minimization problem.

Original: Given these prior connections, we believe that our stable-predictive  template is of independent interest, and could lead to further applications in the future.
Revised: Given these prior connections, we believe that our stable-predictive  template is of independent interest, and could lead to further applications in the future.

Original: Theorem 1.1 extends and strengthens several prior papers in the literature, including the seminal work of Syrgkanis et al.
Revised: Theorem 1.1 extends and strengthens several prior papers in the literature, including the seminal work of Syrgkanis et al.

Original: (2015) that provides accelerated dynamics for coarse correlated equilibrium in normal-form games, as well as the more recent result of Chen & Peng (2020) which showed
Revised: (2015) that provides accelerated dynamics for coarse correlated equilibrium in normal-form games, as well as the more recent result of Chen & Peng (2020) which showed

Original: O ( T  3 / 4 ) convergence to a correlated equilibrium in normal-form games.
Revised: O ( T  3 / 4 ) convergence to a correlated equilibrium in normal-form games.

Original: For the more challenging class of extensive-form games, accelerated rates were previously known only for ﬁnding a Nash equilibrium in the special case of two-player zero-sum games, where an O ( T  3 / 4 ) rate was achieved via a stable-predictive CFR setup (Farina et al., 2019a) and an O ( T  1 ) rate was achieved via optimistic regret minimizers coupled with good distance-generating functions (Farina et al., 2019c).
Revised: For the more challenging class of extensive-form games, accelerated rates were previously known only for ﬁnding a Nash equilibrium in the special case of two-player zero-sum games, where an O ( T  3 / 4 ) rate was achieved via a stable-predictive CFR setup (Farina et al., 2019a) and an O ( T  1 ) rate was achieved via optimistic regret minimizers coupled with good distance-generating functions (Farina et al., 2019c).

Original: The rate of convergence of our dynamics depends polynomially on the size of  the game.
Revised: From a technical standpoint, in order to apply our generic template for accelerated  -regret minimization, we establish two separate ingredients. First, we develop a stable-predictive external regret minimizer for the set of transformations  associated with EFCE. This differs from the construction by Farina et al. (2021a) in that we have to additionally guarantee and preserve the stability—and subsequently the predictivity—throughout the construction.

Original: To achieve this, one of the primary challenges consists of sharply characterizing the stability of ﬁxed points related to trigger deviation functions .
Revised: The second component consists of sharply characterizing the stability of ﬁxed points of trigger deviation functions .

Original: This turns out to be particularly challenging, and direct extensions of prior techniques appear togive only  a bound that is exponential in the size of the game.
Revised: This turns out to be particularly challenging, and direct extensions of prior techniques appear to only give a bound that is exponential in the size of the game.

Original: In this context, one of our key technical contributions is to provide a reﬁned perturbation analysis for a Markov chain consisting of a rank-one stochastic matrix, employing tools that have not been used before in this line of work, and substantially extending the techniques of Chen & Peng (2020).
Revised: In this context, one of our key technical contributions is to provide a reﬁned perturbation analysis for a Markov chain consisting of a rank-one stochastic matrix, employing tools that have not been used before in this line of work, and substantially extending the techniques of Chen & Peng (2020).

Revised: This leads to a rate of convergence that depends polynomially on the description of the game, which is crucial for the applicability of the accelerated dynamics.

Original: Finally, we support our theoretical ﬁndings with experiments on several general-sum benchmarks.
Revised: Finally, we support our theoretical ﬁndings with experiments on several general-sum benchmarks.

Original: Further Related Work.
Revised: Further Related Work.

Original: The line of work on accelerated no-regret learning for Nash equilibrium was pioneered by Daskalakis et al. (2015), showing that one can bypass the adversarial ⌦ ( T  1 / 2 ) barrier for the incurred average regret if both players in a zero-sum game employ an uncoupled variant of the excessive gap technique (Nesterov, 2005), leading to a near-optimal rate of O (log T/T ) .
Revised: The line of work on accelerated no-regret learning for Nash equilibrium was pioneered by Daskalakis et al. (2015), showing that one can bypass the adversarial ⌦ ( T  1 / 2 ) barrier for the incurred average regret if both players in a zero-sum game employ an uncoupled variant of the excessive gap technique (Nesterov, 2005), leading to a near-optimal rate of O (log T/T ) .

Original: Subsequently, Rakhlin & Sridharan (2013a) showed that the optimal rate of O (1 /T ) can be obtained with a remarkably simple variant of Online Mirror Descent which incorporates a prediction term in the update step.
Revised: Subsequently, Rakhlin & Sridharan (2013a) showed that the optimal rate of O (1 /T ) can be obtained with a remarkably simple variant of Online Mirror Descent which incorporates a prediction term in the update step.

Original: While these results only hold for zero-sum games, Syrgkanis et al.
Revised: While these results only hold for zero-sum games, Syrgkanis et al.

Original: (2015) showed that O ( T  3 / 4 ) rate can be obtained for multiplayer general-sum normal-form games.
Revised: (2015) showed that O ( T  3 / 4 ) rate can be obtained for multiplayer general-sum normal-form games.

Original: In a recent result, Chen & Peng (2020) strengthened the regret bounds of Syrgkanis et al.
Revised: In a recent result, Chen & Peng (2020) strengthened the regret bounds of Syrgkanis et al.

Original: (2015) from external to swap regret using the celebrated construction of Blum & Mansour (2007).
Revised: (2015) from external to swap regret using the celebrated construction of Blum & Mansour (2007).

Revised: We also acknowledge a recent result of Daskalakis et al.

Original: While the latter involves strategies arising as stationary distributions of a Markov chain, which in general may behave highly nonsmoothly, one of the key observations of Chen & Peng (2020) is that updating each individual regret minimizer via Optimistic Multiplicative Weights  (OMWU) sufﬁces to guarantee stability.
Revised: (2021) which establishes a near-optimal rate of convergence of e O (1 /T ) to a coarse correlated equilibrium when all players employ the Optimistic Multiplicative Weights Update (OMWU) algorithm in a normal-form game. Extending their result to extensiveform games presents considerable technical challenges since their analysis crucially hinges on the closed-form softmax-type structure of OMWU on the simplex.

Original: Correlated equilibrium in extensive-form games is much less understood than Nash equilibrium, although there has been substantial recent progress.
Revised: Correlated equilibrium in extensive-form games is much less understood than Nash equilibrium.

Original: For EFCE we already discussed Celli et al.

Original: (2020) and Farina et al. (2021a). A feasible EFCE can also be computed efﬁciently through a variant of the Ellipsoid algorithm (Papadimitriou & Roughgarden, 2008; Jiang & Leyton-Brown, 2015), and an alternative sampling-based approach was given by Dudík & Gordon (2009).
Revised: A feasible EFCE can also be computed efﬁciently through a variant of the Ellipsoid algorithm (Papadimitriou & Roughgarden, 2008; Jiang & Leyton-Brown, 2015), and an alternative samplingbased approach was given by Dudík & Gordon (2009).

Original: However, those approaches perform poorly in large-scale problems, and do not allow the players to arrive at EFCE via distributed learning.
Revised: However, those approaches perform poorly in large-scale problems, and do not allow the players to arrive at EFCE via distributed learning.

Original: Celli et al.
Revised: Celli et al.

Original: (2019) devised variants of the CFR algorithm that provably convergence to normal-form coarse correlated equilibria , a solution concept much less appealing than EFCE in extensive-form games.
Revised: (2019) devised variants of the CFR algorithm that provably convergence to normal-form coarse correlated equilibria , a solution concept much less appealing than EFCE in extensive-form games Gordon et al.

Original:  Morrill et al.
Revised: Finally, Morrill et al.

Original: (2021a;b) characterize hindsight rationality notions and associate a set of solution concepts with suitable O ( T  1 / 2 ) no-regret learning dynamics.
Revised: (2021a;b) characterize hindsight rationality notions and associate a set of solution concepts with suitable O ( T  1 / 2 ) no-regret learning dynamics.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Extensive-form Games.
Revised: Extensive-form Games.

Original: An extensive-form game is abstracted on a directed and rooted game tree
Revised: An extensive-form game is abstracted on a directed and rooted game tree

Original: T .
Revised: T .

Original: The set of nodes of T is denoted with H ; non-terminal nodes are referred as decision nodes , and are associated with a player who acts by selecting an action from a set of possible actions A ( h ) , where h 2 H represents the decision node.
Revised: The set of nodes of T is denoted with H ; non-terminal nodes are referred as decision nodes , and are associated with a player who acts by selecting an action from a set of possible actions A ( h ) , where h 2 H represents the decision node.

Original: By convention, the set of players [ n ] [ { c } includes a ﬁctitious agent c who “selects” actions according to ﬁxed probability distributions dictated by the nature of the game (e.g., the roll of a dice); this intends to model external stochastic phenomena occurring during the game.
Revised: By convention, the set of players [ n ] [ { c } includes a ﬁctitious agent c who “selects” actions according to ﬁxed probability distributions dictated by the nature of the game (e.g., the roll of a dice); this intends to model external stochastic phenomena occurring during the game.

Original: For a player i 2 [ n ] [ { c } , we let H ( i ) ✓ H be the subset of decision nodes wherein a player
Revised: For a player i 2 [ n ] [ { c } , we let H ( i ) ✓ H be the subset of decision nodes wherein a player

Original: i makes a decision.
Revised: i makes a decision.

Original: The set of leaves
Revised: The set of leaves

Original: Z ✓ H , or equivalently the terminal nodes , correspond to different outcomes; once the game transitions to a terminal node z 2 Z , payoffs are assigned to each player based on a set of normalized utility functions { u ( i ) :
Revised: Z ✓ H , or equivalently the terminal nodes , correspond to different outcomes; once the game transitions to a terminal node z 2 Z , payoffs are assigned to each player based on a set of normalized utility functions { u ( i ) :

Original: Z !
Revised: Z !

Original: [  1 , 1] } i 2 [ n ] .
Revised: [  1 , 1] } i 2 [ n ] .

Original: It will also be convenient to represent with p ( c ) ( z ) the product of probabilities of “chance” moves encountered in the path from the root until the terminal node z 2 Z .
Revised: It will also be convenient to represent with p ( c ) ( z ) the product of probabilities of “chance” moves encountered in the path from the root until the terminal node z 2 Z .

Original: Imperfect Information.
Revised: Imperfect Information.

Original: To capture imperfect information, the set of decision nodes H ( i ) of player i are partitioned into a collection of setsof nodes J ( i ) , which are called information sets .
Revised: To model imperfect information, the set of decision nodes H ( i ) of player i are partitioned into a collection of sets J ( i ) , which are called information sets .

Original: Each information set j 2 J ( i ) groups nodes which cannot be distinguished byplayer i .
Revised: Each information set j 2 J ( i ) groups nodes which cannot be distinguished by i .

Original: Thus, for any nodes h, h 0 

Revised: Thus, for any nodes h, h 0

Original: j we have A ( h ) =
Revised: j we have A ( h ) =

Original: A ( h 0 ) .
Revised: A ( h 0 ) .

Original: As usual, we assume that the game satisﬁes perfect recall : players never forget information once acquired.
Revised: As usual, we assume that the game satisﬁes perfect recall : players never forget information once acquired.

Original: This implies, in particular, that for any nodes h, h 0 2 j the sequence of i ’s actions from the root until h must coincide with the sequence from the root to node h 0 ; otherwise, i would be able to distinguish between the nodes h and h 0 by virtue of perfect recall.

Original: We will also deﬁne a partial order  on J ( i ) , so that j  j 0 , for j, j 0 2 J ( i ) , if there exist nodes h 2 j and h 0 2 j 0 such that the path from the root to h 0 passes throughnode h .
Revised: We will also deﬁne a partial order  on J ( i ) , so that j  j 0 , for j, j 0 2 J ( i ) , if there exist nodes h 2 j and h 0 2 j 0 such that the path from the root to h 0 passes through h .

Original: If j  j 0 , we will say that j is an ancestor of j 0 , or equivalently, j is a descendant of j 0 .
Revised: If j  j 0 , we will say that j is an ancestor of j 0 , or equivalently, j is a descendant of j 0 .

Original: Sequence-form Strategies.
Revised: Sequence-form Strategies.

Original: For a player i 2 [ n ] , an information set j 2 J ( i ) , and an action a 2 A ( j ) , we will denote with  = ( j, a ) the sequence of i ’s actions encountered on the path from the root of the game until (and included) action a .
Revised: For a player i 2 [ n ] , an information set j 2 J ( i ) , and an action a 2 A ( j ) , we will denote with  = ( j, a ) the sequence of i ’s actions encountered on the path from the root of the game until (and included) action a .

Original: For notational convenience, we will use the special symbol ?
Revised: For notational convenience, we will use the special symbol ?

Original: to denote the empty sequence .
Revised: to denote the empty sequence .

Original: Then, i ’s set of sequences is deﬁned as ⌃ ( i ) :
Revised: Then, i ’s set of sequences is deﬁned as ⌃ ( i ) :

Original: = { ( j, a ) : j2 J ( i ) , a 2 A ( j ) } [ { ? } ; we will also use the notation ⌃ ( i )⇤ :
Revised: = { ( j, a ) : j J ( i ) , a 2 A ( j ) } [ { ? } ; we will also use the notation ⌃ ( i )

Revised: ⇤ :

Original: = ⌃ ( i ) \ { ? } .
Revised: = ⌃ ( i ) \ { ? } .

Original: For a given information set j 2 J ( i ) we will use  ( i ) ( j ) 2 ⌃ ( i ) to represent the parent sequence ; i.e. the last sequence encountered by player i before reaching any node in the information set j , assuming that it exists.
Revised: For a given information set j 2 J ( i ) we will use  ( i ) ( j ) 2 ⌃ ( i ) to represent the parent sequence ; i.e. the last sequence encountered by player i before reaching any node in the information set j , assuming that it exists.

Original: Otherwise, we let  ( i ) ( j ) = ? , and we say that j is the root information set of player i .
Revised: Otherwise, we let  ( i ) ( j ) = ? , and we say that j is the root information set of player i .

Original: At a high-level, a strategy for a player speciﬁes a probability distribution for every possible information set encountered in the game tree.
Revised: A strategy for a player speciﬁes a probability distribution for every possible information set encountered in the game tree.

Original: For perfect-recall EFGs, strategies canequivalently be  represented in sequence-form , where each sequence is associated with a value specifying the realization probability of the sequence of information set-action pairs taken by that player in order to reach a given information set-action pair (Koller et al., 1996; Von Stengel, 2002):
Revised: For perfect-recall EFGs, strategies can be equivalently represented in sequence-form :

Original: Deﬁnition 2.1 (Sequence-form Polytope) .
Revised: Deﬁnition 2.1 (Sequence-form Polytope) .

Original: The sequence-form strategy polytope for player i 2 [ n ] is deﬁned as the following (convex) polytope:
Revised: The sequence-form strategy polytope for player i 2 [ n ] is deﬁned as the following (convex) polytope:

Original: [Equation]
Revised: [Equation]

Original: The probability of playing action a at information set j can be obtained by dividing q [( j, a )] by q [  ( i ) ( j )] . Analogously, one can deﬁne the sequence-form strategy polytope for the subtree of the partially ordered set ( J ( i ) ,  ) rooted at j 2 J (( i ) , which will be denoted as Q ( i ) j .
Revised: Analogously, one can deﬁne the sequence-form strategy polytope for the subtree of the partially ordered set ( J ( i ) ,  ) rooted at j 2 J (( i ) , which will be denoted as Q ( i ) j .

Original: Moreover, the set of deterministic sequence-form strategies for player i 2 [ n ] is the set ⇧ ( i )
Revised: Moreover, the set of deterministic sequence-form strategies for player i 2 [ n ] is the set ⇧ ( i )

Original: =
Revised: =

Original: Q ( i ) \ { 0 , 1 } | ⌃ ( i )
Revised: Q ( i ) \ { 0 , 1 } | ⌃ ( i )

Original: | , and similarly for ⇧ ( i ) j .
Revised: | , and similarly for ⇧ ( i ) j .

Original: The joint set of deterministic sequence-form strategies of the players will be represented with ⇧ := ⇥
Revised: The joint set of deterministic sequence-form strategies of the players will be represented with ⇧ := ⇥

Original: i 2 [ n ] ⇧ ( i ) .
Revised: i 2 [ n ] ⇧ ( i ) .

Original: As such, an element ⇡ 2 ⇧ is an n -tuple ( ⇡ (1) , . . .
Revised: As such, an element ⇡ 2 ⇧ is an n -tuple ( ⇡ (1) , . . .

Original: , ⇡ ( n ) ) specifying a deterministic sequence-form strategy for every player i 2 [ n ] .
Revised: , ⇡ ( n ) ) specifying a deterministic sequence-form strategy for every player i 2 [ n ] .

Original: Finally, the utility of player i 2 [ n ] under a proﬁle ⇡ 2 ⇧ can be expressed as
Revised: Finally, the utility of player i 2 [ n ] under a proﬁle ⇡ 2 ⇧ can be expressed as

Original: [Equation]
Revised: [Equation]

Original: Extensive-Form Correlated Equilibrium. We  will work with the deﬁnition of EFCE due to
Revised: We summarized in Table 1 the EFG notation that we will be using most often throughout the paper.

Original: Farina et al.
Revised: An Illustrative Example.

Original: (2019e) (which is equivalent to that ofvon Stengel & Forges (2008)), which in turn is based on the notion of trigger agents (Gordon et al.
Revised: To clarify some of the concepts we have introduced, we illustrate a simple two-player EFG in Figure 1.

Original: Deﬁnition 2.2 (Trigger Agent) . Consider a player i 2 [ n ] , some sequence ˆ  = ( j, a ) 2⌃ ( i ) ⇤ ,  andˆ ⇡ 2 ⇧ ( i ) j . A (ˆ  , ˆ ⇡ ) - trigger agent engages in the game according to the following rules:
Revised: Black nodes belong to player 1 , white round nodes to player 2 , square nodes are terminal nodes (aka leaves), and the crossed node is a chance node. Player 2 has two information sets, J (2) :

Original: • If the action a at information set j has not been recommended, the trigger agent faithfully adheres to the recommendations put forward by the mediator; • Otherwise, if at the information set j the mediator recommends to play action a , the trigger agent gets “trigerred” by the sequence ˆ  =( j, a ) . That is, the agent will henceforth disregard any recommendations, and will instead commit to the continuation strategy ˆ ⇡ .
Revised: =

Original: With this deﬁnition, an EFCE is a probability distribution µ 2  | ⇧ | over ⇧ such that for any player i 2 [ n ] , trigger sequence ˆ  = ( j, a ) 2 ⌃ ( i ) ⇤ , and continuation strategy ˆ ⇡ 2 ⇧ ( i ) j , the expected utility of the (ˆ  , ˆ ⇡ ) -trigger agent is not strictly greater than the expected utility of i when the mediator’s recommendations are followed. Formally, let us ﬁrst introduce trigger deviation functions : Deﬁnition 2.3 (Trigger Deviation Functions) .
Revised: { C , D } , each containing two nodes.

Original: Consider some player i 2 [ n ] , some sequence ˆ  = ( j, a ) 2 ⌃ ( i ) ⇤ , and joint deterministic sequence-form strategies ⇡ 2 ⇧ ( i ) j . A trigger deviation function with respect to a trigger sequence ˆ  and continuation strategy ˆ ⇡ is any linear function f : R | ⌃ ( i ) | ! R | ⌃ ( i ) | with the following properties: • Any strategy ⇡ 2 ⇧ ( i ) which does not prescribe the sequence ˆ  remains invariant.
Revised: This captures the lack of knowledge regarding the action played by player 1 .

Original: That is, f ( ⇡ ) = ⇡ for any ⇡ 2 ⇧ ( i ) such that ⇡ [ˆ  ] = 0 ; • Otherwise, the prescribed sequence ˆ  = ( j, a ) is modiﬁed so that the behavior at j , as well as all its descendants is replaced by the behavior speciﬁed by the continuation strategy: [Equation]
Revised: In contrast, the outcome of the chance move is observed by both players.

Original: for all  2 ⌃ ( i ) and ⇡ 2 ⇧ ( i ) such that ⇡ [ˆ  ] = [Equation] We will let ( i ) :
Revised: At the information set C, player 2 has two possible actions, A ( C ) :

Original: = {  ( i ) ˆ  !
Revised: = { 5 , 6 } .

Original: ˆ ⇡ : ˆ  = ( j, a ) 2 ⌃ ( i ) ⇤ , ˆ ⇡ 2 ⇧ ( i ) j } be the set of all possible linear mappings deﬁning trigger deviation functions for playeri . Deﬁnition 2.4 (EFCE) . For ✏  0 , a probability distribution µ 2 | ⇧ | is an ✏ -approximate EFCE if for every player i 2 [ n ] and every trigger deviation function  ( i ) ˆ  !
Revised: Thus, one possible sequence for player 2 is the pair  =

Original: ˆ ⇡( i ) , it holds that
Revised: ( C , 5 ) 2 ⌃ (2) .

Original: [Equation]
Revised: [Figure]

Original: where ⇡ = ( ⇡ 1 , . .

Original: . , ⇡ n ) 2 ⇧ .

Original: A probability distribution µ 2  | ⇧ | is an EFCE if it is a 0 -EFCE. Theorem 2.5 (Farina et al. (2021a)) . For every player i 2 [ n ] , let ⇡ ( i ) , 1 , . . .
Revised: [Figure]

Original: , ⇡ ( i ) ,T 2 ⇧ ( i ) be a sequence of deterministic sequence-form strategies whose cumulative ( i ) -regret is R ( i ) ,T with respect to the sequence of linear utility functions

Original: [Equation]
Revised: [Table]

Original: Then, the empirical frequency of play µ 2  | ⇧ | is an ✏ - EFCE , where ✏ := 1 T max i 2 [ n ] R ( i ) ,T .

Original: Regret,  -Regret and Optimistic Regret Minimization
Revised: Regret,  -Regret and Optimistic Regret Minimization.

Original: Consider a convex and compact set X ✓ R d representing the space of strategies of some agent.
Revised: Consider a convex and compact set X ✓ R d representing the space of strategies of some agent.

Original: In the online decision making framework, a regret minimizer R can be thought of as a black-box device which interacts with the external environment via the following two basic subroutines:
Revised: In the online decision making framework, a regret minimizer R can be thought of as a black-box device which interacts with the external environment via the following two basic subroutines:

Original: • R .
Revised: • R .

Original: N EXT S TRATEGY () :
Revised: N EXT S TRATEGY () :

Original: The regret minimizer returns the strategy x t 2 X at time t ; • R .
Revised: The regret minimizer returns the strategy x t 2 X at time t ; • R .

Original: O BSERVE
Revised: O BSERVE

Original: U TILITY ( ` t ) :
Revised: U TILITY ( ` t ) :

Original: The regret minimizer receives as feedback a  utility function ` t : X ! R , and may alter its internal state accordingly. It is assumed that the utility function is linear, so that ` [Equation] [Equation]
Revised: The regret minimizer receives as feedback a linear utility function ` t : X 3 x 7!

Original:  , for a (uniquely-deﬁned [Equation]
Revised: h ` t , x i , and may alter its internal state accordingly.

Original: The decision making is online in the sense thatat each time t the regret minimizer can adapt to previously received information, but no information about future utilities is available.
Revised: The decision making is online in the sense that the regret minimizer can adapt to previously received information, but no information about future utilities is available.

Original: The error of a regret minimizer is typically measured in terms of external regret , deﬁned as follows:
Revised: The error of a regret minimizer is typically measured in terms of external regret , deﬁned, for a time horizon T , as follows:

Original: [Equation]
Revised: [Equation]

Original: where T represents the time horizon.

Original: That is, the performance of the online algorithm is compared with the best ﬁxed strategy in hindsight .
Revised: That is, the performance of the online algorithm is compared with the best ﬁxed strategy in hindsight .

Original: A regret minimizer is called Hannan consistent if, under any sequence of utility functions, the regret grows sublinearly with T ; i.e. R =

Original: o

Original: ( T ) .

Original:  -Regret.
Revised:  -Regret.

Original: A conceptual generalization of the concept of external regret is the so-called  - regret .
Revised: A conceptual generalization of the concept of external regret is the so-called  - regret .

Original: Speciﬁcally, in this framework the performance of the learning algorithm is measured based on a set of transformations  : X !
Revised: Speciﬁcally, in this framework the performance of the learning algorithm is measured based on a set of transformations  : X !

Original: X , leading to the notion of cumulative  -regret:
Revised: X , leading to the notion of cumulative  -regret:

Original: [Equation]
Revised: [Equation]

Original: Observe that when the set of transformations  coincides with the set of constant functions, one recovers theordinal notion of (external) regret given in Equation (6).
Revised: When the set of transformations  coincides with the set of constant functions, one recovers the notion of external regret given in Equation (3).

Original: However,  -regret is substantially more expressive and yields a more appealing notion of hindsight rationality, incorporating the notion of swap regret (Blum & Mansour, 2007).An emerging subﬁeld of online learning (Chiang et al.
Revised: However,  -regret is substantially more expressive and yields a more appealing notion of hindsight rationality (Gordon et al., 2008), incorporating the notion of swap regret (Blum & Mansour, 2007).

Original: ; Rakhlin & Sridharan (2013a)) studies the improved performance guarantees one can obtain if the utility functions observed by the no-regret algorithm possess additional structure , usually in the form of small variation .

Original: For our purposes, we will use the following deﬁnition, which is a slight modiﬁcation of the RVU property introduced by (Syrgkanis et al., 2015, Deﬁnition 3).
Revised: We will employ the following deﬁnition, which is a slight modiﬁcation of the RVU property introduced by (Syrgkanis et al., 2015, Deﬁnition 3).

Original: Deﬁnition 2.6 (Stable-predictivity) .
Revised: Deﬁnition 2.2 (Stable-predictivity) .

Original: Let R be a regert minimizer and let k · k be a norm.
Revised: Let R be a regert minimizer and let k · k be a norm.

Original: R is said to be  - stable with respect to k · k if for all t  2 , the strategies output by R satisfy
Revised: R is said to be  - stable with respect to k · k if for all t  2 , the strategies output by R satisfy

Original: [Equation]
Revised: [Equation]

Original: Moreover, it is said to be ( ↵ ,  ) - predictive with respect to k · k, where ↵ : N ! R > 0 and  2 R > 0 , if at all t  1 its regret R T satisﬁes 
Revised: Moreover, it is said to be ( ↵ ,  ) - predictive with respect to k · k if for all t  1 its regret R T satisﬁes [Equation]

Original: [Equation]
Revised: [Equation]

Original: no matter the sequence of utility vectors ` 1 , . . .
Revised: no matter the sequence of utility vectors ` 1 , . . .

Original: , ` T , where k · k ⇤ is the dual norm of k · k .
Revised: , ` T , where k · k ⇤ is the dual norm of k · k .

Original: Optimistic Follow the Regularized Leader.
Revised: Optimistic Follow the Regularized Leader.

Original: The class of algorithms termed Optimistic Follow the

Original: Regularized Leader (OFTRL) constitutes a simple variant of FTRL, incorporating a prediction into the optimization step.

Original: To be more precise, let d be a 1 -strongly convex function with respect to some norm k · k , and ⌘ > 0 the learning rate .
Revised: Let d be a 1 -strongly convex function with respect to some norm k · k , and ⌘ > 0 the learning rate .

Original: OFTRL’s update rule takes the following form:
Revised: OFTRL’s update rule takes the following form:

Original: [Equation]
Revised: [Equation]

Original: where x 1 :=
Revised: where x 1 :=

Original: arg min x 2 X d ( x ) .
Revised: arg min x 2 X d ( x ) .

Original: Syrgkanis et al.
Revised: Syrgkanis et al.

Original: (2015) established the following property:
Revised: (2015) established the following property:

Original: Lemma 2.7 (Syrgkanis et al. (2015, Proposition 7 and Lemma 20) 1 ) . (OFTRL)with learning rate ⌘ is 2 ⌘ -stable and ( ⌦ d / ⌘ , ⌘ ) -predictive with respect to any norm k · k with respect to which d is 1 -strongly convex, where ⌦ d is the range of d on X , that is, ⌦ d := max x , x 0 2 X { d ( x )  d ( x 0 ) } .
Revised: Lemma 2.3. (OFTRL) is 2 ⌘ -stable and ( ⌦ d / ⌘ , ⌘ ) -predictive with respect to any norm k · k for which d is 1 -strongly convex, where ⌦ d is the range of d on X , that is, ⌦ d := max x , x 0 2 X { d ( x )  d ( x 0 ) } .

Original: In this paper, we consider the entropic regularizer, which is 1 -strongly convex with respect to the `norm.
Revised: In this paper, we consider the entropic regularizer with respect to the simplex d ( x ) := P di =1 x i log x i , which is 1 -strongly convex with respect to the ` 1 norm.

Original: The pair of dual norms in the predictivity bound will therefore be ( k · k 1 , k · k 1 ) .
Revised: The pair of dual norms in the predictivity bound will therefore be ( k · k 1 , k · k 1 ) .

Original: We call this OFTRL setup Optimistic Multiplicative Weights Updates (OMWU).
Revised: We call this OFTRL setup Optimistic Multiplicative Weights Updates (OMWU).

Revised: Extensive-Form Correlated Equilibrium.

Original: A CCELERATING  -R EGRET M INIMIZATION VIA O PTIMISM
Revised: We will work with the deﬁnition of EFCE due to Farina et al. (2019e), which is equivalent to that of von Stengel & Forges (2008). First, let us introduce the concept of a trigger deviation function .

Revised: Deﬁnition 2.4.

Revised: Consider some player i 2 [ n ] , a sequence ˆ  = ( j, a ) 2 ⌃ ( i ) ⇤ , and joint sequenceform strategies ⇡ 2 ⇧ ( i ) j .

Revised: A trigger deviation function with respect to a trigger sequence ˆ  and continuation strategy ˆ ⇡ is any linear function f : R | ⌃ ( i ) | !

Revised: R | ⌃ ( i ) | with the following properties.

Revised: • Any strategy ⇡ 2 ⇧ ( i ) which does not prescribe the sequence ˆ  remains invariant.

Revised: That is, f ( ⇡ ) =

Revised: ⇡ for any ⇡ 2 ⇧ ( i ) such that ⇡ [ˆ  ] = 0 ; • Otherwise, the prescribed sequence ˆ  = ( j, a ) is modiﬁed so that the behavior at j , as well as all its descendants is replaced by the behavior speciﬁed

Revised: by the continuation strategy:

Revised: [Equation]

Revised: for all  2 ⌃ ( i ) and ⇡ 2 ⇧ ( i ) such that ⇡ [ˆ  ] = 1 .

Revised: We will let ( i ) :

Revised: = {

Revised: [Equation]

Revised: i )

Revised: [Equation]

Revised:  !

Revised: ˆ ⇡ : ˆ  =

Revised: [Equation]

Revised: j, a ) 2 ⌃ ( i ) ⇤ , ˆ ⇡ 2 ⇧ ( i ) j } be the set of all possible linear mappings deﬁning trigger deviation functions for player i .

Revised: We are ready to introduce the concept of EFCE.

Revised: Deﬁnition 2.5 (EFCE) .

Revised: For ✏  0 , a probability distribution µ 2  | ⇧ | is an ✏ -approximate EFCE if for every player i 2 [ n ] and every trigger deviation function  ( i )

Revised: ˆ  !

Revised: ˆ ⇡( i ) , it holds that

Revised: [Equation]

Revised: where ⇡ = ( ⇡ 1 , . .

Revised: . , ⇡ n ) 2 ⇧ .

Revised: A probability distribution µ 2  | ⇧ | is an EFCE if it is a 0 -EFCE.

Revised: Theorem 2.6 (Farina et al.

Revised: (2021a)) .

Revised: For every player i 2 [ n ] , let ⇡ ( i ) , 1 , . . .

Revised: , ⇡ ( i ) ,T 2 ⇧ ( i ) be a sequence of deterministic sequence-form strategies whose cumulative ( i ) -regret is R ( i ) ,T with respect to the sequence of linear utility functions

Revised: [Equation]

Revised: Then, the empirical frequency of play µ 2  | ⇧ | is an ✏ - EFCE , where ✏ := 1 T max i 2 [ n ] R ( i ) ,T .

Revised:  A CCELERATING  -R EGRET M INIMIZATION VIA O PTIMISM

Original: In this section we develop a general template for accelerated  -regret minimization for general sets, and then we instantiate the template for dynamics for EFCE.
Revised: In this section we develop a general template for accelerated  -regret minimization for general sets, and then we instantiate the template for dynamics for EFCE.

Original: Our approach combines a framework of
Revised: Our approach combines a framework of

Original: Gordon et al.
Revised: Gordon et al.

Original: with the framework of stable-predictive (aka.
Revised: with the framework of stable-predictive (aka.

Original: optimistic) regret minimization.
Revised: optimistic) regret minimization.

Original: As in Gordon et al.
Revised: As in Gordon et al.

Original: (2008), in our template we combine 1) a regret minimizer that outputs a linear transformation  t 2  at every time t , and 2) a ﬁxed-point oracle for each  t 2  .
Revised: (2008), in our template we combine 1) a regret minimizer that outputs a linear transformation  t 2  at every time t , and 2) a ﬁxed-point oracle for each  t 2  .

Original: However, in our framework, we further require that (2) is stable (in the sense of Deﬁnition 2.6).
Revised: However, in our framework, we further require that 2) is stable (in the sense of Deﬁnition 2.2).

Original: To make (2) stable, we will focus on regret minimizers that have the following property.
Revised: To achieve this, we will focus on regret minimizers that have the following property:

Original: Deﬁnition 3.1.
Revised: Deﬁnition 3.1.

Original: Consider a set of functions  such that  ( X ) ✓ X for all  2  , and a no-regret algorithm R  for the set of transformations  which returns a sequence  t 2  .
Revised: Consider a set of functions  such that  ( X ) ✓ X for all  2  , and a no-regret algorithm R  for the set of transformations  which returns a sequence  t 2  .

Original: We say that R  is ﬁxed point G -stable , for G  0 , if the following conditions hold:
Revised: We say that R  is ﬁxed point G -stable , for G  0 , if the following conditions hold:

Original: • Every  t admits a ﬁxed point.
Revised: • Every  t admits a ﬁxed point.

Original: That is, there exists x t 2 X such that  t ( x t ) =
Revised: That is, there exists x t 2 X such that  t ( x t ) =

Original: x t .
Revised: x t .

Original: • For any x t
Revised: • For any x t

Original: such that x t =
Revised: such that x t =

Original:  t ( x t ) , there exists x t +1 with x t
Revised:  t ( x t ) , there exists x t +1 with x t

Original: +1 =
Revised: +1 =

Original:  t +1 ( x t +1 )
Revised:  t +1 ( x t +1 )

Original: such that k x t +1  x t k  G .
Revised: such that k x t +1  x t k  G .

Original: We will show how to construct an accelerated  -regret minimizer starting from the following:
Revised: We will show how to construct an accelerated  -regret minimizer starting from the following:

Original: R  : A  -stable ( ↵ ,  ) -predictive ﬁxed point G -stable regret minimizer for  ; 
Revised: R  : A  -stable ( ↵ ,  ) -predictive ﬁxed point G -stable regret minimizer for  ; 2.

Original:  e x , G, ✏ ) :
Revised: S TABLE FPO RACLE (  ; e x , G, ✏ ) :

Original: A stable ﬁxed point oracle which returns a point x 2 X such that (i) k
Revised: A stable ﬁxed point oracle which returns a point x 2 X such that (i) k

Original:  ( x )  x k  ✏ , and (ii)
Revised:  ( x )  x k  ✏ , and (ii)

Original: k x  e x k  G (the existence of such a ﬁxed point is guaranteed by the ﬁxed point G -stability assumption for the regret minimizer).
Revised: k x  e x k  G (the existence of such a ﬁxed point is guaranteed by the ﬁxed point G -stability assumption for the regret minimizer).

Original: Given these two components, our next theorem builds a stable-predictive  -regret minimizer.
Revised: Given these two components, our next theorem builds a stable-predictive  -regret minimizer.

Original: Theorem 3.2 (Accelerated  -Regret Minimization) .
Revised: Theorem 3.2 (Accelerated  -Regret Minimization) .

Original: Consider a  -stable ( ↵ ,  ) -predictive regret minimizer R  for a set of linear transformations  , with respect to the ` 1 norm k · k 1 .
Revised: Consider a  -stable ( ↵ ,  ) -predictive regret minimizer R  for a set of linear transformations  , with respect to the ` 1 norm k · k 1 .

Original: More- over, assume that
Revised: More- over, assume that

Original: R  is ﬁxed point G -stable with respect to  .
Revised: R  is ﬁxed point G -stable with respect to  .

Original: Then, if we have access to a
Revised: Then, if we have access to a

Original: S  , we can construct a G -stable algorithm with  -regret R T bounded as
Revised: S TABLE FPO RACLE , we can construct a G -stable algorithm with  -regret R T bounded as

Original: [Equation]
Revised: [Equation]

Original: where ✏ t is the error of S TABLE FPO RACLE at time t , and D ` is an upper bound on the ` 1 norm of ` t ’s. It is also assumed that k x k 1  1 for all x 2 X .
Revised: where ✏ t is the error of S TABLE FPO RACLE at time t , and D ` is an upper bound on the ` 1 norm of ` t ’s. It is also assumed that k x k 1  1 for all x 2 X .

Original: The proof is similar to that of Gordon et al.
Revised: The proof is similar to that of Gordon et al.

Original: and is included in Appendix B.
Revised: and is included in Appendix B.

Original: This framework reduces performing accelerated  -regret minimization to (i) developing a stable-predictive regret minimizer for the set ( i ) , for any player i 2 [ n ] , and (ii) establishing a S TABLE FPO RACLE .

Original: In this context, Section 3.1 is concerned with the former task, while Section 3.2 with the latter.

Original: 3. C ONSTRUCTING A S TABLE -P REDICTIVE R EGRET M INIMIZER FOR ( i )
Revised: 3. C ONSTRUCTING A S TABLE -P REDICTIVE R EGRET M INIMIZER FOR ( i )

Original: Here we develop a regret minimizer for the set co ( i ) , the convex hull of the set of trigger deviation functions.
Revised: Here we develop a regret minimizer for the set co ( i ) , the convex hull of the set of trigger deviation functions.

Original: Given that co ( i ) ◆ ( i ) , this will immediately imply a regret minimizer for the set ( i ) .
Revised: Given that co ( i ) ◆ ( i ) , this will immediately imply a regret minimizer for the set [Figure] [Figure] [Figure]

Original: Farina et al.
Revised: ( i ) [Figure] R ( i ) (Theorem 3.4) ( i ) [Figure]

Revised: Regret Minimizer for Q ( i )

Revised: [Figure]

Revised: ( i )

Revised: [Figure]

Revised: ( i )

Revised: [Figure]

Revised: Fixedpoint

Revised: ( i ) .

Revised: An overview of the algorithm is given in Figure 2.

Revised: Farina et al.

Original: (2021a) observed that the set co ( i ) can be evaluated in two stages.
Revised: (2021a) observed that the set co ( i ) can be evaluated in two stages.

Original: First, for a ﬁxed sequence ˆ  = ( j, a ) 2 ⌃ ( i )
Revised: First, for a ﬁxed sequence ˆ  = ( j, a ) 2 ⌃ ( i )

Original: ⇤ we deﬁne the set 
Revised: ⇤ we deﬁne the set ( i )

Revised: ˆ  := co n

Original: [Equation]
Revised: [Equation]

Original: o ; then, we take the convex hull of all ( i ) ˆ  , that is, co ( i ) =
Revised: o ; then, we take the convex hull of all ( i ) ˆ  , that is, co ( i ) =

Original: co { ( i ) ˆ  : ˆ  2 ⌃ ( i ) ⇤ } .
Revised: co { ( i ) ˆ  : ˆ  2 ⌃ ( i ) ⇤ } .

Original: Correspondingly, we ﬁrst develop a stable-predictive regret minimizer for the set ( i ) ˆ  , for any ˆ  2 ⌃ ( i ) ⇤ , and these individual regret minimizers are then combined using a regret circuit to conclude the construction in Theorem 3.5.
Revised: Correspondingly, we ﬁrst develop a stable-predictive regret minimizer for the set ( i ) ˆ  , for any ˆ  2 ⌃ ( i ) ⇤ , and these individual regret minimizers are then combined using a regret circuit to conclude the construction in Theorem 3.4.

Original: All the omitted poofs and pseudocode for this section are included in Appendix B.1.
Revised: All the omitted poofs and pseudocode for this section are included in Appendix B.1.

Original: Stable-Predictive Regret Minimizer for the set ( 
Revised: Stable-Predictive Regret Minimizer for the set ( i )

Original: [Equation]
Revised: ˆ  .

Original: Consider a sequence ˆ  2 ⌃ ( i ) ⇤ .
Revised: Consider a sequence ˆ  2 ⌃ ( i ) ⇤ .

Original: Farina et al.
Revised: Farina et al.

Original: (2021a) observed that the set of transformations
Revised: (2021a) observed that the set of transformations

Original: [Equation]
Revised: [Equation]

Original:  o is the image of Q ( i ) j under the afﬁne mapping h ( i )
Revised: j o is the image of Q ( i ) j under the afﬁne mapping h ( i )

Revised: ˆ  : q 7!

Original: ˆ  :  ( i ) 
Revised:  ( i ) ˆ  !

Original: ˆ  ! ⇡ .
Revised: q .

Original: Hence, it is well-known that a regret minimizer for ( i ) ˆ  can be constructed starting from a regret minimizer for Q ( i ) j .
Revised: Hence, it is well-known that a regret minimizer for ( i ) ˆ  can be constructed starting from a regret minimizer for Q ( i ) j .

Original: We now show that the same can be said if one restricts to stable-predictive regret minimizers.
Revised: We now show that the same can be said if one restricts to stable-predictive regret minimizers.

Original: In particular, we have the following.
Revised: In particular, we have the following.

Original: Proposition 3.3.
Revised: Proposition 3.3.

Original: Consider a player i 2 [ n ] and any trigger sequence ˆ  =
Revised: Consider a player i 2 [ n ] and any trigger sequence ˆ  =

Original: ( j, a ) 2 ⌃ ( i ) ⇤ .
Revised: ( j, a ) 2 ⌃ ( i ) ⇤ .

Original: There exists an algorithm which constructs a deterministic regret minimizer R ( i ) ˆ  with access to a K -stable
Revised: There exists an algorithm which constructs a deterministic regret minimizer R ( i ) ˆ  with access to a K -stable

Original: ( A T , B ) -predictive deterministic regret minimizer R ( i ) Q for the set Q ( i ) j , such that R ( i )
Revised: ( A T , B ) -predictive deterministic regret minimizer R ( i ) Q for the set Q ( i ) j , such that R ( i )

Original: ˆ  is K -stable and ( A T , B ) -predictive.
Revised: ˆ  is K -stable and ( A T , B ) -predictive.

Original: Moreover, if the routines N EXT S TRATEGY and

Original: O BSERVE U TILITY of R ( i ) Q run in linear time on | ⌃ ( i ) j | , the complexity of R ( i ) ˆ  is O ( | ⌃ ij | ) . In Appendix A we describe a stable-predictive variant of CFR for the set Q ( i ) j , for each j 2 J ( i ) , following the construction of Farina et al. (2019a).
Revised: In Appendix A we describe a stable-predictive variant of CFR for the set Q ( i ) j , for each j 2 J ( i ) , following the construction of Farina et al. (2019a).

Original: Stable-Predictive Regret Minimizer for co ( i ) .
Revised: Stable-Predictive Regret Minimizer for co ( i ) .

Original: The next step consists of combining the regret minimizers ( i ) ˆ  , for all ˆ  2 ⌃ ( i ) ⇤ , to a composite regret minimizer for the set co ( i ) .
Revised: The next step consists of combining the regret minimizers ( i ) ˆ  , for all ˆ  2 ⌃ ( i ) ⇤ , to a composite regret minimizer for the set co ( i ) .

Original: To this end, wewill employ regret circuits (Farina et al., 2019d), and in particular the following construction: Proposition 3.4 (Farina et al. (2019d)) . Consider a collection of sets X 1 , . .
Revised: To this end, we employ regret circuits (Farina et al., 2019d), leading to the main result of this section:

Original: X m , and let R i be a regret minimizer for the set X i , for each

Original: i 2 [ m ] .

Original: Moreover, let R  be a regret minimizer for the m -simplex  m .

Original: A regret minimizer

Original: R co for the set co {X 1 , . . .

Original: , X m } can be constructed as follows:

Original: • R co .

Original: N EXT S TRATEGY obtains the next strategy x ti of each regret minimizer R i , as well as the next strategy  t

Original: =

Original: (  t 1 , . . .

Original: ,  tm ) 2  m of R  , and returns the strategy  t 1 x t 1 + · · · +  tm x tm .

Original: • R co .

Original: O BSERVE

Original: U TILITY ( L t ) forwards the function L t to each of the regret minimizers R 1 , . . .

Original: , R m , while it also forwards the utility (  1 , . .

Original: [Equation]

Original: Then, if R T 1 , . .

Original: , R Tm is the regret accumulated by the regret minimizers R 1 , . . .

Original: , R m , and R T  is the regret of R  , then the cumulative regret of the composite regret minmizers R co can be bounded as

Original: [Equation]

Original: We are now ready to establish the main result of this section.

Original: Theorem 3.5.
Revised: Theorem 3.4.

Original: Consider a  -stable ( ↵ ,  ) -predictive regret minimizer R ( i )  for the the simplex  | ⌃ ( i ) ⇤ | , and K -stable ( A, B ) -predictive regret minimizers 
Revised: Consider a  -stable ( ↵ ,  ) -predictive regret minimizer R ( i )  for the the simplex  | ⌃ ( i ) ⇤ | , and K -stable ( A, B ) -predictive regret minimizers R ( i )

Original: R ( i ) ˆ  for each ˆ  2 ⌃ ( i ) ⇤ , all with respect to the pair of norms ( k · k 1 , k · k 1 ) .
Revised: ˆ  for each ˆ  2 ⌃ ( i ) ⇤ , all with respect to the pair of norms ( k · k 1 , k · k 1 ) .

Original: Then, there exists an algorithm which constructs a regret minimizer R ( i ) for the set co ( i ) such that (i) R ( i ) is O
Revised: Then, there exists an algorithm which constructs a regret minimizer R ( i ) for the set co ( i ) such that (i) R ( i ) is O

Original: ( K + | ⌃
Revised: ( K + | ⌃

Original: ( i ) |  ) -stable, and (ii) under any sequence of linear utility functions
Revised: ( i ) |  ) -stable, and (ii) under any sequence of linear utility functions

Original: L 1 , . . .
Revised: L 1 , . . .

Original: , L T the regret incurred can be bounded as
Revised: , L T the regret incurred can be bounded as

Original: [Equation]
Revised: [Equation]

Original: where k L t k 1  D L .
Revised: where k L t k 1  D L .

Original: Moreover, if the routines N EXT S TRATEGY and

Original: O BSERVE

Original: U TILITY of R ( i )  and R ( i )

Original: ˆ  , for each ˆ  2 ⌃ ( i ) ⇤ , run in linear time on | ⌃ ( i ) | , then the complexity of R ( i ) is O ( | ⌃ ( i ) )

Original: | 2 ) .

Original: 3. S TABILITY OF THE F IXED P OINTS
Revised: 3. S TABILITY OF THE F IXED P OINTS

Original: In this subsection we complete the construction of the ( i ) -regret minimizer by establishing a stable ﬁxed point oracle for any  2 co ( i ) .
Revised: In this subsection we complete the construction of the ( i ) -regret minimizer by establishing a stable ﬁxed point oracle for any  2 co ( i ) .

Original: All of the proofs of this section are included in Appendix B.2.
Revised: All of the proofs of this section are included in Appendix B.2.

Original: Multiplicative Stability.
Revised: Multiplicative Stability.

Original: A sequence { z t } , with z t 2 R d  0 , is said to be  -multiplicative-stable if (1   ) z t  1{+
+} i
Revised: A sequence { z t } , with z t 2 R d  0 , is said to be  -multiplicative-stable if (1   ) z t  {+
+} i

Original: [Equation]
Revised: [Equation]

Original: t  1 i , for any i 2 [ d ] , and for all t  2 .
Revised: , for any i 2 [ d ] , and for all t  2 .

Original: Chen & Peng (2020) observed that a Markov chain whose associated transition matrix is multiplicatively stable will also have a stable stationary distribution in the sense of Deﬁnition 2.6.

Original: Importantly, this notion of multiplicative stability is guaranteed by OMWU (see Lemma B.1).
Revised: Importantly, this notion of multiplicative stability is guaranteed by OMWU (see Lemma B.2).

Original: Thus, if D ( i ) is the depth of i ’s actions and D ( i ) x is an upper bound on the ` 1 norm in the treeplex, we can show the following:
Revised: Thus, if D ( i ) is the depth of i ’s actions and D ( i ) x is an upper bound on the ` 1 norm in the treeplex, we can show the following:

Original: Lemma 3.6.
Revised: Lemma 3.5.

Original: When each regret minimizer R ( i ) ˆ  is constructed using predictive CFR instantiated with OMWU with learning rate ⌘ (Theorem A.4) such that
Revised: When each regret minimizer R ( i ) ˆ  is constructed using predictive CFR instantiated with OMWU with learning rate ⌘ (Theorem A.4) such that

Original: for all ˆ  2 ⌃ ( i ) ⇤ , the output sequence is
Revised: for all ˆ  2 ⌃ ( i ) ⇤ , the output sequence is

Original: O ( ⌘ ( D ( i ) )
Revised: O ( ⌘ ( D ( i ) )

Original: 2 D ( i ) x
Revised: 2 D ( i ) x

Original: D ` ) -multiplicatively-stable.
Revised: D ` ) -multiplicatively-stable.

Original: Moreover, if the regret minimizer R ( i )  is realized using
Revised: Moreover, if the regret minimizer R ( i )  is realized using

Original: OMWU with learning rate ⌘ , it will output an O ( ⌘ | ⌃ ( i ) | D ` ) -multiplicatively-stable sequence.
Revised: OMWU with learning rate ⌘ , it will output an O ( ⌘ | ⌃ ( i ) | D ` ) -multiplicatively-stable sequence.

Original: This characterization will be crucial for establishing the stability of the ﬁxed points.
Revised: This characterization will be crucial for establishing the stability of the ﬁxed points.

Original: In particular, following the approach of Farina et al.
Revised: In particular, following the approach of Farina et al.

Original: (2021a), let us introduce the following deﬁnitions:
Revised: (2021a), let us introduce the following deﬁnitions:

Original: Deﬁnition 3.7.
Revised: Deﬁnition 3.6.

Original: Consider a player i 2 [ n ] and let J ✓ J ( i ) be a subset of i ’s information sets.
Revised: Consider a player i 2 [ n ] and let J ✓ J ( i ) be a subset of i ’s information sets.

Original: We say than J is a trunk of J ( i ) if, for every j 2 J , all predecessors of j are also in J .
Revised: We say than J is a trunk of J ( i ) if, for every j 2 J , all predecessors of j are also in J .

Original: Deﬁnition 3.8.
Revised: Deﬁnition 3.7.

Original: Consider a player i 2 [ n ] , a trunk J ✓ J ( i ) , and  2 co ( i ) .
Revised: Consider a player i 2 [ n ] , a trunk J ✓ J ( i ) , and  2 co ( i ) .

Original: A vector x 2 R | ⌃ ( i ) | is a J -partial ﬁxed point of  if the following conditions hold:
Revised: A vector x 2 R | ⌃ ( i ) | is a J -partial ﬁxed point of  if the following conditions hold:

Original: • x [ ? ] = 1 and x [  ( i ) ( j )]
Revised: • x [ ? ] = 1 and x [  ( i ) ( j )]

Original: = P a 2 A ( j ) x
Revised: = P a 2 A ( j ) x

Original: [( j, a )] , for all j 2 J ; •  ( x )[ ? ]
Revised: [( j, a )] , for all j 2 J ; •  ( x )[ ? ]

Original: = x [ ? ] = 1 , and  ( x )[( j, a )]
Revised: = x [ ? ] = 1 , and  ( x )[( j, a )]

Original: = x [( j, a )] , for all j 2 J , and a 2 A ( j ) .
Revised: = x [( j, a )] , for all j 2 J , and a 2 A ( j ) .

Original: An important property is that a J -partial ﬁxed point can be efﬁciently “promoted” to a J [ { j ⇤ } - partial ﬁxed point by computing the stationary distribution of a certain Markov chain.
Revised: An important property is that a J -partial ﬁxed point can be efﬁciently “promoted” to a J [ { j ⇤ } - partial ﬁxed point by computing the stationary distribution of a certain Markov chain.

Original: However, a signiﬁcant concern is whether this ﬁxed point operation can potentially cause a substantial degradation in terms of stability.
Revised: However, a signiﬁcant concern is whether this ﬁxed point operation can potentially cause a substantial degradation in terms of stability.

Original: One of our key results is that the associated Markov chain has a particular structure, which enables us to substantially improve the stability bound and thereby obtain a polynomial degradation in stability.
Revised: One of our key results is that the associated Markov chain has a particular structure, which enables us to substantially improve the stability bound and thereby obtain a polynomial degradation in stability.

Original: More precisely, we show the following technical lemma.
Revised: More precisely, this boils down to the following technical lemma.

Original: Lemma 3.9.
Revised: Lemma 3.8.

Original: Let M and M 0 be transition matrices of m -state Markov chains such that M = v 1 > + C and M 0{+
+} = v 0 1 > + C 0 , where C , C 0 , v , v 0 have strictly positive entries.
Revised: Let M and M 0 be transition matrices of m -state Markov chains such that M = v 1 > + C and M {+
+} = v 0 1 > + C 0 , where C , C 0 , v , v 0 have strictly positive entries.

Original: Moreover, let ⇡ and ⇡ 0 be the (unique) stationary distributions of M and M 0 respectively.
Revised: Moreover, let ⇡ and ⇡ 0 be the (unique) stationary distributions of M and M 0 respectively.

Original: Then, if  the entries of the matrices C and C 0 are  -multiplicatively-close, (ii) the entries of the vectors v and v 0 are  -multiplicatively-close, and (iii) the sum of the entries of v and v 0 are  -multiplicatively-close, then ⇡ and ⇡ 0 are (  + O (  m ))
Revised: Then, if (i) the entries of the matrices C and C 0 are  -multiplicatively-close, (ii) the entries of the vectors v and v 0 are  -multiplicatively-close, and (iii) the sum of the entries of v and v 0 are  -multiplicatively-close, then ⇡ and ⇡ 0 are (  + O (  m ))

Original: -multiplicatively-close, for a sufﬁciently small 
Revised: -multiplicatively-close, for a sufﬁciently small 

Original: = O (1 /m ) .
Revised: = O (1 /m ) .

Original: Using a slight variant of this result (Corollary B.9), we manage to obtain the following:
Revised: Using a slightly more general result (Corollary B.10), we manage to obtain the following:

Original: Proposition 3.10.
Revised: Proposition 3.9.

Original: Consider a player i 2 [ n ] , and let  =
Revised: Consider a player i 2 [ n ] , and let  =

Original: P ˆ  2 ⌃ ( i ) ⇤  [ˆ  ]  ( i ) ˆ  !
Revised: [Equation] ( i ) ˆ  !

Original: q ˆ  be a transformation in co ( i ) such that the sequence of  t ’s and q t ˆ  ’s is  -multiplicatively-stable, for all ˆ  2 ⌃ ( i ) ⇤ .
Revised: q ˆ  be a transformation in co ( i ) such that the sequence of  t ’s and q t ˆ  ’s is  -multiplicatively-stable, for all ˆ  2 ⌃ ( i ) ⇤ .

Original: If x t is a  -multiplicatively-stable J -partial ﬁxed point sequence, there is an algorithm which computes a ( J [ { j ⇤ } ) -partial ﬁxed point ( x t ) 0 of  such that the sequence of ( x 0 ) t ’s is (  + O (  |A ( j ⇤ ) | )) - multiplicatively-stable, for any sufﬁciently small 
Revised: If x t is a  -multiplicatively-stable J -partial ﬁxed point sequence, there is an algorithm which computes a ( J [ { j ⇤ } ) -partial ﬁxed point ( x t ) 0 of  such that the sequence of ( x 0 ) t ’s is (  + O (  |A ( j ⇤ ) | )) - multiplicatively-stable, for any sufﬁciently small 

Original: = O (1 / |A
Revised: = O (1 / |A

Original: ( j ⇤ ) | ) .
Revised: ( j ⇤ ) | ) .

Original: Thus, using our technical lemma, we manage to bypass the substantial overhead of the term  |A
Revised: Thus, using our technical lemma, we manage to bypass the substantial overhead of the term  |A

Original: ( j ⇤ ) | , which would follow using techniques similar to Chen & Peng (2020).
Revised: ( j ⇤ ) | , which would follow using techniques similar to Chen & Peng (2020).

Original: This turns out to be crucial for obtaining a polynomial dependence on the size of the game.
Revised: This turns out to be crucial for obtaining a polynomial dependence on the size of the game.

Original: Finally, we can inductively employ this proposition to show the overall stability of the ﬁxed points:
Revised: Finally, we can inductively employ this proposition to show the overall stability of the ﬁxed points:

Original: Theorem 3.11.
Revised: Theorem 3.10.

Original: Consider a player i 2 [ n ] , and let  =P ˆ [Equation]
Revised: Consider a player i 2 [ n ] , and let  =

Original: ⌃

Original: [Equation]
Revised: [Equation]

Original: 
Revised:

Original: [Equation]
Revised: [Equation]

Original: q [Equation]  be a transformation in co ( i ) such that the sequence of  t ’s and q t ˆ  ’s is  -multiplicatively-stable, for all ˆ  2 ⌃ ( i ) ⇤ .
Revised: be a transformation in co ( i ) such that the sequence of  t ’s and q t ˆ  ’s is  -multiplicatively-stable, for all ˆ  2 ⌃ ( i ) ⇤ .

Original: Then, there exists an algorithm which computes a ﬁxed point q t 2 Q ( i ) of  such that the sequence of q t ’s is O ( 
Revised: Then, there exists an algorithm which computes a ﬁxed point q t 2 Q ( i ) of  such that the sequence of q t ’s is O ( 

Original: |A ( i ) | D ( i ) ) -multiplicatively-stable, where |A
Revised: |A ( i ) | D ( i ) ) -multiplicatively-stable, where |A

Original: ( i ) | := max j 2 J ( i ) |A
Revised: ( i ) | := max j 2 J ( i ) |A

Original: ( j ) | , and for a sufﬁciently small 
Revised: ( j ) | , and for a sufﬁciently small 

Original: = O (1 / ( |A ( i ) | D ( i ) )) .
Revised: = O (1 / ( |A ( i ) | D ( i ) )) .

Original: Finally, if we use the stability values derived in Lemma 3.6, we arrive at the following conclusion:
Revised: Finally, if we use the stability values derived in Lemma 3.5, we arrive at the following conclusion:

Original: Corollary 3.12.
Revised: Corollary 3.11.

Original: For  = O (( D ( i ) x ( D ( i ) ) 2 + | ⌃ ( i ) | ) |A
Revised: For  = O (( D ( i ) x ( D ( i ) ) 2 + | ⌃ ( i ) | ) |A

Original: ( i )
Revised: ( i )

Original: | D ( i ) D ` ) , the sequence of ﬁxed points will be ( ⌘ ) -multiplicatively-stable, for any sufﬁciently small ⌘ = O (1 /  ) .
Revised: | D ( i ) D ` ) , the sequence of ﬁxed points will be ( ⌘ ) -multiplicatively-stable, for any sufﬁciently small ⌘ = O (1 /  ) .

Original: Putting It All Together.
Revised: Putting Everything Together.

Original: Finally, having established these ingredients, we can use the template of Theorem 3.2 to obtain Theorem 1.1, as we formally show in Appendix B.3.The pseudocode for the ﬁnal algorithm is given in Appendix B.
Revised: Finally, having established these ingredients, we can use the template of Theorem 3.2 to obtain Theorem 1.1, as we formally show in Appendix B.3.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section we experimentally investigate the performance of our stable-predictive algorithm compared to two other popular approaches based on a CFR-style decomposition of regrets into local regret-minimization problems: the existing algorithm by Farina et al.
Revised: In this section we experimentally investigate the performance of our stable-predictive algorithm compared to two other popular approaches based on a CFR-style decomposition of regrets into local regret-minimization problems: the existing algorithm by Farina et al.

Original: (2021b) instantiated with (i) regret matching + (RM + ) (Tammelin, 2014) for each simplex (in place of regret matching), and (ii) using the vanilla MWU algorithm for each simplex.
Revised: (2021a) instantiated with (i) regret matching + (RM + ) (Tammelin, 2014) for each simplex (in place of regret matching), and (ii) using the vanilla MWU algorithm for each simplex.

Original: In accordance to the theoretical predictions, the stepsize for OMWU is set as ⌘ t = ⌧ · t  1 / 4 (cf.
Revised: In accordance to the theoretical predictions, the stepsize for OMWU is set as ⌘ t = ⌧ · t  1 / 4 (cf.

Original: Corollary B.12), and for MWU it is set as ⌘ t = ⌧ · t  1 / 2 , where the parameter ⌧ is chosen by picking the best-performing value among { 0 . 01 , 0 . 1 , 1 , 10 , 100 } .
Revised: Corollary B.13), and for MWU it is set as ⌘ t = ⌧ · t  1 / 2 , where the parameter ⌧ is chosen by picking the best-performing value among { 0 . 01 , 0 . 1 , 1 , 10 , 100 } .

Original: In particular, we evaluate their performance based on the following popular benchmark games: (i) a three-player variant of Kuhn poker (Kuhn, 1950); (ii) a two-player bargaining game known as Sheriff (Farina et al.,
Revised: In particular, we evaluate their performance based on the following popular benchmark games: (i) a three-player variant of Kuhn poker (Kuhn, 1950); (ii) a two-player bargaining game known as Sheriff (Farina et al.,

Original: 2019e)—a benchmark game introduced speciﬁcally for the study of correlated equilibria; and (iii) a three-player version of Liar’s dice (Lis ý et al., 2015).
Revised: 2019e)—a benchmark game introduced speciﬁcally for the study of correlated equilibria; and (iii) a three-player version of Liar’s dice (Lis ý et al., 2015).

Original: A detailed description of each of the three game instances is available in Appendix D.
Revised: A detailed description of each of the three game instances is available in Appendix D.

Original: [Figure]
Revised: [Figure]

Original: Figure 1 shows the performance of each of the three learning dynamics for computing EFCE.
Revised: Figure 3 shows the performance of each of the three learning dynamics for computing EFCE.

Original: On the x -axis we plot the number of iterations performed by each algorithm,(one iteration costs the same for every algorithm, up to constant factors), and on the y -axis we plot the EFCE gap, deﬁned as the maximum advantage that any player can gain by defecting optimally from the mediator’s recommendations. 
Revised: On the x -axis we plot the number of iterations performed by each algorithm, and on the y -axis we plot the EFCE gap, deﬁned as the maximum advantage that any player can gain by defecting optimally from the mediator’s recommendations. It should be noted that one iteration costs the same for every algorithm, up to constant factors.

Original: We see that on every game, OMWU performs better than or on par with RM + and MWU.
Revised: We see that on every game, OMWU performs better than or on par with RM + and MWU.

Original: On Sheriff, OMWU performs signiﬁcantly better than both RM + and MWU, by about an order of magnitude.
Revised: On Sheriff, OMWU performs signiﬁcantly better than both RM + and MWU, by about an order of magnitude.

Original: One caveat to these results is that we did not use two tricks that help CFR + in two-player zero-sum EFG solving: alternation and linear averaging.
Revised: One caveat to these results is that we did not use two tricks that help CFR + in two-player zero-sum EFG solving: alternation and linear averaging.

Original: These tricks are known to retain convergence guarantees in that context (Tammelin et al., 2015; Farina et al., 2019b; Burch et al., 2019), but it is unclear if they still guarantee convergence in the EFCE setting.
Revised: These tricks are known to retain convergence guarantees in that context (Tammelin et al., 2015; Farina et al., 2019b; Burch et al., 2019), but it is unclear if they still guarantee convergence in the EFCE setting.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We described uncoupled no-regret learning dynamics so that if all agents play T repetitions of the game according to the dynamics, the correlated distribution of play is an O ( T  3 / 4 ) -approximate
Revised: We described uncoupled no-regret learning dynamics so that if all agents play T repetitions of the game according to the dynamics, the correlated distribution of play is an O ( T  3 / 4 ) -approximate

Original: EFCE.
Revised: EFCE.

Original: This substantially improves over the prior best rate of O ( T  1 / 2 ) .
Revised: This substantially improves over the prior best rate of O ( T  1 / 2 ) .

Original: One of our conceptual contributions is to connect the line of work on optimistic regret minimization with the framework of  -regret.
Revised: One of our conceptual contributions is to connect the line of work on optimistic regret minimization with the framework of  -regret.

Original: One of our main technical contributions is to characterize the stability of the ﬁxed points associated with trigger deviation functions through a reﬁned perturbation analysis of a certain structured Markov chain, which may be of independent interest.
Revised: One of our main technical contributions is to characterize the stability of the ﬁxed points associated with trigger deviation functions through a reﬁned perturbation analysis of a certain structured Markov chain, which may be of independent interest.

Original: Finally, experiments conducted on standard benchmarks corroborated our theoretical ﬁndings.
Revised: Finally, experiments conducted on standard benchmarks corroborated our theoretical ﬁndings.