Original: L INEAR L AST - ITERATE C ONVERGENCE IN C ONSTRAINED S ADDLE - POINT O PTIMIZATION
Revised: L INEAR L AST - ITERATE C ONVERGENCE IN C ONSTRAINED S ADDLE - POINT O PTIMIZATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Optimistic Gradient Descent Ascent (OGDA) and Optimistic Multiplicative
Revised: Optimistic Gradient Descent Ascent (OGDA) and Optimistic Multiplicative

Original: Weights Update (OMWU) for saddle-point optimization have received growing attention due to their favorable last-iterate convergence.
Revised: Weights Update (OMWU) for saddle-point optimization have received growing attention due to their favorable last-iterate convergence.

Original: However, their behaviors for simple bilinear games over the probability simplex are still not fully understood — previous analysis lacks explicit convergence rates, only applies to an exponentially small learning rate, or requires additional assumptions such as the uniqueness of the optimal solution.
Revised: However, their behaviors for simple bilinear games over the probability simplex are still not fully understood — previous analysis lacks explicit convergence rates, only applies to an exponentially small learning rate, or requires additional assumptions such as the uniqueness of the optimal solution.

Original: In this work, we signiﬁcantly expand the understanding of last-iterate convergence for OGDA and OMWU in the constrained setting.
Revised: In this work, we signiﬁcantly expand the understanding of last-iterate convergence for OGDA and OMWU in the constrained setting.

Original: Speciﬁcally, for OMWU in bilinear games over the simplex, we show that when the equilibrium is unique, linear last-iterate convergence is achievable with aconstant learning rate, which improves the result of (Daskalakis & Panageas, 2019b) under the same assumption.
Revised: Speciﬁcally, for OMWU in bilinear games over the simplex, we show that when the equilibrium is unique, linear last-iterate convergence is achieved with a learning rate whose value is set to a universal constant , improving the result of (Daskalakis & Panageas, 2019b) under the same assumption.

Original: We then signiﬁcantly extend the results to more general objectives and feasible sets for the projected OGDA algorithm, by introducing a sufﬁcient condition under which OGDA exhibits concrete last-iterate convergence rates with a constant learning rate.
Revised: We then signiﬁcantly extend the results to more general objectives and feasible sets for the projected OGDA algorithm, by introducing a sufﬁcient condition under which OGDA exhibits concrete last-iterate convergence rates with a constant learning rate whose value only depends on the smoothness of the objective function.

Original: We show that bilinear games over any polytope satisfy this condition and OGDA converges exponentially fast even without the unique equilibrium assumption .
Revised: We show that bilinear games over any polytope satisfy this condition and OGDA converges exponentially fast even without the unique equilibrium assumption .

Original: Our condition also holds for strongly-convex-strongly-concave functions, recovering the result of (Hsieh et al., 2019).
Revised: Our condition also holds for strongly-convex-stronglyconcave functions, recovering the result of (Hsieh et al., 2019).

Original: Finally, we provide experimental results to further support our theory.
Revised: Finally, we provide experimental results to further support our theory.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Saddle-point optimization in the form of min x max y f
Revised: Saddle-point optimization in the form of min x max y f

Original: ( x , y ) dates back to (Neumann, 1928), where the celebrated minimax theorem was discovered.
Revised: ( x , y ) dates back to (Neumann, 1928), where the celebrated minimax theorem was discovered.

Original: Due to advances of Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) (which itself is a saddle-point problem), the question of how to ﬁnd a good approximation of the saddle point, especially via an efﬁcient iterative algorithm, has recently gained signiﬁcant research interest.
Revised: Due to advances of Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) (which itself is a saddle-point problem), the question of how to ﬁnd a good approximation of the saddle point, especially via an efﬁcient iterative algorithm, has recently gained signiﬁcant research interest.

Original: Simple algorithms such as Gradient Descent Ascent (GDA) and Multiplicative Weights Update (MWU) are known to cycle and fail to converge even in simple bilinear cases (see e.g., (Bailey & Piliouras, 2018) and (Cheung & Piliouras, 2019)).
Revised: Simple algorithms such as Gradient Descent Ascent (GDA) and Multiplicative Weights Update (MWU) are known to cycle and fail to converge even in simple bilinear cases (see e.g., (Bailey & Piliouras, 2018) and (Cheung & Piliouras, 2019)).

Original: Many recent works consider resolving this issue via simple modiﬁcations of standard algorithms, usually in the form of some extra gradient descent/ascent steps.
Revised: Many recent works consider resolving this issue via simple modiﬁcations of standard algorithms, usually in the form of some extra gradient descent/ascent steps.

Original: This includes Extra-Gradient methods (EG) (Liang & Stokes, 2019; Mokhtari et al., 2019b), Optimistic Gradient Descent Ascent (OGDA) (Daskalakis et al., 2018;
Revised: This includes Extra-Gradient methods (EG) (Liang & Stokes, 2019; Mokhtari et al., 2020b), Optimistic Gradient Descent Ascent (OGDA) (Daskalakis et al., 2018;

Original: Gidel et al., 2019; Mertikopoulos et al., 2019), Optimistic Multiplicative Weights Update (OMWU) (Daskalakis & Panageas, 2019b; Lei et al., 2020), and others.
Revised: Gidel et al., 2019; Mertikopoulos et al., 2019), Optimistic Multiplicative Weights Update (OMWU) (Daskalakis & Panageas, 2019b; Lei et al., 2021), and others.

Original: In particular, OGDA and OMWU are suitable for the repeated game setting where two players repeatedly propose x t and y t and receive only ∇ x f ( x t , y t ) and ∇ y f ( x t , y t ) respectively as feedback, with the goal of converging to a saddle point or equivalently a Nash equilibrium using game theory terminology.
Revised: In particular, OGDA and OMWU are suitable for the repeated game setting where two players repeatedly propose x t and y t and receive only ∇ x f ( x t , y t ) and ∇ y f ( x t , y t ) respectively as feedback, with the goal of converging to a saddle point or equivalently a Nash equilibrium using game theory terminology.

Original: One notable beneﬁt of OGDA and OMWU is that they are also no-regret algorithms with important applications in online learning, especially when playing against adversarial opponents (Chiang et al., 2012; Rakhlin & Sridharan, 2013).
Revised: One notable beneﬁt of OGDA and OMWU is that they are also no-regret algorithms with important applications in online learning, especially when playing against adversarial opponents (Chiang et al., 2012; Rakhlin & Sridharan, 2013).

Original: Despite considerable progress, especially those for the unconstrained setting, the behavior of these algorithms for the constrained setting, where x and y are restricted to closed convex sets X and
Revised: Despite considerable progress, especially those for the unconstrained setting, the behavior of these algorithms for the constrained setting, where x and y are restricted to closed convex sets X and

Original: Y respectively, is still not fully understood.
Revised: Y respectively, is still not fully understood.

Original: This is even true when f is a bilinear function and
Revised: This is even true when f is a bilinear function and

Original: X and Y are simplex, known as the classic two-player zero-sum games in normal form, or simply matrix games .
Revised: X and Y are simplex, known as the classic two-player zero-sum games in normal form, or simply matrix games .

Original: Indeed, existing convergence results on the last iterate of OGDA or OMWU for matrix games are unsatisfactory — they lack explicit convergence rates (Popov, 1980; Mertikopoulos et al., 2019), only apply to exponentially small learning rate thus not reﬂecting the behavior of the algorithms in practice (Daskalakis & Panageas, 2019b), or require additional conditions such as uniqueness of the equilibrium or a good initialization (Daskalakis & Panageas, 2019b).
Revised: Indeed, existing convergence results on the last iterate of OGDA or OMWU for matrix games are unsatisfactory — they lack explicit convergence rates (Popov, 1980; Mertikopoulos et al., 2019), only apply to exponentially small learning rate thus not reﬂecting the behavior of the algorithms in practice (Daskalakis & Panageas, 2019b), or require additional conditions such as uniqueness of the equilibrium or a good initialization (Daskalakis & Panageas, 2019b).

Original: Motivated by this fact, in this work, we ﬁrst improve the last-iterate convergence result of OMWU for matrix games.
Revised: Motivated by this fact, in this work, we ﬁrst improve the last-iterate convergence result of OMWU for matrix games.

Original: Under the same unique equilibrium assumption as made by Daskalakis &
Revised: Under the same unique equilibrium assumption as made by Daskalakis &

Original: Panageas (2019b)
Revised: Panageas (2019b)

Original: , we show linear convergence with a concrete rate in terms of the Kullback-Leibler divergence between the last iterate and the equilibrium, using aconstant learning rate.
Revised: , we show linear convergence with a concrete rate in terms of the Kullback-Leibler divergence between the last iterate and the equilibrium, using a learning rate whose value is set to a universal constant .

Original: We then signiﬁcantly extend our results and consider OGDA for general constrained and smooth convex-concave saddle-point problems, without the uniqueness assumption.
Revised: We then signiﬁcantly extend our results and consider OGDA for general constrained and smooth convex-concave saddle-point problems, without the uniqueness assumption.

Original: Speciﬁcally, we start with proving an average duality gap convergence of OGDA at the rate of O (1 / √ T ) after T iterations.
Revised: Speciﬁcally, we start with proving an average duality gap convergence of OGDA at the rate of O (1 / √ T ) after T iterations.

Original: Then, to obtain a more favorable last-iterate convergence in terms of the distance to the set of equilibria, we propose a general sufﬁcient condition on X , Y , and f , called
Revised: Then, to obtain a more favorable last-iterate convergence in terms of the distance to the set of equilibria, we propose a general sufﬁcient condition on X , Y , and f , called

Original: Saddle-Point Metric Subregularity (SP-MS), under which we prove concrete last-iterate convergence rates, all with a constant learning rate and without further assumptions.
Revised: Saddle-Point Metric Subregularity (SP-MS), under which we prove concrete last-iterate convergence rates, all with a constant learning rate and without further assumptions.

Original: Our last-iterate convergence results of OGDA greatly generalize that of (Hsieh et al., 2019, Theorem 2), which itself is a consolidated version of results from several earlier works.
Revised: Our last-iterate convergence results of OGDA greatly generalize that of (Hsieh et al., 2019, Theorem 2), which itself is a consolidated version of results from several earlier works.

Original: The key implication of our new results is that, by showing that matrix games satisfy our SP-MS condition, we provide by far the most general last-iterate guarantee with a linear convergence for this problem using OGDA.
Revised: The key implication of our new results is that, by showing that matrix games satisfy our SP-MS condition, we provide by far the most general last-iterate guarantee with a linear convergence for this problem using OGDA.

Original: Compared to that of OMWU, the convergence result of OGDA holds more generally even when there are multiple equilibria.
Revised: Compared to that of OMWU, the convergence result of OGDA holds more generally even when there are multiple equilibria.

Original: More generally, the same linear last-iterate convergence holds for any bilinear games over polytopes since they also satisfy the SP-MS condition as we show.
Revised: More generally, the same linear last-iterate convergence holds for any bilinear games over polytopes since they also satisfy the SP-MS condition as we show.

Original: To complement this result, we construct an example of a bilinear game with a non-polytope feasible set where OGDA provably does not ensure linear convergence, indicating that the shape of the feasible set matters .
Revised: To complement this result, we construct an example of a bilinear game with a non-polytope feasible set where OGDA provably does not ensure linear convergence, indicating that the shape of the feasible set matters .

Original: Finally, we also provide experimental results to support our theory.
Revised: Finally, we also provide experimental results to support our theory.

Original: In particular, we observe that
Revised: In particular, we observe that

Original: OGDA generally converges faster than OMWU for matrix games, despite the facts that both provably converge exponentially fast and that OMWU is often considered more favorable compared to
Revised: OGDA generally converges faster than OMWU for matrix games, despite the facts that both provably converge exponentially fast and that OMWU is often considered more favorable compared to

Original: OGDA when the feasible set is the simplex.
Revised: OGDA when the feasible set is the simplex.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Average-iterate convergence.
Revised: Average-iterate convergence.

Original: While showing last-iterate convergence has been a challenging task, it is well-known that the average-iterate of many standard algorithms such as GDA and MWU enjoys a converging duality gap at the rate of O (1 / √ T ) (Freund & Schapire, 1999).
Revised: While showing last-iterate convergence has been a challenging task, it is well-known that the average-iterate of many standard algorithms such as GDA and MWU enjoys a converging duality gap at the rate of O (1 / √ T ) (Freund & Schapire, 1999).

Original: A line of works show that the rate can be improved to O (1 /T ) using the “optimistic” version of these algorithms such as OGDA and OMWU (Rakhlin & Sridharan, 2013; Daskalakis et al., 2015; Syrgkanis et al., 2015).
Revised: A line of works show that the rate can be improved to O (1 /T ) using the “optimistic” version of these algorithms such as OGDA and OMWU (Rakhlin & Sridharan, 2013; Daskalakis et al., 2015; Syrgkanis et al., 2015).

Original: For tasks such as training GANs, however, average-iterate convergence is unsatisfactory since averaging large neural networks is usually prohibited.
Revised: For tasks such as training GANs, however, average-iterate convergence is unsatisfactory since averaging large neural networks is usually prohibited.

Original: Extra-Gradient (EG) algorithms.
Revised: Extra-Gradient (EG) algorithms.

Original: The saddle-point problem ﬁts into the more general variational inequality framework (Harker & Pang, 1990).
Revised: The saddle-point problem ﬁts into the more general variational inequality framework (Harker & Pang, 1990).

Original: A classic algorithm for variational inequalities is EG, ﬁrst introduced in (Korpelevich, 1976).
Revised: A classic algorithm for variational inequalities is EG, ﬁrst introduced in (Korpelevich, 1976).

Original: Tseng (1995) is the ﬁrst to prove last-iterate convergence for EG in various settings such as bilinear orsmooth strongly-convex-strongly-concave problems.
Revised: Tseng (1995) is the ﬁrst to show last-iterate convergence for EG in various settings such as bilinear or strongly-convex-strongly-concave problems.

Original: Recent works signiﬁcantly expand the understanding of EG and its variants for unconstrained bilinear problems (Liang & Stokes, 2019), unconstrained strongly-convex-strongly-concave problems (Mokhtari et al., 2019b), and more (Zhang et al., 2019; Lin et al., 2020; Golowich et al., 2020b).
Revised: Recent works signiﬁcantly expand the understanding of EG and its variants for unconstrained bilinear problems (Liang & Stokes, 2019), unconstrained strongly-convex-strongly-concave problems (Mokhtari et al., 2020b), and more (Zhang et al., 2019; Lin et al., 2020; Golowich et al., 2020b).

Original: The original EG is not applicable to a repeated game setting where only one gradient evaluation is possible in each iteration.
Revised: The original EG is not applicable to a repeated game setting where only one gradient evaluation is possible in each iteration.

Original: Moreover, unlike OGDA and OMWU, EG is shown to have linear regret against adversarial opponents, and thus it is not a no-regret learning algorithm
Revised: Moreover, unlike OGDA and OMWU, EG is shown to have linear regret against adversarial opponents, and thus it is not a no-regret learning algorithm

Original: (Bowling, 2005;
Revised: (Bowling, 2005;

Original: Golowich et al., 2020a).
Revised: Golowich et al., 2020a).

Original: However, there are “single-call variants” of EG that address these issues.
Revised: However, there are “single-call variants” of EG that address these issues.

Original: In fact, some of these versions coincide with the OGDA algorithm under different names such as modiﬁed Arrow–Hurwicz method (Popov, 1980) and “extrapolation from the past” (Gidel et al., 2019).
Revised: In fact, some of these versions coincide with the OGDA algorithm under different names such as modiﬁed Arrow–Hurwicz method (Popov, 1980) and “extrapolation from the past” (Gidel et al., 2019).

Original: Apart from OGDA, other single-call variants of EG include Reﬂected Gradient (Malitsky, 2015; Cui & Shanbhag, 2016; Malitsky & Tam) and Optimistic Gradient (Daskalakis et al., 2018;
Revised: Apart from OGDA, other single-call variants of EG include Reﬂected Gradient (Malitsky, 2015; Cui & Shanbhag, 2016; Malitsky & Tam) and Optimistic Gradient (Daskalakis et al., 2018;

Original: Mokhtari et al., 2019a).
Revised: Mokhtari et al., 2020a).

Original: These variants are all equivalent in the unconstrained setting but differ in the constrained setting.
Revised: These variants are all equivalent in the unconstrained setting but differ in the constrained setting.

Original: To the best of our knowledge, none of the existing results for any single-call variant of EG covers the constrained bilinear case (which is one of our key contributions).
Revised: To the best of our knowledge, none of the existing results for any single-call variant of EG covers the constrained bilinear case (which is one of our key contributions).

Original: Error Bounds and Metric Subregularity To derive linear convergence for variational inequality
Revised: Error Bounds and Metric Subregularity To derive linear convergence for variational inequality

Original: problems, error bound method is a commonly used technique (Pang, 1997; Luo & Tseng, 1993).
Revised: problems, error bound method is a commonly used technique (Pang, 1997; Luo & Tseng, 1993).

Original: For example, it is a standard approach to studying the last-iterate convergence of EG algorithms (Tseng, 1995; Hsieh et al., 2020; Azizian et al., 2020).
Revised: For example, it is a standard approach to studying the last-iterate convergence of EG algorithms (Tseng, 1995; Hsieh et al., 2020; Azizian et al., 2020).

Original: An error bound method is associated with an error function that gives every point in the feasible set a measure of sub-optimality that is lower bounded by the distance of the point to the optimal set up to some problem dependent constant.
Revised: An error bound method is associated with an error function that gives every point in the feasible set a measure of sub-optimality that is lower bounded by the distance of the point to the optimal set up to some problem dependent constant.

Original: If such a error function exists, linear convergence can be obtained.
Revised: If such a error function exists, linear convergence can be obtained.

Original: The choice of the error function depends on the feasible region, the objection function, and the algorithm.
Revised: The choice of the error function depends on the feasible region, the objection function, and the algorithm.

Original: Common error functions include natural residual functions (Iusem et al., 2017; Malitsky, 2019) and gap functions (Larsson &
Revised: Common error functions include natural residual functions (Iusem et al., 2017; Malitsky, 2019) and gap functions (Larsson &

Original: Patriksson, 1994; Solodov & Tseng, 2000; Chen et al., 2017).
Revised: Patriksson, 1994; Solodov & Tseng, 2000; Chen et al., 2017).

Original: Our method to derive the last-iterate convergence for OGDA can also be viewed as an error bound method.
Revised: Our method to derive the last-iterate convergence for OGDA can also be viewed as an error bound method.

Original: Metric subregularity is another important concept to derive linear convergence via some Lipschitz behavior of a set-valued operator (Leventhal, 2009; Liang et al., 2016; Alacaoglu et al., 2019; Latafat et al., 2019).
Revised: Metric subregularity is another important concept to derive linear convergence via some Lipschitz behavior of a set-valued operator (Leventhal, 2009; Liang et al., 2016; Alacaoglu et al., 2019; Latafat et al., 2019).

Original: Metric subregularity is closely related to error bound methods (Kruger, 2015).
Revised: Metric subregularity is closely related to error bound methods (Kruger, 2015).

Original: In fact, as we prove in Appendix F, one special case of our condition SP-MS (that allows us to show linear convergence) is equivalent to metric subregularity of an operator deﬁned in terms of the normal cone of the feasible set and the gradient of the objective.
Revised: In fact, as we prove in Appendix F, one special case of our condition SP-MS (that allows us to show linear convergence) is equivalent to metric subregularity of an operator deﬁned in terms of the normal cone of the feasible set and the gradient of the objective.

Original: This is also the reason why we call our condition
Revised: This is also the reason why we call our condition

Original: Saddle-Point Metric Subregularity.
Revised: Saddle-Point Metric Subregularity.

Original: Although metric subregularity has been extensively used in the literature, to the best of our knowledge, our work is the ﬁrst to use this condition to analyze OGDA.
Revised: Although metric subregularity has been extensively used in the literature, to the best of our knowledge, our work is the ﬁrst to use this condition to analyze OGDA.

Original: OGDA and OMWU.
Revised: OGDA and OMWU.

Original: Recently, last-iterate convergence for OGDA has been proven in various settings such as convex-concave problems (Daskalakis et al., 2018), unconstrained bilinear problems (Daskalakis & Panageas, 2018; Liang & Stokes, 2019), strongly-convex-strongly-concave problems (Mokhtari et al., 2019b), and others (e.g. (Mertikopoulos et al., 2019)).
Revised: Recently, last-iterate convergence for OGDA has been proven in various settings such as convex-concave problems (Daskalakis et al., 2018), unconstrained bilinear problems (Daskalakis & Panageas, 2018; Liang & Stokes, 2019), strongly-convex-strongly-concave problems (Mokhtari et al., 2020b), and others (e.g. (Mertikopoulos et al., 2019)).

Original: However, the behavior of OGDA and OMWU for the constrained bilinear case, or even the special case of classic matrix games, appears to be much more mysterious and less understood.
Revised: However, the behavior of OGDA and OMWU for the constrained bilinear case, or even the special case of classic matrix games, appears to be much more mysterious and less understood.

Original: Cheung &
Revised: Cheung &

Original: Piliouras (2020) provide an alternative view on the convergence behavior of OMWU by studying volume contraction in the dual space.
Revised: Piliouras (2020) provide an alternative view on the convergence behavior of OMWU by studying volume contraction in the dual space.

Original: Daskalakis & Panageas (2019b) show last-iterate convergence of OMWU for matrix games under a uniqueness assumption and without a concrete rate.
Revised: Daskalakis & Panageas (2019b) show last-iterate convergence of OMWU for matrix games under a uniqueness assumption and without a concrete rate.

Original: Although it is implicitly suggested in (Daskalakis & Panageas, 2019b;a) that a rate of O (1 /T 1 / 9 ) is possible, it is still not clear how to choose the learning rate appropriately from their analysis.
Revised: Although it is implicitly suggested in (Daskalakis & Panageas, 2019b;a) that a rate of O (1 /T 1 / 9 ) is possible, it is still not clear how to choose the learning rate appropriately from their analysis.

Original: As mentioned, our results for OMWU signiﬁcantly improve theirs, with a clean linear convergence rate using a constant learning rate under the same uniqueness assumption, while our results for OGDA further remove the uniqueness assumption.
Revised: As mentioned, our results for OMWU signiﬁcantly improve theirs, with a clean linear convergence rate using a constant learning rate under the same uniqueness assumption, while our results for OGDA further remove the uniqueness assumption.

Original:  N OTATIONS AND P RELIMINARIES
Revised:  N OTATIONS AND P RELIMINARIES

Original: We consider the following constrained saddle-point problem: min x ∈X max y ∈Y f
Revised: We consider the following constrained saddle-point problem: min x ∈X max y ∈Y f

Original: ( x , y ) , where X and Y are closed convex sets, and f is a continuous differentiable function that is convex in x for any ﬁxed y and concave in y for any ﬁxed x .
Revised: ( x , y ) , where X and Y are closed convex sets, and f is a continuous differentiable function that is convex in x for any ﬁxed y and concave in y for any ﬁxed x .

Original: By the celebrated minimax theorem (Neumann, 1928), we have min x ∈X max y ∈Y f
Revised: By the celebrated minimax theorem (Neumann, 1928), we have min x ∈X max y ∈Y f

Original: ( x , y ) = max y ∈Y min x ∈X f ( x , y ) .
Revised: ( x , y ) = max y ∈Y min x ∈X f ( x , y ) .

Original: The set of minimax optimal strategy is denoted by X ∗ = argmin x ∈X max y ∈Y f ( x , y ) , and the set of maximin optimal strategy is denoted by Y ∗ =
Revised: The set of minimax optimal strategy is denoted by X ∗ = argmin x ∈X max y ∈Y f ( x , y ) , and the set of maximin optimal strategy is denoted by Y ∗ =

Original: argmax y ∈Y min x ∈X f ( x , y ) .
Revised: argmax y ∈Y min x ∈X f ( x , y ) .

Original: It is well-known that X ∗ and Y ∗ are convex, and any pair ( x ∗ , y ∗ ) ∈
Revised: It is well-known that X ∗ and Y ∗ are convex, and any pair ( x ∗ , y ∗ ) ∈

Original: X ∗ × Y ∗ is a Nash equilibrium satisfying f ( x ∗ , y ) ≤ f ( x ∗ , y ∗ ) ≤ f ( x , y ∗ ) for any ( x , y ) ∈
Revised: X ∗ × Y ∗ is a Nash equilibrium satisfying f ( x ∗ , y ) ≤ f ( x ∗ , y ∗ ) ≤ f ( x , y ∗ ) for any ( x , y ) ∈

Original: X × Y .
Revised: X × Y .

Original: For notational convenience, we deﬁne Z = X
Revised: For notational convenience, we deﬁne Z = X

Original: × Y and similarly Z ∗ =
Revised: × Y and similarly Z ∗ =

Original: X ∗ × Y ∗ .
Revised: X ∗ × Y ∗ .

Original: For a point z =
Revised: For a point z =

Original: ( x , y ) ∈
Revised: ( x , y ) ∈

Original: Z , we further deﬁne f ( z ) =
Revised: Z , we further deﬁne f ( z ) =

Original: f
Revised: f

Original: ( x , y ) and F ( z ) = ( ∇ x f ( x , y ) , −∇ y f ( x , y )) .
Revised: ( x , y ) and F ( z ) = ( ∇ x f ( x , y ) , −∇ y f ( x , y )) .

Original: Our goal is to ﬁnd a point z ∈ Z that is close to the set of Nash equilibria Z ∗ , and we consider three ways of measuring the closeness.
Revised: Our goal is to ﬁnd a point z ∈ Z that is close to the set of Nash equilibria Z ∗ , and we consider three ways of measuring the closeness.

Original: The ﬁrst one is the duality gap , deﬁned as α f ( z ) = max y (cid:48)
Revised: The ﬁrst one is the duality gap , deﬁned as α f ( z ) = max y (cid:48)

Original: ∈Y f ( x , y (cid:48) )
Revised: ∈Y f ( x , y (cid:48) )

Original: − min x (cid:48)
Revised: − min x (cid:48)

Original: ∈X f ( x (cid:48) , y ) , which is always non-negative since max y (cid:48)
Revised: ∈X f ( x (cid:48) , y ) , which is always non-negative since max y (cid:48)

Original: ∈Y f ( x , y (cid:48) ) ≥ f ( x , y ) ≥ min x (cid:48) ∈X f ( x (cid:48) , y ) .
Revised: ∈Y f ( x , y (cid:48) ) ≥ f ( x , y ) ≥ min x (cid:48) ∈X f ( x (cid:48) , y ) .

Original: The second one is the distance between z and Z ∗ .
Revised: The second one is the distance between z and Z ∗ .

Original: Speciﬁcally, for any closed set A , we deﬁne the projection operator Π A as Π A ( a ) =
Revised: Speciﬁcally, for any closed set A , we deﬁne the projection operator Π A as Π A ( a ) =

Original: argmin a (cid:48) ∈A (cid:107) a − a (cid:48) (cid:107) (throughout this work (cid:107)·(cid:107) represents Lnorm).
Revised: argmin a (cid:48) ∈A (cid:107) a − a (cid:48) (cid:107) (throughout this work (cid:107)·(cid:107) represents Lnorm).

Original: The squared distance between z and Z ∗ is then deﬁned as dist 2 ( z , Z ∗ )
Revised: The squared distance between z and Z ∗ is then deﬁned as dist 2 ( z , Z ∗ )

Original: = (cid:107)
Revised: = (cid:107)

Original: z − Π Z ∗ ( z ) (cid:107) 2 .
Revised: z − Π Z ∗ ( z ) (cid:107) 2 .

Original: The third one is only for the case when X and Y are probability simplices, and z ∗ =
Revised: The third one is only for the case when X and Y are probability simplices, and z ∗ =

Original: ( x ∗ , y ∗ ) is the unique equilibrium.
Revised: ( x ∗ , y ∗ ) is the unique equilibrium.

Original: In this case, we use the sum of Kullback-Leibler divergence KL ( x ∗ , x ) +
Revised: In this case, we use the sum of Kullback-Leibler divergence KL ( x ∗ , x ) +

Original: KL ( y ∗ , y ) to measure the closeness between z = ( x , y ) and z ∗ , where KL ( x , x (cid:48) )
Revised: KL ( y ∗ , y ) to measure the closeness between z = ( x , y ) and z ∗ , where KL ( x , x (cid:48) )

Original: =
Revised: =

Original: (cid:80) i x i ln x i x (cid:48)
Revised: (cid:80) i x i ln x i x (cid:48)

Original: i .
Revised: i .

Original: With a slight abuse of notation, we use KL ( z , z (cid:48) ) to denote KL ( x , x (cid:48) )
Revised: With a slight abuse of notation, we use KL ( z , z (cid:48) ) to denote KL ( x , x (cid:48) )

Original: + KL ( y , y (cid:48) ) .
Revised: + KL ( y , y (cid:48) ) .

Original: Other notations.
Revised: Other notations.

Original: We denote the ( d − 1) -dimensional
Revised: We denote the ( d − 1) -dimensional

Original: probability simplex as ∆ d = { u ∈ R d + : (cid:80) di =1 u i = 1 } .
Revised: probability simplex as ∆ d = { u ∈ R d + : (cid:80) di =1 u i = 1 } .

Original: For a convex function ψ , the corresponding Bregman divergence is deﬁned as
Revised: For a convex function ψ , the corresponding Bregman divergence is deﬁned as

Original: D ψ ( u , v ) =
Revised: D ψ ( u , v ) =

Original: ψ ( u ) − ψ
Revised: ψ ( u ) − ψ

Original: ( v ) −(cid:104)∇ ψ ( v ) , u − v (cid:105) .
Revised: ( v ) −(cid:104)∇ ψ ( v ) , u − v (cid:105) .

Original: If ψ is γ -strongly convex in a domain, then D ψ ( u , v ) ≥ γ 2 (cid:107) u − v (cid:107) 2 for any u , v in that domain.
Revised: If ψ is γ -strongly convex in a domain, then D ψ ( u , v ) ≥ γ 2 (cid:107) u − v (cid:107) 2 for any u , v in that domain.

Original: For u ∈ R d , we deﬁne supp ( u ) =
Revised: For u ∈ R d , we deﬁne supp ( u ) =

Original: { i : u i > 0 } .
Revised: { i : u i > 0 } .

Original: Optimistic Gradient Descent Ascent (OGDA).
Revised: Optimistic Gradient Descent Ascent (OGDA).

Original: Starting from an arbitrary point ( (cid:98) x 1 , (cid:98) y 1 ) =
Revised: Starting from an arbitrary point ( (cid:98) x 1 , (cid:98) y 1 ) =

Original: ( x 0 , y 0 ) from Z , OGDA with step size η > 0 iteratively computes the following for t = 1 , 2 , . . .
Revised: ( x 0 , y 0 ) from Z , OGDA with step size η > 0 iteratively computes the following for t = 1 , 2 , . . .

Original: [Equation]
Revised: [Equation]

Original: Note that there are several slightly different versions of the algorithm in the literature, which differ in the timing of performing the projection.
Revised: Note that there are several slightly different versions of the algorithm in the literature, which differ in the timing of performing the projection.

Original: Our version is the same as those in (Chiang et al., 2012;
Revised: Our version is the same as those in (Chiang et al., 2012;

Original: Rakhlin & Sridharan, 2013).
Revised: Rakhlin & Sridharan, 2013).

Original: It is also referred to as “single-call extra-gradient” in (Hsieh et al., 2019), but it does not belong to the class of “extra-gradient” methods discussed in (Tseng, 1995;
Revised: It is also referred to as “single-call extra-gradient” in (Hsieh et al., 2019), but it does not belong to the class of “extra-gradient” methods discussed in (Tseng, 1995;

Original: Liang & Stokes, 2019; Golowich et al., 2020b) for example.
Revised: Liang & Stokes, 2019; Golowich et al., 2020b) for example.

Original: Also note that OGDA only requires accessing f via its gradient.
Revised: Also note that OGDA only requires accessing f via its gradient.

Original: In fact, only one gradient at the point ( x t , y t ) is needed for iteration t .
Revised: In fact, only one gradient at the point ( x t , y t ) is needed for iteration t .

Original: This aspect makes it especially suitable for a repeated game setting, where in each round, one player proposes x t while another player proposes y t .
Revised: This aspect makes it especially suitable for a repeated game setting, where in each round, one player proposes x t while another player proposes y t .

Original: With only the information of the gradient from the environment ( ∇ x f ( x t , y t ) for the ﬁrst player and ∇ y f ( x t , y t ) for the other), both players can execute the algorithm.
Revised: With only the information of the gradient from the environment ( ∇ x f ( x t , y t ) for the ﬁrst player and ∇ y f ( x t , y t ) for the other), both players can execute the algorithm.

Original: Optimistic Multiplicative Weights Update (OMWU).
Revised: Optimistic Multiplicative Weights Update (OMWU).

Original: When the feasible sets X and Y are probability simplices ∆ M and ∆ N for some integers M and N , OMWU is another common iterative algorithm to solve the saddle-point problem.
Revised: When the feasible sets X and Y are probability simplices ∆ M and ∆ N for some integers M and N , OMWU is another common iterative algorithm to solve the saddle-point problem.

Original: For simplicity, we assume that it starts from the uniform distributions ( (cid:98) x 1 , (cid:98) y 1 ) =
Revised: For simplicity, we assume that it starts from the uniform distributions ( (cid:98) x 1 , (cid:98) y 1 ) =

Original: ( x 0 , y 0 ) = (cid:0) 1 M M , 1 N N (cid:1) , where 1 d is the all-one vector of dimension d .
Revised: ( x 0 , y 0 ) = (cid:0) 1 M M , 1 N N (cid:1) , where 1 d is the all-one vector of dimension d .

Original: Then OMWU with step size η > 0 iteratively computes the following for t = 1 , 2 , . . .
Revised: Then OMWU with step size η > 0 iteratively computes the following for t = 1 , 2 , . . .

Original: , x 
Revised: , x t,i =

Original: [Equation]
Revised: [Equation]

Original: OMWU and OGDA as Optimistic Mirror Descent Ascent
Revised: OMWU and OGDA as Optimistic Mirror Descent Ascent

Original: OMWU and OGDA can be viewed as special cases of Optimistic Mirror Descent Ascent .
Revised: OMWU and OGDA can be viewed as special cases of Optimistic Mirror Descent Ascent .

Original: Speciﬁcally, let regularizer ψ ( u ) denote the negative entropy (cid:80)
Revised: Speciﬁcally, let regularizer ψ ( u ) denote the negative entropy (cid:80)

Original: i u i
Revised: i u i

Original: ln u
Revised: ln u

Original: i for the case of OMWU and (half of) the L 2 norm square 12 (cid:107) u (cid:107) 2 for the case of OGDA (so that D ψ ( u , v ) is KL ( u , v ) and 12 (cid:107) u − v (cid:107) 2 respectively).
Revised: i for the case of OMWU and (half of) the L 2 norm square 12 (cid:107) u (cid:107) 2 for the case of OGDA (so that D ψ ( u , v ) is KL ( u , v ) and 12 (cid:107) u − v (cid:107) 2 respectively).

Original: Then using the shorthands z t = ( x t , y t ) and (cid:98) z t = ( (cid:98) x t , (cid:98) y t ) and recalling the notation deﬁned earlier:
Revised: Then using the shorthands z t = ( x t , y t ) and (cid:98) z t = ( (cid:98) x t , (cid:98) y t ) and recalling the notation deﬁned earlier:

Original: Z = X
Revised: Z = X

Original: × Y and F ( z )
Revised: × Y and F ( z )

Original: =
Revised: =

Original: ( ∇ x f ( x , y ) , −∇ y f ( x , y )) , one can rewrite OMWU/OGDA compactly as
Revised: ( ∇ x f ( x , y ) , −∇ y f ( x , y )) , one can rewrite OMWU/OGDA compactly as

Original: [Equation]
Revised: [Equation]

Original: By the standard regret analysis of Optimistic Mirror Descent, we have the following important lemma, which is readily applied to OMWU and OGDA when ψ is instantiated as the corresponding regularizer.
Revised: By the standard regret analysis of Optimistic Mirror Descent, we have the following important lemma, which is readily applied to OMWU and OGDA when ψ is instantiated as the corresponding regularizer.

Original: The proof is mostly standard (see e.g., (Rakhlin & Sridharan, 2013, Lemma 1)).
Revised: The proof is mostly standard (see e.g., (Rakhlin & Sridharan, 2013, Lemma 1)).

Original: For completeness, we include it in Appendix B.
Revised: For completeness, we include it in Appendix B.

Original: Lemma 1.
Revised: Lemma 1.

Original: Consider update rules Eq.
Revised: Consider update rules Eq.

Original: (1) and Eq.
Revised: (1) and Eq.

Original: (2) and deﬁne dist 2 p ( z , z (cid:48) )
Revised: (2) and deﬁne dist 2 p ( z , z (cid:48) )

Original: = (cid:107) x − x (cid:48) (cid:107) 2 p + (cid:107)
Revised: = (cid:107) x − x (cid:48) (cid:107) 2 p + (cid:107)

Original: y − y (cid:48) (cid:107) 2 p .
Revised: y − y (cid:48) (cid:107) 2 p .

Original: Suppose that ψ satisﬁes D ψ ( z , z (cid:48) ) ≥ 12 dist 2 p ( z , z (cid:48) ) for some p ≥ 1 , and F satisﬁes dist 2 q ( F ( z ) , F ( z (cid:48) ))
Revised: Suppose that ψ satisﬁes D ψ ( z , z (cid:48) ) ≥ 12 dist 2 p ( z , z (cid:48) ) for some p ≥ 1 , and F satisﬁes dist 2 q ( F ( z ) , F ( z (cid:48) ))

Original: ≤ L 2 dist 2 p ( z , z (cid:48) ) for q ≥ 1 with 1 p + 1 q = 1 .
Revised: ≤ L 2 dist 2 p ( z , z (cid:48) ) for q ≥ 1 with 1 p + 1 q = 1 .

Original: Also, assume that η ≤ 18 L .
Revised: Also, assume that η ≤ 18 L .

Original: Then for any z ∈ Z and any t ≥ 1 , we have
Revised: Then for any z ∈ Z and any t ≥ 1 , we have

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  C ONVERGENCE R ESULTS FOR OMWU
Revised:  C ONVERGENCE R ESULTS FOR OMWU

Original: In this section, we show that for a two-player zero-sum matrix game with a unique equilibrium,
Revised: In this section, we show that for a two-player zero-sum matrix game with a unique equilibrium,

Original: OMWU with a constant learning rate converges to the equilibrium exponentially fast.
Revised: OMWU with a constant learning rate converges to the equilibrium exponentially fast.

Original: The assumption and the algorithm are the same as those considered in (Daskalakis & Panageas, 2019b), but our analysis improves theirs in two ways.
Revised: The assumption and the algorithm are the same as those considered in (Daskalakis & Panageas, 2019b), but our analysis improves theirs in two ways.

Original: First, we do not require the learning rate to be exponentially smaller than some problem-dependent quantity.
Revised: First, we do not require the learning rate to be exponentially smaller than some problem-dependent quantity.

Original: Second, we explicitly provide a linear convergence rate.
Revised: Second, we explicitly provide a linear convergence rate.

Original: In Section 5, we further remove the uniqueness assumption and signiﬁcantly generalize the results by studying OGDA.
Revised: In Section 5, we further remove the uniqueness assumption and signiﬁcantly generalize the results by studying OGDA.

Original: In a matrix game we have X = ∆ M , Y = ∆ N , and f ( z ) =
Revised: In a matrix game we have X = ∆ M , Y = ∆ N , and f ( z ) =

Original: x (cid:62) Gy for some matrix
Revised: x (cid:62) Gy for some matrix

Original: G ∈ [ − 1 , 1] M × N .
Revised: G ∈ [ − 1 , 1] M × N .

Original: To show the last-iterate convergence of OMWU, we ﬁrst apply Lemmawith D ψ ( u , v ) =
Revised: To show the last-iterate convergence of OMWU, we ﬁrst apply Lemmawith D ψ ( u , v ) =

Original: KL ( u , v ) , z = z ∗ (the unique equilibrium of the game matrix G ) and ( p, q ) =
Revised: KL ( u , v ) , z = z ∗ (the unique equilibrium of the game matrix G ) and ( p, q ) =

Original: (1 , ∞ ) .
Revised: (1 , ∞ ) .

Original: The constant L can be chosen as 1 since dist 2 ∞ ( F ( z ) , F ( z (cid:48) ))
Revised: The constant L can be chosen as 1 since dist 2 ∞ ( F ( z ) , F ( z (cid:48) ))

Original: =
Revised: =

Original: max i | ( G ( y − y (cid:48) ))
Revised: max i | ( G ( y − y (cid:48) ))

Original: i | 2 + max j | ( G (cid:62) ( x − x (cid:48) ))
Revised: i | 2 + max j | ( G (cid:62) ( x − x (cid:48) ))

Original: j | 2 ≤ (cid:107) y − y (cid:48) (cid:107) 

Revised: j | 2 ≤ (cid:107) y − y (cid:48) (cid:107)

Original: + (cid:107) x − x (cid:48) (cid:107) 21 = dist 21 ( z , z (cid:48) ) .
Revised: + (cid:107) x − x (cid:48) (cid:107) 21 = dist 21 ( z , z (cid:48) ) .

Original: Also notice that
Revised: Also notice that

Original: F ( z t ) (cid:62) ( z t − z ∗ ) =
Revised: F ( z t ) (cid:62) ( z t − z ∗ ) =

Original: f ( x t , y t ) − f ( x ∗ , y t )
Revised: f ( x t , y t ) − f ( x ∗ , y t )

Original: + f ( x t , y ∗ ) − f ( x t , y t )
Revised: + f ( x t , y ∗ ) − f ( x t , y t )

Original: = f
Revised: = f

Original: ( x t , y ∗ ) − f ( x ∗ , y t ) ≥by the optimality of z ∗ .
Revised: ( x t , y ∗ ) − f ( x ∗ , y t ) ≥by the optimality of z ∗ .

Original: Therefore, we have when η ≤ 18 , 
Revised: Therefore, we have when η ≤ 18 , KL

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] 
Revised: [Equation] [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: Deﬁning Θ t = KL ( z ∗ , (cid:98) z t )
Revised: Deﬁning Θ t = KL ( z ∗ , (cid:98) z t )

Original: + 116 KL ( (cid:98) z t , z t − 1 ) and ζ t = KL 
Revised: + 116 KL ( (cid:98) z t , z t − 1 ) and ζ t = KL (

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) , we rewrite the above as
Revised: ) , we rewrite the above as

Original: [Equation]
Revised: [Equation]

Original: From Eq.
Revised: From Eq.

Original: (3) it is clear that the quantity Θ t is always non-increasing in t due to the non-negativity of ζ t .
Revised: (3) it is clear that the quantity Θ t is always non-increasing in t due to the non-negativity of ζ t .

Original: Furthermore, the more the algorithm moves between round t and round t + 1 (that is, the larger ζ t is), the more Θ t decreases.
Revised: Furthermore, the more the algorithm moves between round t and round t + 1 (that is, the larger ζ t is), the more Θ t decreases.

Original: To establish the rate of convergence, a natural idea is to relate ζ t back to Θ t or Θ t +1 .
Revised: To establish the rate of convergence, a natural idea is to relate ζ t back to Θ t or Θ t +1 .

Original: For example, if we can show ζ t ≥ c Θ t +1 for some constant c > 0 , then Eq.
Revised: For example, if we can show ζ t ≥ c Θ t +1 for some constant c > 0 , then Eq.

Original: (3) implies Θ t +1 ≤ Θ t − 15 c 16 Θ t +1 , which further gives Θ t +1 ≤ (cid:0) 1 + 15 c 16 (cid:1) − 1 Θ t .
Revised: (3) implies Θ t +1 ≤ Θ t − 15 c 16 Θ t +1 , which further gives Θ t +1 ≤ (cid:0) 1 + 15 c 16 (cid:1) − 1 Θ t .

Original: This immediately implies a linear convergence rate for Θ t as well as KL ( z ∗ , (cid:98) z t ) since KL ( z ∗ , (cid:98) z t ) ≤ Θ t .
Revised: This immediately implies a linear convergence rate for Θ t as well as KL ( z ∗ , (cid:98) z t ) since KL ( z ∗ , (cid:98) z t ) ≤ Θ t .

Original: Moreover, notice that to ﬁnd such c , it sufﬁces to ﬁnd a c (cid:48) > 0 such that ζ t ≥ c (cid:48) KL ( z ∗ , (cid:98)
Revised: Moreover, notice that to ﬁnd such c , it sufﬁces to ﬁnd a c (cid:48) > 0 such that ζ t ≥ c (cid:48) KL ( z ∗ , (cid:98)

Original: z t +1 ) .
Revised: z t +1 ) .

Original: This is because it will then give ζ t ≥ 116 KL ( (cid:98) z t +1 , z t )
Revised: This is because it will then give ζ t ≥ 116 KL ( (cid:98) z t +1 , z t )

Original: + 1516 ζ t ≥ 116 KL ( (cid:98)
Revised: + 1516 ζ t ≥ 116 KL ( (cid:98)

Original: z t +1 , z t ) + 15 c (cid:48)
Revised: z t +1 , z t ) + 15 c (cid:48)

Original: 16 KL ( z ∗ , (cid:98)
Revised: 16 KL ( z ∗ , (cid:98)

Original: z t +1 ) ≥ min { 1 , 15 c (cid:48) 16 } Θ t +1 , and thus c (cid:44) min { 1 , 15 c (cid:48) 16 } satisﬁes the condition.
Revised: z t +1 ) ≥ min { 1 , 15 c (cid:48) 16 } Θ t +1 , and thus c (cid:44) min { 1 , 15 c (cid:48) 16 } satisﬁes the condition.

Original: From the discussion above, we see that to establish the linear convergence of KL ( z ∗ , (cid:98) z t ) , we only need to show that there exists some c (cid:48) > 0 such that KL ( (cid:98) z t +1 , z t )
Revised: From the discussion above, we see that to establish the linear convergence of KL ( z ∗ , (cid:98) z t ) , we only need to show that there exists some c (cid:48) > 0 such that KL ( (cid:98) z t +1 , z t )

Original: + KL ( z t , (cid:98) z t ) ≥ c (cid:48)
Revised: + KL ( z t , (cid:98) z t ) ≥ c (cid:48)

Original: KL ( z ∗ , (cid:98)
Revised: KL ( z ∗ , (cid:98)

Original: z t +1 ) .
Revised: z t +1 ) .

Original: The high-level interpretation of this inequality is that when (cid:98) z t +1 is far from the equilibrium z ∗ (i.e.,
Revised: The high-level interpretation of this inequality is that when (cid:98) z t +1 is far from the equilibrium z ∗ (i.e.,

Original: KL ( z ∗ , (cid:98) z t +1 ) is large), the algorithm should have a large move between round t and t + 1 making
Revised: KL ( z ∗ , (cid:98) z t +1 ) is large), the algorithm should have a large move between round t and t + 1 making

Original: KL ( (cid:98) z t +1 , z t ) + KL ( z t , (cid:98)
Revised: KL ( (cid:98) z t +1 , z t ) + KL ( z t , (cid:98)

Original: z t ) large.
Revised: z t ) large.

Original: In our analysis, we use a two-stage argument to ﬁnd such a c (cid:48) .
Revised: In our analysis, we use a two-stage argument to ﬁnd such a c (cid:48) .

Original: In the ﬁrst stage, we only show that KL ( (cid:98) z t +1 , z t )
Revised: In the ﬁrst stage, we only show that KL ( (cid:98) z t +1 , z t )

Original: + KL ( z t , (cid:98)
Revised: + KL ( z t , (cid:98)

Original: z t ) ≥ c (cid:48)(cid:48) KL ( z ∗ , (cid:98)
Revised: z t ) ≥ c (cid:48)(cid:48) KL ( z ∗ , (cid:98)

Original: z t +1 ) 2 for some c (cid:48)(cid:48) > 0 , and use it to argue a slower convergence rate KL ( z ∗ , (cid:98) z t ) =
Revised: z t +1 ) 2 for some c (cid:48)(cid:48) > 0 , and use it to argue a slower convergence rate KL ( z ∗ , (cid:98) z t ) =

Original: O (cid:0) 1 t (cid:1) .
Revised: O (cid:0) 1 t (cid:1) .

Original: Then in the second stage, we show that after (cid:98) z t and z t become close enough to z ∗ , we have KL ( (cid:98) z t +1 , z t )
Revised: Then in the second stage, we show that after (cid:98) z t and z t become close enough to z ∗ , we have KL ( (cid:98) z t +1 , z t )

Original: + KL ( z t , (cid:98) z t ) ≥ c (cid:48)
Revised: + KL ( z t , (cid:98) z t ) ≥ c (cid:48)

Original: KL ( z ∗ , (cid:98)
Revised: KL ( z ∗ , (cid:98)

Original: z t +1 ) for some c (cid:48) > 0 .
Revised: z t +1 ) for some c (cid:48) > 0 .

Original: This kind of two-stage argument might be reminiscent of that used by Daskalakis & Panageas (2019b); however, the techniques we use are very different.
Revised: This kind of two-stage argument might be reminiscent of that used by Daskalakis & Panageas (2019b); however, the techniques we use are very different.

Original: Speciﬁcally, Daskalakis & Panageas (2019b) utilize tools of “spectral analysis” similar to (Liang & Stokes, 2019) and show that
Revised: Speciﬁcally, Daskalakis & Panageas (2019b) utilize tools of “spectral analysis” similar to (Liang & Stokes, 2019) and show that

Original: the
Revised: the

Original: OMWU update can be viewed as a “contraction mapping” with respect to a matrix whose eigenvalue is smaller than 1.
Revised: OMWU update can be viewed as a “contraction mapping” with respect to a matrix whose eigenvalue is smaller than 1.

Original: Our analysis, on the other hand, leverages analysis of online mirror descent, starting from the “one-step regret bound” (Lemma 1) and making use of the two negative terms that are typically dropped in online learning analysis.
Revised: Our analysis, on the other hand, leverages analysis of online mirror descent, starting from the “one-step regret bound” (Lemma 1) and making use of the two negative terms that are typically dropped in the analysis.

Original: Importantly, our analysis does not need an exponentially small learning rate required by (Daskalakis & Panageas, 2019b).
Revised: Importantly, our analysis does not need an exponentially small learning rate required by (Daskalakis & Panageas, 2019b).

Original: Thus, unlike their results, our learning rate is kept as a  constant in all stages.
Revised: Thus, unlike their results, our learning rate is kept as a universal constant in all stages.

Original: The arguments above are formalized below:
Revised: The arguments above are formalized below:

Original: Lemma 2.
Revised: Lemma 2.

Original: Consider a matrix game f
Revised: Consider a matrix game f

Original: ( x , y )
Revised: ( x , y )

Original: =
Revised: =

Original: x (cid:62) Gy with X = ∆ M , Y =
Revised: x (cid:62) Gy with X = ∆ M , Y =

Original: ∆ N , and G ∈ [ − 1 , 1] M × N .
Revised: ∆ N , and G ∈ [ − 1 , 1] M × N .

Original: Assume that there exists a unique Nash equilibrium z ∗ and η ≤ 18 .
Revised: Assume that there exists a unique Nash equilibrium z ∗ and η ≤ 18 .

Original: Then, there exists a constant C 1 > 0 that depends on G such that for any t ≥ 1 , OMWU ensures
Revised: Then, there exists a constant C 1 > 0 that depends on G such that for any t ≥ 1 , OMWU ensures

Original: [Equation]
Revised: [Equation]

Original: Also, there is a constant ξ > 0 that depends on G (deﬁned in Deﬁnition 2) such that as long as max {(cid:107)
Revised: Also, there is a constant ξ > 0 that depends on G (deﬁned in Deﬁnition 2) such that as long as max { [Equation]

Original: z
Revised: z

Original: [Equation] 
Revised: [Equation] [Equation]

Revised: z

Original: [Equation]
Revised: [Equation]

Original:  ≤ ηξ 10 , then
Revised: } ≤ ηξ 10 , then

Original: [Equation]
Revised: [Equation]

Original: for another constant C 2 > 0 that depends on G .
Revised: for another constant C 2 > 0 that depends on G .

Original: With Lemma 2 and the earlier discussion, the last-iterate convergence rate of OMWU is established:
Revised: With Lemma 2 and the earlier discussion, the last-iterate convergence rate of OMWU is established:

Original: Theorem 3.
Revised: Theorem 3.

Original: For a matrix game f ( x , y ) =
Revised: For a matrix game f ( x , y ) =

Original: x (cid:62) Gy with a unique Nash equilibrium z ∗ , OMWU with a learning rate η ≤ 18 guarantees KL ( z ∗ , z t ) ≤ C 3 (1 + C 4 ) − t , where C 3 , C 4 > 0 are some constants depending on the game matrix G .
Revised: x (cid:62) Gy with a unique Nash equilibrium z ∗ , OMWU with a learning rate η ≤ 18 guarantees KL ( z ∗ , z t ) ≤ C 3 (1 + C 4 ) − t , where C 3 , C 4 > 0 are some constants depending on the game matrix G .

Original: Proofs for this section are deferred to Appendix D, where all problem-dependent constants are speciﬁed as well.
Revised: Proofs for this section are deferred to Appendix D, where all problem-dependent constants are speciﬁed as well.

Original: 1 To the best of our knowledge, Theorem 3 gives the ﬁrst last-iterate convergence result for OMWU with a concrete linear rate.
Revised: 1 To the best of our knowledge, Theorem 3 gives the ﬁrst last-iterate convergence result for OMWU with a concrete linear rate.

Original: We note that the uniqueness assumption is critical for our analysis, and whether this is indeed necessary for OMWU is left as an important future direction.
Revised: We note that the uniqueness assumption is critical for our analysis, and whether this is indeed necessary for OMWU is left as an important future direction.

Original:  C ONVERGENCE R ESULTS FOR OGDA
Revised:  C ONVERGENCE R ESULTS FOR OGDA

Original: In this section, we provide last-iterate convergence results for OGDA, which are much more general than those in Section 4.
Revised: In this section, we provide last-iterate convergence results for OGDA, which are much more general than those in Section 4.

Original: We propose a general condition subsuming many well-studied cases, under which OGDA enjoys a concrete last-iterate convergence guarantee in terms of the L 2 distance between z t and Z ∗ .
Revised: We propose a general condition subsuming many well-studied cases, under which OGDA enjoys a concrete last-iterate convergence guarantee in terms of the L 2 distance between z t and Z ∗ .

Original: The results in this part can be specialized to the setting of bilinear games over simplex, but the unique equilibrium assumption made in Section 4 and in (Daskalakis & Panageas, 2019b) is no longer needed.
Revised: The results in this part can be specialized to the setting of bilinear games over simplex, but the unique equilibrium assumption made in Section 4 and in (Daskalakis & Panageas, 2019b) is no longer needed.

Original: Throughout the section we make the assumption that f is L -smooth:
Revised: Throughout the section we make the assumption that f is L -smooth:

Original: Assumption 1.
Revised: Assumption 1.

Original: For any z , z (cid:48) ∈
Revised: For any z , z (cid:48) ∈

Original: Z , (cid:107) F ( z ) − F ( z (cid:48) )
Revised: Z , (cid:107) F ( z ) − F ( z (cid:48) )

Original: (cid:107) ≤ L (cid:107)
Revised: (cid:107) ≤ L (cid:107)

Original: z − z (cid:48) (cid:107) holds.
Revised: z − z (cid:48) (cid:107) holds.

Original: To introduce our general condition, we ﬁrst provide some intuition by applying Lemma 1 again.
Revised: To introduce our general condition, we ﬁrst provide some intuition by applying Lemma 1 again.

Original: Letting ψ ( u ) = 12 (cid:107) u (cid:107) 2 in Lemma 1, we get that for OGDA, for any z ∈ Z and any t ≥ 1 ,
Revised: Letting ψ ( u ) = 12 (cid:107) u (cid:107) 2 in Lemma 1, we get that for OGDA, for any z ∈ Z and any t ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: z [Equation] [Equation] ( z
Revised: z (cid:107) 2 − (cid:107) (cid:98)

Original: [Equation]
Revised: z t +1 − z (cid:107) 2 − (cid:107) (cid:98) z t +1 − z t (cid:107) 2 − 1516 (cid:107) z t − (cid:98)

Original: ) ≤ [Equation] (cid:98)
Revised: z t (cid:107) 2 + 116 (cid:107) (cid:98)

Original: [Equation]
Revised: z t − z t − 1 (cid:107) 2 .

Original: Now we instantiate the inequality above with z = Π Z ∗ ( (cid:98)
Revised: Now we instantiate the inequality above with z = Π Z ∗ ( (cid:98)

Original: z t ) ∈
Revised: z t ) ∈

Original: Z ∗ .
Revised: Z ∗ .

Original: Since z = Π Z ∗ ( (cid:98) z t ) is an equilibrium, we have F ( z t ) (cid:62) ( z t − z ) ≥ f ( x t , y t ) − f ( x , y t )+ f ( x t , y ) − f ( x t , y t ) =
Revised: Since z = Π Z ∗ ( (cid:98) z t ) is an equilibrium, we have F ( z t ) (cid:62) ( z t − z ) ≥ f ( x t , y t ) − f ( x , y t )+ f ( x t , y ) − f ( x t , y t ) =

Original: f ( x t , y )
Revised: f ( x t , y )

Original: − f ( x , y t ) ≥ 0 by the convexity/concavity of f and the optimality of z , and thus (cid:107) 
Revised: − f ( x , y t ) ≥ 0 by the convexity/concavity of f and the optimality of z , and thus (cid:107) [Equation] (cid:107) (cid:98) z [Equation]

Revised: −

Original: [Equation]
Revised: [Equation]

Revised: (cid:107) 2 − 1516 (cid:107)

Revised: z t − (cid:98) z

Revised: [Equation]

Revised: (cid:107)

Revised: [Equation]

Revised: 116 (cid:107) (cid:98)

Original:  Further noting that the left-hand side is lower bounded by dist 2 ( (cid:98) z t +1 , Z ∗ ) by deﬁnition, we arrive at
Revised: z t − z [Equation] (cid:107) 2 . Further noting that the left-hand side is lower bounded by dist 2 ( (cid:98) z t +1 , Z ∗ ) by deﬁnition, we arrive at

Original: [Equation]
Revised: [Equation]

Original: Similarly, we deﬁne Θ t = (cid:107) (cid:98)
Revised: Similarly, we deﬁne Θ t = (cid:107) (cid:98)

Revised: z t − Π Z

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: and rewrite the above as
Revised: and rewrite the above as

Original: [Equation]
Revised: [Equation]

Original: As in Section 4, our goal now is to lower bound ζ t by some quantity related to dist 2 ( (cid:98) z t +1 , Z ∗ ) , and then use Eq.
Revised: As in Section 4, our goal now is to lower bound ζ t by some quantity related to dist 2 ( (cid:98) z t +1 , Z ∗ ) , and then use Eq.

Original: (4) to obtain a convergence rate for Θ t .
Revised: (4) to obtain a convergence rate for Θ t .

Original: In order to incorporate more general objective functions into the discussion, in the following Lemma 4, we provide an intermediate lower bound for ζ t , which will be further related to dist 2 ( (cid:98) z t +1 , Z ∗ ) later.
Revised: In order to incorporate more general objective functions into the discussion, in the following Lemma 4, we provide an intermediate lower bound for ζ t , which will be further related to dist 2 ( (cid:98) z t +1 , Z ∗ ) later.

Original: Lemma 4.
Revised: Lemma 4.

Original: For any t ≥ 0 and z (cid:48) ∈ Z with z (cid:48) (cid:54) =
Revised: For any t ≥ 0 and z (cid:48) ∈ Z with z (cid:48) (cid:54) =

Original: (cid:98) z t +1 , OGDA with η ≤ 18 L ensures
Revised: (cid:98) z t +1 , OGDA with η ≤ 18 L ensures

Original: [Equation]
Revised: [Equation]

Original: where [ a ] + (cid:44) max { a, 0 } , and similarly, for z (cid:48) (cid:54) =
Revised: where [ a ] + (cid:44) max { a, 0 } , and similarly, for z (cid:48) (cid:54) =

Original: z t +1 ,
Revised: z t +1 ,

Original: [Equation]
Revised: [Equation]

Original: We note that a direct consequence of Lemma 4 is an “average duality gap” guarantee for OGDA when Z is bounded:
Revised: We note that a direct consequence of Lemma 4 is an “average duality gap” guarantee for OGDA when Z is bounded:

Original: [Equation]
Revised: [Equation]

Original: where D (cid:44) sup
Revised: where D (cid:44) sup

Original: z , z (cid:48) ∈Z (cid:107)
Revised: z , z (cid:48) ∈Z (cid:107)

Original: z − z (cid:48) (cid:107) is the diameter of Z (the duality gap may be undeﬁned when Z is unbounded).
Revised: z − z (cid:48) (cid:107) is the diameter of Z (the duality gap may be undeﬁned when Z is unbounded).

Original: We are not aware of any previous work that gives this result for the constrained case.
Revised: We are not aware of any previous work that gives this result for the constrained case.

Original: See Appendix E for the proof of Eq.
Revised: See Appendix E for the proof of Eq.

Original: (7) and comparisons with previous works.
Revised: (7) and comparisons with previous works.

Original: However, to obtain last-iterate convergence results, we need to make sure that the right-hand side of
Revised: However, to obtain last-iterate convergence results, we need to make sure that the right-hand side of

Original: Eq.
Revised: Eq.

Original: (5) is large enough.
Revised: (5) is large enough.

Original: Motivated by this fact, we propose the following general condition on f and Z to achieve so.
Revised: Motivated by this fact, we propose the following general condition on f and Z to achieve so.

Original: Deﬁnition 1 (Saddle-Point Metric Subregularity (SP-MS)) .
Revised: Deﬁnition 1 (Saddle-Point Metric Subregularity (SP-MS)) .

Original: The SP-MS condition is deﬁned as: for any z ∈ Z\Z ∗ with z ∗ =
Revised: The SP-MS condition is deﬁned as: for any z ∈ Z\Z ∗ with z ∗ =

Original: Π Z ∗ ( z ) ,
Revised: Π Z ∗ ( z ) ,

Original: [Equation]
Revised: [Equation]

Original: holds for some parameter β ≥ 0 and C > 0 .
Revised: holds for some parameter β ≥ 0 and C > 0 .

Original: We call this condition Saddle-Point Metric Subregularity because the case with β = 0 is equivalent to one type of metric subregularity in variational inequality problems, as we prove in Appendix F.
Revised: We call this condition Saddle-Point Metric Subregularity because the case with β = 0 is equivalent to one type of metric subregularity in variational inequality problems, as we prove in Appendix F.

Original: The condition is also closely related to other error bound conditions that have been identiﬁed for variational inequality problems (e.g., Tseng (1995); Gilpin et al.
Revised: The condition is also closely related to other error bound conditions that have been identiﬁed for variational inequality problems (e.g., Tseng (1995); Gilpin et al.

Original: ; Malitsky (2019)).
Revised: ; Malitsky (2019)).

Original: Although these works have shown that under similar conditions their algorithms exhibit linear convergence, to the best of our knowledge, there is no previous work that analyzes OGDA or other no-regret learning algorithms using such conditions.
Revised: Although these works have shown that under similar conditions their algorithms exhibit linear convergence, to the best of our knowledge, there is no previous work that analyzes OGDA or other no-regret learning algorithms using such conditions.

Original: SP-MS covers many standard settings studied in the literature.
Revised: SP-MS covers many standard settings studied in the literature.

Original: The ﬁrst and perhaps the most important example is bilinear games with a polytope feasible set, which in particular includes the classic two-player matrix games considered in Section 4.
Revised: The ﬁrst and perhaps the most important example is bilinear games with a polytope feasible set, which in particular includes the classic two-player matrix games considered in Section 4.

Original: Theorem 5.
Revised: Theorem 5.

Original: A bilinear game f ( x , y )
Revised: A bilinear game f ( x , y )

Original: =
Revised: =

Original: x (cid:62) Gy with X ⊆ R M and Y ⊆
Revised: x (cid:62) Gy with X ⊆ R M and Y ⊆

Original: R N being polytopes and
Revised: R N being polytopes and

Original: G ∈ R M × N satisﬁes SP-MS with β = 0 .
Revised: G ∈ R M × N satisﬁes SP-MS with β = 0 .

Original: We emphasize again that different from Lemma 2, Theorem 5 does not require a unique equilibrium.
Revised: We emphasize again that different from Lemma 2, Theorem 5 does not require a unique equilibrium.

Original: Note that we have not provided the concrete form of the parameter C in the theorem (which depends on X , Y , and G ), but it can be found in the proof (see Appendix G).
Revised: Note that we have not provided the concrete form of the parameter C in the theorem (which depends on X , Y , and G ), but it can be found in the proof (see Appendix G).

Original: The next example shows that strongly-convex-strongly-concave problems are also special cases of our condition.
Revised: The next example shows that strongly-convex-strongly-concave problems are also special cases of our condition.

Original: Theorem 6.
Revised: Theorem 6.

Original: If f is strongly convex in x and strongly concave in y , then SP-MS holds with β = 0 .
Revised: If f is strongly convex in x and strongly concave in y , then SP-MS holds with β = 0 .

Original: Next, we provide a toy example where SP-MS holds with β > 0 .
Revised: Next, we provide a toy example where SP-MS holds with β > 0 .

Original: Theorem 7.
Revised: Theorem 7.

Original: Let X
Revised: Let X

Original: =
Revised: =

Original: Y (cid:44) { ( a, b ) : 0 ≤ a, b ≤ 1 , a + b = 1 } , n > 2 be an integer, and f ( x , y ) =
Revised: Y (cid:44) { ( a, b ) : 0 ≤ a, b ≤ 1 , a + b = 1 } , n > 2 be an integer, and f ( x , y ) =

Original: x 2 n 1 − x 1 y 1 − y 2 n 1 .
Revised: x 2 n 1 − x 1 y 1 − y 2 n 1 .

Original: Then SP-MS holds with β = 2 n − 2 .
Revised: Then SP-MS holds with β = 2 n − 2 .

Original: With this general condition, we are now able to complete the loop.
Revised: With this general condition, we are now able to complete the loop.

Original: For any value of β , we show the following last-iterate convergence guarantee for OGDA.
Revised: For any value of β , we show the following last-iterate convergence guarantee for OGDA.

Original: Theorem 8.
Revised: Theorem 8.

Original: For any η ≤ 18 L , if SP-MS holds with β = 0 , then OGDA guarantees linear last-iterate convergence:
Revised: For any η ≤ 18 L , if SP-MS holds with β = 0 , then OGDA guarantees linear last-iterate convergence:

Original: [Equation]
Revised: [Equation]

Original: on the other hand, if the condition holds with β > 0 , then we have a slower convergence:
Revised: on the other hand, if the condition holds with β > 0 , then we have a slower convergence:

Original: [Equation]
Revised: [Equation]

Original: where C 5 (cid:44) min (cid:110) 16 η 2 C 

Revised: where C 5 (cid:44) min (cid:110) 16 η 2 C

Original: [Equation]
Revised: [Equation]

Original: We defer the proof to Appendix I and make several remarks.
Revised: We defer the proof to Appendix I and make several remarks.

Original: First, note that based on a convergence result on dist 2 ( z t , Z ∗ ) , one can immediately obtain a convergence guarantee for the duality gap
Revised: First, note that based on a convergence result on dist 2 ( z t , Z ∗ ) , one can immediately obtain a convergence guarantee for the duality gap

Original: α f ( z t ) as long as f is also Lipschitz.
Revised: α f ( z t ) as long as f is also Lipschitz.

Original: This is because α f ( z t ) ≤ max x (cid:48) , y (cid:48)
Revised: This is because α f ( z t ) ≤ max x (cid:48) , y (cid:48)

Original: f
Revised: f

Original: ( x t , y (cid:48) )
Revised: ( x t , y (cid:48) )

Original: − f ( x ∗ , y (cid:48) )
Revised: − f ( x ∗ , y (cid:48) )

Original: + f ( x (cid:48) , y ∗ )
Revised: + f ( x (cid:48) , y ∗ )

Original: − f ( x (cid:48) , y t ) ≤ O ( (cid:107) x
Revised: − f ( x (cid:48) , y t ) ≤ O ( (cid:107) x

Original: t − x ∗ (cid:107) + (cid:107) y t − y ∗ (cid:107) )
Revised: t − x ∗ (cid:107) + (cid:107) y t − y ∗ (cid:107) )

Original: =
Revised: =

Original: O (cid:16)(cid:113) dist 2 ( z t , Z ∗ ) (cid:17) , where ( x ∗ , y ∗ )
Revised: O (cid:16)(cid:113) dist 2 ( z t , Z ∗ ) (cid:17) , where ( x ∗ , y ∗ )

Original: =
Revised: =

Original: Π Z ∗ ( z t ) .
Revised: Π Z ∗ ( z t ) .

Original: While this leads to stronger guarantees compared to Eq.
Revised: While this leads to stronger guarantees compared to Eq.

Original: (7), we emphasize that the latter holds even without the SP-MS condition.
Revised: (7), we emphasize that the latter holds even without the SP-MS condition.

Original: Second, our results signiﬁcantly generalize (Hsieh et al., 2019, Theorem 2) which itself is a consolidated version of several earlier works and also shows a linear convergence rate of OGDA under a condition stronger than our SP-MS with β = 0 as discussed earlier.
Revised: Second, our results signiﬁcantly generalize (Hsieh et al., 2019, Theorem 2) which itself is a consolidated version of several earlier works and also shows a linear convergence rate of OGDA under a condition stronger than our SP-MS with β = 0 as discussed earlier.

Original: More speciﬁcally, our results show that linear convergence holds for a much broader set of problems.
Revised: More speciﬁcally, our results show that linear convergence holds for a much broader set of problems.

Original: Furthermore, we also show slower sublinear convergence rates for any value of β > 0 , which is also new as far as we know.
Revised: Furthermore, we also show slower sublinear convergence rates for any value of β > 0 , which is also new as far as we know.

Original: In particular, we empirically verify that OGDA indeed does not converge exponentially fast for the toy example deﬁned in Theorem 7 (see Appendix A).
Revised: In particular, we empirically verify that OGDA indeed does not converge exponentially fast for the toy example deﬁned in Theorem 7 (see Appendix A).

Original: Last but not least, the most signiﬁcant implication of Theorem 8 is that it provides by far the most general linear convergence result for OGDA for the classic two-player matrix games, or more generally bilinear games with polytope constraints, according to Theorem 5 and Eq.
Revised: Last but not least, the most signiﬁcant implication of Theorem 8 is that it provides by far the most general linear convergence result for OGDA for the classic two-player matrix games, or more generally bilinear games with polytope constraints, according to Theorem 5 and Eq.

Original: Compared to recent works of (Daskalakis & Panageas, 2018; 2019b) for matrix games (on OGDA or OMWU), our result is considerably stronger: 1) we do not require a unique equilibrium while they do; 2) linear convergence holds for any initial points (cid:98) z 1 , while their result only holds if the initial points are in a small neighborhood of the unique equilibrium (otherwise the convergence is sublinear in the beginning); 3) our only requirement on the step size is η ≤ 18 L , 3 while they require an exponentially small η , which does not reﬂect the behavior of the algorithms in practice.
Revised: Compared to recent works of (Daskalakis & Panageas, 2018; 2019b) for matrix games (on OGDA or OMWU), our result is considerably stronger: 1) we do not require a unique equilibrium while they do; 2) linear convergence holds for any initial points (cid:98) z 1 , while their result only holds if the initial points are in a small neighborhood of the unique equilibrium (otherwise the convergence is sublinear initially); 3) our only requirement on the step size is η ≤ 18 L , 4 while they require an exponentially small η , which does not reﬂect the behavior of the algorithms in practice.

Original: Even compared with our result in Section 4, we see that for OGDA, the unique equilibrium assumption is not required, and we do not have an initial phase of sublinear convergence as in Lemma 2.
Revised: Even compared with our result in Section 4, we see that for OGDA, the unique equilibrium assumption is not required, and we do not have an initial phase of sublinear convergence as in Lemma 2.

Original: In Appendix A, we empirically show that OGDA often outperforms OMWU when both are tuned with a constant learning rate. 
Revised: In Appendix A, we empirically show that OGDA often outperforms OMWU when both are tuned with a constant learning rate. [Figure]

Original: One may wonder what happens if a bilinear game has a non-polytope constraint.
Revised: One may wonder what happens if a bilinear game has a non-polytope constraint.

Original: It turns out that in this case, SP-MS may only hold with β > 0 , due to the following example showing that linear convergence provably does not hold for OGDA when the feasible set has a curved boundary.[Figure]
Revised: It turns out that in this case, SP-MS may only hold with β > 0 , due to the following example showing that linear convergence provably does not hold for OGDA when the feasible set has a curved boundary.

Original: Theorem 9.
Revised: Theorem 9.

Original: There exists a bilinear game with a non-polytope feasible set such that SP-MS holds with β = 3 , and dist 2 ( z t , Z ∗ ) =
Revised: There exists a bilinear game with a non-polytope feasible set such that SP-MS holds with β = 3 , and dist 2 ( z t , Z ∗ ) =

Original: Ω(1 /t 2 ) holds for OGDA .
Revised: Ω(1 /t 2 ) holds for OGDA .

Original: This example indicates that the shape of the feasible set plays an important role in last-iterate convergence, which may be an interesting future direction to investigate, This is also veriﬁed empirically in our experiments (see Appendix A).
Revised: This example indicates that the shape of the feasible set plays an important role in last-iterate convergence, which may be an interesting future direction to investigate, This is also veriﬁed empirically in our experiments (see Appendix A).

Original:  E XPERIMENTS FOR M ATRIX G AMES
Revised:  E XPERIMENTS FOR M ATRIX G AMES

Original: In this section, we provide empirical results on the performance of OGDA and OMWU for matrix games on probability simplex.
Revised: In this section, we provide empirical results on the performance of OGDA and OMWU for matrix games on probability simplex.

Original: 4 We include more empirical results in other settings in Appendix A.
Revised: We include more empirical results in other settings in Appendix A.

Original: We set the size of the game matrix to be 32 × 32 , then generate a random matrix with each entry G ij drawn uniformly at random from [ − 1 , 1] , and ﬁnally rescale its operator norm to 1 .
Revised: We set the size of the game matrix to be 32 × 32 , then generate a random matrix with each entry G ij drawn uniformly at random from [ − 1 , 1] , and ﬁnally rescale its operator norm to 1 .

Original: With probability 1 , the game has a unique Nash Equilibrium (Daskalakis & Panageas, 2019b).
Revised: With probability 1 , the game has a unique Nash Equilibrium (Daskalakis & Panageas, 2019b).

Original: We compare the performances of OGDA and OMWU.
Revised: We compare the performances of OGDA and OMWU.

Original: For both algorithms, we choose a series of different learning rates and compare their performances, as shown in Figure 1.
Revised: For both algorithms, we choose a series of different learning rates and compare their performances, as shown in Figure 1.

Original: The x -axis represents time step t , and the y -axis represents ln( KL ( z ∗ , z t ))
Revised: The x -axis represents time step t , and the y -axis represents ln( KL ( z ∗ , z t ))

Original: (we observe similar results using dist 2 ( z ∗ , z t ) or the duality gap as the measure; see Appendix A.1).
Revised: (we observe similar results using dist 2 ( z ∗ , z t ) or the duality gap as the measure; see Appendix A.1).

Original: Note that here we approximate z ∗ by running
Revised: Note that here we approximate z ∗ by running

Original: OGDA for much more iterations and taking the very last iterate.
Revised: OGDA for much more iterations and taking the very last iterate.

Original: We also verify that the iterates of
Revised: We also verify that the iterates of

Original: OMWU converge to the same point as OGDA.
Revised: OMWU converge to the same point as OGDA.

Original: From Figure 1, we see that all curves eventually become a straight line, supporting our linear convergence results.
Revised: From Figure 1, we see that all curves eventually become a straight line, supporting our linear convergence results.

Original: Generally, the slope of the straight line is larger for a larger learning rate η .
Revised: Generally, the slope of the straight line is larger for a larger learning rate η .

Original: However, the algorithm diverges when η exceeds some value (such as 11 for the case of OMWU).
Revised: However, the algorithm diverges when η exceeds some value (such as 11 for the case of OMWU).

Original: Comparing
Revised: Comparing

Original: OMWU and OGDA, we see that OGDA converges faster, which is also consistent with our theory if one compares the bounds in Theorem 3 and Theorem 8 (with the value of the constants revealed in the proofs).
Revised: OMWU and OGDA, we see that OGDA converges faster, which is also consistent with our theory if one compares the bounds in Theorem 3 and Theorem 8 (with the value of the constants revealed in the proofs).

Original: We ﬁnd this observation interesting, since OMWU is usually considered more favorable for problems deﬁned over the simplex, especially in terms of regret minimization.
Revised: We ﬁnd this observation interesting, since OMWU is usually considered more favorable for problems deﬁned over the simplex, especially in terms of regret minimization.

Original: Our experiments suggests that, however, in terms of last-iterate convergence, OGDA might perform even better than
Revised: Our experiments suggest that, however, in terms of last-iterate convergence, OGDA might perform even better than

Original: OMWU.
Revised: OMWU.

Revised: A CKNOWLEDGMENTS

Revised: The authors would like to thank the anonymous reviewers for providing highly constructive comments which bring about signiﬁcant improvement of the result during the rebuttal phase.

Revised: CL would like to thank Yu-Guan Hsieh for many helpful discussions on error bounds and metric subregularity

Revised: The authors are supported by NSF Awards IIS-1755781 and IIS-1943607.

Original: A M ORE E XPERIMENT R ESULTS A. M ORE EMPIRICAL RESULTS FOR M ATRIX G AMES
Revised: A M ORE E XPERIMENT R ESULTS A. M ORE EMPIRICAL RESULTS FOR M ATRIX G AMES

Original: Here, we provide more plots for the same matrix game experiment described in Section 6.
Revised: Here, we provide more plots for the same matrix game experiment described in Section 6.

Original: Specifically, the left plot in Figure 2 shows the convergence with respect to ln (cid:107) z t − z ∗ (cid:107) , while the right plot shows the convergence with respect to the logarithm of the duality gap ln
Revised: Specifically, the left plot in Figure 2 shows the convergence with respect to ln (cid:107) z t − z ∗ (cid:107) , while the right plot shows the convergence with respect to the logarithm of the duality gap ln

Original: ( α f ( z t ))
Revised: ( α f ( z t ))

Original: =
Revised: =

Original: ln (cid:0) max j ( G (cid:62) x t ) j − min i ( Gy t ) i (cid:1) .
Revised: ln (cid:0) max j ( G (cid:62) x t ) j − min i ( Gy t ) i (cid:1) .

Original: One can see that the plots are very similar to those in Figure
Revised: One can see that the plots are very similar to those in Figure

Original: [Figure]
Revised: [Figure]

Original: of the curves.
Revised: the curves.

Original: For OMWU, η ≥ 11 makes the algorithm diverge.
Revised: For OMWU, η ≥ 11 makes the algorithm diverge.

Original: The plot conﬁrms the linear convergence of OMWU and OGDA, although OGDA is generally observed to converge faster than
Revised: The plot conﬁrms the linear convergence of OMWU and OGDA, although OGDA is generally observed to converge faster than

Original: OMWU.
Revised: OMWU.

Original: A. M ATRIX G AME ON C URVED R EGIONS
Revised: A. M ATRIX G AME ON C URVED R EGIONS

Original: Next, we conduct experiments on a bilinear game similar to the one constructed in the proof of
Revised: Next, we conduct experiments on a bilinear game similar to the one constructed in the proof of

Original: Theorem 9.
Revised: Theorem 9.

Original: Speciﬁcally, the bilinear game is deﬁned by
Revised: Speciﬁcally, the bilinear game is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: For any positive integer n , the equilibrium point of this game is (0 , 0) for both x and y .
Revised: For any positive integer n , the equilibrium point of this game is (0 , 0) for both x and y .

Original: Note that in
Revised: Note that in

Original: Theorem 9, we prove that OGDA only converges at a rate no better than Ω(1 /t 2 ) in this game when n = 2 .
Revised: Theorem 9, we prove that OGDA only converges at a rate no better than Ω(1 /t 2 ) in this game when n = 2 .

Original: Figure 3 shows the empirical results for various values of n .
Revised: Figure 3 shows the empirical results for various values of n .

Original: In this ﬁgure, we plot (cid:107) z t − z ∗ (cid:107) versus time step t in log-log scale.
Revised: In this ﬁgure, we plot (cid:107) z t − z ∗ (cid:107) versus time step t in log-log scale.

Original: Note that in a log-log plot, a straight line with slope s implies a convergence rate of order O ( t s ) , that is, a sublinear convergence rate.
Revised: Note that in a log-log plot, a straight line with slope s implies a convergence rate of order O ( t s ) , that is, a sublinear convergence rate.

Original: It is clear from Figure 3 that
Revised: It is clear from Figure 3 that

Original: OGDA indeed converges sublinearly for all n , supporting our Theorem 9.
Revised: OGDA indeed converges sublinearly for all n , supporting our Theorem 9.

Original: A. S TRONGLY - CONVEX - STRONGLY - CONCAVE G AMES
Revised: A. S TRONGLY - CONVEX - STRONGLY - CONCAVE G AMES

Original: In this section, we use the same experiment setup for strongly-convex-strongly-concave games in (Lei et al., 2020), where f[Equation] x
Revised: In this section, we use the same experiment setup for strongly-convex-strongly-concave games in (Lei et al., 2021), where f

Original: [Equation]
Revised: [Equation]

Original: y ) 
Revised: ) =

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: y
Revised: y

Original: [Equation]
Revised: [Equation]

Original:  1 , and X = Y (cid:44) { ( a, b ) , 0 ≤ a, b ≤ 1 , a + b = 1 } .
Revised: 2 x 1 y 1 , and X = Y (cid:44) { ( a, b ) , 0 ≤ a, b ≤ 1 , a + b = 1 } .

Original: The equilibrium point is (0 , 1) for both x and y .
Revised: The equilibrium point is (0 , 1) for both x and y .

Original: In Figure 4, we present the log plot of (cid:107) z t − z ∗ (cid:107) versus time step t and compare OGDA with OMWU using different learning rates as in Appendix A.1.
Revised: In Figure 4, we present the log plot of (cid:107) z t − z ∗ (cid:107) versus time step t and compare OGDA with OMWU using different learning rates as in Appendix A.1.

Original: The straight line of OGDA implies that OGDA algorithm converges exponentially fast, supporting
Revised: The straight line of OGDA implies that OGDA algorithm converges exponentially fast, supporting

Original: Theorem 6 and Theorem 8.
Revised: Theorem 6 and Theorem 8.

Original: Also note that here, OGDA outperforms OMWU, which is different from the empirical results shown in (Lei et al., 2020).
Revised: Also note that here, OGDA outperforms OMWU, which is different from the empirical results shown in (Lei et al., 2021).

Original: We hypothesize that this is because they use a different version of OGDA.
Revised: We hypothesize that this is because they use a different version of OGDA.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: A. A N E XAMPLE WITH β > 0 FOR SP-MS
Revised: A. A N E XAMPLE WITH β > 0 FOR SP-MS

Original: We also consider the toy example in Theorem 7, where f ( x , y )
Revised: We also consider the toy example in Theorem 7, where f ( x , y )

Original: =
Revised: =

Original: x 2 n 1 − x 1 y 1 − y 2 n 1 for some integer n ≥ 2 and X =
Revised: x 2 n 1 − x 1 y 1 − y 2 n 1 for some integer n ≥ 2 and X =

Original: Y (cid:44) { ( a, b ) , 0 ≤ a, b ≤ 1 , a + b = 1 } .
Revised: Y (cid:44) { ( a, b ) , 0 ≤ a, b ≤ 1 , a + b = 1 } .

Original: The equilibrium point is (0 , 1) for both x and y .
Revised: The equilibrium point is (0 , 1) for both x and y .

Original: We prove in Theorem 7 that SP-MS does not hold for β = 

Revised: We prove in Theorem 7 that SP-MS does not hold for β =

Original: but does hold for β = 2 n − 2 .
Revised: but does hold for β = 2 n − 2 .

Original: The point-wise convergence result is shown in Figure 5, which is again a log-log plot of (cid:107) z t − z ∗ (cid:107) versus time step t .
Revised: The point-wise convergence result is shown in Figure 5, which is again a log-log plot of (cid:107) z t − z ∗ (cid:107) versus time step t .

Original: One can observe that the convergence rate of OGDA is sublinear, supporting our theory again.
Revised: One can observe that the convergence rate of OGDA is sublinear, supporting our theory again.

Original: A. M ATRIX G AMES WITH M ULTIPLE N ASH E QUILIBRIA
Revised: A. M ATRIX G AMES WITH M ULTIPLE N ASH E QUILIBRIA

Original: Finally, we provide empirical results for OGDA and OMWU in matrix games with multiple Nash equilibria, even though theoretically we only prove linear convergence results for OMWU assuming that the Nash equilibrium is unique.
Revised: Finally, we provide empirical results for OGDA and OMWU in matrix games with multiple Nash equilibria, even though theoretically we only prove linear convergence results for OMWU assuming that the Nash equilibrium is unique.

Original: We consider the following game matrix
Revised: We consider the following game matrix

Original: G =   −     −  −    −   −−  −   .
Revised: G =   −     −  −    −   −−  −   .

Original: The value of G is 0 .
Revised: The value of G is 0 .

Original: To verify this, consider x 0 = y 

Revised: To verify this, consider x 0 = y

Original: = (cid:2)  0 0 (cid:3) .
Revised: = (cid:2)  0 0 (cid:3) .

Original: Then we have for max y ∈ ∆ 5 x (cid:62) 

Revised: Then we have for max y ∈ ∆ 5 x (cid:62)

Original: G y =
Revised: G y =

Original: min x ∈ ∆ 5 x (cid:62) G y 

Revised: min x ∈ ∆ 5 x (cid:62) G y

Original: = 0 .
Revised: = 0 .

Original: Direct calculation gives the following set of Nash equilibria.
Revised: Direct calculation gives the following set of Nash equilibria.

Original: [Equation]
Revised: [Equation]

Original: Figure 6 shows the point-wise convergence result.
Revised: Figure 6 shows the point-wise convergence result.

Original: Π Z ∗ ( z t ) is the projection of z t on the set of
Revised: Π Z ∗ ( z t ) is the projection of z t on the set of

Original: Nash qquilibria.
Revised: Nash qquilibria.

Original: One can observe from the plots that both OGDA and OMWU achieve linear convergence rate in this example.
Revised: One can observe from the plots that both OGDA and OMWU achieve linear convergence rate in this example.

Original: We thus conjecture that the uniqueness assumption for Theoremcan be further relaxed.
Revised: We thus conjecture that the uniqueness assumption for Theoremcan be further relaxed.

Original: [Figure]
Revised: [Figure]

Original: B L EMMAS FOR O PTIMISTIC M IRROR D ESCENT
Revised: B L EMMAS FOR O PTIMISTIC M IRROR D ESCENT

Original: We prove Lemma 1 in this section.
Revised: We prove Lemma 1 in this section.

Original: To do so, we use the following two lemmas.
Revised: To do so, we use the following two lemmas.

Original: Lemma 10.
Revised: Lemma 10.

Original: Let A be a convex set and u (cid:48)
Revised: Let A be a convex set and u (cid:48)

Original: =
Revised: =

Original: argmin u (cid:48) ∈A {(cid:104) u (cid:48) , g (cid:105)
Revised: argmin u (cid:48) ∈A {(cid:104) u (cid:48) , g (cid:105)

Original: + D ψ ( u (cid:48) , u ) } .
Revised: + D ψ ( u (cid:48) , u ) } .

Original: Then for any u ∗ ∈ A ,
Revised: Then for any u ∗ ∈ A ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Since D ψ ( u (cid:48) , u ) =
Revised: Since D ψ ( u (cid:48) , u ) =

Original: ψ ( u (cid:48) )
Revised: ψ ( u (cid:48) )

Original: − ψ ( u ) −(cid:104)∇ ψ ( u ) , u (cid:48) − u (cid:105) , by the ﬁrst-order optimality condition of u (cid:48) , we have
Revised: − ψ ( u ) −(cid:104)∇ ψ ( u ) , u (cid:48) − u (cid:105) , by the ﬁrst-order optimality condition of u (cid:48) , we have

Original: [Equation]
Revised: [Equation]

Original: On the other hand, notice that the right-hand side of Eq.
Revised: On the other hand, notice that the right-hand side of Eq.

Original: (10) is
Revised: (10) is

Original: [Equation]
Revised: [Equation]

Original: Therefore, Eq.
Revised: Therefore, Eq.

Original: (10) is equivalent to (cid:104) g + ∇ ψ (u
Revised: (10) is equivalent to (cid:104) g + ∇ ψ (

Original: [Equation]
Revised: [Equation]

Original: [Equation] ∇ ψ ( u ) , u ∗ − u (cid:48) (cid:105) ≥ 0 , which we have already shown above.
Revised: ) − ∇ ψ ( u ) , u ∗ − u (cid:48) (cid:105) ≥ 0 , which we have already shown above.

Original: Lemma 11.
Revised: Lemma 11.

Original: Suppose that ψ satisﬁes D ψ ( x , x (cid:48) ) ≥ 12 (cid:107) x − x (cid:48) (cid:107) 2 p for some p ≥ 1 , and let u , u 1 , u 2 ∈
Revised: Suppose that ψ satisﬁes D ψ ( x , x (cid:48) ) ≥ 12 (cid:107) x − x (cid:48) (cid:107) 2 p for some p ≥ 1 , and let u , u 1 , u 2 ∈

Original: A (a convex set) be related by the following:
Revised: A (a convex set) be related by the following:

Original: [Equation]
Revised: [Equation]

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: where q ≥ 1 and 1 p + 1 q = 1 .
Revised: where q ≥ 1 and 1 p + 1 q = 1 .

Original: Proof.
Revised: Proof.

Original: By the ﬁrst-order optimality conditions of u 1 and u 2 , we have
Revised: By the ﬁrst-order optimality conditions of u 1 and u 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Summing them up and rearranging the terms, we get
Revised: Summing them up and rearranging the terms, we get

Original: [Equation]
Revised: [Equation]

Original: By the condition on ψ , we have (cid:104)∇ ψ ( u 1 ) , u 1 − u 2 (cid:105) ≥ ψ ( u 1 ) − ψ ( u 2 ) + 12 (cid:107) u 1 − u 2 (cid:107) 2 p and (cid:104)∇ ψ ( u 2 ) , u 2 − u 1 (cid:105) ≥ ψ ( u 2 ) − ψ ( u 1 ) + 12 (cid:107) u 1 − u 2 (cid:107) 2 p .
Revised: By the condition on ψ , we have (cid:104)∇ ψ ( u 1 ) , u 1 − u 2 (cid:105) ≥ ψ ( u 1 ) − ψ ( u 2 ) + 12 (cid:107) u 1 − u 2 (cid:107) 2 p and (cid:104)∇ ψ ( u 2 ) , u 2 − u 1 (cid:105) ≥ ψ ( u 2 ) − ψ ( u 1 ) + 12 (cid:107) u 1 − u 2 (cid:107) 2 p .

Original: Summing them up we get (cid:104)∇ ψ ( u 1 ) − ∇ ψ ( u 2 ) , u 1 − u 2 (cid:105) ≥ (cid:107) u 1 − u 2 (cid:107) 2 p .
Revised: Summing them up we get (cid:104)∇ ψ ( u 1 ) − ∇ ψ ( u 2 ) , u 1 − u 2 (cid:105) ≥ (cid:107) u 1 − u 2 (cid:107) 2 p .

Original: Combining this with Eq.
Revised: Combining this with Eq.

Original: (11) we get (cid:104) u
Revised: (11) we get (cid:104) u

Original: 2 − u 1 , g 1 − g 2 (cid:105) ≥ (cid:107) u 1 − u 2 (cid:107) 2 p .
Revised: 2 − u 1 , g 1 − g 2 (cid:105) ≥ (cid:107) u 1 − u 2 (cid:107) 2 p .

Original: Since (cid:104) u 2 − u 1 , g 1 − g 2 (cid:105) ≤ (cid:107) u 1 − u 2 (cid:107) p (cid:107) g 1 − g 2 (cid:107) q by H¨older’s inequality, we further get (cid:107) u 1 − u 2 (cid:107) p ≤ (cid:107) g 1 − g 2 (cid:107) q .
Revised: Since (cid:104) u 2 − u 1 , g 1 − g 2 (cid:105) ≤ (cid:107) u 1 − u 2 (cid:107) p (cid:107) g 1 − g 2 (cid:107) q by H¨older’s inequality, we further get (cid:107) u 1 − u 2 (cid:107) p ≤ (cid:107) g 1 − g 2 (cid:107) q .

Original: Proof of Lemma 1.
Revised: Proof of Lemma 1.

Original: Considering Eq.
Revised: Considering Eq.

Original: (2), and using Lemma 10 with u = (cid:98) z t , u (cid:48)
Revised: (2), and using Lemma 10 with u = (cid:98) z t , u (cid:48)

Original: = (cid:98) z t +1 , u ∗ =
Revised: = (cid:98) z t +1 , u ∗ =

Original: z , and g = ηF ( z t ) , we get ηF ( z t ) (cid:62) ( (cid:98)
Revised: z , and g = ηF ( z t ) , we get ηF ( z t ) (cid:62) ( (cid:98)

Original: z t +1 − z ) ≤ D ψ ( z , (cid:98) z t ) − D ψ ( z , (cid:98) z t +1 )
Revised: z t +1 − z ) ≤ D ψ ( z , (cid:98) z t ) − D ψ ( z , (cid:98) z t +1 )

Original: − D ψ ( (cid:98) z t +1 , (cid:98) z t ) .
Revised: − D ψ ( (cid:98) z t +1 , (cid:98) z t ) .

Original: Considering Eq.
Revised: Considering Eq.

Original: (1), and using Lemma 10 with u = (cid:98) z t , u (cid:48) =
Revised: (1), and using Lemma 10 with u = (cid:98) z t , u (cid:48) =

Original: z t , u ∗ = (cid:98) z t +1 , and g = ηF ( z t − 1 ) , we get ηF ( z t − 1 ) (cid:62) ( z t − (cid:98) z t +1 ) ≤ D ψ
Revised: z t , u ∗ = (cid:98) z t +1 , and g = ηF ( z t − 1 ) , we get ηF ( z t − 1 ) (cid:62) ( z t − (cid:98) z t +1 ) ≤ D ψ

Original: ( (cid:98) z t +1 , (cid:98) z t ) − D
Revised: ( (cid:98) z t +1 , (cid:98) z t ) − D

Original: ψ
Revised: ψ

Original: ( (cid:98) z t, z
Revised: ( (cid:98) z t z

Original: ) − D( z
Revised: − D

Original: (cid:98)
Revised: (cid:98)

Original: z

Original: Summing up the two inequalities above, and adding η ( F ( z t ) − F [Equation] z [Equation]
Revised: Summing up the two inequalities above, and adding η ( F ( z t ) − F ( z t − 1 )) (cid:62) ( z t − (cid:98) z t +1 ) to both sides, we get

Original: [Equation]

Original: z t −

Original: [Equation]

Original: z t +1 ) to both sides, we get

Original: [Equation]
Revised: [Equation]

Original: Using Lemma 11 with u = (cid:98) x t , u 1 = x t , u 2 = (cid:98) x
Revised: Using Lemma 11 with u = (cid:98) x t , u 1 = x t , u 2 = (cid:98) x

Original: t +1 , g 1 = η ∇ x f ( z t − 1 ) and g 2 =
Revised: t +1 , g 1 = η ∇ x f ( z t − 1 ) and g 2 =

Original: η ∇ x f ( z t )
Revised: η ∇ x f ( z t )

Original: [Equation]
Revised: [Equation]

Original: we get (cid:107) x t − (cid:98) x
Revised: we get (cid:107) x t − (cid:98) x

Original: t +1 (cid:107) p ≤ η (cid:107)∇ x f ( z t − 1 ) −∇ x f ( z t ) (cid:107) q .
Revised: t +1 (cid:107) p ≤ η (cid:107)∇ x f ( z t − 1 ) −∇ x f ( z t ) (cid:107) q .

Original: Similarly, we have (cid:107) y t − (cid:98)
Revised: Similarly, we have (cid:107) y t − (cid:98)

Original: y t +1 (cid:107) p ≤ η (cid:107)∇ y f ( z t ) − ∇ y f
Revised: y t +1 (cid:107) p ≤ η (cid:107)∇ y f ( z t ) − ∇ y f

Original: ( z t − 1 ) (cid:107) q .
Revised: ( z t − 1 ) (cid:107) q .

Original: Therefore, by H¨older’s inequality, we have
Revised: Therefore, by H¨older’s inequality, we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (by our choice of η )
Revised: (by our choice of η )

Original: Continuing from Eq.
Revised: Continuing from Eq.

Original: (12), we then have
Revised: (12), we then have

Original: [Equation]
Revised: [Equation]

Original: (by the assumption on ψ )
Revised: (by the assumption on ψ )

Original: [Equation]
Revised: [Equation]

Original: This concludes the proof.
Revised: This concludes the proof.

Original: C A N A UXILIARY L EMMA ON R ECURSIVE FORMULAS
Revised: C A N A UXILIARY L EMMA ON R ECURSIVE FORMULAS

Original: Here, we provide an auxiliary lemma that gives an explicit bound based on a particular recursive formula.
Revised: Here, we provide an auxiliary lemma that gives an explicit bound based on a particular recursive formula.

Original: This will be useful later for deriving the convergence rate.
Revised: This will be useful later for deriving the convergence rate.

Original: Lemma 12.
Revised: Lemma 12.

Original: Consider a non-negative sequence { B t } t =1 , 2 , ··· that satisﬁes for some p > 0 and q > 0 ,
Revised: Consider a non-negative sequence { B t } t =1 , 2 , ··· that satisﬁes for some p > 0 and q > 0 ,

Original: • B t +1 ≤ B t − qB p +1 t +1 , ∀ t
Revised: • B t +1 ≤ B t − qB p +1 t +1 , ∀ t

Original: ≥• q (1 + p ) B p 1 ≤ 1 .
Revised: ≥• q (1 + p ) B p 1 ≤ 1 .

Original: Then B t ≤ ct − 1 p , where c = max (cid:26) B 1 , (cid:16) 2 qp (cid:17) 1 p (cid:27) .
Revised: Then B t ≤ ct − 1 p , where c = max (cid:26) B 1 , (cid:16) 2 qp (cid:17) 1 p (cid:27) .

Original: Proof.
Revised: Proof.

Original: We ﬁrst prove that B t +1 ≤ B t − q 2 B p +1 t .
Revised: We ﬁrst prove that B t +1 ≤ B t − q 2 B p +1 t .

Original: Notice that since B t are all non-negative, by the ﬁrst condition, we have B t +1 ≤ B t ≤ · · · ≤ B 1 .
Revised: Notice that since B t are all non-negative, by the ﬁrst condition, we have B t +1 ≤ B t ≤ · · · ≤ B 1 .

Original: Using the fundamental theorem of calculus, we have
Revised: Using the fundamental theorem of calculus, we have

Original: [Equation]
Revised: [Equation]

Original: and thus
Revised: and thus

Original: [Equation]
Revised: [Equation]

Original: By rearranging, we get
Revised: By rearranging, we get

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is because q (1 + p ) B pt ≤ q (1 + p ) B p 1 ≤ 1 .
Revised: where the last inequality is because q (1 + p ) B pt ≤ q (1 + p ) B p 1 ≤ 1 .

Original: Below we use induction to prove B t ≤ ct − 1 p , where c = max (cid:26) B 1 , (cid:16) 2 qp (cid:17) 1 p (cid:27) .
Revised: Below we use induction to prove B t ≤ ct − 1 p , where c = max (cid:26) B 1 , (cid:16) 2 qp (cid:17) 1 p (cid:27) .

Original: This clearly holds for t = 1 .
Revised: This clearly holds for t = 1 .

Original: Suppose that it holds for 1 , . . .
Revised: Suppose that it holds for 1 , . . .

Original: , t .
Revised: , t .

Original: Note that the function f ( B t ) =
Revised: Note that the function f ( B t ) =

Original: (cid:0) 1 − q 2 B pt (cid:1) B t is increasing in B t as f (cid:48)
Revised: (cid:0) 1 − q 2 B pt (cid:1) B t is increasing in B t as f (cid:48)

Original: ( B t )
Revised: ( B t )

Original: = 1 − q ( p +1)
Revised: = 1 − q ( p +1)

Original: B pt ≥ 1 − q ( p +1)
Revised: B pt ≥ 1 − q ( p +1)

Original: B p 1 ≥ 0 .
Revised: B p 1 ≥ 0 .

Original: Therefore, we apply the induction hypothesis and get
Revised: Therefore, we apply the induction hypothesis and get

Original: [Equation]
Revised: [Equation]

Original: by the deﬁnition of c )
Revised: by the deﬁnition of c )

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is by the fundamental theorem of calculus:
Revised: where the last inequality is by the fundamental theorem of calculus:

Original: [Equation]
Revised: [Equation]

Revised: 1 − 1 p .

Original: This completes the induction.
Revised: This completes the induction.

Original: D P ROOFS OF L EMMA 2 AND T HEOREM
Revised: D P ROOFS OF L EMMA 2 AND T HEOREM

Original: In this section, we consider f ( x , y ) =
Revised: In this section, we consider f ( x , y ) =

Original: x (cid:62) Gy with X = ∆ M and Y =
Revised: x (cid:62) Gy with X = ∆ M and Y =

Original: ∆ N being simplex and
Revised: ∆ N being simplex and

Original: G ∈ [ − 1 , 1] M × N .
Revised: G ∈ [ − 1 , 1] M × N .

Original: We assume that G has a unique Nash equilibrium z ∗ =
Revised: We assume that G has a unique Nash equilibrium z ∗ =

Original: ( x ∗ , y ∗ ) .
Revised: ( x ∗ , y ∗ ) .

Original: The value of the game is denoted as ρ = min x ∈X max y ∈Y x (cid:62) Gy = max y ∈Y min x ∈X x (cid:62)
Revised: The value of the game is denoted as ρ = min x ∈X max y ∈Y x (cid:62) Gy = max y ∈Y min x ∈X x (cid:62)

Original: Gy = x ∗(cid:62) Gy ∗ .
Revised: Gy = x ∗(cid:62) Gy ∗ .

Original: Before proving Lemma 2 and Theorem 3, in Section D.1, we deﬁne some constants for later analysis; in Section D.2, we state more auxiliary lemmas, which are useful when proving Lemma 2 and
Revised: Before proving Lemma 2 and Theorem 3, in Section D.1, we deﬁne some constants for later analysis; in Section D.2, we state more auxiliary lemmas, which are useful when proving Lemma 2 and

Original: Theorem 3 in Section D.3.
Revised: Theorem 3 in Section D.3.

Original: D. S OME P ROBLEM - DEPENDENT C ONSTANTS
Revised: D. S OME P ROBLEM - DEPENDENT C ONSTANTS

Original: First, we deﬁne a constant ξ that is determined by G .
Revised: First, we deﬁne a constant ξ that is determined by G .

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: [Equation]
Revised: [Equation]

Original: The fact ξ ≤ 1 can be shown by:
Revised: The fact ξ ≤ 1 can be shown by:

Original: [Equation]
Revised: [Equation]

Original: while the fact ξ > 0 is a direct consequence of Lemma C.3 of Mertikopoulos et al.
Revised: while the fact ξ > 0 is a direct consequence of Lemma C.3 of Mertikopoulos et al.

Original: (2018), stated below.
Revised: (2018), stated below.

Original: Lemma 13 (Lemma C.3 of Mertikopoulos et al.
Revised: Lemma 13 (Lemma C.3 of Mertikopoulos et al.

Original: Let G ∈ R M × N be a game matrix for a two-player zero-sum game with value ρ .
Revised: Let G ∈ R M × N be a game matrix for a two-player zero-sum game with value ρ .

Original: Then there exists a Nash equilibrium ( x ∗ , y ∗ ) such that ( Gy ∗ )
Revised: Then there exists a Nash equilibrium ( x ∗ , y ∗ ) such that ( Gy ∗ )

Original: i = ρ ∀ i ∈ supp ( x ∗ ) , ( Gy ∗ )
Revised: i = ρ ∀ i ∈ supp ( x ∗ ) , ( Gy ∗ )

Original: i > ρ ∀ i / ∈ supp ( x ∗ ) , ( G (cid:62) x ∗ )
Revised: i > ρ ∀ i / ∈ supp ( x ∗ ) , ( G (cid:62) x ∗ )

Original: i = ρ ∀ i ∈ supp
Revised: i = ρ ∀ i ∈ supp

Original: ( y ∗ ) , ( G (cid:62) x ∗ )
Revised: ( y ∗ ) , ( G (cid:62) x ∗ )

Original: i < ρ ∀ i / ∈ supp ( y ∗ ) .
Revised: i < ρ ∀ i / ∈ supp ( y ∗ ) .

Original: Below, we deﬁne V ∗ ( Z )
Revised: Below, we deﬁne V ∗ ( Z )

Original: =
Revised: =

Original: V ∗ ( X ) × V ∗ ( Y ) , whereV ∗
Revised: V ∗ ( X ) × V ∗ ( Y ) , where

Original: [Equation]
Revised: [Equation]

Original: X [Equation] { x : x ∈ ∆ M , supp
Revised: } and

Original: [Equation]
Revised: [Equation]

Original: x ) ⊆ supp ( x ∗ ) } and

Original: V ∗ ( Y ) (cid:44) { y : y ∈ ∆ N , supp (

Original: y ) ⊆

Original: supp ( y ∗ ) } .

Original: Deﬁnition 3.
Revised: Deﬁnition 3.

Original: [Equation]
Revised: [Equation]

Original: Note that in the deﬁnition of c x and c y , the outer minimization is over an open set, which may make the deﬁnition problematic as the optimal value may not be attained.
Revised: Note that in the deﬁnition of c x and c y , the outer minimization is over an open set, which may make the deﬁnition problematic as the optimal value may not be attained.

Original: However, the following lemma shows that c x and c y are well-deﬁned.
Revised: However, the following lemma shows that c x and c y are well-deﬁned.

Original: Lemma 14.
Revised: Lemma 14.

Original: c x and c y are well-deﬁned, and 0 <
Revised: c x and c y are well-deﬁned, and 0 <

Original: c x , c y ≤ 1 .
Revised: c x , c y ≤ 1 .

Original: Proof.
Revised: Proof.

Original: We ﬁrst show c x and c y are well-deﬁned.
Revised: We ﬁrst show c x and c y are well-deﬁned.

Original: To simplify the notations, we deﬁne x ∗ min (cid:44) min i ∈ supp ( x ∗ ) x ∗ i and X (cid:48)
Revised: To simplify the notations, we deﬁne x ∗ min (cid:44) min i ∈ supp ( x ∗ ) x ∗ i and X (cid:48)

Original: (cid:44) { x : x ∈ ∆ M , (cid:107) x − x ∗ (cid:107) 1 ≥ x ∗ min } , and deﬁne y ∗ min and Y (cid:48) similarly.
Revised: (cid:44) { x : x ∈ ∆ M , (cid:107) x − x ∗ (cid:107) 1 ≥ x ∗ min } , and deﬁne y ∗ min and Y (cid:48) similarly.

Original: We will show that
Revised: We will show that

Original: [Equation]
Revised: [Equation]

Original: which are well-deﬁned as the outer minimization is now over a closed set.
Revised: which are well-deﬁned as the outer minimization is now over a closed set.

Original: Consider c x , it sufﬁces to show that for any x ∈ ∆ M such that x (cid:54) =
Revised: Consider c x , it sufﬁces to show that for any x ∈ ∆ M such that x (cid:54) =

Original: x ∗ and (cid:107) x − x ∗ (cid:107) 1 < x ∗ min , there exists x (cid:48) ∈
Revised: x ∗ and (cid:107) x − x ∗ (cid:107) 1 < x ∗ min , there exists x (cid:48) ∈

Original: ∆ M such that (cid:107) x (cid:48) − x ∗ (cid:107) 

Revised: ∆ M such that (cid:107) x (cid:48) − x ∗ (cid:107)

Original: = x ∗ min and
Revised: = x ∗ min and

Original: [Equation]
Revised: [Equation]

Original: In fact, we can simply choose x (cid:48)
Revised: In fact, we can simply choose x (cid:48)

Original: = x ∗ + ( x − x ∗ ) · x [Equation] x ∗ [Equation]
Revised: = x ∗ + ( x − x ∗ ) · x ∗ min (cid:107) x − x ∗ (cid:107) 1 .

Original: We ﬁrst argue that x (cid:48) is still in ∆ M .
Revised: We ﬁrst argue that x (cid:48) is still in ∆ M .

Original: For each j ∈ [ K ] , if x j − x ∗ j ≥ 0 , we surely have x (cid:48)
Revised: For each j ∈ [ K ] , if x j − x ∗ j ≥ 0 , we surely have x (cid:48)

Original: j ≥ x ∗ j + 0 ≥ 0 ; otherwise, x ∗ j > x j ≥ 0 and thus j ∈ supp ( x ∗ ) and x ∗ j ≥ x ∗ min , which implies x (cid:48)
Revised: j ≥ x ∗ j + 0 ≥ 0 ; otherwise, x ∗ j > x j ≥ 0 and thus j ∈ supp ( x ∗ ) and x ∗ j ≥ x ∗ min , which implies x (cid:48)

Original: j ≥ x ∗ j − | x j − x ∗ j | · 
Revised: j ≥ x ∗ j − | x j − x ∗ j | · x

Original: [Equation]
Revised: [Equation]

Original:  − x [Equation] 1 ≥ x ∗ j − x ∗ min ≥ 0 .
Revised: (cid:107) x − x ∗ (cid:107) 1 ≥ x ∗ j − x ∗ min ≥ 0 .

Original: In addition, (cid:80) j x (cid:48)
Revised: In addition, (cid:80) j x (cid:48)

Original: j = (cid:80) j x ∗ j = 1 .
Revised: j = (cid:80) j x ∗ j = 1 .

Original: Combining these facts, we have x (cid:48) ∈
Revised: Combining these facts, we have x (cid:48) ∈

Original: ∆ M .
Revised: ∆ M .

Original: Moreover, according to the deﬁnition of x (cid:48) , (cid:107) x (cid:48) − x ∗ (cid:107) 

Revised: Moreover, according to the deﬁnition of x (cid:48) , (cid:107) x (cid:48) − x ∗ (cid:107)

Original: =
Revised: =

Original: x
Revised: x

Original: ∗ min holds.
Revised: ∗ min holds.

Original: Also, since x ∗ − x and x ∗ − x (cid:48) are parallel vectors, Eq.
Revised: Also, since x ∗ − x and x ∗ − x (cid:48) are parallel vectors, Eq.

Original: (13) is satisﬁed.
Revised: (13) is satisﬁed.

Original: The arguments above show that the c x in Deﬁnition 3 is a well-deﬁned real number.
Revised: The arguments above show that the c x in Deﬁnition 3 is a well-deﬁned real number.

Original: The case of c y is similar.
Revised: The case of c y is similar.

Original: Now we show 0 <
Revised: Now we show 0 <

Original: c x , c y ≤ 1 .
Revised: c x , c y ≤ 1 .

Original: The fact that c x , c y ≤ 1 is a direct consequence of G being in [ − 1 , 1] M × N .
Revised: The fact that c x , c y ≤ 1 is a direct consequence of G being in [ − 1 , 1] M × N .

Original: Below, we use contradiction to prove that c y > 0 .
Revised: Below, we use contradiction to prove that c y > 0 .

Original: First, if c y < 0 , then there exists y (cid:54) =
Revised: First, if c y < 0 , then there exists y (cid:54) =

Original: y ∗ such that x ∗(cid:62)
Revised: y ∗ such that x ∗(cid:62)

Original: Gy ∗ < x ∗(cid:62)
Revised: Gy ∗ < x ∗(cid:62)

Original: Gy .
Revised: Gy .

Original: This contradicts with the fact that ( x ∗ , y ∗ ) is the equilibrium.
Revised: This contradicts with the fact that ( x ∗ , y ∗ ) is the equilibrium.

Original: On the other hand, if c y = 0 , then there is some y (cid:54) =
Revised: On the other hand, if c y = 0 , then there is some y (cid:54) =

Original: y ∗ such that
Revised: y ∗ such that

Original: [Equation]
Revised: [Equation]

Original: Consider the point y (cid:48)
Revised: Consider the point y (cid:48)

Original: =
Revised: =

Original: y ∗ + ξ 

Revised: y ∗ + ξ

Original: ( y − y ∗ )
Revised: ( y − y ∗ )

Original: (recall the deﬁnition of ξ in Deﬁnition 2 and that 0 < ξ ≤ 1 ), which lies on the line segment between y ∗ and y .
Revised: (recall the deﬁnition of ξ in Deﬁnition 2 and that 0 < ξ ≤ 1 ), which lies on the line segment between y ∗ and y .

Original: Then, for any x ∈ X , x
Revised: Then, for any x ∈ X , x

Original: [Equation]
Revised: [Equation]

Revised: (using G ij ∈ [ − 1 , − 1] for the ﬁrst part and y (cid:48) =

Original: Gy
Revised: y

Original: [Equation]
Revised: [Equation]

Original: (using G ij ∈ [ − 1 , − 1] for the ﬁrst part and y (cid:48) =

Original: y ∗ + [-ξ 
-]
Revised: + [-ξ 
-][Equation]

Original: ( y − y ∗ ) for the second) 
Revised: ( y − y ∗ ) for the second) ≥

Original: [Equation]
Revised: [Equation]

Original: (using Eq.
Revised: (using Eq.

Original: (14) and ( Gy ∗ )
Revised: (14) and ( Gy [Equation]

Original:  i = ρ for all i ∈ supp ( x
Revised: ) i = [Equation] for all i ∈ supp ( x

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (using y (cid:48) − y
Revised: (using y (cid:48) − y

Original: [Equation]
Revised: [Equation]

Original: ( y − y
Revised: ( y − y

Original: [Equation]
Revised: [Equation]

Original: ) and (cid:107) y − y
Revised: ) and (cid:107) y − y

Original: [Equation]
Revised: [Equation]

Original: (cid:107)
Revised: (cid:107)

Original: [Equation]

Original: by the deﬁnition of

Original: [Equation]
Revised: [Equation]

Original: This shows that min x ∈X x (cid:62) Gy (cid:48) ≥ ρ , that is, y (cid:48) (cid:54) =
Revised: This shows that min x ∈X x (cid:62) Gy (cid:48) ≥ ρ , that is, y (cid:48) (cid:54) =

Original: y ∗ is also a maximin point, contradicting that z ∗ is unique.
Revised: y ∗ is also a maximin point, contradicting that z ∗ is unique.

Original: Therefore, c y > 0 has to hold, and so does c x > 0 by the same argument.
Revised: Therefore, c y > 0 has to hold, and so does c x > 0 by the same argument.

Original: Finally, we deﬁne the following constant that depends on G :
Revised: Finally, we deﬁne the following constant that depends on G :

Original: Deﬁnition 4.
Revised: Deﬁnition 4.

Original: [Equation]
Revised: [Equation]

Original: D. A UXILIARY L EMMAS
Revised: D. A UXILIARY L EMMAS

Original: All lemmas stated in this section is for the case f ( x , y ) =
Revised: All lemmas stated in this section is for the case f ( x , y ) =

Original: x (cid:62) Gy with Z = ∆ M × ∆ N and a unique Nash equilibrium z ∗ =
Revised: x (cid:62) Gy with Z = ∆ M × ∆ N and a unique Nash equilibrium z ∗ =

Original: ( x ∗ , y ∗ ) .
Revised: ( x ∗ , y ∗ ) .

Original: Lemma 15.
Revised: Lemma 15.

Original: For any z ∈ Z , we have
Revised: For any z ∈ Z , we have

Original: [Equation]
Revised: [Equation]

Original: for C = min { c x , c y } ∈ (0 , 1] .
Revised: for C = min { c x , c y } ∈ (0 , 1] .

Original: Proof.
Revised: Proof.

Original: Recall that ρ = x ∗(cid:62)
Revised: Recall that ρ = x ∗(cid:62)

Original: Gy ∗ is the game value and note that
Revised: Gy ∗ is the game value and note that

Original: [Equation]
Revised: [Equation]

Original: which completes the proof
Revised: which completes the proof

Original: [Equation]
Revised: [Equation]

Original: Lemma 16.
Revised: Lemma 16.

Original: For any z ∈ Z
Revised: For any z ∈ Z

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Using the deﬁnition of the Kullback-Leibler divergence, we have
Revised: Using the deﬁnition of the Kullback-Leibler divergence, we have

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality is by the concavity of the ln( · ) function
Revised: where the ﬁrst inequality is by the concavity of the ln( · ) function

Original: [Equation]
Revised: [Equation]

Original: and the second inequality is because ln(1 + u ) ≤ u .
Revised: and the second inequality is because ln(1 + u ) ≤ u .

Original: Considering i ∈ supp ( x ∗ ) and i / ∈ supp ( x ∗ ) separately in the last summation
Revised: Considering i ∈ supp ( x ∗ ) and i / ∈ supp ( x ∗ ) separately in the last summation

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: The case for KL ( y ∗ , y ) is similar.
Revised: The case for KL ( y ∗ , y ) is similar.

Original: Combining both cases ﬁnishes the proof of the ﬁrst inequality (recall that KL ( z ∗ , z ) is deﬁned as KL ( x ∗ , x ) + KL ( y ∗ , y ) ).
Revised: Combining both cases ﬁnishes the proof of the ﬁrst inequality (recall that KL ( z ∗ , z ) is deﬁned as KL ( x ∗ , x ) + KL ( y ∗ , y ) ).

Original: The second inequality is straightforward:
Revised: The second inequality is straightforward:

Original: [Equation]
Revised: [Equation]

Original: Lemma 17.
Revised: Lemma 17.

Original: For η ≤ 18 , OMWU guarantees 34 (cid:98) z t,i ≤ z t,i ≤ 43 (cid:98) z t,i and 34 (cid:98) z t,i ≤ (cid:98)
Revised: For η ≤ 18 , OMWU guarantees 34 (cid:98) z t,i ≤ z t,i ≤ 43 (cid:98) z t,i and 34 (cid:98) z t,i ≤ (cid:98)

Original: z t
Revised: z t

Original: +

Revised: +

Original: ,i ≤ 43 (cid:98) z t,i .
Revised: ,i ≤ 43 (cid:98) z t,i .

Original: Proof.
Revised: Proof.

Original: This is shown directly by the update of (cid:98) x t :
Revised: This is shown directly by the update of (cid:98) x t :

Original: [Equation]
Revised: [Equation]

Original: So by the condition on η , we have 34 (cid:98) x t,i ≤ exp( − 2 η ) · [Equation]
Revised: So by the condition on η , we have 34 (cid:98) x t,i ≤ exp( − 2 η ) · (cid:98) x

Original: x t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: i ≤
Revised: i ≤

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ≤
Revised: ≤

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: x t
Revised: 2 η ) ·

Original: [Equation]
Revised: [Equation]

Original:  ≤ 43 (cid:98) x t,i .
Revised: x t,i ≤ 43 (cid:98) x t,i .

Original: The cases for x t , (cid:98) y t and y t are similar.
Revised: The cases for x t , (cid:98) y t and y t are similar.

Original: Lemma 18.
Revised: Lemma 18.

Original: For any two probability vectors u , v , if for every entry i , 12 u i ≤ v i ≤ 32 u i , then 13 (cid:80) i ( v i − u i )
Revised: For any two probability vectors u , v , if for every entry i , 12 u i ≤ v i ≤ 32 u i , then 13 (cid:80) i ( v i − u i )

Original: 2 u
Revised: 2 u

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: ≤ 14 .
Revised: ≤ 14 .

Original: Proof.
Revised: Proof.

Original: Using the deﬁnition of the Kullback-Leibler divergence, we have
Revised: Using the deﬁnition of the Kullback-Leibler divergence, we have

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality is because ln(1+ a ) ≤ a − 13 a 2 for − 12 ≤ a ≤ 12 , and the second inequality is because ln(1 + a ) ≥ a − a 2 for − 12 ≤ a ≤ 12 .
Revised: where the ﬁrst inequality is because ln(1+ a ) ≤ a − 13 a 2 for − 12 ≤ a ≤ 12 , and the second inequality is because ln(1 + a ) ≥ a − a 2 for − 12 ≤ a ≤ 12 .

Original: The third inequality is by using the condition | u i − v i | ≤ 12 u i .
Revised: The third inequality is by using the condition | u i − v i | ≤ 12 u i .

Original: Lemma 19.
Revised: Lemma 19.

Original: For all i ∈ supp ( z ∗ ) and t , OMWU guarantees (cid:98) z t,i ≥ (cid:15) ( (cid:15) is deﬁned in Deﬁnition 4).
Revised: For all i ∈ supp ( z ∗ ) and t , OMWU guarantees (cid:98) z t,i ≥ (cid:15) ( (cid:15) is deﬁned in Deﬁnition 4).

Original: Proof.
Revised: Proof.

Original: Using Eq.
Revised: Using Eq.

Original: (3), we have
Revised: (3), we have

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the last equality is because (cid:98) z
Revised: where the last equality is because (cid:98) z

Original: 1 = z 

Revised: 1 = z

Original: = ( 1 M M , [Equation] N ) .
Revised: = ( 1 M M , 1 N N ) .

Original: Then, for any i ∈ supp ( z ∗ ) , we have
Revised: Then, for any i ∈ supp ( z ∗ ) , we have

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: we conclude for all t and
Revised: we conclude for all t and

Original: i ∈ supp ( z ∗ ) , (cid:98) z t,i satisﬁes
Revised: i ∈ supp ( z ∗ ) , (cid:98) z t,i satisﬁes

Original: [Equation]
Revised: [Equation]

Original: D. P ROOFS OF L EMMA 2 AND T HEOREM
Revised: D. P ROOFS OF L EMMA 2 AND T HEOREM

Original: Proof of Lemma 2.
Revised: Proof of Lemma 2.

Original: Below we consider any z (cid:

Revised: Below we consider any z (cid:

Original: ) ∈
Revised: ) ∈

Original: Z such that supp ( z (cid:48) )
Revised: Z such that supp ( z (cid:48) )

Original: ⊆ supp ( z ∗ )
Revised: ⊆ supp ( z ∗ )

Original: [Equation]
Revised: [Equation]

Original: that is, z (cid:48) ∈
Revised: that is, z (cid:48) ∈

Original: V ∗ ( Z ) .
Revised: V ∗ ( Z ) .

Original: Considering Eq.
Revised: Considering Eq.

Original: [Equation]
Revised: [Equation]

Original: and using the ﬁrst-order optimality condition of (cid:98) z t +1 , we have
Revised: and using the ﬁrst-order optimality condition of (cid:98) z t +1 , we have

Original: [Equation]
Revised: [Equation]

Original: where ψ ( z ) =
Revised: where ψ ( z ) =

Original: (cid:80)
Revised: (cid:80)

Original: i
Revised: i

Original: z
Revised: z

Original: i ln
Revised: i ln

Original: z
Revised: z

Original: i .
Revised: i .

Original: Rearranging the terms and we get
Revised: Rearranging the terms and we get

Original: [Equation]
Revised: [Equation]

Original: The left hand side of Eq.
Revised: The left hand side of Eq.

Original: (16) is lower bounded as
Revised: (16) is lower bounded as

Original: [Equation]
Revised: [Equation]

Original: ; ( η ≤ 1 / 8 ) on the other hand, the right hand side of Eq.
Revised: ( η ≤ [Equation] / 8 ) on the other hand, the right hand side of Eq.

Original: (16) is upper bounded by (cid:88)
Revised: (16) is upper bounded by (cid:88)

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: ,i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: (cid:98) [Equation] ( ln(1 + a )
Revised: ln

Original: [Equation]
Revised: [Equation]

Original: a and Lemma 17)
Revised: and Lemma 17)

Original: Combining the bounds on the two sides of Eq.
Revised: Combining the bounds on the two sides of Eq.

Original: (16), we get
Revised: (16), we get

Original: [Equation]
Revised: [Equation]

Original: Since z (cid:48) can be chosen as any point in V ∗ ( Z ) , we further lower bound the left-hand side above using
Revised: Since z (cid:48) can be chosen as any point in V ∗ ( Z ) , we further lower bound the left-hand side above using

Original: Lemma 15 and get
Revised: Lemma 15 and get

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the last inequality uses (cid:15) ≤ 1 .
Revised: where the last inequality uses (cid:15) ≤ 1 .

Original: With the help of Eq.
Revised: With the help of Eq.

Original: (17), below we prove the desired inequalities.
Revised: (17), below we prove the desired inequalities.

Original: Case 1.
Revised: Case 1.

Original: General case.
Revised: General case.

Original: [Equation]
Revised: [Equation]

Original: [Table] 
Revised: [Table] [Equation] [Table] [Equation]

Revised: [Table]

Original: [Equation] 
Revised: [Equation] [Table] [Equation] [Table]

Revised: [Equation]

Original: [Table] 
Revised: [Table] [Equation] [Table] [Equation]

Revised: [Table]

Original: [Equation] 
Revised: [Equation] [Table] [Equation] [Table]

Revised: [Equation]

Original: [Table] 
Revised: [Table] [Equation] [Table] [Equation]

Original:  [Equation]
Revised: [Table] [Equation] [Table] [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table][Equation] [Table] [Equation]
Revised: [Table]

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Lemma 19)

Original: This proves the ﬁrst part of the lemma with C 1 = (cid:15) 4 C 2 / 64 .
Revised: This proves the ﬁrst part of the lemma with C 1 = (cid:15) 4 C 2 / 64 .

Original: Case 2.
Revised: Case 2.

Original: The case when max {
Revised: The case when max {

Original: [Equation]
Revised: [Equation]

Original: Below we continue to bound (cid:80) i
Revised: Below we continue to bound (cid:80) i/

Original: [Equation]
Revised: [Equation]

Original: i .
Revised: i .

Original: By the assumption, we have (cid:107) y t − y ∗ (cid:107) 1 ≤ ηξ 10 , which by Lemma 13 and Deﬁnition 2 implies ∀ i ∈ supp ( x ∗ ) , ( Gyt ) i ≤ ( Gy ∗ )
Revised: By the assumption, we have (cid:107) y t − y ∗ (cid:107) 1 ≤ ηξ 10 , which by Lemma 13 and Deﬁnition 2 implies ∀ i ∈ supp ( x ∗ ) , ( Gy

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 10 ≤
Revised: Gy

Original: [Equation]
Revised: [Equation]

Original: ξ 10 , ∀ i / ∈ supp ( x ∗ ) , ( Gy t ) i ≥ ( Gy ∗ )
Revised: [Equation] ∀ i / ∈ supp ( x ∗ ) , ( Gy t ) i ≥ [Equation]

Original: i −
Revised: Gy [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: 10 ≥ ρ + ξ−
Revised: ξ

Original: [Equation]
Revised: [Equation]

Original: 10 ≥ ρ +9 ξ 10 . We also have (cid:107)
Revised: +

Original: [Equation]
Revised: [Equation]

Original:  x t − x ∗
Revised: We also have (cid:107) (cid:98) x t − x ∗

Original: [Equation]
Revised: [Equation]

Original: 1 ≤
Revised: 1 ≤

Original: [Equation]
Revised: [Equation]

Original: 10 , so (cid:80)
Revised: 10 , so (cid:80)

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: / ∈ supp ( x ∗ ) (cid:98) x t
Revised: supp [Equation] x ∗ ) (cid:98) x t

Original: [Equation]
Revised: [Equation]

Original: j ≤ ηξ 10 .
Revised: j ≤ ηξ 10 .

Original: Then, for i / ∈ supp ( x ∗ ) , we have
Revised: Then, for i / ∈ supp ( x ∗ ) , we have

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is because exp( − 0 .
Revised: where the last inequality is because exp( − 0 .

Revised: [Equation]

Original: 8 u ) 1 − 0 .
Revised: ) 1 − 0 .

Original: 1 u ≤ 1 − 0 .
Revised: 1 u ≤ 1 − 0 .

Original: 5 u for u ∈ [0 , 1] .
Revised: 5 u for u ∈ [0 , 1] .

Original: Rearranging gives
Revised: Rearranging gives

Original: [Equation]
Revised: [Equation]

Original: where the last step uses Lemma 17.
Revised: where the last step uses Lemma 17.

Original: The case for (cid:98) y t is similar, so we have
Revised: The case for (cid:98) y t is similar, so we have

Original: [Equation]
Revised: [Equation]

Original: Combining this with Eq.
Revised: Combining this with Eq.

Original: (18), we get
Revised: (18), we get

Original: [Equation]
Revised: [Equation]

Original: Now we combine two lower bounds of KL ( (cid:98) z t +1 , z t ) + KL ( z t , (cid:98) z t ) .
Revised: Now we combine two lower bounds of KL ( (cid:98) z t +1 , z t ) + KL ( z t , (cid:98) z t ) .

Original: Using an intermediate step in
Revised: Using an intermediate step in

Original: Case
Revised: Case

Original: [Equation]
Revised: [Equation]

Original: and Eq.
Revised: and Eq.

Original: (19), we get
Revised: (19), we get

Original: [Equation]
Revised: [Equation]

Original:  ≥ (cid:15) 3 η 2 C 2 ξ   1 (cid:15) (cid:107) (cid:98) 
Revised: ≥ (cid:15) 3 η 2 C 2 ξ   1 (cid:15) (cid:107) (cid:98) z

Original: z t +1 − z ∗ (cid:107) 21 + (cid:88) i/ ∈ supp ( z ∗ ) (cid:98) z t +1 ,i 
Revised: t +1 − z ∗ (cid:107) 21 + (cid:88) i/ ∈ supp ( z ∗ ) (cid:98) z t +1 ,i 

Original:  ( ξ ≤ 1 , C ≤ 1 , and (cid:15) ≤ 1 ) ≥ (cid:15) 3 η 2 C 2 ξ
Revised:  ( ξ ≤ 1 , C ≤ 1 , and (cid:15) ≤ 1 ) ≥ (cid:15) 3 η 2 C 2 ξ

Original: KL ( z ∗ , (cid:98) z t +1 ) .
Revised: KL ( z ∗ , (cid:98) z t +1 ) .

Original: (Lemma 16 and Lemma 19)
Revised: (Lemma 16 and Lemma 19)

Original: This proves the second part of the lemma with C 2 = (cid:15) 3 C 2 ξ 2 / 128 .
Revised: This proves the second part of the lemma with C 2 = (cid:15) 3 C 2 ξ 2 / 128 .

Original: Now we are ready to prove Theorem 3.
Revised: Now we are ready to prove Theorem 3.

Original: Proof of Theorem 3.
Revised: Proof of Theorem 3.

Original: As argued in Section 4, with Θ t = KL ( z ∗ , (cid:98) z t ) + 116 KL ( (cid:98)
Revised: As argued in Section 4, with Θ t = KL ( z ∗ , (cid:98) z t ) + 116 KL ( (cid:98)

Original: z t , z t − 1 ) and ζ t =
Revised: z t , z t − 1 ) and ζ t =

Original: KL ( (cid:98) z t +1 , z t ) + KL ( z t , (cid:98) z t ) , we have (see Eq.
Revised: KL ( (cid:98) z t +1 , z t ) + KL ( z t , (cid:98) z t ) , we have (see Eq.

Original: Θ t +1 ≤ Θ t − 1516 ζ t .
Revised: Θ t +1 ≤ Θ t − 1516 ζ t .

Original: We the proceed as,
Revised: We the proceed as,

Original: [Equation]
Revised: [Equation]

Original: by Lemma 17 and Lemma 18)
Revised: (by Lemma 17 and Lemma 18)

Original: [Equation]
Revised: [Equation]

Original: 64 as shown in the proof of Lemma 2)
Revised: 64 as shown in the proof of Lemma 2)

Original: [Equation]
Revised: [Equation]

Original: Therefore, Θ t +1 ≤ Θ t − 15 η 2 C 1Θ 2 t +1 ≤ Θ t − 15 η 2 C 1 64+ln MN Θ 2 t
Revised: Therefore, Θ t +1 ≤ Θ t − 15 η 2 C 1Θ 2 t +1 ≤ Θ t − 15 η 2 C 1 64+ln MN Θ 2 t

Original: +1 .
Revised: +1 .

Original: Also, recall (cid:98) z
Revised: Also, recall (cid:98) z

Original: 1 = z 

Revised: 1 = z

Original: = ( 1 M M , 1 N N ) and thus Θ 1 = KL ( z ∗ , (cid:98) z 1 ) ≤ ln( MN ) .
Revised: = ( 1 M M , 1 N N ) and thus Θ 1 = KL ( z ∗ , (cid:98) z 1 ) ≤ ln( MN ) .

Original: Therefore, the conditions of Lemma 12 are satisﬁed with p = 1 and q = 15 η 2 C 1 64+ln( MN ) , and we conclude that
Revised: Therefore, the conditions of Lemma 12 are satisﬁed with p = 1 and q = 15 η 2 C 1 64+ln( MN ) , and we conclude that

Original: [Equation]
Revised: [Equation]

Original: where C (cid:48)
Revised: where C (cid:48)

Original: =
Revised: =

Original: max (cid:110) ln( MN ) , 128+2ln( MN )
Revised: max (cid:110) ln( MN ) , 128+2ln( MN )

Original: 15 η 
Revised: 15 η 2 C

Original: [Equation]
Revised: [Equation]

Revised: =

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Next we prove the main result.
Revised: Next we prove the main result.

Original: Set T 0 = 12800 C (cid:48)
Revised: Set T 0 = 12800 C (cid:48)

Original: η [Equation]
Revised: η 2 ξ 2 .

Original: For t ≥ T 0 , we have using Pinsker’s inequality,
Revised: For t ≥ T 0 , we have using Pinsker’s inequality,

Original: [Equation]
Revised: [Equation]

Original: Therefore, when t ≥ T 0 , the condition of the second part of Lemma 2 is satisﬁed
Revised: Therefore, when t ≥ T 0 , the condition of the second part of Lemma 2 is satisﬁed

Original: [Equation]
Revised: [Equation]

Original: and we have
Revised: and we have

Original: [Equation]
Revised: [Equation]

Original: (by Lemma 2)
Revised: (by Lemma 2)

Original: [Equation]
Revised: [Equation]

Original: 128 as shown in the proof of Lemma 2)
Revised: 128 as shown in the proof of Lemma 2)

Original: Therefore, when t ≥ T 0 , Θ [Equation] Θ [Equation]
Revised: Therefore, when t ≥ T 0 , Θ t +1 ≤ Θ t −

Original: [Equation]
Revised: [Equation]

Original: Θ t +1 , which further leads to Θ
Revised: Θ t +1 , which further leads to Θ

Original: [Equation]
Revised: [Equation]

Original: where the second inequality uses Eq.
Revised: where the second inequality uses Eq.

Original: The inequality trivially holds for t < T 0 as well, so it holds for all t .
Revised: The inequality trivially holds for t < T 0 as well, so it holds for all t .

Original: We ﬁnish the proof by relating KL ( z ∗ , z t ) and Θ t +1 .
Revised: We ﬁnish the proof by relating KL ( z ∗ , z t ) and Θ t +1 .

Original: Note that by Lemma 16, Lemma
Revised: Note that by Lemma 16, Lemma

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: Lemma 19, we have
Revised: Lemma 19, we have

Original: [Equation]
Revised: [Equation]

Original: We continue to bound the last term as
Revised: We continue to bound the last term as

Original: [Equation]
Revised: [Equation]

Original: Pinsker’s inequality
Revised: inequality)

Original: [Equation]
Revised: [Equation]

Original: Combining everything, we get
Revised: Combining everything, we get

Original: [Equation]
Revised: [Equation]

Original: 0 − t − 1, which completes the proof
Revised: − t − 1, which completes the proof

Original: [Equation]
Revised: [Equation]

Original: E P ROOFS OF L EMMA 4 AND THE S UM - OF - DUALITY - GAP B OUND
Revised: E P ROOFS OF L EMMA 4 AND THE S UM - OF - DUALITY - GAP B OUND

Original: Proof of Lemma 4.
Revised: Proof of Lemma 4.

Original: Below we consider any z (cid:48) (cid:54) =
Revised: Below we consider any z (cid:48) (cid:54) =

Original: (cid:98)
Revised: (cid:98)

Original: z t +1 ∈ Z .
Revised: z t +1 ∈ Z .

Original: Considering Eq.
Revised: Considering Eq.

Original: (1) with D ψ ( u , v ) =
Revised: (1) with D ψ ( u , v ) =

Original: 12 (cid:107) u − v (cid:107)
Revised: 12 (cid:107) u − v (cid:107)

Original: [Equation]
Revised: [Equation]

Original: and using the ﬁrst-order optimality condition of (cid:98) z t +
Revised: and using the ﬁrst-order optimality condition of (cid:98) z t +

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Rearranging the terms and we get
Revised: Rearranging the terms and we get

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Here, for both block, the third step uses H¨older’s inequality and the smoothness condition Assumption 1, and the last step uses the condition η ≤ 1 / (8 L ) .
Revised: Here, for both block, the third step uses H¨older’s inequality and the smoothness condition Assumption 1, and the last step uses the condition η ≤ 1 / (8 L ) .

Original: Upper bounding the left-hand side of the two inequalities by (cid:107) (cid:98) z t +1 − (cid:98)
Revised: Upper bounding the left-hand side of the two inequalities by (cid:107) (cid:98) z t +1 − (cid:98)

Original: z t (cid:107)(cid:107) (cid:98) z t
Revised: z t (cid:107)(cid:107) (cid:98) z t

Original: +

Revised: +

Original: −
Revised: −

Original: z (cid:48) (cid:107) and (cid:107) z t +1 − (cid:98) z t +1 (cid:107)(cid:107)
Revised: z (cid:48) (cid:107) and (cid:107) z t +1 − (cid:98) z t +1 (cid:107)(cid:107)

Original: z t +1 − z
Revised: z t +1 − z

Original: [Equation]
Revised: [Equation]

Original: (cid:107) respectively and then rearranging, we get
Revised: (cid:107) respectively and then rearranging, we get

Original: [Equation]
Revised: [Equation]

Revised: 1 − z (cid:48) (cid:107) (cid:18) (cid:107) (cid:98)

Original: Therefore
Revised: z t +1 − (cid:98) z t (cid:107) [Equation] 1 8 (cid:107) z t −

Revised: [Equation]

Revised: t +

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: [Equation]

Revised: (cid:48)

Original: [Equation]
Revised: [Equation]

Revised: , (cid:107) z t +1 − z (cid:48) (cid:107) (cid:18) (cid:107) z t +1 − (cid:98) z t +1 (cid:107) + 1 8 (cid:107) z t − z t

Revised: +1 (cid:107) (cid:19) ≥ ηF ( z t +1 ) (cid:62) ( z t +1 − z (cid:48) ) .

Original:  we have 
Revised: Therefore, we have [Equation] η [Equation]

Revised: + (cid:107)

Original: [Equation]
Revised: [Equation]

Original: Finally, by the triangle inequality and the fact ( a + b ) 2 ≤ 2 a 

Revised: Finally, by the triangle inequality and the fact ( a + b ) 2 ≤ 2 a

Original: + 2 b 2 , we have
Revised: + 2 b 2 , we have

Original: [Table]
Revised: [Table]

Original: [Equation]

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: which ﬁnishes the proof.
Revised: which ﬁnishes the proof.

Original: Next, we use Eq.
Revised: Next, we use Eq.

Original: (4) and Eq.
Revised: (4) and Eq.

Original: (6) to derive a result on the convergence of “average duality gap” across time.
Revised: (6) to derive a result on the convergence of “average duality gap” across time.

Original: First, we use the following lemma to relate the right-hand side of Eq.
Revised: First, we use the following lemma to relate the right-hand side of Eq.

Original: (6) to the duality gap of z t .
Revised: (6) to the duality gap of z t .

Original: Lemma 20.
Revised: Lemma 20.

Original: Let Z be closed and bounded.
Revised: Let Z be closed and bounded.

Original: Then for any z ∈ Z , we have α f ( z ) ≤ max 
Revised: Then for any z ∈ Z , we have α f ( z ) ≤ max [Equation] F ( z [Equation]

Revised: (cid:62) ( z − z (cid:48)

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: This is a direct consequence of the convexity of f ( · , y ) and the concavity of f ( x , · ) : αf ( z
Revised: This is a direct consequence of the convexity of f ( · , y ) and the concavity of f ( x , · ) : α

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: With Lemma 20, the following theorem can be proven straightforwardly.
Revised: With Lemma 20, the following theorem can be proven straightforwardly.

Original: Theorem 21.
Revised: Theorem 21.

Original: Let Z be closed and bounded.
Revised: Let Z be closed and bounded.

Original: Then OGDA with η ≤ 18 L ensures 1 T (cid:80)
Revised: Then OGDA with η ≤ 18 L ensures 1 T (cid:80)

Original: Tt =

Revised: Tt =

Original: α f ( z t ) =
Revised: α f ( z t ) =

Original: O (cid:16) Dη √ T (cid:17) for any T , where D (cid:44) sup z , z (cid:48) ∈Z (cid:107)
Revised: O (cid:16) Dη √ T (cid:17) for any T , where D (cid:44) sup z , z (cid:48) ∈Z (cid:107)

Original: z − z (cid:48) (cid:107) .
Revised: z − z (cid:48) (cid:107) .

Original: Proof.
Revised: Proof.

Original: We ﬁrst bound the sum of squared duality gap as (recall ζ t = (cid:107)
Revised: We ﬁrst bound the sum of squared duality gap as (recall ζ t = (cid:107)

Original: [Equation]
Revised: [Equation]

Original: Finally, by Cauchy-Schwarz inequality, we get 1 T (cid:80)
Revised: Finally, by Cauchy-Schwarz inequality, we get 1 T (cid:80)

Original: Tt =

Revised: Tt =

Original: α f ( z t ) ≤ [-1 T (cid:113) T (cid:80) Tt =
 α f ( z t ) 2 =-]
Revised: α f ( z t ) ≤ [-1 T (cid:113) T (cid:80) Tt =
 α f ( z t ) 2 =-][Equation]

Original: O (cid:16) Dη √ T (cid:17) .
Revised: O (cid:16) Dη √ T (cid:17) .

Original: This theorem indicates that α f ( z t ) is converging to zero.
Revised: This theorem indicates that α f ( z t ) is converging to zero.

Original: A rate of α f ( z t ) =
Revised: A rate of α f ( z t ) =

Original: O ( Dη √ t ) would be compatible with the theorem, but is not directly implied by it.
Revised: O ( Dη √ t ) would be compatible with the theorem, but is not directly implied by it.

Original: In a recent work, Golowich et al.
Revised: In a recent work, Golowich et al.

Original: (2020b) consider the unconstrained setting and show that the extra-gradient algorithm obtains the rate
Revised: (2020b) consider the unconstrained setting and show that the extra-gradient algorithm obtains the rate

Original: α f ( z t ) =
Revised: α f ( z t ) =

Original: O ( Dη √ t ) , under an extra assumption that the Hessian of f is also Lipschitz (since
Revised: O ( Dη √ t ) , under an extra assumption that the Hessian of f is also Lipschitz (since

Original: Golowich et al.
Revised: Golowich et al.

Original: (2020b) study the unconstrained setting, their duality gap
Revised: (2020b) study the unconstrained setting, their duality gap

Original: α f is deﬁned only with respect to the best responses that lie within a ball of radius D centered around the equilibrium).
Revised: α f is deﬁned only with respect to the best responses that lie within a ball of radius D centered around the equilibrium).

Original: Note that the extra-gradient algorithm requires more cooperation between the two players compared to
Revised: Note that the extra-gradient algorithm requires more cooperation between the two players compared to

Original: OGDA and is less suitable for a repeated game setting.
Revised: OGDA and is less suitable for a repeated game setting.

Original: F T HE E QUIVALENCE BETWEEN SP-MS AND M ETRIC S UBREGULARITY
Revised: F T HE E QUIVALENCE BETWEEN SP-MS AND M ETRIC S UBREGULARITY

Original: In this section, we formally that show our SP-MS condition with β = 0 is equivalent to metric subregularity.
Revised: In this section, we formally that show our SP-MS condition with β = 0 is equivalent to metric subregularity.

Original: Before introducing the main theorem, we introduce several deﬁnitions.
Revised: Before introducing the main theorem, we introduce several deﬁnitions.

Original: We let Z ∗ ⊆
Revised: We let Z ∗ ⊆

Original: Z ⊆ R K ( Z ∗ and Z follow the same deﬁnitions as in our main text).
Revised: Z ⊆ R K ( Z ∗ and Z follow the same deﬁnitions as in our main text).

Original: First, we deﬁne the elementto-set distance function d :
Revised: First, we deﬁne the elementto-set distance function d :

Original: Deﬁnition 5.
Revised: Deﬁnition 5.

Original: The element-to-set distance function d : R K × 2 R K
Revised: The element-to-set distance function d : R K × 2 R K

Original: → R is deﬁned as d ( z , S ) =
Revised: → R is deﬁned as d ( z , S ) =

Original: inf z (cid:48) ∈S (cid:107)
Revised: inf z (cid:48) ∈S (cid:107)

Original: z − z (cid:48) (cid:107) .
Revised: z − z (cid:48) (cid:107) .

Original: The deﬁnition of metric subregularity involves a set-valued operator T : Z → 2 R K , which maps an element of Z to a set in R K .
Revised: The deﬁnition of metric subregularity involves a set-valued operator T : Z → 2 R K , which maps an element of Z to a set in R K .

Original: Deﬁnition 6.
Revised: Deﬁnition 6.

Original: A set-valued operator T is called metric subregular at (¯ z , v ) for v ∈ T (¯ z ) if there exists κ > 0 and a neighborhood Ω of ¯ z such that
Revised: A set-valued operator T is called metric subregular at (¯ z , v ) for v ∈ T (¯ z ) if there exists κ > 0 and a neighborhood Ω of ¯ z such that

Original: [Equation]
Revised: [Equation]

Original: for all z ∈ Ω , where x ∈ T − 1 ( v ) ⇔ v ∈ T ( x ) .
Revised: for all z ∈ Ω , where x ∈ T − 1 ( v ) ⇔ v ∈ T ( x ) .

Original: If Ω = Z , we call T globally metric subregular .
Revised: If Ω = Z , we call T globally metric subregular .

Original: The following deﬁnition of normal cone is also required in the analysis:
Revised: The following deﬁnition of normal cone is also required in the analysis:

Original: Deﬁnition 7.
Revised: Deﬁnition 7.

Original: The normal cone of Z at point z is N ( z ) =
Revised: The normal cone of Z at point z is N ( z ) =

Original: { g | g (cid:62) ( z (cid:48) − z ) ≤ 0 , ∀ z (cid:

Revised: { g | g (cid:62) ( z (cid:48) − z ) ≤ 0 , ∀ z (cid:

Original: ) ∈
Revised: ) ∈

Original: Z} (we omit its dependence on Z for simplicity).
Revised: Z} (we omit its dependence on Z for simplicity).

Original: Equivalently, N ( z ) is the polar cone of the convex set
Revised: Equivalently, N ( z ) is the polar cone of the convex set

Original: Z − z (a property that we will use in the proof).
Revised: Z − z (a property that we will use in the proof).

Original: Now we are ready to show that our SP-MS condition with β = 0 is equivalent to metric subregularity of the operator N + F , deﬁned via: ( N + F )( z ) = { g + F ( z ) | g ∈ N ( z ) } .
Revised: Now we are ready to show that our SP-MS condition with β = 0 is equivalent to metric subregularity of the operator N + F , deﬁned via: ( N + F )( z ) = { g + F ( z ) | g ∈ N ( z ) } .

Original: Theorem 22.
Revised: Theorem 22.

Original: Let z ∗
Revised: Let z ∗

Original: ∈
Revised: ∈

Original: Z ∗ .
Revised: Z ∗ .

Original: Then the following two statements are equivalent:
Revised: Then the following two statements are equivalent:

Original: • ( N + F ) is globally metric subregular at ( z ∗ , 0 ) with κ > 0 ; • For all z ∈ Z\Z ∗ , max z (cid:48)
Revised: • ( N + F ) is globally metric subregular at ( z ∗ , 0 ) with κ > 0 ; • For all z ∈ Z\Z ∗ , max z (cid:48)

Original: ∈Z F ( z ) (cid:62) ( z − z (cid:48) )
Revised: ∈Z F ( z ) (cid:62) ( z − z (cid:48) )

Original: (cid:107)
Revised: (cid:107)

Original: z − z (cid:48) (cid:107) ≥ κd ( z , Z ∗ ) .
Revised: z − z (cid:48) (cid:107) ≥ κd ( z , Z ∗ ) .

Original: Proof.
Revised: Proof.

Original: Let T = N + F .
Revised: Let T = N + F .

Original: Notice that
Revised: Notice that

Original: [Equation]
Revised: [Equation]

Original: Therefore, 0 ∈ T ( z ∗ ) indeed holds, and we have T − 1 ( 0 ) =
Revised: Therefore, 0 ∈ T ( z ∗ ) indeed holds, and we have T − 1 ( 0 ) =

Original: Z ∗ .
Revised: Z ∗ .

Original: This means that the ﬁrst statement in the theorem is equivalent to
Revised: This means that the ﬁrst statement in the theorem is equivalent to

Original: [Equation]
Revised: [Equation]

Original: This inequality holds trivially when z ∈ Z ∗ .
Revised: This inequality holds trivially when z ∈ Z ∗ .

Original: Thus, to complete the proof, it sufﬁces to prove that d ( 0 , N
Revised: Thus, to complete the proof, it sufﬁces to prove that d [Equation] 0 , N

Original: [Equation]
Revised: [Equation]

Original: z )
Revised: z )

Original: + F ( z ))
Revised: [Equation] ( z ))

Original: =
Revised: =

Original: max z (cid:48) ∈Z F ( z ) (cid:62) ( z − z (cid:48) )
Revised: max z (cid:48) ∈Z F ( z ) (cid:62) ( z − z (cid:48) )

Original: (cid:107)
Revised: (cid:107)

Original: z − z (cid:48) (cid:107) for z ∈ Z\Z ∗ .
Revised: z − z (cid:48) (cid:107) for z ∈ Z\Z ∗ .

Original: To do so, note that d ( 0 , N ( z ) [Equation] F ( z ))
Revised: To do so, note that d ( 0 , N ( z ) + F ( z ))

Original: =
Revised: =

Original: d ( − F ( z ) , N ( z ))
Revised: d ( − F ( z ) , N ( z ))

Original: =
Revised: =

Original: (cid:107) [Equation] F ( z ) − Π N ( z ) ( − F
Revised: (cid:107) − F ( z ) − Π N ( z ) ( − F

Original: ( z )) (cid:107) =
Revised: ( z )) (cid:107) =

Original: (cid:107) Π N ◦
Revised: (cid:107) Π N ◦

Original: ( z ) ( − F ( z )) (cid:107) where N ◦ ( z ) =
Revised: ( z ) ( − F ( z )) (cid:107) where N ◦ ( z ) =

Original: { g | g (cid:62) n ≤ 0 , ∀ n ∈ N ( z ) } is the polar cone of N ( z ) and the last step is by
Revised: { g | g (cid:62) n ≤ 0 , ∀ n ∈ N ( z ) } is the polar cone of N ( z ) and the last step is by

Original: Moreau’s theorem.
Revised: Moreau’s theorem.

Original: Now consider the projection of − F ( z ) onto the polar cone N ◦ ( z ) :
Revised: Now consider the projection of − F ( z ) onto the polar cone N ◦ ( z ) :

Original: [Equation]
Revised: [Equation]

Original: where the last equality is because N ◦ ( z ) is a cone.
Revised: where the last equality is because N ◦ ( z ) is a cone.

Original: Next, we ﬁnd the ¯ z ∗ and λ ∗ that realize the last argmin operator: notice that the objective is increasing in F ( z ) (cid:62)
Revised: Next, we ﬁnd the ¯ z ∗ and λ ∗ that realize the last argmin operator: notice that the objective is increasing in F ( z ) (cid:62)

Original: ¯ z , so ¯ z ∗ =
Revised: ¯ z , so ¯ z ∗ =

Original: argmin ¯ z ∈N ◦ ( z ): (cid:107) ¯ z (cid:107) =1 (cid:8) F ( z ) (cid:62) ¯
Revised: argmin ¯ z ∈N ◦ ( z ): (cid:107) ¯ z (cid:107) =1 (cid:8) F ( z ) (cid:62) ¯

Original: z (cid:9) , and thus λ ∗ = − F ( z ) (cid:62) ¯ z ∗ when F ( z ) (cid:62) ¯ z ∗ ≤ 0 and λ ∗ =otherwise.
Revised: z (cid:9) , and thus λ ∗ = − F ( z ) (cid:62) ¯ z ∗ when F ( z ) (cid:62) ¯ z ∗ ≤ 0 and λ ∗ =otherwise.

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: Note that N ( z ) is the polar cone of the conic hull of Z − z .
Revised: Note that N ( z ) is the polar cone of the conic hull of Z − z .

Original: Therefore, N ◦ ( z ) =
Revised: Therefore, N ◦ ( z ) =

Original: ( ConicHull ( Z − z )) ◦◦ =
Revised: ( ConicHull ( Z − z )) ◦◦ =

Original: ConicHull ( Z − z ) and max
Revised: ConicHull ( Z − z ) and max

Original: [Equation]
Revised: [Equation]

Original: Finally, note that when z ∈ Z\Z ∗ , we have max z (cid:48) ∈Z F ( z ) (cid:62) ( z − z (cid:48) ) > 0 .
Revised: Finally, note that when z ∈ Z\Z ∗ , we have max z (cid:48) ∈Z F ( z ) (cid:62) ( z − z (cid:48) ) > 0 .

Original: Combining all the facts above, we have shown d ( 0 , N ( z ) + F ( z ))
Revised: Combining all the facts above, we have shown d ( 0 , N ( z ) + F ( z ))

Original: =
Revised: =

Original: max z (cid:48) ∈Z F ( z )(cid:62) ( z − z (cid:48) )
Revised: max z [Equation] ∈Z F ( z )

Original: (cid:107)
Revised: [Equation]

Original: z − z (cid:48) (cid:107) .
Revised: z − z (cid:48) (cid:107) .

Original: G P ROOF OF T HEOREM
Revised: G P ROOF OF T HEOREM

Original: Proof of Theorem 5.
Revised: Proof of Theorem 5.

Original: Let ρ = min x ∈X max y ∈Y x (cid:62)
Revised: Let ρ = min x ∈X max y ∈Y x (cid:62)

Original: Gy
Revised: Gy

Original: = max y ∈Y min x ∈X x (cid:62) Gy be the game value.
Revised: = max y ∈Y min x ∈X x (cid:62) Gy be the game value.

Original: In this proof, we prove that there exists some c > 0 such that
Revised: In this proof, we prove that there exists some c > 0 such that

Original: [Equation]
Revised: [Equation]

Original: (20) for all x ∈ X .
Revised: [Equation] for all x ∈ X .

Original: Similarly we prove
Revised: Similarly we prove

Original: [Equation]
Revised: [Equation]

Original: for all y ∈ Y .
Revised: for all y ∈ Y .

Original: Assume that the diameter of the polytope is D < ∞ .
Revised: Assume that the diameter of the polytope is D < ∞ .

Original: Then combining the two proves 
Revised: Then combining the two proves max z

Original: [Equation]
Revised: [Equation]

Original: z
Revised: z

Original: [Equation]
Revised: [Equation]

Original: meaning that SP-MS holds with β = 0 .
Revised: meaning that SP-MS holds with β = 0 .

Original: We break the proof into following several claims.
Revised: We break the proof into following several claims.

Original: Claim 1.
Revised: Claim 1.

Original: If X , Y are polytopes, then X ∗ and Y ∗ are also polytopes.
Revised: If X , Y are polytopes, then X ∗ and Y ∗ are also polytopes.

Original: Proof of Claim 1.
Revised: Proof of Claim 1.

Original: Note that X ∗ = (cid:8) x ∈ X : max y ∈Y x (cid:62) Gy ≤ ρ (cid:9) .
Revised: Note that X ∗ = (cid:8) x ∈ X : max y ∈Y x (cid:62) Gy ≤ ρ (cid:9) .

Original: Since Y is a polytope, the maximum is attained at vertices of Y .
Revised: Since Y is a polytope, the maximum is attained at vertices of Y .

Original: Therefore, X ∗ can be equivalently written as (cid:8) x ∈ X : max y ∈V ( Y ) x (cid:62)
Revised: Therefore, X ∗ can be equivalently written as (cid:8) x ∈ X : max y ∈V ( Y ) x (cid:62)

Original: Gy ≤ ρ (cid:9) , where V ( Y ) is the set of vertices of Y .
Revised: Gy ≤ ρ (cid:9) , where V ( Y ) is the set of vertices of Y .

Original: Since the constraints of
Revised: Since the constraints of

Original: X ∗ are all linear constraints, X ∗ is a polytope.
Revised: X ∗ are all linear constraints, X ∗ is a polytope.

Original: With Claim 1, we without loss of generality write X ∗ asX ∗ = (cid:8) [Equation] M : a
Revised: With Claim 1, we without loss of generality write X ∗ as

Original: [Equation]
Revised: [Equation]

Original: i x ≤ b i , for i = 1 , . . .

Original: , L, c

Original: [Equation]
Revised: [Equation]

Original: x [Equation] d i , for i = 1 , . . . , K (cid:9) , where the a (cid:62) i x ≤ b i constraints come from x ∈ X and the c (cid:62) i x ≤ d i constraints come from max y ∈V ( Y ) x (cid:62) 
Revised: where the a (cid:62) i x ≤ b i constraints come from x ∈ X and the c (cid:62) i x ≤ d i constraints come from max y ∈V ( Y ) x (cid:62) Gy ≤ ρ .

Original: Gy ≤ ρ .

Original: Below, we refer to a (cid:62) i x ≤
Revised: Below, we refer to a (cid:62) i x ≤

Original: b i as the feasibility constraints , and c (cid:62) i x ≤ d i as the optimality constraints .
Revised: b i as the feasibility constraints , and c (cid:62) i x ≤ d i as the optimality constraints .

Original: In fact, one can identify the i -th optimality constraint as c i =
Revised: In fact, one can identify the i -th optimality constraint as c i =

Original: Gy ( i ) and d i = ρ , where y ( i ) is the i -th vertex of Y .
Revised: Gy ( i ) and d i = ρ , where y ( i ) is the i -th vertex of Y .

Original: This is based on our construction of X ∗ in the proof of Claim 1.
Revised: This is based on our construction of X ∗ in the proof of Claim 1.

Original: Therefore, K = |V ( Y ) | .
Revised: Therefore, K = |V ( Y ) | .

Original: Since Eq.
Revised: Since Eq.

Original: (20) clearly holds for x ∈ X ∗ , below, we focus on an x ∈ X\X ∗ , and let x ∗ (cid:44) Π X ∗ ( x ) .
Revised: (20) clearly holds for x ∈ X ∗ , below, we focus on an x ∈ X\X ∗ , and let x ∗ (cid:44) Π X ∗ ( x ) .

Original: We say a constraint is tight at x ∗ if a (cid:62) i x ∗ =
Revised: We say a constraint is tight at x ∗ if a (cid:62) i x ∗ =

Original: b i or c (cid:62) i x ∗ =
Revised: b i or c (cid:62) i x ∗ =

Original: d i .
Revised: d i .

Original: Below we assume that there are (cid:96) tight feasibility constraints at and k tight optimality constraints at x ∗ .
Revised: Below we assume that there are (cid:96) tight feasibility constraints at and k tight optimality constraints at x ∗ .

Original: Without loss of generality, we assume these tight constraints correspond to i
Revised: Without loss of generality, we assume these tight constraints correspond to i

Original: = 1 , . .
Revised: = 1 , . .

Original: , (cid:96) and i = 1 , . . . ,
Revised: , (cid:96) and i = 1 , . . . ,

Original: k respectively.
Revised: k respectively.

Original: That is, a (cid:62) i x ∗ =
Revised: That is, a (cid:62) i x ∗ =

Original: b i , for i = 1 , . . .
Revised: b i , for i = 1 , . . .

Original: , (cid:96), c
Revised: , (cid:96), c

Original: [Equation]
Revised: [Equation]

Original: i x ∗ = 
Revised: i [Equation] = [Equation]

Original: d i , for i = 1 , . ..
Revised: i , for i = 1 , . .

Original: , k.
Revised: , k.

Original: Claim 2.
Revised: Claim 2.

Original: x violates at least one of the tight optimality constraint at x ∗ .
Revised: x violates at least one of the tight optimality constraint at x ∗ .

Original: Proof of Claim 2.
Revised: Proof of Claim 2.

Original: We prove this by contradiction.
Revised: We prove this by contradiction.

Original: Suppose that x satisﬁes all k tight optimality constraints at x ∗ .
Revised: Suppose that x satisﬁes all k tight optimality constraints at x ∗ .

Original: Then x must violates some of the remaining K − k optimality constraints (otherwise x ∈ X ∗ ).
Revised: Then x must violates some of the remaining K − k optimality constraints (otherwise x ∈ X ∗ ).

Original: Assume that it violates constraints K − n + 1 , . .
Revised: Assume that it violates constraints K − n + 1 , . .

Original: , K for some 1 ≤ n ≤ K − k .
Revised: , K for some 1 ≤ n ≤ K − k .

Original: Thus, we have the following:
Revised: Thus, we have the following:

Original: [Equation]
Revised: [Equation]

Original: [Equation] for [Equation] K
Revised: i =

Original: [Equation]
Revised: [Equation]

Original: n

Original: [Equation] [Equation] K. Recall that c (cid:62) i x ∗ ≤ d i for i = 1 , . .
Revised: Recall that c (cid:62) i x ∗ ≤ d i for i = 1 , . .

Original: , K − n and c (cid:62) i x ∗ <
Revised: , K − n and c (cid:62) i x ∗ <

Original: d i for all
Revised: d i for all

Original: i
Revised: i

Original: = K − n + 1 , . .
Revised: = K − n + 1 , . .

Original: , K .
Revised: , K .

Original: Thus, there exists some x (cid:48) that lies strictly between x and x ∗ that makes all constraints hold (notice that x and x ∗ both satisfy all feasibility constraints), which contradicts with Π X ∗ ( x ) = x ∗ . 
Revised: Thus, there exists some x (cid:48) that lies strictly between x and x ∗ that makes all constraints hold (notice that x and x ∗ both satisfy all feasibility constraints), which contradicts with Π X ∗ ( x ) = x ∗ . Claim 3.

Original: [Equation]
Revised: [Equation]

Original: Proof of Claim 3.
Revised: Proof of Claim 3.

Original: Recall that we identify c i with Gy ( i ) and d i = ρ .
Revised: Recall that we identify c i with Gy ( i ) and d i = ρ .

Original: Therefore, 
Revised: Therefore, max

Original: [Equation]
Revised: [Equation]

Original: where the last equality is because c (cid:62) i x ∗
Revised: where the last equality is because c (cid:62) i x ∗

Original: =
Revised: =

Original: d i for i = 1 , . . .
Revised: d i for i = 1 , . . .

Original: , k .
Revised: , k .

Original: Recall from linear programming literature Davis (2016a;b) that the normal cone of X ∗ at x ∗ is expressed as follows: 
Revised: Recall from linear programming literature Davis (2016a;b) that the normal cone of X ∗ at x ∗ is expressed as follows: [Equation] p [Equation]

Revised: (cid:88) i =

Original: [Equation]
Revised: [Equation]

Revised: q

Revised: [Equation]

Revised: c

Revised: [Equation]

Revised: p

Revised: [Equation]

Revised: [Equation]

Original:  The normal cone of X ∗ at x ∗ consists of all outgoing normal vectors of X ∗ originated from x ∗ .
Revised: q [Equation] (cid:41) . The normal cone of X ∗ at x ∗ consists of all outgoing normal vectors of X ∗ originated from x ∗ .

Original: Clearly, x − x ∗ belongs to N x ∗ .
Revised: Clearly, x − x ∗ belongs to N x ∗ .

Original: However, besides the fact that x − x ∗ is a normal vector of X ∗ , we also have the additional constraints that x ∈ X .
Revised: However, besides the fact that x − x ∗ is a normal vector of X ∗ , we also have the additional constraints that x ∈ X .

Original: We claim that in our case, x − x ∗ lies in the following smaller cone (which is a subset of N x ∗ ):
Revised: We claim that in our case, x − x ∗ lies in the following smaller cone (which is a subset of N x ∗ ):

Original: Claim 4.
Revised: Claim 4.

Original: x − x ∗ belongs to
Revised: x − x ∗ belongs to

Original: [Equation]
Revised: [Equation]

Original: i [Equation] p [Equation]
Revised: Proof of Claim 4.

Original: a [Equation] i
Revised: As argued above, x − x ∗ ∈ N x ∗ , and thus x − x ∗ can be expressed as

Original: [Equation]
Revised: [Equation]

Original: ∀ j
Revised: ki

Original: [Equation]
Revised: [Equation]

Original: Proof of Claim 4.

Original: As argued above, x − x ∗ ∈ N x ∗ , and thus x − x ∗ can be expressed as (cid:80) (cid:96)i =1 p i a i + (cid:80) ki

Original: =1 q i c i with p i ≥ 0 , q i ≥ 0 .
Revised: 1 q i c i with p i ≥ 0 , q i ≥ 0 .

Original: To prove that x − x ∗ ∈ M x ∗ , we only need to prove that it satisﬁes the additional constraints, that is,a (cid:62) [Equation] ( x
Revised: To prove that x − x ∗ ∈ M x ∗ , we only need to prove that it satisﬁes the additional constraints, that is,

Original: [Equation]
Revised: [Equation]

Original: , ∀ i = 1 , . . .

Original: , (cid:96).

Original: This is shown by noticing that for all i = 1 , . . .
Revised: This is shown by noticing that for all i = 1 , . . .

Original: , (cid:96) ,
Revised: , (cid:96) ,

Original: [Equation]
Revised: [Equation]

Original: (the i -th constraint is tight at x∗ )
Revised: the [Equation] -th constraint is tight at x

Original: [Equation]
Revised: [Equation]

Original: ( x ∈ X )

Original: Claim 5.
Revised: Claim 5.

Original: x − x ∗ can be written as (cid:80) (cid:96)i =1 p
Revised: x − x ∗ can be written as (cid:80) (cid:96)i =1 p

Original: i a i + (cid:80) ki
Revised: i a i + (cid:80) ki

Original: =1 q i c
Revised: =1 q i c

Original: i with 0 ≤ p i , q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) for all i and some problem-dependent constant C (cid:48) <
Revised: i with 0 ≤ p i , q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) for all i and some problem-dependent constant C (cid:48) <

Original: ∞ .
Revised: ∞ .

Original: Proof of Claim 5.
Revised: Proof of Claim 5.

Original: Notice that x − x ∗ (cid:107) x − x ∗ (cid:107) ∈
Revised: Notice that x − x ∗ (cid:107) x − x ∗ (cid:107) ∈

Original: M x ∗ (because 0 (cid:54) =
Revised: M x ∗ (because 0 (cid:54) =

Original: x − x ∗ ∈ M x ∗ and M x ∗ is a cone).
Revised: x − x ∗ ∈ M x ∗ and M x ∗ is a cone).

Original: Furthermore, x − x ∗ (cid:107) x − x ∗ (cid:107) ∈ { v ∈ R M : (cid:107) v (cid:107) ∞ ≤ 1 } .
Revised: Furthermore, x − x ∗ (cid:107) x − x ∗ (cid:107) ∈ { v ∈ R M : (cid:107) v (cid:107) ∞ ≤ 1 } .

Original: Therefore, x − x ∗ (cid:107) x − x ∗ (cid:107) ∈
Revised: Therefore, x − x ∗ (cid:107) x − x ∗ (cid:107) ∈

Original: M x ∗ ∩ { v ∈ R M : (cid:107) v (cid:107) ∞ ≤ 1 } , which is a bounded subset of the cone M x ∗ .
Revised: M x ∗ ∩ { v ∈ R M : (cid:107) v (cid:107) ∞ ≤ 1 } , which is a bounded subset of the cone M x ∗ .

Original: Below we argue that there exists a large enough C (cid:48) > 0 such that
Revised: Below we argue that there exists a large enough C (cid:48) > 0 such that

Original: [Equation]
Revised: [Equation]

Original:  To see this, ﬁrst note that P is a polytope.
Revised: P . To see this, ﬁrst note that P is a polytope.

Original: For every vertex (cid:98) v of P , the smallest C (cid:48) such that (cid:98) v belongs to the left-hand side is the solution of the following linear programming:
Revised: For every vertex (cid:98) v of P , the smallest C (cid:48) such that (cid:98) v belongs to the left-hand side is the solution of the following linear programming:

Original: [Equation]
Revised: [Equation]

Original: Since (cid:98) v ∈ M x ∗ , this linear programming is always feasible and admits a ﬁnite solution C (cid:48) (cid:98) v < ∞ .
Revised: Since (cid:98) v ∈ M x ∗ , this linear programming is always feasible and admits a ﬁnite solution C (cid:48) (cid:98) v < ∞ .

Original: Now let C (cid:48)
Revised: Now let C (cid:48)

Original: = max (cid:98) v ∈V ( P ) C (cid:48)
Revised: = max (cid:98) v ∈V ( P ) C (cid:48)

Original: (cid:98) v , where V ( P ) is the set of all vertices of P .
Revised: (cid:98) v , where V ( P ) is the set of all vertices of P .

Original: Then since any v ∈ P can be expressed as a convex combination of points in V ( P ) , v can be also be expressed as (cid:80)[Equation] with
Revised: Then since any v ∈ P can be expressed as a convex combination of points in V ( P ) , v can be also be expressed as (cid:80)

Original: [Equation]
Revised: [Equation]

Original: , q [Equation] ≤ C (cid:48) .
Revised: (cid:48) .

Original: To sum up, x − x ∗ (cid:107) x − x ∗ (cid:107) can be represented as (cid:80) (cid:96)i =1 p
Revised: To sum up, x − x ∗ (cid:107) x − x ∗ (cid:107) can be represented as (cid:80) (cid:96)i =1 p

Original: i a i + (cid:80) ki
Revised: i a i + (cid:80) ki

Original: =1 q i c
Revised: =1 q i c

Original: i with 0 ≤ p i , q i ≤ C (cid:48) .
Revised: i with 0 ≤ p i , q i ≤ C (cid:48) .

Original: This further implies that x − x ∗ can be represented as (cid:80) (cid:96)i =1 p
Revised: This further implies that x − x ∗ can be represented as (cid:80) (cid:96)i =1 p

Original: i a i + (cid:80) ki
Revised: i a i + (cid:80) ki

Original: =1 q i c
Revised: =1 q i c

Original: i with 0 ≤ p i , q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) .
Revised: i with 0 ≤ p i , q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) .

Original: Notice that C (cid:48) only depends on the set of tight constraints at x ∗ .
Revised: Notice that C (cid:48) only depends on the set of tight constraints at x ∗ .

Original: Finally, we are ready to combine all previous claims and prove the desired inequality.
Revised: Finally, we are ready to combine all previous claims and prove the desired inequality.

Original: Deﬁne A i (cid:44) a (cid:62) i ( x − x ∗ ) and C i (cid:44) c (cid:62) i ( x − x ∗ ) .
Revised: Deﬁne A i (cid:44) a (cid:62) i ( x − x ∗ ) and C i (cid:44) c (cid:62) i ( x − x ∗ ) .

Original: By Claim 5, we can write x − x ∗ as (cid:80) (cid:96)i =1 p
Revised: By Claim 5, we can write x − x ∗ as (cid:80) (cid:96) i =1 p

Original: i a i + (cid:80) ki
Revised: i a i + (cid:80) k

Original: =1 q i c
Revised: i =

Original:  i with 0 ≤ p i , q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) , and thus, (cid:96) (cid:88) i
Revised: 1 q i c i with 0 ≤ p i , q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) , and thus, (cid:96) (cid:88) i

Original: = p i
Revised: = p i

Original: A
Revised: A

Original: i + k (cid:88) i
Revised: i + k (cid:88) i

Original: =
Revised: =

Original: q i C i
Revised: q i C i

Original: =
Revised: =

Original: (cid:32) (cid:96) (cid:88) i
Revised: (cid:32) (cid:96) (cid:88) i

Original: = p i a i + k (cid:88)
Revised: = p i a i + k (cid:88)

Original: i =
Revised: i =

Original: q i c i (cid:33) (cid:62) ( x − x ∗ ) =
Revised: q i c i (cid:33) (cid:62) ( x − x ∗ ) =

Original: (cid:107) x − x ∗ (cid:107) 2 .
Revised: (cid:107) x − x ∗ (cid:107) 2 .

Original: On the other hand, since x − x ∗ ∈ M x ∗ by Claim 4, we have (cid:96) (cid:88) i
Revised: On the other hand, since x − x ∗ ∈ M x ∗ by Claim 4, we have (cid:96) (cid:88) i

Original: = p
Revised: = p

Original: i
Revised: i

Original: A i
Revised: A i

Original: = (cid:96) (cid:88)
Revised: = (cid:96) (cid:88)

Original: i = p
Revised: i = p

Original: i a (cid:62) i ( x − x ∗ )
Revised: i a (cid:62) i ( x − x ∗ )

Original: ≤and k (cid:88) i =
Revised: ≤and k (cid:88) i =

Original: q i C i ≤ (cid:18) max i ∈{ 1 ,...,k } C
Revised: q i C i ≤ (cid:18) max i ∈{ 1 ,...,k } C

Original: i (cid:19) k (cid:88) i =
Revised: i (cid:19) k (cid:88) i =

Original: [Equation]
Revised: q i ≤ (cid:18) max i ∈{ 1 ,...,k } C i (cid:19) kC (cid:48)

Original:  where in the ﬁrst inequality we use the fact p i ≥ 0 , and in the second inequality we use the fact max i ∈{ 1 ,...,k } C i > 0 (by Claim 2) and 0 ≤ q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) .
Revised: (cid:107) x − x ∗ (cid:107) , where in the ﬁrst inequality we use the fact p i ≥ 0 , and in the second inequality we use the fact max i ∈{ 1 ,...,k } C i > 0 (by Claim 2) and 0 ≤ q i ≤ C (cid:48) (cid:107) x − x ∗ (cid:107) .

Original: Combining the three inequalities above, we get
Revised: Combining the three inequalities above, we get

Original: [Equation]
Revised: [Equation]

Original: Then by Claim 3,
Revised: Then by Claim 3,

Original: [Equation]
Revised: [Equation]

Original: Note that k and C (cid:48) only depend on the set of tight constraints at the projection point x ∗ , and there are only ﬁnitely many different sets of tight constraints.Therefore, we conclude that there exists a constant c > 0 such that max y [Equation] x
Revised: Note that k and C (cid:48) only depend on the set of tight constraints at the projection point x ∗ , and there are only ﬁnitely many different sets of tight constraints.

Original: [Equation] ≥ c [Equation] x − x ∗ (cid:107) holds for all x and x ∗ , which completes the proof.
Revised: Therefore, we conclude that there exists a constant c > 0 such that max y (cid:48) ∈Y (cid:0) x (cid:62) Gy (cid:48) − ρ (cid:1) ≥ c (cid:107) x − x ∗ (cid:107) holds for all x and x ∗ , which completes the proof.

Original: H P ROOF OF T HEOREM 6 AND T HEOREM
Revised: H P ROOF OF T HEOREM 6 AND T HEOREM

Original: Proof of Theorem 6.
Revised: Proof of Theorem 6.

Original: Suppose that f is γ -strongly-convex in x and γ -strongly-concave in y , and let ( x ∗ , y ∗ )
Revised: Suppose that f is γ -strongly-convex in x and γ -strongly-concave in y , and let ( x ∗ , y ∗ )

Original: ∈
Revised: ∈

Original: Z ∗ .
Revised: Z ∗ .

Original: Then for any ( x , y ) we have
Revised: Then for any ( x , y ) we have

Original: [Equation]
Revised: [Equation]

Original: Summing up the two inequalities, and noticing that f ( x , y ∗ ) − f ( x ∗ , y ) ≥ 0 for any ( x ∗ , y ∗ )
Revised: Summing up the two inequalities, and noticing that f ( x , y ∗ ) − f ( x ∗ , y ) ≥ 0 for any ( x ∗ , y ∗ )

Original: ∈
Revised: ∈

Original: Z ∗ , we get
Revised: Z ∗ , we get

Original: [Equation]
Revised: [Equation]

Original: and therefore, for z / ∈ Z ∗ ,
Revised: and therefore, for z / ∈ Z ∗ ,

Original: [Equation]
Revised: [Equation]

Original: which implies SP-MS with β = 0 and C = γ/ 2 .
Revised: which implies SP-MS with β = 0 and C = γ/ 2 .

Original: Proof of Theorem 7.
Revised: Proof of Theorem 7.

Original: First, we show that f has a unique Nash Equilibrium z ∗ =
Revised: First, we show that f has a unique Nash Equilibrium z ∗ =

Original: ( x ∗ , y ∗ ) =
Revised: ( x ∗ , y ∗ ) =

Original: As f is a strictly monotone decreasing function with respect to y 1 , we must have y ∗ 1 = 0 and y ∗ 2 = 1 .
Revised: As f is a strictly monotone decreasing function with respect to y 1 , we must have y ∗ 1 = 0 and y ∗ 2 = 1 .

Original: In addition, if x = (0 , 1) , max y ∈Y f ( x , y ) = − min y ∈Y y 2 n= 0 .
Revised: In addition, if x = (0 , 1) , max y ∈Y f ( x , y ) = − min y ∈Y y 2 n= 0 .

Original: If x (cid:54) = (0 , 1) , then by choosing y ∗ = (0 , 1) , f ( x , y ∗ )
Revised: If x (cid:54) = (0 , 1) , then by choosing y ∗ = (0 , 1) , f ( x , y ∗ )

Original: =
Revised: =

Original: x 2 n 1 > 0 .
Revised: x 2 n 1 > 0 .

Original: Therefore, we have x ∗ =
Revised: Therefore, we have x ∗ =

Original: (0 , 1) , which proves that the unique Nash Equilibrium is x ∗ = (0 , 1) , y ∗ = (0 , 1) .
Revised: (0 , 1) , which proves that the unique Nash Equilibrium is x ∗ = (0 , 1) , y ∗ = (0 , 1) .

Original: Second, we show that f satisﬁes SP-MS with β = 2 n − 2 .
Revised: Second, we show that f satisﬁes SP-MS with β = 2 n − 2 .

Original: In fact, for any z = ( x , y ) (cid:54) =
Revised: In fact, for any z = ( x , y ) (cid:54) =

Original: z ∗ , we have
Revised: z ∗ , we have

Original: [Equation]
Revised: [Equation]

Original: (Jensen’s inequality)
Revised: Jensen’s inequality

Original: [Equation]
Revised: [Equation]

Original: Note that (cid:107) z − z ∗ (cid:107) =
Revised: Note that (cid:107) z − z ∗ (cid:107) =

Original: (cid:112) x 21 + (1 − x 2 ) 2 + y 21 + (1 − y 2 ) 

Revised: (cid:112) x 21 + (1 − x 2 ) 2 + y 21 + (1 − y 2 )

Original: = (cid:112) 2 x 21 + 2 y 21 .
Revised: = (cid:112) 2 x 21 + 2 y 21 .

Original: Therefore, we have
Revised: Therefore, we have

Original: F ( z ) (cid:62) ( z − z ∗ ) (cid:107)
Revised: F [Equation] z − z ∗ [Equation]

Original: z − z ∗ (cid:107) ≥ n 2 2 n − 2 (cid:107) z − z ∗ (cid:107) 2 n − 1 .
Revised: ≥ n 2 2 n − 2 (cid:107) z − z ∗ (cid:107) 2 n − 1 .

Original: This shows that f satisﬁes SP-MS with β = 2 n − 2 and
Revised: This shows that f satisﬁes SP-MS with β = 2 n − 2 and

Original: C = n 2 2 n − 2 .
Revised: C = n 2 2 n − 2 .

Original: I P ROOF OF T HEOREM
Revised: I P ROOF OF T HEOREM

Original: Proof of Theorem 8.
Revised: Proof of Theorem 8.

Original: As argued in Section 5, with Θ t = (cid:107) (cid:98)
Revised: As argued in Section 5, with Θ t = (cid:107) (cid:98)

Original: z t − Π Z ∗ ( (cid:98) z t ) (cid:107) 2 + 116 (cid:107) (cid:98)
Revised: z t − Π Z ∗ ( (cid:98) z t ) (cid:107) 2 + 116 (cid:107) (cid:98)

Original: z
Revised: z

Original: t − z t − 1 (cid:107) 2 , ζ t = (cid:107) (cid:98) z t
Revised: t − z t − 1 (cid:107) 2 , ζ t = (cid:107) (cid:98) z t

Original: +1 − z t (cid:107) 

Revised: +1 − z t (cid:107)

Original: + (cid:107) z t − (cid:98)
Revised: + (cid:107) z t − (cid:98)

Original: z t (cid:107) 2 , we have (see Eq.
Revised: z t (cid:107) 2 , we have (see Eq.

Original: [Equation]
Revised: [Equation]

Original: Below, we relate ζ t to Θ t +1 using the SP-MS condition, and then apply Lemma 12 to show
Revised: Below, we relate ζ t to Θ t +1 using the SP-MS condition, and then apply Lemma 12 to show

Original: [Equation]
Revised: [Equation]

Original: if β >
Revised: t

Original: [Equation]
Revised: [Equation]

Revised: if β > 0 , (22)

Original: where
Revised: where

Original: C 5 = min (cid:110) 16 η 2 C 2, 12 (cid:111) as deﬁned in the statement of the theorem.
Revised: C 5 = min (cid:110) 16 η 2 C 2, 12 (cid:111) as deﬁned in the statement of the theorem.

Original: This is enough to prove the theorem since
Revised: This is enough to prove the theorem since

Original: [Equation]
Revised: [Equation]

Original: Next, we prove Eq.
Revised: Next, we prove Eq.

Original: We ﬁrst show a simple fact by Eq.
Revised: We ﬁrst show a simple fact by Eq.

Original: [Equation]
Revised: [Equation]

Original: Notice that
Revised: Notice that

Original: [Equation]
Revised: [Equation]

Original: ≥ min
Revised: ≥ min

Original: [Equation]
Revised: [Equation]

Original: 32Θ
Revised: 32Θ

Original: [Equation]
Revised: [Equation]

Original: (by H ¨ older’s inequality:
Revised: (by H ¨ older’s inequality

Original: [Equation]
Revised: [Equation]

Original: a
Revised: (recall that C 5 = min

Original: [Equation]
Revised: [Equation]

Original: (recall that C 5 = min {

Original: [Equation]
Revised: [Equation]

Original: deﬁne
Revised: deﬁne

Original: [Equation]
Revised: [Equation]

Original: Combining this with Eq.
Revised: Combining this with Eq.

Original: (21), we get
Revised: (21), we get

Original: [Equation]
Revised: [Equation]

Original: When β = 0 , Eq.
Revised: When β = 0 , Eq.

Original: (24) implies Θ t +1 ≤ (1 + C 5 ) − 1 Θ t , which immediately implies Θ t ≤ (1 +
Revised: (24) implies Θ t +1 ≤ (1 + C 5 ) − 1 Θ t , which immediately implies Θ t ≤ (1 +

Original: C 5 ) − t +1 Θ 1 ≤ 2Θ 1 (1 + C 5 ) − t .
Revised: C 5 ) − t +1 Θ 1 ≤ 2Θ 1 (1 + C 5 ) − t .

Original: When β > 0 , Eq.
Revised: When β > 0 , Eq.

Original: (24) is of the form speciﬁed in Lemma 12 with p = β and q = C (cid:48) .
Revised: (24) is of the form speciﬁed in Lemma 12 with p = β and q = C (cid:48) .

Original: Note that the second required condition is satisﬁed: C (cid:48)
Revised: Note that the second required condition is satisﬁed: C (cid:48)

Original: ( β + 1)Θ β 1 ≤ β +12 · 4 β ≤ 1 .
Revised: ( β + 1)Θ β 1 ≤ β +12 · 4 β ≤ 1 .

Original: Therefore, by the conclusion of Lemma 12,
Revised: Therefore, by the conclusion of Lemma 12,

Original: [Equation]
Revised: [Equation]

Original: β C 5 β (cid:19) 1 β , 4Θ(cid:18)β (cid:19) 1 β (cid:41) t − 1 β ≤ (cid:34)(cid:32) 

Revised: C 5 β (cid:19) 1 β , 4Θ(cid:18)β (cid:19) 1 β (cid:41) t − 1 β ≤ (cid:34)(cid:32)

Original: +(cid:18)β (cid:19) 1 β (cid:33) Θ 1 +(cid:18)
Revised: +(cid:18)β (cid:19) 1 β (cid:33) Θ 1 +(cid:18)

Original: C 5 β (cid:19) 1 β (cid:35) t − 1 β .
Revised: C 5 β (cid:19) 1 β (cid:35) t − 1 β .

Original: J P ROOF OF T HEOREM
Revised: J P ROOF OF T HEOREM

Original: Proof of Theorem 9.
Revised: Proof of Theorem 9.

Original: Consider the following 2 × 2 bilinear game with curved feasible sets:
Revised: Consider the following 2 × 2 bilinear game with curved feasible sets:

Original: [Equation]
Revised: [Equation]

Original: Below, we use Claim 1 - Claim 5 to argue that if the two players start from x 0 = y 

Revised: Below, we use Claim 1 - Claim 5 to argue that if the two players start from x 0 = y

Original: = (cid:98) x 0 = (cid:98) y 

Revised: = (cid:98) x 0 = (cid:98) y

Original: = ( 12 , 14 ) , and use any constant learning rate η ≤ 164 , then the convergence is sublinear in the sense that (cid:107) z t − z ∗ (cid:107) ≥ Ω(1 /t ) .
Revised: = ( 12 , 14 ) , and use any constant learning rate η ≤ 164 , then the convergence is sublinear in the sense that (cid:107) z t − z ∗ (cid:107) ≥ Ω(1 /t ) .

Original: Then, in Claim 6 , we show that in this example, SP-MS holds with β = 3 .
Revised: Then, in Claim 6 , we show that in this example, SP-MS holds with β = 3 .

Original: Claim 1.
Revised: Claim 1.

Original: The unique equilibrium is x ∗ = 0 , y ∗ = 0 .
Revised: The unique equilibrium is x ∗ = 0 , y ∗ = 0 .

Original: When x = 0 , clearly max y (cid:48) ∈Y f ( x , y (cid:48) )
Revised: When x = 0 , clearly max y (cid:48) ∈Y f ( x , y (cid:48) )

Original: = 0 .
Revised: = 0 .

Original: When x (cid:54) = 0 , we prove max y
Revised: When x (cid:54) = 0 , we prove max y

Original: [Equation]
Revised: [Equation]

Original: ∈Y f ( x , y
Revised: ∈Y f [Equation] x , y

Original: [Equation]
Revised: [Equation]

Original: ) >below.
Revised: ) >below.

Original: If x 1 (cid:54) = 0 , we let y (cid:48) 1 = 12 x 1 and y (cid:48) 2 = 14 x 12 (which satisﬁes y (cid:48) ∈ Y ), and thus
Revised: If x 1 (cid:54) = 0 , we let y (cid:48) 1 = 12 x 1 and y (cid:48) 2 = 14 x 12 (which satisﬁes y (cid:48) ∈ Y ), and thus

Original: [Equation]
Revised: [Equation]

Original: If x 1 = 

Revised: If x 1 =

Original: but x 2 (cid:54) =
Revised: but x 2 (cid:54) =

Original: 0 , we let y (cid:48) 1 = 12 , y (cid:48) 2 = 14 , and thus
Revised: 0 , we let y (cid:48) 1 = 12 , y (cid:48) 2 = 14 , and thus

Original: [Equation]
Revised: [Equation]

Original: Thus, max y (cid:48) ∈Y f ( x , y (cid:48) ) > 0 if x (cid:54) = 0 , and x ∗ = 0 is the unique optimal solution for x .
Revised: Thus, max y (cid:48) ∈Y f ( x , y (cid:48) ) > 0 if x (cid:54) = 0 , and x ∗ = 0 is the unique optimal solution for x .

Original: By the symmetry between x and y (because G = − G (cid:62) ), we can also prove that the unique optimal solution for y is y ∗ = 0 .
Revised: By the symmetry between x and y (because G = − G (cid:62) ), we can also prove that the unique optimal solution for y is y ∗ = 0 .

Original: Claim 2.
Revised: Claim 2.

Original: Suppose that x 0 = y 

Revised: Suppose that x 0 = y

Original: = (cid:98) x 0 = (cid:98) y 

Revised: = (cid:98) x 0 = (cid:98) y

Original: = ( 12 , 14 ) .
Revised: = ( 12 , 14 ) .

Original: Then, at any step t ∈ [ T ] , we have x t =
Revised: Then, at any step t ∈ [ T ] , we have x t =

Original: y t and (cid:98) x t = (cid:98) y t , and all x t , y t , (cid:98) x t , (cid:98) y t belong to { u ∈ R 2 : u 2 = u 21 } .
Revised: y t and (cid:98) x t = (cid:98) y t , and all x t , y t , (cid:98) x t , (cid:98) y t belong to { u ∈ R 2 : u 2 = u 21 } .

Original: We prove this by induction.
Revised: We prove this by induction.

Original: The base case trivially holds.
Revised: The base case trivially holds.

Original: Suppose that for step t , we have x t = y t , (cid:98) x t = (cid:98) y t , and x t , y t , (cid:98) x t , (cid:98) y t ∈ { u ∈ R 2 : u 2 = u 21 } .
Revised: Suppose that for step t , we have x t = y t , (cid:98) x t = (cid:98) y t , and x t , y t , (cid:98) x t , (cid:98) y t ∈ { u ∈ R 2 : u 2 = u 21 } .

Original: Then consider step t + 1 .
Revised: Then consider step t + 1 .

Original: According to the dynamic of OGDA, we have
Revised: According to the dynamic of OGDA, we have

Original: [Equation]
Revised: [Equation]

Original: η (cid:20) − y t,y t,(cid:21)(cid:27) =
Revised: (cid:20) − y t,y t,(cid:21)(cid:27) =

Original: Π X (cid:26)(cid:20) (cid:98) x 
Revised: Π X (cid:26)(cid:20) (cid:98) x t +1 ,

Revised: [Equation]

Original: t +1 , 1 + ηy t, 2 , (cid:98) x t +1 , 2 − ηy t,(cid:21)(cid:27) , (cid:98) y t
Revised: ηy t, 2 , (cid:98) x t +1 , 2 − ηy t,(cid:21)(cid:27) , (cid:98) y t

Original: +1 =
Revised: +1 =

Original: Π Y (cid:26) (cid:98)
Revised: Π Y (cid:26) (cid:98)

Original: y t + η (cid:20) x 
Revised: y t [Equation] η (cid:20) x t,

Revised: [Equation]

Original: t,− x t,(cid:21)(cid:27) =
Revised: x t,(cid:21)(cid:27) [Equation]

Original: Π Y (cid:26)(cid:20)(cid:98) y t, 1 + ηx t,(cid:98)
Revised: (cid:98) [Equation]

Original:  y t, 2 − ηx t,(cid:21)(cid:27) , y t
Revised: (cid:21)(cid:27) [Equation] y [Equation]

Original: +1 = Π Y (cid:26) (cid:98)
Revised: = [Equation] Y (cid:26) (cid:98)

Revised: [Equation]

Original: y t +1 + η (cid:20) x t,− x t,(cid:21)(cid:27) =
Revised: (cid:20) [Equation] x t,(cid:21)(cid:27) [Equation]

Original: Π Y (cid:26)(cid:20)(cid:98) y t +1 , 1 + ηx t,(cid:98) 
Revised: (cid:26)(cid:20)(cid:98) [Equation] t,(cid:98) [Equation]

Original: y t +1 , 2 − ηx t,(cid:21)(cid:27) .
Revised: t +1 , 2 − ηx t,(cid:21)(cid:27) .

Original: According to induction hypothesis, we have (cid:98) x t +1 = (cid:98) y t +1 , which further leads to x t +1 =
Revised: According to induction hypothesis, we have (cid:98) x t +1 = (cid:98) y t +1 , which further leads to x t +1 =

Original: y t +1 .
Revised: y t +1 .

Original: Now we prove that for any (cid:20) xx(cid:21) such that x 1 ≥ 0 , x 2 ≤ 14 and x 2 < x 12 , (cid:20) xx(cid:21) =
Revised: Now we prove that for any (cid:20) xx(cid:21) such that x 1 ≥ 0 , x 2 ≤ 14 and x 2 < x 12 , (cid:20) xx(cid:21) =

Original: Π X (cid:26)(cid:20) xx(cid:21)(cid:27) satisﬁes
Revised: Π X (cid:26)(cid:20) xx(cid:21)(cid:27) satisﬁes

Original: that x 21 = x 2 .
Revised: that x 21 = x 2 .

Original: Otherwise, suppose that x 21 < x 2 .
Revised: Otherwise, suppose that x 21 < x 2 .

Original: Then according to the intermediate value theorem, there exists (cid:20)(cid:101) x(cid:101) x(cid:21) that lies in the line segment of (cid:20) xx(cid:21) and (cid:20) xx(cid:21) such that (cid:101) x 21 = (cid:101) x 2 .
Revised: Then according to the intermediate value theorem, there exists (cid:20)(cid:101) x(cid:101) x(cid:21) that lies in the line segment of (cid:20) xx(cid:21) and (cid:20) xx(cid:21) such that (cid:101) x 21 = (cid:101) x 2 .

Original: Moreover, as x 1 ≥ 0 , (cid:101) x 1 ≥ 0 , x 2 ≤ 14 , (cid:101) x 2 ≤ 14 , we know that (cid:20)(cid:101) x 1 (cid:101) x(cid:21) ∈ X .
Revised: Moreover, as x 1 ≥ 0 , (cid:101) x 1 ≥ 0 , x 2 ≤ 14 , (cid:101) x 2 ≤ 14 , we know that (cid:20)(cid:101) x 1 (cid:101) x(cid:21) ∈ X .

Original: Therefore, we have (cid:107) (cid:101) x − x (cid:107) < (cid:107) x − x (cid:107) , which leads to contradiction.
Revised: Therefore, we have (cid:107) (cid:101) x − x (cid:107) < (cid:107) x − x (cid:107) , which leads to contradiction.

Original: Now consider (cid:98) x t +1 .
Revised: Now consider (cid:98) x t +1 .

Original: According to induction hypothesis, we have ( (cid:98) x t, 1 + ηy t, 2 ) 2 ≥ (cid:98) x 2 t, 1 = (cid:98) x t, 2 ≥ (cid:98) x t, 2 − ηy t, 1 .
Revised: According to induction hypothesis, we have ( (cid:98) x t, 1 + ηy t, 2 ) 2 ≥ (cid:98) x 2 t, 1 = (cid:98) x t, 2 ≥ (cid:98) x t, 2 − ηy t, 1 .

Original: If equalities hold, trivially we have (cid:98) x 2 t
Revised: If equalities hold, trivially we have (cid:98) x 2 t

Original: +1 , 1 = (cid:98) x
Revised: +1 , 1 = (cid:98) x

Original: 2 t, 1 = (cid:98) x t, 2 = (cid:98) x t +1 , 2 according to Eq.
Revised: 2 t, 1 = (cid:98) x t, 2 = (cid:98) x t +1 , 2 according to Eq.

Original: Otherwise, as (cid:98) x t, 1 + ηy t, 2 ≥ 0 , (cid:98) x t, 2 − ηy t, 1 ≤ 14 , according to the analysis above, we also have (cid:98) x 2 t +1 , 1 = (cid:98) x
Revised: Otherwise, as (cid:98) x t, 1 + ηy t, 2 ≥ 0 , (cid:98) x t, 2 − ηy t, 1 ≤ 14 , according to the analysis above, we also have (cid:98) x 2 t +1 , 1 = (cid:98) x

Original: t +1 , 2 .
Revised: t +1 , 2 .

Original: Applying similar analysis to (cid:98) y t +1 , x t +1 and
Revised: Applying similar analysis to (cid:98) y t +1 , x t +1 and

Original: y t
Revised: y t

Original: +1 ﬁnishes the induction proof.
Revised: +1 ﬁnishes the induction proof.

Original: Claim 3.
Revised: Claim 3.

Original: With η ≤ 164 , the following holds for all t ≥ 1 ,
Revised: With η ≤ 164 , the following holds for all t ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: We prove the claim by induction on t .
Revised: We prove the claim by induction on t .

Original: The case t = 1 trivially holds.
Revised: The case t = 1 trivially holds.

Original: Suppose that Eq.
Revised: Suppose that Eq.

Original: (26) and Eq.
Revised: (26) and Eq.

Original: (27) hold at step t .
Revised: (27) hold at step t .

Original: Now consider step t + 1 .
Revised: Now consider step t + 1 .

Original: Induction to get Eq.
Revised: Induction to get Eq.

Original: According to Claim 2, we have
Revised: According to Claim 2, we have

Original: [Equation]
Revised: [Equation]

Original: and (cid:98) x t +1 =
Revised: and (cid:98) x t +1 =

Original: ( u, u 2 ) for some u ∈ [0 , 1 / 2] .
Revised: ( u, u 2 ) for some u ∈ [0 , 1 / 2] .

Original: Using the deﬁnition of the projection function, we have
Revised: Using the deﬁnition of the projection function, we have

Original: [Equation]
Revised: [Equation]

Original: Now we show that argmin u ∈ [0 , 12 ] g ( u ) = argmin u ∈ R g ( u ) .
Revised: Now we show that argmin u ∈ [0 , 12 ] g ( u ) = argmin u ∈ R g ( u ) .

Original: Note that
Revised: Note that

Original: [Equation]
Revised: [Equation]

Original: Therefore, when u > 12 , using x t, 1 ≤ 12 , we have
Revised: Therefore, when u > 12 , using x t, 1 ≤ 12 , we have

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which means g ( u ) > g ( 12 ) .
Revised: which means g ( u ) > g ( 12 ) .

Original: On the other hand, when u < 0 , using (cid:98) x t, 1 ≤ 12 , we have
Revised: On the other hand, when u < 0 , using (cid:98) x t, 1 ≤ 12 , we have

Original: [Equation]
Revised: [Equation]

Original: (30) which means g ( u ) > g (0) .
Revised: (30) which means g ( u ) > g (0) .

Original: Combining Eq.
Revised: Combining Eq.

Original: (29) and Eq.
Revised: (29) and Eq.

Original: (30), we know that argmin u ∈ [0 , 12 ] g ( u ) = argmin u ∈ R g
Revised: (30), we know that argmin u ∈ [0 , 12 ] g ( u ) = argmin u ∈ R g

Original: ( u ) .
Revised: ( u ) .

Original: Therefore, (cid:98) x t +1 , 1 is the unconstrained minimizer of convex function g ( u ) , which means ∇ g ( (cid:98) x
Revised: Therefore, (cid:98) x t +1 , 1 is the unconstrained minimizer of convex function g ( u ) , which means ∇ g ( (cid:98) x

Original: t +1 , 1 ) = 0 .
Revised: t +1 , 1 ) = 0 .

Original: Below we use contradiction to prove that (cid:98) x t +1 , 1 ≥ (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 .
Revised: Below we use contradiction to prove that (cid:98) x t +1 , 1 ≥ (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 .

Original: If (cid:98) x t +1 , 1 < (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 , we use Eq.
Revised: If (cid:98) x t +1 , 1 < (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 , we use Eq.

Original: (28) and get 
Revised: (28) and get ∇ g ( (cid:98) [Equation]

Original: [Equation]
Revised: [Equation]

Original: which leads to contradiction.
Revised: which leads to contradiction.

Original: Similarly, if (cid:98) x t +1 , 1 > (cid:98) x t [Equation] 1 + 4 η (cid:98) x 2 t, 1 , we have
Revised: Similarly, if (cid:98) x t +1 , 1 > (cid:98) x t, 1 + 4 η (cid:98) x 2 t, 1 , we have

Original: [Equation]
Revised: [Equation]

Original: The calculations above conclude that
Revised: The calculations above conclude that

Original: [Equation]
Revised: [Equation]

Original: Induction to get Eq.
Revised: Induction to get Eq.

Original: Similarly, we have
Revised: Similarly, we have

Original: [Equation]
Revised: [Equation]

Original:  and ∇ h ( x t +1 , 1 ) = 0 .
Revised: , and ∇ h ( x t +1 , 1 ) = 0 .

Original: If x t +1 , 1 < 12 (cid:98) x t +1 , 1 , we have ∇ h ( x t +1 , 1 ) = 2( xt +1 , 1 − (cid:98) x t +1 , 1 − ηx
Revised: If x t +1 , 1 < 12 (cid:98) x t +1 , 1 , we have ∇ h ( x t +1 , 1 ) = 2( x

Original: [Equation]
Revised: [Equation]

Original: t, 1 )

Original: [Equation]
Revised: [Equation]

Original: 4 x t +1 , 1 (cid:0) x 2 t +
Revised: −

Original: [Equation]
Revised: [Equation]

Original: 1 + ηx t
Revised: ≤ 0 .

Original: [Equation]
Revised: [Equation]

Original:  1 − (cid:98) x2 t +1 , 1 (cid:1) < − (cid:98) x t + [Equation] 1 −
Revised: If x t +1 , 1 > 2 (cid:98) x t +1 , 1 , we also have ∇ h ( x t +1 , 1 ) = ( x

Original: [Equation]
Revised: [Equation]

Original: ηx 2 t,
Revised: +

Original: [Equation]
Revised: [Equation]

Original:  xt +
Revised: (cid:98) x

Original: [Equation]
Revised: [Equation]

Original: 1 (cid:98) x 2 t+
Revised: t

Original: [Equation]
Revised: [Equation]

Revised: (cid:1)

Original: [Equation]
Revised: [Equation]

Original: (cid:98)x t + [Equation] 
Revised: (cid:98) [Equation] (cid:98) x

Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: ( x t +1 , 1 < 12 (cid:98) x t +1 , 1 ) ≤ 0 .
Revised: [Equation] [Equation] x t [Equation]

Original: ( η ≤ 164 , x t, 1 ≤ 12 )
Revised: [Equation]

Original: If x t +1 , 1 > 2 (cid:98) x t +1 , 1 , we also have ∇ h ( x t +1 , 1 ) =
Revised: [Equation]

Original: ( x t +1 , 1 − (cid:98) x
Revised: [Equation] (cid:98) [Equation]

Original: t +1 , 1 − ηx 2 t, 1 ) + 4 x t +1 , 1 (cid:0) x 2 t
Revised: [Equation] η ( [Equation]

Original: +1 , 1 + ηx t, 1 − (cid:98) x 2 t +1 , 1 (cid:1) > 2 (cid:98) x
Revised: [Equation] η [Equation] x

Original: t +1 , 1 − 2 ηx 2 t, 1 + 24 (cid:98) x 3 t +1 , 1 + 8 η (cid:98) x 
Revised: [Equation] x [Equation] (Eq

Original: t +1 , 1 x t,( x t +1 , 1 > 2 (cid:98) x t +1 , 1 ) ≥ 2 (cid:98) x t +1 , 1 − 2 ηx 2 t, 1 + 24 (cid:98) x 3 t +1 , 1 + 8 η ( (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 ) x t,(Eq.
Revised: [Equation]

Original: (31)) ≥ 2 (cid:98) x
Revised: (31)) ≥ 2 (cid:98) x

Revised: [Equation]

Revised: [Equation]

Original: t +1 , 1 − 2 ηx 2 t, 1 + 24 (cid:98)
Revised: 1 − 2 ηx 2 t [Equation] [Equation] 24 (cid:98)

Original: [Equation]
Revised: [Equation]

Original: Both lead to contradiction.
Revised: Both lead to contradiction.

Original: Therefore, we conclude that x t +1 ∈ [ 12 (cid:98) x
Revised: Therefore, we conclude that x t +1 ∈ [ 12 (cid:98) x

Original: t +1 , 1 , 2 (cid:98) x
Revised: t +1 , 1 , 2 (cid:98) x

Original: t +1 , 1 ] , which ﬁnishes the induction proof.
Revised: t +1 , 1 ] , which ﬁnishes the induction proof.

Original: Claim 4.
Revised: Claim 4.

Original: [Equation]
Revised: [Equation]

Original: The case t = 1 holds trivially.
Revised: The case t = 1 holds trivially.

Original: For t ≥ 2 , we prove this by contradiction.
Revised: For t ≥ 2 , we prove this by contradiction.

Original: Using the deﬁnition of the projection function, we have:
Revised: Using the deﬁnition of the projection function, we have:

Original: [Equation]
Revised: [Equation]

Original: Similar to the analysis in Claim 3 , we have argmin u ∈ [ 0 , 12 ] h ( u ) =
Revised: Similar to the analysis in Claim 3 , we have argmin u ∈ [ 0 , 12 ] h ( u ) =

Original: argmin u ∈ R h ( u ) , which means that ∇ h ( x t +1 , 1 ) = 0 .
Revised: argmin u ∈ R h ( u ) , which means that ∇ h ( x t +1 , 1 ) = 0 .

Original: Note that η ≤ 164 and 0 ≤ (cid:98) x t, 1 ≤ 12 , according to Eq.
Revised: Note that η ≤ 164 and 0 ≤ (cid:98) x t, 1 ≤ 12 , according to Eq.

Original: (26) and Eq.
Revised: (26) and Eq.

Original: (27), we have
Revised: (27), we have

Original: [Equation]
Revised: [Equation]

Original: which means that
Revised: which means that

Original: [Equation]
Revised: [Equation]

Original: If x t +1 , 1 < (cid:98) x
Revised: If x t +1 , 1 < (cid:98) x

Original: t +1 , 1 − 4 η (cid:98) x 2 t +1 , 1 , we show that ∇ h ( x t +1 , 1 ) < 0 .
Revised: t +1 , 1 − 4 η (cid:98) x 2 t +1 , 1 , we show that ∇ h ( x t +1 , 1 ) < 0 .

Original: In fact,
Revised: In fact,

Original: [Equation]
Revised: [Equation]

Original: [Table] 
Revised: [Table] [Equation] [Table] [Equation]

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table] 
Revised: [Table] [Equation] [Table] [Equation]

Revised: [Table]

Original: [Equation] 
Revised: [Equation] [Table] [Equation] [Table]

Original:  [Table]
Revised: [Equation] [Table] [Equation] [Table]

Original: [Equation] 
Revised: [Equation] [Table] [Equation] [Table]

Original:  [Table]
Revised: [Equation] [Table] [Equation] [Table]

Original: [Equation] 
Revised: [Equation] [Table] [Equation] [Table]

Original:  [Table]
Revised: [Equation] [Table] [Equation] [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Revised: t

Original: [Equation]
Revised: [Equation]

Original: ≤
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: ( (cid:98) x
Revised: ( (cid:98) x

Original: [Equation]
Revised: [Equation]

Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: ≤ 0 , which leads to contradiction.
Revised: which leads to contradiction.

Original: Therefore, we show that x t, 1 ≥ (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 for all t ≥ 1 .
Revised: Therefore, we show that x t, 1 ≥ (cid:98) x t, 1 − 4 η (cid:98) x 2 t, 1 for all t ≥ 1 .

Original: Claim 5.
Revised: Claim 5.

Original: If η ≤ 164 , we have (cid:107) z t − z ∗ (cid:107) ≥ Ω(1 /t ) .
Revised: If η ≤ 164 , we have (cid:107) z t − z ∗ (cid:107) ≥ Ω(1 /t ) .

Original: Now we are ready to prove (cid:107) z t − z ∗ (cid:107) ≥ Ω(1 /t ) .
Revised: Now we are ready to prove (cid:107) z t − z ∗ (cid:107) ≥ Ω(1 /t ) .

Original: First we show (cid:98) x t, 1 ≥ 12 t for all t ≥ 1 by induction.
Revised: First we show (cid:98) x t, 1 ≥ 12 t for all t ≥ 1 by induction.

Original: The case t = 1 trivially holds.
Revised: The case t = 1 trivially holds.

Original: Suppose that it holds at step t .
Revised: Suppose that it holds at step t .

Original: Considering step t + 1 , we have
Revised: Considering step t + 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Claim3 )
Revised: Claim

Original: [Equation]
Revised: [Equation]

Original: 12 t ≤ (cid:98)x t
Revised: [Equation] [Equation] (cid:98)

Original: [Equation]
Revised: [Equation]

Original: 1 ≤ 12 , and x − 116 x 2 is increasing when x ≤ 8 )
Revised: t [Equation] [Equation] 12 , and x − 116 x 2 is increasing when x ≤ 8 )

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Therefore, (cid:98) x t, 1 ≥ 12 t , ∀ t ≥ 1 .
Revised: Therefore, (cid:98) x t, 1 ≥ 12 t , ∀ t ≥ 1 .

Original: This, by Claim 4 and the analysis above, shows that
Revised: This, by Claim 4 and the analysis above, shows that

Original: [Equation]
Revised: [Equation]

Original: Note that according to Claim 1 , x ∗ = 0 .
Revised: Note that according to Claim 1 , x ∗ = 0 .

Original: Therefore, we have (cid:107) z t − z ∗ (cid:107) ≥ x t, 1 ≥ 1 2( t +1) , which ﬁnishes the proof.
Revised: Therefore, we have (cid:107) z t − z ∗ (cid:107) ≥ x t, 1 ≥ 1 2( t +1) , which ﬁnishes the proof.

Original: Claim 6.
Revised: Claim 6.

Original: In this example, SP-MS holds with β = 3 .
Revised: In this example, SP-MS holds with β = 3 .

Original: This can be seen by the following: max z (cid:48) ∈Z
Revised: This can be seen by the following: max z (cid:48) ∈Z

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: z )
Revised: z )

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (cid:48) )
Revised: (cid:48) )

Original: (cid:107)
Revised: (cid:107)

Original: [Equation]

Original: (cid:48) (cid:107)

Original: [Equation]
Revised: [Equation]

Original: max
Revised: max

Original: [Equation]
Revised: [Equation]

Original: (cid:48) ∈Z F ( z ) 
Revised: z ) [Equation] ) = max

Original: [Equation]
Revised: [Equation]

Original: ( z
Revised: ∈X

Original: [Equation]
Revised: [Equation]

Original: (cid:48) ) = max x (cid:48) ∈X , y(cid:48) ∈Y (cid:8)
Revised: y

Original: [Equation]
Revised: [Equation]

Original: (cid:62) Gy (cid:48) − x (cid:48)(cid:62)
Revised: ∈Y (cid:8)

Original: [Equation]
Revised: [Equation]

Original: (cid:9) = max x (cid:48) ∈X , y (cid:48) ∈Y {− x 1 y (cid:48)
Revised: = max [Equation] (cid:48) ∈X , y (cid:48) ∈Y {− x 1 y (cid:48)

Original: 2 + x 2 y (cid:48) 

Revised: 2 + x 2 y (cid:48)

Original: + x (cid:48) 1 y 2 − x (cid:48) 2 y 1 } ≥ − x 1 x 22 + x 

Revised: + x (cid:48) 1 y 2 − x (cid:48) 2 y 1 } ≥ − x 1 x 22 + x

Original: + y 22 − y 

Revised: + y 22 − y

Original: y(picking y (cid:48) 

Revised: y(picking y (cid:48)

Original: = x 2 , y (cid:48) 2 = x 22 , x (cid:48) 1 = y 2 , x (cid:48) 2 = y 22 )
Revised: = x 2 , y (cid:48) 2 = x 22 , x (cid:48) 1 = y 2 , x (cid:48) 2 = y 22 )

Original: ≥2 x 22 + 1 2 y( x 1 , y 1 ≤ 12 ) ≥ (cid:0) x 41 + x 42 + y 41 + y 42 (cid:1) ( x 2 ≥ { x 21 , x 22 } , y 2 ≥ { y 21 , y 22 } ) ≥ (cid:0) x 21 + x 22 + y 21 + y 22 (cid:1)(Cauchy-Schwarz)
Revised: ≥2 x 22 + 1 2 y( x 1 , y 1 ≤ 12 ) ≥ (cid:0) x 41 + x 42 + y 41 + y 42 (cid:1) ( x 2 ≥ { x 21 , x 22 } , y 2 ≥ { y 21 , y 22 } ) ≥ (cid:0) x 21 + x 22 + y 21 + y 22 (cid:1)(Cauchy-Schwarz)

Original: =16 (cid:107) z − z ∗ (cid:107) 4 , ( z ∗ = (0 , 0 , 0 , 0) ) which implies β = 3 .
Revised: =16 (cid:107) z − z ∗ (cid:107) 4 , ( z ∗ = (0 , 0 , 0 , 0) ) which implies β = 3 .