Original: ST AY -ON- THE -R IDGE : G UARANTEED C ONVERGENCE TO L OCAL M INIMAX E QUILIBRIUM IN N ONCONVEXN ONCONCAVE G AMES
Revised: ST AY -ON- THE -R IDGE : G UARANTEED C ONVERGENCE TO L OCAL M INIMAX E QUILIBRIUM IN N ONCONVEXN ONCONCAVE G AMES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Min-max optimization problems involving nonconvex-nonconcave objectives have found important applications in adversarial training and other multi-agent learning settings.
Revised: Min-max optimization problems involving nonconvex-nonconcave objectives have found important applications in adversarial training and other multi-agent learning settings.

Original: Yet, no known gradient descent-based method is guaranteed to converge to (even local notions of) min-max equilibrium in the nonconvex-nonconcave setting.
Revised: Yet, no known gradient descent-based method is guaranteed to converge to (even local notions of) min-max equilibrium in the nonconvex-nonconcave setting.

Original: For all known methods, there exist relatively simple objectives for which they cycle or exhibit other undesirable behavior different from converging to a point, let alone to some game-theoretically meaningful one (Vlatakis-Gkaragkounis et al., 2019; Hsieh et al., 2021).
Revised: For all known methods, there exist relatively simple objectives for which they cycle or exhibit other undesirable behavior different from converging to a point, let alone to some game-theoretically meaningful one Vlatakis-Gkaragkounis et al. Hsieh et al.

Original: The only known convergence guarantees hold under the strong assumption that the initialization is very close to a local min-max equilibrium (Wang et al., 2019).
Revised: The only known convergence guarantees hold under the strong assumption that the initialization is very close to a local min-max equilibrium Wang et al.

Original: Moreover, the afore-described challenges are not just theoretical curiosities.
Revised: Moreover, the afore-described challenges are not just theoretical curiosities.

Original: All known methods are unstable in practice, even in simple settings.
Revised: All known methods are unstable in practice, even in simple settings.

Original: We propose the ﬁrst method that is guaranteed to converge to a local min-max equilibrium for smooth nonconvex-nonconcave objectives.
Revised: We propose the ﬁrst method that is guaranteed to converge to a local min-max equilibrium for smooth nonconvex-nonconcave objectives.

Original: Our method is secondorder and provably escapes limit cycles as long as it is initialized at an easy-to-ﬁnd initial point.
Revised: Our method is secondorder and provably escapes limit cycles as long as it is initialized at an easy-to-ﬁnd initial point.

Original: Both the deﬁnition of our method and its convergence analysis are motivated by the topological nature of the problem.
Revised: Both the deﬁnition of our method and its convergence analysis are motivated by the topological nature of the problem.

Original: In particular, our method is not designed to decrease some potential function, such as the distance of its iterate from the set of local min-max equilibria or the projected gradient of the objective, but is designed to satisfy a topological property that guarantees the avoidance of cycles and implies its convergence.
Revised: In particular, our method is not designed to decrease some potential function, such as the distance of its iterate from the set of local min-max equilibria or the projected gradient of the objective, but is designed to satisfy a topological property that guarantees the avoidance of cycles and implies its convergence.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Min-max optimization lies at the foundations of Game Theory (von Neumann, 1928), Convex
Revised: Min-max optimization lies at the foundations of Game Theory von Neumann (1928), Convex

Original: Optimization (Dantzig, 1951; Adler, 2013) and Online Learning (Blackwell, 1956; Hannan, 1957;
Revised: Optimization Dantzig (1951a); Adler (2013) and Online Learning Blackwell (1956); Hannan (1957);

Original: Cesa-Bianchi & Lugosi, 2006), and has found many applications in theoretical and applied ﬁelds including, more recently, in adversarial training and other multi-agent learning problems (Goodfellow et al., 2014;
Revised: Cesa-Bianchi & Lugosi (2006), and has found many applications in theoretical and applied ﬁelds including, more recently, in adversarial training and other multi-agent learning problems Goodfellow et al.

Original:  Madry et al., 2018; Zhang et al., 2019).
Revised: (2014); Madry et al. ; Zhang et al.

Original: In its general form, it can be written as
Revised: In its general form, it can be written as

Original: [Equation]
Revised: [Equation]

Original: where Θ and Ω are convex subsets of the Euclidean space, and f is continuous.
Revised: where Θ and Ω are convex subsets of the Euclidean space, and f is continuous.

Original: Eq. (1) can be viewed as a model of a sequential-move game wherein a player who is interested in minimizing f chooses θ ﬁrst, and then a player who is interested in maximizing f chooses ω after seeing θ .
Revised: Equation (1) can be viewed as a model of a sequential-move game wherein a player who is interested in minimizing f chooses θ ﬁrst, and then a player who is interested in maximizing f chooses ω after seeing θ .

Original: A solution to (1) corresponds to a Nash equilibrium of this sequential-move game.
Revised: Solving (1) corresponds to an equilibrium of this sequential-move game.

Original: We may also study the simultaneous-move game with the same objective f wherein the minimizing player and the maximizing player choose θ and ω simultaneously.
Revised: We may also study the simultaneous-move game with the same objective f wherein the minimizing player and the maximizing player choose θ and ω simultaneously.

Original: The Nash equilibrium of the simultaneous-move game, also called a min-max equilibrium , is a pair ( θ (cid:63) , ω (cid:63) )
Revised: The Nash equilibrium of the simultaneous-move game, also called a min-max equilibrium , is a pair ( θ (cid:63) , ω (cid:63) )

Original: ∈
Revised: ∈

Original: Θ × Ω such that f ( θ (cid:63) , ω (cid:63) ) ≤
Revised: Θ × Ω such that f ( θ (cid:63) , ω (cid:63) ) ≤

Original: f ( θ, ω (cid:63) ) , for all θ ∈ Θ and f ( θ (cid:63) , ω (cid:63) ) ≥ f
Revised: f ( θ, ω (cid:63) ) , for all θ ∈ Θ and f ( θ (cid:63) , ω (cid:63) ) ≥ f

Original: ( θ (cid:63) , ω ) , for all ω ∈ Ω .
Revised: ( θ (cid:63) , ω ) , for all ω ∈ Ω .

Original: [Equation]
Revised: [Equation]

Original: It is easy to see that a Nash equilibrium of the simultaneous-move game also constitutes a Nash equilibrium of the sequential-move game, but the converse need not be true.
Revised: It is easy to see that a Nash equilibrium of the simultaneous-move game also constitutes a Nash equilibrium of the sequential-move game, but the converse need not be true Jin et al.

Original: Here, we focus on solving the (harder) simultaneous-move game.
Revised: Here, we focus on solving the (harder) simultaneous-move game.

Original: In particular, we study the existence of dynamics which converge to solutions of the simultaneous-move game, namely the existence of methods that make incremental updates to a pair ( θ t , ω t ) so as the sequence ( θ t , ω t ) converges, as t → ∞ , to some ( θ ∗ , ω ∗ ) satisfying (2) or some relaxation of it.
Revised: In particular, we study the existence of dynamics which converge to solutions of the simultaneous-move game, namely the existence of methods that make incremental updates to a pair ( θ t , ω t ) so as the sequence ( θ t , ω t ) converges, as t → ∞ , to some ( θ ∗ , ω ∗ ) satisfying equation 2 or some relaxation of it.

Original: This problem has been extensively studied in the special case where Θ and Ω are convex and compact and f is convex-concave — i.e. convex in θ for all ω and concave in ω for all θ .
Revised: This problem has been extensively studied in the special case where Θ and Ω are convex and compact and f is convex-concave — i.e. convex in θ for all ω and concave in ω for all θ .

Original: In this case, the set of Nash equilibria of the simultaneous-move game is equal to the set of Nash equilibria of the sequential-move game, and these sets are non-empty and convex (von Neumann, 1928).
Revised: In this case, the set of Nash equilibria of the simultaneous-move game is equal to the set of Nash equilibria of the sequential-move game, and these sets are non-empty and convex von Neumann (1928).

Original: Even in this simple setting, however, many natural dynamics surprisingly fail to converge: gradient descentascent , as well as various continuous-time versions of follow-the-regularized-leader , not only fail to converge to a min-max equilibrium, even for very simple objectives, but may even exhibit chaotic behavior (Mertikopoulos et al., 2018;
Revised: Even in this simple setting, however, many natural dynamics surprisingly fail to converge: gradient descentascent , as well as various continuous-time versions of follow-the-regularized-leader , not only fail to converge to a min-max equilibrium, even for very simple objectives, but may even exhibit chaotic behavior Mertikopoulos et al.

Original: Vlatakis-Gkaragkounis et al., 2019; Hsieh et al., 2021).
Revised: Vlatakis-Gkaragkounis et al. Hsieh et al.

Original: In order to circumvent these negative results, an extensive line of work has introduced other algorithms, such as extragradient (Korpelevich, 1976) and optimistic gradient descent (Popov, 1980), which exhibit last-iterate convergence to the set of min-max equilibria in this setting; see e.g. Daskalakis et al.
Revised: In order to circumvent these negative results, an extensive line of work has introduced other algorithms, such as extragradient Korpelevich (1976) and optimistic gradient descent Popov (1980), which exhibit last-iterate convergence to the set of min-max equilibria in this setting; see e.g. Daskalakis et al.

Original: ; Daskalakis & Panageas (2018); Mazumdar & Ratliff (2018); Raﬁque et al.
Revised: ; Daskalakis & Panageas (2018); Mazumdar & Ratliff (2018); Raﬁque et al.

Original: Hamedani & Aybat (2018); Adolphs et al.
Revised: Hamedani & Aybat (2018); Adolphs et al.

Original: ; Daskalakis & Panageas (2019);
Revised: ; Daskalakis & Panageas (2019);

Original: Liang & Stokes (2019); Gidel et al.
Revised: Liang & Stokes (2019); Gidel et al.

Original: ; Mokhtari et al.
Revised: ; Mokhtari et al.

Original: ; Abernethy et al.
Revised: ; Abernethy et al.

Original: ; Golowich et al.
Revised: ; Golowich et al.

Original: (2020b;a).
Revised: (2020b;a);

Original:  Alternatively, one may take advantage of the convexity of the problem, which implies that several no-regret learning procedures, such as online gradient descent, exhibit average -iterate convergence to the set of min-max equilibria (Cesa-Bianchi & Lugosi, 2006; Shalev-Shwartz, 2012; Bubeck & 
Revised: Gorbunov et al. (2022); Cai et al.. Alternatively, one may take advantage of the convexity of the problem, which implies that several no-regret learning procedures, such as online gradient descent, exhibit average -iterate convergence to the set of min-max equilibria Cesa-Bianchi & Lugosi (2006);

Original: Cesa-Bianchi, 2012; Shalev-Shwartz & Ben-David, 2014; Hazan, 2016).
Revised: Shalev-Shwartz (2012); Bubeck & Cesa-Bianchi (2012); Shalev-Shwartz & Ben-David (2014); Hazan (2016).

Original: Moreover, Lin et al.
Revised: Beyond the convex/concave setting Lin et al.

Original:  Kong & Monteiro (2021); Ostrovskii et al.
Revised: (2020); Kong & Monteiro (2021); Ostrovskii et al.

Original: (2021) show that convexity with respect to one of the two players is enough to design algorithms that exhibit average-iterate convergence to min-max equilibria.
Revised: (2021) show that convexity with respect to one of the two players is enough to design algorithms that exhibit average-iterate convergence to min-max equilibria while Diakonikolas et al. (2021) and

Revised: Pethick et al.

Revised: (2022) provide convergence results for weak Minty variational inequalities .

Original: Our focus in this paper is on the more general case where f is notassumed to be convex-concave, i.e. it may fail to be convex in θ for all ω , or may fail to be concave in ω for all θ , or both.
Revised: Our focus in this paper is on the more general case where f is not convex-concave, i.e. it may fail to be convex in θ for all ω , or may fail to be concave in ω for all θ , or both.

Original: We call this general setting where neither convexity with respect to θ nor concavity with respect to ω is assumed, the nonconvex-nonconcave setting.
Revised: We call this general setting where neither convexity with respect to θ nor concavity with respect to ω is assumed, the nonconvexnonconcave setting.

Original: This setting presents some substantial challenges.
Revised: This setting presents some substantial challenges.

Original: First, min-max equilibria are not guaranteed to exist, i.e. for general objectives there may be no ( θ (cid:63) , ω (cid:63) ) satisfying (2); this happens even in very simple cases, e.g. when Θ = Ω
Revised: First, min-max equilibria are not guaranteed to exist, i.e. for general objectives there may be no ( θ (cid:63) , ω (cid:63) ) satisfying equation 2; this happens even in very simple cases, e.g. when Θ = Ω

Original: = [0 , 1] and f ( θ, ω ) =
Revised: = [0 , 1] and f ( θ, ω ) =

Original: ( θ − ω ) 2 .
Revised: ( θ − ω ) 2 .

Original: Second, it is NP -hard to determine whether a min-max equilibrium exists (Daskalakis et al., 2021) and, as is easy to see, it is also NP -hard to compute Nash equilibria of the sequential-move game (which do exist under compactness of the constraint sets).
Revised: Second, it is NP -hard to determine whether a min-max equilibrium exists Daskalakis et al. (2021) and, as is easy to see, it is also NP -hard to compute Nash equilibria of the sequential-move game (which do exist under compactness of the constraint sets).

Original: For these reasons, the optimization literature has targeted the computation of local and/or approximate solutions in this setting (Daskalakis &
Revised: For these reasons, the optimization literature has targeted the computation of local and/or approximate solutions in this setting Daskalakis &

Original: Panageas, 2018; Mazumdar & Ratliff, 2018; Jin et al., 2019; Wang et al., 2019; Daskalakis et al., 2021; Mangoubi & Vishnoi, 2021).
Revised: Panageas (2018); Mazumdar & Ratliff (2018); Jin et al. Wang et al. [Equation] ; Daskalakis et al

Revised: [Equation]

Revised: (2021); Mangoubi & Vishnoi (2021).

Original: This is the approach we also take in this paper, targeting the computation of ( ε, δ ) -local min-max equilibria , which were proposed in Daskalakis et al.
Revised: This is the approach we also take in this paper, targeting the computation of ( (cid:15), δ ) -local min-max equilibria , which were proposed in Daskalakis et al.

Original: These are approximate and local Nash equilibria of the simultaneous-move game, deﬁned as feasible points ( θ (cid:63) , ω (cid:63) ) which satisfy a relaxed and local version of (2), namely:
Revised: These are approximate and local Nash equilibria of the simultaneous-move game, deﬁned as feasible points ( θ (cid:63) , ω (cid:63) ) which satisfy a relaxed and local version of equation 2, namely:

Original: [Equation]
Revised: [Equation]

Original: for all ω ∈ Ω such that (cid:107) [Equation] Besides being a natural concept of local, approximate min-max equilibrium, an attractive feature of ( ε, δ ) -local min-max equilibria is that they are guaranteed to exist when f is Λ -smooth and the locality parameter, δ , is chosen small enough in terms of the smoothness, Λ , and the approximation parameter, ε , namely whenever δ ≤ (cid:113) 2 ε Λ .
Revised: Besides being a natural concept of local, approximate min-max equilibrium, an attractive feature of ( (cid:15), δ ) -local min-max equilibria is that they are guaranteed to exist when f is Λ -smooth and the locality parameter, δ , is chosen small enough in terms of the smoothness, Λ , and the approximation parameter, (cid:15) , namely whenever δ ≤ (cid:113) 2 (cid:15) Λ .

Original: Indeed, in this regime of parameters the ( ε, δ ) -local min-max equilibria are in correspondence with the approximate ﬁxed points of the Projected Gradient
Revised: Indeed, in this regime of parameters the ( (cid:15), δ ) -local min-max equilibria are in correspondence with the approximate ﬁxed points of the Projected Gradient

Original: Descent/Ascent dynamics.
Revised: Descent/Ascent dynamics.

Original: Thus, the existence of the former can be established by invoking Brouwer’s ﬁxed point theorem to establish the existence of the latter.
Revised: Thus, the existence of the former can be established by invoking Brouwer’s ﬁxed point theorem to establish the existence of the latter.

Original: (Theorem 5.1 of Daskalakis et al.
Revised: (Theorem 5.1 of Daskalakis et al.

Original: There are a number of existing approaches which would be natural to use to ﬁnd a solution ( θ (cid:63) , ω (cid:63) ) satisfying (3) and (4), but all run into signiﬁcant obstacles.
Revised: There are a number of existing approaches which would be natural to use to ﬁnd a solution ( θ (cid:63) , ω (cid:63) ) satisfying equation 3 and equation 4, but all run into signiﬁcant obstacles.

Original: First, the idea of averaging, which can be leveraged in the convex-concave setting to obtain provable guarantees for otherwise chaotic algorithms, such as online gradient descent, no longer works, as it critically uses Jensen’s inequality which needs convexity/concavity.
Revised: First, the idea of averaging, which can be leveraged in the convex-concave setting to obtain provable guarantees for otherwise chaotic algorithms, such as online gradient descent, no longer works, as it critically uses Jensen’s inequality which needs convexity/concavity.

Original: On the other hand, negative results abound for last-iterate convergence: Hsieh et al.
Revised: On the other hand, negative results abound for lastiterate convergence: Hsieh et al.

Original: (2021) show that a variety of zeroth, ﬁrst, and second order methods may converge to a limit cycle, even in simple settings.
Revised: (2021) show that a variety of zeroth, ﬁrst, and second order methods may converge to a limit cycle, even in simple settings.

Original: Vlatakis-Gkaragkounis et al.
Revised: Vlatakis-Gkaragkounis et al.

Original: (2019) study a particular class of nonconvex-nonconcave games and show that continuous-time gradient descent-ascent (GDA) exhibits recurrent behavior.
Revised: (2019) study a particular class of nonconvex-nonconcave games and show that continuous-time gradient descentascent (GDA) exhibits recurrent behavior.

Original: Furthermore, common variants of gradient descent-ascent, such as optmistic GDA (OGDA) or extra-gradient (EG), may be unstable even in the proximity of local min-max equilibria, or converge to ﬁxed points that are not local min-max equilibria (Daskalakis & Panageas, 2018; Jin et al., 2019).
Revised: Furthermore, common variants of gradient descent-ascent, such as optmistic GDA (OGDA) or extra-gradient (EG), may be unstable even in the proximity of local min-max equilibria, or converge to ﬁxed points that are not local min-max equilibria Daskalakis & Panageas (2018); Jin et al.

Original: While there do exist algorithms, such as F OLLOW -T HE -R IDGE proposed by Wang et al.
Revised: While there do exist algorithms, such as F OLLOW -T HE -R IDGE proposed by Wang et al.

Original: (2019), which provably exhibit local convergence to a (relaxation of) local min-max equilibrium, these algorithms do not enjoy global convergence guarantees, and no algorithm is known with guaranteed convergence to a local min-max equilibrium.
Revised: (2019), which provably exhibit local convergence to a (relaxation of) local min-max equilibrium, these algorithms do not enjoy global convergence guarantees, and no algorithm is known with guaranteed convergence to a local min-max equilibrium.

Original: These negative theoretical results are consistent with the practical experience with min-maximization of nonconvex-nonconcave objectives, which is rife with frustration as well.
Revised: These negative theoretical results are consistent with the practical experience with min-maximization of nonconvex-nonconcave objectives, which is rife with frustration as well.

Original: A common experience is that the training dynamics of ﬁrst-order methods are unstable, oscillatory or divergent, and the quality of the points encountered in the course of training can be poor; see e.g. Goodfellow (2016); Metz et al.
Revised: A common experience is that the training dynamics of ﬁrst-order methods are unstable, oscillatory or divergent, and the quality of the points encountered in the course of training can be poor; see e.g. Goodfellow (2016); Metz et al.

Original: ; Daskalakis et al.
Revised: ; Daskalakis et al.

Original: ; Mescheder et al.
Revised: ; Mescheder et al.

Original: (2018); Daskalakis & Panageas (2018); 
Revised: (2018); Daskalakis & Panageas (2018); Mazumdar &

Original: Mazumdar & Ratliff (2018); Mertikopoulos et al.
Revised: Ratliff (2018); Mertikopoulos et al.

Original: ; Adolphs et al.
Revised: ; Adolphs et al.

Original: In light of the failure of essentially all known algorithms to guarantee convergence, even asymptotically, to local min-max equilibria, we ask the following question: Is there an algorithm which is guaranteed to converge to a local min-max equilibrium in the nonconvex-nonconcave setting (Wang et al., 2019)?
Revised: In light of the failure of essentially all non-trivial, i.e., non brute-force, algorithms to guarantee convergence, even asymptotically, to local min-max equilibria, we ask the following question: Is there any local-search algorithm which is guaranteed to converge to a local min-max equilibrium in the nonconvex-nonconcave setting?

Original: [Equation]
Revised: (see

Original: the left most dots of the corresponding color. The blue and orange trajectories converge to a limit cycle while the green trajectory converges to an equilibrium.
Revised: Table 1)

Original: Our Contribution.
Revised: 1. O UR C ONTRIBUTION

Original: In this work we answer the above question in the afﬁrmative: we propose a second-order method that is guaranteed to converge to a local minmax equilibrium (Theorem 1). 
Revised: In this work we answer the above question in the afﬁrmative: we propose a second-order method that is guaranteed to converge to a local min-max equilibrium (Theorem 1). .

Original: Our algorithm, called ST AY -ON- THE -R IDGE or STON’R, has some similarity to F OLLOW -T HE 
Revised: Our algorithm, called ST AY -ON- THE -R IDGE or STON’R, has some similarity to F OLLOW -T HE -R IDGE or FTR, which only converges locally.

Original: -R IDGE or FTR, which only converges locally and to a relaxed notion of min-max equilibrium.
Revised: STON’R is the ﬁrst method guaranteed to local min-max equilibrium beyond the brute-force grid-search in the non-convex/non-concave setting.

Original: Both the structure of our algorithm and its global convergence analysis are motivated by the topological nature of the problem, as established by Daskalakis et al.
Revised: Both the structure of our algorithm and its global convergence analysis are motivated by the topological nature of the problem, as established by Daskalakis et al.

Original: (2021) who showed that the problem iscomputationally (and mathematically) equivalent to Brouwer ﬁxed point computation.
Revised: (2021) who showed that the problem is equivalent to Brouwer ﬁxed point computation.

Original: In particular, the structure and analysis of STON’R are not based on a potential function argument but on a parity argument , akin to thecombinatorial argument used to prove the existence of Brouwer ﬁxed points.
Revised: In particular, the structure and analysis of STON’R are not based on a potential function argument but on a parity argument (see Section 4), akin to the argument used to prove the existence of Brouwer ﬁxed points.

Original: In Figure 1, we give an example where Gradient Descent/Ascent and Optimistic GDA, whose trajectories are shown in blue and orange respectively, converge to  a limit cycle while STON’R, whose trajectory is shown in green , successfully converges to a local min-max equilibrium of  the objective.
Revised: The main challenge of our work is to prove that there exists an algorithm that uses only local information of the objective function f , i.e., only its second derivative, while satisfying the topological properties that are necessary to guarantee global convergence. In order to understand the main technical contributions of our paper we need ﬁrst to introduce the main steps of showing the convergence using a topological argument in Section 4. Then in Section 5.4 we provide a sketch of our proof and we highlight the technical difﬁculties that we face. [Table]

Original: An example min-max optimization problem for which FTR fails to converge globally is when the objective is f ( θ, ω )
Revised: N a s h E q . [Table] many

Original: = − θω − 120 ω 2 + 220 S (( θ 2 + ω 2 ) / 2) · ω 2 , where θ, ω ∈ [ − 1 , 1] and S is the smooth-step function S ( θ ) = 3 θ 2 − 2 θ 3 , for θ ∈ [0 , 1] , S ( θ ) = 0 for θ ≤ 0 and S ( θ ) = 1 for θ ≥ 1 .
Revised: [Table] applicable [Table] existence same as above

Original: In this case, STON’R converges to a local min-max equilibrium while FTR algorithm gets stuck at the point θ = − 1 , ω = − 1 .
Revised: [Table] complexity same as above [Table] L o ca l N a s h

Original: We refer to Appendix A for more simulations and detailed comparison of GDA, OGDA, EG, FTR, and STON’R in these two examples.
Revised: E q . convergentdynamics same as above many [Table] This paper

Original:  S OLUTION C ONCEPT
Revised:  S OLUTION C ONCEPT

Original: First, a standard notation that we use is this: if m is a natural number then [ m ] = { 1 , . . .
Revised: We begin with formulating our problem in the more general framework of variational inequalities .

Original: , m } . Our main goal in this paper is to design optimization methods that have guaranteed convergence to local min-max equilibria of smooth objectives in the nonconvex-nonconcave setting. For exposition simplicity, we formulate this problem in the language of non-monotone variational inequalities.
Revised: This simpliﬁes our deﬁnitions and notations and also makes our result applicable to more general settings such as multi-player concave games Rosen (1965).

Original: Variational Inequalities (VI).
Revised: Variational Inequalities (VI).

Original: For K ⊆ R n , consider a continuous map V : K → R n .
Revised: For K ⊆ R n , consider a continuous map V : K → R n .

Original: We say that x ∈ K is a solution of the variational inequality VI( V, K ) iff: V ( x ) (cid:62) · ( x − y ) ≤ 0 for all y ∈ K .
Revised: We say that x ∈ K is a solution of the variational inequality VI( V, K ) iff: V ( x ) (cid:62) · ( x − y ) ≥ 0 for all y ∈ K .

Original: It is well known that ﬁnding local min-max equilibria of smooth objectives can be expressed as a non-monotone VI problem.
Revised: It is well known that ﬁnding local min-max equilibria of smooth objectives can be expressed as a non-monotone VI problem.

Original: Speciﬁcally, consider the min-max optimization problem (1), take
Revised: Speciﬁcally, consider the min-max optimization problem (1), take

Original: K =
Revised: K =

Original: Θ × Ω and simplify notation by using x ∈ K to denote points ( θ, ω ) ∈ K .
Revised: Θ × Ω and simplify notation by using x ∈ K to denote points ( θ, ω ) ∈ K .

Original: Call the subset of coordinates of x identiﬁed with θ the “ minimizing coordinates” and the subset of coordinates of x identiﬁed with ω the “ maximizing coordinates.”
Revised: Call the subset of coordinates of x identiﬁed with θ the “ minimizing coordinates” and the subset of coordinates of x identiﬁed with ω the “ maximizing coordinates.”

Original: Then deﬁne V : K → R n as follows:
Revised: Then deﬁne V : K → R n as follows:

Original: For j ∈ [ n ] : set V j ( x ) :
Revised: For j ∈ [ n ] : set V j ( x ) :

Original: = − ∂f ( x ) ∂x j , if j is minimizing, and V j ( x ) :=
Revised: = − ∂f ( x ) ∂x j , if j is minimizing, and V j ( x ) :=

Original: ∂f ( x ) ∂x j , otherwise.
Revised: ∂f ( x ) ∂x j , otherwise.

Original: With these deﬁnitions, it is easy to see that computing ( ε, δ ) -local min-max equilibria of smooth objectives, i.e. points satisfying (3) and (4), can be reduced to ﬁnding solutions to VI( V, K ) .
Revised: Computing ( ε, δ ) -local min-max equilibria of smooth objectives, i.e. points satisfying (3) & (4), can be reduced to ﬁnding solutions to VI( V, K ) .

Original: (In fact, ﬁnding even an approximate VI solution x satisfying V ( x ) (cid:62) ( x − y ) ≥ − α, ∀ y ∈ K , would sufﬁce as long as α > 0 is small enough.
Revised: In fact, ﬁnding even an approximate VI solution x satisfying V ( x ) (cid:62) ( x − y ) ≥ − α, ∀ y ∈ K , would sufﬁce as long as α > 0 is small enough.

Original: For more details see Theorem 5.1 of Daskalakis et al.
Revised: For more details see Theorem 5.1 of Daskalakis et al.

Original: In view of the above, for the remainder of the paper we focus on solvingnon-monotone variational inequality problems.
Revised: Hence, for the rest of the paper we focus on solving variational inequality problems.

Original: For simplicity of expositionthroughout wewill take our constraint set to be
Revised: For simplicity of exposition we take our constraint set to be

Original: K = [0 , 1] n .
Revised: K = [0 , 1] n .

Original: In this case there is a simple characterization of the solutions to VI( V, K ) .
Revised: In this case there is a simple characterization of the solutions to VI( V, K ) .

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: We call a coordinate i  at point x ∈ [0 , 1] n, zero-satisﬁed if V i ( x ) = 0 , 2.
Revised: We call a coordinate i satisﬁed at point x ∈ [0 , 1] n if one of the following holds:

Original:  boundary-satisﬁed if ( V i ( x ) ≤ 0 and x i = 0) or ( V i ( x ) ≥ 0 and x i = 1) , 3. satisﬁed if i is zero- or boundary- satisﬁed and unsatisﬁed if it is not satisﬁed.
Revised: i is zero-satisﬁed at x , i.e, V i ( x ) = 0 , or 2. i is boundary-satisﬁed at x , i.e, ( V i ( x ) ≤ 0 and x i = 0) or ( V i ( x ) ≥ 0 and x i = 1) .

Original: Lemma 1 (Proof in Appendix D) .
Revised: Lemma 1 (Proof in Appendix D) .

Original: x is a solution of VI( V, [0 , 1] n ) iff j is satisﬁed at x , ∀ j ∈ [ n ] .
Revised: x is a solution of VI( V, [0 , 1] n ) iff j is satisﬁed at x , ∀ j ∈ [ n ] .

Original: Finally, in the rest of the paper we make the following assumptions for V :
Revised: Finally, in the rest of the paper we make the following assumptions for V :

Original: (cid:73) ( Λ -Lipschitz) (cid:107) V ( x ) − V( y ) (cid:107)
Revised: ( Λ -Lipschitz) (cid:107) V ( x ) − V

Original:  2 ≤ Λ · (cid:107) x − y (cid:107) 2 , for all x, y ∈ [0 , 1] n (cid:73) ( L -smooth) (cid:107) J
Revised: ( y ) (cid:107) 2 ≤ Λ · (cid:107) x − y (cid:107) 2 , for all x, y ∈ [0 , 1] n . ( L -smooth) (cid:107) J

Original: ( x ) − J ( y ) (cid:107) F ≤ L · (cid:107) x − y (cid:107) 2 , for all x, y ∈ [0 , 
Revised: ( x ) − J ( y ) (cid:107) F ≤ L · (cid:107) x − y (cid:107) 2 , for all x, y ∈ [0 , 1] n .

Original: ] n where J is the Jacobian of V, and (cid:107) A (cid:107) F denotes the Frobenious norm of the matrix A .
Revised: where J is the Jacobian of V, and (cid:107) A (cid:107) F denotes the Frobenious norm of the matrix A .

Original:  ST AY -ON- THE -R IDGE : H IGH -L EVEL D ESCRIPTION
Revised:  ST AY -ON- THE -R IDGE : H IGH -L EVEL D ESCRIPTION

Original: In this section we describe our algorithm and discuss the main design ideas leading to its convergence properties presented in Section 5.
Revised: In this section we describe our algorithm and discuss the main design ideas leading to its convergence properties presented in Section 5.

Original: As explained in the previous section, our goal is to ﬁnd a point x such that every coordinate i ∈ [ n ] is satisﬁed at x according to the Deﬁnition 1.
Revised: As explained in the previous section, our goal is to ﬁnd a point x such that every coordinate i ∈ [ n ] is satisﬁed at x according to the Deﬁnition 1.

Original: Our algorithm is initialized atpoint x (0) =
Revised: Our algorithm is initialized at x (0) =

Original: , 0) where a number of coordinates may be unsatisﬁed.

Original: The goal of the algorithm is to satisfy all unsatisﬁed coordinates one-by-one in lexicographic order (although, as we will see, coordinates may go from being satisﬁed to being unsatisﬁed in the course of the algorithm).
Revised: The goal of the algorithm is to satisfy all unsatisﬁed coordinates one-by-one in lexicographic order (although, as we will see, coordinates may go from being satisﬁed to being unsatisﬁed in the course of the algorithm).

Original: We say that our algorithm “starts epoch i at point x ” iff all coordinates ≤ i − 1 are satisﬁed at x and the algorithm’s immediate goal is to ﬁnd a point x (cid:48)(cid:54) =
Revised: We say that our algorithm “starts epoch i at point x ” iff all coordinates ≤ i − 1 are satisﬁed at x and the algorithm’s immediate goal is to ﬁnd a point x (cid:48)

Revised: (cid:54) =

Original: x that satisﬁes all coordinates ≤ i , namely:
Revised: x that satisﬁes all coordinates ≤ i , namely:

Original: Goal of epoch i , starting at point x : ﬁnd x (cid:48) (cid:54) =
Revised: Goal of epoch i , starting at point x : ﬁnd x (cid:48) (cid:54) =

Original: x satisfying all coordinates ≤ i .
Revised: x satisfying all coordinates ≤ i .

Original: We now describe how the algorithm tries to meet the afore-described goal.

Original: So let us assume that, at time t , our algorithm starts epoch i at point x ( t ) .
Revised: Let us assume that, at time t , our algorithm starts epoch i at point x ( t ) .

Original: Postponing full details to Section 5.1, where we describe our algorithm in detail, for simplicity of exposition let us  assumein this section that, at x ( t ) , all coordinates ≤ i − 1 are zero-satisﬁed, i.e. V j ( x ( t ))
Revised: Let us also assume that, at x ( t ) , all coordinates ≤ i − 1 are zero-satisﬁed (see Section 5.1 for the general case), i.e., V j ( x ( t ))

Original: [-= 
 for-] all j ≤ i − 1 .
Revised: [-= 
 for-]=for all j ≤ i − 1 .

Original: To achieve the goal of epoch i starting at x ( t ) , our algorithm will try to ﬁnd a point x (cid:48)
Revised: Our algorithm tries to achieve the goal of epoch i starting at x ( t ) as follows

Original: (cid:54) =

Original: x ( t ) where all coordinates ≤ i −remain zero-satisﬁed and coordinate i is also satisﬁed as follows:

Original:  • First, it will try to ﬁnd such a point in the connected subset S i ( x ( t )) ⊆
Revised: : • Our algorithm tries to ﬁnd such a point inside the connected subset S i ( x ( t )) ⊆

Original: [0 , 1] n that containsx ( t ) and all points z satisfying the following: (a) all coordinates ≤ i − 1 are zero-satisﬁed at z , and (b) for all j ≥ i + 1 ,
Revised: [0 , 1] n that contains all points z satisfying the following: (a) all coordinates ≤ i − 1 are zero-satisﬁed at z , and (b) for all j ≥ i + 1 ,

Original: z j = x j ( t ) .
Revised: z j = x j ( t ) .

Original: • Next, let us describe how our algorithm navigates S i ( x ( t )) in the hopes of identifying a point x (cid:48)
Revised: • Our algorithm navigates S i ( x ( t )) in the hopes of satisfying the goal of epoch

Original: (cid:54) =
Revised: i .

Original: x ( t ) where all coordinates ≤ i are satisﬁed. A natural approach is to  run a continuoustime dynamics { z ( τ ) } τ ≥ 0 that is initialized at z (0) =
Revised: A natural approach is to navigate S i ( x ( t )) is to run a continuous-time dynamics { z ( τ ) } τ ≥ 0 that is initialized at z (0) =

Original: x ( t ) and moves inside S i ( x ( t )) .
Revised: x ( t ) and moves inside S i ( x ( t )) .

Original: What are possible directions of movementfor such dynamics so that it stays within S i ( x ( t )) ?
Revised: What are possible directions of movement so that our dynamics stay within S i ( x ( t )) ?

Original: If the dynamics is at some point z ∈ S i ( x ( t )) , it will remain in this set if it moves, inﬁnitessimally, in a unit direction d satisfying the following constraints:
Revised: If the dynamics is at some point z ∈ S i ( x ( t )) , it will remain in this set if it moves, inﬁnitessimally, in a unit direction d satisfying the following constraints:

Original: d j = 0 , for all j ≥ i + 1 ; / * this is so that (b) in the deﬁnition of S i ( x ( t ))is maintained */ 2.
Revised: d j = 0 , for all j ≥ i + 1 ; /* to guarantee (b) in the deﬁnition of S i ( x ( t ))

Original: ( ∇ V j ( z )) (cid:62) · d = 0 , for all j ∈ { 1 , . . .
Revised: ( ∇ V j ( z )) (cid:62) · d = 0 , for all j ∈ { 1 , . . .

Original: , j − 1 } .
Revised: , j − 1 } .

Original: /* this is so that (a)is maintained */
Revised: /* to guarantee (a) */

Original: Notice that 1 and 2 specify n − 1 constraints on n variables.
Revised: Notice that 1 and 2 specify n − 1 constraints on n variables.

Original: We will place mild assumptions on f so that there is a unique, up to a sign ﬂip, unit direction satisfying these constraints.
Revised: We will place mild assumptions on V so that there is a unique, up to a sign ﬂip, unit direction satisfying these constraints (see

Original: (Speciﬁcally see Assumption 1 in Section 5.2, where our main result is formally stated.)
Revised: Assumption 1).

Original: Moreover,  wewill specify a way tobreak ties so that we choose one of the two unit directions satisfying our constraints.(Speciﬁcally this is done in part 3 of Deﬁnition 2 in Section 5.1.) Let us denote by
Revised: Moreover, in Deﬁnition 2 we specify a rule to choose one of the two unit directions satisfying our constraints.

Original:  D i ( z ) theunit direction that our tie-breaking rule selects at z .
Revised: We denote by D i ( z ) the direction that our tie-breaking rule selects at z .

Original: • With the above choices, the continuous-time dynamics ˙
Revised: • With the above choices, the continuous-time dynamics ˙

Original: z ( τ ) =
Revised: z ( τ ) =

Original: D i
Revised: D i

Original: ( z ( τ )) , initialized at z (0) =
Revised: ( z ( τ )) , initialized at z ( ) =

Original: x ( t ) , is well-deﬁned.
Revised: x ( t ) , is well-deﬁned.

Original: We follow this dynamics until the earliest time that one of the following happens (if both events happen at the same time we will say that the good event happened): – (Good Event): the dynamics stops at a point x (cid:48) (cid:54) =
Revised: We follow this dynamics until the earliest time that one of the following happens: – (Good Event): the dynamics stops at a point x (cid:48) (cid:54) =

Original: x ( t ) where coordinate i is satisﬁed; – (Bad Event): the dynamics stops at a point x (cid:48) lying on the boundary of [0 , 1] n (and if it were to continue it would violate the constraints).
Revised: x ( t ) where coordinate i is satisﬁed; – (Bad Event): the dynamics stops at a point x (cid:48) lying on the boundary of [0 , 1] n (and if it were to continue it would violate the constraints).

Original: So we have described what our algorithm does if, at time t , it starts epoch i atsome point x ( t ) .
Revised: So we have described what our algorithm does if, at time t , it starts epoch i at x ( t ) .

Original: Suppose x (cid:48) is the point where the continuous-time dynamics executed during epoch i terminates.
Revised: Suppose x (cid:48) is the point where our dynamics executed during epoch i terminates.

Original: If the good event happened, coordinate i is satisﬁed at x (cid:48) , and our algorithm starts epoch
Revised: If the good event happened, coordinate i is satisﬁed at x (cid:48) , and our algorithm starts epoch

Original: i + 1 at x (cid:48) .
Revised: i + 1 at x (cid:48) .

Original: If the bad event happened, our algorithm will in fact start epoch i − 1 at point x (cid:48) .
Revised: If the bad event happened, our algorithm will in fact start epoch i − 1 at point x (cid:48) .

Original: What does this mean?
Revised: What does this mean?

Original: That it will run the continuous-time dynamics corresponding to epoch
Revised: That it will run the continuous-time dynamics corresponding to epoch

Original: i − 1 on the set S i − 1 ( x (cid:48) )
Revised: i − 1 on the set S i − 1 ( x (cid:48) )

Original: starting at x (cid:48) in order to ﬁnd some point x (cid:48)(cid:48) (cid:54) =
Revised: starting at x (cid:48) in order to ﬁnd some point x (cid:48)(cid:48) (cid:54) =

Original: x (cid:48) where all coordinates ≤ i − 1 are satisﬁed.
Revised: x (cid:48) where all coordinates ≤ i − 1 are satisﬁed.

Original: It may fail to do this, in which case it will start epoch i − 2 next.
Revised: It may fail to do this, in which case it will start epoch i − 2 next.

Original: Or it may succeed, in which case, it will start epoch i , and so on so forth until (as we will show!)
Revised: Or it may succeed, in which case, it will start epoch i , and so on so forth until (as we will show!)

Original: all coordinates will be satisﬁed.
Revised: all coordinates will be satisﬁed.

Original: The high-level pseudocode of our algorithm is given below. Dynamics STay
Revised: The high-level pseudocode of our algorithm is given in Dynamics 1.

Original:  High-Level Description 1: Initially x (0) ← (0 , . . . , 0) , i ← 1 , t ← 0 .
Revised: Dynamics 1 STay-ON-the-Ridge (STON’R) — High-Level Description 1: Initially x (0) ← (0 , . . . , 0) , i ← 1 , t ← 0 .

Original: 2: while x ( t ) is not a VI solution do 3:
Revised: 2: while x ( t ) is not a VI solution do 3:

Original: Initialize epoch
Revised: Initialize epoch

Revised: i ’s continuous-time dynamics, ˙ z ( τ ) =

Original:  i’s
Revised: D i

Original: -time
Revised: ( z ( τ )) , at z (0) =

Original: , at
Revised: x ( t ) .

Original: 4: while exit condition of this dynamics has not been reached do 5:
Revised: 4: while exit condition of this dynamics has not been reached do 5:

Original: Execute ˙ z ( τ ) =
Revised: Execute ˙ z ( τ ) =

Original: D i
Revised: D i

Original: ( z ( τ )) forward in time.
Revised: ( z ( τ )) forward in time.

Original: 6: end while 7:
Revised: 6: end while 7:

Original: Set x ( t + τexit ) =
Revised: Set x ( t + τ ) =

Original: z ( τexit )  (where τ exit is time exit condition was met).
Revised: z ( τ ) for all τ ∈ [0 , τ exit ] (where τ exit is time exit condition was met).

Original: 8: if x ( t + τ exit ) (cid:54) =
Revised: 8: if x ( t + τ exit ) (cid:54) =

Original: x ( t ) and coordinate i is satisﬁed at x ( t + τ exit ) then 9:
Revised: x ( t ) and coordinate i is satisﬁed at x ( t + τ exit ) then 9:

Original: Update the epoch i ← i + 1 .
Revised: Update the epoch i ← i + 1 .

Original: 10: else 11: (Bad event happened so) move to the previous epoch i ← i − 1 .
Revised: 10: else 11: (Bad event happened so) move to the previous epoch i ← i − 1 .

Original: 12: end if 13:
Revised: 12: end if 13:

Original: Set t ← t + τ exit .
Revised: Set t ← t + τ exit .

Original: 14: end while 15: return x  t )
Revised: 14: end while 15: return x ( t )

Original: At this point we have described an algorithm that explores the space in a natural way in its effort to satisfy coordinates, but it is unclear why it wouldsucceed in eventually satisfying all of them, how it would escape cycles, and how it would not get stuck at non-equilibrium points.
Revised: At this point we have described an algorithm that explores the space in a natural way in its effort to satisfy coordinates, but it is unclear why it would eventually satisfy all of them, how it would escape cycles, and how it would not get stuck at non-equilibrium points.

Original: Importantly, there is no quantity that seems to be consistently improving during the execution of the algorithm.
Revised: Importantly, there is no quantity that seems to be consistently improving during the execution of the algorithm.

Original: For example, the number of satisﬁed coordinates might decrease during the algorithm’s execution.

Original: How we can show this algorithm converges since no quantity seems to be consistently improving during its execution?
Revised: How we can show convergence since no quantity seems to be consistently improving?

Original: To show the convergence of our algorithm we need to use a different kind of argument than the classical arguments used in optimization which are based on some quantity improving.

Original: In particular, we use a topological argument that we describe in Section 4.

Original: Before that we give an example that illustrates the execution of our algorithm in a simple 2-dimensional setting.

Original:  A T OPOLOGICAL A RGUMENT OF C ONVERGENCE
Revised:  A T OPOLOGICAL A RGUMENT OF C ONVERGENCE

Original: As discussed in Section 3, there seems to be no clear potential function that decreases in the course of our algorithm’s execution, which we could track to show that it converges.

Original: Indeed, even the number of satisﬁed coordinates might decrease in the course the algorithm’s execution as we explained in

Original: Section 3. So how we can show that our algorithm converges? Our main idea  is to use  topological arguments that have beensuccessfully employed to show the convergence of other equilibrium computation algorithms. In the celebrated Lemke & Howson (1964) algorithm, e.g., the following argument is used to prove the algorithm’s convergence.
Revised: Our main idea to show the convergence of the STON’R algorithm is to use a topological argument illutrated in Lemma 2 that has been employed to show the convergence of other equilibrium computation algorithms such as the elebrated Lemke-Howson algorithm Lemke & Howson (1964).

Original: Lemma 2.
Revised: Lemma 2.

Original: Let G =
Revised: Let G =

Original: ( N, E ) be a directed graph such that every node has in-degree at most 1 and out-degree at most 1 .
Revised: ( N, E ) be a directed graph such that every node has in-degree at most 1 and out-degree at most 1 .

Original: If there exists some node v ∈ N with in-degree 0 and out-degree 1 , then there is unique directed path starting at v and ending at some v (cid:48) ∈ N that has in-degree 1 and out-degree 0 .
Revised: If there exists some node v ∈ N with in-degree 0 and out-degree 1 , then there is unique directed path starting at v and ending at some v (cid:48) ∈ N that has in-degree 1 and out-degree 0 .

Original: The proof of Lemma 2 is straightforward, as Figure 2 illustrates.
Revised: The proof of Lemma 2 is straightforward, as Figure 1 illustrates.

Original: The lemma suggests the following recipe for proving the convergence of some deterministic, iterative algorithm, with update rule v t +1 ← F ( v t ) , whose iterates lie in a ﬁnite set N :
Revised: The lemma suggests a recipe for proving the convergence of some deterministic, iterative algorithm, with update rule v t +1 ← F ( v t ) , whose iterates lie in a ﬁnite set N :

Original: Deﬁne adirected graph G whose vertex set is V and edge set is E = { ( u, v ) | u (cid:54) = v and v = F ( u ) } , i.e. there is a directed edge from u to v iff v is different from u and v is reached after one iteration of the algorithm starting at u .2.
Revised: Deﬁne a graph G with vertices N and edges E = { ( u, v ) | u (cid:54) = v and v = F ( u ) } , i.e., there is an edge from u to v iff v (cid:54) = u and v is reached after an iteration of the algorithm starting at u .

Original: Argue that every vertex of G has in-degree ≤1 .
Revised: Argue that every vertex of G has in-degree ≤

Original: It is clear that every vertex has out-degree ≤ 1 .3.
Revised: It is clear that every vertex has out-degree ≤ 1 .

Revised: Show that the algorithm can

Revised: be

Original: Show that the algorithm can be initialized at some v 0 that has in-degree 0 and out-degree 1 . 4.
Revised: initialized at some v 0 that has in- degree 0 and out-degree Employ

Original: Employ Lemma 2 to argue that if the algorithm is initialized at v 0 it must, eventually, arrive at some node v end whose out-degree is 0 .
Revised: Lemma 2 to argue that if the algorithm is initialized at v 0 it must, eventually , arrive at some node v end whose out-degree is 0 .

Original: Out-degree 0 means that v end =
Revised: Out-degree 0 means that v end =

Original: F ( v end ) .
Revised: F ( v end ) .

Original: The above prove that if the algorithm starts at v 0 it is guaranteed to converge.
Revised: The above prove that if the algorithm starts at v 0 it is guaranteed to converge.

Original: Having this topological argument in place we are ready to formally describe our algorithm and argue its convergence.

Original: In the course of our description  wewill be sure to specify a ﬁnite set ofpoints V that will act as the nodes  of theﬁnite graph that we will construct to employ the above convergence argument.
Revised: In the course of the description of the algorithm and its convergence proof in Section 5, we specify a ﬁnite set of nodes N of the graph that we will construct to employ the above convergence argument.

Original: Intuitively, these are all the points at which our algorithm can possibly start an epoch.
Revised: Intuitively, these are all the points at which our algorithm can possibly start a new epoch.

Original: The map F ( · ) that we use to construct our graph is the outcome of the continuous-time process that our algorithm execute when it starts an epoch at such a point.
Revised: The map F ( · ) that we use to construct our graph is the outcome of the continuous-time process that our algorithm execute when it starts an epoch at such a point.

Original:  D ETAILED D ESCRIPTION OF STON’R AND M AIN R ESULT
Revised:  D ETAILED D ESCRIPTION OF STON’R AND M AIN R ESULT

Original: We provide a formal description of our algorithm, state our main convergence theorem, andprovide the main components of its proof building on the ideas from Section 4.
Revised: We provide a formal description of our algorithm (Section 5.1), state our main convergence theorem (Section 5.2), and the main components of its proof building on the ideas (Section 5.4).

Original: 5. STON’R: D ETAILED D ESCRIPTION
Revised: 5. STON’R: D ETAILED D ESCRIPTION

Original: We provide a detailed description of our algorithm, building on the framework from Section 3. To simplify our exposition in that section, we only described the behavior of the algorithm when it starts an epoch at some x where  coordinates ≤ i − 1 are zero-satisﬁed  and its goal is to identify some x (cid:48)
Revised: In Section 3 we focused on the epochs where all coordinates ≤ i − 1 are zero-satisﬁed at the initial point x and the goal is to identify some x (cid:48)

Original: (cid:54) =
Revised: (cid:54) =

Original: x at which coordinates ≤ i are satisﬁed.
Revised: x all coordinates ≤ i are satisﬁed.

Original: To achieve this goal the algorithm executed a continuous-time dynamics constrained by keeping all coordinates ≤ i − 1 zero-satisﬁed.
Revised: To achieve this, we execute a continuous-time dynamics constrained by keeping all coordinates ≤ i − 1 zero-satisﬁed.

Original: However, in the course of its execution the algorithm might be hitting the boundary in its effort to satisfy coordinates.
Revised: However, in the course of these dynamics we be hit the boundary.

Original: So,in the general case, when it starts a new epoch, some coordinates will be zero-satisﬁed and some will be boundary-satisﬁed.
Revised: So, when we start a new epoch, some coordinates will be zero-satisﬁed and some will be boundary-satisﬁed.

Original: As such, what the algorithm will do in the general case during some epoch is execute a continuous-time dynamics constrained by keeping the zero-satisﬁed coordinates zero-satisﬁed and the boundary-satisﬁed coordinates at the right boundary.
Revised: In that general case, the algorithm needs to execute a continuous-time dynamics constrained by keeping the zero-satisﬁed coordinates zero-satisﬁed as well as the boundary-satisﬁed coordinates at the right boundary.

Original: More precisely, the epochs of our algorithm are, in fact, indexednot only by some coordinate i but also by a subset of coordinates S ⊆
Revised: Namely, the epochs are indexed by some coordinate i ∈ [ n ] and a subset of coordinates S ⊆

Original: [ i − 1] that are zero-satisﬁed at the point x where the epoch starts.
Revised: [ i − 1] that are zero-satisﬁed at the point x where the epoch starts.

Original: The goal of the epoch is the following.
Revised: The goal of each epoch is the following.

Original: Goal of epoch ( i, S ) , starting at point x (where S ⊆
Revised: Goal of epoch ( i, S ) , starting at point x (where S ⊆

Original: [ i − 1] , coordinates in S are zero-satisﬁed and coordinates in [ i − 1] \ S are boundary-satisﬁed): ﬁnd x (cid:48) (cid:54) =
Revised: [ i − 1] , coordinates in S are zero-satisﬁed and coordinates in [ i − 1] \ S are boundary-satisﬁed): ﬁnd x (cid:48) (cid:54) =

Original: x where all coordinates ≤ i are satisﬁed, all coordinates in S are zero-satisﬁed and all coordinates in [ i − 1] \ S are boundary-satisﬁed.
Revised: x where all coordinates ≤ i are satisﬁed, all coordinates in S are zero-satisﬁed, and all coordinates in [ i − 1] \ S are boundary-satisﬁed.

Original: As in our high-level description in Section 3, epoch ( i, S ) starting at x might achieve its goal or end before it achieves its goal.
Revised: Epoch ( i, S ) starting at x might achieve its goal or end before it achieves its goal.

Original: In both cases, a new epoch will start.
Revised: In both cases, a new epoch will start.

Original: Now, how does the algorithm try to achieve its goal in some epoch? Similar to the special case discussed in Section 3, in the general case considered here the algorithm will execute a continuous-time dynamics that maintains all the coordinates j ∈ S zero-satisﬁed, all the coordinates j ∈
Revised: Within each epoch our algorithm executes a continuous-time dynamics that maintains all the coordinates j ∈ S zero-satisﬁed, all the coordinates j ∈

Original: [ i − 1] \ S boundary-satisﬁed, and leaves all coordinates [ n ] \ ( { i } ∪ S ) unchanged.
Revised: [ i − 1] \ S boundary-satisﬁed, and leaves all coordinates [ n ] \ [ i ] unchanged.

Original: The following deﬁnition captures the tangent unit vector of the curve that this continuous-time dynamics travels. Deﬁnition 2.
Revised: Deﬁnition 2 (Tangent Unit Vector of Epoch ( i, S ) ) .

Original: Given i ∈ [ n ] ,a set of coordinates S = { s 1 , . . .
Revised: Let i ∈ [ n ] , S = { s 1 , . . .

Original: , s m } ⊆
Revised: , s m } ⊆

Original: [ i − 1] , andsome point x  , we say that a unit vector d ∈ R n is admissible iff it satisﬁes the following:
Revised: [ i − 1] , and x ∈ [0 , 1] n , we say that a unit vector d ∈ R n is admissible if:

Original: d j = 0 , for all j / ∈ S ∪ { i } .
Revised: d j = 0 , for all j / ∈ S ∪ { i } , and 2.

Original: ∇ V j ( x ) (cid:62) · d = 0 , for all j ∈ S . 3.
Revised: ∇ V j ( x ) (cid:62) · d = 0 , for all j ∈ S , and

Original: The sign of (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ∂V s 1 ( x ) ∂x s∂V s 2 ( x ) ∂x s. . .
Revised: the sign of (cid:12)(cid:12)(cid:12) [Equation] [Equation]

Original: ∂V sm ( x ) ∂x sd s... ... ... ... ... ∂V s 1 ( x ) ∂x sm ∂V s 2 ( x ) ∂x sm . . . ∂V sm ( x ) ∂x sm d s m ∂V s 1 ( x ) ∂x
Revised: [Equation]

Original: i ∂V s 2 ( x ) ∂x i . . .
Revised: [Equation] [Equation]

Original: ∂V sm ( x ) ∂x i d i (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) equals  sign (cid:0) ( − 1) | S |(cid:1) .
Revised: [Equation] (cid:12)(cid:12)(cid:12) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) equals the sign of ( − 1) | S | .

Original: If there is a unique unit direction satisfying the above constraints, we denote that direction D iS ( x ) .
Revised: If there is a unique unit direction satisfying the above constraints, we denote that direction D i S ( x ) .

Revised: Conditions 1 and 2 above describe a line in R n and condition 3 speciﬁes a direction on this line.

Original: We will place mild assumptions on V so that D iS ( x ) is(uniquely) deﬁned for all x ∈ [0 , 1] n where coordinates S are zero-satisﬁed.
Revised: We will place some assumptions on V so that D iS ( x ) is deﬁned for all x ∈ [0 , 1] n where coordinates S are zero-satisﬁed (see Assumption 1).

Original: (Speciﬁcally see Assumption 1 in Section 5.2, where our main result is formally stated.)

Original: With this deﬁnition in place, when our algorithm starts epoch ( i, S ) at point x , it will execute the continuous-time dynamics ˙
Revised: Now, when we start epoch ( i, S ) at point x , we will execute the continuous-time dynamics ˙

Original: z ( τ ) =
Revised: z ( τ ) =

Original: D iS ( z ( τ )) , initialized at z (0) =
Revised: D iS ( z ( τ )) , initialized at z (0) =

Original: x , forward in time.
Revised: x , forward in time.

Original: The algorithm executes this dynamics until the earliest time τ exit such that z ( τ exit ) is an exit point, as per the deﬁnition below.
Revised: We this dynamics until the earliest time τ exit such that z ( τ exit ) is an exit point according to the next deﬁnition.

Original: Deﬁnition 3.
Revised: Deﬁnition 3.

Original: Suppose i ∈ [ n ] , S ⊆
Revised: Suppose i ∈ [ n ] , S ⊆

Original: [ i − 1] and x (cid:48) is a point where coordinates in S are zero-satisﬁed and coordinates in [ i − 1] \ S are boundary-satisﬁed.
Revised: [ i − 1] , at x (cid:48) ∈ [0 , 1] n , the coordinates in S are zero-satisﬁed at x (cid:48) , the coordinates in [ i − 1] \ S are boundary-satisﬁed at x (cid:48) .

Original: Then x (cid:48) is an exit point for epoch ( i, S )
Revised: Then x (cid:48) is an exit point for epoch ( i, S )

Original: iff it satisﬁes one of the following:
Revised: iff it satisﬁes one of the following:

Original: • (Good Exit Point) :
Revised: • (Good Exit Point) :

Original: Coordinate i is satisﬁed at x (cid:48) , i.e., V i ( x (cid:48) )
Revised: Coordinate i is satisﬁed at x (cid:48) , i.e., V i ( x (cid:48) )

Original: = 0 , or x (cid:48)
Revised: = 0 , or x (cid:48)

Original: i = 

Revised: i =

Original: and V i ( x (cid:48) )
Revised: and V i ( x (cid:48) )

Original: < 0 , or x (cid:48)
Revised: < 0 , or x (cid:48)

Original: i = 1 and V i ( x (cid:48) ) > 0 .
Revised: i = 1 and V i ( x (cid:48) ) > 0 .

Original: • (Bad Exit Point) : 
Revised: • (Bad Exit Point) : ∃ j ∈ S ∪ { i } s

Original: For some j ∈ S ∪{ i } , it holds that ( D iS ( x (cid:48) ))
Revised: t. ( D iS ( x (cid:48) ))

Original: j > 0 and x (cid:48)
Revised: j > 0 and x (cid:48)

Original: j = 1 , or ( D iS ( x (cid:48) ))
Revised: j = 1 , or ( D iS ( x (cid:48) ))

Original: j <and x (cid:48)
Revised: j < 0 and x (cid:48)

Original: j = 0 ; in other words, if the dynamics for epoch ( i, S ) were to continue from x (cid:48) onward, they would violate the constraints.
Revised: j = 0 , i.e., if the dynamics of epoch ( i, S ) were to continue from x (cid:48) , they would violate the constraints.

Original: • (Middling Exit Point) : For some j ∈ [ i − 1] \ S , it holds that V j ( x (cid:48) ) {+= 
+}
Revised: • (Middling Exit Point) : ∃ j ∈ [ i − 1] \ S s.t. V j ( x (cid:48) ) {+= 
+}

Original: = 0 andone of the following holds: ∇ V j ( x (cid:48) )
Revised: and

Original:  (cid:62) D iS ( x (cid:48) ) > 0 and x (cid:48) j = 0 , or  ∇ V j ( x (cid:48) ) (cid:62) D iS ( x (cid:48) )
Revised: ( ∇ V j ( x (cid:48) ) (cid:62) D iS ( x (cid:48) ) > 0 and x (cid:48) j = 0) or ( ∇ V j ( x (cid:48) ) (cid:62) D iS ( x (cid:48) )

Original: < 0 and x (cid:48)
Revised: < 0 and x (cid:48)

Original: j = 1 ; in other words, if the dynamics for epoch ( i, S ) were to continue from x (cid:48) onward, some boundary-satisﬁed coordinate would become unsatisﬁed.
Revised: j = 1) , i.e., if the dynamics for epoch ( i, S ) were to continue from x (cid:48) , some boundary-satisﬁed coordinate would become unsatisﬁed.

Original: We will place mild assumptions on V so that there can be a unique j triggering the condition of Bad
Revised: We will place some assumptions on V so that there can be a unique j triggering the condition of Bad

Original: Exit Pointin Deﬁnition 3 and there can be a unique j triggering the Middling Exit Point condition, when x (cid:48) is a point where coordinates in S are zero-satisﬁed and coordinates in [ i − 1] \ S are boundary-satisﬁed. (Speciﬁcally see Assumptions 2 and 3 in Section 5.2.)
Revised: Exit Point and there can be a unique j triggering the Middling Exit Point condition (see Assumptions 2).

Original: Here are the actions that weneed to take if one of the above exit conditions has been reached happen.
Revised: Below we describe the actions that we take when one of the above exit conditions is triggered.

Original: Action at Good Events.
Revised: Action at Good Events.

Original: In case of a good event, we start epoch
Revised: In case of a good event, we start epoch

Original: ( i +1 , S (cid:48) )
Revised: ( i +1 , S (cid:48) )

Original: at x (cid:48) , where S (cid:48)
Revised: at x (cid:48) , where S (cid:48)

Original: =
Revised: =

Original: S ∪{ i } , if i is zero-satisﬁed at x (cid:48) , and S (cid:48) =
Revised: S ∪{ i } , if i is zero-satisﬁed at x (cid:48) , and S (cid:48) =

Original: S , if i is boundary-satisﬁed at x (cid:48) .
Revised: S , if i is boundary-satisﬁed at x (cid:48) .

Original: Action at Bad Events.
Revised: Action at Bad Events.

Original: In case of a bad event, note that the coordinate j responsible for the condition in the bad eventto trigger must lie in S ∪ { i } because in all other coordinates ( D iS ( x (cid:48) ))
Revised: In case of a bad event, note that the coordinate j responsible for the condition in the bad event must belong to S ∪ { i } because in all other coordinates ( D iS ( x (cid:48) ))

Original: j = 0 by deﬁnition.
Revised: j = 0 by deﬁnition.

Original: Depending on which j triggers the condition of the bad event we do one of the following:
Revised: Our action depends on which j triggers the bed event as follows:

Original: (1) if the triggering j = i , then we start epoch
Revised: (1) if the triggering j = i , then we start epoch

Original: ( i − 1 , S \ { i − 1 } ) at x (cid:48) .
Revised: ( i − 1 , S \ { i − 1 } ) at x (cid:48) , otherwise (2)

Original: (2) if the triggering j (cid:54) =
Revised: if the triggering j (cid:54) =

Original: i , then we start epoch
Revised: i , then we start epoch

Original: ( i, S \ { j } ) at x (cid:48) .
Revised: ( i, S \ { j } ) at x (cid:48) .

Original: Action at Middling Events.
Revised: Action at Middling Events.

Original: In the case of a middling event, we start epoch
Revised: In this case, we start epoch

Original: ( i, S ∪ { j } ) at x (cid:48) (because the coordinate jthat trigger this event is both zero- and boundary-satisﬁed at x (cid:48)
Revised: ( i, S ∪ { j } ) at x (cid:48) because the coordinate j is both zero- and boundary-satisﬁed at x (cid:48)

Original: so we add it to S to constrain the dynamics to zero satisfy
Revised: so we add j to S to keep it zero-satisﬁed next.

Original: it next. With all the above, our algorithm is easy to describe precisely as follows.
Revised: Combining the above rules we get a full description of our algorithm in Dynamics 2.

Original: In Section B we do a step-by-step analysis of what the algorithmwould do for a simple  min-max optimization problem.
Revised: In Appendix B we do a step-by-step execution of this algorithm for a simple 2D min-max optimization problem.

Original: Dynamics 2 STay-ON-the-Ridge (STON’R)
Revised: Dynamics 2 STay-ON-the-Ridge (STON’R)

Original: 1: Initially x (0) ← (0 , . . . , 0) , i ← 1 , S ← ∅ , t ← 0 .
Revised: 1: Initially x (0) ← (0 , . . . , 0) , i ← 1 , S ← ∅ , t ← 0 .

Original:  while x ( t ) is not a VI solution do 
Revised: 2: while x ( t ) is not a VI solution do 3:

Original: 3: Initialize epoch ( i, S ) ’s continuous-time dynamics, ˙
Revised: Initialize epoch ( i, S ) ’s continuous-time dynamics, ˙

Original: z ( τ ) =
Revised: z ( τ ) =

Original: D iS ( z ( τ )) , at z ( =
Revised: D iS ( z ( τ )) , at z (0) =

Original: x ( t ) .
Revised: x ( t ) .

Original: 4: while z ( τ ) is not an exit point as per Deﬁnition 3 do 5:
Revised: 4: while z ( τ ) is not an exit point as per Deﬁnition 3 do 5:

Original: Execute ˙ z ( τ ) =
Revised: Execute ˙ z ( τ ) =

Original: D iS ( z ( τ )) forward in time.
Revised: D iS ( z ( τ )) forward in time.

Original: 6: end while 7:
Revised: 6: end while 7:

Original: Set x ( t + τexit ) =
Revised: Set x ( t + τ ) =

Original: z ( τexit )  (where τ exit is earliest time z ( τ ) became an exit point) .
Revised: z ( τ ) for all τ ∈ [0 , τ exit ] (where τ exit is the time z ( τ ) became an exit point) .

Original: 8: if x ( t + τ exit ) is (Good Exit Point) as in Deﬁnition 3 then 9: if i is zero-satisﬁed at x ( t + τ exit ) then 10:
Revised: 8: if x ( t + τ exit ) is (Good Exit Point) as in Deﬁnition 3 then 9: if i is zero-satisﬁed at x ( t + τ exit ) then 10:

Original: Update S ← S ∪ { i } .
Revised: Update S ← S ∪ { i } .

Original: 11: end if 12:
Revised: 11: end if 12:

Original: Update i ← i + 1 .
Revised: Update i ← i + 1 .

Original: 13: else if x ( t + τ exit ) is a (Bad Exit Point) as in Deﬁnition 3 for j
Revised: 13: else if x ( t + τ exit ) is a (Bad Exit Point) as in Deﬁnition 3 for j

Original: = i then 14:
Revised: = i then 14:

Original: Update i ← i − 1 and S ← S \ { i − 1 } .
Revised: Update i ← i − 1 and S ← S \ { i − 1 } .

Original: 15: else if x ( t + τ exit ) is a (Bad Exit Point) as in Deﬁnition 3 for j (cid:54)
Revised: 15: else if x ( t + τ exit ) is a (Bad Exit Point) as in Deﬁnition 3 for j (cid:54)

Original: =
Revised: =

Original: i then 16:
Revised: i then 16:

Original: Update S ← S \ { j } .
Revised: Update S ← S \ { j } .

Original: 17: else if x ( t + τ exit ) is a (Middling Exit Point) as in Deﬁnition 3 for j <
Revised: 17: else if x ( t + τ exit ) is a (Middling Exit Point) as in Deﬁnition 3 for j <

Original: i then 18:
Revised: i then 18:

Original: Update S ← S ∪ { j } .
Revised: Update S ← S ∪ { j } .

Original: 19: end if 20:
Revised: 19: end if 20:

Original: Set t ← t + τ exit .
Revised: Set t ← t + τ exit .

Original: 21: end while 22: return x ( t )
Revised: 21: end while 22: return x ( t )

Original: 5. O UR A SSUMPTIONS AND O UR M AIN T HEOREM
Revised: 5. O UR A SSUMPTIONS AND O UR M AIN T HEOREM

Original: Assumptions.

Original: We next present the assumptions on V that are needed for our convergence proof.
Revised: We next present the assumptions on V that are needed for our convergence proof.

Original: We discuss these assumptions further in Appendix C explaining why they are mild.
Revised: We discuss these assumptions further in Appendix A where we present some high level reasons why they are mild.

Original: Assumption 1.
Revised: Assumption 1.

Original: There existpositive real numbers0 < σ min < σ max so that the following holds: For all x ∈ [0 , 1] n and set of coordinates S = { s 1 , . . .
Revised: There exist real numbers σ max > σ min > 0 such that: for all x ∈ [0 , 1] n and for all S = { s 1 , . . .

Original: , s m }  , if V (cid:96) ( x ) = [-
-] for all (cid:96) ∈ S , then the singular values of the m × m matrix[Equation]
Revised: , s m } ⊆ [ n ] , if V (cid:96) ( x ) = [-
-]0 for all (cid:96) ∈ S , then the singular values of the m × m matrix

Original: [Equation] [Equation]
Revised: J KS ( x ) are greater than σ min and less than σ max , where

Original: [Equation]
Revised: [Equation]

Original:  
Revised: Assumption 1 ensures that the direction D iS ( · ) of Deﬁnition 2 is uniquely deﬁned (see Lemma 11 in Appendix 8).

Original: Assumption 2.
Revised: Assumption 2.

Original: For any x ∈ [0 , 1] n , set of coordinates S = { s 1 , . . . , s m } , and i/ ∈  S  : If ( V (cid:96) ( x ) = 0 for all (cid:96) ∈ S) and( x (cid:96) ∈ { 0 , 1 } for all (cid:96) / ∈ S ∪ { i }) then there is at most one coordinate j ∈ S ∪ { i } such that x j = 0 or x j = 1  .
Revised: For all x ∈ [0 , 1] n , for all i ∈ [ n ] , and for all S ⊆ [ i − 1] : if V (cid:96) ( x ) = 0 ∀ (cid:96) ∈ S and x (cid:96) ∈ { 0 , 1 } ∀ (cid:96) / ∈ S ∪ { i } then there is at most one coordinate j ∈ S ∪ { i } such that x j ∈ { 0 , 1 } .

Original: One may ensure Assumption 2 holds by restricting the domain of each variable i in the subset [ α i , 1− β i ] of [0 , 1] , where α i , β i are uniformly random in [0 , (cid:15) ] .
Revised: Assumption 2 ensures that any time at most one coordinate can trigger a middling or a bad event. To see this, imagine there are two different coordinates j 1 , j 2 triggering a bad event at x , then x j 1 ∈ { 0 , 1 } , x j 2 ∈ { 0 , 1 } and V j 1 ( x ) =

Original: For details we refer to Section C.{+V j 2 ( x ) = 
 and therefore Assumption 2 is violated. A similar observation applies for middling events. See also Lemma 8 and Lemma 10 in the Appendix.+}
Revised: {+V j 2 ( x ) = 
 and therefore Assumption 2 is violated. A similar observation applies for middling events. See also Lemma 8 and Lemma 10 in the Appendix.+}

Original: Assumption 3.
Revised: Assumption 3.

Original: For all: (i) collection of coordinates S = ( s 1 , . . .
Revised: For all x ∈ [0 , 1] n , for all i ∈ [ n ] , for all S ⊆

Original: , s m ) , (ii) coordinate i / ∈ S , (iii) point x ∈ [0 , 1]n such that( V (cid:96) ( x ) = [-
 for all-] (cid:96) ∈ S) and( x (cid:96) ∈ { 0 , 1 } for all (cid:96) / ∈ S ∪ { i } ), and (iv) vector ( d s 1 , . . .
Revised: [ i − 1] such that V (cid:96) ( x ) = [-
 for all-]0 ∀ (cid:96) ∈ S and x (cid:96) ∈ { 0 , 1 } ∀ (cid:96) / ∈ S ∪ { i } , and for all vectors ( d s 1 , . . .

Original: , d s m , d i ) satisfying the equations, ∇ S ∪{ i } V j ( x ) (cid:62) · ( d s 1 , . . .
Revised: , d s m , d i ) satisfying the equations, ∇ S ∪{ i } V j ( x ) (cid:62) · ( d s 1 , . . .

Original: , d s m , d i )
Revised: , d s m , d i )

Original: = 

Revised: =

Original: for all j ∈ S, we have that d j (cid:54) =
Revised: for all j ∈ S, we have that d j (cid:54) =

Original: if x j = 0 or x j = 1 .
Revised: if x j = 0 or x j = 1 .

Revised: Assumption 3 ensures that we can determine whether a coordinate begins or stops being satisﬁed by looking at the Jacobian of V .

Revised: For example, consider a coordinate j such that x j = 0 and V j ( x ) = 0 .

Revised: If also D iS ( x ) (cid:62) V j ( x ) = 0 then higher-order information is needed in order to determine whether the direction D iS ( · ) makes the coordinate j satisﬁed or unsatisﬁed (see Lemma 4 in the Appendix).

Original: We are now ready to state our main theorem.
Revised: We are now ready to state our main theorem.

Original: The full proof of Theorem 1 is presented in Appendix E.

Original: Theorem 1.
Revised: Theorem 1.

Original: Under Assumptions 1, 2, and 3, there exists some ¯ T
Revised: Under Assumptions 1, 2, and 3, there exists some ¯ T

Original: = ¯ T ( σ min , σ max , n ) > 0 such that 
Revised: = ¯ T ( σ min , σ max , n, L, Λ) >such that ST AY -O N - THE -R

Original: ST AY -O N - THE -R IDGE (Dynamics 2) will stop, at some time T ≤ ¯ T , at some point x ( T ) ∈ [0 , 1] n that is a solution of VI( V, [0 , 1] n ) .
Revised: IDGE (Dynamics 2) will stop, at some time T ≤ ¯ T , at some point x ( T ) ∈ [0 , 1] n that is a solution of VI( V, [0 , 1] n ) .

Original: Remark 1 (Discrete-time Algorithm) .
Revised: Remark 1 (Discrete-time Algorithm) .

Original: It is possible to combine the proof of Theorem 1 with standard numerical analysis techniques to show the convergence of a simple discrete version of the dynamics assuming that the step size is small enough.
Revised: It is possible to combine the proof of Theorem 1 with standard numerical analysis techniques to show the convergence of a simple discrete version of the dynamics assuming that the step size is small enough.

Original: For more details about this we refer to Appendix K.
Revised: For more details about this we refer to Appendix J.

Revised: 5. S IMULATED 2-D IMENSIONAL E XAMPLE

Revised: Figure

Revised: In Figure 2 we present the behavior of the main existing algorithms for min-max optimization in the 2-dimensional minmax problem with objective f ( θ, ω ) :

Revised: = − θω − 120 · ω 2 + 220 ·

Revised: S (cid:16) θ 2 + ω 2(cid:17) · ω 2 , where S ( z ) is the smooth-step function equal to 0 for z ≤ 0 , 1 for z ≥ 1 and z 2 − 2 z 3 otherwise.

Original: 5. S KETCH OF P ROOF OF T HEOREM
Revised: With blue we observe the behavior of GDA, EG, and OGDA that have the same behavior in this example when initialized at ( − 0 . 5 , − 1) . With orange we observe the behavior of the follow-the-ridge (FtR) algorithm initialized at ( − 0 . 5 , − 0 . 5) and with green we observe the behavior of STON’R. As we can see GDA, EG, OGDA are getting trapped to a cycle whereas FtR hits the boundary at ( − 1 , − 1) that does not correspond to an equilibrium point.

Revised: Our algorithm is the only one that directly converges to the equilibrium following a very short path.

Revised: In Appendix C we provide a more detailed explanation of this example and we observe similar behavior for different initializations of GDA, EG,

Revised: OGDA, and FtR.

Revised: 5. S KETCH OF P ROOF OF T HEOREM

Original: A sketch of our proof of Theorem 1 comes from the recipe that we described in Section 4.
Revised: For a sketch of our proof of Theorem 1 we follow the recipe that we described in Section 4.

Revised: During this proof sketch we highlight some technical challenges that we face.

Revised: (The full proof can be found in

Revised: Appendix E.) 1.

Original: We start with the deﬁnition of the set of nodes N .
Revised: We start with the deﬁnition of the set of nodes N .

Original: The set N contains triples of the form ( i, S, x ) where i ∈ [ n ] , S is a subset of [ i − 1] and x ∈ [0 , 1] n that satisﬁes the following:
Revised: The set N contains triples of the form ( i, S, x ) where i ∈ [ n ] , S is a subset of [ i − 1] and x ∈ [0 , 1] n that satisﬁes the following:

Original: • (a) all coordinates in S are zero-satisﬁed, (b) all coordinates in [ i − 1] \ S are boundary-satisﬁed, (c) x j = 0 for all j ≥ i + 1 , and either (d1) x
Revised: (a) all coordinates in S are zero-satisﬁed, (b) all coordinates in [ i − 1] \ S are boundary-satisﬁed, (c) x j = 0 for all j ≥ i + 1 , and either (d1) x

Original: i = 0 or (d2) x is an exit point for epoch ( i, S ) according to Deﬁnition 3 1 .
Revised: i = 0 or (d2) x is an exit point for epoch ( i, S ) according to Deﬁnition 3 1 .

Original: We show in the Appendix the size of N is ﬁnite (see Lemma 3).
Revised: Our ﬁrst technical challenge is to show that the size of N is ﬁnite (see Lemma 3 in the Appendix).

Original: Next we describe a mapping F : N → N as in Section 4.
Revised: Next we describe a mapping F : N → N .

Original: Let ( i, S, x ) ∈ N  we use the dynamics ˙
Revised: Let ( i, S, x ) ∈ N , we use the dynamics ˙

Original: z = D iS ( z ) with initial condition z (0) =
Revised: z = D iS ( z ) with initial condition z (0) =

Original: x and we ﬁnd the minimum time τ exit such that z ( τ exit ) is an exit point.
Revised: x and we ﬁnd the minimum time τ exit such that z ( τ exit ) is an exit point.

Original: We then update i , S to i (cid:48) , S (cid:48)
Revised: We then update i , S to i (cid:48) , S (cid:48)

Original: according to the rules for actions on exit points of Section 5.1 and we deﬁne F (( i, S, x ))
Revised: according to the rules for actions on exit points of Section 5. and we deﬁne F (( i, S, x ))

Original: =
Revised: =

Original: ( i (cid:48) , S (cid:48) , z ( τ exit )) .
Revised: ( i (cid:48) , S (cid:48) , z ( τ exit )) .

Original: As we show in the appendix the dynamics ˙
Revised: One of our main technical challenges is to show that the dynamics ˙

Original: z = D iS ( z ) have a unique solution under our assumptions and hence F is well deﬁned (Lemma 9).
Revised: z = D iS ( z ) have a unique solution under our assumptions and hence F is well deﬁned (see Lemma 4 in the Appendix).

Original: The set N and the mapping F deﬁne the directed graph G  aswe described in Section 4 that is guaranteed to have vertices with out-degree at most 1 .
Revised: The set N and the mapping F deﬁne the directed graph G , as described in Section 4, that is guaranteed to have vertices with out-degree at most 1 .

Original: We also show that any v ∈ V with out-degree 0 is an equilibrium point (Lemma 5 and 6).
Revised: We also show that any v ∈ V with out-degree 0 is an equilibrium point (see Lemma 4 in the Appendix).

Original: To show that the in-degree is at most 1 too we  show that we can actually solve the dynamics backwards in time.
Revised: To show that the in-degree is at most 1 , we face our next technical challenge which is to show that we can actually solve the dynamics backwards in time.

Original: In particular, if we specify z (0) and there is the smallest time τ exit such that z ( − τ exit ) is an exit point then z ( − τ exit ) is uniquely determined.
Revised: In particular, if we specify z (0) and there is the smallest time τ exit such that z ( − τ exit ) is an exit point then z ( − τ exit ) is uniquely determined.

Original: This means that there exists F − 1 : N → N such that if v (cid:48) =
Revised: This means that there exists F − 1 : N → N such that if v (cid:48) =

Original: F ( v ) then F − 1 ( v (cid:48) )
Revised: F ( v ) then F − 1 ( v (cid:48) )

Original: = v which means that no vertex in N can have in-degree more than 1 (Lemma 7).
Revised: = v which means that no vertex in N can have in-degree more than 1 (see Lemma 5 in the Appendix).

Original: We show that v 0 =
Revised: We show that v 0 =

Original: (1 , ∅ , (0 , . . . , 0))
Revised: (1 , ∅ , (0 , . . . , 0))

Original: ∈ N . Also, if run the dynamics ˙
Revised: ∈ N and that if run the dynamics ˙

Original: z = D 1 ∅ ( z ) backwards in time starting at z (0)
Revised: z = D 1 ∅ ( z ) backwards in time starting at z (0)

Original: = 

Revised: =

Original: then we get outside [0 , 1] n and so v 0 has in-degree 0 .
Revised: then we get outside [0 , 1] n and so v 0 has in-degree 0 .

Original: We also show that the dynamics ˙
Revised: We also show that the dynamics ˙

Original: z = D 1 ∅ ( z ) can move forward in time and stay inside [0 , 1] n so v 0 has out-degree(Lemma 7).
Revised: z = D 1 ∅ ( z ) can move forward in time and stay inside [0 , 1] n so v 0 has out-degree(see Lemma 6 in the Appendix).

Original: The above show that our algorithm converges 
Revised: The above show that our algorithm converges according to Section 4.  C ONCLUSIONS Summary.

Revised: In this work we propose a novel local-search algorithm, called STON’R, that is guaranteed to converge to local min-max equilibrium in the general case of non-convex non-concave objectives.

Revised: To the best of our knowledge STON’R is the ﬁrst method, beyond trivial brute-force, that is guaranteed to ﬁnd a local min-max equilibrium starting from a simple initialization.

Revised: We remark that existing min-max optimization methods required either convexity (resp.

Revised: concavity) in one of the players or an initialization very close to the optimal point in order to guarantee convergence.

Revised: Finally, our approach differs from existing methods in the fundamental way that both its design and analysis are based on topological rather than potential arguments.

Revised: We believe that these types of arguments can play an important role in the future of multi-agent machine learning.

Revised: Comparison with Brute-Force.

Revised: Since we assume that V is a Lipschitz function and that K is an n -dimensional hypercube, it is not hard to see that there exists a small enough discretization of the space such that the brute-force search over all the discrete points is guaranteed to ﬁnd a solution.

Revised: Such brute-force algorithms exist in most of the optimization problems like solving linear programs or ﬁnding Nash equilibria in normal form games.

Revised: These trivial algorithms suffer from the curse of dimensionality even in very simple instances and hence they are almost never useful.

Revised: Instead localsearch algorithms such as simplex or Lemke-Howson Lemke & Howson (1964) have been extremely successful in practice because they converge very fast in the majority of real world instances although in the worst-case their complexity is the same as the brute-force.

Revised: Our contribution is to provide the ﬁrst such algorithm for the fundamental problem of nonconvex-nonconcave min-max optimization and we believe that it will play an important role in the future of multi-agent optimization in machine learning.