Original: A daptive E xtra -G radient M ethods for M in -M ax O ptimization and G ames
Revised: A daptive E xtra -G radient M ethods for M in -M ax O ptimization and G ames

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A bstract
Revised: A bstract

Original: We present a new family of min-max optimization algorithms that automatically exploit the geometry of the gradient data observed at earlier iterations to perform more informative extra-gradient steps in later ones.
Revised: We present a new family of min-max optimization algorithms that automatically exploits the geometry of the gradient data observed at earlier iterations to perform more informative extra-gradient steps in later ones.

Original: Thanks to this adaptation mechanism, the proposed methods automatically detect  the problem’s smoothness properties, without requiring any prior tuning by the optimizer.
Revised: Thanks to this adaptation mechanism, the proposed methods automatically detect whether the problem is smooth or not, without requiring any prior tuning by the optimizer.

Original: As a result, the algorithm simultaneously achieves the optimal smooth / non-smooth min-max convergence rates, i.e., it converges to an ε -optimal solution within O (1 /ε ) iterations in smooth problems, and within O (1 /ε 2 ) iterations in non-smooth ones.
Revised: As a result, the algorithm simultaneously achieves order-optimal convergence rates, i.e., it converges to an ε -optimal solution within O (1 /ε ) iterations in smooth problems, and within O (1 /ε 2 ) iterations in non-smooth ones.

Original: Importantly, these guarantees do not require any of the standard boundedness or Lipschitz continuity conditions that are typically assumed in the literature; in particular, they apply even to problems with divergent loss functions (such as log-likelihood learning and the like).
Revised: Importantly, these guarantees do not require any of the standard boundedness or Lipschitz continuity conditions that are typically assumed in the literature; in particular, they apply even to problems with singularities (such as resource allocation problems and the like).

Original: This“o ﬀ -the-shelf” adaptation is achieved through the use of a geometric apparatus based on Finsler metrics and a suitably chosen mirror-prox template that allows us to derive sharp convergence rates for thefamily of methods at hand.
Revised: This adaptation is achieved through the use of a geometric apparatus based on Finsler metrics and a suitably chosen mirror-prox template that allows us to derive sharp convergence rates for the methods at hand.

Original:  I ntroduction
Revised:  I ntroduction

Original: The surge of recent breakthroughs in generative adversarial networks (GANs) [24], robust reinforcement learning [48], and other adversarial learning models [32] has sparked renewed interest in the theory of min-max optimization problems and games.
Revised: The surge of recent breakthroughs in generative adversarial networks (GANs) [20], robust reinforcement learning [43], and other adversarial learning models [28] has sparked renewed interest in the theory of min-max optimization problems and games.

Original: In this broad setting, it has become empirically clear that, ceteris paribus, the simultaneous training of two (or more) antagonistic models faces drastically new challenges relative to the training of a single one.
Revised: In this broad setting, it has become empirically clear that, ceteris paribus, the simultaneous training of two (or more) antagonistic models faces drastically new challenges relative to the training of a single one.

Original: Perhaps the most prominent of these challenges is the appearance of cycles and recurrent (or even chaotic) behavior in min-max games.
Revised: Perhaps the most prominent of these challenges is the appearance of cycles and recurrent (or even chaotic) behavior in min-max games.

Original: This has been studied extensively in the context of  bilinear games [12, 15, 19, 22, 23, 36, 37, 45, 47], and the methods proposed to overcome cycling and recurrent behaviors typically focus on mitigating the rotational component of min-max games that is at the root of these phenomena.
Revised: This has been studied extensively in the context of learning in bilinear games, in both continuous [16, 32, 42] and discrete time [12, 18, 19, 33], and the methods proposed to overcome recurrence typically focus on mitigating the rotational component of min-max games.

Original: The method with the richest history in this context is the extra-gradient (EG) algorithm of Korpelevich [29] and its variants – for an appetizer, see [15, 33, 34, 40, 42, 50, 51] and references therein.
Revised: The method with the richest history in this context is the extra-gradient (EG) algorithm of Korpelevich [25] and its variants.

Original: Originally applied to  variational inequalities, the EG algorithm exploits the Lipschitz smoothness of the problem and, if coupled with an iterate averaging scheme – the mirror-prox (MP) template [40] – it achieves an O (1 / T ) rate of convergence in smooth, convex-concave min-max problems.
Revised: Originally designed to solve variational inequalities, the EG algorithm exploits the Lipschitz smoothness of the problem and, if coupled with a Polyak–Ruppert averaging scheme, it achieves an O (1 / T ) rate of convergence in smooth, convex-concave min-max problems [36].

Original: This rate is known to be tight [39, 44] but, in order to achieve it, the original method requires the problem’s
Revised: This rate is known to be tight [35, 41] but, in order to achieve it, the original method requires the problem’s

Original: Lipschitz constant to be known in advance.
Revised: Lipschitz constant to be known in advance.

Original: If the problem is not Lipschitz  (or the algorithm is run with a vanishing step-size schedule), the method’s rate of convergence drops to O (1 / √ T ) [40].
Revised: If the problem is not Lipschitz smooth (or the algorithm is run with a vanishing step-size schedule), the method’s rate of convergence drops to O (1 / √ T ).

Original: Our contributions.
Revised: Our contributions.

Original: Our aim in this paper is to provide an algorithm that automatically adapts to smooth / non-smooth min-max problems and games, and achieves optimal rates in both classes without requiring any  tuning by the optimizer.
Revised: Our aim in this paper is to provide an algorithm that automatically adapts to smooth / non-smooth min-max problems and games, and achieves order-optimal rates in both classes without requiring any prior tuning by the optimizer.

Original: In this regard, we propose a ﬂexible algorithmic scheme, which we call UniProx, and which exploits gradient data observed at earlier iterations to perform more informative extra-gradient steps in later ones.
Revised: In this regard, we propose a ﬂexible algorithmic scheme, which we call AdaProx, and which exploits gradient data observed at earlier iterations to perform more informative extra-gradient steps in later ones.

Original: Thanks to this mechanism, and to the best of our knowledge, UniProx is the ﬁrst algorithm that simultaneously achieves the following:
Revised: Thanks to this mechanism, and to the best of our knowledge, AdaProx is the ﬁrst algorithm that simultaneously achieves the following

Original: An O (cid:0) 1 / √ T (cid:1) convergence rate in non-smooth problems and O (1 / T ) in smooth ones.
Revised: An O (cid:0) 1 / √ T (cid:1) convergence rate in non-smooth problems and O (1 / T ) in smooth ones.

Original: Convergence in min-max problems and games where
Revised: Convergence in min-max problems and games where

Original: the standard boundedness / Lipschitz continuity conditions required in the literature do not hold.
Revised: the standard boundedness / Lipschitz continuity conditions required in the literature do not hold.

Original: “O ﬀ -the-shelf” applicability without requiring prior knowledge of the problem’s parameters.
Revised: No prior knowledge of the problem’s parameters (e g., whether the problem’s deﬁning vector ﬁeld is smooth or not, its smoothness modulus if it is, etc.).

Original: [Table]
Revised: [Table]

Original: The methodwe propose achieves the above by fusing the following ingredients: a ) a family of local norms – a Finsler metric – capturing any singularities in the problem at hand; b ) a suitable mirror-prox template for the algorithm; and c ) an adaptive step-size policy in the spirit of Rakhlin & Sridharan [51].
Revised: Our proposed method achieves the above by fusing the following ingredients: a ) a family of local norms – a Finsler metric – capturing any singularities in the problem at hand; b ) a suitable mirror-prox template; and c ) an adaptive step-size policy in the spirit of Rakhlin & Sridharan [46].

Original: As an extra feature of our analysis, we also show that the sequence of iterates generated by the algorithm converges, thus providing an appealing alternative to iterate averaging in cases where the method’s “last iterate” is more appropriate.
Revised: We also show that, under a suitable coherence assumption, the sequence of iterates generated by the algorithm converges, thus providing an appealing alternative to iterate averaging in cases where the method’s “last iterate” is more appropriate (for instance, if using AdaProx to solve non-monotone problems).

Original: Related works.
Revised: Related works.

Original: There have been several works improving on the guarantees of the original extragradient / mirror-prox template.
Revised: There have been several works improving on the guarantees of the original extragradient / mirror-prox template.

Original: We review the most relevant of these works below; for convenience, we also tabulate these contributions in Table 1 above.
Revised: We review the most relevant of these works below; for convenience, we also tabulate these contributions in Table 1 above.

Original: Because many of these works appear in the literature on variational inequalities [18], we also use this language in the sequel.
Revised: Because many of these works appear in the literature on variational inequalities [15], we also use this language in the sequel.

Original: In unconstrained problems with an operator that is locally Lipschitz continuous (but not necessarily globally so), the golden ratio algorithm (GRAAL) [34] achieves convergence without requiring prior knowledge of the problem’s Lipschitz parameter.
Revised: In unconstrained problems with an operator that is locally Lipschitz continuous (but not necessarily globally so), the golden ratio algorithm (GRAAL) [30] achieves convergence without requiring prior knowledge of the problem’s Lipschitz parameter.

Original: However, GRAAL provides no rate guarantees for non-smooth problems  and hence, a fortiori, no interpolation guarantees either.
Revised: However, GRAAL provides no rate guarantees for non-smooth problems – and hence, a fortiori, no interpolation guarantees either.

Revised: By contrast, such guarantees are provided in problems with a bounded domain by the generalized mirror-prox (GMP) algorithm of [50] under the umbrella of Hölder continuity.

Original: Moreover, nothing is known about the convergence of GRAAL  inconstrained problems with divergent operators.
Revised: Still, nothing is known about the convergence of GRAAL / GMP in problems with singularities (i.e., when the problem’s deﬁning vector ﬁeld blows up at a boundary point of the problem’s domain).

Original: Convergence guarantees for problems with divergent operators have been provided in a recent series of papers [1, 21, 55] exploiting a “Bregman continuity” or “Lipschitz-like” condition.
Revised: Singularities of this type were treated in a recent series of papers [1, 17, 51] by means of a “Bregman continuity” or “Lipschitz-like” condition.

Original: These methods achieve optimal rates in the smooth case, without requiring any knowledge of the problem’s parameters.
Revised: These methods are order-optimal in the smooth case, without requiring any knowledge of the problem’s smoothness modulus.

Original: However, like GRAAL, they do not provide“o ﬀ -the-shelf” adaptivity to the non-smooth regime or any rate interpolation guarantees.
Revised: On the other hand, like GRAAL (but unlike GMP), they do not provide any rate interpolation guarantees between smooth and non-smooth problems.

Original: To the best of our knowledge, the only method that simultaneously achieves an O (1 / √ T ) rate in non-smooth problems and an O (1 / T ) rate in smooth ones is the recent algorithm of Bach & Levy [2].
Revised: Another method that simultaneously achieves an O (1 / √ T ) rate in non-smooth problems and an O (1 / T ) rate in smooth ones is the recent algorithm of Bach & Levy [2].

Original: The BL algorithm employs an adaptive, AdaGrad-like step-size policy which allows the method to interpolate between the two regimes – and this, even with noisy gradient feedback.
Revised: The BL algorithm employs an adaptive, AdaGrad-like step-size policy which allows the method to interpolate between the two regimes – and this, even with noisy gradient feedback.

Original: On the negative side, the BL algorithm requires a bounded domain with a (Bregman) diameter that is known in advance; as a result, its theoretical guarantees do not apply to unbounded problems.
Revised: On the negative side, the BL algorithm requires a bounded domain with a (Bregman) diameter that is known in advance; as a result, its theoretical guarantees do not apply to unbounded problems.

Original: In addition, the BL algorithm makes crucial use of boundedness and Lipschitz continuity requirements; extending the BL method beyond this standard framework is a highly non-trivial endeavor which formed a big part of this paper’s motivation.
Revised: In addition, the BL algorithm makes crucial use of boundedness and Lipschitz continuity; extending the BL method beyond this standard framework is a highly non-trivial endeavor which formed a big part of this paper’s motivation.

Original:  P roblem S etup and B lanket A ssumptions
Revised:  P roblem S etup and B lanket A ssumptions

Original: We begin in this section by reviewing some basics for min-maxoptimization problems and games.
Revised: We begin in this section by reviewing some basics for min-max problems and games.

Original: Min-max / Saddle-point problems.
Revised: Min-max / Saddle-point problems.

Original: A min-max game is a saddle-point problem of the form
Revised: A min-max game is a saddle-point problem of the form

Original: [Equation]
Revised: [Equation]

Original: Φ [Equation] where Θ , Φ are convex subsets of some ambient real space and L : Θ × Φ → (cid:146) is the problem’s loss function .
Revised: where Θ , Φ are convex subsets of some ambient real space and L : Θ × Φ → (cid:146) is the problem’s loss function .

Original: In the game-theoretic interpretation of (SP), the player controlling θ seeks to minimize
Revised: In the game-theoretic interpretation of (SP), the player controlling θ seeks to minimize

Original: L ( θ, φ ) for any value of the maximization variable φ , while the player controlling φ seeks to maximize
Revised: L ( θ, φ ) for any value of the maximization variable φ , while the player controlling φ seeks to maximize

Original: L ( θ, φ ) for any value of the minimization variable θ .
Revised: L ( θ, φ ) for any value of the minimization variable θ .

Original: Accordingly, solving (SP) consists of ﬁnding a
Revised: Accordingly, solving (SP) consists of ﬁnding a

Original: Nash equilibrium (NE), i.e., an action proﬁle ( θ ∗ , φ ∗ ) ∈
Revised: Nash equilibrium (NE), i.e., an action proﬁle ( θ ∗ , φ ∗ ) ∈

Original: Θ × Φ such that
Revised: Θ × Φ such that

Original: [Equation]
Revised: [Equation]

Original: [Equation] By the minimax theorem of von Neumann [56], Nash equilibria are guaranteed to exist when Θ , Φ are compact and L is convex-concave
Revised: By the minimax theorem of von Neumann [52], Nash equilibria are guaranteed to exist when Θ , Φ are compact and L is convex-concave

Original: (i.e., convex in θ and concave in φ ).
Revised: (i.e., convex in θ and concave in φ ).

Original: Much of our paper is motivated by the question of calculating a Nash equilibrium ( θ ∗ , φ ∗ ) of (SP) in the context of von
Revised: Much of our paper is motivated by the question of calculating a Nash equilibrium ( θ ∗ , φ ∗ ) of (SP) in the context of von

Original: Neumann’s theorem; we expand on this below.
Revised: Neumann’s theorem; we expand on this below.

Original: Games.
Revised: Games.

Original: Going beyond the min-max setting, a continuous game in normal form is deﬁned as follows: First, consider a ﬁnite set of players N = { 1 , . .
Revised: Going beyond the min-max setting, a continuous game in normal form is deﬁned as follows: First, consider a ﬁnite set of players N = { 1 , . .

Original: . , N } , each with their own action space
Revised: . , N } , each with their own action space

Original: K i ∈ (cid:146)
Revised: K i ∈ (cid:146)

Original: d
Revised: d

Original: i (assumed convex).
Revised: i (assumed convex).

Original: During play, each player selects an action x
Revised: During play, each player selects an action x

Original: i from K i with the aim of minimizing a loss determined by the ensemble x (cid:66) ( x i ; x − i ) (cid:66) ( x 1 , . . .
Revised: i from K i with the aim of minimizing a loss determined by the ensemble x (cid:66) ( x i ; x − i ) (cid:66) ( x 1 , . . .

Original: , x N ) of all players’ actions.
Revised: , x N ) of all players’ actions.

Original: In more detail, writing K (cid:66) (cid:81)
Revised: In more detail, writing K (cid:66) (cid:81)

Original: i K i for the game’s total action space, we assume that the loss incurred by the i -th player is (cid:96) i ( x i ; x − i ), where (cid:96) i : K
Revised: i K i for the game’s total action space, we assume that the loss incurred by the i -th player is (cid:96) i ( x i ; x − i ), where (cid:96) i : K

Original: → (cid:146) is the player’s loss function .
Revised: → (cid:146) is the player’s loss function .

Original: In this context, a Nash equilibrium is any action proﬁle x ∗ ∈ K that is unilaterally stable , i.e., (cid:96) i ( x ∗ i ; x ∗− i ) ≤ (cid:96) i ( x i ; x ∗− i ) for all x i
Revised: In this context, a Nash equilibrium is any action proﬁle x ∗ ∈ K that is unilaterally stable , i.e., (cid:96) i ( x ∗ i ; x ∗− i ) ≤ (cid:96) i ( x i ; x ∗− i ) for all x i

Original: ∈ K i and all i ∈ N .
Revised: ∈ K i and all i ∈ N .

Original: (NE)
Revised: [Equation]

Original: If each K i is compact and (cid:96) i is convex in x i , existence of Nash equilibria is guaranteed by the theorem of Debreu [16].
Revised: If each K i is compact and (cid:96) i is convex in x i , existence of Nash equilibria is guaranteed by the theorem of Debreu [13].

Original: Given that a min-max problem can be seen as a two-player zero-sum game with (cid:96) 1 = L , (cid:96) 2 = − L , von Neumann’s theorem may in turn be seen as a special case of Debreu’s; in the sequel, we describe a ﬁrst-order characterization of Nash equilibria that encapsulates both.
Revised: Given that a min-max problem can be seen as a two-player zero-sum game with (cid:96) 1 = L , (cid:96) 2 = − L , von Neumann’s theorem may in turn be seen as a special case of Debreu’s; in the sequel, we describe a ﬁrst-order characterization of Nash equilibria that encapsulates both.

Original: The vector ﬁeld formulation and variational inequalities.

Original: In most cases of interest, the players’ loss functions are individually subdi ﬀ erentiable on a subset X of K with ri K ⊆ X ⊆ K [25, 52].
Revised: In most cases of interest, the players’ loss functions are individually subdi ﬀ erentiable on a subset X of K with ri K ⊆ X ⊆ K [21, 47].

Original: This means that there exists a (possibly discontinuous) vector ﬁeld V i : X → (cid:146) 
Revised: This means that there exists a (possibly discontinuous) vector ﬁeld V i : X → (cid:146) d

Original: d i such that
Revised: i such that

Original: [Equation]
Revised: [Equation]

Original: for all x ∈ X , x (cid:48) ∈ K [25].
Revised: for all x ∈ X , x (cid:48) ∈ K [21].

Original: In the simplest case, if (cid:96) i is di ﬀ erentiable at x , then V i ( x ) can be interpreted as the gradient of (cid:96) i with respect to x i .
Revised: In the simplest case, if (cid:96) i is di ﬀ erentiable at x , then V i ( x ) can be interpreted as the gradient of (cid:96) i with respect to x i .

Original: The raison d’être of the more general deﬁnition (2) is that it allows us to treat non-smooth loss functions that are common in machine learning (such as
Revised: The raison d’être of the more general deﬁnition (2) is that it allows us to treat non-smooth loss functions that are common in machine learning (such as

Original: L 1 -regularized losses).
Revised: L 1 -regularized losses).

Original: We make this distinction precise below:
Revised: We make this distinction precise below:

Original: If there is no continuous vector ﬁeld V i ( x ) satisfying (2), the problem is called non-smooth .
Revised: If there is no continuous vector ﬁeld V i ( x ) satisfying (2), the game is called non-smooth .

Original: If there is a continuous vector ﬁeld V i ( x ) satisfying (2), the problem is called smooth .
Revised: If there is a continuous vector ﬁeld V i ( x ) satisfying (2), the game is called smooth .

Original: Remark.
Revised: Remark.

Original: We stress here that the adjective “smooth” refers to the game itself: for instance, if (cid:96) ( x ) =
Revised: We stress here that the adjective “smooth” refers to the game itself: for instance, if (cid:96) ( x ) =

Original: | x | for x ∈ (cid:146) , the game is not smooth and any V satisfying (2) is discontinuous at 0.
Revised: | x | for x ∈ (cid:146) , the game is not smooth and any V satisfying (2) is discontinuous at 0.

Original: In this regard, the above boils down to whether the (individual) subdi ﬀ erential of each (cid:96) i admits a continuous selection. 
Revised: In this regard, the above boils down to whether the (individual) subdi ﬀ erential of each (cid:96) i admits a continuous selection. Resource allocation

Original: In both cases (smooth and non-smooth), we have the following characterization of  Nash equilibria:
Revised: and equilibrium problems. The notion of a Nash equilibrium captures the unilateral minimization of the players’ individual loss functions.

Original: Proposition 1.
Revised: In many pratical cases of interest, a notion of equilibrium is still relevant, even though it is not necessarily attached to the minimization of individual loss functions. Such problems are known as “equilibrium problems” [15, 26]; to avoid unnecessary generalities, we focus here on a relevant problem that arises in distributed computing architectures (such as GPU clusters and the like). To state the problem, consider a distributed computing grid consisting of N parallel processors that serve demands arriving at a rate of ρ per unit of time (measured e.g., in ﬂop / s). If the maximum processing rate of the i -th node is µ i (without overclocking), and jobs are bu ﬀ ered and served on a ﬁrst-come, ﬁrst-served (FCFS) basis, the mean time required to process a unit demand at the i -th node is given by the Kleinrock M / M / 1 response function

Original:  x ∗ ∈ X is a Nash equilibrium if and only if it satisﬁes the variational inequality
Revised: τ i ( x i ) = 1 / ( µ i − x i ), where x i denotes the node’s load [5].

Original: [Equation]
Revised: Accordingly, the set of feasible loads that can be processed by the grid is X (cid:66) { ( x 1 , . . . , x N ) : 0 ≤ x i < µ i , x 1 + · · · + x N = ρ } . In this context, a load proﬁle x ∗ ∈ X is said to be balanced if no inﬁnitesimal process can be better served by bu ﬀ ering it at a di ﬀ erent node [40]; formally, this amounts to the so-called Wardrop equilibrium condition

Original: where, in obvious notation, we set V ( x  ) =
Revised: τ i ( x ∗ i ) ≤ τ j ( x ∗ j ) for all i , j ∈ N with x ∗ i > 0 .

Original:  (V 1 ( x  ), . .
Revised: (WE) We note here a crucial di ﬀ erence between (WE) and (NE): if we view the grid’s computing nodes as “players”, the constraint (cid:80) i x i = ρ means that there is no allowable unilateral deviation ( x ∗ i ; x ∗− i )

Original: , V N ( x )) . 
Revised: (cid:55)→ ( x i ; x ∗− i ) with x i (cid:44) x ∗ i . As a result, (NE) is meaningless as a requirement for this equilibrium problem. As we discuss below, this resource allocation problem will require the full capacity of our framework. Variational inequalities.

Revised: Importantly, all of the above problems can be restated as a variational inequality of the form

Revised: [Equation]

Revised: In the above, V : X

Revised: → (cid:146) d is the problem’s deﬁning vector ﬁeld and is given as follows:

Revised: In minmax problems and games, V is deﬁned via (2), as described in Sections 2.1 and 2.2; otherwise, in equilibrium problems of the form (WE), the components of V are given by V

Original:  This equivalent formulation is quite common in the literature on min-max / equilibrium problems [17, 18, 30, 35], and it is often referred to as the “vector ﬁeld formulation” [3, 9, 27].
Revised: i = τ i (we leave the details of this veriﬁcation to the reader). This equivalent formulation is quite common in the literature on min-max / equilibrium problems [14, 15, 26, 31], and it is often referred to as the “vector ﬁeld formulation” [3, 8, 23].

Original: Its usefulness lies in that it allows us to abstract away from the underlying game-theoretic complications (multiple indices, individual subdi ﬀ erentials, etc.) and provides a unifying framework for a wide range of problems in machine learning, signal processing, operations research, and many other ﬁelds [18, 53].
Revised: Its usefulness lies in that it allows us to abstract away from the underlying game-theoretic complications (multiple indices, individual subdi ﬀ erentials, etc.) and provides a unifying framework for a wide range of problems in machine learning, signal processing, operations research, and many other ﬁelds [15, 48].

Original: For this reason, our analysis will focus almost exclusively on solving (VI), and we will treat V and
Revised: For this reason, our analysis will focus almost exclusively on solving (VI), and we will treat V and

Original: X ⊆ (cid:146) d , d = (cid:80) i
Revised: X ⊆ (cid:146) d , d = (cid:80) i

Original: d i , as the problem’s primitive data.
Revised: d i , as the problem’s primitive data.

Original: Merit functions and monotonicity.
Revised: Merit functions and monotonicity.

Original: A widely used assumption in the literature on equilibrium problems and variational inequalities is the monotonicity condition
Revised: A widely used assumption in the literature on equilibrium problems and variational inequalities is the monotonicity condition

Original: [Equation]
Revised: [Equation]

Original: In single-player games, monotonicity is equivalent to convexity of the optimizer’s loss function; in min-max games, it is equivalent to L being convex-concave [30]; etc.
Revised: In single-player games, monotonicity is equivalent to convexity of the optimizer’s loss function; in min-max games, it is equivalent to L being convex-concave [26]; etc.

Original: In the absence of monotonicity, approximating an equilibrium is PPAD-hard [14], so (MC) will be our blanket assumption.
Revised: In the absence of monotonicity, approximating an equilibrium is PPAD-hard [11], so we will state most of our results under (MC).

Original: Now, to assess the quality of a candidate solution ˆ x ∈ X , we will employ the restricted merit function
Revised: Now, to assess the quality of a candidate solution ˆ x ∈ X , we will employ the restricted merit function

Original: [Equation]
Revised: [Equation]

Original: where the “ test domain ” C is a nonempty convex subset of X [18, 28, 42].
Revised: where the “ test domain ” C is a nonempty convex subset of X [15, 24, 38].

Original: The motivation for this is provided by the following proposition:
Revised: The motivation for this is provided by the following proposition:

Original: Proposition 2.
Revised: Proposition 1.

Original: Let C be a nonempty convex subset of X .
Revised: Let C be a nonempty convex subset of X .

Original: Then: a ) Gap C (ˆ x ) ≥ 

Revised: Then: a ) Gap C (ˆ x ) ≥

Original: whenever ˆ x ∈ C ; and b ) if Gap C (ˆ x ) = 0 and C contains a neighborhood of ˆ x, then ˆ x is a solution of (VI) .
Revised: whenever ˆ x ∈ C ; and b ) if Gap C (ˆ x ) = 0 and C contains a neighborhood of ˆ x, then ˆ x is a solution of (VI) .

Original: Proposition 2 generalizes an earlier characterization by Nesterov [42] and justiﬁes the use of Gap C ( x ) as a merit function for (VI); to streamline our presentation, we defer the proof to the paper’s supplement.
Revised: Proposition 1 generalizes an earlier characterization by Nesterov [38] and justiﬁes the use of Gap C ( x ) as a merit function for (VI); to streamline our presentation, we defer the proof to the paper’s supplement.

Original: Moreover, to avoid trivialities, we will also assume that the solution set X ∗ of (VI) is nonempty and we will reserve the notation x ∗ for solutions of (VI).
Revised: Moreover, to avoid trivialities, we will also assume that the solution set X ∗ of (VI) is nonempty and we will reserve the notation x ∗ for solutions of (VI).

Original: Together with monotonicity, this will be our only blanket assumption.
Revised: Together with monotonicity, this will be our only blanket assumption.

Original:  T he E xtra -G radient A lgorithm and its L imits
Revised:  T he E xtra -G radient A lgorithm and its L imits

Original: Perhaps the most widely used solution method for games and variational inequalities (VIs) is the extra-gradient (EG) algorithm of Korpelevich [29] and its variants [33, 50, 51].
Revised: Perhaps the most widely used solution method for games and variational inequalities (VIs) is the extra-gradient (EG) algorithm of Korpelevich [25] and its variants [29, 45, 46].

Original: This algorithm has a rich history in optimization, and it has recently attracted considerable interest in the ﬁelds of machine learning and AI; for an overview, see [9, 15, 22, 26, 27, 37, 38] and references therein.
Revised: This algorithm has a rich history in optimization, and it has recently attracted considerable interest in the ﬁelds of machine learning and AI; for an overview, see [8, 12, 18, 22, 23, 33, 34] and references therein.

Original: In its simplest form, the algorithm proceeds recursively as
Revised: In its simplest form, the algorithm proceeds recursively as

Original: [Equation]
Revised: [Equation]

Original: where Π ( x ) =
Revised: where Π ( x ) =

Original: arg min x (cid:48)
Revised: arg min x (cid:48)

Original: ∈ X (cid:107) x (cid:48) − x (cid:107) is the Euclidean projection on X , V t (cid:66) V ( X t ) for t = 1 , 3 / 2 , . .
Revised: ∈ X (cid:107) x (cid:48) − x (cid:107) is the Euclidean projection on X , V t (cid:66) V ( X t ) for t = 1 , 3 / 2 , . .

Original: . , and γ t > 0, is the method’s step-size.
Revised: . , and γ t > 0, is the method’s step-size.

Original: Then, running (EG) for T iterations, the algorithm returns the “ergodic average”
Revised: Then, running (EG) for T iterations, the algorithm returns the “ergodic average”

Original: [Equation]
Revised: [Equation]

Original: In this setting, the main guarantees for (EG) date back to [40] and can be summarized as follows:
Revised: In this setting, the main guarantees for (EG) date back to [36] and can be summarized as follows:

Original: For non-smooth problems (discontinuous V ): Assume V is bounded , i e., there exists some 
Revised: For non-smooth problems (discontinuous V ): Assume V is bounded , i.e., there exists some M > 0 such that

Original: M > 0 such that

Original: [Equation]
Revised: [Equation]

Revised: - ��� - ��� ��� ��� ��� - ��� - ��� ��� ��� ��� θ ϕ ����� - ���������������� ( ����������� ) - ��� - ��� ��� ��� ��� - ��� - ��� ��� ��� ��� θ ϕ ����� - �������������� ( ����������� ) - ��� - ��� ��� ��� ��� - ��� - ��� ��� ��� ��� θ ϕ ������� ( ����������� )

Original: Then, if (EG) is run with a step-size of the form γ t ∝ 1 / √ t , we have
Revised: Then, if (EG) is run with a step-size of the form γ t ∝ 1 / √ t , we have

Original: [Equation]
Revised: [Equation]

Original: For smooth problems (continuous V ):
Revised: For smooth problems (continuous V ):

Original: Assume V is L-Lipschitz continuous [Equation] i[Equation]
Revised: Assume V is L-Lipschitz continuous , i

Original: e.
Revised: e.,

Original: [Equation]

Original: [Equation]

Original: ) (cid:107)

Original: [Equation]
Revised: [Equation]

Original: for all x

Original: [Equation] x [Equation] Then, if (EG) is run with a constant step-size γ < 1 / L , we have
Revised: Then, if (EG) is run with a constant step-size γ < 1 / L , we have

Original: [Equation]
Revised: [Equation]

Original: Remark.
Revised: Remark.

Original: In the above, (cid:107)·(cid:107) is tacitly assumed to be the standard Euclidean norm.
Revised: In the above, (cid:107)·(cid:107) is tacitly assumed to be the standard Euclidean norm.

Original: Non-Euclidean considerations will play a crucial role in the sequel, but they are not necessary for the moment.
Revised: Non-Euclidean considerations will play a crucial role in the sequel, but they are not necessary for the moment.

Original: Importantly, the distinction between smooth and non-smooth problems cannot be lifted: the bounds (5) and (6) are tight in their respective problem classes and they cannot be improved without further assumptions [39, 44].
Revised: Importantly, the distinction between smooth and non-smooth problems cannot be lifted: the bounds (5) and (6) are tight in their respective problem classes and they cannot be improved without further assumptions [35, 41].

Original: Moreover, we should also note the following:
Revised: Moreover, we should also note the following:

Original: The algorithm changes drastically from the non-smooth to the smooth case: non-smoothness requiresa vanishing γ t  , but such a step-size cannot achieve an accelerated O (1 / T ) rate.
Revised: The algorithm changes drastically from the non-smooth to the smooth case: non-smoothness requires γ t ∝ 1 / √ t , but such a step-size cannot achieve a fast O (1 / T ) rate.

Original: If (EG) is run with a constant step-size, L must be known in advance; otherwise, running (EG) with an ill-adapted step-size ( γ > 1 / L ) could lead to non-convergence.
Revised: If (EG) is run with a constant step-size, L must be known in advance; otherwise, running (EG) with an ill-adapted step-size ( γ > 1 / L ) could lead to non-convergence.

Original: We illustrate thisconvergence failure of (EG) in Fig.
Revised: We illustrate this failure of (EG) in Fig.

Original: As we discussed in the introduction, our aim in the sequel will be to provide a single, universal algorithm that simultaneously achieves the following: a ) an  O (cid:0) 1 / √ T (cid:1) convergence rate in non-smooth problems and O (1 / T ) in smooth ones; b ) convergence in problems where the boundedness / Lipschitz continuity conditions (BD) / (LC) no longer hold; and c ) achieves all this without prior knowledge ofany of the problem’sparameters.
Revised: As we discussed in the introduction, our aim in the sequel will be to provide a single, adaptive algorithm that simultaneously achieves the following: a ) an order-optimal O (cid:0) 1 / √ T (cid:1) convergence rate in non-smooth problems and O (1 / T ) in smooth ones; b ) convergence in problems where the boundedness / Lipschitz continuity conditions (BD) / (LC) no longer hold; and c ) achieves all this without prior knowledge of the problem’s

Original: - ��� - ��� ��� ��� ��� - ��� - ��� ��� ��� ��� θ ϕ ����� - ���������������� ( ����������� ) - ��� - ��� ��� ��� ��� - ��� - ��� ��� ��� ��� θ ϕ ����� - �������������� ( ����������� ) - ��� - ��� ��� ��� ��� - ��� - ��� ��� ��� ��� θ ϕ ������� ( ����������� )
Revised: parameters

Original:  R ate I nterpolation : the E uclidean C ase
Revised:  R ate I nterpolation : the E uclidean C ase

Original: As a prelude to our main result, we provide in this section an adaptive version of (EG) that achieves the “best of both worlds” in the Euclidean setting of Section 3, i.e., an O (cid:0) 1 / √ T (cid:1) convergence rate in problems satisfying (BD), and an O (1 / T ) rate in problems satisfying (LC).
Revised: As a prelude to our main result, we provide in this section an adaptive version of (EG) that achieves the “best of both worlds” in the Euclidean setting of Section 3, i.e., an O (cid:0) 1 / √ T (cid:1) convergence rate in problems satisfying (BD), and an O (1 / T ) rate in problems satisfying (LC).

Original: Our starting point is the observation that, if the sequence X t produced by (EG) converges to a solution of (VI), the di ﬀ erence
Revised: Our starting point is the observation that, if the sequence X t produced by (EG) converges to a solution of (VI), the di ﬀ erence

Original: [Equation]
Revised: [Equation]

Original: (7) must itself become vanishingly small if V is (Lipschitz) continuous.
Revised: must itself become vanishingly small if V is (Lipschitz) continuous.

Original: On the contrary, if V is discontinuous , this di ﬀ erence may remain bounded away from zero (consider for example the L 1 loss (cid:96) ( x ) =
Revised: On the contrary, if V is discontinuous , this di ﬀ erence may remain bounded away from zero (consider for example the L 1 loss (cid:96) ( x ) =

Original: | x | near 0).
Revised: | x | near 0).

Original: Based on this observation, we consider the adaptive step-size policy:
Revised: Based on this observation, we consider the adaptive step-size policy:

Original: [Equation]
Revised: [Equation]

Original:  The intuition behind (8) is as follows: If V is not smooth and lim inf t →∞ γ t > 0, then γ t will vanish at a Θ (cid:0) 1 / √ t (cid:1) rate, which is the optimal step-size schedule for problems satisfying (BD) but not (LC).
Revised: [Equation] The intuition behind (8) is as follows: If V is not smooth and lim inf t →∞ γ t > 0, then γ t will vanish at a Θ (cid:0) 1 / √ t (cid:1) rate, which is the optimal step-size schedule for problems satisfying (BD) but not (LC).

Original: Instead, if V satisﬁes (LC) and X t converges to a solution x ∗ of (VI), it is plausible to expect that the inﬁnite series (cid:80) t δ 2 t is summable, in which case the step-size γ t will not vanish as t → ∞ .
Revised: Instead, if V satisﬁes (LC) and X t converges to a solution x ∗ of (VI), it is plausible to expect that the inﬁnite series (cid:80) t δ 2 t is summable, in which case the step-size γ t will not vanish as t → ∞ .

Original: Furthermore, since δ t is deﬁned in terms of successive gradient di ﬀ erences, it automatically exploits the variation of the gradient data observed up to time t , so it can be expected to adjust to the “local”
Revised: Furthermore, since δ t is deﬁned in terms of successive gradient di ﬀ erences, it automatically exploits the variation of the gradient data observed up to time t , so it can be expected to adjust to the “local”

Original: Lipschitz constant of V around a solution x ∗ of (VI).
Revised: Lipschitz constant of V around a solution x ∗ of (VI).

Original: Our step-size policy and motivation are similar in spirit to the “predictable sequence” approach of [51].
Revised: Our step-size policy and motivation are similar in spirit to the “predictable sequence” approach of [46].

Original: However, making our reasoning precise (especially the summability of (cid:80) t δ 2 t in the smooth case) involves considerable conceptual and technical di ﬃ culties that we present in detail in the supplement.
Revised: However, making our reasoning precise (especially the summability of (cid:80) t δ 2 t in the smooth case) involves considerable conceptual and technical di ﬃ culties that we present in detail in the supplement.

Original: For now, we only state (without proof) our main result for problems satisfying (BD) or (LC).
Revised: For now, we only state (without proof) our main result for problems satisfying (BD) or (LC).

Original: Theorem 1.
Revised: Theorem 1.

Original: Let C be a compact neighborhood of a solution of (VI)  and let H = sup x ∈ C (cid:107) X 1 − x (cid:107) 
Revised: Suppose V satisﬁes (MC) , let C be a compact neighborhood of a solution of (VI) , and let H = sup x ∈ C (cid:107) X 1 − x (cid:107) 2 .

Original: Then, if (EG) is run with the adaptive step-size policy (8) , we have: 
Revised: If (EG) is run with the adaptive step-size policy (8) , we have: a ) [Equation]

Original: a ) If V satisﬁes (BD)
Revised: V [Equation] BD

Original: [Equation]
Revised: [Equation]

Original: b ) If V satisﬁes (LC
Revised: b ) If V satisﬁes (LC)

Original: [Equation]
Revised: [Equation]

Original: We stress here that Theorem 1 holds even for problems with an unbounded domain (diam( X ) = ∞ ).
Revised: Theorem 1 (which is proved in the sequel as a special case of Theorem 2) should be compared to the corresponding results of Bach & Levy [2]. In the non-smooth case, [2] provides a bound of the form ˜ O ( α MD / √ T ) with D 2 = 12 max x ∈ X (cid:107) x (cid:107) 2 − 12 min x ∈ X (cid:107) x (cid:107) 2 (recall that [2] only treats problems with a bounded domain), and α = max { M / M 0 , M 0 / M } where M 0 is an initial estimate of M . The worst-case value of α is O

Original: In fact, even when X is bounded, the adaptive policy (8) does not require any prior knowledge of its diameter (which might be di ﬃ cult to estimate).
Revised: ( M ) when good estimates are not readily available; in this regard, (9a) essentially replaces the O ( D ) constant of Bach & Levy [2] by O ( M ).

Original: These limitations are both present in  the work of 
Revised: Since D = ∞ in problems with an unbounded domain, Theorem 1 provides a signiﬁcant improvement in this regard. In terms of L , the smooth guarantee of Bach & Levy [2] is ˜ O ( α 2 LD 2 / T ), so the multiplicative constant in the bound also becomes inﬁnite in problems with an unbounded domain. In our case, Dis replaced by H (which is also ﬁnite) times an addiitional multiplicative constant which is increasing in M and L (but is otherwise asymptotic, so it is not included in the statement of Theorem 1).

Original:  Bach & Levy [2], so Theorem 1 provides a signiﬁcant improvement in this regard. Going beyond this improvement, in the next sections we drop even the Euclidean regularity requirements (BD) / (LC), and we provide a corresponding rate interpolation result that does not require either condition.
Revised: This removes an additional limitation in the results of Bach & Levy [2]; going beyond this improvement, in the next sections we drop even the Euclidean regularity requirements (BD) / (LC), and we provide a corresponding rate interpolation result that does not require either condition.

Original:  F insler R egularity
Revised:  F insler R egularity

Original: To motivate our analysis outside the setting of (BD) / (LC), consider the generalized Kullback–Leibler (KL) loss
Revised: To motivate our analysis outside the setting of (BD) / (LC), consider the vector ﬁeld

Original: [Equation]
Revised: [Equation]

Original: where p ∈ [0 , 1] d is a ﬁxed target point (typically a ground truth proﬁle). This loss function is widely used in Poisson inverse problems [6], logistic regression and log-likelihood models [20], experiment design [46], and many other ﬁelds of [-statistical learning. The KL loss is well-deﬁned for all x ∈ [0 , 1] d such that x i = 0 when p i = 
 and its gradient is V ( x ) =-]
Revised: (10) which corresponds to the distributed computing problem of Section 2.3 plus a regularization term designed to limit the activation of [-statistical learning. The KL loss is well-deﬁned for all x ∈ [0 , 1] d such that x i = 0 when p i = 
 and its gradient is V ( x ) =-]computing nodes at low loads.

Original: (1 − p i / x i ) i = 1 ,..., d for all x ∈ X = (0 , 1] d . Consequently, (BD) and (LC) both fail for the KL loss because (cid:107) V ( x ) (cid:107) → ∞ whenever x i → 0 + (note here that X is convex but neither open nor closed).
Revised: Clearly, we have (cid:107) V ( x ) (cid:107) → ∞ whenever x i → 0 + , so (BD) and (LC) both fail (the latter even if λ = 0).

Original: On the other hand, if we consider the “local” norm (cid:107) v (cid:107) x , ∗ = (cid:80) di
Revised: On the other hand, if we consider the “local” norm (cid:107) v (cid:107) x , ∗ = (cid:80) di

Original: = 1  x i  | v i | , we have (cid:107) V ( x ) (cid:107) x , ∗ ≤2 d  , so V is bounded relative to (cid:107)·(cid:107) x , ∗ .
Revised: = 1 ( µ i − x i ) | v i | , we have (cid:107) V ( x ) (cid:107) x , ∗ ≤ d + λ (cid:80) di = 1 µ i , so V is bounded relative to (cid:107)·(cid:107) x , ∗ .

Original: This observation motivates the use of a local – as opposed to global – norm on X ; formally, this amounts to equipping X with a Finsler metric [4, 11].
Revised: This observation motivates the use of a local – as opposed to global – norm, which we deﬁne formally as follows:

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: A Finsler metric on a convex subset X of (cid:146) d is a continuous function F : X × (cid:146) d
Revised: A Finsler metric on a convex subset X of (cid:146) d is a continuous function F : X × (cid:146) d

Original: → (cid:146) + which satisﬁes the following properties for all x ∈ X and all z , z (cid:48) ∈ (cid:146) d 
Revised: → (cid:146) + which satisﬁes the following properties for all x ∈ X and all z , z (cid:48) ∈ (cid:146) d :

Original: Subadditivity: F ( x ; z + z (cid:48) ) ≤ F ( x ; z ) + F ( x ; z (cid:48) ).
Revised: Subadditivity: F ( x ; z + z (cid:48) ) ≤ F ( x ; z ) + F ( x ; z (cid:48) ).

Original: Absolute homogeneity: F ( x ; λ z ) = | λ | F ( x ; z ) for all λ ∈ (cid:146) .
Revised: Absolute homogeneity: F ( x ; λ z ) = | λ | F ( x ; z ) for all λ ∈ (cid:146) .

Original: Positive-deﬁniteness: F ( x ; z ) ≥ 0 with equality
Revised: Positive-deﬁniteness: F ( x ; z ) ≥ 0 with equality

Original: if and only if z = 0.
Revised: if and only if z = 0.

Original: Given a Finsler metric on X , the induced primal / dual local norms on X are respectively deﬁned as (cid:107) z (cid:107) x
Revised: Given a Finsler metric on X , the induced primal / dual local norms on X are respectively deﬁned as (cid:107) z (cid:107) x

Original: = F ( x ; z ) and (cid:107) v (cid:107) x , ∗ = max {(cid:104) v, z (cid:105) :
Revised: = F ( x ; z ) and (cid:107) v (cid:107) x , ∗ = max {(cid:104) v, z (cid:105) :

Original: F ( x ; z ) = 1 } (11) for all x ∈ X and all z , v ∈ (cid:146) d .
Revised: F ( x ; z ) = 1 } (11) for all x ∈ X and all z , v ∈ (cid:146) d .

Original: A Finsler metric on X is further called regular when (cid:107) v (cid:107) x (cid:48) , ∗ / (cid:107) v (cid:107) x , ∗ = 1 + O ( (cid:107) x (cid:48)
Revised: A Finsler metric on X is further called regular when (cid:107) v (cid:107) x (cid:48) , ∗ / (cid:107) v (cid:107) x , ∗ = 1 + O ( (cid:107) x (cid:48)

Original: − x (cid:107) x ) for all x , x (cid:48) ∈ X , v ∈ (cid:146) d .
Revised: − x (cid:107) x ) for all x , x (cid:48) ∈ X , v ∈ (cid:146) d .

Original: Finally, for simplicity, we will also assume in the sequel that (cid:107)·(cid:107) x ≥ µ (cid:107)·(cid:107) for some µ > 0 and all x ∈ X (this last assumption is for convenience only, as the norm could be redeﬁned to (cid:107)·(cid:107)
Revised: Finally, for simplicity, we will also assume in the sequel that (cid:107)·(cid:107) x ≥ ν (cid:107)·(cid:107) for some ν > 0 and all x ∈ X (this last assumption is for convenience only, as the norm could be redeﬁned to (cid:107)·(cid:107)

Original: x ← (cid:107)·(cid:107) x
Revised: x ← (cid:107)·(cid:107) x

Original: + µ (cid:107)·(cid:107) without a ﬀ ecting our results).
Revised: + ν (cid:107)·(cid:107) without a ﬀ ecting our theoretical analysis).

Original: When X is equipped with a regular Finsler metric as above, we will say that it is a Finsler space .
Revised: When X is equipped with a regular Finsler metric as above, we will say that it is a Finsler space .

Original: (cid:73) Example 5.1.
Revised: Example 5.1.

Original: Let F ( x ; z )
Revised: Let F ( x ; z )

Original: = (cid:107) z (cid:107) where (cid:107)·(cid:107) denotes the reference norm of X = (cid:146) d .
Revised: = (cid:107) z (cid:107) where (cid:107)·(cid:107) denotes the reference norm of X = (cid:146) d .

Original: Then the axioms of a Finsler space are satisﬁed trivially.
Revised: Then the properties of Deﬁnition 1 are satisﬁed trivially.

Original: (cid:73) Example 5.2.
Revised: [Equation] Example 5.2.

Original: A more interesting example of a Finsler structure on X = (0 , 1] d is provided by the metric (cid:107) z (cid:107) x = max i |
Revised: For a more interesting example of a Finsler structure, consider the set X = (0 , 1] d and the metric (cid:107) z (cid:107) x = max i |

Original: z i | / x i , z ∈ (cid:146) d , x ∈ X .
Revised: z i | / x i , z ∈ (cid:146) d , x ∈ X .

Original: In this case (cid:107) v (cid:107) x , ∗ = (cid:80) di
Revised: In this case (cid:107) v (cid:107) x , ∗ = (cid:80) di

Original: = 1 x i | v i | for all v ∈ (cid:146) d , and the only axiom left to verify is that of regularity.
Revised: = 1 x i | v i | for all v ∈ (cid:146) d , and the only property of Deﬁnition 1 that remains to be proved is that of regularity.

Original: To that end, we have
Revised: To that end, we have

Original: [Equation]
Revised: [Equation]

Original: Hence, by dividing by (cid:107) v (cid:107) x , ∗ , we readily get (cid:107) v (cid:107) x (cid:48) , ∗ / (cid:107) v [Equation] x [Equation]
Revised: Hence, by dividing by (cid:107) v (cid:107) x , ∗ , we readily get (cid:107) v (cid:107) x (cid:48) , ∗ / (cid:107) v (cid:107) x , ∗ ≤ 1 + (cid:107) x − x (cid:48) (cid:107) x i.e., (cid:107)·(cid:107) x is regular in the sense of Deﬁnition 1.

Original: [Equation]

Original: (cid:107) x

Original: [Equation]

Original: x

Original: [Equation]

Original: (cid:107) x

Original: i [Equation] e., (cid:107)·(cid:107) x is regular in the sense of Deﬁnition 1. As we discuss in the sequel, this metric plays an important role for  problemswith loss functions of thegeneralized KL form (10).
Revised: As we discuss in the sequel, this metric plays an important role for distributed computing problems of the form presented in Section 2.3.

Original: [Equation]
Revised: [Equation]

Original: With all this in hand, we will say that a vector ﬁeld V : X → (cid:146) d is
Revised: With all this in hand, we will say that a vector ﬁeld V : X → (cid:146) d is

Original: Metrically bounded if there exists some M > [-
-] such that
Revised: Metrically bounded if there exists some M > [-
-]0 such that

Original: [Equation]
Revised: [Equation]

Original: Metrically smooth if there exists some L >  such that
Revised: Metrically smooth if there exists some L > 0 such that

Original: [Equation]
Revised: [Equation]

Original: The notion of metric boundedness / smoothness extends that of ordinary boundedness / Lipschitz continuity to a Finsler context.
Revised: The notion of metric boundedness / smoothness extends that of ordinary boundedness / Lipschitz continuity to a Finsler context; note also that, even though neither side of (MS) is unilaterally symmetric under the change x ↔ x (cid:48) , the condition (MS) as a whole is .

Original: Our next example, based on the KL loss function (10), shows that this extension is proper , i.e., (BD) / (LC) may both fail while (MB) / (MS) both hold:(cid:73) Example 5.3. 
Revised: Our next example shows that this extension is proper , i.e., (BD) / (LC) may both fail while (MB) / (MS) both hold: Example 5.3. Consider the change of variables x i (cid:32) 1 − x i /µ i in the resource allocation problem of Section 2.3.

Original: Let V  ( x ) =  (1 / x  1 , . . . , 1 / x d ) for  x ∈ X = (0 , 1] d .
Revised: Then, writing V i ( x ) = − (1 / x i ) − λ 1 { x i < 1 } for the transformed ﬁeld (10) under this change of variables, we readily get V i ( x ) →

Original: Obviously, since the components of V ( x ) diverge to inﬁnity as x i → 0, both (BD) and (LC) fail to hold for any global norm on (cid:146) d .
Revised: −∞ as x i → 0 + ; as a result, both (BD) and (LC) fail to hold for any global norm on (cid:146) d .

Original: On the other hand, under the local norm (cid:107) z (cid:107) x = max i | z | i / x i of Example 5.2, we have:
Revised: Instead, under the local norm (cid:107) z (cid:107) x = max i | z | i / x i , we have:

Original:  V is metrically bounded with M = d : (cid:107) V ( x ) (cid:107) x , ∗ = (cid:80) di
Revised: For all λ ≥ 0, V satisﬁes (MB) with M = d (1 + λ ): (cid:107) V ( x ) (cid:107) x , ∗ ≤ (cid:80) di

Original: = 1 x i · (1 / x i  )=
Revised: = 1 x i · (1 / x i + λ )

Original: d . 2. V is metrically smooth with L =d : indeed, for all x , x (cid:48) ∈ X , we have (cid:107) V ( x (cid:48) ) − V ( x ) (cid:107) x , ∗ = (cid:80) di [Equation]
Revised: =

Original: 1 x i (cid:12)(cid:12)(cid:12)(cid:12) [Equation]
Revised: d (1 + λ ).

Original: x (cid:48) i − 1 x i (cid:12)(cid:12)(cid:12)(cid:12) = (cid:80) di = 1 | x 
Revised: For λ = 0, V satisﬁes (MS) with L = d : indeed, for all x , x (cid:48) ∈ X , we have

Original: [Equation]
Revised: [Equation]

Original:  T he U ni P rox A lgorithm and its G uarantees
Revised:  T he A da P rox A lgorithm and its G uarantees

Original: The method.
Revised: The method.

Original: We are now in a position to deﬁne a family of algorithms that is capable of interpolating between the optimal smooth / non-smooth convergence rates for solving (VI) without requiring either (BD) or (LC).
Revised: We are now in a position to deﬁne a family of algorithms that is capable of interpolating between the optimal smooth / non-smooth convergence rates for solving (VI) without requiring either (BD) or (LC).

Original: To do so, the key steps in our approach will be to ( i ) equip X with a suitable
Revised: To do so, the key steps in our approach will be to ( i ) equip X with a suitable

Original: Finsler structure (as in Section 5); and ( ii ) replace the Euclidean projection in (EG) with a suitable “Bregman proximal” step that is compatible with the chosen Finsler structure on X .
Revised: Finsler structure (as in Section 5); and ( ii ) replace the Euclidean projection in (EG) with a suitable “Bregman proximal” step that is compatible with the chosen Finsler structure on X .

Original: We begin with the latter (assuming that X is equipped with an arbitrary Finsler structure):
Revised: We begin with the latter (assuming that X is equipped with an arbitrary Finsler structure):

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: We say that h : (cid:146) d → (cid:146) ∪ {∞} is a Bregman-Finsler function on X if:
Revised: We say that h : (cid:146) d → (cid:146) ∪ {∞} is a Bregman-Finsler function on X if:

Original: h is convex, lower semi-continuous (l.s.c.), and cl(dom h ) =
Revised: h is convex, lower semi-continuous (l.s.c.), and cl(dom h ) =

Original: cl( X ).
Revised: cl( X ).

Original: The subdi ﬀ erential of h admits a continuous selection ∇ h ( x ) ∈
Revised: The subdi ﬀ erential of h admits a continuous selection ∇ h ( x ) ∈

Original: ∂ h ( x ) for all x ∈ dom ∂ h .3.
Revised: ∂ h ( x ) for all x ∈ dom ∂ h .

Original: h is strongly convex , i e., there exists some K > 0 such that 
Revised: h is strongly convex , i.e., there exists some K > 0 such that [Equation]

Original: [Equation]
Revised: [Equation]

Original: for all x ∈ dom ∂ h and all x (cid:48) ∈ dom h .
Revised: for all x ∈ dom ∂ h and all x (cid:48) ∈ dom h .

Original: The Bregman divergence induced by h is deﬁned for all x ∈ dom ∂ h , x (cid:

Revised: The Bregman divergence induced by h is deﬁned for all x ∈ dom ∂ h , x (cid:

Original: ) ∈ dom h as
Revised: ) ∈ dom h as

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (15) and the associated prox-mapping is deﬁned for all x ∈ dom ∂ h and y ∈ (cid:146) d as
Revised: and the associated prox-mapping is deﬁned for all x ∈ dom ∂ h and y ∈ (cid:146) d as

Original: [Equation]
Revised: [Equation]

Original: Deﬁnition 2 is fairly technical, so some clariﬁcations are in order.
Revised: Deﬁnition 2 is fairly technical, so some clariﬁcations are in order.

Original: First, to connect this deﬁnition with the Euclidean setup of Section 4, the prox-mapping (16) should be seen as the Bregman equivalent of a Euclidean projection step, i.e., Π ( x + y ) (cid:33)
Revised: First, to connect this deﬁnition with the Euclidean setup of Section 4, the prox-mapping (16) should be seen as the Bregman equivalent of a Euclidean projection step, i.e., Π ( x + y ) (cid:33)

Original: P x ( y ).
Revised: P x ( y ).

Original: Second, the main di ﬀ erence between Deﬁnitionand other deﬁnitions of Bregman proximal mappings in the literature [5, 7, 8, 10, 28, 41, 42, 54] is that h is assumed strongly convex relative to a local norm – not a global norm.
Revised: Second, a key di ﬀ erence between Deﬁnitionand other deﬁnitions of Bregman proximal mappings in the literature [4, 6, 7, 9, 24, 37, 38, 49] is that h is assumed strongly convex relative to a local norm – not a global norm.

Original: This “locality” will play a crucial role in allowing the proposed methods to adapt to the geometry of the problem.
Revised: This “locality” will play a crucial role in allowing the proposed methods to adapt to the geometry of the problem.

Original: For concreteness, we provide below an example that expands further on Examples 5.2 and 5.3:(cid:73) Example 6.1.
Revised: For concreteness, we provide below an example that expands further on Examples 5.2 and 5.3: Example 6.1.

Original: Consider the local norm (cid:107) z (cid:107) x = max i |
Revised: Consider the local norm (cid:107) z (cid:107) x = max i |

Original: z i | / x i on X = (0 , 1] d and let h ( x ) =
Revised: z i | / x i on X = (0 , 1] d and let h ( x ) =

Original: (cid:80) di
Revised: (cid:80) di

Original: = 1 1 / x i on (0 , 1] d .
Revised: = 1 1 / x i on (0 , 1] d .

Original: We then have
Revised: We then have

Original: [Equation]
Revised: [Equation]

Original: (17) i.e., h is 1-strongly convex relative to (cid:107)·(cid:107) x on X .
Revised: i.e., h is 1-strongly convex relative to (cid:107)·(cid:107) x on X .

Original: [Equation]
Revised: [Equation]

Original: With all this is in place, the extra-gradient method can be adapted to our present setting as follows:
Revised: With all this is in place, the extra-gradient method can be adapted to our current setting as follows:

Original: [Equation]
Revised: [Equation]

Original: with V t =
Revised: with V t =

Original: V ( X t ), t = 1 , 3 / 2 , . .
Revised: V ( X t ), t = 1 , 3 / 2 , . .

Original: , as in Section 3.
Revised: , as in Section 3.

Original: In words, this method builds on the template of (EG) by ( i ) replacing the Euclidean projection with a mirror step; ( ii ) replacing the global norm in (8) with a dual Finsler norm evaluated at the algorithm’s leading state X t + 1 / 2 .
Revised: In words, this method builds on the template of (EG) by ( i ) replacing the Euclidean projection with a mirror step; ( ii ) replacing the global norm in (8) with a dual Finsler norm evaluated at the algorithm’s leading state X t + 1 / 2 .

Original: The ﬁrst of these two steps isalso the deﬁning ingredient of the mirror-prox (MP) algorithm of Nemirovski [40]; the name “UniProx” has beeen chosen precisely because the proposed method can be seen as a mirror-prox method with universal rate interpolation and convergence guarantees. We expand on this below. Trajectory convergence.
Revised: The ﬁrst of these two steps is the main ingredient of the mirror-prox (MP) algorithm of Nemirovski [36]; the name “AdaProx” has beeen chosen precisely because the proposed method can be seen as a mirror-prox method that adapts between the smooth and non-smooth regimes.

Original: We begin our analysis with a trajectory convergence result that governs the actual iterates of the method:

Original: Theorem 2.

Original: Suppose that V satisﬁes (MB) or (MS) .

Original: If Gap C ( x ∗ ) < 0 whenever C is relatively closed in X and disjoint from X ∗ , then the iterates X t of UniProx converge to a solution of (VI) .

Original: The importance of this result is twofold: First, it ratiﬁes our initial bet that X t converges, so the di ﬀ erence sequence δ t = (cid:107) V t + 1 / 2 − V t (cid:107) 2 X t + 1 / 2 , ∗ must vanish when (MS) holds (we provide an explicit

Original: [Figure]

Original: ���� � ���� ������� � �� ��� ���� �� � �� - � ����� ����� ����� � �������� �� �����������

Original: ○

Original: ○

Original: ○ ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○

Original: ○○ □ □ □ □ □□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□ □ □□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□□ △ △ △ △ △△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△△ ○ ����� - �������� ( ������������� ) ����� - �������� ( ������������� ) □ ���� - ���� ( ������������� ) ���� - ���� ( ������������� ) △ ������� ( ������������� ) ������� ( ������������� ) � �� ��� ���� �� � ����� ���� ��� �� �������� �� ����������� statement of this below).

Original: Second, in many practical applications, it is more common to harvest the “last iterate” X t of an algorithm rather than ¯ X T ; Theorem 2 justiﬁes this implementation choice.

Original: The proof of Theorem 2 relies on non-standard arguments, so we provide a sketch below while relegating details to the supplement.

Original: The ﬁrst step is to show that X t visits any neighborhood of a solution point x ∗ ∈

Original: X ∗ inﬁnitely often (this is where the assumption on Gap C ( x ∗ ) is used; note however that this assumption is not used in the sequel).

Original: The second is to use this trapping property in conjunction with a suitable “energy inequality” to establish convergence via the use of a quasi-Fejér technique as in [13]; this part is detailed in a separate appendix.

Original: Convergence rate analysis.
Revised: Convergence speed.

Original: We are ﬁnally in a position to state our main result for UniProx:
Revised: With all this in hand, our main result for AdaProx can be stated as follows:

Original: Theorem 3.
Revised: Theorem 2.

Original: Let C be a compact neighborhood of a solution of (VI)  and set H = sup x ∈ C D ( x , X 1 )
Revised: Suppose V satisﬁes (MC) , let C be a compact neighborhood of a solution of (VI) , and set H = sup x ∈ C D ( x , X 1 )

Original: Then, the UniProx algorithm enjoys theuniversal guarantees:
Revised: Then, the AdaProx algorithm enjoys the guarantees:

Original: a ) If V satisﬁes (MB):
Revised: a ) If V satisﬁes (MB)

Original: [Equation]
Revised: [Equation]

Original: b ) If V satisﬁes (MS) :
Revised: b ) If V satisﬁes (MS) :

Original: [Equation]
Revised: [Equation]

Revised: For the constants that appear in Eq.

Revised: (18), we refer the reader to the discussion following Theorem(of course, since Theorem 1 is a special case of Theorem 2, it is not surprising that the same remarks apply).

Original: The proof of Theorem 3 is quite intricate, so we defer it to the paper’s supplement.
Revised: As for the proof of Theorem 2, it is quite intricate, so we defer it to the paper’s supplement.

Original: The key elementthroughout is the determination of the asymptotic behavior of the adaptive step-size policy γ t in the non-smooth and smooth regimes, i.e., under (MB) and (MS) respectively.
Revised: We only mention here that its key element is the determination of the asymptotic behavior of the adaptive step-size policy γ t in the non-smooth and smooth regimes, i.e., under (MB) and (MS) respectively.

Original: At a very high level, (MB) guarantees that the di ﬀ erence sequence δ t is bounded, which implies in turn that (cid:80) Tt = 1 γ t
Revised: At a very high level, (MB) guarantees that the di ﬀ erence sequence δ t is bounded, which implies in turn that (cid:80) Tt = 1 γ t

Original: = Ω ( √ T ) and eventually yields the bound (18a) for the algorithm’s ergodic average ¯ X T .
Revised: = Ω ( √ T ) and eventually yields the bound (18a) for the algorithm’s ergodic average ¯ X T .

Original: On the other hand, if (MS) kicks in, we can show the much ﬁner result that δ t is actually square summable:
Revised: On the other hand, if (MS) kicks in, we have the following ﬁner result:

Original: Lemma 1.
Revised: Lemma 1.

Original: Assume V satisﬁes (MS) .
Revised: Assume V satisﬁes (MS) .

Original: Then, a ) γ t decreases monotonically to a strictly positive limit γ ∞ =
Revised: Then, a ) γ t decreases monotonically to a strictly positive limit γ ∞ =

Original: lim t →∞ γ t > 0 ; and b ) the sequence δ t is square summable; in particular, (cid:80) ∞ t = 1 δ 2 t = 1 /γ 2 ∞ − 1 .
Revised: lim t →∞ γ t > 0 ; and b ) the sequence δ t is square summable: in particular, (cid:80) ∞ t = 1 δ 2 t = 1 /γ 2 ∞ − 1 .

Original: By means of this lemma (which we prove in the paper’s supplement), it follows that (cid:80) Tt = 1 γ t ≥ γ ∞ T = Ω ( T ).
Revised: By means of this lemma (which we prove in the paper’s supplement), it follows that (cid:80) Tt = 1 γ t ≥ γ ∞ T = Ω ( T ).

Original: Because the algorithm’s rate of convergence is controlled by this quantity, it ultimately follows that UniProx enjoys anaccelerated O (1 / T ) rate of convergence under (MS).
Revised: Because the algorithm’s rate of convergence is controlled by this quantity, it ultimately follows that AdaProx enjoys an O (1 / T ) rate of convergence under (MS).

Original: However, the details of the ensuing calculations are quite complicated, so we defer them to the supplement.
Revised: However, the details of the ensuing calculations are quite complicated, so we defer them to the supplement.

Revised: Trajectory convergence.

Revised: In complement to Theorem 2, we also provide a trajectory convergence result that governs the actual iterates of the AdaProx algorithm:

Original: Conclusions.
Revised: Theorem 3. Suppose that (cid:104) V ( x ) , x − x ∗ (cid:105) < 0 whenever x ∗ is a solution of (VI) and x is not. If, in addition, V satisﬁes (MB) or (MS) , the iterates X t of AdaProx converge to a solution of (VI) . [Figure]

Revised: The importance of this result is that, in many practical applications (especially in non-monotone problems), it is more common to harvest the “last iterate” of the method ( X t ) rather than its ergodic average ( ¯ X T ); as such, Theorem 3 provides a certain justiﬁcation for this design choice.

Revised: The proof of Theorem 3 relies on non-standard arguments, so we relegate it to the supplement.

Revised: Structurally, the ﬁrst step is to show that X t visits any neighborhood of a solution point x ∗ ∈

Revised: X ∗ inﬁnitely often (this is where the coherence assumption (cid:104) V ( x ) , x − x ∗ (cid:105) is used).

Revised: The second is to use this trapping property in conjunction with a suitable “energy inequality” to establish convergence via the use of a quasi-Fejér technique as in [10]; this part is detailed in a separate appendix.

Original: The appealing properties of UniProx are illustrated in  a series of numerical experiments in Fig.
Revised: We conclude in this section with a numerical illustration of the convergence properties of AdaProx in two di ﬀ erent settings: a ) bilinear min-max games; and b ) a simple Wasserstein GAN in the spirit of Daskalakis et al. [12] with the aim of learning an unknown covariance matrix.

Revised: Bilinear min-max games.

Revised: For our ﬁrst set of experiments, we consider a min-max game of the form of the form L ( θ, φ )

Original: Together with our theoretical analysis, Fig. 2 conﬁrms that UniProx can provide a fruitful template for adaptive min-max optimization methods that attain optimal rates in smooth / nonsmooth problems, without requiring any tuning by the optimizer.
Revised: = ( θ − θ ∗ ) (cid:62) A ( φ − φ ∗ ) with θ, φ ∈ (cid:146) 100 and A ∈ (cid:146) 100 × (cid:146) 100 . To test the convergence of AdaProx beyond the “full gradient” framework, we ran the algorithm with stochastic gradient signals of the form V t = V ( X t ) + U t where U t is drawn i.i.d.

Revised: from a centered Gaussian distribution with unit covariance matrix.

Revised: We then plotted in Fig.

Revised: 2 the squared gradient norm (cid:107) V ( ¯ X T ) (cid:107) 2 of the method’s ergodic average ¯ X T after T iterations (so values closer to zero are better).

Revised: For benchmarking purposes, we also ran the extra-gradient (EG) and Bach–Levy (BL) algorithms [2] with the same random seed for the simulated gradient noise.

Revised: The step-size parameter of the

Revised: EG algorithm was chosen as γ t = 0 .

Revised: 025 / √ t , whereas the BL algorithm was run with diameter and gradient bound estimation parameters D 0 = .

Original: {+5 and M 
+}
Revised: {+5 and M 
+}

Revised: = 2 .

Original: {+5 respectively (both determined after a hyper-parameter search since the only theoretically allowable values are D 
+}
Revised: {+5 respectively (both determined after a hyper-parameter search since the only theoretically allowable values are D 
+}

Revised: = M 0 = ∞ ; interestingly, very large values for D 0 and M 0 did not yield good results).

Revised: The experiment was repeated S =times, and AdaProx gave consistently faster convergence rates.

Original: This opens the door to several future research directions, from adaptation to other regimes (strong monotonicity, noise, etc.) to applications to non-convex / non-concave games and reinforcement learning problems.
Revised: Covariance matrix learning. Going a step further, we also considered the covariance learning game [Equation] The goal here is to generate data drawn from a centered Gaussian distribution with unknown covariance Σ ; in particular, this model follows the Wasserstein GAN formulation of Daskalakis et al.

Revised: [12] with generator and discriminator respectively given by G ( z ) =

Revised: θ z

Revised: and D ( x ) =

Revised: x (cid:62) φ x (no clipping).

Revised: For the experiments, we took d = 100, a mini-batch of m = 128 samples per update, and we ran the EG, BL and AdaProx algorithms as above, tracing the square norm of V as a measure of convergence.

Revised: Since the problem is non-monotone, there are several disjoint equilibrium components so the algorithms’ behavior is considerably more erratic; however, after this initial warm-up phase,

Revised: AdaProx again gave the faster convergence rates.

Original: E thical C oncerns
Revised: E thical C oncerns

Original: Our paper is primarily theoretical and, as such, we do not feel that it presents any ethical concerns to the community.
Revised: Our paper is primarily theoretical and, as such, we do not feel that it presents any ethical concerns to the community.

Original: Our main objective was to design a universal algorithm for min-max optimization problems and games that simultaneously achieves the following: a ) optimal rates in smooth / non-smooth problems; b ) convergence beyond the standard boundedness / Lipschitz continuity conditions; and c ) “o ﬀ -the-shelf” applicability without requiring prior tuning by the optimizer.
Revised: Our main objective was to design an adaptive algorithm for min-max optimization problems and games that simultaneously achieves the following: a ) order-optimal rates in smooth / non-smooth problems; b ) convergence beyond the standard boundedness / Lipschitz continuity conditions; and c ) “o ﬀ -the-shelf” applicability without requiring prior tuning by the optimizer.

Original: The proposed method provides a ﬂexible, freely available toolkit that can prove beneﬁcial to researchers and practitioners in the ﬁeld(from GAN training to variational inequalities) and we cannot see anypossible ethical concerns arising from its use
Revised: The proposed method provides a ﬂexible, freely available toolkit that can prove beneﬁcial to researchers and practitioners in the ﬁeld and we do not foresee any ethical concerns arising from its use