Accelerated Single-Call Methods for Constrained Min-Max Optimization
Abstract
We study first-order methods for constrained min-max optimization.
Existing methods either requires two gradient calls or two projections in each iteration, which may be costly in applications.
In this paper, we first show that the Optimistic Gradient (OG) method, a single-call single-projection algorithm, has O ( 1 √ T ) convergence rate for inclusion problems with operators that satisfy the weak
Minty variation inequality (MVI).
Our second result is the first single-call single-projection algorithm – the Accelerated Reflected Gradient (ARG) method that achieves the optimal O ( 1 T ) convergence rate for inclusion problems that satisfy negative comonotonicity.
Both the weak MVI and negative comonotonicity are well-studied assumptions and capture a rich set of non-convex non-concave min-max optimization problems.
Finally, we show that the Reflected Gradient (RG) method, another single-call single-projection algorithm, has O ( 1 √ T ) last-iterate convergence rate for constrained convex-concave min-max optimization, answering an open problem of [26].
1. Introduction
Various Machine Learning applications, from the generative adversarial networks (GANs) (e g., [1, 20]), adversarial examples (e g., [33]), robust optimization (e g., [4]), to reinforcement learning (e.g., [10, 16]) can be captured by constrained min-max optimization.
Unlike the well-behaved convex-concave setting, these modern ML applications often require solving non-convex non-concave min-max optimization problems in high dimensional spaces.
Unfortunately, the general non-convex non-concave setting is intractable even for computing a local solution [13, 25, 44].
Motivated by the intractability, researchers turn their attention to non-convex non-concave settings with structure .
Significant progress has been made for several interesting structured non-convex non-concave settings, such as the ones that satisfy the weak Minty variation inequality (MVI)
(Definition 2) [15, 46] and the ones that satisfy the more strict negatively comonotone condition (Definition 3) [6, 30].
These algorithms are variations of the celebrated extragradient (EG) method [29], an iterative first-order method.
Similar to the extragradient method, these algorithms all require two oracle calls per iteration, which may be costly in practice.
We investigate the following important question in this paper:
Can we design efficient single-call first-order methods for
[Equation]
We provide an affirmative answer to the question.
We first show that a single-call method known as the Optimistic Gradient (OG) method [26] is applicable to all non-convex non-concave settings that satisfy the weak MVI .
We then provide the Accelerated Reflected Gradient (ARG) method
Algorithm
Single-Call?
Constraints?
Non-Monotone
Comonotone weak MVI
Normal
EG+ [15] ✗ ✗
O ( 1 √ T ) O ( 1 √ T )
CEG+ [46] ✗ ✓
O ( 1 √ T ) O ( 1 √ T )
OG[ This paper ] ✓ ✓
O ( 1 √ T ) O ( 1 √ T )
Accelerated
FEG [30] ✗ ✗
O ( 1 T )
AS [6] ✗ ✓
O ( 1 T )
ARG [ This paper ] ✓ ✓
O ( 1 T )
A ✓ in [Table] ?” means the algorithm works in the constrained setting.
The convergence rate is in terms of the operator norm (in the unconstrained setting) and the residual (in the constrained setting).
that achieves the optimal convergence rate in all non-convex non-concave settings that satisfy the negatively comonotone condition .
Single-call methods have been studied in the convex-concave settings [26] but not for the more general non-convex non-concave settings.
See Table 1 for comparisons between our algorithms and other algorithms from the literature.
1.1. Our Contributions
Throughout the paper, we adopt the more general and abstract framework of inclusion problems, which includes constrained min-max optimization as a special case.
More specifically, we consider the following problem.
Inclusion Problem.
Given E = F + A
where F : R n → R n is a single-valued (possibly nonmonotone) operator and A : R n ⇒ R n is a set-valued maximally monotone operator, the inclusion problem is defined as follows find z ∗ ∈
Z such that 0 ∈ E ( z ∗ ) = F ( z ∗ ) + A ( z ∗ ) . (IP)
As shown in the following example, we can interpret a min-max optimization problem as an inclusion problem.
Example 1 (Min-Max Optimization)
The following structured min- max optimization problem captures a wide range of applications in machine learning such as GANs, adversarial examples, robust optimization, and reinforcement learning:
[Equation]
where f ( · , · ) is possibly non-convex in x and non-concave in y .
Regularized and constrained minmax problems are covered by appropriate choices of lower semi-continuous and convex functions g and h .
Examples include the ℓ 1 -norm, the ℓ 2 -norm, and the indicator function of a closed convex feasible set.
Let z
=
( x, y ) , if we define F ( z )
=
( ∂ x f ( x, y ) , − ∂ y f ( x, y )) and A ( z )
=
( ∂g ( x ) , ∂h ( y )) , where A is maximally monotone, then the first-order optimality condition of (1) has the form of an inclusion problem.
[13] shows that without any assumption on the operator E = F + A , the problem is intractable.
The most well understood setting is when E is monotone, i.e., ⟨ u − v, z − z ′ ⟩ ≥ 0 for all z, z ′ and u ∈ E ( z ) , v ∈ E ( z ′ ) , which captures convex-concave min-max optimization.
Motivated by nonconvex non-concave min-max optimization, we consider the two most widely studied families of non-monotone operators: (i) negatively comonotone operators and (ii) operators satisfy the less restrictive weak MVI.
See Section 2 for more detailed discussion on their relationship.
Here are the main contributions of this paper.
Contribution 1: We provide an extension of the Optimistic Gradient (OG) method for inclusion problems when the operator E = F + A satisfies the weak MVI.
More specifically, we prove that OG has a O ( 1 √ T ) convergence rate (Theorem 4) matching the state of the art algorithms [15, 46].
Importantly, our algorithm only requires a single oracle call to F and a single call to the resolvent of A .
a
Next, we provide an accelerated single-call method when the operator satisfies the stronger negatively comonotone condition.
Contribution 2: We design an accelerated version of the Reflected Gradient (RG) [8, 9, 26, 34] method that we call the Accelerated Reflected Gradient (ARG) method, which has the optimal O ( 1 T ) convergence rate for inclusion problems whose operators E = F + A are negatively comonotone (Theorem 5).
Note that O ( 1 T ) is the optimal convergence rate for any first-order methods even for monotone inclusion problems [14, 54].
Importantly, ARG only requires a single oracle call to F and a single call to the resolvent of A .
Finally, we resolve an open question from [26].
Contribution 3: We show that the Reflected Gradient (RG) method has a last-iterate convergence rate of O ( 1 √ T ) for constrained convex-concave min-max optimization (Theorem 6).
[26] show that the RG algorithm asymptotically converges but fails to obtain a concrete rate.
We strengthen their result to obtain a tight finite convergence rate for RG.
1.2. Related Works
We provide a brief discussion of the most relevant and recent results on nonconvex-nonconcave min-max optimization here and defer the discussion on related results in the convex-concave setting to Appendix A. We also refer readers to [2, 17, 49] and references therein for a comprehensive literature review on inclusion problems and related variational inequality problems.
Structured Nonconvex-Nonconcave Min-Max Optimization.
Since in general nonconvex-nonconcave min-max optimization problems are intractable, recent works study problems under additional assumptions.
The Minty variational inequality (MVI) assumption (also called coherence or variationally stablility), which covers all quasiconvex-concave and starconvex-concave problems, is well-studied in e.g., [11, 31, 32, 35, 50, 56].
Extragradient-type algorithms has O ( 1 √ T ) convergence rate for problems that satisfies MVI [11].
[15] proposes a weaker assumption called weak MVI , which includes both MVI or negative comonotonicity [3] as special cases.
Under the weak MVI, the EG+ [15] and OGDA+ [5] algorithms have O ( 1 √ T ) convergence rate in the unconstrained setting.
Recently, [46] generalizes EG+ to CEG+ algorithm, achieving the same convergence rate in the general (constrained) setting.
To the best of our knowledge, the OG algorithm is the only single-call single-resolvent algorithm with O ( 1 √ T ) convergence rate when we only assume weak MVI (Theorem 4).
The result for accelerated algorithms in the nonconvex-nonconcave setting is sparser.
For negatively comonotone operators, optimal O ( 1 T ) convergence rate is achieved by variants of the EG algorithm in the unconstrained setting [30] and in the constrained setting [6].
To the best of our knowledge, the ARG algorithm is the first efficient single-call single-resolvent method that achieves the accelerated and optimal O ( 1 T ) convergence rate in the constrained nonconvex-nonconcave setting (Theorem 5).
We summarize previous results and our results in Table 1.
2. Preliminaries
Basic Notations.
Throughout the paper, we focus on the Euclidean space R n equipped with inner product ⟨· , ·⟩ .
We denote the standard ℓ 2 -norm by ∥ · ∥ .
For any closed and convex set Z ⊆ R n , Π Z [ · ] : R n → Z denotes the Euclidean projection onto set Z such that for any z ∈ R n , Π Z [ z ] = argmin z ′ ∈Z ∥
z − z ′ ∥ .
We denote B ( z, r ) the ℓ 2 -ball centered at z with radius r .
Monotone Operator.
We recall some standard definitions and results on monotone operators here and refer the readers to [2, 48, 49] for more detailed introduction.
A set-valued operator A :
R n ⇒ R n maps each point z ∈ R n to a subset A ( z ) ⊆
R n .
We denote the graph of A as
Gra( A ) :
= { ( z, u ) :
u ∈ A ( z ) } and the zeros of A as Zer( A ) =
{ z : 0 ∈ A ( z ) } .
The inverse operator of A is denoted as A − 1 whose graph is Gra( A − 1 ) =
{ ( u, z ) : ( z, u ) ∈
Gra( A ) } .
For two operators A and B , we denote A + B to be the operator with graph Gra
( A + B )
= { ( z, u A + u B ) : ( z, u A )
∈
Gra( A ) , ( z, u B ) ∈
Gra( B ) } .
We denote the identity operator as I : R n → R n .
We say operator A is single valued if | A ( z ) | ≤ 1 for all z ∈ R n .
Single-valued operator A is L -Lipschitz if
[Equation]
Moreover, we say A is non-expansive if it is 1 -Lipschitz.
Definition 1 ((Maximally) monotonicity)
An operator A : R n ⇒ R n is monotone if (cid:10) [Equation]
[Equation]
Moreover, A is maximally monotone if A is monotone and Gra( A ) is not properly contained in the graph of any other monotone operators.
Non-Monotone Operator.
Definition 2 (Weak MVI [15, 46])
An operator A : R n ⇒ R n satisfies weak MVI if for some z ∗
∈ Zer( A )
, there exists ρ ≤⟨ u, z − z ∗ ⟩ ≥ ρ ∥ u ∥ 2 , ∀
( z, u ) ∈
Gra( A ) .
Definition 3 (Comonotonicity [3])
An operator A : R n ⇒ R n is ρ -comonotone if
[Equation]
When A is ρ -comonotone for ρ > 0 , then A is also known as ρ -cocoercive, which is a stronger condition than monotonicity.
When A is ρ -comonotone for ρ < 0 , then A is non-monotone.
Weak
MVI with ρ = 0 is also know as MVI, coherence, or variational stability.
Note that the weak MVI is implied by negative comonotonicity.
We refer the readers to [30, Example 1], [15, Section 2.2] and [46, Section 5] for examples of min-max optimization problems that satisfy the two conditions.
2.1. Problem Formulation
Inclusion Problem.
We say z is an ϵ -approximate solution to an inclusion problem (IP) if
[Equation]
Throughout the paper, we study IP problems under the following assumption.
Assumption
In the setup of IP,
there exists z ∗ ∈ Zer( E ) , i e., 0 ∈ F ( z ∗ ) + A ( z ∗ ) .
F is L -Lipschitz.
A is maximally monotone.
When F is monotone, we refer to the corresponding IP problem as a monotone inclusion problem, which covers convex-concave min-max optimization.
In the more general non-monotone setting, we would study problems that satisfy negative comonotonicity or weak MVI.
Assumption
In the setup of IP, E = F + A is ρ -comonotone, i.e.,
[Equation]
Assumption
In the setup of IP, E = F + A satisfies weak MVI with ρ ≤ 0 , i.e., there exists z ∗ ∈ Zer( E ) ,
[Equation]
Note that by definition, the parameter ρ in both assumption must be O ( 1 L ) .
An important special case of inclusion problem is the variational inequality problem.
Variational Inequality.
Let Z ⊆ R n be a closed and convex set and F : R n → R n be a singlevalued operator.
The variation inequality (VI) problem associated with Z and F is stated as find z ∗ ∈
Z such that ⟨ F ( z ∗ ) , z ∗ − z ⟩ ≤ 0 , ∀ z ∈ Z .
[Equation]
Note that VI is a special case of IP when A = N Z =
∂
I Z is the normal cone operator:
[Equation]
The general formulation of VI unifies many problems such as convex optimization, min-max optimization, computing Nash equilibria in multi-player concave games, and is extensively-studied since 1960s [17].
Definitions of the convergence measure for VI and the classical algorithms, EG and PEG, are presented in Appendix B.
2.2. Convergence Measure
We focus on a strong convergence measure called the tangent residual , defined as
[Equation]
It is clear by definition that r tanF,A ( z ) ≤ ϵ implies z is an ϵ -approximate solution to the inclusion (IP) problem, and also an ( ϵ · D ) approximate strong solution to the corresponding variational inequality (VI) problem when Z is bounded by D .
Moreover, the tangent residual is an upper bound of other notion of residuals in the literature such as the natural residual r natF,A [14] or the forward-backward residual r fbF,A [55] as shown in Fact 1.
We defer the proof to Appendix B.4.
Note that in the unconstrained setting where A = 0 , these residuals are all equivalent to the operator norm ∥ F ( z ) ∥ .
Fact 1 Let A be a maximally monotone operator and F be an single-valued operator.
Then for any z ∈ R n and α > 0 ,
[Equation]
3. Optimistic Gradient Method for Weak MVI Problems
In this section, we consider an extension of the Optimistic Gradient (OG) algorithm
[12, 26, 38, 39, 45] for inclusion problems: given arbitrary starting point z − 12 =
z 0 ∈ R n and step size η > 0 , the update rule is
[Equation]
For t ≥ 1 , the update rule can also be written as z
[Equation]
+
[Equation]
+ ηF ( z t − 12 )] , which coincides with the forward-reflected-backward algorithm [36].
We remark that the update rule of OG is different from the Optimistic Gradient Descent/Ascent (OGDA) algorithm
(also known as Past Extra Gradient (PEG) algorithm )
[47], which is single-call but requires two projections in each iteration.
Previous results for OG only hold in the convex-concave (monotone) setting.
The main result in this section is that OG has O ( 1 √ T ) convergence rate even for nonconvex-nonconcave min-max optimization problems that satisfy weak MVI, matching the state of the art results achieved by two-call methods [15, 46].
Remarkably, OG only requires single call to F and single call to the resolvent J ηA in each iteration.
The main result is shown in Theorem 4.
The proof relies on a simple yet important observation that z t − z t +1 η
[Equation]
Theorem 4 Assume Assumption 1 and 3 hold with ρ ∈ ( − 1 12 √ 3 L , 0] .
Consider the iterates of (OG) with step size η ∈ (0 , 12 L ) satisfying C
=
+ 2 ρη − 2 η 2 L 2 > 0 (existence of such η is guaranteed by Fact 2).
Then for any T ≥ 1 ,
[Equation]
where H 2 = ∥
z 1 − z ∗ ∥ 2 + 14 ∥
z 12 − z
∥ 2 .
4. Accelerated Reflected Gradient For Negatively Comonotone Problems
In this section, we propose a new algorithm called the Accelerated Reflected Gradient (ARG) algorithm.
We prove that ARG enjoys accelerated O ( 1 T ) last-iterate convergence rate for inclusion problems with comonotone operators (Theorem 5).
Note that the lower bound Ω( 1 T ) holds even for the special case of convex-concave min-max optimization [14, 54].
Our algorithm is inspired by the Reflected Gradient (RG) algorithm [8, 9, 26, 34] for monotone variational inequalities.
Starting at initial points z − 1 =
z 0 ∈ Z , the update rule of RG with step size η > 0 is as follows: for t = 0 , 1 , 2 , · · ·
[Equation]
We propose the following Accelerated Reflected Gradient (ARG) algorithm, which is a single-call single-resolvent first-order method.
Given arbitrary initial points z 0 =
z 12 ∈ R n and step size η > 0 , ARG sets z 1 =
J ηA [ z 0 − ηF ( z 0 )] and updates for t = 1 , 2 , · · ·
[Equation]
We use the idea from Halpern iteration [23] to design the accelerated algorithm (ARG).
This technique for deriving optimal first-order methods is also called Anchoring and receives intense attention recently [6, 14, 30, 51, 52, 54].
We defer detailed discussion on these works to Appendix A.
We remark that the state of the art result from [6] is a two-call variant of the EG algorithm.
Thus to the best of our knowledge, ARG is the first single-call single-resolvent algorithm with optimal convergence rate for general inclusion problems with comonotone operators.
Theorem 5 Assume Assumption 1 and 2 hold with ρ ∈ [ − 1 100 L , 0] , then the accelerated reflected gradient (ARG) algorithm with constant step size η > 0 satisfying Inequality (11) has the following last-iterate convergence rate: for any T ≥ 1 ,
[Equation]
where
[Equation]
We defer the proof to Appendix D.
5. Last-Iterate Convergence Rate of Reflected Gradient
In this section, we show that the Reflected Gradient (RG) algorithm [8, 9, 26, 34] has a last-iterate convergence rate of O ( 1 √ T ) with respect to tangent residual and gap function (see Definition 8) for solving monotone variational inequalities (Theorem 6).
We defer the proof to Appendix E
Theorem 6 For a variational inequality problem (VI) associated with a closed convex set Z and a monotone and L -Lipschitz operator F with a solution z ∗ , the (RG) algorithm with constant step size η ∈ (0 , 1 (1+
[Equation]
L ) has the following last-iterate convergence rate: for any T ≥ 1 ,
[Equation]
where H 2 = 5 ∥
z 0 − z
[Equation]
∥ 2 +
[Equation]
∥ F ( z
[Equation]
∥ 2 and λ =
[Equation]
We remark that the convergence rate of RG is slower than ARG and other optimal first-order algorithms even in the monotone setting.
Nevertheless, understanding its last-iterate convergence rate is still interesting: (1) RG is simple and largely used in practice; (2) Last-iterate convergence rates of simple classic algorithms such as EG and RG are mentioned as open problems in [26].
The question is recently resolved for EG [7, 21] but remains open for RG; (3) Compared to EG, RG requires only a single call to F and a single projection in each iteration.
Additional Related Works
A.1 Convex-Concave and Monotone Setting
A.2 Nonconvex-Nonconcave Setting
B Additional Preliminary
B.1 Normal Cone and Resolvent
B.2 Gap Function
B.3 Classical Algorithms for Variationaly Inequalities
B.4 Proof of Fact
C
Missing
Proofs
in
Section
D
Missing
Proofs in Section
D
.1 Potential Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D.2 Approximately Non-Increasing Potential . . . . . . . . . . . . . . . . . . . . . . .
D.3 Bouding Potential at Iteration 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D.4 Proof of Theorem 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
E Missing Proofs in Section
E.
Non-increasing Potential . .
E.
Best-Iterate Convergence . .
E.
Proof of Theorem 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
F Auxiliary Propositions
Appendix A. Additional Related Works A.1. Convex-Concave and Monotone Setting
In the convex-concave setting, a weak convergence measure is the gap function (Definition 8).
It is well-known that classic extragradient-type methods such as EG and PEG have O ( 1 T ) averageiterate convergence rate in terms of gap function [26, 39–41] and the rate is optimal [43].
But the gap function or average-iterate convergence is not meaningful in the nonconvex-nonconcave setting.
For convergence in terms of the residual in the constrained setting, EG and PEG has a slower rate of O ( 1 √ T ) for best-iterate convergence [17, 26, 29, 47] and the more desirable last-iterate convergence [7, 22].
We remark that the last-iterate convergence rate of the reflected gradient (RG) algorithm was unknown.
The O ( 1 √ T ) rate is tight for p -SCIL algorithms [19], a subclass of firstorder methods that includes EG, PEG, and many of its variations, but faster rate is possible for other first-order methods.
Accelerated Convergence Rate in Residual.
Recent results with accelerated convergence rates in terms of the residual are based on Halpern iteration [23] (also called Anchoring ).
The vanilla
Halpern iteration has O ( 1 T ) convergence rate for cocoercive operators (stronger than monotonicity) [14, 28].
Recently, a line of works contributed to provide O ( 1 T ) convergence rate for monotone operators in the constrained setting.
[14, 55] provide double-loop algorithms with O ( log TT ) convergence rate for monotone operators in the constrained setting.
In the unconstrained setting ( A = 0 ), [54] propose the Extra Anchored Gradient (EAG) algorithm, the first efficient algorithm with O ( 1 T ) convergence rate for monotone operators.
They also establish matching lower bound for first-order methods.
[30] generalize EAG to Fast Extragradient (FEG), which works even for negatively comonotone operators but still in the unconstrained setting.
Analysis for variants of EAG and FEG in the unconstrained setting is provided in [51, 52].
Recently, [6] close the open problem by proving the projected version of EAG has O ( 1 T ) convergence rate.
They also propose the accelerated forward-backward splitting (AS) algorithm, a generalization of FEG, which has O ( 1 T ) convergence rate for negatively comonotone operators in the constrained setting.
A.2. Nonconvex-Nonconcave Setting
This paper study structured nonconvex-nonconcave optimization problems from the general perspective of operator theory and focus on global convergence under weak MVI and negative comonotonicity.
There is a line of works focusing on local convergence , e.g., [18, 24, 27, 37].
Another line of works focus on problems satisfying different structural assumptions, such as the Polyak Łojasiewicz condition [42, 53].
Appendix B. Additional Preliminary B.1. Normal Cone and Resolvent
Normal Cone.
We denote N Z : Z → R n to be the normal cone operator such that for z ∈ Z ,
N Z ( z ) = { a : ⟨ a, z ′ − z ⟩ ≤ 0 ,
∀ z
′ ∈ Z} .
Define the indicator function
[Equation]
It is not hard to see that the subdifferential operator ∂
I Z = N Z .
A useful fact is that if z = Π Z [ z ′ ] , then λ ( z ′ − z
) ∈ N Z
( z ) for any λ ≥ 0 .
Some useful properties of the resolvent are summarized in the following proposition.
Proposition 7 If A is maximally monotone, then J A satisfies the following.
The domain of J A is R n .
J A is non-expansive and single-valued on R n .
If z = J A ( z ′ ) , then z ′ − z
∈
A ( z ) .
If c ∈ A ( z ) , then z = J A ( z + c ) .
When A = N Z is the normal cone operator for some closed convex set Z , then J ηA =
Π Z is the Euclidean projection onto Z for all η > 0 .
B.2. Gap Function
A standard suboptimality measure for the variationaly inequalitt (VI) problem is the gap function defined as G AP Z ,F ( z ) :
= max z ′ ∈Z ⟨ F ( z ) , z − z ′ ⟩ .
Note that when the feasible set Z is unbounded, approximating the gap function is impossible: consider the simple unconstrained saddle point problem min x ∈ R max y ∈ R xy , which has a unique saddle point (0 , 0) but any other point has an infinitely large gap.
A refined notion is the following restricted gap function [41], which is meaningful for unbounded Z .
Definition 8 (Restricted Gap Function)
Given a closed convex set Z , a single-valued operator
F , and a radius D , the restricted gap function at point z ∈ Z is
[Equation]
where B ( z, D ) is a Euclidean ball centered at z with radius D .
In the rest of the paper, we call G AP Z ,F,D the gap function (or gap) for convenience.
The following
Lemma relates ∥ F ( z ) + c ∥ where c ∈ N Z ( z ) , and the gap function.
Lemma 9 Let Z be a closed convex set Z and F be a monotone and L -Lipschitz operator.
For any z ∈ Z and c ∈ N Z ( z ) , we have
[Equation]
Proof The proof is straightforward.
Since c ∈ N Z ( z ) , we have ⟨ c, z − z ′ ⟩ ≥ 0 for any z ′ ∈ Z .
Therefor,
[Equation]
B.3. Classical Algorithms for Variationaly Inequalities
The Extragradient Algorithm [29].
Starting at initial point z 0 ∈ Z , the update rule of EG is: for t = 0 , 1 , 2 , · · ·
[Equation]
At each step t ≥ 0 , the EG algorithm makes an oracle call of F ( z t ) to produce an intermediate point z t + 12 (a gradient descent step if F = ∂f is the gradient of some function f ), then the algorithm makes another oracle call F ( z t + 12 ) and updates z t to z t
+1 .
In each step, EG needs two oracle calls to F and two projections Π Z .
The Past Extragradient Algorithm [47]
Starting at initial point z
=
z − 12 ∈ Z , the update rule of PEG with step size η > 0 is: for t = 0 , 1 , 2 , · · ·
[Equation]
PEG
[Equation]
Note that PEG is also known as the Optimistic Gradient Descent/Ascent (OGDA) algorithm in the literature.
The update rule of PEG is similar to (EG) but only requires a single call to F in each iteration.
Both of EG and PEG perform two projections in every iteration.
B.4. Proof of Fact
Proof For any c ∈ A ( z ) , we have
[Equation]
( J A is non-expansive) and
[Equation]
Thus both r tanF,A ( z ) and r fbF,A,α ( z ) are smaller than r tanF,A ( z )
= min c ∈ A ( z ) ∥ F ( z ) + c ∥ .
Appendix C. Missing Proofs in Section
We present the proof of Theorem 4 here.
Proof From the update rule of (OG)
[Equation]
we have the following identity (see also [26, Appendix B]): for any p ∈ Z ,
[Equation]
Since z t + 12 =
J ηA [ z t − ηF ( z t − 12 )]
[Equation]
we have
[Equation]
12 ) by Proposition 7.
Then
[Equation]
Set p =
z ∗ .
By the weak MVI assumption
[Equation]
we have
[Equation]
Define c = 12 − 2 η 2 L 2 > 0 .
We have identity
[Equation]
[Equation]
Combining Equation (2) and (3) and using ∥ a + b ∥ 2 ≤ 2 ∥ a ∥ 2 + 2 ∥ b ∥ 2 , we have ∥ z t +1 − z ∗ ∥
[Equation]
Using the update rule of OG and L -Lipschitzness of F , we have that for any t ≥ 0 ,
[Equation]
Moreover, using ∥ a + b ∥ 2 ≤ 2 ∥ a ∥ 2 + 2 ∥ b ∥ 2 and Equation (6) , we have that for any t ≥ 1 , (cid:13)(cid:13)(cid:13)
z t + 12 − z t
[Equation]
12 (cid:13)(cid:13)(cid:13)
[Table]
which imples
[Equation]
[Table]
[Equation]
Combining Equation (4), (5), (6)
[Equation]
and (7), we have that for all t ≥ 1 .
[Equation]
Telescoping the above inequality and using c = 12 − 2 η 2 L 2 and ηL < 12 , we get
[Equation]
Note that z 0 is the same as z − 12 .
This completes the proof.
Fact 2 For any L > 0 and ρ > − 1 12 √ 3 L .
There exists η ∈ (0 , 12 L ) such that
[Equation]
[Equation]
Proof Let η = 1 2 √ 3 L , then Inequality (8) holds whenever
[Equation]
Appendix D. Missing Proofs in Section
To prove Theorem 5, we apply a potential function argument.
We first show the potential function is approximately non-increasing and then prove that it is upper bounded by a term independent of
T .
As the potential function at step t is also at least Ω( t 2 ) · r tan ( z t ) 2 , we conclude that ARG has an
O ( 1 T ) convergence rate .
D.1. Potential Function
Recall the update rule of ARG:
z 0 =
z 12 ∈ R n are initial points and z 1 =
J ηA [ z 0 − ηF ( z 0 )] ; for t ≥ 1 ,
[Equation]
Recall that when A is the normal cone of a closed convex set Z , the resolvent J A is equivalent to Euclidean projection to set Z .
Hence, if we apply the ARG algorithm to solve monotone VI problems, the algorithm uses a single call to operator F and a single projection to Z per iteration.
Here we allow A to be an arbitrary maximally monotone operator, and the ARG algorithm becomes a single-call single-resolvent algorithm in this more general setting.
Next, we specify the potential function.
Define
[Equation]
By update rule we have c t ∈ A ( z t ) for all t ≥ 1 .
The potential function we track in iterate t ≥ 1 is defined as
[Equation]
D.2.
Approximately Non-Increasing Potential
Fact 3 For any L > 0 and ρ ≥ − 1 100 L .
There exists η > 0 such that
[Equation]
[Equation]
Moreover, every η > 0 satisfies (11) also satisfies ρη ≥ − 14 .
Proof Rewriting (11), we get
[Equation]
Let x = ηL and f ( x ) =
x (72 x 2 − 1) 4+8 x 2 .
Since f ( 112 ) = − 3292 < − 1100 .
We know η = 112 L satisfies (11).
Moreover, rewritng (11) and using ηL > 0 , we get
[Equation]
We show in the following lemma that V t is approximately non-increasing.
Lemma
In the same setup as Theorem 5, for any t ≥ 1 , we have
[Equation]
Proof The plan is to show that V t − V t +1 plus a few non-positive terms is still ≥ − 18 ·∥ ηF ( z t +1 ) + ηc t +1 ∥ 2 , which certifies the claim.
Two Positive Terms.
Since F + A is ρ -comonotone, we have
[Equation]
Since F is L -Lipschitz, we have
[Equation]
Denote p = 172 .
Multiplying the above inequality with 1 − ρ 5 η > 0 and rearranging terms, we get
[Equation]
Sum-of-Squares Identity.
We show an equivalent formulation z t + 12 and z t +1 using definitions of ηc
[Equation]
1 − z t − ηF ( z t − 12 ) + 1 t ( z 0 − z t − 1 ) and ηc t +
[Equation]
z
[Equation]
We also have
[Equation]
Next, we simplify
[Equation]
using the second identity in Proposition 16: replace x 0 with z 0 ; for k ∈ [4] , replace x k with z t − 1+ kand replace y k with ηF ( z t − 1+ k 2 ) ; replace u 2 with ηc t ; replace u 4 with ηc t +1 ; replace k with t ; replace p with q .
Note that x 3 = x 2 − y 1 − u 2 + 1 k +1 ( x 0 − x 2 ) and x 4 = x 2 − y 3 − u 4 ++ 1 k
+
( x 0 − x 2 ) hold due to the above equivalent formations of z t + 12 and z t +1 .
Expression (18) and (19) appear on both sides of the following equation.
[Equation]
Since ∥ a ∥ 2 + ⟨ a
[Equation]
b ⟩ = ∥
a + b
[Equation]
− ∥ b
∥ 2 4 , we have
Expression (16) + Expression (17)
[Equation]
≥ −
[Equation]
Now it remains to show that the sum of Expression (15), (18)
[Equation]
and (19) is non-negative.
Multiplyingt ( t +1) and replacing p = 172 , we get
[Equation]
Denote
[Equation]
It is not hard to check that B 1 − B 2 =
[Equation]
B 3 − B 4 ) :
[Equation]
Thus we have
[Equation]
The last inequality holds by the choice of η as shown in Fact
[Equation]
D.3. Bouding Potential at Iteration
Lemma 11 Let F be a L -Lipschitz operator and
A be a maximally monotone operator.
For any z 0 = z 12 ∈ R n , η ∈ (0 , 12 L ) , and z 1 = J ηA [ z 0 − ηF ( z 0 )] , we have the following 1.
[Equation]
η · r tanF,A ( z 0 ) .
∥ ηF ( z 1 ) + ηc 1 ∥ ≤ (1 + ηL ) ∥
z 1 − z
∥ 3.
V 1 ≤ 4 ∥
z 1 − z
∥ 2 where V 1 is defined in (10)
Proof For any c ∈ A ( z 0 ) , due to non-expansiveness of J ηA , we have
[Equation]
Thus ∥ z 1 − z
∥ ≤ η · r tanF,A ( z 0 ) .
By definition of V 1 in (10), we have
[Equation]
We bound ∥ ηF ( z 1 ) + ηc 1 ∥ first.
Note that by definition, we have ηc 1 = z 0 − ηF ( z 0 ) − z 1 .
Thus we have
[Equation]
triangle inequality
[Equation]
Then we can apply the bound on ∥ ηF ( z 1 ) + ηc 1 ∥ to bound V 1 as follows:
[Equation]
where we use L -Lipschitzness of F and Cauchy-Schwarz inequality in the first inequality; we use ∥ ηF ( z 1 ) + ηc 1 ∥ ≤ (1 + ηL ) ∥
z 1 − z
∥ in the second inequality; we use ηL ≤ 12 in the last inequality.
D.4. Proof of Theorem
We first show that the potential function V t =
Ω( t 2 · r tan ( z t ) 2 ) .
Lemma
In the same setup as Theorem 5, for any t ≥ 1 , we have
[Equation]
Proof Since 0 ∈ F ( z ∗ ) + A ( z ∗ ) , by ρ -comonotonicity of F + A and Fact 3, we have
[Equation]
By definition of V t in (10), for any t ≥ 1 , we have
V t
[Equation]
where in the second last inequality we we apply ⟨ a, b ⟩ ≥ − α 4 ∥ a ∥ 2 − 1 α ∥ b ∥ 2 with a = √ t ( ηF ( z t ) + ηc t ) , b = √ t ( z ∗ − z 0 )
[Equation]
and α = t +
[Equation]
Proof [Proof of Theorem 5] It is equivalent to prove that for every T ≥ 1 , we have
[Equation]
From Lemma 11, we have
[Equation]
So the theorem holds for T = 1 .
For any T ≥ 2 , by Lemma 12 we have
[Equation]
By subtracting 18 ∥ ηF ( z T ) + ηc T ∥ 2 from both sides of the above inequality, we get
[Equation]
which is in the form of Proposition 17 with C
= H 2 and p = 19 .
Thus we have for any T ≥
[Equation]
Appendix E. Missing Proofs in Section
To prove Theorem 6, our analysis is based on a potential function argument and can be summarized in the following three steps.
(1) We construct a potential function and show that it is non-increasing between two consecutive iterates; (2) We prove that the RG algorithm has a best-iterate convergence rate, i.e., for any T ≥ 1 , there exists one iterate t ∗ ∈
[ T ] such that our potential function at iterate t ∗ is small; (3) We combine the above steps to show that the the last iterate has the same convergence guarantee as the best iterate and derive the O ( 1 √ T ) last-iterate convergence rate.
E.1. Non-increasing Potential
Potential Function.
We denote
[Equation]
Note that according to the update rule of RG, z t +1 =
Π Z [ z t − ηF ( z t + 12 )] , so c t +1 ∈ N Z ( z t +1 ) .
The potential function we adopt is P t defined as
[Equation]
Lemma
In the same setup of Theorem 6, P t ≥ P t +1 for any t ≥ 1 .
Proof The plan is to show that P t − P t +1 plus a few non-positive terms is non-negative, which certifies that P t − P t +1 ≥ 0 .
Three Non-Positive Terms.
Since F is monotone, we have
[Equation]
Since F is L -Lipschitz and 0 < η <
[Equation]
L < 12 L , we have
[Equation]
By definition, we have c t +1 ∈ N Z ( z t +1 ) and c t ∈ N Z
( z t ) .
Since the normal cone operator N Z is maximally monotone, we have
[Equation]
Sum-of-Squares Identity.
We use the following equivalent formations of z t + 1 2 and z t +1 .
[Equation]
The following identity holds according to Proposition 16.
To see this, we replace x k with z t − 1+ k 2 ; replace y k with ηF ( z t − 1+ k 2 ) ; replace u 2 with ηc t ; replace u 4 with ηc t +1 ; also note that x 3 = x 2 − y 1 − u 2 and x 4 = x 2 − y 3 − u 4 hold due to the above equivalent formations of z t + 12 and z t +1 .
[Equation]
The right-hand side of the above equality is clearly ≥ 0 , thus we conclude P t − P t +1 ≥ 0 .
E.2. Best-Iterate Convergence
In this section, we show that for any T ≥ 1 , there exists some iterate t ∗ such that P t ∗ =
O ( 1 T ) , which is implied by (cid:80)
Tt =1 P t =
O (1) .
To prove this, we first show (cid:80) T t =1 ∥
z t + 12 − z t ∥ 2 = (cid:80) T t =1 ∥
z t − z t − 1 ∥ 2 =
O (1) and then relate (cid:80)
Tt =1 P t to these two quantities.
Lemma
In the same setup of Theorem 6, for any T ≥ 1 , we have
[Equation]
Proof First note that by the update rule of RG, we have z t + 1 2 =
z t − z t − 1 thus z t + 1 2 − z t =
z t − z t − 1 .
Therefore, it suffices to only prove the inequality for (cid:80)
Tt =1 ∥
z t + 12 − z t ∥ 2 .
From the proof of [26, Lemma 2], for any t ≥ 1 and p ∈
Z , we have
[Equation]
We set p =
z ∗ to be a solution of the variational inequality (VI) problem in the above inequality.
Note that
[Equation]
where the last equality holds since z t + 12 =
z t − z t − 1 .
Also note that ⟨ F ( z ∗ ) , z t − z ∗ ⟩ ≥ 0 for all t ≥ 0 since z t ∈ Z and z ∗ is a solution to (VI).
Combing Inequality (26) and Inequality (27), telescoping the terms for t = 1 , 2 , · · · , T , and dividing both sides by 1 − (1 + √ 2) ηL > 0 , we get
[Equation]
To get a cleaner constant that only relies on the starting point z
= z 12 , we further simplify the three terms on the right-hand side.
Note that since η < 12 L and z 1 =
Π Z [ z 0 − ηF ( z 0 )] .
[Equation]
Thus we have
[Equation]
Moreover,
[Equation]
Thus ∥ z 1 − z
[Equation]
This completes the proof
[Equation]
Lemma
In the same setup of Theorem 6, for any T ≥ 1 , we have
[Equation]
Proof We first show an upper bound for P t
[Equation]
Summing the above inequality of t = 1 , 2 , · · · T , we get
T (cid:88)
[Equation]
The second last inequality holds by Lemma 14.
The last inequality holds since ∥
z 1 − z
∥ 2 ≤ 4 L 2 ∥ F ( z 0 ) ∥ 2 ≤ H 2 .
Recall that λ
= (cid:114)
[Equation]
η 2 L 2 (1 − (1+ √ 2) ηL ) .
This completes the proof.
E.3. Proof of Theorem
Fix any T ≥ 1 .
From Lemma 13, we know that the potential function P t is non-increasing for all t ≥ 1 .
Lemma 15 guarantees that the sum of potential functions (cid:80) Tt =1 P t is upper bounded by λ 2 H 2 L 2 , where λ 2 = 6(1+3 η 2 L 2 )
η 2 L 2 (1 − (1+ √ 2) ηL ) .
Combining the above, we can conclude that the potential function at the last iterate P T is upper bounded by λ 2 H 2 L 2 T .
Since P T =
∥ F ( z T ) + c T ∥ 2 +
∥ F ( z T ) − F ( z T − 12 )
∥ 2 , we obtain the last-iterate convergence rate r tanF,Z ( z T ) 2 ≤ ∥ F ( z T ) + c T ∥ 2 ≤ λ 2 H 2 L 2 T .
The convergence rate on ∥ F ( z T ) + c T ∥ 2 implies a convergence rate on the gap function
G AP Z,F,D ( z T ) by Lemma 9:
[Equation]
Appendix F. Auxiliary Propositions
Proposition 16 (Two Identities) Let ( x k ) k ∈ [4] , ( y k ) k ∈ [4] , x 0 , u 2 and u 4 be arbitrary vectors in
R n .
Let k ≥ 1 and q ∈ (0 , 1) be two real numbers.
If the following two equations holds:
[Equation]
then the following identity holds:
[Equation]
If the following two equations holds:
[Equation]
then the following identity holds:
[Equation]
Proof We verify the two identities by M ATLAB .
Code is available at the following two anonymous links: https://pastebin.com/MLDWWpxE and https://pastebin.com/NCsAyyZf .
Proposition 17 ([6]) Let { a k ∈ R + } k ≥ 2 be a sequence of real numbers.
Let C 1 ≥ 0 and p ∈ (0 , 13 ) be two real numbers.
If the following condition holds for every k ≥ 2 ,
[Equation]
then for each k ≥ 2 we have
[Equation]
Proof We prove the statement by induction.
Base Case: k = 2 .
From Inequality (28), we have
[Equation]
Thus, Inequality (29) holds for k = 2 .
Inductive Step: for any k ≥ 3 .
Fix some k ≥ 3 and assume that Inequality (29) holds for all 2 ≤ t ≤ k − 1 .
We slightly abuse notation and treat the summation in the form (cid:80) 2 t =3 as 0.
By
Inequality (28), we have
[Equation]
Induction assumption on Inequality (29))
[Equation]
This complete the inductive step.
Therefore, for all k ≥ 2 , we have a k ≤ 4 · C 1 1 − 3 p · 1 k 2 .