Accelerated Algorithms for Monotone Inclusion and Constrained Nonconvex-Nonconcave Min-Max Optimization
Abstract
We study monotone inclusions and monotone variational inequalities, as well as their generalizations to non-monotone settings.
We first show that the Extra Anchored Gradient (EAG) algorithm, originally proposed by Yoon and Ryu [37] for unconstrained convex-concave min-max optimization, can be applied to solve the more general problem of Lipschitz monotone inclusion.
More specifically, we prove that the EAG solves Lipschitz monotone inclusion problems with an accelerated convergence rate of O ( 1 T ) , which is optimal among all first-order methods [9, 37].
Our second result is an accelerated forward-backward splitting algorithm (AS), which not only achieves the accelerated O ( 1 T ) convergence rate for all monotone inclusion problems, but also exhibits the same accelerated rate for a family of general (non-monotone) inclusion problems that concern negative comonotone operators.
As a special case of our second result, AS enjoys the O ( 1 T ) convergence rate for solving a non-trivial class of nonconvex-nonconcave min-max optimization problems.
Our analyses are based on simple potential function arguments, which might be useful for analysing other accelerated algorithms.
1. Introduction
We study the constrained single-valued monotone inclusion problem and the monotone variational inequality , as well as their generalizations in non-monotone settings.
Given a closed convex set
Z ⊆ R n and a single-valued and monotone operator F : Z → R n , i.e., (cid:10) F ( z ) [Equation] F
[Equation]
, z
[Equation] (cid:11) ≥ 0 , [Equation] , the constrained single-valued monotone inclusion problem (MI) consists in finding a z ∗ ∈
Z such that
[Equation]
where I Z ( · ) is the indicator function for set Z , 1 and ∂
I Z ( · ) is the subdifferential operator of I Z .
The corresponding monotone variational inequality shares the same input, and asks for a z ∗ ∈
Z such that
[Equation]
The monotone inclusion problem (MI) and the related monotone variational inequality play a crucial role in mathematical programming, providing unified settings for the study of optimization and equilibrium problems.
They also serve as computational frameworks for numerous important applications in fields such as economics, engineering, probability and statistics, and machine learning [2, 12, 31].
Although the set of exact solutions to the monotone inclusion problem (MI) coincides with the set of exact solutions to the corresponding variational inequality, the approximate solutions to these two problems differ due to different performance measures.
An approximate solution to the monotone inclusion problem (MI) must have a small natural residual, 3 while an approximate solution to the variational inequality only satisfies a weaker condition, i.e., its gap function is small.
Indeed, it is well-known that an approximate solution to the monotone inclusion problem (MI) is also an approximate solution to the monotone variational equality, but the reverse is not true in general.
An important special case of the monotone inclusion problem (MI) is the convex-concave minmax optimization problem:
[Equation]
where X and Y are a closed convex sets in R n x and R n y respectively, and f ( · , · ) is smooth, convex in x , and concave in y .
Besides its central importance in game theory, convex optimization, and online learning, the convex-concave min-max optimization problem has recently received a lot of attention from the machine learning community due to several novel applications such as the generative adversarial networks (GANs) (e.g., [1, 14]), adversarial examples (e.g., [24]), robust optimization (e.g., [4]), and reinforcement learning (e.g., [6, 11]).
Given the importance of the monotone inclusion problem (MI) , it is crucial to understand the following open question.
What is the optimal convergence rate achievable by a first-order method for monotone inclusions?
We provide the first algorithm that achieves the optimal convergence rate and further extend it to inclusion problems with negatively comonotone operators , an important family of non-monotone operators.
This generalization allows us to obtain the optimal convergence rate for a family of structured nonconvex-nonconcave min-max optimization problems.
Prior to our work, even for the special case of convex-concave min-max optimization, the optimal convergence rate is only known for the relatively weak notion of duality gap [26, 27], which is also difficult to generalize to nonconvex-nonconcave settings, see [37] for more discussion.
1.1. Our Contributions
A point z ∈ Z is an ϵ -approximate solution to the constrained single-valued monotone inclusion problem (MI) if
[Equation]
where B ( 0 , ϵ ) is the ball with radius ϵ centered at 0 .
As we argue in Section 2.3, this is equivalent to the tangent residual of z , a notion introduced in [5], being no more than ϵ .
Our first contribution provides an answer to question (*).
Contribution 1: We extend the Extra Anchored Gradient algorithm (EAG), originally proposed by Yoon and Ryu [37] for unconstrained convex-concave min-max problems, to solve constrained single-valued monotone inclusion problems (MI) .
Note that constrained convexconcave min-max optimization is a special case.
We show in Theorem 1 that (EAG) finds an O ( LT ) -approximate solution in T iterations for monotone inclusions (MI) , where L is the
Lipschitz constant of the operator F .
The convergence rate we obtain for (EAG) matches the lower bound by [9, 37], and is therefore optimal for any first-order method.
For the second part of the paper, we go beyond the monotone case and study inclusion problems (CMI) with operators that are not necessarily monotone and only satisfy the weaker ρ - comonotoncity (Assumption 2) condition.
Given a single-valued, L -Lipschitz, and possibly non-monotone operator
F and a set-valued maximally monotone operator A , we denote E = F + A .
The inclusion problem (CMI) consists in finding a point z ∗ ∈ R n that satisfies
[Equation]
under the assumption that E is a set-valued negatively comonotone operator.
The inclusion problem (CMI) captures (MI) (when ρ = 0 and A = ∂ I Z ) and a class of nonsmooth nonconvex-nonconcave min-max optimization problems (when ρ < 0 and A chosen appropriately, see Example 2).
Our second contribution is a new algorithm that achieves accelerated rate for solving (CMI).
Contribution 2: We design an accelerated forward-backward splitting algorithm (AS) that finds a O ( LT ) -approximate solution to the inclusion problem (CMI) in T iterations as long as
E = F + A is a ρ -comonotone operator with ρ > − 12 · L .
See Theorem 2 for a formal statement.
Our algorithm is inspired by the Fast Extra-Gradient (FEG) algorithm
In particular, (AS) is identical to FEG in the unconstrained setting, i.e., A ( z ) = 0 for all z , and can be viewed as a generalization of FEG to accommodate arbitrary set-valued maximally monotone operator A .
Our result is the first to obtain accelerated rate for constrained nonconvex-nonconcave min-max optimization and inclusion problems with negatively comonotone operators.
Splitting methods are a family of algorithms for solving inclusion problems where the goal is to find a zero of an operator that can be represented as the sum of two or more operators.
These methods only invoke each operator individually rather than their sum directly, and hence the name.
There is a rich literature on splitting methods.
See [32] and the references therein for more detailed discussion.
1.2. Related Work
There is a vast literature on inclusion problems and variational inequalities, e.g., see [2, 12, 31] and the references therein.
We summarize most relevant results in Table 1 and provided more detailed discussion in Appendix A.
[Table]
AS [ This paper ]
[Table]
2. Preliminaries
We consider the Euclidean Space ( R n , ∥ · ∥ ) , where ∥ · ∥ is the ℓ 2 norm and ⟨· , ·⟩ denotes inner product on R n .
Basic Notions about Monotone Operators
[Table]
A set-valued operator A : R n ⇒ R n maps z ∈ R n to a subset A ( z ) ⊆
R n .
We say A is single-valued if | A ( z ) | ≤ 1 for all z ∈ R n .
Tha graph of an operator A is defined as Gra A = { ( z, u ) : z ∈ R n ,
u ∈ A
( z ) } .
The inverse operator of A is denoted as A − 1 whose graph is Gra A − 1 = { ( u, z ) :
( z, u ) ∈
Gra A } .
For two operators A and B , we denote
A + B as the operator with graph Gra A + B = { ( z, u A + u B ) : ( z,
u A ) ∈
Gra A , ( z, u B ) ∈
Gra B } .
We denote the identity operator as I : R n → R n .
For L ∈ (0 , ∞ ) , a single-valued operator A : R n → R n is L -Lipschitz if
[Equation]
Moreover, A is non-expansive if it is 1 -Lipschitz.
A set-valued operator A : R n ⇒ R n is monotone if
[Equation]
For a closed convex set Z ⊆ R n and point z ∈ R n , we denote the normal cone operator as
N Z :
[Equation]
∈
[Equation]
Define the indicator function
I Z
[Equation]
if z ∈ Z , + ∞ otherwise .
Then it is not hard to see that the subdifferential operator ∂
I Z = N Z .
The projection operator Π Z : R n → R n is defined as Π Z [ z ] := argmin z ′ ∈Z ∥
z − z ′
∥ 2 .
Maximally Monotone Operator.
A is maximally monotone if A is monotone and there is no other monotone operator B such that Gra A ⊂ Gra B .
When f : R n → R is a convex closed proper function, then the subdifferential operator ∂f is maximally monotone.
Therefore, ∂ I Z = N Z is maximally monotone.
We denote the resolvent of an operator A as J A :
= ( I + A ) − 1 .
ρ -comonotonicity.
A generalized notion of monotonicity is the ρ -comonotonicity [3]: For ρ ∈ R , an operator A : R n ⇒ R n is ρ -comonotone if
[Equation]
Note that when A is 0 -comonotone, then A is monotone.
If A is ρ -comonotone for ρ > 0 , we also say A is ρ -cocoercive (a stronger assumption than monotonicity).
When A satisfies negative comonotonicity, i.e., ρ -comonotonicity with ρ < 0 , then A is possibly non-monotone.
Negative comonotonicity is the focus of this paper in the non-monotone setting.
2.1. Monotone Inclusion and Variational Inequality
Constrained single-valued monotone Inclusion.
Given a closed convex set Z ⊆ R n and a single-valued monotone operator F , the constrained single-valued monotone inclusion problem is to find a point z ∗ ∈ R n that satisfies
[Equation]
We focus on monotone inclusion problems of Lipschitz operators that have a solution.
Assumption 1 In (MI) problem,
F is monotone and L -Lipschitz on Z , i.e., (cid:10) F ( z ) − F ( z ′ ) , z − z ′ (cid:11) ≥ 0 and (cid:13)(cid:13) F ( z )
[Equation]
F ( z ′ )
(cid:13)(cid:13)
[Equation]
[Equation] [Equation]
Z . 2.
There exists a solution z ∗ ∈
Z such that 0 ∈ F ( z ∗ ) + ∂
I Z ( z ∗ ) .
We define the Minty Variational Inequality problem (MVI) and the Stampacchia Variational Inequality problem (SVI) and show how their solutions relate in Appendix B.
Inclusion Problems with Negatively Comonotone Operators.
We study inclusion problem (CMI) with (non)-monotone operators that satisfies ρ - comonotoncity (Assumption 2), which captures (MI) (when ρ = 0 ) and a class of nonconvex-nonconcave min-max optimization problems (when ρ < 0 ).
Given a single-valued and possibly non-monotone operator
F and a set-valued maximally monotone operator A , we denote E = F + A .
The inclusion problem is to find a point z ∗ ∈ R n that satisfies
[Equation]
Similar to (MI) , we say z is an ϵ -approximate solution to (CMI) if
[Equation]
We summarize the assumptions on (CMI) below.
Assumption 2 In (CMI) ,
F : R n → R n is L -Lipschitz.
A : R n ⇒ R n is maximally monotone.
E = F + A is ρ -comonotone, i.e., there exists ρ ≤ 0 such that
[Equation]
There exists a solution z ∗ ∈ R n such that 0 ∈ E ( z ∗ )
[Equation]
The formulation of (CMI) provides a unified treatment for a range of problems, such as min-max optimization and multi-player games.
We present one detailed example in Example 2 in Appendix B and refer readers to [12] for more examples.
2.3. Convergence Criteria
An appropriate convergence criterion is the tangent residual defined in [5] r tanF,A ( z ) :
= min c ∈ A ( z ) ∥ F ( z ) + c ∥ .
It is not hard to see that r tanF,A ( z ) ≤ ϵ implies that z is an ϵ -approximate solution to (CMI) .
If
A = ∂ I Z , and Z is bounded and has diameter no more than D , then z is an ϵ -approximate solution to (MI) .
Another commonly-used convergence criterion that captures the stationarity of a solution is the natural residual r natF,A :=
∥
z − J A [ z − F ( z )] ∥ .
Note that z ∗ is a solution to (CMI)
iff z ∗ =
J A [ z ∗ − F ( z ∗ )] .
The definition of the natural residual for (MI) is similar: r natF,∂ I Z :=
∥
z − Π Z [ z − F ( z )] ∥ .
Fact 1 In (CMI) , r natF,A ( z ) ≤ r tanF,A ( z ) .
We provide a proof of Fact 1 in Appendix B.
In this paper, we state our convergence rates in terms of the tangent residual r tanF,A ( z ) , which implies (i) convergence rates in terms of the natural residual r natF,A ( z ) , and (ii) z is an approximate solution to (CMI) or (MI) .
3. Optimal Monotone Inclusion via EAG
In this section, we study the constrained single-valued monotone inclusion problem (MI) with closed convex feasible set Z ⊆ R n and monotone and L -Lipschitz operator F , as summarized in Assumption 1.
We analyse the (projected) Extra Anchored Gradient Method (EAG) , which is proposed by [37] in the unprojected form for Z = R n .
Let z 0 ∈
Z be an arbitrary starting point and { z k , z k + 12 } k ≥ 0 be the iterates of (EAG) with step size η > 0 , whose update rule is as follows:
[Equation]
Theorem 1 Suppose Assumption 1 holds.
Let z 0 ∈
Z be any starting point and { z k , z k + 12 } k ≥ 0 be the iterates of (EAG) with step size η ∈ (0 , 1 √ 3 L ) .
Denote D := ∥
z 0 − z ∗ ∥ 2 .
Then for any T ≥ 1 , r tanF, Z ( z T ) 2 ≤ η 2 L 2 (1 − 3 η 2 L 2 ) ·
D 2 L
T
We prove a more general version of Theorem 1 in Appendix C.3 where we also provide convergence rates for the Minty Variational Inequality and the Stampacchia Variational Inequality problem.
4. Accelerated Algorithm for Inclusion Problems with Negatively Comonotone Operators
In this section, we focus on the inclusion problem (CMI) with a single-valued L -Lipschitz (possibly non-monotone) operator F and a set-valued maximally monotone operator A , where F + A satisfies ρ -comonotonicity with ρ > − 12 L (Assumption 2).
We propose an accelerated forward-backward splitting algorithm (AS) that is applicable to any maximally monotone operator A .
Our algorithm (AS) generalizes previous algorithms such as EAG and FEG, which are limited to the unconstrained setting.
We show that (AS) enjoys the optimal convergence rate of O ( 1 T ) via a potential function argument.
In splitting methods, the update steps in each iteration of can be categorized as the forward steps , where they involve the forward evaluation of a single-valued operator, or backward steps , where they require computing the resolvent of an operator.
Accelerated Forward-Backward Splitting (AS).
Given any initial point z 0 ∈ R n and step size η > 0 , (AS) sets c 0 = 0 and updates { z k + 12 , z k +1 , c k +1 } k ≥ 0 as follows: for k ≥ 0 ,
[Equation]
Note that by definition we have c k ∈ A ( z k ) for all k ≥ 1 .
Our algorithm is inspired by FEG [20].
In particular, when A ( z ) = 0 for all z , i.e., the unconstrained setting,
c k is always 0 , and our algorithm is identical to FEG.
It is worth noting that (AS) only requires the computation of the resolvent for operator A and only computes it once in each iteration.
When A is the subdifferential of the indicator function of a closed convex set Z , the resolvent operator is exactly the projection to Z .
Compared to (EAG) , (AS) has an additional feature that it only performs one projection (rather than two) in each iteration when applied to (MI) .
Kim [17] achieves the same accelerated rate in the monotone setting, i.e.,
F + A is monotone, by assuming the ability to compute the resolvent of the operator F + A (i.e., the accelerated proximal point method) or the resolvents of both F and A (i.e., the Accelerated
Douglas-Rachford method).
These resolvents could be substantially more difficult to compute in many applications than the resolvent of A , for example, when A = ∂
I Z .
We prove Theorem 2 in
Appendix D.1.
Theorem 2 Suppose Assumption 2 holds for some ρ ∈ ( − 12 L , 0] .
Let z 0 ∈ R n be any starting point and { z k , z k + 12 } k ≥ 1 be the iterates of (AS) with step size η ∈ ( max (0 , − 2 ρ ) , 1 L ) .
Then for any T ≥ 1 ,
[Equation]
Remark 3 To interpret the convergence rate, one can think of a properly selected η such that ( η + 2 ρ ) L is an absolute constant, and the rate is O
( H 20 L 2 T 2 ) .
Introduction
Our Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Preliminaries
Monotone Inclusion and Variational Inequality . . . . . . . . . . . . . . . . . . .
Inclusion Problems with Negatively Comonotone Operators. . . . . . . . . . . . .
Convergence Criteria . .
Optimal Monotone Inclusion via EAG
Accelerated Algorithm for Inclusion Problems with Negatively Comonotone Operators
A Detailed Related Works
A.1 Convex-Concave and Monotone Settings . . . . . . . . . . . . . . . . . . . . . . .
A.2 Nonconvex-Nonconcave Min-Max Optimization and Inclusions with Non-Monotone
Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B Additional Preliminaries
C Missing Proofs in Section 3: Optimal Monotone Inclusion via EAG
C.
Warm Up: Unconstrained Case
C
Convergence of EAG with Arbitrary Convex Constraints
C.3 Proof of Theorem
D Missing Proofs in Section 4: Accelerated Algorithm for Inclusion Problems with Negatively Comonotone Operators
D.1 Proof of Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
E Auxiliary Propositions
Appendix A. Detailed Related Works
There is a vast literature on inclusion problems and variational inequalities, e.g., see [2, 12, 31] and the references therein.
We only provide a brief discussion of the most relevant and recent results.
A.1. Convex-Concave and Monotone Settings Convergence in Gap Function.
Nemirovski and Nesterov [26, 27] show that the average iterate of extragradient-type methods has O ( 1 T ) convergence rate in terms of gap function defined as max z ′ ∈Z ⟨ F ( z ′ ) , z − z ′ ⟩ , which means that their result only provides an approximate solution to the weak solution.
The O ( 1 T ) rate is optimal for first-order methods due to the lower bound by Ouyang and Xu [28].
Convergence
of the Extragradient Method in Stronger Performance Measures.
For stronger performance measures such as the norm of the operator (when Z = R n ) or the residual (in constrained setting), classical results [12, 19] show that the best-iterate of the extragradient method converges at a rate of O ( 1 √ T ) .
Recently, the same convergence rate is shown to hold even for the last-iterate of the extragradient method [5, 15].
Although O ( 1 √ T ) convergence on the residual is optimal for all p -SCIL algorithms [13], a subclass of first-order methods that includes the extragradient method and many of its variations, faster rate is possible for other first-order methods.
Faster Convergence Rate in Operator Norm or Residual.
We provide a brief overview of results that achieve faster convergence rate in terms of the operator norm or residual.
Note that these results also imply essentially the same convergence rate in terms of the gap function.
The literature here is rich and fast-growing, we only discuss the ones that are close related to our paper.
Recent results show accelerated rates through Halpern iteration [16] or a similar mechanism – anchoring.
Implicit versions of Halpern iteration have O ( 1 T ) convergence rate [17, 21, 29] for monotone operators and explicit variants of Halpern iteration achieve the same convergence rate when F is cocoercive [9, 17].
Diakonikolas [9] also provide a double-loop implementation of the algorithm for monotone operators at the expense of an additional logarithmic factor in the convergence rate.
Yoon and Ryu [37] propose the extra anchored gradient (EAG) method, which is the first explicit method with accelerated O ( 1 T ) rate in the unconstrained setting for monotone operators.
They also established a matching Ω( 1 T ) lower bound that holds for all first-order methods.
Convergence analysis of past extragradient method with anchoring in the unconstrained setting is provided in [36].
Lee and Kim [20] proposed a generalization of EAG called fast extraradient (FEG), which applies to comonotone operators and improves the constants in the convergence rate, but their result only applies to the unconstrained setting.
Very recently, Tran-Dinh [35] studies the connection between
Halpern iteration and Nesterov accelerated method, and provides new algorithms for monotone operators and alternative analyses for EAG and FEG in the unconstrained setting.
In Theorem 1, we show the projected version of EAG has O ( 1 T ) convergence rate under arbitrary convex constraints, achieving the optimal convergence rate for all first-order methods in the constrained setting.
A.2. Nonconvex-Nonconcave Min-Max Optimization and Inclusions with Non-Monotone Operators
Many practical applications of min-max optimization in modern machine learning, such as GANs and multi-agent reinforcement learning, are nonconvex-nonconcave.
Without any additional structure, the problem is intractable [8].
Hence, recent works study nonconvex-nonconcave min-max optimization problems under several structural assumptions.
We only introduce the definitions in the unconstrained setting, as that is the setting considered by several recent results, and all convergence rates are with respect to the the operator norm.
The Minty variational inequality (MVI) condition (also called coherence or variationally stable): there exits z ∗ such that
[Equation]
is studied in e.g., [7, 22, 23, 25, 34, 38].
Extragradient-type algorithms has O ( 1 √ T ) convergence rate for Lipschitz operators that satisfy the MVI condition [7].
Diakonikolas [10] proposes a weaker condition called weak MVI : there exits z ∗ and ρ < 0 such that
[Equation]
The weak MVI condition includes both MVI and negative comonotonicity [3] as special cases.
Diakonikolas [10] proposes the EG+ algorithm, which has O ( 1 √ T ) convergence rate under the weak
MVI condition in the unconstrained setting.
Recently, Pethick et al [30] generalized EG+ to CEG+ algorithm which has O ( 1 √ T ) convergence rate under weak MVI condition in general (constrained) setting.
The result for accelerated algorithms in the nonconvex-nonconcave setting is sparser.
FEG achieves O ( 1 T ) convergence rate for comonotone operators in the unconstrained setting [20].
In general (constrained) setting with comonotone operators, implicit methods such as the proximal point algorithm has been shown to converge [3, 18].
To the best of our knowledge, (AS) is the first explicit and efficient method that achieves the accelerated and optimal O ( 1 T ) convergence rate in the constrained nonconvex-nonconcave setting (Theorem 2).
We summarize previous results and our results in Table 1.
Appendix B. Additional Preliminaries
Maximally Monotone Operator.
When A is maximally monotone, useful properties of J A (See e.g., [31, 33]) include:
J A is well-defined on R n ; 2.
J A is non-expansive thus single-valued; 3.
when z =
J A ( z ′ ) , then z ′ − z
∈
A ( z ) ; 4.
when A = ∂ I Z is the normal cone operator of a closed convex set Z ⊆ R n , then J ηA =
Π Z is the projection operator for all η > 0 .
Monotone Inclusion
Remark
We remind readers that in general, monotone inclusion problem is to find z ∗ ∈ R n such that 0 ∈ A ( z ∗ ) , where A is a set-valued maximally monotone operator.
To ease notation, we sometimes refer to the constrained single-valued monotone inclusion problem stated in (MI) as the monotone inclusion problem .
Variational Inequality.
A closely related problem to (MI) is the monotone variational inequality (VI) with operator F and feasible set Z , which has two variants.
The Stampacchia Variational
Inequality (SVI) problem is to find z ∗ ∈
Z such that
[Equation]
Such z ∗ is called a strong solution to VI.
The Minty Variational Inequality (MVI) problem is to find z ∗ ∈
Z such that
[Equation]
Such z ∗ is called a weak solution to VI.
When F is continuous, then every solution to (MVI) is also a solution to (SVI).
When F is monotone, every solution to (SVI) is also a solution to (MVI) and thus the two solution sets are equivalent.
Moreover, the solution set to (MI) is the same as the solution set to (SVI).
Approximate Solutions.
We say z ∈ Z is an ϵ -approximate solution to (MI) if
[Equation]
where we use B ( u, r ) to denote a ball in R n centered at u with radius r .
We say z ∈ Z is an ϵ -approximate solution to (SVI) or (MVI) if
[Equation]
When F is monotone, it is clear that every ϵ -approximate solution to (SVI) is also an ϵ -approximate solution to (MVI); but the reverse does not hold in general.
When F is monotone and Z is bounded by D , then any ϵD -approximate solution to (MI) is an ϵ -approximate solution to (SVI) [9, Fact 1].
Note that when Z is unbounded, neither (SVI) nor (MVI) can be approximated.
If we restrict the domain to be a bounded subset of (possibly unbounded) Z , then we can define the (restricted) gap functions as
[Equation]
The O ( 1 T ) convergence rate for extragradient-type algorithm [26, 27] is provided in terms of G AP MV I F,D ( z ) , which means convergence to an approximate weak solution.
Prior to our work, the O ( 1 T ) convergence rate on G AP SV I F,D ( z ) was only known in the unconstrained setting [37].
When F is monotone, then the tangent residual r tanF,D ( z ) ≤ ϵD (definition in section 2.3) implies G AP SV I F,D ( z ) ≤ ϵ [5,
Lemma 2].
Therefore, our result also implies an O ( 1 T ) convergence rate on G AP SV I F,D ( z ) when Z is arbitrary convex set (Theorem 1).
Example 1 (Gap function is weaker than natural residual) Consider an instance of the Monotone VI problem on the identity operator F ( x )
= x in Z = [0 , 1] .
• Observe that the natural residual on x ∈ Z is ∥ x − Π Z [ x − F ( x )]
∥ = x . • Moreover, since Z = [0 , 1] , observe that for any x ∈ Z and D ≥ 0 ,
[Equation]
As a result, any algorithm with O ( 1 T ) convergence rate with respect to the gap function only implies a O ( 1 √ T ) convergence rate for the corresponding (MI) or the natural residual.
Inclusion Problems with Negatively Comonotone Operators
Example 2 (Min-Max Optimization)
The following structured min-max optimization problem captures a wide range of applications in machine learning such as GANs, adversarial examples, robust optimization, and reinforcement learning:
[Equation]
where f ( · , · ) is possibly non-convex in x and non-concave in y .
Regularized and constrained minmax problems are covered by appropriate choices of lower semicontinuous and convex functions g and h .
Examples include ℓ 1 -norm, ℓ 2 -norm, and indicator function of a convex feasible set.
Let z
=
( x, y ) , if we define F ( z )
=
( ∂ x f ( x, y ) , − ∂ y f ( x, y )) and A ( z )
=
( ∂g ( x ) , ∂h ( y )) , where A is maximally monotone, then the first-order optimality condition of (1) has the form of (CMI) .
See [20, Example 1] for examples of nonconvex-nonconcave conditions that are implied by negative comonotonicity.
Proof
of Fact
Proof For any c ∈ A ( z ) , we have
[Equation]
(non
[Equation]
expansiveness of J A )
Thus r natF,A ( z ) ≤ min c ∈ A ( z )
∥ F ( z ) + c ∥ = r tanF,A ( z ) .
Appendix C. Missing Proofs in Section 3: Optimal Monotone Inclusion via EAG
Our analysis is based on the following potential function V k : for k ≥ 1 ,
[Equation]
where
[Equation]
From the update rule of (EAG) , we know c k ∈ N Z
( z k ) .
Thus ∥ F ( z k ) + c k ∥ ≥ min c ∈ N Z ( z k ) ∥ F ( z k ) + c ∥ = r tan ( z k ) .
In Theorem 9, we show V k is “approximately” non-increasing, i.e., V k +1 ≤ V k
+ O (1) · ∥ ηF ( z k +1 )
+ ηc k +1 ∥ 2 for all k ≥ 1 .
From this property, we get O ( 1 T ) last-iterate convergence rate in terms of ∥ F ( z T ) + c T ∥ and thus the same convergence rate in r tan ( z T ) (Theorem 1).
Potential functions in a similar form as V k have been used to analyse (MI) by Diakonikolas [9] for the Halpern iteration algorithm, and by Yoon and Ryu [37] for (EAG) in the unconstrained setting ( Z = R n ).
We emphasize that we use a different potential function with different analysis.
Diakonikolas [9] studied the Halpern iteration with operator P :
= I − J F + ∂ I Z , which is 12cocoercive but can not be computed efficiently in general.
She showed that the follwoing potential function is non-increasing.
[Equation]
which leads to O ( 1 T ) -approximate solution to (MI) after T iterations.
However, since P can not be computed efficiently in general, the algorithm needs O (log( 1 ϵ )) oracle queries for an O ( ϵ ) - approximate value of P in each iteration, thus total oracle complexity O ( LDϵ · log( 1 ϵ )) for an ϵapproximate solution to (MI) .
In contrast, we use operator F in the potential function V k , and we prove V k is only ”approximately” non-increasing (see Theorem 5 and 9).
Moreover, (EAG) needs only 2 oracle queries in each iteration and achieves optimal O ( LDϵ ) oracle complexity for an ϵapproximate solution to (MI) (Theorem 1) matching the lower complexity bound Ω( LDϵ ) [9].
Yoon and Ryu [37] studied convergence of (EAG) for (MI) in the unconstrained setting ( Z =
R n ).
The specific algorithm they analysed uses anchoring term 1 k +2 ( z 0 − z k ) while we use 1 k +1 ( z 0 − z k )
(see Remark 10 for more discussion on the choice of the constant in the anchoring term).
They use the following potential function
[Equation]
where B k = k + 1 , and A k
= O ( k 2 ) is updated adaptively in a sophisticated way for each k .
Their potential function P k is more complicated compared to V k as we choose B k
= k and A k = k ( k +1) .
For the analysis, their proof of the monotonicity of P k is relatively involved.
In contrast, we use a simple proof to show that V k is “approximately” non-increasing (Theorem 5) which suffices to establish the O ( 1 T ) convergence rate.
Moreover, our analysis can be naturally extended to the constrained setting where Z ⊆ R n is an arbitrary closed convex set (Theorem 9).
C.1. Warm Up: Unconstrained Case
We begin with the unconstrained setting Z
= R n , which illustrates our main idea and proof techniques.
Yoon and Ryu [37] also analyse the unconstrained setting but our proof is much simpler.
In the unconstrained setting, c k = 0 by definition.
Thus
[Equation]
It is not hard to see that V 1 ≤ ( η 2 L 2 + 2 ηL )
∥
z 0 − z ∗ ∥ 2 : since the update rule for z 12 and z 1 of (EAG) coincides with the update rule of EG, by [5, Theorem 1], we have ∥ ηF ( z 1 ) ∥ 2 ≤ ∥ ηF ( z 0 ) ∥ 2 ≤ η 2 L 2 ∥
z 0 − z ∗ ∥ 2 ; by [19] and [12, Lemma 12.1.10 ], we have ∥
z 1 − z ∗ ∥ ≤ ∥ z 0 − z ∗ ∥
[Equation]
Theorem 5 Suppose Assumption 1 holds with Z = R n .
Then for any k ≥ 1 , (EAG) with any step size η ∈ (0 , 1 L ) satisfies V k +1 ≤ V k
+ η 2 L 2 1 − η 2 L 2 ∥ ηF ( z k +1 )
∥ 2 .
Proof Since F is monotone and L -Lipschitz, we have the following inequalities
[Equation]
and
[Equation]
[Table]
We simplify them using the update rule of (EAG) .
In particular
[Equation]
we replace z k − z k +1 with ηF ( z k + 12 ) − 1 k +1 ( z 0 − z k ) and z k + 12 − z k +1 with ηF ( z k + 12 ) − ηF ( z k ) .
[Equation]
It is not hard to verify that the following identity holds
[Equation]
Note that k +12 k ≤ 1 holds for all k ≥ 1 .
Thus, V k +1 ≤ V k + η 2 L 2 1 − η 2 L 2 ∥ ηF (
z k +1 )
∥ 2 .
Lemma
For all k ≥ 2 ,
[Equation]
Moreover, when η ∈ (0 , 1 √ 3 L ) , we have
[Equation]
Proof Fix any k ≥ 2 .
By definition, we have
[Equation]
Using Theorem 5, we have
[Equation]
Combining the two inequalities above and the fact that V 1 ≤ (2 ηL + η 2 L 2 )
∥
z 0 − z ∗ ∥ 2 yields the first inequality in the statement.
When ηL ∈ (0 , 1 √ 3 ) , we have η 2 L 2 1 − η 2 L 2 ≤ 12 ≤ k 4 for k ≥ 2 .
Hence the second inequality in the statement holds.
Theorem 7 Suppose Assumption 1 holds with Z = R n .
Let z 0 ∈ R n be arbitrary starting point and { z k , z k + 12 } k ≥ 0 be the iterates of (EAG) with any step size η ∈ (0 , 1 √ 3 L ) .
Denote
D := ∥
z 0 − z ∗ ∥ .
Then for any T ≥ 1 ,
[Equation]
If we set η = 13 L , then ∥ F ( z T ) ∥ 2 ≤
[Equation]
D 2 L 2 T.
Proof Note that the second inequality is implied by the first inequality since G AP SV I F,D ( z ) ≤ D · ∥ F ( z T ) ∥ [5, Lemma 2].
Denote a k :=
∥ ηF ( z k ) 2 ∥ ∥
z 0 − z ∗ ∥ 2 .
It suffices to prove for all k ≥ 1 ,
[Equation]
Since the update rule for z 12 and z 1 of (EAG) coincides with the update rule of EG, by [5, Theorem 1], we have ∥ ηF ( z 1 ) ∥ 2 ≤ ∥ ηF ( z 0 ) ∥ 2 ≤ η 2 L 2 ∥
z 0 − z ∗ ∥ 2 and thus a 1 ≤ η 2 L 2 < 13 .
Thus (5) holds for k = 1 .
From Lemma 6, we know for k ≥ 2 ,
[Equation]
Applying Proposition 16 with C 1 = (1 + ηL ) 2 and p =
η 2 L 2 < 13 completes the proof.
C.2. Convergence of EAG with Arbitrary Convex Constraints
In this section, we show how to extend the analysis in the unconstrained setting to the arbitrary convex constrained setting.
Recall that the potential function V k and c k are defined in (2).
We use the fact that c k ∈ N Z
( z k ) extensively.
Proposition 8 V 1 ≤ (1+ ηL + η 2 L 2 )(2+2 ηL + η 2 L 2 ) 1 − η 2 L∥ z 0 − z ∗ ∥ 2 .
Proof We first upper bound ∥ ηF ( z 1 ) + ηc 1 ∥ and ∥ z 1 − z 0 ∥ .
Note that z 12 , z 1 are updated exactly as original EG.
By definition, we have
[Equation]
where in the last inequality we use a well-known result regarding EG [
[Equation]
Lemma 12.1.10 ]: ∥
[Equation]
Note that in the above sequence of inequalities, we also prove that ∥ z 1 − z
∥ ≤
[Equation]
By definition of V 1 and the above upper bound for ∥ ηF ( z 1 ) + ηc 1 ∥ and ∥ z 1 − z 0 ∥ , we have
[Equation]
Theorem 9 Suppose Assumption 1 holds.
Let z 0 ∈
Z be any starting point and { z k , z k + 12 } k ≥ 0 be the iterates of (EAG) with step size η ∈ (0 , 1 L ) .
Then for any k ≥ 1 , V k +1 ≤
[Equation]
∥ ηF (
[Equation]
k
[Equation]
+ ηc k +1 ∥ 2 , where V k and c k are defined in (2) .
Proof We first present several inequalities.
From the monotonicity and L -Lipschitzness of F , we have
[Equation]
Since z k + 12 = Π Z [ z k − ηF ( z k ) + 1 k +1 ( z 0 − z k )]
[Equation]
we can infer that z k − ηF ( z k ) + 1 k +
( z 0 − z k ) − z k + 12 ∈ N Z
( z k + 12 ) .
Moreover, by definition of c k and c k +1 , we know c k ∈ N Z
( z k ) and c k +1 ∈ N Z
( z k +1 ) .
Therefore, we have
[Equation]
The following identity holds when we substitute ηc k +1 on both sides using
ηc k +1 =
z k − ηF ( z k + 12 )+ 1 k +1 ( z 0 − z k ) − z k +1 , which follows from the definition.
The correctness of the identity follows from
Identity (19) in Proposition 15: we treat x 0 as z 0 ; x t as z k + t − 12 for t ∈ { 1 , 2 , 3 } ; y t as ηF ( z k + t − 12 ) for t ∈ { 1 , 2 , 3 } ; u 1 as ηc k and u 3 as ηc k +1 ; p as η 2 L 2 and q as k .
[Equation]
Since ∥ a ∥ 2 + ⟨ a, b ⟩ = ∥ a + b 2 ∥ 2 − ∥ b ∥ 2 4 , we have
Expression (12) + Expression (13)
[Equation]
where A = (cid:113) (1 − η 2 L 2 ) k
( k +1) 2 η 2 Land B = (cid:113) η 2 L 2 ( k +1)
2(1 − η 2 L 2 ) k .
Since k ≥ 1 , we have k +12 k ≤ 1 .
Hence, we have V k +1 ≤ V
[Equation]
+
[Equation]
Remark 10 The proof of Theorem 9 naturally extends to the following algorithm and potential function: Fix any z 0 ∈ Z and η ∈ (0 , 1 L ) , δ ≥ 0 .
Update z 12 , z 1 , c 1 , V 1 as (EAG) and for k ≥ 1 :
[Equation]
Since the identity in Proposition 15 holds for any q ̸ = 0 , we only need to change every k to be k + δ in the proof of Theorem 9.
It is possible that a choice of δ > 0 leads to a better upper bound (better constant) than δ = 0 which is chosen for (EAG) , but we do not optimize over δ here.
Lemma
For k ≥ 2 ,
[Equation]
Moreover
[Equation]
when η ∈ (0 , 1 √ 3 L )
[Equation]
then
[Equation]
Proof Fix any k ≥ 2 .
By definition
[Equation]
we have
[Equation]
F
[Equation]
∂ I Z is monotone and 0 ∈ F
[Equation] [Equation]
According to Theorem
[Equation]
V t +1 − V t ≤
[Equation]
2 L 2 ∥ ηF
[Equation]
z t +1 ) + ηc t +1 ∥ 2 for all t ≥ 1 .
Through a telescoping sum, we obtain the following inequality:
[Equation]
The first inequality in the statement follows from the combination of Inequality (14) and (15).
The second inequality in the statement follows from the fact that η 2 L 2 1 − η 2 L 2 ≤ 1 2 ≤ k 4 when η ∈ (0 , 1 √ 3 L ) .
C.3. Proof of Theorem
Theorem 12 [More general version of Theorem 1.]
Suppose Assumption 1 holds.
Let z 0 ∈
Z be any starting point and { z k , z k + 12 } k ≥ 0 be the iterates of (EAG) with step size η ∈ (0 , 1 √ 3 L ) .
Denote
D := ∥
z 0 − z ∗ ∥ 2 .
Then for any T ≥ 1 ,
[Equation]
Proof
The bound on G AP SV I F,D
( z T ) follows from the bound on r tanF,A ( z T ) since G AP SV I F,D ( z T ) ≤ D · r tanF,A ( z T ) [5, Lemma 2].
Denote a k
[Equation]
It suffices to prove that for all k ≥ 1 ,
[Equation] [Equation]
Note that from the proof of Proposition 8, we have
[Equation]
which implies a 1 ≤ (1+ ηL + η 2 L 2 )
1 − η 2 L≤ 6 .
Thus (16) holds for k = 1 .
From (8), we also have
[Equation]
Thus by Lemma 11, we have
[Equation]
⇒
[Equation]
Applying Proposition 16 with C 1 = 11 and p =
η 2 L 2 ∈ (0 , 13 ) , we have (16) holds for any k ≥ 2 .
This completes the proof.
Appendix D. Missing Proofs in Section 4: Accelerated Algorithm for Inclusion Problems with Negatively Comonotone Operators
To analyze (AS), we adopt the following potential function: for k ≥ 1 ,
[Equation]
k
[Equation]
The potential function is the same as the one used in the analysis for FEG [20] when c k is always 0 , and we properly adapted for non-zero c k ’s. The convergence analysis builds on the following two properties of the potential function: Proposition 13 establishes an upper bound of U 1 ; Lemmashows U k +1 ≤ U k for all k ≥ 1 .
Proposition 13 U 1 ≤ (1+ ηL )(3+ ηL )
[Equation]
Proof Note that z 12 =
z 0 and z 1 =
J ηA [ z 0 − ηF ( z 0 )] .
Thus we have ηc 1 =
z 0 − ηF ( z 0 ) − z 1 .
We first bound ∥ ηF ( z 1 ) + ηc 1 ∥ as follows:
[Equation]
Then we can bound U 1 as follows:
[Equation]
Cauchy-Schwarz
[Equation] [Equation]
Moreover
[Equation]
for any c ∈ A ( z 0 )
[Equation]
we have
[Equation]
Hence ∥ z 1 − z
∥ ≤ η · min
[Equation]
∈ A ( z 0 )
∥ F ( z
[Equation]
+ c ∥
[Equation]
η ·
[Equation]
tanF
[Equation]
A (
[Equation]
Lemma
Assume Assumption 2 holds for some ρ .
Let z 0 ∈ R n be any initial point and { z k , z k + 12 } k ≥ 1 be the iterates of (AS) with step size η > − 2 ρ .
Then for all k ≥ 1 , we have
U k +1 ≤ U k .
Proof Fix any k ≥ 1 .
We first present several inequalities.
Since F is L -Lipschitz, we have
[Equation]
Additionally, since F + A is ρ -comonotone, c k ∈ A ( z k ) , and
c k +1 ∈ A ( z k +1 ) , we have
[Equation]
The following identity holds due to Identity (20) in Proposition 15: we treat x 0 as z 0 ; x t as z k + t − 1 2 for t ∈ { 1 , 2 , 3 } ; y t as ηF ( z k + t − 1 2 ) for t ∈ { 1 , 2 , 3 } ; u 1 as ηc k , and u 3 as ηc k +1 ; p as η 2 L 2 , q as k , and r as ρη .
Note that by the update rule of (AS) , we have ηc k
= z k +
[Equation]
and by definition, we have ηc
k +1 =
z k + 1 k
+
( z 0 − z k ) − ηF ( z k + 12 ) − 2 k ρη k +1 · ( ηF ( z k )+ ηc k ) − z k +1 .
[Equation]
Hence, U k +1 ≤ U k .
D.1. Proof of Theorem
Fix any T ≥ 1 .
According to Lemma 14, we have U T ≤ U 1 .
Then by definition of U T , we have
[Equation]
In the first inequality, we use the fact that z ∗ is a solution of the (CMI) with the ρ -comonotone operator F + A .
In the second inequality, we use ⟨ a, b ⟩ ≥ − δ 4 ∥ a ∥ 2 − 1 δ ∥ b ∥ 2 for δ > 0 .
By Proposition 13 and the assumption that ηL < 1 , U 1 ≤ (1+ ηL )(3+ ηL )·
[Equation]
4 ∥ z 1 − z
∥ 2 ≤ 4 L 2 · r tan ( z 0 ) 2 .
Therefore,
[Equation]
Appendix E. Auxiliary Propositions
Proposition 15 Let x 0 , x 1 , x 2 , x 3 , y 1 , y 2 , y 3 , u 1 , u 3 be arbitrary vectors in R n .
Let q > 0 , p > 0 , and r ̸ = − 12 be real numbers.
If u 3 = x 1 − y 2 + 1 q +1 ( x 0 − x 1 ) − x 3 , then the following identity holds:
[Equation]
If u
[Equation]
x 1 + 1 q +1 ( x 0 − x
[Equation]
− qq +1 (1+2 r
[Equation]
y 1 − xqq +
(1+2 r ) and u
3 = x 1 + 1 q +1 ( x 0 − x [Equation] − y 2 −
[Equation]
then the following identity holds:
[Equation]
Proof We verify the two identities using M ATLAB .
Readers can find the verification code here or go to the next url: https://github.com/weiqiangzheng1999/Accelerated-Non-Monotone-Inclusion .
Proposition 16 Let { a k ∈ R + } k ≥ 2 be a sequence of real numbers.
Let C 1 ≥ 0 and p ∈ (0 , 13 ) be two real numbers.
If the following condition holds for every k ≥ 2 ,
[Equation] [Equation]
then for each k ≥ 2 we have
[Equation]
Proof We prove the statement by induction.
Base Case: k = 2 .
From Inequality (21), we have
[Equation]
Thus, Inequality (22) holds for k = 2 .
Inductive Step: for any k ≥ 3 .
Fix some k ≥ 3 and assume that Inequality (22) holds for all 2 ≤ t ≤ k − 1 .
We slightly abuse notation and treat the summation in the form (cid:80) 2 t =3 as 0.
By
Inequality (21), we have
[Equation]
Induction assumption on Inequality (22))
[Equation]
This complete the inductive step.
Therefore, for all k ≥ 2 , we have a k ≤ 4 · C 1 1 − 3 p · 1 k 2 .