Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions
Abstract
We study stochastic monotone inclusion problems, which widely appear in machine learning applications, including robust regression and adversarial learning.
We propose novel variants of stochastic Halpern iteration with recursive variance reduction.
In the cocoercive—and more generally Lipschitz-monotone—setup, our algorithm attains ✏ norm of the operator with O ( 1 ✏ 3 ) stochastic operator evaluations, which signiﬁcantly improves over state of the art O ( 1 ✏ 4 ) stochastic operator evaluations required for existing monotone inclusion solvers applied to the same problem classes.
We further show how to couple one of the proposed variants of stochastic Halpern iteration with a scheduled restart scheme to solve stochastic monotone inclusion problems with O ( log(1 / ✏ ) ✏) stochastic operator evaluations under additional sharpness or strong monotonicity assumptions.
Introduction
Recent trends in machine learning (ML) involve the study of models whose solutions do not reduce to optimization but rather to equilibrium conditions .
Standard examples include generative adversarial networks, adversarially robust training of ML models, and training of ML models under notions of fairness.
It turns out that several of these equilibrium conditions (including, but not limited to, ﬁrst-order stationary points, saddle-points, and Nash equilibria of minimax games) can be cast as solutions to a monotone inclusion problem, which is deﬁned as the problem of computing a zero of a (maximal) monotone operator F : R d !
R d (see (MI) for a formal deﬁnition).
In the context of min-max optimization problems, monotone inclusion reduces to a stationarity condition, which for unconstrained problems boils down to ﬁnding a point with small gradient norm.
Of particular interest to machine learning are stochastic versions of these problems, in which the operator F is not readily available, but can only be accessed through a stochastic oracle b
F .
Such are the settings mentioned above, where the deﬁnitions of equilibria involve expectations over continuous high-dimensional spaces.
The corresponding problem, known as the stochastic monotone inclusion , has not been thoroughly studied, particularly in the context of its stochastic oracle complexity.
Understanding stochastic oracle complexity of monotone inclusion in all standard settings with
Lipschitz operators, from the algorithmic aspect, is the main motivation of this work.
36th
1.1 Contributions
We study three main classes of stochastic monotone inclusion problems with Lipschitz operators, deﬁned by the assumptions made about the operator itself: (i) cocoercive class, which is the most restricted class, but nevertheless fundamental for understanding monotone inclusion, as it relates to the problem of ﬁnding a ﬁxed point of a nonexpansive (1-Lipschitz) operator; (ii) Lipschitz monotone class, which is perhaps the most basic class arising in the study of smooth convex-concave min-max optimization problems; and (iii) Lipschitz monotone class with an additional sharpness property of the operator.
Sharpness is a widely studied property of optimization problems, often referred to as the “local error bound” condition, which is weaker than strong convexity and roughly corresponds to the problem landscape being curved outside of the solution set (see [41] for a survey of classical results).
From an algorithmic standpoint, we consider variants of classical Halpern iteration [21], which was originally introduced for solving ﬁxed point equations with nonexpansive operators.
Variants of this iteration have recently been shown to lead to (near-)optimal ﬁrst-order oracle complexity for all aforementioned standard problem classes in deterministic settings [12, 13, 53].
However, to the best of our knowledge, stochastic variants of these methods have received very limited attention prior to our work.
The only results we are aware of are for a two-step extragradient-like variant of
Halpern iteration in negative comonotone Lipschitz settings [29] and which show that when variance of operator estimates is bounded by order- ✏ 2 k in iteration k, the method attains operator norm ✏ after O ( 1 ✏ ) iterations.
However, [29] does not discuss how such variance control would be obtained.
Simple mini-batching, as we show, only leads to O ( 1 ✏ 4 ) stochastic oracle complexity.
We show that existing variants of the Halpern iteration [12, 51] can be effectively combined with recursive variance reduction [32] to obtain O ( 1 ✏ 3 ) stochastic oracle complexity in the cocoercive and
Lipschitz monotone setups.
We then show that the complexity can be further reduced to O  1 ✏ 2 log( 1 ✏ )  under an additional sharpness assumption about the operator.
The last bound is unimprovable in terms of the dependence on ✏ , due to existing lower bounds, as we argue for completeness in Appendix A.
To the best of our knowledge, our work is the ﬁrst to use variance reduction to reduce stochastic oracle complexity of monotone inclusion (small gradient norm in min-max optimization settings), and the attained bounds are the best achieved to date for direct methods.
1.2 Techniques
Inspired by the potential function originally used by [12] and later used either in the same or slightly modiﬁed form by [13, 29, 51, 53], we adapt this potential function-based argument to account for stochastic error terms arising due to the stochastic oracle access to the operator.
We ﬁrst show that in the cocoercive minibatch setting, this argument only leads to O ( 1 ✏ 4 ) stochastic oracle complexity, and it is unclear how to improve it directly, as the analysis appears tight.
We then combine the cocoercive variant of Halpern iteration [12] with the PAGE estimator [32] to reduce the stochastic oracle complexity to O ( 1 ✏ 3 ) .
The same variance reduced estimator is also used in conjunction with the two-step extrapolated variant of Halpern iteration introduced by [51], as a direct application of
Halpern iteration is not known to converge on the class of Lipschitz monotone operators.
While the basic ideas in our arguments are simple, their realization requires addressing major technical obstacles.
First, the variance reduced estimator that we use [32] was originally devised for smooth nonconvex optimization problems, where it was coupled with a stochastic variant of gradient descent.
This is signiﬁcant, because the proof relies on a descent lemma, which allows cancelling the error arising from the variance of the estimator by the “descent” part.
Such an argument is not possible in our setting, as there is no objective function to descend on.
Instead, our analysis relies on an intricate inductive argument that ensures that the expected norm of the operator is bounded in each iteration, assuming a suitable bound on the variance of the estimator.
To obtain our desired result for the variance, we propose a data-dependent batch allocation in PAGE estimator [32] (see Corollary 2.2), which scales proportionally to the squared distance between successive iterates, similar to [3].
We inductively argue that the squared distance between successive iterates arising in the batch size of the estimator reduces at rate 1 k 2 in expectation .
This allows us to further certify that the estimators do not only remain accurate, but their variance decreases as O ( ✏ 2 /k ) , where k is the iteration count.
In the context of the potential function argument, unlike in the deterministic settings, we do not establish that the potential function is non-increasing, even in expectation.
The stochastic error terms that arise due to the stochastic nature of the operator evaluations are controlled by taking slightly smaller step sizes than in the vanilla methods from [12, 51], which allows us to “leak” negative quadratic terms that are further used in controlling the stochastic error.
The argument for controlling the value of the potential function is itself coupled with the inductive argument for ensuring that the expected operator norm remains bounded.
Finally, while applying a restarting strategy is standard under sharpness conditions [46], obtaining the claimed stochastic oracle complexity result of O
[Equation]
requires a rather technical argument to bound the total number of stochastic queries to the operator.
1.3 Related work
Monotone inclusion and variational inequalities.
Variational inequality problems were originally devised to deal with approximating equilibria.
Their systematic study was initiated by [50].
The relationship between variational inequalities and min-max optimization was observed soon after [44], while one of the earliest papers to study solving monotone inclusion as a generalization of variational inequalities, convex and min-max optimization, and complementarity problems is [45].
For a historical overview of this area and an extensive review of classical results, see [17].
In the case of monotone operators, standard variants of variational inequality problems (see Section 2) and monotone inclusion are equivalent—their solution sets coincide.
This is a consequence of the celebrated Minty Theorem [35].
However, there is a major difference between these problems when it comes to solving them to a ﬁnite accuracy.
In particular, on unbounded domains, approximating variational inequalities is meaningless, whereas monotone inclusion remains well-deﬁned.
This is most readily seen from the observation that mapping from min-max optimization, variational inequalities correspond to primal-dual gap guarantees, while monotone inclusion corresponds to a guarantee in gradient norm.
For a simple bilinear function f ( x, y ) =
xy which has the unique min-max solution at ( x, y )
= (0 , 0) , the primal-dual gap is inﬁnite for any point other than (0 , 0) , while the gradient remains ﬁnite and is a good proxy for measuring quality of a solution.
Further, even on bounded domains or using restricted gap functions on unbounded domains as in e.g., [37], optimal oracle complexity guarantees for approximate monotone inclusion imply optimal complexity guarantees for approximately satisﬁed variational inequalities (see, e.g., [12]).
The opposite does not hold in general.
In particular, in deterministic settings, standard algorithms such as the celebrated extragradient [28, 36], dual extrapolation [37], or Popov’s method [42] that have the optimal oracle complexity O ( 1 ✏ ) for approximating variational inequalities are suboptimal for monotone inclusion and attain oracle complexity of the order O ( 1 ✏ 2 ) [13, 20].
Halpern iteration.
Halpern iteration is a classical ﬁxed point iteration originally introduced by [21], and studied extensively in terms of both its asymptotic and non-asymptotic convergence guarantees [8, 26, 27, 31, 33, 52].
The ﬁrst tight nonasymptotic convergence rate guarantee of 1 /t was obtained in [33, 47].
This rate was also matched by [25].
The usefulness of Halpern iteration for solving monotone inclusion problems was ﬁrst observed by [12], 1 who showed that its variants can be used to obtain near-optimal oracle complexity results for all standard classes of monotone inclusion problems with Lipschitz operators also studied in this work.
The near-tightness (up to poly-logarithmic factors) of the results from [12] was certiﬁed using lower bound reductions from min-max optimization lower bounds introduced by [39].
These lower bounds were made tight for the cocoercive setup in [13].
The generalization of Halpern iteration from the cocoercive to Lipschitz monotone setup in [12] utilized approximating what is known as the resolvent operator, which led to a double-loop algorithm and an additional log(1 / ✏ ) in the resulting complexity.
This log factor was shaved off in [53], who introduced a two-step variant of Halpern iteration, inspired by the extragradient method of [28].
The results of [12, 53] were further extended to other classes of Lipschitz operators by [29, 51].
Except for [29] which considered controlled variance as discussed above, all of the existing results only targeted deterministic settings.
Stochastic settings and variance reduction.
Vanilla stochastic gradient methods have constant variance of stochastic gradients, which creates a bottleneck in the convergence rate.
To improve the convergence rate, in the past decade, powerful variance reduction techniques have been proposed.
For strongly convex ﬁnite-sum problems, SAG [48], which used a biased stochastic estimator of the full gradient, was the ﬁrst stochastic gradient method with a linear convergence rate.
[24] and [11] improved [48] by proposing unbiased estimators of SVRG-type and SAGA-type, respectively.
Such unbiased estimators were further combined with Nesterov acceleration [2, 49], or applied to nonconvex ﬁnite-sum/inﬁnite-sum problems [30, 43].
For nonconvex stochastic (inﬁnite-sum) problems, SARAH [38] and SPIDER [18, 54, 55] estimators were proposed to attain the optimal oracle complexity of O (1 / ✏ 3 ) for ﬁnding an ✏ -approximate stationary point.
Both estimators are referred to as “recursive” variance reduction estimators, as they are biased when taking expectation w.r.t.
current randomness but unbiased w.r.t.
all the randomness in history.
PAGE [32] and STORM [10] signiﬁcantly simpliﬁed SARAH and SPIDER in terms of reducing the number of loops and avoiding large minibatches, respectively.
[3] further extended this line of work by incorporating second-order information and dynamic batch sizes.
In the setting of min-max optimization and variational inequalities/monotone inclusion, variance reduction has primarily been used for approximating variational inequalities, corresponding to the primal-dual gap in min-max optimization; see, for example [1, 5, 6, 23, 34, 40].
Under strong monotonicity (or sharpness in the case of [34]), such results generalize to monotone inclusion; however, to the best of our knowledge, there have been no results that address monotone inclusion under the weaker assumptions considered in this work.
In the context of monotone inclusion with
Lipschitz operators, the tightest complexity result that we are aware of is O ( 1 ✏ 4 ) , due to [14], and it applies to a more general class of structured non-monotone Lipschitz operators, for the best iterate.
The same oracle complexity can be deduced for the last iterate of a two-step variant of Halpern from [29, Theorem 6.1], using mini-batching.
All the results in our work are also for the last iterate.
Preliminaries
We consider a real d -dimensional normed space ( R d , k · k ) , where k · k is induced by an inner product associated with the space, i.e., k · k = p h · , · i .
Let U ✓ R d be closed and convex; in the unconstrained case, U ⌘ R d .
When U is bounded, D = max u , v 2 U k u
v k denotes its diameter.
Classes of monotone operators.
We say that an operator F : R d !
R d is
monotone, if 8 u , v 2 R d , h F ( u )  F ( v ) , u  v i  0 .
L -Lipschitz continuous for some L > 0 , if 8 u , v 2 R d , k F ( u )  F ( v ) k
 L k u  v k . 3.
-cocoercive for some  > 0 , if 8 u , v 2 R d , h F ( u )  F ( v ) , u  v i   k F ( u )  F ( v ) k 2 . 4.
µ -strongly monotone for some µ > 0 , if 8 u , v 2 R d , h F ( u )  F ( v ) , u  v i  µ k u  v k 2 .
Note that we can easily specialize these deﬁnitions to the set U by restricting u , v to be from U .
Throughout the paper, the minimum assumption that we make about an operator F is that it is monotone and Lipschitz.
Observe that any  -cocoercive operator is monotone and 1  -Lipschitz.
The converse to this statement does not hold in general.
Monotone inclusion and variational inequalities.
Monotone inclusion asks for u ⇤ such that
[Equation]
where I U is the indicator function of the set U and @
I U ( · ) denotes the subdifferential of I U .
If F is continuous and monotone, the solution set to (MI) is the same as the solution set of the
Stampacchia Variational Inequality (SVI) problem, which asks for u ⇤ 2 U such that
[Equation]
Further, when F is monotone, the solution set of (SVI) is equivalent to the solution set of the Minty
Variational Inequality (MVI) problem consisting in ﬁnding u ⇤ such that
[Equation]
We assume throughout the paper that a solution to monotone inclusion (MI) exists, which implies that solutions to both (SVI) and (MVI) exist as well.
Existence of solutions follows from standard results and is guaranteed whenever e.g., U is compact, or, if there exists a compact set U 0 such that Id  1 L F maps U 0 to itself, where Id is the identity map [17].
As remarked in the introduction, in unbounded setups it is generally not possible to approximate (MVI) and (SVI), whereas approximating (MI) is quite natural: we only need to ﬁnd u such that 0 2 F ( u ) + @
I U ( u ) + B ( ✏ ) , where 0 denotes the zero vector and B ( ✏ ) denotes the centered ball of radius ✏ .
Stochastic access to the operator.
We consider the stochastic setting for monotone inclusion problems.
More speciﬁcally, we make the following assumptions for stochastic queries to F. These assumptions are made throughout the paper, without being explicitly invoked.
Assumption 1 (Unbiased samples with bounded variance) .
For each query point x 2 U , we observe b
F ( x , z ) where z ⇠ P z is a random variable that satisﬁes the following assumptions:
[Equation]
and
[Equation]
Assumption 2 (Multi-point oracle) .
We can query a set of points ( x 1 , . . .
, x n ) and receive b
F ( x 1 , z ) , . . .
, b
F ( x n , z ) where z ⇠ P z .
Assumption 3 (Lipschitz in expectation) .
E z
[Equation]
We note that complexity results of the paper will bound the total number of queries made to this oracle.
In particular, if multiple query points and/or multiple samples z are used in a single iteration, our complexity is given by the sum of all those queries throughout all iterations of the method.
Also,
Assumption 3 is primary with parameter L , by which F is also L -Lipschitz using Jensen’s inequality.
PAGE variance-reduced estimator.
We now summarize a variant of the PAGE estimator, originally developed for smooth nonconvex optimization by [32], adapted to our setting.
In particular, given queries to b
F , we deﬁne the variance reduced estimator e
F ( u k ) for k  1 by
[Equation]
where p
= 1 , z ( k )
i i.i.d.
⇠ P z , and S ( k )and S ( k )are the sample sizes at iteration k .
Observe that
Assumption 2 guarantees that we can query b
F at u k and u k  1 using the same random seed.
Our analysis will make use of conditional expectations, and to that end, we deﬁne natural ﬁltration F k by F k :
=  ( { e
F ( u j ) } j  k ) ; namely F k contains all the randomness that arises in the deﬁnitions of e
F ( u j ) for j  k. Following a similar argument as in [32], we recursively bound the variance of the estimator e
F , as summarized in the following lemma.
The proof is provided in Appendix B.
Lemma 2.1.
Let F be a monotone operator accessed via stochastic queries b
F , under Assumptions 1– 3.
Then, the variance of e
F deﬁned by Eq.
(2.1) satisﬁes the following recursive bound: for all k  1 ,
[Equation]
With the choices of p k , S ( k ) 1 , S ( k )speciﬁed in the following corollary and using induction with the inequality from Lemma 2.1, we obtain the following bound on the variance.
Corollary 2.2.
Given a target error ✏ > 0 , if for all k  1 , p k =
[Equation]
⌃
[Equation]
2 p 2 k ✏⌥ , then E ⇥ e
F ( u k )  F ( u k )
⇤  ✏ 2 k .
Stochastic Halpern iteration for cocoercive operators
In this section, we consider the setting of 1 L -cocoercive operators F.
While cocoercivity is a strong assumption that implies that an operator is both Lipschitz and monotone (as discussed in Section 2), it is nevertheless the most basic setup for studying the Halpern iteration.
In particular, while Halpern iteration can be applied directly to the nonexpansive counterpart of a cocoercive operator F (i.e., to the linear transformation Id  2 L F , where 1 L is an upper bound on the cocoercivity parameter of F ), convergence does not seem possible to establish for the more general class of Lipschitz monotone operators.
We begin this section by providing a generic proof of stochastic oracle complexity, which we then use to brieﬂy illustrate how to obtain O ( 1 ✏ 4 ) oracle complexity with a simple minibatch stochastic estimator of F .
We then show how to improve this bound to O ( 1 ✏ 3 ) by applying the proposed variant of the PAGE estimator from Eq.
(2.1) to Halpern iteration.
The stochastic variant of Halpern iteration that we consider is deﬁned by
[Equation]
where e
F is a stochastic (possibly biased) estimator of F ,  k +1 = ⇥ ( 1 k ) is the step size, and
L k +1  L is a parameter of the algorithm.
Compared to the classical iteration u k +
=
k +1 u
+ (   k +1 ) T ( u k ) , where T : R d !
R d is a nonexpansive (1-Lipschitz) map [21], T is replaced by the mapping Id  2 L k + e
F , which is stochastic and may not be nonexpansive (as the stochastic estimate e
F of F is not guaranteed to be cocoercive even when F is).
Compared to the iteration variant considered by [12], the access to the monotone operator is stochastic and we also take slightly larger (by a factor of 2) values of L k +1 to bound the stochastic error terms.
Our argument for bounding the total number of stochastic queries to F is based on the use of the following potential function C k =
A k L k k F ( u k ) k
+ B k h F ( u k ) , u k  u 0 i , where { A k } k  1 and { B k } k  1 are positive and non-decreasing sequences of real numbers, while the step size  k is deﬁned by  k :=
B k
A k + B k .
Such potential function was previously used for the deterministic case of Halpern iteration in [12, 13].
Observe that even though we make oracle queries to b
F , the potential function
C k and the ﬁnal bound we obtain are in terms of the true operator value F.
Compared to the analysis of Halpern iteration in the deterministic case [12, 13], our analysis for the stochastic case needs to account for the error terms caused by accessing F via stochastic queries and is based on an intricate inductive argument.
A generic bound on iteration complexity, under mild assumptions about the estimator e
F, is summarized in Theorem 3.1.
The proof is in Appendix C.
Theorem 3.1.
Given an arbitrary u 0 2 R d , suppose that iterates u k evolve according to Halpern iteration from Eq.
(3.1) for k  1 , where L k =
L
and  k = 1 k +1 .
Assume further that the stochastic estimate e
F ( u ) is unbiased for u = u 0 and E [ k F ( u 0 )  e
F ( u 0 ) k 2 ]
 ✏ 2 8 .
Given ✏ > 0 , if for all k  1 , we have that E
[Equation]
, then for all k  1 ,
[Equation]
) where ⇤
=
76 L k u 0  u
⇤ k
and ⇤ 1 =q 23 .
As a result, stochastic Halpern iteration from Eq.
(3.1) returns a point u k such that E [ k F ( u k ) k ]  4 ✏ after at most N = d 2 ⇤
✏ e =
O
L k u 0  u ⇤ k ✏  iterations.
We remark that the previous result states an iteration complexity bound under a rather high accuracy assumption for the operator estimators at each iteration.
In order to attain these accuracy requirements, we could either use a minibatch at every iteration, or use variance reduction.
In what follows we explore both approaches.
We further remark that we made no effort to optimize the constants in the bound above, and thus the constants are likely improvable.
Finally, observe that due to the required low error for the estimates E [ k F ( u k )  e
[Equation]
, we can certify by Chebyshev bound that P [ k F ( u k )  e
F ( u k ) k
✏ ]  1 k .
In particular, after O ( 1 ✏ ) iterations, if we have k ˜ F ( u k ) k  ✏ (which holds in expectation), then k F ( u k ) k is also O ( ✏ ) with probability at least 1  ✏ .
This is particularly important for practical implementations, where a stopping criterion can be based on the value of k e
F ( u k ) k , which, unlike k F ( u k ) k , can be efﬁciently evaluated.
3.1 Stochastic oracle complexity with a simple mini-batch estimate
A direct consequence of Theorem 3.1 is that a simple estimator e
F ( u k ) =
1 S k
P
S k i
=
b
F ( u k , z ( k ) i ) leads to the overall O ( 1 ✏ 4 ) oracle complexity, as stated below while the proof is deferred to Appendix C.
Corollary 3.2.
Under the assumptions of Theorem 3.1, if e
F ( u k ) =
1 S k
P
S k i
=
b
F ( u k , z ( k ) i ) , where b
F ( u k , z ( k ) i ) satisﬁes Assumption 1 and z ( k ) i i.i.d.
⇠ P z , then setting S k =  2 ( k +1) ✏for all k guarantees that E [ k F ( u k ) k ]  4 ✏ after at most O   2 L 2 k u 0  u
⇤ k 2 ✏ queries to b
F .
3.2 Improved oracle complexity via variance reduction
We now consider using the recursive variance reduction method from Eq.
(2.1) to obtain the variance bound required in Theorem 3.1, as summarized in Algorithm 1.
Of course, in practice, k u 0  u
⇤ k is not known, and instead of running the algorithm for a ﬁxed number of iterations N, one could run it, for example, until reaching a point with k e
[Equation]
Input: u 0 2 R d , k u 0  u ⇤ k , L, ✏ > 0 ,  ;
Initialize: ⇤
=
76 L k u 0  u ⇤ k ✏ , N = d 2 ⇤
✏ e , S (0) =
d 8  2 ✏ 2 e , e
F ( u 0 )
= 1 S (0)
P
S (0)1 i =
b
F ( u 0 , z (0) i ) ; for k = 1 : N do u k = 1 k +1 u
[Equation]
Compute e
F ( u k ) based on Eq.
Return: u N
Notice that convergence is guaranteed by Theorem 3.1; however it does not directly address the problem of the oracle complexity (as batch sizes depend on successive iterate distances).
To resolve this issue, we ﬁrst provide a bound on k u k  u k  1 k as in Lemma 3.3, while the proof is deferred to Appendix C, and make the appropriate parameter settings for the estimator from Eq.
It is now possible to apply Theorem 3.1 to obtain the improved O ( 1 ✏ 3 ) stochastic oracle complexity bound.
Lemma 3.3.
Given an arbitrary initial point u 0 2 R d , let { u k } k  1 be the sequence of points produced by Algorithm 1.
Assume further that  k = 1 k
+1 , L k = 2 L for all k  0 .
Then,
[Equation]
Moreover, if for 1  i  k  1 , all of the following conditions hold (same as in Theorem 3.1): (i) E
[ k F ( u i ) k ]  ⇤
i + ⇤ 1 ✏ , where ⇤
=
76 L k u 0  u
⇤ k
and ⇤ 1 =q 23 , (ii)
E ⇥
F ( u i )  e
F ( u i )
⇤  ✏ 2 i , and (iii) ✏  ⇤
k , then E [ k u k  u k  1 k 2 ] =
O ⇣ k u 0  u ⇤ k 2 k⌘ .
Corollary 3.4.
Given arbitrary u 0 2 R d and ✏ > 0 , consider u N returned by Algorithm 1.
Then,
E [ k F ( u N ) k ]  4 ✏ with expected O (  2 L k u 0  u ⇤ k
+ L 3 k u 0  u ⇤ k 3 ✏) oracle queries to b
F .
Proof.
Let m k be the number of stochastic queries made by the estimator from Eq.
(2.1) in iteration k .
Using Corollary 2.2, we have
[Equation]
where the ﬁrst equality holds because S ( k )is measurable w.r.t.
F k  1 and the only random choice that remains is the selection of the estimator in Eq.
(2.1) determined by probabilities p k and p k .
Taking expectation with respect to all randomness on both sides, rearranging the terms, and using the fact that d x e  x + 1 for any x 2 R , we obtain E [ m k +1 ]  8  2 ✏ 2 + 16(1  p k )
L 2 E [ k u k
u k  1 k 2 ] p 2 k ✏+ 2 .
Recalling that p k =
k
+
=
O
( 1 k ) and E
[ k u k  u k  1 k 2 ]
= O  k u 0  u ⇤ k 2 k by Lemma 3.3, it follows that E
[ m k +1 ]
=
O
[Equation]
⇤
[Equation]
2 ✏
[Equation]
.
As, by Theorem 3.1, the total number of iterations to attain 4 ✏ norm of the operator in expectation is N =
[Equation]
and m 0 = S (0)
=
O
[Equation]
✏ , the total number of queries to
[Equation]
is
[Equation]
We note in passing that the running time guarantee of this algorithm is of Las Vegas-type: despite its iteration number being surely bounded by
[Equation]
, the batch sizes (in particular
S ( k ) 2 ) are random, and are not universally bounded.
We further argue that Algorithm 1 can be extended to constrained settings by deﬁning the operator mapping as in [12] and modifying the variance-reduced stochastic estimator accordingly based on the projection of e
F .
We show that the newly deﬁned operator mapping is also cocoercive while the variance of the modiﬁed estimator is bounded by the variance of e
F , so arguments from Theorem 3.
and Corollary 3.4 extend to this case.
This modiﬁed estimator need not be unbiased (as neither is e
F ); however, this is irrelevant to our analysis as it does not require unbiasedness.
For completeness, a detailed extension to the constrained case is provided in Appendix C.2.
Monotone and Lipschitz setup
Throughout this section, we assume that F is monotone and L -Lipschitz.
While the previous section addresses the cocoercive setup using the classical version of Halpern iteration adapted to cocoercive operators, it is unclear how to directly generalize this result to the setting with monotone Lipschitz operators.
In the deterministic setting, generalization to monotone Lipschitz operators can be achieved through the use of a resolvent operator (see [12]).
However, such an approach incurs an additional log(1 / ✏ ) factor in the iteration complexity coming from approximating the resolvent and it is further unclear how to generalize it to stochastic settings, as the properties of the stochastic estimate e
F of F do not readily translate into the same or similar properties for the resolvent of e
F. Instead of taking the approach based on the resolvent, we consider a recently proposed two-step variant of Halpern iteration [51], adapted here to the stochastic setting.
The variant uses extrapolation and is deﬁned by
[Equation]
where  k 2 [0 , 1) , ⌘ k > 0 , and e
F is deﬁned by (2.1).
The resulting algorithm with a complete parameter setting is provided in Algorithm 2.
To analyze the convergence of the extrapolated Halpern variant from Eq.
(4.1), we use the potential function V k
=
A k k F ( u k ) k
+ B k h F ( u k ) ,
u k  u
i + c k L 2 k u k  v k  1 k 2 , previously used by [51], where A k , B k and c k are positive parameters to be determined later.
Observe that this is essentially the same potential function as C k , corrected by the quadratic term c k L 2 k u k  v k  1 k 2 to account for error terms appearing in the analysis of the two-step variant from Eq.
Similarly as in the cocoercive setup, the potential function is not monotonically non-increasing, due to the error terms that arise due to the stochastic access to F. Bounding these error terms requires a careful technical argument, and is the main technical contribution of this section.
Due to space constraints, the complete technical argument is deferred to Appendix D, while the main results are stated below.
Theorem 4.1.
Given an arbitrary initial point u 0 2 R d and target error ✏ > 0 , assume that the iterates u k evolve according to Algorithm 2 for k  1 .
Then, for all k  2 ,
[Equation]
where ⇤
=
( L 2 ⌘ 0 ⌘ +1)
k
u 0  u ⇤ k 2 ⌘and ⇤ 1 = 5 ( 1+ M ⌘⌘ 0 ) M ⌘.
In particular, E
[Equation] [Equation]
⇤  2 ⇤ 1 ✏ 2 = O ( ✏ 2 ) after at most N = ⌃ p ⇤
p
⇤ 1 ✏ ⌥ = O
[Equation]
iterations.
The total number of oracle queries to b
F is O
[Equation]
in expectation.
Input
[Equation]
Initialize: v  1 = u 0 , S (  1)=
[Equation] [Equation]
Set
[Equation]
P
[Equation]
, z
[Equation]
[Equation]
) i ) , where z (  1)
i i.i.d.
⇠ P z ; for k = 1 : N do v
[Equation]
Compute e
F ( v k  1 ) based on Eq.
[Equation]
Return: u N
Faster convergence under a sharpness condition
We now show that by restarting Algorithm 2, we can achieve the O  1 ✏ 2 log 1 ✏  oracle complexity under a milder than strong monotonicity µ -sharpness condition : for all u 2 U , h F ( u )  F ( u ⇤ ) , u  u
⇤ i  µ k u  u
⇤ k 2 .
The scheme is summarized in Algorithm 3, and the proof is deferred to Appendix E.
Algorithm
Input: v  1 = u 0 2 R d , k u 0  u ⇤ k , 0 < ⌘ 0  1 3 p 3 L , L, µ , ✏ > 0 ,  ;
Initialize: M = 9 L 2 , ⌘ = ⌘ 0 ( 1  2 M ⌘ 20 ) M ⌘, N = l log ⇣
p 6 k u 0  u ⇤ k 2 ✏ ⌘m ; for k = 1 : N do
Call Algorithm 2 with initialization v ( k )  1 = u ( k )
= u k  1 , ✏ k = µ ✏ p M ⌘ q 5 ( 1+ M ⌘⌘ 0 ) , and
S (  1)= S (0) =
d 8  2 ✏ 2 k e , for K = l 4 p L 2 ⌘ 0 ⌘ +1 µ ⌘ m iterations, and return u k ;
Return: u N
Theorem 5.1.
Given L -Lipschitz and µ -sharp F and the precision parameter ✏ , Algorithm 3 outputs u N with E [ k u N  u ⇤ k 2 ]  ✏ 2 as well as E
⇥ k F ( u N ) k
⇤  L 2 ✏ 2 after N = O ⇣
Lµ log k u 0  u ⇤ k ✏ ⌘ iterations with at most O ⇣  2 ( µ + L )log( k u 0  u ⇤ k / ✏ )+ L 3 k u 0  u ⇤ k 2 µ 3 ✏⌘ queries to b
F in expectation.
Numerical experiments and discussion
We now illustrate the empirical performance of stochastic Halpern iteration on robust least square problems.
Speciﬁcally, given data matrix A 2 R n ⇥ d and noisy observation vector b 2 R n subject to bounded deterministic perturbation  with k  k  ⇢ , robust least square (RLS) minimizes the worst-case residue as min x 2 R d max  : k  k ⇢
k
Ax  y k 22 with y
= b +  [16].
We consider solving MI induced from RLS with Lagrangian relaxation where u = ( x , y ) T and
F ( u ) =
r x L  ( x , y ) , r y L  ( x , y )
T for L  ( x , y ) = 1 2 n k A x  y k 2 2   2 n k y  b k 2 2 .
We use a real-world superconductivity dataset [22] from UCI Machine Learning Repository [15] for our experiment, which is of size 21263 ⇥ 81 .
To ensure the problem is concave in y , we need that  > 1; in the experiments, we set  = 1 . 5 .
For the experiment, we compare Halpern, E-Halpern, and Restarted E-Halpern algorithms with gradient descent-ascent (GDA), extragradient (EG) [28], and Popov’s method [42] in stochastic settings.
Even though our theoretical results for Restarted
[Figure]
E-Halpern require scheduled restarts based on known problem parameters, in the implementation, to avoid complicated parameter tuning and illustrate empirical performance, we restart E-Halpern whenever the norm of stochastic estimator e
F used in E-Halpern halves.
All Halpern variants are implemented with PAGE estimator considered in our paper; all other algorithms are implemented using minibatches.
Additionally, we compare E-Halpern with the PAGE estimator against E-Halpern with single-sample and mini-batch estimators.
We plot the (empirical) operator norm k F ( u ) k against the number of stochastic operator evaluations.
Note that evaluations of k F ( u ) k are only used for plotting but not for running any of the algorithms.
We use the same random initialization and tune the batch sizes and step sizes (to the values achieving fastest convergence under noise) for each method by grid search.
We use constant batch sizes and constant step sizes for GDA, EG, and Popov.
We also choose the batch sizes of PAGE estimator to ensure E [ k F ( u k )  e
F ( u k ) k 2 ]  O ( 1 k ) , which handles error accumulation [29] and early stagnation of stochastic Halpern iteration.
We implement all the algorithms in Python and run each algorithm using one CPU core on a macOS machine with Intel
2.3GHz
Dual Core i5 Processor and 8GB RAM.
We observe that (i) in Figure 1(a) both Halpern and E-Halpern exhibit faster convergence to approximate stationary points (with much smaller gradient norm after same number of gradient evaluations) than other algorithms, and restarting E-Halpern provides additional speedup, validating our theoretical insights; (ii) in Figure 1(b), E-Halpern with PAGE estimator displays faster convergence compared to other two estimators, in agreement with our theoretical analysis.
Conclusion
We introduced stochastic variance reduced variants of Halpern iteration for addressing monotone inclusion problems.
Our work addresses all standard classes of Lipschitz monotone problems and achieves improved stochastic oracle complexity guarantees, all for the last iterate.
Subsequent to this work, [7] obtained near-optimal bounds for the cases considered in this work, by reducing the Lipschitz monotone case to the Lipschitz strongly monotone case, using regularization.
It is an open question to obtain such near-optimal bounds with a direct method, without the use of regularization.
Acknowledgements
XC and CS were supported in part by the NSF grant 2023239.
CG’s research was partially supported by INRIA Associate Teams project, FONDECYT 1210362 grant, ANID Anillo ACT210005 grant, and National Center for Artiﬁcial Intelligence CENIA FB210017, Basal ANID.
Part of this work was done while CG was at the University of Twente.
JD was supported by the NSF grant 2007757, by the
Ofﬁce of Naval Research under contract number N00014-22-1-2348, and by the Wisconsin Alumni
Research Foundation.
Part of this work was done while JD and CS were visiting Simons Institute for the Theory of Computing.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] The contributions are summarized in Introduction (Section 1) and fully supported by detailed proofs in the main body and the appendix.
(b) Did you describe the limitations of your work?
[Yes] Some speciﬁc examples are: (i) the algorithm being of the Las Vegas-type, in Section 3.2 and (ii) the dependence on problem parameters other than ✏ and  being likely suboptimal in Appendix A. (c) Did you discuss any potential negative societal impacts of your work?
[N/A]
This is a theoretical paper with preliminary numerical results.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 2.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Sections 3–5 in the main body and the appendix.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See the url in Section 6.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 6.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We use the same initialization for all methods, and tune the parameters of each method to be optimal individually.
Details can be found in the README from the url in Section 6.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 6. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 6.
(b) Did you mention the license of the assets?
[Yes] See Section 6.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] Our code is included in the url in Section 6.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A]
The data we used is from online public repository.
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A]
The data we used is from online public repository.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]