Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We study stochastic monotone inclusion problems, which widely appear in machine learning applications, including robust regression and adversarial learning.
We propose novel variants of stochastic Halpern iteration with recursive variancereduction.
In the cocoercive—and more generally Lipschitz-monotone—setup, ouralgorithm attains ✏ norm of the operator with O ( 1 ✏ 3 ) stochastic operator evaluations, which signiﬁcantly improves over state of the art O ( 1 ✏ 4 ) stochastic operatorevaluations required for existing monotone inclusion solvers applied to the sameproblem classes.
We further show how to couple one of the proposed variants ofstochastic Halpern iteration with a scheduled restart scheme to solve stochasticmonotone inclusion problems with O ( log(1 / ✏ ) ✏) stochastic operator evaluations under additional sharpness or strong monotonicity assumptions.
Finally, we argue viareductions between different problem classes that our stochastic oracle complexitybounds are tight up to logarithmic factors in terms of their ✏ -dependence.
Introduction
Recent trends in machine learning (ML) involve the study of models whose solutions do not reduce tooptimization but rather to equilibrium conditions .
Standard examples include generative adversarialnetworks, adversarially robust training of ML models and training of ML models under notionsof fairness.
It turns out that several of these equilibrium conditions (including, but not limited to,ﬁrst-order stationary points, saddle-points and Nash equilibria of minimax games) can be cast assolutions to a monotone inclusion problem, which is deﬁned as the problem of computing a zero ofa (maximal) monotone operator F : R d !
R d (see (MI) for a formal deﬁnition).
In the context ofmin-max optimization problems, monotone inclusion reduces to a stationarity condition, which forunconstrained problems boils down to ﬁnding a point with small gradient norm.
Of particular interest to machine learning are stochastic versions of these problems, in which theoperator F is not readily available, but can only be accessed through a stochastic oracle b
F .
Such arethe settings mentioned above, where the deﬁnitions of equilibria involve expectations over continuoushigh-dimensional spaces.
The corresponding problem, known as the stochastic monotone inclusion ,has not been thoroughly studied, particularly in the context of its stochastic oracle complexity.
Understanding stochastic oracle complexity of monotone inclusion in all standard settings with
Lipschitz operators, from the algorithmic aspect, is the main motivation of this work.
1.1 Contributions
We study three main classes of stochastic monotone inclusion problems with Lipschitz operators,deﬁned by the assumptions made about the operator itself: (i) cocoercive class, which is the mostrestricted class, but nevertheless fundamental for understanding monotone inclusion, as it relates to the problem of ﬁnding a ﬁxed point of a nonexpansive (1-Lipschitz) operator; (ii) Lipschitz monotoneclass, which is perhaps the most basic class arising in the study of smooth convex-concave min-maxoptimization problems; and (iii) Lipschitz monotone class with an additional sharpness property ofthe operator.
Sharpness is a widely studied property of optimization problems, often referred to as the“local error bound” condition, which is weaker than strong convexity and roughly corresponds to theproblem landscape being curved outside of the solution set (see [38] for a survey of classical results).
From an algorithmic standpoint, we consider variants of classical Halpern iteration [19], which wasoriginally introduced for solving ﬁxed point equations with nonexpansive operators.
Variants ofthis iteration have recently been shown to lead to (near-)optimal ﬁrst-order oracle complexity forall aforementioned standard problem classes in deterministic settings [10, 11, 50].
However, to thebest of our knowledge, stochastic variants of these methods have received very limited attentionprior to our work.
The only results we are aware of are for a two-step extragradient-like variantof Halpern iteration in co-monotone Lipschitz settings [26] and which show that when variance ofoperator estimates is bounded by order- ✏ 2 k in iteration k, the method attains operator norm ✏ after
O ( 1 ✏ ) iterations.
However, [26] does not discuss how such variance control would be obtained.
Simple mini-batching, as we show, only leads to O ( 1 ✏ 4 ) stochastic oracle complexity.
We show that existing variants of the Halpern iteration [10, 48] can be effectively combined withrecursive variance reduction [29] to obtain O ( 1 ✏ 3 ) stochastic oracle complexity in the cocoercive and
Lipschitz monotone setups.
We then show that the complexity can be further reduced to O  1 ✏ 2 log( 1 ✏ )  under an additional sharpness assumption about the operator.
The latter two bounds can be certiﬁedto be near-optimal in ✏ , by reductions between different problem classes, as argued in Appendix A.
To the best of our knowledge, our work is the ﬁrst to use variance reduction to reduce stochasticoracle complexity of monotone inclusion (small gradient norm in min-max optimization settings),and the attained bounds are the best achieved to date.
1.2 Techniques
Inspired by the potential function originally used by [10] and later used either in the same or slightlymodiﬁed form by [11, 26, 48, 50], we adapt this potential function-based argument to account forstochastic error terms arising due to the stochastic oracle access to the operator.
We ﬁrst show thatin the cocoercive minibatch setting, this argument only leads to O ( 1 ✏ 4 ) stochastic oracle complexity,and it is unclear how to improve it directly, as the analysis appears tight.
We then combine thecocoercive variant of Halpern iteration [10] with the PAGE estimator [29] to reduce the stochasticoracle complexity to O ( 1 ✏ 3 ) .
The same variance reduced estimator is also used in conjunction withthe two-step extrapolated variant of Halpern iteration introduced by [48], as a direct application of
Halpern iteration is not known to converge on the class of Lipschitz monotone operators.
While the basic ideas in our arguments are simple, their realization requires addressing major technicalobstacles.
First, the variance reduced estimator that we use [29] was originally devised for smoothnonconvex optimization problems, where it was coupled with a stochastic variant of gradient descent.
This is signiﬁcant, because the proof relies on a descent lemma, which allows cancelling the errorarising from the variance of the estimator by the “descent” part.
Such an argument is not possiblein our setting, as there is no objective function to descend on.
Instead, our analysis relies on anintricate inductive argument that ensures that the expected norm of the operator is bounded in eachiteration, assuming a suitable bound on the variance of the estimator.
To obtain our desired result forthe variance, we propose a data-dependent batch allocation in PAGE estimator [29] (see Corollary2.2), which scales proportionally to the squared distance between successive iterates, similar to [3].
We inductively argue that the squared distance between successive iterates arising in the batch size ofthe estimator reduces at rate 1 k 2 in expectation .
This allows us to further certify that the estimators donot only remain accurate, but their variance decreases as O ( ✏ 2 /k ) , where k is the iteration count.
In the context of the potential function argument, unlike in the deterministic settings, we do notestablish that the potential function is non-increasing, even in expectation.
The stochastic error termsthat arise due to the stochastic nature of the operator evaluations are controlled by taking slightlysmaller step sizes than in the vanilla methods from [10, 48], which allows us to “leak” negativequadratic terms that are further used in controlling the stochastic error.
The argument for controllingthe value of the potential function is itself coupled with the inductive argument for ensuring that theexpected operator norm remains bounded.
Finally, while applying a restarting strategy is standard under sharpness conditions [43], obtainingthe claimed stochastic oracle complexity result of O  1 ✏ 2 log
[Equation]
1 ✏ )  requires a rather technical argumentto bound the total number of stochastic queries to the operator.
1.3 Related Work
Monotone Inclusion and Variational Inequalities.
Variational inequality problems were originally devised to deal with approximating equilibria.
Their systematic study was initiated by [47].
The relationship between variational inequalities and min-max optimization was observed soonafter [41], while one of the earliest papers to study solving monotone inclusion as a generalization ofvariational inequalities, convex and min-max optimization, and complementarity problems is [42].
For a historical overview of this area and an extensive review of classical results, see [15].
In the case of monotone operators, standard variants of variational inequality problems (see Section 2)and monotone inclusion are equivalent—their solution sets coincide.
This is a consequence of thecelebrated Minty Theorem [32].
However, there is a major difference between these problems whenit comes to solving them to a ﬁnite accuracy.
In particular, on unbounded domains, approximatingvariational inequalities is meaningless, whereas monotone inclusion remains well-deﬁned.
Thisis most readily seen from the observation that mapping from min-max optimization, variationalinequalities correspond to primal-dual gap guarantees, while monotone inclusion corresponds toa guarantee in gradient norm.
For a simple bilinear function f ( x, y ) =
xy which has the uniquemin-max solution at ( x, y )
= (0 , 0) , the primal-dual gap is inﬁnite for any point other than (0 , 0) ,while the gradient remains ﬁnite and is a good proxy for measuring quality of a solution.
Further,even on bounded domains or using restricted gap functions on unbounded domains as in e.g., [34],optimal oracle complexity guarantees for approximate monotone inclusion imply optimal complexityguarantees for approximately satisﬁed variational inequalities (see, e.g., [10]).
The opposite does nothold in general.
In particular, in deterministic settings, standard algorithms such as the celebratedextragradient [25, 33], dual extrapolation [34], or Popov’s method [39] that have the optimal oraclecomplexity O ( 1 ✏ ) for approximating variational inequalities are suboptimal for monotone inclusionand attain oracle complexity of the order
O ( 1 ✏ 2 ) [11, 18].
Halpern iteration.
Halpern iteration is a classical ﬁxed point iteration originally introduced by[19], and studied extensively in terms of both its asymptotic and non-asymptotic convergenceguarantees [24, 28, 30, 49].
The ﬁrst tight nonasymptotic convergence rate guarantee of 1 /t wasobtained in [30, 44].
This rate was also matched by an alternative method proposed by [23].
The usefulness of Halpern iteration for solving monotone inclusion problems was ﬁrst observedby [10], 1 who showed that its variants can be used to obtain near-optimal oracle complexity resultsfor all standard classes of monotone inclusion problems with Lipschitz operators also studied in thiswork.
The near-tightness (up to poly-logarithmic factors) of the results from [10] was certiﬁed usinglower bound reductions from min-max optimization lower bounds introduced by [36].
These lowerbounds were made tight for the cocoercive setup in [11].
The generalization of Halpern iteration from the cocoercive to Lipschitz monotone setup in [10]utilized approximating what is known as the resolvent operator, which led to a double-loop algorithmand an additional log(1 / ✏ ) in the resulting complexity.
This log factor was shaved off in [50], whointroduced a two-step variant of Halpern iteration, inspired by the extragradient method of [25].
Theresults of [10, 50] were further extended to other classes of Lipschitz operators by [26, 48].
Exceptfor [26] which considered controlled variance as discussed above, all of the existing results onlytargeted deterministic settings.
Stochastic Settings and Variance Reduction.
Vanilla stochastic gradient methods have constantvariance of stochastic gradients, which creates a bottleneck in the convergence rate.
To improve theconvergence rate, in the past decade, powerful variance reduction techniques have been proposed.
For strongly convex ﬁnite-sum problems, SAG [45], which used a biased stochastic estimator ofthe full gradient, was the ﬁrst stochastic gradient method with a linear convergence rate.
[22] and[9] improved [45] by proposing unbiased estimators of SVRG-type and SAGA-type, respectively.
Such unbiased estimators were further combined with Nesterov acceleration [2, 46], or appliedto nonconvex ﬁnite-sum/inﬁnite-sum problems [27, 40].
For nonconvex stochastic (inﬁnite-sum)problems, SARAH [35] and SPIDER [16, 51, 52] estimators were proposed to attain the optimaloracle complexity of O (1 / ✏ 3 ) for ﬁnding an ✏ -approximate stationary point.
Both estimators arereferred to as “recursive” variance reduction estimators, as they are biased when taking expectationw.r.t.
current randomness but unbiased w.r.t.
all the randomness in history.
PAGE [29] and STORM[8] signiﬁcantly simpliﬁed SARAH and SPIDER in terms of reducing the number of loops andavoiding large minibatches, respectively.
[3] further extended this line of work by incorporatingsecond-order information and dynamic batch sizes.
In the setting of min-max optimization and variational inequalities/monotone inclusion, variancereduction has primarily been used for approximating variational inequalities, corresponding to theprimal-dual gap in min-max optimization; see, for example [1, 5, 6, 21, 31, 37].
Under strongmonotonicity (or sharpness in the case of [31]), such results generalize to monotone inclusion;however, to the best of our knowledge, there have been no results that address monotone inclusionunder the weaker assumptions considered in this work.
In the context of monotone inclusion with
Lipschitz operators, the tightest complexity result that we are aware of is O ( 1 ✏ 4 ) , due to [12], and itapplies to a more general class of structured non-monotone Lipschitz operators, for the best iterate.
The same oracle complexity can be deduced for the last iterate of a two-step variant of Halpern from[26, Theorem 6.1], using mini-batching.
All the results in our work are also for the last iterate.
Preliminaries
We consider a real d -dimensional normed space ( R d , k · k ) , where k · k is induced by an inner productassociated with the space, i.e., k · k = p h · , · i .
Let U ✓ R d be closed and convex; in the unconstrainedcase, U ⌘ R d .
When U is bounded, D = max u , v 2 U k u
v k denotes its diameter.
Classes of Monotone Operators.
We say that an operator F : R d !
R d is
monotone, if 8 u , v 2 R d , h F ( u )  F ( v ) , u  v i  0 .
L -Lipschitz continuous for some L > 0 , if 8 u , v 2 R d , k F ( u )  F ( v ) k
 L k u  v k .
-cocoercive for some  > 0 , if 8 u , v 2 R d , h F ( u )  F ( v ) , u  v i   k F ( u )  F ( v ) k 2 .
µ -strongly monotone for some µ > 0 , if 8 u , v 2 R d , h F ( u )  F ( v ) , u  v i  µ k u  v k 2 .
Note that we can easily specialize these deﬁnitions to the set U by restricting u , v to be from U .
Throughout the paper, the minimum assumption that we make about an operator F is that it ismonotone and Lipschitz.
Observe that any  -cocoercive operator is monotone and 1  -Lipschitz.
Theconverse to this statement does not hold in general.
Monotone Inclusion and Variational Inequalities.
Monotone inclusion asks for u ⇤ such that
[Equation]
where I U is the indicator function of the set U and @
I U ( · ) denotes the subdifferential of I U .
If F is continuous and monotone, the solution set to (MI) is the same as the solution set of the
Stampacchia Variational Inequality (SVI) problem, which asks for u ⇤ 2 U such that(
[Equation]
Further, when F is monotone, the solution set of (SVI) is equivalent to the solution set of the Minty
Variational Inequality (MVI) problem consisting in ﬁnding u ⇤ such that
[Equation]
We assume throughout the paper that a solution to monotone inclusion (MI) exists, which implies thatsolutions to both (SVI) and (MVI) exist as well.
Existence of solutions follows from standard resultsand is guaranteed whenever e.g., U is compact, or, if there exists a compact set U 0 such that Id  1 L Fmaps U 0 to itself, where Id is the identity map [15].
As remarked in the introduction, in unboundedsetups it is generally not possible to approximate (MVI) and (SVI), whereas approximating (MI) isquite natural: we only need to ﬁnd u such that 0 2 F ( u ) + @
I U ( u ) + B ( ✏ ) , where 0 denotes the zerovector and B ( ✏ ) denotes the centered ball of radius ✏ .
Stochastic Access to the Operator.
We consider the stochastic setting for monotone inclusionproblems.
More speciﬁcally, we make the following assumptions for stochastic queries to F. Theseassumptions are made throughout the paper, without being explicitly invoked.
Assumption 1 (Unbiased samples with bounded variance) .
For each query point x 2 U , we observeb
F ( x , z ) where z ⇠ P z is a random variable that satisﬁes the following assumptions:
[Equation]
and
[Equation]
Assumption 2 (Multi-point oracle) .
We can query a set of points ( x 1 , . . .
, x n ) and receive
[Equation]
z ⇠ P z
[Equation]
Assumption 3 (Lipschitz in expectation) .
E z
[Equation]
We note that complexity results of the paper will bound the total number of queries made to thisoracle.
In particular, if multiple query points and/or multiple samples z are used in a single iteration,our complexity is given by the sum of all those queries throughout all iterations of the method.
Also,
Assumption 3 is primary with parameter L , by which F is also L -Lipschitz using Jensen’s inequality.
PAGE Variance-Reduced Estimator.
We now summarize a variant of the PAGE estimator, originally developed for smooth nonconvex optimization by [29], adapted to our setting.
In particular,given queries to b
F , we deﬁne the variance reduced estimator e
F ( u k ) for k  1 by
[Equation]
w
[Equation]
p
[Equation] [Equation]
where p
= 1 , z ( k )
i i.i.d.
⇠ P z , and S ( k )and S ( k )are the sample sizes at iteration k .
Observe that
Assumption 2 guarantees that we can query b
F at u k and u k  1 using the same random seed.
Ouranalysis will make use of conditional expectations, and to that end, we deﬁne natural ﬁltration F kby F k :
=  ( { e
F ( u j ) } j  k ) ; namely F k contains all the randomness that arises in the deﬁnitions ofe
F ( u j ) for j  k. Following a similar argument as in [29], we recursively bound the variance of theestimator e
F , as summarized in the following lemma.
The proof is provided in Appendix B.
Lemma 2.1.
Let F be a monotone operator accessed via stochastic queries b
F , under Assumptions 1–3.
Then, the variance of e
F deﬁned by Eq.
(2.1) satisﬁes the following recursive bound: for all k  1 ,
[Equation]
With the choices of p k , S ( k ) 1 , S ( k )speciﬁed in the following corollary and using induction with theinequality from Lemma 2.1, we obtain the following bound on the variance.
Corollary 2.2.
Given a target error ✏ > 0 , if for all k  1 , p k
=
k
+1 , S ( k ) ⌃ 8  2 p k ✏⌥ , S ( k )⌃ 8 L 2 k u k
u k  1 k 2 p 2 k ✏⌥ , then E ⇥ e
F ( u k )  F ( u k )
⇤  ✏ 2 k .
Stochastic Halpern Iteration for Cocoercive Operators
In this section, we consider the setting of 1 L -cocoercive operators F.
While cocoercivity is a strongassumption that implies that an operator is both Lipschitz and monotone (as discussed in Section 2),it is nevertheless the most basic setup for studying the Halpern iteration.
In particular, while Halperniteration can be applied directly to the nonexpansive counterpart of a cocoercive operator F (i.e., tothe linear transformation Id  2 L F , where 1 L is an upper bound on the cocoercivity parameter of F ),convergence does not seem possible to establish for the more general class of Lipschitz monotoneoperators.
We begin this section by providing a generic proof of stochastic oracle complexity, whichwe then use to brieﬂy illustrate how to obtain O ( 1 ✏ 4 ) oracle complexity with a simple minibatch stochastic estimator of F .
We then show how to improve this bound to O ( 1 ✏ 3 ) by applying theproposed variant of the PAGE estimator from Eq.
(2.1) to Halpern iteration.
The stochastic variant of Halpern iteration that we consider is deﬁned by
[Equation]
where e
F is a stochastic (possibly biased) estimator of F ,  k +1 = ⇥ ( 1 k ) is the step size, and
L k +1  L is a parameter of the algorithm.
Compared to the classical iteration u k +
=
k +1 u 0 + [-
-]224 (   k +1 ) T ( u k ) , where T : R d !
R d is a nonexpansive (1-Lipschitz) map [19], T is replacedby the mapping Id  2 L k + e
F , which is stochastic and may not be nonexpansive (as the stochasticestimate e
F of F is not guaranteed to be cocoercive even when F is).
Compared to the iterationvariant considered by [10], the access to the monotone operator is stochastic and we also take slightlylarger (by a factor of 2) values of L k +1 to bound the stochastic error terms.
Our argument for bounding the total number of stochastic queries to F is based on the use ofthe following potential function C k
=
A k L k k F ( u k ) k
+ B k h F ( u k ) , u k  u 0 i , where { A k } k  1 and{ B k } k  1 are positive and non-decreasing sequences of real numbers, while the step size  k is deﬁnedby  k :=
B k
A k + B k .
Such potential function was previously used for the deterministic case of Halperniteration in [10, 11].
Observe that even though we make oracle queries to b
F , the potential function
C k and the ﬁnal bound we obtain are in terms of the true operator value F.
Compared to the analysis of Halpern iteration in the deterministic case [10, 11], our analysis for thestochastic case needs to account for the error terms caused by accessing F via stochastic queries andis based on an intricate inductive argument.
A generic bound on iteration complexity, under mildassumptions about the estimator e
F, is summarized in Theorem 3.1.
The proof is in Appendix C.
Theorem 3.1.
Given an arbitrary u 0 2 R d , suppose that iterates u k evolve according to Halperniteration from Eq.
(3.1) for k  1 , where L k =
L
and  k = 1 k +1 .
Assume further that the stochasticestimate e
F ( u ) is unbiased for u = u 0 and E [ k F ( u 0 )  e
F ( u 0 ) k 2 ]
 ✏ 2 8 .
Given ✏ > 0 , if for allk  1 , we have that E
[Equation]
[Equation]
✏ 2 k , then for all k  1 ,
[Equation]
where ⇤
=
76 L k u 0  u
⇤ k
and ⇤ 1 =q 23 .
As a result, stochastic Halpern iteration from Eq.
(3.1)returns a point u k such that E [ k F ( u k ) k ]  4 ✏ after at most N = d 2 ⇤
✏ e =
O
[Equation]
k u 0  u ⇤ k ✏  iterations.
We remark that the previous result states an iteration complexity bound under a rather high accuracyassumption for the operator estimators at each iteration.
In order to attain these accuracy requirements,we could either use a minibatch at every iteration, or use variance reduction.
In what follows weexplore both approaches.
We further remark that we made no effort to optimize the constants in thebound above, and thus the constants are likely improvable.
Finally, observe that due to the required low error for the estimates E [ k F ( u k )  e
F ( u k ) k 2 ]  ✏ 2 k ,we can certify by Chebyshev bound that P [ k F ( u k )  e
F ( u k ) k
✏ ]  1 k .
In particular, after ⌦ ( 1 ✏ )iterations, with probability at least 1  ✏ , both k F ( u k ) k and k e
F ( u k ) k are O ( ✏ ) in expectation.
Thisis particularly important for practical implementations, where a stopping criterion can be based onthe value of k e
F ( u k ) k , which, unlike k F ( u k ) k , can be efﬁciently evaluated.
3.1 Stochastic Oracle Complexity With a Simple Mini-batch Estimate
A direct consequence of Theorem 3.1 is that a simple estimator e
[Equation]
F ( u k , z ( k ) i ) leadsto the overall O ( 1 ✏ 4 ) oracle complexity, as stated below while the proof is deferred to Appendix C.
Corollary 3.2.
Under the assumptions of Theorem 3.1, if e
F ( u [Equation]
[Equation]
P
S k i
=
b
F ( u k , z ( k ) i ) , whereb
F ( u k , z ( k ) i ) satisﬁes Assumption 1 and z ( k ) i i.i.d.
⇠ P z , then setting S k =  2 ( k +1) ✏for all k  0guarantees that E [ k F ( u k ) k ]  4 ✏ after at most O   2 L 2 k u 0  u
⇤ k 2 ✏ queries to b
F .
3.2 Improved Oracle Complexity via Variance Reduction
We now consider using the recursive variance reduction method from Eq.
(2.1) to obtain the variancebound required in Theorem 3.1, as summarized in Algorithm 1.
Of course, in practice, k u 0  u ⇤ k isnot known, and instead of running the algorithm for a ﬁxed number of iterations N, one could run it,for example, until reaching a point with k
[Equation]
Input: u 0 2 R
[Equation]
k u
[Equation]
u
[Equation]
k , L, ✏
[Equation]
[Equation]
Initialize: ⇤
=
76 L k u 0  u ⇤ k ✏
[Equation]
for k = 1 : N do u k =
[Equation]
+
[Equation]
[Equation] p k = 2 k +1 , S ( k )
[Equation]
Compute e
F ( u k ) based on Eq.
Return: u N
Notice that convergence is guaranteed by Theorem 3.1; however it does not directly address theproblem of the oracle complexity (as batch sizes depend on successive iterate distances).
To resolvethis issue, we ﬁrst provide a bound on k u k  u k  1 k , and making the appropriate parameter settings 269 for the estimator from Eq.
(2.1), it is now possible to apply Theorem 3.1 to obtain the improved
O ( 1 ✏ 3 ) stochastic oracle complexity bound, as stated below while the proof is deferred to Appendix C.
[-[ k F ( u i ) k ]  ⇤ 
-]
[-i + ⇤ 1 ✏ , where ⇤ 
-]
[-⇤  ✏ 2 i , and (iii) ✏  ⇤ 
-]
[-(3.1), we have u k  u k  
-]
[- k ( u 0  u k  1 ) 
-]
Corollary 3.3.
Given arbitrary u 0 2 R d and ✏ > 0 , consider u N returned by Algorithm 1.
Then,
E [ k F ( u N ) k ]  4 ✏ and Algorithm 1 makes O (  2 L k u 0  u ⇤ k
+ L 3 k u 0  u ⇤ k 3 ✏) oracle queries to b
F .
[-Recalling that p k = 
-]
[-+
-]
[-= O  k u 0  u ⇤ k 2 k by Lemma 3.3, it followsthat E [ m k +
-]
We note in passing that the running time guarantee of this algorithm is of Las Vegas-type: despiteits iteration number being surely bounded by ⌃
[Equation]
✏  , the batch sizes (in particular
S ( k ) 2 ) are random, and are not universally bounded.
We further argue that Algorithm 1 can be extended to constrained settings by deﬁning the operatormapping as in [10] and modifying the variance-reduced stochastic estimator accordingly based onthe projection of e
F .
We show that the newly deﬁned operator mapping is also cocoercive while thevariance of the modiﬁed estimator is bounded by the variance of e
F , so arguments from Theorem 3.1and
Corollary 3.3 extend to this case.
This modiﬁed estimator needs not be unbiased (as neither ise
F ); however, this is irrelevant to our analysis as it does not require unbiasedness.
For completeness, adetailed extension to the constrained case is provided in Appendix C.2.
Monotone and Lipschitz Setup
Throughout this section, we assume that F is monotone and L -Lipschitz.
While the previous sectionaddresses the cocoercive setup using the classical version of Halpern iteration adapted to cocoerciveoperators, it is unclear how to directly generalize this result to the setting with monotone Lipschitzoperators.
In the deterministic setting, generalization to monotone Lipschitz operators can be achievedthrough the use of a resolvent operator (see [10]).
However, such an approach incurs an additionallog(1 / ✏ ) factor in the iteration complexity coming from approximating the resolvent and it is furtherunclear how to generalize it to stochastic settings, as the properties of the stochastic estimate e
F of Fdo not readily translate into the same or similar properties for the resolvent of e
F. Instead of takingthe approach based on the resolvent, we consider a recently proposed two-step variant of Halperniteration [48], adapted here to the stochastic setting.
The variant uses extrapolation and is deﬁned by
[Equation]
where  k 2 [0 , 1) , ⌘ k > 0 , and e
F is deﬁned by (2.1).
The resulting algorithm with a completeparameter setting is provided in Algorithm 2.
[-A k k F ( u k ) k 
-]
[-u k  u 
-]
[Equation]
Initialize: v  1 = u
[Equation]
S (
[Equation]
Set
[Equation]
) , where z (  1)
i i.i.d.
⇠ P z ; for k = 1 : N do
[Equation]
Compute e
F ( v k
[Equation]
) based on Eq.
[Equation]
Return: u N
To analyze the convergence of the extrapolated Halpern variant from Eq.
(4.1), we use the potentialfunction V k = A k k F [Equation]
2 , previously used by [48], 298 where A k , B k and c k are positive parameters to be determined later.
Observe that this is essentiallythe same potential function as C k , corrected by the quadratic term c k L 2 k u k  v k  1 k 2 to account 300 for error terms appearing in the analysis of the two-step variant from Eq.
Similarly as inthe cocoercive setup, the potential function is not monotonically non-increasing, due to the errorterms that arise due to the stochastic access to F. Bounding these error terms requires a carefultechnical argument, and is the main technical contribution of this section.
Due to space constraints,the complete technical argument is deferred to Appendix D, while the main results are stated below.
Theorem 4.1.
Given an arbitrary initial point u 0 2 R d and target error ✏ > 0 , assume that theiterates u k evolve according to Algorithm 2 for k  1 .
Then, for all k  2 ,
[Equation]
where ⇤
=
( L 2 ⌘ 0 ⌘ +1)
k u
[Equation]  u ⇤ k 2 ⌘and ⇤ 1 = 5 ( 1+ M ⌘⌘ 0 ) M ⌘.
In particular, E ⇥ k F ( u N {+) k 
+}
+
[Equation]
{+⇤  2 ⇤ 1 ✏ 2 = O ( ✏ 2 ) after at most N = ⌃ p ⇤ 
 p ⇤ 1 ✏+}
[Equation]
k
[Equation]
iterations.
The total number of oracle queries to
[Equation]
.
Faster Convergence Under a Sharpness Condition
We now show that by restarting Algorithm 2, we can achieve the O  1 ✏ 2 log 1 ✏  oracle complexity undera milder than strong monotonicity µ -sharpness condition : for all u 2 U , h F ( u )  F ( u ⇤ ) , u  u
⇤ i µ
k u  u
⇤ k 2 .
The scheme is summarized in Algorithm 3, and the proof is deferred to Appendix E.
[-Call Algorithm 2 with initialization v ( k )  1 = u ( k ) 
-]
Theorem 5.1.
Given L -Lipschitz and µ -sharp F and the precision parameter ✏ , Algorithm 3 outputsu N with E [ k u N  u
⇤ k 2 ]  ✏ 2 as well as E ⇥ k F
( u N ) k
⇤  L 2 ✏ 2 after N = O ⇣
[Equation]
⌘ iterations with at most O ⇣
[Equation]
⌘ queries to b
F .
Numerical Experiments and Discussion
We now illustrate the empirical performance of stochastic Halpern iteration on robust leastsquare problems.
Additional experiments can be found in Appendix ?? . Speciﬁcally, givendata matrix A 2 R n ⇥ d and noisy observation vector b 2 R n subject to bounded deterministic perturbation  with k  k  ⇢ , robust least square (RLS) minimizes the worst-case residue as Input: v  1 = u 0 2 R d , k u 0  u ⇤ k , 0 < ⌘ 0  1 3 p 3 L , L, µ , ✏ > 0 ,  ; Initialize: M = 9 L 2 , ⌘ = ⌘ 0 ( 1  2 M ⌘ 20 ) M ⌘, N = l log ⇣
p 6 k u 0  u ⇤ k 2 ✏ ⌘m ; for k = 1 : N do
{+Call Algorithm 2 with initialization v ( k )  1 = u ( k ) 
+}
= u k  1 , ✏ k = µ ✏ p M ⌘ q 5 ( 1+ M ⌘⌘ 0 ) , and
S (  1)= S (0) =
d 8  2 ✏ 2 k e , for K = l 4 p L 2 ⌘ 0 ⌘ +1 µ ⌘ m iterations, and return u k ;
Return: u N
[Figure]
min x 2 R d max  : k  k ⇢ k
Ax  y k 22 with y
= b +  [14].
We consider solving MI induced from RLSwith Lagrangian relaxation where u = ( x , y ) T and F ( u ) =
r x L  ( x , y ) , r y L  ( x , y )
T for
L  ( x , y ) =
12 n k A x  y k 22   2 n k y  b k 22 .
We use a real-world superconductivity dataset [20] from 325 UCI Machine Learning Repository [13] for our experiment, which is of size⇥ 81 .
To ensure 326 the problem is concave in y , we need that  > 1; in the experiments, we set  = 1 . 5 .
For the experiment, we compare Halpern, E-Halpern, and Restarted E-Halpern algorithms withgradient descent-ascent (GDA), extragradient (EG) [25], and Popov’s method [39] in stochasticsettings.
Even though our theoretical results for Restarted E-Halpern require scheduled restarts basedon known problem parameters, in the implementation, to avoid complicated parameter tuning andillustrate empirical performance, we restart E-Halpern whenever the norm of stochastic estimator e
Fused in E-Halpern halves.
All Halpern variants are implemented with PAGE estimator consideredin our paper; all other algorithms are implemented using minibatches.
Additionally, we compare
E-Halpern with the PAGE estimator against E-Halpern with single-sample and mini-batch estimators.
We report and plot the (empirical) operator norm k F ( u ) k against the number of stochastic operatorevaluations.
Note that evaluations of k F ( u ) k are only used for plotting but not for running any ofthe algorithms.
We use the same random initialization and tune the batch sizes and step sizes (to thevalues achieving fastest convergence under noise) for each method by grid search.
We use constantbatch sizes and constant step sizes for GDA, EG, and Popov.
We also choose the batch sizes of PAGEestimator to ensure E [ k F ( u k )  e
F ( u k ) k 2 ]  O ( 1 k ) , which handles error accumulation [26] andearly stagnation of stochastic Halpern iteration.
We implement all the algorithms in Python and runeach algorithm using one CPU core on a macOS machine with Intel 2.3GHz
Dual Core i5 Processorand 8GB RAM.
We observe that (i) in Figure 1(a) both Halpern and E-Halpern exhibit faster convergence to approximate stationary points (with much smaller gradient norm after same number of gradient evaluations)than other algorithms, and restarting E-Halpern provides additional speedup, validating our theoreticalinsights; (ii) in Figure 1(b), E-Halpern with PAGE estimator displays faster convergence compared toother two estimators, in agreement with our theoretical analysis.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] The contributions are summarized in Introduction (Section 1) and fully supported by detailed proofs in the main body and the appendix.
(b) Did you describe the limitations of your work?
[Yes] Some speciﬁc examples are: (i) the algorithm being of the Las Vegas-type, in Section 3.2 and (ii) the dependence on problem parameters other than ✏ and  being likely suboptimal in Appendix A.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
This is a
purely theoretical paper.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 2.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Sections 3–5 inthe main body and the appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See supplemental materials.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 6.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We use the same initialization for all methods, and tune [Table] the parameters of each method to be optimal individually.
[Table]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)? [Yes] See Section 6.
[Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 6.
[Table]
(b) Did you mention the license of the assets?
[Yes] See Section 6.
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
[Table]
Our code is included in the supplemental material.
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[N/A]
The data we used is from online public repository.
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[N/A]
The data we used is from online public
[Table]
repository.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table] (c) Did you include the estimated hourly wage paid to participants and the total amount [Table]
spent on participant compensation?
[N/A]
[Table]