Sifting through the noise: Universal first-order methods for stochastic variational inequalities

√ T ) for absolute noise profiles , and O ( 1/T ) for relative ones . Subsequently , we drop all prior knowledge requirements ( the absolute/relative variance of the randomness affecting the problem , the operator ’ s cocoercivity constant , etc . ) , and we analyze an adaptive instance of the method that gracefully interpolates between the above rates – i.e. , it achieves O ( 1/ √ T ) and ( 1/T ) in the absolute and relative cases , respectively . To our knowledge , this is the first universality result of its kind in the literature and , somewhat surprisingly , it shows that an extra-gradient proxy step is not required to achieve optimal rates . 1 Introduction . This paper focuses on solving variational inequality problems of the form Find x∗ ∈ Rd such that ⟨A ( x∗ ) , x− x∗⟩ ≥ 0 for all x ∈ Rd , ( VI ) where A : Rd → Rd is a monotone cocoercive operator , i.e. , ⟨A ( y ) −A ( x ) , y − x⟩ ≥ β∥A ( y ) −A ( x ) ∥2 for some β > 0 and all x , y ∈ Rd . ( CC ) The study of variational inequalities is a classical topic in optimization that provides a powerful and elegant unifying framework for a broad spectrum of “ convex-structured ” problems – including 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . convex minimization , saddle-point problems , and games [ 6 , 13 ] . In particular , such problems have recently attracted considerable attention in the fields of machine learning ( ML ) and data science because of their potential applications to generative adversarial networks [ 16 ] , multi-agent and robust reinforcement learning [ 40 ] , auction theory [ 47 ] , and many other areas of interest where the minimization of a single empirical loss function does not suffice . The golden standard for solving ( VI ) is provided by first-order methods : these methods can be run with computationally cheap updates that only require ( noisy ) access to A , so they are ideal for problems with very high dimensionality and moderate-to-low precision needs ( as is typically the case in ML ) . More precisely , when A is monotone cocoercive as above , the min-max optimal convergence rate for solving ( VI ) is O ( 1/T ) after T oracle calls , and it is achieved by the extra-gradient / mirrorprox algorithm [ 25 , 34 ] with Polyak-Rupert averaging [ 42 ] . However , this method requires access to a perfect oracle ; if the method is run with an imperfect , stochastic first-order oracle , its convergence rate drops to O ( 1/ √ T ) [ 23 ] , and this rate can not be improved without additional assumptions [ 36 ] . One case where the O ( 1/ √ T ) convergence rate can be improved is when the underlying operator is strongly monotone – i.e. , the RHS of ( CC ) is replaced by α∥y − x∥2 for some α > 0 . In this case , we can obtain a fast O ( 1/T ) rate with a rapidly decreasing step-size [ 17 ] ; however , this acceleration requires knowledge of the strong monotonicity modulus , and there is no known way to adapt to it . In particular , if a stochastic method that has been fine-tuned for strongly monotone operators is run on a merely monotone problem , its rate of convergence suffers a catastrophic drop to O ( 1/ log T ) . These considerations naturally lead to two key research questions : 1 . Are there any conditions for the method ’ s oracle that would close the stochastic-deterministic convergence gap outlined above ? 2 . Is it possible to design a class of methods that are capable of adapting to the quality of the oracle , and that achieve order-optimal rates without prior knowledge of the problem ’ s parameters ? Our contributions in the context of related work . Our goal in this paper is to provide a range of positive answers to the above questions , both in terms of the required oracle conditions , as well as methods that are able to gracefully interpolate between an O ( 1/T ) and an O ( 1/ √ T ) rate depending on the setting at hand . With regard to the first question , our point of departure is the “ relative noise ” framework of Polyak [ 41 ] , in which the variance of the oracle is upper bounded by the square norm of the operator at the queried point . This noise model is particularly relevant in coordinate descent methods for unconstrained problems as well as applications to control theory and signal processing where the operator is calculated based on actual , physical measurements that are only accurate up to a percentage of their true value . In recent applications to ML , this noise model has also been studied in the context of overparametrization [ 39 ] , representation learning [ 51 ] , and multi-agent learning [ 28 ] . Finally , this oracle model has also been studied under the umbrella of multiplicative noise [ 21 ] or growth conditions [ 8 , 45 , 48 , 50 ] , and it is known to improve the convergence rate of stochastic gradient algorithms with non-adaptive step-sizes , even in non-smooth problems [ 14 ] . With regard to the second question , we introduce a flexible first-order algorithmic template that includes as special cases the dual averaging [ 38 ] , dual extrapolation [ 37 ] and optimistic gradient methods [ 43 , 44 ] , and which accounts for both adaptive and non-adaptive variants thereof . Our contributions can then be summarized as follows : 1 . For oracles with bounded variance , we show that the proposed methods achieve an O ( 1/ √ T ) rate of convergence if run with a non-adaptive , decreasing step-size . 2 . In the relative noise model , this rate improves to O ( 1/T ) , and it is achieved with a constant step-size that does need to be tuned as a function of T . 3 . Finally , we provide an adaptive step-size rule that allows the method to achieve a fast , O ( 1/T ) rate under relative noise , and an order-optimal O ( 1/ √ T ) rate in the absolute noise case . Importantly , our work shows that an extra-gradient mechanism is not required to obtain a fast O ( 1/T ) rate , as this can be achieved by vanilla dual-averaging methods with a constant step-size . This is an elegant consequence of the interplay between cocoercivity and the relative noise model ; to the best of our knowledge , the only other work considering these models in tandem is the very recent paper [ 28 ] . Lipschitz Cocoercive + rel . noise Vt Ergodic Last Iterate Ergodic Last Iterate Adapt . dual averaging 0 1/ √ T [ 11 ] Unknown 1/T Asym . Adapt . dual extrapolation AXt + rel.noise 1/ √ T [ 44 ] Unknown 1/T Asym . Adapt . optimistic gradient AXt−1/2 + rel.noise 1/ √ T [ 12 ] Unknown 1/T Asym . Table 1 : The best known convergence rates in stochastic monotone VIs with our contributions highlighted in gray . Adaptive refers to our particular adaptive step-size choice in ( Adapt ) . We obtain various schemes with particular choices of Vt. For the nomenclature , please refer to Section 3.2 . Our work closes several open threads in [ 28 ] , which requires a vanishing relative noise level to obtain faster convergence in models with relative noise . A summary of our results in the context of related work can be found in Table 1 , and we also elaborate on related work in greater detail in the paper ’ s appendix . 2 Problem setup and preliminaries . Examples and motivation . Throughout the sequel , we will focus on solving the variational inequality problem ( VI ) . For completeness ( and a certain degree of posterity ) , we briefly mention some examples below , and we defer to [ 13 , 46 ] for a panoramic survey of the field . Example 1 ( Convex Minimization ) . If A = ∇f for some convex function f , the solutions of ( VI ) are precisely the minimizers of f . Example 2 ( Min-Max Problems ) . If A = ( ∇x1L , −∇x2L ) for some convex-concave function L ( x1 , x2 ) , then the solutions of ( VI ) coincide with the ( global ) saddle points of L. More precisely , x∗ = ( x∗1 , x ∗ 2 ) is a solution of ( VI ) if and only if it holds that L ( x∗1 , x2 ) ≤ L ( x∗1 , x∗2 ) ≤ L ( x1 , x∗2 ) for all x1 ∈ X1 , x2 ∈ X2 . ( SP ) In this case , ( VI ) is sometimes referred to as the “ vector field formulation ” of ( SP ) . Example 3 ( Monotone games ) . Going beyond the min-max setting , a continuous game in normal form is defined as follows : First , consider a finite set of players N = { 1 , . . . , N } , each with their own action space Xi = Rdi . During play , each player selects an action xi from Xi with the aim of minimizing a loss ℓi ( xi ; x−i ) determined by the ensemble x : = ( xi ; x−i ) : = ( x1 , . . . , xN ) of all players ’ actions . In this context , a Nash equilibrium is any action profile x∗ ∈ X that is unilaterally stable , i.e. , ℓi ( x ∗ i ; x ∗ −i ) ≤ ℓi ( xi ; x∗−i ) for all xi ∈ Xi and all i ∈ N . ( NE ) The corresponding operator associated to the game is A ( x ) = ( ∇xiℓi ( xi ; x−i ) ) i∈N . If A is monotone , then the game is itself called monotone , and its Nash equilibria coincide with the solutions of ( VI ) , cf . [ 7 , 13 , 27 , 30–33 , 46 ] and references therein . Regularity conditions . As we discussed in the introduction , our blanket regularity assumption for ( VI ) is that the defining operator A is β-cocoercive in the sense of ( CC ) ; for a panoramic overview of cocoercive operators we refer the reader to [ 6 ] . Some further comments for the cocoercivity condition are in order . First , one may easily observe that if A is β-cocoercive , it is also 1/β-Lipschitz . The converse does not hold for the general setting of operators ; however , when A is the gradient of a smooth convex function , this is indeed the case [ 5 ] . Moreover , even though cocoercivity implies that A is monotone , it does not imply that it is strictly monotone – a condition which is usually invoked to ensure the existence and uniqueness of solutions to ( VI ) . Therefore , to avoid pathologies , we make the following assumption for our setting : Assumption 1 . The set X ∗ = { x∗ ∈ Rd : x∗ is a solution of ( VI ) } is non-empty . Together with cocoercivity , the existence of a solution will be our only blanket assumption in the sequel . The gap function . With the above setup in hand , a widely used performance measure in order to evaluate a candidate solution of ( VI ) is the so-called restricted gap function : GapC ( x̂ ) = sup x∈C ⟨A ( x ) , x̂− x⟩ , ( Gap ) where the `` test domain '' C is a non-empty compact subset of Rd . The motivation for this choice of merit function is that it characterizes the solutions of the ( VI ) via its zeros . Formally , we have the following : Proposition 1 . Let C be a non-empty convex subset of Rd . Then , the following holds

Summary: This paper proposes universal adaptive stochastic gradient-based methods for variational inequalities.  There are at least three remarkable contributions. Firstly, it unifies several popular dual averaging algorithms, including stochastic dual averaging, stochastic dual extrapolation, and stochastic optimistic dual averaging. Secondly, with an AdaGrad-type stepsize that is parameter-free, the algorithms achieve an $O(1/\sqrt{T})$ complexity (for the restricted gap) when the stochastic gradients have absolutely bounded variances and an $O(1/T)$ complexity when the the stochastic gradients have relatively bounded variances, without tweaking the hyperparameters. This gives universal and adaptive algorithms. Lastly, the paper shows the asymptotic convergence of the last iterate, which is usually the output in practice.