Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers

We present a new , stochastic variant of the projective splitting ( PS ) family of algorithms for monotone inclusion problems . It can solve min-max and noncooperative game formulations arising in applications such as robust ML without the convergence issues associated with gradient descent-ascent , the current de facto standard approach in ML applications . Our proposal is the first version of PS able to use stochastic gradient oracles . It can solve min-max games while handling multiple constraints and nonsmooth regularizers via projection and proximal operators . Unlike other stochastic splitting methods that can solve such problems , our method does not rely on a product-space reformulation of the original problem . We prove almost-sure convergence of the iterates to the solution and a convergence rate for the expected residual . By working with monotone inclusions rather than variational inequalities , our analysis avoids the drawbacks of measuring convergence through the restricted gap function . We close with numerical experiments on a distributionally robust sparse logistic regression problem . 1 INTRODUCTION . The most prominent application of optimization in ML is empirical risk minimization . However , inspired by the success of GANs ( Goodfellow et al. , 2014 ) . , ML practitioners have developed more complicated min-max and adversarial optimization formulations ( Yu et al. , 2021 ; Kuhn et al. , 2019 ; Shafieezadeh-Abadeh et al. , 2015 ; Sinha et al. , 2018 ; Lin et al. , 2020 ; Namkoong & Duchi , 2016 ; Huang et al. , 2017 ; Wadsworth et al. , 2018 ; Zhang et al. , 2018 ; Edwards & Storkey , 2015 ; Celis & Keswani , 2019 ) . Solving these multi-player games leads to issues not seen when minimizing a single-player loss function . The competitive nature of a game leads to rotational dynamics that can cause intuitive gradient-based methods to fail to converge ( Gidel et al. , 2019 ; Daskalakis et al. , 2018 ; Hsieh et al. , 2020 ) . A mathematical framework underlying both convex optimization and saddle-point problems is the monotone inclusion problem ; see Ryu & Boyd ( 2016 ) for an introduction . Methods developed for monotone inclusions will converge for convex-concave , games as they are explicitly designed to handle such problems ’ governing dynamics . In recent years , monotone inclusion methods and theory have started to receive attention in the ML community ( Diakonikolas , 2020 ; Liu et al. , 2021 ; Ryu et al. , 2020 ; Pathak & Wainwright , 2020 ) , with a focus on monotone variational inequalities , which form a special case of monotone inclusions ( Antonakopoulos et al. , 2019 ; Gidel et al. , 2019 ; Daskalakis et al. , 2018 ; Hsieh et al. , 2020 ; Mertikopoulos et al. , 2019 ) . The most prevalent methods for solving min-max games in ML are variants of gradient descent-ascent ( GDA ) . This method alternates between a gradient-descent step for the minimizing player and a gradient-ascent step for the maximizing player . Unfortunately , GDA requires additional assumptions to converge on convex-concave games , and it even fails for some simple 2D bilinear games ( Gidel et al. , 2019 , Prop . 1 ) . While there have been several approaches to modify either GDA ( Chavdarova et al. , 2021 ; Grnarova et al. , 2021 ; Balduzzi et al. , 2018 ) or the underlying game objective ( Mescheder et al. , 2018 ; Nagarajan & Kolter , 2017 ; Mescheder et al. , 2017 ) to ensure convergence , this paper instead develops a method for solving monotone inclusions that can naturally handle game dynamics . Our approach builds upon the recently proposed projective splitting ( PS ) method with forward steps ( Johnstone & Eckstein , 2020b ) . PS is designed specifically for solving monotone inclusions , thus does not fall prey to the convergence issues that plague GDA , at least for convex-concave games . PS is within the general class of projective splitting methods invented by Eckstein & Svaiter ( 2008 ) and developed further in Eckstein & Svaiter ( 2009 ) ; Alotaibi et al . ( 2014 ) ; Combettes & Eckstein ( 2018 ) ; Eckstein ( 2017 ) ; Johnstone & Eckstein ( 2019 ; 2021 ; 2020a ) . These methods work by creating a separating hyperplane between the current iterate and the solution and then moving closer to the solution by projecting the current iterate onto this hyperplane ( see Section 3 for an overview ) . Other than being able to natively handle game dynamics , the primary advantage of PS is that it fully splits problems involving an arbitrary number of regularizers and constraints . “ Full splitting ” means that the method can handle multiple regularizers and constraints through their respective individual proximal and projection operators , along with the smooth terms via gradients . What makes this useful is that many of the regularizers used in ML have proximal operators that are relatively easy to compute ; see for example Parikh & Boyd ( 2013 ) . Despite these advantages , the preexisting PS framework has a significant drawback : it requires deterministic gradient oracles . This feature makes it impractical for application to large datasets for which stochastic oracles may be the only feasible option . Contributions The primary contribution of this work is a new projective splitting algorithm that allows for a stochastic gradient oracle . We call the method stochastic projective splitting ( SPS ) . Our method “ fully splits ” the monotone inclusion problem Find z ∈ Rd s.t . 0 ∈ ∑n i=1Ai ( z ) +B ( z ) , ( 1 ) where B is monotone and L-Lipschitz and each Ai is maximal monotone and typically set valued , usually arising from a constraint or a nonsmooth regularizer in the underlying optimization problem or game ; see for example Ryu & Boyd ( 2016 ) for definitions . For some example ML applications of ( 1 ) , see Section 2 and Appendix A . Here , an algorithm that “ fully splits ” ( 1 ) means one whose computational steps each involve only the individual operators A1 , . . . , An , B . Ours is the first method that can accomplish full splitting without a product-space reformulation that recasts ( 1 ) as a two-operator problem on a higher-dimensional space , a tactic whose disadvantages are discussed in Appendix F.7 . Our method interrogates the Lipschitz operator B through a stochastic oracle . Previous methods splitting ( 1 ) have either required a deterministic oracle for B , or have made far more restrictive assumptions on the noise or the operators ( Briceño-Arias & Combettes , 2011 ; Combettes & Pesquet , 2012 ; Malitsky & Tam , 2020 ; Bot et al. , 2019 ; Van Dung & Vu , 2021 ) than we will require below . However , the stochastic methods of Alacaoglu et al . ( 2021 ) and Böhm et al . ( 2020 ) , when combined with a product-space reformulation , can solve ( 1 ) when all the Ai are subdifferentials of convex functions ; see Section 6 . When moving away from a deterministic gradient oracle in projective splitting , a key difficulty is that the generated hyperplanes do not guarantee separation between the solution and the current point . We solve this issue by relaxing the projection : we only update each iterate in the direction of the noisy projection and scale its movement by a decreasing stepsize that allows for control of the stochastic error . Using the framework of stochastic quasi-Fejér monotonicity ( Combettes & Pesquet , 2015 ) , we prove almost-sure convergence of the final iterate and do not require averaging of the iterates ( Theorem 1 , Section 5 ) . We also provide a non-asymptotic convergence rate for the approximation residual ( Theorem 2 , Section 5 ) . A special case of SPS is the recently-developed Double Stepsize Extragradient Method ( DSEG ) ( Hsieh et al. , 2020 ) . When n = 0 and therefore only B is present in ( 1 ) , DSEG and SPS coincide . Thus , our method extends DSEG to allow for regularizers and constraints . Our analysis also provides a new interpretation for DSEG as a special case of projective splitting . Our nonasymptotic convergence rate for SPS also applies to DSEG under no additional assumptions . By contrast , the original convergence rate analysis for DSEG requires either strong monotonicity or an error bound . We close with numerical experiments on a distributionally robust sparse logistic regression problem . This is a nonsmooth convex-concave min-max problem which can be converted to ( 1 ) with n = 2 set-valued operators . On this problems class , SPS compares well to the possible alternative splitting methods . Non-monotone problems The work of Hsieh et al . ( 2020 ) included a local convergence analysis for DSEG applied to locally monotone problems . For min-max problems , if the objective is locally convex-concave at a solution and DSEG is initialized in close proximity , then for small enough stepsizes it converges to the solution with high probability . It is possible to extend this result to SPS , along with our convergence rate analysis . This result is beyond the scope of this work , but Appendix J provides a proof sketch . 2 BACKGROUND ON MONOTONE INCLUSIONS . Since they are so important to SPS , this section provides some background material regarding monotone inclusions , along with their connections to convex optimization , games , and ML . Appendix G discusses their connections to variational inequalities . For a more thorough treatment , we refer to Bauschke & Combettes ( 2017 ) . See Appendix A for a longer discussion of the applications of monotone inclusions to ML along with several examples . Fundamentals Let f : Rd → R ∪ { ∞ } be closed , convex , and proper ( CCP ) . Recall that its subdifferential ∂f is given by ∂f ( x ) .= { g : f ( y ) ≥ f ( x ) + g > ( y − x ) } . The map ∂f has the property u ∈ ∂f ( x ) , v ∈ ∂f ( y ) =⇒ ( u− v ) > ( x− y ) ≥ 0 , and any point-to-set map having this property is called a monotone operator . A monotone operator T is called maximal if no additional points can be included in the image T ( x ) of any x ∈ Rd without violating the above property ( Bauschke & Combettes , 2017 , Def . 20.20 ) . Subgradient maps of CCP functions are maximal ( Bauschke & Combettes , 2017 , Thm . 20.25 ) . A minimizer of f is any x∗ such that 0 ∈ ∂f ( x∗ ) . This is perhaps the simplest example of a monotone inclusion , the problem of finding x such that 0 ∈ T ( x ) , where T is a monotone operator . If f is smooth , then ∂f ( x ) = { ∇f ( x ) } for all x , and the monotone inclusion 0 ∈ ∂f ( x ) is equivalent to the first-order optimality condition 0 = ∇f ( x ) . Under certain regularity conditions ( Bauschke & Combettes , 2017 , Cor . 16.5 ) , minimizing a sum of CCP functions f1 , . . . , fn is equivalent to solving the monotone inclusion formed from the sum of their subdifferentials : x∗ ∈ arg min x∈Rd n∑ i=1 fi ( x ) ⇐⇒ 0 ∈ n∑ i=1 ∂fi ( x ∗ ) . ( 2 ) As throughout this paper for all set addition operations , the summation on the right-hand side of ( 2 ) is the Minkowski sum ∑n i=1 Si = { ∑n i=1 si | si ∈ Si ∀ i ∈ 1 .. n } . For a convex set X , a constraint x ∈ C for some convex set C may be imposed by setting one of the fi to be the indicator function ιC , defined by ιC ( x ) = 0 for x ∈ C and ιC ( x ) = +∞ for x 6∈ C. Indicator functions of closed convex sets are CCP ( Bauschke & Combettes , 2017 , Ex . 1.25 ) , and the subgradient map of ιC is also referred to as the normal cone map NC of C ( Bauschke & Combettes , 2017 , Def . 6.37 ) . Multiple constraints may be imposed by including multiple indicator functions in ( 2 ) . ML applications The form ( 2 ) can be used to model ML problems with multiple constraints and/or nonsmooth regularizers , including sparse and overlapping group lasso ( Jacob et al. , 2009 ) , sparse and low-rank matrix estimation problems ( Richard et al. , 2012 ) , and rare feature selection ( Yan & Bien , 2020 ) ; see Pedregosa & Gidel ( 2018 ) for an overview . Games Consider a two-player noncooperative game in which each player tries to selfishly minimize its own loss , with each loss depending on the actions of both players . Typically , the goal is to find a Nash equilibrium , in which neither player can improve its loss by changing strategy : x∗ ∈ arg min x∈Θ F ( x , y∗ ) and y∗ ∈ arg min y∈Ω G ( x∗ , y ) . ( 3 ) Assuming that the admissible strategy sets Θ ⊆ Rdx and Ω ⊆ Rdy are closed and convex and that F and G are differentiable , then writing the first-order necessary conditions for each optimization problem in ( 3 ) yields 0 ∈ [ ∇xF ( x∗ , y∗ ) ∇yG ( x∗ , y∗ ) ] + ( NΘ ( x ∗ ) ×NΩ ( y∗ ) ) . ( 4 ) IfG = −F , then ( 3 ) is a min-max game . If F is also convex in x and concave in y , thenB : ( x , y ) 7→ ( ∇xF ( x , y ) , −∇yF ( x , y ) ) > is monotone1 on Rdx+dy ( Rockafellar , 1970 ) . In many applications , B is also Lipschitz continuous . In this situation , ( 4 ) is a monotone inclusion involving two operators B and NΘ×Ω , with B being Lipschitz . Using the simultaneous version of GDA on ( 3 ) is equivalent to applying the forward-backward method ( FB ) ( Bauschke & Combettes , 2017 , Thm . 26.14 ) to ( 4 ) . However , convergence of FB requires that the operator B be cocoercive ( Bauschke & Combettes , 2017 , Def . 4.10 ) , and not merely Lipschitz ( Bauschke & Combettes , 2017 , Thm . 26.14 ) . Thus , simultaneous GDA fails to converge for ( 3 ) without additional assumptions ; see Gidel et al . ( 2019 , Prop . 1 ) for a simple counterexample . Regularizers and further constraints may be imposed by adding more operators to ( 4 ) . For example , if one wished to apply a ( nonsmooth ) convex regularizer r : Rdx → R ∪ { +∞ } to the x variables and a similar regularizer d : Rdy → R ∪ { +∞ } to the y variables , one would add the operator A2 : ( x , y ) 7→ ∂r ( x ) × ∂d ( y ) to the right-hand side of ( 4 ) . ML applications of games Distributionally robust supervised learning ( DRSL ) is an emerging framework for improving the stability and reliability of ML models in the face of distributional shifts ( Yu et al. , 2021 ; Kuhn et al. , 2019 ; Shafieezadeh-Abadeh et al. , 2015 ; Sinha et al. , 2018 ; Lin et al. , 2020 ; Namkoong & Duchi , 2016 ) . Common approaches to DRSL formulate the problem as a min-max game between a learner selecting the model parameters and an adversary selecting a worst-case distribution subject to some ambiguity set around the observed empirical distribution . This min-max problem is often further reduced to either a finite-dimensional saddlepoint problem or a convex optimization problem . DRSL is a source of games with multiple constraints/regularizers . One such formulation , based on Yu et al . ( 2021 ) , is discussed in the experiments below . The work in Namkoong & Duchi ( 2016 ) uses an ambiguity set based on f -divergences , while Sinha et al . ( 2018 ) introduce a Lagrangian relaxation of the Wasserstein ball . When applied to models utilizing multiple regularizers ( Jacob et al. , 2009 ; Richard et al. , 2012 ; Yan & Bien , 2020 ) , both of these approaches lead to min-max problems with multiple regularizers . Other applications of games in ML , although typically nonconvex , include generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ; Arjovsky et al. , 2017 ; Loizou et al. , 2020 ; 2021 ; Mishchenko et al. , 2020 ) , fair classification ( Wadsworth et al. , 2018 ; Zhang et al. , 2018 ; Edwards & Storkey , 2015 ; Celis & Keswani , 2019 ) , and adversarial privacy ( Huang et al. , 2017 ) . Resolvents , proximal operators , and projections A fundamental computational primitive for solving monotone inclusions is the resolvent . The resolvent of a monotone operator A is defined to be JA . = ( I + A ) −1 , where I is the identity operator and the inverse of any operator T is simply T−1 : x 7→ { y : Ty 3 x } . If A is maximal monotone , then for any ρ > 0 , JρA is single valued , nonexpansive , and has domain equal to Rd ( Bauschke & Combettes , 2017 , Thm . 21.1 and Prop . 23.8 ) . Resolvents generalize proximal operators of convex functions : the proximal operator of a CCP function f is proxρf ( t ) . = arg min x∈Rd { ρf ( x ) + ( 1/2 ) ‖x− t‖2 } . It is easily proved that proxρf = Jρ∂f . Like proximal operators , resolvents generalize projection onto convex sets : if f = ιC , then JρNC = proxρf = projC for any ρ > 0 . In many ML applications , proximal operators , and hence resolvents , are relatively straightforward to compute . For examples , see Parikh & Boyd ( 2013 , Sec . 6 ) . Operator splitting methods Operator splitting methods attempt to solve monotone inclusions such as ( 1 ) by a sequence of operations that each involve only one of the operators A1 , . . . , An , B . Such methods are often presented in the context of convex optimization problems like ( 2 ) , but typically apply more generally to monotone inclusions such as ( 1 ) . In the specific context of ( 1 ) , each iteration of such a method ideally handles each Ai via its resolvent and the Lipschitz operator B by explicit ( not stochastic ) evaluation . This is a feasible approach if the original problem can be decomposed in 1Sufficient conditions for the monotonicity of ( 4 ) in the case where G 6= −F are discussed in e.g . Scutari et al . ( 2014 ) ; Briceño-Arias & Combettes ( 2013 ) . such a way that the resolvents of each Ai are relatively inexpensive to compute , and full evaluations of B are possible . Although not discussed here , more general formulations in which matrices couple the arguments of the operators can broaden the applicability of operator splitting methods .

Summary: The paper focuses on the stochastic variant of the projective splitting (PS) algorithm. With a specific focus on monotone inclusion problems, the authors propose a novel separable algorithm featured by the ability to handle multiple constraints and non-smooth regularizers. Compared with similar approaches on variational inequality which is a special case of monotone inclusions, this paper uses a more direct error metric than the restricted gap function. Moreover, although with a slower convergence rate, this paper is the first discussion under the general discontinuous monotone inclusion case.