High Probability Generalization Bounds for Minimax Problems with Fast Rates

√ n ) , where n is the sample size . In this paper , we provide improved generalization analyses and obtain sharper high probability generalization bounds for almost all existing generalization measures of minimax problems . We then use the improved learning bounds to establish high probability generalization bounds with fast rates for classical empirical saddle point ( ESP ) solution and several popular gradient-based optimization algorithms , including gradient descent ascent ( GDA ) , stochastic gradient descent ascent ( SGDA ) , proximal point method ( PPM ) , extra-gradient ( EG ) , and optimistic gradient descent ascent ( OGDA ) . In summary , we provide a systematical analysis of sharper generalization bounds of minimax problems . 1 INTRODUCTION . Minimax learning problems have achieved great success over a broad range of learning tasks in machine learning , with examples including reinforcement learning ( Du et al. , 2017 ; Dai et al. , 2018 ) , robust optimization ( Chen et al. , 2017 ; Namkoong & Duchi , 2017 ) , adversarial learning ( Goodfellow et al. , 2014 ) , distributed computing ( Razaviyayn et al. , 2020 ; Shamma , 2008 ; Mateos et al. , 2010 ) , and AUC maximization ( Lei & Ying , 2021b ) , to just name a few . This framework is formulated as a zero-sum game characterized as two groups of decision variables , one for minimization and one for maximization . The coupling of the two groups of variables makes analysis of minimax problems more complex than the standard statistical learning theory setting , with only one minimization operator ( Lei et al. , 2021 ) . Researchers have designed various optimization algorithms , for instance , gradient descent ascent ( GDA ) , stochastic gradient descent ascent ( SGDA ) , proximal point method ( PPM ) , extra-gradient ( EG ) , and optimistic gradient descent ascent ( OGDA ) , to solve the minimax optimization problem ( Farnia & Ozdaglar , 2021 ) . Current theoretical research in ML literature is mainly devoted to the convergence rate and optimality of these minimax optimization algorithms in different setting , such as convex-concave settings ( Nemirovski et al. , 2008 ) , nonconvex-concave setting ( Rafique et al. , 2018 ) , strongly convex-strongly-concave setting ( Balamurugan & Bach , 2016 ) , and nonconvex-nonconcave setting ( Liu et al. , 2021 ; Yang et al. , 2020 ) . In contrast , there is far less work on the generalization performance analysis , which is an important measure to indicate the performance of the learned model based on training samples when generalized to the test data . To the best of our knowledge , there is only three work on the generalization bounds of minimax optimization algorithms ( Zhang et al. , 2021a ; Farnia & Ozdaglar , 2021 ; Lei et al. , 2021 ) . Among them , ( Zhang et al. , 2021a ) studies the generalization bounds for ESP solution to minimax problems , ( Farnia & Ozdaglar , 2021 ) analyzes the generalization properties of several gradient-based optimization algorithms : GDA , SGDA , GDmax and PPM , and ( Lei et al. , 2021 ) provides a systematical generalization analysis of SGDA . However , in the above-mentioned papers , almost all generalization bounds are derived in expectation . Only two high probability bounds exist , proposed in ( Lei et al. , 2021 ) . Unfortunately , they are of slow order O ( 1/ √ n ) . It is known that the high probability bound is beneficial to understand the robustness of optimization algorithms ( Bousquet et al. , 2020 ; Klochkov & Zhivotovskiy , 2021 ) and is much more challenging to be derived ( Bousquet et al. , 2020 ; Lei et al. , 2021 ) . In this paper , our goal is to provide the sharper high probability generalization bounds for minimax learning problems . We leverage the lens of algorithmic stability , which is also served as an important tool in ( Zhang et al. , 2021a ; Farnia & Ozdaglar , 2021 ; Lei et al. , 2021 ) . Our contributions are summarized below . 1 . In view of the coupling construction between the minimization variable and the maximization variable , minimax learning problems have many generalization measures ( Lei et al. , 2021 ; Farnia & Ozdaglar , 2021 ; Zhang et al. , 2021a ) . In this paper , we provide improved stability analyses for almost all existing generalization measures , based on which we establish sharper high probability generalization bounds . These developed learning bounds can be employed to derive generalization bounds with fast rates for stable minimax learning algorithms . 2 . The generalization performance of the ESP solution and gradient-based optimization algorithms stands a central place in the learning theory of minimax problems ( Lei et al. , 2021 ) . In this paper , we develop high probability generalization bounds with fast rates for ESP solution and several popular gradient-based optimization algorithms : GDA , SGDA , PPM , EG , and OGDA . Overall , we provide a systematical analysis of sharper generalization bounds for minimax learning problems . The rest of this paper is organized as follows . In Section 2 , we briefly discuss the related work . Preliminaries about minimax problems are introduced in Section 3 . We provide the main theorem in Section 4 . Applications are shown in Section 5 . In the last , Section 6 concludes the paper . 2 RELATED WORK . Algorithmic stability . Algorithmic stability is a fundamental concept in learning theory ( Bousquet & Elisseeff , 2002 ) , which has a deep connection with learnability ( Rakhlin et al. , 2005 ; ShalevShwartz & Ben-David , 2014 ; Shalev-Shwartz et al. , 2010 ) . A training algorithm is stable if small changes in the training set lead to small differences in the output predictions of the trained model . Different algorithmic stability measures have been developed , including uniform stability ( Bousquet & Elisseeff , 2002 ; Feldman & Vondrak , 2018 ; 2019 ; Klochkov & Zhivotovskiy , 2021 ; Hardt et al. , 2016 ; Lei et al. , 2020 ) , uniform argument stability ( Liu et al. , 2017 ; Bassily et al. , 2020 ) , hypothesis stability ( Bousquet & Elisseeff , 2002 ; Charles & Papailiopoulos , 2018 ) , hypothesis set stability ( Foster et al. , 2019 ) , on average stability ( Shalev-Shwartz et al. , 2010 ; Lei & Ying , 2020 ; Kuzborskij & Lampert , 2018 ; Zhang et al. , 2021b ; Lei & Ying , 2021a ) , locally elastic stability ( Deng et al. , 2021 ) , collective stability ( London , 2017 ) , and PAC-Bayesian stability ( Li et al. , 2020 ) . These stability measures have been extensively studied in the generalization analysis of the standard statistical learning theory setting ( Chen et al. , 2018 ; Zhang et al. , 2021b ) . Several stability measures have also been extended to minimax learning problems , for instance , weak stability , argument stability , and uniform stability ( Farnia & Ozdaglar , 2021 ; Zhang et al. , 2021a ; Lei et al. , 2021 ) . In related work ( Farnia & Ozdaglar , 2021 ; Zhang et al. , 2021a ; Lei et al. , 2021 ) , they mostly focus on the expectation form of these stability measures since they are to derive bounds in expectation . In this paper , we will focus on the last two measures , which are often used when establishing high probability generalization bounds ( Feldman & Vondrak , 2018 ; 2019 ; Klochkov & Zhivotovskiy , 2021 ) . Convergence analysis . Convergence analysis has been widely studied in different settings , including convex-concave learning ( Nemirovski , 2005 ; Nedic & Ozdaglar , 2009 ; Mokhtari et al. , 2020 ; Cherukuri et al. , 2017 ; Mokhtari et al. , 2019 ; Balamurugan & Bach , 2016 ; Hsieh et al. , 2019 ; Yan et al. , 2020 ; Lin et al. , 2020b ; Wang & Li , 2020 ; Yoon & Ryu , 2021 ) , nonconvex-concave learning ( Rafique et al. , 2018 ; Kong & Monteiro , 2019 ; Luo et al. , 2020 ; Grnarova et al. , 2017 ; Thekumparampil et al. , 2019 ; Lu et al. , 2020 ; Namkoong & Duchi , 2016 ; Sanjabi et al. , 2018 ; Nouiehed et al. , 2019 ; Lin et al. , 2020a ; Sinha et al. , 2017 ; Chen et al. , 2021 ) , and nonconvex-nonconcave learning ( Heusel et al. , 2017 ; Balduzzi et al. , 2018 ; Daskalakis & Panageas , 2018 ; Mertikopoulos et al. , 2019 ; Loizou et al. , 2020 ; Yang et al. , 2020 ; Liu et al. , 2021 ; Lin et al. , 2018 ; Diakonikolas et al. , 2021 ; Wang et al. , 2020 ; Loizou et al. , 2021 ; Fiez & Ratliff , 2021 ) . There are so many studies on convergence , considering the length limit , the references listed here are not complete . Please refer to the related references concerning the above work . We investigate the generalization performance of minimax problems instead of the convergence behavior . Note that the convergence analysis also plays an essential role in this paper , formalized as strong PD empirical risk ( please refer to Defi- nition 1 ) , which is defined on the function value difference and referred to as optimization error or primal-dual gap in some convergence literature ( Lei et al. , 2021 ; Nemirovski , 2005 ; Mokhtari et al. , 2019 ; 2020 ) . 3 PRELIMINARIES . Let X and Y be two parameter spaces in Rd . Let P be a probability measure defined on a sample space Z . We define f : X ×Y ×Z 7→ R and consider the following minimax optimization problem min x∈X max y∈Y F ( x , y ) : = Ez∼P [ f ( x , y ; z ) ] . ( 1 ) The above minimax objective represents a expectation of a cost function f ( x , y ; z ) for minimization variable x , maximization variable y and data variable z . Unfortunately , we typically are not available to the underlying distribution P. In practice , F is approximated by the corresponding empirical risk . Let S = { z1 , ... , zn } be a dataset which are independent drawn according to P , the empirical risk is defined as FS ( x , y ) = 1 n n∑ i=1 f ( x , y ; zi ) . ( 2 ) Let the output of a ( randomized ) algorithmA on a dataset S beA ( S ) : = ( Ax ( S ) , Ay ( S ) ) ∈ X ×Y . Since A ( S ) is just an empirical approximated solution of the true minimax optimization problem , we are interested in studying how well A ( S ) generalizes to the unseen data . As claimed in ( Farnia & Ozdaglar , 2021 ; Lei et al. , 2021 ) , the coupling between the minimization variable and the maximization variable in ( 1 ) makes minimax problems have many different generalization performance measures . These measures are collected in ( Lei et al. , 2021 ) . For better readability , we use their symbols . Let E be the expectation with respect to ( w.r.t . ) the randomness of algorithm A and the dataset S. These generalization measures are listed below . Definition 1 . ( Lei et al. , 2021 ) There are four groups of generalization measures . 1 ( Primal Measures . ) The primal population risk of a model x is defined as R ( x ) = supy∈Y F ( x , y ) , and the corresponding primal empirical risk is defined as RS ( x ) = supy∈Y FS ( x , y ) . Then , when using empirical risk RS ( x ) to bound R ( x ) , we call this error of the model x the primal generalization error . While using optimal infx∈X R ( x ) to bound R ( x ) , we call this error of the model x the excess primal population risk . 2 ( Plain Measure . ) When using FS ( x , y ) to bound F ( x , y ) , we call the this error of a model ( x , y ) the plain generalization error . 3 ( Strong Measures . ) The strong primal-dual ( PD ) population risk of a model ( x , y ) is defined as 4s ( x , y ) = sup y′∈Y F ( x , y′ ) − inf x′∈X F ( x′ , y ) , and the corresponding strong PD empirical risk is defined as 4sS ( x , y ) = sup y′∈Y FS ( x , y ′ ) − inf x′∈X FS ( x ′ , y ) . Then , the strong PD generalization error4s ( x , y ) −4sS ( x , y ) of the model ( x , y ) is defined as ( sup y′∈Y F ( x , y′ ) − sup y′∈Y FS ( x , y ′ ) ) + ( inf x′∈X FS ( x ′ , y ) − inf x′∈X F ( x′ , y ) ) . 4 ( Weak Measures . ) The weak PD population risk of a ( randomized ) model ( x , y ) is defined as 4w ( x , y ) = sup y′∈Y E [ F ( x , y′ ) ] − inf x′∈X E [ F ( x′ , y ) ] , and the corresponding weak PD empirical risk is defined as 4wS ( x , y ) = sup y′∈Y E [ FS ( x , y′ ) ] − inf x′∈X E [ FS ( x′ , y ) ] . Then , the weak PD generalization error4w ( x , y ) −4wS ( x , y ) of the model ( x , y ) is defined as ( sup y′∈Y E [ F ( x , y′ ) ] − sup y′∈Y E [ FS ( x , y′ ) ] ) + ( inf x′∈X E [ FS ( x′ , y ) ] − inf x′∈X E [ F ( x′ , y ) ] ) . Remark 1 . We provide some discussions for the four groups of measures . ( 1 . Primal Measures : ) In the context of GANs , the primal population risk R ( x ) represents a divergence measure between the learned and true distributions , and in the context of adversarial training it represents the learner ’ s risk under adversarial perturbations ( Farnia & Ozdaglar , 2021 ) . One would be interested in the relationship between R ( x ) and its corresponding empirical risk RS ( x ) , and the relationship between R ( x ) and its infimum infx′∈X R ( x′ ) . ( 2 . Plain Measure : ) This generalization measure is a direct extension of the standard generalization error in the minimization optimization . ( 3 . Strong Measures : ) 4sS ( x , y ) is referred to as the primal-dual gap in the optimization literature . 4s ( x , y ) is the primal-dual gap of the population risk . 4s ( x , y ) − 4sS ( x , y ) studies the difference between the population primal-dual gap and its empirical counterpart . ( 4 . Weak Measures : ) The difference between the strong and weak measures is that weak measures take the expectation over the randomness of the dataset and the algorithm , for instance , supy′∈Y F ( x , y ′ ) − infx′∈X F ( x′ , y ) in the strong measures and supy′∈Y E [ F ( x , y′ ) ] − infx′∈X E [ F ( x′ , y ) ] in the weak measures . Therefore , the upper bounds of weak measures hold in expectation , while the upper bounds of strong measures hold uniformly for any dataset . Denote the Lp norm of a random variable z ∈ Z as ‖z‖p = ( Ez|z|p ) 1/p . Let ‖ · ‖ be the Euclidean norm and 〈· , ·〉 be the inner product . A differentiable function g : W 7→ R is called µ-stronglyconvex in w if the following inequality holds for every w1 , w2 : g ( w1 ) − g ( w2 ) ≥ 〈∇g ( w2 ) , w1 −w2〉+ µ 2 ‖w1 −w1‖2 , where∇ is the gradient operator . We say g is µ-strongly-concave if −g is µ-strongly-convex . Definition 2 . Let g : X × Y 7→ R. Assume that X and Y are convex feasible sets . Then 1. g is µ-strongly-convex-strongly-concave ( µ-SC-SC ) if g ( · , y ) is µ-strongly-convex for any y ∈ Y and g ( x , · ) is µ-strongly-concave for any x ∈ X . 2. g is convex-concave ( C-C ) if g is 0-SC-SC . We then introduce the definition of algorithmic stability this paper used . Algorithmic stability plays an important role in studying the generalization behavior of a learning algorithm . Intuitively , an algorithm A : Zn 7→ ( X , Y ) is said to be stable if the output model ( Ax ( S ) , Ay ( S ) ) is insensitive to perturbations . Let S′ be a neighboring dataset that differs at most one single example to S. Definition 3 ( Algorithmic Stability ) . Let A be a learning algorithm and > 0 . 1 . We say A is -uniformly-stable if for any training datasets S , S′ ∈ Zn we have sup z [ f ( Ax ( S ) , Ay ( S ) ; z ) − f ( Ax ( S′ ) , Ay ( S′ ) ; z ) ] ≤ . 2 . We say A is -argument-stable if for any training datasets S , S′ ∈ Zn we have ‖Ax ( S ) −Ax ( S′ ) ‖+ ‖Ay ( S ) −Ay ( S′ ) ‖ ≤ . From Definition 3 , one can see that the uniform stability measures the sensitivity of the function values , while the argument stability measures the sensitivity of the arguments . We finally introduce two standard assumptions in minimax problems . Assumption 1 implies f is Lipschitz continuous w.r.t . both x and y , while Assumption 2 implies f is smooth w.r.t . ( x , y ) . Assumption 1 ( Lipschitz continuity ) . Let L > 0 . Assume that for any x ∈ X , y ∈ Y and z ∈ Z , f ( x , y ; z ) satisfies ‖∇xf ( x , y ; z ) ‖ ≤ L and ‖∇yf ( x , y ; z ) ‖ ≤ L. Assumption 2 ( Smoothness ) . Let β > 0 . Assume that for any x1 , x2 ∈ X , y1 , y2 ∈ Y and z ∈ Z , f ( x , y ; z ) satisfies∥∥∥∥ ( ∇xf ( x1 , y1 ; z ) −∇xf ( x2 , y2 ; z ) ∇yf ( x1 , y1 ; z ) −∇yf ( x2 , y2 ; z ) ) ∥∥∥∥ ≤ β ∥∥∥∥ ( x1 − x2y1 − y2 ) ∥∥∥∥ . Under Assumption 1 , the argument stability implies the uniform stability . Therefore , the argument stability is the main stability measure that we will focus on .

Summary: This paper consider a stochastic minimax problem of the form $\min_x \max_y F(x,y)$ where $F(x,y) = \mathbb{E}[f(x,y,z)]$. The goal is to obtain ``fast'' rates of $O(1/n)$ with a training set of $n$ i.i.d. $z_1,\dots,z_n$ in high probability. In this setting there are a variety of convergence measures one might consider, most notably perhaps the gap $\Delta(x,y) = \max_{\hat y} F(x,\hat y) - \min_{\hat x} F(\hat x, y)$. For these measures the fast rates are achieved for various specific problems such as strongly-convex-concave saddle-point detection. The main technique is to generalize classical results on algorithmic stability developed for stochastic minimization towards the stochastic minimax problem, and then to demonstrate that a variety of algorithms (mostly variants on gradient descent) are stable in particular settings.