What is a Good Metric to Study Generalization of Minimax Learners?
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Minimax optimization has served as the backbone of many machine learning (ML)problems.
Although the convergence behavior of optimization algorithms has beenextensively studied in minimax settings, their generalization guarantees in thestochastic setting, i.e., how the solution trained on empirical data performs on theunseen testing data, have been relatively underexplored.
A fundamental questionremains elusive: What is a good metric to study generalization of minimax learners?
In this paper, we aim to answer this question by ﬁrst showing that primal risk , auniversal metric to study generalization in minimization, fails in simple examplesof minimax problems.
Furthermore, another popular metric, the primal-dual risk ,also fails to characterize the generalization behavior for minimax problems withnonconvexity, due to non-existence of saddle points.
We thus propose a new metricto study generalization of minimax learners: the primal gap , to circumvent theseissues.
Next, we derive generalization bounds for the primal gap in nonconvexconcave settings.
As byproducts of our analysis, we also solve two open questions:establishing generalization bounds for primal risk and primal-dual risk in thestrong sense, i.e., without strong concavity or assuming that the maximization andexpectation can be interchanged, while either of these assumptions was needed inthe literature.
Finally, we leverage this new metric to compare the generalizationbehavior of two popular algorithms – gradient descent-ascent (GDA) and gradientdescent-max (GDMax) in stochastic minimax optimization.
Introduction
Stochastic minimax optimization, a classical and fundamental problem in operations research andgame theory, involves solving the following problem:
[Equation]
More recently, such minimax formulations have received increasing attention in machine learning,with signiﬁcant applications in generative adversarial networks (GANs)
[Goodfellow et al., 2014],adversarial learning [Madry et al., 2017], and reinforcement learning [Chen and Wang, 2016, Dai et al.
Most existing works have focused on the optimization aspect of the problem, i.e., studying therates of convergence, robustness, and optimality of several algorithms, in various minimax settingsincluding convex-concave
[Nemirovski et al., 2009, Monteiro and Svaiter, 2010], nonconvex-concave[Lin et al., 2020, Raﬁque et al., 2018], and certain special nonconvex-nonconcave [Nouiehed et al.
,2019, Yang et al., 2020] problems.
However, the optimization aspect is not the only venue for understanding the success of minimaxoptimization in machine learning.
In particular, as in classical supervised learning, which is usuallystudied as a minimization problem [Hastie et al., 2009], the out-of-sample generalization performance is a key metric for evaluating the learned models.
The study of generalization guarantees in minimaxoptimization (and related machine learning problems) has not received signiﬁcant attention untilrecently [Arora et al., 2017, Feizi et al., 2020, Yin et al., 2019, Lei et al., 2021, Farnia and Ozdaglar,2021, Zhang et al., 2021b].
Speciﬁcally, existing works along this line have investigated two types ofgeneralization guarantees: uniform convergence generalization bounds, and algorithm-dependentgeneralization bounds.
The former is more general and irrespective of the optimization algorithmsbeing used, while the latter is usually ﬁner and really explains what happens in practice, whenoptimization algorithms play an indispensable role.
In fact, the former might not be able to explaingeneralization in deep learning [Nagarajan and Kolter, 2019], making the latter a more favorablemetric for understanding the success of minimax optimization in machine learning.
Algorithm-dependent generalization for minimax optimization has been studied recently in [Farniaand Ozdaglar, 2021, Lei et al., 2021, Xing et al., 2021, Yang et al., 2022, Li and Liu, 2021].
Thesepapers build on the algorithmic stability framework developed in [Bousquet and Elisseeff, 2002].
Inparticular, these works have studied primal risk and/or (variants of) primal-dual risk under differentconvexity and smoothness assumptions of the objective.
Primal risk (see formal deﬁnition in §2) is anatural extension of risk from minimization problems.
Primal-dual risk, on the other hand, is deﬁnedsimilarly but based on the duality gap of the solution, which makes sense when the saddle-pointexists (i.e., min and max can be interchanged).
Based on these metrics, [Farnia and Ozdaglar, 2021,
Lei et al., 2021, Li and Liu, 2021], compare the performance of speciﬁc algorithms, e.g., gradientdescent-ascent (GDA) and gradient descent-max (GDMax).
We provide a more thorough literaturereview and comparison of results in Appendix A.
Although these metrics are natural extensions of generalization metrics from the minimization setting,they might not be the most suitable ones for studying generalization in stochastic minimax optimization, especially in the nonconvex settings that is pervasive in machine/deep learning applications.
Inparticular, we are interested in the following fundamental question:
What is a good metric to study generalization of minimax learners?
In this paper, we make an initial attempt to answering this question, by identifying the inadequaciesof existing metrics, and proposing a new metric, the primal gap that overcomes these inadequacies.
We then provide generalization bounds for the newly proposed metric, and discuss how it capturesinformation not included in the other existing metrics.
We summarize our contributions as follows.
Contributions.
First, we introduce an example through which we identify the inadequacies ofprimal risk , a well-studied metric for minimax generalization, in capturing the generalization behaviorof nonconvex-concave minimax problems.
Second, to address the issue, we propose a new metric –the primal gap , which provably avoids the issue in the example, and derive its generalization bounds.
Next, we leverage this new metric to compare the generalization behavior of GDA and GDMax,two popular algorithms for minimax optimization and GAN training, and answer the question ofwhen does GDA generalize better than GDMax?
Moreover, we also address two open questionsin the literature: establishing generalization bounds for primal risk and primal-dual risk withoutstrong concavity or assuming that the maximization and expectation can be interchanged, whileeither of these assumptions was needed in the literature [Farnia and Ozdaglar, 2021, Lei et al., 2021,
Xing et al., 2021, Yang et al., 2022] (see Section §E due to space limitation).
Finally, under certainassumptions of the max learner, our results also generalize to the nonconvex-nonconcave setting.
Preliminaries2.1 Problem formulation
In this paper, we consider the following (stochastic) minimax problem:
[Equation]
[Equation]
We make the following assumption on the sets W and Θ throughout the paper.
Assumption 1.
W and Θ are convex, closed sets, and we further assume that W is compact with(cid:107) w (cid:107) ≤ M ( W ) , ∀ w ∈ W .
Here M ( W ) is a constant dependent on the set W .
Let r ( w, θ ) =
E z ∼ P
z
f
( w, θ ; z ) .
For a training dataset S = { z 1 , · · · , z n } with n > 1 i.i.d.
samplesdrawn from P z , we deﬁne r S ( w, θ )
= 1 n (cid:80) ni
=1 f ( w, θ ; z i ) .
Next, we deﬁne the following quantity:
Deﬁnition 1 (Primal Risk (Empirical/Population)) .
Primal population risk is given byr ( w ) = max θ ∈ Θ E z ∼ P z f ( w, θ ; z ) , and the primal empirical risk is given by: r S ( w ) =max θ
[Equation]
n
[Equation]
=1 f ( w,
[Equation]
i ) .
Throughout this paper, we use ( w S , θ S ) to denote a solution of the min-max problem:min w ∈ W max θ ∈ Θ r S ( w, θ ) .
Notice that ( w S , θ S ) need not be a saddle point of r S .
Furthermore, weuse ( w ∗ , θ ∗ ) to denote a solution of min w ∈ W max θ ∈ Θ r ( w, θ ) .
Once again, notice that ( w ∗ , θ ∗ ) maynot be a saddle point of r .
The goal in Problem (1) is to minimize the primal population risk r ( w ) .
Note that this function canbe decomposed as
[Equation]
In practice, we only have access to r S ( w, θ ) and our goal is to design algorithms for minimizing r ( w )using dataset S .
Suppose A is a learning algorithm initialized at ( w, θ ) =
We deﬁne ( w AS , θ AS )to be the output of Algorithm A using dataset S .
From Equation (2), it is clear if we ensure r S ( w AS ) as well as r ( w AS ) − r S ( w AS ) are small, this wouldguarantee that r ( w AS ) is small, which is the goal of Problem (1).
Note that we can always ensure thatr S ( w AS ) is small by using a good optimization
Algorithm A (if the problem is tractable).
The maingoal in the study of generalization is therefore to estimate the generalization error r ( w AS ) − r S ( w AS ) .
Deﬁnition 2.
The generalization error for the primal risk is deﬁned as:
[Equation]
Here the expectations are taken over the randomness in the dataset S , as well as any randomnessused in the Algorithm A .
This metric has been used to study generalization in stochastic minimization problems, i.e., whenthe maximization set Θ is a singleton, as well as several recent works in minimax optimization (see[Hardt et al., 2016, Farnia and Ozdaglar, 2021]).
We are interested in when the solution to the empirical problem w AS has good generalization behavior ,i.e., when E [ r ( w AS ) − min w ∈ W r ( w )] is small - w AS is an approximate minimizer of the primalpopulation risk r .
In the next subsection, we brieﬂy describe why the generalization error of theprimal risk ζ Pgen ( A ) is a good measure to study generalization behavior in minimization problems.
2.1.1 ζ Pgen ( A ) for minimization problems
Consider a minimization problem of the form
[Equation]
In this case, we deﬁne the (minimization) primal risk (population and empirical version respectively)as: m ( w ) =
E z ∼ P z g ( w ; z ) , and m S ( w ) = 1 n (cid:80) n i =1 g ( w ; z i ) .
The generalization error ζ P,mingen ( A )for the (minimization) primal risk is the same as in Deﬁnition 2 using the (minimization) primal risk.
Assume that the generalization error of the primal risk for an Algorithm A is small, say ζ P,mingen ( A ) ≤ (cid:15) .
This implies that (from Deﬁnition 2): E [ m ( w AS )] ≤ E [ m S ( w AS )]
+ (cid:15) .
Note that the expectation iswith respect to S and A .
Now, in order to show that w AS has good generalization behavior, we ﬁrstsee that:
[Equation]
However, note that for minimization problems, since E [ m S ]
= m , we have 1 : min w ∈ W m ( w ) ≥
E [min w ∈ W m S ( w )] , which gives us:
[Equation]
Therefore, for minimization problems, if the generalization error for primal risk is small, we havegood generalization behavior of the solution to the empirical risk minimization.
In the next subsection,we highlight some results in the literature which discusses bounds on generalization error of theprimal risk.
These results depend on the concept of algorithmic stability which we discuss next.
2.2 Stability of algorithms
Stability analysis is a powerful tool to analyse the generalization behavior of algorithms (see [Bousquetand Elisseeff, 2002]).
In this section, we will review some deﬁnitions and theoretical results aboutstability bounds existing in the current literature.
More speciﬁcally, in this paper, we adopt thefollowing deﬁnition of stability:
Deﬁnition 3.
Suppose that A is a randomized algorithm for solving the minimax problem.
We deﬁne( w AS , θ AS ) as the output of Algorithm A using dataset S .
We say S and S (cid:48) are neighboring datasetif they defer only in one sample.
An Algorithm A is deﬁned to be (cid:15)
-stable
if (cid:107) w AS − w AS (cid:48) (cid:107) ≤ (cid:15) ,(cid:107) θ
AS − θ AS (cid:48) (cid:107) ≤ (cid:15) for any neighboring datasets S and S (cid:48) .
From [Hardt et al., 2016], we have the following basic result for the generalization error of r S ( w, θ ) :
Theorem 1 ([Hardt et al., 2016]) .
Consider the (stochastic) minimization problem deﬁned in (4) .
Suppose g ( · ; z ) is L -Lipschitz continuous, i.e., (cid:107) g ( w 1 ; z ) − g ( w 2 ; z ) (cid:107) ≤ L (cid:107) w 1 − w 2 (cid:107) , ∀ w 1 , w 2 ∈ W .
Then, for an (cid:15) -stable
Algorithm A , we have | E S E A
[ r ( w AS ) − r S ( w AS )]
| ≤ L(cid:15) .
However, this theorem cannot be directly extended to analyze the generalization behavior of minimaxlearners because we have an additional maximization step before taking expectation.
In this paper, we ﬁrst show that good bounds for ζ Pgen ( A ) is not sufﬁcient to conclude the generalization behaviour in minimax problems.
We then propose a new measure, the primal gap ζ PGgen ( A ) whichis more insightful to understand generalization behavior in minimax problems.
We provide bounds forζ PGgen ( A ) in the nonconvex-concave setting (which includes the more restrictive nonconvex - stronglyconcave case) and as a byproduct, we also derive bounds for ζ Pgen ( A ) for the nonconvex-concavecase, which to the best of our knowledge, is an open problem.
Primal Gap: A New Metric to Study Generalization
The main idea which leads to the success of ζ Pgen ( A ) as a metric to study generalization for minimization learners is that E [min w ∈ W r S ( w )] ≤ min w ∈ W r ( w ) , which is no longer the case in the minimaxcase.
In fact, in this section, we show that a good bound for the generalization error of primal riskdoes not imply good generalization behavior in minimax problems.
We then propose a new metric,the primal gap, which circumvents these issues and provides insights into the generalization behaviorin minimax problems.
3.1 Is ζ Pgen ( A ) a good metric?
We provide an example where the generalization error of the primal risk is small, but the ﬁnal solutionto the empirical problem has poor generalization behavior.
In this example, the minimizer of r S ( w )is suboptimal for r ( w ) with high probability, and E S [ r ( w S ) − r ( w ∗ )] is large.
Example 1 (Analytical example) .
Let y ∼ N (0 , 1 / √ n ) be a Gaussian random variable in R .
Deﬁnethe truncated Gaussian variable z ∼ P z as follows:
z = y if | y | < λ log n/ √ n and z = λ log n/ √
nif y ≥
λ log n/ √ n .
Let f ( w, θ ; z ) = 12 w 2 − (cid:0) 12 n 2 θ 2 − zθ + 1 (cid:1) w , where w ∈ W =
[0 , 1] , θ ∈ Θ =[ − λn, λn ] with a sufﬁciently large λ > 0 , and z i ∼ P z be i.i.d truncated Gaussian variables.
Then,we have r S ( w, θ )
=
12 w 2 − (cid:16) 12 n 2 θ 2 − (cid:80) ni
=
z i n θ +
[Equation]
w , and
[Equation]
Note that this leads to the primal population risk function: r ( w ) =
12 w 2 − w .
It is not hard to see that we always have r S ( w ) ≥ r ( w ) .
Note that this means ζ Pgen ( A ) < 0 , and thuswe have good generalization of primal risk.
However, we can prove that for large enough λ ,
[Equation]
This means that w S has a constant error compared to w ∗ in terms of the population risk, despite thatthe generalization error of the primal risk is small.
This phenomenon is due to that min w ∈ W r S ( w ) −min
w ∈ W r ( w ) >
c for some c > 0 , and hence minimizing r S ( w ) is very different from minimizingr ( w ) .
This example shows that the generalization error of primal risk is not a good measure to studygeneralization in minimax learners.
The main drawback is that E S [min w r S ( w ) − min
w ∈ W r ( w )] ≤ 0no longer hold for minimax problems while it always holds for minimization problems.
We nowintroduce another more practical example, from GAN training, to further illustrate this point.
Example 2 (GAN-training example) .
Suppose that we have a real distribution P r in R d whichcan be represented as G ∗ ( y ) with y ∈ R k drawn from a standard Gaussian distribution P 0 anda mapping G ∗ :
R k
→ R d .
For an arbitrary generator G , we deﬁne P G to be the distribution ofthe random variable G ( y ) with y ∼ P 0 .
So our goal is to ﬁnd a generator G such that P G = P r .
The generative adverserial network (GAN) is a popular tool for solving this problem.
Consider a
Generative Adversarial Network (GAN) with generator G and discriminator D .
The goal of GANtraining is to ﬁnd a pair of generator G and a discriminator D that solves the min-max problem:
[Equation]
where Φ : R → R is concave, monotonically increasing and Φ( u )
=
−∞ for u ≤ 0 .
To connect to the minimax formulation in (1) , we note that z
= ( x, y ) , and P z
= P r × P 0 .
Also, we denote r ( G, D ) =
E
[Equation]
r Φ( D ( x )) + E y ∼ P
Φ(1 − D ( G ( y )))] to be the population risk.
We now give the empirical version of this problem.
Let S 1 = { x 1 , · · · , x n } and S 2 = { y 1 , · · · , y n } .
Let S
= S 1 ∪ S 2 and r S ( G, D ) =
1 n ( (cid:80) ni
=
Φ( D ( x i ) + Φ(1 − D ( G ( y i )))) .
We assume that P G w has the same support set as P r .
Moreover, we assume that (cid:107) w − w ∗ (cid:107) ≤ 0 .and
G w ( y ) is 1 -Lipschitz w.r.t.
w for any y .
Then we have (see Theorem B.1 in Arora et al.
[Equation]
which is bounded away from 0 .
Note that in this example, we also have E S [min w r S ( w ) − min w r ( w )] > 0 , implying that ζ Pgen ( A )might not be a good metric.
To address this issue, we next deﬁne a new metric to study thegeneralization of minimax learners.
3.2 Primal gap
The empirical and population versions of the primal gap are deﬁned as follows:
Deﬁnition 4 (Primal gap (empirical/population)) .
The population primal gap is deﬁned as∆( w ) =
r ( w ) − min w ∈ W r ( w ) , and the empirical primal gap is deﬁned as ∆ S ( w ) =
r S ( w ) −min
w ∈ W r S ( w ) .
Notice that these two primal gaps can always take 0 at w S = arg
min w ∈ W r S ( w ) and w ∗ =
arg min w ∈ W r ( w ) respectively even if the saddle point of problem 2 does not exist.
Next, we deﬁnethe expected generalization error of this primal gap as follows:
Deﬁnition 5.
The generalization error for the primal gap is deﬁned as
[Equation]
Notice that we have the decomposition
[Equation]
If E S E A [ r S ( w AS ) − min w ∈ W r S ( w )] is small and ζ PGgen ( A ) is small (or large), then
E S E A [ r ( w AS ) −min
w ∈ W r ( w )] is small (or large).
Next, we provide bounds on ζ PGgen ( A ) for stable algorithms A , and show that in Example 1, ζ PGgen ( A )cannot be small (unlike ζ Pgen ( A ) ).
3.3 Relationship between generalization and stability
We provide bounds for the generalization error of the primal gap (Deﬁnition 5) for (cid:15) − stable algorithms
A .
We will focus on the nonconvex-concave case where the following assumptions are madethroughout the rest of the paper.
Assumption 2.
The function f in Problem (1) is nonconvex-concave, i.e. f ( w, · ; z ) is a concavefunction for all w ∈ W and for all z .
Next we deﬁne the notion of capacity , which will play a key role in the bounds we derive for ζ PGgen ( A ) .
Deﬁnition 6 (Capacity) .
For any w ∈ W and any constraint set Θ , we deﬁne
[Equation]
We deﬁne the capacities C e and C p as:
[Equation]
For the speciﬁc constraint set in Problem 1, we denote the capacities as C e and C p respectively.
The norm of the parameter is usually viewed as the metric for the complexity of learning models.
Infact, the norm of the optimal solution determines the Rademacher complexity of the function class instatistical learning theory [Vapnik, 1999].
Moreover, in deep learning, minimum-norm solution ofoverparameterized neural networks is well-known to enjoy better generalization performance [Zhanget al., 2021a].
Hence, we view the capacity constant C e and C p as natural metrics to capture themodel complexity for the best response of the max learner, when using the empirical data set andpopulation data respectively.
Now, we are ready to discuss the relationship between the stability bound and the generalizationerror of algorithms in nonconvex-concave minimax problems.
All proofs have been deferred to theappendix.
We make the following assumptions throughout the paper:
Assumption 3.
The gradient of f is (cid:96) -Lipschitz-continuous for all z , i.e.,(cid:107)∇
[Equation]
Moreover, ﬁxing w ∈ W , the partial gradient ∇ θ f ( w, · ; z ) is (cid:96) θθ -Lipschitz continuous with respectto θ for all z , i.e., (cid:107)∇ θ f ( w, θ 1 ; z ) − ∇ θ f ( w, θ 2 ; z ) (cid:107) ≤
[Equation] [Equation]
∀
[Equation] [Equation]
Assumption 4.
For any Θ 1 ⊆ Θ , we assume that f is L (Θ 1 ) -Lipschitz-continuous with respectto w ∈ W, θ ∈ Θ 1 for all z , i.e., (cid:107) f
( w 1 , θ 1 ; z ) − f ( w 2 , θ 2 ; z ) (cid:107) ≤ L (
[Equation]
1 )( (cid:107) w 1 − w 2 (cid:107) + (cid:107) θ 1 −θ 2 (cid:107) ) , ∀ w 1 , w 2 ∈ W, ∀ θ 1 , θ 2 ∈ Θ 1 , and the gradient ∇ f ( w, θ ; z ) is uniformly bounded as(cid:107)∇ w,θ f ( w, θ ; z ) (cid:107) ≤ L (Θ 1 ) for all z and w ∈ W, θ ∈ Θ 1 .
Moreover, f ( w ∗ , · ; z ) is L ∗ θ Lipschitz continuous with respect to θ where w ∗ ∈ arg min w ∈ W r ( w ) .
We also deﬁne L :=
L ( B (0 , 2 C p +1) ∩ Θ)and L r :=
L ( B (0 , r ) ∩ Θ) , where B ( v, r ) denotes the l 2 -ball with radius r centered at v .
Note that we can decompose the generalization error of the primal gap as follows:
[Equation]
Next, we provide a bound on the generalization error for the primal risk ζ Pgen ( A ) .
To the best of ourknowledge, this is the ﬁrst bound for ζ Pgen ( A ) in the nonconvex-concave setting.
Lemma 1.
The generalization error of the primal risk of an (cid:15) − stable Algorithm A for a min-maxproblem with concave maximization problem can be bounded by ζ Pgen ( A ) ≤ (cid:113) 8 C 2 p (cid:96)L · √ (cid:15) .
We show that this dependence on (cid:15) is tight in the Appendix Section C.2.
Since we already have the generalization error for the primal risk E S E A [ r ( w AS ) − r S ( w AS )] from
Lemma 1, we only need to estimate
[Equation]
[Primal
[Equation]
The following theorem gives the generalization bound of the primal gap using the upper bound from
Lemma 1 and bounding the Primal Min Error in Equation (8).
Note that in minimization case, wealways have ζ PMgen ≤ 0 .
But this is not always true in minimax case as shown in Example 1.
So weneed to estimate this error in minimax problems.
Theorem 2.
Suppose Algorithm A is (cid:15)
-stable.
The generalization bound of the primal gap is given by
[Equation]
Note that the ﬁrst term is from the generalization of the primal risk, as shown in Lemma 1, and thesecond term is an upper bound for the Primal Min Error in Equation (8).
We show the tightness ofthe bound in Section C.4.
3.4 Revisiting Example 1
Recall Example 1 in the Section 3.1.
In this example, we have that the primal risk generalizes well,but the solution w S does not, thereby indicating that the generalization error of the primal risk maynot be a good metric to study generalization behavior of minimax problem.
Note that in Example1, we have ζ PGgen ( A ) ≥ 0 .
02 (implying bad generalization) while ζ Pgen ( A ) ≤ 0 (implying generalizewell).
We already know that this algorithm can not generalize well.
Therefore, ζ PGgen ( A ) can capturethe generalization better than ζ Pgen ( A ) .
Therefore, the primal min error ζ PMgen also plays an importantrole in analysing the generalization for minimax problems.
As shown in the appendix (Proposition 3),we have
[Equation]
On the other hand, it is easy to compute that L ∗ θ = λ log n/ √ n and C e = λn log n .
Therefore, by Theorem 2, we have an upper bound for the primal min error (see Equation (8)):
[Equation]
W r S ( w ) − min
w ∈ W r ( w )] ≤ 4 L ∗ θ C e / √ n = 4 log n, which is tight up to a log factor according to (9).
Therefore, the primal gap has a constant generalization error which is consistentwith the observation that the solution to the empirical problem does not have good generalizationbehavior.
3.5 Nonconvex-nonconcave case
In this section, we extend our results to the nonconvex-nonconcave setting.
We will show that undercertain assumptions on the inner maximization problem, we can derive generalization error boundsfor the primal risk and primal gap in terms of algorithmic stability.
Due to space limitation, we deferthe details and justiﬁcations of the assumptions (speciﬁcally Assumptions 6 and 7) to Appendix C.5.
Furthermore, we assume that f ( · , · ; z ) is L -Lipschitz 2 continuous in W × Θ .
This, along with
Assumptions 6 and 7, allows us to derive the generalization error bounds of the primal risk and primalgap in terms of algorithmic stability.
Lemma 2.
Suppose that Assumption 6 holds.
If a minimax learning Algorithm A is an (cid:15) -stablealgorithm, we have ζ Pgen ( A ) ≤ L(cid:15)
+ (cid:112)
Lλ p √ (cid:15).
Similarly, we can derive the generalization bound for the primal gap given the above assumptions.
Theorem 3.
Suppose Assumptions 6 and 7 hold.
Then we have ζ PGgen ( A ) ≤ ζ Pgen ( A ) + √
Lλ e (cid:14) √ n.
The proof of these two results can be found in Appendix C.5.
Comparison of GDA and GDMax
In Section 3.3, we provide generalization bounds for the primal gap for any (cid:15)
− stable algorithm.
Inthis section, we focus on two algorithms in particular – GDA and GDMax.
These two algorithms aredescribed in Algorithms 1 and 2 in Appendix D.
We note that though analyzing the optimization properties of GDA/stochastic GDA for solvingthe empirical minimax problem is an important topic, our focus in this paper is on studying thegeneralization behavior of these algorithms.
We assume that the empirical version of minimaxproblem we consider can be solved by GDA and GDMax, i.e., we assume that GDA and GDMaxsatisfy the following assumption:
Assumption 5.
Let A be a minimax learner, such as GDA or GDMax.
Then we assume that A hasthe following convergence rate: E A [ r S ( w t ) − min w ∈ W r S ( w )] ≤ ( φ A ( M ( W ))
+
φ A ( C e )) /ψ
A ( t ) ,where M ( W ) is the maximum of the norms of w , φ A ( s ) , ψ A ( s ) are nonnegative, increasing functionsthat tend to inﬁnity as s → ∞ .
For simplicity, throughout this section, we assume that (cid:107) f
( w, θ ; z ) (cid:107) ≤ 1 for all w, θ , and z .
The next theorem provides a bound for the population primal gap ∆( w AS ) :
= r ( w AS ) − min w ∈ W r ( w ) .
Note that the goal of any algorithm is to make this gap as small as possible.
For an Algorithm A and subsets W 0 ⊆ W, Θ 0 ⊆
Θ , we deﬁne A ( W 0 , Θ 0 ) as the algorithm whichrestricts Algorithm A to solve (1) under constraint sets W 0 and Θ 0 .
Speciﬁcally, A ( W, Θ) is just A .
Theorem 4.
Let w A,tS , θ A,tS be the t -th iterate generated by Algorithm A using dataset S .
Assume that { θ A,t S } ⊆
Θ 0 =
Θ Aθ for t ≤ T with probability 1 − δ (due to the randomness in S ) and
B (0 , C p ) ⊆
Θ Aθ .
Here B ( v, r ) denotes the l 2 -ball with radius r centered at v .
Let A
= A ( W, Θ 0 ) .
Then after T iterations of Algorithm A , the population primal gap can be bounded as:
[Equation]
where ζ Pgen ( A 0 ) =
E S E A [ r ( w A 0 ,T S )
− r S ( w A 0 ,T S )] is the generalization error of the primal risk of
Algorithm A 0 .
Remark 1.
Theorem 4 builds a closer connection between generalization behavior and the dynamicsof minimax learner A .
More restricted the the max learner is, better the the generalization behavior.
We make this clear in the comparison of GDA and GDMax by analyzing the three terms in Theorem 4.
4.1 Analyzing the term ζ Pgen ( A 0 )
First, we study the generalization bound of the primal risk, i.e., ζ Pgen in Theorem 4.
For GDA, we canestimate ζ Pgen by using Lemma 1.
Therefore, it sufﬁces to estimate the stability of GDA.
We do thisand bound ζ Pgen ( A 0 ) for GDA in the following lemma:
Lemma 3.
Let c 0 = max { α 0 , β 0 } , If we use diminishing stepsizes α t
= α 0 /t and β t = β 0 /t for
GDA for T iterations, we have the stability bound (cid:15) GDA ≤ 2 L Θ GDAθ T c 0 (cid:96) /
( n(cid:96) ) .
Therefore, for GDAwith diminishing stepsizes α 0 /t, β 0 /t run for T iterations (denoted by GDA T ), the generalizationerror of the primal risk can be bounded by: ζ Pgen ( GDA T ) ≤ ( L Θ GDAθ ) 3 / 2 (cid:113) 8 C 2 p /(cid:96)
(cid:112) T c 0 (cid:96) /n.
However for GDMax, we can not compute a uniform stability bound that vanishes as n goes toinﬁnity.
In fact, we can show from the following simple example that ζ Pgen ( GDMax ) can be aconstant independent of n , which means that for the case where r ( w, θ ) is nonconvex-concave, thegeneralization bound for primal risk of GDMax can be undesirable.
Example 3 (Constant generalization error of primal risk for GDMax) .
Consider a dataset S with n elements.
Deﬁne the objective function: f ( w, θ ; z ) =
(cid:0) wn 2 − z (cid:1) θ − θ 2 2 n , where w ∈ W = [ − n √ n, n √ n ] , θ ∈ Θ
= R and z is drawn from the uniform distribution over {− 1 / √ n, 1 / √ n } .
We have
[Equation]
and r ( w ) =
w 2 2 n 2 .
Therefore, min w ∈ W r ( w ) = 0 .
From the deﬁnition of the function f and the sets
W and Θ , we have (cid:96) =
1 /n 2 , L = O (1 / √ n ) .
Note that one step of GDMax can attain the minimizer of r S ( w ) (since it is a one dimensionalquadratic problem), i.e., w S = n (cid:80) ni
=1 z i and r S ( w S )
= 0 .
Furthermore, we have E S r ( w S ) =
E [ ( (cid:80) ni
=
z i )2] = 1 / 2 > 0 .
Therefore, ζ Pgen ( GDMax ) =
E [ r ( w S ) − r S ( w S )]
= 1 / 2 > 0 cannot bemade small.
Therefore, from Lemma 3 and Example 3, we see that the bound for the expected Population primalgap contains the term ζ Pgen which cannot be bounded for GDMax, whereas can be bounded for GDAwhich leads us to conclude that GDA generalizes better than GDMax for such problems.
However,it is possible to bound ζ Pgen ( GDMax ) in certain problems, and in this case the two other terms in
Theorem 4 become crucial.
We analyze them next.
4.2 Analyzing the other two terms of Theorem
As shown in Example 1, sometimes the GDMax can have a good generalization bound of theprimal risk.
Therefore, we need to analyze the other two terms in Theorem 4, i.e., ( φ A ( M w ) +φ A ( C e (Θ Aθ ))) /ψ
A ( T ) and L ∗ θ C e (Θ Aθ ) / √ n .
For these two terms, since L ∗ θ is ﬁxed, the constant
C e (Θ Aθ ) is the key term which differentiates the performance of different algorithms.
By deﬁnition, the constant C e (Θ GDMaxθ ) for GDMax is nearly C e (See Deﬁnition 6).
Therefore, thepopulation primal gap after T steps of GDMax is dominated by C e if C e is large.
However, the setΘ GDAθ for GDA can be much smaller than Θ , which implies that C e (Θ GDAθ ) can be much smallerthan C e .
This phenomenon can be seen from Example 1: If we perform one step of GDMax withprimal stepsize 1 , we can attain w 1 = w S .
Then E S [ r ( w 1 S ) − min w ∈ W r ( w )] ≥ 0 .
005 from (7).
For
GDA, we can see that w 1 = 1 after one step of GDA with stepsize 1 .
Therefore, GDA generalizesbetter than GDMax.
Generally, we have the following estimate of C e (Θ GDAθ ) .
Lemma 4.
Let L 0 = max z (cid:107)∇ f ( w 0 , θ 0 ; z ) (cid:107) .
Let c
= max { α 0 , β 0 } , If we use diminishing stepsizesα t =
α 0 /t and β t = β 0 /t for GDA, then after T steps we have (cid:107) θ t (cid:107) ≤ T c 0 (cid:96) L 0 /(cid:96) for t ∈ [ T ] .
Therefore, if C e is much larger than C p , using GDA with C p ≤ T c 0 (cid:96) L 0 /(cid:96)
≤ C e is better than
GDMax.
We make this more concrete in the context of GAN training next.
4.3 GAN training
We now study the speciﬁc case of GAN training to explore why GDA might generalize better than GDMax.
This is numerically veriﬁed in the literature, such as Farnia and Ozdaglar [2021].
Speciﬁcally, we revisit Example 2, and consider a special case: D is restricted to be a overparametrized linear function with respect to θ .
Deﬁne the descriminator D ( x ) =
Φ T ( x ) v + b 0 , where Φ( x ) =
[Φ 1 ( x ) , · · · , Φ m ( x )] T ∈ R m is the feature matrix and b 0 ∈ R .
Also suppose that G is parametrized by w and G ∗ =
G w ∗ .
Then the GAN problem can be written as min w ∈ W max θ ∈ Θ r ( w, θ ) , where
[Equation]
Here θ = ( v, b 0 ) .
Assume that (cid:113) σ max (cid:0) E x ∼ P Gw Φ( x )Φ T ( x ) (cid:1) ≤ ¯ σ max / √ m , where σ max ( · ) denotes the largest singular value of a matrix and ¯ σ max > 0 is a constant.
Also assume that
E x ∼ P Gw Φ( x )Φ T ( x ) is full rank.
Also, we assume that | φ (cid:48) ( λ ) | ≤ L φ for any λ ∈ [0 , 1] .
Therefore,we have E [
(cid:107)∇ θ f ( w, θ ; z ) (cid:107) 2 ]
≈ L 2 φ ¯ σ 2max .
Then it is reasonable to assume that (cid:107)∇ f (cid:107) ≤ O (1) .
Lemma 5.
Suppose Φ( x ) is sub-Gaussian and the matrix
[Equation]
] is full column rank ( m > n ) with probability 1 .
Then with probability at least 1 − Cδ with someconstant C , we have (cid:107) θ S ( w ∗ ) (cid:107) ≥ Ω( √ n ) ,
where θ S ( w ∗ ) ∈ arg max θ ∈ Θ r S ( w ∗ , θ ) .
Now, for θ ∈ arg max θ (cid:48) ∈
Θ r ( w ∗ , θ (cid:48) ) , it can be easily seen that v = 0 , b 0 = 1 / 2 in this case.
Therefore, C p ≈ 1 / 2 .
Finally, combining the previous discussion on GDA in Lemma 4, and usingthe fact that C e is large from Lemma 5, we see from Theorem 4 that GDA can generalize better than
GDMax.
More detailed discuss of the GAN-training example and Lemma 5 can be found in Section
D.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes]( b ) Did you describe the limitations of your work? [Yes]
(c)
Did you
discuss any potential negative societal impacts of your work
[N/A]
d) Have you
read
the ethics review guidelines and ensured
that your
paper
conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]
[Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[N/A]
[Table]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
Supplementary Materials for“What
is a Good Metric to Study Generalization of Minimax Learners?
A Related Work
Algorithms for minimax optimization.
There is a vast literature on algorithms for minimaxoptimization.
The most popular algorithms include the Extragradient (EG), the Optimistic Gradient Descent Ascent (OGDA) and the Gradient Descent Ascent(GDA) and their variants.
The
EG algorithms introduced in Korpelevich [1976], has been analyzed in several papers including
Monteiro and Svaiter [2010], Mokhtari et al.
[2020a,b], Golowich et al.
[2020b] for (strongly)convex-(strongly)concave problems.
Another popular algorithm is the optimistic gradient descent ascent(OGDA) introduced in Popov [1980] and has been analyzed in several recent works including
Daskalakis et al.
[2017], Hsieh et al.
[2019], Golowich et al.
[2020a].
Once again, all these worksfocus on the (strongly)convex-(strongly)concave setting.
Stochastic versions of these algorithms insimilar settings have also been analyzed in several papers including Nemirovski et al.
[2009], Hsiehet al.
[2019], Fallah et al.
A few papers including Lin et al.
[2020], Zhang et al.
Huang et al.
[2022], Zhang et al.
[2021c], Ostrovskii et al.
[2021b], Kong et al.
[2019], Zhang et al.[2020] analyze gradient based algorithms in the nonconvex-(strongly)concave cases.
Some papersincluding Raﬁque et al.
[2018], Yang et al.
[2021], Ostrovskii et al.
[2021a], Grimmer et al.
[2020]analyze special cases of nonconvex-nonconcave (like nonconvex-PL) for algorithms like GDA andits variants.
However, in this paper, we are interested in the generalization performance of thesealgorithms.
We summarize below the most related literature that studies the generalization behaviorin minimax optimization problems.
Algorithm-independent generalization.
Speciﬁc to the machine learning problems of GAN andadversarial training, there have been several papers studying the uniform convergence generalizationbounds.
[Arora et al., 2017] establish a uniform convergence generalization bound which dependson the number of discriminator parameters.
[Wu et al., 2019] connect the stability-based theory todifferential privacy ([Shalev-Shwartz et al., 2010]) in GANs and numerically study the generalizationbehavior in GANs.
[Zhang et al., 2017, Bai et al., 2018] analyze the Rademacher complexity ofthe players to show the uniform convergence bounds for GANs.
In the simpler Gaussian setting,[Feizi et al., 2020] and [Schmidt et al., 2018] derive bounds for GANs and adversarial training,respectively.
The uniform convergence bounds for adversarial training have also been studied underseveral statistical learning frameworks, e.g., PAC-Bayes [Farnia et al., 2018], Rademacher complexity[Yin
et al., 2019], margin-based [Wei and Ma, 2019], and VC analysis [Attias et al., 2019].
Recently,[Zhang et al., 2021b] investigate the generalization of empirical saddle point (ESP) solution instrongly-convex-concave problems using a stability-based approach.
Note that these results are notspeciﬁc to the optimization algorithms being used.
Algorithm-dependent generalization.
Algorithm speciﬁc generalization bounds for minimaxoptimization have attracted increasing attention.
Based on the algorithmic stability framework in[Bousquet and Elisseeff, 2002], [Farnia and Ozdaglar, 2021] has established generalization bounds ofstandard gradient descent-ascent and proximal point algorithms under the strongly-convex-stronglyconcave setting, and those of stochastic GDA and GDMax under the nonconvex-strongly concavesetting.
Concurrently, [Lei et al., 2021] derives high-probability generalization bounds for bothstrongly-convex-strongly-concave and convex-strongly-concave settings, with possibly nonsmoothobjectives, also through the lens of algorithmic stability.
Recently, Li and Liu [2021] improvesthe high probability generalization bounds in terms of different stability measures and gives acomprehensive discussion on the generalization bound for different algorithms.
These works hingedon the metrics of primal risk and primal-dual risk .
As shown in the present work, the former is notnecessarily suitable to characterize the generalization behavior of minimax optimization, while thelatter might not make sense when the saddle point does not exists, i.e., the max and min cannot beinterchanged, a common scenario in machine learning (see Jin et al.
Following this line ofwork, [Xing et al., 2021] provide generalization bounds speciﬁcally for adversarial training, whichis essentially the primal risk, also using the algorithmic stability framework.
Recently, [Yang et al. ,2022] study the generalization of stochastic GDA under differential privacy constraints.
[Table] [Table]
and
[Table] [Table]
[Table] [Table]
[Table] [Table]
A.1 Existing related results
From [Farnia and Ozdaglar, 2021], we have the following theorem showing the connection betweenstability and generalization for minimax problems.
Theorem 5 ([Farnia and Ozdaglar, 2021]) .
Consider an Algorithm A which is (cid:15) -stable.
We have thefollowing two claims:
If the maximization and the expectation can be swapped when computing r ( w ) , then
[Equation]
If f ( · , · ; z ) is nonconvex-strongly-concave and f is µ -strongly-concave with respect to θ , then
[Equation]
Remark 2.
In [Lei et al., 2021], the authors proved a generalization bound in a weak sense, i.e., they consider the weak duality gap:
[Equation]
However, notice that the expectation is inside the min and max operators.
It does not deal with thecoupling of the maximization and expectation.
Remark 3.
According to Theorem 5, the generalization bound for ζ Pgen scales with the conditionnumber κ θ , and therefore cannot give useful bounds in the absence of strong concavity (whenκ θ → ∞ ).
Remark 4.
The generalization bounds for ζ Pgen of algorithms for problems in terms of stabilitywithout strong concavity is still open to the best of our knowledge.
As mentioned in [Lei et al.
,2021], ﬁnding generalization bounds without the strong concavity assumption is an interesting openproblem.
B Analysis of Example
In this section, we analyze the toy example given in Example 1.
Proposition 1.
For the risk function and data distribution given in Example 1, we have
[Equation]
for any w ∈ W .
Proof.
For a ﬁxed w , r ( w ) =
w 2 / 2 − w .
On the other hand,
[Equation]
Therefore, we have the desired result.
Next, we prove that | (cid:80) ni
=
z i | will stay in the interval [0 . 5 , λ ] with high probability.
Lemma 6.
For large enough λ > 2 , we have
[Equation]
Proof.
Let y i ∼ N (0 , 1 / √ n ) , i = 1 , · · · , n be n i.i.d.
variables.
Then (cid:80) ni
=
y i ∼ N (0 , 1) .
According to the table of Normal distribution, we have Pr( | (cid:80) ni
=1 y i | ∈ [0 .
5 , λ ]) ≥ 0 .
By the deﬁnition of z i , we have
Pr
[Equation]
For the ﬁrst term, we have
[Equation]
[Equation]
Taking λ sufﬁciently large yields the desired result, where the ﬁrst inequality is because of theunion bound and the second inequality is due to the tail bound of Normal distribution.
Therefore,
Pr( | (cid:80) ni
=
z i | ∈ [0 .
5 , λ ]) > 0 .
4 for sufﬁciently large n .
The second statement follows similarly,noting from the table of Normal distribution that Pr( | (cid:80) ni
=1 y i | ∈ [0 .
5 , λ ]) ≥ 0 .
Proposition 2.
For sufﬁciently large λ > 0 , we have
[Equation]
Proof.
If | (cid:80) ni
=
z i | ∈ [0 .
5 , λ ] , we have
[Equation]
In this case, we have
[Equation]
by direct calculation.
Therefore, we have
[Equation]
[Equation]
21) where the ﬁrst inequality is because of (18) and the fact that r ( w S ) − min w ∈ W r ( w ) ≥ 0 for any
S .
Proposition 3.
For sufﬁciently large λ > 0 , we have:
[Equation]
for Example 1.
Proof.
If | (cid:80) ni
=
z i | ≥ λ > 2 , we have w S = 0 and hence r S ( w S )
= 0 .
If | (cid:80) ni
=
z i | ≤ λ , we have
[Equation]
Therefore, min w ∈ W r S ( w ) ≥ min w ∈ W r ( w ) for any S .
By Lemma 6, we can prove that
Pr( | (cid:80) ni
=
z i | ∈ [2 , λ ]) ≥ 0 .
01 for sufﬁciently large λ .
Notice that for | (cid:80) ni
=
z i | ∈ [2 , λ ] , r S ( w S )
− min w ∈ W r ( w ) = 1 / 2 .
Therefore, we have
[Equation]
This completes the proof.
C Deferred Results and Proofs in Section 3C.1 Proof of Lemma 1
In this subsection, we assume that A is an (cid:15)
-stable algorithm.
For any w ∈ W , let Θ S ( w ) =
arg max θ ∈ Θ r S ( w, θ ) and Θ( w ) =
arg max θ ∈ Θ r ( w, θ ) be the solution sets of the problems.
Letθ ( w ) be any element in Θ( w ) .
Then
[Equation]
where the inequality is because r S ( w AS , θ S ( w AS )) ≥ r S ( w AS , θ ) for any θ .
Let f be µ -strongly concavewith respect to θ .
We denote the condition number by κ θ = (cid:96) θθ /µ .
In the strongly concave case, Θ( w ) has a unique element θ ( w ) , which is κ θ -Lipschitz continuouswith respect to w (see [Lin et al., 2020]).
Then, deﬁning ˜ f
( w, z ) =
f ( w, θ ( w ); z ) , the minimax problem reduces to the usual minimizationproblem on the function ˜ f .
The stability and the Lipschitz continuity of θ ( w ) with respect to wyield the generalization bound of L √ κ 2 + 1 (cid:15) .
This is the result shown in Theorem 1 of [Farnia and
Ozdaglar, 2021].
However, if the maximization problem is not strongly concave, we lose the Lipschitz continuity andthe uniqueness.
To overcome this difﬁculty, we deﬁne an approximate maximizer ¯ θ ( w ) to r ( w, θ ) .
Concretely speaking, we deﬁne ¯ θ ( w ) to be the point after s steps of gradient ascent for the functionr ( w, · ) with a stepsize 1 /(cid:96) θθ and being initialized at 0 .
Then we have the following lemma:
Lemma 7.
For any w ∈ W , we have
(cid:107) ¯ θ ( w ) − ¯ θ ( w (cid:48) ) (cid:107) ≤ s (cid:96)(cid:96) θθ (cid:107) w − w (cid:48) (cid:107) .
r ( w ) − r ( w, ¯ θ ( w )) ≤ (cid:96) θθ C 2 p /s .
Proof.
To prove the ﬁrst part, let θ
= θ (cid:48)
= 0 .
Deﬁne θ t , θ (cid:48) t recursively as follows:
[Equation]
[Equation]
and
[Equation]
We prove (cid:107) θ t − θ (cid:48) t (cid:107) ≤ t (cid:96)(cid:96) θθ (cid:107) w − w (cid:48) (cid:107) by induction.
For t = 0 , θ 0 − θ (cid:48)
= 0 .
Assume the inductionhypothesis (cid:107) θ t − 1 − θ (cid:48)
t − 1 (cid:107) ≤ ( t − 1) (cid:96)(cid:96) θθ (cid:107) w − w (cid:48) (cid:107) holds.
We have
[Equation]
[Equation]
+
[Equation]
∇
[Equation] [Equation] [Equation]
≤
[Equation]
[Equation]
≤
[Equation] [Equation]
where the ﬁrst inequality follows from the triangle inequality, the second inequality follows fromnon-expansiveness of gradient ascent for concave functions and the (cid:96) -Lipschitz continuity of ∇ r , andthe third inequality follows from the induction hypothesis.
Therefore, letting t = s completes the proof of the ﬁrst part.
The second part of this lemma is just theconvergence result for gradient ascent on smooth concave functions (see e.g., [Nesterov, 2013]).
Consider a virtual algorithm ¯
A : for any S , the algorithm returns w = w AS
and θ = ¯ θ ( w AS ) .
Lemma 8.
The stability of this virtual algorithm is (cid:15)
[Equation]
Proof.
It is direct from the ﬁrst part of Lemma 7.
Then we have the generalization bound of r S ( w, θ ) :
Lemma 9.
We have
[Equation]
Proof.
For any z , by Assumption 4, we have
[Equation]
The result follows directly from the standard stability theory in [Hardt et al., 2016].
Now we are ready to derive the generalization bound of the Primal Risk for an Algorithm A with(cid:15) -stability.
First, we have
[Equation]
where the ﬁrst inequality is because r S ( w AS ) = max θ r S ( w AS , θ ) , the second inequality is because ofthe second part of Lemma 7 and the last inequality is because of Lemma 9.
Optimizing over s , thegeneralization error is bounded by ζ Pgen ( A ) ≤ (cid:113) 8 L(cid:96)C 2 p · √ (cid:15) .
This completes the proof.
C.2 Tightness of the bound for Primal Risk
Consider the following risk function: f ( w, θ ; z )
=
(cid:112) n/(cid:15) (( w/ ( n √ n(cid:15) )
− z ) θ − θ 2 / (2 n √ n(cid:15) )) , where w ∈ W = [ − λ(cid:15) √ n log n, λ(cid:15) √ n log n ] and θ ∈ Θ = R .
The sample z is drawn from the uniform distribution over {− 1 / √ n, 1 / √ n } .
Then we have r ( w ) =
(cid:112) n/(cid:15) ( w 2 / (2 (cid:15)n √ n )) and r S ( w ) =
(cid:112) n/(cid:15) (( (cid:15)n √ n )( w/ ( (cid:15)n √ n ) − (cid:80) ni
=
z i /n )
Now we have (cid:96) =
√ n/(cid:15) ( (cid:15)n √ n ) , C p = λ(cid:15) √ n log n and
L = √ n/(cid:15) √ n .
If we perform one-step of GDMax with stepsize 1 /(cid:96) r S where (cid:96) r S
=
(cid:112) n/(cid:15)/ ( (cid:15)n √ n ) , then we attain w S = arg min w ∈ W r S ( w ) .
The stability bound of the GDMax is (cid:15) .
Therefore, the generalization error of the primal risk is estimated as:
[Equation]
On the other hand, (cid:80) ni =
z i /n ∈ [ − λ log n/n, λ log n/n ] holds with probability at least 1 − C/n λ by
Hoefding inequality.
Let ¯ w S
= (cid:15) √ n ( (cid:80) z i ∈ S
z i ) .
Then with probability at least 1 − C/n λ , w S = ¯ w S
Notice that E S [ r ( ¯ w S ) − r S ( ¯ w S )]
=
E S [ √ n · ( √ (cid:15)n √ n ) · ( (cid:80) z
i ∈ S
z i /n )
2 ] = √ (cid:15) .
It is not hard to show that | r ( ¯ w S ) | ≤ n √ (cid:15) , r S ( ¯ w S )
= 0 , | r ( w S ) | ≤ 2 n √ (cid:15) and | r S ( w S ) | ≤ 2 n √ (cid:15) .
Then we have
E S [ r ( ¯ w S ) − r S ( ¯ w S )] − E S
[ r ( w S ) − r S ( w S )]
≤ 5 Cn √ (cid:15)/n λ .
Therefore, E [ r ( w S ) − r S ( w S )] ≥ √ (cid:15)/ 2 for sufﬁciently large λ and n .
Then in this example we have √ (cid:15)/ 2 ≤ ζ Pgen ( A ) ≤ 8 λ log n √ (cid:15).
For (cid:15) ≤ 1 /n τ +1 , we have log [Equation] log( [Equation]
/(cid:15) ) . Therefore, the estimate ζ Pgen ≤ λ √ (cid:15) log(1 /(cid:15) ) /
( τ + 1) is tight up to a log(1 /(cid:15) ) factor.
C.3 Proof of Theorem
Recall that the empirical primal gap is deﬁned as
[Equation]
and the population primal gap is given by
[Equation]
Suppose we are given an (cid:15)
-stable Algorithm A .
We then want to derive the generalization error
[Equation]
[Equation] AS )] . Since we already have the generalization error for the primal risk E S E A [ r ( w AS ) − r S ( w AS )] in
Theorem 1, we only need to estimate
[Equation]
to get a generalization bound on the primal gap.
Lemma 10.
Let w ∗ ∈ arg
min w ∈ W r ( w ) .
Suppose that f ( w ∗ , · ; z ) is L ∗ θ Lipschitz continuous with respect to θ .
Then we have
[Equation]
Proof.
We use similar techniques as in the proof of Lemma 1.
Step 1.
We deﬁne an approximate maximizer ˜ θ S of the function r S ( w ∗ , · ) .
˜ θ
S is attained byperforming s steps of gradient ascent to r S ( w ∗ , · ) with stepsize 1 /(cid:96) θθ and being initialized at 0 .
Similar to Lemma 7, we have the following lemma:
Lemma 11.
We have the following properties:
(cid:107) ˜
θ S − ˜
θ S (cid:48) (cid:107) ≤ 2 sL ∗ θ / ( n(cid:96) θθ ) .2.
r S ( w ∗ ) − r S ( w ∗ , ˜ θ S ) ≤ (cid:96) θθ C 2 e /s .
Proof.
The proof is similar to the proof of Lemma 7.
To prove the ﬁrst part, let ˜ θ
= ˜
θ (cid:48)
= 0 .
Deﬁne ˜ θ t , ˜ θ (cid:48) t recursively as follows:
[Equation]
and
[Equation]
We prove (cid:107) ˜ θ t − ˜ θ
[Equation]
t (cid:107) ≤
L ∗ θ / ( n(cid:96) θθ ) by induction.
For t = 0 , ˜ θ 0 − ˜ θ (cid:48)
= 0 .
Assume the inductionhypothesis (cid:107) ˜ θ t [Equation] 1 − ˜ θ
[Equation] 1 (cid:107) ≤ ( t − 1)
L [Equation] θ / ( n(cid:96) θθ ) holds.
We have
[Equation]
where the ﬁrst inequality follows from the triangle inequality, the second inequality follows from nonexpansiveness of gradient ascent for concave functions and the L ∗ θ -Lipschitz continuity of f ( w ∗ , · ; z ) ,and the third inequality follows from the induction hypothesis.
Therefore, letting t = s completes the proof of the ﬁrst part.
The second part of this lemma is just theconvergence result for gradient ascent on smooth concave functions (see e.g., [Nesterov, 2013]).
We then deﬁne the virtual algorithm ˜
A given by w ˜ AS = w ∗ and θ ˜ AS = ˜ θ S .
Since the output argumentw of ˜ A is always w ∗ , the stability of ˜
A only depends on ˜ θ S .
Then the stability bound of this virtualalgorithm is given in the following lemma:
Lemma 12.
The stability of Algorithm ˜
A is given by (cid:15) sta
( ˜ A )
= 2 sL ∗ θ / ( n(cid:96) θθ ) .
Then by the standard stability theory in [Hardt et al., 2016], we have
[Equation]
Step 2.
We have
[Equation]
where (i) follows from the deﬁnition of w ∗ , θ ∗ , (ii) follows since w S minimizes r S ( w ) , (iii) followsfrom Lemma 11, and (iv) follows from the optimality of θ ∗ given w ∗ .
Then by (22), we have
[Equation]
which completes the proof.
The ﬁnal statement of the theorem follows from Lemma 10 and Theorem 1.
C.4 Tightness of Bound for Primal Min Error
We can construct an example with C e , L ∗ θ independent of n and the upper bound for ζ PMgen is tight upa log factor.
Consider the following example:
• Let f ( w, θ ; z ) = 1 M ( w 2 / 2 + w (log 2 nθ 2 / (2 K 2 ) + n log nzθ
K + 1)) , where w ∈ W =[ − 1 , 1] , θ ∈ Θ =
[ − λK, λK ] for some arbitrary constants K > 0 and M > 0 .
z isdrawn from a truncated Gaussian distribution.
Concretely speaking, let y ∼ N (0 , 1 / √ n ) .
Then z = y if | y | ≤ λ log n/ √ n and z = λ log n/ √ n if
y ≥
λ log n/ √ n .
No-
tice that Mf ( w, θ ; z ) =
h ( w, θ (cid:48) ; z ) , where θ (cid:48)
= n log nθ/K , h ( w, θ (cid:48) ; z ) =
w 2 / 2 +w (( θ (cid:48) )
2 / (2 n 2 ) + zθ (cid:48) + 1) , θ (cid:48)
= n log nθ/K ∈ [ − λn log n, λn log n ] .
Notice that h is just
the risk function in Example 1 in the paper.
Therefore, we can estimate the lower bound of thePrimal Min Error corresponding to f using the result in Example.
The lower bound of Primal
Min Error of the problem in Example 1 (corresponding to h ) is 0 .
Then the Primal-Min Error (corresponding to f ) ζ PMgen ( A ) =
E S min w ∈ W r S ( w ) − min w ∈ W r ( w ) ≥ 0 .
005 /M .
On the other hand, it is not hard to have L ∗ θ = λ √ n log 2 n/MK , C e = λK .
Therefore,
L ∗ θ C e = λ 2 √ n log 2 n/M .
Let
M = √ n log 2 n .
Then L ∗ θ = λ/K .
If K does not depend on n , L ∗ θ , C e do not depend on n .
Therefore, the Primal-Min Error
ζ PMgen ( A ) satisﬁes 0 .
005 λ 2 ( L ∗ θ C e ) / (log 2 n √ n ) = 0 .
( L ∗ θ C e ) /
( ML ∗ θ C e ) ≤ ζ PMgen ( A ) ≤
( L ∗ θ C e ) / √ n , which is tight up to a log 2 n .
• If we let M = 1 and K = 1 , then ζ PMgen ( A ) ≤ λ 2 log 2 n as discussed in the paper.
Thisupper bound means that we can not attain an arbitrary accuracy δ > 0 .
The lower bound,
i
e., ζ PMgen ( A ) ≥ 0 .
005 λ 4 , also implies that we can not attain an arbitrary accuracy.
• If we let M = 1 / ( λ 2 log 2 n ) and K = 1 , L ∗ θ C e / √ n = 1 .
This upper bound implies
that we can not let ζ PMgen smaller than arbitrariy required accuracy.
Return to the lower
bound, i.e., ζ PMgen ( A ) ≥ 0 .
005 / ( λ 2 log 2 n ) .
If we want to attain an accuracy δ , the required
sample complexity is 2 λ/ √ δ , which is larger than a polynomial size and hence is still viewed
as intractable.
In this sense, the upper bound and the lower bound do not make a majordifference.
• Combining the two points above, we can conclude that in terms of sample complexity, ourbound is tight (up to logarithmic factors).
C.5 Results in §3.5 for the Nonconvex-Nonconcave Case
For the nonconvex-nonconcave setting, we assume that the maximization problem can be solved bygradient ascent for any w .
Note that this includes the case where the maximization problem has thegradient-domination structure (see for example Raﬁque et al.
We ﬁrst make the following assumptions on the inner maximization problem:
Assumption 6.
For any γ > 0 , there exists an algorithm which outputs θ γP ( w ) , for the innermaximization problem max θ ∈ Θ r ( w, θ ) , satisfying the following conditions:
r ( w ) − r ( w, θ γP ( w )) ≤ γ .2.
(cid:107) θ γP ( w ) − θ γP ( w (cid:48) )
(cid:107) ≤ λ p γ (cid:107)
w − w (cid:48) (cid:107) with some constant λ p > 0 for all w, w (cid:48) ∈ W .
Assumption 7.
For any γ > 0 , there exists an algorithm which outputs θ γE ( S ) , for the innermaximization problem max θ ∈ Θ r S ( w ∗ , θ ) , satisfying the following conditions:
r S ( w ∗ ) − r S ( w ∗ , θ γE ( S )) ≤ γ .
For any neighboring dataset S, S (cid:48) , we have (cid:107) θ γE ( S ) − θ γE ( S (cid:48) )
(cid:107) ≤ λ e nγ with some constantλ e > 0 .
The following lemma gives sufﬁcient conditions for these two assumptions to hold.
Lemma 13.
Consider constants D e ≥ γ and D p ≥ γ .
Suppose that gradient ascent with diminishing stepsizes c 0 /t for the problem max θ ∈ Θ r ( w, θ ) has convergence rate r ( w ) − r ( w, θ s ) ≤ D p /s .
Then we deﬁne θ γp ( w ) by performing s = D p /γ steps of gradient ascent.
Then, θ γp ( w ) satisﬁes Assumption Suppose that gradient ascent with
constant stepsize c 0 for the problem max θ
∈
Θ r (
w
θ )
has
convergence
rate r (
w
−
r (
w, θ
s )
≤ D
p η s
for
some constant
0 <
η
< 1 .
Then we
deﬁne
θ γp ( w ) by s =
log
( D p
/γ / log 1 /η steps of gradient ascent.
Then, θ γp ( w ) satisﬁes
Assumption 6.
Suppose gradient ascent with diminishing stepsizes c 0 /t for the problem max θ ∈ Θ r S ( w, θ ) has convergence rate r S ( w ) − r S ( w, θ s ) ≤ D p /s .
Then we deﬁne θ γe ( S ) by performing
s = D e /γ steps of gradient ascent.
Then, θ γe ( S ) satisﬁes Assumption Suppose
gradient ascent with constant stepsize c 0 for the problem max θ ∈ Θ r S ( w, θ ) has convergence rate r S ( w −
r S ( w θ s )
≤ D
e η
s for some constant 0 < η < 1 .
Then
wedeﬁne θ γe ( w ) by s = log( D e /γ ) / log(1 /η ) steps of gradient ascent
Then , θ γe ( S ) satisﬁes
Assumption 7.
Proof.
We only prove the ﬁrst part of this lemma and the others can be proved similarly.
Let s =
[ D p /γ ]
+ 1 , where [ r ] denotes the largest integer no more than r .
To prove the ﬁrst part, let θ
= θ (cid:48)
= 0 .
Deﬁne θ t , θ (cid:48) t recursively as follows:
[Equation]
and
[Equation]
We prove (cid:107) θ t − θ
[Equation]
t (cid:107) ≤ t (cid:96)(cid:96) θθ
(cid:107) w − w (cid:48)
(cid:107) by induction.
For t = 0 , θ 0 − θ (cid:48)
= 0 .
Assume the inductionhypothesis (cid:107) θ t − 1 − θ (cid:48)
t − 1 (cid:107) ≤ ( t − 1) (cid:96)(cid:96) θθ (cid:107) w − w (cid:48) (cid:107) .
We have
[Equation] [Equation]
[Equation] [Equation]
Here the ﬁrst inequality follows from the triangle inequality
[Equation]
the second inequality follows from the (cid:96) θθ − Lipschitz continuity of ∇ θ r and (cid:96) -Lipschitz continuity of ∇ r .
Therefore, we have
[Equation]
Let δ t = (cid:107) θ t − θ (cid:48) t (cid:107) .
Then by the above recursion
[Equation]
we have
[Equation]
Using the inequalities e a ≥ 1 + a and (cid:80) ti =1 1 /i ≤ log t
[Equation]
we have
[Equation]
Letting t = s yields
[Equation]
Since D p > γ , we have
[Equation]
[Equation]
[Equation]
[Equation]
D p /γ.
Hence,
[Equation]
Setting λ p = 2 D p (cid:96)/(cid:96) θθ yields the desired result.
Input do Input do 2:
[Equation]
w θ
=
[Equation]
Θ
[Equation]
w
S
Remark 5.
Note that for some practical nonconvex optimization problems in machine learning,gradient descent indeed converges to the global minima at a reasonably fast rate, e.g., in trainingdeep overparametrized neural networks [Du et al., 2019], robust least squares problems [El Ghaouiand Lebret, 1997], phase retrieval and matrix completion [Ma et al., 2019].
Our Assumptions 6 and7 can be viewed as an abstract summary of some benign properties of gradient descent for certainnonconvex optimization problems.
Furthermore, we assume that f ( · , · ; z ) is L -Lipschitz 3 continuous in W × Θ .
This, along with
Assumptions 6 and 7, allows us to derive the generalization error bounds of the primal risk and primalgap in terms of algorithmic stability.
Lemma 14.
Suppose that Assumption 6 holds.
If a minimax learning Algorithm A is an (cid:15) -stable
algorithm, we have
[Equation]
Proof.
This is similar to the proof of Lemma 1.
We ﬁrst deﬁne the virtual algorithm ¯
A which outputs ( w AS , θ γp ( w AS )) .
By Assumption 6, it can be easily seen that ¯ A is (1+ λ p /γ )
(cid:15) -stable.
Then by Theorem 1, we have
[Equation]
This gives us:
[Equation]
Taking γ = (cid:112)
Lλ p √ (cid:15) , we have
[Equation]
This completes the proof.
Similarly, we can derive the generalization bound for the primal gap given the above assumptions.
Theorem 6.
Suppose Assumptions 6 and 7 hold.
Then we have
[Equation]
The proof of this theorem is similar to the proof of Lemma 14 and Theorem 2 and hence omitted.
D Proof in Section 4D.1 Proof of Theorem 4
First, we have
[Equation]
Furthermore, by Assumption 5 and Theorem 2, we have
[Equation]
Next, notice that the output of A 0 is equal to the output of A with probability at least 1 − δ and (cid:107) r ( w ) (cid:107) ≤ 1 .
Therefore, we have
[Equation]
which gives the desired result.
D.2 Proof of Lemma 3
Deﬁne δ t = (cid:107) ( w tS , θ tS ) − ( w tS (cid:48) , θ tS (cid:48) )
(cid:107) .
We haveδ t +1 ≤ (1 + c 0 (cid:96)/t ) δ t + 2 c 0 L Θ GDAθ /nt.
Therefore,
[Equation]
32) which completes the proof.
D.3 Proof of Lemma
For a ﬁxed dataset S , let g t = ∇ r S ( w t , θ t ) and d t = (cid:107) ( w 0 , θ 0 ) − ( w t , θ t ) (cid:107) .
Then we have g t ≤ L
+ d t (cid:96) and d t +1 ≤ d t + c 0 g t /t .
Substituting the ﬁrst inequality into the second one, we have
[Equation]
which gives us
[Equation]
Multiplying this inequality from 0 to T − 1 yields
[Equation]
which completes the proof.
D.4 Proof of Lemma
Let u =
[1 , 1 , · · · , 1 , 0 , · · · , 0] T ∈ R 2 n .
Then θ S ( w ) satisﬁes Q TS θ S ( w ) =
u − b 0 e , where e =[1 , 1 , · · · , 1] T ∈ R 2 n .
It can be easily seen that (cid:107) u − b
e (cid:107) ≥ √ n/ 2 .
We can also show that σ max ( Q S ) ≤ 2 σ max · σ max ( P ) , where P ∈ R 2 n × m is full row-rank andindependent rows.
Moreover, every row of P has covariance matrix I m / √ m .
Then by random matrixtheory (see Vershynin [2010]), we have σ max ( P ) ≤ O ( √ m/ √ m − C √ n/ √ m + log(1 /δ ) / √ m ) =
O (1) with probability 1 − Cδ .
Therefore, we have θ S ( w ) ≥ Ω( √ n ) .
[Figure]
D.5 Experiments on GAN-training
In this section, we provide some numerical results to corroborate our theoretical ﬁndings.
D.5.1 Setup
We train a GAN on MNIST data using two algorithms – GDA and GDMax.
Since the stability isimproved by using adaptive methods like Adam, we use Adam-descent-ascent (ADA) and Adamdescent-max (ADMax) instead.
ADA simultaneously trains the generator and the discriminator, while
ADMax trains the optimal discriminator for each generator step.
We simulate this by taking 10 stepsof ascent for every descent step.
Figure 1 plots the images generated by GANs trained using thesetwo algorithms.
Finally, in Figure 2, we plot the norms of the discriminator trained by these twoalgorithms.
D.5.2 Results
Figure 1 plots the images generated by GANs trained using GDA and GDMax (using Adam insteadof the simple gradient step).
As predicted by the theory in Section D, we can see that GDA producesbetter images than the corresponding GAN trained using GDMax.
Furthermore, the claim that
C e >> C p can be seen from Figure 2 where we see that the norm of the discriminator trained using
GDMax is much larger than the norm of the discriminator trained using GDA.
This follows from theresults in Section 4.2.
GDMax trains the discriminator to exactly distinguish between the empiricaldata generated by the true and fake distributions.
Therefore, when they are nearly the same, theirempirical distributions would be close as well.
This would imply that the discriminator would need tohave a very large slope (Lipschitz constant) to exactly distinguish between the two empirical datasets,and this in turn leads to a large discriminator norm (which captures the Lipschitz constant of thediscriminator).
E Generalization Error for Primal-Dual Risk
If the saddle point exists, the primal-dual risk is often a good measure of generalization:
Iteration count
[Figure]
[Figure]
[Figure]
N o r mo f
D i s c
r
i m i n a t
o r w e
i g h t s 1e
GDMax
GDA
Deﬁnition 7.
The empirical and population primal-dual (PD) risks are deﬁned as:
[Equation]
and
[Equation]
A point ( w, θ ) is called a saddle point of r S (or r ) if ∆ PDS ( w, θ ) = 0 (or ∆ PD ( w, θ ) = 0 ).
Furthermore, if a saddle point ( w S , θ S ) exists for r S ( · , · ) , we have w S = min w ∈ W r S ( w ) .
Moreover,if w S ∈ arg min w ∈ W r S ( w ) and θ S ∈ arg max θ ∈ Θ r S ( w S , θ ) , then ( w S , θ S ) is a saddle point ofr S ( · , · ) .
Notice that if we can ﬁnd an approximate saddle point ( w S , θ S ) of r S ( w, θ ) , i.e., ∆ PDS ( w S , θ S ) < (cid:15)and guarantee that ∆ PD ( w S , θ S ) − ∆ PDS ( w S , θ S ) is small, we can guarantee that ∆( w S , θ S ) issmall and therefore ( w S , θ S ) is an approximate saddle point of r ( · , · ) .
Hence if the saddle point existsfor r S ( · , · ) , the primal-dual risk can be a good measure for the generalization of a solution.
We deﬁnethe expected generalization error for the primal-dual risk as follows:
Deﬁnition 8.
The generalization error for the primal-dual risk is deﬁned as
[Equation]
E.1 The generalization of the primal-dual risk for convex-concave problems
Similar to Deﬁnition 6, we deﬁne the W -capacity as follows:
Deﬁnition 9 (W-Capacity) .
Let
[Equation]
, and
[Equation]
The W -capacities C we and C wp are deﬁned as
[Equation]
Next, we also deﬁne the following:
Deﬁnition 10.
Let f − ( θ, w ; z ) =
− f ( w, θ ; z ) .
We ﬁrst have
[Equation] [Equation]
Furthermore, we deﬁne:
[Equation]
Now, we have the following bound for the generalization error of the primal-dual risk, ζ PDgen ( A ) foran (cid:15)
-stable Algorithm A :
Theorem 7.
Suppose that Algorithm A is (cid:15) -stable.
The generalization error ζ PDgen ( A ) for convexconcave problem, i.e., when f ( · , · ; z ) is convex-concave for all z , is bounded by:
[Equation]
Proof.
Notice that
[Equation]
The two terms can be bounded by Lemma 1 respectively.
By Lemma 1, we have
[Equation]
and
[Equation]
Combining these two inequalities yields the desired result.
E.2 ζ PDgen ( T ) for the proximal point algorithm
In this section, we only study the generalization behavior of the proximal point algorithm (PPA) ((See
Equation (3) in [Farnia and Ozdaglar, 2021])).
By Farnia and Ozdaglar [2021], the stability of Tsteps of PPA can be bounded as follows:
Lemma 15 ([Farnia and Ozdaglar, 2021]) .
The stability of T steps of PPA can be bounded by(cid:15) ≤ ( C e + C we ) T/n .
Therefore, substituting the result of Lemma 15 in Theorem 7, we have the following generalizationbound for T steps of PPA:
Theorem 8.
After T steps of PPA, the generalization error of the primal-dual gap can be bounded by:
[Equation]
E.3 The population primal-dual risk of PPA
Finally, we give the population primal-dual risk after T steps of PPA.
By Mokhtari et al.
[2020b], wehave the following convergence result of PPA.
Lemma 16 ([Mokhtari et al., 2020b]) .
Let ( w tS , θ tS ) be the iterates obtained after t iterations of proximal point algorithm on the function r S ( · , · ) and ¯ w tS = 1 t (cid:80) ti =1 w iS , ¯ θ tS = 1 t (cid:80) ti =1 θ iS be the averaged iterates.
Then we have
[Equation]
Combining Lemma 16 and Theorem 8, we have the following result:
Theorem 9.
Let ( w t S , θ t S ) be the iterates obtained after t iterations of proximal point algorithm on the function r S ( · , · ) and ¯ w tS = 1 t (cid:80) ti =1 w iS , ¯ θ tS = 1 t (cid:80) ti =1 θ iS be the averaged iterates.
Then, the expected population primal-dual risk at the point ( ¯ w tS , ¯ θ tS ) can be bounded by:
[Equation]