What is a Good Metric to Study Generalization of Minimax Learners?
Abstract
Minimax optimization has served as the backbone of many machine learning problems.
Although the convergence behavior of optimization algorithms has been extensively studied in minimax settings, their generalization guarantees in the stochastic setting, i.e., how the solution trained on empirical data performs on the unseen testing data, have been relatively underexplored.
A fundamental question remains elusive:
What is a good metric to study generalization of minimax learners?
In this paper, we aim to answer this question by ﬁrst showing that primal risk , a universal metric to study generalization in minimization, fails in simple examples of minimax problems.
Furthermore, another popular metric, the primal-dual risk , also fails to characterize the generalization behavior for minimax problems with nonconvexity, due to non-existence of saddle points.
We thus propose a new metric to study generalization of minimax learners: the primal gap , to circumvent these issues.
Next, we derive generalization bounds for the primal gap in nonconvexconcave settings.
As byproducts of our analysis, we also solve two open questions: establishing generalization bounds for primal risk and primal-dual risk in the strong sense, i.e., without strong concavity or assuming that the maximization and expectation can be interchanged, while either of these assumptions was needed in the literature.
Finally, we leverage this new metric to compare the generalization behavior of two popular algorithms – gradient descent-ascent (GDA) and gradient descent-max (GDMax) in stochastic minimax optimization.
Introduction
Stochastic minimax optimization, a classical and fundamental problem in operations research and game theory, involves solving the following problem:
[Equation]
More recently, such a minimax formulation has received increasing attention in machine learning, with signiﬁcant applications in generative adversarial networks (GANs)
[Goodfellow et al., 2014], adversarial learning [Madry et al., 2017], and reinforcement learning [Chen and Wang, 2016, Dai et al., 2018].
Most existing works have focused on the optimization aspect of the problem, i.e., studying the rates of convergence, robustness, and optimality of algorithms for solving an empirical version of the problem where it approximates the expectation by an average over a sampled dataset, in various minimax settings including convex-concave
[Nemirovski et al., 2009, Monteiro and Svaiter, 2010], nonconvex-concave
[Lin et al., 2020, Raﬁque et al., 2018], and certain special nonconvex-nonconcave [Nouiehed et al., 2019, Yang et al., 2020] problems.
However, the optimization aspect is not sufﬁcient to achieve the success of stochastic minimax optimization in machine learning.
In particular, as in classical supervised learning, which is usually studied as a minimization problem [Hastie et al., 2009], the out-of-sample generalization performance is a key metric for evaluating the learned models.
The study of generalization guarantees in minimax optimization (and related machine learning problems) has not received signiﬁcant attention until recently [Arora et al., 2017, Feizi et al., 2020, Yin et al., 2019, Lei et al., 2021, Farnia and Ozdaglar, 2021, Zhang et al., 2021b].
Speciﬁcally, existing works along this line have investigated two types of generalization guarantees: uniform convergence generalization bounds, and algorithm-dependent generalization bounds.
The former is more general and irrespective of the optimization algorithms being used, while the latter is usually ﬁner and really explains what happens in practice, when optimization algorithms play an indispensable role.
In fact, the former might not be able to explain generalization performance in deep learning, e.g., these bounds can increase with the training dataset size and easily become vacuous in practice [Nagarajan and Kolter, 2019], making the latter a more favorable metric for understanding the success of minimax optimization in machine learning.
Algorithm-dependent generalization for minimax optimization has been studied recently in [Farnia and Ozdaglar, 2021, Lei et al., 2021, Xing et al., 2021, Yang et al., 2022].
These papers build on the algorithmic stability framework developed in [Bousquet and Elisseeff, 2002], which are further investigated in [Hardt et al., 2016].
In particular, these works have studied primal risk and/or (variants of) primal-dual risk under different convexity and smoothness assumptions of the objective.
Primal risk (see formal deﬁnition in §2) is a natural extension of the deﬁnition of risk from minimization problems.
Primal-dual risk, on the other hand, is deﬁned similarly but based on the duality gap of the solution.
It is know that it is well-deﬁned and can be optimized to zero only when the global saddle-point exists (i.e., min and max can be interchanged).
Based on these metrics, [Farnia and
Ozdaglar, 2021, Lei et al., 2021] compare the performance of speciﬁc algorithms, e.g., gradient descent-ascent (GDA) and gradient descent-max (GDMax).
We provide a more thorough literature review in Section A, and a detailed comparison in Table A.
Although these metrics are natural extensions of generalization metrics from the minimization setting, they might not be the most suitable ones for studying generalization in stochastic minimax optimization, especially in the nonconvex settings that is pervasive in machine/deep learning applications, where the global saddle-point might not exist.
In particular, we are interested in the following fundamental question:
What is a good metric to study generalization of minimax learners 2 ?
In this paper, we make an initial attempt to answering this question, by identifying the inadequacies of the existing metric, and proposing a new metric, the primal gap that overcomes these inadequacies.
We then provide generalization error bounds for our new metric, and discuss how it captures information not included in the other existing metrics, as well as discussing the tightness of the bounds.
We summarize our contributions as follows.
Contributions.
First, we introduce an example through which we identify the inadequacies of primal risk , a well-studied metric for generalization in stochastic minimax optimization, in capturing the generalization behavior of nonconvex-concave minimax problems.
Second, to address the issue, we propose a new metric – the primal gap , which provably avoids the issue in the example, and derive its generalization error bounds.
Next, we leverage this new metric to compare the generalization behavior of GDA and GDMax, two popular algorithms for minimax optimization and GAN training, and answer the question of when does GDA generalize better than GDMax?
Moreover, we also address two open questions in the literature: establishing generalization error bounds for primal risk and primal-dual risk without strong concavity or assuming that the maximization and expectation can be interchanged, while at least one of these assumptions was needed in the literature [Farnia and Ozdaglar, 2021, Lei et al., 2021, Xing et al., 2021, Yang et al., 2022].
Finally, under certain assumptions of the max learner, our results also generalize to the nonconvex-nonconcave setting.
Preliminaries 2.1 Problem formulation
In this paper, we consider the following (stochastic) minimax problem:
[Equation]
We make the following assumption on the sets W and Θ throughout the paper.
Assumption 1.
W and Θ are convex, closed sets, and we further assume that W is compact with (cid:107) w (cid:107) ≤ M ( W ) , ∀ w ∈ W .
Here M ( W ) is a constant dependent on the set W .
Let r ( w, θ ) =
E z ∼ P
z
f
( w, θ ; z ) .
For a training dataset S = { z 1 , · · · , z n } with n i.i.d.
variables drawn from P z , we deﬁne r S ( w, θ )
= 1 n (cid:80) ni
=1 f ( w, θ ; z i ) .
Next, we deﬁne the following quantity:
Deﬁnition 1 (Primal risk (empirical/population)) .
Primal population risk is given by
[Equation]
) , and the primal empirical risk is given by: r S ( w
[Equation]
=
[Equation]
Throughout this paper, we use ( w S , θ S ) to denote a solution of the minimax problem: min w ∈ W max θ ∈ Θ r S ( w, θ ) .
Notice that ( w S , θ S ) need not be a global saddle-point of r S .
Furthermore, we use ( w ∗ , θ ∗ ) to denote a solution of min w ∈ W max θ ∈ Θ r ( w, θ ) .
Once again, notice that ( w ∗ , θ ∗ ) may not be a saddle point of r .
The goal in Problem (1) is to minimize the primal population risk r ( w ) .
Note that this function can be decomposed as
[Equation]
In practice, we only have access to r S ( w, θ ) , and our goal is to design algorithms for minimizing r ( w ) using dataset S .
Suppose A is a learning algorithm initialized at ( w, θ ) =
We deﬁne ( w AS , θ AS ) to be the output of Algorithm A using dataset S .
From Equation (2), it is clear if we ensure r S ( w AS ) as well as r ( w AS ) − r S ( w AS ) are small, this would guarantee that r ( w AS ) is small, which is the goal of Problem (1).
Note that we can always ensure that r S ( w AS ) is small by using a good optimization
Algorithm A (if the problem is tractable).
The main goal in the study of generalization is therefore to estimate the generalization error of the primal risk, as deﬁned below.
Deﬁnition 2.
The generalization error for the primal risk is deﬁned as: ζ Pgen ( A ) =
E S E A [ r ( w AS ) − r S ( w AS )] .
Here the expectations are taken over the randomness in the dataset S , as well as any randomness used in the Algorithm A .
This metric has been used to study generalization in stochastic minimization problems, i.e., when the maximization set Θ is a singleton, as well as several recent works in stochastic minimax optimization (see [Hardt et al., 2016, Farnia and Ozdaglar, 2021, Lei et al., 2021]).
We are interested in the question of when the solution to the empirical problem w AS has good generalization behavior , i.e., when E [ r ( w AS ) − min w ∈ W r ( w )] is small – w AS is an approximate minimizer of the primal population risk r .
In the next subsection, we brieﬂy describe why the generalization error of the primal risk ζ Pgen ( A ) is a good measure to study the generalization behavior in minimization problems.
2.1.1 ζ Pgen ( A ) for minimization problems
Consider a stochastic optimization problem of the form
[Equation]
We deﬁne the (minimization) primal risk (population and empirical version respectively) as: r ( w ) =
E z ∼ P z g ( w ; z ) , and r S ( w ) = 1 n (cid:80) ni
=1 g ( w ; z i ) .
The generalization error ζ P,mingen ( A ) for the (minimization) primal risk is the same as in Deﬁnition 2 using the (minimization) primal risk.
Assume that the generalization error of the primal risk for an Algorithm A is small, say ζ P,mingen ( A ) ≤ (cid:15) .
This implies that (from Deﬁnition 2): E [ r ( w AS )] ≤ E [ r S ( w AS )] + (cid:15) .
Note that the expectation is with respect to S and A .
Now, in order to show that w AS has good generalization behavior, we ﬁrst see that:
[Equation]
However, note that for minimization problems, since E [ r S ] =
r , we have that 4 min w ∈ W r ( w ) ≥
E [min w ∈ W r S ( w )] , which gives us:
[Equation]
Therefore, for minimization problems, if the generalization error for primal risk is small, the solution to the empirical risk minimization problem has good generalization behavior.
Next, we highlight some results in the literature which discusses generalization error bounds of the primal risk.
These results depend on the concept of algorithmic stability we use later.
2.2 Stability of algorithms
Stability analysis is a powerful tool to analyze the generalization behavior of algorithms (see [Bousquet and Elisseeff, 2002]).
In this section, we will review some deﬁnitions and theoretical results about stability bounds existing in the current literature.
More speciﬁcally, in this paper, we adopt the following deﬁnition of stability:
Deﬁnition 3 ( (cid:15) -stable Algorithm) .
Suppose that A is a randomized algorithm for solving the stochastic minimax problem.
We deﬁne ( w AS , θ AS ) as the output of Algorithm A using dataset S .
We say S and S (cid:48) are neighboring dataset if they defer only in one sample.
An Algorithm A is deﬁned to be (cid:15)
-stable if E A (cid:107) w AS − w AS (cid:48) (cid:107) ≤ (cid:15) and E A (cid:107) θ
AS − θ AS (cid:48) (cid:107) ≤ (cid:15) for any neighboring datasets S and S (cid:48) .
[Hardt et al., 2016] gives the following basic result for the generalization error of r S ( w ) .
Theorem 1 ([Hardt et al., 2016]) .
Consider the (stochastic) minimization problem deﬁned in 3.
Suppose g ( · ; z ) is ¯ L -Lipschitz continuous, i.e., ∀ z , it holds that (cid:107) g ( w 1 ; z ) − g ( w 2 ; z ) (cid:107) ≤ ¯ L (cid:107) w 1 − w 2 (cid:107) , ∀ w 1 , w 2 ∈ W .
Then, for an (cid:15) -stable
Algorithm A , we have | E S E A
[ r ( w AS ) − r S ( w AS )]
| ≤ ¯ L(cid:15) .
Unfortunately, Theorem 1 cannot be directly extended to analyze the generalization behavior of minimax learners, because we have an additional maximization step before taking expectation.
Under certain additional conditions, primal risk can be a valid metric for minimax learners.
We provide theoretical justiﬁcations in Section B, instantiated by an adversarial training example.
We deal with the more general cases next.
Primal Gap: A New Metric to Study Generalization
The main idea which leads to the success of ζ Pgen ( A ) as a metric to study generalization for minimization learners is that E [min w ∈ W r S ( w )] ≤ min w ∈ W r ( w ) , which is no longer the case in the minimax case.
In fact, in this section, we show that a good bound for the generalization error of primal risk does not imply good generalization behavior in minimax problems.
We then propose a new metric, the primal gap, which circumvents these issues and provides insights into the generalization behavior in minimax problems.
3.1 Primal risk can fail for minimax learners
We provide an example where the generalization error of the primal risk is small, but the ﬁnal solution to the empirical problem has poor generalization behavior.
In this example, the minimizer of r S ( w ) is suboptimal for r ( w ) with high probability, and E S [ r ( w S ) − r ( w ∗ )] is large.
Example 1 (Analytical example) .
Let y ∼ N (0 , 1 / √ n ) be a Gaussian random variable in R .
Deﬁne the truncated Gaussian variable z ∼ P z as follows:
z = y if | y | < λ log n/ √ n and z = λ log n/ √ n if y ≥ λ log n/ √ n .
Let f ( w, θ ; z ) = 12 w 2 − (cid:0) 12 n 2 θ 2 − zθ + 1 (cid:1) w , where w ∈ W =
[0 , 1] , θ ∈ Θ =
[ − λn, λn ] with a sufﬁciently large λ > 0 , and z i ∼ P z be i.i.d truncated Gaussian variables.
Then, we have r S ( w, θ )
=
12 w 2 − (cid:16) 12 n 2 θ 2 − (cid:80) ni
=
z i n θ + 1 (cid:17) w , and
[Equation]
Note that this leads to the primal population risk function: r ( w ) =
12 w 2 − w .
It is not hard to see that we always have r S ( w ) ≥ r ( w ) .
Note that this means ζ Pgen ( A ) ≤ 0 , and thus we have a small generalization error for primal risk.
However, we can prove that for large enough λ ,
[Equation]
This means that w S has a constant error compared to w ∗ in terms of the population risk, despite that its generalization error is small.
This phenomenon is due to that min w ∈ W r S ( w ) − min w ∈ W r ( w )
>
c for some c > 0 , and hence minimizing r S ( w ) is very different from minimizing r ( w ) .
This example shows that the generalization error of primal risk is not a good measure to study generalization in minimax learners.
The main drawback is that min w r S ( w ) and min w r ( w ) can be very different.
We now introduce another more practical example, from GAN training, to further illustrate this point.
Example 2 (GAN-training example) .
Suppose that we have a real distribution P r in R d which can be represented as G ∗ ( y ) with y ∈ R k drawn from a standard Gaussian distribution P 0 and a mapping
G ∗ : R k
→ R d .
For an arbitrary generator G , we deﬁne P G to be the distribution of the random variable G ( y ) with y ∼ P 0 .
So our goal is to ﬁnd a generator G such that P G = P r .
GAN is a popular tool for solving this problem.
Consider a GAN with generator G , parametrized by w and discriminator D parametrized by θ .
The goal of GAN training is to ﬁnd a pair of a generator G and a discriminator D that solves the minimax problem:
[Equation]
where φ : R → R is concave, monotonically increasing and φ ( u )
=
−∞ for u ≤ 0 .
To connect to the minimax formulation in (1) , we note that z
= ( x, y ) , and P z
= P r × P 0 .
Also, we denote
[Equation] [Equation]
[Equation]
] to be the population risk.
We now give the empirical version of this problem.
Let
S 1 = { x 1 , · · · , x n } and S 2 = { y 1 , · · · , y n } .
Let S
= S 1 ∪ S 2 and r S ( w, θ )
=n ( (cid:80) ni
=1 φ ( D θ ( x i ) + φ (1 − D θ ( G w ( y i )))) .
We assume that P G w has the same support set as
P r .
Moreover, we assume that (cid:107) w − w ∗ (cid:107) ≤ 0 .
5 and G w ( y ) is 1 -Lipschitz w.r.t.
w for any y .
Here w ∗ denotes the parameter for which G w ∗ =
G ∗ .
Then, combining Theorem B.1 in [Arora et al., 2017] and the Lipschitz continuity of G w ( y ) as well as (cid:107) w − w ∗ (cid:107) ≤ 0 .
5 , we have that the distance between the sets S 1 and { G w ( y 1 ) , G w ( y 2 ) , · · · , G w ( y n ) } will be larger than 0 .
6 with probability greater than 1 − O ( n 2 /e d ) .
Now, if n is only of polynomial size of d , the optimal discriminator for disjoint datasets outputs 1 on one dataset, and 0 on the other.
On the other hand, when w = w ∗ , the optimal discriminator for the population problem outputs 1 / 2 for any sample it receives.
Combining these two results, we have: E S [min w ∈ W r S ( w ) − min w ∈ W r ( w )] ≥ (1 − δ ) (2 φ (1) − 2 φ (1 / 2)) , which is bounded away from 0 .
Note that in this example, we also have E S [min w r S ( w ) − min w r ( w )] > 0 , implying that using ζ Pgen ( A ) might not be a good way to characterize the generalization behavior in GAN training.
To address this issue, we next deﬁne a new metric, the primal gap, and use its generalization error to study the generalization of minimax learners.
3.2 Primal gap to the rescue
The population and empirical versions of the primal gap are deﬁned as follows:
Deﬁnition 4 (Primal gap (empirical/population)) .
The population primal gap is deﬁned as ∆( w ) =
r ( w ) − min w ∈ W r ( w ) , and the empirical primal gap is deﬁned as ∆ S ( w ) =
r S ( w ) − min w ∈ W r S ( w ) .
Notice that these two primal gaps can always take 0 at w S ∈ arg min w ∈ W r S ( w ) and w ∗ ∈ arg min w ∈ W r ( w ) respectively even if the saddle point of problem (2) does not exist.
Next, we deﬁne the expected generalization error of this primal gap as follows:
Deﬁnition 5.
The generalization error for the primal gap is ζ PGgen ( A ) =
E S E A [∆( w AS ) − ∆ S ( w AS )] .
Notice the fact that
[Equation]
If E S E A [ r S ( w AS ) − min w ∈ W r S ( w )] is small and ζ PGgen ( A ) is small (or large), then
E S E A [ r ( w AS ) − min w ∈ W r ( w )] is small (or large).
Now we provide bounds on ζ PGgen ( A ) for a stable algorithm A , and show that in Example 1, ζ PGgen ( A ) cannot be small (unlike ζ Pgen ( A ) ).
3.3 Relationship between generalization and stability
We provide bounds for the generalization error of the primal gap (Deﬁnition 5) for an (cid:15) -stable
Algorithm A .
We will focus on the nonconvex-concave case where the following assumptions are made throughout the rest of the paper.
Assumption 2.
The function f in Problem (1) is nonconvex-concave, i.e., f ( w, · ; z ) is a concave function for all w ∈ W and for all z .
Next we deﬁne the notion of capacity , which will play a key role in the bounds we derive for ζ PGgen ( A ) .
Deﬁnition 6 (Capacity) .
For any w ∈ W and any constraint set Θ , we deﬁne
[Equation]
We deﬁne the capacities C p and C e as:
[Equation]
where dist( p, S ) denotes the distance between a point p to a set S in Euclidean space, i.e., dist( p, S ) :
= inf q ∈S (cid:107) p − q (cid:107) 2 .
For the constraint set in Problem (1) , we succinctly denote the capacities as C p and C e , respectively.
The norm of the model parameter (its distance to 0 ) is usually viewed as the metric for the complexity of the model.
In fact, the norm of the optimal solution determines the Rademacher complexity of the function class in statistical learning theory [Vapnik, 1999].
Moreover, in deep learning, minimumnorm solution of overparameterized neural networks is well-known to enjoy better generalization performance [Zhang et al., 2021a].
Hence, we view the capacity constant C e and C p as natural metrics to capture the model complexity for the best response of the max learner, i.e., the power of the maximizer, when using the empirical data set and population data respectively.
Now, we are ready to discuss the relationship between the stability bound and the generalization error of algorithms in nonconvex-concave minimax problems.
All proofs have been deferred to the appendix.
We make the following assumptions throughout the paper:
Assumption 3.
The gradient of f is (cid:96) -Lipschitz-continuous for all z , i.e., for all z (cid:107)∇ f ( w 1 , θ 1 ; z ) − ∇ f ( w 2 , θ 2 ; z ) (cid:107) ≤ (cid:96) ( (cid:107) w 1 − [Equation] (cid:107) + (cid:107)
[Equation]
−
[Equation]
2 (cid:107) ) , ∀ w 1 , w 2 ∈ W, ∀ θ 1 , θ 2 ∈ Θ .
Moreover, ﬁxing w ∈ W , the partial gradient ∇ θ f ( w, · ; z ) is (cid:96) θθ -Lipschitz continuous with respect to θ for all z , i.e., (cid:107)∇ θ f ( w, θ 1 ; z ) − ∇ θ f ( w, θ 2 ; z ) (cid:107) ≤ (cid:96) θθ (cid:107) θ 1 − θ 2 (cid:107) , ∀ w ∈ W, ∀ θ 1 , θ 2 ∈ Θ .
Assumption 4.
For any Θ 1 ⊆ Θ , we assume that f is L (Θ 1 ) -Lipschitz-continuous with respect to w ∈ W, θ ∈ Θ 1 for all z , i.e., (cid:107) f
( w 1 , θ 1 ; z ) − f ( w 2 , θ 2 ; z ) (cid:107) ≤ L (Θ 1 )( (cid:107) w 1 − w 2 (cid:107) + (cid:107) θ 1 − θ 2 (cid:107) ) , ∀ w 1 , w 2 ∈ W, ∀ θ 1 , θ 2 ∈ Θ 1 , and the gradient ∇ f ( w, θ ; z ) is uniformly bounded as (cid:107)∇ w,θ f ( w, θ ; z ) (cid:107) ≤ L (Θ 1 ) for all z and w ∈ W, θ ∈ Θ 1 .
Moreover, f ( w ∗ , · ; z ) is L ∗ θ -Lipschitz continuous with respect to θ where w ∗ ∈ arg min w ∈ W r ( w ) .
We also deﬁne L :=
L ( B (0 , 2 C p +1)
∩ Θ) and L r :=
L ( B (0 , r ) ∩ Θ) , where B ( v, r ) denotes the l 2 -ball with radius r centered at v .
Note that we can decompose the generalization error of the primal gap as follows:
[Equation]
We now provide a generalization error bound for the primal risk ζ Pgen ( A ) .
To the best of our knowledge, it is the ﬁrst bound for ζ Pgen ( A ) in the nonconvex-concave (not strongly concave) setting.
Lemma 1.
The generalization error of the primal risk of an (cid:15) -stable
Algorithm A for a minimax problem with concave maximization problem can be bounded by ζ Pgen ( A ) ≤ (cid:113) 4 L(cid:96)C 2 p · √ (cid:15) + (cid:15)L .
We show that this dependence on (cid:15) is tight in Section D.2.
Since we already have the generalization error for the primal risk E S E A [ r ( w AS ) − r S ( w AS )] from Lemma 1, we only need to estimate
E
[Equation]
The following theorem gives the generalization bound of the primal gap using the upper bound from
Lemma 1 and bounding the Primal Min Error in Equation (7).
Theorem 2.
Suppose Algorithm A is (cid:15)
-stable.
The generalization error bound of the primal gap is given by ζ
[Equation]
The ﬁrst term in the bound above is from the generalization bound of the primal risk, as shown in Lemma 1.
Note that the bound in Lemma 1 only involves C p , as the key in the analysis is to upper-bound the population risk r ( w AS ) , which requires bounding the power of the maximizer using the population capacity C p .
This reﬂects the intuition that the power of the maximizer should affect the generalization behavior of minimax learners, and the stronger the maximizer is, the harder for the learner to generalize.
On the other hand, the bound in Theorem 2 additionally involve C e , the empirical capacity.
Technically, C e (instead of C p ) appears since we need to bound min w r S ( w ) (deﬁned on the empirical dataset) in the Primal Min Error term in (7).
We show the tightness of this bound in Section D.4.
The appearance of C e reﬂects the intuition that the difference between the maximizers of the empirical and population risks should make a difference in characterizing the generalization of minimax learners.
This intuition cannot be captured by the generalization error of the primal risk, as in Lemma 1.
Note that in the minimization case, the Primal Min Error can be upper-bounded directly by zero, and such a distinction disappears, making primal risk a valid metric.
3.4 Revisiting Example
In Example 1, we have that the primal risk generalizes well, but the solution w S does not, thereby indicating that the generalization error of the primal risk may not be a good metric to study generalization behavior of minimax problem.
Note that in Example 1, we have ζ PGgen ( A ) ≥ 0 .
02 while ζ Pgen ( A ) ≤ 0 .
We already know that this algorithm can not generalize well.
Therefore, ζ PGgen ( A ) can capture the generalization behavior better than ζ Pgen ( A ) .
This shows that the primal min error ζ PMgen also plays an important role in analysing the generalization for minimax problems.
As shown in the appendix (Proposition 4), we have
[Equation]
On the other hand, it is easy to compute that L ∗ θ = λ log n/ √ n and C e = λn log n .
Therefore, by Theorem 2, we have an upper bound for the Primal Min Error (see (7)): E S [min w ∈ W r S ( w ) − min
w ∈ W r ( w )] ≤ 4 L ∗ θ C e / √ n = 4 log n, which is tight up to a log factor according to (8).
Therefore, the primal gap has a constant generalization error which is consistent with the observation that the solution to the empirical problem does not have good generalization behavior.
3.5 Nonconvex-nonconcave case
In this section, we extend our results to the nonconvex-nonconcave setting.
We will show that under certain assumptions on the inner maximization problem, we can derive generalization error bounds for the primal risk and primal gap in terms of algorithmic stability.
We make the following assumptions on the inner maximization problem:
Assumption 5.
For any γ > 0 , there exists an algorithm which outputs θ γP ( w ) , for the inner maximization problem max θ ∈ Θ r ( w, θ ) , satisfying the following conditions: 1) r ( w − r
( w, θ γ
P
( w )) ≤ γ
2) (cid:107) θ γP ( w ) − θ γP ( w (cid:48) )
(cid:107) ≤ λ p γ (cid:107)
w − w (cid:48) (cid:107) with some constant λ p > 0 for all w, w (cid:48) ∈ W .
Assumption 6.
For any γ > 0 , there exists an algorithm which outputs θ γE ( S ) , for the inner maximization problem max θ ∈ Θ r S ( w ∗ , θ ) , satisfying the following conditions: 1)
r
S
( w ∗ ) − r S ( w ∗ , θ γE ( S )) ≤ γ . 2)
For any
neighboring
dataset S and S (cid:48) , (cid:107) θ γE ( S ) − θ γE ( S (cid:48) )
(cid:107) ≤ λ e nγ for some λ e > 0 .
The following lemma gives sufﬁcient conditions for these two assumptions to hold.
Lemma 2.
Consider constants D e ≥ γ and D p ≥ γ .
Suppose gradient ascent with diminishing stepsizes c 0 /t for the problem max θ ∈ Θ r ( w, θ ) has convergence rate r ( w ) − r ( w, θ s ) ≤ D p /s .
Then we deﬁne θ γp ( w ) by performing s = D p /γ steps of gradient ascent.
Then, θ γp ( w ) satisﬁes Assumption 5. 2.
Suppose gradient ascent with constant stepsize c 0 for the problem max θ ∈ Θ r ( w, θ ) has convergence rate r ( w ) − r ( w, θ s ) ≤ D p η s for some constant 0 < η < 1 .
Then we deﬁne θ γp ( w ) by s = log( D p /γ ) / log(1 /η ) steps of gradient ascent.
Then, θ γp ( w ) satisﬁes Assumption 5. 3.
Suppose gradient ascent with diminishing stepsizes c 0 /t for the problem max θ ∈ Θ r S ( w, θ ) has convergence rate r S ( w ) − r S ( w, θ s ) ≤ D p /s .
Then we deﬁne θ γe ( S ) by performing s
= D e /γ steps of gradient ascent.
Then, θ γe ( S ) satisﬁes Assumption 6. 4.
Suppose gradient ascent with constant stepsize c 0 for the problem max θ ∈ Θ r S ( w, θ ) has convergence rate r S ( w ) − r S ( w, θ s ) ≤ D e η s for some constant 0 < η < 1 .
Then we deﬁne θ γe ( w ) by s = log( D e /γ ) / log(1 /η ) steps of gradient ascent.
Then, θ γe ( S ) satisﬁes Assumption 6.
Remark 1.
Note that for some practical nonconvex optimization problems in machine learning, gradient descent indeed converges to the global minima at a reasonably fast rate, e.g., in training deep overparametrized neural networks [Du et al., 2019], robust least squares problems [El Ghaoui and Lebret, 1997], phase retrieval and matrix completion [Ma et al., 2019].
Our Assumptions 5 and 6 can be viewed as an abstract summary of some benign properties of gradient descent for certain nonconvex optimization problems.
Furthermore, we assume that f ( · , · ; z ) is L -Lipschitz 5 continuous in W × Θ .
This, along with
Assumptions 5 and 6, allows us to derive the generalization error bounds of the primal risk and primal gap in terms of algorithmic stability.
Lemma 3.
Suppose that Assumption 5 holds.
If a minimax learner A is an (cid:15) -stable
algorithm, we have ζ Pgen ( A ) ≤ L(cid:15)
+ (cid:112)
Lλ p √ (cid:15).
Similarly, we can derive the generalization bound for the primal gap given the above assumptions.
Theorem 3.
Suppose Assumptions 5 and 6 hold.
Then we have ζ PGgen ( A ) ≤ ζ Pgen ( A ) + √
Lλ e (cid:14) √ n.
The proof of this theorem is similar to the proof of Lemma 3 and Theorem 2 and hence omitted.
Comparison of GDA and GDMax
In Section 3.3, we provide generalization bounds for the primal gap for any (cid:15)
-stable algorithm.
In this section, we focus on two algorithms in particular – GDA and GDMax.
These two algorithms are described in Algorithms 1 and 2 in Appendix E.
We note that though analyzing the optimization properties of GDA/stochastic GDA for solving the empirical minimax problem is an important topic, our focus in this paper is on studying the generalization behavior of these algorithms.
We assume that the empirical version of the stochastic minimax problem can be solved by GDA and GDMax, i.e., we assume that GDA and GDMax satisfy the following assumption:
Assumption 7.
Let A be a minimax learner, such as GDA or GDMax.
Then we assume that A has the following convergence rate: E A [ r S ( w t ) − min w ∈ W r S ( w )] ≤ ( φ A ( M ( W ))
+
φ A ( C e )) /ψ
A ( t ) , where M ( W ) is the maximum of the norms of w , and φ A ( s ) , ψ A ( s ) are nonnegative, increasing functions that tend to inﬁnity as s → ∞ .
For simplicity, throughout this section, we assume that (cid:107) f
( w, θ ; z ) (cid:107) ≤ 1 for all w, θ , and z .
The next theorem provides a bound for the population primal gap ∆( w AS ) :
= r ( w AS ) − min w ∈ W r ( w ) .
Note that the goal of any algorithm is to make this gap as small as possible.
For an Algorithm A and subsets W 0 ⊆ W, Θ 0 ⊆
Θ , we deﬁne A ( W 0 , Θ 0 ) as the algorithm which restricts A to solve (1) under constraint sets W 0 and Θ 0 .
Speciﬁcally, A ( W, Θ) is just A .
Theorem 4.
Let w A,tS , θ A,tS be the t -th iterate generated by Algorithm A using dataset S .
Assume that { θ A,tS } ⊆
Θ 0 =
Θ Aθ for t ≤ T with probability 1 − δ (due to the randomness in S ) and
B (0 , C p ) ⊆
Θ Aθ .
Here B ( v, r ) denotes the l 2 -ball with radius r centered at v .
Let A
= A ( W, Θ 0 ) .
Then after T iterations of Algorithm A , the population primal gap can be bounded as:
[Equation]
where ζ Pgen ( A 0 ) =
E S E A [ r ( w A 0 ,T S )
− r S ( w A 0 ,T S )] is the generalization error of the primal risk of
Algorithm A 0 .
Remark 2.
Theorem 4 builds a closer connection between generalization behavior and the dynamics of the minimax learner A .
It shows that suitable restriction to the max learner can lead to better minimax learner, in terms of generalization.
We make this clear in the comparison of GDA and
GDMax by analyzing the three terms in Theorem 4.
4.1 Analyzing the term I
First, we study the generalization error bound of the primal risk, i.e., ζ Pgen in Theorem 4.
For GDA, we can estimate ζ Pgen by using Lemma 1.
Therefore, it sufﬁces to estimate the stability of GDA.
We do this in the following lemma:
Lemma 4.
Let c 0 = max { α 0 , β 0 } , If we use diminishing stepsizes α t
= α 0 /t and β t = β 0 /t for
GDA for T iterations, we have the stability bound (cid:15) GDA ≤ 2 L Θ GDAθ T c 0 (cid:96) /
( n(cid:96) ) .
Now, since we have a bound for ζ Pgen ( A ) for an (cid:15) -stable Algorithm A in Lemma 1, we can substitute the stability bound for GDA from Lemma 4 in this expression to get a bound on ζ Pgen ( GDA ) for
GDA.
We do this in the next proposition.
We can bound ζ Pgen ( A 0 ) for GDA by substituting the stability bound in Lemma 4 into Lemma 1 (letting (cid:15) =
(cid:15) GDA ).
Proposition 1.
Let c 0 = max { α 0 , β 0 } and assume that f ( · , · ; z ) is L Θ GDAθ -Lipschitz-continuous inside the set W × Θ GDAθ .
For GDA with diminishing stepsizes α 0 /t, β 0 /t run for T iterations (denoted by GDA T ), the generalization error of the primal risk can be bounded by:
[Equation]
However, for GDMax, we can not compute a uniform stability bound that vanishes as n goes to inﬁnity.
In fact, we can show from the following simple example that ζ Pgen ( GDMax ) can be a constant that is independent of n , which means that for the case where r ( w, θ ) is nonconvex-concave, the generalization error of primal risk of GDMax can be undesirable.
Example 3 (Constant generalization error of primal risk for GDMax) .
Consider a dataset S with n elements.
Deﬁne the objective function: f ( w, θ ; z ) =
(cid:0) wn 2 − z (cid:1) θ − θ 2 2 n , where w ∈ W = [ − n √ n, n √ n ] , θ ∈ Θ
= R and z is drawn from the uniform distribution over {− 1 / √ n, 1 / √ n } .
We have r S ( w ) = n 2 2 (cid:0) wn 2 − 1 n (cid:80) ni
=
z i (cid:1) 2 , and r ( w ) =
w 2 2 n 2 .
Therefore, min w ∈ W r ( w ) = 0 .
From the deﬁnition of the function f and the sets W and Θ , we have (cid:96) =
1 /n 2 , L = O (1 / √ n ) .
One step of GDMax can attain the minimizer of r S ( w ) (since it is a one dimensional quadratic problem), i.e., w S = n (cid:80) ni
=1 z i and r S ( w S )
= 0 .
Furthermore, we have E S r ( w S ) =
E [ ( (cid:80) ni
=
z i )2] = 1 / 2 > 0 .
Thus, ζ Pgen ( GDMax ) =
E [ r ( w S ) − r S ( w S )]
= 1 / 2 > 0 cannot be made small.
Therefore, from Proposition 1 and Example 3, we see that the bound for the expected population primal gap contains the term ζ Pgen which cannot be bounded for GDMax, whereas can be bounded for
GDA which leads us to the conclusion that GDA generalizes better than GDMax for such problems.
However, it is possible to bound ζ Pgen ( GDMax ) in certain problems, and in this case the other terms in Theorem 4 become crucial.
We analyze them next.
4.2 Analyzing the term II
As shown in Example 1, sometimes GDMax can have a good generalization bound for the primal risk.
Therefore, we need to analyze the other two terms in Theorem 4, i.e., ( φ A ( M w ) + φ A ( C e (Θ Aθ ))) /ψ
A ( T ) and L ∗ θ C e (Θ Aθ ) / √ n .
For these two terms, since L ∗ θ is ﬁxed, the constant
C e (Θ Aθ ) is the key term which differentiates the performance of different algorithms.
By deﬁnition, the constant C e (Θ GDMaxθ ) for GDMax is nearly C e (See Deﬁnition 6).
Therefore, the population primal gap after T steps of GDMax is dominated by C e if C e is large.
However, the set Θ GDAθ for GDA can be much smaller than Θ , which implies that C e (Θ GDAθ ) can be much smaller than C e .
This phenomenon can be seen from Example 1: If we perform one step of GDMax with primal stepsize 1 , we can attain w 1 = w S .
Then E S [ r ( w 1 S ) − min w ∈ W r ( w )] ≥ 0 .
005 from (6).
For
GDA, we can see that w 1 = 1 after one step of GDA with stepsize 1 .
Therefore, GDA generalizes better than GDMax.
Generally, we have the following estimate of C e (Θ GDAθ ) .
Lemma 5.
Let L 0 = max z (cid:107)∇ f ( w 0 , θ 0 ; z ) (cid:107) .
Let c
= max { α 0 , β 0 } .
If we use diminishing stepsizes α t = α 0 /t and β t = β 0 /t for GDA, then after T steps we have (cid:107) θ t (cid:107) ≤ T c 0 (cid:96) L 0 /(cid:96) for t ∈
[ T ] .
Therefore, if C e is much larger than C p , using GDA with C p ≤ T c 0 (cid:96) L 0 /(cid:96)
≤ C e is better than
GDMax.
We make this more concrete in the context of GAN training next.
4.3 GAN training
We now study the speciﬁc case of GAN training to explore why GDA might generalize better than GDMax.
This is numerically veriﬁed in the literature, such as Farnia and Ozdaglar [2021].
Speciﬁcally, we revisit Example 2, and consider a special case: D is restricted to be a overparametrized linear function with respect to θ .
Deﬁne the descriminator D ( x ) =
Φ T ( x ) v + b 0 , where Φ( x ) =
[Φ 1 ( x ) , · · · , Φ m ( x )] T ∈ R m is the feature matrix and b 0 ∈ R .
Also suppose that G is parametrized by w and G ∗ =
G w ∗ .
Then the GAN problem can be written as min w ∈ W max θ ∈ Θ r ( w, θ ) , where
[Equation]
Here θ = ( v, b 0 ) .
Assume that (cid:113) σ max (cid:0) E x ∼ P Gw Φ( x )Φ T ( x ) (cid:1) ≤ ¯ σ max / √ m , where σ max ( · ) denotes the largest singular value of a matrix and ¯ σ max > 0 is a constant.
Also assume that
E x ∼ P Gw Φ( x )Φ T ( x ) is full rank.
Also, we assume that | φ (cid:48) ( λ ) | ≤ L φ for any λ ∈ [0 , 1] .
Therefore, we have E [
(cid:107)∇ θ f ( w, θ ; z ) (cid:107) 2 ]
≈ L 2 φ ¯ σ 2max .
Then it is reasonable to assume that (cid:107)∇ f (cid:107) ≤ O (1) .
Lemma 6.
Suppose Φ( x ) is sub-Gaussian and the matrix
Q
[Equation] [Equation]
x n ) Φ( G [Equation] ( y 1 )) · · · Φ( G w ( y n ))] is full column rank ( m > n ) with probability 1 .
Then with probability at least 1 − Cδ with some constant C , we have (cid:107) θ S ( w ∗ ) (cid:107) ≥ Ω( √ n ) ,
where θ S ( w ∗ ) ∈ arg max θ ∈ Θ r S ( w ∗ , θ ) .
Now, for θ ∈ arg max θ (cid:48) ∈
Θ r ( w ∗ , θ (cid:48) ) , it can be easily seen that v = 0 , b 0 = 1 / 2 in this case.
Therefore, C p ≈ 1 / 2 .
Finally, combining the previous discussion on GDA in Lemma 5, and using the fact that C e is large from Lemma 6, we see from Theorem 4 that GDA can generalize better than
GDMax.
More detailed discussion of the GAN-training and Lemma 6 can be found in Section E.
Conclusions
In this paper, we ﬁrst demonstrate the shortcomings of one popular metric, the primal risk, in terms of characterizing the generalization behavior of minimax learners.
We then propose a new metric, the primal gap, whose generalization error overcomes these shortcomings and captures the generalization behavior of algorithms that solve stochastic minimax problems.
Finally, we use this newly proposed metric to study the generalization behavior of two different algorithms – GDA and GDMax, and study cases where GDA has a better generalization behavior than GDMax.
Future directions include further investigation of the proposed new metric, the primal gap, and deriving its (tighter) generalization error bounds in other structured stochastic minimax optimization problems in machine learning.
Acknowledgments and Disclosure of Funding
Research was sponsored by the United States Air Force Research Laboratory and the United States Air
Force Artiﬁcial Intelligence Accelerator and was accomplished under Cooperative Agreement Number
FA8750-19-2-1000.
The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the ofﬁcial policies, either expressed or implied, of the United States Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. S.P. acknowledges support from MathWorks Engineering Fellowship.
A.O. and K.Z. were supported by MIT-DSTA grant 031017-00016.
K.Z. also acknowledges support from Simons-Berkeley Research
Fellowship.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e
g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]