Paper: Uniform Localized Convergence and Sharper Generalization Bounds for Minimax Problems (ICLR-2024)

Reviewer comment (Novelty):
the novelty of the proof techniques is not clear to me. Most of them are just incremental compared with the existing work

Author response:
There are two mainly challenges in our work in generalization bounds for minimax problems. On one hand, comparing with uniform convergence in [Zhang et al. 2022], we use a novel uniform localized convergence techniques [Xu and Zeevi, 2020] to construct a functional w.r.t. loss functions on minimax problems. This two layer structure involves difficulties. On the other hand, it is noteworthy that the optimal point $y^*(x) := argmax_{y \in \mathcal{Y}} F(x, y)$ for a given $x$ differs from $y_S^*(x) := argmax_{y \in \mathcal{Y}} F_S(x, y)$, thus introducing an additional error term $|| y^*(x) - y_S^*(x) ||$. Compared to [Zhang et al. 2022], they only need to bound this term with $O(\frac{1}{\sqrt{n}})$. But we need to reach the upper bound of order $O(\frac{1}{n})$ under certain assumptions.

For applications, we found that all the related optimization papers in minimax problems focused on the iteration complexity (or gradient complexity). As a result, they only require $\mathbb{E}[||\nabla \Phi(x_T)||]$ for average of round $T$ with an expected outcome. We were compelled to derive the high probability empirical optimization bound ourselves using classical optimization methods under SC-SC conditions. 

Notice that even under SC-SC settings, achieving this for SGDA remains difficult. Drawing inspiration from [Lei et al., 2021]'s proof of Primal-Dual Risk optimization bound, we eventually derive the optimization bound for primal risk. The proofs for excess risks in applications differ from minimization problems [Li et al., 2021] and pose challenges. These challenges stem from errors in the inner and outer layers of minimax problems. Consequently, we can only achieve a result close to $O(1/n^2)$.

Reviewer comment (Presentation):
The paper lacks a cohesive narrative, as the extensions presented are not tightly interconnected and could be considered in isolation. Specifically, Section 5 appears unrelated to the preceding sections, making the overall structure disjointed. This fragmentation complicates the assessment of the paper's technical contributions.

Author response:
Answer: I believe our work is natural and connected. As we introduced in the introduction, we first introduce a novel ``uniform localized convergence'' framework using generic chaining to obtain better generalization error bounds. For further applications, we all know that excess error is a combination of generalization error and optimization error. So we hope to get better excess error through improved generalization error. Since there is currently no work about the high-probability optimization error under the NC-SC setting, we can only fall back on the excess error under the SC-SC setting. It is precisely due to the improvement of our generalization error that under the SC-SC condition, our excess error reaches $O(1/n^2)$, which is better than best existing results in SC-SC condition ($O(1/n)$) (See comparison in Table 1).

Reviewer comment (Theory):
How do the extensions contribute to a unified narrative on the generalization of minimax optimization?

Author response:
Answer:  Apart from the state-of-the-art results, we introduce a novel generic chaining framework into minimax problems to relaxed the assumptions, propose a minimax version of Bernstein condition, use variance-type concentration inequality and address numerous challenges within the minimax problems, primarily related to the connections between inner and outer layers, the norm of the gradient or the sharper bounds.

Reviewer comment (Novelty):
What constitutes the paper's key technical contribution, and why is it challenging?

Author response:
Answer: We have discussed in Introduction and Remark 6. [Zhang et al. 2022] firstly introduced a expectation generalization error for primal functions in minimax problems using complexity. Naturally, we want to create a high-probability version, preferably using local methods to introduce variance information and obtain a tighter upper bound. As Reviewer WZTS stats, we can certainly continue with the traditional localized approach and solve the problem with covering numbers. However, these technologies require additional bounded assumptions (Assumption 2), or need certain distributional assumptions for unbounded condition. For example, [Mei et al. 2018] introduced the 'Hessian statistical noise' assumption when using covering numbers. Fortunately, [Xu and Zeevi, 2020] developed a novel ``uniform localized convergence'' framework using generic chaining for the minimization problems and [Li and Liu, 2021] extended it to analyze stochastic algorithms.

This novel framework can not only relax the bounded (or specific distribution) assumptions but also impose fewer restrictions on the surrogate function for the localized method, enabling us to design the measurement functional to achieve a sharper bound. Consequently, we introduce this remarkable framework into minimax problems. Our generalization bound uses weaker assumptions comparing with [Zhang et al. 2022] and is sharper in some conditions due to our utilization of variance information.

Introducing this new framework into minimax problems is not straightforward. [Zhang et al. 2022] indeed established a connection between inner and outer layers with the loss of primal functions, but we need do this with a new generic chaining approach. Furthermore, while [Zhang et al. 2022] only needed to bound the error caused by the connection between inner and outer layer with $O(\frac{1}{\sqrt{n}})$. We need to introduce the variance for a sharper bound, necessitating the use variance-type concentration inequalities.

Next, we turn to applications. Firstly, for a sharper excess risk bound, we need to introduce the PL-SC condition to establish a connection between excess risk and the gradient of primal functions, leading to our results for ESP. Unfortunately, for GDA and SGDA algorithms, we found that all the related optimization papers in minimax problems focused on the iteration complexity (or gradient complexity). As a result, they only require $E[||\nabla \Phi(x_T)||]$ for average of round $T$ with an expected outcome. We were compelled to derive the high probability empirical optimization bound ourselves using classical optimization methods under SC-SC conditions. 

Notice that even under SC-SC settings, achieving this for SGDA remains difficult. Drawing inspiration from [Lei et al. 2021]'s proof of Primal-Dual Risk optimization bound, we eventually derive the optimization bound for primal risk. Additionally, in nonconvex minimization problems, several results bound the $T$-th empirical optimization error with high probability. Their main approach involves constructing proper martingale difference sequences, which also presents a challenge in the minimax settings. This remains a topic for future research.

The proofs for excess risks in applications differ from minimization problems and pose challenges. These challenges stem from errors in the inner and outer layers of minimax problems. Consequently, we can only achieve a result close to $O(\frac{1}{n^2})$.

Reviewer comment (Novelty):
How does Theorem 6 differ from existing research, specifically [Lin et al., 2020]?

Author response:
Answer: [Lin et al., 2020] derived the empirical optimization error. Our paper in Theorem 6 derives the \emph{population} optimization error, which can be decomposed into the empirical optimization error and the generalization error. This definition is given in Preliminaries and our paper mainly studies the generalization error.

Reviewer comment (Theory):
Why is the generalization error analysis limited to the SC-SC setting for GDA and SGDA, rather than extending to the NC-SC setting?

Author response:
Answer: We have discussed in above the definition of SC-SC assumption and Remark 10, although the generalization bounds we proved are in NC-SC settings, we require the SC-SC assumptions to derive the empirical optimization error bound of primal functions, to gain the high probability excess risk bound. Existing optimization results do not have a high probability optimization error bound under NC-SC conditions.

Reviewer comment (Theory):
Given that the PL condition is a natural extension of strong convexity, how does the analysis in Section 5.1 differ from that in the SC-SC setting? Is a minor modification of SC-SC results sufficient?

Author response:
Answer: On one hand, compared with the results of other works under stronger SC-SC conditions, we obtained the best results. On the other hand, although PL-SC condition is a natural extension of the SC-SC condition. Since PL condition does not necessarily satisfy convexity, properties related to convexity cannot be used in the proof process, which makes the proof difficult.

Reviewer comment (Writing):
I encountered the term, "almost" $O(\cdot)$, in the paper, but I'm uncertain about its exact meaning. Does this imply that the order of certain elements is not calculated with high accuracy?

Author response:
The expression``almost'' is used to describe the excess risk bound we finally get, which is $O(1/(n(n-c)))$ where $c$ is a constant depending on $\beta, \mu_x, \mu_y$ and $\log(1/\delta)$. We use this word to emphasize this constant term. When $n$ is large, it can be understood as $O(1/n^2)$, which is better than best previous result $O(1/n)$.

Reviewer comment (Theory):
Certain remarks in the paper are challenging to comprehend due to their presentation. For instance, Remark 5 suggests that specific quantities are small in Theorem 3 and (8), but it does not specify their orders or the precise conditions under which they are considered small.

Author response:
When we refer to these terms as ''tiny'', we mean that the order can be $O(\frac{1}{n})$ under certain assumption. In fact, in application section (Section 5), we include the assumption that the optimal population primal function $\Phi(x^*) = O(\frac{1}{n})$ allowing us to derive the result to be approximately $O(\frac{1}{n^2})$. In this situation, these terms are actually ''tiny''.

Reviewer comment (Theory):
In Remark 8, it is stated that $\phi(x^*) = O(\frac{1}{n})$ is a common occurrence. Can you provide examples to illustrate this assertion?

Author response:
Thanks for pointing that out. The assumption of an optimal population primal function $\Phi(x^*) = O(\frac1n)$ is common and weak. This assumption is natural as $F(x^*, y^*)$ represents the minimal population risk and many researches such as (Srebro et al, 2010; Zhang et al, 2017; Lei and Ying, 2020) were used to achieve tighter bound.  

References

Nathan Srebro, Karthik Sridharan, and Ambuj Tewari. Optimistic rates for learning with a smooth loss. 2010.

Lijun Zhang, Tianbao Yang, and Rong Jin. Empirical risk minimization for stochastic convex optimization. 2017.

Yunwen Lei and Yiming Ying. Fine-grained analysis of stability and generalization for stochastic gradient descent. 2020.

Reviewer comment (Theory):
Are there more examples, beyond the one in Remark 2, where the Bernstein condition is met but not the bounded gradient norm? It would be beneficial to see this condition applied in more varied contexts.

Author response:
Yes, we have discussed in Remark 1 when introducing standard Bernstein condition.      ``Bernstein condition is milder than the bounded assumption of random variables and is also satisfied by various unbounded variables. For example, a random variable is sub-exponetial if it satisfies Bernsteion condition (Wainwright, 2019).''

Reviewer comment (Theory):
Regarding the potentially very large sample size, I am concerned that the feasibility of such a condition seems questionable, especially with large sample sizes, potentially diminishing the practical relevance of the results.

Author response:
Theorem 1 (our main Theorem) don't need the potentially very large sample size. For Theorem 3, large sample size is assumed to remove the dependence on the dimension $d$. Of course, we can also relax this assumption of sample size, but the final result will have dimension $d$.

Reviewer comment (Theory):
In the NC-SC setting, the optimal point $x^*$ may not be unique (should be a set $X^*$), while here the authors implicitly assume the uniqueness. I think a better notation can be $\text{dist}(x, X^*)$.

Author response:
Thanks for your suggestion. We will revise in the final version.