Paper: Stability and Generalization in Free Adversarial Training (ICLR-2024)

Reviewer comment (Theory):
The theoretical support for FreeAT having a smaller generalization gap than VanillaAT could be more rigorous. Specifically, while Theorem 2 presents pessimistic results for the convergence of VanillaAT, it is unclear whether this bound is tight. It is unclear whether the convergence difference between vanilla and free AT is due to the algorithm itself or some artifacts of the proof technique. While experiment results support this intuition, the paper would benefit from some additional explanations. It would be even better if some lower bounds could be provided for $\mathcal{E}\_{\textrm{gen}} (A\_{\textrm{Vanilla}})$.

Author response:
**Re**: The reviewer is right that the generalization bound in Theorems 2 and 4 could be pessimistic in the sense that we only assume bounded Lipschitz and smoothness coefficients of the learning objective function. We would like to clarify that we did not make a definite theoretical statement on the generalization comparison of vanilla and free adversarial training, since our stability-based generalization bounds *only suggest* a lower expected generalization gap for free AT in comparison to vanilla AT. As pointed out by the reviewer, a generalization lower-bound would shed light on the tightness of the bounds in Theorems 2, 4. However, to the best of our knowledge, the question of a non-trivial lower-bound on the *expected* generalization gap (the quantity bounded in the stability-based approach) is still an open problem for both nonconvex-concave and nonconvex-nonconcave min-max learning problems. 

To investigate the discussed implications of our theoretical bounds, we performed numerical experiments on standard datasets and neural net architectures, which is similar to other theoretical works on the generalization of adversarial learning algorithms. The results of the numerical experiments are consistent with our hypothesis on the comparison of generalization error between vanilla and free AT algorithms.

Reviewer comment (Experiments):
The relationship between a smaller generalization gap and better transferability is unclear. The motivation for the experiment setting of transferring attacks from a robust model to a standard model is weak. I suggest moving the transferability analysis to the Appendix (it's still good to have them) and making space for Table 2, which supports your main claim.

Author response:
**Re**: As recommended by the reviewer, we deferred the numerical results on the attack transferability to the Appendix, and used the space to present the numerical results on the role of the size of training set $n$, which was suggested by the reviewer.

Reviewer comment (Experiments):
Since the proposed convergence bounds depend on the dataset size $n$, this paper would benefit from some empirical comparisons between free and vanilla AT with different $n$ values.

Author response:
**Re**: Following the reviewer’s suggestion, we performed numerical experiments by randomly sampling a subset of size $n \in \lbrace 10000, 20000, 30000, 40000, 50000\rbrace$ from CIFAR-10 and CIFAR-100 training sets, and trained ResNet18 neural nets using those training sets. The numerical results of our comparison of the generalization gaps between free and vanilla AT are presented in Figure 4 and Appendix B.3 of the revised submission. The results indicate that the generalization gap of free AT is decreasing considerably faster than vanilla AT with dataset size $n$. We thank the reviewer for the great suggestion.

Reviewer comment (Novelty):
While the stability results for the free AT method are first-ever-known, the proof techniques seem to be incremental and the paper did not illustrate clearly what the main technical contribution is, particularly considering there is a considerable amount of work on stability analysis.

Author response:
**Re**: Free AT is a widely used variant of adversarial training, mainly due to its higher speed in application to large-scale learning problems. While the free AT method has been extensively used in many applications, the existing theory literature on adversarial training methods has not focused on the specific properties of free AT algorithm. In our work, we focus on the generalization analysis of models learned by free AT, which, to the best of our knowledge, has not been studied in previous works.

Regarding the reviewer’s comment on our work’s technical contribution, we note that our generalization guarantee for free AT (Theorem 4) does not follow from any existing stability-based generalization bound for min-max learning frameworks. Specifically, our generalization analysis accounts for the following properties of the free AT method, which have not been considered in the related works: 

1- Re-initialization of the max variable (adversarial perturbations) after the completion of optimization steps for every batch of training data\
2- Utilizing the normalized gradient (instead of vanilla gradient) for the gradient ascent step of solving the maximization sub-problem\
3- Using mini-batch stochastic optimization for updating min and max variables at every iteration

The generalization analysis under the above conditions and the comparison of generalization performance between vanilla vs. free adversarial training are our work’s main technical contributions, which we believe are novel results that do not follow from any existing bound in the literature.

Reviewer comment (Theory):
It is not clear what the free AT method aims to minimize or optimize. The objective function of the vanilla AT method is given on page 3, i.e. $R_S(w)$ or $R(w)$. From the pseudo-code of Algorithm 3, there are two random samplings—one for mini-batch and one for $\{\delta_j\}$—and then $w$ and $\delta$ are updated by gradient descent and ascent, respectively. In this sense, does the free AT method aim to minimize the following objective  
$$ \min_w \max_\delta {1\over n } \sum_{j=1}^n \int_{\delta_j\in \Delta} h(w,\delta_j; x_j,y_j)$$

Author response:
**Re**: The free AT algorithm has been originally proposed in [3] to solve the standard min-max formulation of adversarial training problems. The reviewer’s question points to an interesting research direction on the effects of the optimization steps in free AT on the target min-max optimization problem. While this problem sounds an interesting direction to explore on free AT, we think our stability-based generalization analysis is orthogonal to this research direction.

Please note that our main objective is to compare the generalization error of free vs. vanilla AT methods, and to perform a fair comparison we should use the same generalization error metric for both these AT algorithms. According to the standards in the adversarial learning literature, we chose this generalization metric to be the difference between the worst-case training and test error under norm-bounded perturbations. Therefore, to ensure a fair comparison between free and vanilla adversarial training, we have to use the same generalization metric for the analysis of free AT. Any different selection of the generalization metric for free AT would have led to an unfair comparison of the generalization error between the two methods.

Reviewer comment (Writing):
What is the definition of $\Delta$?

Author response:
**Re**: As we stated at the beginning of Section 3.1, $\Delta$ is the set of all possible perturbations, which is usually an $L_2$-norm or $L_\infty$-norm bounded ball of some radius $\varepsilon$.

Reviewer comment (Writing):
Given $S$, $w=A(S)$ is a random variable or constant?

Author response:
**Re**: As we stated in Section 3, $A$ is a potentially randomized algorithm that takes a dataset $S$ as input and outputs model parameter $w=A(S)$. Since the randomness of $A$ may come from random initialization or random batch selection, $\mathbb{E}_A$ is taking the expectation over such randomness, and $w=A(S)$ will therefore be a random vector.

Reviewer comment (Writing):
What's the definition of $S'$ here?

Author response:
**Re**: Following our statement in Definition 1, $S’$ is any dataset that has the same size as $S$ and differs from $S$ in at most one example.

Reviewer comment (Writing):
What is the definition of "$A$ is $\epsilon$-uniformly stable"?

Author response:
**Re**: Please refer to Definition 1 for the formal definition of “$\epsilon$-uniformly stable”, which follows from references [1,2].

Reviewer comment (Evaluation):
I **guess** the theory is developed over "randomized algorithm $A$" (and Gibbs loss?), i.e., the output of $A(S)$ is random weights, which is distributed by a posterior. However, this paper only presents empirical results based on deterministic weights. How can these empirical findings provide support for the theoretical results?

Author response:
**Re**: Free AT is a widely used variant of adversarial training methods due to its higher speed in training the neural net classifier. While the free AT method has been extensively used in several applications, the existing theoretical analysis of adversarial training have not focused on the properties of free AT algorithm. In our work, we focus on the generalization analysis of models learned by the free AT approach, which, to the best of our knowledge, has not been studied in the literature. We show that due to the simultaneous updating nature of free AT, it has significantly better generalization performance compared to vanilla AT in theoretical analysis, and the numerical results are consistent with the theory. Please see our response to the first comment of Reviewer T7Bj for more information about our technical contributions.

Reviewer comment (Theory):
If $A(S)$ is a random variable, given $S$, what are the specific posterior distributions of $A_{AVanilla}(S)$ and $A_{Free}(S)$? What is the difference between these posterior distributions? Where does the randomness of $A_{AVanilla}(S)$ come from?

Author response:
**Re**: In our analysis, both $A_{Vanilla}$ and $A_{Free}$ are stochastic optimization algorithms and the training dataset is also random. So the randomness of $A$ could come from both random initialization and random batch selection.

Reviewer comment (Evaluation):
It seems there are not some interesting insights from the theoretical and empirical results in this work.

Author response:
**Re**: Free AT is a widely used variant of adversarial training methods due to its higher speed in training the neural net classifier. While the free AT method has been extensively used in several applications, the existing theoretical analysis of adversarial training have not focused on the properties of free AT algorithm. In our work, we focus on the generalization analysis of models learned by the free AT approach, which, to the best of our knowledge, has not been studied in the literature. We show that due to the simultaneous updating nature of free AT, it has significantly better generalization performance compared to vanilla AT in theoretical analysis, and the numerical results are consistent with the theory. Please see our response to the first comment of Reviewer T7Bj for more information about our technical contributions.

Reviewer comment (Theory):
This paper is dedicated to generalization performance analysis of existing adversarial training methods and reveals some interesting points. Nevertheless, there is a lack of deep insights on the new advanced designs of adversarial training from the generalization bounds. The authors should have discussed the insights/guidance from the theoretical findings, or discussed certain limitations of the algorithmic stability approach itself.

Author response:
**Re**: In the revised version, we have added a remark after Theorem 4 to discuss the implications of the theoretical bound on Free AT. We note that Theorem 4 suggests how the stepsize parameters for the max and min variables affect the generalization gap of free AT, where by reducing the number of maximization steps $m$ and step size $\alpha_\delta$ one can achieve a lower generalization error.

Reviewer comment (Evaluation):
From the experimental results, e.g., Figure 1, it appears that the reduced generalization error gap of free adversarial training is mainly due to the higher training error. Assuming the generalization error gap maintains, it is unclear if the test error can be further reduced when the training error is reduced. The authors should add some comments on this.

Author response:
**Re**: We agree with the reviewer that if we continue training the neural networks by applying free AT for more epochs, the training error might further decrease and the generalization gap might be larger. This phenomenon is essentially consistent with the stability analysis – our stability-based generalization bounds consider the number of iterations $T$ as a parameter and grow sublinearly with $T$. To make a fair comparison between free and vanilla AT from the stability perspective, we use the same number of training iterations in our experiments, and we observe a lower generalization gap and sometimes 2%-5% improvement on the robust test accuracy by free AT. Please note that the dependence of the generalization gap on the iteration number $T$ is consistent with the numerical findings of the references (e.g. the discussion in [1]).

Reviewer comment (Theory):
It would expect that new training/regularization methods could be proposed given the obtained generalization error bounds. Otherwise, the impact of the theoretical findings of this work is quite limited. A thorough discussion would be helpful and beneficial. It would be also interesting to know the potential connection between generalization gap and the robustness against adversarial attacks.

Author response:
**Re**: The simultaneous nature of min-max updates of free AT can be extended to other established robust training methods such as TRADES [2]. Based on this comment, we have performed numerical experiments on the application of a similarly defined ‘‘free’’ version of TRADES (which we call Free-TRADES). The results of our numerical experiments are presented in the revised Appendix B.6. The numerical results indicate that Free-TRADES can similarly gain a better generalization performance using the simultaneous min and max optimization steps. The theoretical analysis of the proposed Free–TRADES will be an interesting future direction to our work, which we have discussed in the revised conclusion section.