Balancing Generalization and Robustness in Adversarial Training via Steering through Clean and Adversarial Gradient Directions

Authors: haoyu  tong
xiaoyu  zhang
yulin  jin
jian  lou
kai  wu
xiaofeng  chen
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 16:52:53

Abstract: Adversarial training (AT) is a fundamental method to enhance the robustness of Deep Neural Networks (DNNs) against adversarial examples. While AT achieves improved robustness on adversarial examples, it often leads to reduced accuracy on clean examples. Considerable effort has been devoted to handling the trade-off from the perspective of \textit{input space}. However, we demonstrate that the trade-off can also be illustrated from the perspective of the \textit{gradient space}. In this paper, we propose Adversarial Training with Adaptive Gradient Reconstruction (\textit{AGR}), a novel approach that balances generalization (accuracy on clean examples) and robustness (accuracy on adversarial examples) in adversarial training via steering through clean and adversarial gradient directions. We first introduce a ingenious technique named Gradient Orthogonal Projection in the case of negative correlation gradients to adjust the adversarial gradient direction to reduce the degradation of generalization. Then we present a gradient interpolation scheme in the case of positive correlation gradients for efficiently increasing the generalization without compromising the robustness of the final obtained. Rigorous theoretical analysis prove that our \textit{AGR} has lower generalization error upper bounds indicating its effectiveness.  Comprehensive experiments empirically demonstrate that \textit{AGR} achieves excellent capability of balancing generalization and robustness, and is compatible with various adversarial training methods to achieve superior performance. Our codes are available at: \url{https://github.com/RUIYUN-ML/AGR}.

Main paper: 
 introduction: 
 Deep neural networks (DNNs) have gained widespread adoption in myriad multimedia processing fields thanks to their exceptional performance, such as image recognition [16, 20] , text generation [1, 2] , speech recognition [34, 61] and object detection [19, 50] . However, due to widely recognized vulnerabilities caused by various attacks, the security concerns and privacy protection issues for AI-backed multimedia systems are increasingly aggravating [12, 31, 32, 60] . Currently, a wide range of real-world applications have been shown to be vulnerable to adversarial examples (AEs) [13, 18, 21, 40] , which adds tiny perturbations to clean examples that are imperceptible to human perception and give a false prediction. There exist considerable well-established methods for adversarial example generation such as FGSM [13] , PGD [35] , C&W [6] , AutoAttack [9] , etc. The introduction of adversarial examples has spurred the development of numerous techniques to ensure the trustworthiness of multimedia processing [14, 58, 59] , research on the adversarial attack defense mechanisms has been a trending topic in the multimedia field [15, 46] , among which adversarial training [35, 45, 48, 55] has been widely accepted as an effective defense.Although AT and its variants have notably improved model robustness, they inevitably compromise generalization performance compared to standard training methodologies and vice versa. Recently, there has been a host of research dedicated to the trade-off between generalization and robustness [30, 37-39, 42, 44] , such as leveraging the redundant capacity [63] , instance reweighting [56, 57] , adding a regularisation term for the loss function [41, 49] , or leveraging mixed data [62] . However, most of the aforementioned works utilize only the information brought by the input space to improve the model generalization or robustness, paying little attention to the information in the gradient space and even less attention to the dynamics of the gradient space, which is also very important in influencing the model performance. In order to thoroughly investigate the trade-off issue from the perspective of gradient space, we conducted an experiment with the results depicted in Figure 1 . We observe that there exist various levels of correlation between the gradient direction of the clean and adversarial robust loss during the adversarial training process, and even a negative correlation between their directions in some cases, which exactly explains that the adversarial training updating process leads to the gradual rise of the robustness and the degradation of the generalization. We perceive this finding as an indication of the existence of the tradeoff problem, which can be explained from the perspective of the difference in gradient directions between clean loss and adversarial robust loss. In light of this phenomenon of trade-off observed from the gradient direction, we ask the intuitive question: "Can we exploit these gradient directions to control or improve the trade-off in the training process dynamically?" As an answer to the question, we are tackling the trade-off problem from a novel perspective by dividing the clean and adversarial directions of the gradient into two cases during adversarial training.• In the negative correlation case, since the reduction in generalization comes from this situation, it is possible to adjust the direction of the adversarial gradient to remove that negative effect and maintain a positive correlation with the original gradient direction. • In the positive correlation case, correlation-based gradient interpolation can be employed to enhance generalization, paying high attention to increasing generalization and low attention to maintaining robustness in high correlation and vice versa.Based on the above inspiration, we propose a novel adversarial training method based on adjusting the direction of the gradients called Adversarial Training with Adaptive Gradient Reconstruction (AGR) that aims to improve the generalization without compromising the robustness of the final obtained. Our framework divides the parameter update process into two cases based on the cosine similarity between gradient pairs (see Figure 2 ). 1) Negative correlation case, we propose the Gradient Orthogonal Projection (GOP) to decompose the adversarial gradient orthogonality into two parts, one aligned with the negative direction of the clean gradient and the other orthogonal to it. This achieves the desired effect that the new gradient direction causes no degradation of generalization; 2) Positive correlation case, we propose an adaptive gradient interpolation scheme named Gradient Interpolation Based on Cosine Similarity (GICS), which makes use of the similarity of the gradients as the weights of the interpolation. With this interpolation scheme, the adversarial training process can dynamically pay more (or less) attention to robustness and less (or more) attention to generalization in the presence of low (or high) correlation, achieved by assigning interpolation weights corresponding to the gradient. This ensures efficient and dynamic improvement of generalization without compromising the robustness of the final obtained in adversarial training. The main contributions of our work can be summarized as follows:• We analyze the trade-off problem that exists during adversarial training from a novel perspective, i.e., gradient space, utilizing this perspective as a breakthrough point to further control or improve the trade-off.AGR to increase the generalization without compromising the robustness of the final obtained. Additionally, this approach is compatible with most of the existing adversarial training methods to achieve outstanding performance. • We theoretically prove that our AGR has lower generalization error bounds compared to other AT methods. We also conducted extensive experiments to evaluate the AGR against five state-of-the-art adversarial attacks, demonstrating superior performance in handling the trade-off. 
related work: 
 Adversarial Attacks. Adversarial examples [40] have become a prevalent attack method imposing tiny perturbations to the model inputs that drive the target model mispredicts outputs. There has been a considerable amount of literature on the methods of their generation since their discovery. One of the earliest and most wellknown methods is the Fast Gradient Sign Method (FGSM) [13] .Most subsequent adversarial attacks have been proposed based on this approach. Iterative FGSM (I-FGSM) [23] acts as an extended variant of the FGSM, which adds small perturbations by iteratively using the FGSM. PGD [35] is to randomly initialize a point on the input neighborhood as the starting point and then apply I-FGSM. Unlike FGSM [13] , C&W [6] is an optimization-based attack that transforms the attack problem into a minimization problem with constraints, instead of making use of the model's gradient information to generate perturbations. Adversarial Defense. In order to address and mitigate the potential threat of adversarial example attacks, manifold defense methods have been introduced and proposed. These include adversarial training [35, 45, 48, 55] , adversarial detection [7, 51] , certified adversarial robustness [8, 24, 26] and adversarial purification [3, 36, 52] . Among these techniques, adversarial training has emerged as a particularly effective defense method that involves training a model to improve its robustness by utilizing adversarial examples as training data.There are numerous approaches to adversarial training, each with its own unique strengths. Among them, PGD-AT [35] is the earliest and most widely accepted method, which improves model robustness by using maximization with PGD attack. TRADES [55] proposes a surrogate loss by analyzing the upper bound of the robust error, which pushes the decision boundary away from the examples. AWP [48] reveals a clear correlation between weight landscape and robust generalization gap. By smoothing the weight loss landscape, the generalization gap can be effectively reduced to improve robustness. CFA [47] investigates the effects of adversarial perturbations on different classes, and improves the robustness by customizing the perturbation configurations of different classes during adversarial training. The complete clean loss function in standard training is defined as:EQUATION)where 𝑛 is the number of the training data, 𝑓 (𝒘; •) is the model (neural network), ℓ (•) is the loss function (e.g., the cross-entropy (CE) loss).For adversarial training, we denote the adversarial (or robust) loss as:EQUATIONwhere 𝒙 ′ is considered an adversarial example that falls within the 𝜖-ball, bounded by 𝐿 𝑝 -norm and centered at the clean example 𝒙. 
overview: 
 Adversarial training has been demonstrated to be an effective technique for enhancing a model's robustness against adversarial examples. However, one potential drawback of this approach is that it may decline the model classification accuracy on clean examples. This is because the model may become overly focused on defending against adversarial attacks, which can negatively impact its ability to correctly classify clean examples.To address this issue, we propose a novel adversarial training approach with Adaptive Gradient Reconstruction (AGR). Specifically, AGR introduces orthogonal projection and gradient interpolation in adversarial training, which modifies the gradient of robust loss during the adversarial training process. As a result, AGR improves standard accuracy without compromising the robustness of the final obtained. Essentially, AGR allows the model to better balance its attention between defending against adversarial attacks and correctly classifying clean examples.We first explore the application of orthogonal projection methods in adversarial training. This exploration provides a metric for distinguishing between positive and negative correlation gradient pairs. For convenience, we denote the gradient of the latter concerning L 𝑠𝑡𝑑 (𝒘) and L 𝑎𝑑𝑣 (𝒘) as L 𝑛 and L 𝑎𝑑𝑣 , respectively. Definition 3.1. The cosine similarity between the gradients ∇L 𝑛 and ∇L 𝑎𝑑𝑣 is When the Ψ(•) < 0, it indicates that the gradient is in the opposite direction for standard and adversarial training. We refer to this gradient pair as a negative correlation gradient pair. Whereas, when Ψ(•) > 0, we call the gradient pair a positive correlation gradient pair. We also conduct a simple experiment in Figure 1 to show that there are many cases where the Ψ(•) < 0 during the model training process.Ψ(∇L 𝑛 , ∇L 𝑎𝑑𝑣 ) = ∇ L 𝑛 •∇ L 𝑎𝑑𝑣 ∥ ∇ L 𝑛 ∥ 2 ∥ ∇ L 𝑎𝑑𝑣 ∥ 2 .In the adversarial training process, updating the model in the direction of ∇L 𝑎𝑑𝑣 will significantly affect its predictive performance at point 𝒙 ′ , but will result in a smaller change to the prediction of 𝒙 ′ along the orthogonal to ∇L 𝑎𝑑𝑣 . We consider the effect of both directions ∇L 𝑎𝑑𝑣 and ∇L 𝑛 on the performance of the model simultaneously. Let us use cosine similarity as a measure between gradient pairs (i.e., Ψ(∇L 𝑛 , ∇L 𝑎𝑑𝑣 )), then we can divide the gradient pairs into two types, positive correlation gradient pairs or negative correlation gradient pairs, corresponding to Ψ(∇L 𝑛 , ∇L 𝑎𝑑𝑣 ) ≥ 0 or Ψ(∇L 𝑛 , ∇L 𝑎𝑑𝑣 ) < 0, respectively. Subsequently, we propose the gradient orthogonal projection and gradient interpolation methods to improve the generalization for the two cases (See Figure 2 ). To consider the orthogonalization process in more detail, we will orthogonalize the gradient for each parameter, we have:EQUATION𝑎𝑑𝑣 , ..., ∇LEQUATIONEQUATION𝑛 , ..., ∇LEQUATIONwhere 𝑐 is the number of the parameters.As depicted in Figure 1 , we demonstrate that there exists a few instances where the Ψ(∇L(𝑖 ) 𝑛 , ∇L (𝑖 )𝑎𝑑𝑣 ) < 0 during adversarial training. It suggests that this update moving will increase the robustness but decrease the generalization. To reduce the loss of generalization, we propose to use "orthogonalize" to refine the gradient ∇L (𝑖 ) 𝑎𝑑𝑣 . We now introduce the Gradient Orthogonal Projection (GOP) clearly and concisely, using a formulaic approach. We first orthogonally project ∇L as follows:𝒈 𝑖 1 = ⟨∇L (𝑖 ) 𝑎𝑑𝑣 , ∇L (𝑖 ) 𝑛 ⟩ ⟨∇L (𝑖 ) 𝑛 , ∇L (𝑖 ) 𝑛 ⟩ ∇L (𝑖 ) 𝑛 .(EQUATIONIn this scenario, the vectors of 𝒈 𝑛 as the moving direction, we denote 𝑃𝑟𝑜 𝑗 (•) as follows:EQUATIONIt is worth noting that we take 𝒈 𝑖 2 as the gradient of the i-th parameter update for this model does not cause much decrease in robustness as it is positively correlated with ∇L (𝑖 )𝑎𝑑𝑣 (i.e., Ψ(𝒈 𝑖 2 , ∇L 𝑎𝑑𝑣 ) > 0).In the case where Ψ(∇L 𝑎𝑑𝑣 ) > 0 is inappropriate, and we next consider a gradient interpolation method for this scenario.While there exists a series of works that have used the GOP to solve a variety of problems. For example, Bryniarski et al. [5] explores a new attack technique aimed at constructing adversarial examples that satisfy multiple constraints simultaneously through GOP. Farajtabar et al. [11] proposes to use it to solve the catastrophic forgetting in continual learning. Li et al. [27] introduces subspace learning to federated unlearning via the orthogonal projection. To the best of our knowledge, we are the first work to pursue the improvement of generalization through gradient orthogonal projection.Let's focus on improving generalization during training. After performing gradient orthogonal projection in adversarial training, the model achieves higher standard accuracy and consistent robustness compared to AT. However, it is worth noting that the increase in standard accuracy is often not very significant. Figure 1 shows that during the training process, only a small fraction of iterations have Ψ(•) values less than 0.Overall, there are still a large number of positive correlations in the direction of the gradient here, and we will make reasonable utilization of this gradient information to further handle the trade-off. Our main idea is to consider gradient interpolation based on cosine similarity between ∇L (𝑖 )𝑛 . Through interpolation, we can obtain a gradient that is very correlated with both gradient vectors, which can effectively balance the robustness and generalization. Thus, we propose the Gradient Interpolation Based on Cosine Similarity (GICS), the interpolation of the formula can be written as,EQUATIONwhere 𝑐𝑠 is the cosine similarity between ∇L (𝑖 )𝑎𝑑𝑣 and ∇L (𝑖 ) 𝑛 . The above interpolation method is able to dynamically handle the balance between generalization and robustness by using the cosine similarity between ∇L (𝑖 )𝑛 as interpolation weighting factors. When the cosine similarity between gradients is larger, due to the larger 𝑐𝑠 this interpolation method will consider ∇L (𝑖 ) 𝑛 more and efficiently gain generalization without loss of robustness. When the cosine similarity between gradients becomes small, the interpolation method will give priority to ∇L (𝑖 ) 𝑎𝑑𝑣 to prevent the decrease in robustness. Thus, it achieves the effect of improving generalization without loss of robustness. And when Ψ(∇L(𝑖 ) 𝑛 , ∇L (𝑖 )𝑎𝑑𝑣 ) is less than 0, we do not opt for using gradient interpolation. We reason that the gradient pairs are negatively correlated at this moment, and the interpolated gradient will retain a small similarity to the clean gradients or adversarial robust gradients. This similarity makes it challenging to improve the generalization and robustness of the model.No matter which interpolation method, it will lead to a certain difference between the new gradient direction and the original ∇L (𝑖 ) 𝑎𝑑𝑣 , which will cause the model to fail to obtain the robustness in time, so we choose to take GICS after the model obtains the maximum robustness (e.g., in the subsequent experiments we start using GICS at 150 epochs) and in order to finally maintain higher robustness, we choose to perform a gradient clipping on the interpolated gradient in 𝑙 2 norm, i.e., the 𝐶𝑙𝑖𝑝 operation is formulated as 𝑮/𝑚𝑎𝑥 (1, ∥𝑮 ∥ 2 𝐶 ), for a clipping threshold 𝐶. It is well documented that catastrophic overfitting of AT is related to the magnitude of the gradient paradigm, and reducing the magnitude of the gradient paradigm can effectively suppress catastrophic forgetting [29] . To better facilitate understanding, the pseudo-code for AGR is presented in Supplementary. 
theoretical analysis: 
 In this section, we provide a general result (Theorem 4.1) to demonstrate the effectiveness of AGR in improving generalization while maintaining robustness during each iteration. Then, we use Rademacher complexity to bound the generalization error of AGR. Now, we consider generalization in terms of two parts of AGR to obtain the following results: Theorem 4.1 shows that both in gradient orthogonal projection and cosine similarity interpolation operations, it is at least guaranteed that the direction of the gradient update causes no increase in clean loss and adversarial robust loss. Detailed notations and proof can be found in Supplementary. Next, we consider the following setting of Rademacher complexity.There has an unknown distribution D over the input space Z, from which we drawn 𝑁 examples i.i.d from D to form the standard training dataset 𝑆 = {𝑧 1 , 𝑧 2 , ..., 𝑧 𝑁 } where 𝑧 𝑖 = (𝑥 𝑖 , 𝑦 𝑖 ). Similarly, we denote the adversarial training dataset 𝑆 ′ drawn from the distribution T . We formulate the population risk and empirical risk as:EQUATIONEQUATIONwhere ℓ (•, •) represents the loss function.In our setting, AGR utilizes the standard training dataset and adversarial training dataset during adversarial training denoted as 𝑆 and 𝑆 ′ , respectively. Further, AGR maintains the value of L 𝑠𝑡𝑑 without decreasing during the optimization of L 𝑎𝑑𝑣 , thus we could view 𝑓 𝐴𝐺𝑅 as learned from 𝑆 and 𝑆 ′ , (10) where F represents the function space and 𝜆 denotes the proportion between the standard training dataset and the adversarial training dataset. Following this, we give the definition of empirical Rademacher complexity and derive the generalization error of AGR based on Rademacher complexity. Definition 4.2. Given a unknown distribution D and a function space F , let 𝑆 = {𝑧 𝑖 } 𝑁 𝑖=1 denotes the training dataset drawn i.i.d from D and {𝜎 𝑖 } 𝑁 𝑖=1 be the independent random variables set drawn uniformly from {-1, 1}. Then, the empirical Rademacher complexity of F on the set 𝑆 is defined to be: (13) In our setting, the ratio of clean to adversarial examples for AGR was kept at 1:1 (i.e., 𝜆 = 0.5 and 𝑁 𝑆 = 𝑁 𝑎 ). For conventional adversarial training methods, which usually utilize only the adversarial training dataset, its generalization error can be viewed as Eq. 13, while the error of AGR is shown in Eq. 12. It is worth noting that the TRADES method also utilizes the clean examples, and in subsequent experiments, we point out that AGR with TRADES as a special case of TRADES is dynamically adjusting the hyperparameters of the loss function of TRADES. Overall, the generalization error of 𝑓 𝐴𝐺𝑅 and 𝑓 𝑆𝐴 are bounded by the empirical training risk, distributed error, and estimation error. The empirical training risk can be optimized to be infinitely small. The distribution error of 𝑓 𝑆𝐴 is two times that of 𝑓 𝐴𝐺𝑅 . The remaining term in the inequality with respect to 𝑁 𝑎 and 𝑁 𝑠 is denoted the estimation error, which tends to 0 when the training dataset size is infinitely large. Therefore, our AGR can achieve a much smaller generalization error. The proof of Theorem 4.3 can be found in Supplementary.𝑓 𝐴𝐺𝑅 = arg min 𝑓 ∈ F 𝑅 𝑆+𝑆 ′ (𝑓 ) = arg min 𝑓 ∈ F 𝜆𝑅 𝑆 (𝑓 ) + (1 -𝜆)𝑅 𝑆 ′ (𝑓 ),EQUATION 
experiments: 
 In this section, to verify the effectiveness, efficiency, and feasibility of our proposed AGR, we conduct extensive comparative experiments. Firstly, we show that our approach combined with currently available adversarial training methods (i.e., AT [35] , TRADES Table 1 : The standard accuracy (SA) and robust accuracy of PreAct-ResNet-18 and WideResNet-34 trained on CIFAR10, CIFAR100, and Tiny Imagenet datasets under 𝑙 ∞ = 8/255 against white-box attacks across different defense mechanisms, i.e., AT [35] , TRADES [55] , MART [45] , and AGR (%). (𝜆 = 1/6) [55] , MART [45] , AWP [48] , Avmixup [25] can significantly improve standard accuracy while ensuring robustness against various attacks. Furthermore, we conducted many ablation studies to confirm the effectiveness of individual components of our methodology. We use the standard accuracy and robustness accuracy as metrics to measure model performance.Datasets. For all experiments, we evaluate the standard accuracy and robustness of the proposed AGR on CIFAR10 [22] , CIFAR-100 [22] , and Tiny Imagenet [10] , which are three well-known datasets of natural images used in computer vision research. These datasets were divided into two parts, the training set, and the validation set, with a ratio of 5:1. For the data augmentations, we apply 32 × 32 random crops with 4-pixel zero padding, random horizontal flip, and cutout.Training. For CIFAR10/100 and Tiny Imagenet, we utilize PreAct-ResNet-18 [17] and WideResNet-34 [53] architecture as the primary model for evaluation. For the setting of hyperparameters for adversarial training, we train the PreAct-ResNet-18 and WideResNet-34 for 200 epochs by SGD with momentum 0.9, and weight decay of 5×10 -4 . The learning rate is initially 0. step size to 2/255, and applied a limit of 𝑙 ∞ constraint over 10 steps. More detailed settings can be found in the Supplementary. Attacks. For white-box attacks, we consider the four typical attacks below: FGSM [13] , PGD-20 [35] , PGD-100 [35] , and 𝐶𝑊 ∞ [6] . For black-box attacks, we choose SPSA attack [43] and AutoAttack [9] , which contains a black box attack called square attack [4] and three white-box attacks. To ensure consistency in the experimental results, the mean of three experiment repetitions was employed for all results. Impact on standard accuracy. Table 1 shows that our proposed AGR method, when combined with diverse adversarial training methods, can significantly enhance the standard accuracy of the model on CIFAR10, CIFAR100, and Tiny Imagenet. Specifically, when applied in conjunction with adaptive gradients reconstruction, AGR achieves an impressive standard accuracy of 86.22% on CIFAR10 with PreAct-ResNet-18. This is a significant improvement compared to the baseline AT, which only manages to achieve 84.86% accuracy on clean images, resulting in a gap of 1.36%. Despite the gap in standard accuracy, the robustness accuracy of both methods is quite comparable, with only a marginal difference of 0.49%. The identical experimental results can also be observed on the Tiny Imagenet dataset. Robustness against white-box attacks. To verify the reliability of our approach, we conducted a thorough evaluation of its robustness against a range of white-box attacks. Specifically, we considered various attacks with the same norm constraints (e.g., 𝑙 ∞ = 8/255). The results, as presented in Table 1 , demonstrate that our proposed method, which incorporates adaptive gradient reconstruction, consistently maintains exceptional robustness across all evaluated attacks. Even in the worst-case scenario, where the model was trained with TRADES on CIFAR100, there was only a 0.45% reduction in robustness. Over the Tiny Imagenet dataset, we also demonstrate the effectiveness of our method, which performs well in improving generalization. In addition to this, we further consider other adversarial training methods in conjunction with AGR, such as AWP [48] , Avmixup [25] . The results are shown in Figure 3 , where the proposed method is effective in improving generalization while maintaining robustness.To demonstrate the robustness of our method against different white-box adversarial attacks, we evaluate our method against FGSM, PGD-100, and C&W. As shown in Table 1 , our method is efficient in maintaining robustness against diverse adversarial attacks. We can observe that the combination of our AGR method also maintains reasonable robustness against unseen perturbations. Robustness against black-box attacks. We also conducted two black-box attack tests on our approach, i.e., query-based attack SPSA and AutoAttack, assessing robustness by a black-box attack (Square Attack) and three white-box attacks. Table 1 and Figure 3 illustrate our approach's effectiveness in defending against blackbox attacks. Visualization results. To visualize that our method improves generalization, we show the decision boundary of AGR in Figure 4 and the weight loss landscape of the model trained with vanilla AT and AGR in Figure 5 . For decision boundary, we randomly select an image and generate 1K random directions, applying continuously perturbations with a fixed step size (𝛼 = 1/255) along each direction until the model's prediction for the image changes. We record the number of steps required to change the prediction result of the example in 1K directions in Figure 4 in ascending order. The decision boundary of our method is further away from the example than the baseline. Weight loss landscape is a commonly used measure to describe the generalization gap in standard training [28, 33] . We can observe that our approach has a much flatter loss compared to 6 , we show that our method achieves a remarkably smooth loss trained by vanilla AT and TRADES on CIFAR-10/100 datasets. The loss change result on Tiny Imagenet can be found in Supplementary. Robustness against unseen attacks. We have conducted evaluations for other threat models. Table 2 reports the adversarial robustness using PRN-18 under 𝑙 2 and 𝑙 1 threat models, which indicates that the proposed method is also effective for other threat models.We aim to provide a thorough analysis of the proposed AGR method by conducting substantial ablation studies on each individual component. Our primary objective is to evaluate the effectiveness of the Gradient Orthogonal Projection (GOP) and Gradient Interpolation Based on Cosine Similarity (GICS) of AGR for standard accuracy enhancement, respectively. Evaluation on gradient orthogonal projection. To evaluate GOP's ability to improve standard accuracy, we tested models trained with and without GOP, comparing it to vanilla AT, TRADES, and MART under consistent experimental settings. The results, as shown in Table 3 , indicate that GOP significantly enhances standard accuracy in all adversarial training. Notably, TRADES and MART with GOP show greater improvement than vanilla AT, likely due to the frequency projection during training. In AT-GOP, GOP frequency is 2.03%, while in TRADES and MART, it reaches 14.93% and 6.09%, respectively. Evaluation on gradient interpolation. We conducted a comparative experiment with three gradient interpolation methods: average gradients, fixed scale interpolation, and our GICS (Section 3.4). Table 4 shows that our method significantly improves standard accuracy, but in some cases, it results in decreased robustness compared to Case 2.In particular, we point out that the interpolation method in TRADES is actually changing the hyperparameters 𝜆 of the loss function of TRADES, whereas our method (i.e., Case 3) dynamically adjusts 𝜆 so that when robustness is lost too much, interpolation yields a larger 𝜆 to ensure that robustness is not lost. In this way, we can get a better balance between standard accuracy and robustness accuracy compared to fixed interpolation weights. The results of Case 2 with different 𝜏 values are shown in Figure 7 . As 𝜏 increases, robust accuracy increases while standard accuracy decreases, indicating a static trade-off where the gradient focuses more on robustness over generalization. In Case 3, this trade-off is dynamically managed by adjusting the interpolation weight based on the gradient directions of clean and adversarial losses. As shown by the dotted line in Figure 7 , Case 3 achieves a better balance between robustness and generalization compared to Case 2, often providing higher generalization with comparable robustness. Evaluation on time and memory consumption. Here we demonstrated the time and GPU memory consumption of our proposed method compared to the vanilla AT method in Figure 8 . Time loss and memory usage are still increased compared to the original method. 
conclusion: 
 In this paper, we perform a comprehensive study on how to balance generalization and robustness in adversarial training. From a novel perspective, we proposed Adversarial Training with Adaptive Gradients Reconstruction (AGR) to implement the gradient information of clean and adversarial examples to dynamically handle the trade-off between generalization and robustness in order to improve standard accuracy. Extensive experiments show that our method has an excellent performance in improving generalization while maintaining robustness. 
 
 Appendix:  


Decision: The paper received three positive reviews, but reviewers mainly focus on its relevance to the multimedia field. The authors provided a detailed explanation of this issue in their rebuttal. Therefore, I am inclined to accept the paper. However, upon acceptance, the authors should emphasize the paper’s relevance to the multimedia field.

Reviews: review: paper_summary: paper_summary: To trade-off between clean and robust accuracy from the gradient space, this paper proposes Adversarial Training with Adaptive Gradient Reconstruction (AGR). This method optimizes adversarial gradient direction and interpolation scheme to increase generalization without compromising robustness. Besides, theoretical analysis and empirical experiments demonstrate that AGR effectively balances generalization and robustness, achieving better performance.
strength_weakness: strength_weakness: For presentation, the paper is well organized and interesting. The proposed method is easy to follow, which have visual description. For theoretical contribution, the author proposed and analyzed a theorem on the generalization properties of AGD. Third, this paper proposes a method from the perspective of gradients that can enhance the clean and robust accuracy of models, which may inspire progress in other related works. A detailed evaluations are conducted; the results show the proposed method enhances the performance of existing adversarial training as a plug-in.
limitations: limitations: 1.	Some definitions are not consistent, such as “example” and “sample,” which leads confusion during reading. 
2.	The improvement in robust accuracy is limited, particularly when against attack in larger datasets and AutoAttack. 
3.	Considering that the proposed method requires computing gradients with respect to both clean and adversarial examples, it may require increased computational costs. Therefore, it is necessary to conduct comparative experiments or analyses regarding training time to determine the competitiveness in efficiency. Additionally, the authors can also provide constructive suggestions to relieve this issue.
4.	I recommend authors include pseudocode descriptions of the algorithm and link them to the corresponding formulas to assist readers in better understanding.
paper_summary: paper_summary: This paper presents a comprehensive study on how to balance generalization and robustness in adversarial training. They first propose adversarial training with adaptive gradient reconstruction (AGR), which utilizes the gradient information of both clean and adversarial examples to dynamically handle the trade-off between generalization and robustness, thus improving the criterion accuracy. Since I am not very familiar with the field, I will be adjusting my scores based on the ratings of other reviewers.
strength_weakness: strength_weakness: 1.	The article proposes a very new perspective to improve the generalization ability and performance of the model through gradient orthogonal projection. I think it is very innovative.
2.	The experimental part of the article is very detailed and argues very favorably for the validity of the model.
limitations: limitations: 1.	The article suggests that the performance of testing clean data is degraded under robust modeling, and I would like to know what kind of data the authors hope to be able to achieve excellent performance with, and how it can be demonstrated in the experiments?
2.	What is the strong connection between articles and multimedia?
3.	What was the original purpose of introducing the gradient space to improve the model's ability to generalize to adversarial data? How does the experimental component demonstrate the model's ability to generalize?
paper_summary: paper_summary: The authors introduce AGR, a novel method aimed at balancing generalization and robustness in adversarial training, and evaluate its effectiveness across various benchmark methods and datasets.
strength_weakness: strength_weakness: 1. The perspective is novel and interesting. The author explores the gradient space between standard and adversarial examples and uses GOP and GICS to adjust negative and positive correlations.

2. The proposed methodology stands out as technically sound and effective both theoretically and empirically.

3. The experimental results convincingly validate the approach.

4. The paper is overall well-written and easy to follow.
limitations: limitations: While the current version of the paper is satisfactory, addressing the following points would still be meaningful.

1. Figure 1 illustrates that the ratio of positive to negative gradients varies by layer. Conducting a layer-dependent ablation study could yield insightful findings about how different layers contribute to the effectiveness of AGR.

2. The results in Figure 3 indicate that incorporating AGR results in diminished robustness with AWP. Could the author provide an intuitive explanation for this phenomenon?

3. It would be beneficial to test on adaptive attacks, such as BPDA and Lagrange. 

4. This paper does not involve any multimedia or multimodal elements.