Fast Adversarial Training against Sparse Attacks Requires Loss Smoothing

Authors: Xuyang Zhong, Yixiao Huang, Chen Liu
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: adversarial robustness, sparse attack, fast adversarial training

Abstract:
This paper studies fast adversarial training against sparse adversarial perturbations. We highlight the challenges faced when employing $1$-step attacks on $l_0$ bounded perturbations for fast adversarial training, including degraded performance and the occurrence of catastrophic overfitting (CO). We highlight that CO in $l_0$ adversarial training is caused by sub-optimal perturbation locations of $1$-step attack, which is distinct from other cases. Theoretical and empirical analyses reveal that the loss landscape of $l_0$ adversarial training is more craggy compared to its $l_\infty$, $l_2$ and $l_1$ counterparts. Moreover, we corroborate that the craggy loss landscape can aggravate CO. To address these issues, we propose Fast-LS-$l_0$ that incorporates soft label and the trade-off loss function to smooth the adversarial loss landscape. Extensive experiments demonstrate our method can overcome the challenge of catastrophic overfitting, achieves state-of-the-art performance and narrows down the performance gap between $1$-step and multi-step adversarial training against sparse attacks.

Decision: reject
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1729218577660

thread_type:
official_review

<thread_object>
creator: reviewer_vsuq

<content>
summary:
This paper explores the challenges and solutions for catastrophic overfitting within 1-step adversarial training targeted at sparse adversarial perturbations bounded by the  $L_0$  norm. 
Through a comprehensive analysis, the authors illustrate that the CO issue in $L_0$ norm stems from a more craggy loss landscape compared to its $L_{\infty}$,  $L_2$, and  $L_1$ counterparts.
To address these challenges, the authors propose incorporating soft labels and a trade-off loss function, strategies specifically designed to smooth the adversarial loss landscape associated with $ L_0$  norm 1-step adversarial training.

strengths:
This work is the first to investigate fast adversarial training in the context of $L_0$ bounded perturbations.

The authors successfully demonstrate that the CO issue in the $L_0$ norm is caused by sub-optimal perturbation locations,  rather than sub-optimal perturbation magnitudes via some interesting ablation studies.

This study conducts extensive experiments, including ImageNet and Transformer-based architectures.

weaknesses:
Theoretical analysis indicates that large $\left\|\boldsymbol{\delta}_1-\boldsymbol{\delta}_2\right\|$ can intensify the gradient discontinuity, and  $L_0$ norm has the largest upper bound. 
However, directly comparing these upper bounds may not be a fair comparison due to the naturally larger freedom in change magnitudes associated with the $L_0$  norm. 
Could authors provide some empirical results of $\left\|\boldsymbol{\delta}_1-\boldsymbol{\delta}_2\right\|$ among different norms to support their claim?

It would be interesting to examine the performance of Expectation over Transformation (EOT) [1] and Backward Pass Differentiable Approximation (BPDA) [1] on the authors’ proposed method, which can effectively exclude gradient masking in 1-step adversarial training.

Recent advances in addressing CO by preventing loss landscape distortion with other norms [2,3,4,5] may also prove beneficial in alleviating CO in the $L_0$ norm and should be discussed.

[1] Athalye, A., Carlini, N., & Wagner, D. (2018, July). Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning (pp. 274-283). PMLR.\
[2] Grabinski, J., Jung, S., Keuper, J., & Keuper, M. (2022, October). Frequencylowcut pooling-plug and play against catastrophic overfitting. In European Conference on Computer Vision (pp. 36-57). Cham: Springer Nature Switzerland.\
[3] Lin, R., Yu, C., & Liu, T. (2024). Eliminating catastrophic overfitting via abnormal adversarial examples regularization. Advances in Neural Information Processing Systems, 36.\
[4] Rocamora, E. A., Liu, F., Chrysos, G., Olmos, P. M., & Cevher, V. Efficient local linearity regularization to overcome catastrophic overfitting. In The Twelfth International Conference on Learning Representations.\
[5] Lin, R., Yu, C., Han, B., Su, H., & Liu, T. Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependency. In Forty-first International Conference on Machine Learning.

questions:
Refer to weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730167435702

thread_type:
official_review

<thread_object>
creator: reviewer_wg55

<content>
summary:
This work studies the fast adversarial training, usually the single-step adversarial training (AT), in the setting of $\ell_0$-bounded sparse adversarial perturbation. It first identifies the phenomenon of catastrophic overfitting in $\ell_0$ fast AT and the accounts for it, sub-optimal perturbation locations and non-smoothness of the loss landscape. It then proposes to incorporate soft label and trade-off loss function into $\ell_0$ fast AT to smooth loss landscape to mitigate catastrophic overfitting.

strengths:
1. The presentation is easy to follow
2. The proposed method is well-motivated.
3. The experimental results verify the effectiveness of the proposed method in mitigating catastrophic overfitting in $\ell_0$ fast AT.

weaknesses:
1. The studied problem, catastrophic overfitting in $\ell_0$ fast AT, is quite narrow. It considers a very specific case, $\ell_0$, of the $\ell_p$ adversarial setting. The impact of this work's conclusions on the entire field, adversarial machine learning, is therefore limited.
2. The analytical framework and the findings are similar to the existing works of analyzing overfitting in AT. Some of them are already cited in this work, while some else are missing. For example, [1] also attributes overfitting in AT to the degradation of training adversary and the complication of loss landscape throughout training, and proposes to integrate loss landscape smooth techniques into AT to mitigate overfitting. However, I acknowledge that some conlcusion of this work, like the one about sub-optimal perturbation location, seems to be unique to the setting of $\ell_0$ and useful.  
3. The mitigation methods, soft label and trade-off loss function, are proposed by the exiting works, so the methodological contribution of this work is the discovery of the effectiveness of existing methods in mitigating catastrophic overfitting in $\ell_0$ fast AT, but not something novel like proposing a new method. However, as mentioned in Weakness 2, the idea of mitigating overfitting in AT through loss smoothing has been already explored in the existing works, despite that different smoothing techniques are employed.

[1] Li, Lin, and Michael Spratling. "Understanding and combating robust overfitting via input loss landscape analysis and regularization." Pattern Recognition 136 (2023): 109229.

questions:
Please find above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730561838868

thread_type:
official_review

<thread_object>
creator: reviewer_jqqa

<content>
summary:
This paper investigates FAT against sparse adversarial perturbations bounded by the $L_0$ norm, highlighting the challenges associated with FAT, including performance degradation and catastrophic overfitting phenomenon. The authors propose Fast-LS-$l_0$, which incorporates soft labels and a trade-off loss function to smooth the adversarial loss landscape. Experimental results show the effectiveness of the proposed method against sparse attacks.

strengths:
1.	Research on FAT under $L_0$ norm constraints is relatively limited, and the authors address this gap by proposing an innovative method to mitigate performance degradation and catastrophic overfitting in FAT.

2.	The authors provide detailed theoretical insights into the connection between the non-smooth adversarial loss landscape and catastrophic overfitting, giving clarity to the existing issues from the theoretical perspective.

3.	The paper is well-organized and clearly explained, and extensive experiments across diverse datasets validate the robustness of the proposed method.

weaknesses:
1.	The novelty is somewhat limited, as many techniques, including soft labels and TRADES loss, are widely used in adversarial defense. The overall method may appear as a straightforward integration of existing techniques.
2.	The application scenario is not clearly defined, particularly why robustness under the $L_0$ norm constraint is essential. Can this adversarial training effectively enhance defense against one-pixel attacks?
3.	The application scenario is unclear, and it is unclear why the model needs to be more robust under the $L_0$ norm constraint. Can this adversarial training enhance the defense capability under the one-pixel attack? 
4.	Whilemost current methods are based on $L_{\infty}$ norm constraints, the author should show the performance of these methods under $L_0$ constraints, such as FAST-GA[1], FAST-BAT[2], FAST-PCO[3].

[1]. Andriushchenko, M. et.al “Understanding and improving fast adversarial training” NIPS, 2020  
[2]. Zhang Y. et.al “Revisiting and advancing fast adversarial training through the lens of bi-level optimization” PMLR, 2022  
[3]. Wang Z. et.al “Preventing Catastrophic Overfitting in Fast Adversarial Training: A Bi-level Optimization Perspective” ECCV, 2024

questions:
1.	Can Fast-LS-$l_0$ be adapted for other sparse attacks, such as those bounded by $L_1$?
2.	Does the trade-off loss function parameter $\alpha$ need to be adjusted across different datasets and tasks?
3.	Could you further elaborate on the necessity of adversarial training under the $L_0$ constraint?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731912194981

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (1/2)

comment:
We would like to sincerely thank the reviewers and ACs for their engagement and constructive review. 

We are very pleased that the reviewers recognized our work as “an innovative method to mitigate performance degradation and catastrophic overfitting in FAT under $\ell_0$ norm” (Jqqa) and "well-motivated" (wG55). We are also encouraged to see that all three reviewers found our proposed method effective and extensively validated across diverse datasets. Additionally, we appreciate the acknowledgment of our theoretical analysis, described as “detailed and clear” (Jqqa), as well as the recognition of our ablation studies for demonstrating the uniqueness of catastrophic overfitting under the $\ell_0$ norm from other norms as “interesting” (vSUq) and "useful" (wG55).

The following are our responses to the most common comments from reviewers. Please refer to the point-to-point responses to each reviewer for further details.

**1. Novelty**

We would like to emphasize that the rationale and advantages of employing soft labels and the trade-off loss function differ significantly in the context of $l_0$ norms. **Specifically, the use of soft labels and trade-off loss functions is intended to improve generalization and to balance the robustness-utility trade-off in scenarios involving other norms such as $l_1$​, $l_2$, and $l_\infty$​. However,  in terms of fast adversarial training under $l_0$​ constraints, these techniques are crucial for achieving meaningful performance, particularly without suffering from catastrophic overfitting.** Without them, fast adversarial training against $l_0$​-bounded perturbations usually suffer from catastrophic overfitting, leading to only trivial performance outcomes (see Fig. 1). In contrast, fast adversarial training using other norms can still achieve competitive results through various techniques, such as random start [1], GradAlign [2], and ATTA [3]. These methods, however, are ineffective in the $l_0$​ scenario (see Table 6).

To substantiate our claims, we analyze the phenomenon of **catastrophic overfitting** in fast $l_0$​ adversarial training in Section 3, concluding that **it arises from suboptimal perturbation locations and a significantly more challenging loss landscape compared to other norms**. In Section 4, we propose that **soft labels and the trade-off loss function can provably enhance first- and second-order smoothness, respectively**, which are crucial to mitigate catastrophic overfitting. Numerical results support this assertion (see Fig. 6).

Motivated by these analyses and observations, in Section 5.1, we evaluate various combinations of methods that incorporate soft labels and the trade-off loss function. Our findings indicate that the integration of TRADES, SAT, and N-FGSM achieves the highest performance. **Notably, our method attains the best robust accuracy of 63.0%**, whereas other smoothing techniques, such as NuAT [4] and MART [5], report robust accuracies of only 51.9% and 48.0%, respectively (see Table 6).

In summary, this work is first to investigate the **unique** challenges and solutions in accelerated adversarial training against $l_0$ bounded perturbations, **it achieves the state-of-the-art performance based on the most comprehensive robustness evaluation so far**.

**2. Application Scenarios**

In contrast to $l_\infty$ or $l_2$ bounded perturbations, $l_0$ bounded perturbations are sparse and quite common in physical scenarios, including broken pixels in LED screens to fool object detection models and adversarial stickers on road signs to make an auto-driving system fail [6-10]. In this regard, **training a robust model against $l_0$ bounded perturbations is essential to realistic scenarios with high-security needs, such as auto-driving systems and access control systems.**

Although $l_0$ bounded perturbations have been studied in many existing works [6-10], most of them focus on generating adversarial perturbations instead of the defense against these attacks. Moreover, fast adversarial training against $l_0$ bounded perturbations is challenging due to the suboptimality of one-step attacks and the craggy loss landscape induced by $l_0$ bounded perturbations. **What’s worse, the existing fast adversarial training methods against $l_1$, $l_2$, $l_\infty$ attacks, like random start [1], GradAlign [2], and ATTA [3], cannot avoid catastrophic overfitting and thus are invalid in the $l_0$ scenario.** Therefore, it is necessary to develop an effective fast adversarial training approach against sparse attacks to fill the gap.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper addresses an important issue in fast adversarial training against sparse perturbations, providing a specific solution to mitigate catastrophic overfitting through loss smoothing techniques like soft labels and trade-off loss functions. However, the proposed approach lacks significant technical novelty, as it primarily builds on existing methods and combines them without introducing a fundamentally new framework. While the experimental results demonstrate effectiveness under sparse attacks, the paper’s scope is narrow, focusing exclusively on this specific setting without offering broader insights or applications to other adversarial scenarios. The theoretical claims about the challenging loss landscape are compelling but insufficiently supported by empirical evaluations, particularly under stronger adaptive attack scenarios such as BPDA or EoT. Furthermore, the real-world relevance of defending against sparse perturbations remains limited, and the comparison with baseline methods raises concerns about fairness and comprehensiveness despite subsequent hyperparameter tuning. Overall, the contribution does not meet the bar for acceptance due to its constrained impact and insufficient novelty.