Nasty Adversarial Training:  A Probability Sparsity Perspective for Robustness Enhancement
Authors: Yuhang Zhou
Zhongyun Hua
Zhaoquan Gu
Keke Tang
Rushi Lan
Yushu Zhang
Qing Liao
Leo Yu Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: adversarial training
adversarial robustness
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: The vulnerability of deep neural networks to adversarial examples poses significant challenges to their reliable deployment. Among existing empirical defenses, adversarial training and robust distillation have proven the most effective. In this paper, we identify a property originally associated with model intellectual property, i.e., probability sparsity induced by nasty training, and demonstrate that it can also provide interpretable improvements to adversarial robustness. 
We begin by analyzing how nasty training induces sparse probability distributions and qualitatively explore the spatial metric preferences this sparsity introduces to the model. Building on these insights, we propose a simple yet effective adversarial training method, nasty adversarial training (NAT), which incorporates probability sparsity as a regularization mechanism to boost adversarial robustness. Both theoretical analysis and experimental results validate the effectiveness of NAT, highlighting its potential to enhance the adversarial robustness of deep neural networks in an interpretable manner.

Published as a conference paper at ICLR 2026
NASTY ADVERSARIAL TRAINING: A PROBABILITY SPARSITY PERSPECTIVE FOR ROBUSTNESS ENHANCE-
MENT
Yuhang Zhou
Zhongyun Hua
School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China (238951015) @stu.. hit. .ed..ddu.... en
School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China Chonguungy @hit edu.edu.co cn
Zhaoquan Gu
Keke Tang
School Computer Science and Technology Harbin Institute of Technology, Shenzhen
Cyberspace Institute Advanced Technology Guangzhou University
Peng Cheng Laboratory
Guangzhou, China
Shenzhen, China
(tangbohutib) e gma com
{guzhaoquan) @hit edw..e.duuccom en
Rushi Lan
Yushu Zhang
School of Computer Science and Information Security Guilin University of Electronic Technology
School of Computing and Artificial Intelligence Jiangxi University of Finance and Economics Nanchang, China
Guilin, China
(alaan2016@@163..0
Lyushuu(nuace edu.
Qing Liao
Leo Yu Zhang School of Information and Communication Technology Griffith University
School Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China (liaoqing) Chit. edu. en
Brishane, Australia [leo 2hang) edu au
ABSTRACT
The vulnerability of deep neural networks to adversarial examples poses signifi- cant challenges their reliable deployment. Among empirical defenses, adver- sarial training and robust distillation remain the most effective. In this paper, we identify a property originally studied in the context of model intellectual prop- erty protection, i.e., probability sparsity induced by nasty training, and reveal its potential enhance adversarial robustness in an interpretable manner. We first analyze how nasty training drives models toward sparse probability distributions and qualitatively explore the spatial metric preferences introduced by such spar- sity. Building on these insights, we propose nasty adversarial training (NAT), simple yet effective adversarial training framework that incorporates probability sparsity regularization mechanism to strengthen robustness. Both theoretical analysis and extensive experiments demonstrate the effectiveness of NAT, showing that probability sparsity not only improves adversarial resilience but also provides interpretability to the robustness gains.
INTRODUCTION
Despite their success in a wide range of computer vision tasks, deep neural networks (DNNs) re- main highly vulnerable to adversarial attacks, raising serious challenges to their reliability and cure deployment. By introducing carefully crafted perturbations to input, adversaries can easily mislead models into producing incorrect predictions. As these perturbations often imperceptible

---

Published as a conference paper at ICLR 2026
or barely noticeable to the human eye, such vulnerabilities pose significant risks in safety- critical applications such as autonomous driving, where manipulated traffic signs may cause catastrophic accidents (Eykholt al. 2018).
Adversarial defense aims to preserve robustness under such attacks. Although early defense meth- ods (Xie al., 2019a Liao al. 2018 Papernot et al. 2016 Xie et al. 2017a] reported promising robustness, they were later shown to rely on "obfuscated gradients" (Athalye et al. 2018) and proved
ineffective against adaptive adversaries. Currently, the most reliable empirical defenses are adver-
sarial training (AT) (Madry et al., 2017; Jia et al., 2022a Zhou & Hua. 2024 Zhang et al. 2202aa]
Yu-Hang et 2025) and robustness distillation (RD) (Zhu et al. 2021 Zi et al., 2021] Goldblum
et [020]] Zhou et al.). AT improves robustness by dynamically generating adversarial exam-
ples during training, while RD extends this paradigm with improved inference efficiency, making suitable for deployment on resources- constree-conttrainee platforms such as edge devices.
Although AT has many variants, few studies have explored robustness attribution from the perspec- tive of the model's output probability distribution. Nasty training (NT) (Ma et al., 2021]. which regularizes models using a nasty adversary, provides an intriguing perspective by suggesting that
probability sparsity may also contribute to adversarial robustness. The original goal of NT is to train a teacher model resistant to imitation by a student model. Subsequent work by Ma et al. (2022) theoretically linked this effect to probability sparsity, showing that sparse probability distributions can hinder distillation. However, NT does not explicitly address adversarial robustness and largely overlooks the potential impact of probability sparsity in this context. We argue that probability sparsity reflects greater inter-class separation in the output logits, offering a spatial-metric interpre- tation of robustness. Employing this insight could unlock additional potential in adversarial training. Additional detailed related work is provided in the appendix.
In this paper, we first investigate the under- lying cause of sparse probability outputs in NT, attributing to high-order power optimization. We then qualitatively ana- lyze the spatial metric properties induced by these sparse probability distributions, including enhanced class separability and larger attack tolerance in the classification layer. Motivated on these insights, we propose nasty adversarial training (NAT), a simple yet efficient adversarial defense that utilizes probability sparsity as reg- ularization mechanism to enhance adver- sarial robustness. NAT introduces an aux- iliary adversary model that applies nasty regularization alongside the primary train- ing objective. The target model achieves probability sparsity by maximizing the di- vergence between its outputs and those the adversary model, while simulta- neously learning discriminative ability by minimizing divergence with the ground truth. Experimental experiments demonstrate that NAT achieves otate-of-the-ar adversarial robust- ness with minimal overhead. Our main contributions are summarized as follows:
Target Model
Target Logits
Macerioe Stmsary
SinglePakk
Cleon Dom
Mirieeiee
Backbone
Advemaniia
Ground Trush
Data
Advenarry
Advemsnial
Counterpart
Logits
Mulsi Peak Dioza
Figure 1: Framework of the proposed NAT. NAT en- hances the target model's training by introducing an auxiliary adversary model in addition to the primary training loss. Probability sparsity regularization (i.e.. probabilistic rectification) is achieved by maximizing the divergence between the output distributions of the target model and the adversary model.
(1) We analyze the attribution of probability sparsity in NT and reveal its spatial metric benefits, providing interpretable insights into adversarial robustness.
(2) We propose NAT, novel adversarial defense that integrates NT an effective AT regularizer.
(3) We empirically validate that NAT achieves state-of-ther tort----tttt-ttttee adversarial robustness with low com- putational overhead. Ablation studies and discussions further highlight its effectiveness.
2

---

Published as a conference paper at ICLR 2026
2 METHODOLOGY
PRELIMINARY
Nasty Training (NT). The original objective NT is to train a target teacher model fo, that resists distillation by student model fe.. where 8g and 0g represent their respective model parameters. In standard knowledge distillation, the student model is trained to approximate the outputs of the teacher model. This process can be formalized as:
min (x
Σ
KT. (ov, (pior (ai)) ,ro (pios (x:)))
(I)
(1 - a)XE (psig (oi)) .vi).
where KL(-) and (E(-) denote the KL-divergence and cross-entropy loss, respectively. The hyper- parameter a balances the trade-off between distillation and classification objectives. or, (.) denotes the "softmax temperature" function (Ma et al. 2021,, which reduces the standard softmax o-) when T, = 1. Here, тN and Ta are softmax temperatures, X' is the target distribution, and represents the output probabilities.
In NT, to prevent the teacher model from being distilled by the student model, an auxiliary adver- sary model fo., which is vanilla-trained counterpart of the target model in the original setting, introduced. This adversary model enables the target teacher model to learn task-relevant knowl- edge and maximize its output discrepancy from the adversary model. This training objective can be formalized as:
XE (psa, (4x)) ...)
mii (rii)))
(2)
-.KL (ot. (pso, (ei)) ora (plo, (a:)))
where wa is itively, the adversary model fo, acts surrogate for the student model fe.- Instead of allowing the student to distill knowledge by aligning with the teacher, NT reduces the similarity between the teacher and the surrogate adversary's outputs. Subsequently, Ma al. (2022) further provided theoretical analysis, showing that the sparsity of the teacher's output probabilities the key factor that prevents successful distillation.
that balances the target cross-entropy loss and the nasty loss. Intu-
The objective of adversarial attacks is to identify an optimal perturbation 6 that causes well-trained target model fe, make incorrect predictions the perturbed input, thereby maximizing the model's loss on the true label. This can be formalized as:
Adversarial Attack.
arg max C(o,, (x 62). s.t. ||62|1p < e,
(3)
where E denotes the perturbation budget, represents the target loss, typically the cross-entropy loss.
Adversarial training. AT, adversarial examples dynamically generated during training process and incorporated into the learning process. Through a min-max optimization framework, the model learns gain adversarial robustness against internal maximization (i.e., adversarial attacks). This process can be formalized as:
min B
max C(x + dz, y: |1621l1
(4)
2.2 NASTY ADVERSARIAL TRAINING
We propose NAT, which incorporates probabilistic sparsity into AT through nasty regularization. Intuitively, NAT extends standard AT by introducing an auxiliary adversary model for the target model, with both models jointly optimized under adversarial training. The overall training objective
3

---

Published as a conference paper at ICLR 2026
is defined as:
Σ XE (o (zi)),
min 0c
WoKL (Tra (pre, (zi)) or (psea (ai)))
S.L. X aa arg arg max XE (a (psa, (a:)) v.) Vx V
where X' denotes the set of adversarial examples corresponding to the original input distribution. Following the original NT setup, we use a vanilla-trained counterpart the target teacher as the adversary model. This configuration allows the target model to learn sparse output probability dis- tributions while simultaneously improving adversarial robustness. In the next section, we analyze the origin of sparsity in NT and qualitatively explain how probabilistic sparsity contributes to ro- bustness by shaping spatial relationships the output space.
3 IN-DEPTH ANALYSIS
In this section, present two key analyses to explain how NAT induces probabilistic sparsity and strengthens robustness: (1) the origin of probabilistic sparsity in NT, and (2) its influence on the model robustness from the perspective of spatial structure.
3.1 ATTRIBUTION OF PROBABILITY SPARSITY
primarily focused on the effect of probability sparsity on knowledge distillation, but did not investigate why NT induces such sparsity. In this paper, we further explore the origin of probability sparsity in NT by performing Taylor expansion of the adversary regular- ization term. Specificallyy, beyond the primary classification objective (i.e., cross-entropy loss), we reformulate the adversary regularization term in NT follows:
Previous work (Ma et al. 2022
L
Σ
Σ
log
P.
(6)
Uiee log (g(,e)
4hio log (4t
N
where N denotes the number of samples and C' the number of categories. gf and of are the output logits of the target model and adversary model for the ich example and oth category, respectively. Our analysis focuses the second term, since the first term is independent of the adversary model and therefore does not capture its effect. Let A Lh - kie- we expand log(ff.) around g. using a Taylor series:
1
log (ff..) log (gf.e)
A
(A9 (A),))
(vf.e)
(-1)(K+1)
(A).)
K(at.)
Following the assumption of Ma et al. (0022) we let dh df. + A where A denotes the gap between of and df.- Each of of Di and Ag can be regarded functions of the input Ti i.c., i.c. of (xi,c) (at.c) and A (ar..) Although (f, ) is abstract and difficult to express explicitly, it induces an implicit functional dependence between the cross-entropy loss and of since both of and of functions of the input Ti.e- As result, the cross-entropy can be rewritten as T( ofe A A.). enabling Taylor expansion. Based this formulation, the regularization term
4

---

Published as a conference paper at ICLR 2026
can be approximated as:
Naaty
Σ
N C
Σ
(96
4i,c)
Σ
N C
Σ
(q
)
N
2N
1
c-1
1-1c cl
1
N
(gf.c
(8)
3.V
(g),c)
N
(g gh.0)
KN
(2% )1,
The Taylor expansion of the nasty loss provides an intuitive explanation of the mechanism underly- ing NAT. Beyond the primary cross-entropy loss, NT introduces higher-order regularization term. The first-order term, Σ -e- ( - -(,c), can be ignored since the probability outputs of both (1
encour- ages the target model to maximize its output discrepancy with the adversary, especially on non-target classes where the target assigns lower probabilities. Since these smaller probabilities appear in the denominator, they induce larger regularization weights, thereby amplifying the effect of the loss on non-target classes. Although higher-order odd terms may introduce effects opposite to the desired behavior, they are typically suppressed by the preceding even terms, whose coefficients dominate.
models sum to a constant value of 1. The second-order term, IN 2
91.c
In standard settings, the adversary model adopts the same architecture the target model and typ- ically produces a "single peak + uniform distribution" " prediction pattern (Ma et al.] (022). This behavior arises from the use of one-hot labels in the cross-entropy loss, where the ground truth dis- tribution is characterized by "single peak + uniform zero". The training behavior induced by NT can thus be summarized follows: the cross-entropy loss drives the model to concentrate probabil- ity mass on the target class, while the nasty regularization term discourages it from replicating the adversary's 's uniform distribution across non-target classes. As a result, the target model redistributes the probability mass previously spread uniformly across all non-target classes onto small subset of classes that are closer to the target (i.e., more compressible), ultimately leading to sparse output probability distribution.
Our later experiments further show that the non-target class probabilities tend to be compressed onto semantically related classes (e.g., cat and dog), indicating that the model captures more generaliz- able semantics. practice, even if the adversary's output deviates from the "single peak + uniform distribution" assumption, the differentiation--riiee interaction between the adversary model and the target model still enforces sparse probability distribution in the target model. However, the result- ing peaks of the target model may not align with the adversary's multi-peak outputs, reflecting different allocation of probability mass.
3.2 BENEFITS ON SPATIAL METRIC RELATIONSHIPS
The sparsity of output probabilities and the spatial metric behavior show obvious logical connection. Specifically, probability sparsity indicates that the model strongly favors the target class by assigning it substantially larger logits, while assigning much smaller logits non-target classes. Let i and denote the target class and a non-target class, respectively, with (w, b.) and (wy, b,) representing the weights and bias of their corresponding linear classifiers. For a model exhibiting probability sparsity, we have:
Wa + 6; Wir by.
Here, >> rather than > is used emphasize the existence sufficiently large margin in a nasty- trained model, due to the saturation regions of activation functions such as Sigmoid and Sofimax. This property relates to two key spatial metrics: (1) the distance from data points to the decision boundary, and (2) the minimum distance between classification boundaries. Both are measured by the magnitude of the output logits. Therefore, Equation 9 jiplics that robust models exhibit larger margins to the decision boundary and greater separation between hyperplanes, thereby achieving improved adversarial robustness. For clarity, we formalize the computation of these two distances in high- dimennsoinn space.
Distance from Data Points to the Decision Boundary. To compute the distance from a data point to the decision boundary, model the linear classification layer as hyperplane in high-dimensional
5

---

Published as a conference paper at ICLR 2026
space. The distance from data point H; to the decision boundary of class is then given by the standard seintto-hyperplane distance formula:
[e + I; + bel
Ddata to bound
(10)
[00e 2
Analysis. Intuitively, larger logits indicate greater distances from the decision boundary. Although this distance depends on the norm of the classification layer's weights (1wwel12) weight magnitudes are typically constrained by L2 regularization and thus vary within a limited range. Compared to the substantial shifts in logits caused by the saturation behavior of the Softmax function, variations in the model parameters of much smaller magnitude. As result, greater probability sparsity yields substantially larger distances between data points and their corresponding classification boundaries. In the experiment section, empirically validate this hypothesis by quantifying the average dis- tance of all samples within each class to the ten decision boundaries.
Minimum Distance Between Classification Boundaries. Furthermore, for the same data point Fi the larger logit gap observed in the nasty-trained model can be attributed to greater discrepancy between classification parameters, defined as:
weiiheeJ2 = ||0, ww/l2-
(11)
This increased weight discrepancy leads to larger inter-class distances in the classification layer. Specifically, the linear layer of deep model can be regarded as set of high-dimensional vectors. Although the exact distances between such vectors cannot be directly computed,the shortest dis- tance between the corresponding class decision boundaries can be approximated using projection geometry. Specifically, the classification weight vectors for different categories can be treated as non-intersecting lines, and their shortest distance can be calculated as follows:
(1) Calculate the difference between the weight vectors: 7 W/ Wi--
(2) Normalize the weight vectors to obtain unit directions: d; = The
wig
d
Twy
(3) Estimate the shortest distance between the classification directions of classes i and as:
Diu shortest = || (7 d:) dd|12,
(12)
where y x d is the scalar projection of y onto the direction d,, and (7 - di) de is the corresponding projection vector.
Thus, the shortest distance between two classifier boundaries (e.g., and j) can expressed as:
Diuj shortest
(20;
Wi
d:
(13)
Analysis. Intuitively, larger inter-class weight gaps yield greater shortest distances between classi- fication boundaries, making it more difficult for data points to shift between classes. As result, adversaries must apply larger perturbations (i.e., higher attack budget) to induce a misclassifica- tion. To empirically validate this theoretical insight, we will later conduct qualitative analysis in the experiments section, examining the relationship between inter-class weight differences and the distances between different classification boundaries.
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Attacks Details. Following common defense evaluation settings (Zhang et al. 2024b Yue et al., 2024 Yin et al.) 2024: Li et al. 2024 Zi et al. 2021 Zhou & Hual 2024| Zhou et al.,, we evaluate NAT against several widely used adversarial attacks: PGD with 10, 20, 50, and 100 steps (Madry et al. 2017), CW (Carlini & Wagner 2017). and Auto/Attck (Croce & Hein 2020). For CIFARIO and 100, the Leg norm attack budget is set to c = 8/25 3/255, with perturbation step size 77 = 2/2 2/255 and iterations K 10. For ImageNet100, we set 0.03 and 7) 2/255.
Training Details. Following Ma et al. 2001 2022). we choose a normally trained adversary model with the same architecture as the target model. The nasty regularization coefficient wa is set to

---

Published as a conference paper at ICLR 2026
0.006, which is empirically validated optimal in our ablation study. For the target model, SGD optimizer with momentum 0.9 and weight decay 5 x 10-4. Training is conducted for 300 epochs with initial learning rate of 0.1, docayed by a factor of 10 at epochs 160 and 240. Batch sizes are 512 for CIFAR datasets and 128 for ImageNet. All experiments are performed on NVIDIA GeForce RTX 4090 GPUs using PyTorch 1.12.1. The adversary model is set the vanilla-trained counterpart of the target model under the basic setting, and we will discuss the model selection later. Furthermore, adversarial data augmentation, we adopt the same augmentation strategy as in WWag et al. 20023) demonstrating the superiority of our NAT framework under identical conditions.
ensure fair comparison with recent defenses that leverage diffusion-based
Datasets and Backbones. We evaluate NAT on two standard benchmark datasets, CIFAR- 10 and CIFAR-10K 100, which are widely used for adversarial robustness evaluation (Zhang et al. 2024b) Yue et al. 2024. Yin et al. 2024 Li et al. 20244) Zi et al., 2021 Zhou & Hua 2024). To further validate its scalability, we also test NAT on higher-resolution dataset, ImageNet
CIFARIO & CIFARIO0. For both datasets, we employ ResNet-18 (He et al.) 2016) and Wide- ResNet-34-10 10, which are commonly employed in adversarial defense evaluations.
ImageNet100. For ImageNer100, we use ViT-Small (Alexey
2020) as the backbone.
Our experimental comparisons primarily benchmark NAT against AGAIN (Jia et al.) 2022a) and LAS-AT (inn et al., (023), which currently achieves the state-of-the-ar performance (Zhang et al.) 2024b, Yue et al.) 2024 Yin et al. 2024 Li et al., 2024; Zi et al., 202I). All reported results correspond to the best outcomes over three independent runs.
Table 1: Test accuracy and robustness on CIFAR-10 and CIFAR- 100 using Wide-ResNete 34-
CIFARIO PGI3-5
CIFARIO0
Clean
PGD-10
PGI3-20
CRW
AA
Avg.
Clean
PGD-1D
PGD-20 PGI3-50
C&W
AA
Avg.
PGD-AT TRADES
85.17
56.07
55.00
54.88 88
53.91
51.67
59.46
60.89
$2.19
31 31.69
31:45
30.1
27.
35,
29.20
27.05
25.94
SNT
87.97
28.1
27.17
24.57
AWP
85.57
58.92
58. 13
57.92
56.03
53.90
61.74
60.38
34.13
33.86
33.65
31.12
28, 86
37.0
LEGAT
88
56.25
54.66
54. 30
54.29
52.23
59.99
60:64
35.13
34.75
34.62
30.65
29. 33
37.52
LAS-AWP
87.74
61.39
60.16
59.79
58.22
55.52
58,50
64.89
37.11
36.36
36.13
33.92
30.77
39.86
52.95
36.79
36.22
30.85
NAT (best)
(89.10
(63.43
(62.24
(61.77
(52
65.88
(62.72
(36.43
(35.90
(35,52
(32.32
(90.13 39.22
10.13)
0.211)
1.0.17)
1.0.25)
1.0.30)
1..41)
1.0.19)
0.29)
1.0.88
1.0.61)
1.052)
10.71)
63.66
35.
35.51
29.14
NAT (last)
(87.15
(64.8)
86
(63.42
162.79
(49 87 65.44
(34
(35.20
(34.97
(30.01
(28.45 37.88
20.14)
-0211)
20.16)
10.27)
20.29)
10.45)
20.14)
20.30)
20.43
20.58)
20.54)
10.70)
4.2
MAIN RESULTS
We report the results for CIFARIO and CIFARI00 on WRN-34-10 in Table and on ResNet-18 in Table 2 The mean and standard deviation are reported in parentheses (mean+ For compara- tive methods, the ResNet- 18 results are taken from Yin et al. (2023), while the WRN-34-10 results are taken from Jia et al. (202aa). NAT is further evaluated on ImageNet100 (as presented in Ap- pendix B). along with black-box attack settings (as presented in Appendix C). Results consistently show that NAT provides superior adversarial robustness, regardless of whether the backbone is convolutional neural network or a ViT, and whether the dataset is low-resolution (CIFAR) or high- resolution (lmageNet). In the Appendix D) we further supplement the discussion on the effectiveness and superiority of NAT against adaptive attacks. Additionally, results NAT with EDM-based data- augmentation provide in the Appendix These results demonstrate that NAT is compatible with EDM- based augmentation, and can further exploit its potential to achieve even stronger robustness, compared with other defenses under the same augmentation conditions. Also, digureBddemonssrates the significant probability sparsity of Nasty VT & AT compared to Normal VT & AT.
4.3 ABLATION STUDY
We perform ablation studies from multiple perspectives to evaluate the convenience, efficiency, and effectiveness of NAT. All ablation experiments are conducted on CIFAR-10.
7

---

Published as a conference paper at ICLR 2026
Table 2: Test accuracy and robustness on CIFAR-I0 and 100 dataset using ResNee-18 18.
CIFARIO
CIFAR10) Clean PGDD-10PGD-20PGD--5000
Clean PGD-10PPD--00GGI P0D C&W
Avg.
PGD-1100 C&W
AA
Avg.
PGD-AT 81.25
46.88
46..56
44.85
44.76
15.75
41.69
50.67
62.34
21.24
21.38
21.05
21.01
22.15
19.76
26.99
MART
81.61
52.38
51.28
50.93
50.80
47.77
46.09
54.40
55.14
28.52
28.08
27.79
27.91
25.65
24.04
31.01
TRADES
83.64
52.05
50.67
50.38
50.20
49.68
48.41
55.00
58.18
28.71
28.25
28.10
27.99
24.22
24,03
31.35
FAT
87.32
45.80
43.53
43.11
42.98
43.50
40.26
57
18.35
19.31
17.38
24.57
LBGAT
52.05
49.04
29
56.78
12.84
32
34.26
CAS
86.24
51.38
51.49
51.77
51,04
53.66
46.69
56.03
64.04
16
31
31.26
31,02
34,82
24.40
35.53
AWP
79.45
55.04
54.47
54.36
$4,30
51.17
49.40
56.88
54.00
31.78
31.49
31.44
31.74
28.20
26.
33.54
LAS-AT
82.39
54.74
53.70
53,70
53.72
51.96
49.94
57.16
58.38
12. 32
31 89
31.82
31.77
28.48
26.
34.5
AGAIN-AWP
86.52
59.99
59.35
59.11
58.85
61.19
51.89
62.41
64.51
35.58
35. 44
35.39
35.08
40.02
28.69
39.24
90.86 (
60.19
50.18
36.82
28.82
NAT (best)
90.77
62.12
60.81
59.91
59.72
62.24
(50.00 63.85
(63.77
(36.
(36.39
(36.27
(34.3)
(37.77
(28 39.26
1:0.15)
110.22)
1..27)
1.0.9)
1.0.22)
1.0.4)
10.46)
1.222)
1 1.022)
10.33)
10.41)
0.556)
1.0.51) 1.0.3)
60.00
34.55
27.17
NAT (last)
(90.07
(61.72 (59.72
(59.26
(58.57
(48.05
63.01
03
(34.30
(34.29
(33.92
(33.91
(36.42 (26.97 37.56
10.18) 1.0.22
10.21)
1 10.36)
1.029)
10.44) 10.51)
10.23)
10.33)
10.41)
10.39) 10.57)
1.0.8) 1.0.8)
Selection of the Papaee-ar x. In the original NT, the default value of À is set to 0.08. However, this choice may not be optimal un- der adversarial training. To investigate its ef- feet, we vary À in the range [0, 0.12] with step size of 0.02 and report the results in Fig- ure 2 The results indicate that for both the best and last models, accuracy and robustness first increase and then decrease, reaching the peak performance at X 0.06. Importantly, across all tested values, introducing nasty regu- larization consistently improves robustness, as shown from the markedly poorer performance observed at X 0.
Reults of Best Models
Results of Last Models
Cleze
(lem
PGD-10
80
PGGD-20
Auc. Rsh.
PG2D-10
IXi0--0
PCDD too
PGD-100
Auc.
CWW
90
Figure 2: Ablation study of x CIFARI0 dataset.
Impact of Adversary Model Structure. In the original setup, the adversary model is configured as the naturally trained counterpart of the target model. We further investigate the effect of varying adversary model architectures on NAT in Appendix [F] Overall, we find that adversary models with different structures consistently contribute robustness improvements, offering flexible options for NAT. Please refer to Appendix F for detailed discussion.
Impact of Adversary Model State. We analyze the impact of different adversary model parameter including random initialization, vanilla training, adversarial training, and SAM regularization. Overall, while all parameter states contribute to robustness improvements, some do not exhibit the characteristic "single peak uniform distribution" probability pattern. De- tailed discussions are provided in Appendix G
states on NAT in Appendix G
4.4 VERIFICATION OF SPATIAL METRIC RELATIONSHIPS.
We quantitatively validate the conclusions presented in the in-depth analysis section, including (1) the increased distances from data points to classification boundaries, (2) the enlarged inter-class weight gaps, and (3) the greater shortest distances between classification hyperplanes. All quantifi- cation results are achieved on the test set.
Distance from Data Point to Classification Boundary. We measure the distance of all samples from class each classification boundaries, as shown in Figure 6 To better illustrate the metric relationship between data points and classification boundaries, we remove the absolute value oper- ation in Equation TO Intuitively, the correctly classified target class should exhibit larger positive distance, whereas non-target classes should yield smaller or even negative distances. The robust model indeed shows greater distances to the decision boundaries compared to standard models. Moreover, the robust model tends to assign positive logits (i.e., positive distances) to semantically related classes (e.g., dog and cat), while dissimilar categories (e.g.. automobile and ship) consistently yield negative distances. This behavior indicates that NAT does not simply overfit the training data but tends to capture invariant semantic structures shared across similar categories while preserving higher inter-class separability.
8

---

Published as a conference paper at ICLR 2026
(4) Normal VT
Normal AT
(c) Nasty VT
(d) Nasty AT
0.75
Aventu Da Pro
Anchie: Prnh
Prch
0.25
0.00
Class
Figure 3: Verification of sparsity of model output probability.
AT
Neavy
ASTANAC 20120
Figure 4: Symmetric matrix of the shortest distance among each boundary.
Clean
PGD-10
PGD-20
PGD-50
CW
AA
VT
AT
NAT
Figure 5: UMAP visualization of feature distributions for Vanilla Trained model (VT), Adversarial Trained model(AT) and Nasty Adversarial Trained model (NAT).
Inter-Class Gap in the Weights of the Classification Layer. We use the L2 norm to quantify the magnitude of inter-class weight differences, ignoring directional effects. As shown in Figure 7 the results indicate that the nasty model produces substantially larger inter-class weight separations in the classification layer compared to the normal model.
Shortest Distance Among Linear Layers. Following the in-depth analysis, we compute the short- inter-class distances in the classification layers of both natural and robust models under normal and nasty settings. As shown in the symmetric matrices of Figure NAT consistently exhibits substantially larger shortest inter-class distances compared to standard models.
These findings validate our hypothesis regarding the spatial metric preference: adversaries must expend larger attack budget force misclassification, thereby enhancing robustness. In addition, a visualization of feature distributions using UMAP, provided in Figure further confirms NAT's spatial preference, characterized by larger inter-class separation and smaller intra-class compactness.
5 FURTHER DISCUSSION
We further examine two additional questions: (a) Does employing ensemble model the adver- sary model provide further gains for NAT? (b) Can NAT be extended to robustness distillation, and if so, does it achieve similar improvements hinder the student model's learning process in the

---

Published as a conference paper at ICLR 2026
(3) Nomal VT
(b) Normal
(c) Nasty VT
(d) Nasty
-50
Class Iodcs
Clas Endox
Clacs indes
Figure Average distance of class samples in CIFARIO dataset to all classification boundaries.
Figure 7: Symmetric matrix of L2 distances between classification layer weights.
nasty training? Our findings suggest that while the ensemble adversary model provides marginal improvements. is not necessary. Moreover, NAT can provide a degree of undistillable protection for robust teachers, though further investigation needed strengthen this property. We also com- pare NAT against explicit regularizers such as entropy regularization and logits norm regularization, highlighting its irrelacceability. In addition, provide brief discussion of NAT's training cost, showing that it introduces only minimal overhead. Finally, we outline the limitations of NAT. More detailed analyses of these issues presented in Appendix H M
CONCLUSION
In this paper, we propose nasty adversarial training (NAT), simple yet effective regularizer for ad- versarial training that leverages the probabilistic sparsity prior. We first provide theoretical analysis of how NAT induces probabilistic sparsity and examine its role in enhancing robustness from a met- ric perspective. Extensive experiments demonstrate that NAT achieves state-of-the-ard adversarial robustness. Additional analyses further highlight its simplicity, efficiency, and irreplaceability.
ETHICS STATEMENT
Our research on NAT addresses open problem in Al using public datasets. We declare no potential conflicts interest, and the work raises no issues related to bias, fairness, privacy, security, or legal compliance. The study, which is purely methodological and requires no IRB approval, is conducted with the sole intention of improving model robustness against adversarial attacks and presents no malicious ethical threats.
ACKNOWLEDGMENTY
This work was supported in part by the National Natural Science Foundation of China under Grants 62572150, 62372137, and 62472117, the Guangdong Basic and Applied Basic Research Foundation under Grant 1054A151 the Major Key Project of PCL under Grant PCL2024A05, and the Shenzhen Science and Technology Program under Grant JCYI20230807094411000
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Dosovitskiy Alexey, An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In International conference on machine learning, pp. 274-283. PMLR, 2018.
Song Bai, Yingwei Li, Yuyin Zhou, Qizhu Li, and Philip HS Torr. Adversarial metric attack and defense for person re-identifiiation. IEEE Transactions Pattern Analysis and Machine Intelli- gence,
2020.
Quentin Bouniot, Romaric Audigier, and Angelique Loesch. Vulnerability of person re- identification models to metric adversarial attacks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 794-795, 2020.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint
2017.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium security and privacy (sp). pp. 39-57. Ieee, 2017.
Pin-Yu Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi. and Cho-Jui Hsich. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pp. 15-26, 2017.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse caramcter-free attacks. In International conference machine learning, pp. 2206- 2216. PMLR, 2020.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEFE conference computer vision and pattern recognition, pp. 9185-9193, 2018.
Ziyi Dong, Pengxu Wei, and Liang Lin. Adversarially aware robust object detector. In European Conference on Computer Vision, pp. 297-313. Springer, 2022.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-worid attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1625-1634, 2018
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 3996-4003, 2020. 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 1rXiv:1412.65777 2014.
Sven Gowal, Chongli Qin, Jonathan Uesato. Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Sven Gowal, Sylvestre-Alvies Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. Advances in neural information processing systems, 34:4218-4233, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Zhichao Huang. Yanbo Fan, Chen Liu, Weizhong Zhang, Yong Zhang, Mathieu Salzmann, Sabine Süsstrunk, and Jue Wang. Fast adversarial training with adaptive step size. IEEE Transactions on Image Processing, 2023.
11

---

Published as a conference paper at ICLR 2026
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang. and Xiaochun Cao. Las-at: adversarial training with learnable attack strategy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13398-13408, 2022a.
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Jue Wang, and Xiaochun Cao. Boosting fast adversarial training with learnable adversarial initialization. IEEE Transactions on Image Processing, 31: 4417-4430, 2022b.
Hyeyoon Lee, Kanghyun Choi, Dain Kwon, Sunjong Park, Mayoore Selvarasa Jaiswal, Noseong Park, Jonghyun Choi, and Jinho Lee. Datafreeshield: Defending adversarial attacks without training data. arXiv preprint arXiv:2406. 15635, 2024.
Yuezun Li, Daniel Tian, Ming-Chin Ching Chang, Xiao Bian, and Siwei Lyu. Robust adversarial pertur- bation on deep proposal-bes -based models. arXiv preprint arXiv: arXiv: /809.89092 2018.
Zhuorong Li, Daiwei Yu, Lina Wei, Canghong Jin, Yun Zhang, and Sixian Chan. Soften to defend: Towards adversarial robustness via self-guided lahel refinement. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 24776-24782 2024.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1778-1787, 2018.
Haoyu Ma, Tianlong Chen, Ting-Kun Kuei Hu, Chenyu You, Xiaohui Xie, and Zhangyang Wang. Undis- tillable: Making a nasty teacher that cannot teach students. arXiv preprint arXiv:2105.07381, 2021.
Haoyu Ma, Yifan Huang, Hao Tang, Chenyu You, Deying Kong, and Xiaohui Xie. Sparse logits suffice to fail knowledge distillation. In ICLR 2022 Workshop on PAIR (extacciicincmmm 2Struct: Privacy, Accountability, Interpretability. Robustness, Reasoning on Structured Data, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1006:::770606000 2017.
Seyed-Mohsen Moosavi-Dezfooli: Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 2574-2582, 2016.
Tianyu Pang. Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag tricks for adversarial training. arXiv preprint arXiv:2010.00467, 2020.
Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, and Shuicheng Yan. Robustness and accuracy could be reconcilable by (proper) definition. In International conference on machine learning, pp. 17258- 17277. PMLR, 2022.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), pp. 882-597. IEEE, 2016.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint 1rXiv:2103.01946, 2021.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International conference machine learning, pp. 8093-8104. PMLR, 2020.
Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mittal. Robust learning meets generative models: Can proxy distributions improve ad- versarial robustness? arXiv preprint 214.02200 2021.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International conference on learning representations, 2019.
12

---

Published as a conference paper at ICLR 2026
Yuzheng Wang, Zhaoyu Chen, Dingkang Yang, Pinxue Guo, Kaixun Jiang, Wenqiang Zhang, and Lizhe Qi. Out of thin air: Exploring frta-free adversarial robustness distillation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 5776-5784, 2024.
Zekai Wang. Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In International conference on machine learning, pp. 36246-36263. PMLR, 2023.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. Advances in neural information processing systems, 33:2958-2969 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprint 17/Xiv/7/11.011 2017a.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In Proceedings of the IEFE interna- fional conference on computer vision, pp. 1369-1378 2017b.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 501-509, 2019a.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2730-2739, 2019b.
Shenglin Yin, Kelu Yao, Sheng Shi, Yangzhou Du, and Zhen Xiao. Again: Adversarial training with attribution span enlargement and hybrid feature fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2023.
Shenglin Yin, Zhen Xiao, Mingxuan Song, and Jieyi Long. Adversarial distillation based on slack matching and attribution region alignment. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 24605 246144, 2024.
Wang Yu-Hang, Junkang Guo, Aolei Liu, Kaihao Wang. Zaitong Wu, Zhenyu Liu, Wenfei Yin, and Jian Liu. Taet: Two-stage adversarial equalization training on long-tailed distributions. In Pro- ceedings of the Computer Vision and Puttern Recognition Conference, pp. 15476-15485, 2025.
Xinli Yue, Ningping Mou, Qian Wang, and Lingchen Zhao. Revisiting adversarial training under long-tailed distributions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24492-24501, 2024.
Haichao Zhang and Jianyu Wang. Towards adversarially robust object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 421-430, 2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent Bl Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference machine learning. pp. 7472-7482. PMLR, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez- Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 2017.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International conference on machine learning, pp. 11278-11281 PMLR, 2020a.
Lei Zhang, Yuhang Zhou, Yi Yang, and Xinbo Gao. Meta invariance defense towards generalizable robustness to unknown adversarial attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024a.
Linjun Zhang, Zhun Deng, Kenji Kawaguchi, Amirata Ghorbani, and James Zou. How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.048/9 2020b.
13

---

Published as a conference paper at ICLR 2026
Yanghao Zhang, Tianle Zhang, Ronghui Mu, Xiaowei Huang, and Wenjie Ruan. Towards fairness- aware adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024b.
Yuhang Zhou and Zhongyun Hua. Defense without forgetting: Continual adversarial defense with anisotropic & isotropic pseudo replay. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24263-24272. 2024.
Yuhang Zhou, Yushu Zhang, Leo Yu Zhang, and Zhongyun Hua. Derd: Data-free adversarial ro- bustness distillation through self-adversarial teacher group. In ACM Multimedia 2024.
Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. arXiv preprint arXiv:2106.0493 2021.
Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distil- lation: Robust soft labels make student better. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16443-16452, 2021.
Junhua Zou, Zhisong Pan, Junyang Qiu, Xin Liu, Ting Rui, and Wei Li. Improving the transferability of adversarial examples with resized-diverse-innee European Conference on Computer Vision, pp. 563-635577 563-579. Springer, 2020.
and region fitting. In
APPENDIX
A RELATED WORKS
A.1 ADVERSARIAL ATTACKS
Adversarial attacks aim to introduce imperceptible perturbations to input data, causing well-trained model to produce incorrect predictions. Depending the amount of information accessible to the attacker, adversarial attacks are generally categorized into white-box and black-box settings. In white-box attacks, the attackers has fully access the target model. Common methods clude gradient-based approaches (Goodfellow et al.) 2014 Dong et al.) 2018 Madry et al.) 2017). classifier-bart based methods (Moosavi-Dezfooli al.] 2016), and optimization-
techniques (Car- lini & Wagner, (017).. In contrast, black-box attacks assume limited prior knowledge of the target model and are typically classified into score-based (Chen al. 2017), decision-based (Brendel et al.) 2017), and consfer-based attacks (Xie et al. (019b] Zou et al. 2000)). Among them, transfer-based attacks involve training a surrogate model to simulate the target model's 's behavior and are widely used to evaluate the black-box adversarial robustness of DNNs.
A.2 ADVERSARIAL DEFENSES
Adversarial defenses aim improve model's accuracy on adversarial examples. AT Goodffllow et al. 2014 Zhang et al. 2019 Dong et al. 20188 Madry et al. 2017 7 Jia et al., 2022a Wang et al. 2019 widely regarded as one of the most effective defense. RD Gooldllum et al. 2020 Zhu al,
2021
Zi et al., [202T) is later proposed to transfer robustness from a large, robust model to a smaller.
more efficient student model. Recently, AT are further explored, such as fairness issue (Zhang al. 2024b), attention maps (Yin et al. 2023). and learnable strategies (Jia et al.) 2022a). Data-free ad- versarial defense [Wang et al. 2024| Zhou et al. Lee et al. 2024) are explored to achieve adversarial robustness in scenarios with limited data. Additionally, internal maximization methods like fast ad- versarial training (FAT) (Jia et al. 2022b Huang et al. 2023) have been developed to enhance the efficiency of AT without significantly sacrificing robustness. In this paper, focus AT and pro- pose a simple yet effective regularization strategy from the perspective of output probability sparsity, which achieves state-of-the-art adversarial robustness with little additional computational overhead.
14

---

Published as a conference paper at ICLR 2026
Table 3: The results of several advanced adversarial defense using the EDM-bbsed data augmenta- tion.
Dataset
Architecture
Method
Generated
Batch
Epochi
Clean
AA
WRN-34-20
Rice et al.
2020
128
85.34
53.42
WRN-34-10
Zhang ct al
2020
128
120
84.52
53.51
WRN-34-20
Pang et al
2020
X
128
110
86.43
54.39
WRN-34-10
2020)
128
200
85.36
56.17
WRN-70-16
Gowal
2020)
512
85.29
57.14
WRN-34-10
(Sehwag ct ai
20721
10M
128
200
87.00
60.60
(Rebufh ct al
2021)
IM
1024
800
87.33
60.73
Pang al.
2022)
IM
512
400
88.10
61.51
Gowal et
2021
100M
1024
2000
87.50 63.38
WRN-28-10
IM
512
400
91.12 63.35
IM
1024
800
91.43 63.96
Wang al.
2023
CIFAR-10
50M
2048
1600
92.27 67.17
(lo., 8/255)
20M
2048
2400
92.44 67.31
Pang 2022
IM
512
88.57
63.74
Rebuth ct
202
1024
88.54
64.20
Gowal et al.
2021
100M
1024
2000
88.74
66.11
WRN-70-16
IM
512
400
91.98
65.54
Wang al
2023
5M
512
800
92.58
67.92
5OM
1024
2000
93.25
70.69
WRN-34-10
Wu et al.
2020
128
200
60.38
28.86
WRN-70-16
Gowal cf al
2020
512
200
60.86
30.03
WRN-34-10
Schhag et al.
2021
IM
128
200
65.90
31.20
128
200
0
WRN-34-10
AutoAug
128
512
300
89.15
52.95
WRN-34-10
Ours
IM
1024
600
90.44
55,41
5M
1024
600
91.01
57.24
Pang al.
2022
IM
512
400
62.08
31.40
Rebulli
202
IM
1024
800
62.41
32.06
WRN-28-10
IM
512
400
68.06
33.65
Wang et al.
2023
50M
2048
1600
72.58
38.83
Pang et al.
2022
IM
512
400
63.99
33.65
CIFAR-100
WRN-70-16
Rebuth et
2021
IM
1024
800
63.56
34.64
IM
512
400
70.21
38.69
(loo 8/255)
Wang et al.
2023
50M
1024
2000
75.22
42.67
128
200
0
WRN-34-10
AutoAug
128
200
0
512
300
62.87
30.85
WRN-34-10
Ours
IM
1024
800
64.74
33.08
5M
1024
800
66.63
36.71
B RESULTS ON IMAGENET- 100
In this section, we report the robustness performance of NAT on ImageNer100. Due to the lack of direct comparisons, we primarily compare NAT with natural models and adversarial training models. The experimental results are shown in Table Under the same experimental setup, com- pared to smaller datasets like CIFAR1O, NAT demonstrates significant robustness increments on ImageNet100.
C RESULTS ON BLACK-BOR ATTACKS
In this section, provide simple evaluation of NAT's adversarial robustness against black-box attacks based on CIFARIO. We mainly follow the black-box attack settings in RSLAD, which test both the transfer attack and query-based attack. For transfer attack, generate the adversarial examples by PGD-20 and CW on an adversarially pre-trained surrogate ResNet-50. The maximum perturbation also set to 8/255. For query-based attack, use the strong Square attack. The
15

---

Published as a conference paper at ICLR 2026
Table 4: Test accuracy and robustness of the ImageNet-100 dataset ViT-small.
Clean
PGD-10
PGD-20
PGD-50
WGD-100
C&W
AA
Ave.
Vanilla Training
96.23
8.23
6.54
4.58
4.14
7.97
1.68
18.48
PGD-AT
95.69
64.44
62.32
39.80
39.71
41.64
37.12
54.38
NAT (best)
94.92
67.07
66.84
43.22
44.16
49.82
45.44
58.78
NAT (last)
94.87
64.77
66.37
42.13
45.07
48.53
44.40
58.01
Table 5: Black-box robustness on CIFAR1O.
ResNet-18
MobileNetV2
Methods
PGDs
CW
Square
PGDs
CW
Square
SAT
60.84
60.52
54.27
60.46
59.83
53.94
ARD
63.49
63.05
56.89
62.13
61.85
55.60
IAD
62.78
62.26
56.62
61.57
61.25
55.45
Trades
62.20
61.75
S5.13
60.90
60.23
53.46
RSLAD
64.11
63.84
57.90
63.30
63.20
56.70
LAS-AT
66.42
65.41
60.21
66.14
65.42
58.87
NAT
67.19
66.77
60.23
67.82
66.49
59.21
Table 6: Evaluation on the adaptive attacks.
Backhones Defenses
APGD-10 APGD-20 APGD-50 APGD-100
CI
100 Ave. APGD-10 APGD-20 APGD-50 APGD-106 Ave.
CTFARI00
VT
12.17
9.88
9.02
8.88
9.98
2.91
2.68
2.63
2.51
2.68
WRN3410
AT-PGD
$7.57
54.52
53.36
53.14
54.64
28.21
28.00
27.95
27.80
27.99
NAT
61.48
59.02
58.27
58.11
59.22
33.14
32.41
32.01
31.77
32.33
VT
10.37
9.17
8.77
8.71
9.25
4.47
4.09
3.95
3.90
4.10
ResNer18
AT-PGD
53.77
52.07
$1.60
51.48
52.23
27.33
26.83
26.67
26.65
26.87
NAT
61.00
58.25
58.11
57.75
58.77
33.62
32.72
32.14
32.01
32.62
experimental results are shown in Table 5 The results in the table include both adversarial training methods and robustness distillation methods. To maintain consistency, we performed NAT training on MobileNetV2 for evaluation. It can be observed that our proposed NAT significantly outperforms the provided mainstream adversarial training methods and robustness distillation methods in black- box attacks, showing its superiority.
D EVALUATION ON THE ADAPTIVE ATTACKS
In this section, discuss the effectiveness of NAT against adaptive attacks. This discussion is based on the assumption that the attacker is fully aware of the defender's NAT defense paradigm and, consequently, uses the objective loss function of NAT the target function for their attacks. The objective function of an adaptive attacker can be formalized as:
arg max [XE (a (pro, (x 8)) v) W., KL (or. (pro, (x ()+ Tr. (P.o. (x (x+6))) S0
(14)
Such attack strategy is constructed using the PGD framework, with its hyperparameters remaining consistent with those specified in the main body of the paper. We similarly evaluate the effective- ness of NAT on both ResNet- 18 and WideResNeet-44-11 10 architectures, utilizing the CIFAR-10 and CIFAR 100 datasets, and compare its performance against both Vanilla Training and PGD-AT. The experimental results are summarized in Table 6
NAT consistently demonstrates effective robustness against adaptive attacks, significantly outperforming PGD-AT. This robust performance underscores the reliability of NAT's defensive capabilities. In essence, unlike early defenses that rely random- ization, NAT remains fundamentally rooted in the adversarial training paradigm, which inherently avoids the pitfalls associated with "obfuscated gradients"..
16

---

Published as a conference paper at ICLR 2026
E COMBINING NAT WITH EDM-BASED DATA AUGMENTATION
In this section, we provide the experimental results of NAT using EDM-based data augmenta- tion (Wang al. 2023), with comparisons to state-of-the-arr (SOTA) adversarial defenses under the same data augmentation strategy. The results are shown in Table 3 According to the conclusion of [Wang et al. (2023), under conditions of large-scale data augmentation, larger batch size and more training epochs can significantly improve model performance (Wang et al. [202]). Still, with identical data augmentation conditions (i.e., generated data number, batch size, and epoch number), NAT can achieve optimal adversarial robustness performance. This demonstrates the inherent supe- riority of NAT, confirms its compatibility with such augmentation, and reveals how they mutually explore each other's ' potential for enhancing adversarial robustness.
Table The impact different adversary model architecture. The target backbone WRN-34-10.
Clean PGD-10 PGD-20
PGD-50
PGD-100 C&W
AA
Ave.
WRN-34-10
89.15
63.69
62.34
62.05
62.01
65.10
52.95
65.32
Adversary Model Structure
DN121
87.95
66.00
64.39
63.62
63.42
65.46
48.47
65.61
RN18
88.09
66.73
64.88
64.09
63.00
67.36
48.43
66.08
RN50
88.12
68.04
66.71
66.15
65.08
67.47
46.67
66.89
Ensemble Models
88.94
69.12
67.34
67.19
66.37
67.84
46.00 67.64
Normal Adversary
(b) SAM Adversary
(c) Robust Adversrry
(d) Nasty Adversary
Avenge Prob
Avenge Poob.
Avenge Prob.
Avcrage Frah.
Figure 8: For the NAT with different adversary model structures, the output probability of "cat" class.
Table The impact different adversary model parameter state.
The target backbone is WRN-34-
10.
Clean
PGD-10
PGD-20
PGD-50
C&W
AA
Ave.
VT
89.15
63.69
62.34
62.05
65.10
$2.95
65,88
Para. State
SAM
84.44
67.63
63.71
63.39
65,01
47.69
65.31
AT
85.90
67.66
65.96
64.92
63.83
53.42
66.94
Nasty
82.94
68.16
64.14
64.02
63.04
49.53
65.30
Table Training time statistics for one epoch.
CIFARIO
CIFAR100
ResNet-18
WRN-34-10
ResNet-18
WRN-34-10
Vanilla Training
15.3 s
2 90 .2s
15.7s
101.3s
Adversarial Training
108.6s
1047.2
110.9s
1197.2s
MID
(Zhang et al. 2024a)
2530+10s
3460420s
NAT
129.8 s
1165.8
127.3s
1328.2s
F IMPACT OF ADVERSARY MODEL STRUCTURE
In the original setup of NT, the adversary model is configured as a vanilla-trained model with the same structure the target model. Here, discuss the impact of adversary models with different
17

---

Published as a conference paper at ICLR 2026
Table 10: The impact of NT to robustness distillation.
CClen
PGD-10
PGD-20
PGD-50
PGD-100
C&W
AA
Ave.
From AT Teacher 89.23
50.57
50.19
50.09
48.86
47.15
45.77
54.55
RSLAD
From NAT Teacher 87.97
47.50
47.24
47.20
47.15
48.86
42.64
52.65
From AT Teacher 79.43
38.84
38.21
39.93
39.87
36.61
34.12
43.85
ARD
From NAT Teacher 77.67
37.66
37.52
37.46
37.45
38.78
30.43
42.42
From AT Teacher 74.72
42.01
41.91
41.89
41.82
44.41
37.55
46.33
IAD
From NAT Teacher
74.29
41.64
41.61
41.58
41.48
43.39
36.25
45.74
Table 11: Comparing NAT with explicit regularizers.
Methods
Clean
PGDI0
PGD20
PGD50
PGD-100
C&W
AA
Avg.
VT
95.03
11.72
10.52
10.13
10.09
11.42
7.81
22.38
NT
94.37
16.57
14.65
13.38
12.92
13.04
8.84
24.82
AT
84.25
46.88
46.56
44.85
44.76
45.75
41.69
50.67
AT+LASSO
88.06
41.45
39.77
39.08
38.78
40.18
30.54
45.40
AT+EMR
90.4
55.63
54.67
54.32
54.22
53.04
47.77
58.57
AT+NNR
92.13
53.83
52.59
52.1
51.97
52.04
46.53
57.31
AT+mixup
82.5
47.66
46.88
46.57
46.55
48.49
38.07
50.95
NAT
90.86
62.37
60.94
60.19
59.91
62.54
50.18
63.85
structures on the robustness increments. We select WRN-34-10 as the backbone the target model for ablation analysis and DenseNet-121. ResNet- 18, and ResNet-50 the backbones of the adversary model, in order to analyze the effects of different structures and different capacities. For ease of comparison, model. The experimental results, as shown in Table 7 indicate that various structures of adversary models, whether isomorphic or heterogeneous, can provide a stable robustness increment the tar- get model. Moreover, adversary models with larger capacities seem to demonstrate better robustness increments for the target model. For instance, ResNet50, serving as the nasty model, can provide better robustness increments compared to ResNet-18 18 the target model.
also report results for the adversary model that is isomorphic to the target
G IMPACT OF ADVERSARY MODEL STATE
In the original setup, the adversary model is configured natural model with the same structure as the target model. Here, we discuss the impact of different model parameter states on robustness improvement, which includes random initialization, vanilla training, adversarial training, SAM reg- ularization, and Nasty model. The experimental results are shown in Table g demonstrating that different model parameters all provide robustness improvements for NAT.
An interesting question is that the nasty model no longer shows the "single peak uniform distri- bution" probability distribution, yet still provides robustness improvement. We speculate that the output probabilities of the nasty model exhibit "a few peaks & uniform zero distribution". thus allowing the target model achieve sparse probability distribution by moving away from the uni- form zero distribution. However, the non-target class peaks of the target model might have different labels compared the non-target class peaks when using a regular model as the adversary.
We validated this hypothesis in Figure g For the "cat" class, when a normal model is used as the adversary model, the suboptimal peaks are "dog" and "horse." When the nasty model used as the adversary model, the suboptimal peak concentrates on "dog". Similarly, suboptimal peak variations also appear when SAM models and Robust models are used as adversary models. However, All the setups can achieve sparse probability distribution and robustness improvement model. Such results confirm our hypothesis: different adversary models may lead the model to learn different suboptimal classes, but they consistently cause the target model learn sparse probability distribution. The discussion above may also partially explains why feeding the adversary model with both natural examples and adversarial examples provide significant adversarial robustness
the target
18

---

Published as a conference paper at ICLR 2026
improvement for the target model, even though the outputs of the adversary model adversarial examples may not satisfy the "single peak" assumption.
H ANALYSIS OF TIME COSTS
In this subsection, we briefly discuss the computational cost of NAT. In recent studies adver- sarial robustness, MID (Zhang et al. 2024a) provides an intuitive quantification of computational time cost by measuring the time required to train epoch. Following the setup in MID, also present a simple quantification of the computational time cost, as shown in Table Intuitively, NAT introduces only limited additional computation compared to standard adversarial training, primarily attributed to the inference time of the adversary model. We further demonstrate that the architec- tural design of the adversary model has negligible impact on the target model. is worth mentioning that the time cost reported in Table 9 does not include the time cost of performing Vanilla Training (VT) on the adversary model. This omission is primarily based on the following considerations: (a) The structure and training settings of the adversary model are flexible and can generally bring performance gains to the model (as shown in Table 7 and Table S) (b) Compared to adversarial training, the time cost of vanilla training is negligible. In cases where the target model and the ad- versary model share the same architecture, the time cost reported for "vanilla training" in the first row can serve reference for the VT time cost of the adversary model. In conclusion, argue that the extra computational cost NAT remains controllable and acceptable compared to standard adversarial training.
1 DOES ENSEMBLE ADVERSARY MODEL HELP?
In machine learning, a natural approach to model enhancement is ensemble learning. In this section, we discuss the robustness performance of NAT when the adversary model is an ensemble model. Specifically, we choose ResNet- 18, DenseNet12) and VGGI1 to form the ensemble model. We use the same ensemble method A, guiding the nasty regularization with the mean of the logits output by the three models. The experimental results are shown in the last line of Table 7
Compared to the standard setup, the ensemble model shows slight incremental advantage. This is an expected result, as models of any structure tend to exhibit a "single peak + uniform distribution" probability output preference. Averaging the logits of different models further smooths the uniform distribution. We have analyzed that moving away from this uniform distribution can bring robustness improvements to NAT. However, given the significantly increased computational cost, this slight increment may suggest that an ensemble adversary model is not necessary.
J TRANSFER NAT TO ROBUSTNESS DISTILLATION
In this section, we discuss an open question: Can NAT be transferred robust distillation achieve incremental improvements? Or does it, like regular nasty model, prevent the robust model from being distilled by student models?
As we mentioned, the original purpose of NT is to make the nasty model difficult for student models to learn simply. However, whether this property can be transferred NAT is still unclear. If this property is transferable, NAT could also serve as a method to protect the copyright of robust models, preventing robust teachers from being distilled into student models without authorization. We briefly discuss this issue in this section.
Specifically, we use WRN-34-10 trained with standard AT and NAT as the robust teacher models, and randomly initialized ResNet-18 18 as the student model. We apply RSLAD, ARD, and IAD as robustness distillation methods to perform robustness distillation on the student model. The exper- imental results are shown in Table 10 As teacher model, although the NAT teacher shows better robustness performance than the AT teacher, its student model's 's robustness performance is relatively poor. This indicates that NAT has a certain tendency avoid being learned, which can also serve as a method to protect the intellectual property of robust teachers. However, this protective effect is still far from the ideal of an unlearnable teacher, the student model still acquires considerable level
19

---

Published as a conference paper at ICLR 2026
of robustness performance. There remains significant space for research in the intellectual property protection of adversarial robust models.
K COMPARING NAT WITH EXPLICIT REGULARIZERS
NAT can be considered integrating probability sparsity regularization into AT. In this section, explain the differences between NAT and several other explicit regularization methods, as well as the advantages of NAT.
To begin with, we need to clarify the distinction between the proposed probability sparsity and model sparsity. From the conceptual perspective, model sparsity generally refers to the sparsity of model parameters, which reduces the redundancy of the model, while probability sparsity means that the probabilities of irrelevant categories, calculated based on the model's output logits, approach zero. From the effect perspective, model sparsity aims to decrease model complexity and reduce redundant features, thereby enhancing generalization. The effect of probability sparsity more intuitive: it induces the target model to output probabilities only for the target category or similar categories, while reducing confidence in irrelevant categories. This allows the model to learn more generalized and semantically coherent feature representations and classification preferences. Thus, there an essential difference between the two. A common regularization method for model sparsity is LASSO regularization, which can be combined with adversarial training and formalized as:
Lxr ALASSO
W
(15)
where ÂLASSO is set as 0.0001. We compare the experimental results of NAT and Lasso regularization in Table TT] where NAT still shows better results. This proves that LASSO regularization cannot achieve the probability sparsity that NAT can provide, making it difficult to attain similar robustness increments.
Furthermore, in order to demonstrate the irreplaceability of NAT, we explain the differences between NAT and directly constraining the probability outputs. One intuitively potential method to achieve probability sparsity is to directly add explicit regularization at the output end, such as minimizing the output entropy:
L LxT + λEMR
Σ
pi log
(16)
which we call entropy minimization regularization(EMRR) later, Also, one can regularize negative of the L2 norm of the output values as:
C = LAT ANNR
(17)
which we call negative norm regularizattin(NNNR) later.
Intuitively, both EMR and NNR induce the model to output sharp probability distributions and achieve optimal solutions when the model outputs one-hot probabilities. Although they also achieve probability sparsity in form, this sparsity is suboptimal. 'The overly rigid one-hot constraint re- sembles overfitting. EMR and NNR amplify the cross-entropy loss's "winner takes all" preference, causing the model to be overly confident in the current training data and label, which results in a loss of generalization capability. At the same time, the strict constraints may amplify the impact of noise. In contrast, NAT does not explicitly require the model to output very rigid one-hot labels; instead, induces the model to allocate certain probability outputs for similar classes (e.g., cats and dogs) and output smaller probabilities for less related categories (e.g., cats and airplanes) in adaptive manner, thus learning robust features that generalize across categories. To validate this perspective, compare NAT with the performance of these two regularizers, and the experimental results shown in Table II where λEMR = 0.001 and ANNr = 0.001. Although EMR and NNR can bring certain increments AT, NAT still demonstrates optimal results, proving that it cannot be simply replaced by methods that constrain output probabilities.
Furthermore, one may think about not constraining the model to learn one-hot output tendencies but rather allowing the model's output to approach a soft label. However, obtaining such soft la- bels is costly, especially when the desired suboptimal classes similar to the target class. Such
20

---

Published as a conference paper at ICLR 2026
Table 12: Evaluate NAT on person Re-ldentification.
Datasets
Defense
Clean
8/255-16
FNA
10/255-16
8/255-16
SMA
10V25-16
8/255-16
IPGSM
10255-16
None
78. 13.49992.00 01
0.200011 17
0.1800.44
0.27A1.26
0.2080.11
1,25//195
1,09/1 66
Market
AMI AMD&NAT
69V8 24
8:57/18.14
4.37,941
22.85/35.69
15.21/21.37
17.97/3465
11.74/23.32 34
68.92/88.25
12.64/2542
8.66/15.43
26.14/38.05
22.07/27.42
22.33/36.79
13.9028.93
Table 13: Evaluate NAT on object detection.
Attacks
Clean
losScls
loss
DAG
RAP
Standard
72.1
1.5
0.0
0.3
0.6
MTD
47.2
28.2
30.7
26.7
43.5
MTD&NAT
48.4
30.7
31.3
27.8
44.7
soft labels require extensive manual identification of which classes are "similar classes". Robust- ness distillation (Goldblum et al. 2020; Zhu et al. 202I; Zi et al. 202T) uses soft labels from the teacher's output to guide the student model. However, the reliability of the soft labels depends on the robustness of the teacher model. Reliable regularization is still needed guide the training of the teacher model. An alternative regularization to obtain soft label is to use mixup (Zhang et al.) 2017 (0220b) to generate augmented samples and soft labels. Compared to the adaptive allocation capability of NAT, mixup soft labels is manually set and do not guarantee that suboptimal classes are always categories similar the target class. We also report the experimental results in Table where the ratio of mixup is set to [0.7, 0.3] and randomly sampled in the current batch. Compared to mixup, NAT still demonstrates superiority.
Finally, we need to point out that although probability sparsity helps with adversarial robustness, directly applying sparsity regularization to vanilla training (which is actually the original nasty training (Ma et al. 202I)) does not achieve sufficient and reliable adversarial robustness. This is because such models can only learn sparse probabilistic outputs but still cannot acquire robust rep- resentations. The role of probability sparsity is further guide the semantic interpretability and generalization of robust knowledge based on the model's learning of that robust knowledge. Results of NT in Table [1 validate the above viewpoint.
L PERFORMANCE ON NON-CLASSIFICATION TASKS
We further argue that NAT, through the orthogonalization of linear layers, can introduce a degree of feature space separation in the target model, and such property is transferable to other tasks. To further discuss the effectiveness of NAT for non-classification tasks, we consider two tasks dis- tinct from pure classification: (1) Person Re-identification (classification metrie learning), and (2)
Object detection (classification + regression).
First, for the person re-identification task, adopt the common setup combining the cross-entropy loss and the triplet loss. We investigate commonly used adversarial metric attacks in RelD, including FNA Bouniot et (2020). SMA Bouniot et al. (2020), and IFGSM (e.g., AMA Bai et al. (200 ) The experimental results are shown in Table 12 The effectiveness of NAT for metric models trained with cross-entropy loss is foreseeable. As stated in the theoretical analysis, we argue that the prob- abilistic sparsity of NAT primarily stems from the high-order power constraint on the output logits, which can subsequently regularize the metric relationships the feature space. Therefore, NAT can also bring performance gains to metric learning tasks.
Secondly, for the object detection task, we adopt the setup from Dong et al. (002)) ) Zhang & Wang (209)) and reproduce the results of Table 2 in in/Zhag & Wang (2019) on the PASCAL VOC dataset, based on which we further provide a preliminary validation
the efficacy of combining NAT with Multi-task Domain (MTD) training (since method in [3] is based generative reconstruction, which is not conducive to the integration and fair comparison with NAT). As defined in [4], the MTD setup here refers to the defender performing min-max adversarial training by leveraging both the classifi- cation loss and the localization loss during training. We follow the standard NAT configuration in this experiment, applying the adversary model and NAT procedure only the classification loss.
21

---

Published as a conference paper at ICLR 2026
The attack methods employed include classification -ooss-ollsyyyyy attack (losscls). recalization-loss--one attack (loss DAG Xie et al. (2017b), and RAP Li et al. (2018). The experimental results are shown in Table T3 which demonstrate that NAT not only brings robustness increments to the ad- versarial training framework based on the classification loss, but also generalizes to enhance the framework utilizing the localization loss. This outcome provides preliminary evidence that NAT can contribute to the adversarial robustness of object detection task.
In summary, argue that NAT can stably bring robustness increments adversarial training for non-classification tasks.
M LIMITATIONS
NAT can be regarded as general improvement to commonly used adversarial, with its main contri- butions stemming from simple yet effective modifications adversarial training and the exploration of robust sparsity attribution. The limitations of this work primarily lie in the evaluation and com- parison on larger datasets (e.g.. the full ImageNet-Ik). On one hand, this due to the massive computational complexity involved, and on the other hand, because the primary evaluation proto- cols within the adversarial robustness community have predominantly focused on datasets such as CIFAR, while benchmarks for ImageNet- kk remain relatively underdeveloped and establishing such a benchmark may be huge task. Furthermore, extending our method to other downstream tasks could represent interesting direction for future exploration. We leave these explorations for future work.
THE USE OF LARGE LANGUAGE MODELS (LLMS)
LLMs are used sparingly and only to assist with proofreading and improving the linguistic fluency of few sections of this paper (such Related Work, Experiments, and Appendix), i.e., aid or polish writing. All scientífic contributions and core idea are the work of the authors. This assistance poses no issues of scientific ethics misconduct.
22


Reviews:
[{"review_id": "vkzk4dwovn", "reviewer": "Reviewer_2bUB", "summary": "This paper introduces Nasty Adversarial Training (NAT), a defense framework for adversarial attacks. NAT integrates the concept of probability sparsity, which was originally proposed in the context of distillation resistance, into traditional adversarial training. The paper aims to demonstrate how enforcing sparsity in the output probability distribution can enhance the robustness of models by increasing the inter-class separability and widening decision margins in the logit space.", "questions": "1. Can authors quantify the relationship between measured probability sparsity and empirical robustness (e.g., correlation between entropy and adversarial accuracy)?\n2. How does NAT perform under adaptive attacks specifically designed to exploit the auxiliary adversary structure?\n3. Could the “probabilistic sparsity” be approximated directly (e.g., via entropy regularization) instead of adversary-based NAT?\n4. How sensitive is NAT to adversary model mismatch or overfitting? Would a partially shared backbone improve stability?\n5. Does the spatial metric benefit persist for non-classification tasks (e.g., detection or segmentation)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper use “nasty training” as a robustness regularizer.\n2. This paper have solid theoretical reasoning and thorough experimental validation.\n3. This method improves robustness while maintaining efficiency and interpretability.\n4. This paper have good exposition of probability sparsity and its spatial interpretation.\n5. This method can be easily integrated into existing AT pipelines with negligible cost.", "weaknesses": "1. The Taylor expansion and sparsity explanation rely on approximations; formal proofs or bounds are lacking.\n2. The “spatial metric” benefits are qualitatively visualized but lack quantitative metrics (e.g., explicit margin distributions).\n3. Only standard PGD/CW/AA considered — might not generalize to adaptive threat models.\n4. The authors compare to entropy/logit norm regularization briefly but not in depth.\n5. The term “nasty” may be unconventional for robustness literature and could obscure broader relevance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dPcOwQ8ha3", "reviewer": "Reviewer_gn7C", "summary": "This paper introduces a method that integrate the nasty training (NT) into AT to strengthen robustness. They further analyze the probability sparsity of NT which has potential in adversarial training. And the experiments show the effectiveness of the method.", "questions": "1.\tCould you add some up-to-date SOTA methods to make a more comprehensive comparison?\n2.\tCould you present the mean and standard deviation of your method at least in main table? And could you change the class index in Figure 5 to integer?\n3.\tNAT utilizes a VT for training. For the fairness, does NAT cover the expenses of the VT in Table 8?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. This paper analyzes the probability sparsity from NT and combined it with AT, by increasing decision boundary margins result in a higher cost for adversarial attacks, which shows the promising performance. And has in-depth theoretical interpretable analysis.\n2. The experiment is comprehensive with CNN and ViT under different white and black attacks. And the ablations show the sparsity and effectiveness.", "weaknesses": "1. The main results in Table 1 and Table 2 were not compared with some SOTA method, such as LWTA [1], IKL-AT [2] and DCS [3].\n2. All reported results correspond to the best outcomes over three independent runs, but there is no report of the mean and standard deviation of the results. And the class index in Figure 5 should be integer.\n\n[1] Stochastic local winner-takes-all networks enable profound adversarial robustness, 2021.\n\n[2] Decoupled kullback-leibler divergence loss, NeurIPS 2024.\n\n[3] Adversarial Robustness via Deformable Convolution with Stochasticity, ICML 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TXa9zWv5Uk", "reviewer": "Reviewer_AFqW", "summary": "The paper proposed a method for improving adversarial training which incorporates nasty training. The method uses two networks: a target network which we would like to be more robust to adversarial example and a vanilla trained network. The target network is trained to be a \"nasty teacher\" version of the vanilla trained network. The intuition is that this keeps the top-1 prediction correct but sparse so top-N predictions are very low probability and therefore far away in the decision space of the model. This should make the model robust to perturbations of the input. The paper provides interesting theoretical analysis of the solution and empirical results show that it does work.", "questions": "This was a very interesting paper which I think makes a good contribution. The idea is new and it's an insightful application of nasty training which makes sense from the theoretical motivation in the paper. The primary thing I think is missing is more/more convincing results. Based on the discussion I expected there to be a pretty clear improvement from the proposed method but on the presented CIFAR results, it didn't look like a dramatic change. Also I am not sure that CIFAR results are reflective of real conditions so the method should really be tested on something more comprehensive. It also it wasn't clear to me from the paper how much longer it takes to incorporate the proposed training loop after already going through adversarial training on the target model.\n\n**Specific Questions** \n- Does the method work on datasets beyond CIFAR? ImageNet for example?\n- What is the total retraining time for this method? Both with and without the required adversarial training that occurs before the nasty training.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Interesting and new idea: nasty training was intended for another purpose so its a good application\n- Good analysis: the intuitive and formal analysis was interesting and mostly convincing\n- Mostly good results: based on the results the method clearly does work", "weaknesses": "- The results seem to focus on CIFAR datasets which may not be fully representative of real world conditions (See Maiya et al. \"Unifying the Harmonic Analysis of Adversarial Attacks and Robustness\")\n- No analysis of training time", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)