Long-tailed Adversarial Training with Self-Distillation
Authors: Seungju Cho
Hongsin Lee
Changick Kim
Venue: ICLR 2025 Poster (2025)
Keywords: Adversarial Robustness
Adversarial Training
Long-Tail Distribution Learning
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Adversarial training significantly enhances adversarial robustness, yet superior performance is predominantly achieved on balanced datasets.
 Addressing adversarial robustness in the context of unbalanced or long-tailed distributions is considerably more challenging, mainly due to the scarcity of tail data instances. 
 Previous research on adversarial robustness within long-tailed distributions has primarily focused on combining traditional long-tailed natural training with existing adversarial robustness methods.
 In this study, we provide an in-depth analysis for the challenge that adversarial training struggles to achieve high performance on tail classes in long-tailed distributions.
 Furthermore, we propose a simple yet effective solution to advance adversarial robustness on long-tailed distributions through a novel self-distillation technique.
 Specifically, this approach leverages a balanced self-teacher model, which is trained using a balanced dataset sampled from the original long-tailed dataset.
Our extensive experiments demonstrate state-of-the-art performance in both clean and robust accuracy for long-tailed adversarial robustness, with significant improvements in tail class performance on various datasets.
We improve the accuracy against PGD attacks for tail classes by 20.3, 7.1, and 3.8 percentage points on CIFAR-10, CIFAR-100, and Tiny-ImageNet, respectively, while achieving the highest robust accuracy.

Published as a conference paper at ICLR 2025
LONG-TAILED ADVERSARIAL TRAINING WITH SELF- DISTILLATION
Seungju Cho', Hongsin Lee", Changick Kim
Korea Advanced Institute of Science and Technology (KAIST) (joyga, hongsinC4, changick @kaiskkkkkitt..... ac.
ABSTRACT
Adversarial training significantly enhances adversarial robustness, yet superior performance is predominantly achieved on balanced datasets. Addressing ad- versarial robustness in the context of unbalanced or long-tailed distributions is considerably more challenging, mainly due to the scarcity of tail data instances. Previous research on adversarial robustness within long-tailed distributions has primarily focused on combining traditional long-tailed natural training with exist- ing adversarial robustness methods. In this study, we provide an in-depth analysis for the challenge that adversarial training struggles to achieve high performance on tail classes in long-tailed distributions. Furthermore, we propose a simple yet effective solution to advance adversarial robustness on long-tailed distributions through a novel self-distillation technique. Specifically, this approach leverages balanced self-teacher model, which is trained using balanced dataset sam- pled from the original long-tailed dataset. Our extensive experiments demonstrate of-te-o-thee art performance in both clean and robust accuracy for long-tailed ad- versarial robustness, with significant improvements in tail class performance on various datasets. We improve the accuracy against PGD attacks for tail classes by 20.3, 7.1, and 3.8 percentage points on CIFAR-10. CIFAR-100, and Tiny- ImageNet, respectively, while achieving the highest robust accuracy.
INTRODUCTION
Recent studies have highlighted the vulnerabilities inherent in deep learning models when subjected
to adversarial attacks (Goodfellow et al. 2014 Carlini & Wagner 2017] Madry et al., 2017 Athalye
eet al. 2018). These attacks exploit subtle changes in input data that can lead to drastically ll incor-
rect predictions, undermining model reliability in critical applications (Ma et al. 2021 Grigorescu
al. 2020 Wang et al.) 2023). As result, research efforts have focused on enhancing robustness
against such adversarial threats, with various strategies being explored (Das et al. 2017 Xie et al.)
2019 Cohen et al.] [2119] Carmon et al. [219)] Zhang al. [022]] Jin al. (023). 3. Among these,
adversarial training (Goodfellow et al. 20144] Madry et al. 2017 has proven to be one of the most effective methods for enhancing model robustness (Pang et al. 2020 Bai et al., 2021a Wei et al., 2023). However, many existing studies primarily validate their approaches on balanced datasets, overlooking the practical scenarios where data is inherently imbalanced or long-tailed. This gap underscores the need for novel adversarial training methodologies capable of addressing these more complex data distributions effectively.
While numerous studies (Cao al. 2019a Cui et al. 2019 Kang al. 2019 Zhou et al. 2020
Li et al. 2021 Alshammari et al. 2022] Du et al. (023) have addressed long-tailed distributions without considering robustness, the intersection of adversarial training and long-tailed distributions (Wu et al. 2021 Li et al.) 2023) Yue et al. (202) has received far less attention. Existing research in this area primarily combines traditional long-taile tailed classification techniques with basic adversarial
training methods, such as PGD adversarial training (Madry et al. 2017) and TRADES loss (Zhang al. 2019) with balanced softmax (Wu et al. 2021: Yue et al. 2024). Despite combining such methods, existing approaches still demonstrate low performance on tail classes with fewer samples in long-tailed distributions. We find that their high robustness primarily stems from the improved
"These authors contributed equally.

---

Published as a conference paper at ICLR 2025
19
Ours
Ours
AWP
Atto accuracy (96)
Tail PGD accuracy (%)
18
16
17
MART
AT-8SL
RoBal
REAT
16
TRADES
REAT AT-BSL
PGD-AT
RoBal TRADES
PGD-AT
38
44
26
31
Clean accuracy (%)
Tail clean accuracy (%)
(a) Performance on all classes
(h) Performance on tail classes
Figure 1: (a) The overall clean accuracy and AutoAttack (Croce Hein 2020 accuracy of vari- ous adversarial training methods (blue circles) and long-tailed adversarial training methods (green circles) using the ResNet-18 (He et al. 2016a) architecture on CIFAR- -0---- (Krizhevsky et al.) 2009). (b) The clean accuracy and 20-step PGD attack (Madry et al. 2017) accuracy on tail classes for the same set methods. Our method (yellow star) surpasses all existing methods, achieving notable improvement tail classes.
robustness of head classes, which have larger number of samples. This highlights the need for more advanced research adversarial training in long-tailed distributions.
In this paper, we provide an in-depth analysis of why adversarial training in long-tailed distributions particularly challenging, focusing on the performance on tail classes. Through theoretical anal- ysis, we show that adversarial training causes more severe performance degradation in tail classes compared natural training. This highlights the inherent difficulty of achieving high robustness in long-tail distributions, especially for the tail classes. Building on these insights, we propose novel two-step framework designed improve tail class robustness under adversarial training on long-tailed distribution.
Our framework consists of constructing a balanced dataset from a given unbalanced dataset and employing seff-distillation. We first create sub-dataset where each class contains equal number of data samples, referred to as the balanced sub-dataset. Then, we adversarially train a self-teacher model on this balanced dataset, achieving higher robustness in tail classes than models trained on the full long-tailed tailed dataset. Subsequently, we apply self deff-ditillation using the balanced self-teacher model to improve tail class performance, resulting in significant gains baseline models. As shown in Figure Ta our method achieves the highest accuracy against AutoAttack (Croce & Hein 2020) and demonstrates significant performance improvements, particularly tail classes in Figure
Here are our key contributions:
We conduct an in-depth analysis to explain why adversarial training on long-tailed datasets results poor tail class performance. Our findings show that, despite adversarial training, tail class robustness is even lower than natural training.
Based on these insights, we propose a novel two-step adversarial training approach specif- ically designed for long-tailed datasets. This method improves upon basclines that merely combine existing long-tailed classification techniques with adversarial training.
Our approach achieves state-of-the performance in adversarial training on long-tailed datasets across various architectures, datasets, and imbalance ratios. leading to significant enhancements both clean and robust accuracy, with particularly notable improvements tail classes.
2

---

Published as a conference paper at ICLR 2025
2
RELATED WORKS
2.1 ADVERSARIAL TRAINING AND DISTILLATION
In response adversarial attacks (Goodiellow et al. 2014 Carlini Wagner 2017 Madry et al. 2017 Athalye et al.) 2018), adversarial training (Goodfellow et al., [014]] Madry et al. (017) em- pirically stands out as one of the most effective. Adversarial training defines optimization as a min- max problem, where inner maximization generates adversarial inputs, and outer minimization trains the model on these adversarial samples. TRADES (Zhang et al. 2019) incorporates the Kullback- Leibler (KL) divergence loss between the logits of clean and adversarial images. MART (Wang) ct al. 2020 introduces per-sample weights based on the confidence of each sample. These two methods are used baseline methods for other recent adversarial training research (Qin et al.
2019 Wu et al. 2020: Bai et al. 2021b: Jin et al. 2022 Tack et al. 2022 Jin et al. 2023 Wei et al, 2023
The superior performance of adversarial training is primarily observed in large architecture net- works, motivating research efforts to improve performance in smaller architectures using techniques such as distillation. Adversarial Robust Distillation (ARD) (Goldblum et al.) 2020) proposes a loss function that guides the adversarial output of the student model towards the natural output of the teacher, similar to TRADES (Zhang et al. 2019). Robust Soft Labels Adversarial Distillation (RSLAD) (Zi et al. )021) leverages teacher logits to improve performance through inner maxi- mization in adversarial training. Many other studies leverage the teacher's logits (Zhu et al.) 2021 Maroto et al. [022] HHung et al.] 2023) and gradients (Lee et al. (023) to distill robustness into the
student model.
While these adversarial training and distillation studies have achieved strong robustness, they have only been conducted on balanced datasets where each class has equal number of samples. This differs significantly from the real-world data configurations we encounter, highlighting the necessity of adversarial training or distillation for unbalanced datasets.
2.2 LONG-TAILED RECOGNITION
Extensive research has been conducted to address the performance imbalance inherent in -oilet distribution datasets. Prominent methods include oversampling the minority tail data (Chawla la al.) 2002 Han et al.) 2005) and increasing the weight of the minority classes (Cui et al.) 2019 Zhhan al. 2021). D. Although these methods are intuitive, they pose a risk of overlitting on the tail classes and can degrade feature extraction performance (Kang et al. 2019 Zhou et al. (0020). A more effec- tive approach, decoupled learning (Kang et al., 2019 Zhou et al. 2020 Alshammari et al. 2022), separates feature learning from classification mitigate such issues. Moreover, logit compensa- tion methods have been proposed recently, introducing relatively larger margins between different
classes based on prior class frequencies (Cao al. 2019a Kang al. 2019 Menon et al. 2020
Ren et al. 2020 Tan et al.) 2020).
2.3 LONG-TAILED ADVERSARIAL TRAINING
RoBal (Wu et al., 2021 the first paper to address adversarial robustness long-tailed classifica- tion. RoBal applies adversarial training with TRADES regularization (Zhang et al. 2019) alongside long-tailed techniques such as balanced softmax (Ren et al., 2020) and class-aware margin (Cao) et al. 2019b). Moreover, it provides detailed insights into which modules are effective for long- tailed adversarial training. REAT (Li et al. 2023) aimed to achieve balanced performance by uti- lizing class-wise weights to generate adversarial examples and expanding the feature space of tail class data. AT-BSL (Yue et al. 2024) revisited the RoBal paper to analyze the necessity of various modules and concluded that only the balanced softmax loss (BSL) is sufficient without the need for
complex modules as follows:
c Σ, ezitbr
CBsL(f(z'). log
where a' is an adversarially perturbed input of æ. 2f = f(x'); is i-th logits of the adversarial input, by log(n:), 7 is hyperparameter and n, is the number of examples in the i-th class. The
3

---

balanced softmax is a commonly used loss in addressing long-tail tail problems to boost the performance of tail classes (Ren et al. 2020). However, its drawback lies in adjusting the importance of tail classes based the number of data. In other words, more than balanced softmax is needed to Published as a conference paper at ICLR 2025
address robustness concerns for tail classes adequately.
There has been in-depth analysis of robustness degradation in tail classes compared to head classes adversarially robust long-tailed distributions. In this paper, we aim to improve the overall performance of existing long-tailed adversarial training by achieving sufficient robustness of tail classes.
ANALYSIS
3.1 PRELIMINARY
Let f represent the classification model, which maps the input data space X' to the output label space V, Lee. f X - y. For specific instance of X and y, we use a E X and y y and (21 , where is the dimension of æ.
Definition 1. For a classifier f(), the overall standard evoor Real.) of classifier f(.) is defined as Raalf) = Pr(f(x) + v),
and its robust error Rob() ) is
Rm) = Pr(36 with 110||0 S e s.t. f(x + 8) + y)
where Pr(.) means probability and e a non-negative perturbation boundary.
For simplicity, we denote by (.at(.) the natural classifier that minimizes standard error, and by frot(.) the robust classifier that minimizes robust error. Additionally, we denote the standard error and robust error for a given class k as Rea and R. (f), respectively.
3.2 THEORETICAL ANALYSIS
Let the long-tain dataset for a binary classification task data S with imbalance ratio r 2 1, i.e., the ratio of the number of instances in the head class = +1) the number of instances in the tail class (y - 1) is r. We assume Gaussian mixture distribution, which is similar to Xu et al. (2021); Lee 2024) follows.
+1,
W.P F
Lid N 1),
1 w.p
21.
24
where 7) is a constant that determines the standard deviation of the Gaussian distribution. We address binary classification problem on the above dataset, and then we obtain the following linear function (w, with weight w with bias b.
fw)((x) sign
Σ wkaF +b)
(3)
1R-1
Figure 2: Logistic regression on binary data Equation with different imbalance ratio (IR).
4

---

Published as a conference paper at ICLR 2025
According to Lemma I and Lemma 2 Appendix A each weight w1, u2..... wn of optimal (nat- ural. robust) classifier has the same weight, L.e. W1 = W2 = =... wn. We derive the standard and robust error for the tail class of each optimal classifier as follows.
Theorem 1. For a data distribution S. the optimal natural classifier Shat and robust classifier Frob exhibit the following standard and robust errors for the tail class -1 with perturbation margin O <E < v,, respectively:
Rid (Frat) = (-vion+ hr
-Vvi(n -c) -c++ In Redh (rrot) = = (-V7(1
(fret)) =
(4)
2/10
2//nn/
Inr
lnr
R Reac (Frot) = (-vinn 2v1(n -e)
6) -)++ 2/7(1 (5)
A detailed proof of Theorem D can be found in Appendix [ From Theorem we can easily infer that both the natural and robust errors of the tail class for both the natural and robust classifiers increase monotonically with respect to the imbalance ratio r. Building upon this, we present the following corollary:
Corollary 1. Adversarial training on long-tailed datasets exacerbates the vulnerability of the tail class, making them even less robust than under natural training:
Rids (fob))
RRob
A proof of Corollary is trivial according eq. 4 and eq. (5) in Theorem !
3.3 EMPIRICAL ANALYSIS
In Figure 2 we present a visualization of the theoretical
100
analysis in 2-dimensional space. The data were sam- pled from Gaussian distributions with = 1 and n = 2 following Equation (2, considering three different imbal-
ance ratios (IR=1, 2, 5). The figure highlights the decision boundaries formed by both natural and adversarial train- ing (c = 0.5). As the imbalance ratio increases, the deci- sion boundary of the adversarially trained model becomes more distorted, reflecting the model's increased sensi- tivity to adversarial perturbations in the minority class.
Jssows the test accuracy of the tail class (y = 11) of the logistic regressions. The results indicate a clear trend: the imbalance ratio grows, the test accuracy for the tail class drops across both natural and adversar- ial training scenarios. More notably, adversarial train- ing consistently produces more robust errors than natural training. These results align with theoretical predic- tions: imbalanced data amplifies clean and robust errors for the tail class, and adversarial training further exacer- bates robust errors.
Adversarial Vain hatural lest
batural Than Advenanisl
Adversorial Tain Adversorial Test
imbalance
Tail class natural and robust accuracy with respect to natural and ad- versarial training with different imbal- ance ratios (IR) in Figure 2
Figure 3:
We further experiment long-tailed CIFAR-
10 dataset on various imbalance ratios. We con- duct both standard and adversaraalia train- ing and compare the robust accuracy against PGD attack with 6 2/255 of the entire and tail classes. As shown in Figure 4aj the ro- bust performance of the adversarially-trainedd model across various imbalance ratios is su- perior to that of the naturally-trainer model, which shows trivial results. However, in Fig- ure [bb] the natural model exhibits better ro- bust performance than the adversarially-trained
Accuracy (95)
Accuracy (%)
30
16
10
Imbalance Ratio (I)
Imbalance Ratio (IR)
(a) Whole classes.
(b) Tail classes.
Figure 4: Clean and robust accuracy of natural and robust models.
5

---

Published as a conference paper at ICLR 2025
Algorithm Main Algorithm
Input: Long- tailed dataset D, batch size N, epochs T. learning rate hyperparameters a, 7 and balanced self-teacher training parameters (batch size NB. epochs TB, and learning rate μB) Robust model on long-tailed dataset 'D
Output:
balanced self-teacher training
Make balanced dataset DB by re-sampling D with hyperparameter " Randomly initialize 8g. the weights of balanced self-teacher fB for epoch to Ta do for mini batch { ye Iili in DB do for i to NB do x= (G y:)
PGD attack
end for
8B +- -MB Nu
ΣN Vog (c
end for end for
# main training Randomly initialize 8, the weights of training model f for epoch to T do for mini batch (æc,, y: W in do for N do
x DG, y:) end for
PGD attack
1 E V [nsnn(((a)))))
Balanced softmax loss
ta (M-CkD(((E)),f(t)))
Self-distillation
end for end for
model in the tail classes. This experiment supports the results of Corollary D which clearly demon- strates that while adversarial training generally helps to improve robustness, it could exacerbate performance degradation in the tail classes of long-tailed distributions.
METHOD
We examine the impact of unbalanced datasets on performance disparity, particularly noting that this disparity becomes more pronounced during robust training compared to natural training. To address this issue, we propose simple yet effective self-distillation framework.
4.1 LIMITATIONS OF EXISTING BALANCED SOFTMAX APPROACHES
Balanced softmax (Wu et al. 2021 Yue al. 2024 is powerful method that effectively addresses the issue of tail-class robustness under adversarial training. These works demonstrate that applying Balanced Softmax improves tail-class robustness. However, shown in Table 6 while Balanced Softmax prevents the robustness of tail classes, still falls short compared to naive PGD training on balanced dataset in terms of tail-class robustness. Moreover, our experiments in Table [ Table and Table 3 demonstrate that tail-class robustness under adversarial attacks remains notably lower than that of head classes. This observation underscores the necessity of additional strategies to explicitly improve tail-class robustness.
4.2 TRAINING SELF-TEACHER TO GUIDE TAIL CLASS ROBUSTNESS
To address the limitations of Balanced Softmax and improve tail-class robustness, we construct a balanced sub-dataset D3 by up-sampling tail classes and down-sampling head classes. Specifically, suppose the number of samples each class is n, < n2 < < nc. then we construct dataset where each class contains 7 ny where 7 > is a hyperparameter of adjusting the

---

Published as a conference paper at ICLR 2025
Table 1: The clean accuracy and robustness for various algorithms using ResNet-18 on CIFAR-IO- LT. T-Clean and T-PGD are clean and PGD-20 accuracy on tail class.
Best Checkpoint
Last Checkpoint
Method
Clean
PGD)
AA
T-Clean
T-PGD
Clean
PGD
AA
T-Clean
T-PGD
PGD-AT
52.71
29.30
27.57
12.7
1.0
56.39
26.98
25.81
20.8
2.2
TRADES
45.79
28.66
27.01
6.7
0.8
47.10
28.00
26.45
6.4
0.6
MART
44.03
29.36
27.59
5.0
0.5
47.33
28.08
26.55
10.9
1.0
AWP
51.69
32.42
30.35
5.3
0.2
51.89
32.42
30.35
10.9
0.6
RoBal
70.54
35.33
28.83
70.4
33.1
72.80
28.04
25.00
67.7
15.9
REAT
68.34
35.98
32.45
69.5
29.1
68.32
28.67
26.68
55.7
11.6
AT-BSL
68.43
35.87
32.27
63.2
22.0
67.60
29.40
27.46
50.1
8.7
Ours
70.81
38.85
34.32
73.8
36.9
71.74
37.80
33.74
74.7
36.2
number DB. Using DB, we perform robust training with PGD, resulting in a self-teacher model that is more robust to tail classes compared to models trained on imbalanced datasets. The balanced self-teacher transfers its tail robustness to student model via adversarial knowledge distillation Lee et al. (202)), detailed in Algorithm Through this process, the proposed method addresses the insufficient tail-class robustness Balanced Softmax, enhancing the model's 's robustness on tail classes while maintaining overall robustness.
EXPERIMENTS
5.1 EXPERIMENT SETTINGS
Dataset. We conducted experiments using long-tailed distribution datasets: CIFAR-IO-LT, CIFAR-
100-LT (Krizhevsky et al. 2009) and Tiny-ImagNNet-LI (Le & Yang 2015), with various imbal- ance ratios (IR), primarily set at 50 for CIFAR-IO-LT, 10 for CIFAR- LT and Tiny- Iny-ImageNett-TT. Random crop and random horizontal flip were applied, while other augmentations were not utilized unless specified.
Training details. We employed ResNet-18 (He et al. 201 and WideResNeet33410 (Zagoruyko & Komodakis (016) architectures for CIFAR- and results for WideResNet-344 10 are included the appendix. For Tint-ImageNet-LT,, we employed PreActResNet-118 (He et al. 2016bb). Initially, we trained balanced self-teacher using the same model architecture for 30 epochs using a batch size of 32 with a balanced dataset, resampled by the original long-tailed dataset with y = IR/2. In the main training phase, trained for 100 epoch using batch size of 128 with self- distillation from the balanced self-teacher. We utilized SGD optimization to train both the balanced self-teacher and the main model, setting the learning rate 0.1 and weight decay to 5 10-1 1 We used an epsilon boundary of 8/255, a commonly used setting in adversarial training, and employed 10-step PGD attack during training.
Comparison models. As comparison models, we utilized PGD-AT (Madry et al. 2017). TRADES (Zhang al. 2219), MART (Wang al. 2020), and AWP (Wu et al. 2020). representing prominent approaches of AT. Additionally, we followed RoBal (Wu et al. (021). REAT (Li et al. 2023). and AT-BSL Yue et al.] 2024). which focus on long-tailes adversarial training. For long-tailed AT implementation, we meticulously followed the setting of existing methods such learning rate, batch size, weight decay, etc.
Evaluation. Evaluation metrics included clean accuracy, accuracy under 20-step PGD attack, and AutoAttack (AA) accuracy (Croce Hein 2020). Additionally, we assessed clean and 20-step PGD attack accuracy specifically for tail classes, denoted as T-Clean and T-PGD, respectively. In 10-LT, the performance evaluation focused on the last class, while
and Tiny- ImageNet-LT evaluated the performance of the tail 10 and 20 classes of 100 and 200 classes, respectively. We measured performance at both the best and last epoch based on the accuracy under the 20-step PGD attack.

---

Published as a conference paper at ICLR 2025
Table The clean accuracy and robustness for various algorithms using ResNet- 18 on CIFAR-100- LT. T-Clean and T-PGD are clean and PGD-20 accuracy on the tail class group.
Best Checkpoint
Last Checkpoint
Method
Clean
PGD
AA
T-Clean
T-PGD
Clean
PGD
AA
T-Clean
T-PGD
PGD-AT
42.73
17.31
15.44
20.4
7.7
43.09
15.07
14.05
22.4
6.7
TRADES
38.83
19.05
16.06
16.5
7.3
39.63
18.86
16.18
16.1
MART
38.57
19.90
17.10
16.6
7.7
40.31
17.07
15.21
19.5
7.2
AWP
40.46
21.85
18.58
16.2
8.5
40.15
21.71
18.33
16.2
8.6
RoBal
44.27
19.67
16.78
18.4
8.0
46.46
16.28
14.73
23.3
6.7
REAT
45.73
18.22
15.82
32.2
11.4
45.53
15.64
14.27
33.0
10.5
AT-BSL
45.38
18.04
15.73
33.1
12.4
45.48
15.36
14.07
31.5
9.1
Ours
46.13
22.42
18.73
38.9
17.9
47.22
21.82
18.53
37.9
17.6
Table 3: The clean accuracy and robustness for various algorithms using PreActResNet-18 on Tiny- ImageNet-LT.
Best Checkpoint
Last Checkpoint
Method
Clean
PGD
AA
T-Clean
T-PGD
Clean
PGD
AA
T-Clean
T-PGD
PGD-AT
34.89
14.17
10.98
15.8
5.4
36.55
9.69
8.45
23.2
5.0
TRADES
33.76
13.71
10.00
15.4
6.4
32.97
12.50
9.61
16.0
6.0
MART
31.15
15.45
11,94
14.4
6.0
32.91
12.42
10.32
17.8
7.6
AWP
32.28
15.09
11.27
14.2
5.6
32.13
13.95
11.10
14.4
6.6
RoBal
35.25
14.01
10.44
15.0
4.4
37.97
10.51
8.64
21.8
4.8
REAT
38.37
15.25
11.99
33.0
12.6
38.48
10.58
9.07
31.6
8.8
AT-BSL
38.38
15.39
11.85
30.8
13.0
38.41
10.25
8.90
32.4
7.0
Ours
38.44
17.02
12.57
36.6
16.0
49.37
14.09
11.15
33.8
12.6
5.2
MAIN RESULTS
We demonstrated excellent performance across all datasets, including CIFAR-10-LT, CIFAR-100 100- LT, and Tiny-Imageeee the WideResNeet34-110 architecture can be found in Table Table the appendix. Particularly noteworthy is the substantial improvement performance for tail classes. When adversarial training methods such PGD-AT, TRADES, MART, and AWP naively applied long-tailed datasets, overall performance remains reasonable compared to Robal, REAT, AT-BSL, but performance for the tail classes notably suffers. For instance, while AWP exhibits superior performance compared to RoBal, the clean accuracy and robust accuracy for tail classes are significantly low. Long-tailed adversarial training methods such RoBal, REAT, and AT-BSL. show considerable improvement in tail class performance compared to other adversarial training methods. However, when compared to the performance of the entire class, is still evident that the performance remains imbalanced. In contrast, our method shows significant improvement in the performance of the tail classes, resulting minimal difference compared to the performance of the entire classes. Additionally, we achieved overall better performance than the baseline at both the best and last checkpoints.
[able Table and Table respectively. The experimental results on
5.3 ABLATION
In this section, we conduct further experiments to corroborate our main contribution.
5.3.
AUGMENTATION
Following the inclusion of various augmentation experiments outlined in AT-BSL, conducted experiments applying RandAugment (RA) (Cubuk et al.] (0200) and AutoAugment (AuA) (Cubuk) et al. 2019) in Table 4 While applying augmentation led to overall performance improvements, the
8

---

Published as a conference paper at ICLR 2025
Table The clean accuracy and robustness with augmentation using ResNet-18 CIFAR-I00-LT. T-Clean and T-PGD are clean and 20-step PGD accuracy on the tail class group.
Best Checkpoint
Last Checkpoint
Method
Clean
PGD
AA
T-Clean
T-PGD)
Clean
PGD
AA
T-Clean
T-PGD
Robal
44.27
19.67
16.78
18.4
8.0
46.46
16.28
14.73
23.3
6.7
Robal-RA
44.64
20.11
17.02
15.8
7.7
47.62
18.63
16.05
19.2
7.4
Robal-AuA
45.87
20.24
17.05
17.4
6.7
47.42
19.32
16.30
18.2
7.4
Reat
45.38
18.04
15.73
33.1
12.4
45.48
15.36
14.07
31.5
9.1
Reat-RA
46.94
21.71
18.02
33.3
14.9
50.41
20.33
17.46
36.6
14.7
Reat-AuA
47.86
23.09
19.43
34.0
16.7
50.56
22.20
18.60
36.5
16.5
AT-BSL
45.38
18.04
15.73
33.1
12.4
45.48
15.36
14.07
31.5
9.1
AT-BSL-RA
48.38
22.18
18.58
34.7
16.8
50.33
20.29
17.42
37.1
14.6
AT-BSL-AuA
47.30
22.78
18.66
34.2
16.4
50.57
21.98
18.45
36.8
16.3
Ours
46.13
22.42
18.73
38.9
17.9
47.22
21.82
18.53
37.9
17.6
Ours-RA
48.78
23.58
19.30
34.9
17.2
50.98
22.43
18.80
38.2
16.9
Ours-AuA
50.14
24.60
20.08
36.4
18.0
50.46
24.32
20.28
37.6
18.5
Table The clean accuracy and robustness with different Imbalance Ratio(IR) using ResNet- on CIFAR-100-LT. T-Clean and T-PGD are clean and 20-step of PGD accuracy the tail class group.
Best Checkpoint
Last Checkpoint
IR
Method
Clean
PGD
AA
T-Clean
T-PGD
Clean
PGD
AA
T-Clean
T-PGD
RoBal
33.52
14.56
12.27
1.9
0.9
34.81
12.16
11.02
4.6
1.7
REAT
26.62
13.73
10.64
9.2
4.1
36.51
12.28
11.15
19.1
4.4
AT-BSL
30.06
13.80
10.91
10.3
4.2
36.46
12.07
11.21
17.8
4.4
Ours
38.09
16.65
13.58
14.8
5.2
38.56
16.08
13.52
19.2
5.8
RoBal
40.08
16.91
14.28
12.1
5,4
41.28
13.96
12.70
15.5
5.1
REAT
33.17
15.82
13.01
20.0
9.2
41.73
13.79
12.58
29.8
7.7
AT-BSL
41.70
15.51
13.62
30.7
10.4
41.41
13.49
12.48
27.1
7.9
Ours
42.54
19.66
16.36
33.2
13.3
42.64
19.24
15.97
32.5
13.7
RoBal
44.27
19.67
16.26
16.7
7.6
46.46
16.28
14.73
23.3
6.7
REAT
45.73
18.22
15.82
34.4
12.2
45.53
15.64
14.27
33.0
10.5
10
AT-BSL
45.38
18.04
15.73
33.1
12.4
45.48
15.36
14.07
31.5
9.1
Ours
47.22
21.82
18.53
37.9
17.6
47.22
21.82
18.53
37.9
17.6
RoBal
49.49
21.66
18.59
26.1
11.9
51.56
18.15
16.58
34.7
9.9
REAT
49.48
21.95
18.98
39.3
18.3
49.76
18.19
16.65
40.7
14.1
5
AT-BSL
49.75
21.53
18.65
42.3
18.2
49.41
18.12
16.56
40.0
13.7
Ours
50.77
24.13
20.10
44.0
20.9
51.92
25.00
21.14
43.8
20.4
best results were achieved when augmentation was applied to our method. Our method consistently outperformed baselines robustness with augmentation setting including tail class performance with augmentation.
5.3.2 DIFFERENT IMBALANCE RATIO
In Table 5 we conducted experiments using different imbalance ratios (IR). As the IR increases, the number of tail classes decreases, leading to decreased robustness. In all cases, method outperforms the baseline in terms of both overall and tail robustness. This indicates that our proposed framework generally performs well across different IR settings.

---

Published as a conference paper at ICLR 2025
The clean accuracy and robustness using ResNet-18 on CIFAR-I00-LT. T-Clean and T- PGD represent clean and 20-step PGD accuracy on tail class.
Table 6:
Dataset
Method
Clean
PGD
T-Clean
T-PGD
Robal
44.27
19.67
18.4
8.0
D
Reat
45.73
18.22
32.2
11.4
BSL
45.38
18.04
33.1
12.4
DB
PGD-AT
35.71
14.94
34.6
14.1
5.3.3 EFFECT OF BALANCED SUBSET
To evaluate the effect of the balanced subset DB. we compare the performance of mod- els trained with PGD-AT on Dg against Robal, Reat, and BSL trained on D, which in- corporate techniques like balanced softmax to address long-tailed distributions. For simplic- ity, we denote PGD-AT D as the model trained with PGD-AT on the balanced subset DB. As shown in Table 6 while the PGD-AT model trained on DB achieved the lowest overall per- formance, it demonstrated the best results for tail classes, T-Clean and T-PGD. This suggests that even with fewer training epochs, the balanced subset is effective for improving performance on tail classes. In Figure [5, apply the same methods with dif- ferent teachers where the performance is summa- rized in Table 6 Interestingly, the best results were achieved when we utilize PGD-AT Dg as the teacher model. despite having the lowest overall per- formance. Specifically, the performance on the tail classes highlights the effectiveness of the teacher trained on the balanced subset, as demonstrates superior performance on the tail class compared to other methods. This underscores the utility of the balanced subset improving tail class performance, Additionally, although trained the teacher for self-dist distilstillatoon using a simple method, PGD-AT, de- veloping more effective teacher remains area for future work.
Accuracy (86)
38
110 DCC
Teacher
Figure 5: Clean and robust accuracy cording to different teachers. Robal, Reat, and Bsl were trained with 100 epochs, while PGD-AT D used teacher trained with 30 epochs.
6 CONCLUSION
Building the observation that adversarial training methods inherently struggle with tail classes, we propose effective strategies to address the lower performance on these classes. We first train balanced self-teacher and subsequently perform knowledge distillation from this self-teacher. This approach leads to significant improvements in long-tailes adversarial training, enhancing both over- all robustness and tail class robustness.
Discussion It is well known that adversarial training varies in difficulty across classes, and perfor- mance also differs by class. This presents fairness issue, indicating that in robustness, not only the number of data points but also the intrinsic difficulty of each class plays role. While this paper focuses solely on data quantity, designing robust models that account for class-level fairness remains an area for future work.
REFERENCES
Shaden Alshammari, Yu- Yu-Xiong Wang. Deva Ramanan, and Shu Kong. Long- Long-tailed recognition via weight balancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6897-6907, 2022.
10

---

Published as a conference paper at ICLR 2025
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity: Circumventing defenses to adversarial examples. In International conference on machine learning, pp. 202-2742222 283. PMLR, 2018.
Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. Recent advances in adversarial training for adversarial robustness. arXiv preprini arXiv:2102.013:: 2021a.
Yang Bai, Yuyuan Zeng, Yong Jiang, Shu-Tao Xia, Xingjun Ma, and Yisen Wang. Improving adversarial robustness via channel-wiss activation suppressing. arXiv preprint arXiv:2103.08307, 2021b.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with abbel-istributtin--aarr margin loss. Advances neural information processing systems, 32, 2019a.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with tabel-distribution-aaar margin loss. Advances neural information processing systems, 32, 2019b.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. leee, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. Advances neural information processing systems, 32, 2019.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206- 2216. PMLR, 2020.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp- 1 113-123, 2019.
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 702-703, 2020.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. on effective number samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2268-9277, 2019.
loss based
Nilaksh Das, Madhuri Shanbhogue, Shang- Tse Chen, Fred Hohman, Li Chen, Michael E Kounavis, and Duen Horng Chau. Keeping the bad guys out: Protecting and vaceinating deep learning with jpeg compression. arXiv preprint arXiv: 10t.::/000000000 2017.
Fei Du, Peng Yang, Qi Jia, Fengtao Nan, Xiaoting Chen, and Yun Yang. Global and local mix- ture consistency cumulative learning for long-tailed visual recognitions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15814 158233, 2023.
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In
Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.
2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3):362-386 2020.
11

---

Published as a conference paper at ICLR 2025
Hui Han, Wen- Yuan Wang. and Bing-Huan Mao. Borderline smote: new samp-ingg method in imbalanced data sets learning. International conference on intelligent computing, pp. 878-887. Springer, 2005.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pailern recognition, pp- 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual net- works. Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Nether- lands, October 11-14, 2016, Proceedings. Part IV 14, pp. 630 645. Springer, 2016b.
Bo Huang, Mingyang Chen, Yi Wang. Junda Lu, Minhao Cheng, and Wei Wang. Boosting accuracy and robustness of student models via adaptive adversarial distillation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2023.
Gaojie Jin, Xinping Yi, Wei Huang, Sven Schewe, and Xiaowei Huang. Enhancing adversarial training with second-order statistics of weights. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15273-15283, 2022.
Gaojie Jin, Xinping Yi, Dengyu Wu, Ronghui Mu, and Xiaowei Huang. Randomized adversarial training via taylor expansion. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 16447-16457, 2023.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition, arXiv preprint 1rX1v:1910.09217. 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Hongsin Lee, Seungju Cho, and Changick Kim. Indirect gradient matching for adversarial robust distillation. arXiv preprint arXiv:2312.0322 2023.
Hyungyu Lee, Saehyung Lee, Hyemi Jang, Junsung Park, Ho Bae, and Sungroh Yoon. Dafa: Distance-aware fair adversarial training. arXiv preprint arXiv:2401.12532, 2024.
Guanlin Li, Guowen Xu, and Tianwei Zhang. Alleviating the effect of data imbalance on adversarial training, 2023.
Tianhao Li, Limin Wang, and Gangshan Wu. Self supervision to distillation for long-tailed visual recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 630-639, 2021.
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu. Under- standing adversarial attacks deep learning based medical image analysis systems. Pattern Recognition, 10:107333,, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.066 2017.
Javier Maroto, Guillermo Ortiz-Jiménez, and Pascal Frossard. On the benefits of knowledge distilla- tion for adversarial robustness. CoRR, abs/2203.07159, 2022. doi: 10.4850/ARX1..203007159
(185550/aXii1 URL h 2203. 07159
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. arXiv preprint arXiv.2007.07314. 2020.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag tricks for adversarial
training. arXiv preprint 0.00022010.00467, 2020.
12

---

Published as a conference paper at ICLR 2025
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. Advances in Neural Information Processing Systems. 32, 2019.
Jiawei Ren, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, et al. Balanced meta-softmax for long- tailed visual recognition. Advances in neural information processing systems, 33:4175-4186, 2020.
Jihoon Tack, Sihyun Yu, Jongheon Jeong, Minseon Kim, Sung Ju Hwang, and Jinwoo Shin. Con- sistency regularization for adversarial robustness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8414-8422, 2022.
Jingru Tan, Changbao Wang, Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, and Junjie Yan. Equalization loss for long-tailed object recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11662-11671 2020.
Ningfei Wang, Yunpeng Luo, Takami Sato, Kaidi Xu, and Qi Alfred Chen. Does physical adversarial example really matter to autonomous driving? towards system-level effect of adversarial object evasion attack. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 1441244227 2023.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2020.
Zeming Wei, Yifei Wang, Yiwen Guo, and Yisen Wang. Cfa: Class-wise calibrated fair adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 8193-8201, 2023.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. Advances in Neural Information Processing Systems, 33:2958-221 29699, 2020.
Tong Wu, Ziwei Liu, Qingqiu Huang, Yu Wang, and Dahua Lin. Adversarial robustness under long- tailed distribution. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8668, June 2021.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 501-509, 2019.
Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust to be fair: Towards fairness in adversarial training. In International conference on machine learning, pp. 11492- 11501. PMLR, 2021.
Xinli Yue, Ningping Mou, Qian Wang, and Lingchen Zhao. Revisiting adversarial training under long-tailed distributions. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition (CVPR), pp. 24492-24501, June 2024.
Sergey Zagoruyko and Nikos Komodakis. arXiv: 146, 2016.
Wide residual networks.
arXiv preprint
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482. PMLR, 2019.
Qingzhao Zhang, Shengtuo Hu, Jiachen Sun, Qi Alfred Chen, and Z Morley Mao. On adversarial robustness of trajectory prediction for autonomous vehicles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 15159-15168, 2022.
Songyang Zhang, Zeming Li, Shipeng Yan, Xuming He, and Jian Sun. Distribution alignment: A unified framework for long-tail visual recognition. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 2361-2370, 2021.
13

---

Published as a conference paper at ICLR 2025
Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen. Bbn: Bilateral-brance network with cumulative learning for ence on computer vision and pattern recognition, pp. 9719-9728, 2020.
visual recognition. In Proceedings of the IEEE/CVF confer-
Jianing Zhu, Jiangchao Yao. Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, and Hongxia Yang. Reliable adversarial distillation with unreliable teachers. arXiv preprint arXiv:2106.0499 2021.
Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distil- lation: Robust soft labels make student better. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 16443-16452, 2021.
A THEORITICAL PROOF
Given the data distribution S, an optimal natural classifier that minimizes the overall standard error has optimal weight that satisfies wi U22 Wn.
Lemma 1.
Proof. Let's assume, for the sake of contradiction, that the optimal weights do satisfy the given condition. In other words, for some * and i, E (1,2,...,, ,n), we assume there exist my < w.. Then, obtain the following standard error
Rn) Pr -1)- Pr
Σ ktin
+b +0
Pr(y +1) Pr
Σ
WhN (, w w <
However, if we define new classifier f', which has the same weight vector as classfier f but uses wy to replace ws. The resulting standard error for the new classifier f' can be obtained as follows
n
Rnai (f') = Pr(y -1) - Pr
Σ
k
PPryy +1)-
+ (o
Σ
wt
<0
Given that w; < Wjj the f' has smaller enror than f, which contradicts the assumption that f is the optimal classifier with the least error.
Given the data distribution S, an optimal robust classifier that minimizes the robust error has optimal weight that salisfies wy = W2 Wp,
Lemma 2.
Similar to the Lemmal 1 it can be easily proved with the same argument.
A.1 PROOF OF THEOREM
Proof. By the Lemma I the optimal classifier frut for standard error has optimal weight of w1 W2 W,, For simplicity, we assume 12-norm of w 1, Le., w = (L/vn,1/ 1/,,, 1...1/// /n). following existing works Xu al. (2021): Lee al. (202)). Then, the standard errors of fnat can
14

---

Published as a conference paper at ICLR 2025
be formulated follows.
Real (fnot) Pr(y +11 (nnot) Pr(y = -1) (frat)
r
Pr (f(x)) + -1 +
1
Pr (f(x) # y1y=-1)
r+1
r+1
1 N(+,, 1) +b<+ + r+1 Σ
1
Pr Σ
Pr
r r+1
1
(- r+1
20-vnn +
(9)
Here, & represents the cumulative distribution function of the standard normal distribution. To
determine the optimal value of b, we solve the equation n'R (fnot)))))) //ot)/ O.
d'Rnat(fnat) db
r ++1 26 Van - b) + r+1 Φ( -in + +b) =
r
-r- pr.((-viu)
b) + o(-vv(1i vin +
(- b)p exp (-v + b)2
-x-exp
(10)
Here, ó represents the standard normal distribution function. Therefore, the optimal biat for natural classifier is follows,
In
naat 2/nn
(11)
By using the optimal natural classifier, the standard error of the tail class can be formulated as follows,
Ridd (Reas) - 10 (-Vin) + 2//n)
ln
(12)
Then, the robust error of the tail class with optimal natural classifier can be calculated as follows, Rob (Frae) Pr(E6 with ||0||oc. Se s.t. liac + * ply = 1)
= Pr
1
(x; + +e) + breat > 0)
Pr
Vn N'( (-1 + c,, brat >0)
Pr (N 1) -Vn(n- e) + brat)
In
-Vn(n- c)+ -)
(13)
2
Similarly, based the Lemma the optimal classifier frah for robust error has optimal weight of wi = W2 = ... Wn = 1/V n. Therefore, the robust errors of frab can be formulated as follows with adversarial noise e satisfying 0 <<<> <
Reos (Frot) = Pr(y = +1) Ridh (Fros) + Pr(yy = -1) Roo((Fro))
r+1 Pr
N(+7 - e, (, +,10)b00))))))
vn
+ r+1
Pr
vn - N(-97
Pr (N(0, 1) < - -v(y -b)
1+1
+ r+1
Pr(((()1)) < -v - t) bb)
r+1 -0(-viln <) b) + ++1 Φ(- 6) +b)
(14)
15

---

Published as a conference paper at ICLR 2025
To determine the optimal value of b, solve the equation (frob))(rrrl//// dRrob(frob) r
db r+1 ((-va(1) 6) - b) r r+1 o6 Vn((I - c) -r .r.0(- ( (n(11 - c) - b) + S0
vn(11 - () b)
-) c) (- b)2 + exp
1 vū(n b)2 =
-r. exp Therefore, the optimal 6, for robust classifier is follows,
(15)
lnr
b
(16)
Then, the standard and robust error of the tail class with optimal robust classfier can be formulated as follows.
In
R
-vin 2//(111
(17)
R Roob(Sroa)
(-vil(n- - +)+ 2/n(n -)
In
(18)
B ADDITIONAL EXPERIMENTS
B. EXPERIMENTS ON ANOTHER ARCHITECTURE.
We conducted experiments using WideResNNet34-110 10. Similar to the results of ResNet- 18 in the main paper, our method consistently demonstrated superior performance. Notably, on both CIFAR-10-LE and CIFAR-I00-LT datasets, significant performance improvements were observed in both T-Clean and T-PGD settings. While RoBal exhibited a marginally higher clean accuracy in the case of the best checkpoint on CIFAR- 1O-LT. the difference compared our method is negligible. However, our method achieved approximately 5% point improvement in robust accuracy against auto attack on CIFAR-100 1---TT.. the CIFAR- 100-II dataset, our method demonstrated the best performance in terms of both clean accuracy and robustness across all classes. Additionally, the improvements in T- Clean and T-PGD demonstrate that our method is more suitable for handling long-tail distributions.
Table 7: The clean accuracy and robustness for various algorithms using W1deResNet-34-10 on CIFAR-10-LT.
Best Checkpoint
Last Checkpoint
Method
Clean
PGD
AA
T-Clean
T-PGD Clean
PGD
AA
T-Clean
T-PGD
PGD-AT
58.86
30.57
29.43
18.5
2.1
59.10
26.3
25.66
19,0
1.9
TRADES
51.93
30.45
29.20
4.5
0.3
55.36
27.62
26.99
19.2
2.9
MART
48.92
31.45
29.85
9.5
0.9
54.81
27.25
26.29
23.1
2.0
AWP
51.69
32.42
30.35
5.3
0.2
51.89
29.19
27.45
10.9
0.6
RoBal
74.46
32.82
25.72
71.5
22.8
70.03
24.74
23.09
50.6
5.7
REAT
73.16
33.45
28.71
66.4
20.8
64.11
25.90
25.00
31.7
3.6
AT-BSL
73.23
35.08
32.26
66.4
18.9
66.23
26.87
25.98
40.6
4.3
Ours
73.97
39.25
35.97
74.3
33.7
72.38
31.15
29.10
60.4
12.7
B.2
ADDITIONAL EXPERIMENT OF MORE TRAINING EPOCHS
Since we employed additional training epochs for self-distillation, also trained the baselines with more epochs and summarized the results in Table The results showed that increasing the training epochs for the baselines did not lead to performance improvements; in REAT, performance declined when more training epochs were utilized. As a result, it is clear that the efficacy of our approach is not solely consequence of increasing the number of training epochs.
16

---

Published as a conference paper at ICLR 2025
Table 8: CIFAR-100-LT.
The clean accuracy and robustness for various algorithms using WidcResNet-34-10 on
Best Checkpoint
Last Checkpoint
Method
Clean
PGD
AA
T-Clean
T-PGD
Clean
PGD
AA
T-Clean
T-PGD
PGD-AT
47.48
19.36
17.79
25.4
8.7
46.09
16.51
15.67
24.3
7.2
TRADES
42.67
20.89
18.42
18.3
6.9
43,99
18.53
17.51
19.9
7.4
MART
41.54
21.52
18.83
19.2
9.2
43.08
17.00
15.84
22.8
7.8
AWP
45.53
23.23
19.92
20.4
7.9
47.05
21.97
19.21
23.1
8.8
RoBal
49.06
18.23
16.79
27.6
9,4
46.92
15.48
14.69
28.0
6.8
REAT
49.06
20.00
18.08
34.4
12.2
47.65
16.95
15.60
33.6
9.8
AT-BSL
50.05
18.96
17.10
38.3
13.3
47.95
16.40
15.31
32.2
9.5
Ours
50.55
23.43
20.16
38.4
19.5
50.87
22.21
19.44
42.5
18.4
Table The clean accuracy and robustness for various algorithms using ResNet-18 18 on CIFAR-100- LT. T-Clean and T-PGD are clean and PGD-20 accuracy on the tail class group.
Best Checkpoint
Last Checkpoint
Method
Clean
PGD
AA T-Clean T-PGD Clean PGD
AA T-Clean T-PGD
RoBal (100 epochs)
44.27 19.67 16.78
18.4
8.0 46.46 16.28 14.73
23.3
6.7
RoBal (200 epochs)
44.20 19.70 17.01
17.5
8.1 45.60 15.06 14.03
24.1
6.7
REAT (100 epochs)
45.73
18.22 15.82
32.2
11.4 45.53 15.64
14.27
33.0
10.5
REAT (200 epochs)
44.67 16.48 14.53
29.6
10.3 44.80 14.86 13.64
28.9
7.9
AT-BSL (100 epochs)
45.38
18.04 15.73
33.1
12.4
45.48
15.36
14.07
31.5
9.1
AT-BSL (200 epochs)
45.01
17.19 14.56
28.9
9.7 44.04
14.23 13.23
28.5
7.2
Ours
46.13
22.42 18.73
38.9
17.9 47.22 21.82 18.53
37.9
17.6
20.0
20.0
45.00
46.00
47.76
47.10
47.36
20.41
20.53
20.75
21.76
21.53
10.0
46.14
47.20
47.58
47.54
47.52
10.0
19.53
19.07
19.63
19.09
19.97
46.
47.23
48.17
47.52
47.78
18.49
18.34
18:37
18.71
18.43
46.75
48.01
47.55
47.69
48. 28
17.45
17.00
16.85
16.82
17.30
47.72
47.77
46.99
47.21
48.61
16.01
16.08
16.01
16.38
15.97
x/d
1/3
r/4
n/3
r/2
(a) Clean accuracy
(b) Robust accuracy
Figure 6: Hyperparmeter sensitivity of entire class
20.0
33.80
38.60
36.70
38.00
37.20
20.0
14.90
16.20
16.00
18.00
17.40
10.0
34.70
36.20
36.50
37.00
38.00
10.0
14.10
14.30
13.90
15.40
15.30
5.0
35.10
37.10
36.50
35.90
36.70
5,
13.50
13.10
13.60
13.90
13.90
2.0
10.30
10.60
r/4
7/3
r/2
7/4
n/3
r/2
y
(a) Clean accuracy
(b) Robust accuracy
Figure 7: Hyperparmeter sensitivity of tail class
17

---

Published as a conference paper at ICLR 2025
B.3 SENSITIVITY OF HYPERPARAMETER
In Figure 6 we experiment with the sensitivity of the self-distillation weight parameter, and the sampling rate, 7, where r' an imbalance ratio between the class with the largest number of samples and the class with the smallest number of samples. We can see that as increases, robustness improves, but clean accuracy slightly decreases. This indicates a trade-off between robustness and clean accuracy, which is expected given the use of adversarial distillation techniques. In the case of y, it did not significantly impact performance. However, when y is larger-meaning more samples used to train the self-teacher-bolt both clean accuracy and robustness showed improvement.
In Figure 7 we plot the tail class performance. In this case, we observed that as a increases, i.e., as the weight of the loss for knowledge distillation from the balanced self-teache increases, the clean and robust performance of the tail class improves. The sensitivity to was not significant.
B.4 VARIANCE OF MULTIPLE RUNS
Table 10: The clean accuracy and robustness for various algorithms using ResNet18 8 on CIFAR-100 LT.
Runs
Clean
PGD
T-Clean
T-PGD
46.13
22.42
38.9
17.9
2
46.57
22.23
37.5
17.7
46.47
22.27
37.8
17.8
4
46.59
22.12
38.8
17.8
46.01
22.52
38.9
18.0
Average
46.35
22.31
38.38
17.84
Standard deviation
0.27
0.15
0.68
0.11
In Table TO we conducted five experiments and computed the mean and standard deviation to as- sess the impact of randomness. The results show that the standard deviations are relatively small, indicating that the model's performance is consistent across different runs. This suggests that the observed improvements are robust and not significantly influenced by random fluctuations in the training process.
B.5 CLASS- WISE ROBUSTNESS.
35
Durs
Reot
30
Robil
Robust Accuracy (%)
25
20
15
class1019 19
class20-29 29
class30-2 39
class40-4t 49
dass50- 59
class60- 69
clas70-79 79
class80 89
class90- 99
class-94
Classes
Figure 8: Class-wise robustness.
In Figure g we divided the classes CIFAR-100 into 10 groups and measured the robustness across them. As move from class O class 99, the number of data points decreases. Our
18

---

Published as a conference paper at ICLR 2025
method demonstrated higher robustness across all class groups compared to the baseline. Notably, achieved the best performance in all groups except the first group. In contrast, Robal showed strong performance the first group (head classes) but the worst performance on the last group (tail classes).
19


Reviews:
[{"review_id": "mlCnoO0iF7", "reviewer": "Reviewer_61WA", "summary": "This paper improves model robustness against adversarial attacks in long-tailed learning. \n\nThe authors observed that adversarial training, compared with normal training, further sacrifices minority class accuracy over majority classes. This observation is interesting and quite intuitive after some thoughts. It could be inspiring to further works. \n\nBased on this observation, the authors proposed to use stronger constrains than previous methods to balance the accuracy among different classes. Specifically, previous methods uses balance-aware loss functions to balance accuracy in long-tailed adversarial training. The authors added resampling (when training the teacher model), another common technique to handle data imbalance, on top of those previous works. The overall method is limited in technical novelty. \n\nExperimental results show good improvements over previous methods. The experimental settings raise some concerns to me. \n\nOverall, I think this is a marginal paper. I lean toward accepting primarily due to the good numerical results shown under the specific experiment setting used by this paper.", "questions": "Please see above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The observation that adversarial training further harms minority classes compared to normal training is interesting and may inspire further works. \n\n2. The performance gain under the specific experiment setting is good.", "weaknesses": "1. The technical novelty is limited. The two methods used to handle long-tail learning in this paper are resampling (when training the teacher model) and a balance-aware loss function proposed by previous works. \n\n2. The proposed method has a noticeable limitation: It requires to train a teacher model on balanced data. This greatly limits its practical application. In most cases, the powerful large model used as teacher models (e.g., foundation models downloaded from Hugging faces) are trained on real-world datasets which are typically unbalanced. It is very hard to get an off-the-shelf model that is \"balanced\". This requires the users to train the balanced model on their own. As a training algorithm, this is considerable complexity overhead. Moreover, it is not discussed in the paper how the teacher model could affect the distillation. Does the re-sampling method matter? Does the size of the teach model matter? Does the re-sampled dataset has to be perfectly balanced or it could have some imbalance? If it is the later case, how much imbalance it could have?\n\n3. Table 5 is not interpretable. Which part is CIFAR10 and which part is CIFAR100?\n\n4. The experiment setting is limited. The largest imbalance ratio is 50 for CIFAR10 and 10 for CIFAR100 and ImageNet. However, on both datasets, it is common to experiment with much stronger imbalance, with the ratio up to 100 [1]. Experimenting with stronger imbalance ratio is important since it is a more challenging case for re-sampling based methods. Under the stronger ratio, I assume the training of the teacher model would face difficulty cause the minority classes have few samples and simple up-sampling might not help too much. \n\n[1] Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jYg8TuvnEI", "reviewer": "Reviewer_pWdA", "summary": "In this paper, the authors first provide a theoretical analysis explaining why adversarial training leads to robustness degradation in tail classes. Motivated by these findings, they propose a simple empirical solution to mitigate this issue. Specifically, they employ a distillation-based approach in which a balanced version of the long-tailed dataset is used to adversarially train a teacher model. This teacher model is then used to train a student model on the original long-tailed dataset. The authors conduct experiments across multiple datasets and various settings, such as with and without augmentations, different imbalance ratios, and more.", "questions": "- Please see weaknesses and clarify the imbalance ratio better, and conduct statistical tests for comparison with other methods.\n- It’s unclear why the authors report the AutoAttack accuracy on the whole dataset but not for the tailed classes. I would like to see the AutoAttack accuracy on the tailed classes for Figure 1 and all relevant tables.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The key strengths of the paper are:\n- The paper is mostly written well.\n- The authors conduct comprehensive experiments, as well as provide theoretical motivations.\n- While the improvement methods use standard techniques like balanced softmax and indirect gradient matching, it's interesting to see that a simple prior distillation step enhances performance.", "weaknesses": "- The concept of Imbalance Ratio in the paper is not well explained. While I understand how it applies to a binary setup, it’s unclear what an imbalance ratio of 50 means for CIFAR-10-LT compared to an imbalance ratio of 10. I suggest that the authors include the number of samples in the least populated class for clarity.\n- Related to this, the core assumption is that training on a balanced dataset will enhance feature learning for the tail classes. However, the proposed method may negatively impact overall performance if the long-tail ratio is extreme (e.g., 5 samples per class). This could lead to ineffective training of the teacher network, and the distillation loss may then degrade performance. I would be interested to hear the authors' thoughts on this.\n- Additionally, given the smaller sample size (especially for tail classes), it would be helpful to report the mean along with error metrics, such as standard deviation, and to conduct statistical tests to verify that the proposed method significantly improves performance compared to other methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ueobi7yXQ5", "reviewer": "Reviewer_ffZU", "summary": "This paper introduces a method to improve adversarial robustness on long-tailed distributions. The primary contributions include a two step adversarial training paradigm which involve, (i) adversarially training a teacher model on a balanced sub-dataset obtained from the original unbalanced dataset, and (ii) adversarially training a student model on the unbalanced original dataset using knowledge distillation from the teacher model and an additional balanced softmax loss. Another notable contribution is a theoretical analysis demonstrating how traditional adversarial training methods fail to improve robust accuracy of tail classes even when compared to non-adversarially trained methods.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The authors effectively justify the need for adversarial training methods tailored to long-tailed distributions through their theoretical analysis (Section 3.2) and empirical evaluations (Section 3.3).\n\n- An intriguing result is the observation that adversarial training unexpectedly reduces robust performance on tail classes compared to non-adversarial methods when addressing long-tailed distributions.\n\n- The experimental results demonstrate substantial improvements in both clean and robust accuracy on tail classes, outperforming existing adversarial training (AT) methods such as TRADES, MART, and AWP, as well as long-tailed AT methods like RoBal, REAT, and AT-BSL.\n\n- These performance gains are consistently observed across multiple datasets and architectural frameworks, highlighting the method's potential usability.", "weaknesses": "- The paper does not sufficiently explain why their proposed method outperforms existing AT methods focused on long-tailed distributions. Particularly, why should Knowledge Distillation be a good choice for handling adversarial-robustness on long-tailed datasets?\n\n- The incorporation of knowledge distillation (KD) and balanced softmax loss appears somewhat arbitrary. E.g., KD has been already explored for training of long-tailed datasets [1,2,3]; balanced softmax has been used in AT for long-tailed data [4,5]; KD for AT is explored in [6]. However, I agree that KD is not used for all AT and long-tailed datasets simultaneously. Hence, I would like to get a better justification of the proposed method for clarity.\n\n- The absence of a dedicated subsection that develops a theoretical or intuitive motivation for the proposed method diminishes the overall impact and comprehensibility of the paper. Providing a more thorough theoretical framework or intuitive explanations would significantly strengthen the paper’s contributions and clarity.\n\n- The proof of Lemma 1 in Appendix A needs to be a bit more rigorous and show how having equal weights minimizes the variance of $\\sum_{k=1}^n w_k x_k + b$ using Cauchy-Schwartz, and hence minimises the natural error.\n\n- Some inconsistency in the naming of the balanced dataset, throughout the paper, but specifically on page 6, line 317 onwards, notation shifts from D_b to D_B.\n\n\n  \n\nOverall, the paper presents strong experimental results and provides sound theoretical analysis to motivate the original problem statement—adversarial training for long-tailed distributions. Therefore, I provide a score of 6 (Weak Accept). If the authors address the concerns outlined in the Weakness section, I would be willing to raise my score.\n\n  \n\n[1] T. Li, L. Wang, and G. Wu, “Self supervision to distillation for long-tailed visual recognition,” in IEEE International Conference on Computer Vision, 2021, pp. 630–639.\n\n[2] Y.-Y. He, J. Wu, and X.-S. Wei, “Distilling virtual examples for long-tailed recognition,” in IEEE International Conference on Computer Vision, 2021, pp. 235–244.\n\n[3] H. Rangwani, P. Mondal, M. Mishra, A. R. Asokan, and R. V. Babu, “Deit-lt: Distillation strikes back for vision transformer training on long-tailed datasets,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 23396–23406.\n\n[4] TongWu, Ziwei Liu, Qingqiu Huang, YuWang, and Dahua Lin. Adversarial robustness under longtailed distribution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8659–8668, June 2021.\n\n[5] Xinli Yue, Ningping Mou, Qian Wang, and Lingchen Zhao. Revisiting adversarial training under long-tailed distributions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24492–24501, June 2024.\n\n[6] Hongsin Lee, Seungju Cho, and Changick Kim. Indirect gradient matching for adversarial robust distillation. arXiv preprint arXiv:2312.03286, 2023.\n\n\n--------------------\n\n**Post Rebuttal**:\n\n\nThank you for addressing my concerns and providing additional clarity in the rebuttal. I appreciate the authors' efforts to strengthen the motivation for their method, particularly with respect to the usage of KD alongside the Balanced Softmax loss.\n\n> ⁠Consequently, we argue that if the teacher is robust to the tail classes, then using KD enables the student model to mimic the teacher’s robustness and perform similarly well in the tail classes. There are various ways to make the teacher robust to tail classes, but in our case, we provided theoretical proof showing that a teacher trained on a balanced dataset is more robust to tail classes compared to one trained on an unbalanced dataset.\n\nI am satisfied with the justification provided for the approach, as well as the addition of these explanations to Sections 4.1 and 4.2 of the paper. This effectively addresses my primary concern about the motivation behind the method.\n\nAs a result, I am increasing my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UUt8f7q1MZ", "reviewer": "Reviewer_Uh2k", "summary": "The paper's main contribution is the two-phase adversarial training of deep neural networks (ResNet-18) on CIFAR 10/100 and TynyImageNet datasets. Phase one is training on a Balanced dataset (e.g., 30 epochs), and Phase two is the Knowledge distillation on the long-tailed imbalanced dataset to improve the robustness of the model on a long-tailed imbalanced dataset. \n\nFor this contribution, the paper presents a theoretical analysis of Theorem 1, which says robustness training will further impact (worsen) the accuracy of models on long-tailed datasets. Most of the iterative analysis is in the part of the Appendix rather than the main part of the paper. The proof of Corollary indicates that it is trivial according to equations 4 and 5, which appears to indicate that the readers should figure it out themselves. \n\nThe main results show improvement compared to some SOTA algorithms. The author's results are shown to be the best in all cases. A typical training set was used for these results.", "questions": "Mention which dataset is used for  Figure 3. \n\nA comparison with balanced datasets to determine where adversarial training improves the performance of adversarial tests could be included in Figure 3. \n\nIn Figure 3, at IR =1, explain why both natural and adversarial training produces the same test results on the adversarial test data. \n\nExplain what loss function is used for $L_{KD}$ in Algorithms 1were used.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The two-phase training methodology uses the \"self-distillation\" approach, where phase one is trained on balanced datasets, and this trained model is used for knowledge distillation in the second phase to make models robust on long-trailed datasets. The performance of this approach is shown to be better performance. That is, the results shown surpass the mentioned SOTA results on RestNet18.", "weaknesses": "Explain why the proof of the theorem is not generic for any iterative learning processes. Also, a step-by-step explanation of how the weight values affect robustness accuracy in the proof is required. \n\nInclude a visual or mathematical explanation connecting Figure 2 and Theorem 1 to clarify their relationship. \n\nThe definition of variables $\\Phi$ and $r$ in Equations 4 and 5 are required. \n\nFor reproducibility, the authors should indicate the availability of codes in the experimentation section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)