Imbalanced Adversarial Training with Reweighting
Authors: Wentao Wang
Han Xu
Xiaorui Liu
Yaxin Li
Bhavani Thuraisingham
Jiliang Tang
Venue: ICLR 2022 Submitted (2022)
Keywords: imbalanced data
robustness
adversarial training

Abstract: Adversarial training has been empirically proven to be one of the most effective and reliable defense methods against adversarial attacks. However, the majority of existing studies are focused on balanced datasets, where each class has a similar amount of training examples. Research on adversarial training with imbalanced training datasets is rather limited. As the initial effort to investigate this problem, we reveal the facts that adversarially trained models present two distinguished behaviors from naturally trained models in imbalanced datasets: (1) Compared to natural training, adversarially trained models can suffer much worse performance on under-represented classes, when the training dataset is extremely imbalanced. (2) Traditional reweighting strategies may lose efficacy to deal with the imbalance issue for adversarial training. For example, upweighting under-represented classes will drastically hurt the model’s performance on well-represented classes, and as a result, finding an optimal reweighting value can be tremendously challenging. In this paper, to further understand our observations, we theoretically show that the poor data separability is one key reason causing this strong tension between under-represented and well-represented classes. Motivated by this finding, we propose Separable Reweighted Adversarial Training (SRAT) to facilitate adversarial training under imbalanced scenarios, by learning more separable features for different classes. Extensive experiments on various datasets verify the effectiveness of the proposed framework.

Under review as a conference paper at ICLR 2022
IMBALANCED REWEIGHTING
ADVERSARIAL
TRAINING
WITH
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial training has been empirically proven to be one of the most effective and reliable defense methods against adversarial attacks. However, the majority of existing studies are focused on balanced datasets, where each class has similar amount of training examples. Research on adversarial training with imbalanced training datasets is rather limited, As the initial effort to investigate this problem, we reveal the facts that adversarially trained models present two distinguished behaviors from naturally trained models in imbalanced datasets: (1) Compared to natural training, adversarially trained models can suffer much worse performance on under-represented classes, when the training dataset is extremely imbalanced. (2) Traditional reweighting strategies may lose efficacy to deal with the imbalance issue for adversarial training. For example, upweighting under-represented classes will drastically hurt the model's performance well-represented classes, and a result, finding an optimal reweighting value can be tremendously challenging. In this paper, further understand our observations, we theoretically show that the poor data separability is one key reason causing this strong tension between under- represented and well-represented classes. Motivated by this finding, we propose Separable Reweighted Adversarial Training (SRAT) to facilitate adversarial training under imbalanced scenarios, by learning more separable features for different classes. Extensive experiments various datasets verify the effectiveness of the proposed framework.
INTRODUCTION
The existence of adversarial samples (Szegedy al. 2013, Goodfellow et al. 2014) has risen huge concerns on applying deep neural network (DNN) models into security-critical applications, such as autonomous driving (Chen al.) 2015) and video surveillance systems (Kurakin et al. 2016). As countermeasures against adversarial attacks, adversarial training (Madry et al.) 2017 Zhang jet al. 2019 Wang et al., 2019) has been empirically proven to be one of the most effective and reliable defense methods. In general, it can be formulated to minimize the model's 's average error on adversarially perturbed input examples (Madry et al. 2017). Although promising to improve the model's robustness, most existing adversarial training methods assume that the number of training examples from each class is equally distributed. However, datasets collected from real-world applications typically have imbalanced distribution (Everingham al., 2010 Lin al.) (014). Hence, it is natural to ask: What is the behavior of adversarial training under imbalanced scenarios? Can we directly apply existing imbalanced learning strategies natural training to tackle the imbalance issue for adversarial training? Recent studies find that adversarial training usually presents distinct properties from natural training. For example, compared natural training, adversarially trained models suffer more from the overfitting issue (Schmidt et al., 2018), and they tend to present strong wlssswise performance disparities, even the training examples are uniformly distributed over different classes (Xu al. (2000a). Imagine that if the training data distribution is highly imbalanced, these properties of adversarial training can be greatly exaggerated and make it extremely difficult to be applied in practice. Therefore, is necessary but challenging to answer aforementioned questions.
As the initial effort to study the imbalanced problem in adversarial training, in this work, we first investigate the performance of existing adversarial training under imbalanced settings. As preliminary study shown in Section[..] we apply both natural training and PGD adversarial train- ing (Madry al. 2017) on multiple imbalanced training datasets constructed from CIFAR10 10 training

---

Under review as a conference paper at ICLR 2022
unaall NOCUPREY USDE GCOPCY
everal DOUSEeY under ecourayy
under accurayy
60--
100
20
(a) Natural Training Standard Acc. (b) Adv. Training Standard Acc.
(c) Adv. Training Robust Acc.
Figure 1:
Class-wise performance of natural & adversarial training using an imbalanced CIFARIO.
dataset (Krizhevsky et al. 2009) and evaluate trained models' performance on class-balanced test dataset. From the preliminary results, we observe that, compared to naturally trained models, adver- sarially trained models always present very low standard & robust accuracy on under-represented classes. This observation suggests that adversarial training is more sensitive imbalanced data distribution than natural training. Thus, when applying adversarial training in practice, imbalance learning strategies should always be considered for help.
As a result, we explore potential solutions which can handle the imbalance issue for adversarial training. In this work, we focus on studying the behavior of the reweighting strategy (He & Ma 2013) and leave other strategies such as resampling (Estabrooks et al. 2004) for one future work. In Section 2.2] we apply the reweighting strategy to adversarial training with varied weights assigning to one under-represented class and evaluate trained models' performance. From the results, we observe that, in adversarial training, increasing weights for an under-represented class can substantially improve the standard & robust accuracy on this class, but drastically hurt the model's performance on the well-represented class. This finding indicates that the performance of adversarially trained models is very sensitive the reweighting manipulations and it could be very hard to figure out an eligible reweighting strategy which is optimal for all classes.
is also worth noting that, in natural training, we find that upweighting the under-represented class increases model's standard accuracy on this class but only slightly hurts the accuracy on the well-represented class, even when adopting large weight for the under-represent class. To further investigate the possible reasons leading to different behaviors of the reweighing strategy in natural and adversarial training, we visualize their learned features (in Figure 3. and observe that features learned by the ad adversarially trained model of different classes tend to mix together while they are well separated for the naturally trained model. This observation motivates us to theoretically show that when the given data distribution has poor data separability, upweighting classes will hurt the model's performance on well-represented classes. Motivated by our theoretical understanding, we propose novel framework Separable Reweighted Adversarial Training (SRAT) to facilitate the reweighting strategy in imbalanced adversarial training by enhancing the separability of learned features. Through extensive experiments, validate the effectiveness of SRAT.
PRELIMINARY STUDY
2.1 THE BEHAVIOR OF ADVERSARIAL TRAINING
In this subsection, we conduct preliminary studies examine the performance of PGD adversarial training (Madry et al., 2017). Following previous works (Cui et al., 2019; Cao et al. 2019), we construct an imbalanced CIFAR10 (Krizhevsky et al.) 2009) training dataset, where each of the first classes (a.k.a. well-represented classes) has 5,000 training examples and each of the last 5 classes (a.k.a. under-represented classes) has 50 training examples.
Figure I shows the performance of naturally and adversarially trained models using a ResNet18 (He al., 2016) architecture. From the figure, we can observe that, compared with natural training, PGD adversarial training will result in larger performance gap between well-represented classes and under-represented classes. For example, in natural training, the ratio between the average standard
In this work, we denote standard accuracy as model's accuracy on the imputs without perturbations and robust accuracy model's 's accuracy on the input samples which are adversarially perturbed. Without clear clarification, consider the perturbation constrained by 30 -nom 8/255.
2

---

Under review as a conference paper at ICLR 2022
accuracy of well-represented classes (brown) and under-represented classes (violet) is about 2:1, while in adversarial training. this ratio expands to 16:1. Moreover, for adversarial training, it has extremely under-represented classes. There are 3 out of the 5 under-represented classes with 0% standard & robust accuracy. As a conclusion, the performance of adversarial training is easier be affected by imbalanced distribution than natural training and suffers classes. More results are reported in Appendix A. which further support our findings.
poor performance
on under-represented
2.2 THE REWEIGHTING STRATEGY IN NATURAL TRAINING V.S. IN ADVERSARIAL TRAINING
The preliminary study in Section 2.1 demonstrates that it is highly demanding to adjust the original adversarial training methods to accommodate imbalanced data distribution. Next, investigate the effectiveness of adopting the reweighting strategy (He & Ma 2013) in adversarial training. Our experiments conducted under binary classification setting, where the training dataset contains two classes that are randomly selected from CIFARIO dataset, with each class having 5,000 and 50 training examples respectively. Based on this training dataset, we arrange multiple trails of (reweighted) natural training and (reweighted) adversarial training, with the weight ratio between the represeepreeented class and well-represented class ranging from 1:1 to 200:1.
80
60
26-
undo aocrrey
JRGET OCCERDY
1:30
23
100
150
200
110
50. 25
150
7001
110
25
20
15
100
350
200
(a) Natural Training Standard Acc.
(b) Adv. Training Standard Acc.
(c) Adv. Training Robust Acc.
Figure 2: Class-wise performance of reweighted natural & adversarial training in binary classification.
Figure 2 shows the experimental results with training data sampled from the classes "cat" and "horse". As demonstrated in Figure 2 increasing the weight for the under-represented class (horse) will drastically increase the model's performance on this class, while also immensely decreasing the performance on the well-represented class (cat), For example, when increasing the weight ratio from 1:1 to 150:1, the standard accuracy of the under-represented class is improved from 0% to 60% and its robust accuracy from 0% to 50%. However, the standard accuracy on the well-represented class drops from 100% to 60%, and its robust accuracy drops from 100% 50%. These results illustrate that adversarial training's 's performance can be significantly affected by the reweighting strategy. As a result, the reweighting strategy in this setting can hardly help improve the overall performance no matter which weight ratio is chosen, because the model's performance always presents strong tension between these two classes. More experiments using different binary imbalanced datasets are reported in Appendix A.2 where have similar observations.
THEORETICAL ANALYSIS
observe that in natu- ral training, the reweighting strategy can only make small impact on the classes' performance. This phenomenon has been extensively studied by recent works (Byrd & Lipton| 2019 Xu et al.) 2021), where they find that linear clas- sifier optimized by SGD on a linearly separable data will converge to the solu- tion of the hard-margin support vector machine (Noble) 2006). In other words, as long the data can be well separated, reweighting will not make huge influ- ence the finally trained models.
In Section 2.2
Cleon Exarplee pla
Cleon Exampie
welrrmeeeete Lece- cass
voented uedeereanooeee Koe caus
(a) Natural Training.
(b) Adversarial Training.
Figure 3: t-SNE visualization of learned features.
3

---

Under review as a conference paper at ICLR 2022
Inspired by their conclusions, hypothesize that, the adversarially trained models separate the data poorly, their performance is highly sensitive to the reweighting strategy. As a direct validation of hypothesis, in Figure 3 visualize the learned (penultimate layer) features of the imbalanced training examples used in the binary classification problem in Section 2.2 We find that adversarially trained models do present obviously poorer separability the learned features. Next, we theoretically analyze the impact of reweighting on linear models which are optimized under poorly separable data and provide all detailed proof in Appendix A..3
Binary Classification Problem. To construct the theoretical study, we focus on a binary classification problem, with a Gaussian mixture distribution D which defined as:
dim=d
if = +1 and μ (7) 1m..... (N(-
[N(W,
y ~~ 1-1,+1
if
where the two classes' centers (# E R3)) with each dimension have mean value in (17 >> 0) and variance o². Formally, we define the data separability as S n/o².. Intuitively, when larger, it suggests that two classes are well separated. Previous work (Byrd & Lipton 2019) also closely studied this term to describe data separability.
Besides, we assume the imbalanced training dataset satisfying the condition Pr.(y = +1) K Pr.(y = 1) and > I, which indicates the imbalance ratio between two classes. During test, we assume two classes have the equal probability to appear. Under the data distribution D, we will discuss the performance of linear classifiers f(x) = sign(w" x b) where w and are the weight and bias terms of the model f. If reweighting strategy is involved, we define the model upweights the under-represented class "-1" by p.
Lemma 3.1 Under the data distribution DD as defined in Eq. 7 with an imbalanced ratio K and reweight ratio p. the optimal classifier which minimizes the (reweighted) empirical risk:
# Pr. Pr. -yy f arg min (
Pr(f(()))))))))))))))))))))))))) xly==1) pr(y==1)) (2)
has the solution: w and b = log( c
da² 7
log( ()
Lemma 3.T indicates that the final optimized classifier has a weight vector equal to 1 and its bias term b only depends K, and the data separability S. In the following, we first focus one special setting when p = 1, which is the original ERM model without reweighting. Specifically, we aim to compare the behavior of linear models when they can poorly separate data (like adversarial trained models) or they can well separate data (like naturally trained models).
Theorem 3.1 Under two data distributions (x()),())) E DD and (xl2), yy(2)) E D2 with different separabilities S1 > S2, let f and f2 be the optimal non-reweighted classifiers (p = 1) under D and D2, respectively. Given the imbalance ratio K is large enough, we have:
(x()f(x(())) + 10 -1) Pr.(fT (x + (1) (1) (1= +1)
(3)
< (x + y (2))) -11 Pn( (2)) + (1))) +1).
Intuitively, Theorem suggests that when the data separability S is low (such as D2), the optimized classifier (without reweighting) can intrinsically have larger error difference between the under- represented class "-1" and the well-represented class "+1". Similar to the observation in Section 2. and Figure 3 adversarially trained models present a weak ability to separate data, and they also present strong performance gap between the well-represented class and under-represented class. Conclusively, Theorem 3.1 indicates that the poor ability to separate the training data can be one important reason which leads to the strong performance gap of adversarially trained models.
Next, we consider the case when the reweighting strategy is applied. In particular, we compare the impact of upweighting the under-represented class the performance of well-represented class.
Theorem 3.2 Under two data distributions (x(1,, ())) E Dj and (e(2),y(1)) E D2 with different separabilities S S2. let fi and J be the optimal non-reweighted classifiers (p 1) under D
4

---

Under review as a conference paper at ICLR 2022
and D2. respectively, and let fi" and f/2" be the optimal reweighted classifiers under D1 and Di
given the optimal reweighting ratio (p = K). Given the imbalance ratio K large enough, we have:
Pr:(fi" (r()) + 10 (2) = +11 Pr: (r.(f( * y 1y +1)
(4)
Pr (cl21) + 11111) = =1) Pr + (y +1).
As Theorem 3.2 shows, when the data distribution has poorer data separability (such as D:2), upweight- ing the under-reppe readerr-rreeeened class can cause greater hurt on the performance of the well-represented class. It is also consistent with our empirical findings about adversarial training models. Since the ad- versarially trained models poorly separate the data (Figure 3). upweighting the under-represented class always drastically decreases the performance of the well-represented class (Section 2.2). Through the discussions in both Theorem 3.] and Theorem 3.2, we conclude that the poor separability can be one important reason which makes adversarial training and its reweighted variants extremely difficult to achieve good performance under imbalance data distribution. Therefore, in the next section, we will explore potential solutions which can facilitate the reweighting strategy in adversarial training.
4 SEPARABLE REWEIGHTED ADVERSARIAL TRAINING (SRAT)
The observations from both preliminary study and theoretical understandings indicate that more sepa- rable data will advance the reweighting strategy in adversarial training under imbalanced scenarios. Thus, this section, we present framework, Separable Reweighted Adversarial Training (SRAT), which enables the effectiveness of the reweighting strategy in adversarial training under imbalanced scenarios by increasing the separability in the learned latent feature space.
4.1 REWEIGHTED ADVERSARIAL TRAINING
Given an input example (z, 3), adversarial training (Madry al. 2017) aims to obtain robust model So that can make the same prediction y for an adversarial example x', generated by applying an adversarially perturbation on x. The adversarial perturbations are typically bounded by small value
under norm, i.e., || =|Ip S E.
As indicated in Section 2.1 adversarial training cannot applied in imbalanced scenarios directly, as it presents very low performance under-represented classes. To tackle this problem, a natural idea is to integrate existing imbalanced learning strategies proposed in natural training, such as reweighting, into adversarial training to improve the trained model's performance those under--epresented classes. Hence, the reweighted adversarial training can be defined as
min 7
max pilpp
(.C(((()))..)
(5)
where w; is weight value assigned for each input sample (x,, yi) based on the example size of the class (Ir, belongs to or some properties of (aa, ye In most existing adversarial training
methods (Madry et al. 2017 Zhang et al., 2019) Wang et al. 2019), the cross entropy (CE) loss is adopted the loss function -..-). However, the CE loss could be suboptimal in imbalanced settings and some new loss functions designed for imbalanced learning specifically, such as Focal loss (Lin) cet al. 2017) and LDAM loss (Cao et al. 2019), have been proven their superiority in natural traiming. Hence, besides CE loss, Focal loss and LDAM loss can also be adopted as the loss function 2(;)) in Eq. (5).
4.2
INCREASING FEATURE SEPARABILITY
Our preliminary study indicates that only reweighted adversarial training cannot work well under imbalanced scenarios, and the reweighting strategy in adversarial training behaves very differently from natural training. Hence, in order to facilitate the reweighting strategy in adversarial training under imbalanced scenarios, inspired by our theoretical analysis, we equip a feature separation loss with our SRAT method to enforce the learned feature space separable as possible. More specifically, the goal of our feature separation loss to make the learned features of examples from the same class well clustered while from different classes well separated. Hence, adjusting the decision boundary via
5

---

Under review as a conference paper at ICLR 2022
the reweighting strategy fit muder-representeer classes' examples more will not hurt well-represented classes drastically. The feature separation loss is formally defined as:
exp(z/ x//T)
(seplat)
P(()
log
E exp(2) 2//T))
(6)
PEP(1)
where z/ is the feature representation of the adversarial example ai of 3ú, E R+ is a scalar temperature parameter, P(t) denotes the set of input examples belonging to the same class with i; and A(t) indicates the set of all input examples except z/- Our proposed feature separation loss (-ep(-) is inspired by the supervised contrastive loss proposed in (Khosla et al. 2020). The main difference is, instead of applying data augmentation techniques to generate two different views of each data example and feeding the model with augmented data examples, our feature separation loss directly takes the adversarial example xi of each data example x; as input.
4.3 TRAINING SCHEDULE
By combining the feature separation loss with the reweighted adversarial training, the final object function for Separable Reweighted Adversarial Training (SRAT) defined as:
1 7i
n
min
max s
we wx v) + Aaeep(a)).
where À is a hyper-parameter to balance the contributions of two terms.
In practice, in order to better take advantage of the reweighting strategy in our SRAT method, we adopt a deferred reweighting training schedule (Cao et al. 2019). Specifically, before annealing the learning rate, SRAT first trains model without introducing the reweighting strategy and then applies reweighting into model training process with a smaller learning rate. Since SRAT enables to learn more separable feature space, comparing with applying the reweighting strategy from the beginning of training, this deferred reweighting training schedule enables the reweighting strategy to obtain more benefits from our SRAT method. The training algorithm for SRAT is shown in Appendix A.4
EXPERIMENT
In this section, we perform experiments to validate the effectiveness of our SRAT method. We first compare SRAT with several representative imbalanced learning methods in adversarial training under various imbalanced scenarios and then conduct ablation study to understand SRAT more deeply.
5.1 EXPERIMENTAL SETTINGS
Datasets. We conduct experiments on multiple imbalanced training datasets artificially created from three benchmark image datasets CIFAR 10, CIFAR 100 (Krizhevsky et al. 2009) and SVHN (Netzer ct al. 2011) with diverse imbalanced distributions. Specifically. we consider two different imbalance types: Exponential (Exp) imbalance (Cui et al. 2019) and Step imbalance (Buda et al. 2018). For Exp imbalance, the number of training examples of each class will be reduced according to an exponential function n 147'' where the class index, n, is the number of training examples in the original training dataset for class i and 7 E (0, 1). We categorize half classes with most frequent example sizes in the imbalanced training dataset as well-represented classes and the remaining half classes as under- redreseepre classes. For Step imbalance, we follow the similar process adopted in Section 2.1 Moreover, we denote imbalance ratio K as the ratio between training example sizes of the most frequent and least frequent class. We construct different imbalanced datasets "Step-10", "Step- 100", "Exp-10" 10" and "Exp-1001 100", by adopting different imbalanced types (Step or Exp) with different imbalanced ratios (K 10 or K 100) to train models, and evaluate model's performance on the original uniformly distributed test datasets of CIFAR1O, CIFAR 100 and SVHN correspondingly. More detailed information about imbalanced training sets be found in Appendix A.5
Baseline methods. We implement several representative and ofte-of-theear imbalanced learning methods (or their combinations) into adversarial training baseline methods. These methods include: (1) Focal loss (Focal); (2) LDAM loss (LDAM); (3) Class-balanced reweighting (CB- Reweight) (Cui et al. 2019), where each example is reweighted proportionally by the inverse of

---

Under review as a conference paper at ICLR 2022
Table 1: Performance comparison on the CIFARIO Step-10 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
63,26 + 0.59
40.62 = 1.10
36.96 0.36
14.23 0.83
Focal
63.57 + 0.92
41.17
36.89 0.36
14.25 + 0.97
LDAM
57.08 + 1.16
2.20
37.18 0.56
12,44 = 0.93
CB-Reweight
73.30 + 0.30
74,80 + 0.88
41.34 0044
42.15 + 1.42
CB-Focal
73.42 0.29
74.35 + 1.39
41.34 - -023
41.80 1.24
DRCB-CE
75.89 0.23
70.55 = 1.10
39.93 0.24
33.33
1.42
DRCB-Foccl
74.61 + 0.35
67.06 + 1.37
37.91 = 0.24
29.50 + 1.31
DRCB-LDAM
72.95 + 0.08
75.42 + 1.83
45.23 0.19
44,98 + 1.90
SRAT-CE
76,69 + 0.33
73.07 - 0.63
41.02 0.49
36.57 = 0.92
SRAT-Focal
75.41 + 0.69
74,91 + 0.70
42.05 0.52
41.28 = 0.82
SRAT-LDAM
73.99 + 0.52
76.63 + 0.39
45.60 0.18
45.95 0.51
the effective number of its class; (4) Class-balancee balanced Focal loss (CB-Focal) (Cui et al. 2019). combination of Class-balancee method and Focal loss, where well-classified examples will be downweighted while hard-classified examples will be upweighted controlled by their corresponding effective number; (5) deferred reweighted CE loss (DRCB-CE), where deferred reweighting training schedule is applied based on the CE loss; (6) deferred reweighted
Focal loss (DRCB-
Focal), where a deferred reweighting training schedule applied based on the CB-Focal loss; (7) deferred reweighted Class-balanced LDAM loss (DRCB-LDAM) (Cao et al. 2019), where a deferred reweighting training schedule is applied based on the CB-LDAM loss. In addition, we also include the original PGD adversarial training method using cross entropy loss (CE) in our experiments.
Our proposed methods. We evaluate three variants of our proposed SRAT method with different implementations of the prediction loss C(.,-) in Eq. (5), i.e., CE loss, Focal loss and LDAM loss. The variant utilizing CE loss is denoted as SRAT-CE, and, similarly, other two variants are denoted as SRAT-Focal and SRAT-LDAM, respectively. For all these three variants, Class-balanced method (Cui ct al. 2019) is adopted to set weight values within the deferred reweighting training schedule.
Implementation details. All aforementioned methods are implemented using Pytorch library DeepRobust (Li ct al. 2020). For CIFARIO/CIFAL in training are calculated by PGD-10, with perturbation budget e 8/ 8/255 and step size 2/255; in evaluation, we report robust accuracy under 2 -norm 8/255 attacks generated by PGD-20 on Resnet- 18 (He et al. 2016) models. For SVHN based datasets, the settings step size y to 1/255 in both training and evaluation, as suggested in (Wu et al. 2020). We set the total training epochs to 200 and the initial learning rate to 0, and decay the learning rate at epoch 160 and 180 with the ratio 0.01. The deferred reweighting strategy will be applied starting from epoch 160.
based datasets, the adversarial examples used
similar, excepts set
5.2 PERFORMANCE COMPARISON
Table and Table 2 show the performance comparison on two different imbalanced CIFARIO datasets. In these two tables, we use bold values to denote the highest accuracy among all methods and use the underline values indicate our SRAT variants which achieve the highest accuracy among their corresponding baseline methods utilizing the same loss function for making predictions. Due to the limited space, we report the performance comparison on other imbalanced datasets created from CIFARIO, CIFAR100 and SVHN datasets in Appendix A.6
From Table D and Table 2 we can make the following observations. First, compared to baseline methods, our SR AT method obtains improved performance in terms of both overall standard & robust accuracy under almost all imbalanced settings. More importantly. SRAT makes significant improvements on those under-represented classes, especially under the extremely imbalanced settings. For the CIFAR1O Step-100 dataset, our SRAT-Focal method improves the standard accuracy on under-represented classes from 21.81% % achieved by the best baseline method utilizing Focal loss to 5 51.83% and robust accuracy from 3.24% to 15.89%. These results demonstrate that our SRAT method able to obtain more robustness under imbalanced settings. Second, the performance gap among
Thhe effective number defined as the volume of examples and can be calculated by (1 - 88' - B)) where 3 E [0, 1) is a hyperparameter and n, denotes the number of examples of class i.
'The implementations can be found via pettta://gite thw.ttttthu.e..e http:///////ooooooco//oreeaeeeee
7

---

Under review as a conference paper at ICLR 2022
Table 2: Performance comparison on the CIFARIO Step-100 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
47.29 + 0.32
9.03 + 0.99
30.39 0.24
1.62 0.41
Focal
47.36 + 0.19
9.03 = 0.52
30.12 0
1.45 = 0.12
LDAM
42,49 + 0.62
0.85 0.46
30.80 0031
0.05 = 0.06
CB-Reweight
37.68 + 1.18
19.64
25.58 0062
10.33 + 0.82
CB-Focal
15.44 3.85
0.00 0.00
14.46 3.16
0.00 0.00
DRCB-CE
53.40 + 1.20
1.00
28.31 = 0.59
3.35 0.56
DRCB-Focal
52.75 + 0.96
21.81 + 2.27
27.78 0.49
3.24 0.57
DRCB-LDAM
61,60 + 0.44
50.69 + 2.27
31,37 0.45
16.25 + 2.04
SRAT-CE
60.04 + 1.16
41.71
30.00 0.80
12.25 + 1.43
SRAT-Focal
62.93 + 1.10
51.83 + 3,33
28.38 1.00
15.89 + 3.15
SRAT-LDAM
63.13 + 1.17
52.73 + 3.23
33,51 0.68
18.89 + 0.59
three SRAT variants are mainly caused by the gap between the loss functions in these methods. As shown these two tables, DRCB-LDAM typically performs better than DRCE-CE and DRCB-Focaa and similarly, SRAT-LDAM outperforms SRAT-CE and SRAT-Focal under the same settings.
5.3 ABLATION STUDY
In this subsection, we provide ablation study to understand our SRAT method more comprehensively.
Feature space visualization. In order to facilitate the reweighting strategy in adver- sarial training under the im- balanced setting, we present feature separation loss in our SRAT method. The main goal of the feature separation loss is to enforce the learned feature space much sepa- rated as possible. For check- ing whether the feature sepa- ration loss can work as expected, we apply t-SNE (Van der Maaten & Hinton 2008 to visualize
Chean tharcaes
Dean bu
(a) CE.
(b) DRCB-LDAM.
(c) SRAT-LDAM.
Figure 4: t-SNE visualization of feature learned by different methods.
the latent feature space learned by our SRAT-LDAM method as well as by original PGD adversarial training method (CE) and DRCB-LDAM method in Figure 4
As shown in Figure 4 the feature space learned by our SRAT-LDAM method is more separable than two baseline methods, which demonstrates that, with our feature separation loss, the adversarially trained model able learn much better features and thus SRAT can achieve superiority performance.
Impact of weight values. As in all SRAT variants, we adopt the Class-balanced method (Cui et al. 2019) to assign different weights to different classes. To explore how the assigned weights impact the performance of SRAT. we conduct experiments using "Step-100" dataset to the change of model's performance using different reweighting values. Specifically, we assign well- represented classes with weight 1 and change the weight for under-represenntd classes from 10 to 200. The experimental results are shown in Figure 5 Here, we use approximation integer 78 to denote the weight calculated by the Class-balanced method when the imbalance ratio equals 100.
swa ca Star ALL pwir boal Dondeed
Siateface Ailal ACc Staft lnbest
70
20
10
25
30
75
300
13C
200
Figure 5: The impact of weights.
From Figure 5] we can obverse that, for all SRAT variants, the model's standard accuracy is increased with the increasing of the weights for redreseepeeeented classes. However, the robust accuracy for these three methods do not synchronize with the change of their standard accuracy. When increasing the weights for under-representedd classes, robust accuracy of SRAT-LDAM is almost unchanged and of SRAT-CE and SRAT-Focal even has slight decrease. As trade-off, using a relative large weight, such as 78 or 100, SRAT can obtain satisfactory performance on both standard & robust accuracy.
8

---

Under review as a conference paper at ICLR 2022
Impact of hyper-parameter A. In our SRAT method, the con- tributions of feature separation loss and prediction loss are con- trolled by hyper-parameter X. In this part, we study how this hyper-parameter affects the per- formance of SRAT. In experi- ments, we evaluate the models' performance of all SRAT variants with different values of À used in training process on both "Step--00" and 'Exp-100" datasets.
outer Symert
sas tor
Accurce Onl
000511
Va
Valuda for
Figure 6: The impact of the Exp-100 100.
(a) Step- 100.
(b)
A.
As shown in Figure the performance of all SRAT variants are not very sensitive with the choice of A. However, a large value of X, such as 8, may hurt the model's performance.
6 RELATED WORK
Adversarial Robustness. The vulnerability of DNN models to adversarial examples has been verified by many existing successful attack methods (Goodfellow et al. 2014 Carlini & Wagner 2017). To improve model robustness against adversarial attacks, various defense methods have been proposed (Madry et al. 2017 Raghunathan et al. 2018 Cohen et al. 2019). Among them, adversarial training has been proven to be one of the most effective defense methods (Athalye et al., 2018). Adversarial training can be formulated as solving a min-max optimization problem where the outer minimization process enforces the model to be robust adversarial examples, generated by the inner maximization process via some existing attacking methods like PGD (Madry et al. (017). Based on adversarial training. several variants, such as TRADES (Zhang et al. 2019). MART (Wang
itt al. 2019). have been presented to improve the model's performance further. More details about adversarial robustness can be found in recent surveys (Chakraborty et al.) 2018 Xu et al. 2020b).
Since almost all studies of adversarial training are focused on balanced datasets, it's worthwhile to investigate the performance of adversarial training methods on imbalanced training datasets.
Imbalanced Learning. Most existing works of imbalanced training can be roughly classified into two categories, i.e., re-sampling and reweighting. Re-sampling methods aim to reduce imbalance level through either over-sampling examples from under-represented classes (Buda et al.] 2018 Byrd & Lipton| 2019) or under-sampling examples from well-represented classes (Japkowicz &
Stephen 2002 Drummond et al.) 2003 He & Garcial 2009). reweighting methods allocate different
weights for different classes or even different examples. For example, Focal loss (Lin et al.) 2017)
enlarges the weights of wrongly -lassssidd examples while reducing the weights of well-classifed examples in the standard cross entropy loss; and LDAM loss (Cao et al. 2019) regularizes the under- represented classes more strongly than the well-represented classes to attain good generalization on under-represented classes. More details about imbalanced learning can be found in recent surveys (He) & Ma 2013 Johnson J013Johhsoo & Khoshgoftaar 2019). The majority of existing methods focused on the nature traming scenario and their trained models will be crashed when facing adversarial attacks (Szegedy al. 2013) Goodfellow et al., 2014). Hence, this paper, we develop a novel method that can defend adversarial attacks and achieve well-pleasing performance under imbalance settings.
7 CONCLUSION
In this work, we first empirically investigate the behavior of adversarial training under imbalanced settings and explore potential solutions to assist adversarial training tackling the imbalanced issue. As neither adversarial training itself nor adversarial training with reweighting can work well under imbalanced settings, we further theoretically verify that the poor data separability is one key reason causing the failure of adversarial training based methods. Based on our findings, we propose the Separable Reweighted Adversarial Training (SRAT) framework to facilitate the reweighting strategy in imbalanced adversarial training. We validate the effectiveness of SRAT via extensive experiments. In the future, we plan to examine how other types of defense methods perform under imbalanced scenarios and how other types of balanced learning methods behave under adversarial training.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security; Circumventing defenses adversarial examples. In International Conference on Machine Learning, pp. 273-283. PMLR, 2018.
Matcusz Buda, Atsuto Maki, and Macicj A Mazurowski. A systematic study of the class imbalance problem convolutional neural networks. Neural Networks, 106:249-2555 2018.
Jonathon Byrd and Zachary Lipton. What is the effect of importance weighting in deep learning? In International Conference on Machine Learning, pp. 872-881. PMLR, 2019.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with aabel-iitribution--waae margin loss. arXiv preprint arXiv: (900:0706555 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee sympositm on security and privacy (sp), pp. 39-57. IEEE, 2017.
Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopad- hyay. Adversarial attacks and defences: A survey. arXiv preprint arXiv: 18::.:800000 110.00069, 2018.
Chenyi Chen, Ari Seff, Alain Korhauser, and Jianxiong Xiao. Deepdriving: Learning affordance for direct perception in autonomous driving. In Proceedings of the IEEE international conference on computer vision, pp. 2722-2730, 2015.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Yin Cui, Menglin Jia, Tsung- Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IÉEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9268-99699999 91787,977, 2019.
Chris Drummond, Robert C Holte, al. C4. 5, class imbalance, and cost sensitivity: why under- sampling beats ever-sampling. In Workshop on learning from imbalanced datasets II, volume 11, pp. 1-8. Citeseer, 2003.
Andrew Estabrooks, Tacho Jo, and Nathalic Japkowicz. A multiple resampling method for learning from imbalanced data sets. Computational intelligence, 20(1):18-36 2004.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International jouraaall of computer vision, 88(2): 303-338, 2010.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6573 2014.
Haibo He and Edwardo A Garcia. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9):1263-122 1284, 2009.
Haibo He and Yunqian Ma. Imbalanced learning: foundations, algorithms, and applications. 2013.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition.
pp. 770-778, 2016.
Nathalie Japkowicz and Shaju Stephen. The class imbalance problem: A systematic study. Intelligent data analysis, 6(5):429 449, 2002.
Justin M Johnson and Taghi M Khoshgoftaar. Survey on deep learning with class imbalance. Journal of Big Data, 601):1-54, 2019.
Prannay Khosla, Piotr Teterwak, Chen Wang. Aaron Samaa, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. arXiv preprint arXiv:2004. 11362, 2020.
10

---

Under review as a conference paper at ICLR 2022
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv: 00003,, 2016.
Yaxin Li, Wei Jin, Han Xu, and Jiliang Tang. Deeprobust: A pytorch library for adversarial attacks and defenses. arXiv preprint vrXiv:2005.06149 2020.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection, In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1706. 06083, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
William S Noble. What is a support vector machine? Nature biotechnology, 24(12):1565-1567, 2006.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. arXiv preprint arXiv:1801.09344 2018.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data, arXiv preprint 1804.1880.1122 2018.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1rXiv:1312.6199 2013.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 911), 2008.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 33, 2020.
Da Xu, Yuting Ye, and Chuanwei Ruan. Understanding the role of importance weighting for deep learning. arXiv preprint arXiv: 103. 15209, 2021.
Han Xu, Xiaorui Liu, Yaxin Li, and Jiliang Tang. To be robust or to fair: Towards fairness in adversarial training. arXiv preprint arXiv:2010.06122, 2020a.
Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K Jain. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 1712)151 2020b.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482, PMLR, 2019.
11

---

Under review as a conference paper at ICLR 2022
A APPENDIX
In this section, we provide more details about the proposed SRAT framework, as well as the full experimental results of the preliminary study and evaluation.
A.1 THE BEНAVIIR ADVERSARIAL TRAINING
In order to examine the performance of PGD adversarial training under imbalanced scenarios, we
adversarially train ResNet18 (He et al. 2016) models multiple imbalanced training datasets based
on CI dataset (Krizhessky et al. 2009). Similar with observations we discussed in Section 2.T
shown in Figure 7 Figure and Figure adversarial training produces larger performance gap between well-represented classes and under-represented classes than natural training. Especially, in all imbalanced scenarios, adversarially trained models obtain very low robust accuracy under- represented classes, which proves again that adversarial training cannot be applied in practical imbalanced scenarios directly.
overal accurccy under acourayy
EG
60
90
20
owrO ACCAPRT voli acCerEEy
20
WET BOLLECCY
DOCUTRCY
Care
Cass
(a) Natural Training Standard Acc.
(b) Adv. Training Standard Acc.
(c) Adv. Training Robust Acc.
Figure 7: Class-wise performance of natural & adversarial training under an imbalanced CIFARIO dataset "Step-10""
100
100
accurscy under sccuaAy
60
ACCATRCY
20
(a) Natural Training Standard Acc.
(h) Adv. Training Standard Acc.
Adv. Training Robust Acc.
Figure 8: Class-wise performance of natural & adversarial training under an imbalanced CIFARIO dataset "Exp-100".
200 BOLUNNy wwllsccnccy
80
NO
150
30
UCCAFRCY vel BOLEENCY
24
Closs
Class
(a) Natural Training Standard Acc.
(b) Adv. Training Standard Acc.
(c) Adv. Training Robust Acc.
Figure 9: Class-wise performance of natural & adversarial training under an imbalanced CIFARIO dataset "Exp-10".
12

---

Under review as a conference paper at ICLR 2022
A.2 REWEIGHTING STRATEGY IN NATURAL TRAINING V.S. IN ADVERSARIAL TRAINING
For exploring whether the reweighting strategy can help adversarial training deal with imbalanced issues, we evaluate performance of adversarial trained models using diverse binary imbalanced training datasets with different weights assigning to under-represented class. As shown in Figure T0 Figure I1 Figure 12 for adversarially trained models, increasing the weights assigning to under- under-represented class. However, as
represented class will improve models' performance the same time, the models' performance on well-represented class will be drastically decreased. As comparison, adopting larger weights in naturally trained models will also improve models' performance on under-represented class but only result slight drop in performance on well- represented class. In other words, the reweighting strategy proposed in natural training handle imbalanced problem may only provide limited help in adversarial training, and, hence, new techniques are needed for adversarial training under imbalanced scenarios.
Acuncy
ovecal 1528 SCOUTN/
USSET scourcy
under scoutry
Wiigit
(a) Natural Training Standard Acc. (b) Adv. Training Standard Acc.
(c) Adv. Training Robust Acc.
Figure 10:
Class-wise performance of reweighted natural & adversarial training in binary classifica- tion. ("auto" as well-represented class and "truck" under-represented class).
60-
acorryy
30
vell acCurRCy
under sccurecy
unde Scourecy
Unde RCOUNCE
Waighhs
Weighs Undererpeenn
Wriglt ferunde isonEsiss
(a) Natural Training Standard Acc.
(b) Adv. Training Standard Acc.
(c) Adv. Training Robust Ace.
Figure 11: Class-wise performance of reweighted natural & adversarial training in binary classifica- tion. ("bird" well-represented class and "frog" under-represented class).
100
350
Acussey
60)
overc ocCuROy
wet acouary
unda SOUMAN
Whilghts
1.00
50
75
-50
230
110 34
35 100
193
301
(a) Natural Training Standard Acc.
(b) Adv. Training Standard Acc.
Training Robust Acc.
Class-wise performance of reweighted natural & adversarial training in binary classifica- tion. ("dog" as well-represented class and "deer" under-eepresented class).
Figure 12:
13

---

Under review as a conference paper at ICLR 2022
A.3 PROOFS THE THEOREMS IN SECTION
A.3.1 PROOF OF LEMMA 1.1
Lemma 3.1 Under the data distribution D as defined in Eq. 47). with an imbalanced ratio K and reweight ratio the optimal classifier which minimizes the (reweighted) empirical risk:
f" arg min (2x(f fyy== Pr. Pr.(y=11)
(r) ( ,yv== pr(y=+1)) (2)
has the solution: w and = log( f
do
log ))
Proof (Proof of Lemma 3.1
We will first prove that the optimal model f has parameters wn W2 wa (or 1) by contradiction. We define G = =1, 21...... .] and make the following assumption: for the optimal us and b, we assume if there exist wi < w for i # and i. E G. Then obtain the following standard errors for the class "-1" and the class "+1" R of this classifier f with weight w:
Pr:(f* (x:) + v = Pe(w" o(-
=Pr E wkN(- W.N(- -wN- -b 0}.
(8)
Pn( + sly +1) Pr(w" N(+n,o²) - -b<
=P Σ
WaN(+n, o2) w.N(+n, wyN(+n, -b)
kg
However, if we define a new classier f whose weight w uses wy to replace wa we obtain the errors for the new classifier:
Pr(f(z) # vlu = 1) 1)= PE( Σ waN((-n, LwNN wyN((nno bb> bb>0},
Pr((f(x) vlu +1) ) Pr Σ wwN(+n,, (o, w bb b<
Comparing the errors in Eq. error in each class. Therefore, it contradicts with the assumption that f is the optimal classifier with smallest error. Thus, we conclude for an optimal linear classifier in natural training, it must satisfies Wa (or w 1) if we do not consider the scale of w.
(8) and Eq. 99. w; W.y then the classifier f has smaller standard
W1 W2
Next, calculate the optimal bias term b given 1, where we find an optimal can minimize the (reweighted) empirical risk:
Error (J") = Pr.lS" ( (x) * vlu = - Prly = 1) Pr # vlu = +1) Pr = +1)
x Pr(ff' (x) + vly -1) p+ Pr.(f + v/v +1) KK
= Pr. N (-- 02) b> 0) + K Pr.
N(n. 0²) - 0)
b b+ dn - K Pr(N((0, 1) <
- dr
= PE(N((0 1)
da
do
and we take the derivative with respect to b:
EError
p.
(- (-11) exp(
1 {
b+ du
K
1 expp
1 du p.
ab
2 do
do
/2
do
2 do
When aError 10b O, can calculate the optimal b which gives the minimum value of the empirical error, and we have:
1
P.
do²
log d
b =
log
2
K 17
K 5
14

---

Under review as a conference paper at ICLR 2022
A.3.2 PROOF OF THEOREM
Theorem 3.1 Under two data distributions (x y E Di and ,y())))))))))) E D2 with different
separabilities S1 > S2. let fi and f2 be the optimal non-reweighted classifiers (p = 1) under D and D2, respectively. Given the imbalance ratio K is large enough, we have:
Pr(ff(x!!)) # 11) = -1) - Pr( (x # (0) = +11
(3)
< (r( & -y = -11 - (r( & 1101)(1)) =
Proof 2 (Proof of Theorem Without loss of generality, for distribution D2, D2 with different mean-vart mnean-variance pairs (+m,, of) and (#772, oï), we can only consider the case 1h T22 and of < oz. Otherwise,
of them match the mean vector of the other and will simply rescale not impact the results. Under this definition, the optimal classifier J and f has weight vector w1 W22 1 and bias term b by, with the value demonstrated in Lemma 3.7 Nex,, will prove the Theorem by 2 steps.
Step 1. For the error of class "-1", we have:
d W(-n, ((- -b 0)
= Pe(fi(z(1) # ) 11) Pr.
d
< Pr.
N( W(-1),0
(1-0) oî) - -b1 >>0 0) (because Si > -2))
i=1
Pr.
Σ ((11),0)) o22) - -b 0) (because of < <
Pr((f(( + (2))) y =1).
Step 2. For the error of class "+1", we have:
Pr( + 10) = +1) Pr.
W(n.oj) - < 0)
Po(N 1) < 01 du
(10)
do
= -Pr(N(0,1)< log K) .0
"
)
217
σ1
and similarly,
(x!2)) + 22) = +1) = Px.(00)) < -log( K) -02
17
(11)
2m
02
Note that when K is large enough, i.e. log(K) >
2-1²
we can get the Z-score in Eq.
10 is larger
01.02'
than Eq. TT7. As a result, we have:
(e(ff(x!")) + y = +1) -11 Pr( + y 22
+1).
(12)
By combining Step I and Step 2, we can get the inequality in Theorem 3.1
A.3.3 PROOF OF THEOREM 3.2
Theorem 3.2 Under two data distributions (z()), y0))) E D1 and (z!2), ,))))))))=)))))))))))) E D2 with different
separabilities S₁ > S2, let fi and f2 be the optimal now-reweighted classifiers (p= fp= 1) under Dj and Dz, respectively, and let fi" and f be the optimal reweighted classifiers under DD and D given the optimal reweighting ratio (p K). Given the imbalance ratio K is large enough, we have:
Pr(ff' (x(11) + (1) = +1) (r((f((x())) # (1)) =+1)
(4)
< Pr(ff' (x(2)) + 12) +1) (xx + 12)) +1).
15

---

Under review as a conference paper at ICLR 2022
We first show that under both distribution D1 and D2, the optimal reweighting ratio p is equal to the imbalance ratio K. Based on the results in Eq. ( and calculated model parameters and b, we have the test error (given the model trained by reweight value p):
Proof 3 (Proof of Theorem 3.2
Errorwal(")
Pr(S (x) # yly= -11 Pr(y -1) Pr.(f + wly +1) Px(y +1)
x Pr(N(O0, b b+ Pr(N(O,1) )-d1) do
do
Pr(N(O < log( ( - < <-2 log( k
The value of taking the minimum when its derivative with respect to is equal to O, where get p = K and the bias term Note that the variance values have relation: of < oz. Therefore,
it is easy to get that:
Pr. (x # t 10 +1)= PP:() (nn,o)) <0)
(13)
Pr.
N(n, 0 Pr.[22 (z. 9 +1).
Combining the results in Eq.
12 and
13
we have proved the inequality in Theorem 3.2
A.4 ALGORITHM OF SRAT
The algorithm of our proposed SRAT framework is shown in Algorithm Specifically, in each training iteration, we first generate ad adversarial examples using PGD for examples in the current batch (Line 5). If the current training iteration does not reach predefined starting reweighting epoch Ta, will assign same weights, i.e., (Line 6). Otherwise, the reweighting strategy will be adopted in the final loss function (Line 15), where specific weight w; will be assigned for each adversarial example x; if its corresponding clean example ii comes from an under-represented class.
I for all adversarial examples z; in the current batch
Algorithm Separable Reweighted Adversarial Training (SRAT).
Input: imbalanced training dataset D = {(I;, yt yt)) number of total training epochs T, starting reweighting epoch Ta, batch size N, number of batches M, learning rate ?
Output: An adversarially robust model fe Initialize the model parameters randomly;
2: for epoch 1..... Ta do
3: for mini-batch 1, 1.......... M do
4: Sample mini-batch B = {(x,, y1 J from D:
5: Generate adversarial example o for each af E B;
6: C(fe) 2/2 ΣN m ((fo(/)))))))))))))))))(()) m)) + ACeep(r))
7:
0 - 0 (o
8: end for
9: Optional: 7 +- 7/K
10: end for
11: for epoch Ta....... do
12: for mini-batch = I,..., M do
13:
Sample mini-bateh B {z,, y/ N from D; Generate adversarial example x for each n' E B;
14:
15:
C(fe) 1/4 Σ. max ((foo()) v:) Ao (2.)
16: 0 D- VoL((o)
17: end for
18: Optional: 7 7/N
19: end for
16

---

Under review as a conference paper at ICLR 2022
A.5 DATA DISTRIBUTION OF IMBALANCED TRAINING DATASETS
In our experiments, we construct multiple imbalanced training datasets to simulate various kinds of imbalanced scenarios by combining different imbalance types (i.e., Exp and Step) with different imbalanced ratios (i.e., K = 10 and K = 100). Figure 13 and Figure [4 show the data distribution of all ten-classes imbalanced training datasets used in our preliminary studies and experiments based on CIFARIO (Krizhevsky et al., 2009) and SVHN (Netzer et al. 2011 datasets, respectively.
(a) Step- 10
(b) Step- 100
(c) Exp-10
(d) Exp-100
Figure 13: Data distribution of imbalanced training datasets constructed from CIFARIO dataset.
(a) Step-10
(b) Step-100
(c) Exp-10
(d) Exp-100
Figure 14: Data distribution of imbalanced training datasets constructed from SVHN dataset.
A.6 PERFORMANCE COMPARISON
A.6.1 PERFORMANCE COMPARISON ON IMBALANCED CIFARIO DATASETS
Table 3 and Table show the performance comparison on two imbalanced CIFAR10 datasets. We use bold values to denote the highest accuracy among all methods and use the underline values to indicate our SRAT variants which achieve the highest accuracy among their corresponding baseline methods utilizing the same loss function for making predictions.
From Table 3 and Table 4 we get similar observation that, comparing with baseline methods, our proposed SRAT method can produce a robust model which can achieve improved overall performance when the training dataset is imbalanced. In addition, based on the experimental results Table to Table we find that. compared with the performance improvement between DRCB-LDAM and SRAT-LDAM. the improvement between DRCB-CE and SRAT-CE and the improvement between DRCB-Focal and SRAT-Focal
more obviously. The possible reason behind this phenomenon is, the LDAM loss can also implicitly produce a more separable feature space (Cao et al.) 2019) while CE loss and Focal loss do not conduct any specific operations on the latent feature space. Hence, the feature separation loss contained in SRAT-CE and SRAT-Focal could be more effective on learning separable feature space and facilitate the Focal loss on prediction. However, in SRAT-LDAM, the feature separation loss and LDAM loss may affect each other on learning feature representations and, hence, the effectiveness of the feature separation loss may be counteracted or weakened.
A.6.2 PERFORMANCE COMPARISON ON IMBALANCED SVHN DATASETS
We report the performance comparison various imbalanced SVHN datasets with different imbal- ance types and imbalance ratios from Table 5 to Table
Based on experimental results shown in Table 5 and Table S we find that the robust model trained by our SRAT method can achieve superior overall performance under various imbalanced scenarios, when comparing with their corresponding baselines methods which utilize the same prediction loss function. Hence, experiments conducted on various imbalanced SVHN datasets further verify the effectiveness of our SRAT method under diverse imbalanced scenarios.
17

---

Under review as a conference paper at ICLR 2022
Table 3: Performance comparison on the CIFARIO Exp-10 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
71.95 1 0.52
64.09 1 0.44
37.94 1 0.19
0.19
26.79 E 0.51
Focal
72.06 + 0.78
63.99 + 1.15
37.62 + 0.34
26.27 + 1.04
LDAM
67.39 + 1.00
58.01
2.26
0.32 41.35
28.65 + 0.83
CB-Rewiight
75.17 0.15
76.87 0.69
41.02.1.00
41.67 0.89
CB-Focal
74.73 0.41
76.67.1..
38.86 0.67
42.41 0.56
DRCB-CE
76.25 + 0.09
75.83 0:49
40.02 0.45
37.93 + 0.65
DRCB-Focal
75.36 13.36110.44 0.40
72.72 0.94
37.76 1 0.54
33.83 + 0.68
DRCB-LDAM
73.92 + 0.31
78.53 1: 1.24
46.29 1 0.46
48.81 + 0.54
SRAT-CE
76.74 + 0.15
78.61 0.63
42.39 + 0.71
43.37 + 0.38
SRAT-Focal
75.26 + 0.00
80.52
+ 0.00
42.37 + 0.00
47.22 + 0.00
SRAT-LDAM
74.63 0.00
79.82 1 0.00
46.72 1 0.00
50.38 0.00
Table 4: Performance comparison the CIFARIO Exp- 100 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
48.40 + 0.59
23.04 + 1.15
26.94 0.84
6.17 0.86
Focal
49.16 + 0.61
23.69 + 1.15
26.84 0.59
5.88 0.48
LDAM
48.39 + 0.99
25.69 = 1.35
29.51 = 0.27
8.95 = 0.45
CB-Reweight
57.49 + 0.58
56.47 + 1.67
29.01 = -0.30
26.53 + 1.27
CB-Focal
50.35 + 0.44
60.05 0.53
27.15 0.20
33,56 0.35
DRCB-CE
57.30 + 0.30
37.90 1.23
26.97 0.55
10.57 1.03
DRCB-Focal
54.76 + 0.30
31.79 + 1.30
25.24
0.39
7.81 = 0.87
DRCB-LDAM
62.65 + 0.50
57.19 + 2.10
31.66 0.56
22.11 + 1.70
SRAT-CE
64.29 1 0.46
61.81 + 183
29.99 0.43
24.09 1 0.98
SRAT-Focal
62.57 + 0.47
64,88 + 0.81
30.34 0.67
28.66 - 1.60
SRAT-LDAM
63.11 + 0.08
65,60 1.94
34,22 0.41
32.55 1.70
A.6.3 PERFORMANCE COMPARISON ON IMBALANCED CIFAR 100 DTASETS
Table 9 and Table TO show the performance comparison between our SRAT variants with baseline methods on two imbalanced CIFAR 100 datasets.
As shown in Table and Table TO although the increasing of total number of classes in the dataset brings more challenges on the model training process, our SRAT variants are able to achieve notable improvements on both standard accuracy and robust accuracy.
Therefore, in conclusion, experiments conducted on multiple imbalanced datasets created from three benchmark image datasets CIFAR10, CIFAR100 (Krizhevsky al., (009) and SVHN (Netzer et 2011) with diverse imbalanced distributions verify the effectiveness of our proposed SRAT method under various imbalanced scenarios.
A.7 ABLATION STUDY
Impact of weight values. We study the impact of weight values in Section (53 via evaluating the overall standard accuracy and ro- bust accuracy SRAT trained models with different weights assigning to under-represented classes. Here, we further inves- tigate the impact weight val- ues in our SART method from the class-wise wise perspective. In Fig- ure 15 report the standard & robust accuracy of models trained by our SRAT-LDAM variant with
100
100
Cyard Eccaeee
11.00
100
150
300
13.00
73
350
260
(a) Standard Accuracy
(b) Robust Accuracy.
Figure 15: The impact of weights for the SRAT-LDAM method.
18

---

Under review as a conference paper at ICLR 2022
Table 5: Performance comparison on the SVHN Step-10 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
79.88 1 0.17
67.04 1 0.37
37.62 1 0.10
22.08 E 0.26
Focal
79.96 + 0.15
67.03 + 0.38
37.83 0.17
22.47 + 0.43
LDAM
84.55 + 0.83
74.96 + 1.52
45.80+0.33
31.23 0.70
CB-Rewed CB-Rewiight
79.48 + 0.27
66.07 0.42
37.38 0.20
21.66 + 0.26
CB-Focal
80.29 0.15
67.56
38.10
23.00 1 0.38
DRCB-CE
80.62 + 0.25
68.74
37.25
0.15
22.79 + 0.42
DRCB-Focal
79.11 4 0.15
6S.72 0.44
37.01
0.33
22.02 + 0.55
DRCB-LDAM
87.83 + 0.68
82.63 1 1.17
46.45 1 0.30
35,15 + 0.69
SRAT-CE
82.89 + 0.91
72.79 + 1.93
38.23 + 0.78
24.70 + 1.62
SRAT-Focal
85.32 + 0.19
77.75 + 0.51
39.53 + 0.40
28.41 + 0.99
SRAT-LDAM
87.65 1 0.09
82.62 1 0.25
46.03 1 0.10
34.75 1 0.48
Table 6: Performance comparison on the SVHN Step-100 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
59.61 1.59
26.19 + 3.08
29.57 + 0.59
5.03 + 1.49
Focal
60.58 1.40
28.17
30.27 0.51
5.83 + 1.42
LDAM
65.61 + 0.43
37.13
33.34
8.36 + 0.54
CB-Reweight
60.23 + 1.74
27.68 + 3.49
0.76
5.32 + 1.40
CB-Focal
60.73 :: 0.40
1.66
30.09.0.0
5.75 0.61
DRCB-CE
60.67 1.04
28.36
30.02 1 1.04
5,59 1 1.30
DRCB-Focal
61.65 + 1.21
30.29 +
30.78 + 0.32
7.06 + 0.93
DRCB-LDAM
63.78 + 1.79
33.99 3.50
33.60 + 0.23
7.28 + 1.47
SRAT-CE
63.39 1 0.64
33.85 1.11
29.64 1 0.21
6.11 1 0.29
SRAT-Focal
69.27 + 1.38
45.50 + 3.16
31.58 + 0.70
9.87 + 1.61
SRAT-LDAM
71.56 + 1.25
50.33 + 2.29
33.54 + 0.52
11.63 + 1.09
different weight values on all ten classes, five well-represented classes and five under-representee classes of the Step- 100 CIFARIO dataset separately. Same as before, we assign well-represented classes with weight and change the weight for redreseepreeented classes from 10 200. In addition, we use approximation integer 78 to denote the weight calculated by the Class-balancee method (Cui al. 2019)
As shown in Figure 15 when increasing of the weights for under-represented classes, the model's s performance on under-tepresented classes can be increased greatly, with the cost of relatively small performance dropping well-represented classes, and the overall performance of the model is also increased. This phenomenon can be observed much clear in the standard accuracy case, as shown in Figure| 15a Recall that in our preliminary study (Section 2}. we find the reweighting strategy cannot work well in adversarial training, as it causes a strong tension of the model's performance between well-represented and under-represennee class. As this strong tension does not exist in the model trained by our SRAT method, we believe our SRAT method indeed facilitate the reweighting strategy in adversarial training under imbalanced scenarios.
Impact of imbalance ratio K. In previous experiments, we evaluate the effectiveness of our SRAT method using various imbalanced datasets with imbalance ratio K = 10 or K = 100. To investigate the performance of our SRAT method more comprehensively. in this part, we test our SRAT method on more imbalanced datasets with diverse imbalance ratios. Specifically, we construct series of "Step"" imbalanced CIFAR10 datasets by setting the value the imbalance ratio K from 5 to 100. For comparison, we apply both DRCB-Focas method and our SRAT- Focal variant to train models on those imbalanced datasets and test the trained models' performance on the original uniformly distributed CIFAR 10 test dataset. The experimental results are shown in Figure 16
Imbatance ratio
Figure 16: The impact of imbal- ance ratio K.
19

---

Under review as a conference paper at ICLR 2022
Table 7: Performance comparison on the SVHN Exp-10 10p-10 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
87.54 0.61
82.67 1 1.01
44.12 0.36
35.33 E 0.19
Focal
87.82 + 0.52
83.01 + 0.90
44.88 + 0.55
35.97 + 0.09
LDAM
90.06 + 0.59
86.69 0.70
+ +.68
43.73 +.97
CB-Rec CB-Rewiight
87.66 1 0.59
1.979
44.39
0.57
35.53 + 0.53
CB-Focal
87.86 0.53
82.96
14.6110.0
35.55 0.35
DRCB-CE
88.49 0.55
84.51 0.89
43.82 + 0.46
36.28 + 0.37
DRCB-Focal
87.47 1 0.48
82.78 0.72
42.52 1 0.60
34.31 + 0.54
DRCB-LDAM
91.24 + 0.57
89.65 0.70
52.39 1 0.74
46.71 +: 1.46
SRAT-CE
88.70 + 0.37
44.54 + 0.66
36.59 + 0.68
SRAT-Focal
89.36 + 0.44
85.93 + 0.44
0.44
45.41 + 0.55
38.18 + 0.79
SRAT-LDAM
91.27 0.46
89.55 1 0.66
52.10 1 0.85
46.13 1.23
Table 8: Performance comparison on the SVHN Exp-100 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
72.51 + 0.46
56.30 + 0.85
33.34 + 0.42
16.93 + 00.24
Focal
72.61 + 0.37
56.48 + 0.57
34.09+ 0.37
17.62 + 0.29
LDAM
79.11 0.74
66.86 1 1.14
40.42.0..0
25.18 1.29
CB-Reweight
55.97 0.84
33.36
17.16 0.77
CB-Focal
+ 0.50
34.25 + 0.37
17.90 + 0.53
DRCB-CE
0.53
58.03 1.14
33.52 0.13
17.68 + 0.40
DRCB-Focal
71.95 + 0.09
55.11
33.43 + 0.36
17.63 + 0.49
ORCB-LDAN
80.29 + 0.28
69.23
40.16 + 0.74
24.64 0.77
SRAT-CE
77.11 + 0.48
1.19
34.48 0.19
19.91 = 0.68
SRAT-Focal
81.30 0.91
72.26 1 2.00
36.71 1 0.53
24.84 1.59
SRAT-LDAM
80.71 0.40
70.49 0.71
40.33 + 0.43
25.11 0.58
obverse that, under different imbalanced scenarios, the model trained by SRAT-Focal can always achieve better performance than the one trained by DRCB-Fock method. In other words, the effectiveness of our SRAT method will not be affected by the imbalanced ratio K, which determines the data distribution of the imbalanced training dataset.
From Figure 16
20

---

Under review as a conference paper at ICLR 2022
Table 9: Performance comparison on the CIFARI00 Step- dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
39.90 + 0.11
17.90
0.38
17.88 1 0.32
6,40 = 0.60
Focal
40.10 + 0.27
17.99 + 0.75
17.67 + 0.30
6.40 + 0.18
IDAM
39.34 0.54
0.94
20.95 0.20
7.41 7.41+ 0.37
DRCB-CE
45.21 0.11
33.26
18.36 +
11.15 0.48
DRCB-Focal
44.28 0.15
30.57 0.22
17.30 + 0.39
9.73 + 0.18
DRCB-LDAM
44,70 + 0.46
35.90 0.92
21.80 + 0.12
15.19 + 0.36
SRAT-CE
47.17 0.26
37.81003
21.36 1 0.31
15.41 0.19
SRAT-Focal
46.83 + 0.28
38.10 + 0.58
21.66 + 0.32
16.52 + 0.32
SRAT-LDAM
45.41 + 0.55
36.39 + 0.65
23.15 + 0.15
16.84 + 0.08
Table Performance comparison on the CIFAR100 Exp-10 10 dataset.
Metric
Standard Accuracy
Robust Accuracy
Method
Overall
Under
Overall
Under
CE
41.88 + 0.36
31.30 + 0.57
16.62 + 0.03
11.22 + 0.21
Focal
41.64 +: 0.51
31.02
0.71
16.29 1 0.18
10.97 +: 0.34
LDAM
41.55 + 0.60
31.74
20.20
0.20
14.71 0.51
DRCB-CE
43.89 0.26
37.28
0.29
16.90
13.62 0.14
DRCB-Focal
43.38 0.30
36.17 0.57
+ 0.18
12.56 0.27
DRCB-LDAM
43.36 + 0.48
39.27
0.72
20.36 + 0.30
17.63 + 0.38
SRAT-CE
45.84 0.18
21.20 0.53
19.23 0.36
BRAT-Foca
46.38 + 0.28
42.53 0.79
20.09 + 0.25
17.83 + 0.56
SRAT-LDAM
44.98 + 0.33
40.39 + 0.69
21.83 + 0.33
18.99 + 0.59
21


Reviews:
[{"review_id": "y4uYjWJLlmv", "reviewer": "Reviewer_6i9F", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1.\tThis paper is well written and organized. The proposed method SRAT is well motivated by the empirical observations and theoretical analyses.\n2.\tThe experiments are conducted comprehensively on various imbalanced datasets and the efficacy of SRAT is justified by the improved performance on overall classes and under-represented classes. The visualized feature space clearly demonstrates the influence of feature separation loss.\n\nWeaknesses:\n1.\tThe proposed feature separation loss is somewhat trivial, which is very similar to the objective of self-supervised contrastive learning. There are many other ways to help feature become more separated, e.g., channel-wise activation suppressing [1], large-margin softmax [2]. Therefore, I wonder why the authors here utilize the contrastive loss instead of other methods that help improve feature separability.\n\n[1] Improving Adversarial Robustness via Channel-wise Activation Suppressing, ICLR 2021\n\n[2] Large-Margin Softmax Loss for Convolutional Neural Networks, ICML 2016\n", "summary_of_the_paper": "This paper provides two critical observations. The first is that adversarial training has worse performance on under-represented classes than natural training on imbalanced datasets. The second is that conventional reweighting methods that upweight under-represented classes largely hurt the performance on well-represented classes for adversarial training. Further, the authors theoretically analyze the reason for the above observations on a binary-classification case. Motivated by the theoretical analysis, the authors incorporate standard adversarial training with feature separation loss, namely SRAT, which is empirically validated SRAT can improve the performance on imbalanced datasets.", "summary_of_the_review": "Overall, this paper firstly investigates adversarial training in a new scenario under imbalanced datasets and propose to improve feature separability for adversarial training that should enhance the performance on imbalanced datasets.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "r4XC0CJvUc", "reviewer": "Reviewer_2Vzh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: This paper provide a novel method SRAT for adversarial training on imbalanced dataset, SRAT outperformed the vanilla adversarial training method on imbalanced CIFAR10 benchmark. The authors provide enough details and code for reproducing the results.\n\nWeakness:  The experiments in this paper is poorly conducted. \na) The authors only evaluate their model on CIFAR10 benchmarks, the effect of the proposed methods on large scale benchmarks e.g. ImageNet should also be considered.\nb) Only PGD-10 $l_\\infty$ $8/255$ attacks are evaluated in this paper. Different attacks settings e.g. $\\epsilon = 4/255$, $l_2$-norm attacks, should also be considered. It would also be better if the authors can evaluate SRAT on other types of attacks listed in [a]. I understand that evaluating SRAT on all types of attacks is computationally expensive, but selecting two or three of the SOTA attacks on image classification will increase the strength of the results.\nc) Missing confidence score. The training results of the model is depend on the random initialization and especially, in this paper, the under accuracy is related to the under-represented classes, where the images are sampled randomly from the original class. Thus I believe report the confidence score on under-represented classes with different random seed is necessary.\n\nI have a question regarding to the proof of Lemma 1, the authors omit the detailed calculation of comparison of Eq. (8) and Eq. (9), but from my perspective, the errors of Eq. (9) are less than Eq. (8) is not so obviously. Although I agree that intuitively $w=\\mathbf{1}$ should be the optimal classifier, the detailed discussion should be added for better understanding the proof. Maybe assuming $\\mu = (\\eta,0,…,0)$ makes it easier. \n\nI also don’t understand the proof of theorem 3.2, when calculating the $Error_{test}(f^*)$, the authors canceled $Pr.(y=-1)$ and $Pr(y=+1)$ in the second line of the formula and optimising a proportion of the original $Error_{test}(f^*)$ (in the third line of the formula). However, as $Pr.(y=+1) = KPr.(y=-1)$, there is no way to cancel them together without introducing a constant K as the coefficient of $Pr.(N(0,1)<(b-d\\eta)/(d\\sigma))$. I think the optimising objective should be $Pr.(N(0,1)<-(b+d\\eta)/(d\\sigma))+KPr.(N(0,1)<(b-d\\eta)/(d\\sigma))$ instead of $Pr.(N(0,1)<-(b+d\\eta)/(d\\sigma))+Pr.(N(0,1)<(b-d\\eta)/(d\\sigma))$.\n\nBesides, theorem 3.1 and theorem 3.2 hold with only a large enough imbalance ratio $K$, which is grown exponentially with $\\eta^2/(\\sigma_1\\sigma_2)$. I’m worried about the scale of $K$ in empirical dataset e.g. CIFAR10, due to the curse of dimension in high dimensional space, $\\sigma_1$ and $\\sigma_2$ has to be small enough ($O(1/\\sqrt{d})$) in order to locate in the input space $[0,1]^{d}$, in this case $K$ would become $O(e^d)$, which is invalid comparing to the scale of CIFAR10 dataset. Thus the empirical algorithm of SRAT is not theoretically guaranteed.\n\n[a] Xu et al Adversarial Attacks and Defenses in Images, Graphs and Text: A Review\n", "summary_of_the_paper": "This paper propose a new adversarial training algorithm, Separable Reweighted Adversarial Training (SRAT), for imbalanced datasets.", "summary_of_the_review": "Please check my review details. I would rate the paper as: score 4 weakly rejected\n\nI will consider to improve the score if the authors can address my concerns.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "JRd64FsWTVh", "reviewer": "Reviewer_mB3v", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I find the premise of this paper, and the resulting observations about the effect of dataset imbalance on AT, interesting. My main concerns with the paper are as follows:\n\n(i) Significance of the findings: \n\n- The claims in theorem 3.1 and 3.2 do not seem too surprising to me. These behaviors are clear even when we consider the two extremes: (a) when the two classes are very-well separated there is no difference in their performance (b) when the classes are close by, the majority class will push the decision boundary towards the minority class.\n- In light of this, and given that AT can be viewed as enforcing large margin requirements (i.e., decreasing separation between classes), the fact that adversarially trained models are more significantly affected by dataset imbalance follows fairly naturally.\n\n(ii) Motivation for SRAT:  The authors do not provide sufficient justification for the loss term introduced in (6), or explain why the default cross-entropy loss would not already induce clustering to the extent possible.\n\n(iii) Validity of the experimental results:\n\n- The difference between the overall performance of the proposed method and prior approaches is small and inconsistent (e.g., the SVHN results in the Appendix).\n- The authors propose using a deferred reweighting training schedule with their method, however it is not clear also apply this to the baseline methods.\n- Given that the size of the dataset w.r.t. the minority class is fairly small (50-500) examples, there is likely a great degree of variance between runs (due to different samplings of the dataset). Thus it is not clear whether the improvements are statistically significant. The authors should report confidence intervals across multiple runs to clarify this.\n- Theorem 3.2 predicts that for better separated features, the drop in accuracy on the majority class post re-weighting should be less significant. However from Table 1 and 2, it appears that compared to baselines, the majority class suffers more (similar overall performance and greater improvement on the minority class).\n- Are all methods trained to convergence? The train (standard/robust) accuracy for the majority/minority classes should be reported in the appendix.\n\nOther comments: \n\n- The authors should clarify what \"well\" and \"under\" accuracies refer to in the figure captions.\n- The use of t-SNE to justify claims regarding feature separation can be misleading as has been noted in prior work [[https://distill.pub/2016/misread-tsne/](https://distill.pub/2016/misread-tsne/)]. In general, given the high-dimensional nature of the data, conclusions drawn based on their two-dimensional projections could be incorrect. For instance, from Figure 3, it would seem that the two classes are not separable in feature space for adversarially trained models. However, in Figure 2, it is apparent that a linear classifier trained on this feature space does get good standard accuracy.", "summary_of_the_paper": "This paper studies the effect of adversarial training (AT) in the context of imbalanced datasets. In particular, the authors demonstrate empirically that adversarially trained models tend to have a larger gap in accuracy between well and under represented classes, as compared to their standard counterparts. Moreover, they show that standard reweighting strategies do not considerably alleviate this issue. Motivated by their findings, they propose adding a regularization function to adversarial training to encourage greater feature separation between classes (SRAT), and study its effectiveness on unbalanced datasets.", "summary_of_the_review": "While the premise of this paper is interesting, I believe that both the motivation for the SRAT method, and its empirical validation could be substantially improved.\n\n\n## Post-rebuttal Update\n\nI thank the authors for their response to my comments. I have decided to keep my original score as some of my concerns still hold. In particular:\n1. With the new results that the authors have added which show the baseline losses (CE, LDAM, focal) with deferred reweighting, the difference between the proposed approaches and prior work is even less significant and lies entirely within confidence intervals. \n2. Figure 15 does not address my concern about the over/under accuracies in Table 1 and 2. In particular, if SRAT indeed led to better separation, one would expect a comparable or lower drop in over accuracy compared to prior work. However, this seems not to be the case as per Table 1 and 2. For example, DRCB-focal and SRAT-focal, have similar overall accuracy, but the latter has higher under (and thus lower over) accuracy. More broadly, this links to my other concern that the loss proposed is not sufficiently motivated, and it is unclear that it leads to better clustering of features.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Bt1BXVPvjJ", "reviewer": "Reviewer_9erX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strengths:\n\n-  Experimental validation is conducted both in the step and exponential imbalance.\n-  The motivation for studying the imbalanced datasets and their adversarial training is clear.  \n-  The preliminary study in sec. 2 exhibiting the problem is useful.\n\n# Questions:\n\nThe following parts were not clear while studying the manuscript:\n\n-  Given that several sections of this work are focused on the reweighting strategy, e.g. sec. 2.2, 4.1, I am wondering about its significance in the first place; is it the most popular in training on imbalanced datasets? \n-  The analysis in sec. 3 is on linear models, and I am wondering whether the analysis can be extended to non-linear models (e.g. neural networks used in the experiments) or how it is relevant for the proposed losses in sec. 4. Is this added for intuition only?\n-  In addition, the analysis in sec. 3 is not studied for an adversarial training objective (e.g. as in eq. 1), so how is this informative for the losses in sec. 4? \n-  What is the contribution of the two losses in isolation? That is, what is the significance of the reweighting scheme alone and what is the contribution of the separation loss alone? There is a small ablation study, in Fig. 6, but I believe this should be extended further and analyzed.  \n-  The experimental setup seems rather weak for an empirical paper. Namely: \n    -  Both datasets only include 10 classes, while in practical scenarios many datasets have orders of magnitude more classes. It is recommended to conduct experiments in CIFAR100 or Tiny-Imagenet. \n    -  Both datasets include 32x32 images, while in practice images of higher resolution are used for real-world tasks (that is the stated goal for imbalanced dataset study). \n    -  CIFAR10 and SVHN are artificially imbalanced datasets, would it be possible to extend the experiments to naturally imbalanced datasets? It is possible that having a natural imbalance has a semantic meaning that currently is lost with artificial imbalance. \n    -  Would the results differ in case the attacks had an L_2 norm instead of an L_infinity? Is it possible to include such experiments in the revised version? Such attacks are also quite popular even in PGD. \n    -  What is the computational cost of the separation loss introduced in sec. 4.2? In every iteration it includes a large sum operation in the denominator, which might be costly. \n    -  I notice that the LDAM has a marginal improvement in the case of imbalance ratio 10; is there any intuition about this? Does it work better only under extreme imbalance ratio? \n    -  Is it possible to conduct multiple experiments and report the mean and variance in the tables and the figures? In my experience, adversarial training can have quite some variance.  \n \n\n\n# Minor corrections: \n-  The ‘well accuracy’ in (almost) all the figures is confusing; for most of the figures it could be unified under a unique legend (e.g. in Fig. 1) that can be placed above the image. \n-  ‘The adversarially perturbations’ (sec. 4.1). \n-  ‘have been prove superiority’ (sec. 4.1). \n-  ‘all input examples excepts’ (sec. 4.2). \n-  ‘makes significantly improvement’ (sec. 5.2).\n-  ‘using a relative large weights’ (sec. 5.3). \n", "summary_of_the_paper": "The paper focuses on adversarial training for imbalanced datasets. Imbalanced datasets emerge frequently in real-world applications, while as the paper demonstrates the adversarial training on the under-represented classes results in weak performance. The paper showcases that when the classes are not separable, then a linear model in under-represented classes is weak. Two modifications in adversarial training are proposed to ameliorate that: a) a weighted average of the loss per sample (depending on the class this sample belongs into), b) an additional loss that aims in maximizing the class separation.  ", "summary_of_the_review": "Currently, the paper includes several flaws, but many of them (e.g. the proofreading, or improvement of the legends) can be fixed. If the experimental section is strengthened and the questions clarified, I will consider my rating again. However, I am not convinced yet of the novelty of the proposed model, other than demonstrating that the problem with adversarial training on imbalanced datasets exists. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Three out of the four reviewers raised various concerns on motivation clarify, result significance, and unclear writing. While the authors provided their rebuttals, unfortunately no reviewer seems to have changed their mind. AC reads the paper and agreed this paper perhaps needs major revision before publishing in a major venue. However, the technical ideas are still interesting and promising; the authors are suggested to carefully take into account reviewer comments during revision.
Decision: Reject