Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper propose a new adversarial training algorithm, Separable Reweighted Adversarial Training (SRAT), for imbalanced datasets.
main_review: Strength: This paper provide a novel method SRAT for adversarial training on imbalanced dataset, SRAT outperformed the vanilla adversarial training method on imbalanced CIFAR10 benchmark. The authors provide enough details and code for reproducing the results.

Weakness:  The experiments in this paper is poorly conducted. 
a) The authors only evaluate their model on CIFAR10 benchmarks, the effect of the proposed methods on large scale benchmarks e.g. ImageNet should also be considered.
b) Only PGD-10 $l_\infty$ $8/255$ attacks are evaluated in this paper. Different attacks settings e.g. $\epsilon = 4/255$, $l_2$-norm attacks, should also be considered. It would also be better if the authors can evaluate SRAT on other types of attacks listed in [a]. I understand that evaluating SRAT on all types of attacks is computationally expensive, but selecting two or three of the SOTA attacks on image classification will increase the strength of the results.
c) Missing confidence score. The training results of the model is depend on the random initialization and especially, in this paper, the under accuracy is related to the under-represented classes, where the images are sampled randomly from the original class. Thus I believe report the confidence score on under-represented classes with different random seed is necessary.

I have a question regarding to the proof of Lemma 1, the authors omit the detailed calculation of comparison of Eq. (8) and Eq. (9), but from my perspective, the errors of Eq. (9) are less than Eq. (8) is not so obviously. Although I agree that intuitively $w=\mathbf{1}$ should be the optimal classifier, the detailed discussion should be added for better understanding the proof. Maybe assuming $\mu = (\eta,0,…,0)$ makes it easier. 

I also don’t understand the proof of theorem 3.2, when calculating the $Error_{test}(f^*)$, the authors canceled $Pr.(y=-1)$ and $Pr(y=+1)$ in the second line of the formula and optimising a proportion of the original $Error_{test}(f^*)$ (in the third line of the formula). However, as $Pr.(y=+1) = KPr.(y=-1)$, there is no way to cancel them together without introducing a constant K as the coefficient of $Pr.(N(0,1)<(b-d\eta)/(d\sigma))$. I think the optimising objective should be $Pr.(N(0,1)<-(b+d\eta)/(d\sigma))+KPr.(N(0,1)<(b-d\eta)/(d\sigma))$ instead of $Pr.(N(0,1)<-(b+d\eta)/(d\sigma))+Pr.(N(0,1)<(b-d\eta)/(d\sigma))$.

Besides, theorem 3.1 and theorem 3.2 hold with only a large enough imbalance ratio $K$, which is grown exponentially with $\eta^2/(\sigma_1\sigma_2)$. I’m worried about the scale of $K$ in empirical dataset e.g. CIFAR10, due to the curse of dimension in high dimensional space, $\sigma_1$ and $\sigma_2$ has to be small enough ($O(1/\sqrt{d})$) in order to locate in the input space $[0,1]^{d}$, in this case $K$ would become $O(e^d)$, which is invalid comparing to the scale of CIFAR10 dataset. Thus the empirical algorithm of SRAT is not theoretically guaranteed.

[a] Xu et al Adversarial Attacks and Defenses in Images, Graphs and Text: A Review

summary_of_the_review: Please check my review details. I would rate the paper as: score 4 weakly rejected

I will consider to improve the score if the authors can address my concerns.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper studies the effect of adversarial training (AT) in the context of imbalanced datasets. In particular, the authors demonstrate empirically that adversarially trained models tend to have a larger gap in accuracy between well and under represented classes, as compared to their standard counterparts. Moreover, they show that standard reweighting strategies do not considerably alleviate this issue. Motivated by their findings, they propose adding a regularization function to adversarial training to encourage greater feature separation between classes (SRAT), and study its effectiveness on unbalanced datasets.
main_review: I find the premise of this paper, and the resulting observations about the effect of dataset imbalance on AT, interesting. My main concerns with the paper are as follows:

(i) Significance of the findings: 

- The claims in theorem 3.1 and 3.2 do not seem too surprising to me. These behaviors are clear even when we consider the two extremes: (a) when the two classes are very-well separated there is no difference in their performance (b) when the classes are close by, the majority class will push the decision boundary towards the minority class.
- In light of this, and given that AT can be viewed as enforcing large margin requirements (i.e., decreasing separation between classes), the fact that adversarially trained models are more significantly affected by dataset imbalance follows fairly naturally.

(ii) Motivation for SRAT:  The authors do not provide sufficient justification for the loss term introduced in (6), or explain why the default cross-entropy loss would not already induce clustering to the extent possible.

(iii) Validity of the experimental results:

- The difference between the overall performance of the proposed method and prior approaches is small and inconsistent (e.g., the SVHN results in the Appendix).
- The authors propose using a deferred reweighting training schedule with their method, however it is not clear also apply this to the baseline methods.
- Given that the size of the dataset w.r.t. the minority class is fairly small (50-500) examples, there is likely a great degree of variance between runs (due to different samplings of the dataset). Thus it is not clear whether the improvements are statistically significant. The authors should report confidence intervals across multiple runs to clarify this.
- Theorem 3.2 predicts that for better separated features, the drop in accuracy on the majority class post re-weighting should be less significant. However from Table 1 and 2, it appears that compared to baselines, the majority class suffers more (similar overall performance and greater improvement on the minority class).
- Are all methods trained to convergence? The train (standard/robust) accuracy for the majority/minority classes should be reported in the appendix.

Other comments: 

- The authors should clarify what "well" and "under" accuracies refer to in the figure captions.
- The use of t-SNE to justify claims regarding feature separation can be misleading as has been noted in prior work [[https://distill.pub/2016/misread-tsne/](https://distill.pub/2016/misread-tsne/)]. In general, given the high-dimensional nature of the data, conclusions drawn based on their two-dimensional projections could be incorrect. For instance, from Figure 3, it would seem that the two classes are not separable in feature space for adversarially trained models. However, in Figure 2, it is apparent that a linear classifier trained on this feature space does get good standard accuracy.
summary_of_the_review: While the premise of this paper is interesting, I believe that both the motivation for the SRAT method, and its empirical validation could be substantially improved.


## Post-rebuttal Update

I thank the authors for their response to my comments. I have decided to keep my original score as some of my concerns still hold. In particular:
1. With the new results that the authors have added which show the baseline losses (CE, LDAM, focal) with deferred reweighting, the difference between the proposed approaches and prior work is even less significant and lies entirely within confidence intervals. 
2. Figure 15 does not address my concern about the over/under accuracies in Table 1 and 2. In particular, if SRAT indeed led to better separation, one would expect a comparable or lower drop in over accuracy compared to prior work. However, this seems not to be the case as per Table 1 and 2. For example, DRCB-focal and SRAT-focal, have similar overall accuracy, but the latter has higher under (and thus lower over) accuracy. More broadly, this links to my other concern that the loss proposed is not sufficiently motivated, and it is unclear that it leads to better clustering of features.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper focuses on adversarial training for imbalanced datasets. Imbalanced datasets emerge frequently in real-world applications, while as the paper demonstrates the adversarial training on the under-represented classes results in weak performance. The paper showcases that when the classes are not separable, then a linear model in under-represented classes is weak. Two modifications in adversarial training are proposed to ameliorate that: a) a weighted average of the loss per sample (depending on the class this sample belongs into), b) an additional loss that aims in maximizing the class separation.  
main_review: # Strengths:

-  Experimental validation is conducted both in the step and exponential imbalance.
-  The motivation for studying the imbalanced datasets and their adversarial training is clear.  
-  The preliminary study in sec. 2 exhibiting the problem is useful.

# Questions:

The following parts were not clear while studying the manuscript:

-  Given that several sections of this work are focused on the reweighting strategy, e.g. sec. 2.2, 4.1, I am wondering about its significance in the first place; is it the most popular in training on imbalanced datasets? 
-  The analysis in sec. 3 is on linear models, and I am wondering whether the analysis can be extended to non-linear models (e.g. neural networks used in the experiments) or how it is relevant for the proposed losses in sec. 4. Is this added for intuition only?
-  In addition, the analysis in sec. 3 is not studied for an adversarial training objective (e.g. as in eq. 1), so how is this informative for the losses in sec. 4? 
-  What is the contribution of the two losses in isolation? That is, what is the significance of the reweighting scheme alone and what is the contribution of the separation loss alone? There is a small ablation study, in Fig. 6, but I believe this should be extended further and analyzed.  
-  The experimental setup seems rather weak for an empirical paper. Namely: 
    -  Both datasets only include 10 classes, while in practical scenarios many datasets have orders of magnitude more classes. It is recommended to conduct experiments in CIFAR100 or Tiny-Imagenet. 
    -  Both datasets include 32x32 images, while in practice images of higher resolution are used for real-world tasks (that is the stated goal for imbalanced dataset study). 
    -  CIFAR10 and SVHN are artificially imbalanced datasets, would it be possible to extend the experiments to naturally imbalanced datasets? It is possible that having a natural imbalance has a semantic meaning that currently is lost with artificial imbalance. 
    -  Would the results differ in case the attacks had an L_2 norm instead of an L_infinity? Is it possible to include such experiments in the revised version? Such attacks are also quite popular even in PGD. 
    -  What is the computational cost of the separation loss introduced in sec. 4.2? In every iteration it includes a large sum operation in the denominator, which might be costly. 
    -  I notice that the LDAM has a marginal improvement in the case of imbalance ratio 10; is there any intuition about this? Does it work better only under extreme imbalance ratio? 
    -  Is it possible to conduct multiple experiments and report the mean and variance in the tables and the figures? In my experience, adversarial training can have quite some variance.  
 


# Minor corrections: 
-  The ‘well accuracy’ in (almost) all the figures is confusing; for most of the figures it could be unified under a unique legend (e.g. in Fig. 1) that can be placed above the image. 
-  ‘The adversarially perturbations’ (sec. 4.1). 
-  ‘have been prove superiority’ (sec. 4.1). 
-  ‘all input examples excepts’ (sec. 4.2). 
-  ‘makes significantly improvement’ (sec. 5.2).
-  ‘using a relative large weights’ (sec. 5.3). 

summary_of_the_review: Currently, the paper includes several flaws, but many of them (e.g. the proofreading, or improvement of the legends) can be fixed. If the experimental section is strengthened and the questions clarified, I will consider my rating again. However, I am not convinced yet of the novelty of the proposed model, other than demonstrating that the problem with adversarial training on imbalanced datasets exists.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper provides two critical observations. The first is that adversarial training has worse performance on under-represented classes than natural training on imbalanced datasets. The second is that conventional reweighting methods that upweight under-represented classes largely hurt the performance on well-represented classes for adversarial training. Further, the authors theoretically analyze the reason for the above observations on a binary-classification case. Motivated by the theoretical analysis, the authors incorporate standard adversarial training with feature separation loss, namely SRAT, which is empirically validated SRAT can improve the performance on imbalanced datasets.
main_review: Strengths:

1.	This paper is well written and organized. The proposed method SRAT is well motivated by the empirical observations and theoretical analyses.
2.	The experiments are conducted comprehensively on various imbalanced datasets and the efficacy of SRAT is justified by the improved performance on overall classes and under-represented classes. The visualized feature space clearly demonstrates the influence of feature separation loss.

Weaknesses:
1.	The proposed feature separation loss is somewhat trivial, which is very similar to the objective of self-supervised contrastive learning. There are many other ways to help feature become more separated, e.g., channel-wise activation suppressing [1], large-margin softmax [2]. Therefore, I wonder why the authors here utilize the contrastive loss instead of other methods that help improve feature separability.

[1] Improving Adversarial Robustness via Channel-wise Activation Suppressing, ICLR 2021

[2] Large-Margin Softmax Loss for Convolutional Neural Networks, ICML 2016

summary_of_the_review: Overall, this paper firstly investigates adversarial training in a new scenario under imbalanced datasets and propose to improve feature separability for adversarial training that should enhance the performance on imbalanced datasets.
review_title: null

Meta-review: Three out of the four reviewers raised various concerns on motivation clarify, result significance, and unclear writing. While the authors provided their rebuttals, unfortunately no reviewer seems to have changed their mind. AC reads the paper and agreed this paper perhaps needs major revision before publishing in a major venue. However, the technical ideas are still interesting and promising; the authors are suggested to carefully take into account reviewer comments during revision.
Decision: Reject