Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper focuses on adversarial training with direct attacks that effectively reduce the information about the true classes. It presents interesting observations that adversarial training might hurt robustness performance under limited training sample sizes. Theoretical analysis with main theorems provided on linear classifier case study and the generalizability is discussed. Experimental results on the customized Waterbirds,  CIFAR10, and hand-gesture datasets support the claims.
strength_and_weaknesses: ### Strength
- The paper presents very interesting observations that can potentially arouse wide attention for the practical usage of adversarial training.
- Solid theoretical analysis on linear classifiers with good intuitions provided.
- Experimental results focus on a customized Waterbirds dataset with motion blur and illumination and the results confirm the claims. CIFAR10 and hand-gesture dataset with square mask perturbations are also investigated.

### Weakness
- More types of direct attacks could be potentially evaluated, e.g., physical attacks on stop signs, to make the claims more convincing.
clarity,_quality,_novelty_and_reproducibility: The paper is novel with high-quality theoretical analysis and empirical evaluation.
summary_of_the_review: Overall, the paper provides good observations on adversarial training with the directed attack. The authors provide solid theoretical analysis and sufficient empirical evaluations to support the claims.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors tackle the problem of robust training. They show that when the training set is small robust training can hurt the robust performance of the model. They start by theoretically showing this result for linear classifiers. Using these results they also design directed attacks and show that robust error can indeed increase when the dataset is small. 
strength_and_weaknesses: Strengths
1. The results presented in this paper are interesting and does indeed caution user from blindly applying standard ML techniques for adversarial training. 
2. The theoretical aspect of the paper is sound and I prefer the way the authors draw inspiration from the linear classifiers and extend that to non-linear ones.
3. The experimental results are also neat and clearly convey the message the authors are trying to provide.

Weaknesses
1. While the results are neat, I would have also liked to seen a more detailed discussion regarding in which scenario these learnings would be useful. The experimental results are for image classifiers which usually require a large amount of data both for training from scratch and for transfer learning. Thus it is unclear to me where the gain is useful. As the authors themselves mention, one of the main reasons for this degradation would be catastrophic overlearning which is intuitive on its own. 
2. One potential way to strengthen the findings of this paper would be to have a relation on when this robust error increase happens in terms of d and n. That aspect is currently missing from this paper.
clarity,_quality,_novelty_and_reproducibility: The paper is well written and easy to follow. The ideas are presented well and in a clean manner. While the main results of this paper go against general ML knowledge, some of it can also be considered intuitive. The authors provide some details about the experiments they have done but more details are required to completely reproduce the results.
summary_of_the_review: This paper cautions ML practitioners against blindly applying ML techniques for robust learning. However, the scope of the application may not be large as most current ML techniques use a large dataset to train. This significantly takes away from the contributions of this paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors study the impact of adversarial training on both the clean and robust error in the low-sample regime using a special type of adversarial attacks, termed directed attacks. The main message is that adversarial training may even hurt the robust accuracy, in addition to the well-known effect of reducing the clean accuracy. The authors analyze the linearly-separable binary classification problem, where directed attacks are easily defined in relation to the optimal separator, and use the robust max-$\ell_2$-margin suggesting to decompose the robust error into a clean error term and a susceptibility term. The analysis, along with experiments, is aimed to elucidate the effect of the train/test budgets on each type of error. The findings of this analysis are then extended to the multi-class image classification problem using deep CNNs where directed attacks are generalized to encompass a number of image corruptions. Results on standard and customized image datasets are claimed to corroborate the theoretical and empirical findings from earlier.
strength_and_weaknesses: - Strengths
  - Brings attention to a theoretically-interesting aspect of adversarial training, that could be relevant in practice.
  - Presents theoretical analysis in the linear 2-class case, verified by corresponding experiments.
  - Applies the results to a restricted class of adversarial attacks on image datasets, (claiming to) corroborate the theory.
- Weaknesses
  - The presentation needs significant work ~~,to the point that I'm unable to grasp the core technical points or the empirical evaluations.~~
    - The definition of directed attacks, involving notions of true signal and class information, needs to be solidified with a clear and consistent linking to the Bayes optimal classifier.
    - The qualification of the implemented attacks in the image context as instance of directed attacks needs to be established more rigorously.
    - Important recent work needs to be cited and discussed to better position the present contributions; see below.
clarity,_quality,_novelty_and_reproducibility: The quality and novelty are both good subject to straightening up the presentation. Please see below for some comments and suggestions.

- Title
  - Does the scope of this study, e.g. as outlined in Section 1, actually answer the general question posed in the title?
    - The paper concerns a "new" special class of adversarial attacks, and its findings specifically hold in the low-data regime.
    - As such, it is not clear how the findings reported relate to the mainstream work on adversarial robustness in the absence of those essential conditions.
- Abstract
  - I strongly recommend to limit the use of sentiment words such as "surprising". It appears twice in the abstract.
    - First occurrence is justified with respect to the common belief.
    - Second occurrence is unjustified: it is the opposite of surprising to apply insights from a mathematical proof.
  - Please use the terms perturbations or attacks consistently, especially in the abstract.
- Section 1
  - Could you please motivate the new term given to "directed" attacks?
    - What are they directed to?
    - How does this concept relate to the more commonly used term of "targeted" attacks, where a given misclassification is desired?
    - Perhaps something like "hindering" attacks could be more accurate.
      - Indeed, "corruption attacks" appears early in Section 4.
  - Figure 1: reference to a green curve?
  - Please use the terms robust error, or adversarial error, or robust test error consistently, especially in the introduction.
  - I strongly recommend to rewrite the penultimate paragraph to read more easily. I didn't find the visual effects to be helpful to my understanding, especially as the sentence structure with added citations is a bit complicated already.
    - If absolutely necessary, then "in the low-sample regime" should be included in the punch line.
  - The 2nd point in the contribution list seems like merely a bridge from the 1st to the 3rd, and if so, should probably be removed. If there's more technical novelty to it, e.g., how linear separability maps to the low-data regime, then please clarify. Intuition can make for valuable discussion, but does not stand on its own as a contribution.
- Section 2
  - Equation 1
    - Please define $\mathbb{P}$
    - There is no indication of the transformation type in the definition of $T(x; \epsilon_{te})$
      - This is more of an issue when this notation is used later to indicate the same perturbation set for training and testing writing $T(x; \epsilon_{tr}) = T(x; \epsilon_{te})$
    - The definition given for $\ell$ only applies for class probabilities, and not the sign interpretation for the binary case.
  - Directed attacks
    - attack of the (input $x$)? .. for the model $f_\theta$
    - Is the direction of the optimal decision boundary the reason of calling those directed attacks? If so, how does that correspond to the characterization that such attacks "effectively reduce the information about the true class"? or to the image manipulations as shown in Figure 2?
  - It was probably meant to write $L(z) = \log(1 + e^{-z})$ not $\mathbb{E}^{-z}$.
- Section 3
  - What is the "true signal"?
  - Data model
    - $d - 1 > n$, as in Theorem 3.1, makes $n$ lower than the ambient dimension d minus 1.
    - Please clarify that $e_1$ is the first standard coordinate vector.
    - Please clarify what $u_1$ is.
    - Please clarify the last sentence, seeing that consistent perturbations are not defined and the role of $r$ has not been explained.
      - $r$ is only referred to as the "separation" early in $3.2.
  - Robust max-$\ell_2$-margin classifier
    - What is the relevance of the implicit bias of interpolators to the development?
    - Please motivate the choice of the robust max-margin solution as written in Eq. 4.
  - Main results
    - The statement of Theorem 3.1 needs to be rephrased. It was going in the direction of saying "the $\epsilon_{te}$-robust error ..?" but ended up saying "the following holds"
    - It appears you can safely move the definitions of $\phi_{min}$ and $\phi_{max}$ to the statement of the theorem.
    - Equation 5 is not really usable as an equation with $\tilde{\phi}$ unspecified. Part (a) as presented only serves the body of the paper to establish a monotonic relationship with the robust training budget $\epsilon_{tr}$, and as such, I recommend to present it as such.
    - ~~Please indicate in the statement of the theorem for part (b) that it assumes a fixed $\epsilon_{tr} = \epsilon_{te}$.~~
    - Why the large values of $\epsilon \in [0, 5]$ for the experiments in this section? It seems that for the experiments on images $\epsilon \in [0, 0.5]$.
  - Proof intuition
    - the solution of adversarial training => $\epsilon_{te}$-robust margin ~ Eq. (1)
    - $\epsilon_{tr}$ appears nowhere near Equation 7, yet $\text{Susc}(\theta; \epsilon_{te})$ is defined as the $\epsilon_{tr}$-attack-susceptibility.
      - I recommend you recall Equation 2 explaining how $\epsilon_{tr}$ is implicit in the training objective to obtain the $\theta$ in Equation 7.
    - ~~$\hat{\theta}^{\epsilon_{tr}}$ is used before it was defined.~~
- Nitpicking
  - Please use the technical term consistently
    - "small sample size" (abstract), "low-sample regime" (Section 1, Section 4), "low sample size" (Section 4), "low-sample size" (Section 6).
    - normal vs. Gaussian
  - Section 1
    - Please use the correct citation type, and fix up the punctuation
      - human faces (Wu et al. 2020))
      - translations or corruptions Engstrom et al. (2019)
      - as noted in Zhang et al. (2019) .... ")"
  - Section 3
    - in function of => as a function of (both in the text and the caption of Figure 3)
  - Section 4
    - appply
summary_of_the_review: The authors clearly put significant work, evident by the abundance of content and auxiliary analyses. However, the presentation needs significant work to polish up the key contributions and make the results more accessible.

Following my extended discussion with the authors, and taking the time to look up more of the recent literature, I'm updating my evaluation with the following requests:
- Please revise the definition of directed attacks, involving notions of true signal and class information, with a clear and consistent linking to the Bayes optimal classifier.
- Please include a careful derivation or additional empirical evidence to the qualification of the implemented attacks in the image context as instance of directed attacks.
- Please cite and discuss related recent works in the same vein of the generalization and accuracy-robustness trade-offs of adversarial training, as I list below. It appears the present work is distinguished by more insightful empirical contributions, while similar works are either mostly theoretical or only include limited experiments.
  - I recommend to discuss the following papers, which seemed most relevant to the main questions studied:
>- Dobriban, Edgar, Hamed Hassani, David Hong, and Alexander Robey. "Provable tradeoffs in adversarially robust classification." arXiv preprint arXiv:2006.05161 (2020).
>- Min, Yifei, Lin Chen, and Amin Karbasi. "The curious case of adversarially robust models: More data can help, double descend, or hurt generalization." In Uncertainty in Artificial Intelligence, pp. 129-139. PMLR, 2021.
>- Dong, Chengyu, Liyuan Liu, and Jingbo Shang. "Data Quality Matters For Adversarial Training: An Empirical Study." arXiv preprint arXiv:2102.07437 (2021).
>- Mendonça, Marcele OK, Javier Maroto, Pascal Frossard, and Paulo SR Diniz. "Adversarial training with informed data selection." In 2022 30th European Signal Processing Conference (EUSIPCO), pp. 608-612. IEEE, 2022.
  - I recommend to include a brief citation of the following papers:
>- Attias, Idan, Aryeh Kontorovich, and Yishay Mansour. "Improved generalization bounds for robust learning." In Algorithmic Learning Theory, pp. 162-183. PMLR, 2019.
>- Xing, Yue, Qifan Song, and Guang Cheng. "On the generalization properties of adversarial training." In International Conference on Artificial Intelligence and Statistics, pp. 505-513. PMLR, 2021.

With a better understanding of the positioning of the presented contributions, I am increasing my score with residual concerns regarding the clarity of presentation and the anticipated impact given those clarity issues.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies the peculiar observation that, with very sample sizes, robust training can be detrimental for robust error. The authors show that in a classification task given by a linear model, the robust risk of the max-margin classier can increase as a function of the perturbation size used during robust training. Importantly, they study a new notion of robust training that they call "directed attacks", which specifically targets the informative features of the problem. The authors then show how, in more complex settings where analysis is harder (and not presented here) some of the intuitions gained for the linear case seem to translate as well. The paper is clearly written and organized.

strength_and_weaknesses: This is an interesting paper, which I enjoyed reading very much. The authors clearly study and characterize the simple linear setting, and their conclusions seem to extend, empirically, to more complicated scenarios.

I have a few comments, mostly minor:

1. The definition and analysis of "directed attack" is interesting, as it is a relative new concept (to my knowledge). I understand why restricting their analysis to these cases in particular is important. Yet, could the authors comment more on the differences between regular/traditional adversarial losses (say, L2 or Linfty bounded) and their "directed attack" model? It seems to me that in the linear case with only 1 informative feature, they would be equivalent (except in the Linfty case), but maybe I'm missing something.

2. In their definition of logistic loss, immediately after Eq 2, i think the authors meant $e$ instead of $\mathbb E$ (expectation operator).

3. Commenting on Eq (7), the authors mention that this equation suggests that the robust error "can only be small if both the standard error and susceptibility are small". This is not true, as Eq. (7) is just an upper bound on the robust error. Probably the authors meant "a small standard error and susceptibility imply a small robust error". At the same time, it seems like that for the linear model studied here this bound could be tight?
clarity,_quality,_novelty_and_reproducibility: - Clearly written and presented.
- Novel behavior and analysis.
summary_of_the_review: This paper studies a new (to my knowledge) behavior in adversarial training and shows interesting results.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper shows that in the small-sample regime, adversarial training may actually hurt rather than help. This is demonstrated mathematically in the context of high-dimensional linear classification, and backed up by experiments on some image datasets with perceptible perturbations.
justification_for_why_not_higher_score: The paper has certain weaknesses. The writing is also not as clear as it could be. 
justification_for_why_not_lower_score: Clearly over the acceptance threshold.
Decision: Accept: poster