Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper cites the feature robustness framework of Ilyas et al. 2019 as its starting point. It claims that framework fails to explain overfitting in adversarial training (Note: that paper wasn't claiming to explain overfitting). The paper then proposes a narrower definition of feature robustness (where an attacker and a trainer are given). They claim that it explains overfitting.  

The paper, I think, mischaracterizes Ilyas et al. 2019 definition of feature robustness when describing Limitations. 

The writing is not particularly clear on why the change in definition explains overfitting, which was the original motivation for the paper. The paper acknowledges past work that robust overfitting can be largely "surpassed" (suppressed?) by enforced a smoother landscape with additional regularizations. In the end, it seems the proposed schemed BoAT also ends up doing something similar (Section 3.1: "In order to suppress robust overfitting of the model trainer T after LR decays, we need to put further regularization on its local fitting ability, e.g., by enforcing better landscape flatness.").

So, I am still trying to figure out the key contribution of the paper and new teachings. It could be that the theoretical arguments need improvement to explain overfitting.  

The BoAT training method is a potential contribution since the claim is that it eliminates overfitting even over 500 epochs. But, the authors should compare that with other approaches such as simply picking an earlier saved model (prior to overfitting) and prior techniques that introduce additional regularizations to smooth the landscape (many of which they cite in the 2nd para of the Intro). 

The other issue is efficiency of BoAT training. How long does the training take compared to existing techniques, e.g.,  early stop? I am trying to figure if BoAT is proposed as a practical technique that practitioners should use or is simply a tool to help explain overfitting. 

The Intro states: 

"Specifically, we hypothesize that due to the strong local fitting ability endowed by smaller LR, the model learns false mapping of non-robust features contained in adversarial examples, which opens shortcuts for the test-time attacker and induces large test robust error. "

But, why doesn't standard training also learn false mappings on non-robust features and also cause overfitting? 

In the end, the paper started with an interesting promise of explaining the difference in overfitting between standard training and adversarial training. But, that difference in behavior still remains unclear after reading the paper.

strength_and_weaknesses: 
Strengths: 

I am still trying to figure that out. I wish the paper was clearer in writing. It seems the authors are trying to draw a connection between feature robustness (using their definition, not Ilyas et al.), learning rate decay,  and overfitting. And why the authors think that standard training doesn't run into overfitting but adversarial training does at a fundamental level.

In the end, it is not clear if BoAT is a better algorithm over prior methods for adversarial training.

Weaknesses: 

English writing is poor, making the paper difficult to follow. The abstract could be better written to begin with. 

Limitations on page 3: I didn't understand your argument. I didn't find in Ilyas a discussion or claim regarding overfitting.  Furthermore,  during model training, isn't the classifier changing? And also isn't f changing, if if is part of the model? If it is some abstract function f independent of the trained model, then why you do believe that robustness of f is changing during model training?

Furthermore, I don't see an attacker A in definition (2). 

I don't agree with the characterization of  Ilyas et al. 2019 that they only consider attacker A and fail to view adversarial training as a dynamic min-max game. For instance, in Robust Training para on page 4 of Ilyas et al., they explicitly discuss adversarial training a min-max optimization problem.  

Furthermore, the definition I Alias et al. considers the adversarial space, but does not explicitly mention an attacker strategy. I am still trying to understand your criticism of Ilyas et al. 2019.





clarity,_quality,_novelty_and_reproducibility: I think the paper can be drastically improved. 

Figure out what the key contribution is and really support that well. I am not sure the conceptual framework of robust features from Ilyas et al. contributed to the conceptual understanding since you ended up changing their definitions anyway. 

The writing is tentative. For instance: "Therefore, due to the change of relative strength between A and T , these non-robust features could become robust after LR decay." 

The stated findings are not particularly enlightening. For example: 

"The discrepancy in feature robustness explains why the rise of training robustness does not consistently lead to higher test robustness, leading to a large robust generalization gap. 

It is not clear that one thing explains the other.  What about robust features? Is their overfitting ruled out?
And the above statement begs the question as to why non-robust features cause overfitting.

Performance Experiments: 

How do your findings compare with Yu et al.  in ICML 2022 (Understanding Robust Overfitting of Adversarial Training and Beyond)? They proposed a method called minimum-loss constrained adversarial training (MLCAT).  And what about efficiency of training?
summary_of_the_review: The paper needs to be better written with a  more careful discussion of related work to better position the paper. The paper is also missing some related work that also attempts to explain overfitting and propose techniques to prevent overfitting.  

Additional comments for the authors after the rebuttal: 

I thank the reviewers for detailed explanations. I think the algorithm is a good contribution. But, I still feel that the overall pitch in the Intro and Section 2 should be changed in a future version of the paper. I am not sure Ilyas et al. is the correct baseline to criticize, since it was not directed at providing an explanation on robust overfitting. Other papers that attempt to explain that in other ways would be a better starting point.  In case you use Ilyas et al., perhaps a better pitch would be to say that even if a feature is fundamentally robust,  it can be perceived as non-robust by a given model because of inability to find the correct decision boundary for that feature -- even if one exists.  That way, perhaps Ilyas et al. are not necessarily wrong -- static and dynamic views become more in harmony. I don't think the paper proves that their view was necessarily wrong, but the paper suggests so, which I think is an issue with the current thrust. 

Another area that the paper could  do a better job in  would be to better explain why normal training does not overfit, but adversarial does. Are there situations where normal training would also overfit, based on the understanding provided by the work? Even normal data should have some non-robust features, no -- or is that fundamentally missing in normal datasets?  More in-depth discussion would be useful. 

Overall, I am not changing my evaluation, but  I encourage authors to rethink the overall pitch for presenting the Boat algorithm and add more depth for an improved paper in the future.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work aims to provide a new perspective to explain the robust overfitting and improve the adversarial robustness built upon the proposed viewpoint. To do so, the authors revisit and generalize the static framework of feature robustness. Built upon the proposed framework, the authors claim that the balance between the attacker and the trainer is the key to understanding robust overfitting. Consequently, the authors propose three methods inspired by the framework and demonstrate the effectiveness of these (two, as one is dropped in the main page) methods.
strength_and_weaknesses: Strength:
- The considered symmetrized confusion matrix is interesting and convincing, which is novel to me. 
- The designed objective function is inspiring. Specifically, the authors introduce a KL term, minimizing the KL divergence between the learnable model f_{\theta} and the detached model f_{\phi}, which is actually equal to assigning a soft label to each adversarial example. Thus, it is promising and exciting to explore a novel approach to designing different assignments of soft labels. And, maybe, it has potential benefit to the research of certifiable defense.
- The paper is well-written, easy to follow, and has solid experimental analysis.
- The proposed method is simple yet effective, verified by the experiments.

Weakness:
- A stronger attack for stronger defense is not employed due to its harmful impacts on natural accuracy, but I cannot find any explanations about the observation from the proposed perspective. It is necessary to add the corresponding explanation using the proposed view because methods inspired by the proposed framework are dropped from the main page, making the framework less convincing.
- The definition of Eq. (3) seems confusing. The trainer T is directly applied to update the model, i.e., changing the learned representations, thus, introducing the trainer T, given the representation, seems meaningless, but the trainer T is important for the rest of the work. Hence, the authors should clarify the point. Similarly, the utilized F_f^T is also confusing.

Suggestions:
1 The claimed theory seems to be a hypothesis, as merely experimental analysis is given in the current version.

2 According to the claimed perspective -- the main goal is to prevent the model trainer T from fitting non-robust features too quickly and too adequately --, a straightforward approach can be that the trainer decreases the learning rate progressively from the previous epoch decaying the learning rate by 0.1. Thus, the corresponding experiments are required. However, it is lacking in the current version. 

3 Different from the section on the method, the section on explanation seems weak, I suggest employing more references to convince the claim, the related perspective is the spurious feature perspective, such as CausalADV and group DRO.

4 The ‘feature’ considered in Eq. (2) corresponds to the data space, while the ‘feature’ considered in this work Eq. (3) is more about the representation space, thus, I suggest the authors make these two features distinguishable as they are conceptually different.

[CausalADV] Adversarial robustness through the lens of causality. Zhang et al. 2021

[group DRO] Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. Sagawa et al., 2020

clarity,_quality,_novelty_and_reproducibility: Good clarity, quality, novelty, and reproducibility.
summary_of_the_review: The proposed framework is novel to me, the proposed methods are simple yet effective, and the conducted experiments are convincing.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper begins with an argument that the robustness of a model should also be conditioned on the training process. They then focus on  the one specific aspect of the training process -- learning rate (LR) decays. They found that after LR decays, the confusion matrix becomes more symmetric. The author claimed that this observation helps to understand robust overfitting. Then, they proposed the BoAT loss, which seems to perform better than some prior work in their experiment.
strength_and_weaknesses: Weakness:
- The first paragraph in page 5 is confusing.
  - In Figure 2, there are no measurement on feature, only on classes. But the conclusion is on target-class features. What is the definition of non-robust feature?
  - The relationship between the confusion matrix being symmetric and robust overfitting is unclear.
  - What does A->B and B->A mean in Figure 2?
  - What does accuracy with misclassified labels in Figure 2a mean?
  - Is the confusion matrix computed with adversarial attack?
- Some other parts of the paper that are confusing:
  - Should the output of f be a feature or a label? In the text, it's called feature while in equation 2 and 3, they are compared with label y.
  - The authors claimed "False Non-robust Mapping Opens Shortcuts for Test-time Adversarial Attack". What is a shortcut? Could you measure this "shortcut"? Are there any hypothesis on why this shortcut is related to LR decay?
- What is the learning rate schedule used in Figure 2? Does different scheduling or decay method matters?
- The method used in [1] should also be compared. Their method is effectively make the model locally smooth, which in some way also improves landscape flatness.
- For Equation (4), is the adversarial example computed on the cross-entropy loss or on the BoAT loss? The inner minimization should be included in the equation to make things more clear.
- How is the lambda in Equation (4) chosen?
- This paper discussed a lot on robust overfitting, while in section 3 and 4, there is not discussion on the generalization gap comparing with other methods.
- The hyper parameter choices in Table 4 seems arbitrary.
- I think it is arguable whether adversarial training is SOTA algorithm for adversarial robust training. I would suggest checking out some of the methods in https://robustbench.github.io/
- In the paragraph before Section 4.1: "We train each model for 200 epochs and record the best-epoch (selected based on PGD attack)and final-epoch robust test accuracy.". One should not just select the best epoch on test data. This should be done on a validation set.

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." International conference on machine learning. PMLR, 2019.

clarity,_quality,_novelty_and_reproducibility: No code is provided and the paper is not really clear to me. I don't think I will be able reproduce this work.
summary_of_the_review: This paper is not well written. The flow of the paper is also not logical (e.g. the relationship between the Symmetrized Confusion Matrix and robust overfitting is not explained). There are major flaws in the experiment setup (e.g. selecting the best epoch based on the performance on test data), and the experimental result cannot support all their claims.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Summary

Based on former findings, this paper first proposes a new definition of robust features with taking dynamics into consideration. Focused on the phenomenon that robust overfitting usually starts after learning rate decay, the authors provide an explanation of robust overfitting based on several experiments. New approaches to mitigate robust overfitting are proposed and validated. The main difference from previous studies is their emphasis on the necessity of considering the min-max nature of AT in robust overfitting. 
strength_and_weaknesses: 
Pros

1. The background and motivation of the paper are well-written, and the validation experiments of the proposed hypothesis are convincing.
2. The paper is well-written, clear, and easy to follow.



Cons.

1. The proposed dynamic feature robustness framework needs to be more formally and clearly defined, especially the difference from the former work. For instance, it is encouraged to provide a formal mathematical definition of dynamic, robust, and non-robust features.
2. The mitigation approaches ”adding regularization loss”, “using stronger attacker” and “decreasing learning rate” are well-established tricks and therefore this paper does not bring technical novelty.
3. The proposed methods are only evaluated on 2 architectures in the same model family, which is highly insufficient. More network backbones like MobileNet and VGG should be included.
4. Although the proposed methods lead to improved robustness, it suffers from natural accuracy degradations. It is hard to say whether the BoAT is better compared to other approaches. Also, this needs to be mentioned in the front of this paper, otherwise only mentioning the robustness benefits is misleading.
5. The paper mentions that non-robust features will be much more robust after the learning rate decay. I am very curious about whether the non-robust features will also approach robust features visually.
6. "RO usually happens after the learning rate decay in AT". Actually, this claim is insufficiently supported. Since most previous papers decay the learning rate at the 100th epoch. In order to well support this claim, the authors need to conduct experiments with LR decay at different epochs and show the consistency of RO observations.
clarity,_quality,_novelty_and_reproducibility: Clarity is good.

Quality and Novelty are fine.

Reproducibility is unknown
summary_of_the_review: Good paper with novelty and experiment limitations.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the robust overfitting (RO) problem for adversarial training (AT) which usually happens after the learning rate (LR) decay. In particular, it hypothesizes that RO is caused by a loss of balance (trainer becomes stronger than attacker) after LR decay. It verifies the hypothesis by looking into target-class information in test robust features, bilateral class correlation, and confusion matrix symmetry for models with and without LR decay. Based on this understanding, the authors propose three techniques (model flatness regularization, smaller LR decay and stronger attacker during training) to rebalance the trainer and the attacker. The proposed method BoAT is evaluated against several baselines on two model architectures and three datasets.
strength_and_weaknesses: ## Pros

-neat observations and formulation of the RO problem

-the experiments mostly verify the proposed hypotheses

## Cons

-BoAT has much lower natural accuracy compared with KD+SWA

The results in Table1 seem to suggest BoAT simply trades off natural accuracy and adversarial accuracy compared with existing methods (e.g., KD+SWA) so it might not be very useful in the sense of improving the natural-adversarial accuracy trade-off. I would like to see how the trade-off of natural accuracy and adversarial accuracy look like for different hyper-parameters of BoAT and baselines, and which one lies on the Pareto front.

clarity,_quality,_novelty_and_reproducibility: ## Clarity

The paper is overall clear and well-written. However, the plotting needs to be improved. For example, Figure 2(c) has way too tiny x/y labels. The color in 3(d) is too close for 12/255 and 16/255 to enable one to differentiate one from the other. There are also some typos, for example, in the third to last line of Sec3.3, “under \epsilon = 8/255” seems to be “under \epsilon = 16/255” instead? “purposed BoAT” (end of page 7) should be “proposed BoAT”. Please carefully check the paper for similar errors.
 

## Quality

The claims regarding the RO after LR decay are mostly supported by the experiments.

## Novelty

The insight of the cause for RO after LR decay is novel. The mitigation strategy leveraging self-supervision is also moderately novel. 

## Reproducibility

Although code is not provided, the method seems to be reproducible. 

summary_of_the_review: Overall, the paper provides neat observations on the RO problem in AT and verifies the hypotheses. Its proposed three techniques mitigating RO seems to be functional for alleviating RO but might not be useful as general techniques to improve robustness.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Even though the authors addressed several concerns raised by the reviewers, reviewers remained skeptical of several claims. Overall, this is an interesting contribution, but the authors are strongly encouraged to follow the suggestions below towards improving the paper. Repeated/summarized from the reviews:

1) The overall pitch in the Intro and Section 2 should be changed in a future version of the paper. Papers that attempt to explain  robust overfitting, or even just overfitting. 

2) Along the same lines, spending more effort to explain circumstances/conditions under which normal training does not overfit, but adversarial does, would be important. This would better motivate the study of the proposed method in the adversarial, rather than standard, setting.

3) The relationship between LR decay, dynamic features, and the proposed algorithm is still unclear.  Strengthening the connection between BoAT and dynamic features is also a good idea, and so would be reporting the result under validation-based early stopping in the main paper.
justification_for_why_not_higher_score: Please see above
justification_for_why_not_lower_score: NA
Decision: Reject