Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new knowledge distillation (KD) method for adversarial training. The authors first observed that the soft-labels provided by the teacher gradually becomes less and less reliable during the adversarial training of student model. Based on this observation, they propose to partially trust the soft labels provided by the adversarily pretrained teacher.  
main_review: Pros:
1. The paper is well written and easy to follow. The main claims and key observations are clearly stated.

2. The key observation is very inspiring: The (adversarially pretrained) teacher model's accuracy on adversarial images generated by the student model gradually drops. This provides solid foundation for the main claim of the paper: The student model shouldn't be fully convinced by the soft labels provided by the teacher model in adversarial distillation. This is something important that previous works overlooked. 

3. AKD2+IDA does generally outperform previous methods, although the margin is not that significant. I suggest the authors to try the modification I mentioned in the second bullet in "Cons" if haven't already. It may enlarge the performance improvement based on my intuitions.

Cons:

As I mentioned before, I'm totally convinced with the intuition to partially trust the teacher model's soft label in adversarial distillation. Bellow are some minor concerns.

1. The observation in Figure 1 (a) and Figure 3 (left column) tells us that the teacher model's predictions on the **adversarial** images generated by the student model shouldn't be fully trusted. However, the previous method ARD uses the soft label generated by the teacher network on **clean** images. So, it is not proper to claim that ARD has issue because it fully trusted the teach model's unreliable soft labels. In fact, the soft labels used in ARD are generated by teacher on **clean** images and has been shown to be generally reliable by yourself in the blue curve in Figure 1 (a). With that said, ADK2 does use the soft-label on **adversarial** images. I think it is better to use ADK2 as the starting point to motivate your method, instead of from ARD as in Sec 3.1 of current version. An even more direct way to motivate IDA is to replace the second loss term (ie the "fully trust" KL term) in ADK2 to your "partially trust" KL term, while keeping other terms unchanged. 

2. Most importantly, just like in ARD, IAD in Eq. (3) uses soft labels from teacher on **clean** images, instead of **adversarial** images. As I mentioned in bullet 1, the soft labels on **clean** images are generally trustworthy, as shown by yourself in the paper. So why does IAD partially trust it? It is never motivated, if I understand correctly. 

3. To support your intuitions, the current design of IAD in Eq. (3) might not be the best choice. Specifically, the "Student Introspection" KL loss is weighted by (1-\alpha). This is not so intuitive for me. You have justified the "Teacher Guidance" is not always trustworthy so it is good to weight its loss term with \alpha. However, a trustworthy "Teacher Guidance" doesn't mean an untrustworthy "Student Introspection". So why do you down-weight "Student Introspection" KL loss when \alpha is large? In fact, the  "Student Introspection" KL loss term is just an annealed version of the smoothness loss term in TRADES. As shown in TRADES paper, it regularizes the smoothness of the model which helps robustness. In my view, based on your key motivation to partially trust the **teacher**, the **student** introspection loss term should have a constant weight and instead of dynamically down-weighted as in Eq (3). 

4. There are four loss terms in the best performing method AKD2+IDA. It would be nice if ablation studies can be provided on all those loss terms. 

Overall, I think the key observations in this paper is very inspiring. If proper modifications can be made to address the above issues, I think it would be a good paper. 
summary_of_the_review: Interesting observation and good intuition, but there is a noticeable gap between the designed method and their intuition. I agree the proposed IAD empirically works, but the intuitions provided in the paper to motivate and interpret IAD is problematic.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, a new method called introspective adversarial distillation (IAD) is proposed for conventional adversarial training. Concretely, it targets the unreliable teacher case, where teacher is good at adversarial / natural data or none of it. Experimental results validate the effectiveness of the proposed method towards enhancing adversarial robustness. 
main_review: Pros.

The overall idea is interesting and the proposed method seems simple and practical as well since we only need to adjust the annealing parameter adaptively. Besides, the paper is quite readable and experimental results seem promising. 

Cons. 

There are some issues to be addressed.

- The results of Natural and FGSM are not good. The tradeoff between Natural performance and robustness should be investigated more. 

- In tiny-imagenet and cifar datasets, why IAD + AKD^2 is better than IAD for AA? This phenomenon should be explained more.

- The current evaluation is based on image datasets, like cifar and tiny-imagenet. However, is IAD still working for language datasets?

- It is good to discuss some theoretical justification for IAD, which will make IAD stronger in theory.
summary_of_the_review: Basically, this paper is well written and the proposed method seems promising. I thus would like to lean on the acceptance side.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies a specific distillation scenario, adversarial distillation, to improve the model robustness. Different from the constant sort supervision in the ordinary distillation, the teacher model will become progressively unreliable along with training, since the adversarial data are dynamically searched by the student model and might not be well identified by the teacher model. Therefore, the authors introduce an introspective adversarial distillation, which considers to bootstrap the learning from both the teacher model and itself. A range of experiments demonstrate its superiority on improving the model robustness.
main_review: In summary, the discovery of the specific degeneration in adversarial distillation is interesting, and the authors design an elaborate loss that takes such a drawback in the adversarial distillation into account. A range of experiments on different network architectures, adversarial training backbones (AT or TRADES) and three widely-used datasets demonstrate its superiority in improving the model robustness. Besides, the ablation study regarding the annealing temperature and the warming up has been conducted to provide the view on its working mechanism. 

However, there are still some concerns about this work.
Major Concerns
1)  In Section 3.2, the authors analyze the adversarial data by partitioning them into three groups. Is there the fourth group where the teacher model has wrong predictions on natural data but right prediction on its adversarial counterpart? If it almost does not exist, it will be complete that the authors at least mention this in the paragraph. Besides, for the third group, it is not absolutely consistent with the case of x_2 where it also requires the wrong prediction of the student model. 
2) In Section 4, Eq.(4) can approximately reflect the confidence on the adversarial data. But have the authors considered the difference between group 2 and group 3 in Eq.(4), since they may have different scale and corresponds to different cases.
3) In fact, the proposed Introspective Adversarial Distillation can degenerate to TRADES and ARD by setting some extreme hyperparameters. It will be better to conduct an experiment to plot a accuracy (natural acc and robust acc) surface with varying hyperparameters that includes these three methods, which helps us to better understand the position your method.

Minor Concerns
1) Table 1 is messy. The authors could list each distance term as a column name and compare baselines by considering it or not.
2) The experiments of IAD + AKD^2 is not well explained. I guess it is to show how IAD can help improve the natural accuracy but the authors have not clarified this.

summary_of_the_review: This paper investigates an interesting problem about adversarial distillation, which has some distinct with the ordinary distillation. The authors provides an introspective adversarial distillation to solve the problem. Despite simple, it shows consistent improvement on the model robustness while does not sacrifices the natural accuracy too much. However, the writing about the motivation and the loss designed for each group as well as the experimental parts are not very clear and need to be further improved.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Adversarial training has been developed for many years, and it aims to provide reliable classifiers in the era of deep learning. However, in many ML-driven scenarios, users might not want to retrain their big deep networks using adversarial training. The main reason is that training a reliable/adversarial-robust classifier will cost many computational resources. To address this issue, some methods are proposed to distil the robustness from adversarially pre-trained models, which is more practical than the direct adversarial training. This is also the point this paper focuses on. In my humble opinion, this is a very promising research direction and should be paid more attentions in future. 
Compared to existed works, this paper argues that the adversarial training data of the student model and that of the teacher model are egocentric (respectively generated by themselves) and becoming more adversarial challenging during training, which causes failure of existing works. To address this issue, this paper proposes an Introspective Adversarial Distillation (IAD) to effectively utilize the knowledge from an adversarially pre-trained teacher model. Extensive experiments are conducted on CIFAR-10/CIFAR-100 and the more challenging Tiny-ImageNet datasets to evaluate the efficiency of our IAD.

main_review: Pros:
+ The research direction is promising and more practical than adversarial training in the real world. As I demonstrated in the summary part, in many ML-driven scenarios, users might not want to retrain their big deep networks using adversarial training due to huge computational cost. Thus, it is, IMHO, urgent to consider to distil the robustness of a robust model. 
+ Although the high-level idea is mentioned in previous papers, this paper considers this problem in detail and find a failure case of previous papers. By considering this failure case, this paper proposes a novel framework to complete this task, which is novel and significant to the development of the field.
+ This paper is well-written and easy to follow. Experiments are enough to support the claims made in this paper. A plus should be that TRADES and AT are both considered in the experiments, which verifies that the proposed framework is suitable for existing adversarial training methods.

Cons:
- Since the student model is different from the teacher model, it is expected that adversarial data generated by the student model is different from the data used to train the teacher model. This kind of performance drop also happens in other fields, such as domain adaptation, even adversarial machine learning itself. Thus, it is not clear what the point of Figure 1(a) is. I am not sure if 1(a) is necessary. It is more like the transferability of adversarial attacks.
- Based on the above drawback, it is also unclear if there are other points that make the DISTILLATION fails (except for the distributional discrepancy).
- The formal problem setting is missing. It is better to provide the formal problem setting to make readers understand the problem clearly.
- What is the difference between S(\circ|\tau) and S(\circ)? I did not see any definition of S(\circ|\tau). How does \tau effect on S(\circ|\tau)?
- I would like to see how \tau effects the performance of the proposed method. If we remove \tau, then the “Student Introspection” term is actually from TRADES. Meanwhile, how does \tau effect the performance of TRADES? Are there any papers discussing this point?
- From the experimental results, IAD performs much better than ARD (IMHO, the most direct baseline to IAD), which is very good. I would like to see how \tau effects the performance of IAD compared to ARD (giving the same \tau for ARD and IAD).
- The computational-cost comparison is missing, which is important for distillation-based method.
- Figure 1(b) is a clear figure, but I cannot see the advantages of your method in this figure. In other words, it is unclear why your method can improve the performance when reading 1(b).

summary_of_the_review: In general, considering the significance of the researched problem, this paper can be accepted by the ICLR2022. However, some points should be clarified and strengthened in the revision. I would like to strongly support this paper if my concerns can be fully addressed.
review_title: null

Meta-review: This paper proposes a new knowledge distillation (KD) method for adversarial training. The key observation is inspiring: soft-labels provided by the teacher gradually becomes less and less reliable during the adversarial training of student model. Based on that,  they propose to partially trust the soft labels provided by the teacher in adversarial distillation. 

Reviewers unanimously agree that this paper has clear motivation, well-sorted logic, and neat writing. While some reviewers initially posed concerns on evaluation completeness and detail clarification, they were well addressed during the rebuttal. AC reads the paper/discussion thread and agrees this is a worthy work to get accepted.
Decision: Accept (Poster)