Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes an adversarial defense method that builds a robust classifier by using a min-max optimization in the feature extractor. 

Pros:
1. The proposed idea is interesting and reasonable.
2. The experiment results look good.

Cons:
1. The proposed idea is not novelty enough. There is a similar work [1] talking about the feature scattering to make the model robustness.  The only difference is the paper further uses a discriminator which involves a different network in the training process.
2. The results might not be reliable. while [1] uses a similar idea, it later shows a significant performance drop by using some later proposed adversarial attack method like Autoattack[2]. Specifically, it reduces the performance to from claimed 60.6to 36.64. I suggest the author should try more blackbox based attack or the ensemble attack in Autoattack. 
3. The paper quality needs to be improved. For example, Figures 3  doesn't have x-axis label. I personally find it very confusing on the different x scale used in the figure. 
4. The methods mentioned in non-obfuscated gradients are not sufficient for me to believe there is no such case. (ii) (iii) is quite unrelated to the obfuscated gradients problem. Also, the boundary attack in the appendix looks very confusing. I can't understand what the number represents in the table. By the way, the boundary attack is not a good way to attack the adversarial defended method. Attacks like B&B or FAB are more suitable.





[1] Zhang, Haichao, and Jianyu Wang. "Defense against adversarial attacks using feature scattering-based adversarial training." Advances in Neural Information Processing Systems. 2019.

[2] https://github.com/fra31/auto-attack
review_title: Not enough novelty with potential problems in the evaluation.

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes to leverage an additional adversarial discriminator to distinguish between the clean and perturbed inputs from the representation level.

The empirical evaluation shows promising results in terms of generalizing to unforeseen attacks. 
However, the baselines TRADES is evaluated on ImageNet which is a more challenging task and it would be good to evaluate the performance on large scale dataset for the proposed method as well.
It would be important to compare with related work on improving robustness by learning robust representation [1][2].
In addition, from figure 4, the sensitivity comparison is actually not very clear and it would be good to compute the significance level to show how sensitive the AFD learned representation with quantitative results.

From the methodology perspective, it would be good to analyze the difference between the distributions of benign and adversary representation. Several previous work shows that with only activation patterns or logits it is not enough to distinguish the benign and adversarial instances. It would be important to evaluate and confirm that the learned representation of the benign and adversarial instances can indeed be learned and separated by a trained discriminator, which may lead to further interesting analysis and findings.


[1] Liao, Fangzhou, et al. "Defense against adversarial attacks using high-level representation guided denoiser." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[2] Samangouei, Pouya, Maya Kabkab, and Rama Chellappa. "Defense-gan: Protecting classifiers against adversarial attacks using generative models." ICLR.
review_title: This paper proposes adversarial feature desensitization to generate robust representation 

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:

This paper proposes Adversarial Feature Desensitization (AFD) as a defense against adversarial examples. AFD employs a min-max adversarial learning framework where the classifier learns to encode features of both clean and adversarial images as the same distribution, thereby desensitizing adversarial features. With the aim of fooling a separate discriminator model into categorizing the classifier’s adversarial features as from clean images, the classifier is trained with the standard cross-entropy loss and adversarial loss terms. The authors showed through experiments on MNIST, CIFAR10 and CIFAR100 datasets that AFD mostly outperform previous defenses across different adversarial attacks under white- and black-box conditions.

Pros:
+Strong defense performance
+Novel idea

Cons:
-No discussion on the scalability of AFD defense or results on larger dataset such as imagenet

Recommendation:
The idea is interesting and is backed by strong empirical results. Generally, the paper is well-written and easy to follow. Given AFD employs both a GAN training component and adversarial example generation, I can imagine it to be computationally more expensive than most of the existing defenses. It would be more convincing to show experiments and discussions addressing AFD’s scalability. Apart from this point, there is no major flaw in the paper and I am inclined towards acceptance.


Other questions and comments:
For the black-box attack assessment, are the adversarial examples generated from models that are trained on their respective defense, initialized with different random seeds? It would be good to mention how exactly the black-box attacks are conducted.
 
Some prior defenses have been shown to be prone to transfer black-box attacks, e.g. adversarial examples from a TRADES model, transferred to the AFD defense. It would be more convincing that AFD is not relying on obfuscated gradients with results on this form of black-box attacks.
 
How much is AFD’s performance dependent on the strength of adversarial examples using during training? How many iterations were used for the L-inf attack policy used to perturb the training inputs?
 
I believe the sentence in the Introduction is untrue: “adversarial learning has not yet been successfully applied to the problem of adversarial robustness.” Several studies using adversarial learning for adversarial robustness exist:

“Improved Network Robustness with Adversary Critic” NeurIPS 2018

"A Direct Approach to Robust Deep Learning Using Adversarial Networks" ICLR2019

“What it Thinks is Important is Important: Robustness Transfers through Input Gradients” CVPR 2020”

“GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier” arXiv:1903.02585

--Update after rebuttal--

The reviewer thanks the authors for addressing the key questions and concerns and have updated the confidence score accordingly.

review_title: Review #4

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper is about leveraging the GAN idea to get robust features that are insensitive to adversarial attacks. Specifically, with a shared encoder/embedding, the feature of the adversarial attack are considered as the fake sample, while the feature from the real image is the real sample; by forming an adversarial game in the feature space and by jointly considering the classification loss, the AFD method is proposed. 

The main idea is reasonable. But the current manuscript may not be ready for publication.

The structure of the current manuscript should be revised substantially. For example, too many materials (like experimental results) are given in Appendix, which makes the main manuscript less self-contained; I would suggest rearranging the contents and put the important ones in the main manuscript. 

Figures and their captions should be revised, especially Figures 3 and 4.


In the last paragraph of Page 2, the authors stated that “the proposed method outperforms existing methods by a large margin. However, this might not be true, by considering the performance shown in Tables 1 and 2 and the high variance observed in Table 1 and Figure 4.

In the 5th line of the first paragraph of Section 2, the definition of l_i (x) is different from the one in Algorithm 1. 

In the last paragraph of Page 3, the definition of pi(x, epsilon) is not consistent. Epsilon is missing.

In the last paragraph of Page 3, the output of Da_psi should be real and the ideal discriminator output is {0,1}.

In Algorithm 1, why the policy pi is related to Dc_phi.  

Theorem 1 might not be right. A GAN matches two distributions. Besides, if E() is a linear function, delta could be a vector in the null space of the weight.

Many typos exist. For example, PGD-L_{inf} in Page 5.

In the last paragraph of Page 7, the representation sensitivity Sc is not defined.

review_title: The main idea is reasonable. But the current manuscript may not be ready for publication.

Meta-review: This paper proposes Adversarial Feature Desensitization (AFD) as a defense against adversarial examples. Specifically, following the spirit of GAN and Adversarial Domain Adaptation, an adversarial discriminator is introduced to distinguish clean and perturbed inputs at the representational level. 

This paper receives 3 reject and 1 accept recommendations. On one hand, though the proposed method shares some similarity with the Feature Scattering method at a high level, most of the reviewers still find the proposed method is interesting. The AC also agrees that the paper's organization and typos does not warrant a rejection. 

On the other hand, the reviewers have also raised a few concerns. (i) A more careful discussion on the scalability of the proposed method is needed. (ii) Experiments are mostly focused on small datasets, while results on ImageNet is lacking, which makes the paper less convincing. The authors claim that they are trying to at least run Tiny-ImageNet experiments; however, this set of results are not provided by the end. (iii) A more detailed analysis and visualization on the learned difference between the distributions of benign and adversary representation is needed, since a discriminator is learned here. 

The rebuttal unfortunately did not fully address the reviewers' main concerns. On balance, the AC regrets that the paper cannot be recommended for acceptance at this time. The authors are encouraged to consider the reviewers' comments when revising the paper for submission elsewhere. 
Decision: Reject