Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This paper aims to improve adversarial robustness considering the information about the object shape details with the means of edge maps. Two different strategies are proposed to increase model robustness using the edge maps: i) conduct the adversarial training on the input images, which are concatenated with its corresponding edge map as an additional input channel to the image. Here, the edge maps are recomputed and concatenated to the adversarial inputs after their generation during adversarial training. ii) Utilize a conditional GAN to generate the images from clean data distribution that is conditioned on the edge maps and later the classifier performance is evaluated on the generated images. Here, the authors claim that these two strategies improves the classifier robustness after conducting experiments across 10 different datasets. They also studied the effectiveness of their strategy when combined with background subtraction, the defense against poisoning attacks and robustness against natural image corruptions.

Strengths: 
+ Motivation is clear.
+ The proposed strategies are interesting, explained clearly, easy to follow and are different from existing works. 
+ Rigorous experiments across 10 different datasets are carried out to demonstrate the effectiveness of the proposed strategies.
+ Results demonstrate that the model robustness can be significantly improved using proposed strategies utilizing the edge maps. The improvement not only limited to adversarial robustness but also extends to backdoor attacks and natural image corruptions.

Weaknesses:
-	The major concern lies in the evaluation of the proposed strategies. Here, the authors considers that their method purify the input image before passing it to the model and an adaptive attack against their edge map based defense strategies will likely results in structural damage to the edge map. However, it is crucial to evaluate the proposed defense against an adversarial attack which craft the adversarial examples to produce minimal structural alterations to the edge map but mislead the model predictions. An adversary could potentially optimize the perturbation in such manner and may remain successful in attacking the model.
-	Results on CIFAR-10 and Icons-50 of GAN bases shape defense depicted in Figure 4 do not provide solid evidence on the model robustness. Here, the performance on clean inputs degraded significantly and the improvement seen in perturbed samples might be the result of the trade-off between model robustness and generalization as noted in the literature.
-	The claims on robustness against natural image corruptions using the edge maps seems to be valid only on GTSRB dataset and do not hold true for TinyImageNet and Icons-50 as seen in Figure 6. The robustness of the model with edge maps is similar or on par with the model without edge maps on these two datasets. These results suggest that additional usage of edge maps do not improve model robustness and the improved performance seen on GTSRB could be attributed to the simple nature of the objects in the dataset.

Final thoughts:
The proposed method is clearly motivated. Although the performance gains on adversarial robustness is significant, there are critical points yet to be addressed. Therefore, I marginally accept this paper.

------------------------------------------------------------------------------------------------------------
Post rebuttal:
The authors have devised an adaptive attack to craft the adversarial examples against edge maps and shown that the proposed technique is still remain robust. However, the essence of robustness in this work lies in the BINARIZATION of the input (i.e., binarized edge maps) which is shown in the previous work [1] and need not necessarily attribute to the shape information obtained through edge maps. I recently came across state-of-the-art deep edge detector [2] that produces non-binarized edge maps, which could be interesting for authors to validate their approach using such non-binary inputs. Hence, I maintain my initial rating and marginally accept this paper.
 

[1] ON THE SENSITIVITY OF ADVERSARIAL ROBUSTNESS TO INPUT DATA DISTRIBUTIONS, ICLR 2019

[2] Richer Convolutional Features for Edge Detection, CVPR 2017
review_title: Adversarial robustness using shape details in the form of edge maps

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper studies how to incorporate shape (particularly depth map) into CNN for more robust models. The study focuses on image classification. Specifically, this paper proposes two depth-map-based defense: 1) Edge-guided Adversarial Training (EAT), which use depth map as an additional input 2) GAN-based Shape Defense (GSD), which learns a generator from depth map to reconstructed images, which is then used as net input. Experiments on 10 datasets shows the effectiveness of the proposed two defenses against white-box attacks including FGSM and PGD40. To further demonstrate the effectiveness, the authors also conduct some other experiments: 1) the proposed EAT goes well with two fast AT algorithms; 2) the proposed algorithm can also be used to defend backdoor attack; 3) edge makes CNN more robust to common image corruptions.

I think the topic that tries to explore and understand the connection between shape and CNN is very important and somewhat under-studied. This paper provides some interesting empirical results and insights to the community. 

1. The assumption of this paper is that edge map does not change much under adversarial attacks. I think this relies on two things: A) we use tradition non-deep-net based edge detector like canny edge; B) the adversarial perturbation is pixel-based perturbation. I am curious to see how the proposed algorithm work with B), but for now, I will focus on A) below.

For A), for harder and more realistic datasets, canny may not work well and we may resort to deep-net-based edge detector (like HED). I think the author also mentions this for cifar10 and tinyimagenet, which is only 32x32 and 64x64. This problem likely becomes more severe when we deal with larger real images. But, if we use deep-net-based edge detector, we now break the assumption that edge map does not change much under adversarial attacks. Since it is deep net and so it is fragile. So I am not sure how the proposed method work with deep-net-based edge detectors, on a somewhat more realistic image datasets.

2. For GSD, as mentioned by the authors, it is similar to the two GAN/VAE-based baselines. I am curious to see how it compares with them? 

3. Also for GSD, considering scaling up to more realistic images with more visual patterns, it would be super hard to learn a mapping from pure depth to rgb, since it is ill-posed and under-determined. The two baseline methods do not have this under-determination because their input is rgb images. Also, the learned mapper would be very correlated with and overfitting to the training data, which hinders generalization.

4. How does CW attack perform against the proposed defense?

5. A minor thing about reference. For the fast AT algorithms, to my best knowledge, I think there is a third one "Bilateral Adversarial Training: Towards Fast Training of More Robust Models Against Adversarial Attacks" published in ICCV 2019.



review_title: important topic, some concerns about the proposed methods

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:
- In this paper, the authors try to learn robust models for visual recognition and propose two defense methods, Edge-guided Adversarial Training and GAN-based Shape Defense (GSD), to use shape bias and background subtraction to strengthen the model robustness.

However, I have still some concerns below:
- In summary, the paper is hard to follow and the writting is not clear, such as the detailed motivation of the proposed methods and the structure of this paper.
- For the experiments, a big dataset is needed, such as CIFAR-100. In addition, the results are not convincing, i.e., the evaluation on FGSM and PGD attack is not enough, some gradient-free attacks are needed. 
review_title: Official Blind Review #2 

final_score: rating: 3: Clear rejection
review_content: This paper investigates incorporating shape information in deep neural networks to improve their adversarial robustness. It proposes two methods: the first one is to augment the input with the corresponding edge and then adversarially train a CNN on the augmented input. The second idea is to train a conditional GAN to reconstruct images from edge maps and use the reconstructed image as input to a standard classifier. 

1. The description of the proposed defense in section 3 seems to be limited. It is not clear why the author applied a conditional GAN to reconstruct clean images from edge maps. In other words, what is the motivation for designing GSD on top of EAT?

2. The authors use Canny edge detector to extract edges. Why not use neural network based edge extractors [2] as they give better edges? What is the motivation here? 

3. Considering the possible obfuscated gradient issues of white-box attacks [3], the authors should explicitly describe their efforts to evaluate against strong custom adaptive attacks.

4. In terms of the experiments, the authors claim that they investigated adaptive attack but I did not see any quantitative experiment results. They also claim that any adaptive attack would cause perceptible changes to the edges. This is not an excuse for not doing quantitative study; the authors already considered adversarial perturbations with magnitude as large as 64. Such magnitude can also cause perceptible changes to images as Figure 8 shows.

5. For EAT, what is the performance if the model is not adversarially trained? Why use adversarial training in EAT but not in GSD? I believe these analyses are required for an in-depth understanding of how the proposed defense works.

6. Last but not least, the algorithms proposed in this paper looks similar (almost the same) to this paper [1] from previous year: (a) The edge-guided adversarial training (EST) is basically applying adversarial training on EdgeNetRob in [1]; (b) The GAN-based shape defense (GSD) is exactly the same as EdgeGANRob in [1]; (c) Both of them use canny edge detector to extract edges. Can the authors highlight the differences? If this is a separate paper, given the previous work [1] that already proposed this idea, the contribution of this work seems to be limited. 

[1] Shape Features Improve General Model Robustness. https://openreview.net/forum?id=SJlPZlStwS, 2019.

[2] Richer Convolutional Features for Edge Detection. Liu, et al TPAMI, 2019.

[3] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. Anish et al, ICML 2018.

review_title: Repeated work and limited contributions.

Meta-review: The paper proposed a method for adversarial robustness by considering information from the edge map of the images. Two reviewers point out the similarities of the paper with previous work ([1]) and it is unclear whether the benefits come from binarization of the input or from shape information. As such, the paper is not suggested for publication at this time
Decision: Reject