Shape Defense
Authors: ali borji
Venue:  (2021)
Keywords: adversarial robustness
adversarial defense
adversarial attack
shape
background subtraction

Abstract: Humans rely heavily on shape information to recognize objects. Conversely, convolutional
neural networks (CNNs) are biased more towards texture. This fact
is perhaps the main reason why CNNs are susceptible to adversarial examples.
Here, we explore how shape bias can be incorporated into CNNs to improve their
robustness. Two algorithms are proposed, based on the observation that edges are
invariant to moderate imperceptible perturbations. In the first one, a classifier is
adversarially trained on images with the edge map as an additional channel. At
inference time, the edge map is recomputed and concatenated to the image. In the
second algorithm, a conditional GAN is trained to translate the edge maps, from
clean and/or perturbed images, into clean images. The inference is done over the
generated image corresponding to the input’s edge map. A large number of experiments
with more than 10 data sets have proved the effectiveness of the proposed
algorithms against FGSM and `1 PGD-40 attacks. against FGSM and `$\ell_\infty$ PGD-40 attacks. 
Further, we show that edge information can a) benefit other adversarial training methods, b) be even more effective
in conjunction with background subtraction, c) be used to defend against poisoning
attacks, and d) make CNNs more robust against natural image corruptions
such as motion blur, impulse noise, and JPEG compression, than CNNs trained
solely on RGB images. From a broader perspective, our study suggests that CNNs
do not adequately account for image structures and operations that are crucial for
robustness. The code is available at: https://github.com/[masked].

Under review as a conference paper at ICLR 2021
SHAPE DEFENSE
Anonymous authors Paper under double-blind review
ABSTRACT
Humans rely heavily on shape information to recognize objects. Conversely, con- volutional neural networks (CNNs) are biased more towards texture. This fact is perhaps the main reason why CNNs susceptible to adversarial examples. Here, we explore how shape bias can be incorporated into CNNs to improve their robustness. Two algorithms proposed, based on the observation that edges are invariant to moderate imperceptible perturbations. In the first one, a classifier is adversarially trained on images with the edge map as additional channel. At inference time, the edge map is recomputed and concatenated to the image. In the second algorithm, a conditional GAN is trained to translate the edge maps, from clean and/or perturbed images, into clean images. The inference is done over the generated image corresponding to the input's s edge map. large number of exper- iments with more than 10 data sets demonstrate the effectiveness of the proposed algorithms against FGSM, Lo PGD-40, Carlini-Wagner, Boundary, and adaptive attacks. Further, we show that edge information can a) benefit other adversarial training methods, b) even more effective conjunction with background sub- traction, c) be used to defend against poisoning attacks, and d) make CNNs more robust against natural image corruptions such as motion blur, impulse noise, and JPEG compression, than CNNs trained solely RGB images. From broader perspective, our study suggests that CNNs do not adequately account for image structures and operations that are crucial for robustness. The code is available at: htttts:://giiiii com [masked]
INTRODUCTION
Deep neural networks (LeCun et al. 2015) remain the state of the art across many areas and are employed in a wide range of applications. They also provide the leading model of biological neural networks, especially in visual processing (Kriegeskorte) 2015). Despite the unprecedented success, however, they can be easily fooled by adding carefully-crafted imperceptible noise to normal in- puts (Szegedy et al.] 2014: Goodiellow et al. 2015). This poses serious threats in using them in safety- and sncurity-critice domains. Intensive efforts are ongoing to remedy this problem.
Our primary goal here is to learn robust models for visual recognition inspired by two observa- tions. First, object shape remains largely invariant to imperceptible adversarial perturbations (Fig [ Shape is sign of an object and plays vital role in recognition (Biederman 1987). We rely heavily on edges and object boundaries, whereas CNNs emphasize more on texture (Geirhos et al.) 2018))
Second, unlike CNNs, we recognize objects one at a time through attention and background sub-
traction (e.g., Itti & Koch (2001)). These may explain why adversarial examples are perplexing.
The convolution operation in CNNs is biased towards capturing texture since the number of pixels constituting texture far exceeds the number of pixels that fall on the object boundary. This in turn provides a big opportunity for adversarial image manipulation. Some attempts have been made to emphasize more on edges. for example by utilizing normalization layers (e.g.. contrast and divisive normalization (Krizhevsky et al., 2)12)). Such attempts, however, have not been fully investigated for adversarial defense. Overall, how shape and texture should be reconciled in CNNs continues to be an open question. Here we propose two solutions that can be easily implemented and integrated in existing defenses. We also investigate possible adaptive attacks against them. Extensive experiments across ten datasets, over which shape and texture have different relative importance, demonstrate the effectiveness of solutions against strong attacks. Our first method performs adversarial training on edge-augmented inputs. The second method uses a conditional GAN (Isola et al.) 2017) to translate edge maps to clean images, essentially finding a perturbation-invari transformation.

---

Under review as a conference paper at ICLR 2021
FGSM
PGD-40
odv. -ing
odge mrap
espersp
odge map
sbt. img iing
odge maap
odga mapadd
dif edge map
>20
DeepFool
CW
stapp0 1000
stepe
Figure 1:
Adversarial attacks against ResNer152 over the giant panda image using FGSM
Goodfellow
al. 2015), PGD-40 (Madry et al. 2017) (a=8/255). DeepFool (Moosavi-Dezfooli et al.] 2016) and Carlin-- Wagner (Carlini& & Wagner 2017 7) attacks. The second columns in panels show the difference (La) between the original image (not shown) and the adversarial one (values shifted by 128 and clamped). The edge map (using Canny edge detector) remains almost intact small perturbations. Notice that edges are better preserved for the PGD-40. See Appx. A for more detailed version this figure, and also same using the Sobel method.
There is need for adversarial training (and hence less computation) in this method. Further, and perhaps less surprising, we find that incorporating edges also makes CNNs more robust to natural images corruptions and backdoor attacks. The versatility and effectiveness of these approaches, without significant parameter tuning, is very promising. Ultimately, our study shows that shape is the key to build robust models and opens a new direction for future research in adversarial robustness.
2 RELATED WORK
Here, we provide a brief overview of the closely related research with an emphasis on adversarial defenses. For detailed comments on this topic, please refer to Akhtar & Mian 2018).
Adversarial attacks. The goal of the adversary is to craft an adversarial input 5 E Rd by adding an imperceptible perturbation to the (legitimate) input Id (here in the range [0.1], i.c., ã x + c. Here, we consider two attacks based on the -nom of e, the Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2015), as well as the Projected Gradient Descent (PGD) method (Madry) et al. 2017). Both white-bos box and black-box attacks in the untargeted condition are considered. Deep models are also susceptible to image transformations other than adversarial attacks (e.g., noise, blur), as is shown in Hendrycks & Dietterich (2019) and Azulay & Weiss| (2018).
Adversarial defenses. Recently, there has been a surge of methods to mitigate the threat from adver- sarial attacks either by making models robust to perturbations or by detecting and rejecting malicious inputs. popular defense is adversarial training in which network trained on adversarial ex- amples (Szegedy et al. [2014] Goodfellow et al. 2015). In particular, adversarial training with a PGD adversary remains empirically robust to this day (Athalye et al., 2018). Drawbacks of adversarial training include impacting clean performance, being computationally expensive, and overfitting to the attacks it trained on. Some defenses, such as Feature Squeezing (Xu et al. (017). Feature Denoising (Xie et al. (019)), PixelDefend (Song et al. (017)) JPEG Compression (Dziugaite et al., 2016) and Input Transformation (Guo et al., 2017). attempt to purify the maliciously perturbed images by transforming them back towards the distribution seen during training. MagNet (Meng & Chen 2017) trains a reformer network (one or multiple auto-encoders) to move the adversurial im- age closer the manifold of legitimate images. Likewise, Defense-GAN (Samangouei et al.) 2018)
uses GANs (Goodfellow et al. (014)) to project samples onto the manifold of the generator before classifying them. A similar approach based on Variational AutoEncoders (VAE) is proposed in Li & Jil (2019). Unlike these works which are based on texture (and hence are fragile (Athalye et al., 2018 our GAN-based defense is built upon edge maps. Some defenses are inspired by biology (e.g., Dapello et al. (2020). Li et al. (2019). Strisciuglio et al. (2000) Reddy et al. (2020)).
Shape vs. texture. Geirhos et al. (2018) discovered that CNNs routinely latch on to the object texture, whereas humans pay more attention to shape. When presented with stimuli with conflicting cues (e.g., a cat shape with elephant skin texture; Appx. A), human subjects correctly labeled them based on their shape. In sharp contrast, predictions made by CNNs were mostly based on the texture (See also Hermann & Kornblith (201)).. Similar results are also reported by [aker et al. 20188) |Hermann et al.] (2020) studied the factors that produce texture bias in CNNs and learned that data augmentation plays a significant role to mitigate texture bias. Xiao et al. (2019), in parallel to our work, have also proposed methods to utilize shape for adversarial defense. They perform classification on the edge map rather than the image itself. This a baseline method against which compare our algorithms. Similar to us, they also use GANs to purify the input image.
2

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Edge-guided adversarial training (EAT) for T' epochs, perturbation budget and loss bal- ance ratio a, over dataset of size M for network fo (performed in minibatches in practice). 3 {edge, img, imgedgee] indicates network type and redetect, means edge redetection during training.
for t 1. do
for = 1. M do
// launch adversarial attack (here FGSM and PGD attacks) .44 = clip(æ, + signl V (..((()))))))
if 3 = imgedge & redetect then
2, detect edge(E:)
// recompute and replace the edge map
end if
L {(fo(r,), vn) (1 ((fo(;:), v.) // here 0.5
0 = 0 Vol // /pudaae model weights with some optimizer, e.g, Adam
end for end for
Alg Algorithm 2 GAN-based shape defense (GSD)
// Training
1. Create a dataset of images X = { yi ) ji=1...N including clean and/or perturbed images
2. Extract edge maps (e;) for all images in the dataset
3. Train a conditional GAN Py (2/ec [e] to map edge image e to clean image x // here pix2pix
4. Train classifier (y(aa) map generated image to class label y
// Inference
1. For input image æ, clean or perturbed, first compute the edge image e
2. Then, compute Pe Po(y/a") where x' is the generated image corresponding to
PROPOSED METHODS
Edge-guided Adversarial Training (EAT). The intuition here is that the edge map retains the
structure in the image and helps disambiguate the classification (See Fig. [T.. In its simplest form (Fig. 7 A) in Appx. A; Alg. 1), adversarial training is performed over the 2D (Gray+Edge) or 4D EdGg)) input (i.e., number of channels; denoted as Img+)dge). In slightly more complicated form (Fig. BB). first, for each input (clean or adversarial), the old edge map is replaced with the newly extracted one. The edge map can be computed from the average of only image channels or all available channels (i.e., image plus edge). The latter can sometimes improve the results, since the old edge map (although perturbed; Fig, 10land Appx. B) still contains unaltered shape structures. Then, adversarial training is performed over the new input. The reason behind adversarial training with redetected edges is to expose the network to possible image structure damage. The loss for training is a weighted combination of loss over clean images and loss over adversarial images. At inference time, first, the edge map is computed and then classification is done over the ange-augmented input. As baseline model, we also consider first detecting the input's edge map and then feeding it to the model trained on the edges for classification. We refer to this model as Img2Edge.
GAN-based Shape Defense (GSD). Here, first, a conditional GAN is trained to map the edge image, from clean or adversarial images, to its corresponding clean image (Alg. 2). Any image translation method (here pix2pix by [sola et al. (2017) using this code]) can be employed for this purpose. Next, a CNN is trained over the generated images. At inference time, first, the edge map computed and then classification is done over the generated image for this edge image. The intuition is that the edge map remains nearly the same over small perturbation budgets (See Appx. A). Notice that conditional GAN can also be trained on perturbed images (similar to Samangouei et al. (2018) and Li & Ji (2019) or edge-augmented perturbed images (similar to above).
EXPERIMENTS AND RESULTS
4.1 DATASETS AND MODELS
Experiments are spread across 10 datasets covering a variety of stimulus types. Sample images from datasets are given in Fig 2 Models are trained with cross-entropy loss and Adam optimizer (Kingmal
3

---

Under review as a conference paper at ICLR 2021
& Ba 2014) with batch size of 100, for 20 epochs over MNIST and FashionMNIST, 30 over DogV and 10 over the remaining. Canny method (Canny] 1986 is used for edge detection over all datasets, except DogBreeds for which Sobel is used. Edge detection parameters adjusted for each dataset. We did not carry out an exhaustive hyperparameter search, since we are interested in additional benefits edges may bring rather than training the best possible models.
separately
The first two datasets include MNIST (LeCun et al. 1998) and FashionMNIST (Xiao et al. 2017) A CNN with 2 convolution, 2 pooling, and 2 fc layers is trained. Each of these datasets contains 60K training images (resolution x8x28) and 6K test images over 10 classes. The third dataset, Dog contains 18,085 training and 8,204 test images. Images in this dataset are of varying dimensions. They are resized here to 150x 150 pixels to save computation. A CNN with 4 convolu- tion, 4 pooling, and 2 fe layers trained from scratch.
Over the remaining datasets, we finetune a pre-trained ResNet18 (He) jet al.] (016)) trained over ImageNet (Deng et al. 2009). and normalize im- ages using ImageNet mean and standard deviation.
oSE MNST
Fashion
Oa CIFAR10
Dog CM Breeds
G DogiiCCtt
X4 GTSRB
OME Icons50
Livd Sketch
no Imagenette
Tiny Lot ImageNet
MNST oN
2
20
9
3
Figure 2: Sample images from the datasets. Numbers in parentheses denote the number of classes.
The fourth dataset, CIFAR- 10 (Krizhevsky,2009 con- tains 50K training and 10K test images with resolution of 32x32 which are resized here 64 x for better edge detection. The fifth dataset is DogBreeds (see footnote). It contains 1,421 training and 356 test images at resolution 224 224 over 16 classes. The sixth dataset is GTSRB (Stallkamp) iet al. 2012) and includes 39,209 and 1,2631 training and test images, respectively, over 43 classes (resolution 64 x 64 pixels). The seventh dataset. Icons-50, includes 6,975 training and 3,025 test images over 50 classes (Hendrycks & Dietterich 2019). The original image size is 120x 120 which resized to 64x64. The eighth dataset, Sketch, contains 14K training and 6K test images over 250 classes. Images have size 1111 x 1111 and are resized to 64 in experiments (Eitz et al.) 2012). The ninth and tenth datasets are derived from ImageNet The Imagenette2-166 dataset has 3,925 training and 9,469 test images (resolution 160x 160) over 10 classes (tench, English springer, cas- sette player, chain saw, church, French horn, garbage truck, gas pump, golf ball, and parachute). The Tiny Imagenet dataset has 100K training images (resolution 64 64) and 10K validation images (used here as the test set) over 200 classes
For attacks,
/tttpa://github... http://tw.....aaaaaaa/a////aaaaa cept Boundary attack for which we use http://s/ggttaab..c.c////g/gg///
4.2 RESULTS
4.2.1
EDGE-GUIDED ADVERSARIAL TRAINING
Results over MNIST and CIFAR-10 shown in Tables 1 and 2 respectively. In these experi- ments, edge maps are computed only from the gray-level image (in turn computed from the image channels). Please refer to Appx. B for results over the remaining datasets.
Over MNIST and MashionMNISST robust models trained using edges outperform models trained on gray-level images (the last column). The naturally trained models, however, perform better using gray-level images than edge maps (Orig. model column). Adversarial training with augmented inputs improves the robustness significantly over both datasets, except the FGSM attack on Fashion- MNIST. Over CIFAR- 10, incorporating the edges improves the robustness by a large margin against the PGD-40 attack. At 32/255, the performance of the robust model over clean and perturbed images is raised from (0.316, 0.056) to (0.776, 0.392). On average, the robust model shows 64% improvement over the RGB model (last column in Table 2). Results when using the Sobel edge detector instead of the Canny does not show a significant difference (Table 7 in Appx. B). Over the TinylmageNet dataset, as in CI, classification using edge maps is poor perhaps due to the background clutter. Nevertheless, incorporating edges improves the results. We expect even better
www..aggt..cm//ddggssssessssstio http:///dwgggoosssssscccccoooeos -errers---keells-iiiiioonnn & httt:://gthha..cm///sssamgggggge & https://tiny-imaat www.e//ammmneetettuuuu......mm
4

---

Under review as a conference paper at ICLR 2021
Table I: Results (Top-1 acc) over MNIST. The best accuracy in each column is highlighted in bold. In italics the results of the substitute attack. Epsilon values over 255. We used the loo variants of FGSM and PGD. Img2Edge means applying the Edge model (first row) to edge map of the image.
Orig. model
Roh. model (8)
Rob. model (32)
Roh, model (64)
Average Rob. models
Welcan
32
64
Oiclcan
8
Ollccan
32
Olclcan 64
Edge
0.964
0.925
0.586
0.059
0,973
0.954
0.970
0.892
0.964
0.776
0.921
Img2Eddee
0.960
0,951
0.918
0,971
0.957
0.910
0.957
RGSM
Img
0.973
0.717
0.162
0.976
0.955
0,977
0.892
0.770
0.745
0.919
Img+ Tdgee
0.972
0.941
0.664
0,089
0.97
0.958
0.977
0,902
0.972
0.782
0.928
Redetect
0.950
0.803
0.356
0.962 (0.968)
0.919 (0.947)
0.843 (0.881)
0.941
Img.
Redetected Edge
0.974
0.950
0.970
0.771
0.968
0.228
0,810
Redetect
0.958 (0. 966)
0.929 (0.947)
0,922 (0.925)
0.953
Edge
0.964
0.923
0.345
0.000
00.771
0.949
0.973
0.887
0.955
0.739
0.912
Img2Edde
0.961
0,955
0.934
0,970
0.958
0.927
0.960
Imn
0.973
0.008
0.977
0.957
0,978
0.873
0.963
0.658
0,901
Img+ Edge
0.972
0.938
0.446
0,001
0.978
0.953
0.975
0.879
0.965
0,743
0.915
Redetect
0.900
0.741
0.116
0.960
967)
"
(0.948)
(0.908)
0.932
Img
Redetected Edge
0.975
0.949
0.973
0.649
0.968
0,000
0.752
Redetect
0.958 967)
0.945 (0.958)
0,939 (0.942)
0.960
Table 2: Results over the CIFAR- 10 dataset.
Orig, model
Rob. model (8)
Rob. model (32)
Average
OVelean
8
Wclcan
Velean
32
Rob. models
Edge
0.490
0.060
0.015
0.335
0.323
0,382
0.199
0.360
TmggEdge
0.258
0.258
0,270
0.217
FGSM
Img
0,887
0.359
0.246
0,869
0.668
0,855
0,553
0.736
ling + Edge
0.860
0,366
0.169
0.846
0.611
0.815
0,442
0.679
Redetect
0.399
0.281
0.569 (0.631)
0.417 (0.546)
0.662
Img
Redetected Edge
0.846
0.530
0.832
0.337
0.636
Rodetect
0.702 (0.753)
0.569 (0.678)
0.737
Edge
0.490
0.071
0.000
0.337
0,315
0.142
0.119
0.278
Img2Edge
0,259
0.253
0.274
0,253
0.301
PGD 40
Img
0.887
0,018
0.000
0.807
0.450
0.316
0.056
0.407
Imn + Edge
0.860
0.019
0.000
0.788
0,429
0.176
0.119
0.378
Redetect
0.306
0.093
0.504 (0.6465)
0,150
(0.170)
0.404
Img
Redetected
Edge
0.834
0.155
0.776
0.006
0.443
Redetect
0.661 (0.767)
0.392 (0.700)
0.666
results with more accurate edge detection algorithms (e.g., supervised deep edge detectors). Over these 4 datasets, the final model (i.e., adversarial training using image + redetected edge, and edge redetection at inference time) leads to the best accuracy. The improvement over the image is more pronounced at larger perturbations, in particular against the PGD-40 attack (as expected; Fig. D
Over the Dog VsCat dataset, as in FashionMNIST, the model trained on the edge map is much more robust than the image-only model (Table 8 in Appx. B). Over the DogBreeds dataset, utilizing edges does not improve the results significantly (compared to the image model). The reason could be that texture is more important than shape in this fine-grained recognition task (Table 9 Appx. B). Over GTSRB, Icons-50, and Sketch datasets, image+edge model results in higher robustness than the image-only model, but leads relatively less improvement compared to the edge-only model. Please see Tables 11 [3] and 15 Over the Imagenette2-166 160 dataset TTblle[7), classification using images does better than edges since the texture is very important on this dataset.
Average results over 10 datasets is presented in Fig. 3 (left panel). Combining shape and texture (full model) leads to a substantial improvement in robustness over the texture alone (5.24% imp. against FGSM and 28.76% imp. against PGD-40). Also, image+edge model is slightly more robust than the image-only model. Computing the edge map from all image channels improves the results on some datasets (e.g.. GTSRB and Sketch) but hurts some others (e.g.. CIFAR-10) shown in Appx. B. The right two panels in Fig. show a comparison of natural (Orig. model column in tables; solid lines) inference time leads to enhanced robustness with little to no harm to standard accuracy. Despite the Edge model only being trained on edges from clean images, the Img2Fdge model does better than other naturally-trained models against attacks. The best performance, however, belongs to models trained ad adversarially. Notice that our results set new record on adversarial robustness on some of these datasets even without exhaustive parameter search]]
adversarial training. Natural training with image+edge and redetection at
Robustness against Barlini-Wagner (CW) and Boundary attacks. Performance of our method against le CW attack on MNIST dataset is shown in Appx. J. To make experiments tractable, we set the number of attack iterations to 10. With even 10 iterations, the original Edge and Img mod-
*cf. Zhang al. 2019; the best robust accuracy on CIFAR-10 against PGD attacks is under 60%.
5

---

Under review as a conference paper at ICLR 2021
0.00
0.
0.15
PCD
0121
KDD
3.67
04
0C
0:00
1228
B319
150
bae
04
SEge
0.5
029
Geg.Geeeeactt. Sc Mode
Reaus 7 Nead
100
.025
Figure 3: Left) Average results of the EAT defense on all datasets (last cols. in tables). Middle and Right) Comparison of natural (Orig. model column; solid lines) vs. adversarial training averaged over all datasets.
els are severely degraded. Img2Edge and Img+(Edge Redetect) models, however, remain robust. Adversarial training with CW attack results in robust models in all cases.
Results against the decision-based Boundary attack (Brendel et al. 2017) are shown in Appx. K over MNIST and Fashion MNIST datasets. Edge, Img. and Img+Edge models perform close zero over adversarial images. Img+ Redetect) model remains robust since the Canny edge map does not change much after the attack, as is illustrated in Fig. 29
Robustness against substitute model attacks. Following Papernot et al. (2016), we trained substi- tute models to mimic the robust models (with the same architecture but with RGB channels) using the cross-cntropy loss over the logits of the two networks, for 5 epochs. The adversarial examples crafted for the substitute networks were then fed to the robust networks. Results are shown in italics in Tables 2 and (performed only against the edge-redetect models). We find that this attack is not able to knock off the robust models. Surprisingly, it even improves the accuracy in some cases.
Please refer to Appx. E for more details.
Robustness against adaptive attacks. So far we have been using the Canny edge detector which is non-differentiable. What if the adversary builds differentiable edge detector approximate the Canny edge detector and then utilizes it to craft adversarial examples? To study this, we run two experiments. In the first one, we build the following pipeline using the HED deep edge detector (Xie) & Tu y015): Img - HED - Classifier A CNN classifier (as above) is trained over the HED edges on the Imagenette2-
dataset (See Appx. L). Attacking this classifier with FGSM and PGD-5 (c = 8/255) completely fools the network. The original classifier (Img2Edge here) trained on Canny edges, however, is still largely robust to the attacks (i.e., Img"de- IED - Canny - ClassifierConnyy as shown in Table 29 Notice that the HED edge maps are continuous in the range [0, whereas Canny edge maps are binary, which may explain why easy to fool the HED classifier (See Fig. 33).
Above, we used an off the shelf deep edge detector trained on natural scenes. As can be seen in Appx. L. its generated edge maps differ significantly from Canny edges. What if the adversary trains a model with the (input, output) pair as (input image, Canny edge map) to better approximate the Canny edge detector? In experiment two, we investigate this possibility. We build pipeline con- sisting of a convolutional autoencoder followed by a CNN on MNIST. Details regarding architecture and training procedure are given in Appx. M. As results in Fig. 33 reveal, FGSM and PGD-40 at- tacks against the pipeline are very effective. Passing the adversarial images through Canny and then trained (naturally or adversarially) classifier on Canny edges (i.e., Img2Edge). still leads high accuracy, which means that transfer was not successful. We attribute this feat to the binary output of Canny. Two important point deserve attention. First, here we used the Img2Edge model, which as shown above, is less robust compared to the full model (i.e., img+edge and redetection). Thus, adap- tive attacks may be even less effective against the full model. Second, proposed methods perform better when edge map is less disturbed. For example, as shown in Fig. 33 (bottom), the adaptive attack is less effective against the PGD attack since edges are preserved better.
Analysis of parameter By setting examples (Alg. 1), which is computationally more efficient. However, it results in lower accuracy and robustness compared when 0.5, which means exposing the network to both clean and adversarial images is important (See Table [9] Appx. D). Nevertheless, here again incorporating edges improves the robustness significantly compared to the image-only case.
O, the network will be exposed only to adversarial
Why is this method working? The main reason is that the edge map acts as a checksum, and the network learns (through adversarial training) to rely more the redetected edges when other

---

Under review as a conference paper at ICLR 2021
channels are misleading (See Table 23 This aligns with prior observations such as shortcut learning in CNNs (Geirhos et al. 2020). Also, our approach resembles adversarial patch or backdoor/trojan attacks where the goal is to fool a classifier by forcing it to rely on irrelevant cues. Conversely, here we use this trick to make a model more robust. Also, the Img2Fdge model can purify the input before classifying it. Any adaptive attack against the EAT defense has to alter the edges which most likely will result in perceptible structural damages. See also Figs, [0 & [4 in Appx. A.
4.2.2 CAN-BASED SHAPE DEFENSE
We trained the pix2pix model for 10 epochs over MNIST and FashionMNIST, and for 100 epochs over CIFAR-10 and Icons-50 datasets. Sample generated images are shown in Fig. [8] (Appx. F)- A CNN (same architecture as before) was trained for 10 epochs to classify the generated images. Results are shown in Fig. The model trained over the images generated by pix2pix (solid lines in the figure) compared the model trained over the original clean training set (denoted by the dashed lines). Both models are tested over the clean and perturbed versions of the original test sets of the four datasets. Over MNIST and FashionMNIST datasets, GSD performs on par with the original model on clean test images. It is, however, much more robust than the original model against the attacks. When we trained the pix2pix over the edge maps from the perturbed images, the new CNN models became even more robust (stars in Fig. 4 top panels) We expect even better results with training over edge maps from both intact and perturbed images
Over CIFAR-IO and Icons-50 datasets, generated im- ages are poor. Consequently, GSD underperforms the original model over the original clean images. Over the adversarial inputs, however, GSD wins, especially at high perturbation budgets and against the PGID-40 at- tack. With better edge detection and image generation methods (e.g., using perceptual loss), even better results are expected.
Fashion MNIST
0.5
04
MNIST
00
Why is this method working? The main reason is that cGAN learns function f that is invariant to adversar- ial perturbations. Since the edge map not completely invariant to (especially large) perturbations, one has to train the cGAN on the augmented dataset composed of clean and perturbed images. One advantage of this ap- proach is it computational efficiency since there is no need for adversarial training. Any adaptive attack against this defense has to fool the cGAN which is perhaps not feasible since it will be noticed from the generated im- ages (i.e., cGAN will fail to generate decent images). Compared to other adversarial defenses that utilize GANs (e.g.,
10
GSD. FGSM
Orig, model, FOSM
GSD: PGD-40
Orig, model. PCD-40
Icons-50
x
CIFAR-10
6255
Figure 4: Results of GSD method.
(Samangouei et al. (2018); Li &
Ji (019)), our approach relies less on texture. It can be integrated with these defenses.
5 FAST & FREE ADVERSARIAL TRAINING WITH SHAPE DEFENSE
Here, we examine whether incorporating shape bias can empower other defenses, in particular, a) fast adversarial training by WWong et al. (2020), dubbed FasAT,, and free adversarial training by Shafahi et al. (2019). dubbed FreeAT. Wong et al. trained robust models using a much weaker and cheaper ad adversary to lower the cost of adversarial training. They showed that adversarial training with the FGSM adversary is as effective as PGD-based training. The key idea in Shafahi et al. 's work to simultaneously update both the model parameters and image perturbations in one backward pass, rather than using separate gradient computations at each update step. Please see also Appx. G.
The same CNN architectures as in Wong et al. employed here. For FastAT, we trained three models over MNIST (for 10 epochs), FashionMNIST (for 3 epochs), and CIFAR-10 (for 10 epochs & early-stopping) datasets. For FreeAT, we trained models only over CIFAR-10 for 10 epochs.
Results are shown in Table 3 Using saape FastAT and over MNIST, robust accuracy against PGD-50 grows from 95.5% (image-only model) 98.4% (our full model) at e 0.1 and from
SSimilarly, the edge map classifier used in the Fdg2Fdge model in the previous section (FAT defense) can he trained on edge maps from both clean and adversarial examples improve performance.
7

---

Under review as a conference paper at ICLR 2021
Table 3: Performance of edge-augmented FastAT and FreeAT adversarial defenses over clean and perturbed
images (See Appx. G for extended algorithms). FastAT is trained with the FGSM adversary (e 0.1 or
= 0.3) over MNIST and FaSSionMNIST datasets, and e 8/255 over CIFAR-10). FreeAT is trained over CIFAR-10 with = 8/255 and minihatch replays. CIFAR-10 results are averaged over 3 runs (Appx. G).
PGD attacks use 10 random restarts. The remaining settings and parameters are as in Wong et al. (2020).
MNIST (FastAT)
Fashion MNIST (FastAT)
CIFAR-10 (FastAT)
CIFAR-10
(FreeAT)
0.1
0.3
Avg.
0.1
0,3
Avg.
8/255
Avg.
8/255
Avg.
PGD+50
PGD+50
PGD.50
PGD:50
Acc.
PGD-10
Ace.
PG
Acc.
Edige
0.986
0.940
0.113
0.113
0.538
0.844
0.753
0.786
0.310
0.625
0.582
0,386
0.48-00.679
0.678
0.678
Iang
0.991
0.955
0.985
0.877
0.952
0.835
0.696
0.641
0.000
0.545
0.767
0.381
0.574
0.774
0.449
0.612
Img + Edge
0.988
0.968
0.980
0.922
0.965
0.851
0.780
0.834
0.769
0.809
0,874
0.386
0.630
0.782
0.442
0.612
Redetect
0.977
0.966
0.978
0.823
0.778
0.822
0.393
0.634
0.448
0.615
Img + Red. 0.966
0.986
0.087
0.986
0.000
0.515
0.857
0.262
0.817
0.000
0.484
0.806
0.074
0.470
0.777
0.451
0.614
Redetect
0.984
0.986
0.986
0.855
0.823
0.838
0.416
0.641
0.452
0.615
87.7% to 98.6% at e = 0.3, which are even higher than what is reported by Wong et al. (97.5% at = 0.1 and 88.8% at 0.3). Over FashionMNIST, the improvement is even more pronounced (from 69.6% to 85.5% at = 0.1 and from 0% to 82.3% at = 0.3 ). Over clean images, our full model outperforms other models in most of the cases. Over the CIFAR-IO 10 dataset, the shape-basel extension of the defenses results in high accuracy over both clean and perturbed images (using PGD- 10 attack). compared to the image-only model. We expect similar improvements with the classic PGD adversarial training. Overall, our analyses in this section suggest that exploiting edges is not specific to the particular way we perform adversarial training (Algorithms 1&2), and be extended to other defense methods (e.g.. TRADES algorithm by Zhang et al. (2019)).
6 BACKGROUND SUBTRACTION
Background subtraction (a.k.a foreground detection) is an important mechanism by which humans process scenes and recognize objects. interacts with other mechanisms such as edge and boundary detection. How useful is it for adversarial robustness? In other words, how robust the model will be assuming that the attacker has only access to the foreground object? To find out, we perform an experiment over MNIST and FashionMNIST, for which it is easy to derive the foreground masks. We compare the Img and Edge models (from Section 4.2.I) over the original and noisy (digits placed on white noise background) data, with and without background subtraction and edge detection, against the FGSM attack. Results are shown in 5(g.5) First, both models perform poorly over noisy images with the Edge model doing better. Second, post background subtraction, models much more robust. Third, applying the Edge model to the foreground region leads to almost perfect robustness over MNIST. Even without perfect edge detection, the Edge model does very well over MNSS. This analysis provides an upper bound on the potential benefit from background subtraction on model robustness, assuming that foreground objects can be reliably detected.
HARNESSING BACKDOOR ATTACKS
Proposed mechanisms can also withstand invisible and visible backdoor attacks (Brown al. 2017 Liu et al. 2017). Over MNIST, we planted an invisible C-like patch in half of the 8s and relabeled them as 9. We then trained the Img model on this new dataset. The Img model on test set where all 8s are contaminated (with the patch), classifies almost all of them as 9 (top-left panel in Fig. 5 B). The Edge model, however, correctly classifies them as since edge detection removes the pattern (top-right panel). Thanks to the edge detection, it is also not possible to train the Edge model on the poisoned dataset. A similar experiment on MaShionMNISTT using a different patch, shows similar results (bottom panels in Fig. B)B. In presence of visible patches, the model would not be affected the correct region is identified (via background subtraction) during training or testing (Appx. I).
8 ROBUSTNESS AGAINST NATURAL IMAGE DISTORTIONS
Previous work has shown that ImageNet trained CNNs generalize poorly over a wide range of image distortions (e.g.. Azulay & Weiss 2018)) Dodge& & Karam (2017)) Our objective in this section is to study whether increasing shape bias improves robustness against common image distortions just as it did over adversarial examples. Following Hendrycks & Dietterich (019)), we systematically test how model accuracies degrade if images are corrupted by 15 different types of distortions including brightness, contrast, defocus blur, elastic transform, frost, Gaussian noise, glass blur, impulse noise, JPEG compression, motion blur, pixelatation, shot noise, snow, and zoom blur, at 5 levels of severity. Fig. 19 (Appx. H) shows sample images along with their distortions.
We test the original models (trained naturally on clean training images) as well as the robust models (trained adversarially using Algorithm 1) over the corrupted versions of test sets three
8

---

Under review as a conference paper at ICLR 2021
A
NNIIST
Fashion MNIST
B
8023 Dett: inng
8
1237 PAp Jigel
0.8
2
RED lin rodd
8.3 Pred
Eaga rode
0.-
priginal data
0.5
subtraction
data
- parturbed
ueiTUE Ouet tags
Tent iiee
subtraction
2
02-
2012
2-5 Fred
petutel
055 0. 0.15 02
0.5 03
0
05
01
015
02
025
03
Figure 5: A) Background subtraction together with edge detection improves robustmess (here against the FGSM attack). Noisy data is created by overlaying a digit over white noise (noise x (1 (1-mask)ddgiit) B) De- fending backdoor attacks. An almost invisible pattern (with intensity 10/255 of the digit intensity) is added to half of the samples from class, which are then relabeled as another class. Notice that the Fdge model is confused over the edge maps (right panels) since edge detection removes the pattern. In case of visible backdoor attack, background subtraction can help discard the irrelevant region. See Appx. for more details.
datasets. Results are visualized in Fig. 6. See Appx. H for breakdown results on each dataset and distortion. Two conclusions are drawn. First, incorporating edge information in origi- nal models (and hence increasing shape bias) improves robustness against common image dis- tortions (solid curves in Fig. 6 RGB+Egde > RGB or Edge). Improvement is more notice- able at larger distortions and over datasets with less background clutter (e.g.. Icons-50). This is in alignment with Geirhos et al. (2018) where they showed ResNet-50 trained on the Stylized- ImageNet dataset performs better than the vanilla ResNet-50 on both clean and distorted images. Second, adversarially-traiine
mod-
els (in particular those trained on Img + Edge) are more robust to im- age distortions compared to orig- inal models. In summary, incor- porating edges and adversarial im- ages leads to improved robustness against natural image distortions, despite models not being trained on any of the distortions during train- ing. This in turn suggests that the proposed algorithms indeed rely more shape than texture.
TinylmageNet
GTSRB
Icons-50
Edge
Robust 32 RGB Robust
41
RGBEdde Robunt
Figure 6: Classification accuracy over naturally distorted images.
9 DISCUSSION AND OUTLOOK
Two algorithms are proposed to use shape bias and background subtraction to strengthen CNNs and defend against adversarial attacks and backdoor attacks. To fool these defenses one has to perturb the image such that the new edge map is significantly different from the old one while preserving image shape and geometry, which does not seem be trivial low perturbation budgets. Even though we did not perform an exhaustive parameter search (model architecture, epochs, edge detection, cGAN training, etc.), our results are better than or on par with the state of the art in some cases (e.g., over MNIST and CIFAR datasets). The proposed mechanisms are computationally efficient and excel with higher resolution images and low background clutter. They are also more effective against stronger attacks than weaker ones since strong attacks perturb the image less while being more destructive (e.g.. PGD vs. FGSM; Fig. D Shape defense can also be combined with other defenses to produce robust models without a significant slowdown.
Future work should assess shape defense against adversarial attacks such as e.g., gradient-free at- tacks, decision-basee attacks, sparse attacks (e.g., the one pixel attack (Su et al. 2019)). attacks that perturb only the edge pixels, attacks that manipulate the image structure (Xiao et al. 2018), ad-hoc adaptive attacks, and backdoor (Chen al.) 2017)), well as other Ep norms, and datasets. There might be also other ways to incorporate shape- bias in CNNs, such as 1) augmenting a dataset with edge maps negative images, 2) overlaying texture from some objects onto some others as in Geirhos et al. (2018). and 3) designing normalization layers (Carandini & Heeger 2012). Lastly, the interpretation of the shape defense, in Zhang & Zhu (2019), is another research direction.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Naveed Akhtar and Ajmal Mian. Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access, 6:14410-14430, 2018.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. CoRR, abs/ 1802.00424 2018.
Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize so poorly to small
image transformations? arXiv preprint arXiv: 12177, 2018.
Nicholas Baker, Hongjing Lu, Gennady Erlikhman, and Philip Kellman. Deep convolutional networks do not classify based on global object shape. PLOS computational biology, 14(12): e1006613, 2018.
Irving Biederman. decognition-by-compooe logical review, 94(2):115, 1987.
a theory of human image understanding. Psycho-
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-basse adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv: 2017.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprini arXiv: (rXXv:/712.09665 2017.
John Canny. A computational approach edge detection. IEEE Transactions on pattern analysis and machine intelligence, (6):679-698, 1986.
Matteo Carandini and David Heeger. Normalization as canonical neural computation. Nature Reviews Neuroscience, 13(1):51-62, 2012.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pp. 39-57, 2017.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.055566 2017.
Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David D Cox, and James Di- Carlo. Simulating primary visual cortex at the front of cnns improves robustness to image perturbations. BioRxiv, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- crarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. leee, 2009.
Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recogni- tion performance under visual distortions. In 2017 26th international conference computer communication and networks (ICCCN), pp. 1-7. IEEE, 2017.
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel Roy. A study of the effect of JPG compression on adversarial images. CoRR, abs/1608.00ss 008550. 2016.
Mathias Eitz, James Hays, and Mare Alexa. How do humans sketch objects? ACM Trans. Graph. 2012.
(Proc. SIGGRAPH),
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. arXiv preprint urXiv:1871.1223/ 2018.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. arXiv preprint arXiv:2004.07700 2020.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
10

---

Under review as a conference paper at ICLR 2021
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proc. ICLR, 2015.
Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using input transformations. CoRR, abs/1711.001111 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common ruptions and perturbations. arXiv preprint arXiv:1903. 2019.
Katherine Hermann, Ting Chen, and Simon Kornblith. The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33, 2020.
Katherine L Hermann and Simon Kornblith. Exploring the origins and prevalence of texture bias in convolutional neural networks. arXiv preprint arXiv: 1.09071, 2019.
Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.
Laurent Itti and Christof Koch. Computational modelling of visual attention. Nature reviews neuro- science, 2(3):194-200) 2001.
Diederik Kingma and Jimmy Ba. Adam: A method stochastic optimization.
CoRR,
abs/1412.6980, 2014.
Nikolaus Kriegeskorte. Deep neural networks: new framework for modeling biological vision and brain information processing. Annual review of vision science, 1:417-446, 2015.
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 1998.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep leamnin. Nature, 521(7553):436 444, 2015.
Xiang Li and Shihao Ji. Defense-vae: A fast and accurate defense against adversarial attacks. In Joint European Conference on Machine Learning and Knowledge Discovery Databases, pp. 191-207. Springer, 2019.
Zhe Li, Wieland Brendel, Edgar Walker, Erick Cobos, Taliah Muhammad, Jacob Reimer, Matthias Bethge, Fabian Sinz, Zachary Pitkow, and Andreas Tolias. Learning from brains how regularize machines. In Advances in Neural Information Processing Systems, pp. 9529-9539, 2019.
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. CoRR, abs/1706.06083, 2017.
Dongyu Meng and Hao Chen. Magnet: A two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017. doi: 10.1145/ CCS 2017, Dallas, TX. USA, October 30 -Noemberr 03, 2017, pp. 135-147, 2017. 3133956.3134455.
Seyed-Mohsen Moosavi-Dezia Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. In Proc. CVPR, pp. 2574 2582, 2016.
11

---

Under review as a conference paper at ICLR 2021
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against deep learning systems using adversarial examples. CoRR, abs/1602.02697, 2016.
Manish Vuyyuru Reddy, Andrzej Banburski, Nishka Pant, and Tomaso Poggio. Biologically inspired mechanisms for adversarial robustness. Advances in Neural Information Processing Systems, 33, 2020.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan gate: Protecting classifiers against adversarial attacks using generative models. arXiv preprint 1805.06605. 2018.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 1358-3369, 2019.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv: 10766, 2017.
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. Man vs. computer: Bench- marking machine learning algorithms for traffic sign recognition. Neural networks, 32:323-332, 2012.
Nicola Strisciuglio, Manuel Lopez-Antequera, and Nicolai Petkov. Enhanced robustness of con- volutional networks with push-pull inhibition layer. Neural Computing and Applications, pp- 1-15, 2020.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828 841, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In In Proc. ICLR, 2014.
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.039)) 2020.
Chaowei Xiao, Jun-Yan Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song, Spatially trans- formed adversarial examples. arXiv preprint /0XXvv/80//.000000 2018.
Chaowei Xiao, Mingjie Sun, Haonan Qiu, Han Liu, Mingyan Liu, and Bo Li. Shape features improve general model robustness. 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist novel image dataset for benchmark- ing machine learning algorithms. arXiv preprini urXiv: / 708.077747 2017
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 501-509, 2019.
Saining Xie and Zhuowen Tu. Holistically-nested edge detection. In IEEE International Conference on Computer Vision, 2015.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. CoRR, abs/1 155, 2017.
Hongyang Zhang. Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv preprint 1XX100900..8773 2019.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural net- works. arXiv preprint 1905.0990.. 2019.
12

---

Under review as a conference paper at ICLR 2021
A
ILLUSTRATION OF SHAPE IMPORTANCE IN ADVERSARIAL ROBUSTNESS
image
adversarial
Training
Edge map
adversarial
training
robust model
A
Image
adversarial Input
edge map
Image edge map
edge mapp
adversarial
training
exctrsct odgo
robust model
image
edge moap
image
Inference
edge mapp
bale map
inferencce
exxraaci
edge
concat
robust model
Figure 7: Edge-guided adversarial training (EAT). In its simplest form, adversarial training is per- formed over the 2D EGrey+Edge) or 4D Edge+ input (i.e., number of channels; denoted as Img+Edge). In slightly more complicated form (B), first for each input (clean adversarial), the old edge map is replaced with the newly extracted one. The edge map can be computed from the average of only image channels or all available channels (i.e., image plus edge).
FGSM
PGD-40
dif
dif
DeepPool
SNaDO 1000
the giant panda image using 4 prominent attack types: FGSM (Goodfellow et al., 2015) and PGD-40 (Madry et al. 2017) (a=8/255) for different perturbation budgets {8. 16. 32, 64). well as DeepFool (Moosavi-Dezzooli et al., 2016) and Carlini- Carlini-Wagner (Carlini Wagner 2017). The second column each panel shows the difference (C₂) between the original image (not shown) and the adversarial one (values shifted by 128 and clamped). For DF and CW, values are magnified 20x and then shifted. The edge map (using the Canny edge detector) remains almost intact small perturbations. Notice that edges are better preserved for the PGD-40 attack. See Appx. A for results using the Sobel method.
Figure 8: Adversarial attacks against ResNet152
13

---

Under review as a conference paper at ICLR 2021
FGSM
adv. img img
edge map
edge map adv.
diff. edge
ois
uns
bowd
E-
E-44
PGD-40
E-16
E-4
DeepFool
step==10
CW
steps= 1000
Figure 9: As is in Fig. I in the main text but using the Sobel edge detector. As can be seen edge maps are almost invariant to adversarial perturbation.
14

---

Under review as a conference paper at ICLR 2021
MNIST
==0
= 8
= 32
8 = 64
7
7
image
edge map
7
7
7
7
redetected edge map 7
7
7
GTSRB
image
30
30
30
30
edge map
30
30
30
30
redetected edge map
30
30
0
Figure 10: Illustration of adversarial perturbation over the image well as its edge map. The first row each panel shows the clean or adversarial image (under the FGSM attack). The second row shows the perturbed edge map (i.e., the edge channel of the the 2D or 4D adversarial input). The third row shows the redetected edge map from attacked gray or rgb image (i.e., calculated only from the image channels and excluding the edge map itself).
15

---

Under review as a conference paper at ICLR 2021
Figure 11: Samples images from Sketch and Icons-50 datasets, perturbed with FGSM = 8/255,
and their corresponding edge maps using Canny edge detection.
16

---

Under review as a conference paper at ICLR 2021
+.007 x
a
sign(V ( J x, y))
esign V a ( x v))
"panda" 57.7% confidence
"nematode" 8.2% confidence
"gibbon" 99.3 confidence
correct
+distort
ostrich
correct
+distort
ostrich
Figure 12: Top) Adversarial example generated for the giant panda image using the FGSM
tack (Goodfellow et al.) 2015). Bottom) Adversarial examples generated for AlexNet from Szegedy al. (2014). (Left) correctly predicted sample, (center) difference between correct image, and image predicted incorrectly magnified by 10x (values shifted by 128 and clamped), (right) adversar- ial example (i.e., left image middle image). Even though the left and right images appear visually the same to humans, the left images are correctly classified by DNN classifier while the right im- ages are misclassified as "ostrich, Struthio camelus". Notice that in all of these images the overall image structure and edges preserved.
17

---

Under review as a conference paper at ICLR 2021
A
(a) Texture image
(b) Content image
(c) Texture-shape cue conflict
81.4%
Indian elephant
71.1%
tabby cat
63.9%
Indian elephant
10.3%
indri
17.3%
grey fox
26.4%
indri
8.2%
black
3.3%
Siamese
9.6%
black
B
100 100100000 100 99
97 99 100
100100 98
100 100
87
54
GoogiLoNot
ResNet-50
49
48
AlaxNat
VGG-16 GG
Humans
28
18
original
greyscale
silhouette
edges
texture
C
Figure 13: A) Classification of standard ResNet-50 of (a) a texture image (elephant skin: only texture cues); (b) a normal image of cat (with both shape and texture cues), and (c) an image with texture-shape cue conflict, generated by style transfer between the first two images, B) Accuracy and example stimuli for five different experiments without cue conflict, and C) Sample images from the ImageNet (SIN) dataset created by applying AdaIN style transfer an ImageNet image (left). Figure compiled from Geirhos et al. (2018).
18

---

Under review as a conference paper at ICLR 2021
An example visual illusion simultaneously depicting a portrait of young lady old lady. While fooling humans takes a lot of effort and special skills are needed, deep models are much easier to be fooled. In this example, the artist has carefully added features make the portrait look like an old lady while the new additions will not negatively impact the look of the young lady too much. For example, the right eyebrow of the old lady (marked in red below) does not distort the ear of the young lady too much. See https:/ / imeiium cemeiiimcoom/m/ & jonathan_hhuii adversarial -attacks------ pattacks-b58319at 188118b49777 997b for more details.
Figure 14:
19

---

Under review as a conference paper at ICLR 2021
Shape- Object
Texture Object
Display Image
Choice
ndd Choice
3rd Choice
4th Choice
5th Choice
Can Opener (12.12%)
Electric
Remote
Otter (0%)
Odometer (0.03%)
Guitar (7.66%)
Hook (3.64%)
Control (3.54%)
Corkscrew (3.53%)
Chesapeake Bay Retrever (1.09%)
Ram
Bison (0.06%)
lbex
Ram (31.8%)
Bighorn (18.1%
Hyena (0.48%)
(31.8))
(46.51%)
Sturgeon
Haneycomb
Banded Gocko (5.6%)
Electric Ray (5.57%)
Starfish (15.36%)
Snail (5.56%)
Kite (4.52%)
(0.02%)
(0.48)
Bee
Velvet
Stole
Wool (10.24%)
Velvet (6.97%)
Bonnet (4.53%)
Poncho (4.2%)
(0.04%)
(6.97%)
(29.94%)
Stone Wall (8. (8.2%)
Stone Wall (8.72%)
Bison (0.01%)
Parachute (7.9%)
Tile Roof (5.19%)
Stole (3.6%)
Kitc (3.26%)
Germam Short-
Haired Pomter (7.73%)
Egyptian Cat (6.19%)
Kerty Bluc Terrier (5.6%)
Gorilla (0.03%) (0%)
Dugong
Bluctick (8.53%)
Tabby (5.79%)
Flamingo (0.83%)
Python (0.05%)
Limpkin (13.48%)
Hock (12. (2.46%)
Corkscrew (6.33%)
Bustard (3.8%)
Ostrich (3.63%)
Green Mamba (1.48%)
Custard Apple (6.42%)
Granny Smith (6.8 (6)
Bell Pepper (5.47%)
Gorilla (0%)
Strawberry (19,03%)
Knot (4.5%)
Pencil Sharpener (2.02%)
Ilyena
Cask (0. 19%)
Triccratups (19.8%)
Mortarboar (3.2%)
Podestal (2.4%)
Kite (1.78%)
(0%15%
Rocking Chair (4.8%)
Lobster
Broom (0.8%%
Wool
Knot (4.67%)
Hamper (4.41%)
Stole (3.36%)
(0.02%)
(18.69%)
Figure 15: Classification results based shape texture. The left-most column shows the image presented to model. The second column in each row names the object from which the shape was sampled. The third column names the object from which the textured silhouette was obtained. Probabilities assigned to the object name in columns 2 and 3 are shown as percents below the object label. The remaining five columns show the probabilities (as percents) produced by the network for its top five classifications, ordered left to right in terms of probability. Correct shape classifications the top five are shaded in blue correct texture classifications are shaded orange. Figure from Baker et al. (018).
20

---

Under review as a conference paper at ICLR 2021
B ADDITIONAL RESULTS FOR THE EDGE-AUGMENTED DEFENSE
Results of the shape defense (Algorithm in the main text) over eight datasets. Tables with (*) in their caption have contributed Fig in the main text. some tables, results are computed when the edge map is computed from all image channels, i.e., a gray-level image is first computed by averaging the image channels (Img Edge map) and then new edge map derived.
Table 4: Results over the Fashion MNIST dataset (*)
Orig. model
model (8)
Rob. model (32)
model Ofclcan 64
Dretean
32
64
Welkaa
Welean
32
Rob. models
FGSM
Edge
0.714
0.089
0.750
0.741
Img2Fddee
0.755
0.679
0.452
0.762
0.664
0,420
0.690
Img
0.798
0.670
0.288
0.027
0.798
0.722
0.764
0.584
0.768
0.505
0.690
Iug+EEdge
0.750
Redetect
0.691
0.326
0.053
0.739 (0.761)
0.616 (0.660)
0.491 (0.496)
0.693
Img
Redetected Edge Redetect
0.789
0.719
0.775
0.539
0.762
0.045
0.605
0.739 (0.753)
0.664
0.61 (0.532)
0.721
PGD-40
Edge
0.775
0.711 0.370
0.002
0.783
0.744
0.769
0.661
0.743
0.574
0.712
Ixdge
Iun
0.798
0.659
0.133
0.000
0.792
0.713
0.760
0.515
0.734
0.324
0.649
Img+ Edge
0.809
0.647
0.100
0.000
0.794
0.726
0.765
0.608
0.744
0.368
0.701
Redetect
0.682
0.235 0.014
0.734 (0.760)
0.607
Img
Redetected Edge
0.800
0.717
0.779
0.393
0.771
0.002
0.577
Redelect
0.743 (0.766)
0.694 (0.681)
0.690 (0.50V)
0.746
Table Results over TinyllmgeNeet dataset (*)
Orig. model
Rob. model (8)
Rob. model (32)
Average
models
FGSM
Edge
0.136
0.010
0.001
0.150
0.078
0.098
0.021
0.087
Img2Eddee
0.097
0.096
0.034
0.077
0.105
6.531
0.166
0.074
0.512
0.297
0,488
0,168
0,366
Img Edge
0.522
0.152
0.050
0.508
0.273
0.471
0,148
0.350
Redetect
0.081
0.287 (0.356)
(0.266)
0.357
Img Redetected Edge
0.305
0.204
0.482
0.340
Redetect
0.305 (0.371)
0.171 (0.296)
0.366
PGD-40
Edgee
0.136
0.007
0.000
0.148
0.077
0.039
0,014
0.069
Img2Fddee
0.094
0.092
0.095
0.033
0.079
Imy
0.531
0.019
0.000
0.392
0.150
0.191
0.019
0.188
Img Edge
0.402
0.173
Redetect
0.074
0.009
0.198 (0.353)
0.019 (0.)33)
0.194
Img + Redelected Edge
0.425
0.072
0.328
0.005
0,208
Redetect
0.206 (0.380)
0.073 (0.279)
0.258
21

---

Under review as a conference paper at ICLR 2021
Table 6: Results on CIFAR-10 dataset (edge map computed from 4 channels]
Orig. model
Rob. model (8)
Rob. model (32)
Average Rob. models
Oclcan
Olclean
Welcan
FGSM
Ing+EEdge
0.860
0.366
0.169
0.846
0.611
0.815
0.442
0.679
0.574
Img Redetected Edge
0.848
0.547
0.835
0.351
0.645
Rodetoct
0.696
0,553
0.733
Ing+Eddee Redetect
0,860
0.000
0.789
0.431
0.179
0.135
0.384
0.087
0.501
0.152
0.405
Img
Edge
0.767
Redetect
0.648
0.352
0.651
Table 7: Results on CIFAR dataset using Sobel edge detection (edge map computed from 4 channels]
Orig. model
model
Rob, model (32)
Average Rob. models
Welean
8
(leccan
8
Welcan
FGSM
Img+EEdge Redetect
0.424
0.285
0.645
0.490
0.705
Img Redetected Edge
0,838
0,380
0.842
0.411
0.673
Redetect
0.685
Table 8: Results on VsCat dataset [edge map computed from channels] (*)
Orig. model
Rob. model (8)
Rob, model (32)
Average
(Vllean
32
Weleam
Wcleen
32
Rob, models
FGSM
Edge
0.814
0.633
0.119
0.812
0.757
0,806
0.999
0.843
Img2Edge
0.755
0.584
0.767
0.576
0.740
Img
0.863
0.777
Img+
0.823
0.007
0.000
0.782
0.641
0,808
0.992
0,806
Redetecct
0.043
0.002
0.666
0.986
0.810
Iug
Edge
0.778
Redetect
0.763
0.998
0.850
PGD-40
Edie
0.758
Ing2lddge
0.760
0.568
0,778
0.656
0.754
Img
0.863
0.000
0.000
0.769
0.384
0.500
0.500
0.338
Edge
0.823
0.000
0.000
0.785
0.689
0.816
0.496
0.096
Redetect
0.006
0.000
0.744
0.500
0.711
Img + Redetected
Edge
0.819
0.600
0.817
0.009
0.361
Redetect
0.842
22

---

Under review as a conference paper at ICLR 2021
Table 9; Results on DogBreeds dataset using Sobel edge detection [edge map computed from channels] (*)
Orig. model
Rob, model (8)
Rob, model (32)
Average
Olelcan
32
Wclsan
Weteun
32
Rob. models
Edgge
0,750
0.006
0.031
0.506
0.101
0.413
0.073
0.213
Img2Fdge
0.236
0.194
0.362
0.241
0.380
Inng
0.823
0.449
Img Edge
0,896
0.225
0.098
0.862
0.534
0.820
0.385
0,650
Redetect
0.244
0.171
0.455
0. 0.292
0.607
Img Redetected Edge
0.874
Redetect
0.618
0.419
0.689
PGD-40
Edge
0.000
0.036
1mnge
0,250
0.207
0.301
0.037
0.222
Img
0.899
0.000
0.000
0.793
0.286
0.396
0.025
0.425
Img Edge
0.567
0.406
Redetect
0.008
0.000
0.396
0.065
0.454
Img Redetected Edge
0.772
0.028
0.677
0.000
0.369
0.149
Table 10: Results on DogBreeds dataset using Sobel edge detection (edge map computed from 3 channels]
Orig.
model
Rob. model Weleam
Average Rob. modelss
Olcleun
32
Wcleen
32
FGSM
Inng Edge Redetect
0.812
0.261
0.602
0,258
0.110
0.502
0.275
0.618
Img + Redetected Edge
0.893
0.480
0,848
0.216
0.609
Redetect
0.626
0.388
0.689
Table 11: Results on GTSRB dataset [edge map computed from channels] (*)
Orig. model
Rob. model (8)
Rob. model (32)
Average
Olclean
32
Wcleen
Welemm
32
Rob. models
Edgge
0.938
0.683
0.315
0.947
0.863
0.946
0.701
0.864
Img22ddee
0.501
0.451
0.516
0.469
0.719
Inng
0,955
0.322
0.902
0.562
Img Edge
0
0.624
0.382
0.940
0.842
0.943
0.686
0.853
Redetect
0.592
0.471
0.743
0.626
0,813
Img Redetected Edge
0.925
0.939
0.616
Redettcc
0.844
0.766
0.869
PGD-40
Edige
0.054
0.598
2mgglbdgee
U,501
0.459
0.506
0.462
0.714
Img
0.955
0.189
0.033
0.855
0.495
0.736
0.246
0.583
Img Edge
Redetect
0.526
0.251
0.774
0.514
0.767
Img + Redetected Edge
0.929
0.305
0.893
0.134
0.615
Redetect
0.557
Table 12: Results on GTSRB dataset [edge map computed from channels]
model
Rob, model
Rob, model Welsan
Average
Olelean
32
Welcan
32
Robs models
FGSM
Img Edge
0.951
0.382
0.440
0.943
0.686
0.853
Redetect
0,500
0.395
0.558
0.492
0.733
Img Redetected Edge
0.889
0.891
0.549
0.757
Redetect
0.610
0.577
0.742
23

---

Under review as a conference paper at ICLR 2021
Table 13: Results on Icons-50 dataset fedge map computed from 4 channels] (*)
Orig.
model
model
Average
Olelean
32
Ofcican
O'cloun
32
Rob, models
FGSM
Edge
0.594
Ing2Eddge
0,713
0.690
0.746
0.730
0.817
Imn
0,930
0.495
0.433
0.772
0.789
0.836
0.720
0.779
Inn Edge
0.929
U.569
0,433
0.829
0.818
0.844
0.745
0.809
Redetect
0.470
0.414
0.730
0.732
0,784
Ing Redetected Edge
0.341
0.377
0,849
0.688
0.804
Redetect
0.730
PGD440
Edge
0.883
U-423
0.000
0.902
0.769
0.846
0:404
0.730
Img2Edde
0.683
0.695
0.799
Imy.
0,930
0.341
0.113
0.765
0.663
0.736
0.453
0.654
Imgg Edge
0.929
0.320
0.011
0.800
0.678
0.785
0.366
0.657
Rodetoct
0,248
0.600
0,746
Tmg Redetected Edge
0.838
0.644
0.824
0.097
0.601
Redetect
0.792
0.539
0.748
Table 14: Results on Icons-50 datasct [edge map computed from 3 channels]
Orig. model
Rob. model (8)
Rob. model (32)
Average Rob. models
(CClean
FGSM
ImgeFdge Redoteet
0.929
0.569
0.433
0.829
0.818
0.844
0.745
0.809
0,520
0.460
0.737
0.731
0.785
Img Redetected Edge
0,831
0.788
0.870
0.725
0.804
Redetect
0,783
0.765
0.812
Table Results on Sketch dataset [edge map computed from channels]
Orig. model 8
Roh. model
Roh. model (32)
Average Rob. models
OVelcan
Welsan
8
Welean
32
Edge
0.479
0.167
0.041
0.502
0.343
0.483
0,216
0,386
1mg2Eddee
0.464
0.014
0.494
0.022
0.375
Img
0.532
0.109
0.021
0.530
0.278
0.474
0.144
0.356
Gray Edge
0,486
0.097
0.133
0.286
0.440
0.167
0.352
Rodetect
0.263
0.004
0.355
0.013
0.330
Img + Redetected Edge
0.497
0.180
0,420
0.071
0.292
Redetect
0.416
0,162
0.374
PGD-40
Edge
0.480
0,106
0.508
0.341
0.401
0,330
2Fdge
0.471
0.127
0.499
0.214
0.405
Img
0,532
0.028
0.000
0.538
0.260
0.018
0.000
0.204
Gray + Edge
0.486
0.034
0.000
0.500
0.279
0.026
0.000
0.201
Recletect
0.277
0.024
0.360
0.004
0.223
Img Redetected Edge
0.502
0.121
0.448
0.000
0.268
Redetoct
0.423
0.212
0.396
Table 16: Results on Sketch dataset [edge map computed from channel]
Orig. model
Rob, model
Rob, model (32)
Average Roh. models
0Vclean
Nellea
8
Welean
32
Gray Edge
0.486
0.097
0.019
0.513
0.286
0,440
0.167
0.352
Redetect
0.213
0.005
0.388
0.022
0.341
Img Redetected Edge
0.519
0.296
Redetect
0.397
0.020
0.345
24

---

Under review as a conference paper at ICLR 2021
Table 17: Results on Imagenette2-160 dataset [edge map computed from 4 channels] (*)
Orig.
(8)
Average
Olclean
32
Olclcan
Welean
32
Rob. models
FGSM
Edige
0.780
0.781
0.664
0.245
Img2Edge
0,599
0.598
0.603
0.578
0.656
Img
0.969
0.617
0.409
0.959
0.827
0.466
0.710
0.860
Img Edge
Rodetoct
0.652
0.471
0.812
0.687
0.846
Img
Redetected Edge
0.950
0.747
0.949
0.592
0.810
Redetect
0.834
0.732
0.866
PGD-40
Edge
0,780
0.064
0.000
0.794
0.526
0.577
0.071
0.492
TEdge
0.591
Inng
0,969
0,052
0.005
0.988
0,599
0.808
0.221
0.636
Imn Edgee
0.959
U.045
0.000
0,909
0.558
0.762
0.151
0.595
Img
Redetected Edge
0.944
0.246
0,883
0.046
0.530
Redetect
0.757
0.432
0.754
Table 18: Results Imagenette2- 160 dataset (edge map computed from channels]
Orig. model
Rob. model (8)
Rob. model (32)
Average
Olelcan
32
Ofelean
Welcan
32
Rob. models
Img Edge
0.939
0.613
0.373
0.91
0.801
0.935
0.643
0.833
Redetect
0.611
0.447
0.802
0.673
0,840
Redetected Edge
0952
0.767
0.999
0.596
0.816
Redetect
0.832
0.729
0.865
25

---

Under review as a conference paper at ICLR 2021
C
SUMMARY RESULTS OF USING EDGES FOR ADVERSARIAL DEFENSE
Average over MNIST and FashionMNIST datasets
Average over 10 datasets
fgsm
0.
0.8
0.8
0.6
FGSM
0.5
04
Edge
0.4
Edge
Img2Edge
0.3
Img2Edge
Img
Img
02
Img+ Edge
Img Eddge
Img Edge Redetc.
Img Edge Redetc.
Robust Full Model
Robust Full Model
32
54
4/255
42.55
Pgd
0.8
0.7
06
PGD-40
0.5
0.4
Edge
Edge
Img2Edge
Img2Edge
0.2
Img
ImgeEdge
02
Img Eddge
Img
Img + Edge Redetc.
01
Img Edge Redetc.
Model
0.0
4255
52.55
Figure 16: Comparison of natural (the Orig. column in the tables; solid curves) vs. adversarial training (blue dashed-dot curves). The accuracy at e =0 0 (for adversarial training) is averaged over different robust models (three over MNIST and two over others; corresponding to clean columns in tables). Left column) Average over MNIST and Fashion MNIST datasets, Right column) Average over all datasets. Results show clear advantage of using edges. Over MNIST and FashionMNIST, the model trained on edges alone leads to trade-off between accuracy and robustness. Img+edge model does worse than the Image model but its performance is recovered after adversarial training. Img2Edge model wins over models using natural training. Please see also tables in the main text and Appx. B and the explanation in the main text. Overall, incorporating edge and image together and redetection at inference times leads to higher accuracy and robustness.
26

---

Under review as a conference paper at ICLR 2021
Frg
leghtigee
11
076333
EogteCe:
DoyVicat
Eedit
Debresh
Limu300
FGSM
21
44-
Terrs P9E3T
SPEE
Ioee
Imapyetts)
Rgge
lgg2Fdg
Wib1
14
EeptuCe
(1753
STSRO
CTSRE
Batch
Huugo ciic
PGD-40
25-
34-
W1
Figure 17: Breakdown of natural training (the Orig. row in Tables) over datasets.
27

---

Under review as a conference paper at ICLR 2021
D ANALYSIS OF PARAMETER a IN ALG. (EAT DEFENSE)
Table 19: Results (Top-1 acc.) over MNIST corresponding to a (i.e., adversarial training only on adversarial examples taking part in the loss function). See also Table | in the main text.
Rob. model (8)
Rob, model (32)
Rob. model (64)
Average
0/clean
O/clean
32
O/clean
Rob. models
FGSM
Img+Edge 0.963
0.938
0.959
0.869
0.931
0.684
0.891
Redetect
0.943
0.887
0.727
0,902
Img Redetected Edge
0.963
0.936
0.944
0.588
0.937
0.030
0.733
Redetect
0.948
0.911
0.916
0.937
PGD-40
Edge
0.966
0.940
0.960
0.859
0.928
0.607
0.877
Redetect
0.946
0.883
0.657
0.890
Img + Redetected Edge
0.963
0.933
0.947
0.469
0.936
0.000
0.708
Redetect
0.946
0.913
0.915
0.937
0 (i.e., adversarial training only on adversarial examples taking part in the loss function). See also Table 4 in the main text.
Table 20: Results (Top-1 acc.)
Fashion MNIST corresponding
Rob. model (8)
Rob. model (32)
Rob. model (64)
Average Rob. models
0/clean
8
0/clean
32
0/clean
64
FGSM
Img+Edge 0.756
0.701
0.732
0.619
0.683
0.487
0.663
Redetect
0.707
0.635
0.481
0.666
Img Redefected Edge
0.768
0.705
0.739
0.481
0.693
0.040
0.571
Redetect
0.727
0.660
0.635
0.704
PGD-40
Img+Edge
0.768
0.702
0.749
0.573
0.718
0.432
0.657
Redetect
0.714
0,593
0.510
0.675
Img Redetected Edge
0.778
0.702
0.762
0.414
0.750
0.001
0.568
Redetect
0.725
0.632
0.615
0.710
28

---

Under review as a conference paper at ICLR 2021
E RESULTS OF THE SUBSTITUTE MODEL ATTACK
Table 21: Results of the substitute attack against the robust Img + Edge models (redetect and full model).
MNIST
Fashion MNIST
Tiny Img
8
64
32
32
FGSM
Img- edge model (redetect inference)
Substitute model clean images
0.94
0.9365
0.9314
0.7515
0.7393
0.7311
0.8079
0.7766
0.008
0.008
Substitute model on adversarial images
0.8941
0.5838
0.0992
0.6484
0.3701
0.0967
0.2716
0.2049
0.004
0.003
Robust model clean images
0.9761
0.9766
0.9722
0.7939
0.7692
0.75
0.8463
0.8463
0.508
0.471
Rohust model on adversarial images
0.9623
0.9189
0.842
0.7391
0.6155
0.4908
0.5695
0.4186
0.287
0.161
Robust model on substitute adv. images
0.9678
0.9472
0.8813
0.7609
0.6604
0.4955
0.6307
0.5463
0.356
0.266
Img redetected edge model (redetect Inference)
Substitute model on clean images
0.9381
0.9335
0.9326
0.7513
0,7431
0,7388
0.8
0.7966
0.008
0.008
Substitute model on adversarial images
0.89
0.5696
0,0989
0.6338
0.3663
0.08
0.2879
0.1988
0.004
0.002
Robust model on clean images
0.9742
0.9699
0.9681
0.7891
0.7746
0.7617
0.8456
0.8328
0.495
0.482
Robust model on adversarial images
0.9583
0.9283
0.9216
0.7392
0.664
0.61
0,7032
0.5684
0.380
0.170
Robust model substitute adv. innaee
0.9657
0.5318
0.7528
0.7528
0.296
PGD-40
Img edge model (redetect inference)
Substitute model clean images
0.9391
0.9344
0.9257
0.7531
0.7408
0.7303
0.756
0.194
0.008
0.006
Substiute model on adv. images
0.8906
0.4455
0.0196
0.6473
0.2745
0.0096
0.020
0.003
0,000
0.000
Robust model clean images
0.9782
0.9751
0.9654
0.7938
0.7652
0.7442
0,788
0.179
0.395
0.157
Robust model on adv, images
0.9599
0.9132
0.8039
0.7336
0.6289
0.6068
0.504
0.152
0,242
0.018
Robust model on substitute adv. images
0.9667
0.9477
0.9079
0.7603
0.6656
0.4263
0.646
0.170
0,352
0.103
Img redefected edge model (redetect inference)
Substitute model on clean images
0.9385
0.9363
0.9329
0.7503
0.7471
0.7415
0.804
0.730
0.008
0.008
Substitute model on adv. imuges
0.8888
0.4617
0.0211
0.6458
0.2687
0.01
0.016
0.000
0.000
0.000
Robust model on clean images
0.975
0.9732
0.9682
0.7998
0.7793
0.7715
0.834
0.766
0.425
0.328
Rohust model on ady. images
0.9581
0.9449
0.9386
0.7435
0.6943
0.6902
0.662
0.375
0.206
0.074
model substitute adv. images
0.9665
0.5037
29

---

Under review as a conference paper at ICLR 2021
Analysis of making either the image channel the edge channel in models (i.e., making them zero). Over the original edge augmented model (Img+Edge), the image channel is more important since masking it hurts the model more (compared to the making the edge channel). Conversely, the robust and robust redetect models, masking the edge channel hurts more. This indicates that robust models rely more shape than texture. Models used here adversarially trained against each attack. For example, Img+Edge Robust model is trained separately for e = 8/255. This is the same setup in the main text and tables.
Table 22: Masking channels over MNIST dataset
Img+Edge Model
Img+ Edge Robust
Img+Edge Robust Redefect
8
32
64
32
64
FGSM
Masking Img Channels
0.851
0.841
0.914
0.924
0.850
0,931
0,951
Masking Edge Channel
0.968
0.974
0.975
0.969
0.964
0,942
0.718
PGD
Masking Img Channels
0.851
0.915
0.921
0.920
0.862
0,956
0,956
Masking Edge Channel
0.968
0.975
0.973
0.954
0.970
0.957
0.858
Table 23: Masking channels over Fashion MNIST dataset
Emg+Edge
Img+Edge Edge Robust
Img+Edge Robust Redetect
32
Masking Ing Channels
0.161
0.324
0.526
0.690
0.253
0.560
0.734
Masking Edge Channel
Masking Ing Channels
0.161
0.220
0.585
0.744
0.246
0.677
0.760
Masking Edge Channel
30

---

Under review as a conference paper at ICLR 2021
F SAMPLE GENERATED IMAGES BY THE CONDITIONAL GAN IN GAN-BASED SHAPE DEFENSE (GSD)
GSD: Regular training
MNIST
Fashion MNIST
Icons- 50
CI
9
9
8
8
8
A
5
5
5
1
Original image
edge map
generated image
GSD: Training over adversarial images
MNIST
Fashion MNIST
eps 32/255
eps 64/255
eps 32/255
eps 64/255
7
3
3
5
S
5
5
S
5
4
4
4
2
2
2
3
3
3
7 6
7
7
7
7
adv. image
edge
generated image
map
Figure 18: Top) GSD with a classifier trained on images generated (by pix2pix) only from the edge maps of the clean images, Bottom) GSD with edge maps derived from adversarial examples. Columns from left to right: adversarial images by the FGSM attack, their edge maps, and generated images by pix2pix.
31

---

Under review as a conference paper at ICLR 2021
G SHAPE-BASED EXTENSIONS OF VANILLA PGD ADVERSARIAL TRAINING, FREE ADVERSARIAL TRAINING (FREEAT), AND FAST ADVERSARIAL TRAINING (FASTAT) ALGORITHMS
Algorithm Shape-based PGD adversarial training for 7 epochs, given some radius adversarial step size a and N PGD steps and a dataset size M for a network fo. {edge, img, imgedge) indicates the net.type and redetect_tain mean edge redetection during training.
for / = ....
T do
for = 1.....MMMMM. M do // Perform PGD adversarial attack
8 = 0) or randomly initialized
for NN do
( V VoL(Jo(z; + 80, 8 max(min( 0, e), -t)
â = 5 + a sign
end for
if redetect train & B == imgedge then
8; detect (dge(( // recompute and replace the edge map end if = Vol(Jo(E)), v.) // Update model weights with some optimizer, e.g. SGD
end for end for
Algorithm Shape-based "Free" adversarial training for T epochs, given some radius N mini- batch replays, and dataset of size M for network fo. 8 E {edge, img, imgedge) indicates the net_ype and redetect_traie mean edge redetection during training.
=
hteraee T/N times to account for minibatch replays and run for T total epochs for t 1... T/N do
for = 1. 1....... M do
// Perform simultaneous FGSM adversarial attack and model weight updates T times for N...N do G. x, 5
if redetect_train & == imgedge then
x, (dtect)) // recompute and replace the edge map end if
// Compute gradients for perturbation and model weights simultaneously
V.. Vo Ve(fol
(z.)) y:)
8 = 8 + sign( Vs)
6 max(minn -e)
0 = 0 V // Update model weights with some optimizer. e.g. SGD
end for
end for
end for
32

---

Under review as a conference paper at ICLR 2021
Algorithm Shape-based FGSM adversarial training for 7 epochs, given some radius e, PGD steps, step size a, and dataset of size M for network fa. 3 E {edge, img, imgedge) indicates the net type and redetect train mean edge redetection during training.
for t = 1. do for 1... M do // Perform FGSM adversarial attack
8 Uniform(-c,
8 = + sign(
(osnnVV&lee (VV([a((,, + (0, -t)
8 max(miní
2, = +
if redetect train
imgedge then
T; detect edge(ā:) // recompute and replace the edge map
end if
0 = 0 Vol(f((())) v.) // Update model weights with some optimizer, e.g. SGD
end for end for
Table 24: Performance of the Fast Adversarial Training (FastAT) method over three runs.
Run
Run
Run
Average Clean
Model
Clean
PGD-10 10
Clean
PGD-10 10
Clean
PGD-10
PGD-10
Edge
0.559
0.384
0.581
0. 0.187
0.608
0.586
0.582
0.386
RGB
0.813
0.368
0.598
0.205
0.889
0.569
0.767
0.381
Img Edge
0.863
0.590
0.882
0.334
0.878
0.878
0.874
0.386
Redetect
0,593
0.341
0.245
0,393
RGB Redet. Edge
0.892
0.001
0.817
0.115
0.889
0.105
0.866
0.074
Redetect
0.265
0.656
0.326
0.416
Table 25: Performance of the Free Adversarial Training (FreeAT) method over three runs.
Run
Run
Run
Average Clean
Model
Clean
PGD-10
Clean
PGD-10
Clean
PGD-10
PGD-10
Edge
0.674
0.672
0.704
0.702
0.660
0.659
0.679
0.678
RGB
0.783
0.450
0.768
0.450
0.772
0.447
0.774
0.449
Img Edge
0.784
0.432
0.779
0.447
0.782
0.448
0.782
0.442
Redetect
0.447
0.448
0.449
0.448
RGB Redet. Edge
0.776
0.451
0.776
0.454
0.780
0.447
0.777
0.451
Redetect
0.452
0.456
0.448
0.452
33

---

Under review as a conference paper at ICLR 2021
H
PERFORMANCE OF THE MODELS AGAINST COMMON IMAGE CORRUPTIONS
Tiny ImageNet
GTSRB
Icons-50
Severity
1
2
3
4
2
4
5
2
3
4
5
brightness
contrast
defocus
blur
elastic
transform
fog
frost
gaussian
noise
glass blur
impulse
noise
jpeg
compression
motion blur
pixelate
shot noise
snow
zoom blur
Figure 19: Sample images alongside their corruptions with 5 severity levels.
34

---

Under review as a conference paper at ICLR 2021
Tiny ImageNet
GTSRB
Icons-50
Severity
2
brightness
contrast
defocus
blur
elastic
transform
fog
frost
gaussian
noise
glass blur
impulse
noise
jpeg
compression
motion blur
pixelate
shot noise
snow
zoom blur
Figure 20: Edges for images in Fig.
19
35

---

Under review as a conference paper at ICLR 2021
Edge
RGB
RGBE
ricton SiLY
DIOTL Dur
0.06
02-
0.54
brigtnees
transform
Edge Robust
RGS Robust
RGBEdge Robust
012
0.6
0.
0.0
Edge Robust 32
RGS Robust
RGBEdge Robust 32
0.3
034
02
0.8
Severity
Severit
Stverity
Figure 21: Performance of models against natural image corruptions over the TinylmageNet dataset. Robust models are trained against FGSM attack.
36

---

Under review as a conference paper at ICLR 2021
Edge
dge
0350
06
05
defouu thar
glan Sier
mation
04
8193
0.5
Enggtness
02
0.2
0.35
Avp
Edge Robust
RGB Robust
RGBEdge Robust
0.30
0.4
0.40
0.90
0.35
04
0.89
0.30
0.2
0.55
Edge Robust
Robust 32
AGBEdge Robust
0.-
Top-2 Act.
04-
0.30
23
0.25
02-
0.0
Soverit
Scwerie
Scerriy
Figure 22: Performance of models against natural image corruptions over the GTSRB dataset.
37

---

Under review as a conference paper at ICLR 2021
Edge
PGBE
0.45
0.0
0.5
nolse
419290, relse
debice Baur
glat blar
0.3
0.5
0.00
035
trightnees
Arg
0:5
Edge Robust
Robust
RGBEdge Robust
0.4
0.5
0.2
0.35
0.25
0.5
015
032
010
Edge Robust
RGS Robust
RGBEdge Rebust
035
035
025
0.2
0.20
015
0.2
0.0
Severit
Seceriy
Saaerly
Figure 23: Performance of models against natural image corruptions over the Icons-50 dataset.
38

---

Under review as a conference paper at ICLR 2021
1 EFFECT OF BACKGROUND SUBTRACTION IN ADVERSARIAL DEFENSE
moonioor screen notebook
toaster banana piggy bank spaghetti
Figure 24:
Examples invisible (left) and visible (right) backdoor attacks (e.g., Brown et al. (017 ) Our proposed shape defense easily bypass the invisible backdoor attack since edge detection removes the watermark. Our defense together with background subtraction (akin to gazing single object a time) can also avoid visible backdoor attacks. See also the main text.
39

---

Under review as a conference paper at ICLR 2021
Original data
Noisy data
Background subtraction
sattect
sdd role
deset adye
vottect
Nd5 noln
dotect Ndge
toltar
Adt nose
delect adge
7
S
Z
5
8
6
2
5
3
Img model
S
6
8
8
9
Y
&
2
3
D
8
7
extract /g
Gdd noiso X
desert_cdgg
oxtract 10
000 noto
detect odge
Ietratt 3g
add nolee
detert.ddg
CO
?
Edge model
&
Cra
Figure 25: Application of the Img and Edge models Original, noisy, and backgrouundssubrrateed MNIST digits. Noisy data is created by overlaying a digit over white noise (noise x (1-mask) The FGSM attack is used here. We find that background subtraction together with edge detection improves robustness.
40

---

Under review as a conference paper at ICLR 2021
Original data
Noisy data
Background subtraction
nnbee dedect X
MagH X
deeed Mage
X
Img model
X
X
Edge model
Figure 26: Application of the Img and Edge models over Original, noisy, and background-suubrraeted FashionMNIST data. Noisy data created by overlaying object over white noise (noise (1- tobjec.. The FGSM attack is used here. We find that background subtraction together with edge detection improves robustness.
41

---

Under review as a conference paper at ICLR 2021
Accuracy Epsilon
Accuracy Epsilon
10
MNIST
with bg masking (clean)
masking (clean)
with bg masking (noise)
0.8
w/o masking (noise)
FashionMNIST
0.6
with mesking (clean)
0.5
we bg masking (clean)
masking
0.4
--
wi bg masking (noise)
0.3
02
0.1
0.0
0.00
0.10
0.15
0.25
0.30
0.00
0:10
0.15
0.5
Epsilon
Epsiton
Figure 27: Similar to Fig. in the main text with the difference that here the noise model is trained over the noisy data. Removing the perturbations on the image background (via background subtrac- tion) improves the robustness.
Input image
fg mask
attacked image
masked image
Img model
7
7
7
Edge model
7
7
7
Figure 28: Masking the FGSM attack perturbations (i.c., keeping the altered pixels on the image or edge only regions).
Table 26: Performance of the models (naturally trained and adversarially-traine against the images with only the foreground being impacted/perturban Compared with the results in Table. applying the models the foreground regions improves the accuracy by a large margin.
FGSM
PGD-40
c=
c
c 64
Img (natural training)
0.9598
0.9021
0.7516
0.9598
0.8947
0.7042
Img (adversarial training)
0.9641
0.9237
0.8398
0.9636
0.9139
0.7868
Edge (natural training)
0,9553
0.9199
0.8323
0.9553
0.9213
0.8452
Edge (adversarial training)
0.9686
0.9468
0.9059
0.9686
0.9474
0.9057
42

---

Under review as a conference paper at ICLR 2021
J ROBUSTNESS AGAINST THE CW ATTACK OVER MNIST DATASET
Performance of the the EAT defense against the l2 Carlini- Wagner attack (Carlini & Wagner 2017) with the following parameters:
attack = CW (net, targeted-False, c=1e-4, kappa-0, 10ers=10, 1r=0.001)
Orig. model
Robust model
Average
O/clean
adv.
Ofclean
adv.
Rob. models
Edge
0.964
0.106
0.948
0.798
0.873
Img2Edge
0.962
0.949
0.949
Img
0.973
0.103
0.949
0.856
0,903
Img+Edge
0.972
0.097
0.945
0.845
0.895
Redetect
0.971
0.942
0.944
Img Redetected Edge
0.947
0.819
0.883
Redetect
0.946
0.946
43

---

Under review as a conference paper at ICLR 2021
K ROBUSTNESS AGAINST BOUNDARY ATTACK
Performance of the the edge augmented model against the Boundary attack (Brendel et al. 2017) with the following parameters:
BoundaryAttack Nine,_attacc--ooe, steps=25000, spherical_step-0 01,
source_sted le 07, step step_adaptation=1.., tensorboard=False, saaee__stts_eeerrye
Table 27: Results over 500 images from the MNIST dataset
Orig. model
O/clean
adv. (boundary)
Edge
0.964
0.000
Img
0.973
0.003
Img+Edge
0.972
0.000
Redetect
0.945
"
Img+ Redeteceed Edge (adversarially trained using FGSM 8/255)
0.974
0.001
Redetect
0.965
Table 28: Results over 500 images from the Fashion MNIST dataset
Orig. model adv. (boundary)
0/clean
Edge
0.776
0.005
Img
0.798
0.018
Img+Edge
0.809
0.003
Redetect
0.747
Img+Redetecter Edge (adversarially trained using FGSM = 8/255) Redetect
0.789
0.003
0.770
MNIST
Fashion MNISTT
Adv. Img
7
Edge from Adv. Img
7
Figure 29: Sample images from the Boundary attack.
44

---

Under review as a conference paper at ICLR 2021
L ROBUSTNESS AGAINST ADAPTIVE ATTACKS OVER IMAGENETTE2-160 DATASET
We the PyTorch implementation of the HED edge detector proposed by Xie & Tu 2015). Here, a classifier is first trained on top of the edge maps from the HED. Then, the entire pipeline (Img - HED - Classifier"
attacked to generate an adversarial image. The performance of this classifier is measured on both clean and adversarial images. The adversarial image is also fed to the Canny + Classifier Results are shown in Table below. As it can be seen, adversarial examples crafted for HED fail to completely fool the model trained on Canny edges (i.e., they not transfer).
classifier trained on Canny edge maps (Img -
Table 29: Results over 500 images from the Imagenette2-16/ dataset against the FGSM and PGD-5 = 8/255) attacks.
Orig. model
O/clean adv. (FGSM) adv. (PGD-5)
Img2Edge (Img
HED
Classifier (f ITD)
0.793
0.052
0.003
Img2Edge (Imgadv HED
Canny
Classifier CaniifferCCanngg Canng)
0.767
0.542
0.548
adversarial image
HED edge map
Canny edge map
324
100
370
378
170
340
340
240
100
120
140
20
100
320
180
248
00
224
100
328
Figure 30: Two sample adversarial images (FGSM) along with their edge maps using HED and Canny edge detection methods.
45

---

Under review as a conference paper at ICLR 2021
M ROBUSTNESS AGAINST ADAPTIVE ATTACKS OVER MNIST DATASET
Here, we attempt to explicitly approximate the Canny edge detector using differentiable convolu- tional autoencoder. In our pipeline, classifier (CNN) is stacked after the convolutional autoencoder (with sigmoid output neurons). We first freeze the classifier and train the autoencoder using the MSE loss with (input, output) pair being (image, canny edge map). We then freeze the autoencoder and train the classifier using Cross Entropy loss. After training the network, we then craft adversarial examples for it and feed them a classifier trained on Canny edges (original models or robust mod- els as was mentioned in the main text). Fig. 31 shows the pipeline and some sample approximated edge maps. Fig. 32 shows the architecture details in PyTorch.
The top panel in Fig. 33 shows results using the FGSM and PGD-40 attacks against the pipeline itself, and also against the Img2Edge model (trained over clean edges or adversarial one As can be seen, both attacks are very successful against the pipeline but they do not perform well against the Canny edge map classifier (i.e., crafted adversarial examples for the pipeline do not transfer well to the Imge2Edge trained over Canny Edge map; img Canny -> class label). Notice, that here we only used the model trained on edge maps. is likely to gain even better robustness against the adaptive attacks in using the img+edge+redetect.
The bottom panel in Fig. 33 shows sample adversarial digits (constructed using the adaptive attack) and their edge maps under the FGSM and PGD-40 attacks. Notice how PGD-40 attack preserves the edges (compered to FGSM). This is because it needs less perturbation to fool the classifier. Also, notice that the perturbations shown perceptible which results in edges maps having noise. If limit ourselves to imperceptible perturbations, then edge maps will not change much compared to the original edge maps on clean images.
"Here we used the model adversarially trained at eps=8/255 and test against other perturhations; unlike the main text where trained robust models separately for each epsilon.
46

---

Under review as a conference paper at ICLR 2021
sigmoid activation
Convolutional AutoEncoder
Canny edge map
class label
image
CNN
cross entropy loss
MSE loss
1. Freeze the CNN (requires seruuires_grrd__a False) and train the AutoEncoder
2. Freeze the AutoEncoder (requires __rad_rrd_-_ False) and train the CNN
3. Unfreeze all the network (requires _grad- = True) and attack it
4. Feed the adeversarial image to CNN trained with Canny edge maps
4
9
8
2
Digit
2
Canny
AutoEncoder
9
8
Figure 31: Top: our pipeline to approximate the Canny edge detector and our approach for crafting adversarial examples, Bottom: Sample digits and their generated edge maps.
47

---

Under review as a conference paper at ICLR 2021
combined network
Model definition
class MNIST Net (an. def init (self, net_type tep.tttpeessseee gray )
super (MNIST Net _ee__comined, self). init ()
encoder = nn. Se.Sunnnnt # like the Composition layer you built Conv2d(1, 16, stride=2, padding=1), nn nn.Relu())
nn.Conv2d116, 32, 3, stride=2, padding=1), nn.ReIU()) an.Conv2d(32, 64, 7)
self. .ecodder nn. Sequential
nn con 32, 7),
nn. an.ReIU())
wm.coonTTrannspose2233, 16, 3, stride=2, padding=1, output_padding=-)) an.ReLU(),
mm.ConvTrranspose22(1,, 1, 3, stride=2, padding=1, output (ttput_padding=l), nn.Sigmoid()
self conv1 nn.Conv2d(1, 10, hernel_sizers
calf.conve nn. Cn.Con2dd110, 20, kernel_sizer selv.comv2 drop on.Dropout2d()
self. fcl nn am.Linear(320, 50) self fc2 mn.Linear (50,
def forwa, x):
self. encoderr))))
x x_auto self (elf.deec))r(2)) reconstructed egde
auto viewo, suto.shape(0),1. 28,28)
x ( www.dscolllll.lleeeeeeoeeeaeea.e (x (x_auto), 2))
F.relu(F.max pool2d (self.comv) f
2))
x x.view(-1, 320)
r.relu(seef.ffc1(x)
x dz trainingsself. seainingssll.trrannnnng self.fc2(x)
return x, x auto
Figure 32: PyTorch code our pipeline shown in Fig 31
48

---

Under review as a conference paper at ICLR 2021
Accuracy vs Epsilon (FGSM atatck)
Accuracy vs Epsilon (PGD-40 attack)
auto encoder edge
10
auto encoder
mistt mist edge_robust
miit edge rebust
06
0.8
Accurecy
Accurryy
0.4
0.1
0.2
0.3
0.1
0.2
0.3
Epsilon
Epsilon
FGSM
PGD-40
eps eps= =32/255
777 F
7
7
23-
eps 64/255
7
77
30
Figure 33: Top:
Performance of the adaptive attack, Bottom: Samples adversarial images and their edge maps using the Canny edge detector.
49


Reviews:
[{"review_id": "q7cTTyfraZ9", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper investigates incorporating shape information in deep neural networks to improve their adversarial robustness. It proposes two methods: the first one is to augment the input with the corresponding edge and then adversarially train a CNN on the augmented input. The second idea is to train a conditional GAN to reconstruct images from edge maps and use the reconstructed image as input to a standard classifier. \n\n1. The description of the proposed defense in section 3 seems to be limited. It is not clear why the author applied a conditional GAN to reconstruct clean images from edge maps. In other words, what is the motivation for designing GSD on top of EAT?\n\n2. The authors use Canny edge detector to extract edges. Why not use neural network based edge extractors [2] as they give better edges? What is the motivation here? \n\n3. Considering the possible obfuscated gradient issues of white-box attacks [3], the authors should explicitly describe their efforts to evaluate against strong custom adaptive attacks.\n\n4. In terms of the experiments, the authors claim that they investigated adaptive attack but I did not see any quantitative experiment results. They also claim that any adaptive attack would cause perceptible changes to the edges. This is not an excuse for not doing quantitative study; the authors already considered adversarial perturbations with magnitude as large as 64. Such magnitude can also cause perceptible changes to images as Figure 8 shows.\n\n5. For EAT, what is the performance if the model is not adversarially trained? Why use adversarial training in EAT but not in GSD? I believe these analyses are required for an in-depth understanding of how the proposed defense works.\n\n6. Last but not least, the algorithms proposed in this paper looks similar (almost the same) to this paper [1] from previous year: (a) The edge-guided adversarial training (EST) is basically applying adversarial training on EdgeNetRob in [1]; (b) The GAN-based shape defense (GSD) is exactly the same as EdgeGANRob in [1]; (c) Both of them use canny edge detector to extract edges. Can the authors highlight the differences? If this is a separate paper, given the previous work [1] that already proposed this idea, the contribution of this work seems to be limited. \n\n[1] Shape Features Improve General Model Robustness. https://openreview.net/forum?id=SJlPZlStwS, 2019.\n\n[2] Richer Convolutional Features for Edge Detection. Liu, et al TPAMI, 2019.\n\n[3] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. Anish et al, ICML 2018.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "T1vrN0AoQOW", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper aims to improve adversarial robustness considering the information about the object shape details with the means of edge maps. Two different strategies are proposed to increase model robustness using the edge maps: i) conduct the adversarial training on the input images, which are concatenated with its corresponding edge map as an additional input channel to the image. Here, the edge maps are recomputed and concatenated to the adversarial inputs after their generation during adversarial training. ii) Utilize a conditional GAN to generate the images from clean data distribution that is conditioned on the edge maps and later the classifier performance is evaluated on the generated images. Here, the authors claim that these two strategies improves the classifier robustness after conducting experiments across 10 different datasets. They also studied the effectiveness of their strategy when combined with background subtraction, the defense against poisoning attacks and robustness against natural image corruptions.\n\nStrengths: \n+ Motivation is clear.\n+ The proposed strategies are interesting, explained clearly, easy to follow and are different from existing works. \n+ Rigorous experiments across 10 different datasets are carried out to demonstrate the effectiveness of the proposed strategies.\n+ Results demonstrate that the model robustness can be significantly improved using proposed strategies utilizing the edge maps. The improvement not only limited to adversarial robustness but also extends to backdoor attacks and natural image corruptions.\n\nWeaknesses:\n-\tThe major concern lies in the evaluation of the proposed strategies. Here, the authors considers that their method purify the input image before passing it to the model and an adaptive attack against their edge map based defense strategies will likely results in structural damage to the edge map. However, it is crucial to evaluate the proposed defense against an adversarial attack which craft the adversarial examples to produce minimal structural alterations to the edge map but mislead the model predictions. An adversary could potentially optimize the perturbation in such manner and may remain successful in attacking the model.\n-\tResults on CIFAR-10 and Icons-50 of GAN bases shape defense depicted in Figure 4 do not provide solid evidence on the model robustness. Here, the performance on clean inputs degraded significantly and the improvement seen in perturbed samples might be the result of the trade-off between model robustness and generalization as noted in the literature.\n-\tThe claims on robustness against natural image corruptions using the edge maps seems to be valid only on GTSRB dataset and do not hold true for TinyImageNet and Icons-50 as seen in Figure 6. The robustness of the model with edge maps is similar or on par with the model without edge maps on these two datasets. These results suggest that additional usage of edge maps do not improve model robustness and the improved performance seen on GTSRB could be attributed to the simple nature of the objects in the dataset.\n\nFinal thoughts:\nThe proposed method is clearly motivated. Although the performance gains on adversarial robustness is significant, there are critical points yet to be addressed. Therefore, I marginally accept this paper.\n\n------------------------------------------------------------------------------------------------------------\nPost rebuttal:\nThe authors have devised an adaptive attack to craft the adversarial examples against edge maps and shown that the proposed technique is still remain robust. However, the essence of robustness in this work lies in the BINARIZATION of the input (i.e., binarized edge maps) which is shown in the previous work [1] and need not necessarily attribute to the shape information obtained through edge maps. I recently came across state-of-the-art deep edge detector [2] that produces non-binarized edge maps, which could be interesting for authors to validate their approach using such non-binary inputs. Hence, I maintain my initial rating and marginally accept this paper.\n \n\n[1] ON THE SENSITIVITY OF ADVERSARIAL ROBUSTNESS TO INPUT DATA DISTRIBUTIONS, ICLR 2019\n\n[2] Richer Convolutional Features for Edge Detection, CVPR 2017", "extra_scores": {}, "extra_text": {}}, {"review_id": "RvnyFY9bjY", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies how to incorporate shape (particularly depth map) into CNN for more robust models. The study focuses on image classification. Specifically, this paper proposes two depth-map-based defense: 1) Edge-guided Adversarial Training (EAT), which use depth map as an additional input 2) GAN-based Shape Defense (GSD), which learns a generator from depth map to reconstructed images, which is then used as net input. Experiments on 10 datasets shows the effectiveness of the proposed two defenses against white-box attacks including FGSM and PGD40. To further demonstrate the effectiveness, the authors also conduct some other experiments: 1) the proposed EAT goes well with two fast AT algorithms; 2) the proposed algorithm can also be used to defend backdoor attack; 3) edge makes CNN more robust to common image corruptions.\n\nI think the topic that tries to explore and understand the connection between shape and CNN is very important and somewhat under-studied. This paper provides some interesting empirical results and insights to the community. \n\n1. The assumption of this paper is that edge map does not change much under adversarial attacks. I think this relies on two things: A) we use tradition non-deep-net based edge detector like canny edge; B) the adversarial perturbation is pixel-based perturbation. I am curious to see how the proposed algorithm work with B), but for now, I will focus on A) below.\n\nFor A), for harder and more realistic datasets, canny may not work well and we may resort to deep-net-based edge detector (like HED). I think the author also mentions this for cifar10 and tinyimagenet, which is only 32x32 and 64x64. This problem likely becomes more severe when we deal with larger real images. But, if we use deep-net-based edge detector, we now break the assumption that edge map does not change much under adversarial attacks. Since it is deep net and so it is fragile. So I am not sure how the proposed method work with deep-net-based edge detectors, on a somewhat more realistic image datasets.\n\n2. For GSD, as mentioned by the authors, it is similar to the two GAN/VAE-based baselines. I am curious to see how it compares with them? \n\n3. Also for GSD, considering scaling up to more realistic images with more visual patterns, it would be super hard to learn a mapping from pure depth to rgb, since it is ill-posed and under-determined. The two baseline methods do not have this under-determination because their input is rgb images. Also, the learned mapper would be very correlated with and overfitting to the training data, which hinders generalization.\n\n4. How does CW attack perform against the proposed defense?\n\n5. A minor thing about reference. For the fast AT algorithms, to my best knowledge, I think there is a third one \"Bilateral Adversarial Training: Towards Fast Training of More Robust Models Against Adversarial Attacks\" published in ICCV 2019.\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CAtwMihcqEW", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n- In this paper, the authors try to learn robust models for visual recognition and propose two defense methods, Edge-guided Adversarial Training and GAN-based Shape Defense (GSD), to use shape bias and background subtraction to strengthen the model robustness.\n\nHowever, I have still some concerns below:\n- In summary, the paper is hard to follow and the writting is not clear, such as the detailed motivation of the proposed methods and the structure of this paper.\n- For the experiments, a big dataset is needed, such as CIFAR-100. In addition, the results are not convincing, i.e., the evaluation on FGSM and PGD attack is not enough, some gradient-free attacks are needed. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposed a method for adversarial robustness by considering information from the edge map of the images. Two reviewers point out the similarities of the paper with previous work ([1]) and it is unclear whether the benefits come from binarization of the input or from shape information. As such, the paper is not suggested for publication at this time
Decision: Reject