The shape and simplicity biases of adversarially robust ImageNet-trained CNNs

Adversarial training has been the topic of dozens of studies and a leading method for defending against adversarial attacks . Yet , it remains largely unknown ( a ) how adversarially-robust ImageNet classifiers ( R classifiers ) generalize to out-ofdistribution examples ; and ( b ) how their generalization capability relates to their hidden representations . In this paper , we perform a thorough , systematic study to answer these two questions across AlexNet , GoogLeNet , and ResNet-50 architectures . We found that while standard ImageNet classifiers have a strong texture bias , their R counterparts rely heavily on shapes . Remarkably , adversarial training induces three simplicity biases into hidden neurons in the process of “ robustifying ” the network . That is , each convolutional neuron in R networks often changes to detecting ( 1 ) pixel-wise smoother patterns i.e . a mechanism that blocks highfrequency noise from passing through the network ; ( 2 ) more lower-level features i.e . textures and colors ( instead of objects ) ; and ( 3 ) fewer types of inputs . Our findings reveal the interesting mechanisms that made networks more adversarially robust and also explain some recent findings e.g . why R networks benefit from much larger capacity ( Xie & Yuille , 2020 ) and can act as a strong image prior in image synthesis ( Santurkar et al. , 2019 ) . 1 INTRODUCTION . Given excellent test-set performance , deep neural networks often fail to generalize to out-ofdistribution ( OOD ) examples ( Nguyen et al. , 2015 ) including “ adversarial examples ” , i.e . modified inputs that are imperceptibly different from the real data but change predicted labels entirely ( Szegedy et al. , 2014 ) . Importantly , adversarial examples can transfer between models and cause unseen , all machine learning ( ML ) models to misbehave ( Papernot et al. , 2017 ) , threatening the security and reliability of ML applications ( Akhtar & Mian , 2018 ) . Adversarial training—teaching a classifier to correctly label adversarial examples ( instead of real data ) —has been a leading method in defending against adversarial attacks and the most effective defense in ICLR 2018 ( Athalye et al. , 2018 ) . Besides improved performance on adversarial examples , test-set accuracy can also be improved , for some architectures , when real images are properly incorporated into adversarial training ( Xie et al. , 2020 ) . It is therefore important to study how the standard adversarial training ( by Madry et al . 2018 ) changes the hidden representations and generalization capabilities of neural networks . On smaller datasets , Zhang & Zhu ( 2019 ) found that adversarially-robust networks ( hereafter , R networks ) rely heavily on shapes ( instead of textures ) to classify images . Intuitively , training on pixel-wise noisy images would encourage R networks to focus less on local statistics ( e.g . textures ) and instead harness global features ( e.g . shapes ) more . However , an important , open question is : Q1 : On ImageNet , do R networks still prefer shapes over textures ? It remains unknown whether such shape preference carries over to the large-scale ImageNet ( Russakovsky et al. , 2015 ) , which often induces a large texture bias into networks ( Geirhos et al. , 2019 ) e.g . to separate ∼150 four-legged species in ImageNet . Also , this shape-bias hypothesis suggested by Zhang & Zhu ( 2019 ) seems to contradict the recent findings that R networks on ImageNet act as a strong texture prior i.e . they can be successfully used for many image translation tasks without any extra image prior ( Santurkar et al. , 2019 ) . The above discussion leads to a follow-up question : Q2 : If an R network has a stronger preference for shapes than standard ImageNet networks ( hereafter , S networks ) , will it perform better on OOD distorted images ? Networks trained to be more shape-biased can generalize better to many unseen ImageNet-C ( Hendrycks & Dietterich , 2019 ) image corruptions than S networks , which have a strong texture bias ( Brendel & Bethge , 2019 ) . In contrast , there was also evidence that classifiers trained on one type of images often do not generalize well to others ( Geirhos et al. , 2018 ; Nguyen et al. , 2015 ; Kang et al. , 2019 ) . Importantly , R networks often underperform S networks on original test sets ( Tsipras et al. , 2019 ) perhaps due to an inherent trade-off ( Madry et al. , 2018 ) , a mismatch between real vs. adversarial distributions ( Xie et al. , 2020 ) , or a limitation in architectures—AdvProp helps improving performance of EfficientNets but not ResNets ( Xie et al. , 2020 ) . Most previous work aimed at understanding the behaviors of R classifiers as a function but little is known about the internal characteristics of R networks and , furthermore , their connections to the shape bias and generalization performance . Here , we ask : Q3 : How did adversarial training change the hidden neural representations to make classifiers more shape-biased and adversarially robust ? In this paper , we harness the common benchmarks in ML interpretability and neuroscience—cueconflict ( Geirhos et al. , 2019 ) , NetDissect ( Bau et al. , 2017 ) , and ImageNet-C—to answer the three questions above via a systematic study across three different convolutional architectures—AlexNet ( Krizhevsky et al. , 2012 ) , GoogLeNet ( Szegedy et al. , 2015 ) , and ResNet-50 ( He et al. , 2016 ) — trained to perform image classification on the large-scale ImageNet dataset ( Russakovsky et al. , 2015 ) . Our main findings include:1 1 . R classifiers trained on ImageNet prefer shapes over textures∼67 % of the time ( Sec . 3.1 ) — a stark contrast to the S classifiers , which use shapes at only ∼25 % . 2 . Consistent with the strong shape bias , R classifiers interestingly outperform S counterparts on texture-less , distorted images ( stylized and silhouetted images ) ( Sec . 3.2.2 ) . 3 . Adversarial training makes R networks more robust by ( 1 ) blocking pixel-wise input noise via smooth filters ( Sec . 3.3.1 ) ; ( 2 ) narrowing the input range that highly activates neurons to simpler patterns , effectively reducing the space of adversarial inputs ( Sec . 3.3.2 ) . 4 . Units that detect texture patterns ( according to NetDissect ) are not only useful to texturebased recognition as expected but can be also highly useful to shape-based recognition ( Sec . 3.4 ) . By aligning NetDissect and cue-conflict frameworks , we found that hidden neurons in R networks are surprisingly neither strongly shape-biased nor texture-biased , but instead generalists that detect low-level features ( Sec . 3.4 ) . 2 NETWORKS AND DATASETS . Networks To understand the effects of adversarial training across a wide range of architectures , we compare each pair of S and R models while keeping their network architectures constant . That is , we conduct all experiments on two groups of classifiers : ( a ) standard AlexNet , GoogLeNet , & ResNet-50 ( hereafter , ResNet ) models pre-trained on the 1000-class 2012 ImageNet dataset ; and ( b ) three adversarially-robust counterparts i.e . AlexNet-R , GoogLeNet-R , & ResNet-R which were trained via adversarial training ( see below ) ( Madry et al. , 2018 ) . Training A standard classifier with parameters θ was trained to minimize the cross-entropy loss L over pairs of ( training example x , ground-truth label y ) drawn from the ImageNet training set D : arg min θ E ( x , y ) ∼D [ L ( θ , x , y ) ] ( 1 ) On the other hand , we trained each R classifier via Madry et al . ( 2018 ) adversarial training framework where each real example x is changed by a perturbation ∆ : arg min θ E ( x , y ) ∼D [ max ∆∈P L ( θ , x+ ∆ , y ) ] ( 2 ) 1All code and data will be available on github upon publication . where P is the perturbation range ( Madry et al. , 2018 ) , here , within an L2 norm . Hyperparameters The S models were downloaded from PyTorch model zoo ( PyTorch , 2019 ) . We trained all R models using the robustness library ( Engstrom et al. , 2019 ) , using the same hyperparameters in Engstrom et al . ( 2020 ) ; Santurkar et al . ( 2019 ) ; Bansal et al . ( 2020 ) . That is , adverarial examples were generated using Projected Gradient Descent ( PGD ) ( Madry et al. , 2018 ) with an L2 norm constraint of 3 , a step size of 0.5 , and 7 PGD-attack steps . R models were trained using an SGD optimizer for 90 epochs with a momentum of 0.9 , an initial learning rate of 0.1 ( which is reduced 10 times every 30 epochs ) , a weight decay of 10−4 , and a batch size of 256 on 4 Tesla-V100 GPU ’ s . Compared to the standard counterparts , R models have substantially higher adversarial accuracy but lower ImageNet validation-set accuracy ( Table 1 ) . To compute adversarial accuracy , we perturbed validation-set images with the same PGD attack settings as used in training . Correctly-labeled image subsets : ImageNet-CL Following Bansal et al . ( 2020 ) , to compare the behaviors of two networks of identical architectures on the same inputs , we tested them on the largest ImageNet validation subset ( hereafter , ImageNet-CL ) where both models have 100 % accuracy . The sizes of the three subsets for three architectures—AlexNet , GoogLeNet , and ResNet—are respectively : 17,693 , 24,581 , and 27,343 . On modified ImageNet images ( e.g . ImageNet-C ) , we only tested each pair of networks on the modified images whose original versions exist in ImageNet-CL . That is , we wish to gain deeper insights into how networks behave on correctly-classified images , and then how their behaviors change when some input feature ( e.g . textures or shapes ) is modified . 3 EXPERIMENT AND RESULTS . 3.1 DO IMAGENET ADVERSARIALLY ROBUST NETWORKS PREFER SHAPES OR TEXTURES ? . It is important to know which type of feature a classifier uses when making decisions . While standard ImageNet networks often carry a strong texture bias ( Geirhos et al. , 2019 ) , it is unknown whether their adversarially-robust counterparts would be heavily texture- or shape-biased . Here , we test this hypothesis by comparing S and R models on the well-known cue-conflict dataset ( Geirhos et al. , 2019 ) . That is , we feed “ stylized ” images provided by Geirhos et al . ( 2019 ) that contain contradicting texture and shape cues ( e.g . elephant skin on a cat silhouette ) and count the times a model uses textures or shapes ( i.e . outputting elephant or cat ) when it makes a correct prediction . Experiment Our procedure follows Geirhos et al . ( 2019 ) . First , we excluded 80 images that do not have conflicting cues ( e.g . cat textures on cat shapes ) from their 1,280-image dataset . Each texture or shape cue belongs to one of 16 MS COCO ( Caesar et al. , 2018 ) coarse labels ( e.g . cat or elephant ) . Second , we ran the networks on these images and converted their 1000-class probability vector outputs into 16-class probability vectors by taking the average over the probabilities of the fine-grained classes that are under the same COCO label . Third , we took only the images that each network correctly labels ( i.e . into the texture or shape class ) , which ranges from 669 to 877 images ( out of 1,200 ) for 6 networks and computed the texture and shape accuracies over 16 classes . Results On average , over three architectures , R classifiers rely on shapes ≥ 67.08 % of the time i.e . ∼2.7× higher than 24.56 % of the S models ( Table 2 ) . In other words , by replacing the real examples with adversarial examples , adversarial training causes the heavy texture bias of ImageNet classifiers ( Geirhos et al. , 2019 ; Brendel & Bethge , 2019 ) to drop substantially ( ∼2.7× ) .

Summary: This paper takes a step further to understand the relationships between the adversarial trained CNNs (R-CNNs) and shape-based representation, and delve deeper into the R-CNNs via studying the hidden units. First, it justifies that the R-CNNs prefer shape cues based on random-shuffled, Stylized-ImageNet, and silhouette experiments. Then, it tests R-CNNs on ImageNet-C to show the less connection between the shape-biased and the robustness against common corruptions. Finally, it studies the hidden unit via qualitative tools including NetDissect.