Intriguing Properties of Vision Transformers

1 Introduction . As visual transformers ( ViT ) attract more interest [ 1 ] , it becomes highly pertinent to study characteristics of their learned representations . Specifically , from the perspective of safety-critical applications such as autonomous cars , robots and healthcare ; the learned representations need to be robust and generalizable . In this paper , we compare the performance of transformers with convolutional neural networks ( CNNs ) for handling nuisances ( e.g. , occlusions , distributional shifts , adversarial and natural perturbations ) and generalization across different data distributions . Our in-depth analysis is based on three transformer families , ViT [ 2 ] , DeiT [ 3 ] and T2T [ 4 ] across fifteen vision datasets . For brevity , we refer to all the transformer families as ViT , unless otherwise mentioned . We are intrigued by the fundamental differences in the operation of convolution and self-attention , that have not been extensively explored in the context of robustness and generalization . While convolutions excel at learning local interactions between elements in the input domain ( e.g. , edges and contour information ) , self-attention has been shown to effectively learn global interactions ( e.g. , 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . relations between distant object parts ) [ 5 , 6 ] . Given a query embedding , self-attention finds its interactions with the other embeddings in the sequence , thereby conditioning on the local content while modeling global relationships [ 7 ] . In contrast , convolutions are content-independent as the same filter weights are applied to all inputs regardless of their distinct nature . Given the content-dependent long-range interaction modeling capabilities , our analysis shows that ViTs can flexibly adjust their receptive field to cope with nuisances in data and enhance expressivity of the representations . Our systematic experiments and novel design choices lead to the following interesting findings : • ViTs demonstrate strong robustness against severe occlusions for foreground objects , non-salient background regions and random patch locations , when compared with state-of-the-art CNNs . For instance , with a significant random occlusion of up to 80 % , DeiT [ 3 ] can maintain top-1 accuracy up to ⇠60 % where CNN has zero accuracy , on ImageNet [ 8 ] val . set . • When presented with texture and shape of the same object , CNN models often make decisions based on texture [ 9 ] . In contrast , ViTs perform better than CNNs and comparable to humans on shape recognition . This highlights robustness of ViTs to deal with significant distribution shifts e.g. , recognizing object shapes in less textured data such as paintings . • Compared to CNNs , ViTs show better robustness against other nuisance factors such as spatial patch-level permutations , adversarial perturbations and common natural corruptions ( e.g. , noise , blur , contrast and pixelation artefacts ) . However , similar to CNNs [ 10 ] , a shape-focused training process renders them vulnerable against adversarial attacks and common corruptions . • Apart from their promising robustness properties , off-the-shelf ViT features from ImageNet pretrained models generalize exceptionally well to new domains e.g. , few-shot learning , finegrained recognition , scene categorization and long-tail classification settings . In addition to our extensive experimental analysis and new findings , we introduce several novel design choices to highlight the strong potential of ViTs . To this end , we propose an architectural modification to DeiT to encode shape-information via a dedicated token that demonstrates how seemingly contradictory cues can be modeled with distinct tokens within the same architecture , leading to favorable implications such as automated segmentation without pixel-level supervision . Moreover , our off-the-shelf feature transfer approach utilizes an ensemble of representations derived from a single architecture to obtain state-of-the-art generalization with a pre-trained ViT ( Fig . 1 ) . 2 Related Work . CNNs have shown state-of-the-art performance in independent and identically distributed ( i.i.d ) settings but remain highly sensitive to distributional shifts ; adversarial noise [ 11 , 12 ] , common image corruptions [ 13 ] , and domain shifts ( e.g. , RGB to sketches ) [ 14 ] . It is natural to ask if ViT , that processes inputs based on self-attention , offers any advantages in comparison to CNN . Shao et al . [ 15 ] analyze ViTs against adversarial noise and show ViTs are more robust to high frequency changes . Similarly , Bhojanapalli et al . [ 16 ] study ViT against spatial perturbations [ 15 ] and its robustness to removal of any single layer . Since ViT processes image patches , we focus on their robustness against patch masking , localized adversarial patches [ 17 ] and common natural corruptions . A concurrent work from Paul and Chen [ 18 ] also develops similar insights on robustness of ViTs but with a somewhat different set of experiments . Geirhos et al . [ 9 ] provide evidence that CNNs mainly exploit texture to make a decision and give less importance to global shape . This is further confirmed by CNN ability to only use local features [ 19 ] . Recently , [ 20 ] quantifies mutual information [ 21 ] between shape and texture features . Our analysis indicates that large ViT models have less texture bias and give relatively higher emphasis to shape information . ViT ’ s shape-bias approaches human-level performance when directly trained on stylized ImageNet [ 9 ] . Our findings are consistent with a concurrent recent work that demonstrates the importance of this trend on human behavioural understanding and bridging the gap between human and machine vision [ 22 ] . A recent work [ 23 ] shows that self-supervised ViT can automatically segment foreground objects . In comparison , we show how shape-focused learning can impart similar capability in the image-level supervised ViT models , without any pixel-level supervision . Zeiler et al . [ 24 ] introduce a method to visualize CNN features at different layers and study the performance of off-the-shelf features . In a similar spirit , we study the generalization of off-the-shelf features of ViT in comparison to CNN . Receptive field is an indication of network ’ s ability to model long range dependencies . The receptive field of Transformer based models covers the entire input space , a property that resembles handcrafted features [ 25 ] , but ViTs have higher representative capacity . This allows ViT to model global context and preserve the structural information compared to CNN [ 26 ] . This work is an effort to demonstrate the effectiveness of flexible receptive field and content-based context modeling in ViTs towards robustness and generalization of the learned features . 3 Intriguing Properties of Vision Transformers . 3.1 Are Vision Transformers Robust to Occlusions ? . The receptive field of a ViT spans over the entire image and it models the interaction between the sequence of image patches using self-attention [ 26 , 27 ] . We study whether ViTs perform robustly in occluded scenarios , where some or most of the image content is missing . Occlusion Modeling : Consider a network f , that processes an input image x to predict a label y , where x is represented as a patch sequence with N elements , i.e. , x = { xi } Ni=1 [ 2 ] . While there can be multiple ways to define occlusion , we adopt a simple masking strategy , where we select a subset of the total image patches , M < N , and set pixel values of these patches to zero to create an occluded image , x0 . We refer to this approach as PatchDrop . The objective is then to observe robustness such that f ( x0 ) argmax = y . We experiment with three variants of our occlusion approach , ( a ) Random PatchDrop , ( b ) Salient ( foreground ) PatchDrop , and ( c ) Non-salient ( background ) PatchDrop . Random PatchDrop : A subset of M patches is randomly selected and dropped ( Fig . 2 ) . Several recent Vision Transformers [ 2 , 3 , 4 ] divide an image into 196 patches belonging to a 14x14 spatial grid ; i.e . an image of size 224⇥224⇥3 is split into 196 patches , each of size 16⇥16⇥3 . As an example , dropping 100 such patches from the input is equivalent to losing 51 % of the image content . Salient ( foreground ) PatchDrop : Not all pixels have the same importance for vision tasks . Thus , it is important to study the robustness of ViTs against occlusions of highly salient regions . We leverage a self-supervised ViT model DINO [ 23 ] that is shown to effectively segment salient objects . In particular , the spatial positions of information flowing into the final feature vector ( class token ) within the last attention block are exploited to locate the salient pixels . This allows to control the amount of salient information captured within the selected pixels by thresholding the quantity of attention flow . We select the subset of patches containing the top Q % of foreground information ( deterministic for fixed Q ) and drop them . Note that this Q % does not always correspond to the pixel percentage , e.g. , 50 % of the foreground information of an image may be contained within only 10 % of its pixels . Non-salient ( background ) PatchDrop : The least salient regions of the image are selected following the same approach as above , using [ 23 ] . The patches containing the lowest Q % of foreground information are selected and dropped here . Note this does not always correspond to the pixel percentage , e.g. , 80 % of the pixels may only contain 20 % of the non-salient information for an image . Robust Performance of Transformers Against Occlusions : We consider visual recognition task with models pretrained on ImageNet [ 2 ] . The effect of occlusion is studied on the validation set ( 50k images ) . We define information loss ( IL ) as the ratio of dropped and total patches ( M / N ) . IL is varied to obtain a range of occlusion levels for each PatchDrop methodology . The results ( Top-1 % ) reported in Fig . 3 show significantly robust performance of ViT models against CNNs . In the case of random PatchDrop , we report the mean of accuracy across 5 runs . For Salient and Non-Salient Patchdrop , we report the accuracy values over a single run , since the occlusion mask is deterministic . CNNs perform poorly when 50 % of image information is randomly dropped . For example , ResNet50 ( 23 Million parameters ) achieves 0.1 % accuracy in comparison to DeiT-S ( 22 Million parameters ) which obtains 70 % accuracy when 50 % of the image content is removed . An extreme example can be observed when 90 % of the image information is randomly masked but Deit-B still exhibits 37 % accuracy . This finding is consistent among different ViT architectures [ 2 , 3 , 4 ] . Similarly , ViTs show significant robustness to the foreground ( salient ) and background ( non-salient ) content removal . See Appendix A , B , C , D , and E for further results on robustness analysis . ViT Representations are Robust against Information Loss : In order to better understand model behavior against such occlusions , we visualize the attention ( Fig . 4 ) from each head of different layers . While initial layers attend to all areas , deeper layers tend to focus more on the leftover information in non-occluded regions of an image . We then study if such changes from initial to deeper layers lead to token invariance against occlusion which is important for classification . We measure the correlation coefficient between features/tokens of original and occluded images by using corr ( u , v ) = P i ûiv̂i n , where ûi = ui E [ ui ] ( ui ) , E [ · ] and ( · ) are mean and standard deviation operations [ 29 ] . In our case , random variables u and v refer to the feature maps for an original and occluded image defined over the entire ImageNet validation set . In the case of ResNet50 , we consider features before the logit layer and for ViT models , class tokens are extracted from the last transformer block . Class tokens from transformers are significantly more robust and do not suffer much information loss as compared to ResNet50 features ( Table 1 ) . Furthermore , we visualize the correlation coefficient across the 12 selected superclasses within ImageNet hierarchy and note that the trend holds across different class types , even for relatively small object types such as insects , food items and birds ( Fig . 5 ) . See Appendix F for attention visualizations and G for the qualitative results . Given the intriguing robustness of transformer models due to dynamic receptive fields and discriminability preserving behaviour of the learned tokens , an ensuing question is whether the learned representations in ViTs are biased towards texture or not . One can expect a biased model focusing only on texture to still perform well when the spatial structure for an object is partially lost . Figure 5 : Correlation b/w features/final tokens of original and occluded images for 50 % Random Drop . Results are averaged across classes for each superclass .

Summary: The paper systematically studies ViT’s performance on occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. Experiments show that ViT models are relatively more robust to occlusions, perturbations and domain shifts, and less sensitive to local textures. Off-the-shelf features from ViT models yield high accuracy on several downstream classification datasets.