Intriguing Properties of Vision Transformers
Authors: Muzammal Naseer
Kanchana Ranasinghe
Salman Khan
Munawar Hayat
Fahad Khan
Ming-Hsuan Yang
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: Vision Transformers
Auto-segmentation
Off-the-shelf-features
Robustness
Shape-Modeling

Abstract: Vision transformers (ViT) have demonstrated impressive performance across numerous machine vision tasks. These models are based on multi-head self-attention mechanisms that can flexibly attend to a sequence of image patches to encode contextual cues. An important question is how such flexibility (in attending image-wide context conditioned on a given patch) can facilitate handling nuisances in natural images e.g., severe occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. We systematically study this question via an extensive set of experiments encompassing three ViT families and provide comparisons with a high-performing convolutional neural network (CNN). We show and analyze the following intriguing properties of ViT: (a)Transformers are highly robust to severe occlusions, perturbations and domain shifts, e.g., retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content. (b)The robustness towards occlusions is not due to texture bias, instead we show that ViTs are significantly less biased towards local textures, compared to CNNs. When properly trained to encode shape-based features, ViTs demonstrate shape recognition capability comparable to that of human visual system, previously unmatched in the literature. (c)Using ViTs to encode shape representation leads to an interesting consequence of accurate semantic segmentation without pixel-level supervision. (d)Off-the-shelf features from a single ViT model can be combined to create a feature ensemble,  leading to high accuracy rates across a range of classification datasets in both traditional and few-shot learning paradigms.  We show effective features of ViTs are due to flexible and dynamic receptive fields possible via self-attention mechanisms. Our code will be publicly released.

Intriguing Properties of Vision Transformers
Muzammal Naseer Kanchana Ranasinghe Salman Khan
Munawar Hayat Fahad Shahbaz Khan Ming-Hsuan Yang
Australian National University. "Mohamed bin Zayed University AI, Stony Brook University, Monash University, Linkööingg University, UUniversity of California, Merced, "Ynseei University, V "Google Research muzammal. naseer @anu.. edu, au
Abstract
Vision transformers (ViT) have demonstrated impressive performance across nu- merous machine vision tasks. These models are based on multi-head self self-attention mechanisms that can flexibly attend sequence of image patches to encode con- textual cues. An important question is how such flexibility (in attending image-wide context conditioned on a given patch) can facilitate handling nuisances natural images e.g., severe occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. We systematically study this question via an extensive set of experiments encompassing three ViT families and provide comparisons with a high high-performing convolutional neural network (CNN). We show and analyze the following intriguing properties of VIT: (a) Transformers are highly robust to severe occlusions, perturbations and domain shifts, e.g. retain as high as 60% top-1 accuracy ImageNet even after randomly occluding 80% of the image content. (b) The robustness towards occlusions not due to texture bias, instead we show that ViTs significantly less biased towards local textures, compared CNNs. When properly trained to encode chape-based features, ViTs demonstrate shape recognition capability comparable to that of human visual system, previously unmatched in the literature. (c) Using ViTs to encode shape representation leads to interesting consequence of accurate semantic segmentation without pixel-level supervision. (d) Off-the-shed features from a single ViT model can be combined create feature ensemble, leading to high accuracy rates across a range of classification datasets in both traditional and few-shot learning paradigms. We show effective features of ViTs are due to flexible and dynamic receptive fields possible via self-attention mechanisms. Code: pettss//giie 1ort/1o5X/11...
Js11X.
Introduction
As visual transformers (ViT) attract more interest [1], it becomes highly pertinent study character- isties of their learned representations. Specifically, from the perspective of safety-critical applications such autonomous cars, robots and healthcare; the learned representations need to be robust and generalizable. this paper, we compare the performance transformers with convolutional neural networks (CNNs) for handling nuisances (e.g., occlusions, distributional shifts, adversarial and natural perturbations) and generalization across different data distributions. Our in-depth analysis is based on three transformer familícs, VIT [2], DeiT [3] and T2T [4] across fifteen vision datasets. For brevity, we refer to all the transformer families as VIT, unless otherwise mentioned.
We are intrigued by the fundamental differences in the operation of convolution and self-attention, that have not been extensively explored in the context of robustness and generalization. While convolutions excel at learning local interactions between elements in the input domain (e.g., edges and contour information), self-attention has been shown to effectively learn global interactions (e.g..
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

(a) Occlusion
Discribution Stift (c) Adversarial Patch
Permutation
Suge--t (f) Off-the-shclff
Figure 1: We show intriguing properties of VIT including impressive robustness to (a) severe occlusions, (b) distributional shifts (e.g. stylization remove texture cues), (c) adversarial perturbations, and (d) patch permutations. Furthermore, our VIT models trained to focus on shape cues can segment foregrounds without any pixel-level supervision (e). Finally, off-the-shelf fcatures from ViT models generalize better than CNNs (f).
relations between distant object parts) [5, 6]. Given query embedding, self-attention finds its interactions with the other embeddings in the sequence, thereby conditioning on the local content while modeling global relationships [7]. In contrast, convolutions are content-independent the same filter weights are applied to all inputs regardless of their distinct nature. Given the content onteent-dependntt long-range interaction modeling capabilities, our analysis shows that ViTs can flexibly adjust their receptive field to cope with nuisances in data and enhance expressivity of the representations.
Our systematic experiments and novel design choices lead the following interesting findings:
ViTs demonstrate strong robustness against severe occlusions for foreground objects, non-salient background regions and random patch locations, when compared with state-of-the-an -frt--rrtt CNNs. For instance, with significant random occlusion of up to 80%, DeiT [3] can maintain top-1 accuracy up to ~60% where CNN has zero accuracy, on ImageNet [8] val. set.
When presented with texture and shape of the same object, CNN models often make decisions based on texture [9]. In contrast, ViTs perform better than CNNs and comparable humans on shape recognition. This highlights robustness of ViTs to deal with significant distribution shifts e.g., recognizing object shapes in less textured data such as paintings.
Compared to CNNs, ViTs show better robustness against other nuisance factors such as spatial patch-level permutations, adversarial perturbations and common natural corruptions (e.g., noise, blur, contrast and pixelation artefacts). However, similar to CNNs [10], a shape- training process renders them vulnerable against adversarial attacks and common corruptions. Apart from their promising robustness properties, off-the-shelf ViT features from ImageNet pretrained models generalize exceptionally well to new domains e.g., few-shot learning, fine- grained recognition, scene categorization and long-tail classification settings.
In addition to our extensive experimental analysis and new findings, introduce several novel design choices to highlight the strong potential of ViTs. To this end, we propose an architectural modification DeiT to encode chape-information via dedicated token that demonstrates how seemingly contradictory cues can be modeled with distinct tokens within the same architecture, leading to favorable implications such automated segmentation without pixel-level supervision. Moreover, our off-the-shelf feature transfer approach utilizes an ensemble of representations derived from single architecture obtain state-of-the-ar generalization with pre-trained ViT (Fig. 1).
2 Related Work
CNNs have shown state-of-the-art performance in independent and identically distributed (i.i.d) settings but remain highly sensitive to distributional shifts; adversarial noise [11, 12], common image corruptions [13], and domain shifts (e.g.. RGB to sketches) [14].
natural to ask if VIT, that processes inputs based self-attention, offers any advantages in comparison to CNN. Shao et al. [15] analyze ViTs against adversarial noise and show ViTs are more robust to high frequency changes. Similarly, Bhojanapalli et al. [16] study ViT against spatial perturbations [15] and its robustness to removal of any single layer. Since ViT processes image patches, we focus on their robustness against patch masking, localized adversarial patches [17] and common natural corruptions. concurrent work from Paul and Chen [18] also develops similar insights on robustness of ViTs but with somewhat different of experiments.
Geirhos el al. [9] provide evidence that CNNs mainly exploit texture to make a decision and give less importance to global shape. This is further confirmed by CNN ability to only use local features

---

[19]. Recently, [20] quantifies mutual information [21] between shape and texture features. Our analysis indicates that large ViT models have less texture bias and give relatively higher emphasis to shape information. ViT's shape-bias approaches human-level performance when directly trained on stylized ImageNet [9]. Our findings are consistent with a concurrent recent work that demonstrates the importance of this trend on human behavioural understanding and bridging the gap between human and machine vision [22]. A recent work [23] shows that self-supervised ViT can automatically segment foreground objects. In comparison, we show how shape-focused learning can impart similar capability in the image-level supervised ViT models, without any pixel-level supervision.
Zeiler er al. [24] introduce a method to visualize CNN features at different layers and study the performance of aff-the-shelf features. In a similar spirit, we study the generalization of off fff-the-shelff features of ViT in comparison to CNN. Receptive field is an indication of network's 's ability to model long range dependencies. The receptive field of Transformer based models covers the entire input space, property that resembles handcrafted features [25], but ViTs have higher representative capacity. This allows ViT to model global context and preserve the structural information compared to CNN [26]. This work is effort to demonstrate the effectiveness of flexible receptive field and content-based context modeling in ViTs towards robustness and generalization the learned features.
3 Intriguing Properties of Vision Transformers
3.1 Are Vision Transformers Robust to Occlusions?
The receptive field of a ViT spans over the entire image and it models the interaction between the sequence of image patches using self-attention [26, 27]. We study whether ViTs perform robustly in occluded scenarios, where some or most of the image content missing.
Consider a network f. that processes an input image x to predict a label y. where x is represented patch sequence with N elements, i.e., x {x; [2]. While there can be multiple ways to define occlusion, we adopt a simple masking strategy, where we select a subset of the total image patches, M N, and set pixel values of these patches image, x'. We refer to this approach as PatchDrop. The objective is then to observe robustness such that f(x') (ra')argmax y. We experiment with three variants of our occlusion approach, (a) Random PatchDrop, (b) Salient (foreground) PatchDrop, and (c) Non-salient (background) PatchDrop.
Ocelusion Modeling:
create occluded
Random PatchDrop: A subset of M patches is randomly selected and dropped (Fig. 2). Several recent Vision Transformers [2. 3, 4] divide an image into 196 patches belonging to a 14x 14 spatial grid; Le. an image of size 224x 244x224x3 3 is split into 196 patches, each of size 16x 16x3. As example, dropping 100 such patches from the input is equivalent to losing 51% of the image content.
Salient (foreground) PatchDrop: Not all pixels have the same importance for vision tasks. Thus, it is important to study the robustness of VIT's against occlusions of highly salient regions. We leverage supervisd ViT model DINO [23] that is shown to effectively segment salient objects. In particular, the spatial positions of information flowing into the final feature vector (class token) within the last attention block are exploited to locate the salient pixels. This allows to control the amount of salient information captured within the selected pixels by thresholding the quantity of attention flow.
We select the subset of patches containing the top Q% of foreground information (deterministic for fixed Q) and drop them. Note that this Q% does not always correspond to the pixel percentage. e.g.. 50% of the foreground information of an image may be contained within only 10% of its pixels.
Non-salient (buckground) PatchDrop: The least salient regions of the image are selected following the same approach as above, using [23). The patches containing the lowest Q% of foreground information are selected and dropped here. Note this does not always correspond to the pixel percentage, e.g., 80% of the pixels may only contain 20% of the non-salient information for an image.
Figure 2: An example image with its oc-
Original Image
Ramdom PatchDrop
Salieni PatchDrup
Non: Salien PaichDrop
cluded versions (Random, Salient and Non-
Salient). The occluded images are cor- rectly classified by Deit-S [3] but mis- classified by ResNet50 [28]. Pixel values in occluded (black) regions set to zero.
3

---

accunecy 183)
Bandom PatchOrop
Random PatchGrop
Becunery (N)
Random PatchGrop
Random PetchOrop
Accuacy Top-2
8.
Redhviss
burt
VITL
Squaerster
Accumnyy
belli
vita
Yar
%
X
X
Information Loan (%)
Information Loan (%)
Infermation Loan (N)
Information Loan (S4)
Selient PetchOrop
Selient PatchDrop
Selient PutchOrop
Salient PutchOrop
Recuweyy Topa (N)
Arouner Top (N)
Becures Topa (N)
Accuant Top- (S))
Deraetet123
Delto
VIEa VITS
TPT 24
Squeesaltet
VomtE an
Peolecho
Rapleeto
Information Loon (1b)
Information Loos (%)
Information Less (%)
Information (%)
Salient PatchDrop
Salient PatchDrop
PatchDrop
Salient PatchDrop
Rcouracy Tap (M)
AOCHAN Tapa (6)
Accurmoc Top (%)
Accuracy topa
ura VIT-S Restwine
R
Deltz
vplzz Jn
8
Rechwito
information Lown
Information Lows (%)
Information Loan
Information Losa (S)
Robustness against object occlusion in images is studied under three PatchDrop settings (see Sec 3.1). (left) We study the robustness of CNN models occlusions. and identify ResNet50 strong baseline. (mid-left) We compare the DeiT model family against ResNet50 exhibiting their superior robustness object occlusion. (mid-right) Comparison against ViT model family. (right) Comparison against T2T model family.
Figure 3:
Robust Performance of Transformers Against Occlusions: We consider visual recognition task with models pretrained on ImageNet [2]. The effect of occlusion studied on the validation set (50k images). We define information loss (IL) as the ratio of dropped and total patches (M / /N.. IL is varied to obtain a range of occlusion levels for each PatchDrop methodology. The results (Top-1 %) reported in Fig. 3 show significantly robust performance of ViT models against CNNs. In the case of random PatchDrop, we report the mean of accuracy across runs. For Salient and Non-Salient Patchdrop, we report the accuracy values over single run, since the occlusion mask is deterministic. CNNs perform poorly when 50% of image information is randomly dropped. For example, ResNet50 (23 Million parameters) achieves 0.1 % accuracy in comparison to DeiT-S (22 Million parameters) which obtains 70% accuracy when 50% of the image content is removed. An extreme example can be observed when 90% of the image information is randomly masked but Deit-B still exhibits 37% accuracy. This finding is consistent among different ViT architectures [2, 3, 4]. Similarly, ViT's show significant robustness to the foreground (salient) and background (non-salient) content removal. See Appendix A, B, C, D, and E for further results on robustness analysis.
VIT Representations are Robust against Information Loss: In order to better understand model behavior against such occlusions, we visualize the attention (Fig. 4) from each head of different layers. While initial layers attend to all areas, deeper layers tend to focus more the leftover information in non-occluded regions of an image. We then study if such changes from initial to deeper layers lead to token invariance against occlusion which is important for classification. We measure the correlation coefficient between features/tokens of original and occluded images by using corr (u, v) = Σ, dipq
-==E|us], ET} and (-)) are mean and standard deviation operations [29]. In our case, random variables u and refer to the feature maps for an original and occluded image defined over the entire ImageNet validation set. In the case of ResNet50, we consider features before the logit layer and for ViT models, class tokens are extracted from the last transformer block. Class tokens from transformers are significantly more robust and do not suffer much information loss as compared to ResNet50 features (Table 1). Furthermore, we visualize the correlation coefficient across the 12 selected superclasses within ImageNet hierarchy and note that the trend holds across different class types, even for relatively small object types such as insects, food items and binds (Fig. 5). See Appendix F for attention visualizations and G for the qualitative results.
where vi a(u.)
Given the intriguing robustness of transformer models due to dynamic receptive fields and discrim- inability preserving behaviour of the learned tokens, an ensuing question is whether the learned representations in ViTs are biased towards texture or not. One can expect biased model focusing only texture to still perform well when the spatial structure for an object is partially lost.

---

Luves 01
Layas
Luver
Lawes
Lowu
Lowar
Figure 4: Attention maps (averaged over the entire ImageNet val. sct) relevant cach head in multiple layers an ImageNet pre-trained DeiT-B model. images are occluded (Random PatchDrop) with the same mask (bottom right). Observe how later layers clearly attend to non-occluded regions of images to make decision, evidence of the model's 'ss highly dynamic receptive field.
Model
Correlation Coefficient: Random PatchDrop 50% Dropped 75% Dropped
25% Dropped
Cerrelation
ResNet50
0.324:0.16
0.13:0.11
0.07+0.09
VI14
Inl-3
TaT-S
0.83+-0.08
0.67=00.12
0.46:t0.1
DerTo
VIT-L
0.92:10.06
0,81:10.13
0.50:1021
Doit-B
0.90:10.06
0.77+10.10
0.56+00115
T2T-24
0.80:10.10
0.60:10.15
0.3110.17
Senettore
Chitting
Burthe
Food
Forasture
Priouate
Table 1: Correlation coefficient b/w features/final class
Figure S: Correlation b/w features/final tokens of orig- inal and occluded images for 50% Random Drop. Re- sults averaged across classes for each superclass.
tokens of original and occluded images for Random PatchDrop. Averaged across the ImageNet val. set.
3.2 Shape vs. Texture: Can Transformer Model Both Characteristics?
Geirhos al. [9] study shape vs. texture hypothesis and propose training framework enhance shape-bias in CNNs. We first carry out similar analysis and show that VIT models preform with shape-bias much stronger than that of CNN, and comparably the ability of human visual system in recognizing shapes. However, this approach results in significant drop in accuracy on the natural images. To address this issue, we introduce a shape token into the transformer architecture that learns to focus on shapes, thereby modeling both shape and texture related features within the same architecture using distinct set of tokens. As such, we distill the shape information from a pretrained CNN model with high shape-bias [9]. Our distillation approach makes a balanced trade-off between high classification accuracy and strong shape- biis compared the original ViT model.
We outline both approaches below. Note that the measure introduced in [9] used to quantify shape-bias within ViT models and compare against their CNN counterparts.
Training without Local Texture: In this approach, we first remove local texture cues from the training data by creating stylized version of ImageNet [9] named SIN. We then train tiny and small DeiT models [3] on this dataset. Typically, ViTs use heavy data augmentations during training [3]. However, learning with SIN is difficult task due to less texture details and applying further augmentations on stylized samples distorts shape information and makes the training unstable. Thus, train models SIN without applying any augmentation, label smoothing mixup.
We note that ViTs trained on ImageNet exhibit higher shape-bias comparison to similar capacity CNN models e.g,, DeiT-S (22-Million params) performs better than ResNer50 (23-Million params) (Fig. 6, right plot). In contrast, the SIN trained ViTs consistently perform better than CNNs. Interestingly. DeīT-S [3] reaches human-level performance when trained on SIN (Fig, 6, left plot).
5

---

Fraction 'hape' decisions
[0]
Shape Dies
0.4
I (4]
RevWer50
DwrEy
DesEs
Delra
T2T-7
T2E.14
T2R.24
WTTS
Bestetso SIN
WITL
In
DeITY-
DelF' SIN
Figure Shape-bias Analysis: Shape- hias is defined as the fraction of correct decisions based on object shape. (Left) Plot shows shape- texture tradeoff for CNN, ViT and Humans across different object classes. (Right) class- mean shape-bias comparison. Overall, ViT' perform ter than CNN. The shape bias increases significantly when trained on stylized ImageNet (SIN).
0.5
Fraction of "texture' decisions
Restiet50
DeiT.S
Humans
Model
Distilled
Token Type
ImageNet top-1 (%)
Shape Bias
Table 2: Performance comparison of mod- els trained on SIN. ViT produces dynamic fcatures that can be controlled by auxiliary tokens. "cls' represents the class token. During distillation cls and shape tokens converged vastly different solution us- ing the same features as compared to [3].
DeiT-T-SIN
X
cls
40.5
0.87
DeiT-T-SIN
71.8
0.35
DeiT-T-SIN
shape
63.4
0.44
DeiT-S-SIN DeiT-S-SIN
X
cls
52.5
0.93
cls
75.3
0.39
DeiT-S SIN
shape
0.47
Shape Distillation: Knowledge distillation allows to compress large teacher models into smaller student models [30] as the teacher provides guidance to the student through soft labels. We introduce new shape token and adapt attentive distillation [3] to distill shape knowledge from CNN trained on the SIN dataset (ResNet50-SIN [9]). We observe that ViT features are dynamic in nature and can be controlled by auxiliary tokens to focus on the desired characteristics. This means that single ViT model can exhibit both high shape and texture bias at the same time with separate tokens (Table 2). We achieve more balanced performance for classification as well as shape-bias measure when the shape token is introduced (Fig. 7). To demonstrate that these distinct tokens (for classification and shape) indeed model unique features, we compute cosine similarity (averaged over ImageNet val. set) between class and shape tokens of our distilled models, DeiT-T-SIN and DeiT-S-SIN, which turns out to be 0.35 and 0.68, respectively. This significantly lower than the similarity between class and distillation tokens [3]: 0.96 and 0.94 for DeiT-T and Deit-S, respectively. This confirms hypothesis on modeling distinct features with separate tokens within ViTs, a unique capability that cannot be straightforwardly achieved with CNNs. Further, offers other benefits as explain next.
Shape-biased ViT Offers Automated Object Segmen- tation: Interestingly, training without local texture or with shape distillation allows a ViT concentrate on fore- ground objects in the scene and ignore the background (Table 3, Fig. 8). This offers an automated semantic seg- mentation for an image although the model is never shown pixel-wise object labels. That is, shape-bias can be used self-supervision signals for the ViT model to learn dis- tinct shape-related features that help localize the right foreground object. We note that ViT trained without emphasis on shape does not perform well (Table 3).
Class Token
Class
Head
CE-Loss
Patch T Tokens
VIT Blocks
Class Label
Shape
Shape Head
Token
A
ResNet (trained SIN)
Distilition Loss
Image
Figure 7: Shape Distillation.
The above results show that properly trained ViT models
offer shape-bias nearly as high as the human's ability to recognize shapes. This leads to question positional encoding is the key that helps ViTs achieve high performance under severe occlusions (as can potentially allow later layers to recover the missing information with just a few image patches given their spatial ordering). This possibility is examined next.

---

Model
Distilled Token Type
Jaccard Index
Deil-T-Random
cls
19.6
DeñT-T
cls
32.2
DeiT-T-SIN
cls
29.4
DeiT-T-SIN
cls
40.0
DeiT-T-SIN
shape
42.2
RaTT-S-Random X
cls
22.0
DeiT-$
X
cls
29.2
DeiT-S-SIN
X
cls
37.5
DeiT-S-SIN
cls
42.0
DeiT-S-SIN
shape
42.4
Table 3: We compute the Jaccard similarity between ground truth and masks generated from the attention maps of VIT models (similar to [23] with threshold 0.9) over the PASCAL-VOC12 validation set. Only class level ImageNet labels are used for training these models, Our results indicate that supervised ViTs can be used for automated segmentation and perform closer to the self-supervised method DINO [23].
DATS
DeiTS-SIN
DeiT-S-INN (Disilled)
Figure 8: Segmentation maps from ViTs. Shape distil- lation performs hetter than standard supervised models.
3.3 Does Positional Encoding Preserve the Global Image Context?
Transformers' ability to process long- sequences in parallel using self-attention [27] (instead of sequential design in RNN [31]) is invariant to sequence ordering. For images, the order of patches represents the overall image structure and global composition. Since ViTs operate on a sequence of images patches, changing the order of sequence e.g., shuffling the patches can destroy the image structure. Current ViTs [2, 3, 4, 26] use positional encoding to preserve this context. Here, we analyze if the sequence order modeled by positional encoding allows ViT to excel under occlusion handling. Our analysis suggests that transformers show high permutation invariance to the patch positions, and the effect of positional encoding towards injecting structural information of images to VIT models is limited (Fig. 10). This observation is consistent with the findings in the language domain [32] as described below.
Sensitivity to Spatial Structure: We remove the structural information within images (spatial relationships) illustrated in Fig. by defining a shuffling operation on input image patches. Fig. 10 shows that the DeiT models [3] retain accuracy better than their CNN counterparts when spatial structure of input images is disturbed. crucial for right classification decisions, and the model does not "recover" global image context using the patch sequence information preserved in the positional encodings. Without encoding, the ViT performs reasonably well and achieves better permutation invariance than a ViT using position
14.
Figure 9: illustration of shuffle operation applied images used to eliminate their structural information. (best viewed zoomed-in)
This also indicates that positional encoding is not absolutely
Encoding
Encoding
Effect of No. of Patches
8o 70
70
Accuracy Top-1 (%)
60
50
Accuracy
10
40
DelT-T
DelT-T-no-per no0-pos
100
Bushiut50
DeiT-T
64 Shuffle Grid Size
8
16
32
196
256
16
32
64 196 256
Shuffle
Shuffle Grid Size
Figure 11: DeiT-T [3] trained different number of image
Figure 10: Models trained on 196 image patches. Top-1 (%) accuracy over ImageNet val. set when patches are shuffled. Note the performance peaks when shuffle grid size equal the original number of patches used during training, since equals only changing position of input patch (and not disturbing the patch content).
patches. Reducing patch size de- creases the overall performance but also increases sensitivity to shuffle grid size.

---

Trained with Augmentaatioss
Trained without Augmentation
DeiT-B
DeiT-S
DeiT-T T2T-24
TnT-S
Augmix
ResNet50
ResNet50-SII
DeiT-T-SIN DeiT-S-SIH
48.5
54.6
71.1
49.1
53.1
65.3
76.7
77.3
94.4
84.0
Table 4: mean Corruption Error (mCE) across common corruptions [13] (lower the better). While ViTs have better robustness compared CNNs, training to achieve higher shape-bias makes both CNNs and ViTs vulnerable natural distribution shifts. models trained with augmentations (ViT CNN) have lower mCE in comparison to models trained without augmentations on ImageNet or SIN.
Figure 12: Robustness against adver- sarial patch attack. ViTs even with less parameters exhibit higher ro- bustness than CNN. Models trained ImageNet are more robust than the ones trained on SIN. Results are aver- aged across five runs of patch attack over ImageNet val. set.
Accuracy Top-1 (96
60
DerT-S
BerT.B
Accuracy Top-1 (%)
Daii-S-SIN RosMec50 SIN
60
DeIT-T
ResMee50
Adversarial Patch w.r.t. Image
Adversarial Patch Image
Accuracy Tap-3 (%)
FGSM
(%)
FGSM
Accuracy Top-1 (%)
2
PGD
PGD
-2
Cab bak5-08
Deb
Detd Raseed
Accurscy Top-1
Des559
Fashad3
15
Perturbation budget
Perturbation budget
Perturbation budget
Perturbation
Figure 13:
Robustness against sample specific attacks including single step, FGSM [34], and multi-step, PGD
[35]. ViTs even with less parameters exhibit a higher robustness than CNN. PGD ran for iterations only. Attacks are evaluated under so norm and e represents the perturbation budget by which each pixel is changed in the input image. Results are reported over the ImageNet val. set.
encoding (Fig. 10). Finally, when the patch size is varied during ViT training, the permutation invariance property is also degraded along with the accuracy on unshuffled natural images (Fig. 11). Overall, we attribute the permutation invariance performance of ViT's to their dynamic receptive field that depends on the input patch and can adjust attention with the other sequence elements such that moderately shuffling the elements does not degrade the performance significantly.
The above analysis shows that just like the texture-bias hypothesis does not apply ViTs, the dependence on positional encodings to perform well under occlusions is also incorrect. This leads us the conclude that ViTs robustness due to its flexible and dynamic receptive field (see Fig. 4) which depends on the content of an input image. We now delve further deep into the robustness of ViT, and study its performance under adversarial perturbations and common corruptions.
3.4 Robustness of Vision Transformers to Adversarial and Natural Perturbations
After analyzing the ability of ViTs to encode shape information (Sec. 3.2), one ensuing question is: Does higher shape-bias help achieve better robusiness? ? Table 4, we investigate this by calculating mean corruption error (mCE) [13] on a variety of synthetic common corruptions (e.g., rain, fog, snow and noise). A ViT with similar parameters as CNN (e.g.. DeiT-S) more robust to image corruptions than ResNet50 trained with augmentations (Augmix [33]). Interestingly, CNNs and ViTs trained without augmentations on ImageNet or SIN are more vulnerable to corruptions. These findings are consistent with [10]. and suggest that augmentations improve robustness against common corruptions.
We observe similar performance against untargeted, universal adversarial patch attack [17] and sample specific attacks including single step, fast gradient sign method (FGSM) [34], and multi-step projected gradient attack known as PGD [35]. Adversarial patch attack [17] is unbounded that can change pixel values at certain location in the input image by any amount, while sample specific attacks [34, 35] are bounded by loc. norm with a perturbation budget e, where represents the amount by which each pixel is changed in the entire image. ViTs and CNN trained on SIN are significantly
8

---

Block
2lock2
Bleck
Table 5: Ablative Study off off-the-shelf fea- ture transfer on three datasets using ImageNet pretrained DeiT-S [3]. A linear classifier is learned only concatenation of class to- kens the combination of class and averaged patch tokens various blocks. note class token from blocks 9-12 are most discrimina- tive (Fig, 15) and have the highest transfer- Accurscy Top (N0)
DelT-T DelT-S
VIT
VIP
VIT
DeiT-B
Classiter
Stand
Clessfer
Shared
2
10
12
VIT Blocks
Figure 14: A single ViT model can provide features en- semble since class token from each block can be processed by the classifier independently. This allows us to identify the most discriminative tokens useful for transfer learning.
Figure 15: Top-1 (%) for ImageNet val. set for class tokens produced by each ViT block. Class to- kens from last few layers exhibit highest perfor- mance indicating the most discriminative tokens.
Blocks
Class
Putch
CUB
Flowers
iNaturalist [39]
Tokens
Tokens
[37]
[38]
Only 12th (last block)
68.16
82.58
38.28
70.66
86.58
41.22
From 1d 12
/
x
72.90
91.38
44,03
73.16
91.27
43.33
9th 12
73.58
90.00
45.15
73.37
90.33
45.12
ability in terms of Top-1 (%) accuracy.
more vulnerable to adversarial attack than models trained on ImageNet (Figs. 12 and 13), due to the shape-bias vs. robustness trade-off [10].
Given the strong robustness properties of VIT as well as their representation capability in terms of shape-bias, automated segmentation and flexible receptive field, we analyze their utility as an off-the-shelf feature extractor to replace CNNs as the default feature extraction mechanism [36].
3.5 Effective Off-the-shelf' Tokens for Vision Transformer
unique characteristic of ViT models that each block within the model generates class token which can be processed by the classification head separately (Fig. 14). This allows us to measure the discriminative ability of each individual block of an ImageNet pre-trained VIT as shown in Fig. 15. Class tokens generated by the deeper blocks are more discriminative and we use this insight to identify an effective ensemble of blocks whose tokens have the best downstream transferability.
Transfer Methodology: As illustrated in Fig. 15, we analyze the block-wise classification accuracy of DeiT models and determine the discriminative information is captured within the class tokens of the last few blocks. As such, we conduct an ablation study for off-the-shelf transfer learning on fine-grained classification dataset CUB [37], Flowers [38] and large scale iNaturalist [39] using DeiT-S [3] as reported in Table 5. Here, we concatenate the class tokens (optionally combined with average patch tokens) from different blocks and train linear classifier to transfer the features to downstream tasks. Note that patch token is generated by averaging along the patch dimension. The scheme that concatenate class tokens from the last four blocks shows the best transfer learning performance. We refer to this transfer methodology as DeïT-S (ensemble). Concatenation of both class and averaged patch tokens from all blocks helps achieve similar performance compared to the tokens from the last four blocks but requires significantly large parameters to train. We find some exception to this on the Flower dataset [38] where using class tokens from all blocks have relatively better improvement (only 1. 1.2%), compared the class tokens from the last four blocks (Table 5). However, concatenating tokens from all blocks also increases the number of parameters e.g., transfer to Flowers from all tokens has 3 times more learnable parameters than using only the last four tokens. We conduct further experimentation with DeiT-S (ensemble) across a broader range of tasks validate hypothesis. We further compare against pre-trained ResNet50 baseline, by using features before the logit layer.
Visual Classification We analyze the transferability of off-the-shelf features across several datasets including Aircraft [40], CUB [37], DTD [41], GTSRB [42], Fungi [43], Places365 [44] and iNaturalist [39]. These datasets are developed for fine-graines recognition, texture classification,

---

Accuracy Top-1 (%%
RasHet50 DaTTS (ensemble))
Datty B00
aecuracy
jessersdde
40-
20-
Oeniigiet
Airorftt
cua
DTO
pungt
Plaurer
aTSRa
Ms coco
MAUST
CIAB-10
CIPA-1900 100
Fungi
VIT features transfer better than CNNs. explore transferability learned repre- sentations using generic classification as well as few-shot classification for out-of-domain tasks. In the case of classification (left), the ImageNet pre-trained ViT' transfer better than their CNN counterparts across tasks. In the case of few-shot learning (right), ImageNet pre-trained ViTs perform better average.
Figure 16:
traffic sign recognition, species classification and scene recognition with 100, 200, 47, 43, 1394, 365 and 1010 classes respectively. We train a linear classifier top of the extracted features over the train split of each dataset, and evaluate the performance on their respective test splits. The ViT features show clear improvements over the CNN baseline (Fig. 16). We note that DeiT-T, which requires about 5 times fewer parameters than ResNet50, performs better among all datasets. Furthermore, the model with the proposed ensemble strategy achieves the best results across all datasets.
We consider meta-dataset [45] designed as a large-scale few-shot learning (FSI.) benchmark containing diverse set of datasets from multiple domains. This includes letters of alphabets, hand-drawn sketches, images of textures, and fine-grained classes making a challenging dataset involving domain adaption requirement well. We follow the standard setting of training on ImageNet and testing on all other datasets which are considered as the downstream tasks.
Few-Shot Learning:
In our experiments, we use network pre- pre-trained for classification on ImageNet dataset to extract features. For each downstream dataset, under the FSL setting, a support set of labelled images is available for every test query. We use the extracted features to learn a linear classifier over the support set for each query (similar to [46]), and evaluate using the standard FSL. protocol defined [45]. This evaluation involves varying number of shots specific for each downstream dataset. On average, the ViT features transfer better across these diverse domains (Fig. 16) in comparison to the CNN baseline. Furthermore, we note that the transfer performance of VIT is further boosted using the proposed ensemble strategy. We also highlight the improvement in QuickDraw, a dataset containing hand-drawn sketches, which aligns with our findings on improved shape-bias of ViT models in contrast to CNN models (see Sec. 3.2 for elaborate discussion).
Discussion and Conclusions
In this paper, we analyze intriguing properties of ViTs in terms of robustness and generalizability. We test with variety of ViT models fifteen vision datasets. All the models trained 4 V100 GPUs. We demonstrate favorable merits of ViTs over CNNs for occlusion handling, robustness to distributional shifts and patch permutations, automatic segmentation without pixel supervision, and robustness against adversarial patches, sample specific adversarial attacks and common corruptions. Moreover, we demonstrate strong transferability of off-the-shelf ViT features a number of down- stream tasks with the proposed feature ensemble from a single ViT model. An interesting future research direction is explore how the diverse range of cues modeled within a single ViT using separate tokens can be effectively combined to complement each other. Similarly, we found that ViTs nnto-seemeenation property stems from their ability encode shape information. We believe that integrating our approach and DINO [23] is worth exploring in the future. To highlight few open research questions: a) Can suff-superiision on stylized ImageNet (SIN) improve segmentation ability of DINO?, and b) Can modified DINO training scheme with texture (IN) based local views and shape (SIN) based global views enhance (and generalize) seto-seementttion capability?
Our current set of experiments are based on ImageNet (ILSVRC'12) 12) pre-trained ViTs, which pose the risk of reflecting potential biases in the learned representations. The data is mostly Western, and encodes several gender/ethnicity stereotypes with under-representation of certain groups [47]. This version of the ImageNet also poses privacy risks. due the unblurred human faces. In future, we will use a recent ImageNet version which addresses the above issues [48].
10

---

Acknowledgments
This work is supported in part by NSF CAREER grant 1149783, and VR starting grant (2016-05543). M. Hayat is supported by Australian Research Council DECRA fellowship DE200101100
References
[1] Salman Khan, Muzammal Nascer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. arXiv preprint arXiv:2101.01 169, 2021.
[2] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth Transformers for image recognition at scale. arXiv preprint arXiv:2010.11999 2020. 1, 4,
16x16 words:
[3] Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.1287,, 2020. 1, 2, 3,4, 6,
[4] Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokens-to-token vit: Training vision transformers from scratch imagenet. arXiv preprint arXiv:2101111866 2021. 4.7.
[5] Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, and Jonathon Shlens. Stand- alone self attention vision models. arXiv preprint arXiv:I906.05000 2019.
[6] Han Hu, Zheng Zhang, Zhenda Xie, and Stephen Lin. Local relation networks for image recognition. In IEEE Conference on Computer Vision and Pottern Recognition, pages 4444-473, 2019.
[7] Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, and Jonathon Shlens. Scaling local self-attention for parameter efficient visual backbones. arXiv preprint 103.v:2103.1273), 2021.
[8] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115(3):211222) 12222,22222222 2015.
[9] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet- trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv: 12231, 2018. 2, 3, 5, 6, 18
[10] Chaithanya Kumar Mummadi, Ranjitha Subramaniam, Robin Hutmacher, Julien Vitay, Volker Fischer, and Jan Hendrik Metzen. Does enhanced shape bias improve neural network robustness to common corruptions? In International Conference on Learning Representations, 2021. 2, 8,9
[11] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312.6199, 2013. 2
[12] Muzammal Naseer, Salman H Khan, Harris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. Advances in Neural Information Processing Systems, 2019. 2
[13] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions
and perturbations. arXiv preprint arXiv:/903 /2261, 2019. 2,8 8
[14] Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In IEEE Conference on Computer Vision and Pattern Recognition, pages 5542-5550, 2017.
[15] Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin- Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of visual transformers. arXiv preprint arXiv:2103. 15670, 2021. 2
[16] Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li. Thomas Unterthiner, and Andreas Veit. Understanding robustness of transformers for image classification. urXiv preprint arXiv:2103.1 14586. 2021. 2
[17] Tom B Brown, Dandelion Mané, Aurko Roy. Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint 17X16:17.2.09666, 2017. 2,8

---

[18] Sayak Paul and Pin- Yu Chen. Vision transformers are robust learners. arXiv preprint arXiv:2105.0758/ 2021. 2
[19] Wieland Brendel and Matthias Bethge. Approximating cans with bag surprisingly well on imagenet. arXiv preprint arXiv:1904.00 0904.000000 2019.
models works
[20] Md Amirul Islam, Matthew Kowal, Patrick Esser, Sen Jia, Bjorn Ommer, Konstantinos G Derpanis, and Neil Bruce, Shape or texture: Understanding discriminative features in cnns, arXiv preprint arXiv:2101. 1604, 2021. 3
[21] David V Foster and Peter Grassberger. Lower bounds on mutual information. Physical Review E, 83(1):110101, 2011. 3
[22] Shikhar Tuli, Ishita Dasgupta, Erin Grant, and Thomas L. Griffiths. Are convolutional neural networks or transformers more like human vision? arXiv preprint arXiv: wwxiv2105.00799, 2021. 3
[23] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint arXiv:2 104.:22004442000 14294, 2021. 3, 7, 10
[24] Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, pages 818-833. Springer, 2014. 3
[25] Chuan Yang, Lihe Zhang, Huchuan Lu, Xiang Ruan, and Ming-Hsuan Yang. Saliency detection via graph-base based manifold ranking. In IEEE Conference Computer Vision and Puttern Recognition, pages 3166-3173, 2013. 3
[26] Yuxin Mao, Jing Zhang, Zhexiong Wan, Yuchao Dai, Aixuan Li, Yunqiu Lv, Xinyu Tian, Deng-Ping Fan, and Nick Barnes. Transformer transforms salient object detection and camouflaged object detection, arXiv preprint arXiv:2104. 10127, 2021. 3,7
[27] Ashish Vaswani, Noam Shazeer. Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. arXiv preprint 1rXiv:1706.03762, 2017. 3,
[28] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016. 3
[29] David Forsyth. Probability and statistics for computer science. Springer, 2018. 4
[30] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:/503.02531 2015. 6
[31] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 1735:1735-1788 1997. 7
[32] Kazuki Irie, Albert Zeyer, Ralf Schlüter, and Hermann Ney. Language modeling with deep transformers. arXiv preprint arXiv: 04226, 2019. 7
[33] Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Iaksshmnnarayanan. AugMix: A simple data processing method to improve robustness and uncertainty, International Conference on Learning Representations, 2020. 8
[34] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples.
In International Conference on Learning Representations, 2014. 8
[35] Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. 8
[36] A. Razavian, Hossein Azizpour, J. Sullivan, and S. Carlsson. Can features off-the-shelf: An astounding baseline for recognition. In IEFE Conference on Computer Vision and Pattern Recognition Workshops,
pages 512-519, 2014. 9
[37] P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-200 -022010-01, California Institute of Technology, 2010. 9
[38] M-E, Nilsback and A. Zisserman. Automated flower classification over a large number of classes, In Indian Conference Computer Vision, Graphics and Image Processing, 2008.
12

---

[39] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig. Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In IEEE Conference on Computer Vision and Pattern Recognition, pages 8769-8778, 2018.
[40] S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013.
[41] M. Cimpoi, S. Maji, Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In JEEE Conference on Computer Vision and Pattern Recognition, pages 3606 3613, 2014.
[42] Sebastian Houben, Johannes Stallkamp, Jan Salmen, Mare Schlipsing. and Christian Igel. Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark. International Joint Conference on Neurul Networks, 2013. 9
[43] Brigit Schroeder and Yin Cui. Fgvex fungi classification challenge 2018. g github. com/ visipedio/ fgucx_ fungs_ comp, 2018.9 9
[44] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intellignnce,
(000:-112-4400 2017. 9
[45] Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. Meta-dataset: A dataset of datasets for lcarning to learn from few examples. http://aarttiorrggaas////0000000 10
0bss/193300300 2019.
[46] Yonglong Tian, Yue Wang. Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola. Rethinking sewtshot image classification: a good embedding all you need? arXiv preprint 1rXiv:2003.11539 2020. 10
[47] Kaiyu Yang. Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky, Towards fairer datasets: Filtering and balancing the distribution the people subtree in the imagenet hierarchy. In ACM Conference on Faimess,, Accountability, and Transparency, pages
2020.
[48] Kaiyu Yang. Jacqueline Yau, Fei-Fei. Deng, and Olga Russakovsky, study faae obfuscation imagenet. arXiv preprint 103.:::103306/999 2021. 10
[49] Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. IEEE International Conference on Computer Vision, 2021. 17
[50] Ilija Radosavovic, Raj Prateck Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. Designing network design spaces, In JEEE Conference on Computer Vision and Pattern Recognition, 2020. 17
13


Reviews:
[{"review_id": "vj5PYT6bu8a", "reviewer": "Reviewer_KJfC", "summary": "The paper systematically studies ViT’s performance on occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. Experiments show that ViT models are relatively more robust to occlusions, perturbations and domain shifts, and less sensitive to local textures. Off-the-shelf features from ViT models yield high accuracy on several downstream classification datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the paper is clear and well-written. Experiments are thorough and fair. I don’t see any red flags during my review, except for a few questions and suggestions as follows:\n-\tL190 the authors discovered that shape biased ViT offers automated object segmentation. A recent work, [1], shows this property as well by self-supervised training without any labels. Exploring or discussing this more would be good.\n-\tL138, the authors claimed that “class token preserves information”. It is a confusing statement as the authors may be simply stating a fact, or it could be that the authors argued the class token itself was essential. Either way it is somewhat inaccurate. The experimented architecture is (a) ViT + class token, and (b) CNN + feature map. If (a) is better than (b), it’s obvious that it doesn’t imply class token is better than feature map. \n-\tAll CNN experiments are done with ResNet50. However, ViT models are the latest cool models whereas ResNet was proposed 6 years ago. Using state-of-the-art CNNs, some of which are highly similar to ResNet but yield much better accuracy, such as RegNetY [2], is strongly encouraged.\n-\tTable 7 studies transfer accuracy using off-the-shelf features from various ViT stages. Intuitively, of course, the final layer’s features are the best ones, but the study is only done on CUB dataset, which is notorious for being not reliable. I would suggest the authors to make the study more rigorous, however small or intuitive the study’s result may yield.\n-\tIn the y axis of figures, some of them use percentage (e.g., 50% accuracy), while some of them use decimal (e.g., 0.5 accuracy). Please make it consistent over the whole paper. Figure 9 and 10 have the wrong unit, i.e., 0.5% accuracy. \n\n\n[1] Emerging Properties in Self-Supervised Vision Transformers, Caron et al.\n[2] Designing Network Design Spaces, Radosavovic et al.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "tuDV6lKe-Wz", "reviewer": "Reviewer_yAfc", "summary": "This paper explores a variety of properties of ViTs about occlusions, shapes and texture bias, perturbations, robustness and domain shifts. The authors empirically demonstrate above merits of ViTs compared with CNNs by sufficient experiments on fifteen vision datasets. Besides, an architectural modification to DeiT for shape distillation and a transfer approach utilizing an ensemble of representations with a pre-trained ViTs are proposed as new design choices. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality：\nThis paper shows some intriguing properties of ViTs. Besides, the authors also propose two design choices for ViTs for different tasks. Although some properties are intriguing, the experiment design in this paper basically follows the discussion of these properties of CNNs, which leads to the lack of novelty.\n\n### Quality\nThe submission is technically sound and is a complete piece of work. It also provides attention maps for explanation. However, only experimental explanations for these properties are given, and theoretical proof is lacking.\n\n### Clarity\nThe paper is well organized with clear tables and figures and easy to follow.\n\n### Significance:\nThis paper shows some intriguing properties of ViTs, which may be conducive to further understanding of this new architecture. The amount of experiments is sufficient, the arrangement of experiments is reasonable, and the results are convincing. \n\nAs mentioned in line 234, you attribute the robustness of ViT to its flexible and dynamic receptive field by exclusion. I would like to know if you can provide direct experiments to prove this conclusion.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do provide description of the limitations mentioned and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}, {"review_id": "puYStzd0i-4", "reviewer": "Reviewer_Xmdv", "summary": "This paper analyzes intriguing properties of Vision Transformers (ViTs) in the context of robustness and generalizability compared to CNNs. In particular, they show that (i) ViTs are highly robust to severe occlusions and perturbations, (ii) ViTs are significantly less biased towards texture, (iii) ViTs that encode shape representation provides accurate semantic segmentation without labels, (iv) we can ensemble off-the-shelf tokens from a single ViT model for transfer learning to a wide range of tasks. They conduct their experiments on three transformer families (ViT, DeiT, T2T) across 15 vision datasets.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While existing papers regarding transformers in vision focus on the architectural novelty or applying them to novel tasks, this paper provides a thorough and extensive study on the properties of vision transformers regarding robustness.\n\nI find it particularly intriguing that a single ViT can model conflicting features (e.g. both texture and shape bias) using separate tokens, which is a property that CNNs cannot achieve. This property can be particularly useful in increasing the downstream transferability to a more wide range of tasks. Not all downstream tasks require the same set of invariances, e.g. invariance to rotation can be beneficial in view-independent aerial image recognition but harmful in tasks such as detecting which way is up in a photograph [1]. In these cases, we could simply ensemble different blocks whose tokens encode different invariances, as how others did in Section 3.5.\n\nThe paper is well-written and clear except for minor typos: (271) comapre -> compare. \n\nOverall, I believe this is a solid work that will be helpful to the community.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I appreciate that the authors pointed out that their analysis is based on “ImageNet (ILSVRC ‘12)”  pre-trained vision transformer which suffer its own biases (e.g. dominantly Western) as well as privacy risks. Therefore, their in-depth analysis is limited to the scope of this training set and may very well reflect the potential biases in the learned representations.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "14"}, "extra_text": {}}, {"review_id": "X75bAVJ6xe7", "reviewer": "Reviewer_C6Em", "summary": "This paper analyzes several intriguing properties of vision transformers (ViTs): First, it shows vision transformers have strong robustness against severe occlusions for foreground objects, non-salient background regions, and random patch locations, which are significantly better than the CNN counterparts. Second, vision transformers demonstrate better awareness of the shape whereas CNNs make decisions mainly based on texture. A shape distillation strategy is further proposed to improve the ViTs for a balance between classification performance and shape bias. Third, vision transformers achieve better robustness against spatial patch-level permutations, adversarial perturbations, and common natural corruptions. Finally, vision transformers transfer better than CNNs on visual classification and few-shot learning. A training scheme using multi-block class tokens (and/or average patch tokens) as a feature ensemble is applied to further boost the performance.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths / Contribution:\n\nThis paper provides a comprehensive analysis of the properties of vision transformers, mainly focusing on robustness, shape/texture awareness, and transferability. \n\n1. In terms of the robustness evaluation, it is observed that ViTs (DeiT, ViT) are generally more robust than CNNs (ResNet, VGG, DenseNet, SqueezeNet) on three types of occlusions: Foreground objects, non-salient background regions, random patch locations. A more thorough and solid analysis in the supplementary shows that the better robustness is observed on various grid settings (from 4x4 to 16x16), which does not need to match the input grid (14x14) in the model (although the matched grid leads to a larger gap between ViTs and CNNs). This observation is interesting since it shows each token can still extract partially meaningful information from occluded patches and contribute to reasonable classification performance. Additionally, experiments on correlation coefficients also (partially) demonstrate that the class token in ViTs retains more information than the features in ResNet after applying occlusion to input.\n\n     Moreover, some preliminary results on the adversarial patch attack also pinpoint that ViTs could have higher robustness than CNNs against untargeted, universal adversarial patches.\n\n2. This paper examines the shape-bias of ViTs and CNNs and finds out the ViTs have a better awareness of shape in decisions, even with an explicit shape-oriented training scheme (i.e. use stylized ImageNet). To have a better balance between the classification performance and shape bias, the shape distillation using an auxiliary shape token is proposed. The shape distillation also helps the automated object segmentation task. \n\n3. The authors attempt to validate if the position encoding in ViTs can preserve the global image context (e.g. global shape structure) by shuffling the image patches. However, I think this section is not well organized, as mentioned in the weaknesses section below.\n\n4. Since ViTs naturally have multiple class tokens from all blocks, this paper proposes a feature ensemble approach that combines several class tokens for better generalization. Evaluations on transfer learning tasks including visual classification and few-shot learning demonstrate that using top several class tokens can greatly improve the feature transfer performance on a wide range of tasks.\n\nIn addition, the paper is well written, properly organized, and easy to read.\n\nWeaknesses: \n1. In Line 138, the paper uses correlation coefficient to measure the amount of preserved information in class token. However, the detailed calculation of this coefficient is not provided. In addition, it mentions \"In the case of ResNet50, we consider features before the logit layer\", where I am not sure if the features in ResNet are comparable with the class token (are both the feature vectors before the last linear layer for classification?)\n\n2. In Sec. 3.3, the authors would like to \"analyze if the sequence order modeled by positional encoding allows ViT to excel under occlusion handling\". However, a simple comparison of ViT and ViT (w/o position encoding) on different occlusion settings is missing. In addition, the shuffling patches experiments suggest that transformers demonstrate high permutation invariance to the patch positions, and a claim \"the effect of positional encoding towards injecting structural information of images models is limited\" has been made. However, the Fig. 9 (right) shows a clear gap between ViT and ViT w/o pos, especially when the shuffle grid size (196) matches the input grid size, which seems contradictory to the claim.\n\n3. In Sec. 3.4, it analyzes the robustness of ViTs under adversarial and natural perturbations and attempts to find its connection with shape-bias. However, only one certain type of adversarial attack is used (untargeted, universal adversarial patch in white-box setting). The conclusion would be more convincing if more adversarial attack methods are employed in the experiment.\n\n4. Typo(s): \n\n    Line 200: perofrm -> perform \n\n    Line 276: specie classification -> species classification\n\n    Figure 9: Y-axis is in %, thus the values should be 0, 10, 20, ..., 80 instead of 0, 0.1, ..., 0.8\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have mentioned the risk of the potential bias in the ImageNet pretrained ViTs and provided solutions.", "review_text": "", "extra_scores": {"ethical_concerns": "I think there is no significant ethical issue with this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers agree that this is a solid paper that should be accepted. The analysis of vision transformers vs convnets provided in the paper is through, interesting and potentially useful for practitioners. The authors mostly addressed the (not very numerous) reviewer' concerns. Clear accept.
Decision: Accept (Spotlight)