Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper analyzes intriguing properties of Vision Transformers (ViTs) in the context of robustness and generalizability compared to CNNs. In particular, they show that (i) ViTs are highly robust to severe occlusions and perturbations, (ii) ViTs are significantly less biased towards texture, (iii) ViTs that encode shape representation provides accurate semantic segmentation without labels, (iv) we can ensemble off-the-shelf tokens from a single ViT model for transfer learning to a wide range of tasks. They conduct their experiments on three transformer families (ViT, DeiT, T2T) across 15 vision datasets.

main_review: While existing papers regarding transformers in vision focus on the architectural novelty or applying them to novel tasks, this paper provides a thorough and extensive study on the properties of vision transformers regarding robustness.

I find it particularly intriguing that a single ViT can model conflicting features (e.g. both texture and shape bias) using separate tokens, which is a property that CNNs cannot achieve. This property can be particularly useful in increasing the downstream transferability to a more wide range of tasks. Not all downstream tasks require the same set of invariances, e.g. invariance to rotation can be beneficial in view-independent aerial image recognition but harmful in tasks such as detecting which way is up in a photograph [1]. In these cases, we could simply ensemble different blocks whose tokens encode different invariances, as how others did in Section 3.5.

The paper is well-written and clear except for minor typos: (271) comapre -> compare. 

Overall, I believe this is a solid work that will be helpful to the community.


limitations_and_societal_impact: I appreciate that the authors pointed out that their analysis is based on “ImageNet (ILSVRC ‘12)”  pre-trained vision transformer which suffer its own biases (e.g. dominantly Western) as well as privacy risks. Therefore, their in-depth analysis is limited to the scope of this training set and may very well reflect the potential biases in the learned representations.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper systematically studies ViT’s performance on occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. Experiments show that ViT models are relatively more robust to occlusions, perturbations and domain shifts, and less sensitive to local textures. Off-the-shelf features from ViT models yield high accuracy on several downstream classification datasets.
main_review: Overall, the paper is clear and well-written. Experiments are thorough and fair. I don’t see any red flags during my review, except for a few questions and suggestions as follows:
-	L190 the authors discovered that shape biased ViT offers automated object segmentation. A recent work, [1], shows this property as well by self-supervised training without any labels. Exploring or discussing this more would be good.
-	L138, the authors claimed that “class token preserves information”. It is a confusing statement as the authors may be simply stating a fact, or it could be that the authors argued the class token itself was essential. Either way it is somewhat inaccurate. The experimented architecture is (a) ViT + class token, and (b) CNN + feature map. If (a) is better than (b), it’s obvious that it doesn’t imply class token is better than feature map. 
-	All CNN experiments are done with ResNet50. However, ViT models are the latest cool models whereas ResNet was proposed 6 years ago. Using state-of-the-art CNNs, some of which are highly similar to ResNet but yield much better accuracy, such as RegNetY [2], is strongly encouraged.
-	Table 7 studies transfer accuracy using off-the-shelf features from various ViT stages. Intuitively, of course, the final layer’s features are the best ones, but the study is only done on CUB dataset, which is notorious for being not reliable. I would suggest the authors to make the study more rigorous, however small or intuitive the study’s result may yield.
-	In the y axis of figures, some of them use percentage (e.g., 50% accuracy), while some of them use decimal (e.g., 0.5 accuracy). Please make it consistent over the whole paper. Figure 9 and 10 have the wrong unit, i.e., 0.5% accuracy. 


[1] Emerging Properties in Self-Supervised Vision Transformers, Caron et al.
[2] Designing Network Design Spaces, Radosavovic et al.

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper explores a variety of properties of ViTs about occlusions, shapes and texture bias, perturbations, robustness and domain shifts. The authors empirically demonstrate above merits of ViTs compared with CNNs by sufficient experiments on fifteen vision datasets. Besides, an architectural modification to DeiT for shape distillation and a transfer approach utilizing an ensemble of representations with a pre-trained ViTs are proposed as new design choices. 
main_review: ### Originality：
This paper shows some intriguing properties of ViTs. Besides, the authors also propose two design choices for ViTs for different tasks. Although some properties are intriguing, the experiment design in this paper basically follows the discussion of these properties of CNNs, which leads to the lack of novelty.

### Quality
The submission is technically sound and is a complete piece of work. It also provides attention maps for explanation. However, only experimental explanations for these properties are given, and theoretical proof is lacking.

### Clarity
The paper is well organized with clear tables and figures and easy to follow.

### Significance:
This paper shows some intriguing properties of ViTs, which may be conducive to further understanding of this new architecture. The amount of experiments is sufficient, the arrangement of experiments is reasonable, and the results are convincing. 

As mentioned in line 234, you attribute the robustness of ViT to its flexible and dynamic receptive field by exclusion. I would like to know if you can provide direct experiments to prove this conclusion.

limitations_and_societal_impact: The authors do provide description of the limitations mentioned and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper analyzes several intriguing properties of vision transformers (ViTs): First, it shows vision transformers have strong robustness against severe occlusions for foreground objects, non-salient background regions, and random patch locations, which are significantly better than the CNN counterparts. Second, vision transformers demonstrate better awareness of the shape whereas CNNs make decisions mainly based on texture. A shape distillation strategy is further proposed to improve the ViTs for a balance between classification performance and shape bias. Third, vision transformers achieve better robustness against spatial patch-level permutations, adversarial perturbations, and common natural corruptions. Finally, vision transformers transfer better than CNNs on visual classification and few-shot learning. A training scheme using multi-block class tokens (and/or average patch tokens) as a feature ensemble is applied to further boost the performance.
main_review: Strengths / Contribution:

This paper provides a comprehensive analysis of the properties of vision transformers, mainly focusing on robustness, shape/texture awareness, and transferability. 

1. In terms of the robustness evaluation, it is observed that ViTs (DeiT, ViT) are generally more robust than CNNs (ResNet, VGG, DenseNet, SqueezeNet) on three types of occlusions: Foreground objects, non-salient background regions, random patch locations. A more thorough and solid analysis in the supplementary shows that the better robustness is observed on various grid settings (from 4x4 to 16x16), which does not need to match the input grid (14x14) in the model (although the matched grid leads to a larger gap between ViTs and CNNs). This observation is interesting since it shows each token can still extract partially meaningful information from occluded patches and contribute to reasonable classification performance. Additionally, experiments on correlation coefficients also (partially) demonstrate that the class token in ViTs retains more information than the features in ResNet after applying occlusion to input.

     Moreover, some preliminary results on the adversarial patch attack also pinpoint that ViTs could have higher robustness than CNNs against untargeted, universal adversarial patches.

2. This paper examines the shape-bias of ViTs and CNNs and finds out the ViTs have a better awareness of shape in decisions, even with an explicit shape-oriented training scheme (i.e. use stylized ImageNet). To have a better balance between the classification performance and shape bias, the shape distillation using an auxiliary shape token is proposed. The shape distillation also helps the automated object segmentation task. 

3. The authors attempt to validate if the position encoding in ViTs can preserve the global image context (e.g. global shape structure) by shuffling the image patches. However, I think this section is not well organized, as mentioned in the weaknesses section below.

4. Since ViTs naturally have multiple class tokens from all blocks, this paper proposes a feature ensemble approach that combines several class tokens for better generalization. Evaluations on transfer learning tasks including visual classification and few-shot learning demonstrate that using top several class tokens can greatly improve the feature transfer performance on a wide range of tasks.

In addition, the paper is well written, properly organized, and easy to read.

Weaknesses: 
1. In Line 138, the paper uses correlation coefficient to measure the amount of preserved information in class token. However, the detailed calculation of this coefficient is not provided. In addition, it mentions "In the case of ResNet50, we consider features before the logit layer", where I am not sure if the features in ResNet are comparable with the class token (are both the feature vectors before the last linear layer for classification?)

2. In Sec. 3.3, the authors would like to "analyze if the sequence order modeled by positional encoding allows ViT to excel under occlusion handling". However, a simple comparison of ViT and ViT (w/o position encoding) on different occlusion settings is missing. In addition, the shuffling patches experiments suggest that transformers demonstrate high permutation invariance to the patch positions, and a claim "the effect of positional encoding towards injecting structural information of images models is limited" has been made. However, the Fig. 9 (right) shows a clear gap between ViT and ViT w/o pos, especially when the shuffle grid size (196) matches the input grid size, which seems contradictory to the claim.

3. In Sec. 3.4, it analyzes the robustness of ViTs under adversarial and natural perturbations and attempts to find its connection with shape-bias. However, only one certain type of adversarial attack is used (untargeted, universal adversarial patch in white-box setting). The conclusion would be more convincing if more adversarial attack methods are employed in the experiment.

4. Typo(s): 

    Line 200: perofrm -> perform 

    Line 276: specie classification -> species classification

    Figure 9: Y-axis is in %, thus the values should be 0, 10, 20, ..., 80 instead of 0, 0.1, ..., 0.8

limitations_and_societal_impact: Yes, the authors have mentioned the risk of the potential bias in the ImageNet pretrained ViTs and provided solutions.
review_title: null

Meta-review: The reviewers agree that this is a solid paper that should be accepted. The analysis of vision transformers vs convnets provided in the paper is through, interesting and potentially useful for practitioners. The authors mostly addressed the (not very numerous) reviewer' concerns. Clear accept.
Decision: Accept (Spotlight)