Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation
Authors: Yao Qin
Chiyuan Zhang
Ting Chen
Balaji Lakshminarayanan
Alex Beutel
Xuezhi Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Robustness
Distributional shift
Vision transformers

Abstract: We investigate the robustness of vision transformers (ViTs) through the lens of their special patch-based architectural structure, i.e., they process an image as a sequence of image patches. We find that ViTs are surprisingly insensitive to patch-based transformations, even when the transformation largely destroys the original semantics and makes the image unrecognizable by humans. This indicates that ViTs heavily use features that survived such transformations but are generally not indicative of the semantic class to humans. Further investigations show that these features are useful but non-robust, as ViTs trained on them can achieve high in-distribution accuracy, but break down under distribution shifts. From this understanding, we ask: can training the model to rely less on these features improve ViT robustness and out-of-distribution performance? We use the images transformed with our patch-based operations as negatively augmented views and offer losses to regularize the training away from using non-robust features. This is a complementary view to existing research that mostly focuses on augmenting inputs with semantic-preserving transformations to enforce models' invariance. We show that patch-based negative augmentation consistently improves robustness of ViTs on ImageNet based robustness benchmarks across 20+ different experimental settings. Furthermore, we find our patch-based negative augmentation are complementary to traditional (positive) data augmentation techniques and batch-based negative examples in contrastive learning. 

Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation
Yao Qin
Chiyuan Zhang
Ting Chen
Balaji Lakshminarayanan
Alex Beutel
Xuezhi Wang
Google Research
Abstract
We investigate the robustness of vision transformers (ViTs) through the lens of their special patch-based architectural structure, i.e., they process an image as a sequence of image patches. We find that ViTs are surprisingly insensitive to patch- based transformations, even when the transformation largely destroys the original semantics and makes the image unrecognizable by humans. This indicates that ViTs heavily use features that survived such transformations but are generally not indicative of the semantic class to humans. Further investigations show that these features are useful but non-robust, as ViTs trained on them can achieve high in-distribution accuracy, but break down under distribution shifts. From this un- derstanding, we ask: can training the model to rely less on these features improve VIT robustness and formed with our patch-based baseed operations as negatively augmented views and offer losses to regularize the training away from using non-robust features. This is complementary view to existing research that mostly focuses on augmenting in- puts with semantic-preservingg transformations to enforce models' invariance. We show that patch-based negative augmentation consistently improves robustness of ViTs ImageNet based robustness benchmarks across 20+ different experi- mental settings. Furthermore, we find our patch-based negative augmentation are complementary traditional (positive) data augmentation techniques and batch- based negative examples in contrastive learning.
performance? We use the images trans-
Introduction
Building vision models that robust, i.e., that maintain accuracy distribution images, is increasingly a requirement to trusting vision models and a strong benchmark for progress in the field. Recently, Vision Transformers (ViT', Dosovitskiy al. (2021)) sparked great interest in the literature, as a radically new model architecture offering significant accuracy improvements and with hope of new robustness benefits. Over the past decade, there has been extensive work on understanding the robustness of convolution-based neural architectures, as the dominant design for visual tasks; researchers have explored adversarial robustness (Szegedy al., 2013), domain generalization (Xiao et al., 2021: Khani & Liang, 2021), feature biases (Brendel & Bethge, 2019; Geirhos et al., 2018: Hermann et al., 2020). As result, with the new promise of vision transformers, it is critical to understand their properties and in particular their robustness. Recent early studies (Naseer et al., 2021: Paul & Chen, 2021: Bhojanapalli al., 2021) have found ViTs be more robust than ConvNets in some scenarios, with the hypothesis that the non-local attention based interactions enabled ViTs capture more global and semantic features. In contrast, we add this
unexpected and out-of-
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Class: keeshond
Cont: 0,982
Cont:
Class:
P-Shuffle
P-lfill Roplace Rato: 25%
PJlfill Roplace Rate: 37.5%
Patch Sizc: x2
Patch Size: 48
Patch Size: 32 32 Patch Size: 48
Figure Patch-based transformations largely destroy images to unrecognizable to humans whereas ViT recognizes them as the original class (e.g., keeshond or magpie) with high confidence. Visualization of patch-based transformations. On the top of each image, we display the predicted confidence score of ViT-B/16 pretrained on ImageNet-21k and finetuned on ImageNet-Ik.
line of research showing a different side of the challenge: we find that ViTs are still vulnerable to relying on non-robust features, and propose an algorithm to mitigate it for better iitoo--distribution performance.
Specifically, we first conduct a study on the robustness properties of ViTs and show that ViTs still rely on specific non-robust features. We with the architectural traits of ViTs ViTs operate on non-overlapping image patches and allow long range interaction between patches even in lower layers. hypothesized in recent studies (Naseer al., 2021; Paul & Chen, 2021: Bhojanapalli et al., 2021) that the non-local attention based interactions contribute to better robustness of ViTs than ConvNets. To study the ability of ViTs to integrate global semantics structures across patches, we design and apply patch-based image transformations, such as random patch rotation, shuffling, and background-infiling (Figure 1). Those transformations destroy the spatial relationship between patches and corrupted the global semantics, and the resultant images are often visually unrecog- nizable. However, we find that ViTs are surprisingly insensitive to these transformations and can make highly accurate predictions that survive such transformations but are generally not indicative the semantic class to humans. Going one step further, we find that those features are useful but not robust, as VITs trained them achieved high in di-triibbution accuracy, but suffered significantly on robustness benchmarks.
these transformed images. This suggests that ViTs use features
With this understanding of ViTs' reliance on non-robust features captured by patch-based transfor- mations, we aim mitigate this shortcoming by answering the following questions: (a) how can we train VìT's to not rely such features? and (b) will reducing reliance on such features meaningfully improve distriibuiiinn performance and not sacrifice in-distribution accuracy? past robust training algorithms encourage the smoothness of model predictions on augmented im- ages with semantic preserving transformations (Hendrycks et al., 2020; Cubuk et al., 2019). How- ever, the aatch-based transformations deliberately destroy the semantic meaning and only leave non- robust features. Taking inspiration from recent research on generative modeling (Sinha al., 2020), propose family of robust training algorithms based on busch negative augmentations that regularize the training from relying on non-robust features surviving patch-based transformations. Through extensive evaluation ods consistently improve the robustness of the trained ViTs. Furthermore, our patch-based negative augmentation be combined with the traditional (positive) data augmentation to boost the per- formance further. With this we get a more complete picture: training models both to be insensitive spurious changes (as in positive augmentation) but also to not rely non-robust features (as in negative augmentation) together can meaningfully improve robustness of ViTs.
majority of
wide of ImageNet-based benchmarks, find that meth-
Our key contributions are as follows:
Understanding Non-Robust Features in VIT: We show that VITs heavily rely on non-robust. features surviving patch-based transformations but are not indicative of the semantic classes to humans.

---

Patch wise Shuffie
Patch wise Rotate
Potch wise Infill
Clean
Tep-1 Accurary (%6
VTT-8/16
VIT-8/16
VT-B/16
VIT-8/32
B/T
VTT6/32
WT--L/16
WIT-L/16
WT-L/16
1/32
V/2
L/32
64
96
32
64
96
0.4
0.3
0.2
Patch Size
Patch Size
Replace Rate
ViTs can rely features surviving patch-based transformations maintain high accu- racy, even after images have been heavily transformed to be largely unrecognizable. Top-1 accuracy of ViT models when tested on patch-based transformed images using the semantic class of the cor- responding clean image as ground-truth. The test accuracy on ImageNet- Ik validation set is shown on the right. All ViT models are pre-trained on ImageNet-21k and fine-tuned on ImageNet-Ik.
Figure 2:
We propose set of patch-based operations as negatively augmented views, com- plementary existing works that focus on semantic-pre preeerriien ("positive") augmentations, to regularize the training away from using these specific non-robust features;
Modeling:
Improved Robustness of ViT: We show across 20+ experimental settings that our proposed patch-based negative augmented views can consistently improve ViT's robustness and com- plementary to "positive" augmentation techniques as well as batch-based negative examples in contrastive learning.
2 Preliminaries
Vision Transformers Vision transformers (Dosovitskiy et al., 2021) are family of architec- tures adapted from Transformers natural language processing (Vaswani et al., 2017), which di- rectly process visual tokens constructed from image patch embeddings. To construct visual tokens, VIT (Dosovitskiy et al., 2021) first splits an image into a grid of patches. Fach patch linearly pro- jected into hidden representation vector, and combined with a positional embedding. A learnable class token is also added. Transformers (Vaswani et al., 2017) are then directly applied on this set of visual and class token embeddings as if they are word embeddings in the original Transformer formulation. Finally, a linear projection of class token is used to calculate the class probability.
Model Variants We consider ViT models pretrained either ILSVRC-2012 ImageNet- 1k. with 133 million images or ImageNet-2Ik,, with 14 million images (Russakovsky et al., 2015). All models are fine-tuned on ImageNet-Ik 1k dataset. We adopt the notations used in (Dosovitskiy al., 2021) to denote model size and input patch size. For example, ViT-B/16 denotes the "Base" model variant with input patch size 16 x 16.
Robustness Benchmarks To evaluate models' robustness, we mainly focus on three ImageNet- based robustness benchmarks, ImageNet-A (Hendrycks et al., 2019), ImageNet-C (Hendrycks & Dietterich, 2019a), ImageNet-R (Hendrycks et al., 2021). Specifically, ImageNet-A contains chal- lenging natural images from a distribution unlike ImageNet training distribution, which can easily fool models to make misclassification. ImageNet-C consists of 19 types of conruptions that are widely
frequently encountered natural images and each corruption has levels of severity. used to measure models' robustness under distributional shift. ImageNet-R is composed of images obtained by artistic rendition of ImageNet classes, c.g., cartoons, and is widely used to evaluate model's robustness out-of-distribution data.
3 Understanding the Robustness of Vision Transformers
Recent works (Naseer et al., 2021; Paul & Chen, 2021; Bhojanapalli et al., 2021) have shown that vision transformers achieve better robustness compared to standard convolutional networks. One explanation is that the attention mechanism can capture better global structures. To investigate VIT has successfully taken advantage of the long range interactions between patches, design series of patch- patch-based transformations which significantly destroys the global structure of images. The patch-based transformations (see Fig. 1) are:
3

---

ImageNet-1k 1k
ImageNantAA
Imagohet C (Cauusiar No se-5)
Imagetett-
64.1
1.0
-2.5
Kohulfte (pswab)
K-Rccate (ps=1b)
Korill (mrw0.54)
80
50
P-Shuffie (ps=32)
80
P P-Rosate (p5=32)
90
P-nfll (m=0.42)
P-Shuffic
Robotc (p5-48)
Pinnlll (m=00550)
600
60
60
60
40
40
46
40
37.9
-7.
20
20
26.7
-12.2
-4.5
20
-22
2
-.9
20
VTT8/6
P.Sh.ftte
P-Rotate
Pimiil
MTT0/16
P.Shutte
P-Rctate
Pinnt
VIT-8/66 P.Sruttee
P-Ratane
Pinnil
NTT8/16 P.Shultle
P-Rotate
Pintil
Figure 3: Features preserved in patch-based transformations are useful but non-robust as training ViT on them impedes robustness. Top-1 Accuracy (%) on ImageNet- Ik validation set and Ima- geNet robustness datasets: ImageNet-A, ImageNet-C. ImageNet-R, RR The baseline model is ViT-B/16 in (Dosovitskiy et al., 2021) trained on original images. Other models are trained on patch-based transformed images, e.g.. "P-Shuffle" stands for ViT-B/16 model trained on patch-based shuffled images. Numbers above the bars are either accuracy (c.g., ViT-B/16) or the max accuracy difference between each model family and the baseline ViT-B/16. The patch size in P- P-Shuffle and P-Rotate and replacement ratio in P-Infil is denoted by "ps" and "rr" respectively.
Patch-based Shuffle (P-Shuffle): we randomly shuffle the input image patches to change their positions'.
Patch-based Rotate (P-Rotate): randomly select rotation degree from the set Ω {09, 90,900, 1800 } and rotate each image patch independently. Patch-based Infill (P-Infill): replace the image patches in the center region of an image with the patches on the image boundary".
Each patch-based transformation performed to a single image. We make sure the patch size of our patch-based transformation is multiple of the input image patch of ViT so that the content within each patch well-maintained. For P-infill, we use "replace rate" to denote the ratio of replaced patches in the center over the total number of patches in image. Examples of transformed images are shown in Fig. (see Appendix E for more examples). In most cases, it is challenging to recognize the semantic classes after those transformations.
Do ViTs rely on features not indicative of the semantic classes to humans? ? To validate ViTs be- have similarly as humans on these patch-based transformed images, we evaluate ViT models (Doso- vitskiy al., 2021) on these patch- based transformed image. Specifically, we apply each patch- based transformation to ImageNet- Ik validation set and report the test accuracy of each VIT on the transformed images. The test accuracy is computed by using the semantic class of the correspond- ing original image the ground-truth. As shown in Figure 2, the accuracies achieved by ViT's are significantly higher than random guessing (0.1%) addition, as shown Figure 1, VIT gives these patch-base based transformed images highly confident prediction even when the transformation largely destroys the semantics and make the image unrecognizable by humans." This strongly indicates that ViT models heavily rely the features that survive these transformations to make a prediction. However, these features are not indicative of the semantic class to humans.
Do features preserved in patch-based transformations impede robustness? Taking one step further, we want to know the features preserved by simple patch-based transformations, which are not indicative of the semantic class to humans, result in robustness issues. To this end, we train vision transformer, e.g., ViT-B/16, on patch-based transformed images with original semantic class assigned as their ground-truth. Note that all the training images are patch-based transformed images. In this way, we force the model to fully exploit the features preserved in patch- based transformations. In addition, make sure the "only" difference between our models and the baseline ViT-B/16 (Dosovitskiy et al., 2021) is the training images. Then, test the model ImageNet-1k validation set and three robustness benchmarks, ImageNet-A, ImageNet-C and ImageNet-R without any transformation.
reuse all the training details and hyperparameters in (Dosovitskiy et al., 2021) to
PP-Shuffee is equivalent to shuffling the position embeddings.
Forr example, given an image with size 384 x 384, input patch size is 16 x 16 and replace rate 0.25, in total have 576 patches æi.j. where i and j denotes the row and column index and 1 S i.j S 24. The patches the center æm..: 7 I m, n S 18 are replaced by the remaining patches. Similar patterns observed when ViT models are pretrained on ImageNet-II.

---

First, we can observe that for the baseline ViT-B/16, compared to in-distribution accuracy, all the out-of-distribution accuracies have suffered from sigmificant drop (the 4 blue bars in Figure 3). This trend has been observed both for ViTs and convolution-based networks (Paul & Chen, 2021; Zhai et al., 2021). Second, if we compare the accuracy between the baseline model and models trained on patch-based transformations (i.e., the difference between the blue bar and one of the red/green/orange bars in Figure 3), we find that ViTs' in-distribution accuracy drops only slightly, but the robustness drop is significant when models are trained on these patch-based transformations. Take P-Shuffle as an example, the model trained on patch-based shuffled images can still achieve 79.1% accuracy ImageNet- Ik, only 5 percentage point (pp) drop in in-distribution accuracy. In contrast, the accuracy drop on robustness datasets much more significant, e.g., 17pp on ImageNet- R. The deterioration rate in robustness is close to 50% of the baseline ViT-B/16. This strongly suggests that the features preserved in patch-based transformations are sufficient for high-accurate in-distribution prediction but not robust under distributional shifts.
Taking the above results together, we conclude that even though ViTs are shown to more robust than ConvNets in previous studies (Naseer et al., 2021; Paul & Chen, 2021), they still heavily rely on features that are not indicative of the semantic classes to humans. These features, captured by patch-based transformations, are useful but non-robust, as ViTs trained on them achieve high in- distribution accuracy but suffer significantly on robustness benchmarks.
With this knowledge we now ask: based on these understandings. can we train ViTs to not rely on such non- non-robust features? And if we do, will it improve their robustness?
4 Improving the Robustness of Vision Transformers
Based on the key observations that the patch-based transformations encode features that contribute to the non-robustness of ViTs, we propose a negative augmentation procedure to regularize ViTs from relying on such features. To this end, we use the images transformed with our patch-based operations as negatively augmented views. Then we design negative loss regularizations to prevent the model from using those non-robust features preserved in paich-based transformations.
Specifically, given a clean image æ, generate its negative view, denoted as ã, by ap- plying a patch-based transformation to x. We call it negative augmentation, in contrast with the standard (positive) augmentation that are semantic preserving. Let (B: Eie,)) y log softmax(f(z; 0)) represent the cross-entropy loss function used to train a vi- sion transformer with parameters O, where B is minibatch of clean examples, and y denotes the ground-truth label. The loss negative views Lneg [B, B; 0) can be easily added to the cross- entropy loss Lee (B; () via
Lee (B: + Regg [B, B;O).
where À is coefficient balancing the importance between clean training data as well as patch-based negative augmentation. Below, we introduce uniform loss and E, loss to leverage negative views.
Uniform loss Many existing data augmentation techniques (Cubuk et al., 2019, 2020; Hendrycks et al., 2020) use one-hot labels for semantic-preservvngg augmented data to enforce the invariance of the model prediction. In contrast, the semantic classes of our generated patch-based negative augmented data are visually unrecognizable, as shown in Figure 1. Therefore, we propose to use uniform labels instead for those negative augmentations. Specifically, the loss function on negative views that we optimize at each training step can be formulated as:
Eney BB B: 0)
E
Σ
( ÿ log softmax(f((;00))
(2)
B
where ÿ denotes the uniform distribution: Th = 1 where K is the total number of classes. f(;; 0) denotes the function mapping the input image into the logit space.
lg Loss An alternative to pre- pre-assuming labels for negative augmentation is to add the constraints on the logit space (or the space of predicted probability). Inspired by existing work (Kannan et al., 2018; Zhang et al., 2019; Hendryeks et al., 2020) which provides an extra regularization term en- couraging similar logits between clean and "positive" augmented counterparts, we instead encourage the logits of clean examples and their corresponding negative augmentations to be far away. this way, prevent the model from relying on the non-robust features preserved in negative views.
5

---

Table I: Top-1 accuracies for VIT models pre-trained and fine-tuned on ImageNet- Ik with or without the proposed negative augmentation. Best results highlighted in bold.
Model
IeageNet-1k
ImageNet-A
ImageNet-C
ImageNet-R
ViT-B/32 (Dosovitskiy et al., 2021)
72.5
3.7
46.7
19.4
P-Rotate / Uniform
72.9 (+0.4)
3.7 (+0.0)
47.9 (+1.2)
19.9 (+0.5)
P-Rotate L.2
73.2 (+0.7)
3.8 (+0.1)
48.4 (+1.7)
20.4 (+1.0)
ViT-B/16 (Dosovitskiy et al., 2021)
77.6
6.7
50.8
20.3
P-Rotate / Uniform
78.2 (+0.6)
7.0 (+0.3)
52.4 (+1.6)
21.4 (+1.1)
+ P-Rotate
77.8 (+0.2)
6.7 (40.0)
51.6 (+0.8)
21.0 (+0.7)
Specifically, we maximize the lg distance between the predicted probability of clean examples and their corresponding negative views. The loss on negative views, therefore, can be formulated as:
negg B:
Σ EeeB.eck
|softmax( I
(a; 0)) - softmax(f(ē; 0))122
(3)
Here the &g distance computed over the predicted probability rather than the logits (a; 0) because empirically we observe that maximizing the difference of logits can cause numerical instability.
5 Experiments
Experimental setup We follow Dosovitskiy et al. (2021) to first pre-train all the models with image size 224 x 224 and then fine-tune the models with higher resolution 384 384. We reuse all their training hyper-parameters, including batch size, weight decay, and training epochs (see Appendix B for details). For the extra loss coefficient À in Eqn. 1 sweep it from the set {0.5, 1, 1.5) and choose the model with the best hold-out validation performance. Please refer to Appendix C for the chosen hyperparameters for each model. We make sure our proposed models and our implemented baselines are trained with exactly the same settings for fair comparison.
5.1 Effective in improving robustness of vanilla ViTs
First, we apply our proposed patch- based transformations a ViT model pre-trained and fine-tuned on ImageNet-Ik, Ik. The extra loss regularization on negative views is used in both pre-training and fine-tuning stages prevent the model from learning non-robust features preserved in patch-based transformations. We use "Transformation / Regularization" to denote pair of patch-based negative augmentation and loss regularization. For examples, "P-Rotate / Uniform" means that we use P- Rotate to generate the negative views and use uniform loss to regularize the training. We display the results in Table 1, where we can clearly see that our proposed patch-based negative augmentation effectively improves the out-of-distribution robustness on ImageNet-C and ImageNet-R. Note the improvement small over ImageNet-A when the baseline almost does not work, but becomes more siginificant when the baseline improves (e.g.. Table 5).
Larger improvement on small-scale datasets We also apply our patch-based negative augmen- tation to ViT-B/16 on CIFAR- (Krizhevsky, 2009) in the fine-tuning stage and find that it can also significantly improve the robustness on the corrupted dataset CIFAR-100-C (Hendrycks & Di- etterich, 2019b), which includes 19 different corruption types with 5 different corruption severities. For example, P-Shuffle Uniform can boost the in-distribution accuracy of ViT-B/I on CIFAR-100 from 91.8% to 92.6% (+0.8) and the robustness accuracy CIFAR-100 100-C can also be improved from 74.6% to 77.0% (+2.4), as shown in Table 10 in Appendix.
5.2 Complementary to traditional (""osittiee") data augmentation
To investigate if our proposed patch-based negative augmentation is complementary to traditional ("positive") data augmentation, we apply our patch-based negative transformation on top of tra- ditional data augmentation: Rand-Augment (Cubuk et al., 2020), which is widely used in vision transformers (Touvron et al., ;021; Mao et al., 2021), and AugMix (Hendrycks et al., 2020), which specifically proposed to improve models' robustness under distributional shift. Crucially, we fol- low (Hendrycks et al., 2020) to exclude transformations used in "positive" data augmentation which overlap with corruption types in ImageNet-C (Hendrycks & Dietterich, 2019a). Therefore, the of transformations used in Rand-Augment and Aug is disjoint with the corruptions in ImageNet-C.
When combine "negative" and "positive" augmentation, the cross-entropy loss Lce (B+::0 () in Eqn. 1 is computed over "positive" examples B+ {a ... ...
x using either Rand-Augment or

---

Table 2:
Patch-based negative augmentation is complementary to "positive" data augmenta- tion. Top-1 accuracies for ViT-B/16 pre-trained and fine-tuned on ImageNet-Ik using Rand- Augment (Cubuk al., 2020) or AugMix (Hendrycks al., 2020). The proposed negative aug- mentation is added on top of either positive augmentation. Best results are highlighted in bold.
Model
ImageNet- Ik ImageNet-A
ImageNet-C
ImageNet-R
Kand-Augment (Cubuk al., 2020)
79.1
7.2
55.2
23.0
+ P-Shfflle/ / Uniform
79.3
7.7 (+0.5)
56.2 (+1.0)
23.4 (+0.4)
+ P-Rotate / Uniform
79.3
8.1 (+0.9)
56.4 (+0.8)
23.8 (+0.8)
+ P-Infill / Uniform
79.2
7.8 (+0.6)
56.4 (+1.2)
24.0 (+1.0)
+ P-Shufflc/ L.2
78.9
7.5 (+0.3)
55.6 (+0.4)
22.6 (-0.4)
+ P-Rotate /
79.1
7.9 (+0.7)
56.7 (+1.5)
23.8 (+0.8)
+ P-Infill LL2
78.8
7.4 (+0.2)
55.4 (+0.2)
23.2 (+0.2)
AugMix (Hendrycks et al., 2020)
78.8
7.7
57.8
24.9
P-Shufile Uniform
79.2
8.0 (+0.3)
58.6 (+0.8)
25.7 (+0.8)
+ P-Rotate Uniform
79.1
8.2 (+0.5)
58.5 (+0.7)
25.7 (+0.8)
+ P-Infill / Uniform
79.3
8.3 (+0.6)
58.4 (+0.6)
25.7 (+0.8)
+ P-Shuffle L12
78.8
7.9 (+0.2)
58.3 (+0.5)
25.7 (+0.8)
+ P-Rotate / L2
79.0
8.3 (+0.6)
58.8 (+1.0)
26.0 (+1.1)
+ P-Infill / L2
79.0
7.9 (+0.2)
58.5 (+0.7)
25.6 (+0.7)
Table 3: Effects of patch- basch negative augmentation on five different levels of corruption severities on ImageNet-C. Best results are highlighted in bold. See Table 12 in Appendix for full results.
ImageNet-C Corruption Severity Level
Model
2
3
4
5
(Cubuk et al., 2020)702
70.4
63.7
57.9
48,2
36.1
+ P-Rotate / Uniform
71.1 (+0.7)
64.6 (+0.9)
59.0 (+1.1)
50.0 (+1.8)
37.6 (+1.5)
P-Rotate / 1.2
71.1 (+0.7)
64.8 (+1.1)
59.5 (+1.6)
50.1 (+1.9)
37.8 (+1.7)
AugMix (Hendrycks et al., 2020)
71.4
65.2
60.5
51.9
40.2
+ P-Rotate / Uniform
71.7 (+0.3)
65.7 (+0.5)
61.1 (+0.6)
52.7 (+0.8)
41.4 (+1.2)
+ P-Rotate / 1.2
71.9 (+0.5)
66.0 (+0.8)
61.5 (+1.0)
52.9 (+1.0)
41.6 (+1.4)
AugMix. Meanwhile, the loss regularization on negative views in Eqn. I is computed over neg- atively transformed version of xt That is: for Va+ E B+, we apply our patch-based negative transformation obtain its negative version and then use the negative example to compute the loss regularization Lneg. The positive data augmentation is only used in pre-training stage as we ob- serve it is slightly better than using them for both stages (Please see more detailed discussion in Appendix D)4) and Steiner et al. (2021) have the similar observation. Instead, we apply our negative augmentation in both stages, as is the best design choice as discussed in Appendix D.3.
As shown in Table 2, we see that when our patch-based negative augmentations are applied to either Rand-Augment or AugMix, we can consistently improve the robustness of vision transformers across all three robustness benchmarks. This is particularly noteworthy as both Rand- Al--ggeueett and AugMix are already designed to significantly improve the robustness of vision models. Yet, we see that patch-based negative augmentation fur robustness benefits. This suggests that robustness of vision models was not adequately addressed by "positive" data augmentation and that basch negative augmentation is complementary to these traditional approaches.
To have clear pieture how negative augmentation improves robustness data with different de- grees of corruptions, we also display the top-1 accuracy on ImageNet-C with five different levels of corruption severity in Table 3. The general trend emerges: the corruption level goes higher, the benefit of negative examples becomes larger. This suggests our proposed negative examples are especially useful when data is highly corrupted.
5.3 Complementary to batch-based negative examples in contrastive loss
In addition, we also investigate if our proposed basch- negative augmentation can be incor- porated into the traditional contrastive loss formulation and if they are complementary and provide additional value on top of the batch-based negative examples traditionally used (Oord et al., 2018; Chen et al., 2020a). We will first describe how to use contrastive loss as regularization term and then introduce how to integrate patch-based negative augmentation into contrastive loss.
7

---

Table 4: Complementary to basch-based negative examples in contrastive loss. Top-1 accuracies of VIT-B/16 pretrained and fine-tuned on ImageNet- 1k with and without P-Rotate.
Model
ImageNet-Ik Ik
ImageNet-A
ImageNet-C
ImageNet-R
ViT-B/16 + Contrastive*
78.7
8.1
53.5
22.8
ViT-B/16 P-Rotate Contrastive
78.9
8.6 (+0.5)
54.1 (+0.6)
23.6 (+0.8)
Rand- Aund-Atgeett-- + Contrastive*
79.7
8.9
57.6
24.7
Rand- Rand-Augment P-Rotate / Contrastive
79.9
9.4 (+0.5)
58.4 (+0.8)
25.4 (+0.7)
Table 5: Patch-based negative augmentation helpful even with large-scale pretraining. Top-1 accuracies of ViT-B/16 pretrained on ImageNet-2IK and finetuned on ImageNet-1k, lk. standaa deviation are reported over independent runs.
Model
ImageNet-Ik
ImageNet-A
ImageNet-C
ImageNet-R
Rand Augment (Cubuk et al., 2020)
84,4+0.0
28.7=:0.2
67.2+0.0
38.7+0.1
+ P-Shuffle/ Uniform
84.5
29.9
67.7
38,9
+ P-Shuffle L2
84.5
29.710.3
68.0+1.0
39.6:10.2
Following supervised contrastive loss proposed in (Khosla et al., 2020), for an example æ; E B, we create positive set P, {ær, BB, {2;; Hy, = y:) with all the examples in the minibatch IB sharing the same class as æ;. The anchor æ; is excluded from its positive set IP, Next, all the examples in the minibatch IB with a different class æ; are used negative examples. Let the candidate set Q = IB [ the contrastive loss can be expressed as
Loont (B;0)
1
1
Σ log Exreo exp(sim(2; xT)/T)²
exp(sim(2,,
(4)
B
Psl
BIEB
æyEP.
where 7 is the temperature and sim (a, x;)) [o(=;:0) [|s(")00]I computes the cosine similarity between g( 0) and g(æy;; 0), and g(:; 0) denotes the representation learned by the penultimate layer of the classifier. We do not use a learnable projection head as in contrastive representation learning (Chen al., 2020a,b) to avoid extra network parameters. The contrastive loss Ccont used as the regularization term Cneg in Eqn 1. We denote this stronger baseline as "Contrastive"",
g(ap.:0T (((
To integrate our proposed patch-based negative examples into contrastive loss, we expand the neg- ative set in contrastive loss, which now composed two types of negative examples: 1) all the examples in the minibatch B with different class 2;, 2) the patch-based negatively transformed in total have B| |Pll negative pairs, where [B] is the batch size and [P:] is the cardinality of the positive set P.. Accordingly, the candi- date set in Eqn 4 becomes Q = B U B) {x; In Table 4, we can see that even if we add the patch- based negative augmentation on top of this stronger contrastive baseline, we can still consistently achieve extra improvement across robustness benchmarks. This shows our proposed patch-based negative augmentation also complementary to batch-based negative examples in contrastive loss in improving models' robustness. Similar trend holds on other settings, see Table 14 in Appendix. Comparing to the other two negative loss regularizations, uniform and lg loss, we suggest readers to use our proposed contrastive loss in practice since it incorporates the extra benefit of constraining the embeddings of positive pairs to be similar and consistently performs better.
images Iे B. Therefore, for each anchor I,,
5.4 Robustness improvements even under larger pre-training datasets
We further investigate if our proposed method can scale up to larger datasets and continues to be necessary and valuable. To this end, we test if our proposed patch-based negative augmentation still helps robustness when models pre-trained on ImageNet-21k (10x larger than ImageNet- 1k), where the baseline performance is higher. Take P-Shuffle as an example, we display the results in Table with negative augmentation both pre-training and fine-tuning stages. We that even when the pre-training dataset is significantly increased, our patch-based negative augmentation can still further improve the robustness of VIT. This demonstrates that approach is valuable at scale and improves models' robustness from an angle orthogonal to larger pre-training data.
Considering training models from scratch ImageNet-Ik Ik 21k is very costly, we take the strongest baseline as an example to independently run multiple times. As shown in Table 5, the small standard deviations suggests the improvement is statistically significant. In addition, we have presented 20+ different experimental settings in the paper where our proposed patch-based negative examples consistently improves robustness.
Mean and standard deviation:
8

---

5.5 Patch-based negative augmentation reduces texture bias
Geirhos et al. (2018) observed that unlike humans, CNNs rely more local information (e.g., texture) rather than more global information (e.g.. shape) to make a classification. Since our patch- based transformations largely destroy the global structure (e.g., shape), we want to investigate if the non-robust features surviving patch-based transformation overlap with local texture biases. To this end, evaluate ViT-B/16 trained patch-based transformations on Conflict Stimuli bench- mark (Geirhos et al., 2018), and we see that ViT's trained only on patch-based transformation have 4.9pp to 31 31. increase on texture bias (Figure 5 in Appendix). This suggests that the useful but non-robust features preserved in patch-based transformation are indeed overlapped with the local texture bias. In addition, using our patch-negative augmentation can also some extent reduce models' reliance on local texture bias, e.g., we decrease the texture accuracy from 7 71.7% to 66.5% for ViT-B/16 (Table Appendix).
Discussions
Does VIT become more robust w.r.f. transformed images? We further evaluate ViTs trained with our robust training algorithms on the patch-based transformed images. We found all three losses on negative views can successfully reduce the prediction accuracy of ViTs to be close random guess with the original semantic classes as the ground-truth. In other words, our robust training algorithms make ViTs behave similarly as humans on those patch-based transformed images.
Are high confidence predictions w.r.t. patch trans- formed images due to non-overlapping patch embed- dings? To answer this, we test original ViT from Doso- vitskiy et al. (2021) on the eatch-transformed images where the patch size of the patch-based transformations ps; not perfectly aligned with the input patch size p8;, e.g., ps, is not a multiple of ps;. This can approximate the scenario when patch embeddings are extracted from over- lapped patches. As shown in Table 11 in Appendix, when the patch size of the pateh-based transformation (ps, = 24) and the input patch size (ps, = 16) of ViT are not perfectly aligned, the test accuracy on the patch-based shuffled images decreases from 84. % to 32.9%. This is
VIT- B/16 Attention map
P-Rotate/ Uniform Allention map
Jellyfish
Paper Towel
Jellyfish
Shiold
Fire Scroen
Shiold
lower than the case that the patch sizes are aligned, e.g.. the test accuracy is 57.8% when ps, 32, but still sig- nificantly higher than humans (a random guess is close to 0.1%) well comparable CNN- based network 9.2%. Therefore, we conjecture the high confidence w.r.t. trans- formed images partially from non-overlapping patch emhedding in ViT's.
Axclol
Goldfish
Axcloti
Starfish
Ocarina
Starfish
Attention visualization of images misclassified by ViT-B/16 (mid- dle column) and correctly classified by patch-based negative augmentation (P- Rotate) using uniform loss (third col- umn) on ImageNet-R. The ground truth (first column) or the predicted class (second and third column) are displayed the bottom of each image.
Figure 4:
When does patch-based negative augmentation help? To answer this question, we follow (Dosovitskiy et al., 2021) visualize the attention maps of models trained with/without our proposed negative data augmentation. In Figure 4, can effectively 1) help models attend to the object to make a correct prediction whereas the standard ViT fails (top two rows), and 2) mitigate models' reliance on unrobust biases, e.g., local bias, color bias, texture bias, etc. For example, the cartoon Axolotl (third row) is misclassified by the standard ViT as Goldfish due to color similarity and the Starfish (bottom row) is incorrectly recognized as Ocarina because of local resemblance. This is aligned with other signals in previous sections that standard ViT relies on unrobust biases to make a prediction, whereas our negative augmentation effectively encourages the model to rely less on these unrobust features. In- terestingly, for the bottom two rows, the standard ViT has successfully attended to the object in both cases but still failed to make correct predictions, indicating that the attention on the object does not ensure model is relying on semantically meaningful and robust features to make a prediction.
see that our negative augmentation

---

7 Related Work
Vision transformers (Dosovitskiy et al., 2021; Touvron et al., 2021) are a family of Transformer models ( (Vaswani et al., 2017) that directly process visual tokens constructed from image patch em- bedding. Unlike convolutional neural networks (LeCun et al., 1989; Krizhevsky et al., 2012: He et al., 2016) that assume locality and translation invariance in their architectures vision transform- ers have no such assumptions and can exchange information globally. A few recent studies find pretrained vision transformers are at least as robust as the ResNet counterparts (Bhojanapalli et al., 2021), and possibly more robust (Naseer et al., 2021; Paul & Chen, 2021). Our work studies a specific aspect of robustness pertaining patch-based visual tokens in VIT, and show it may lead to a generalization gap. Different from (Naseer et al., 2021) which also shows ViTs are insensitive to patch operations such as shuffle and occlusion, we design different types of patch-based transforma- tions and develop deeper understanding that the features preserved in patch- patch-based transformations non-robust. Further, propose mitigation strategy based on these patch-based transforma- tions to increase robustness of vision transformers. Another orthogonal line of improving robustness of vision transformers is to develop deeper understanding of the self-attention mechanisms in vi- sion transformers (Gu et al., 2022) and further advances the neural network architectures (Zhou al., 2022).
Data augmentation widely used in computer vision models to improve model perfor- mance (Howard, 2013; Szegedy et al., 2015; Cubuk et al., 2020, 2019; Touvron et al., 2021). How- ever, most of the existing data augmentations are "positive" semantic being preserved after the transformation. In this work, we explore "negative" data aug- mentation operations based on patches, where we encourage the representations of transformed example to be different from the original ones. Most related to work in this direction is the work of Sinha et al. (2020). Although the concept of negative augmentation was proposed in their work, they only apply it for generative and unsupervised modeling without consideration of robustness. In contrast, our work focuses on discriminative and supervised modeling, and demonstrate how such negative examples can reveal specifie robustness issues and such augmentation approaches can offer robustness improvements under large-scale pretraining settings.
the sense they assume the class
Our work is also related to contrastive learning (Wu et al., 2018; Hjelm et al., 2019; Oord et al., 2018; He et al., 2020: Tian et al., 2020). The increasing number of negative pairs has shown to be important for representation learning in self-supervised contrastive learning (Chen et al., 2020a), where different images serve as negative examples for each other, and supervised contrastive learn- ing (Khosla al., 2020), where images with different classes used negative examples. Unlike the traditional setting of representation learning, our proposed contrastive loss serves as a regular- ization term with patch-based negative augmentations extra negative data points.
Conclusion
Through this research we have found concrete evidences of ViTs relying non-robust features to make predictions and shown that this reliance is limiting out-of-distribution robustness. This opens multiple exciting new lines of research. First, our methodology for analyzing the robustness of ViTs provides a valuable recipe for future research. Through designing patch-wise, semantie-destroying transformations that ViTs are insensitive to. identified which non-robust features models rely on. Second, through negative augmentations during training we reduced the ViTs' reliance on such non-robust features, and improved the out-of-distribution performance of ViTs significantly, with- out harming in-distribution accuracy. This approach shows the potential for further improving the robustness of ViTs. While we have identified multiple such non-robust features in ViTs, we be- lieve discovering and addressing more is a promising direction for continued progress toward robust vision transformers and vision models in general.
Acknowledgements
The authors would like to thank Alexey Dosovitskiy, Lucas Beyer and Neil Houlsby for helpful discussions on vision transformer systems. We also want to thank the reviewers for their useful comments.
10

---

References
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and An- dreas Veit. Understanding robustness of transformers for image classification. arXiv preprint arXiv:2103. 14586, 2021.
Wieland Brendel and Matthias Bethge. Approximating cnns with bag-ofellcallfeettre models works surprisingly well on imagenet. In The International Conference on Learning Represen- tations, 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International Conference on Machine Learning, pp. 1597-1607. PMLR, 2020a.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self-supervised models are strong semi-supervised learners. Advances in Neural Information Processing Systems, 2020b.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 113-123, 2019.
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 702-703, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An im- age is worth 16x 16 words: Transformers for image recognition at scale, International Conference on Learning Representations, 2021.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. In International Conference on Learning Representations, 2018.
Jindong Gu. Volker Tresp, and Yao Qin. Are vision transformers robust to patch perturbations? In European Conference on Computer Vision, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Kaiming He. Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In International Conference on Learning Representations, 2019a.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In Proceedings of the International Conference Learning Repre- sentations, 2019b.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. arXiv preprint 1907.0/ 2019.
Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty. In International Conference on Learning Representations, 2020.
Dan Hendryeks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: critical analysis of out-of distrriiittiitoo generalization. Interna- tional Conference on Computer Vision, 2021.

---

Katherine L Hermann, Ting Chen, and Simon Korbllith. The origins and prevalence of texture bias in convolutional neural networks. In Advances In Neural Information Processing Systems. 2020.
R Devon Hjelm, Alex Fedorov, Samuel Lavoie- Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. International Conference on Learning Representations, 2019.
Andrew G Howard. Some improvements on deep convolutional neural network based image classi- fication. arXiv preprint arXiv: 2013.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373, 2018.
Fereshte Khani and Percy Liang. Removing spurious features can hurt accuracy and affect groups disproportionately. In Proceedings of the 2021 ACM Conference Fairness, Accountability, and Traanppareny, pp. 196-205, 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances In Neural Information Processing Systems, 2020.
Diederik P. Kingma and Jimmy Ba. 0bss/412.6990, 2015.
Adam: A method for stochastic optimization. CoRR,
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical reporr, University of Toronto, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances in Neural Information Processing Systems, 25:1097-1105, 2012.
Yann LeCun, Bernhard Boser, John S Denker; Donnie Henderson, Richard Howard, Wayne Hub- bard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neurul Computation, 1(4):541-551, 1989.
Xiaofeng Mao, Gege Qi, Yuefeng Chen, Xiaodan Li, Shaokai Ye, Yuan He, and Hui Xue. Rethinking the design principles robust vision transformer. arXiv preprint arXiv:2105.07926. 2021.
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. arXiv preprint 1054997.05. 2021.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arX1v:1807.03748, 2018
Sayak Paul and Pin-Yu Chen. Vision transformers are robust learners. arXiv preprint 105.0:2105.07581 2021.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-255 2015. doi: 10.100 022---1---816-...
Abhishek Sinha, Kumar Ayush, Jiaming Song, Burak Uzkent, Hongxia Jin, and Stefano Ermon. Negative data augmentation. In International Conference on Learning Representations, 2020.
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2 106. 10270, 2021.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
12

---

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9, 2015.
Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In European Conference on Computer Vision, pp. 776-794. Springer, 2020.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference Machine Learning, pp. 10347-10357. PMLR, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, pp. 5998-60980 6008, 2017.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 3733-3742, 2018.
Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise signal: The role of image backgrounds object recognition. International Conference on Learning Representations, 2021.
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers, 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning. 2019.
Daquan Zhou, Zhiding Yu, Enze Xie, Chaowei Xiao, Animashree Anandkumar, Jiashi Feng, and Jose M Alvarez. Understanding the robustness in vision transformers. In International Conference on Machine Learning, pp. 27378-27399 PMLR, 2022.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A] Not
Applicable
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or URL)? [Yes] See Supple- mentary Materials.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix.
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Section 5.4.
(d) Did you include the total amount of compute and the type of resources used type of GPUs, internal cluster, cloud provider)? [No]
13

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're re using/curatingg [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "V5YLegKmexW", "reviewer": "Reviewer_9VjA", "summary": "This paper studies the robustness of ViT models. The authors find that ViT models can keep high in-distribution accuracy even when training on images with heavy patch-based transformations such as shuffle, rotate, and infill, which indicates that ViTs rely on the features that survived these transformations. However, these features are non-robust and render a significant accuracy drop on the out-of-distribution datasets. Based on such observation, the authors propose negative augmentation to improve the model robustness against data distribution shift. The algorithm first generates negatively augmented training samples using the patch-based transformation. It then incorporates the Uniform loss and L2 loss to regularize the training away from learning these non-robust features. The authors conduct extensive experiments to demonstrate the effectiveness of the proposed approach. The negative augmentation is complementary to existing positive augmentation such as AugMix and batch-based negative examples in contrastive loss, and can further improve the robustness on top of that.", "questions": "1. In equation 1, for the loss on the negative augmented training samples, is it L_{neg}(B, \\tilde(B); \\theta) or L_{neg}(\\tilde(B); \\theta)? Do you use the clean dataset twice in the training? If so, what is the reason for doing so? \n\n2. For the experiments that use Uniform loss, how will the performance change if you use the regular cross-entropy loss on the negative augmented examples?\n\n3. In Table 1, Uniform loss achieves the best performance on ViT-B/16 while the L2 loss achieves the best performance on ViT-B/32, what is the reason behind this? In practice, how do we decide which loss to use? Also, is there any strategy to pick the best type of patch-based transformation for negative augmentation?\n\n4. Did you try to train models with the combination of Uniform loss and L2 loss, will that further improve the robustness or undermine the robustness? \n\n5. Can you please explain the setting difference between lines 306-307 and lines 133-135?\n\n6. Another related direction to enhance model robustness is to improve the model architecture (e.g. Daquan Zhou et.al, Understanding The Robustness in Vision Transformers). It would be desirable if the authors could discuss this orthogonal research direction in the related work.\n\n7. A small typo in line 35: showingthat --> showing that", "limitations": "Please refer to the questions above.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper studies an important topic --  model robustness against OOD data, and proposes an effective approach to improve the robustness. \n2. The authors demonstrate an intriguing property of ViTs that they are resilient to patch-based transformations. They further show that the features that survived such transformations are not robust, which is a novel discovery. \n3. Based on the observation they propose negative augmentation and two losses to train models away from using the non-robust features. The proposed approach is technically sound and easy to understand.\n4. The authors perform extensive experiments to evaluate the effectiveness of the proposed method, the results are convincing and support their claims.\n5. The authors also show that the proposed method can collaborate with positive data augmentation and batch-based negative examples in contrastive loss and further improve the robustness.\n6. The paper is well-organized and easy to follow.\n\nWeakness:\n1. For some experiment results, the authors didn't explicitly explain the rationale and possible reasons. Please refer to the Questions for details.\n2. Missing discussion about related works in the same field. Please refer to the Questions for details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "O8HCqUbkjIK", "reviewer": "Reviewer_QKPp", "summary": "This is an interesting paper that discusses the robustness of the Transformer’s patch-wise architectural structure. The authors experimentally show that patch-based transformations largely destroy images to be unrecognizable to humans, but ViT recognizes them as the original class (e.g., keeshond or magpie) with high confidence as shown in Figure 1. Based on this investigation, they proposed patch-based negative augmentation and experimentally demonstrated that it consistently improves robustness of ViTs on ImageNet based robustness benchmarks across 20+ different experimental conditions.", "questions": "Please answer to my comments above.", "limitations": "Nothing special.", "rating": 5, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n+ This is an interesting paper that discuss its robustness to the patch-wise data destroy. The motivation is very clear.\n+ The patch-based negative augmentation based on the above observation is interesting and reasonable.\n+ The experimental validation is throughout.\n\nWeakness:\n- The performance improvement from the baseline is not very large. Therefore, I admit that the concept itself is interesting, but I wonder how much this negative augmentation would give impacts to the community.\n- Although it is claimed that the proposed method outperforms the baseline, the best condition differs from dataset to dataset. The authors should discuss the theoretical background of which approach is better in which case. If the authors could provide a strategy to choose the best setting, that would be great.\n- The authors may want to show some specific cases where the proposed method helps a lot, not only discussing the average accuracy.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GMGFhCnP1a5", "reviewer": "Reviewer_BLbX", "summary": "In this paper, the authors use the special patch-based data processing pipeline of ViTs to study their robustness. Authors find that ViTs tend to overfit on some unrecognizable features in the patch, making them not robust to distribution shifts. To alleviate this issue, the authors propose to use patch-based negative augmentations to train the ViTs and boost their robustness. Experiments show that the proposed patch-based negative augmentations can boost the robustness of ViTs and are complementary to traditional (positive) data augmentation techniques and batch-based negative examples in contrastive learning. ", "questions": "- A curious question is whether the observed phenomenon exists in other vision tasks (e.g., detection and segmentation)? ", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work. ", "rating": 5, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The observation on overfitting and its reason is interesting. \n- The proposed method is simple and effective in boosting the trained ViTs robustness. \n- Detailed evaluations are provided to show the performance improvement. \n- Writing is clear and easy to follow \n\nWeakness:\n- Only evaluated on traditional ViTs, how about more recent ViTs? Will more intense use of convolution layers alleviate such overfitting issues mentioned? ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "FeT0A_dDohk", "reviewer": "Reviewer_5kdm", "summary": "This paper investigates a small but non-obvious problem: vision transformers (ViTs) are known to be non-sensitive to patch shuffling, in contrast to humans. Could we make ViTs more robust by making them sensitive to patch shuffling (and other patch-based transformations)? The authors performed extensive experiments, and the benefits of doing so is decent and consistent across various settings. ", "questions": "1. Is the negative augmentation done to every batch, or every N batches? \n2. It will help readers understand whether/how the ViT attends to different patterns after applying negative augmentation, using some attribution visualizations.\n\n", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The investigated problem involves a basic aspect of ViT training. The benefits of this method can be cheaply incorporated to various downstream tasks.\n2. The experiments are very extensive and convincing.\n\nWeaknesses:\n1. Some of the most significant improvements are achieved with patch-rotate (e.g. Table 1). But the insensitivity of vit to patch-rotate seems to be a result of the \"whole-image rotation\" augmentation (part of RandAug)? This is quite different from the insensitivity of vit to patch-shuffling, which is due to the invariance of transformers to token orders. So I think patch-shuffling is more inherent and more interesting to vits. \n2. I'm still not totally sure that the shuffle invariance of ViTs has only negative effects and should be completely eliminated. For example, in MAE [a], ViTs can recognize an image even when most patches are masked. Given this, it's natuaral that when patches are randomly shuffled, ViTs can still recognize the image. Therefore, I'm a bit concerned that maybe there are deeper explanations of the shuffle invariance, instead of just \"exploiting texture/local patterns biases\". \n\n[a] Masked Autoencoders Are Scalable Vision Learners Kaiming, CVPR 2022.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept