Shift Invariance Can Reduce Adversarial Robustness

1 Introduction . In adversarial attacks ( Szegedy et al. , 2013 ) against classifiers , an adversary with knowledge of the trained classifier makes small perturbations to a test image or even to objects in the world ( Eykholt et al. , 2018 ; Wu et al. , 2020b ) that change the output . Such attacks threaten the deployment of deep learning systems in many critical applications , from spam filtering to self-driving cars . Despite a great deal of study , it remains unclear why neural networks are so susceptible to adversarial attacks . We show that invariance to circular shifts in Convolutional Neural Networks ( CNNs ) can be one cause of this lack of robustness . All reference to shifts will refer to circular shifts . To motivate this conclusion we study in detail a simple example . Indeed , one of our contributions is to present perhaps the simplest possible example in which adversarial attacks can occur . Figure 1 shows a two class classification problem in which each class consists of a single image , a white or black dot on a gray background . We train either a fully connected ( FC ) network or a CNN that is designed to be fully shift-invariant to distinguish between them . Since each class contains only a single image , we measure adversarial robustness as the l2 distance to an adversarial example produced by a DDN attack ( Rony et al. , 2019 ) , using the training image as the starting point . The figure shows that the CNN is much less robust than the FC network , and that the robustness of the CNN drops precipitously with the image size . In Sections 3 and 4 we explain this result theoretically . In Section 3 we study the effect of shift invariance on the margin of linear classifiers , for linearly separable data . We call a linear classifier ∗Equal contribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . ( a ) Clean Samples ( b ) Adversarial Examples FCCNN ( c ) Distance to generate adversarial examples vs. image width 𝑑 CNNFC CNTK ( ൗ1 𝑑 ) Figure 1 : In a binary classification problem , with each image as a distinct class ( a ) , a FC network requires an average L2 distance close to 1 to construct an adversarial example while a shift-invariant CNN only requires approximately 1√ d , where d is the input dimension . ( b ) -left shows adversarial examples for the CNN . On the top , for example , the adversarial example looks just like the clean sample for the white dot class , but is classified as belonging to the black dot class . ( b ) -right shows that `` adversarial '' examples for the FC network just look like gray images . ( c ) shows accuracy under an adversarial attack , where the L2 norm of the allowed adversarial perturbation is shown in the horizontal axis . The curve labeled CNN shows results for a real trained network , while CNTK shows the theoretically derived prediction of 1√ d for an infinitely wide network . shift invariant when it places all possible shifts of a signal in the same class . We prove that for a shift invariant linear classifier the margin will depend only on differences in the DC ( constant ) components of the training signals . It follows that for the two classes shown in Figure 1 , the margin of a linear , shift invariant classifier will shrink in proportion to 1√ d , where d is the number of image pixels . Next , in Section 4 , we prove that under certain assumptions , a CNN whose architecture makes it shift invariant produces a linear decision boundary for the example in Figure 1 , explaining its lack of adversarial robustness . We draw on recent work that characterizes infinitely wide neural networks as kernel methods , describing FC networks with the neural tangent kernel ( NTK ) ( Jacot et al. , 2018 ) and CNNs with a convolutional neural tangent kernel ( CNTK ) ( Arora et al. , 2019 ; Li et al. , 2019 ) . Our proof uses these kernels , and applies when the assumptions of this prior work hold . However , the results in Figure 1 are produced with real networks that do not satisfy these assumptions , suggesting that our results are more general . We feel that it is valuable to produce such a simple example in which adversarial vulnerability provably occurs and can be fully understood . Still , it is reasonable to question whether shift invariance can affect robustness in networks trained on real data . In Section 5 we show experimentally that it can . First , we compare the robustness of FC and shift-invariant networks on MNIST and Fashion-MNIST . On these smaller datasets , FC networks attain reasonably good test accuracy on clean data . We find that indeed , FC networks are also quite a bit more robust than shift-invariant CNNs . We also show that if we reduce the degree of shift invariance of a CNN , we increase its adversarial robustness on these datasets . Next we show that on the SVHN dataset , FC networks are more robust than ResNet architectures suggesting that this phenomenon extends to realistic convolutional networks . Finally , we consider industrial strength networks trained on CIFAR-10 and ImageNet . We note that ResNet and VGG are shift-invariant to a degree , while AlexNet , FCNs and transformer based networks are much less so . This suggests that ResNet and VGG will also be less robust than AlexNet , FCNs , and transformer networks , which we confirm experimentally . Finally , we use simple synthetic data to explore the connection between shift invariance and robustness . We show that when shift invariance effectively increases the dimensionality of the training data , it produces less robustness , and that this does not occur when shift invariance does not increase dimensionality . This is in line with recent studies that connect higher dimensionality to reduced robustness . However , we also show , at least in a simple example , that higher dimensionality only affects robustness when it also affects the margin of the data . The main contribution of our paper is to show theoretically that shift invariance can undermine adversarial robustness , and to show experimentally that this is indeed the case on realworld networks .. This enhances our understanding of robustness , and suggests that we can improve robustness by judiciously reducing invariances that are not needed for good performance . Our code can be found at https : //github.com/SongweiGe/shift-invariance-adv-robustness . 2 Related Work . In this section we survey prior work on the origin of adversarial examples and briefly discuss work on shift-invariance in CNNs , and on the NTKs . 2.1 The Cause of Adversarial Examples . One common explanation for adversarial examples lies in the curse of dimensionality . Goodfellow et al . ( 2015 ) argues that even with a linear classifier a lack of adversarial robustness can be explained by high input dimension . They suggest that large gradients naturally occur as the input dimension increases , assuming that the magnitude of weights remains constant . Following this , ( Simon-Gabriel et al. , 2019 ) provides theoretical and empirical evidence that gradient magnitude in neural networks grows as O ( √ d ) instead of O ( d ) , where d is the input dimension . Gilmer et al . ( 2018 ) analyzes the case of two classes consisting of concentric spheres , and shows that adversarial examples arise when the spheres are of high dimension . Shafahi et al . ( 2019 ) provides a general discussion of how high dimensional data can lead to adversarial examples can be inevitable by showing that a small perturbation in a high dimensional space can cover the whole space . Khoury & Hadfield-Menell ( 2019 ) argues theoretically and empirically that adversarial examples arise when data lies on a lower dimensional manifold in a high dimensional space . Other studies show that adversarial examples could arise due to properties of the data . Wang et al . ( 2020 ) argues that adversarial examples can be explained by the dependence of the model on the high frequency components of the image data . Tsipras et al . ( 2019 ) shows that to achieve high accuracy one must sacrifice robustness due to biases in the features . Follow-up work ( Schmidt et al. , 2018 ) shows that the sample complexity requirements for an adversarially robust classifier to achieve good generalization are much higher than for a normal classifier . Furthermore , a recent study ( Chen et al. , 2020 ) shows that in some cases more data actually undermines adversarial robustness and also provides a more complete picture of this line of research . Apart from the data , some papers look for explanations in the model . Nakkiran ( 2019 ) conjectures that the reason adversarial robustness is hard to achieve is that current classifiers are still not complex enough . Shah et al . ( 2020 ) shows that in many cases networks learn simple decision boundaries , with small margins , rather than more complex ones with large margins , which induces adversarial vulnerability . Daniely & Shacham ( 2020 ) shows that for most ReLU networks with random weights , most samples x admit an adversarial perturbation at distance O ( ‖x‖d ) , where d is the input dimension . Shamir et al . ( 2019 ) shows that for ReLU networks , adversarial examples arise as a consequence of the geometry of Rn and show they can find targeted adversarial examples with m l0 distance , when networks are designed to distinguish between m classes . Galloway et al . ( 2019 ) provides empirical evidence that using batch normalization can reduce adversarial robustness . Madry et al . ( 2018 ) and Wu et al . ( 2020a ) have discussed the relationship between robustness and network width . Kamath et al . ( 2020 ) examine the interplay between rotation invariance and robustness . They empirically show that by inducing more rotational invariance using data augmentation in CNNs and Group-equivariant CNNs the adversarial robustness of these models decreases . This work is especially relevant to our results that shift invariance can reduce robustness ( A quite different notion of invariance is also explored in Jacobsen et al . ( 2019 ) and subsequent work ) . 2.2 CNNs and Shift-Invariance . Several papers have pointed out that the modern CNN architectures are not fully shift invariant for reasons such as padding mode , striding , and padding size . ( Zhang , 2019 ) improves the local shift invariance to global invariance with the idea of “ antialiasing ” . ( Kayhan & Gemert , 2020 ) proposes to undermine the ability of CNNs to exploit the absolute location bias by using full convolution . ( Chaman & Dokmanic , 2021 ) proposes the use of a novel sub-sampling scheme to remove the effects of down-sampling caused by strides and achieve perfect shift invariance . ( Alsallakh et al. , 2021 ) show padding mechanism can cause spatial bias in CNNs . ( Azulay & Weiss , 2019 ) relates a lack of shift invariance to a failure to respect the sampling theorem . 2.3 Neural Tangent Kernel . Neural networks are usually highly non-convex . But recent studies have shown that when the networks are over-parameterized , their parameters stay close to the initialization during the training , which makes the analysis of the convergence and generalization of neural networks tractable ( Allen-Zhu et al. , 2019a , b ; Du et al. , 2018 , 2019a ) . When optimizing the quadratic loss using gradient descent , the dynamics of an infinitely wide neural network can be well described by a kernel regression , using the Neural Tangent Kernel ( NTK ) ( Jacot et al. , 2018 ) . This kernel only depends on the network architecture and initialization . Subsequent studies extend NTK beyond the fully connected network to various well-known architectures ( Yang , 2020 ) including using the CNTK kernel for Convolutional Neural Networks ( Arora et al. , 2019 ; Li et al. , 2019 ) , the GNTK for Graph Neural Networks ( Du et al. , 2019b ) , and the RNTK for Recurrent Neural Networks ( Alemohammad et al. , 2021 ) . Earlier works with these kernels have shown a superior performance over traditional kernels ( Arora et al. , 2020 ) while later theoretical and empirical studies show that NTKs are closely related to traditional kernels ( Geifman et al. , 2020 ; Chen & Xu , 2021 ) . The salient advantage of NTK is its ability as a tool to deliver tractable analysis of deep neural networks . Many interesting studies have been done to demystify the puzzling behaviors of deep neural networks using this tool . ( Basri et al. , 2019 ; Rahaman et al. , 2019 ; Basri et al. , 2020 ) study the inductive bias of the neural network in terms of the frequencies of the learned function . ( Tancik et al. , 2020 ) further apply these discoveries and random fourier features to empower neural networks to learn high-frequency signals more easily . ( Huang et al. , 2020 ) uses NTK to understand the importance of the residual connection proposed in ResNet ( He et al. , 2016 ) . In this paper , we use NTK and CNTK to help us understand the behavior of FC and Convolutional Neural Networks in toy examples and draw a connection between CNN and adversarial examples .

Summary: The adversarial attack is one of the critical problems in current machine learning research that causes end-users to mistrust models deployed for high-stakes decision-making applications. The paper identifies the "shift-invariance" property of neural networks to be a possible reason behind their high sensitivity to adversarial attacks. The paper shows that the margin between the classes for a shift-invariant linear classifier only depends on the DC component of the signals. Further, the authors show that in simple cases,  fully connected and shift-invariant neural networks learn linear decision boundaries. Finally, experimental evaluations on real datasets and architectures show that shift invariance reduces adversarial robustness.