Invariance vs Robustness of Neural Networks

1 INTRODUCTION . Neural networks achieve state of the art accuracy on several standard datasets used in image classification . However , their performance in the wild depends on how well they can handle natural or non-adversarial transformations of input seen in real-world data as well as known deliberate , adversarial attacks created to fool the model . Natural or non-adversarial transformations seen in real-world images include translations , rotations , and scaling . Convolutional Neural Networks ( CNNs ) are translation-invariant or shift-invariant by design . Invariance to other symmetries , and especially rotations , have received much attention recently , e.g. , Harmonic Networks ( H-Nets ) by Worrall et al . ( 2016 ) , cyclic slicing and pooling by Dieleman et al . ( 2016 ) , Transformation-Invariant Pooling ( TI-Pooling ) by Laptev et al . ( 2016 ) , Group-equivariant Convolutional Neural Networks ( GCNNs ) by Cohen & Welling ( 2016 ) , Steerable CNNs by Cohen & Welling ( 2017 ) , Deep Rotation Equivariant Networks ( DREN ) by Li et al . ( 2017 ) , Rotation Equivariant Vector Field Networks ( RotEqNet ) by Marcos et al . ( 2017 ) , Polar Transformer Networks ( PTN ) by Esteves et al . ( 2018 ) . For a given symmetry group G , a G-equivariant network learns a representation or feature map at every intermediate layer such that any transformation g ∈ G applied to an input corresponds to an equivalent transformation of its representations . Any model can improve its invariance to a given group of symmetries through sufficient training augmentation . Equivariant models use efficient weight sharing and require smaller sample complexity to achieve better invariance . Equivariant models such as CNNs and GCNNs too generalize well to progressively larger random rotations , but only when their training data is augmented similarly . Adversarial attacks on neural network models are certain , deliberate changes to inputs that fool a highly accurate model but are unlikely to fool humans . Given any neural network model , Szegedy et al . ( 2013 ) show how to change the pixel values of images only slightly so that the change is almost imperceptible to human eye but makes highly accurate models misclassify . Szegedy et al . ( 2013 ) find these adversarial pixel-wise perturbations of small magnitude by maximizing the prediction error of a given model using box-constrained L-BFGS . Goodfellow et al . ( 2015 ) propose Fast Gradient Sign Method ( FGSM ) that applies the adversarial perturbation as x′ = x+ sign ( ∇xJ ( θ , x , y ) ) , where x is the input , y represents the target , θ represents the model parameters , and J ( θ , x , y ) is the loss function used to train the network . Moreover , Goodfellow et al . ( 2015 ) propose adversarial training , or training augmented with points ( x′ , y ) , as a way to improve adversarial robustness of a model . Subsequent work has introduced multi-step variants of FGSM , notably , an iterative method by Kurakin et al . ( 2017 ) and Projected Gradient Descent ( PGD ) attack by Madry et al . ( 2018 ) . Given any model , these attacks produce adversarial perturbation for every test image x from a small ` ∞-ball around it , namely , each pixel value xi is perturbed within [ xi − , xi + ] . PGD attack does so by solving an inner optimization by projected gradient descent over ` ∞-ball of radius around x to approximate the optimal perturbation . Adversarial training with PGD perturbations improves the adversarial robustness of models , and it is one of the best known defenses to make models robust to perturbations of bounded ` ∞ norm on MNIST and CIFAR-10 datasets , as shown in Madry et al . ( 2018 ) and Athalye et al . ( 2018 ) . Recent work has looked at simultaneous robustness to multiple adversarial attacks . Engstrom et al . ( 2017 ) show that adversarial training with PGD makes CNNs robust against perturbations of bounded ` ∞ norm but an adversarially chosen combination of a small rotation and a translation can still fool these models nevertheless . Recent work of Schott et al . ( 2019 ) shows that PGD adversarial training is a good defense against perturbations of bounded ` ∞ norm but can be broken with adversarial perturbations of small ` 0 or ` 2 norm that are also imperceptible to humans or have little semantic meaning for humans . Schott et al . ( 2019 ) show how to build models for MNIST dataset that are simultaneously robust to perturbations of small ` 0 , ` 2 and ` ∞ norms . 1.1 PROBLEM FORMULATION AND OUR RESULTS . Let f be a neural network classifier trained on a training set of labeled images . The accuracy of f is the fraction of test inputs x for which the predicted label f ( x ) matches the true label y . For a given transformation T of the image space , f is said to be T -invariant if the predicted label remains unchanged after the transformation T for all inputs , i.e. , f ( Tx ) = f ( x ) , for all inputs x . If f is T -invariant and f ( x + a ) 6= f ( x ) , for some input x and its adversarial perturbation a with ‖a‖p ≤ , then by invariance f ( Tx + Ta ) = f ( x + a ) 6= f ( x ) = f ( Tx ) . Let T be a translation , rotation , or more generally abstracted as a permutation of coordinates in Tramèr & Boneh ( 2019 ) , then ‖Ta‖p = ‖a‖p ≤ . Hence , Ta is an adversarial perturbation for input Tx of small ` p norm . When a change of variables maps x to Tx by a permutation of coordinates , then the gradient and T operators can be swapped . In other words , grad ( f ) at Tx is the same as T applied to grad ( f ) at x . This gives a 1-1 correspondence between FGSM ( and PGD ) perturbations of x and Tx , respectively , with ` p norm at most . As a corollary , the ` p fooling rate for any T -invariant classifier f on the transformed data { Tx : x ∈ X } must be equal to its ` p fooling rate on the original data X . A subtlety kicks in when f is not truly invariant , that is , f ( Tx ) = f ( x ) , for most inputs x but not all x . Define the rate of invariance of a classifier f to a transformation T as the fraction of test images whose predicted labels after transformation T remains unchanged , i.e. , f ( Tx ) = f ( x ) . For a class of transformations , e.g. , rotations upto degree [ −θ◦ , +θ◦ ] , we define the rate of invariance as the average rate of invariance over transformations T in this class . The rate of invariance is 100 % iff the model f is truly invariant . When f is not truly invariant , the interplay between the invariance under transformations and robustness under adversarial perturbations of small ` p-norm is subtle . This interplay is exactly what we investigate . In this paper , we study neural network models and the simultaneous interplay between their rate of invariance for random rotations between [ −θ◦ , +θ◦ ] , and their adversarial robustness to pixel-wise perturbations of ` ∞ norm at most . Measuring the robustness of a model to adversarial perturbations of ` p norm at most is NP-hard as shown in Katz et al . ( 2017 ) and Sinha et al . ( 2018 ) . Athalye et al . ( 2018 ) compare most of the known adversarial attacks to argue that PGD is among the strongest . Therefore , we use accuracy on adversarially perturbed test data using PGD attack as a proxy for adversarial robustness . Unlike previous studies by Engstrom et al . ( 2017 ) and Tramèr & Boneh ( 2019 ) , we do not fix the magnitude of pixel-wise adversarial perturbations ( e.g. , say = 0.3 ) nor limit ourselves to small rotations up to ±30◦ . Another important difference is we consider random rotations instead of adversarial rotations . We compute the rate of invariance of a given model on inputs rotated by a random angle between [ −θ◦ , +θ◦ ] , for θ varying in the range [ 0 , 180 ] . Similarly , we normalize the underlying dataset , and compute the accuracy of a given model on adversarially perturbed test inputs by PGD and FGSM adversarial attacks of ` ∞ norm at most , for varying in the range [ 0 , 1 ] . We empirically study the following : ( a ) change in ` ∞ adversarial robustness as we improve only the rate of rotation invariance using training augmentation with progressively larger rotations , ( b ) change in invariance as we improve only adversarial robustness using PGD adversarial training with progressively larger ` ∞-norm of pixel-wise perturbations . We study equivariant models , StdCNNs and GCNNs , as well as LeNet and ResNet model from Madry et al . ( 2018 ) . Equivariant models , especially GCNNs , when trained with random rotation augmentations come very close to being truly rotation invariant ; see Cohen & Welling ( 2016 ) . Adversarially trained LeNet and ResNet models with PGD are among the best known ` ∞ adversarially robust models on MNIST and CIFAR-10 , respectively , as shown in Madry et al . ( 2018 ) , and reconfirmed by Athalye et al . ( 2018 ) . In other words , these models essentially represent the two separate solutions known currently for achieving invariance and adversarial robustness , respectively . Our two main observations are as follows . ( i ) Equivariant models ( StdCNNs and GCNNs ) progressively improve their rate of rotation invariance when their training is augmented with progressively larger random rotations but while doing so , their ` ∞ adversarial robustness drops progressively . This drop or trade-off is very significant on MNIST . ( ii ) Adversarially trained LeNet and ResNet models from Madry et al . ( 2018 ) trained using progressively larger ` ∞-norm attacks improve their adversarial robustness but while doing so , their rate of invariance to random rotations upto ±θ◦ drops progressively . On these adversarially trained models we observe that the change in the rate of invariance is marginal for small rotations . We empirically observe that the average distance of the test points to the decision boundary reduces when models are trained with larger rotations . This could explain observation ( i ) . We empirically observe that adversarial perturbations are nearly orthogonal to perturbations caused by small rotations of test points . This could explain the marginal change in the rate of invariance for small rotations , mentioned in ( ii ) . Related Work Schott et al . ( 2019 ) study simultaneous robustness to adversarial perturbations of small ` 0 , ` 2 , and ` ∞-norm . Tramèr & Boneh ( 2019 ) show an impossibility result by exhibiting data distribution where no model can have substantially better-than-random accuracy for binary classification simultaneously against both ` ∞ and ` 1 perturbations , and also against both ` ∞ perturbations and spatial perturbations , given by an adversarial permutation of coordinates that models an adversarially chosen combination of a small rotation and a small translation . They show an empirical validation of this claim on MNIST and CIFAR-10 datasets for simultaneous robustness against ` ∞ adversarial perturbation and an adversarially chosen combination of translation upto ±3 pixels and rotation upto ±30◦ . Intuitively and theoretically , it has been argued in Engstrom et al . ( 2019 ) and Tramèr & Boneh ( 2019 ) that small , adversarial pixel-wise perturbations and small , adversarial geometric transformations are essentially dissimilar attacks focusing on different features , due to which a simultaneous solution to both may be inherently difficult . They do not postulate any gradual trade-off between invariance and robustness . They do not consider group-equivariant models such as GCNNs , a natural choice for invariance to geometric transformations .

Summary: This paper examines the interplay between the related ideas of invariance and robustness in deep neural network models. Invariance is the notion that small perturbations to an input image (such as rotations or translations) should not change the classification of that image. Robustness is usually taken to be the idea that small perturbations to input images (e.g. noise, whether white or adversarial) should not significantly affect the model's performance. In the context of this paper, robustness is mostly considered in terms of adversarial perturbations that are imperceptible to humans and created to intentionally disrupt a model's accuracy. The results of this investigation suggests that these ideas are mostly unrelated: equivariant models (with architectures designed to encourage the learning of invariances) that are trained with data augmentation whereby input images are given random rotations do not seem to offer any additional adversarial robustness, and similarly using adversarial training to combat adversarial noise does not seem to confer any additional help for learning rotational invariance. (In some cases, these types of training on the one hand seem to make invariance to the other type of perturbations even worse.)