I NVARIANCE VS . R OBUSTNESS OF N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Neural networks achieve state of the art accuracy on many standard datasets used in image classiﬁcation.
One would like to design neural network models whose performance is not affected by natural (or non-malicious) geometric transformations as well as adversarial (or malicious) perturbations.
Previous work has studied generalization to natural geometric transformations (e.g., rotations) as invariance , and generalization to adversarial perturbations as robustness .
In this paper, we examine the interplay between invariance and robustness .
We empirically study the following two cases: (a) change in adversarial robustness as we improve only the invariance using equivariant models and training augmentation, (b) change in invariance as we improve only the adversarial robustness using adversarial training.
We observe that the rotation invariance of equivariant models (StdCNNs and GCNNs) improves by training augmentation with progressively larger rotations but while doing so, their adversarial robustness does not improve, or worse, it can even drop signiﬁcantly on datasets such as MNIST.
As a plausible explanation for this phenomenon we observe that the average perturbation distance of the test points to the decision boundary decreases as the model learns larger and larger rotations.
On the other hand, we take adversarially trained LeNet and ResNet models which have good (cid:96) ∞ adversarial robustness on MNIST and CIFAR-10, and observe that adversarially training them with progressively larger norms keeps their rotation invariance essentially unchanged.
In fact, the difference between test accuracy on unrotated test data and on randomly rotated test data upto ± θ ◦ , for all θ ∈ [0 , 180] , remains essentially unchanged after adversarial training .
As a plausible explanation for the observed phenomenon we show empirically that the principal components
of adversarial perturbations and perturbations given by small rotations are nearly orthogonal
I NTRODUCTION
Neural networks achieve state of the art accuracy on several standard datasets used in image classiﬁcation.
However, their performance in the wild depends on how well they can handle natural or non-adversarial transformations of input seen in real-world data as well as known deliberate, adversarial attacks created to fool the model.
Natural or non-adversarial transformations seen in real-world images include translations, rotations, and scaling.
Convolutional Neural Networks (CNNs) are translation-invariant or shift-invariant by design.
Invariance to other symmetries, and especially rotations, have received much attention recently, e.g., Harmonic Networks (H-Nets) by Worrall et al.
(2016), cyclic slicing and pooling by Dieleman et al.
(2016), Transformation-Invariant Pooling (TI-Pooling) by Laptev et al. (2016),
Group-equivariant Convolutional Neural Networks (GCNNs) by Cohen & Welling (2016), Steerable
CNNs by Cohen & Welling (2017), Deep Rotation Equivariant Networks (DREN) by Li et al.
Rotation Equivariant Vector Field Networks (RotEqNet) by Marcos et al.
Polar Transformer
Networks (PTN) by Esteves et al.
For a given symmetry group G , a G -equivariant network learns a representation or feature map at every intermediate layer such that any transformation g ∈ G applied to an input corresponds to an equivalent transformation of its representations.
Any model can improve its invariance to a given group of symmetries through sufﬁcient training augmentation.
Equivariant models use efﬁcient weight sharing and require smaller sample complexity to achieve better invariance.
Equivariant models such as CNNs and GCNNs too generalize well to progressively larger random rotations but only when their training data is augmented similarly.
Adversarial attacks on neural network models are certain, deliberate changes to inputs that fool a highly accurate model but are unlikely to fool humans.
Given any neural network model, Szegedy et al.
(2013) show how to change the pixel values of images only slightly so that the change is almost imperceptible to human eye but makes highly accurate models misclassify.
Szegedy et al.
ﬁnd these adversarial pixel-wise perturbations of small magnitude by maximizing the prediction error of a given model using box-constrained L-BFGS.
Goodfellow et al.
(2015) propose Fast Gradient Sign
Method (FGSM) that applies the adversarial perturbation as x (cid:48)
=
x + (cid:15) sign ( ∇ x J ( θ, x, y )) , where x is the input, y represents the target, θ represents the model parameters, and J ( θ, x, y ) is the loss function used to train the network.
Moreover, Goodfellow et al. (2015) propose adversarial training, or training augmented with points ( x (cid:48) , y ) , as a way to improve adversarial robustness of a model.
Subsequent work has introduced multi-step variants of FGSM, notably, an iterative method by Kurakin et al.
(2017) and Projected Gradient Descent (PGD) attack by Madry et al.
Given any model, these attacks produce adversarial perturbation for every test image x from a small (cid:96) ∞ -ball around it, namely, each pixel value x i is perturbed within [ x i − (cid:15), x i + (cid:15) ] .
PGD attack does so by solving an inner optimization by projected gradient descent over (cid:96) ∞ -ball of radius (cid:15) around x to approximate the optimal perturbation.
Adversarial training with PGD perturbations improves the adversarial robustness of models, and it is one of the best known defenses to make models robust to perturbations of bounded (cid:96) ∞ norm on MNIST and CIFAR-10 datasets, as shown in Madry et al.
(2018) and Athalye et al.
Recent work has looked at simultaneous robustness to multiple adversarial attacks.
Engstrom et al.
(2017) show that adversarial training with PGD makes CNNs robust against perturbations of bounded (cid:96) ∞ norm but an adversarially chosen combination of a small rotation and a translation can still fool these models nevertheless.
Recent work of Schott et al.
(2019) shows that PGD adversarial training is a good defense against perturbations of bounded (cid:96) ∞ norm but can be broken with adversarial perturbations of small (cid:96) 0 or (cid:96) 2 norm that are also imperceptible to humans or have little semantic meaning for humans.
Schott et al.
(2019) show how to build models for MNIST dataset that are simultaneously robust to perturbations of small (cid:96) 0 , (cid:96) 2 and (cid:96) ∞ norms. 1. P ROBLEM FORMULATION AND OUR RESULTS
Let f be a neural network model (classiﬁer) trained on a training set of labelled images. The accuracy of f is the fraction of test inputs X correctly classiﬁed by f . For a given transformation T of the image space, f is said to be T -invariant if f ( Tx ) = f ( x ) , for all inputs x .
If f is T -invariant and f ( x + a ) (cid:54) = f ( x ) , for some input x and its adversarial perturbation of a small l p norm, then by invariance f ( Tx + Ta ) = f ( x + a ) (cid:54) = f ( x ) =
f
( Tx ) .
Let T be a translation, rotation, or more generally, a permutation of coordinates (see Tram ` er & Boneh (2019)), then || a || p = ||
Ta || p .
Thus,
Ta is an adversarial perturbation for input Tx of small l p norm.
When a change of variables maps x to Tx by a permutation of coordinates, then gradient and T can be swapped.
In other words, ∇ ( f ) at Tx is the same as T applied to ∇ ( f ) at x .
Using this, we can get a 1-1 correspondence between
FGSM (and PGD) perturbations of x and Tx , respectively, with l p norm (cid:15) .
As a corollary, the l p fooling rate for any T -invariant classiﬁer f on the transformed data { Tx | x ∈ X } must be equal to its l p fooling rate on the original data X .
A subtlety kicks in when f is not truly invariant, that is, f ( Tx ) =
f ( x ) , for most inputs x but not all x .
Then the interplay between the classiﬁers accuracy under the transformation T and under adversarial perturbations A is subtle.
This is what we investigate in this paper.
The above statements for a single transformation T (e.g., 90-degree clockwise rotation) also hold when T comes from a family of transformations (e.g., random rotation up to ± θ degree).
In this paper, we study the simultaneous interplay between invariance of models to images randomly rotated by an angle between [ − θ ◦ , + θ ◦ ] , and their adversarial robustness to pixel-wise perturbations of (cid:96) ∞ norm at most (cid:15) .
Using the above language we study the interplay between transformations,
R θ , rotations up to [ − θ ◦ , + θ ◦ ] and transformations, A (cid:15) , adversarial perturbations with (cid:96) p norm atmost (cid:15) .
In order to study these questions we ﬁrst need to address how to measure the pixel-wise robustness, since computing that is NP-hard .
PGD is known to be a strong adversarial attack, Athalye et al.
So we use adversarial accuracy against PGD attack as a proxy for robustness.
Unlike previous studies e.g., Engstrom et al. (2017), we do not ﬁx the magnitude (cid:15) (e.g., say (cid:15) = 0 . 3 ) of pixel-wise perturbations nor limit ourselves to small rotations upto ± 30 ◦ .
Instead, we compute the accuracy of a given model on input rotated by a random angle between [ − θ ◦ , + θ ◦ ] for θ in the range [0 , 180] .
Similarly, we compute the accuracy of a given model for an adversarial attack of (cid:96) ∞ norm at most (cid:15) for (cid:15) in the range [0 , 1] (and the underlying dataset is normalized).
We empirically study the following: (a) change in (cid:96) ∞ adversarial robustness as we improve only rotation invariance using equivariant models (StdCNNs and GCNNs) and training augmentation with progressively larger rotations, (b) change in invariance as we improve only adversarial robustness using PGD adversarial training with progressively larger (cid:96) ∞ -norm of pixel-wise perturbations.
We observe that the rotation invariance of equivariant models (StdCNNs and GCNNs) improves by training augmentation with progressively larger rotations but while doing so, their adversarial robustness does not improve, or worse, it can even drop signiﬁcantly on datasets such as MNIST.
On the other hand, we take adversarially trained LeNet and ResNet models which have good (cid:96) ∞ adversarial robustness on MNIST and CIFAR-10, and observe that adversarially training them with progressively larger norms keeps their rotation invariance essentially unchanged.
In fact, the difference between test accuracy on unrotated test data and on randomly rotated test data upto ± θ ◦ , for all θ ∈ [0 , 180] , remains essentially unchanged after adversarial training with progressively larger bounds on the (cid:96) ∞ norm of pixel-wise perturbations.
We support these two observations with plausible empirical reasons for their occurrence - distance to the decision boundary of the test points reducing in the ﬁrst case, and orthogonality of adversarial perturbations with perturbations caused by small rotations of test points.
Related Work Schott et al.
(2019) study simultaneous robustness to adversarial perturbations of small (cid:96) 0 , (cid:96) 2 , and (cid:96) ∞ -norm.
Tram ` er & Boneh (2019) show an impossibility result by exhibiting data distribution where no model can have substantially better-than-random accuracy for binary classiﬁcation simultaneously against both (cid:96) ∞ and (cid:96) 1 perturbations, and also against both (cid:96) ∞ perturbations and spatial perturbations, given by an adversarial permutation of coordinates that models an adversarially chosen combination of a small rotation and a small translation.
They show an empirical validation of this claim on MNIST and CIFAR-10 datasets for simultaneous robustness against (cid:96) ∞ adversarial perturbation and an adversarially chosen combination of translation upto ± 3 pixels and rotation upto ± 30 ◦ .
R OTATION INVARIANCE VS . (cid:96) ∞ ADVERSARIAL ROBUSTNESS
In this section, we present our main result about the interplay between rotation invariance and (cid:96) ∞ adversarial robustness of models on MNIST, Fashion MNIST, and CIFAR-10 data.
In Subsection 2.2, we study how the robustness of equivariant models (StdCNNs and GCNNs) to (cid:96) ∞
adversarial perturbations changes as they become more rotation invariant through training augmentation with random rotations of progressively larger degree.
In Subsection 2.3, we study how the rotation invariance of LeNet and ResNet based architectures change as they become robust to (cid:96) ∞ adversarial perturbations through PGD adversarial training.
The models used in Subsection 2.2 were shown to achieve good rotational invariance with augmentation by Cohen & Welling (2016).
The LeNet and
ResNet models used in Subsection 2.3 were shown to be adversarially robust by Madry et al.
(2018) and reconﬁrmed in recent work by Athalye et al.
Before we begin our experiments we ﬁrst give details of the plots that our experiments will yield.
Rotation invariance means that the predicted labels of an image and any of its rotations should be the same.
Since most datasets are centered, we restrict our attention to rotations around the center of the image.
One way to quantify rotation invariance is to look at the accuracy of a given model on randomly rotated test data.
This is visualized by plotting the accuracy on test inputs rotated by a random angle between [ − θ ◦ , + θ ◦ ] , as θ varies in the range [0 , 180] .
We call this the rotation invariance proﬁle of the given model.
Adversarial robustness means that the predicted labels of an image and its adversarial pixel-wise perturbation should be the same.
Adversarial robustness of a given model to a ﬁxed adversarial attack (e.g., FGSM, PGD) and
a ﬁxed (cid:96) ∞ norm (say, 0 ≤ (cid:15) ≤ 1 ) is quantiﬁed by the accuracy of the given model on perturbed test data with a perturbation of (cid:96) ∞ norm at most (cid:15) generated by the attack.
The resulting plot of the accuracy of the model as (cid:15) varies over [0 , 1] is the robustness proﬁle of the given model.
Convention used in the legends of our ﬁgure: We use the following convention in the legends of our plots.
A coloured line labeled A/B indicates that the training data is augmented with random rotations from [ − A ◦ , A ◦ ] and the test data is augmented with random rotations from [ − B ◦ , B ◦ ] .
If
A (resp.
B ) is zero it means the training data (resp.
test data) is unrotated.
2. I MPROVING ROTATION INVARIANCE
Any model improves its rotation invariance proﬁle when its training data is augmented with progressively larger random rotations.
Equivariant models (e.g., GCNNs, H-Nets, PTNs, RotEqNets) are designed to achieve better rotation invariance thought clever weight sharing.
However, models such as GCNNs and H-Nets cannot achieve rotation invariance, if their training data is not sufﬁciently augmented.
This appears to be folklore so we do not elaborate on this in the main paper.
In Appendix E we conﬁrm this on many popular rotation equivariant models and also show empirically that they become equivariant to rotations by training with a small sample of data augmented with random rotations
[Figure]
For any ﬁxed θ ∈ [0 , 180] , we take an equivariant model, namely, StdCNN or GCNN, and augment its training data by random rotations from [ − θ ◦ , + θ ◦ ] .
Figures 1 and 2 show how the robustness proﬁles of StdCNN and GCNN models change on MNIST, Fashion MNIST, and CIFAR-10 datasets, as we increase the degree θ used in training augmentation of the model (thereby improving their rotation invariance).
We use PGD adversarial attack for MNIST and Fashion MNIST, FGSM and
PGD attack for CIFAR-10 dataset.
FGSM is a weaker attack for these models on CIFAR-10.
In the above experiments, the test data is unrotated but adversarially perturbed.
For completeness,
Figures 9, 10 in Appendix B, show the robustness proﬁle of the same networks when the test is also augmented with random rotations before applying adversarial perturbations.
Our experiments show that training augmentation with progressively larger rotations improves the rotation invariance of StdCNNs and GCNNs.
For example the red line in Figure 20(bottom) for
CIFAR-10 in Appendix E.2 shows that the accuracy of GCNNs drops below 70% when it is not trained with rotations but when the test has random rotations upto ± 30 ◦ .
However when the training data is augmented with random rotations upto ± 30 ◦ the accuracy improves to about 88% as can be seen from the black line in the same ﬁgure.
However training with rotations does not improve their (cid:96) ∞ adversarial robustness, and can even worsen as in the case of MNIST.
On MNIST, the (cid:96) ∞ adversarial robustness of StdCNNs and GCNNs drops sharply when they are trained with larger rotation augmentations.
When GCNN’s are trained on MNIST with rotations upto 30 ◦ (light blue line in Figure 2 (top left))
their accuracy drops to 90% under a PGD attack with (cid:15) = 0 .
3 , whereas their accuracy is about 97% with the same PGD attack when they are not trained with random rotations (dark blue line in Figure 2 (top left)).
We observe the same trend when the test is augmented with rotations, Figure 10 (left) in Appendix B.
It is alarming to note that even seemingly innocuous training augmentation with random rotations can affect adversarial robustness of models.
2. E FFECT OF (cid:96) ∞ ADVERSARIAL TRAINING ON ROTATION INVARIANCE
The most common approach to improve adversarial robustness is adversarial training, i.e., training the model on adversarially perturbed training data.
Adversarial training with PGD attack is one of the strongest known defenses on MNIST and CIFAR-10 datasets, see Athalye et al.
For any ﬁxed (cid:15) ∈ [0 , 1] and any ﬁxed θ ∈ [0 , 180] , we adversarially train our models (LeNet and
ResNet) with PGD adversarial perturbations with (cid:96) ∞ budget (cid:15) , and then plot their rotation invariance proﬁles.
Each colored line in Figure 3 corresponds to a model adversarially trained with a different value of (cid:15) .
Tsipras et al.
(2019) point out that the test accuracy on the original data drops after adversarial training, and our plots conﬁrm this - the y co-ordinate of the points where the curves begin decreases with increasing (cid:15) .
In Figure 3, we observe that the entire invariance proﬁle essentially make a parallel shift downward for LeNet and ResNet based models on MNIST and CIFAR-10 datasets, respectively.
In other words, adversarial training does not affect the relative drop in accuracy when test data is augmented with progressively larger rotations.
D ECISION BOUNDARY DISTANCE AND O RTHOGONALITY
In order to understand what could be the possible reasons behind our empirical observations in
Section 2.2 and Section 2.3 we perform more experiments.
3. A VERAGE PERTURBATION DISTANCE TO THE BOUNDARY .
For each test image, adversarial attacks ﬁnd perturbations of the test point with small (cid:96) ∞ norm that would change the prediction of the given model.
Most adversarial attacks do so by ﬁnding the directions in which the loss function of the model changes the most.
In order to explain why these networks become vulnerable to pixel-wise attacks as they learn more rotations, we see how the distance of the test points to the decision boundary changes as the networks learn larger rotations.
This is abstractly depicted in Figure 4 where we show the distance of a test point x 0 to the boundary
D 0 (resp.
D 180 ) when the model is trained with zero (resp. 180 ◦ ) rotations.
We use the L 2 attack vectors obtained by DeepFool (Moosavi-Dezfooli et al., 2016) for the datapoints under attack.
We take the norm of this attack vector as an approximation to the shortest distance of the test point to the decision boundary.
For each of the test points we collect the perturbation vectors given by DeepFool attack and report the average perturbation distance.
We plot this average distance as the datasets are augmented with larger rotations.
Our experiments show that as the networks learn larger rotations with augmentation, the average perturbation distance falls.
So as (symmetric) networks become invariant to rotations, they are more vulnerable to pixel-wise attacks.
The plots in Figures 5, 6, 7, 8 show this for StdCNNs and GCNNs on MNIST and CIFAR-10.
Figures 17 and 18 in Appendix C show the plots for Fashion-MNIST.
To make our point we plot the accuracy of these networks and also the average perturbation distance of the test points alongside in one ﬁgure.
The blue line in Figure 5(left) shows the accuracy of a StdCNN on MNIST when both the training data and test data are augmented with θ , as θ ranges from 0 to 180 .
The green line in
Figure 5(left) shows the accuracy of the StdCNN model when the train is augmented with random
, and
The red line Figure 5(right) shows the average perturbation distance of the unrotated test points when the network is trained with rotations
upto θ .
The green line shows the average perturbation distance of test points which are augmented with rotations upto
θ - this is about 5 when θ is 0 ◦ (the point on the y -axis where the curves begin).
As the network is trained with random rotations up to 180 ◦ the average perturbation distance of the augmented test drops below 3.5.
Figure 5(left) shows that that the PGD accuracy has dropped from around 85% for the network at 0 ◦ to 30% at 180 ◦ (the corresponding green line on the left).
The fact that the red line is above the green line is also reﬂected in Figure 11 in Appendix C. When the test is perturbed by PGD, the accuracy of the
StdCNN with training data augmented with rotations is better when the test is not augmented with rotations than if the test were also augmented with rotations.
3. O RTHOGONALITY OF A DVERSARIAL PERTURBATIONS & INVARIANT ROTATIONAL PERTURBATIONS
In general we expect neural networks to classify x and a small rotation x (cid:48) of x , identically.
This would be particularly so if the dataset has some inherent rotations and the labels remain the same under such rotations.
Writing x (cid:48)
=
x
+ ∆( x ) we say ∆( x ) is an invariant rotational perturbation
[Table]
[Table]
for x .
For a network trained to be equivariant to rotations, this rotational perturbation of x could well be orthogonal to an adversarial perturbation of x , since an adversarial perturbation changes the label whereas an invariant rotational perturbation does not change the label.
We empirically show that this is the case for MNIST and CIFAR-10.
For each test input ﬁnd we ﬁnd its PGD perturbation and stack these adversarial perturbations as rows of an adversarial matrix.
We ﬁnd the top 5 right singular vectors of this adversarial matrix.
We rotate each input by 4 ◦ / 2 ◦ for MNIST/CIFAR-10 and subtract the original image from it.
We stack such invariant rotational perturbations as rows of an invariance matrix.
We ﬁnd the top 5 right singular vectors of this invariance matrix.
We empirically observe that the principal angles between these two 5-dimensional subspaces are close to 90 ◦ .
This may explain why in Section 2.3 the rotation invariance proﬁle just shifts down.
For completeness we recall the deﬁnition of principal angles in Appendix D.
Table 1 gives the principal angles of MNIST trained on LeNet and Table 2 gives the principal angles for CIFAR-10 trained on ResNet as given by Madry et al.
C ONCLUSION
We observe that as equivariant models (StdCNNs and GCNNs) are trained with progressively larger rotations their rotation invariance improves but at the cost of their adversarial robustness.
On the other hand, adversarial training with perturbations of progressively increasing norms improves the robustness of LeNet and ResNet and also keeps the invariance unchanged.
A plausible explanation of the ﬁrst observation is that the average perturbation distance of the test points to the boundary decreases as StdCNNs and GCNNs learn larger rotations.
A plausible explanation for the second observation is that the singular vectors of adversarial perturbations are orthogonal to the singular vectors of invariant rotational perturbations.
A PPENDIX A D ETAILS OF EXPERIMENTS
All experiments performed on neural network-based models were done using MNIST, Fashion
MNIST and CIFAR-10 datasets with appropriate augmentations applied to the train/validation/test set.
Data sets MNIST dataset consists of 70 , 000 images of 28 × 28 size, divided into 10 classes.
55 ,used for training, 5 , 000 for validation and 10 , 000 for testing.
Fashion MNIST dataset consists of 70 , 000 images of 28 × 28 size, divided into 10 classes.
55 , 000 used for training, 5 , 000 for validation and 10 , 000 for testing.
CIFAR-10 dataset consists of 60 , 000 images of 32 × 32 size, divided into 10 classes.
40 , 000 used for training, 10 , 000 for validation and 10 , 000 for testing.
Model Architectures For the MNIST and Fashion MNIST based experiments we use the network architecture of GCNN as given in Cohen & Welling (2016).
The StdCNN architecture is similar to the GCNN except that the operations are as per CNNs.
Refer to Table 3 for details.
H-Nets architecture is as given in Worrall et al.
RotEqNet architecture is as given in Marcos et al.
PTN architecture is as given in Esteves et al.
For the CIFAR-10 based experiments we use the VGG16 architecture as given in Simonyan & Zisserman (2014) and its GCNN equivalent is obtained replacing the various layer operations with equivalent GCNN operations as given in Cohen & Welling (2016).
This is similar to how we obtained a GCNN architecture from StdCNN for the MNIST based experiments.
Input training data was augmented with random cropping and random horizontal ﬂips.
For the adversarial training experiments we used the LeNet based architecture for MNIST and the
ResNet architecture for CIFAR-10.
Both these models are exactly as given in Madry et al.
[Table]
A PPENDIX B E FFECT OF INVARIANCE ON ADVERSARIAL ROBUSTNESS AFTER TEST AUGMENTATION
A PPENDIX C D ECISION BOUNDARY DISTANCE , ORTHOGONALITY
A PPENDIX D P RINCIPAL ANGLES
Recall that for two subspaces V
W the ﬁrst principal angle is deﬁned as the minimum angle between two unit vectors v 1 ∈ V , w 1 ∈ W .
The second principal angle is the minimum angle between unit vectors v 2 ∈ V, w 2 ∈ W , with v 2 ⊥ v 1 and w 2 ⊥ w 1 .
The other principal angles are deﬁned similarly.
A PPENDIX E R OTATION INVARIANCE OF EQUIVARIANT NETWORKS
We observe that (a) Rotation equivariant networks are robust to only small degrees of rotations away from the ones present in the training data - this was studied only for CNNs in Weiler et al.
(2018), (b) applying data augmentation increases their invariance to rotations.
) Rotation equivariant net works achieve state of the art results with smaller sample size for training.
E. R OBUSTNESS TO ROTATIONS WITHOUT TRAINING AUGMENTATION
We ﬁrst train all the networks on MNIST and Fashion MNIST with no rotation augmentation and test against inputs augmented with varying range of random rotations from ± 0 ◦ to ± 180 ◦ .
Figure 19(top left) for MNIST and Figure 19(top right) for Fashion MNIST shows the performance of these networks to the entire range of rotations.
We observe that even though these datasets are small and these networks are designed to be invariant to rotations, their performance drops as the range of random rotations in test data increases.
For small range of rotations up to about ± 20 ◦ , the performance of these networks on MNIST remains above 95% indicating that MNIST dataset has some natural rotation augmentation.
On Fashion MNIST, the performance being above 85% when the test is augmented with small rotations up to ± 20 ◦ .
When the test is augmented with rotations beyond 25 ◦ , the accuracy of these networks drops.
We observe that PTN and RotEqNet are more robust than the other networks for MNIST and Fashion MNIST.
On the dataset CIFAR-10 we compare the performance of rotation-equivariant GCNNs with translation-equivariant StdCNNs.
This is shown in Figure 19(bottom).
GCNNs perform better, as expected, but the performance of both GCNNs and StdCNNs degrades to less than 70% accuracy when the test data is augmented with random rotations larger than ± 30 ◦ .
E. R OBUSTNESS WITH TRAINING AUGMENTATION
We train equivariant networks with input augmented with varying range of random rotations from ± 0 ◦ to ± 180 ◦ .
In Figure 20 we plot our results for MNIST (top left), Fashion MNIST (top right) and CIFAR-10 (
E. S AMPLE C OMPLEXITY OF N ETWORKS
To understand the sample complexity of the networks we perform two experiments.
In the ﬁrst we train the networks with varying sample sizes of training set and test them on the entire test set.
And in the second experiment we do the same as the ﬁrst with the inputs in train and test augmented with random rotations in the range [ − 180 ◦ , 180 ◦ ] .
From Figure 21 for MNIST, Figure 22 for Fashion
MNIST and 23 for CIFAR-10, we can see that rotation equivariant networks achieve their best per- formance safely using 10 k - 30 k training samples.
This conﬁrms that rotation equivariant networks can do well with smaller training sample size.
A PPENDIX F A DDITIONAL P LOTS R EQUESTED BY R EVIEWERS F. M ODELS FROM M ADRY ET AL . (2018)
Deﬁne the fooling rate of a classiﬁer f to a transformation T e.g., rotation, as the ratio { x ∈
X | f ( x ) (cid:54) =
f ( T ( x )) } / | X | .
Based on a suggestion of one of the referees, we plot the rotation invariance proﬁles of PGD adversarially trained models in Figure 24 below using fooling rate on the y axis (actually, 1 - fooling rate) - compare with Figure 3, where we plot accuracy on the y axis. The plots in the ﬁgure below are aligned at 0 ◦ .
However the overall trends in the plots in Figureand the plots in Figure 3 are very similar.
On MNIST, adversarial training with PGD with larger (cid:15) results in an increase in fooling rate of
LeNet based model (as given in Madry et al.
with respect to rotations - in Figure 24 (left), the yellow line (PGD with (cid:15) = 0 . 4 ) is below the light blue line (PGD with (cid:15)
= 0 . 1 ).
It would be more accurate to say that the invariance proﬁle curves in Figure 24 (plotted using fooling rate) just undergo a parallel shift downwards with an increase in (cid:15) used for PGD during training, similar to what was observed of the invariance proﬁle curves in Figure 3(left) (plotted using accuracy).
This is true of the ResNet based model (as given in Madry et al.
(2018)) too, as can be seen from Figure(right).
The invariance proﬁle curves are similar to the invariance proﬁle curves in Figure 3(right).
This empirical evidence suggests that the invariance proﬁle of the LeNet/ResNet based model remains essentially unchanged even as they are adversarially trained with progressively larger norms.
To complete the picture we plot both the invariance proﬁle curves and the robustness proﬁle curves of the LeNet and ResNet based model.
It is known that as these models are trained with PGD using larger (cid:15) budget their adversarial robustness increases.
The robustness proﬁle curves of the LeNet model trained with larger PGD budget dominates the robustness proﬁle curve of the same model trained with a smaller PGD budget - the red line in Figure 25 (right), dominates the light blue line.
This is true of the ResNet based model too, as can be seen from Figure 26 (right).
F. S TD CNN S AND GCNN S
To complete the picture we plot the rotation invariance proﬁles of StdCNNs and GCNNs.
So for each model which is trained using data augmented with rotations up to ± θ -degrees, we plot the accuracy when the test data is augmented with random rotations upto ± 180 -degrees.
The plots of their robustness proﬁles are taken from Section 2.1.
As remarked in Section 2.1, training with larger and larger rotations increases the ability of these models to handle larger rotations in the test data.
But this comes at a price of their adversarial robustness.
The black line in Figure 27 (left), shows that the adversarial robustness of a StdCNN which is trained to handle rotations up to ± 180 degrees on MNIST, drops by more than 50%, even when the (cid:15) budget for PGD attack on unrotated MNIST is only 0 .
The black line in Figure 27 (right), shows this models rotational invariance proﬁle - this model can handle larger rotations in the test data.
This can be contrasted with the model depicted by the red line - this StdCNN is trained to handle rotations up to 60 degrees.
The rotational invariance proﬁle of this model is below that of the model depicted by the black line and so it cannot handle large rotations.
However this model can handle adversarial (cid:96) ∞ -perturbations up to 0.3 on unrotated data, with an accuracy more than 80 % - this can be seen from the red line in Figure 27 (left).
The above observations, of there being a tradeoff between handling larger rotations with training augmentation and handling larger adversarial perturbations is seen in GCNNs also.
We show this in
{+Figure 29 and Figure 
+}