Original: R OBUSTNESS AND E QUIVARIANCE OF N EURAL N ETWORKS
Revised: R OBUSTNESS AND E QUIVARIANCE OF N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Neural networks models are known to be vulnerable to geometric transformations as well as small pixel-wise perturbations of input.
Revised: Neural networks models are known to be vulnerable to geometric transformations as well as small pixel-wise perturbations of input.

Original: Convolutional Neural Networks (CNNs) are translation-equivariant but can be easily fooled using rotations and small pixel-wise perturbations.
Revised: Convolutional Neural Networks (CNNs) are translation-equivariant but can be easily fooled using rotations and small pixel-wise perturbations.

Original: Moreover, CNNs require sufﬁcient translations in their training data to achieve translation-invariance.
Revised: Moreover, CNNs require sufﬁcient translations in their training data to achieve translation-invariance.

Original: Recent work by Cohen &
Revised: Recent work by Cohen &

Original: Welling (2016), Worrall et al.
Revised: Welling (2016), Worrall et al.

Original: (2016), Kondor & Trivedi (2018), Cohen & Welling (2017), Marcos et al.
Revised: (2016), Kondor & Trivedi (2018), Cohen & Welling (2017), Marcos et al.

Original: (2017), and Esteves et al.
Revised: (2017), and Esteves et al.

Original: (2018) has gone beyond translations, and constructed rotation-equivariant or more general group-equivariant neural network models.
Revised: (2018) has gone beyond translations, and constructed rotation-equivariant or more general group-equivariant neural network models.

Original: In this paper, we do an extensive empirical study of various rotation-equivariant neural network models  to understand how effectively they learn rotations.
Revised: In this paper, we do an extensive empirical study of various rotation-equivariant neural network models and Standard CNNs to understand how effectively they learn rotations.

Original: This includes Group-equivariant Convolutional Networks (GCNNs) by Cohen & Welling (2016), Harmonic Networks (H-Nets) by Worrall et al.
Revised: This includes Group-equivariant Convolutional Networks (GCNNs) by Cohen & Welling (2016), Harmonic Networks (HNets) by Worrall et al.

Original: (2016), Polar Transformer Networks (PTN) by Esteves et al.
Revised: Polar Transformer Networks (PTN) by Esteves et al.

Original:  and Rotation equivariant vector ﬁeld networks by Marcos et al.
Revised: (2018) and Rotation equivariant vector ﬁeld networks by Marcos et al.

Original: We empirically compare the ability of these networks to learn rotations efﬁciently in terms of their number of parameters, sample complexity, rotation augmentation used in training.
Revised: We empirically compare the ability of these networks to learn rotations efﬁciently in terms of their number of parameters, sample complexity, rotation augmentation used in training.

Original: We compare them against each other as well as Standard CNNs.
Revised: We compare them against each other as well as Standard CNNs.

Original: We observe that as theserotation-equivariant neural networks learn rotations, they instead become more vulnerable to small pixel-wise adversarial attacks, e.g., Fast Gradient 
Revised: We observe that as these neural networks learn rotations, they instead become more vulnerable to small pixel-wise adversarial attacks, e.g., Fast Gradient Sign

Original: Sign Method (FGSM) and Projected Gradient Descent (PGD), in comparison with
Revised: Method (FGSM) and Projected Gradient Descent (PGD).

Original: Standard CNNs.

Original: In other words, robustness to geometric transformations in these models comes at the cost of robustness to small pixel-wise perturbations.
Revised: In other words, robustness to geometric transformations in these models comes at the cost of robustness to small pixel-wise perturbations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Neural network-based models achieve state of the art results on several speech and visual recognition tasks but these models are known to be vulnerable to various adversarial attacks.
Revised: Neural network-based models achieve state of the art results on several speech and visual recognition tasks but these models are known to be vulnerable to various adversarial attacks.

Original: Szegedy et al.
Revised: Szegedy et al.

Original: (2013) show that small, pixel-wise changes that are almost imperceptible to the human eye can make neural networks models grossly misclassify.
Revised: (2013) show that small, pixel-wise changes that are almost imperceptible to the human eye can make neural networks models grossly misclassify.

Original: They ﬁnd a small perturbation so as to maximizes the prediction error of a given model using box-constrained L-BFGS.
Revised: They ﬁnd a small perturbation so as to maximizes the prediction error of a given model using box-constrained L-BFGS.

Original: Goodfellow et al.
Revised: Goodfellow et al.

Original: (2015) propose the Fast Gradient Sign Method (FGSM) as a faster approach to ﬁnd such an adversarial perturbation given by x (cid:48)
Revised: (2015) propose the Fast Gradient Sign Method (FGSM) as a faster approach to ﬁnd such an adversarial perturbation given by x (cid:48)

Original: =
Revised: =

Original: x + (cid:15) sign ( ∇ x J ( θ, x, y )) , where x is the input, y represents the targets, θ represents the model parameters, and J ( θ, x, y ) is the cost used to train the network.
Revised: x + (cid:15) sign ( ∇ x J ( θ, x, y )) , where x is the input, y represents the targets, θ represents the model parameters, and J ( θ, x, y ) is the cost used to train the network.

Original: Subsequent work has introduced multi-step variants of FGSM, notably, an iterative method by Kurakin et al.
Revised: Subsequent work has introduced multi-step variants of FGSM, notably, an iterative method by Kurakin et al.

Original: (2017) and Projected Gradient Descent (PGD) by Madry et al.
Revised: (2017) and Projected Gradient Descent (PGD) by Madry et al.

Original: On visual tasks, the adversarial perturbation must come from a set of images that are perceptually similar to a given image.
Revised: On visual tasks, the adversarial perturbation must come from a set of images that are perceptually similar to a given image.

Original: Goodfellow et al. (2015) and Madry et al.
Revised: Goodfellow et al. (2015) and Madry et al.

Original: (2018) study adversarial perturbations from the (cid:96) ∞ -ball around the input x , namely, each pixel value is perturbed by a quantity within [ − (cid:15), + (cid:15) ] .
Revised: (2018) study adversarial perturbations from the (cid:96) ∞ -ball around the input x , namely, each pixel value is perturbed by a quantity within [ − (cid:15), + (cid:15) ] .

Original: Broadly, all the above-mentioned adversarial attacks are model-dependent.
Revised: Broadly, all the above-mentioned adversarial attacks are model-dependent.

Original: Tramer et al.
Revised: Tramer et al.

Original: (2017) also mention model-agnostic perturbations using the direction of the difference between the intra-class means.
Revised: (2017) also mention model-agnostic perturbations using the direction of the difference between the intra-class means.

Original: There is a large class of spatial transformations including translations, rotations, scaling that preserve perceptual similarity.
Revised: There is a large class of spatial transformations including translations, rotations, scaling that preserve perceptual similarity.

Original: Convolutional Neural Networks (CNNs) are translation-equivariant by construction.
Revised: Convolutional Neural Networks (CNNs) are translation-equivariant by construction.

Revised: Recent works show that apart from pixel-wise perturbations, geometric transforms like rotations are also possible natural attacks; see Dumont et al.

Revised: (2018), Gilmer et al.

Original: However, Engstrom et al.
Revised: En- gstrom et al.

Original: (2017) show that simple adversarial attacks using rotations and translations can fool CNNs, even when they are adversarially trained to make them robust to (cid:96) p -bounded adversaries.
Revised: (2017) show that simple adversarial attacks using rotations and translations can fool CNNs, even when they are adversarially trained to make them robust to (cid:96) p -bounded adversaries.

Original: They observe that (cid:96) p -bounded and spatial adversarial perturbations have additive or super-additive effect on the performance drop, suggesting that these two types of attacks have no bearing on each other.
Revised: They observe that (cid:96) p -bounded and spatial adversarial perturbations have additive or super-additive effect on the performance drop, suggesting that these two types of attacks have no bearing on each other.

Original: Engstrom et al.
Revised: Engstrom et al.

Original: (2017) also show that CNNs achieve translation invariance only if the training data (or augmentation) contains some amount of translated inputs, however, their accuracy against the worst-case translations is signiﬁcantly worse than the averagecase.
Revised: (2017) also show that CNNs achieve translation invariance only if the training data (or augmentation) contains some amount of translated inputs, however, their accuracy against the worst-case translations is signiﬁcantly worse than the average-case.

Original: CNNs are translation-equivariant but not equivariant with respect to other spatial symmetries such as rotations, reﬂections etc.
Revised: CNNs are translation-equivariant but not equivariant with respect to other spatial symmetries such as rotations, reﬂections etc.

Original: Variants of CNNs to achieve rotation-equivariance and other symmetries have received much attention recently, notably, Harmonic Networks (H-Nets) by Worrall et al.
Revised: Variants of CNNs to achieve rotation-equivariance and other symmetries have received much attention recently, notably, Harmonic Networks (H-Nets) by Worrall et al.

Original: (2016), cyclic slicing and pooling by Dieleman et al.
Revised: (2016), cyclic slicing and pooling by Dieleman et al.

Original: (2016), Tranformation-Invariant Pooling (TIPooling) by Laptev et al.
Revised: (2016), Tranformation-Invariant Pooling (TIPooling) by Laptev et al.

Original: (2016), Group-equivariant Convolutional Neural Networks (GCNNs) by
Revised: (2016), Group-equivariant Convolutional Neural Networks (GCNNs) by

Original: Cohen & Welling (2016), Steerable CNNs by Cohen & Welling (2017), Deep Rotation Equivariant
Revised: Cohen & Welling (2016), Steerable CNNs by Cohen & Welling (2017), Deep Rotation Equivariant

Original: Networks (DREN) by Li et al. (2017), Rotation Equivariant Vector Field Networks (RotEqNet) by
Revised: Networks (DREN) by Li et al. (2017), Rotation Equivariant Vector Field Networks (RotEqNet) by

Original: Marcos et al.
Revised: Marcos et al.

Original: Polar Transformer Networks (PTN) by Esteves et al.
Revised: Polar Transformer Networks (PTN) by Esteves et al.

Original: For our study, we  choose GCNNs as they achieve close to the current state of the art results on MNIST-rot [-
-] and CIFAR10 data sets as reported in Esteves et al.
Revised: For our study, we do the experiments on MNIST, Fashion MNIST and CIFAR10 and choose GCNNs as they achieve close to the current state of the art results on MNIST-rot [-
-]1 and CIFAR10 data sets as reported in Esteves et al.

Original: GCNNs provide good representative networks to understand the effect of (cid:96) p -bounded and spatial transformation adversaries on symmetric networks.
Revised: GCNNs provide good representative networks to understand the effect of (cid:96) p -bounded and spatial transformation adversaries on symmetric networks.

Original: GCNNs use G-convolutions, they have more weight-sharing than regular convolution layers, and they are easy to implement with minimal computational overhead for discrete groups of symmetry generated by translations, reﬂections, and rotations.
Revised: GCNNs use G-convolutions, they have more weight-sharing than regular convolution layers, and they are easy to implement with minimal computational overhead for discrete groups of symmetry generated by translations, reﬂections, and rotations.

Original: We do show similar qualitative trends on Harmonic Networks (H-Nets), Polar Transformer Networks(PTN) and Rotation Equivariant Vector Field Networks (RotEqNet).
Revised: We do show similar qualitative trends on Harmonic Networks (H-Nets), Polar Transformer Networks(PTN) and Rotation Equivariant Vector Field Networks (RotEqNet).

Original:  R OBUSTNESS OFROTATION - EQUIVARIANT NETWORKS
Revised:  R OBUSTNESS OF EQUIVARIANT NETWORKS

Original: We study the robustness of  GCNNs to adversarial attacks based on rotations as well as pixel-wise perturbations on MNISTdata set and compare it with Standard CNNs(StdCNNs).
Revised: We study the robustness of StdCNNs and GCNNs to adversarial attacks based on rotations as well as pixel-wise perturbations for MNIST, Fashion MNIST and CIFAR10 data sets.

Original: To the best of our knowledge this is the ﬁrst study of a rotation-equivariant network towards pixel-wise perturbations.
Revised: To the best of our knowledge this is the ﬁrst study of a rotation-equivariant network towards pixel-wise perturbations.

Original: There are other types of networks like CapsNetSabour et al.
Revised: There are other types of networks like CapsNetSabour et al.

Original: (2017) which do show natural robustness to AffNist dataset 2 though not trained on it.
Revised: (2017) which do show natural robustness to AffNist dataset 2 though not trained on it.

Original: And further derived networks based on EM Routing
Revised: And further derived networks based on EM Routing

Original: Hinton et al.
Revised: Hinton et al.

Original: (2018) which along with better spatial robustness also seem to be robust to FGSM attacks.
Revised: (2018) which along with better spatial robustness also seem to be robust to FGSM attacks.

Original: We have also checked the robustness of CapsNet to small rotations.
Revised: We have also checked the robustness of CapsNet to small rotations.

Original: 2. R OBUSTNESS TO ROTATIONS
Revised: 2. R OBUSTNESS TO ROTATIONS

Original: We study the robustness of equivariant networks to attacks based on rotations on MNIST and compare it with StdCNNs.
Revised: We study the robustness of equivariant networks to attacks based on rotations for MNIST, Fashion MNIST and CIFAR10.

Original: The main takeaways of our empirical results are (a) Rotation equivariant networks are robust to  small degrees of rotations away from the ones present in the training data, (b) applying data augmentation increases their robustness, (c) Rotation equivariant networks do achieve state of the art results with smaller sample size for training.
Revised: The main takeaways of our empirical results are (a) Rotation equivariant networks are robust to only small degrees of rotations away from the ones present in the training data, (b) applying data augmentation increases their robustness, (c) Rotation equivariant networks do achieve state of the art results with smaller sample size for training.

Original: We ﬁrst trained all the networks on  MNIST  with no  augmentation and tested against inputs augmented with varying range of rotations.
Revised: We ﬁrst trained all the networks on MNIST, Fashion MNIST and CIFAR10 respectively with no rotation augmentation and tested against inputs augmented with varying range of random rotations from ± 0 ◦ to ± 180 ◦ .

Original: In Figure (1)(left) we observe the inherent robustness of all the equivariant networks to small(below  40 ◦ ) rotations.
Revised: In Figure (1)(left), (2)(left) and (14)(left) we observe the inherent robustness of all the equivariant networks to small(below ± 40 ◦ ) rotations.

Original: Their accuracy is always greater than StdCNNs with accuracy well above 98% for rotations upto 20 ◦ .
Revised: Most equivariant networks (except HNets for Fashion MNIST) accuracy is always greater than StdCNNs.

Original: We also did the same experiment for CapsNet(see Figure 12) and found their values to be sandwiched between GCNNs and HNets.
Revised: We also did the same experiment for CapsNet(see Figure 13) with MNIST and found their values to be sandwiched between GCNNs and HNets.

Original: Figure (1)(right) shows the performance of these networks to the entire range of rotations.
Revised: Figure (1)(right), (2)(right) and (14)(right) shows the performance of these networks to the entire range of rotations.

Original: Here we observe PTN and RotEqNet to be more robust than the other networks  and their accuracy remains safely above 60%.
Revised: Here we observe PTN and RotEqNet to be more robust than the other networks for MNIST and Fashion MNIST. While GCNN is better than StdCNN for CIFAR10.

Revised: In Figure 3 we observe that since MNIST contains some natural rotation augmentation in the training data, equivariant networks are able to utilize this augmentation to become more invariant to small rotations than StdCNNs.

Revised: However, such augmentation is not present in Fashion MNIST and

Revised: CIFAR10, so they donot perform signiﬁcantly better than StdCNNs

Original: 2.1. R OBUSTNESS WITH TRAINING AUGMENTATION
Revised: 2.1. R OBUSTNESS WITH TRAINING AUGMENTATION

Original: We trained the equivariant networks with input augmented with varying range of rotations.
Revised: We trained the equivariant networks with input augmented with varying range of random rotations from ± 0 ◦ to ± 180 ◦ .

Original: We observe that their accuracy for the respective range of  rotation augmentation remains above 98%,
Revised: We observe that when we compare the networks accuracy when trained with unrotated data against training with data augmented with ± 180 ◦ range of random rotation augmentations only on MNIST rotation-equivariant networks achieve almost( ≈ 1% difference) the same performance while there is a gap of 3-5% for Fashion MNIST and CIFAR10. This is evident in Figure 4 where we see a change in slope of the black line from ± 0 ◦ to ± 180 ◦ . The change is much more signiﬁcant for StdCNNs which is evident from the change in slope of the brown line from ± 0 ◦ to ± 180 ◦ .

Original: 2. S AMPLE C OMPLEXITY OF N ETWORKS
Revised: From this it is very clear that rotation-equivariant networks depend very heavily on augmentation to learn even small rotations. Since, even with augmentation there is a gap between the accuracy of the network against that trained without augmentation, it strongly indicates that even these networks trained with augmentation rotations itself can be adversarial for the network. Similar observation is made for StdCNNs by Engstrom et al. S AMPLE C OMPLEXITY OF N ETWORKS

Original: To understand the sample complexity of the networks, we perform two experiments.
Revised: To understand the sample complexity of the networks, we perform two experiments.

Original: In the ﬁrst we train the networks with varying sample sizes of MNIST training set and test them on the entire
Revised: In the ﬁrst we train the networks with varying sample sizes of MNIST training set and test them on the entire

Original: MNIST test set.
Revised: MNIST test set.

Original: And in the second experiment we do the same as the ﬁrst with the inputs in train and test augmented with  [ − 180 ◦ , 180 ◦ ] range of rotations.
Revised: And in the second experiment we do the same as the ﬁrst with the inputs in train and test augmented with random rotations in the range [ − 180 ◦ , 180 ◦ ] .

Original: From Figure 3, we can see that rotation equivariant networks achieve their best performance safely using 10 k - 30 k training samples.
Revised: From Figure 5, we can see that rotation equivariant networks achieve their best performance safely using 10 k - 30 k training samples.

Original: This conﬁrms that rotation equivariant networks exploit symmetry and reduce training sample size
Revised: This conﬁrms that rotation equivariant networks do reduce training sample size.

Original: 2. R OBUSTNESS TO P IXEL - WISE PERTURBATIONS
Revised: 2. R OBUSTNESS TO P IXEL - WISE PERTURBATIONS

Original: From Figure 2 we observe that  with augmentation the networks become robust to rotations.
Revised: From Figure 4 we observed that only with augmentation the networks become robust to rotations.

Original: With networksmade robust to rotations with respective augmentation we now study  the vulnerability of StdCNNs and GCNNs to pixel-wise perturbations. We ﬁrst motivate the comparison of the two networks by considering pixel-wise attack on architectures as give in Table 1. We observe that as GCNNs become more robust to rotations they are more vulnerable to pixel-wise attacks, more so than StdCNNs.
Revised: Hence, we augment the networks with varying range of rotation from ± 0 ◦ to ± 180 ◦ and study in detail the vulnerability of StdCNNs and GCNNs to pixel-wise perturbations e.g. FGSM and PGD for MNIST, Fashion MNIST and CIFAR10 datasets.

Original: For a fairer comparison we make them roughly equivalent on parameter count by doubling the number of ﬁlters in StdCNNs and reducing the number of ﬁlters in GCNNs.

Original: Even with this change the relative behaviour of GCNNs to StdCNNs remains the same. In parallel we also show what happens when the networks are adversarially trained and tested with/without adversarial perturbations. We ﬁrst give complete details for the networks attacked with FGSM  and follow it up withPGD. For a ﬁner analysis of their vulnerability we plot their accuracy drop as a function of (cid:15) budget used in the attack for each network made robust to a speciﬁc range of rotations.
Revised: For MNIST and Fashion MNIST, we give the accuracy for the networks attacked with FGSM at various values of (cid:15) in the range [0 , 1] with and without test augmentation and follow it up with a comparion at (cid:15) = 0 .

Original: We augment both the train and test with random rotations from the same ﬁxed range to handle spatial perturbations, and see that GCNNs not only outperform StdCNNs but also maintain accuracy above 98% even for rotations in the larger range of 180 ◦ . With this setup, we check the vulnerability of the networks to FGSM perturbed inputs, with (cid:15) =0 .
Revised: For CIFAR10 we do the unrotated versus rotated test augmentation comparison at (cid:15) =

Original: Figure 4 clearly demonstrates that as GCNNs become more robust to larger rotations they become more vulnerable to FGSM attack. However, for small rotations GCNNs are deﬁnitely more robust to FGSM. but changed to make them parameter equivalent. Next, we check their performance with FGSM adversarial training and unperturbed test.
Revised: Accuracy drop with varying (cid:15)

Original: From Figure 5(left), as expected, GCNNs outperform StdCNNs in every count. However, when their parameter counts are roughly the same, we see in Figure 5(right) thatStdCNNs perform as well as GCNNs.
Revised: It is clear from the above experiments that as GCNNs become more robust to larger rotations, they become more vulnerable to pixel-wise attacks, in comparison to StdCNNs.

Original: Finally, we check the performance of these networks with both train and test data FGSM perturbed.

Original: Here also we also we see in Figure 6(left) that GCNNs perform better than StdCNNs.

Original: Once again, when their parameter counts are roughly the same, StdCNNs perform as well as GCNNs as shown in Figure 6(right).

Original: Similar to the FGSM attack studied above, we now check and compare the vulnerability ofthese networks to the stronger PGD attack(with (cid:15) = 0 .
Revised: We do a ﬁner analysis of the attacks with varying (cid:15) values, (cid:15) being the maximum perturbation allowed for the attack.

Original: 3 ) once they are made robust to rotation with augmentation. IntheplotsFigure 7(left) and 7(center) we compare the performance of the individual networks towards FGSM and PGD. And, as expected, we observe that inboth the networks PGD is a stronger attack than FGSM.
Revised: Plots in

Original: From Figure 7(right) we see that as in the case of FGSM attack, GCNNs are more vulnerable to PGD attack than StdCNNs as they become more robust to larger rotations. For smaller rotations GCNNs are still more robust to PGD than StdCNNs. This trend is similar when GCNNs and  StdCNNs are made roughly parameter count equivalent.
Revised: Figure 6 and 9 are for FGSM attack on StdCNNs with changing (cid:15) for MNIST and Fashion MNIST, respectively and Figure 12(left) is for FGSM attack on StdCNNs with (cid:15)

Original: These observations are given in Figure 8. In Figure 9, we also have checked the vulnerability of RotEqNet to pixel-wise perturbations like FGSM and PGD. There is no clear trend as in GCNNs.
Revised: = 0 .

Original: RotEqNet seem to be severely affected by

Original: FGSM even at no rotation augmentation. Their accuracy changes roughly from 40% to 60% as we augment the network from 0 ◦ to 180 ◦ . Accuracy drop with varying (cid:15) It’s clear from the above experiments that as GCNNs become more robust to larger rotations, they become more vulnerable to pixel-wise attacks, in comparison to StdCNNs.
Revised: 06 for CIFAR10.

Original: We do a ﬁner analysis of the attacks with varying (cid:15) values, (cid:15)being the maximum perturbation allowed for the attack.
Revised: Plots in Figure 7 and 10 are for FGSM attack on GCNNs with changing (cid:15) for MNIST and Fashion MNIST, respectively and 12(right) are for FGSM attack on GCNNs with (cid:15)

Original: Plots in
Revised: = 0 .

Original: Figure 10 are for FGSM and PGD attackon StdCNNs with changing (cid:15) and plots in Figure 11 are for FGSM and PGD attack on GCNNs with changing (cid:15) .
Revised: 06 for CIFAR10. For PGD attack plots please refer to Figure 15, 18 and 21(left) for StdCNNs and 16, 19 and 21(right) for GCNNs in the appendix.

Original: The labels a/a in the legend of the plots denote the [ − a ◦ , a ◦ ] range of  rotations augmented to train/test, respectively.
Revised: The labels a/a in the legend of the plots denote the [ − a ◦ , a ◦ ] range of random rotations augmented to train/test, respectively.

Original: We observe that even for (cid:15) as small as 0.1 the networks exhibit a behaviour similar to that seen from above experiments.
Revised: We observe that even for (cid:15) as small as 0.1 the networks exhibit a behaviour similar to that seen from above experiments.

Original: As GCNNs become robust to larger rotations they become more vulnerable to pixel-wise attacks even for smaller epsilon.
Revised: As GCNNs become robust to larger rotations they become more vulnerable to pixel-wise attacks even for smaller epsilon.

Revised: In Figure 33, we also have checked the vulnerability of RotEqNet for MNIST to pixel-wise perturbations like FGSM and PGD.

Revised: There is no clear trend as in GCNNs.

Revised: RotEqNet seem to be severely affected by FGSM even at no rotation augmentation.

Revised: Their accuracy changes roughly from 40% to 60% as we augment the network from ±

Revised: 0 ◦ to ± 180 ◦ range of random rotations.

Original:  D ETAILS OF EXPERIMENTS
Revised:  D ETAILS OF EXPERIMENTS

Original: All experiments performed on neural network-based models were done using  MNIST dataset with appropriate augmentations applied to the train/validation/test set.
Revised: All experiments performed on neural network-based models were done using MNIST, Fashion MNIST and CIFAR10 datasets with appropriate augmentations applied to the train/validation/test set.

Original: Data sets MNIST 3 dataset consists of 70 , 000 images of 28 × 28 size, divided into 10 classes.
Revised: Data sets MNIST 3 dataset consists of 70 , 000 images of 28 × 28 size, divided into 10 classes.

Revised: 55 , 000 used for training, 5 , 000 for validation and 10 , 000 for testing.

Revised: Fashion MNIST 4 dataset consists of 70 , 000 images of 28 × 28 size, divided into 10 classes.

Original: 55 , 000 used for training, 5 , 000 for validation and 10 , 000 for testing. {+CIFAR10 
+}
Revised: 55 , 000 used for training, 5 ,for validation and 10 , 000 for testing. {+CIFAR10 
+}

Revised: dataset consists of 60 , 000 images of 32 × 32 size, divided into 10 classes.

Revised: 40 , 000 used for training, 10 , 000 for validation and 10 , 000 for testing.

Original: Model Architectures For the MNIST  based experiments we use the 7 layer architecture of GCNN similar to Cohen & Welling (2016).
Revised: Model Architectures For the MNIST and Fashion MNIST based experiments we use the 7 layer architecture of GCNN similar to Cohen & Welling (2016).

Original: The StdCNN architecture is similar to the GCNN except that the operations are as per CNNs.
Revised: The StdCNN architecture is similar to the GCNN except that the operations are as per CNNs.

Original: Refer to Table 1 for details.
Revised: Refer to Table 1 for details.

Original: RotEqNet architecture is as given in Marcos et al.
Revised: RotEqNet architecture is as given in Marcos et al.

Original: [Table]
Revised: For the CIFAR10 based experiments we use the ResNet18 architecture as in He et al. (2016) and its equivalent in GCNN as given in Cohen & Welling (2016).

Revised: Input training data was augmented with random cropping and random horizontal ﬂips by default for all experiments.

Original: [Table]
Revised: [Table]

Revised: [Equation]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We observe that the robustness to geometric transformations in equivariant networks comes at the cost of their robustness to pixel-wise adversarial perturbations.
Revised: We observe that the robustness to geometric transformations in equivariant networks comes at the cost of their robustness to pixel-wise adversarial perturbations.

Original: We do an extensive comparative study of various equivariant network models ranging from StdCNNs to GCNNs, HNets, PTNs,
Revised: We do an extensive comparative study of various equivariant network models ranging from StdCNNs to GCNNs, HNets, PTNs,

Original: RotEqNets.
Revised: RotEqNets for MNIST, Fashion MNIST and StdCNNs, GCNNs for CIFAR10 datasets.

Original: We believe that good neural network models should be robust to both geometric transformations and pixel-wise adversarial perturbations, and understanding trade-offs similar to the ones in our paper is an important direction for future work.
Revised: We believe that good neural network models should be robust to both geometric transformations and pixelwise adversarial perturbations and understanding trade-offs similar to the ones in our paper is an important direction for future work

Original: A A PPENDIX
Revised: A A PPENDIX

Revised: [Table]

Revised: [Figure]

Revised: Apart from comparing StdCNN and GCNN with themselves that is when train has no rotation augmentation to train-augmented with rotation, we also compare the two networks by considering pixel-wise attack on architectures as given in Table 1 and 2.

Revised: The networks in Table 2 roughly become parameter equivalent and channel equivalent for each layer in the architecture hence, allowing a fairer comparison between the two.

Revised: We observe that as GCNNs become more robust to rotations they are more vulnerable to pixel-wise attacks, more so than StdCNNs.

Revised: In parallel we also show what happens when the networks are adversarially trained and tested with/without adversarial perturbations.

Revised: Please refer plots 22 to 28 for results on MNIST dataset.

Revised: Refer to plots 29 to 32 for results on Fashion MNIST.