Invariance vs Robustness of Neural Networks

Abstract: Neural networks achieve human-level accuracy on many standard datasets used in image classification. The next step is to achieve better generalization to natural (or non-adversarial) perturbations as well as known pixel-wise adversarial perturbations of inputs. Previous work has studied generalization to natural geometric transformations (e.g., rotations) as invariance, and generalization to adversarial perturbations as robustness. In this paper, we examine the interplay between invariance and robustness. We empirically study the following two cases:(a) change in adversarial robustness as we improve only the invariance using equivariant models and training augmentation, (b) change in invariance as we improve only the adversarial robustness using adversarial training. We observe that the rotation invariance of equivariant models (StdCNNs and GCNNs) improves by training augmentation with progressively larger rotations but while doing so, their adversarial robustness does not improve, or worse, it can even drop significantly on datasets such as MNIST. As a plausible explanation for this phenomenon we observe that the average perturbation distance of the test points to the decision boundary decreases as the model learns larger and larger rotations. On the other hand, we take adversarially trained LeNet and ResNet models which have good \ell_\infty adversarial robustness on MNIST and CIFAR-10, and observe that adversarially training them with progressively larger norms keeps their rotation invariance essentially unchanged. In fact, the difference between test accuracy on unrotated test data and on randomly rotated test data upto \theta , for all \theta in [0, 180], remains essentially unchanged after adversarial training . As a plausible explanation for the observed phenomenon we show empirically that the principal components of adversarial perturbations and perturbations given by small rotations are nearly orthogonal

Reviews and author responses:

This paper examines the interplay between the related ideas of invariance and robustness in deep neural network models. Invariance is the notion that small perturbations to an input image (such as rotations or translations) should not change the classification of that image. Robustness is usually taken to be the idea that small perturbations to input images (e.g. noise, whether white or adversarial) should not significantly affect the model's performance. In the context of this paper, robustness is mostly considered in terms of adversarial perturbations that are imperceptible to humans and created to intentionally disrupt a model's accuracy. The results of this investigation suggests that these ideas are mostly unrelated: equivariant models (with architectures designed to encourage the learning of invariances) that are trained with data augmentation whereby input images are given random rotations do not seem to offer any additional adversarial robustness, and similarly using adversarial training to combat adversarial noise does not seem to confer any additional help for learning rotational invariance. (In some cases, these types of training on the one hand seem to make invariance to the other type of perturbations even worse.)

This paper is mostly clear and reasonably written. However, I do not think that the results of this investigation are significant enough to warrant publication at ICLR. In particular, I'm not sure that I really understand the motivation of this research question. I suppose a full notion of robustness would include invariance to perturbations of all types -- whether adversarial or otherwise -- and one might hope that techniques for encouraging such resilience mutually reinforce each other. However, since they are human perceptible, the perturbations associated with invariance are exactly the features that are not associated with adversarial noise, so I don't see why they should be related at all. 

From a different perspective, invariance is a property of the true data distribution -- a rotated version of an image of a cat is another valid sample from the underlying distribution -- the invariance property is a type of constraint tying together different elements of the data generating distribution. On the other hand, adversarially perturbed images are often thought to be "off the data manifold" -- i.e. not valid samples from the true underlying distribution. Given this perspective, I am confused about why I should expect to see any interplay between these two ideas. The fact that the authors do not find any interplay is reasonable, but I remain confused about why they were investigating this question in the first place. 

Unfortunately, this means that I don't think this work meets the significance criterion for being published at ICLR. 

This paper shows empirically that rotational invariance and l infinity robustness are orthogonal to each other in the training procedure. However, the reviewer has the following concerns,

It is already shown in (Engstrom et al., 2017) that models hardened against l infinity-bounded perturbations are still vulnerable to even small, perceptually minor departures from this family, such as small rotations and translations. What is the message beyond that paper that the authors would like to convey?
The experiments are only on MNIST and CIFAR-10.  Training on a larger dataset  like imagenet would make the experiments more convincing.
Going beyond the observation, what shall we do to improve against different perturbation simultaneously? Or is it an impossible task to improve on both?



This paper analyzes the behaviour of DNN trained with rotated images and adversarial examples. Namely, the paper analyzes the relationship between training with rotated images and the robustness to adversarial perturbations, and vice-versa.

The paper has several technical issues that need to be resolved before drawing any conclusions:

1) “invariance”: this term is not used in the correct way. The fact that the network has the same accuracy when before and after rotation does not mean that the output layer is invariant to rotation. Note invariance in the output layer is a more stringent criterion as it requires that the images get labeled in the same way. The same accuracy can be achieved with completely different labelings of the images. What this paper is evaluation is robustness to rotation vs robustness to adversarial perturbations.

2) It is unclear that Figure 3 is saying that adversarial training does not affect the rotation invariance because there is a general drop of accuracy. The analysis could have been done by evaluating how many images are labelled differently after the rotation, and all the plots will be aligned at 0 degrees.

3) Finding out the robustness to adversarial perturbations is an NP-hard problem. So, for all tested cases in the paper, there could be a perturbation that damaged the model much more than the ones found, which could change the conclusions of the analysis. 

4) The networks compared in the two experiments are different networks. There could be a network dependency.

Also, I find the paper poorly written (eg. in the abstract: "Neural networks achieve human-level accuracy on many standard datasets used in image classification.” -> what does it mean “human-level accuracy”?; "The next step is to achieve better generalization to natural (or non-adversarial) perturbations” -> why is this the next step?). 


Meta-review: This paper examines the interplay between the related ideas of invariance and robustness in deep neural network models. Invariance is the notion that small perturbations to an input image (such as rotations or translations) should not change the classification of that image. Robustness is usually taken to be the idea that small perturbations to input images (e.g. noise, whether white or adversarial) should not significantly affect the model's performance. In the context of this paper, robustness is mostly considered in terms of adversarial perturbations that are imperceptible to humans and created to intentionally disrupt a model's accuracy. The results of this investigation suggests that these ideas are mostly unrelated: equivariant models (with architectures designed to encourage the learning of invariances) that are trained with data augmentation whereby input images are given random rotations do not seem to offer any additional adversarial robustness, and similarly using adversarial training to combat adversarial noise does not seem to confer any additional help for learning rotational invariance. (In some cases, these types of training on the one hand seem to make invariance to the other type of perturbations even worse.)

Unfortunately, the reviewers do not believe the technical results are of sufficient interest to warrant publication at this time. 

Decision: reject