user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Adversarial Examples Are a Natural Consequence of Test Error in Noise

Paper content:
Over the last few years, the phenomenon of adversarial examples --- maliciously constructed inputs that fool trained machine learning models --- has captured the attention of the research community, especially when the adversary is restricted to making small modifications of a correctly handled input.
At the same time, less surprisingly, image classifiers lack human-level performance on randomly corrupted images, such as images with additive Gaussian noise.
In this work, we show that these are two manifestations of the same underlying phenomenon.
We establish this connection in several ways.
First, we find that adversarial examples exist at the same distance scales we would expect from a linear model with the same performance on corrupted images.
Next, we show that Gaussian data augmentation during training improves robustness to small adversarial perturbations and that adversarial training improves robustness to several types of image corruptions.
Finally, we present a model-independent upper bound on the distance from a corrupted image to its nearest error given test performance and show that in practice we already come close to achieving the bound, so that improving robustness further for the corrupted image distribution requires significantly reducing test error.
All of this suggests that improving adversarial robustness should go hand in hand with improving performance in the presence of more general and realistic image corruptions.
This yields a computationally tractable evaluation metric for defenses to consider: test error in noisy image distributions.
State-of-the-art computer vision models can achieve superhuman performance on many image classification tasks.
Despite this, these same models still lack the robustness of the human visual system to various forms of image corruptions.
For example, they are distinctly subhuman when classifying images distorted with additive Gaussian noise BID12 , they lack robustness to different types of blur, pixelation, and changes in brightness BID17 , lack robustness to random translations of the input BID2 , and even make errors when foreign objects are inserted into the field of view BID25 .
At the same time, they also are sensitive to small, worst-case perturbations of the input, so-called "adversarial examples" BID28 .
This latter phenomenon has struck many in the machine learning community as surprising and has attracted a great deal of research interest, while the former seems to inspire less surprise and has received considerably less attention.Our classification models make errors on two different sorts of inputs: those found by randomly sampling from some predetermined distribution, and those found by an adversary deliberately searching for the closest error to a given point.
In this work, we ask what, if anything, is the difference between these two types of error.
Given that our classifiers make errors in these corrupted image distributions, there must be a closest such error; do we find that this closest error appears at the distance we would expect from the model's performance in noise, or is it in fact "surprisingly" close?The
answer to this question has strong implications for the way we approach the task of eliminating these two types of errors. An
assumption underlying most of the work on adversarial examples is that solving it requires a different set of methods than the ones being developed to improve model generalization. The
adversarial defense literature focuses primarily on improving robustness to small perturbations of the input and rarely reports improved generalization in any distribution.We claim that, on the contrary, adversarial examples are found at the same distance scales that one should expect given the performance on noise that we see in practice. We
explore the connection between small perturbation adversarial examples and test error in noise in two different ways.First, in Sections 4 and 5, we provide empirical evidence of a close relationship between test performance in Gaussian noise and adversarial perturbations. We
show that the errors we find close to the clean image and the errors we sample under Gaussian noise are part of the same large set and show some visualizations that illustrate this relationship. (This
analysis builds upon prior work which makes smoothness assumptions on the decision boundary to relate these two quantities.) This
suggests that training procedures designed to improve adversarial robustness might reduce test error in noise and vice versa. We provide
results from experiments which show that this is indeed the case: for every model we examined, either both quantities improved or neither did. In particular
, a model trained on Gaussian noise shows significant improvements in adversarial robustness, comparable to (but not quite as strong as) a model trained on adversarial examples. We also found
that an adversarially trained model on CIFAR-10 shows improved robustness to random image corruptions.Finally, in Section 6, we establish a relationship between the error rate of an image classification model in the presence of Gaussian noise and the existence of adversarial examples for noisy versions of test set images. In this setting
we can actually prove a rigorous, model-independent bound relating these two quantities that is achieved when the error set is a half space, and we see that the models we tested are already quite close to this optimum. Therefore, for
these noisy image distributions, our models are already almost as adversarially robust as they can be given the error rates we see, so the only way to defend against adversarial examples is to reduce test error.In this work we will investigate several different models trained on the MNIST, CIFAR-10 and ImageNet datasets. For MNIST and
CIFAR-10 we look at the naturally trained and adversarially trained models which have been open-sourced by BID22 . We also trained
the same model on CIFAR-10 with Gaussian data augmentation. For ImageNet, we
investigate Wide ResNet-50 trai]ned with Gaussian data augmentation. We were unable to
study the effects of adversarial training on ImageNet because no robust open sourced model exists (we considered the models released in BID29 but found that they only minimally improve robustness to the white box PGD adversaries we consider here). Additional training
details can be found in Appendix A.
We proved a fundamental relationship between generalization in noisy image distributions and the existence of small adversarial perturbations.
By appealing to the Gaussian isoperimetric inequality, we formalized the notion of what it means for a decision boundary to be badly behaved.
We showed that, for noisy images, there is very little room to improve robustness without also decreasing the volume of the error set, and we provided evidence that small perturbations of clean images can also be explained in a similar way.
These results show that small-perturbation adversarial robustness is closely related to generalization in the presence of noise and that future defense efforts can measure progress by measuring test error in different noise distributions.Indeed, several such noise distributions have already been proposed, and other researchers have developed methods which improve generalization in these distributions BID17 BID12 a; BID30 BID35 .
Our work suggests that adversarial defense and improving generalization in noise involve attacking the same set of errors in two different ways -the first community tries to remove the errors on the boundary of the error set while the second community tries to reduce the volume of the error set.
The isoperimetric inequality connects these two perspectives, and suggests that improvements in adversarial robustness should result in improved generalization in noise and vice versa.
Adversarial training on small perturbations on CIFAR-10 also improved generalization in noise, and training on noise improved robustness to small perturbations.In the introduction we referred to a question from BID28 about why we find errors so close to our test points while the test error itself is so low.
We can now suggest an answer: despite what our low-dimensional visual intuition may lead us to believe, these errors are not in fact unnaturally close given the error rates we observe in noise.
There is a sense, then, in which we simply haven't reduced the test error enough to expect to have removed most nearby errors.While we focused on the Gaussian distribution, similar conclusions can be made about other distributions.
In general, in high dimensions, the -boundary measure of a typical set is large even when its volume is small, and this observation does not depend on anything specific about the Gaussian distribution.
The Gaussian distribution is a special case in that we can easily prove that all sets will have large -boundary measure.
BID23 proved a similar theorem for a larger class of distributions.
For other data distributions not every set has large -boundary measure, but under some additional assumptions it still holds that most sets do.
An investigation of this relationship on the MNIST distribution can be found in Gilmer et al. (2018b, Appendix G) .We
believe it would be beneficial for the adversarial defense literature to start reporting generalization in noisy image distributions, such as the common corruption benchmark introduced in BID17 , rather than the current practice of only reporting empirical estimates of adversarial robustness. There
are several reasons for this recommendation.1. Measuring
test error in noise is significantly easier than measuring adversarial robustnesscomputing adversarial robustness perfectly requires solving an NP-hard problem for every point in the test set BID19 . Since BID28
, hundreds of adversarial defense papers have been published. To our knowledge
, only one BID22 has reported robustness numbers which were confirmed by a third party. We believe the
difficulty of measuring robustness under the usual definition has contributed to this unproductive situation. 2. Measuring test
error in noise would also allow us to determine whether or not these methods improve robustness in a trivial way, such as how the robust MNIST model learned to threshold the input, or whether they have actually succeeded in improving generalization outside the natural data distribution. 3. All of the failed
defense strategies we examined failed to improve generalization in noise.For this reason, we should be highly skeptical of defense strategies that only claim improved l p -robustness but do not demonstrate robustness in more general settings. 4. Finally, if the goal
is improving the security of our models in adversarial settings, errors in the presence of noise are already indicative that our models are not secure. Until our models are perfectly
robust in the presence of average-case corruptions, they will not be robust in worst-case settings. The usefulness of l p -robustness
in realistic threat models is limited when attackers are not constrained to making small modifications.The interest in measuring l p robustness arose from a sense of surprise that errors could be found so close to correctly classified points. But from the perspective described
in this paper, the phenomenon is less surprising. Statistical classifiers make a large
number of errors outside the data on which they were trained, and small adversarial perturbations are simply the nearest ones. Table 3 : The models from Section 1
trained and tested on ImageNet with Gaussian noise with standard deviation σ; the column labeled 0 refers to a model trained only on clean images.

assistant: text: Small adversarial perturbations should be expected given observed error rates of models outside the natural data distribution.