Unadversarial Examples: Designing Objects for Robust Vision
Authors: Hadi Salman
Andrew Ilyas
Logan Engstrom
Sai Vemprala
Aleksander Madry
Ashish Kapoor
Venue: NeurIPS 2021 Poster (2021)
Keywords: adversarial robustness
adversarial examples
computer vision

Abstract: We study a class of computer vision settings wherein one can modify the design of the objects being recognized. We develop a framework that leverages this capability---and deep networks' unusual sensitivity to input perturbations---to design ``robust objects,'' i.e., objects that are explicitly optimized to be confidently classified. Our framework yields improved performance on standard benchmarks, a simulated robotics environment, and physical-world experiments. 

Unadversarial Examples: Designing Objects for Robust Vision
Hadi Salman' hady@mit, edu MITt
Andrew Ilyas* ailyas@mit momit.edu edu MIT
Logan Engstrom" engs ingstrommmit edu MIT
Sai Vemprala saihvêmic crosoft. com Microsoft Research
Aleksander Mędry madry@mit edu MIT
Ashish Kapoor akapoor@microsoo Microsoft Research
Abstract
We study class of computer vision settings wherein one can modify the de- sign of the objects being recognized. We develop framework that leverages this capability- and deep networks' unusual sensitivity to imput perturbations- de- sign "robust objects." i.c. objects that are explicitly optimized to be confidently classified. Our framework yields improved performance on standard benchmarks, simulated robotics environment, and physical-worf experiments.
Introduction
Performing reliably on unseen or shifting data distributions a difficult challenge for modern com- puter vision systems. For example, slight rotations and translations of images suffice to reduce the accuracy of state-of-the-art classifiers [ETT+19; ALG+19; KMF18). Similarly, models that attain near human-level performance on benchmarks exhibit significantly degraded performance when faced with even mild image corruptions and transformations [HD19; KSH+19). In fact, when an adversary is allowed to modify inputs directly, standard vision models can be manipulated into predicting arbitrary outputs (cf. adversarial examples [BCM+13; SZS+14J), While robustness in- terventions and additional training data can improve out-of-distribution behavior, they do not fully close the gap between model performance on standard heldout data and on corrutedd/otherwis un- familiar data ITDS+20; HBM+200. The situation is worse still when test-time distribution is under- or mis-specified, which occurs commonly in practice.
How can we change this state of affairs? We propose new approach to image recognition in the face of unforeseen corruptions or distribution shifts. This approach rooted reconsideration of the problem setup itself. Specifically, we observe that in many situations, system designer actually controls, to some extent, the inputs that fed into that model. For example, drone operator seeking to train a landing pad detector can modify the surface of the landing pad; and, roboticist training a perception model to recognize small set of custom objects can slightly alter the texture or design of these objects.
We find that such control over inputs can be leveraged to drastically improve our ability tackle computer vision tasks. In particular, it allows us to turn the input-sensitivity of modern vision systems from weakness into strength. Instead of optimizing inputs mislead models (c.g., as in adversarial examples), we can alter inputs to reinforce correct behavior, yielding what we refer as "unadversarial examples." Indeed, we show that even simple gradient-basee algorithm can
Eqqual contribution.
Work partially completed while at Microsoft Research. Our code is available at attps://githun wow.///ccrosootetuaaveeeeoooeeee
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Clean
Foggy
Dusty
Unadversarial Jet
M
Human--essined
We demonstrate that optimizing objects (e.g., the pictured jet) for pre-trained neural net- works can boost performance and robustness on computer vision tasks. Here, we show an example of classifying an unadversarial jet and standard jet using a pretrained ImageNet model. The model correctly classifies the unadversarial jet even under bad weather conditions (e.g., foggy or dusty), whereas it fails to correctly classify the standard jet.
Figure 1:
successfully construct unadversarial examples variety of vision settings and demonstrate that, by optimizing objects for vision systems (rather than vice-versa), we can significantly improve both in-distribution performance and robustness to unforeseen data shifts and corruptions.
We demonstrate the efficacy of our methods on both standard benchmarks (CIFAR, ImageNet) and robustness-basse benchmarks (ImageNet-C, CIFAR-C) while also comparing them broad set of baselines (e.g., QR codes or heuristically designed patches). To further highlight the practicality of our framework, (a) extend methods designing the texture of three-dimensional objects (rather than patches); (b) deploy unadversarial examples a simulated drone setting; and (c) ensure that the performance improvement yielded by the objects we design actually transfer to the physical world.
2 Motivation and approach
While vision models tend perform well on held-out data drawn from the same distribution as the training data, distr-diitribution inputs can severely degrade this performance. For example, models behave unreliably under distribution shifts induced by new data collection procedures [RRS+19] 19; EIS+20; TE11], 1], synthetic corruptions [HD19; KSH+19), spatial transformations [ETT+19; FF15], as well as under other types of shift.
Given a fixed type of distribution shift, a standard approach to increasing model robustness is explicitly train on regularize with data from the corresponding anticipated test distribution [KSH+19]. For example, Engstrom et al. [ETT+19] find that vision models trained on worst-case rotations and translations end up being fairly robust to rotation and translation- based distribution shifts. However, this approach is not without shortcomings for example, Kang et al. [KSH+19] find that training CIFAR classification models that are robust to #EG-compression in this manner requires significant sacrifice in natural accuracy. Recent works make similar observations in the context of other distribution shift mechanisms like ép adversaries [TSE+19; SZC+ 18; RXY+19] or texture swapping [GRM+19].
These observations give rise more general question: given that performing reliably in the face of constrained, well-specified distribution shifts is already a difficult challenge, how can we attain robustness to broad, unforeseen distribution shifts?
2.1 Leveraging more controlled vision settings
Consider the vision tasks of detecting a landing pad from a drone, or classifying manufacturing components from factory robot. In both these tasks, reliable in-distribution performance is necessity; still, a number of possible distribution shifts may oceur at deployment time. For example, the drone might approach the landing pad at an atypical angle, or have a view obstructed by snow,

---

smoke, or rain. Similarly, the factory robot may encounter objects in unfamiliar poses, or could be equipped with only a low-qualitynnoisy camera.
At first glance, dealing with these issues seems to require tackling the difficult problem of general distribution shift robustness discussed earlier in this section. However, there is in fact critical distinction between the scenarios considered above and vision tasks in their full generality. In par- ticular, in these scenarios and many others, the system designer has control over the physical objects that the model operates on. For instance, the designer of the drone's 's landing algorithm could paint the landing pad bright yellow. machine learning model trained to detect this custom landing pad might then be more effective than a model trained to detect a standard grey pad, especially in low- visibility conditions. Still, the particular choice to paint the landing pad yellow rather ad hoc, and likely rooted in the way humans recognize objects. Meanwhile, an abundance of prior work (e.g., (JBZ+19;; GRM+19: 8L IST+19]) demonstrates that humans and machine learning models tend to use different sets of features to make their decisions. This suggests that rather than relying on human priors, we should instead be asking: how can we build objects that are easily detectable by machine learning models?
2.2 Unadversarial examples
The task of making inputs less recognizable by computer vision systems has been focus of research in adversarial examples. Adversarial examples are small, carefully constructed perturbations to natural images that can induce arbitrary (mis)behavior from machine learning models [BCM+13; SZS+14]. These perturbations are typically constructed as the result of an optimization problem that maximizes the loss of machine learning model with respect to the input, i.e., by solving the optimization problem
8ade = arg max L(fo(x + 5).8). SEA
(1)
where fo is parameterized model (e.g.. neural network with weights 0); is a natural input; is
the corresponding correct label; L is the loss function used to train @ (e.g., cross-entropy loss) and
A is class of permissible perturbations (e.g., norm-bounded perturbations: A {6 ||01|, S }} for some small 6 > 0). Adversarial perturbations are typically crafted via projected gradient descent (PGD) [Nes03] in input space, standard iterative first-order optimization method-prio work in adversarial examples has shown that even a few iterations of PGD suffice to completely change the prediction of many state-of-the-an machine learning systems [MMS+18].
From adversarial examples to unadversarial objects. The goal of this work is to modify the design of objects so that they are more easily recognizable by computer vision systems. If we could specify every pixel of every image that a model encounters at test time, we could draw on the effectiveness of adversarial examples, and construct image perturbations (using PGD) that minimize the loss of the system, e.g., Sunado = arg min L(O, x + d,
SEA
In our setting of interest, however, having such fine-grained access to the test inputs is unrealistic (presumably, if we had precise control over every pixel in the input, we could just directly encode the ground-truth label directly in it). Instead, we have limited control over some physical objects; these objects are in turn captured within images, affected by many signals that are out of our control, such camera artifacts, weather effects, or background scenery.
It turns out that we can still draw techniques from adversarial examples research in this limited- control setting. Specifically, a recent line of work [KGB17: 77 SBB+16; EEF+18a; AEI+18] concerns itself with constructing robust adversarial examples [AEI+18], i.c., physically realizable objects that act as adversarial examples when introduced into a scene in any one of a variety of ways. For example, Sharif et al. [SBB+16] design glasses frames that cause facial recognition models mis- classify faces, Athalye et al. [AEI+18] 18] design -exture 3D models that are misclassified by state-of-the-aat ImageNet classifiers from many angles and viewpoints, and [BMR+18] design ad- versarial patches: stickers that can be placed anywhere on objects causing them to be misclassified. In this paper, we leverage the techniques developed in the above line of work to construct robust un-adversarial piieets-physicallly realizable objects optimized to minimize (rather than maximize) the loss of target classifier. In the next section, we will more concretely discuss our methods for generating unadversarial objects, then outline our evaluation setup.
3

---

Unadv patch
Unady texture
(a) An example unadversarial patch designed for the "tiger" class.
(b) An example unadversarial texture designed for jet 3D mesh (class "warplane") and applied rendered city backgrounds.
Figure 2: Examples of the two considered methods for constructing unadversarial objects.
2.3 Constructing unadversarial objects
In the previous section, we identified class of scenarios where system designer can, to some extent, control the objects that machine learning system model operates on. In these settings, we motivated so-called unadversarial examples as potential way to boost models' overall performance and robustness to distribution shifts. In this section, we present and illustrate two concrete algorithms for constructing unadversarial examples: unadversarial patches and unadversarial textures. In the former, we design sticker or "patch" [BMR+18] that can be placed on the object; the latter, we design the 3D texture of the object (in similar manner to the texture-base adversarial examples of Athalye et al. [AEI+18]). Example results from both techniques are shown Figure For simplicity, we will assume that the task being performed is image classification, but the techniques are directly applicable to other tasks as well. In all cases, we require access to a pre-trained model for the dataset of interest.
Unadversarial patches (cf. Algorithm in Appendix A). To train unadversarial patches (cf. Fig- ure 2a), in addition to the pre-trained model, we require sample access image-label pairs from the dataset of interest. At each iteration, we sample an image-label pair (z, y) from a training set, and place the patch corresponding class y onto the image with random orientation and position?. Since placing the patch is an affine transformation, after each iteration we can compute the gradient of the model's loss with respect the pixels in the patch, and take negative gradient step on the patch parameters. The algorithm terminates when the model's loss on sticker-boosted images plateaus, or after a fixed number of iterations.
Unadversarie textures (ef. Algorithm 2 in Appendix A). To train unadversarial textures (cf. Figure 2b), we do not require sample access to the dataset, but instead a set of 3D meshes for each class of objects that we would like to augment, as well as set of background images that we can use to simulate sampling a scene (these can be images from the dataset of interest, solid-color backgrounds, random patterns, etc.).
For each 3D mesh, our goal optimize 2D texture which improves classifier performance when mapped onto the mesh. At each iteration, we sample a mesh and random background; we then use 3D renderer (Mitsuba [NVZ+19]) map the corresponding texture onto the mesh. We overlay the rendering onto a random background image, and then feed the resulting composed image into the pre-trained classifier, with the label being that of the sampled 3D mesh. Since rendering is typically non-differentiable, we use a linear approximation of the rendering process (cf. Athalye et al. [AE[+18]) in order to compute (this time approximate) gradients of the model's loss with respect to the utilized texture. From there, we apply the same SGD algorithm as we did for the patch case.
3 Experimental evaluation
In order to determine the effectiveness of our proposed framework, we evaluate against a suite of computer vision tasks. Below, we first provide some detail on the precise access model and baselines that will be considered. We then briefly outline the experimental setup of each task, and show that
We allow the patch to be placed anywhere as a matter of convenience: ideally we would only be applying the patch onto the main object itself, but this would require bounding box data that we do not have for most classification datasets.

---

Figure 3: Clean (left) and corresponding corrupted (right) ImageNet images augmented with an unadversarial patch we use such images evaluate the efficacy of unadversarial patches in Section 3.2.
Brightness
Contrast
Defecus
Transtom
Fog
Accuracy (8)
50-
Accuracy (%)
Clean
100
Frost
Gaussian Noise
Glass
Impu Noise
JPLG Compression
Accurscy (%a
100-
50
5
20
45
Patched Portion
Motion Blur
Pixelate
Shot Noise
Snow
Zoom Blur
of the Image (%)
Accuracy (9%)
(a) Performance on ImageNet
Patched Portion the
1%6)
(b) Performance synthetically corrupted data (ImageNet-C)
Figure 4: Accuracy on (a) clean ImageNet images and (b) synthetically corrupted ImageNet-C im- ages function of patch size (given percentage of image area). In (b), each bar denotes the average accuracy over the five severities ImageNet-C, and the horizontal dashed lines report the accuracy the original (non-patched) datasets. Unadversarial patches consistently boost per- formance for both clean and corrupted images, with accuracy monotonically increasing with patch trained without any corruptions non-standard data augmentation in-the-
size. The patches loop (we train with the same augmentations that the pre-trained model itself was trained with).
unadversarial objects consistently improve the performance and robustness of the vision systems tested. For a more detailed account of each experimental setup, see Appendix C.
3.1 Access model and baselines
In many of the settings discussed thus far, system designer alter the objects being recognized but not allowed to alter the the classifier itself. That is, we are optimizing unadversarial objects for a fixed (pre-trained) model. For instance, road engineer may wish to design road signs that are easier to recognize for autonomous vehicles, without being able to train or alter the machine learning models that operate the vehicles. Similarly, roboticist might want to design landing pad that works better for a commercial (pre-trained) drone. We will refer to this setting as the fixed- model setting. On the other hand, sometimes the same entity is able train both the model and the transformations (the discussed factory robot example may fall into this category, for example). In this "free-model" setting, one may be able to boost performance by co-designing the machine learning model and the objects of interest.
In this work, will focus designing unadversarial examples in the fixed-model setting, for the sake of both simplicity and applicability. In particular, any valid algorithm under a fixed-model assumption is also a valid algorithm under the co-design assumption (but the converse is not true). This leaves the task of leveraging even more control in joint optimization settings as a potential avenue for future work.
Baselines. Since we consider the fixed-model setting throughout our paper, the only truly compa- rable baselines are those which do not alter the model being trained. Nonetheless, in order to fully contextualize our results, we will also consider a few baselines that fall outside of our intended ac- cess model (e.g., QR codes). A notable disadvantage of these baselines (that we do not explicitly
5

---

Accuracy (%)
100
Patch-based
50
Class-based
Patch and class match
9 10
CIFAR-10 class
Figure 5: The accuracy of pretrained ResNet-50 on boosted CIFAR-10 when the image and the unadversarial patch used to boost the image have: (1) same class, (2) conflicting classes. For the conflicting classes setting, we report the accuracies based on the class of the patch (red bars), and those based on the class of the image (blue bars). When there is conflicting signal between the patch and the image, the model relies more on the image.
demonstrate below) is that if the "unadversarial signal" (c.g., the QR code) occluded or removed, the entire system fails; this is in contrast to the fixed-model setting, where the traitranne model is able to recognize objects without any unadversarial signals and is instead "boosted" by their pres- ence.
3.2 Clean data and synthetic corruptions
We first test whether unadversarial examples improve the performance of image classifiers on beneh- mark datasets. Using the algorithm described in Section 2.3, we construct unadversarial patches of varying size for pre-trained ResNet-50 classifiers on the CIFAR [Kri09] and ImageNet [RDS+15] datasets. For evaluation, we add these patches at random positions, scales, and orientations to val- idation set images (see Appendix for the exact protocol). As shown in Figure 4a, the pre-trained ImageNet classifier is consistently more accurate the augmented ImageNet images. For example, an unadversarial patch 20 times smaller than ImageNet images boosts accuracy by 26.3% (analogous results for CIFAR are given in Appendix D).
Robustness to synthetic corruptions. Next, we use the CIFAR-C and ImageNet-C datasets [HD19] (consisting of the CIFAR and ImageNet validation sets corrupted in 15 systematic ways) to see whether the addition of unadversarial patches to images confers any corruption robustness.
We use the same patches and evaluation protocol that we used when looking at clean data (to ensure fair evaluation, we apply corruptions boosted images only after the unadversarial patches have been applied). As consequence, at test time neither model nor patch has been exposed to any image corruptions beyond standard data augmentation. As a result, this experiment tests the ability for unadversarially boosted images to withstand completely unforeseen corruptions; we also avoid any potential biases from training on (and thus "overfittin"" to [KSH+ 19]) a specific type of corruption. The results (cf. Figure 4b for ImageNet and Appendix D for CIFAR) indicate that unadversarial patches do improve performance across corruption types; for example, applying an unadversarial patch 5% the size of standard ImageNet image boosts accuracy by an average of 31.7% points across corruptions
The model does not ignore the image in the presence of unadversarial patches.
Recall from our discussion of the fixed-model setting in Section 3.1 that an advantage of designing unadversarial objects without changing the model that the model still works in the absence of the unadversarial signal. We now briefly explore the case where the model is exposed an unadversarial signal for the wrong class. Ideally, we would want the patch to only assist/boost the signal from the original image-in in particular, we do want the patch make the model totally ignore the contents of the image itself. Thus, in cases where the signal from the image and the patch conflict, we would like the classifier predict according to the features present in the image more frequently than the class encoded in the unadversarial patch.
"Since the original corruption benchmarks proposed by [HD19] are only available as pre-computed JPEGs (for which we cannot apply a patch pre-corruption) or CPU-based Python image operations (which were pro- hibitively slow), we re-implemented all 15 corruptions as batched GPU operations; we verified that model accuracies on our corruptions mirrored the original CPU counterparts (i.e., within 1% accuracy). For more details about reimplementation,
code release.

---

Dark jee
jet Bright jet
60-
Dright jet
Acurorth
40-
40-
20-
20-
10
0-
Figure 6: The jet unadversarial example task. We show example conditions under which we evaluate the objects, along with aggregate statistics for how well an ImageNet classifier classifies the objects in different conditions. We find that the classifiers perform consistently better on the unadversarial jet texture over the standard jet texture both standard and distributionally shifted conditions. We also give baseline of a white jet with lighter texture because of the poorly visibility inherent in simulator; we find performed worse than even the standard jet.
UnAdy
UnAdv
UnAdv
(a) Bus
Containes Slip
TraileTTurkk
Figure 7: Additional examples reporting aggregate statistics for how well ImageNet classifier classifies various objects different conditions. Again, we find that the classifiers perform consis- tently better on the unadversarial objects texture over the standard objects.
Indeed, this turns out to be the case. Figure 5 shows that on CIFAR-10, when the signal from an unadversarial patch and the image itself conflict, the model predicts according to the patch only 31.2% of the time on average, and according to the image 60.3% of the time (the accuracy of the model when the patch and image agree 98. 08.93%.)
At first glance, this result may seem to at odds with the near-perfect effectiveness of adversarial patches [BMR+18]. However, the phenomenon observe here can be tied to the subtle difference between the way we train our unadversarial patches and the way one trains targeted adversarial patches. In the former, overlay each patch exclusively onto images from its respective class thus, unadversarial patches are never optimized to be effective when overlaid on different class. In the latter, however, adversarial patches optimized to maximize confidence in particular class on all possible backgrounds, making the patch dominant even when overlaid on an image from a different class.
Baselines. We also compare our results to a variety natural baselines; the most relevant of these is the "best loss image patch," where we use the minimum-loss training image in place of a patch. We compare with this bascline to ensure that our method doing something beyond this naive way to add signal to image. The results are shown in Appendix D, along with comparisons to less sophisticated baselines, such as QR Codes and predefined random Gaussian noise patches.
3.3 Classification in 3D simulation
We now test unadversarial examples in a more practical setting: recognizing 3D objects in high-fidelity simulator. collect meshes corresponding to four ImageNet classes: "warplane," "minibus," "container ship." and "trailer truck," from sketchfab. com. We generate a texture for each object using the unadversarial texture algorithm of Section 2.3, using the ImageNet validation set as the set of backgrounds for the algorithm, and pre-trained ResNet-50 as the classifier.
7

---

Unikd Pae
Typrisi Pod
Typical Padd
Teokcal
2
Severity of Fog
Sevenity of Fog
Figure 8: Drone landing task. On the left we show the unadversarial versus standard landing pads. On the right we show the results for the task when both the standard and unadversarial landing pads are used. We find that the drone consistently takes less time to land, and has higher chance of landing correctly, when detecting the unadversarial landing pad.
To evaluate the resulting textures, we import each mesh into Microsoft AirSim, a high-fidelity three- dimensional simulator; we then test pre-trained ImageNet models' ability to recognize each object with and without the unadversarial texture applied variety of surroundings. We also test each texture's robustness to more realistic weather corruptions (snow and fog) built directly into the sim- ulator (rather than applied as post-processing step). We provide further detail on AirSim and our usage of it in Appendix B. Examples of the images used to evaluate the unadversarial textures, as well as our main results for one the meshes are shown in Figure We find that in both standard and adverse weather conditions, the model consistently performs better the unadversarial texture than on the original. We present similar results for the other three meshes in Figure 7.
3.4 Localization for (simulated) drone landing
We then assess whether unadversarial examples can help outside of the classification setting. Again using AirSim, we set up drone landing task with a perception module that receives as input an axis-aligned acrial image of a landing pad, and tasked with outputing an estimate of the camera's (Ir y)-position relative to the pad. While this task is quite basic, we are particularly interested in studying performance in the presence of heavy (simulated) weather-based corruptions. The drone is equipped with pretrained regression model that localizes the landing pad (described in detail in Appendix B). We optimize an unadversarial texture for the surface of the landing pad best help the drone's 's regression model in localization. Figure shows an example of the landing pad localization task, along with the performance of the unadversarial landing pad compared to the standard pad. The drone landing on the unadversarial pad consistently lands both more reliably.
3.5 Physical-world mnadversarial examples
Finally, we move out of simulation and test whether unadversarial patches that we generate can survive naturally-arisin distribution shift from effects such as real lighting, camera artifacts, and printing imperfections. We use four houschold objects (a toy racecar, miniature plane, coffeepot, and eggnog container), and print out (on standard InkJet printer) the adversarial patch corresponding to the label of each object. We take pictures of the toy with and without the patch taped on using an ordinary cellphone camera, and count the number of poses for which the toy is correctly classified by pre-trained ImageNet classifier. Our results are in Table 9a, and examples of patches are in Figure 9b. Classifying both patched and unpatched images over diverse set of poses, we find that the adversarial patches consistently improve performance even at uncommon object orientations.
4 Related work
Here, we first highlight (and differentiate from) previous works using reference markers to improve recognition and localization. We will then discuss related work in adversarial robustness.
Improving computer vision with fiducial markers. past research, vision-based precision land- ing was initially attempted with classical computer vision based tracking of reference designs

---

Airzlana with patch (85% accuracy)
Cur with patch (86% accuracy)
Class
No Patch
Patch
"racer"
22%
83%
"eggnog" 22%
44%
"coffee pot"
39%
56%
"warplane" 67%
83%
(a) Accuracy pre-trained ResNet- 18 on photographs of real world ob- jects with and without patches.
Airplano without patch (07% acourau))
without patch (2755 acouraeg)
(b) Examples photos of the "warplane" and "racer" physical ob- jects taken with (top) and without (bottom) an unadversarial patch.
Figure 9: whylical-world experiments. We take pictures of objects at diverse orientations while varying the presence of patch on the object. Note that don't 't do any additional data augmentation on the patches, which are the same used in our previous ImageNet benchmark experiment.
[SMS03; LSP09; YSS+14: FZS+17], or on arrangements of fiducial markers [LZT+19]] (a fidu- cial marker is a fixed pattern or object that is placed in scene reference point for location or measure). Several types of visual fiducial markers were also proposed for robust tracking and pose estimation applications through classical vision [Fia05: GMM+ 14; Ols RMM19]. While fiducial markers such as AprilTTags [Ols11] were commonly used in robotics, they suffer from limitations such as orientation uncertainty, accuracy falloff depending on viewing angles, and short detection ranges [AAB+19; WO16], which motivated research into using convolutional neural networks for landing pad detection and pose estimation [NAK+I8; TNN+ 19J. Our work attempts to unify these perspectives by leveraging the expressivity of neural network-based systems to design robust un- adversarial examples. Another key difference between fiducial markers and unadversarial objects is that the former require vision systems to be aware of their presence; in contrast, the latter designed on top of pre-trained systems. This means that we do not require any further non non-standard model training, and that we do not depend as heavily on the unadversarial example being visible.
Adversarial robustness. Our design of unadversarial examples motivated by the success of ad- versarial examples, i.e., minute perturbations to the inputs of machine learning models that can induce nearly arbitrary behavior. Adversarial examples were shown to be effective in seriously hampering machine vision related tasks such as classification [SZS+14; EEF+18a; AEI+18]. 88) ob- ject deteetiio//eemmnnatioo
IEEF+1Sb; AMT18; LK19; XWZ+17] and visual question-answering (VQA) [XCL+18]. Furthermore, prior work has shown that adversarial examples can be constructed even without direct access to the vision system being manipulated [CZS+17; PMG16; PMG+17; IEA+18; IEM18] Synthesized physical adversarial examples were shown to be effective in fooling person detection [TVG19], sign detection for autonomous driving 1SBM+18; BHG+19]. Robotic platforms such manipulators were also shown to be sensitive to vision based adversarial examples [MDB+17], and to specifically designed adversarial objects [WTL+19]] Additionally, recent work [EGS19; NHD+19] shows that can "reprogram" neural networks using adversarial examples, e.g., one can construct a patch that causes a CIFAR-10 classifier to operate as an MNIST classifier.
5 Discussion and conclusions
In this work, we demonstrated that it is possible to design object alternations that boost the cor- responding classifiers' performance, even under strong and unforeseen distribution shift. Indeed, such resulting unadversarial objects are robust to a broad range of data shifts and corruptions, even when these were never seen in training. We view our results as a promising route towards increasing out-of-distribution robustness of computer vision models.
Limitations and future work. One limitation of our method is that it requires differentiability with respect to properties of the input object of interest, or a close proxy (e.g. a differentiable simulator) that can mimick the object and the environment in which the object operates.
Furthermore, as discussed at length in Section 3. 1, in this work we only consider the case where the system designer cannot alter the model being trained. While this carries some inherent advantages (e.g., increased applicability), future work should investigate whether attaining even more accuracy and robustness is possible by jointly optimizing models and unadversarial objects.

---

Finally, we have shown unadversarial examples to be effective only in classification and regression settings. However, the fact that unadversarial examples and adversarial examples share the same underlying generation technique is evidence that unadversarial examples could apply to any system that is vulnerable to adversarial examples. Thus, an avenue of future work could be apply these examples to detection, segmentation, style transfer, and other domains.
Broader impact. On hand, as with any improvement the robustness and reliability of com- puter vision systems, unadversarial examples also carry the potential for misuse (c.g., as part of overreaching surveillance technology). On the other hand, unadversarial examples give end users of machine learning systems-ve who typically have little to no power over the behaviour of the ML systems they interact with the power to improve the performance and robustness of these systems in their respective use cases. For example, in the future someone living in a remote area where roads or traffic signs may look "atypical" can slightly modify their signs to better recognized by autonomous vehicles.
Acknowledgements
We are grateful to lan Engstrom for helping take the photographs for the physical-world experiments.
Work supported in part by the NSF grants CCF- 1553428 and CNS-1115221, Open Philanthropy, and the Microsoft Corporation. This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR001 120C0015.
Research sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-199 177550019--10000 1000. The views and conclusions contained in this document those of the authors and should not be interpreted as representing the official policies, either expressed or im- plied, of the United States Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
10

---

References
[AAB+19] Syed Muhammad Abbas, Salman Aslam, Karsten Berns, and Abubakr Muhammad. "Analysis and Improvements in AprilTag Based State Estimation", In: Sensors (2019). Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. "Synthesizing Robust Adversarial Examples". In: International Conference on Machine Learning (ICML). 2018.
[AEI+18]
[ALG+19] Michael A Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen. "Strike (with) pose: Neural networks are easily fooled by strange poses of familiar objects". In: Conference on Computer Vision and Pattern Recogni- tion (CVPR). 2019.
[AMT18) Anurag Amab, Ondrej Miksik, and Philip HS Torr. "On the robustness of semantic segmentation models to adversarial attacks". In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, pp. 888-897.
[BCM+13] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndié, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. "Evasion attacks against machine learn- ing at test time". In: Joint European conference on machine learning and knowledge discovery in databases (ECML-KDD). 2013.
[BHG+19] Adith Boloor, Xin He, Christopher Gill, Yevgeniy Vorobeychik, and Xuan Zhang- "Simple physical adversarial examples against end-to-end autonomous driving mod- els". In: 2019 IEEE International Conference on Embedded Software and Systems (ICESS). 2019.
[BMR+18] Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Ad- versarial Putch. 2018. arXiv: 1712. 09665 [cs.CV],
[CZS+17] Pin-Yo Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. "Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models". In: Workshop on Artificial Intelligence and Security. 2017.
[EEF+18a]
Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. "Robust Physical-Wonl World Attacks on Ma- chine Learning Models". In: Conference on Computer Vision and Pattern Recognition (CVPR). 2018.
[EEF+18b]
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, Tadayoshi Kohno, and Dawn Song. "Physical Adversarial Ex- amples for Object Detectors". In: CoRR (2018).
[EGS19] Gamaleldin Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. "Adversarial Repro- gramming Neural Networks". In: International Conference on Learning Represen- tations (ICLR). 2019.
[EIS+20]
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Jacob Stein- hardt, and Aleksander Madry. "Identifying Statistical Bias in Dataset Replication". In: International Conference on Machine Learning (ICML). 2020.
[ETT+19]
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. "Exploring the Landscape of Spatial Robustness". In: International Confer- ence on Machine Learning (ICML). 2019.
[FF15]
Alhussein Fawzi and Pascal Frossard. "Manitest: Are classifiers really invariant?" In:
British Machine Vision Conference (BMVC). 2015.
[Fia05]
Mark Fiala. "ARTag, fiducial marker system using digital techniques". In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'0S). 2005.
[FZS+17]
Davide Falanga, Alessio Zanchettin, Alessandro Simovic. Jeffrey Delmerico, and Da- vide Scaramuzza. "Vision-based autonomous quadrotor landing on moving plat- form".. In: 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR). 2017.
[GMM+14]
Sergio Garrido-Jurade Rafael Manuel Jesús Marin- Jiménez. "Automatic generation and detection of highly reliable fiducial markers under occlusion". In: Pattern Recognition (2014).
Francisco José Madrid-Cuevas. and

---

[GRM+19]
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Betbge, Felix A. Wichmann, and Wieland Brendel. "ImageNet- CNNs are biased towards tex- ture; increasing shape bias improves accuracy and robustness." In: International Con- ference on Learning Representations (ICLR). 2019.
[HBM+20] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The Many Faces of Robustness: A Critical Analysis of Out-of-Distriiutiion Generalization. 2020. arXiv: 2006. 16241 [cs.CV].
[HD19]
Dan Hendryeks and Thomas G. Dietterich. "Benchmarking Neural Network Robust- ness to Common Corruptions and Surface Variations". In: International Conference on Learning Representations (ICLR). 2019.
[[A+188]
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. "Black-box Adversar- ial Attacks with Limited Queries and Information". In: International Conference Machine Learning (ICML). 2018.
[[EM18]
Andrew Ilyas, Logan Engstrom, and Aleksander Madry, "Prior Convictions: Black- box Adversarial Attacks with Bandits and Priors". In: International Conference on Learning Representations (ICLR). 2018.
[IST+19]
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. "Adversarial Examples Are Not Bugs, They Are Features". In: Neural Information Processing Systems (NeurIPS). 2019.
[JBZ+19]
Jorn- Henrik Jacobsen, Jens Behrmann, Richard Zemel, and Matthias Bethge. "Exces- sive Invariance Causes Adversarial Vulnerability", In: International Contemporary on Learning Representations. 2019.
[JLT18]
Saumya Jetley, Nicholas Lord, and Philip Torr. "With friends like these, who needs ad- versaries?" In: Advances in Neural Information Processing Systems (NeurIPS). 2018.
[KGB17]
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. "Adversarial Machine Learn- ing at Scale". In: International Conference on Learning Representations (ICLR). 2017.
[KMF18]
Can Kanbak, Seyed- Moosavi-Dezfooli, and Pascal Frossard. "Geometric ro- bustness of deep networks: analysis and improvement". In: Conference on Computer Vision and Pattern Recognition (CVPR). 2018.
[Kri09]
Alex Krizhevsky. "Learning Multiple Layers of Features from Tiny Images". In: Tech- nical report. 2009.
[KSH+19]
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. "Testing Robustness Against Unforeseen Adversaries". In: ArXiv preprint orxiv:1908 1ox.0:9008.00066 2019.
[LK19]]
Mark Lee and Zico Kolter. "On physical adversarial patches for object detection". In: arXiv preprint arXiv: 1906. 1897 (2019).
[LSPP99]
Sven Lange, Niko Sunderhauf, and Peter Protzel. "A vision based onboard approach for landing and position control of an autonomous multirotor UAV in GPS-denied environments". In: 2009 International Conference on Advanced Robotics. 2009.
[LZT+19]] Xuancen Liu, Shifeng Zhang, Jiayi Tian, and Longbin Liu. "An Onboard Vision- Based System for Autonomous Landing of a Low-Cost Quadrotor on a Novel Landing Pad". In: Sensors (2019).
[MDB+17] Marco Melis, Ambra Demontis, Battista Biggio, Gavin Brown, Giorgio Fumera, and Fabio Roli. "Is deep learning safe for robot vision? adversarial examples against the icub humanoid". In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017, pp. 751-759.
[MMS+18] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. "Towards deep learning models resistant to adversarial attacks". In: International Conference on Learning Representations (ICLR). 2018.
[NAK+18]
Phong Ha Nguyen, Muhammad Arsalan, Ja Hyung Koo, Rizwan Ali Naqvi, Noi Quang Truong, and Kang Ryoung Park. "LightDense A fast and accurate marker tracker for autonomous UAV landing by visible light camera sensor on drone". In: Sensors (2018).
[Nes03]
Yurüi Nesterov. Introductory Lectures on Convex Optimization. 2003.
12

---

[NHD+19]
Paarth Neekhara, Shehzeen Hussain, Shlomo Dubnov, and Farinaz Koushanfar. "Ad- versarial Reprogramming of Text Classification Neural Networks". In: Empirical Methods in Natural Language Processing (EMNLP). 2019.
[NVZ+19] Merlin Nimier-David Delio Vicini, Tizian Zelmnr, and Wenzel Jakob. "Mitsuba 2: A Retargetable Forward and Inverse Renderer". In: Transactions on Graphics (Proceed- ings of SIGGRAPH Asia) 38.6 (2019). DOI: 10. (145//355000 3356498.
[OIs1]]
Edwin Olson. " "AprilTag: A robust and flexible visual fiducial system". In: 2011 1 IEEE International Conference on Robotics and Automation. 2011.
[PMG+17]
Nicolas Papernot, Patrick McDaniel, lan Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. "Practical black-box attacks against machine learning". In: Asia Conference Computer and Communications Security, 2017.
[PMG16] Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. "Transferability in Machine Learning: from Phenomena to Black-box Attacks using Adversarial Samples". In: ArXiv preprint arXiv: 07277. 2016.
[RDS+15]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. "ImageNet Large Scale Visual Recognition Challenge". In: In- ternational Journal of Computer Vision (IJCV). 2015.
[RMM19] Francisco Romero-Ramire, Rafael Muñoz-Salinas, and Rafael Medina-Carnice "Fractal Markers: a new approach for long-range marker pose estimation under oc- clusion". In: JEEE Access (2019).
[RRS+19] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. "Do Ima- geNet Classifiers Generalize to ImageNet?" In: International Conference on Machine Learning (ICML). 2019.
[RXY+19] Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C Duchi, and Percy Liang. "Adversarial Training Can Hurt Generalization" In: arXiv preprint arXiv: 1906.00999000... (2019).
[SBB+16]
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. "Accessorize to a Crime: Real and Stealthy Attacks on ofate-of-the-Art Face Recognition". In: Pro- ceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016. 2016, pp. 1528-1540.
[SBM+18] Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Mung Chiang, and Pra- teck Mittal. "Darts: Deceiving autonomous cars with toxic signs". In: arXiv preprint #xXiv:882.004444 (2018).
[SIE+20]
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. "Do Adversarially Robust ImageNet Models Transfer Better?" In: Advances in Neural Information Processing Systems (NeurIPS). 2020.
[SMS03] Srikanth Saripalli, James F Montgomery, and Gaurav S Sukhatme. "Visually guided landing of an unmanned acrial vehicle". In: JEEE transactions on robotics and au- tomation (2003).
[S7C+18]
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, and Yupeng Gao. "Is Robustness the Cost of Accuracy? A Comprehensive Study on the Robustness of 18 Deep Image Classification Models". In: European Conference on Computer Vision (ECCV). 2018
[S]S+14] Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. "Intriguing properties of neural networks". In: ternational Conference on Learning Representaations (ICLR). 2014.
[TDS+20]
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. "Measuring Robustness to Natural Distribution Shifts in Image Classification". In: Neural Information Processing Systems (NeurIPS). 2020.
[TE11]
Antonio Torralba and Alexei A Efros. "Unbiased look at dataset bias". In: CVPR 2011. 2011.
[TNN+19]
Noi Quang Truong, Phong Ha Nguyen, Se Hyun Nam, and Kang Ryoung Park. "Deep learning-bea based super-resolution reconstruction and marker detection for drone land- ing". In: IEEE Access (2019).
13

---

[TSE+19]
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Alek- sander Madry. "Robustness May Be at Odds with Accuracy". In: International Con- ference Learning Representations (ICIR). 2019.
[TVG19] Simen Thys, Wiebe Van Ranst, and Toon Goedemé. "Fooling automated surveillance cameras: adversarial patches to attack person detection". In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2019.
[WO16]
John Wang and Edwin Olson. "AprilTag 2: Efficient and robust fiducial detection". In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2016.
[WTL+19] D. Wang. D. Tseng, Li, Y. Jiang, M. Guo, M. Danielezuk, Mahler, J. Ichnowski, and K. Goldberg. "Adversarial Grasp Objects". In: 2019 IEEE 15th International Con- ference on Automation Science and Engineering (CASE). 2019.
[XCL+18] Xiaojun Xu, Xinyun Chen, Chang Liu, Anna Rohrbach, Trevor Darrell, and Dawn Song. "Fooling vision and language models despite localization and attention mech- anism". In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. pp. 4951-4961.
[XWZ+17] Cihang Xie, Jianyu Wang. Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. "Adversarial examples for semantic segmentation and object detection", In: Proceed- ings of the IEEE International Conference on Computer Vision. 2017, pp. 1369-1378.
[YSS+14) Shaowu Yang, Sebastian A Scherer, Konstantin Schauwecker, and Andreas Zell. "Au- arbitrarily textured landing site using onboard
tonomous landing of MAVs monocular vision". In: Journal of Intelligent & Robotic Systems (2014).
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] The code link is in the abstract and Appendix C.6.
(b) Did you specify all the training details (e.g., data splits, hyperparametens, how they were chosen)? [Yes] See Appendix C.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No] Generating each adversarial patch time-intensive and so we did not randomize over the entire patch creation process. We did average over thousands of images at evaluation time, with number samples detailed each respective section. We note that although there are no emor bars with respect to the patch creation process, the patches consistently perform much better than the baseline (on the order of tens of percentage points).
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix C.5.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] We use ImageNet and CIFAR-10, and we cite them both in the main paper.
(b) Did you mention the license of the assets? [Yes] See Appendix C.4.
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "k981NRrJw0I", "reviewer": "Reviewer_eMCH", "summary": "In this work, the authors leverage techniques from the adversarial examples literature to design patches and textures that aid in classifying an object *correctly.* In a fixed-model setting, they construct patches / textures using gradient-based methods and demonstrate effectiveness on clean CIFAR-10 and ImageNet and various robustness benchmarks such as ImageNet-C (common perturbations). The authors also compare their patches / textures to simple baselines such as QR codes, smaller-sized images from the training set, and predefined fixed patterns.", "questions": "", "limitations": "", "rating": 9, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: Applying gradient-based adversarial perturbations in this setting (aiding the end user who benefits from classification) is unique and interesting. The work is an interesting application of the \"adversarial patch\" concept, \"adversarial reprogramming,\" and PGD. Related work is adequately cited.\n\nQuality: The experiments are well-designed, and the claims are well-supported by the results. The baselines presented in the appendix are thorough. The authors do a good job acknowledging practical constraints by users (i.e. line 101, recognizing that users have only limited control over physical objects). The methods are sound and well-thought out -- for example, the corruptions are applied correctly (after the patch), and the setting where patch & class conflict is studied. Code is attached to the submission and clearly organized at a glance. \n\nClarity: The submission is very clearly written and organized. The introduction and motivations are explained thoroughly and well. There are a few nits:\n\n* line 82: recent work (ex: Ford and Gilmer et al. 2019) describes how adversarial examples do not have to be restricted to \"carefully constructed perturbations.\" Although the authors of this paper implicitly acknowledge different kinds of distribution shifts in section 2, maybe the paper could benefit from changing the statement in line 82 to \"Adversarial examples can include small, carefully constructed perturbations...\"\n\n* figure 5 typo: s/\"red bards\"/\"red bar\"\n\nFinally, it would be very interesting to see how the authors ended up at the patch size they chose. Essentially, how does figure 5 change as a result of the patch size?\n\nSignificance: This work motivates a new line of work into constructing \"helper\" perturbations to aid in classification. The results are compelling, and the open-sourced code hopefully will make it easy for others to build on this work. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and societal impact are addressed adequately in section 5.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "Ubx6kzBdxa", "reviewer": "Reviewer_mos2", "summary": "The paper considers a scenario where a fixed computer vision system (such as CNN based classification or regression models) is given and one tries to adapt the design of objects in order to increase their chance of being correctly detected by the system. This setting is similar to the design of marker systems for computer vision with the difference that the detection algorithm is considered fixed.\n\nTwo approaches are presented to design (i) patches that can be printed and sticked onto objects and (ii) complete textures that cover the surface of 3D objects. Both approaches follow the design of adversarial perturbations with the difference that instead of maximizing the loss of the system, it is now minimized to improve its performance instead of weakening it.\n\nExperiments evaluate how these approaches can improve the overall performance of the given system as well as its robustness against a variety of different corruptions. Experiments are performed on synthetic 2D and 3D data, as well as on real objects, where the designed patches are printed out and attached to physical objects. Overall, they demonstrate consistent and significant improvements over non-modified objects and simple baselines.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**\n\nThe idea to turn the table and use adversarial attacks to ones advantage is quite original and refreshing. Especially the experiments on real objects with attached stickers of the designed patches demonstrate potential use cases for such an approach. While it does not provide much methodological novelty compared to adversarial attacks, it demonstrates some novel, desirable and surprising effects of such a simple change from adversarial examples to unadversarial examples. In particular, the fact that the model still works on unmodified objects and the observation that it prefers the actual object over a conflicting patch have positive consequences for applications. Finally, it will be interesting to see how such an approach can benefit other applications besides classification and regression.\n\n**Presentation**\n\nThe paper is well written and structured. The most important details to understand the idea are clearly presented in the main paper and the supplementary and code provide additional details which should make the results reproducible.\n\n**Strengths**\n\n- The paper contributes an original idea to turn adversarial attacks, which are associated with decreased robustness of CNNs, into something to increase the robustness of CNN based models (in a specific setting).\n- It provides some novel and interesting results regarding conflicting signals that arises from such an unadversarial optimization of patches.\n- It provides an extensive and convincing evaluation, which includes evaluation of overall performance and robustness against corruptions for varying sizes of unadversarial patches and textures, and includes an evaluation of printed patches attached to physical objects. The evaluation shows consistently positive results.\n- The approach is applicable to already deployed systems to boost their performance in the presence of unadversarial markers without any negative effect in the absence of these markers.\n\n**Weaknesses**\n\n- It is not quite clear how realistic the scenario is for applications but this is something hard to judge objectively and I am including it only as a minor point.\n- While the proposed approach has the important advantage of being applicable to pretrained, fixed models, it is still important to get a sense of how well this performs in comparison to more specialized marker approaches. The authors seem to somewhat agree with this by including QR codes as a baseline. However I do not think that this baseline accurately reflects the quality of markers for computer vision. In particular, Grinchuk, O. et al. “Learnable Visual Markers.” NeurIPS (2016) outperform QR codes and moreover, it is very close to \"jointly optimizing models and unadversarial objects\" (l.298) - something which according to the paper  under review should be explored in future works.\n\n**Comments and Suggestions**\n\n- It took me a while to get a concrete understanding of the considered setting. Until l.23 the description is rather abstract and I think it would be quite helpful to provide a concrete example already in the abstract (e.g. that which is actually considered in the paper, sticking a patch onto objects).\n- l.151/Sec. 3.1 Why speak of a \"threat\" model here? My best guess is that this is related to adversarial attacks but in the current work I don't really see a \"threat\" model since there is no adversary (except maybe for image corruptions but I wouldn't call them adversarial).\n- Fig. 1 and its caption provide very little context which makes it hard to understand without already knowing the considered task.\n- Fig. 5 immediately left me wondering if this does not contradict results regarding adversarial patches; my concerns were well addressed in footnote 3 but I think this is a rather important point and might as well be included in the main text.\n- l.217-218 a quick summary of the results that QR code detection is less robust than the proposed unadversarial examples could be included here.\n\n**Rating**\n\nThe paper is well written, contributes an interesting idea and performs a solid evaluation thereof. The main issue that I see is a missing discussion of Grinchuk et al. 2016 and comparison to this work and other, existing marker systems besides QR codes. However, the fact that the proposed approach provides additional advantages over such existing marker systems, in particular the fact that the system gracefully falls back to its usual performance if the \"marker\" is missing without being affected negatively, lessens this drawback somewhat such that I recommend acceptance. Nevertheless, I would still highly recommend to at least discuss these approaches and potentially compare to them.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper discusses limitations regarding the requirement for differentiability and additional aspects that should be explored in future work, such as settings where both markers and models are optimized and the applicability to tasks other than classification and regression. It discusses impacts regarding dual use but also the potential to influence already employed models towards the better.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "11"}, "extra_text": {}}, {"review_id": "NN73hYNByCW", "reviewer": "Reviewer_5UTM", "summary": "This work proposes and studies two techniques to generate unadversarial images (images which cause a computer vision deep learning model to more reliably give accurate predictions). One technique is for creating unadversarial patches which can be overlaid on testing images to increase the robustness of a pre-trained classifier, the other is for creating unadversarial textures for 3D meshes which when applied to 3D meshes. The work provides strong empirical evidence for unadversarial patches on ImageNet and CIFAR, including results where images are corrupted after applying the patches. Evidence is provided for unadversarial textures on simulated data and a small scale real world experiment.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n* The paper is very clearly written, with good figures and relevant additional details provided in the appendix.\n* Strong empirical evidence for the effectiveness of unadversarial patches, including for cases where strong corruptions are applied to the images. The work provides thorough analysis with respect to patch size, simple baseline patch techniques. Further, there is evidence that when a patch is applied to a different class from the intended one it overall does not cause the model to classify according to the patch and ignore the image entirely.\n* The ability to ensure visual classifier robustness for pretrained classifiers is highly desirable, especially for real world applications like fiducial marker tracking. As a result I find this work very well motivated. The drone landing simulated experiment points to the real world utility of the proposed techniques.\n\nWeaknesses\n* The takeaways that can be made from the experiments in AirSim using four ImageNet classes are not entirely clear. This is because the number of 3D models used for each class for these experiments is not specified. The results presented in Figures 6 and 7 are impressive, but it’s currently difficult to understand their significance without knowing the scale of the experiment.\n* A similar weakness regarding the scale holds for the real world experiments. While it’s good to see that the patterns of improving accuracy with adversarial patches hold, experiments with only four objects do not represent strong evidence.\n* While the ImageNet experiments validate the approach, there’s a lack of sufficient evidence for how applicable the proposed techniques are on real data from real world tasks. An important direct application is fiducial marker detection, and an experiment where a CNN-based fiducial marker detector is made more robust as a result of applying unadversarial patch techniques is well within scope. Such an empirical investigation would significantly strengthen this work.\n\nMinor\nTypo in figure 5 where there’s “bards” rather than “bars”\n\n**Update After Discussion Period**\n\nThank you to the authors for engaging in thoughtful discussions with the reviewers. The issues pertaining to the scale of the simulation and real-world experiments remain. Making physical patches and applying them to objects (for the real world), or applying them as textures (in 3D simulation) directly shows the utility of unadversarial patches. The evidence of experiments in these settings would be more significant if there are more instances per category at test time, given that the problem is object classification and not instance classification. I therefore keep my initial rating: 6 - marginally above acceptance threshold.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "Gjl3fJNdeux", "reviewer": "Reviewer_aFae", "summary": "This work introduces and studies \"unadversarial\" input modifications (patches or object textures) designed to *increase* classifier accuracy, particularly in corrupted images. The work is inspired by research on adversarial examples (in particular \"adversarial patches\" which trick a classifier into predicting a particular class when the patch is present in the scene, regardless of what other class might be present).\n\nGiven access to a known, fixed, pre-trained network, a small patch (or the entire object texture) are optimized by gradient descent to increase prediction accuracy. When applied, the method is shown to be quite effective for increasing classification accuracy (and performance on a simple regression task) under a variety of simulated image corruptions (blur, fog, etc.). In addition, the patch method is shown to significantly aid classification even when printed and affixed to real physical objects.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited?*\n\nReliance on prior works is clearly and properly acknowledged throughout the paper. The method in this paper appears to be heavily inspired by \"Adversarial Patches\" [BMR+18], however, in addition to the novel \"unadversarial\" setting, this paper treads new ground in a few areas:\n* Introducing an effective way to optimize \"unadversarial\" object texture (using the method of [AEI+18])\n* Showing robustness to a wide variety of artificial image corruptions\n* Showing robustness to a variety of patch sizes and viewing angles ([BMR+18] were not particularly robust here)\n* Applying the method to a regression setting (drone landing pad localization)\nOne setting that is absent is a \"black box\" network where patch training is performed on one or more known networks, but testing is performed w.r.t. a *previously unseen network*.\n\n*Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?*\n\nThe experiments are well designed and clearly described, and the authors discuss limitations/strengths/weaknesses/relation to prior work throughout the text. The claims are well supported experimentally, with two caveats:\n\n1. *Results are limited to a classification setting:* As acknowledged by the authors, nearly all results are on image classification problems. While classification is a fair and reasonable problem setting, it is not likely to be the primary setting in which the method would find real world application. And there may be non-trivial issues with extending this work to other problems, e.g. object detection. (a) While patches may increase the *saliency* of an object, it may not help (and could even harm) prediction of bounding boxes or instance segmentations. (b) Multiple patches in the same image could conflict with each other or overwhelm other prediction signals. (The results of Fig. 5 are encouraging in that patches are shown to be effective improving classification scores without dominating the visual features of the object during classification, but further experiments are needed.)\n\n2. *Results are limited to a white-box network/dataset setting:* Again this limitation is acknowledged and is not unreasonable (though some cited hypothetical examples, e.g. line 159, would be hard to fit in this setting). The results would be stronger with an examination of the effectiveness of patches learned on Network A when seen by Network B. Also of interest here is whether the dataset is important, e.g. can patches learned with a network trained on KITTI perform well when tested on a network trained for Cityscapes?\n\n*Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)*\n\nThe paper is well written and organized (including supplementary), figures are polished and clear, and it seems adequately informative for reproduction.\n\n*Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?*\n\nWhile there is little methodologically novel here, the task and experiments are quite interesting and raise practical questions for further research. For instance, QR tags (and similar) are quite common in robotics settings; could learning \"unadversarial\" patches be preferable in some settings? If these results hold for black-box network settings, what impact could this have on safety and robustness with autonomous agents? The experiments in this work are a useful start.\n\n*Bottom line*\nI find the paper to be very well written, methodologically straightforward, but experimentally good, and potentially impactful both practically and toward further research. That said, the conclusions would be much more informative with two further experimental settings (object detection and transfer from network A to network B or dataset A to dataset B), and it is difficult to comprehensively judge the importance of this work without them.\n\n*Notes*\n* Line 159: Designing road signs would likely require the \"black box\" network setting, so it doesn't seem to be a good example for the \"fixed network\" setting. Similar in line 309.\n* Fig. 4, Fig. 5, Fig. 6, and Fig. 9: Does \"Accuracy\" refer to \"Top-1 Accuracy\" in each of these cases? Only Fig. 7 specifically names \"Top-1 Accuracy\".\n* Line 181, \"we construct unadversarial patches of varying size\": Is there a single learned patch that is scaled to various sizes when pasted on test images?\n* Fig 5: \"(red bards)\" should be \"(red bars)\"\n* Supplementary, Line 732: \"we fix the -a- set of patches to [a] predefined pattern\"", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The primary limitation addressed by the authors is that the method requires differentiability w.r.t. the patch or texture of interest. Implicit in this the need to access to the weights of a fixed, pre-trained network that is later used (unmodified) at test time. This latter implication is not always clearly considered, particularly when hypothesizing about applications of the work, as listed in the previous section. Additional care should be taken here.\n\nAs noted above, further limitations include demonstrating few results outside of classification problems. In particular, no results are shown with multiple visible patches or where size or pose of the object are predicted.\n\nRegarding societal impact, all methods applied in this paper were first explored in the context of creating *adversarial* examples, designed to trick a classifier. This paper leverages such methods to *improve* classification when subjects can be modified, e.g. by applying a sticker to an object. The paper mentions the potential for misuse in this type of work and also notes some potential benefits when end users have the ability to increase or decrease classifier performance even without being able to control the classifier itself.", "review_text": "", "extra_scores": {"ethical_concerns": "See previous section. The authors acknowledge the ethical implications of this and related works.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "GY_jN07MM_I", "reviewer": "Reviewer_np1y", "summary": "This paper exploits the  known-sensitivity of deep models to perturbation in the input data in order to improve (rather than decrease) the performance of a pre-trained model in a classification task. Specifically, the authors propose to alter the input data (i.e. images)  by 1) adding a patch to the image 2) altering the texture of specific objects. The paper shows experimentally that this improves both in-domain performance and robustness to data corruptions.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pro:\n\n- The idea is novel and intuitive.\n- The paper is well-written, easy to follow and well-motivated.\n- The experimental results shows significant boost in terms of both in-domain performance and corrupted data.\n\nCons.\n- The technical contribution is a bit limited (mostly an extension of known techniques from the adversarial robustness literature).\n\nQuestions:\n- Are the patches/texture specific to the architecture or do they generalize among different architectures and pre-trained models? Some analysis in the generalization capabilities of the unadversarial examples could make the technical contribution stronger. \n\n- Recently, several methods have exploited the idea of \"test-time\" optimization to also improve both in-domain performance and out-of-domain performance. For example [1] and [2] .  While they do not act directly on the input image as here ( e.g. [1] optimizes some parameter of the models, and [2] first project the image to a latent space), the idea is somewhat similar e.g. gradient-based optimization at test time to maximize performance of the pre-trained model.  I would have loved seeing a comparison of those methods vs directly  altering the image (un-adversarial examples) since the last one could be more interpretable  but 1 and 2 does not require to directly modify the image.\n\n[1] Tent: Fully Test-time Adaptation by Entropy Minimization ICLR 2021\n\n[2] Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization  CVPR 2021\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6h"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers agreed that this is a solid contribution to NeurIPS. Reviewers agreed that providing more convincing evidence related to the 3D simulation and physics experiments would have significantly strengthened the paper. 
Decision: Accept (Poster)