S YNTHESIZING R OBUST A DVERSARIAL E XAMPLES
Anonymous authors
Paper under double-blind review
A BSTRACT
Neural network-based classiﬁers parallel or exceed human-level accuracy on many common tasks and are used in practical systems.
Yet, neural networks are susceptible to adversarial examples, carefully perturbed inputs that cause networks to misbehave in arbitrarily chosen ways.
When generated with standard methods, these examples do not consistently fool a classiﬁer in the physical world due to a combination of viewpoint shifts, camera noise, and other natural transformations.
Adversarial examples generated using standard techniques require complete control over direct input to the classiﬁer, which is impossible in many real-world systems.
We introduce the ﬁrst method for constructing real-world 3D objects that consistently fool a neural network across a wide distribution of angles and viewpoints.
We present a general-purpose algorithm for generating adversarial examples that are robust across any chosen distribution of transformations.
We demonstrate its application in two dimensions, producing adversarial images that are robust to noise, distortion, and afﬁne transformation.
Finally, we apply the algorithm to produce arbitrary physical 3D-printed adversarial objects, demonstrating that our approach works end-to-end in the real world.
Our results show that adversarial examples are a practical concern for real-world systems.
I NTRODUCTION
The existence of adversarial examples for neural networks has until now been largely a theoretical concern.
While minute, carefully-crafted perturbations can cause targeted misclassiﬁcation in a neural network, adversarial examples produced using standard techniques lose adversariality when directly translated to the physical world as they are captured over varying viewpoints and affected by natural phenomena such as lighting and camera noise.
This suggests that practical systems may not be at risk because adversarial examples generated using standard techniques are not robust in the physical world.
classiﬁed as turtle classiﬁed as riﬂe classiﬁed as other at every viewpoint by an ImageNet classiﬁer.
An unperturbed model is classiﬁed correctly as a turtle 100% of the time.
See https://youtu.be/YXy6oX1iNoA for a video where every frame is fed through the classiﬁer: the turtle is consistently classiﬁed as a riﬂe.
We show that neural network-based classiﬁers are vulnerable to physical-world adversarial examples.
We introduce a new algorithm for reliably producing physical 3D objects that are adversarial over a distribution of viewpoints.
Figure 1 shows an example of an adversarial object constructed using our approach, where a 3D-printed turtle is consistently classiﬁed as riﬂe by an ImageNet classiﬁer.
In this paper, we demonstrate the efﬁcacy and generality of our method, demonstrating conclusively that adversarial examples are a concern in real-world systems.
1. C HALLENGES
Methods for transforming ordinary two-dimensional images into adversarial examples are wellknown; however, the extent to which these techniques affect speciﬁc real-world systems is still an open question.
Prior work has shown that adversarial examples generated using standard techniques lose their adversarial nature once subjected to minor transformations (Luo et al., 2016;
Lu et al., 2017).
These results suggest that adversarial examples, while serving as interesting phenomena, have no hope of applying to physical-world systems, where transformations such as viewpoint changes and camera noise are inevitable.
Prior techniques attempting to synthesize robust adversarial examples for the physical world have had limited success.
While some progress has been made, current efforts have demonstrated a small number of datapoints on nonstandard classiﬁers, and only in the two-dimensional case, with no clear generalization to three dimensions (further discussed in Section 4).
The entirety of prior work has only explored generating adversarial examples in the two-dimensional case, where “viewpoints” can be approximated by an afﬁne transformations of an original image.
Constructing adversarial examples for the physical world requires the ability to generate entire 3D adversarial objects, which must remain adversarial in the face of complex transformations not applicable to 2D objects, such as 3D rotations and perspective projection.
1. C ONTRIBUTIONS
In this work, we deﬁnitively show that adversarial examples pose a real threat in the physical world.
We propose a general-purpose algorithm for reliably constructing adversarial examples robust over a chosen distribution of transformations, and we demonstrate the efﬁcacy of this algorithm in both the 2D and 3D case.
We succeed in producing physical-world 3D adversarial objects that are robust over a large, realistic distribution of 3D viewpoints, proving that the algorithm produces adversarial three-dimensional objects that are adversarial in the physical world.
Speciﬁcally, our contributions are as follows:
• We develop Expectation Over Transformation (EOT), a novel algorithm that produces single adversarial examples that are simultaneously adversarial over an entire distribution of transformations •
We consider the problem of constructing 3D adversarial examples under the EOT framework, viewing the 3D rendering process as part of the transformation, and we show that the approach successfully synthesizes adversarial objects
• We fabricate adversarial objects and show that they remain adversarial, demonstrating that our approach works end-to-end in the physical world, showing that adversarial examples are of real concern in practical deep learning systems
A PPROACH
First, we present the Expectation Over Transformation (EOT) algorithm, a general framework allowing for the construction of adversarial examples that remain adversarial over a chosen transformation distribution T .
We then describe our end-to-end approach for generating adversarial objects using a specialized application of EOT in conjunction with differentiating through the 3D rendering process.
2. E XPECTATION O VER T RANSFORMATION
When constructing adversarial examples in the white-box case (that is, with access to a classiﬁer and its gradient), we know in advance a set of possible classes Y and a space of valid inputs X to the classiﬁer; we have access to the function P ( y | x ) and its gradient ∇ P ( y | x ) , for any class y ∈ Y and input x ∈ X .
In the standard case, adversarial examples are produced by maximizing the log-likelihood of the target class over a (cid:15) -radius ball around the original image, that is:
[Equation]
This approach has been shown to be both feasible and effective at generating adversarial examples for any given classiﬁer.
However, prior work has shown adversarial examples’ inability to remain adversarial even under minor perturbations inevitable in any real-world observation (Luo et al., 2016;
Lu et al., 2017).
To address this issue, we introduce Expectation Over Transformation (EOT) . The key insight behind
EOT is to model such perturbations within the optimization procedure.
Rather than optimizing the log-likelihood of a single example, EOT uses a chosen distribution T of transformation functions t taking an input x (cid:48) supplied by the adversary to the “true” input t ( x (cid:48) ) perceived by the classiﬁer.
Furthermore, rather than simply taking the norm of x (cid:48) − x to constrain the solution space, EOT instead aims to constrain the effective distance between the adversarial and original inputs, which we deﬁne as:
[Equation]
Intuitively, this is how different we expect the true input to the classifer will be, given our new input.
Then, EOT solves the following optimization problem
[Equation]
In practice, the distribution T can model perceptual distortions such as random rotation, translation, or addition of noise.
However, EOT generalizes beyond simple transformations. EOT ﬁnds examples robust under any perception distribution Q ( · ; x ) parameterized by the generated example x as long as ddx Q ( · ; x ) is well-deﬁned.
The objective function can be optimized by stochastic gradient descent, approximating the derivative of the expected value through sampling transformations independently at each gradient descent step and differentiating through the transformation.
2. S YSTEM O VERVIEW
Given its ability to synthesize robust adversarial examples, we use the EOT framework for generating 2D examples, 3D models, and ultimately physical-world adversarial objects.
Within the framework, however, there is a great deal of freedom in the actual method by which examples are generated, including choice of T , distance metric, and optimization method.
In the 2D case, we design T to approximate a realistic space of possible distortions involved in printing out an image and taking a natural picture of it.
This amounts to a set of random transformations of the form t ( x ) =
Ax + (cid:15) , which are more thoroughly described in Section 3.
In the 3D case, we deﬁne the transformations t from a texture to a perceived image by mapping the texture to a given 3D object, and simulating highly nonlinear functions such as rendering, rotation, translation, and perspective distortion of the object in addition to the perceptual mechanisms used in the 2D case.
Finding adversarial examples across these nonlinear transformations is what allows for the transfer of adversarial examples to the physical world, but also introduces implementation complexities not found in the 2D case.
EOT requires the ability to differentiate though the transformation distribution with respect to the input. In the 3D case, this implies differentiating through the 3D renderer with respect to the texture.
To do this, we model the rendering process as a sparse matrix multiplication between the texture and a coordinate map, which is generated from a standard 3D renderer.
Rather than attempting to differentiate through the complex renderer, we use the renderer to ﬁnd a coordinate map for each rendering, which maps coordinates on the texture to coordinates in the classiﬁer’s ﬁeld of view.
We can then simulate this speciﬁc rendering as a sparse matrix multiplication and differentiate through it; we differentiate through different matrix multiplications at each sampling step.
Once EOT has been parameterized, i.e. once a distribution T is chosen, the issue of actually optimizing the induced objective function remains.
Rather than solving the constrained optimization problem given above, we use the Lagrangian-relaxed form of the problem, as Carlini & Wagner (2017) do in the conventional (non-EOT, single-viewpoint) case:
[Equation]
In order to encourage imperceptibility of the generated images, we set d ( x (cid:48) , x ) to be the (cid:96) 2 norm in the LAB color space, a perceptually uniform color space where Euclidean distance roughly corresponds with perceptual distance.
Note that the E t ∼ T [ || LAB ( t ( x )) − LAB ( t (ˆ x ))
|| 22 ] can be sampled and estimated in conjunction with E [ P ( y | t ( x ))] ; in general, the Lagrangian formulation gives EOT the ability to intricately constrain the search space (in our case, using LAB distance) at insigniﬁcant computational cost (without computing a complex projection).
Our optimization, then, is:
[Equation]
We use projected gradient descent to ﬁnd the optimum, clipping to the set of valid inputs (e.g. [0 , 1] for images).
E VALUATION
We show that we can reliably produce transformation-tolerant adversarial examples in both the 2D and 3D case.
Furthermore, we show that we can synthesize and fabricate 3D adversarial objects, even those with complex shapes, in the physical world: these adversarial objects remain adversarial regardless of viewpoint, camera noise, and other similar real-world factors. Adversariality in this case refers to the propensity of the classiﬁer to predict an attacker-chosen target class given an image or object.
In our evaluation, given a source object x and a set of correct classes { y 1 , . . .
y n } , as well as an attacker-chosen target class y adv and a crafted adversarial example x (cid:48) , we use the following terms in order to characterize the effectiveness of x (cid:48) .
We call a viewpoint “adversarial” if the top output of the classiﬁer is y adv ; “correct” if the adversarial example fails and the classiﬁer outputs one of { y 1 , . . .
, y n } , and “misclassiﬁed” if the classiﬁer predicts a class unrelated to that of the source object (i.e. not in { y 1 , . . . y n } ) but not the target class y adv . By randomly sampling a set of viewpoints, we can evaluate an adversarial example x (cid:48) by examining the proportion of adversarial, correct, and misclassiﬁed viewpoints. We evaluate robust adversarial examples in the 2D and 3D case, and furthermore, we evaluate physical-world 3D adversarial objects.
The two cases are fundamentally different.
In the virtual case, we know that we want to construct adversarial examples robust over a certain distribution of transformations, and we can simply use EOT over that distribution to synthesize a robust adversarial example.
In the case of the physical world, however, we cannot capture the exact distribution unless we perfectly model all physical phenomena.
Therefore, we must approximate the distribution and perform EOT over the proxy distribution.
This works well in practice for producing adversarial objects that remain adversarial under the “true” physical-world distribution, as we demonstrate.
See
Appendix A for the exact parameters of the distributions we use in the 2D, 3D simulation, and 3D physical-world cases.
In all cases, the various transformation parameters are sampled as continuous random variables from a uniform distribution between the minimum and maximum values given, unless otherwise indicated in Appendix A (i.e. Gaussian noise).
In our experiments, we use TensorFlow’s standard pre-trained InceptionV3 classiﬁer (Szegedy et al., 2015) which has 78.0% top-1 accuracy on ImageNet.
In all of our experiments, we use randomly chosen target classes, and we use EOT to synthesize adversarial examples over a chosen distribution.
We measure the (cid:96) 2 distance per pixel between the original and adversarial example (in LAB [Table] [Table] Persian cat
P
[Table]
true
[Table]
P
[Table]
adv
[Table]
P
[Table]
Adversarial: jacamar (cid:96) 2 = 2 .
1 × 10 −
P ( true ) : 0%
P ( adv ) : 91%
[Table]
( true
[Table]
true
[Table]
space), and we also measure classiﬁcation accuracy (percent of randomly sampled viewpoints classiﬁed as the true class) and adversariality (percent of randomly sampled viewpoints classiﬁed as the adversarial class) for both the original and adversarial example.
When working in simulation, we evaluate over a large number of transformations sampled randomly from the distribution; in the physical world, we evaluate over a large number of manually-captured images of our adversarial objects taken over different viewpoints.
3. R OBUST 2D A DVERSARIAL E XAMPLES
In the 2D case, we consider the distribution of transformations that includes rescaling, rotation, lightening or darkening by an additive factor, adding Gaussian noise, and any in-bounds translation of the image.
We take the ﬁrst 1000 images in the ImageNet validation set, randomly choose a target class for each image, and use EOT to synthesize an adversarial example that is robust over the chosen distribution.
For each adversarial example, we evaluate over 1000 random transformations sampled from the distribution at evaluation time.
Table 1 summarizes the results.
On average, the adversarial examples have an adversariality of 96.4%, showing that our approach is highly effective in producing robust adversarial examples.
Figure 2 shows an example of a synthesized adversarial example, along with the classiﬁcation conﬁdence in true and adversarial classes for original and corresponding adversarial images.
See Appendix B for more examples.
3. R OBUST 3D A DVERSARIAL E XAMPLES
We produce 3D adversarial examples by modeling the 3D rendering as a transformation under EOT.
Given a textured 3D object, we optimize over the texture such that the rendering is adversarial from any viewpoint.
We consider a distribution that incorporates different camera distances, lateral trans- [Table] Original: turtle P ( true ) : 97%
P ( adv ) : 0% P ( true [Table] [Table]
P (
[Table]
P
[Table]
Adversarial: jigsaw puzzle (cid:96) 2 = 8 .
9 × 10 −
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) :
[Table]
P ( adv ) :
[Table]
( true
[Table]
true
[Table]
adv
[Table]
lation, rotation of the object, and solid background colors.
We approximate the expectation over transformation by taking the mean loss over batches of size 40; furthermore, due to the computational expense of computing a batch, we reuse up to 80% of the batch at each iteration, but enforce that each batch contain at least 8 new images.
As previously mentioned, the parameters of the distribution we use is speciﬁed in Appendix A, sampled as independent continuous random variables (that are uniform except for Gaussian noise).
We consider 10 complex 3D models, choose 20 random target classes per 3D model, and use EOT to synthesize adversarial textures for the 3D models with minimal parameter search (four constant, pre-chosen λ values were tested across each [3D model, target] pair).
For each of the 200 adversarial examples, we sample 100 random transformations from the distribution at evaluation time.
Table 2 summarizes results, and Figure 3 shows renderings of drawn samples with classiﬁcation probabilities.
See Appendix C for more examples.
The simulated adversarial object have an average adversariality of 83.4% with a long left tail, showing that EOT usually produces highly adversarial objects.
See Appendix C for a plot of the distribution.
3. P HYSICAL A DVERSARIAL E XAMPLES
In order to fabricate physical-world adversarial examples, beyond modeling the 3D rendering process, we need to model physical-world phenomena such as lighting effects and camera noise.
Furthermore, we need to model the 3D printing process: in our case, we use commercially available full-color 3D printing.
With the 3D printing technology we use, we ﬁnd that color accuracy varies between prints, so we model printing errors as well.
We approximate all of these phenomena by a distribution of transformations under EOT.
In addition to the transformations considered for 3D in simulation, we consider camera noise, additive and multiplicative lighting, and per-channel color inaccuracies.
classiﬁed as turtle classiﬁed as riﬂe classiﬁed as other classiﬁed as baseball classiﬁed as espresso classiﬁed as other
[Table] We evaluate physical adversarial examples over two 3D-printed objets: one of a turtle (where we consider any of the 5 turtle classes in ImageNet as the “true” class), and one of a baseball.
The unperturbed 3D-printed objects are correctly classiﬁed as the true class with 100% accuracy over a large number of samples.
Figure 4 shows example photographs of unperturbed objects, along with their classiﬁcations.
We choose target classes for each of the 3D models at random — “riﬂe” for the turtle, and “espresso” for the baseball — and we use EOT to synthesize adversarial examples.
We evaluate the performance of our two 3D-printed adversarial objects by taking 100 photos of each object over a variety of viewpoints 1 .
Figure 5 shows a random sample of these images, along with their classiﬁcations.
Table 3 gives a quantitative analysis over all images, showing that our 3D-printed adversarial objects are strongly adversarial over a wide distribution of transformations.
See Appendix D for more examples.
3. D ISCUSSION
The results and quantative analysis in this section demonstrate the efﬁcacy of EOT and conﬁrm the existence of physical adversarial examples.
Here, we perform a qualitative analysis of the results:
Modeling Perception.
The EOT algorithm as presented in Section 2 presents a general method to construct adversarial examples over a chosen perceptual distribution, but notably gives no guarantees for observations of the image outside of the chosen distribution.
In constructing physical-world adversarial objects, we use a crude, high-variance approximation of the rendering and capture process, and this succeeds in ensuring robustness to a diverse set of environments; see, for example,
Figure 6, which shows the same adversarial turtle in vastly different environments.
In specialized classiﬁed as turtle classiﬁed as riﬂe classiﬁed as other classiﬁed as baseball classiﬁed as espresso classiﬁed as other domains, however, a domain expert may opt to model the perceptual distribution precisely in order to better constrain the space.
Error in Printing.
We also ﬁnd signiﬁcant error in the color accuracy of even state of the art commercially available color 3D printing; Figure 7 shows a comparison of a 3D-printed model along with a printout of the model’s texture, printed on a standard laser color printer.
Still, EOT was able to overcome the problem and produce robust physical-world adversarial objects.
We predict that we could have produced adversarial examples with smaller (cid:96) 2 perturbation with a higher-ﬁdelity printing process.
Semantically Relevant Misclassiﬁcation.
Interestingly, for the majority of viewpoints where the adversarial target class is not the top-1 predicted class, the classiﬁer also fails to correctly predict the source class.
Instead, we ﬁnd that the classiﬁer often classiﬁes the object as an object that is semantically similar to the adversarial target; while generating the adversarial turtle to be classiﬁed as a riﬂe, for example, the second most popular class (after “riﬂe”) was “revolver,” followed by “holster” and then “assault riﬂe.”
Similarly, when generating the baseball to be classiﬁed as an espresso, the example was often classiﬁed as “coffee” or “bakery”.
R ELATED W ORK
State of the art neural networks are vulnerable to adversarial examples (Szegedy et al., 2013).
Researchers have proposed a number of methods for synthesizing adversarial examples in the whitebox scenario (with access to the gradient of the classiﬁer), including L-BFGS (Szegedy et al., 2013), the Fast Gradient Sign Method (FGSM)
(Goodfellow et al., 2015), and a Lagrangian relaxation formulation (Carlini & Wagner, 2017), all for the single-viewpoint case where the adversary directly controls the input to the neural network.
Projected Gradient Descent (PGD) can be seen as a universal ﬁrst-order adversary (Madry et al., 2017).
Moosavi-Dezfooli et al. (2017) show the existence of universal (image-agnostic) adversarial perturbations, small perturbation vectors that can be applied to any image to induce misclassiﬁcation. Their paper proposes an algorithm that ﬁnds perturbations that are universal over images; in our work, we give an algorithm that ﬁnds a perturbation to a single image or object that is universal over a chosen distribution of transformations. In preliminary experiments, we found that universal adversarial perturbations, like standard adversarial perturbations to single images, are not inherently robust to transformation.
In the ﬁrst work on 2D physical-world adversarial examples, Kurakin et al.
(2016) demonstrate the transferability of FGSM-generated adversarial misclassiﬁcation on a printed page.
In their setup, a photo is taken of a printed image with QR code guides, and the resultant image is warped, cropped, and resized to become a square of the same size as the source image before classifying it.
Their results show the existence of 2D physical-world adversarial examples for approximately axis-aligned views, demonstrating that adversarial perturbations produced using FGSM can translate to the physical world and are robust to camera noise, rescaling, and lighting effects.
Kurakin et al.
(2016) do not synthesize targeted physical-world adversarial examples, they do not evaluate other real-world 2D transformations such as rotation, skew, translation, or zoom, and their approach does not translate to the 3D case. Sharif et al. (2016) develop a real-world adversarial attack on a state-of-the-art face recognition algorithm, where adversarial eyeglass frames cause targeted misclassiﬁcation in portrait photos. The algorithm produces robust perturbations through optimizing over a ﬁxed set of inputs: the attacker collects a set of images and ﬁnds a perturbation that minimizes cross entropy loss over the set.
The algorithm solves a different problem than we do in our work: it produces adversarial perturbations universal over portrait photos taken head-on from a single viewpoint, while EOT produces 2D/3D adversarial examples robust over transformations.
The approach also includes a mechanism for enhancing perturbations’ printability using a color map to address the limited color gamut and color inaccuracy of the printer.
Note that this differs from our approach in achieving printability: rather than creating a color map, we ﬁnd an adversarial example that is robust to color inaccuracy.
Our approach has the advantage of working in settings where color accuracy varies between prints, as was the case with our 3D-printer.
Recently, Evtimov et al.
(2017) proposed a method for generating robust physical-world adversarial examples in the 2D case by optimizing over a ﬁxed set of manually-captured images.
However, the approach is limited to the 2D case, with no clear translation to 3D, where there is no simple mapping between what the adversary controls (the texture) and the observed input to the classiﬁer (an image).
Furthermore, the approach requires the taking and preprocessing of a large number of photos in order to produce each adversarial example, which may be expensive or even infeasible for many objects.
Lu et al.
(2017) argued that adversarial examples may not be a practical concern in physical-world systems because adversarial examples generated for the single viewpoint case lose adversariality at viewpoints with differing scale and rotation.
While this is the case with standard adversarial examples as evaluated in the paper and in Luo et al. (2016), we have shown that with EOT, it is in fact possible to construct physical-world adversarial images and objects that are classiﬁed as a chosen target class over a wide range of viewpoints.
C ONCLUSION
Our work shows that adversarial examples pose a practical threat to systems using neural networkbased image classiﬁers.
By introducing EOT, a general-purpose algorithm for creating robust adversarial examples under any chosen distribution, and modeling 3D rendering and printing within the framework of EOT, we succeed in fabricating three-dimensional adversarial objects.
With access only to low-cost commercially available 3D printing technology, we successfully print physical adversarial objects that are strongly classiﬁed as a chosen target class over a variety of angles, viewpoints, and lighting conditions by a standard ImageNet classiﬁer.
A D ISTRIBUTIONS OF T RANSFORMATIONS
Under the EOT framework, we must choose a distribution of transformations, and the optimization produces an adversarial example that is robust under the distribution of transformations.
Here, we give the speciﬁc parameters we chose in the 2D (Table 4), 3D (Table 5)
[Table]
and physical-world case (Table 6).
B R OBUST 2D A DVERSARIAL E XAMPLES
We give a random sample out of our 1000 2D adversarial examples in Figures 8 and 9.
C R OBUST 3D A DVERSARIAL E XAMPLES
We give a random sample out of our 200 3D adversarial examples in Figures 10 and 11 and 12.
We give a histogram of adversariality (percent classiﬁed as the adversarial class) over all 200 examples in Figure
[Table]
Original:
European ﬁre salamander
P ( true ) : 93%
P ( adv ) : 0%
P ( true ) : 91%
P ( adv ) : 0%
P ( true ) : 93%
P ( adv ) : 0%
P ( true ) : 93%
P ( adv ) : 0%
Adversarial: guacamole (cid:96) 2 = 2 . 1 × 10 −
P ( true ) : 0%
P ( adv ) : 99%
P ( true ) : 0%
P ( adv ) : 99%
P ( true ) : 0%
P ( adv ) : 96%
P ( true ) : 0%
P ( adv ) : 95%
Original: caldron
P ( true ) : 75%
P ( adv ) : 0%
P ( true ) : 83%
P ( adv ) : 0%
P ( true ) : 54%
P ( adv ) : 0%
P ( true ) : 80%
P ( adv ) : 0%
Adversarial: velvet (cid:96) 2 = 1 . 9 × 10 −
P ( true ) : 0%
P ( adv ) : 94%
P ( true ) : 0%
P ( adv ) : 94%
P ( true ) : 1%
P ( adv ) : 91%
P ( true ) : 0%
P ( adv ) : 100%
Original: altar
P ( true ) : 87%
P ( adv ) : 0%
P ( true ) : 38%
P ( adv ) : 0%
P ( true ) : 59%
P ( adv ) : 0%
P ( true ) : 2%
P ( adv ) : 0%
Adversarial:
African elephant (cid:96) 2 = 2 .
6 × 10 −
P ( true ) : 0%
P ( adv ) : 93%
P ( true ) : 0%
P ( adv ) : 87%
P ( true ) : 3%
P ( adv ) : 73%
P ( true ) : 0%
P ( adv ) :
[Figure]
Original: barracouta
P ( true ) : 91%
P ( adv ) : 0%
P ( true ) : 95%
P ( adv ) : 0%
P ( true ) : 92%
P ( adv ) : 0%
P ( true ) : 92%
P ( adv ) : 0%
Adversarial: tick (cid:96) 2 = 2 .
7 × 10 −
P ( true ) : 0%
P ( adv ) : 88%
P ( true ) : 0%
P ( adv ) : 99%
P ( true ) : 0%
P ( adv ) : 98%
P ( true ) : 0%
P ( adv ) : 95%
Original: tiger cat
P ( true ) : 85%
P ( adv ) : 0%
P ( true ) : 91%
P ( adv ) : 0%
P ( true ) : 69%
P ( adv ) : 0%
P ( true ) : 96%
P ( adv ) : 0%
Adversarial: tiger (cid:96) 2 = 2 . 1 × 10 −
P ( true ) : 32%
P ( adv ) : 54%
P ( true ) : 11%
P ( adv ) : 84%
P ( true ) : 59%
P ( adv ) : 22%
P ( true ) : 14%
P ( adv ) : 79%
Original: speedboat
P ( true ) : 14%
P ( adv ) : 0%
P ( true ) : 1%
P ( adv ) : 0%
P ( true ) : 1%
P ( adv ) : 0%
P ( true ) : 1%
P ( adv ) : 0%
Adversarial: crossword puzzle (cid:96) 2 = 3 .
4 × 10 −
P ( true ) : 3%
P ( adv ) : 91%
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) : 0%
P ( adv ) : 100%
Original: barrel
P ( true ) : 96%
P ( adv ) : 0%
P ( true ) : 99%
P ( adv ) : 0%
P ( true ) : 96%
P ( adv ) : 0%
P ( true ) : 97%
P ( adv ) : 0%
Adversarial: guillotine (cid:96) 2 = 3 . 1 × 10 −
P ( true ) : 1%
P ( adv ) : 10%
P ( true ) : 0%
P ( adv ) : 95%
P ( true ) : 0%
P ( adv ) : 91%
P ( true ) : 3%
P ( adv ) : 4%
Original: baseball
P ( true ) : 100%
P ( adv ) : 0%
P ( true ) : 100%
P ( adv ) : 0%
P ( true ) : 100%
P ( adv ) : 0%
P ( true ) : 100%
P ( adv ) : 0%
Adversarial: green lizard (cid:96) 2 = 2 . 9 × 10 −
P ( true ) : 0%
P ( adv ) : 66%
P ( true ) : 0%
P ( adv ) : 94%
P ( true ) : 0%
P ( adv ) : 87%
P ( true ) : 0%
P ( adv ) : 94%
Original: turtle
P ( true ) : 94%
P ( adv ) : 0%
P ( true ) : 98% P ( adv ) : 0% P ( true ) : 90% P ( adv ) : 0%
P ( true ) : 97%
P ( adv ) : 0% Adversarial: Bouvierdes Flandres (cid:96) 2 = 1 .
4 × 10 − P ( true ) : 1% P ( adv ) : 1% P ( true ) : 0%
P ( adv ) : 6%
P ( true ) : 0% P ( adv ) : 21% P ( true ) : 0% P ( adv
[Equation]
Original: baseball
P ( true ) : 100% P ( adv ) : 0% P ( true ) : 100% P ( adv ) : 0%
P ( true ) : 100%
P ( adv ) : 0%
P ( true ) : 100%
P ( adv ) : 0%
Adversarial: Airedale (cid:96) 2 = 3 . 4 × 10 −
P ( true ) : 0%
P ( adv ) : 94%
P ( true ) : 0%
P ( adv ) : 6%
P ( true ) : 0%
P ( adv ) : 96%
P ( true ) : 0%
P ( adv ) : 18%
Original: orange
P ( true ) : 73%
P ( adv ) : 0%
P ( true ) : 29%
P ( adv ) : 0%
P ( true ) : 20%
P ( adv ) : 0%
P ( true ) : 85%
P ( adv ) : 0%
Adversarial: power drill (cid:96) 2 = 4 .
3 × 10 −
P ( true ) : 0%
P ( adv ) : 89%
P ( true ) : 4%
P ( adv ) : 75%
P ( true ) : 0%
P ( adv ) : 98%
P ( true ) : 0%
P ( adv ) : 84%
Original: dog
P ( true ) : 1%
P ( adv ) : 0%
P ( true ) : 32%
P ( adv ) : 0%
P ( true ) : 12%
P ( adv ) : 0%
P ( true ) : 0%
P ( adv ) : 0%
Adversarial: bittern (cid:96) 2 = 7 . 4 × 10 −
P ( true ) : 0%
P ( adv ) : 97%
P ( true ) : 0%
P ( adv ) : 91%
P ( true ) : 0% P ( adv ) : 98% P ( true ) : 0% P ( adv ) : 97%
Original: teddybear
P ( true ) : 90% P ( adv ) : 0% P ( true ) : 1% P ( adv ) : 0%
P ( true ) : 98% P ( adv ) : 0% P ( true ) : 5% P ( adv ) : 0%
Adversarial: sock (cid:96) 2 = 3 . 3 × 10 −
P ( true ) : 0%
P ( adv ) : 99%
P ( true ) : 0%
P ( adv ) : 99%
P ( true ) : 0%
P ( adv ) : 98%
P ( true ) : 0%
P ( adv ) : 99%
Original: clownﬁsh
P ( true ) : 46% P ( adv ) : 0% P ( true ) : 14% P ( adv ) : 0%
P ( true ) : 2%
P ( adv ) : 0%
P ( true ) : 65%
P ( adv ) : 0%
Adversarial: panpipe (cid:96) 2 = 4 .
8 × 10 −
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) : 0%
P ( adv ) : 1%
P ( true ) : 0%
P ( adv ) : 12%
P ( true ) : 0%
P ( adv ) : 0%
Original: sofa
P ( true ) : 15%
P ( adv ) : 0%
P ( true ) : 73%
P ( adv ) : 0%
P ( true ) : 1%
P ( adv ) : 0%
P ( true ) : 70%
P ( adv ) : 0%
Adversarial: sturgeon (cid:96) 2 = 1 .
3 × 10 −
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) : 0%
P ( adv ) : 100%
P ( true ) : 0%
P ( adv ) : 100%
D P HYSICAL A DVERSARIAL E XAMPLES
Figure 14 gives all 100 photographs of our adversarial 3D-printed turtle, and Figure 15 gives allphotographs of our adversarial 3D-printed baseball.
[Figure] [Figure]