Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
Abstract
Making classiﬁers robust to adversarial examples is hard.
Thus, many defenses tackle the seemingly easier task of detecting perturbed inputs.
We show a barrier towards this goal.
We prove a general hardness reduction between detection and classiﬁcation of adversarial examples: given a robust detector for attacks at distance (cid:15) (in some metric), we can build a similarly robust (but inefﬁcient) classiﬁer for attacks at distance (cid:15)/ 2 .
Our reduction is computationally inefﬁcient, and thus cannot be used to build practical classiﬁers.
Instead, it is a useful sanity check to test whether empirical detection results imply something much stronger than the authors presumably anticipated.
To illustrate, we revisit 13 detector defenses.
For 11 / 13 cases, we show that the claimed detection results would imply an inefﬁcient classiﬁer with robustness far beyond the state-of-the-art.
1. Introduction
Consider the following claims about two defenses against adversarial examples (Szegedy et al., 2014) on CIFAR-10:
• defense A is a classiﬁer that achieves robust accuracy of 90% under (cid:96) ∞ -perturbations bounded by (cid:15)
= 4 / 255 ; • defense B also has a “detection” option, and achieves robust accuracy of 90% under (cid:96) ∞ -perturbations bounded by (cid:15)
= 8 / 255 (defense B is correct if it classiﬁes a natural example correctly, and either detects or correctly classiﬁes all perturbed examples at distance (cid:15) .)
If you had to take a bet that one of these two (empirical) claims is correct, which one would you choose?
Defense A claims much higher robustness than the current best result achieved with adversarial training (Madry et al.,
Accepted by the ICML 2021 workshop on A
2018), the only empirical defense against adversarial examples that stands the test of time.
Thus, this claim might be met with some initial skepticism and heightened scrutiny.
The claim of defense B is harder to assess, due to a lack of long-standing baselines for robust detectors.
On one hand, detection of adversarial examples has largely been considered to be an easier task than classiﬁcation (Xu et al., 2018; Pang et al., 2021).
On the other hand, defense B claims robustness to perturbations that are twice as large as defense A.
In this paper, we show that the claims about defenses A and
B are, in fact, equivalent!
(up to computational efﬁciency.)
We prove a general hardness reduction between classiﬁcation and detection of adversarial examples.
Given a detector defense that achieves robust risk α for attacks at distance (cid:15) (under any metric), we show how to build an explicit but inefﬁcient classiﬁer that achieves robust risk α for classifying attacks at distance (cid:15)/ 2 .
The reverse implication also holds: a classiﬁer robust at distance (cid:15)/ 2 implies an explicit but inefﬁcient robust detector at distance (cid:15) .
To the authors knowledge, there is no known way of leveraging computational inefﬁciency to build more robust models.
We should thus be as “surprised” by the claim made by defense B as by the claim made by defense A.
Our hardness reduction provides a way of assessing the plausibility of new robust detection claims, by contrasting them with results from the more mature literature on robust classiﬁcation.
To illustrate, we revisit 13 published detection defenses, and show that in 11/13 cases the defense’s robust detection claims would imply an inefﬁcient classiﬁer with robustness far superior to the current state-of-the-art.
Yet, none of these defense papers claim that their results should imply such a breakthrough in robust classiﬁcation .
Using our reduction, it is obvious that most of the 13 detection defenses are claiming stronger robustness than we currently believe feasible.
And indeed, many of these defenses were later shown to have overestimated their robustness (Carlini & Wagner, 2017; Tram`er et al., 2020).
Remarkably, we ﬁnd that for certiﬁed defenses, the state- of-the-art results for provable robust classiﬁcation and detection perfectly match the results implied by our reduction.
For example, Sheikholeslami et al.
(2021) recently proposed a certiﬁed detector on CIFAR-10 with provable robust error that is within 3% of the provable error of the inefﬁcient detector obtained by combining the state-of-the-art robust classiﬁer of Zhang et al.
(2020) and our result.
This gives further credence to our assumption that computational inefﬁciency is of little help towards building more robust models.
In summary, we prove that giving classiﬁers access to a rejection/detection option does not help robustness.
It would be interesting to ﬁnd similar “barriers” for other directions that have been considered to enhance robustness (e.g., the use of randomness, ensembles, denoising functions, etc.)
2. Hardness Reductions Between Robust Classiﬁers and Detectors
In this section, we prove our main result: a reduction between robust detectors and robust classiﬁers, and vice-versa.
We consider a classiﬁcation task with a distribution D over examples x ∈ R d with labels y ∈
[ C ] .
A classiﬁer is a function f : R d → [ C ] .
A detector is a classiﬁer with an extra “rejection” option ⊥ .
The binary indicator function{ A } is 1 if and only if the predicate A is true.
We ﬁrst deﬁne a classiﬁer’s risk , i.e., its classiﬁcation error on unperturbed samples.
Deﬁnition 1 (Risk) .
Let f : R d → [ C ] ∪{⊥} be a classiﬁer (optionally with a detection output ⊥ ).
The risk of f is:
[Equation]
Note that for a detector, rejecting an unperturbed example, f ( x ) = ⊥ , is counted as an error.
For classiﬁers without a rejection option, we deﬁne the robust risk as the risk on worst-case adversarial examples.
Given an input x sampled from D , an adversarial example ˆ x is constrained to being within distance d ( x, ˆ x ) ≤ (cid:15) from x , where d is some arbitrary distance measure.
Deﬁnition 2 (Robust risk) .
Let f : R d → [ C ] be a classiﬁer.
The robust risk at distance (cid:15) is:
[Equation]
For a detector (a classiﬁer with an extra rejection output), we analogously deﬁne the robust risk with detection.
The classiﬁer is now allowed to reject adversarial examples.
Deﬁnition 3 (Robust risk with detection) .
Let f : R d → [ C ] ∪ {⊥} be a classiﬁer with an extra detection output ⊥ .
The robust risk with detection at distance (cid:15) is:
[Equation]
That is, a detector defense f is robust on a natural input x if and only if f classiﬁes x correctly and f either rejects or correctly classiﬁes every input
ˆ x within distance (cid:15) from x .
The requirement that f correctly classify natural examples eliminates pathological defenses that reject all inputs.
We are now ready to introduce our main result, a reduction from a robust detector for adversarial examples at distance (cid:15) , to a robust classiﬁer at distance (cid:15)/ 2 :
Theorem 4 ( (cid:15) -detection implies (cid:15)/ 2 -classiﬁcation) .
Let d ( · , · ) be an arbitrary metric.
Let f be a defense that achieves risk R ( f )
=
α , and robust risk with detection
R (cid:15) adv-det ( f ) = β .
Then, we can construct an explicit (but inefﬁcient) defense g that achieves risk R ( g ) ≤ α and robust risk R (cid:15)/ 2 adv ( g ) ≤ β .
The defense g is constructed as follows on input x :
Run y = f ( x ) .
If y (cid:54) = ⊥ , output y .
Otherwise, ﬁnd an input x (cid:48)
such that d ( x, x (cid:48) ) ≤ (cid:15)/ 2 and f ( x (cid:48) )
(cid:54) = ⊥ .
If such an input x (cid:48) is found, output f ( x (cid:48) ) .
Else, output a random class y ∈ [ C ] .
An intuitive illustration for our construction, and for the proof of the theorem (see below) is in Figure 1.
Our construction is best viewed as an analog of minimum distance decoding in coding theory.
We can view the clean data sampled from D as codewords, and adversarial examples ˆ x as a noisy message with a certain number of errors (where the error magnitude is measured using an arbitrary metric on R d rather than the Hamming distance).
A standard result in coding theory states that if a code can detect α errors, then it can correct α/ 2 errors.
Proof.
First, note that the natural accuracy of g is at least as high as that of f , since g always mimics the output of f when f does not reject an input.
Thus, R ( g ) ≤ R ( f ) = α .
Now, consider an input ( x, y ) ∼ D for which g is not robust at distance (cid:15)/ 2 .
That is, there exists an input ˆ x at distance d ( x, ˆ x ) ≤ (cid:15)/ 2 such that g (ˆ x )
=
ˆ y (cid:54)
=
y .
We will show that the defense f is not robust with detection for x either (for attacks at distance up to (cid:15) .)
By deﬁnition of g , if g (ˆ x ) = ˆ y (cid:54) =
y then either:
• The defense f also misclassiﬁes ˆ x , i
e., f (ˆ x ) = ˆ y .
So f is not robust with detection for x at distance (cid:15) .
• There exists x (cid:48)
such that d (ˆ x, x (cid:48) ) ≤ (cid:15)/ 2 and f ( x (cid:48) )
=
ˆ y .
Note that by the triangular inequality, d ( x, x (cid:48) ) ≤ d ( x, ˆ x ) + d (ˆ x, x (cid:48) ) ≤ (cid:15)/ 2 + (cid:15)/ 2 = (cid:15) , and thus f is not robust with detection for x at distance (cid:15) .
[Equation]
• For all x (cid:48) such that d (ˆ x, x (cid:48) ) ≤ (cid:15)/ 2 , we have f ( x (cid:48) )
= ⊥ (and thus g has output ˆ x at random).
Since d ( x, ˆ x ) ≤ (cid:15)/ 2 , this implies that f ( x ) = ⊥ , and thus f is not robust with detection for x .
In summary, whenever g fails to robustly classify an input x up to distance (cid:15)/ 2 , the defense f also fails to robustly classify x with detection up to distance (cid:15) .
Taking expectations over the entire distribution D concludes the proof.
A corollary to our reduction is that many “information theoretic” results about robust classiﬁers can be directly extended to robust detectors.
For example, the formal tradeoff between robust classiﬁcation and accuracy of Tsipras et al.
(2019), the increased data complexity of robust generalization of Schmidt et al.
(2018), or the tradeoff between robustness to multiple perturbation types of Tram ` er & Boneh (2019), all imply similar tradeoffs for robust detectors.
Indeed, all of these results also apply to inefﬁcient classiﬁers.
A similar argument can be used in the opposite direction, to show that a robust classiﬁer at distance (cid:15)/ 2 implies an inefﬁcient robust detector at distance (cid:15) .
Theorem 5 ( (cid:15)/ 2 -classiﬁcation implies (cid:15) -detection) .
Let d ( · , · ) be an arbitrary metric.
Let g be a defense that achieves robust risk R (cid:15)/ 2 adv ( f )
= β .
Then, we can construct an explicit (but inefﬁcient) defense f that achieves risk
R ( f ) ≤ β and robust risk with detection R (cid:15) adv-det ( f ) ≤ β .
The defense f is constructed as follows on input x :
Run y = g ( x ) .
• Find an input x (cid:48) such that d ( x, x (cid:48) ) ≤ (cid:15)/ 2 and g ( x (cid:48) )
(cid:54) =
y .
The proof of Theorem 5 is in Appendix A.
A main distinction between Theorem 4 and Theorem 5 is that the construction in Theorem 4 preserves clean accuracy, but the construction in Theorem 5 does not.
That is, the constructed robust detector in Theorem 5 has natural accuracy that is equal to the robust classiﬁer’s robust accuracy.
The construction in Theorem 5 can be efﬁciently (but approximately) instantiated by a certiﬁably robust classiﬁer (Wong & Kolter, 2018; Raghunathan et al., 2018).
These defenses can certify that a classiﬁer’s output is constant for all points within some distance of the input.
For an adversarial example ˆ x for g , the certiﬁcation always fails and thus the constructed detector f will reject ˆ x .
If g is robust and the certiﬁcation succeeds, the detector f copies the output of g .
However, a certiﬁed defense may fail to certify a robust input (a false negative), and thus the detector f may reject more inputs than with the “optimal” construction in
Theorem 5.
This reduction from a certiﬁed classiﬁer to a detector is implicit in (Wong & Kolter, 2018, Section 3.1).
3. What Are Detection Defenses Claiming?
We now survey 13 detection defenses, and consider the robust classiﬁcation performance that these defenses implicitly claim (via Theorem 4).
As we will see, in 11/13 cases, the defenses’ detection results imply an inefﬁcient classiﬁer with far better robust accuracy than the state-of-the-art.
These claims are not necessarily wrong .
But given how challenging robust classiﬁcation is proving to be, we have reason to be skeptical of major breakthroughs (even for inefﬁcient classiﬁers).
To compound this, many proposed detection defenses are quite simple, and reject adversarial inputs based on some standard statistical test over a neural network’s features.
It would be particularly surprising if such simple techniques could yield robust classiﬁers (and indeed, many of these defenses have been broken by stronger attacks (Carlini & Wagner, 2017; Tram`er et al., 2020)).
Setup.
We choose 13 detector defenses from the literature (see Table 1).
Our choice for these defenses was somewhat artiﬁcial and pragmatic: we chose defenses that made claims that were easy to translate into a bound on the robust risk with detection R (cid:15) adv-det .
Indeed, some detection defenses simply report a single AUC score, from which we cannot derive a useful bound on the robust risk.
We thus focus on defenses that either directly report a robust error akin to
Deﬁnition 3, or that provide concrete pairs of false-positive and false-negative rates (e.g., a full ROC curve).
In the latter case, we compute a “best-effort” bound on the robust risk
[Table]
[Table] [Table]
[Table] [Table]
with detection as:
[Equation]
where FPR and FNR are the detector’s false-positive and false-negative rates
[Table]
and R ( f ) is the defense’s standard risk (i
[Table]
e., the test error on natural examples).
We note that this union bound may be quite pessimistic, as we might overcount examples that lead to multiple sources of errors (e
[Table]
g., a natural input that is misclassiﬁed and erroneously detected).
The true robustness claim made by these detector defenses might thus be stronger than what we obtain from our bound.
We encourage future defense papers to report the adversarial risk, to facilitate comparisons with robust classiﬁers.
The 13 detector defenses use three datasets: MNIST, CIFAR10 and ImageNet, and consider adversarial examples under the (cid:96) ∞ or (cid:96) 2 norms.
Given a claim of robust detection at distance (cid:15) , we contrast it to a state-of-the-art robust classiﬁcation result for distance (cid:15)/ 2 .
On MNIST with (cid:96) ∞ attacks, we use TRADES (Zhang et al., 2019) and measure robust error with the Square attack (Andriushchenko et al., 2020).
For (cid:96) 2 attacks, we use the model from Tram`er & Boneh (2019) and measure robust error with PGD (Madry et al., 2018).
For CIFAR-10, we use the best model of Rebufﬁ et al.
(2021) (trained without external data), and attack it using AutoAttack (Croce & Hein, 2020).
For ImageNet, we use models and attacks from Engstrom et al.
We also consider two certiﬁed defenses for CIFAR-10: the robust classiﬁer of Zhang et al.
(2020), and a recent certiﬁed detector of Sheikholeslami et al.
Results.
As we can see from table 1, most defenses claim a detection performance that implies a far greater robust accuracy than our current best robust classiﬁers.
In Table 2, we look at the robust accuracy with detection, and standard robust accuracy achieved by certiﬁed defenses (for which the claimed robustness numbers are necessarily mathematically correct).
Remarkably, we ﬁnd that existing results nearly match what is implied by our reduction (up to ± 2% error).
For example, Zhang et al.
(2020) follow a long line of results on robust classiﬁers and achieve 39% robust accuracy on CIFAR-10 for perturbations of (cid:96) ∞ -norm below 4 / 255 .
Together with Theorem 5, this implies an inefﬁcient detector with 39% robust detection accuracy for perturbations of (cid:96) ∞ -norm below 8 / 255 .
The recent work of Sheikholeslami et al.
(2021) nearly matches that bound ( 37% robust accuracy with detection), with a defense that has the advantage of being concretely efﬁcient.
4. Conclusion
We have shown formal reductions between robust classiﬁcation with, and without, a detection option.
Our results show that signiﬁcant progress on one of these two tasks implies similar progress on the other.
This raises the question on whether we should spend our efforts on studying both of these tasks, or focus our efforts on a single one.
On one hand, the two tasks represent different ways of tackling a same goal, and working on either task might result in new techniques or ideas that apply to the other task as well.
On the other hand, our reductions show that unless we make progress on both tasks, work on one of the tasks can merely aim to match the robustness of our inefﬁcient constructions, whilst improving their computational complexity. Acknowledgments
The author would like to thank Nicholas Carlini and Wieland
Brendel for helpful discussions, as well as Alex Ozdemir for suggesting the connection between our results and minimum distance decoding.
A. Proof of Theorem 5.
We restate Theorem 5 here:
Theorem 4 ( (cid:15)/ 2 -classiﬁcation implies (cid:15) -detection) .
Let d ( · , · ) be an arbitrary metric.
Let g be a defense that achieves robust risk R (cid:15)/ 2 adv ( f )
= β .
Then, we can construct an explicit (but inefﬁcient) defense f that achieves risk
R ( f ) ≤ β and robust risk with detection R adv-det ( f ; (cid:15) )
≤ β .
The defense f is constructed as follows on input x :
Run y = g ( x ) .
• Find an input x (cid:48) such that d ( x, x (cid:48) ) ≤ (cid:15)/ 2 and g ( x (cid:48) )
(cid:54) =
y .
If such an input x (cid:48) exists, output ⊥ .
Else, output y .
Proof of Theorem 5.
Note that for any input ( x, y ) for which g is robust at distance (cid:15)/ 2 , no input x (cid:48) above exists and so f ( x ) = y .
Thus, the risk of f is at most β .
Now, consider an input ( x, y ) ∼ D for which f is not robust with detection at distance (cid:15) .
That is, either f ( x ) (cid:54) =
y , or there exists an input ˆ x at distance d ( x, ˆ x ) ≤ (cid:15) such that f (ˆ x ) =
ˆ y / ∈ { y, ⊥} .
We will show that the defense g is not robust for x either (for attacks at distance up to (cid:15)/ 2 .)
If f ( x ) (cid:54) =
y , then by the same argument as above it cannot be the case that g is robust at distance (cid:15)/ 2 for x .
So let us consider the case where f (ˆ x ) =
ˆ y / ∈ { y, ⊥} .
By the deﬁnition of f , this means that for all x (cid:48) at distance at most (cid:15)/ 2 from ˆ x , we have g ( x (cid:48) )
=
ˆ y .
But, note that there exists a point x ∗ that is at distance at most (cid:15)/ 2 from both ˆ x and x .
Since we must have g ( x ∗ )
=
ˆ y , we conclude that g is not robust at distance (cid:15)/ 2 for x .
Taking expectations over the distribution D concludes the proof.