P ROVABLY ROBUST CLASSIFICATION OF ADVERSARIAL EXAMPLES WITH DETECTION
A BSTRACT
Adversarial attacks against deep networks can be defended against either by building robust classiﬁers or, by creating classiﬁers that can detect the presence of adversarial perturbations.
Although it may intuitively seem easier to simply detect attacks rather than build a robust classiﬁer, this has not bourne out in practice even empirically, as most detection methods have subsequently been broken by adaptive attacks, thus necessitating veriﬁable performance for detection mechanisms.
In this paper, we propose a new method for jointly training a provably robust classiﬁer and detector.
Speciﬁcally, we show that by introducing an additional “abstain/detection” into a classiﬁer, we can modify existing certiﬁed defense mechanisms to allow the classiﬁer to either robustly classify or detect adversarial attacks.
We extend the common interval bound propagation (IBP) method for certiﬁed robustness under (cid:96) ∞ perturbations to account for our new robust objective, and show that the method outperforms traditional IBP used in isolation, especially for large perturbation sizes.
Speciﬁcally, tests on MNIST and CIFAR-10 datasets exhibit promising results, for example with provable robust error less than 63 .
63% and 67 .
92% , for 55 .
6% and 66 .
37% natural error, for (cid:15)
= 8 / 255 and 16 / 255 on the CIFAR-10 dataset, respectively.
I NTRODUCTION
Despite popularity and success of deep neural networks in many applications, their performance declines sharply in adversarial settings.
Small adversarial perturbations are shown to greatly deteriorate the performance of neural network classiﬁers, which creates a growing concern for utilizing them in safety critical application where robust performance is key.
In adversarial training, different methods with varying levels of computational complexity aim at robustifying the network by ﬁnding such adversarial examples at each training steps and adding them to the training dataset.
While such methods exhibit empirical robustness, they lack veriﬁable guarantees as it is not provable that a more rigorous adversary, e.g., one that does brute-force enumeration to compute adversarial perturbations, will not be able to cause the classiﬁer to misclassify.
It is thus desirable to provably verify the performance of robust classiﬁers without restricting the adversarial perturbations by inexact solvers, while restraining perturbations to a class of admissible set, e.g., within an (cid:96) ∞ norm-bounded ball.
Progress has been made by ‘complete methods’ that use Satisﬁability Modulo Theory (SMT) or Mixed-Integer Programming (MIP) to provide exact robustness bounds, however, such approaches are expensive, and difﬁcult to scale to large networks as exhaustive enumeration in the worst case is required (Tjeng et al., 2017; Ehlers, 2017; Xiao et al., 2018).
‘Incomplete methods’ on the other hand, proceed by computing a differential upper bound on the worst-case adversarial loss, and similarly for the veriﬁcation violations, with lower computational complexity and improved scalability.
Such upper bounds, if easy to compute, can be utilized during the training, and yield provably robust networks with tight bounds.
In particular, bound propagation via various methods such as differentiable geometric abstractions (Mirman et al., 2018), convex polytope relaxation (Wong & Kolter, 2018), and more recently in (Salman et al., 2019; Balunovic &
Vechev, 2020; Gowal et al., 2018; Zhang et al., 2020), together with other techniques such as semideﬁnite relaxation, (Fazlyab et al., 2019; Raghunathan et al., 2018), and dual solutions via additional veriﬁer networks (Dvijotham et al., 2018) fall within this category.
In particular, recent successful use of Interval Bound Propagation (IBP) as a simple layer-by-layer bound propagation mechanism was shown to be very effective in Gowal et al.
(2018), which despite its light computational complexity exhibits SOTA robustness veriﬁcation.
Additionally, combining IBP in a forward bounding pass with linear relaxation based backward bounding pass (CROWN) Zhang et al.
(2020) leads to improved robustness, although it can be up to 3-10 times slower.
Alternative to robust classiﬁcation, detection of adversarial examples can also provide robustness against adversarial attacks, where suspicious inputs will be ﬂagged and the classiﬁer “rejects/abstains” from assigning a label.
There has been some work on detection of out-of-distribution examples Bitterwolf et al.
(2020), however the situation in the literature on the detection of adversarial examples is quite different from above.
Most techniques that attempt to detect adversarial examples, either by training explicit classiﬁers to do so or by simply formulating “hand-tuned” detectors, still largely look to identify and exploit statistical properties of adversarial examples that appear in practice Smith & Gal (2018); Roth et al.
However, to provide a fair evaluation, a defense must be evaluated under attackers that attempt to fool both the classiﬁer and the detector, while addressing particular characteristics of a given defense, e.g., gradient obfuscation, non-differentability, randomization, and simplifying the attacker’s objective for increased efﬁciency.
A non-exhaustive list of recent detection methods entails randomization and sparsity-based defenses (Xiao et al., 2019; Roth et al., 2019;
Pang et al., 2019b), conﬁdence and uncertainty-based detection (Smith & Gal, 2018; Stutz et al., 2020; Sheikholeslami et al., 2020), transformation-based defenses (Bafna et al., 2018; Yang et al., 2019), ensemble methods (Verma & Swami, 2019; Pang et al., 2019a), generative adversarial training Yin et al.
(2020), and many more.
Unfortunately, existing defenses have largely proven to have poor performance against adaptive attacks (Athalye et al., 2018;
Tramer et al., 2020), necessitating provable guarantees on detectors as well.
Recently Laidlaw & Feizi (2019) have proposed joint training of classiﬁer and detector, however it also does not provided any provable guarantees.
Our contribution .
In this work, we propose a new method for jointly training a provably robust classiﬁer and detector.
Speciﬁcally, by introducing an additional “abstain/detection” into a classiﬁer, we show that the existing certiﬁed defense mechanisms can be modiﬁed, and by building on the detection capability of the network, classiﬁer can effectively choose to either robustly classify or detect adversarial attacks.
We extend the light-weight Interval Bound Propagation (IBP) method to account for our new robust objective, enabling veriﬁcation of the network for provable performance guarantees.
Our proposed robust training objective is also effectively upper bounded, enabling its incorporation into the training procedure leading to tight provably robust performance.
While tightening of the bound propagation may be additionally possible for tighter veriﬁcation, to the best of our knowledge, our approach is the ﬁrst method to extend certiﬁcation techniques by considering detection while providing provable veriﬁcation.
By stabilizing the training, as also used in similar IBP-based methods, experiments on MNIST and CIFAR-10 empirically show that the proposed method can successfully leverage its detection capability, and improves traditional IBP used in isolation, especially for large perturbation sizes.
B ACKGROUND AND RELATED WORK
Let us consider an L -layer feed-forward neural network, trained for a K -class classiﬁcation task.
Given input x , it will pass through a sequential model, with h l denoting the mapping at layer l , recursively parameterized by
[Equation]
where σ l ( . ) is a monotonic activation function, z 0 denotes the input, and z L ∈ R K is the preactivation unnormalized K -dimensional output vector ( n L = K and σ L ( . ) as identity operator), referred to as the logits.
Robust classiﬁers can be obtained by minimizing the worst-case (adversar- ial) classiﬁcation loss, formally trained by the following min-max optimization Madry et al.
[Equation]
where θ denotes network parameters, vector f θ ( x ) =
z L is the logit output for input x , (cid:96) ( . ) is the misclassﬁcation loss, e.g., (cid:96) xent ( . ) deﬁned as the cross-entropy loss, and ∆ (cid:15) denotes the set of permissible perturbations, e.g., for (cid:96) ∞ -norm ball of radius (cid:15) giving ∆ (cid:15) :
= { δ | (cid:107) δ (cid:107) ∞ ≤ (cid:15) } .
Although augmenting the training set with adversarial inputs, obtained by approximately solving the inner maximization in Eq. 2, empirically leads to improved adversarial robustness Madry et al.
(2017); Shafahi et al.
; Wong et al.
; Zhang et al.
(2019), inexact solution of the inner maximization prevents such methods from providing provable guarantees.
In critical applications however, provable veriﬁcation of classiﬁcation accuracy against a given threat model is crucial.
2. PERFORMANCE VERIFICATION AND NETWORK RELAXATION
Given input ( x , y ) , a classiﬁcation network is considered veriﬁably robust if all of its perturbed variations, that is x + δ for ∀ δ ∈ ∆ (cid:15) , are correctly classiﬁed as class y .
Such veriﬁcation can be effectively obtained by
[Equation]
where ,
[Equation]
where c y,i = e y − e i for i
= 1 , 2 , .., K, i (cid:54) =
y , and e i is the standard
i th canonical basis vector.
If p ∗ i > 0 ∀ i (cid:54) =
y , then the classiﬁer is veriﬁably robust at point ( x , y ) as this guarantees that z i ≤
z y ∀ i (cid:54) =
y for all admissible perturbations δ ∈ ∆ (cid:15) .
The feasible set Z L is generally nonconvex, rendering obtaining p ∗ i intractable.
Any convex relaxation of Z L however, will provide a lower bound on p ∗ i , and can be alternatively used for veriﬁcation.
As outlined in Section 1, various relaxation techniques have been proposed in the literature.
Specifically, IBP in (Mirman et al., 2018; Gowal et al., 2018) proceeds by bounding the activation z l of each layer by propagating an element-wise bounding box using interval arithmetic for networks with monotonic activation functions.
Despite its simplicity and relatively small computational complexity (computational requirements for bound propagation for a given input using IBP is equal to two forward passes of the input), it can provide tight bounds once the network is trained accordingly.
Speciﬁcally, starting from the input layer z 0 , it can be bounded for the perturbation class δ ∈ ∆ (cid:15) as z 0 = x − (cid:15) 1 and z 0 = x + (cid:15) 1 , and z l for the following layers can be bounded as
[Equation]
where | .
| is the element-wise absolute-value operator.
The veriﬁcation problem over the relaxed feasible set ˆ Z L :
[Equation]
[Equation]
¯ z L,i } , where Z L ⊆ ˆ Z L is then easily solved as
[Equation]
2. R OBUST TRAINING OF VERIFIABLE NETWORKS
It has been shown that convex relaxation of Z L can also provide a tractable upper bound on the inner maximization in Eq.
While this holds for various relaxation techniques, focusing on the IBP let us deﬁne
[Equation] [Equation]
., J IBP K ] where
[Equation]
with ( θ, (cid:15) ) implicitly inﬂuencing ˆ Z L (dropped for brevity), and upperbound the inner-max in Eq.
[Equation]
By using this tractable upper bound of the robust optimization, network can now be trained by minimize θ
[Equation]
where γ trades natural versus robust accuracy, and κ is scheduled through a ramp-down process to stabilize the training and tightening of IBP Gowal et al.
(2018) (where γ = 1 is selected therein).
V ERIFIABLE CLASSIFICATION WITH DETECTION
In this paper, we propose a new method for jointly training a provably robust classiﬁer and detector.
Speciﬁcally, let us augment the classiﬁer by introducing an additional “abstain/detection”.
This can be readily done by extending the K -class classiﬁcation task to a ( K + 1) -class classiﬁcation, with the ( K + 1) -th class dedicated to the detection task, and the maximum weighted class is ﬁnally chosen as the classiﬁcation output.
The classiﬁer is then trained such that adversarial examples, or ideally any other example that the network would misclassify , are classiﬁed in this abstain class, denoted by a , thus preventing incorrect classiﬁcation.
Formally, the classiﬁer can be denoted as in Eq. 1, with the only difference that the ﬁnal output is
K + 1 dimensional, i.e., z L ∈ R K +1 ; simply by substituting the last fully-connected weight matrix
W L of dimension n L × K with that of dimension n L × ( K + 1) , and similarly for b L .
3. V ERIFICATION PROBLEM FOR CLASSIFICATION WITH ABSTAIN / DETECTION
It is desirable to provably verify performance of the joint classiﬁcation/detection.
In contrast to existing robust classiﬁers, however, on a perturbed image x + δ , the classiﬁcation/detection task is considered successful if the input is classiﬁed either as the correct class y , or as the abstain class a ; as both cases prevent misclassiﬁcation of the adversarially perturbed input as a wrong class.
On clean natural images however, classiﬁcation/detection is considered successful only if it is classiﬁed as the correct class y , and abstaining is considered misclassiﬁcation.
In order to certify performance in adversarial settings, it is now sufﬁcient to verify that the network satisﬁes the following for a given input pair ( x , y ) and δ ∈ ∆ (cid:15) and i = 1 , .., K, i (cid:54) =
y :
[Equation]
where c y :=
e y − e i and c a :=
e a − e i , a denotes the “abstain” class, and the dependence of Z L on ( x , y, (cid:15), θ ) is omitted for brevity.
Veriﬁcation can be done effectively by seeking a counterexample
[Equation]
If π ∗ i ≥ 0 ∀ i (cid:54) =
y , the speciﬁcation is then satisﬁed and the performance is veriﬁed.
Similar to previous veriﬁcation methods, to overcome the non-convexity of the optimization in Eq.
9, one can lower bound the problem by expanding the feasible set Z L ⊆ ˆ Z L , where ˆ Z L is convex, as stated in Theorem 1, and proved in Appendix A.1.
Theorem 1: For any convex ˆ Z L s.t.
Z L ⊆
ˆ Z L , Eq.
9 can be bounded by the convex relaxation
[Equation]
Although Theorem 1 holds for any convex relaxation of Z L , for IBP relaxation in Gowal et al.
(2018) it can be further simpliﬁed by substituting z = W (cid:62)
L z L − 1 + b L , thus not propagating the intervals through the last layer for tighter bounding, and solved analytically as follows.
Theorem 2: The optimization in Eq.
9 can be lower-bounded by the convex optimization
[Equation]
in which ω 1 := W (cid:62) L c y,i , ω 2 := W (cid:62) L ( c a,i − c
y,i ) , ω 3 :=
b (cid:62) L ( c a,i − c y,i ) , ω 4 :=
b (cid:62) L c y,i and convex set ˆ Z L − 1 is a convex relaxation of Z L − 1 on the hidden values at L − 1 .
Furthermore, J i ( x , y ) can be analytically obtained as outlined in Alg.
Note that since η is the dual variable, any selection within the feasible set serves as a (looser but valid) lower bound, while the maximization makes the bound tight; see Appendix A.2 and A.3 for proof and a step-by-step algorithm description.
Similar to other convex relaxation-based veriﬁcation methods, in order for a networks to provide veriﬁable performance, one needs to incorporate bound propagation in training.
[Table] [Table]
on layer
[Table]
: z
[Table]
1 , ¯ z
[Table]
1 , and
[Table]
W L 2: ω
1 = W L c y,i and ω 2 = W L ( c a,i − c y,i ) , ω 3 :=
b (cid:62) L ( c a,i − c y,i ) , and ω 4 :=
b (cid:62) L c y,i 3: ζ = [ ζ 1 , ..., ζ n L ] := − ω 1 / ω 2 and vector of indices s that sorts ζ , i.e., ζ s 1 ≤ · · · ≤ ζ s nL −4: u 1 =
Π s ( ω 1 ◦ z
L − 1 ) , ¯ u 1 =
Π s ( ω 1 ◦ ¯ z
L − 1 ) , u 2 :
= Π s ( ω 2 ◦ z
L − 1 ) , ¯ u 2 :
= Π s ( ω 2 ◦ ¯
z
L − 1 ) where operators ◦ and Π s ( . )
denote element-wise multiplication, and permutation according to indices s , respectively.
[Table]
and
[Table]
for j
[Table] [Table]
do
[Table]
Compute
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Equation]
[Table]
[Equation]
[Table]
computed
[Table]
T RAINING A VERIFIABLE ROBUST CLASSIFICATION WITH DETECTION
In order to train a robust classiﬁer with detection, let us start by formalizing the objective of an adversarial attacker.
Naturally, an adaptive attacker’s objective is to craft perturbation δ such that it simultaneously evades detection and causes misclassiﬁcation.
Formally, this can be tackled by seeking δ such that loss corresponding to the winner of the two classes y and a (higher logit leading to smaller cross-entropy loss) is maximized, i.e.,
[Equation]
12) where (cid:96) xent
( z , c ) denotes the cross-entropy loss for class c =
y and c = a , and I = { 1 , 2 , ..., K, a } denotes the class index set with K + 1 elements.
Let us now deﬁne
[Equation]
(13) in which the inner maximization is closely related to that of the adversarial objective in Eq.
12 with a small difference: loss terms (cid:96) xent \ a and (cid:96) xent \ y are deﬁned as
[Equation]
This small alteration to the cost, while not changing the minimization “winner” between the true class y and rejection class a in Eq.
12 and 13, i.e.,
[Equation]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
favorably inﬂuences the training process.
That is so since, for δ such that, for instance z a <
z y , minimizing L abstainrobust ( x , y ; θ ) during training reduces to minimizing (cid:96)
xent
( f θ ( x + δ ) , y ) which in turn leads to further increasing z y while decreasing the logit value z a ; and similarly, increasing z y while decreasing z y if z y <
z a .
Intuitively however, the true objective of the classiﬁer augmented with detection on adversarial examples is to increase both z y and z a while reducing z j , ∀ j (cid:54)
=
a, y ; thus preventing any gap in between the boundary of the classes a and y , which can potentially lead to successful adaptive attacks.
Hence, minimizing Eq.
12 would be in contrast with the true underlying objective, and Eq.
13 simply prevents the raised issue.
Upon deﬁning L natural ( x , y ; θ ) :
= (cid:96) xent
( f θ ( x ) , y ) and L robust ( x , y ; θ ) :
[Equation]
max δ ∈ ∆ (cid:96) xent ( f θ ( x + δ ) , y ) , we then deﬁne the overall training loss as
[Equation]
[Equation]
where L natural ( x , y ; θ ) captures the misclassiﬁcation loss of the natural (clean) examples, and
L robust ( x , y ; θ ) denotes that of adversarial examples without considering the rejection class, i.e., similar to that of Gowal et al.
(2018), and parameters ( λ 1 , λ 2 ) trade-off clean and adversarial accuracy.
To train a robust classiﬁer, we proceed by minimizing the overall loss Eq.
14, by ﬁrst upperbounding
L robust ( x , y ; θ ) and L abstainrobust ( x , y ; θ ) .
4. U PPERBOUNDING THE TRAINING LOSS
Using Theorem 2, and restricting 0 < η ≤ η ≤ ¯ η < 1 , let us now deﬁne J η,
¯ η i ( x , y ) , where trivially
[Equation]
and can also be solved analytically similar to Theorem 2.
By generalizing the ﬁndings in Wong &
Kolter (2018); Mirman et al.
(2018), we can upper bound the robust optimization problem using our dual problem in Eq. 15, according to the following Theorem, which we prove in Appendix A.4.
Theorem 3:
For any data point ( x , y ) , and (cid:15) > 0 , and for any 0 ≤ η ≤ ¯ η ≤ 1 , the adversarial loss
L abstainrobust ( x , y ; θ ) in Eq.
13 can be upper bounded by
[Equation]
where J (cid:15),θ ( x , y ) is a ( K + 1) -dimensional vector, valued at index i as [ J (cid:15),θ ( x , y )]
i =
J η, ¯ η i ( x , y ) .
Note that maximization over η for obtaining J η, ¯ η i ( x , y ) can be done either by bisection (concave maximization) or by following Alg. 1 and substituting m = min ζ sν ≥ η ν , and
[Equation]
Remark 1 .
Setting η = ¯ η = 0 forces η = 0 which reduces J η, ¯ η i ( x , y ) in Eq.
15 to that in Eq.
5, .i.e,
J IBP i ( x , y )
=
J η, ¯ η i ( x , y ) | η =¯ η =0 , also bounding loss term L natural ( x , y ; θ ) as
[Equation]
Remark 2 .
While setting η = 0 and ¯ η = 1 gives tighter bounds, (and is thus used for the veriﬁcation counterpart in Theorem 2), strictly setting 0 < η ≤ ¯ η < 1 empirically yields better generalization of the network.
This can be intuitively understood by rewriting ω 1 + η ω 2 = W (cid:62) L ( η c a,i + (1 − η ) c
y,i ) which is a convex combination of the veriﬁcation constraints for the correct and the abstain class.
Thus η (cid:54) = 0 (cid:54) = 1 will lead to minimizing a combination of both terms, preventing gaps in between the two classes.
Also, higher values of η increase the inﬂuence of the term corresponding to the abstain case, and vice versa, whose tuning can promote abstaining by considering how desirable such outcome is (or is not).
Utilizing upperbounds in Eq.
16 and Eq.
17, we can proceed to training the network by minimizing the tractable upperbound on the overall loss min
[Equation]
Note that setting λ 1 = 0 and γ = λ 2 - and incorporation of a ramp-down process by parameter κ as detailed in Section 5 - reduces the training in Eq.
18 to that of Gowal et al.
(2018) without detection.
Complexity.
Since given IBP bounds on z L − 1 , the solution to Eq.
16 is analytically available (that is after sorting whose complexity is negligible in comparison with forward pass), computing Eq.imposes the same computational complexity as in IBP, which is twice the normal training procedure, as it requires propagating the upper and lower bounds via forward pass.
E XPERIMENTS
Empirical performance of the proposed robust classiﬁcation with detection on MNIST-10 and
CIFAR-10 datasets is reported in this section, and is compared with the state-of-the-art alternatives.
The training procedure is stabilized as detailed next 1 .
5. S TABILIZING THE TRAINING PROCEDURE
We incorporate the following mechanisms to stabilize the training procedure in our tests, where the ﬁrst two have been previously used in (Gowal et al., 2018) and (Zhang et al., 2020) as well.
Ramp down of κ : To stabilize the trade-off between nominal and veriﬁed accuracy, let us introduce parameter κ in the overall loss by trading the natural and robust loss as
[Equation]
Setting κ = 0 .
5 renders the optimization identical to that in Eq.
During the training however, we incorporate a ramp down procedure where κ starts at value κ start = 1 , thus training the model to ﬁt the natural data, and slowly decreasing it to value κ end = 0 .
5 , similar to that in Gowal et al.
Ramp up of (cid:15) :
It is very important during the training process to start at (cid:15) = 0 and gradually increase it to (cid:15) train , while also setting (cid:15) train larger than (cid:15) test can improve generalization.
Ramp down of η and ¯ η : Setting 0 <
η and ¯ η < 1 helps with better generalization.
Furthermore, setting large η and ¯ η promotes the abstain class in loss term ¯ L abstainrobust by increasing the weight of ωin Eq. 15.
Thus, we can further stabilize the training process through a ramp down procedure where these parameters start at η = η start and ¯ η = ¯ η start , and are gradually reduced to η = η end and ¯ η = ¯ η end , with η end <
η start and ¯ η end < ¯ η start .
Furthermore, although the term ¯ L robust ( x , y ; θ ) could in theory be excluded from the training process, as the term L natural ( x , y ; θ ) prevents the degenerate solution of always classifying all images in the abstain class, it’s inclusion empirically helps the stability of the training process.
5. E MPIRICAL RESULTS ON MNIST AND CIFAR
The classiﬁcation networks are identical to the large network in Gowal et al.
(2018), also detailed in Table 2, trained by minimizing the loss in Eq.
18 with the above stabilizing schemes.
Selection of parameters for each datasets is detailed in Appendix B. Since most recent detector networks have shown very low performance against adaptive attacks, and lack provable performance Tramer et al.
, we only compare the performance with other provable robust classiﬁcation methods, while focusing on the different decomposition in the reported natural and robust accuracy among these two.
As numbers in Table 1 suggest, the proposed detection/classiﬁcation network shows improved robustness against other methods, including IBP in isolation (without the detection capability), specially against larger perturbations in the CIFAR-10 dataset, which intuitively is pleasing: as larger perturbations are naturally more distinguishable, the detection capability of the network is successfully leveraged for improving the adversarial robustness.
Let us now take a closer look at the performance by focusing on the detection capability.
Effectiveness of the detection class.
By nature, the proposed classiﬁcation “adaptively chooses” between (robust) correct classiﬁcation and detection of adversarial or difﬁcult inputs during the training.
This gives rise to two phenomena: (
In veriﬁably robust methods, natural image accuracy declines as robustness improves.
In the proposed approach however, a considerable number of misclassiﬁed natural inputs are in fact abstained on , which in certain applications is more favorable than assigning them to a wrong class, as classiﬁers without detection capability would: compare 30 .
5% abstain and 25 .
6% ‘wrong-class misclassiﬁcation’ (other than abstain and the correct class) in IBP-with-detection, with that of 53 .
7% ‘wrongclass’ misclassiﬁcation in IBP on natural CIFAR-10 images in networks trained for (cid:15) = 8 / 255 .
(2) Regardless of the training procedure, the proposed classiﬁer with detection can still be veriﬁed using veriﬁcation in Eq.
4 to obtain its guaranteed robustness with only considering the correct class.
Thus, comparing this veriﬁcation percentage with that of Eq.
11 highlights the effectiveness of the abstain class in detecting perturbed images and increasing robustness: for instance, using our method 76 .
07% maximum robust error successfully decreases to 63 .
63% by considering the detection capability, on CIFAR-10 trained for (cid:15) = 8 / 255 , compared to 69 .
92% in IBP without detection.
[Table] [Table]
see veriﬁed error rates from table
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
See Fig.
1 for decomposition of the performance metrics of the proposed network over CIFARdataset, demonstrating the effectiveness of the abstain class in detecting “difﬁcult” natural images while also increasing the robustness certiﬁcate over adversarial inputs.
5. N ATURAL VERSUS ADVERSARIAL ERROR TRADEOFF
Reporting a single set point in the Pareto Frontier as reported in Table 1 gives limited understanding on how different methods trade off natural versus robust error.
To address this, a more detailed study on this trade-off in IBP-based robust classiﬁcation with and without detection is discissed here.
In order to get the best performance for IBP-based robust training without detection (that is λ 1 = 0 ), and since it is not known whether varying κ end or λ 2 will lead to a better performance, we have
[Figure] [Figure]
trained the classiﬁcation networks in two ways: (1) setting κ end = 0 .
5 and varying λ 2 ∈ [0 , 3] , and (2) setting λ 2 = 1 and varying κ end ∈ [0 , 0 .
5] , to get multiple set points along the Pareto Frontier.
Similarly, for IBP-with-detection-based classiﬁcation, we have set κ end = 0 .
5 , λ 1 = 0 .
6 , 0 . 8 , 1 . 0 for
MNIST and λ 1 = 1 . 0 for CIFAR-10, and varied λ 2 ∈ [1 4] to get various points along the frontier.
The network is trained for various (cid:15) values, with other training parameters as stated in Appendix B.
Results are plotted in Fig.
2 and 3 (presented in the Appendix due to space limitation).
As shown, the classiﬁer enhanced with detection capability is better able to trade natural and robust accuracy, thus attaining higher robustness by trading small decrease in natural accuracy.
This together with the fact that the natural accuracy decrease is also partly handled by abstaining of such natural images that would have been misclassiﬁed (as one of the original K classes) otherwise, demonstrates the effective utilization of the detection capability in the proposed method.
It is important to note that IBP w/detection allows us to obtain additional regions on this Pareto frontier that traditionalrobust-classiﬁers without detection cannot obtain, and could potentially provide additional gain to what is achievable by other various improvement techniques such as tighter relaxation and bound propagation methods.
C ONCLUSION
We proposed a new method for jointly training a provably robust classiﬁer and detector.
By introducing an additional “abstain/detection” into a classiﬁer, we have proposed a veriﬁcation scheme for classiﬁers with detection under adversarial settings, and shown that such networks can be efﬁciently trained be extending the common IBP relaxation techniques.
The effectiveness of the proposed detection scheme with provable guarantees versus SOTA robust veriﬁable classiﬁcation methods is corroborated by empirical tests on MNIST and CIFAR-10, specially against large perturbations.
A A PPENDIX A. P ROOF OF T HEOREM
Since Z L ∈ ˆ Z L it trivially holds that
[Equation]
The lower bound is now a convex minimization, which can be rewritten as
[Equation]
Deﬁning the slack variables η a ≥ 0 and η y ≥ 0 for the inequality constraints
[Equation]
the Lagrangian can be written as
[Equation]
and minimizing L ( τ, z , η a , η y ) with respect to the primal variable τ , yields η a + η y = 1 .
Deﬁning η :=
η a = 1 − η y
[Equation]
and using the fact that the dual maximization always serves as a lower bound on the primal we get
[Equation]
A. P ROOF OF T HEOREM
Following on the statement of Theorem 1 and by substituting z = W (cid:62) L z L − 1 + b L
[Equation]
we get
[Equation]
which can be reordered as
[Equation]
where ω 1 :=
W L c y,i , ω 2 :=
W L ( c a,i − c
y,i ) , ω 3 :=
b (cid:62) L ( c a,i − c y,i ) , and ω 4 :=
b (cid:62) L c y,i , which then equals
[Equation]
where minimization w.r.t.
z L − 1 is solved by the (this is under the setting for most networks with positive activations, and thus lower bound z l is always non-negative)
[Equation] [Equation]
and can be rewritten as
[Equation]
and can be rewritten as
[Equation]
where “ ◦ ” denotes the elementwise multiplication.
Thus, due to the concavity of the dual, optimal η can be found by evaluationg the objective in between the break points which are given by ζ :=
[ ζ 1 , ..., ζ n L − 1 ] with its j-th element deﬁned as ζ j := − ω 1 ,j /ω 2 ,j .
To do this, let us use s to denote the n L -ary tuple of indices that sorts ζ .
That is
[Equation]
with operator Π s ( . ) denoting the permutation of its arguments according to s , such that ˜ ζ
i = ζ s i ∀ i , and ˜ ζ is sorted in the increasing order .
We can also rewrite the problem by summing over the indices in the sorting set s instead, as
[Equation]
Now let us deﬁne u 1 :
= Π s ( ω 1 ◦ z L − 1 ) , ¯ u 1 :=
Π s ( ω 1 ◦ ¯ z L − 1 ) , u 2 := Π s ( ω 2 ◦ z L − 1 ) , ¯ u 2 :=
Π s ( ω 2 ◦ ¯ z L − 1 )
[Equation]
we get
[Equation]
In order to break the objective of maximization into piece-wise linear programming subproblems
[Equation]
let us ﬁrst identify the (indices of)
ζ s i values that fall in the feasible set 0 ≤ η ≤ 1 by
[Equation]
The overall maximization can now be reduced to piece-wise subproblems over sets ˜ ζ ν ≤ η ≤ ˜ ζ ν + for m − 1 ≤ ν ≤ M as
[Equation]
Since each of these subproblems are maximized at the boundaries of the feasible sets
[Equation]
the overall maximization essentially reduces to evaluation of the following objective function at ( M − m + 3) points η = 0 , ˜ ζ m , ˜
ζ m +1 , · · · , ˜ ζ M − 1 , ˜ ζ M ,g
[Equation]
η
[Equation]
=
[Equation]
Values of g ( η ) can be efﬁciently computed by a forward cumulative sum and forward-backward cumulative sum of u 1 and u 2 , ¯ u 1 and ¯ u 2 , thus imposing the overall complexity which is dominated by the sorting at O ( n L − 1 log( n L − 1 )) in an efﬁcient implementation.
(cid:3)
A. D ESCRIPTION OF A LGORITHM
Here is a step-by-step walk-through for Algorithm 1, with insight on how these steps are performed.
Form vectors ω 1 and ω 2 , which are the last layer values as ω 1 = W L c y,i , ω 2 = W L ( c a,i − c y,i ) , ω 3 :=
b (cid:62) L ( c a,i − c y,i )
and ω 4 :
=
b (cid:62) L c y,i .
Deﬁne ζ =
[ ζ 1 , ..., ζ n L ] :
= − ω 1 / ω 2 and get the vector of indices s that sorts it, i.e., ζ s
[Equation]
nL
[Equation]
Form the element-wise product of ( ω 1 , ω 2 ) with ( z L − 1 , ¯ z L − 1 ) ), and sort them according to the index set s .
[Equation]
Get the lowest and highest indexes ( m, M ) such that the sorted ζ vector value at those indices are in the feasible set, between 0 and 1.
Iterate over the feasible values of η = 0 , ζ s m , ζ s m +1 , · · · , ζ s M − 1 , ζ s M , 1 and compute the corresponding objective values
[Equation]
Return the maximum value of g ( η ) over the evaluated points.
A. P ROOF OF T HEOREM
Let us start by splitting the feasible set into disjoint sets of ˆ
[Equation]
ˆ Z yL
[Equation]
y } where
[Equation]
Proof is carried out by considering z ∈ ˆ Z yL − 1 and z ∈ ˆ Z aL − 1 , separately.
Restricting z ∈ ˆ Z yL − 1 we have (cid:96) xent \ a ( f θ ( x + δ ) , y ) ≤ (cid:96) xent \ y
( f θ ( x + δ ) , a ) which leads to
[Equation]
Loss function (cid:96) xent \ a is the cross entropy loss deﬁned on the K -dimensional vector [ z L, 1 , · · · , z L,K ] and class y , and thus following Wong & Kolter (2018) given its transnational invariance equals
[Equation]
with 1 denoting the ( K + 1) -dimensional vector of all ones.
Given the invariance of (cid:96) xent \ a with respect to z L,a , it can ﬁnally be upperbounded by taking the upperbound for all i indices where i = 1 , ..., K, i (cid:54) = a, y and lowerbound at index i =
y .
Note that for i
=
y , value
[ z L − z L,y 1 ] i = 0 , and a lower bound on other entries i = 1 , ..., K, i (cid:54)
= a, y can be obtained by
[Equation]
where the ﬁrst equality holds since ˆ Z yL − 1 :
= { z L − 1 | z L − 1 ,a < z L − 1 ,y } for z ∈ ˆ Z yL − 1 , second inequality is due to Theorem 2, and third inequality is given by Eq. 15.
Thus, for z ∈ ˆ Z y L − 1 the loss term is now upperbounded by
[Equation]
where
[Equation]
if
[Equation]
otherwise .
[Table]
Similarly, it can be shown that for Thus, for z ∈ ˆ Z aL − 1 the loss term is now upperbounded by
[Equation]
The equality of (cid:96) xent \ y ( − J (cid:15),θ ( x , y ) , a ) = (cid:96) xent \ a ( − J (cid:15),θ ( x , y ) , y ) trivially follows from the fact that [ J (cid:15),θ ( x , y )]
i = 0 for i = a, y .
Thus, since ˆ Z L − 1 = ˆ Z yL − 1 ∪ ˆ Z aL − 1 , the proof is complete.
(cid:3)
B A PPENDIX : E XPERIMENT SET UP
Training parameters and schedules are similar to (Gowal et al., 2018) and (Zhang et al., 2020), and outlined in detail here.
For training the classiﬁer network with architecture given in Table 2, for both datasets, Adam optimizer with learning rate of 5 × 10 − 4 is used.
Unless stated differently, κ is scheduled by a linear ramp-down process, starting at 1 , which after a warm-up perio,d is ramped down to value κ end = 0 .
Value of (cid:15) during the training is also simultaneously scheduled by a linear ramp-up, starting at 0 , and ramped up to the ﬁnal value of (cid:15) train , reported in Tabel 1, and networks are trained
with
a single NVIDIA Tesla V100S GPU.
• For MNIST, the network is trained in 100 epochs with batchsize of 100 (total of 60 K steps).
A warm up period of 3 epochs ( 2 K steps) is used (normal classiﬁcation training with no robust loss), followed up by a ramp-up duration of 18 epochs ( 10 K steps), and the learning rate is decayed × 10 at epochs 25 and 42.
No data augmentation is used.
Furthermore, ﬁxed selection of ¯ η = 0 .
9 and η = 0 .
1 during training is used for this dataset with no ramp-down.
Reported numbers in Table 1 corresponds to λ 1 = 1 and λ 2 = 2 for (cid:15)
= 0 .
3 , and λ 1 = 0 .
6 and λ 2 = 1 for (cid:15) = 0 .
4 respectively.
• For CIFAR10, the network is trained in 3200 epochs with batchsize of 1600 (total of 100 K steps).
A warm up period of 320 epochs ( 10 K steps) is used (normal classiﬁcation training with no robust loss), followed up by a ramp-up duration of 1600 epochs ( 50 K steps), and the learning rate is decayed × 10 at epochs 2600 and 3040 (60k and 90K steps).
Random translations and ﬂips, and normalization of each image channel (using the channel statistics from the train set) is used during training.
Furthermore, during training for all (cid:15) values we have selected ¯ η start = 1 . 0 and ¯ η end = 0 . 9 .
Additionally, η end = 0 .
1 is used during training, with η start = 0 .
1 for (cid:15)
= 2 / 255 (no ramp down), η start = 0 .
3 for (cid:15)
= 8 / 255 , η start = 0 .for
(cid:15) = 12 / 255 , and η start = 0 .
5 for (cid:15)
= 16 / 255 .
The intuition behind these parameters selection lies in Remark 2, as large η values promote the abstain option more, so for large (cid:15) , we start with larger η start as well.
Reported numbers in Tabel 1 correspond to λ 1 =for all (cid:15) values, and λ 2 = 3 . 0 for (cid:15)
= 2 /
255 , λ 2 = 2 .
9 for (cid:15) = 8 / 255 , and λ 2 = 3 . 1 for (cid:15)
= 16 / 255 to insure similar natural accuracy for fair comparison against other methods.
[Figure]
B. P ARETO F RONTIER FOR MNIST DATASET B. E MPIRICAL ATTACK SUCCESS RATE USING PGD ATTACKS
In order to obtain empirical attack success on the trained networks, adversarial perturbations are sought by solving
[Equation]
This attack is indeed an adaptive attack as it aims at circumventing the detection while trying to cause misclassiﬁcation (Tramer et al., 2020).
Perturbations are sought by maximizing this objective using
PGD with 200 -steps for mnist and 500 -steps for CIFAR-10 Madry et al.
(2017), with 10 random restarts.
It is interesting to note that the achieved attack success rate in Table 1 is well below the veriﬁed robust error, further implying the effectiveness of incorporation of the detection mechanism as the true robustness of the system against practical adaptive PGD attacks are considerably stronger in comparison to robust classiﬁcation without detection.