C OST -S ENSITIVE R OBUSTNESS AGAINST A DVERSARIAL E XAMPLES
Anonymous authors
Paper under double-blind review
A BSTRACT
Several recent works have developed methods for training classiﬁers that are certiﬁably robust against norm-bounded adversarial perturbations.
However, these methods assume that all the adversarial transformations provide equal value for adversaries, which is seldom the case in real-world applications.
We advocate for cost-sensitive robustness as the criteria for measuring the classiﬁer’s performance for speciﬁc tasks.
We encode the potential harm of different adversarial transformations in a cost matrix, and propose a general objective function to adapt the robust training method of Wong & Kolter (2018) to optimize for cost-sensitive robustness.
Our experiments on simple MNIST and CIFAR10 models and a variety of cost matrices show that the proposed approach can produce models with substantially reduced cost-sensitive robust error, while maintaining classiﬁcation accuracy.
I NTRODUCTION
Despite the exceptional performance of deep neural networks (DNNs) on various machine learning tasks such as malware detection (Saxe & Berlin, 2015), face recognition (Parkhi et al., 2015) and autonomous driving (Bojarski et al., 2016), recent studies (Szegedy et al., 2014; Goodfellow et al., 2015) have shown that deep learning models are vulnerable to misclassifying inputs, known as adversarial examples , that are crafted with targeted but visually-imperceptible perturbations.
While several defense mechanisms have been proposed and empirically demonstrated to be successful against existing particular attacks (Papernot et al., 2016; Goodfellow et al., 2015), new attacks (Carlini & Wagner, 2017; Tram`er et al., 2017; Athalye et al., 2018) are repeatedly found that circumvent such defenses.
To end this arm race, recent works (Wong & Kolter, 2018; Raghunathan et al., 2018; Wong et al., 2018; Wang et al., 2018) propose methods to certify examples to be robust against some speciﬁc norm-bounded adversarial perturbations for given inputs and to train models to optimize for certiﬁable robustness.
However, all of the aforementioned methods aim at improving the overall robustness of the classiﬁer.
This means that the methods to improve robustness are designed to prevent seed examples in any class from being misclassiﬁed as any other classes.
Achieving such a goal requires producing a perfect classiﬁer, and has, unsurprisingly, remained elusive.
Indeed, Mahloujifar et al.
(2018) proved that if the metric probability space is concentrated, overall adversarial robustness is unattainable for any classiﬁer with initial constant error.
We argue that overall robustness may not be the appropriate criteria for measuring system performance in security-sensitive applications, since only certain kinds of adversarial misclassiﬁcations pose meaningful threats or provide value for adversaries.
Whereas overall robustness places equal emphasis on every adversarial transformation, from a security perspective, only certain transformations matter.
As a simple example, misclassifying a malicious program as benign results in more severe consequences than the reverse.
In this paper, we propose a general method for adapting provable defenses against norm-bounded perturbations to take into account the potential harm of different adversarial class transformations.
Inspired by cost-sensitive learning (Domingos, 1999; Elkan, 2001) for non-adversarial contexts, we capture the impact of different adversarial class transformations using a cost matrix C , where each entry represents the cost of an adversary being able to take a natural example from the ﬁrst class and perturb it so as to be misclassiﬁed by the model as the second class.
Instead of reducing the overall robust error, our goal is to minimize the cost-weighted robust error (which we deﬁne for both binary and real-valued costs in C ).
The proposed method incorporates the speciﬁed cost matrix into the training objective function, which encourages stronger robustness guarantees on cost-sensitive class transformations, while maintaining the overall classiﬁcation accuracy on the original inputs.
Contributions.
We introduce the notion of cost-sensitive robustness (Section 3.1) as a criteria to assess the expected performance of a classiﬁer when facing adversarial examples.
Speciﬁcally, by encoding the consequences of different adversarial transformations into a cost matrix, we propose an objective function for training a cost-sensitive robust classiﬁer (Section 3.2) for any given task.
The proposed method is general in that it can incorporate any type of cost matrix, including both binary and real-valued.
We demonstrate the effectiveness of the proposed cost-sensitive defense model for a variety of cost scenarios on two benchmark image classiﬁcation datasets: MNIST (Section 4.1) and
CIFAR10 (Section 4.2).
Compared with the state-of-the-art overall robust defense model (Wong &
Kolter, 2018), our model achieves signiﬁcant improvements in cost-sensitive robustness for different tasks, while maintaining approximately the same classiﬁcation accuracy on both datasets.
Notation.
We use lower-case boldface letters such as x for vectors and capital boldface letters such as A to represent matrices.
Let [ m ] be the index set { 1 , 2 , . . .
, m } and A ij be the ( i, j ) -th entry of matrix A .
Denote the i -th natural basis vector, the all-ones vector and the identity matrix by e i ,and I respectively.
For any vector x ∈ R d , the (cid:96) ∞ -norm of x is deﬁned as (cid:107) x (cid:107) ∞ =
max i ∈ [ d ] | x i | .
B ACKGROUND
In this section, we provide a brief introduction on related topics, including neural network classiﬁers, adversarial examples, defenses with certiﬁed robustness, and cost-sensitive learning.
2. N EURAL N ETWORK C LASSIFIERS
A K -layer neural network classiﬁer can be represented by a function f : X → Y such that f ( x ) =
f K − 1 ( f K − 2 ( · · · ( f 1 ( x )))) , for any x ∈ X .
For k ∈ { 1 , 2 , . . .
, K − 2 } , the mapping function f k ( · ) typically consists of two operations: an afﬁne transformation (either matrix multiplication or convolution) and a nonlinear activation.
In this paper, we consider rectiﬁed linear unit (ReLU) as the activation function.
If denote the feature vector of the k -th layer as z k , then f k ( · ) is deﬁned as
[Equation]
where W k denotes the weight parameter matrix and b k the bias vector.
The output function f K − 1 ( · ) maps the feature vector in the last hidden layer to the output space Y solely through matrix multiplication: z K =
f K − 1 ( z K − 1 ) =
W K − 1 z K − 1 + b K − 1 , where z K can be regarded as the estimated score vector of input x for different possible output classes.
In the following discussions, we use f θ to represent the neural network classiﬁer, where θ =
{ W 1 , . .
, W K − 1 , b 1 , . . .
, b K − 1 } denotes the model parameters.
To train the neural network, a loss function (cid:80) N i =1 L ( f θ ( x i ) , y i ) is deﬁned for a set of training examples { x i , y i } Ni =1 , where x i is the i -th input vector and y i denotes its class label.
Cross-entropy loss is typically used for multiclass image classiﬁcation.
With proper initialization, all model parameters are then updated iteratively using backpropagation.
For any input example (cid:101) x , the predicted label (cid:98) y is given by the index of the largest predicted score among all classes, argmax j [ f θ ( (cid:101) x )] j .
2. A DVERSARIAL E XAMPLES
An adversarial example is an input, generated by some adversary, which is visually indistinguishable from an example from the natural distribution, but is able to mislead the target classiﬁer.
Since “visually indistinguishable” depends on human perception, which is hard to deﬁne rigorously, we consider the most popular alternative: input examples with perturbations bounded in (cid:96) ∞
-norm (Goodfellow et al., 2015).
More formally, the set of adversarial examples with respect to seed example { x 0 , y 0 } and classiﬁer f θ ( · ) is deﬁned as
[Equation]
where (cid:15) > 0 denotes the maximum perturbation distance.
Although (cid:96) p distances are commonly used in adversarial examples research, they are not an adequate measure of perceptual similarity (Sharif et al., 2018) and other minimal geometric transformations can be used to ﬁnd adversarial examples (Engstrom et al., 2017; Kanbak et al., 2017; Xiao et al., 2018).
Nevertheless, there is considerable interest in improving robustness in this simple domain, and hope that as this research area matures we will ﬁnd ways to apply results from studying simpliﬁed problems to more realistic ones.
2. D EFENSES WITH C ERTIFIED R OBUSTNESS
A line of recent work has proposed defenses that are guaranteed to be robust against norm-bounded adversarial perturbations.
Hein & Andriushchenko (2017) proved formal robustness guarantees against (cid:96) 2 -norm bounded perturbations for two-layer neural networks, and provided a training method based on a surrogate robust bound.
Raghunathan et al.
(2018) developed an approach based on semideﬁnite relaxation for training certiﬁed robust classiﬁers, but was limited to two-layer fullyconnected networks.
Our work builds most directly on Wong & Kolter (2018), which can be applied to deep ReLU-based networks and achieves the state-of-the-art certiﬁed robustness on MNIST dataset.
Following the deﬁnitions in Wong & Kolter (2018), an adversarial polytope Z (cid:15) ( x ) with respect to a given example x is deﬁned as
[Equation]
which contains all the possible output vectors for the given classiﬁer f θ by perturbing x within an (cid:96) ∞ -norm ball with radius (cid:15) .
A seed example, { x 0 , y 0 } , is said to be certiﬁed robust with respect to maximum perturbation distance (cid:15) , if the corresponding adversarial example set A (cid:15) ( x 0 , y 0 ; θ ) is empty.
Equivalently, if we solve, for any output class y targ (cid:54) =
y 0 , the optimization problem, minimize z K [
[Equation]
subject to z K ∈ Z (cid:15)
[Equation]
(2.3) then according to the deﬁnition of A (cid:15) ( x 0 , y 0 ; θ ) in (2.1), { x 0 , y 0 } is guaranteed to be robust provided that the optimal objective value of (2.3) is positive for every output class.
To train a robust model on a given dataset { x i , y i } Ni =1 , the standard robust optimization aims to minimize the sample loss function on the worst-case locations through the following adversarial loss
[Equation]
where L ( · , · ) denotes the cross-entropy loss.
However, due to the nonconvexity of the neural network classiﬁer f θ ( · ) introduced by the nonlinear ReLU activation, both the adversarial polytope (2.2) and training objective (2.4) are highly nonconvex.
In addition, solving optimization problem (2.3) for each pair of input example and output class is computationally intractable.
Instead of solving the optimization problem directly, Wong & Kolter (2018) proposed an alternative training objective function based on convex relaxation, which can be efﬁciently optimized through a dual network.
Speciﬁcally, they relaxed Z (cid:15) ( x ) into a convex outer adversarial polytope (cid:101) Z (cid:15) ( x ) by replacing the ReLU inequalities for each neuron z
= max { (cid:98) z, 0 } with a set of inequalities, z ≥
[Equation] [Equation]
.5) where u, (cid:96) denote the lower and upper bounds on the considered pre-ReLU activation.
1 Based on the relaxed outer bound (cid:101) Z (cid:15) ( x ) , they propose the following alternative optimization problem, minimize z
[Equation] [Equation]
subject to z
[Equation]
(2.6) which is in fact a linear program.
Since Z (cid:15) ( x ) ⊆ (cid:101) Z (cid:15) ( x ) for any x ∈ X , solving (2.6) for all output classes provides stronger robustness guarantees compared with (2.3), provided all the optimal objective values are positive.
In addition, they derived a guaranteed lower bound, denoted by
J (cid:15) (cid:0) x 0 , g θ ( e y
− e y targ ) (cid:1) , on the optimal objective value of (2.6) using duality theory, where g θ ( · ) is a K -layer feedforward dual network (Theorem 1 in Wong & Kolter (2018)).
Finally, according to the properties of cross-entropy loss, they minimize the following objective to train the robust model, which serves as an upper bound of the adversarial loss (2.4): minimize
[Equation]
where g θ ( · ) is regarded as a columnwise function when applied to a matrix.
Although the proposed method in Wong & Kolter (2018) achieves certiﬁed robustness, its computational complexity is quadratic with the network size in the worst case so it only scales to small networks.
Recently, Wong et al.
(2018) extended the training procedure to scale to larger networks by using nonlinear random projections.
However, if the network size allows for both methods, we observe a small decrease in performance using the training method provided in Wong et al.
Therefore, we only use the approximation techniques for the experiments on CIFAR10 ( § 4.2), and use the less scalable method for the MNIST experiments ( § 4.1).
2. C OST -S ENSITIVE L EARNING
Cost-sensitive learning (Domingos, 1999; Elkan, 2001; Liu & Zhou, 2006) was proposed to deal with unequal misclassiﬁcation costs and class imbalance problem in many real-world applications, such as database marketing, fraud detection and medical diagnosis.
The key observation is that cost-blind learning algorithms tend to overwhelm the major class, but the neglected minor class is often our primary interest.
For example, in medical diagnosis misclassifying a rare cancerous lesion as benign is extremely costly.
Various cost-sensitive learning algorithms (Kukar & Kononenko, 1998;
Zadrozny et al., 2003; Zhou & Liu, 2010) has been proposed in literature, but few of them considered adversarial settings.
Dalvi et al.
(2004) studied the naive Bayes classiﬁer for spam detection in the presence of a cost-sensitive adversary, and developed an adversary-aware classiﬁer based on game theory.
Asif et al.
(2015) proposed a cost-senstive robust minimax approach that hardens a linear discriminant classiﬁer with robustness in the adversarial context.
All of these methods are developed for simple linear classiﬁers, which cannot be extended to neural network classiﬁers directly.
In addition, the robustness of their proposed classiﬁer is only examined experimentally based on the performance against some speciﬁc adversary, so does not provide any notion of certiﬁed robustness.
Recently, Dreossi et al.
(2018) advocated for the idea of using application-level semantics in adversarial analysis, however, they didn’t provide a formal method on how to train such classiﬁer.
In contrast, our work provides a practical training method that hardens neural network classiﬁers with certiﬁed cost-sensitive robustness against adversarial perturbations.
T RAINING C OST -S ENSITIVE R OBUST C LASSIFIER
The approach introduced in Wong & Kolter (2018) penalizes all adversarial class transformations equally, even though the consequences of adversarial examples usually depends on the speciﬁc class transformations.
Here, we provide a formal deﬁnition of cost-sensitive robustness ( § 3.1) and propose a general method for training cost-sensitive robust models ( § 3.2).
3. C ERTIFIED C OST -S ENSITIVE R OBUSTNESS
Our approach uses a cost matrix C that encodes the cost of different adversarial examples.
First, we consider the case where there are m classes and C is a m × m binary matrix with C i,j ∈ { 0 , 1 } .
The value C jj (cid:48) indicates whether we care about an adversary transforming a seed input in class j into one recognized by the model as being in class j (cid:48) .
If the adversarial transformation j → j (cid:48) matters,
C jj (cid:48)
= 1 , otherwise C jj (cid:48)
= 0 .
Let Ω j
= { j (cid:48) ∈ [ m ] : C jj (cid:48)
(cid:54) = 0 } be the index set of output classes that induce cost with respect to input class j .
For any j ∈ [ m ] , let δ j =
if Ω j is an empty set, and δ j = 1 otherwise.
We are only concerned with adversarial transformations from a seed class j to target classes j (cid:
) ∈
Ω j .
For any example x in seed class j , x is said to be certiﬁed cost-sensitive robust if the lower bound J (cid:15) ( x , g θ ( e j − e j (cid:48) )) ≥ 0 for all j (cid:48) ∈
Ω j .
That is, no adversarial perturbations in an (cid:96) ∞ -norm ball around x with radius (cid:15) can mislead the classiﬁer to any target class in Ω j .
The cost-sensitive robust error on a dataset { x i , y i } Ni =1 is deﬁned as the number of examples that are not guaranteed to be cost-sensitive robust over the total number of valued seed examples:
[Equation]
where # A represents the cardinality of a set A , and N j is the total number of examples in class j .
Next, we consider a more general case where C is a m × m real-valued cost matrix.
Each entry of C is a non-negative real number, which represents the cost of the corresponding adversarial transformation.
To take into account the different potential costs among adversarial examples, we propose to measure the cost-sensitive robustness by the average certiﬁed cost of adversarial examples.
The cost of an adversarial example x in class j is deﬁned as the sum of all C jj (cid:48)
such that
J (cid:15) ( x , g θ ( e j − e j (cid:48) ))
< 0 .
Intuitively speaking, an adversarial example will induce more cost if it can be adversarially misclassiﬁed as more target classes with high cost.
Accordingly, the robust cost is deﬁned as the total cost of adversarial examples divided by the total number of valued seed examples:
[Equation]
where 1 ( · ) denotes the indicator function.
3. C OST -S ENSITIVE R OBUST O PTIMIZATION
Recall that our goal is to develop a classiﬁer with certiﬁed cost-sensitive robustness as deﬁned in § 3.1, while maintaining overall classiﬁcation accuracy.
According to the guaranteed lower bound,
J (cid:15) (cid:0) x 0 , g θ ( e y
− e y targ ) (cid:1) on (2.6)
, we propose to solve the following robust optimization problem with respect to a neural network classiﬁer f θ : minimize θ
[Equation]
where α ≥ 0 denotes the regularization parameter.
The ﬁrst term in (3.2) denotes the cross-entropy loss for standard classiﬁcation, whereas the second term accounts for the cost-sensitive robustness, which is inspired by the cost-sensitive CE loss (Khan et al., 2018).
Compared with (2.7), we include a regularization parameter α to control the trade-off between classiﬁcation accuracy on original inputs and adversarial robustness.
To provide cost-sensitivity, the loss function selectively penalizes the adversarial examples based on their cost.
For binary cost matrixes, the regularization term penalizes every cost-sensitive adversarial example equally, but has no impact for instances where C jj (cid:48)
= 0 .
For the real-valued costs, a larger value of C jj (cid:48) increases the weight of the corresponding adversarial transformation in the training objective.
Optimization problem (3.2) can be solved efﬁciently using gradient-based algorithms, such as stochastic gradient descent and ADAM (Kinga & Adam, 2015).
E XPERIMENTS
We evaluate the performance of the proposed cost-sensitive robustness training on models for two benchmark image classiﬁcation datasets: MNIST (LeCun et al., 2010) and CIFAR10 (Krizhevsky &
Hinton, 2009).
We compare our results for various cost scenarios with overall robustness training ( § 2.3) as a baseline.
For both datasets, the family of adversarial attacks is speciﬁed as all the adversarial perturbations that are bounded in an (cid:96) ∞ -norm ball.
4. MNIST
We use the same convolutional neural network architecture (LeCun et al., 1998) for MNIST as Wong & Kolter (2018), which includes two convolutional layers, with 16 and 32 ﬁlters respectively, and a two fully-connected layers, consisting of 100 and 10 hidden units respectively.
ReLU activations are applied to each layer expect the last one.
For both our cost-sensitive robust model and the overall robust model, we randomly split the 60,000 training samples into ﬁve folds of equal size, train the classiﬁer over 60 epochs on four of them using the Adam optimizer (Kinga & Adam, 2015) with batch size 50 and learning rate 0.001, and treat the remaining one as validation dataset for model selection.
In addition, we use the (cid:15) -scheduling
and learning rate decay techniques, where we increase (cid:15) from 0 .
05 to the desired value linearly over the ﬁrst 20 epochs and decay the learning rate by 0.
every 10 epochs for the remaining epochs.
Baseline: Overall Robustness.
Figure 1(a) illustrates the learning curves of both classiﬁcation error and overall robust error during training based on robust loss (2.7) with maximum perturbation
[Figure]
[Table]
[Figure]
[Table]
[Figure]
R o b u s t
e rr o r r a t e
[Figure]
distance (cid:15) = 0 .
The model with classiﬁcation error less than 4% and minimum overall robust error on the validation dataset is selected over the 60 training epochs.
The best classiﬁer reaches 3 .
39% classiﬁcation error and 13 .
80% overall robust error on the 10,000 MNIST testing samples.
We report the robust test error for every adversarial transformation in Figure 1(b) (for the model without any robustness training all of the values are 100% ).
The ( i, j ) -th entry is a bound on the robustness of that seed-target transformation—the fraction of testing examples in class i that cannot be certiﬁed robust against transformation into class j for any (cid:15) norm-bounded attack.
As shown in Figure 1(b), the attack vulnerability differs considerably among class pairs.
For instance, only 0 .
26% of seeds in class 1 cannot be certiﬁed robust for target class 9 compare to 10% of seeds from class 9 into class 4.
Binary Cost Matrix.
Next, we evaluate the effectiveness of cost-sensitive robustness training in producing models that are more robust for the important adversarial transformations.
We consider four types of tasks deﬁned by different binary cost matrices that capture different sets of adversarial transformations: single pair : particular seed class s to particular target class t ; single seed : particular seed class s to any target class; single target : any seed class to particular target class t ; and multiple : multiple seed and target classes.
For each setting, the cost matrix is deﬁned as C i,j =
if
i, j ) is selected; C i,j = 0 otherwise.
In general, we expect that the sparser the cost matrix, the more opportunity there is for cost-sensitive robustness to improve on overall robustness.
For the single pair task, we selected three representative adversarial goals based on the original robustness in Figure 1(b): a low vulnerability pair (0,2), medium vulnerability pair (6,5) and high vulnerability pair (4,9).
Similarly, for the single seed and single target tasks we select three examples representing low, medium, and high vulnerability (see Appendix B.1 for results for all classes).
For the multiple task, we consider four variations: (i) the ten most vulnerable seed-target transformations; (ii) ten randomly-selected seed-target transformations; (iii) all the class transformations from odd digit seed to any other class; (iv) all the class transformations from even digit seed to any other class.
Table 1 summarizes the results, comparing the cost-sensitive robust error between the baseline model trained for overall robustness and a model trained using our cost-sensitive robust optimization.
The proposed cost-sensitive robust defense model is trained with (cid:15) = 0 .
2 based on loss function (3.2) and the corresponding cost matrix C (see Appendix B.2 for comparison results with different (cid:15) ).
The regularization parameter α is tuned via cross validation (see Appendix A in the supplementary materials for details).
We report the selected best α , classiﬁcation error and cost-sensitive robust error on the testing dataset.
Our model achieves a substantial improvement on the cost-sensitive robustness compared with the baseline model on all of the considered tasks.
The decrease on cost-sensitive robust error varies from 30% to 90% , and is generally higher for lower sparsity cost matrices.
In particular, our classiﬁer [Table] [Table] reduces the number of cost-sensitive adversarial examples from 198 to 12 on the single target task with digit 1 as the concerned class.
Real-valued Cost Matrices.
Loosely motivated by an forging adversary who obtains value by changing the semantic interpretation of a number, we consider two real-valued cost matrices: smalllarge , where only adversarial transformations from a smaller digit class to a larger one are valued, and the cost of valued-transformation is quadratic with the absolute difference between the seed and target class digits: C ij = 0 if i ≥ j ; C ij
=
( i − j ) 2 otherwise; large-small : only adversarial transformations from a larger digit class to a smaller one are valued: C ij = 0 if i ≤ j ; C ij = ( i − j )otherwise.
We tune α for the cost-sensitive robust model on the training MNIST dataset via cross validation [Table] and set all the other parameters the same as in the binary case.
The certiﬁed robust error for every adversarial transformation on MNIST testing dataset are shown in Figures 2(a) and 2(b), and the classiﬁcation error and robust cost are given in Table 2.
Compared with the heatmap of robust test error based on the overall robust model in Figure 1(b), our trained classiﬁer achieves stronger robustness guarantees on the adversarial transformations that induce costs, especially on those with larger costs.
4. CIFAR
We use the same neural network architecture for the CIFAR10 dataset as Wong et al.
(2018), with four convolutional layers and two fully-connected layers.
For memory and computational efﬁciency, we incorporate the approximation technique based on nonlinear random projection during the training phase (Wong et al.
We train both the baseline model and our model using random projection of 50 dimensions, and optimize the training objective using SGD.
Other parameters such as learning rate and batch size are set as same as those in Wong et al.
Given a speciﬁc task,
[Table]
[Table] [Figure] [Table] we train the proposed cost-sensitive robust classiﬁer on 80% randomly-selected training examples, and tune the regularization parameter α according to the performance on the remaining examples as validation dataset.
The tasks are similar to those for MNIST ( § 4.1), except for the multiple task we cluster the ten CIFAR10 classes into two large groups: animals and vehicles, and consider the cases where only transformations between an animal class and a vehicle class are sensitive, and the converse.
Table 3 shows the comparison results on the testing data based on different robust defense models with (cid:15) = 2 / 255 (see Appendix B.2 for comparison results with larger (cid:15) ).
For all of the aforementioned tasks, our model substantially reduces the cost-sensitive robust error while keeping a lower classiﬁcation error than the baseline.
For the real-valued task, we are concerned with adversarial transformations from seed examples in vehicle classes to other target classes.
In addition, more cost is placed on transformations from vehicle to animal, which is 10 times larger compared with that from vehicle to vehicle.
Figures 2(c) and 2(d) illustrate the pairwise robust test error using overall robust model and the proposed classiﬁer for the aforementioned real-valued task on CIFAR10.
C ONCLUSION
By focusing on overall robustness, previous robustness training methods expend a large fraction of the capacity of the network on unimportant transformations.
We argue that for most scenarios, adversarial transformations have different costs depending on the seed and target class, so robust training methods should be designed to account for these differences.
By incorporating a cost matrix into the training objective, we proposed a general method for producing a cost-sensitive robust classiﬁer, and demonstrate the effectiveness of our method for a variety of cost scenarios on two simple datasets.
There remains a large gap between the small models and limited attacker capabilities for which we can achieve certiﬁable robustness, and the complex models and unconstrained attacks that may be important in practice, but we hope that considering cost-sensitive robustness will be a step towards achieving more realistic robustness goals for important problems.
Cost-Sensitive Robustness against Adversarial Examples
Supplemental Materials
A P ARAMETER T UNING
For experiments on the MNIST dataset, we ﬁrst perform a coarse tuning on regularization parameter α with searching grid { 10 − 2 , 10 − 1 , 10 0 , 10 1 , 10 2 } , and select the most appropriate one, denoted by α coarse , with overall classiﬁcation error less than 4% and the lowest cost-sensitive robust error on validation dataset.
Then, we further ﬁnely tune α from the range { 2 − 3 , 2 − 2 , 2 − 1 , 2 0 , 2 1 , 2 2 , 2 3 } · α coarse , and choose the best robust model according to the same criteria.
Figures 3(a) and 3(b) show the learning curves for task B with digit 9 as the selected seed class based on the proposed cost-sensitive robust model with varying α (we show digit 9 because it is one of the most vulnerable seed classes).
The results suggest that as the value of α increases, the corresponding classiﬁer will have a lower cost-sensitive robust error but a higher classiﬁcation error, which is what we expect from the design of (3.2).
We observe similar trends for the learning curves for the other tasks, so do not present them here.
For the CIFAR10 experiments, a similar tuning strategy is implemented.
The only difference is that we use 35% as the threshold of overall classiﬁcation error for selecting the best α .
[Figure]
t i o n
[Figure]
r
[Figure]
as the selected seed class on MNIST using the proposed model with varying α : (a) learning curves of classiﬁcation error; (b) learning curves of cost-sensitive robust error.
The maximum perturbation distance is speciﬁed as (cid:15)
= 0 .
B O THER E XPERIMENTS
Here, we provide results from some additional experiments that generally conﬁrm the results included in the paper.
B. S INGLE S EED AND T ARGET T ASKS
In addition to the tasks shown in Table 1, we evaluated all the single-seed and single target tasks for
MNIST.
Figures 4(a) and 4(b) demonstrate the comparison results on cost-sensitive robustness between our model and baseline model for each single seed and single target class cost scenario.
Our model achieves signiﬁcant improvements for all of the considered binary tasks.
We remark that the overall classiﬁcation errors of both models are similar in all of the cases, thus we do not present them.
B. V ARYING A DVERSARY S TRENGTH
We also investigate the performance of our model against different levels of adversarial strength by varying the value of (cid:15) that deﬁnes the (cid:96) ∞ ball available to the adversary.
[Figure]
Figures 5(a) and 5(b) show the overall classiﬁcation and cost-sensitive robust error of our best trained model, compared with the baseline model, on MNIST single seed task with digit 9 and
CIFAR single seed task with dog as the seed class of concern, respectively.
In particular, we vary the maximum (cid:96) ∞ perturbation distance (cid:15) from { 0 . 1 , 0 . 15 , 0 . 2 , 0 . 25 } for MNIST and vary (cid:15) from { 2 / 255 , 4 / 255 , 6 / 255 } for CIFAR10.
Under all the considered attack models, the proposed classiﬁer achieves better cost-sensitive adversarial robustness than the baseline, while maintaining similar classiﬁcation accuracy on original data points.
As the adversarial strength increases, the improvement for cost-sensitive robustness over overall robustness becomes more signiﬁcant.
[Figure]
B. C OMPARISON WITH S TANDARD C OST -S ENSITIVE C LASSIFIER
As discussed in Section 2.
[Figure]
prior work on cost-sensitive learning mainly focuses on the nonadversarial setting.
In this section
[Figure]
we investigate the robustness of the cross-entropy based costsensitive classiﬁer proposed in Khan et al.
[Figure]
and compare the performance of their classiﬁer with our proposed cost-sensitive robust classiﬁer.
Given a set of training examples { ( x i , y i ) } Ni = and cost matrix C with each entry representing the cost of the corresponding misclassiﬁcation
[Figure]
the evaluation metric for cost-sensitive learning is deﬁned as the average cost of misclassiﬁcations
[Figure]
or more concretely
[Equation]
where m is the total number of class labels and f θ ( · ) denotes the neural network classiﬁer as introduced in Section 2.1.
In addition, the cross-entropy based cost-sensitive training objective takes the following form:
[Equation]
To provide a fair comparison, we assume the cost matrix used for (B.1) coincides with the cost matrix used for cost-sensitive robust training (3.2) in our experiment, whereas they are unlikely to be the same for real security applications 2 .
In particular, we consider the small-large real-valued task for
MNIST, where the cost matrix C is designed as C ij = 0 .
1 , if i > j ; C ij = 0 , if i = j ; C ij =
( i − j ) 2 , otherwise.
Table B.3 demonstrates the comparison results of different classiﬁers in such setting: baseline standard deep learning classiﬁer, standard cost-sensitive classiﬁer (Khan et al., 2018) trained using (B.1), classiﬁer trained for overall robustness (Wong & Kolter, 2018) and our proposed classiﬁer trained for cost-sensitive robustness.
Compared with baseline, the standard cost-sensitive classiﬁer indeed reduces the misclassiﬁcation cost, but does not provide any improvement on the robust cost, as deﬁned in (3.1) for real-valued cost matrix.
In sharp contrast, our robust training method signiﬁcantly improves the cost-sensitive robustness.
[Table]
C B ACKGROUND ON C OST -S ENSITIVE L EARNING
Cost-sensitive learning (Kukar & Kononenko, 1998; Domingos, 1999; Elkan, 2001) was originally introduced to tackle the problem of unequal misclassiﬁcation costs, which has been an active research ﬁeld in data mining and machine learning for decades.
Witnessing the success of deep learning in machine learning applications, recent works (Liu & Zhou, 2006; Chung et al., 2015; Raj et al., 2016; Wang et al., 2016; Khan et al., 2018; Buda et al., 2018) focused on neural networks as the core classiﬁer for cost-sensitive learning.
Most cost-sensitive learning research is motivated by the class imbalance problem, a special case of general cost sensitivity, while a few of the proposed methods can be generalized beyond class imbalance.
For instance, Liu & Zhou (2006) studied the effect of sampling and threshold moving on training cost-sensitive classiﬁers using neural networks, which are only applicable to class imbalanced datasets.
Nevertheless, there exist neural network based cost-sensitive methods that can incorporate an arbitrary cost matrix into the training procedure.
In particular, Kukar & Kononenko (1998) proposed four approaches to harden neural network classiﬁers with general cost-sensitivity, including modiﬁcations using a cost matrix on testing probabilities, outputs of the networks, learning rate and training loss function.
Among all four methods, they observed that incorporating the cost matrix into training loss is the most effective.
From a cost-sensitive learning perspective, our proposed robust classiﬁer falls into this category, but relies on cross-entropy loss instead of MSE loss for training.
Chung et al.
(2015) proposed a pretrained cost-sensitive deep learning algorithm by replacing the softmax layer with smoothed one-sided regression outputs.
Although their method can deal with an arbitrary cost matrix, it is unclear whether the proposed SVM-based cost-sensitive algorithm can be applied to adversarial settings.
More recently, Khan et al.
(2018) proposed different cost-sensitive surrogate losses to jointly optimize the network parameters and class-sensitive costs for class imbalance problem.
Speciﬁcally, they proved that the proposed cost-sensitive cross entropy (CE) loss satisﬁes desired properties for classiﬁcation, including classiﬁcation calibration (Bartlett et al., 2006) and guess aversion (Beijbom et al., 2014).
Although their method is developed for class imbalanced datasets, it can deal with general cost matrices and achieves improved misclassiﬁcation costs, as illustrated in Section B.3.
Inspired by Khan et al.
, we propose cost-sensitive robust training loss (3.2) by adapting the cost-sensitive CE loss to adversarial settings.
To be more speciﬁc, we modify the overall robustness training objective (2.7) by incorporating the cost information into the guaranteed lower bound, then feed the modiﬁed robust estimates into the cross-entropy loss.
Similar to the effect of cost-sensitive
CE loss for standard classiﬁcation, (3.2) induces a larger penalization for adversarial transformations with larger cost, thus achieving enhanced cost-sensitive robustness.
Replacing cross-entropy loss in (3.2) with other loss functions, such as MSE loss and hinge loss, should also be feasible for inducing cost-sensitive robustness and future research may consider these alternatives.
However, as cross-entropy loss is arguably the most effective and widely-used loss function for multiclass image classiﬁcation based on neural networks (Nielsen, 2015), we believe hardening cross-entropy based classiﬁer with cost-sensitivity will lead to the best performance.
D C HOICE OF C OST M ATRICES
Our goal in the experiments was to evaluate how well a variety of different types of cost matrices can be supported.
MNIST and CIFAR-10 are toy datasets, thus deﬁning cost matrices corresponding to meaningful security applications for these datasets is difﬁcult.
Instead, we selected representative tasks and designed cost matrices to capture them.
Our experimental results show the promise of the cost-sensitive training method works across a variety of different types of cost matrices, so we believe it can be generalized to other cost matrix scenarios that would be found in realistic applications.
We use the “hardness” of adversarial transformation between classes only for choosing representative cost matrices, and the robust error results on the overall-robustness trained model as a measure for transformation hardness.
The transformation hardness implied by the robust error heatmap is generally consistent with intuitions about the MNIST digit classes (e.g., 9 and 4 look similar, so are harder to induce robustness against adversarial transformation), as well as with the visualization results produced by dimension reduction techniques, such as t-SNE (Maaten & Hinton, 2008).
Nevertheless, there is no intrinsic correlation between the adversarial value (for example, in the toy check fraud motivation for MNIST) and transformation difﬁculty, since adversarial value is directional and semantically different digits can look more similar than far apart ones.
For an actual security application, it would be desirable for the system developers to deﬁne the classes in such a way that the valuable adversarial transformations are also the hardest ones to achieve, but whether or not this is possible depends on the application.