Target Training: Tricking Adversarial Attacks to Fail
Authors: Blerta Lindqvist
Venue:  (2021)
Keywords: adversarial machine learning

Abstract: Recent adversarial defense approaches have failed. Untargeted gradient-based attacks cause classifiers to choose any wrong class. Our novel white-box defense tricks untargeted attacks into becoming attacks targeted at designated target classes. From these target classes, we derive the real classes. The Target Training defense tricks the minimization at the core of untargeted, gradient-based adversarial attacks: minimize the sum of (1) perturbation and (2) classifier adversarial loss. Target Training changes the classifier minimally, and trains it with additional duplicated points (at 0 distance) labeled with designated classes. These differently-labeled duplicated samples minimize both terms (1) and (2) of the minimization, steering attack convergence to samples of designated classes, from which correct classification is derived. Importantly, Target Training eliminates the need to know the attack and the overhead of generating adversarial samples of attacks that minimize perturbations. Without using adversarial samples and against an adaptive attack aware of our defense, Target Training exceeds even default, unsecured classifier accuracy of 84.3% for CIFAR10 with 86.6% against DeepFool attack; and achieves 83.2% against CW-$L_2$ (κ=0) attack. Using adversarial samples, we achieve 75.6% against CW-$L_2$ (κ=40). Due to our deliberate choice of low-capacity classifiers, Target Training does not withstand $L_\infty$ adaptive attacks in CIFAR10 but withstands CW-$L_\infty$ (κ=0) in MNIST. Target Training presents a fundamental change in adversarial defense strategy.

Under review as a conference paper at ICLR 2021
TARGET TRAINING: TRICKING ADVERSARIAL ATTACKS TO FAIL
Anonymous authors
Paper under double-blind review
ABSTRACT
Recent adversarial defense approaches have failed. Untargeted gradient-based attacks cause classifiers to choose any wrong class. Our novel white-box de- fense tricks untargeted attacks into becoming attacks targeted at designated target classes. From these target classes, we derive the real classes. The Target Training defense tricks the minimization at the core of untargeted, gradient- based adversarial attacks: minimize the sum of (1) perturbation and (2) classifier adversarial loss. Target Training changes the classifier minimally, and trains it with additional duplicated points (at 0 distance) labeled with designated classes, These differenttl- duplicated samples minimize both terms (I) and (2) of the minimization, steering attack convergence to samples of designated classes, from which correct classification is derived. Importantly, Target Training elim- inates the need to know the attack and the overhead of generating adversarial samples of attacks that minimize perturbations. Without using adversarial sam- ples and against an adaptive attack aware of our defense, Target Training exceeds even default, unsecured classifier accuracy of 84.3% for CIFARIO with 86.6% against DeepFool attack; and achieves 83.2% against CW-L2(R=O) attack. Us- ing adversarial samples, we achieve 75.6% against CW-L2(x=40). Due to our deliberate choice of low-capacity classifiers, Target Training does not withstand Loe adaptive attacks in CIFARIO but withstands CW- (k=0) in MNIST. Target Training presents fundamental change in adversarial defense strategy.
INTRODUCTION
Neural network classifiers are vulnerable to malicious adversarial samples that appear indistinguish- able from original samples (Szegedy et al. 2013), for example, an adversarial attack can make a traffic stop sign appear like speed limit sign (Eykholt al. 2018) to classifier. An adversarial sample created using one classifier can also fool other classifiers (Szegedy et al. 2013 Biggio et al,, 2013). even ones with different structure and parameters (Szegedy al. 2013 Goodfellow al.
2014 Papernot et al., 2016b) Tramer et al. 7b). This transferability of adversarial attacks (Pa- pernot al. 2016b) matters because means that classifier access is not necessary for attacks. The increasing deployment of neural network classifiers in security and safety-critical domains such as traffic (Eykholt al. 2018), autonomous driving (Amodei al., 2016). healthcare (Faust et al., and malware detection (Cui et al. 2018) makes countering adversarial attacks important.
2018).
Gradient-based attacks use the classifier gradient to generate adversarial samples from non- adversarial samples. Gradient- based attacks minimize at the same time classifier adversarial loss and perturbation (Szegedy et al., 2013). though attacks can relax this minimization to allow for big- ger perturbations, for example in the Carlini& Wagner attack (CW) (Carlini & Wagner (017c) for
> 0, in the Projected Gradient Descent attack (PGD) (Kurakin et al. 2016: Madry et al. (017). in FastGradiintMethod (FGSM) (Goodfellow et al. 2014). Other gradient- include DeepFool (Moosavi-Dezfooe et al. 2016). Zeroth order optimization (ZOO) (Chen et al. 2017). Universal Adversarial Perturbation ( (UAP) (Moosavi-Dezfoolii et al. 2017).
adversarial attacks
Many recent proposed defenses have been broken (Athalye et al., 2018: Carlini & Wagner 2016
2017al bi Tramer al.) 2020). They fall largely into these categories: (1) adversarial sample de- tection. (2) gradient masking and obfuscation, (3) ensemble, (4) customized loss. Detection de-
fenses [Meng & Chen 2017 Ma et al. 2018 Li et al. 2019
Hu al. 2019] aim detect, cor-

---

Under review as a conference paper at ICLR 2021
rect reject adversarial samples. Many detection defenses have been broken (Carlini & Wagner
2017b at Tramer et al. 2020). Gradient obfuscation is aimed at preventing gradient-based attacks
from access to the grachent and can be achieved by shattering gradients (Guo et al. 2018 Verma & Swami 2019 Sen et al. 2020), randomness (Dhillon et al. 2018 Li et al. 2019 or vanishing or
exploding gradients (Papernot et al. 20T [Song et al.) 2018 Samangouei et al. 2018). Many gra-
dient obfuscation methods have also been successfully defeated (Carlini & Wagner [2016] Athalyc
al. 20188 Tramer et al., 2020). Ensemble defenses (Tramèr 2017, Verma Swami 2019
Pang et al. 2019 [Sen et al., 2020) have also been broken (Carlini & Wagner 2016] Tramer et al.
2020j. unable to even outperform their best performing component. Customized attack losses defeat
defenses (Tramer et al., 2020) with customized losses (Pang et al., 2020) Verma & Swami (219)) but also, for example ensembles (Sen et al., (020). Even though has not been defeated, Adversarial Training (Kurakin et al.) 2016 (Szegedy et al. 2013) Madry et al. (017) assumes that the attack is known in advance and takes time to generate ad adversarial samples at every iteration. The inability of recent defenses to counter adversarial attacks calls for new kinds of defensive approaches.
In this paper, we make the following major contributions:
We develop Target Training novel, white-box adversarial defense that converts untar- geted gradient-based attacks into attacks targeted at designated target classes, from which correct classes are derived. Target Training is based on the minimization the core of untargeted gradient-based adversarial attacks.
For all attacks that minimize perturbation, we eliminate the need know the attack or to generate adversarial samples during training.
We show that Target Training withstands non- adversarial attacks without resorting to increased network capacity. With default accuracy of 84.3% in CIFARIO, Target Training achieves 86.6% against the DeepFool attack, and 83.2% against the CW-Lz((t-O) attack without using adversarial samples and against adaptive attack aware of defense. Against an adaptive CW-I2(xx=40 attack, we achieve 75.6% while using adversarial sam- ples. Our choice of low-capacity classifiers makes Target Training withstand T. adap- tive attacks, except for ( in MNIST.
We conelude that Adversarial Training might not be defending by populating sparse areas with samples, but by minimizing the same minimization that Target Training minimizes.
2 BACKGROUND AND RELATED WORK
Here, we present the state-of-the-art in adversarial attacks and defenses, as well as a summary.
Notation A k-class neural network classifier that has 0 parameters is denoted by function f(x that takes input æ E IRd and outputs E IR* where is the dimensionality and is the number of classes. An adversarial sample is denoted by Zade. Classifier output y, y/ is the probability that
the input belongs to class i. Norms are denoted as Lg, L2 and Loo.
2.1 ADVERSARIAL ATTACKS
Szegedy et al. (2013) were first to formulate the generation of adversarial samples as a constrained minimization of the perturbation under an Lp norm. Because this formulation can be hard to solve, (Szegedy et al. (2013) reformulated the problem as gradient-based, two-term minimization of the weighted sum of perturbation, and classifier loss. For untargeted attacks, this minimization is:
minimize
c II - =2112 loss (Zadv) Fadu [0, 1]
(Minimization 1)
subject
where f is the classifier, loss, is classifier loss adversarial input, and constant value evaluated
in the optimization. Term (1) is a norm to ensure a small adversarial perturbation. Term (2) utilizes
the classifier gradient to find adversarial samples that minimize classifier adversarial loss.
Mmiimizttinn I is the foundation for many gradient- gradient-basedd attacks, though many tweaks can and have been applied. Some attacks follow Minimization 1 implicitly (Moosavi-Deztooli al. 2016
2

---

Under review as a conference paper at ICLR 2021
and others explicitly (Carlini & Wagner 2017c). The type of L, norm in term (1) of the mini- mization also varies. For example the CW attack (Carlini & Wagner 2017c) uses Lo, Lg and Los. whereas DeepFool (Moosovi-Deztooll et al. 2016) uses the L2 norm. A special perturbation case is the Pixel attack by et al. (201)))) which changes exactly one pixel. Some attacks even exclude term (1) from theiminnni and introduce external parameter control perturbation. The FGSM attack by Goodfellow et al. (2014), for example, uses an e parameter, while the CW attack (Carlini & Wagner 2017c) uses a confidence parameter.
The Fast Gradient Sign Method by Goodfellow et al. (2014) is a simple, Loe-bundeed attack that constructs adversarial samples by perturbing each imput dimension in the direction of the gradient by magnitude of e: Todv = x + 6 signn (V sign(Vyloss(( 3)).
The current strongest attack is CW (Carlini Wagner 2017c). CW customizes Minimization by passing c to the second term, and using it to tune the relative importance of the terms. With a further change of variable, CW obtains an unconstrained minimization problem that allows it to optimize directly through back-propagation. In addition, CW has a parameter for controlling the confidence of the adversarial samples. For x >> O and up to 100, the CW attack allows for more perturbation in the adversarial samples it generates.
The DeepFool attack by Moosavi-Deztooli et al. (2016) follows the Minimization implicitly. DeepFool Gibosavi-Dezfooll et al. (2016) looks at the smallest distance of point from the classifier decision boundary as the minimum amount of perturbation needed to change its classification. Deep- Fool approximates the classifier with a linear one, estimates the distance from the linear boundary, and then takes steps in the direction of the closest boundary until an adversarial sample is found.
Black-box attacks Black-box attacks assume no access to classifier gradients. Such attacks with access to output class probabilities are called score-based attacks, for example the 700 attack (Chen et al. 2017), black-hox variant of the CW attack (Carlini & Wagner) (017c). Attacks with access to only the final class label are recision-based attacks, for example the Boundary (Brendel et al. 2017) and the HopSkiplumpAttack (Chen al. 2019) attacks.
Multi-step attacks The PGD attack (Kurakin et al. 2016) is an iterative method with an a pa- rameter that determines a step-size perturbation magnitude. PGD starts at a random point Zu. projects the perturbation on an Lp-ball B at each iteration: x(j + 1) = Proja(x()) + a =z)), y)). The BIM attack (Kurakin et al. 2016) applies FGSM (Goodfellow al., iteratively with an a step. To find a universal perturbation, UAP (Moosavi-Dezfooli et al. iterates over the images and aggregates perturbations calculated as in DeepFool.
2014
2017
2.2
ADVERSARIAL DEFENSES
Adversarial Training. Adversarial Training (Szegedy et al.) 2013 Kurakin et al., 2016 Madry et al. 2017) is one of the first and few, undefeated defenses. It defends by populating low proba- bility, so-called blind spots (Szegedy et al. 2013) Goodfellow et al. 2014) with adversarial samples labelled correctly, redrawing boundaries. The drawback of Adversarial Training is that it needs to know the attack in advance, and it needs generate adversarial samples during training. The Adversarial Training algorithm 2 in the Appendix is based on Kurakin et al. (2016). Madry et al. (2117) formulate their defense as robust optimization problem, and use adversarial samples to augment the training. Their solution however necessitates high-capacity classifiers bigger models with more parameters.
Detection defenses Such defenses detect adversarial samples implicitly or explicitly. then correct or reject them. So far, many detection defenses have been defeated. For example, ten diverse de- tection methods (other network, PCA, statistical properties) were defeated by attack loss customiza- tion (Carlini& & Wagner) 2017a);; Tramerr et al. (2020) used attack customization against (Hu et al.) 2019 ; attack transfcrability (Carlini & Wagner 2017b) was used against MagNet by Meng & Chen 2017 ; deep feature adversaries (Sabour et al. 2016) against (Roth et al. 2019).
Gradient masking and obfuscation Many defenses that mask or obfuscate the classifier gradient have been defeated (Carlini & Wagner 2016 Athalyy et al., 2018). Athalyy et al. (2018) identify three types of gradient obfuscation: (1) Shattered gradients incorrect gradients caused by non- differentiable components or numerical instability, for example with multiple input transformations by Guo et al. (2018). Athalye et al. (018)) counter such defenses with Backward Pass Differentiable
3

---

Under review as a conference paper at ICLR 2021
Approximation. (2) Stochastic gradients in randomized defenses overeome with Expectation Over Transformation (Athalye et al.] 2017) by Athalye et al. Examples are Stochastic Activation Pruning (Dhillon et al. 2018), which drops layer neurons based on a weighted distribution, and (Xie et al. 2018) which adds a randomized layer to the classifier input. (3) Vanishing or exploding gra- dients are used, for example, in Defensive Distillation (DD) (Papernot et al., 2016a) which reduces the amplitude of gradients of the loss function. Other examples are PixelDefend (Song et al. 2018))
and Defense-GAN (Samangouei et al. 2018). Vanishing or exploding gradients are broken with parameters that avoid vanishing or exploding gradients (Carlini & Wagner 2016).
Complex defenses Defenses combining several approaches, for example (Li al. 2019) which uses detection, randomization. multiple models and losses, can be defeated by focusing on the main defense components (Tramer et al., 2020). 2020) are defeated ensemble defenses combined with numerical instability (Verma & Swami 2019), regularization (Pang al. 2019), or mixed precision on weights and activations (Sen et al. 2020).
2020).(Verma (Verma Swami] 2019; Pang et al.] 2019 Sen et al.)
2.3 SUMMARY
Many defenses have been broken. They focus on changing the classifier. Instead, our defense changes the classifier minimally, but forces attacks change convergence. Target Training is the first defense based on Minimization at the core of untargeted trudient-based adversarial attacks.
TARGET TRAINING
Target Training converts untargeted attacks to attacks targeted at designated classes, from which correct classification is derived. Untargeted gradient-based attacks based on Minimization (on page 2 of the sum of (1) perturbation and (2) classifier adversarial loss. Target Training trains the classifier with samples of designated classes that minimize both terms of the minimization at the same time. These samples are exactly what adversarial attacks look for, based on Minimization I nearby points (at 0 distance) that minimize adversarial loss. For attacks that relax the minimization by removing the perturbation term, we adjust Target Training to use adversarial samples against attacks that do not minimize perturbation.
Target Training eliminates the need to know the attack or to generate adversarial samples against attacks that minimize perturbation. Instead of adversarial samples, we use original samples labelled as designated classes because they have minimum O-distance from original samples. Following, we outline how Target Training minimizes both terms of Minimization siimutaaneously.
Term (1) of the minimization perturbation. Against attacks that do minimize perturbation, such as CW (N 0) and DeepFool, Target Training uses duplicates of original samples in each batch instead of adversarial samples because these samples minimize the perturbation to 0 no other points can have smaller distance. This eliminates the overhead of calculating adversarial samples against all attacks that minimize perturbation. Algorithm [ shows Target Training against attacks that minimize perturbations. Against attacks that do not minimize perturbation, such as CW (k >> 0), PGD and FGSM, Target Training replaces duplicated samples with adversarial samples from the attack. The adjusted algorithm is shown in Algorithm in the Appendix.
Algorithm Target Training of classifier N against attacks that minimize perturbation
Require: Batch size is m, number of dataset classes is k, untrained classifier N with 2 output classes, TRAIN trains classifier on a batch and its ground truth Ensure: Classifier N is Target- Trained against all attacks that minimize perturbation
while training not converged do
B- = [x] x:
D. Get random batch Get batch ground truth Duplicate batch
G yy
B' = xx
I x1 rerg I 1
G' = {y) 7dh y + k ..... nd
+ k} b Duplicate ground truth and increase duplicates by b Train classifier on duplicated batch and new ground truth
TRAIN(N, B', G¹)
end while
4

---

Under review as a conference paper at ICLR 2021
Training without adversarial samples
Training with adversarial samples
Inference
Labels
Labcls
Output vector y Probabilities
4
4
4
4
81
Original batch
3
3
Original batch
3
3
92
5
5
5
5
8/t
1/6
Target Training hatch
8
8
8
8
3/s
7
Classifier
19r 0.5
7
7
7
7
896
3/
Duplicated samples
4
14
samppu /
4
14
911
912
3
13
Adversaria Samss
3
13
Class 17 has the highest probability of all adversarial classes, Attacks converge samples of class minimize udversurial loss.
914
5
15
5
15
915
8
18
8
18
B17 ~ 0.5
918
7
17
7
17
9/9
Figure I: Target Training with and without adversarial samples, and output probabilities at inference. Example images are from the MNIST dataset, smaller batch size shown for brevity. Actual inference output probability values are shown in the Appendix in Table for MNIST, in Table for CIFARIO. 10.
Term of the minimization classifier adversarial loss. In a converged, multi-class classifier, the probabilities of adversarial classes are ~ O with no distinction among them (the real class has high probability). This causes untargeted attacks converge to samples of whichever high- est probability adversarial class, due to the minimization of adversarial loss in term (2). In order force attack convergence to designated classes, classifier would need train so that one of the adversarial classes has higher probability than the other adversarial classes. Which means that classifier would need have two high probability outputs: the real class, and the designated adversarial class.
Target Training. Against attacks that minimize perturbation, Target Training duplicates training samples in each batch and labels duplicates with designated classes. Against attacks that do minimize perturbation, adversarial samples are used instead of duplicates. As result, the real class and the designated class have equal inference probabilities ~ 0.5, as shown Figure [ Since attacks minimize adversarial loss in term (2) of Minimization 1] attacks converge to adversarial sam- ples from the designated class. The same samples also minimize term (1) because the classifier was trained with: samples with 0 distance, or adversarial samples close to original samples. It does not affect Target Training whether the c constant the Minimization Tis at term (I) or term (2) in an at- tack because both terms minimized. Target Training could be extended defend simultaneously against many attacks using a designated class for each type of attack. During training, adversarial samples would be labeled with the corresponding designated class. In this paper, do conduct experiments on simultaneous multi-attack defense.
Model structure and inference. The only change to classifier structure is doubling the number of output classes from & to 2k, loss function remains standard softmax cross entropy. Inference calculation is: C(x) arg max (y, + y1+x), [O... 1)].
EXPERIMENTS AND RESULTS
We assume that the adversary goal is generate adversarial samples that cause untargeted misclassification. We perform white-box evaluations, assuming the adversary has com- plete knowledge of the classifier and how the defense works. In terms of capabilities, assume
Threat model
5

---

Under review as a conference paper at ICLR 2021
that the adversary is gradient- basde has access the CIFARIO and MNIST image domains and is able to manipulate pixels. Against attacks that minimize perturbations, no adversarial samples are used in training and no further assumptions mize perturbations, we assume that the attack is of the same kind as the attack used to generate the adversarial samples used during training. Perturbations are assumed to be c--cosstraieee..
made about attacks. For attacks that do not mini-
Attack parameters For PGD, we use parameters based on the PGD paper Madry et al. 2017 7 steps of size with total 8 for CIFARIO, and 40 steps of size 0.01 with a total e 0.3 for MNIST. For all PGD attacks, we use random initialisations within the e ball, effectively starting PGD attacks from the original images. For CW, we use 1000 iterations by default but we also run experiments with up to 100000 iterations. Confidence values in CW range from 0 to 40. For the ZOO attack, we use parameters based on the ZOO attack paper (Chen al. 2017): binary search steps, 1000 iterations, initial constant selected images from MNIST and CIFARIO test sets, same as done in ZOO (Chen et al., 2017) for untargeted attacks. For FGSM, 0.3, as in (Madry et al., 2017).
0.01. Additionally, for the ZOO attack, we use 200 randomly
Classifier models We purposefully do not use high capacity models, such as ResNet (He et al, 2016), used for example by Madryy et (2017). The reason is to show that Target Training does not necessitate high model capacity to defend against adversarial attacks. The architectures of MNIST and CIFAR datasets are shown in Table 5 in the Appendix, and no data augmentation was used. Default accuracies without attack are 99.1% 1. for MNIST and 84.3% for CIFARIO.
Datasets The MNIST (LeCun et al., 1998) and the CIFARIO (Krizhevsky et al. 2009) datasets are 10-class datasets that have been used throughout previous work. The MNIST (LeCun et al. 1998) dataset has 60,, 28 x 28 x 1 hand-written, digit images. The CIFARIO (Krizhevsky et al. dataset has 70K, 32 x 32 x 3 images. All experimental evaluations are with testing samples.
2009
Tools Adversarial samples generated with CleverHans 3.0.1 (Papernot et al. 2018) for CW (Car-
lini & Wagner 2017c). DeepFool (Moosavi-Dezinn et al. 2016). FGSM (Goodfellow et al., 2014) attacks and IBM Adversarial Robustness 360 Toolbox (ART) toolbox 1.2 (Nicolac et al. 22118) for
other attacks. Target Training is written in Python 3.7.3, using Keras 2.2.4
(Chollet et al. 2015).
4.1 TARGET TRAINING AGAINST ATTACKS THAT MINIMIZE PERTURBATION
We compare Target Training with unsecured classifier as well Adversarial Training, since other defenses have been defeated (Carlini & Wagner 201 7b a 2016 Athalye et al.) 2018 Tramer et al., 2020) successfully. Table T shows that, while not using adversarial samples against adversarial attacks that minimize perturbation, Target Training exceeds by far accuracies by unsecured classifier and Adversarial Training in both CIFARIO and MNIST. In CIFARIO. Target Training even exceeds the accuracy of the unsecured classifier on non-adversarial samples (84.3%) for most attacks.
4.2 TARGET TRAINING AGAINST ATTACKS THAT DO NOT MINIMIZE PERTURBATION
Against adversarial attacks that do not minimize perturbation, Target Training uses ad adversarial sam- ples and performs slightly better than Adversarial Training. We choose Adversarial Training baseline as an undefeated adversarial defense, more details in Section 2 Our implementation of Adversarial Training is based on (Kurakin et al. 2016), shown in Algorithm In Table we show that Target Training performs slightly better than Adversarial Training against attacks that do not minimize perturbation.
In Table 6 in the Appendix, we show that both Target Training and Adversarial Training are able to defend in some cases against attacks, the adversarial samples of which have not been used during training- Target Training in more cases than Adversarial Training.
4.3 TARGET TRAINING PERFORMANCE ON ORIGINAL, NON-ADVERSANN SAMPLES
In Table 7 in the Appendix, show that Target Training exceeds default classifier accuracy original, aoversarrall images when trained without adversarial samples against attacks that min- imize perturbation: 86.7% (up from 84.3%) in 1IFAR10. Furthermore, Table 7 shows that when

---

Under review as a conference paper at ICLR 2021
Table I: Here, we show Target Training performance against attacks that minimize perturbation, for which Target Training does not use adversarial samples, Target Training exceeds performance of default, unsecured classifier. Target Training also exceeds the performance of classifier trained with Adversarial Training that uses adversarial samples. For several attacks, Target Training even exceeds the accuracy of default classifier on aov-adversarial samples in CIFARIO (84.3%).
CIFARIO (84.3%)
MNIST (99.1%)
Target Training
Advers. Training
Default Classifier
Target Training
Advers. Training
Default Classifier
Attack
CW-L22 K. 0, IK iter.
85.6%
22.8%
8.8%
96.3%
5.0%
0.8%
DeepFool
86.6%
24.0%
9.2%
94.9%
5.2%
1.3%
ZOO
89.0%
81.5%
93.0%
96.0%
UAP
86.8%
42.5%
17.2%
98.6%
58.4%
42.
CW-Loo, 0, 1K iter.
84.2%
21.4%
42.0%
96.3%
80.1%
82.1%
Based ZOO paper (Chen et al. (017)) 200 randomly selected test images used for the untargeted ZOO attack. Accuracies for Adversarial Training against ZOO attack are not calculated because the ZOO attack is slow in generating the adversarial samples needed by Adversarial Training.
Table 2:
Using adversarial samples in training, Target Training performs slightly better than Adversarial Training against attacks that do not minimize perturbation.
CIFAR1O (84.3%)
MNIST (99.1%)
Attack
Target Training
Adversarial Training
Unsecured Classifier
Target Training
Adversarial Training
Unsecured Classifier
CW-L2(K = 40)
77.7%
77.4%
8.5%
98.0%
98.0%
0.7%
PGD
76.3%
76.2%
32.7%
92.3%
91.7%
79.7%
FGSM(c = 0.3)
72.1%
71.8%
11.8%
98.0%
98.4%
10.0%
using adversarial samples against attacks that do not minimize perturbation, Target Training equals Adversarial Training performance on original, con-adveraarial images. This holds for both MNIST and CIFARIO.
4.4 SUMMARY OF RESULTS
experiments show that substantially improve performance against attacks that minimize perturbation without using adversarial samples, surpassing even default accuracy for CIFARIO. Section 4.2 experiments show that against attacks that do minimize perturbation, Target Training has slightly better performance than the current non-broken defense, Adversarial Training.
Our Section 4.1
4.5 TRANSFERABILITY ANALYSIS
For a defense to be strong, it needs to be shown to break the transferability of attacks. A good source of adversarial samples for transferability the unsecured classifier (Carlini et al.) 2019). We experiment on the transferability of attacks from the unsecured classifier to classifier secured with Target Training. In Table B] we show that Target Training breaks the transferability of adversarial samples generated by attacks that minimize perturbation. Against attacks that do not minimize perturbation, the transferability broken in MNIST, and partially in CIFAR10.
5 ADAPTIVE EVALUATION
Many recent defenses have failed anticipate attacks that have defeated them (Carlini al. 2019 Carlini & Wagner 201 7a) Athalye et al. 2018). To avoid that, we perform an adaptive evalua-
tion (Carlini et al. 2019 Tramer al. 2020) of our Target Training defense.

---

Under review as a conference paper at ICLR 2021
Table 3: Target Training breaks the transferability of attacks generated from unsecured classifier by maintaining high accuracy against attacks generated using the unsecured classifier in attacks that minimize perturbation for CIFARIO and MNIST. For attacks that do not minimize perturbation, Target Training breaks the transferability in MNIST, and partially in CIFARIO.
CIFARIO (84.3%)
MNIST (99.1%)
Target Training
Unsecured Classifier
Target Training
Unsecured Classifier
Attack
CW-Ly(N 0), IK iter.
69.9%
8.8%
78.3%
0.8%
(I = 0), IK iter.
76.6%
42.0%
93.5%
82.1
DeepFool
74.8%
9.2%
96.5%
1.3%
CW-L2(1 40), 1K iter. PGD
34.7%
8.5%
95.1%
0.7%
36.8%
32.7%
92.2%
79.7%
Whether Target Training could be defeated by methods used to break other defenses. Adap- tive attack approaches (Carlini & Wagner 2017b 2016 Athalye et al. 2018: Tramer et al.) 2020)
used to defeat most current defenses cannot break Target Training defense because we use none of the previous defense approaches: adversarial sample detection, preprocessing, obfuscation, ensem- ble, customized loss, subcomponent, non-differentiable component, or special model layers. We also keep the loss function simple standard softmax cross-entropy and no additional loss.
Adaptive attack against Target Training. We consider an adversarial attack that is aware that classifier uses Target Training. The attack adds a new layer at the top of the classifier, with 20 inputs and 10 outputs, where output is the of input i and i k. The classifier with the extra layer used to generate adversarial samples that are tested on the original classifier. We summarize the results of the adaptive attack in Table 4
Table Target Training performance against adaptive adversarial attack, for CIFARIO and MNIST.
Adversarial images are generated using the extra-layer classifier and tested against the original Target-Traine Trained classifier. Target Training withstands the adaptive attack when the norm is not Loo- When the norm is Lx., Target Training withstands the attack only for MNIST CW-L = 0).
CIFAR1O (84.3%)
MNIST (99.1%)
Target Training
Unsecured Classifier
Target Training
Unsecured Classifier
Attack
CW-L2(n 0), 1K iter.
83.2%
8.8%
95.7%
0.8%
DeepFool
86.6%
9.2%
94.9%
1.3%
CW-Tooo = 0), 1K iter.
20.8%
42.0%
95.7%
82.1
CW-L2(K 40), IK iter. PGD
75.6%
8.5%
93.9%
0.7%
7.1%
32.7%
57.9%
79.7%
Table 4 shows that Target Training withstands the adaptive attack when the norm of the attack is not Lo even surpassing default classifier accuracy on non-adversarial images for DeepFool (L2) in CIFAR10. For based attacks, PGD and CW-L (k = 0), Target Training performace decreases. Withstanding L attacks has previously been shown by Madry et al. (2017) to require high-capacity architecture in classifiers. We suspect that the low capacity of our classifiers causes Target Training to not defend the classifiers against Ta attacks. The capacity of our classifiers was deliberately chosen to be low in order to investigate whether Target Training can defend low- low-cipacity classifiers.
How does Target Training withstand this adaptive attack when the norm not L The addition of the extra layer to the Trriet-Trained classifier effectively converts the classifier into an Adversarial- Trained classifier: the classifier was originally trained with additional nearby samples that are now labelled correctly. As result, the adversarial samples generated from the extra-layer classifier non-adversarial, as in Adversarial Training, and classify correctly in the original classifier. There- fore, what makes Adversarial Training work, also makes Target Training work in this adaptive attack.
8

---

Under review as a conference paper at ICLR 2021
Other adaptive aftack against Target Training. Most current untargeted attacks, including the strongest one, CW, are based on Mimiimmzzation I] But this minimization a simplification of the hard problem of generating adversarial attacks, outlined by [Szegeyy al. (2013). In order to defeat Target Training, attacks would need to solve the much harder problem of generating adversarial attacks without using gradients. This remains open problem.
Iterative attacks. The multi-step PGD (Kurakin et al., 2016) attack decreases Target Training ac- euracy more than single-step attacks, which suggests that our defense is working correctly, according to Carliii et al. (2019).
Transferability. We conduct transferability attacks to show that Target Training can withstand at- tacks that use another classifier to generate adversarial samples. 3 that Target Training breaks the transferability of attacks generated using an unsecured classifier, as defenses should ac- cording to Carlini et al. 2019).
Stronger CW attack leads to better Target Training accuracy. Many attacks fail to defend from attacks when attacks become stronger. Increasing iterations for CW-L2 0) 100-fold from 1K to 100 increases our defense's 's accuracy. In CIFARIO, the accuracy increases from 85.6% to 86. 26.2%. in MNIST from 96.3% to 96.6%. Target Training accuracy increases because higher number of iterations means that the attack converges better samples of the designated classes, leading to higher accuracy.
No gradient masking or obfuscation. The fact that Target Training defends against black-box 700 attack is also an indicator that Target Training does not gradient masking or obfuscation according to Carlinn et al. (2019).
6 DISCUSSION AND CONCLUSIONS
In conclusion, Target Training enables low-capacity classifiers defend against non-Lco attacks, in the case of attacks that minimize perturbation even without using adversarial samples. Target Training achieves this by replacing adversarial samples with original samples, which minimize the adversarial perturbation better. This eliminates the need to know the attack in advance, and the overhead of adversarial samples, for all attacks that minimize perturbation. ous non-broken Adversarial Training defense needs to know the attack and to generate adversarial samples of the attack during training. In addition, Target Training minimizes adversarial loss using designated classes. Our experiments show that Target Training even exceed default accuracy on non-adversarial samples in CIFARIO, when against non-L adaptive attacks that are aware of Target Training defense. We attribute T. adaptive attack performance low classifier capacity. In CIFAR10, 10, Target Training achieves 86.6% against the adaptive DeepFool attack without using adversarial samples, exceeding default accuracy of 84.3%. Against the CW-L2(x=0) adaptive attack and without using adversarial samples, Target Training achieves 83.2%. Against adaptive CW- L2(x=40) attack, we achieve 75.6% while using adversarial samples.
contrast, the previ-
Target Training resilience to non-L adaptive attacks can offer a different explanation for how Adversarial Training defends classifiers. The commonly-accepted explanation that Adversarial Training defends by populating low-density areas with adversarial samples labeled correctly. How- ever, the adaptive attack in Section 5 fails for CW-Lz(K 0) and DeepFool attacks when using what has become an Adversarial- Trained classifier trained with only original samples instead of clas-
adversarial samples. The failure the adaptive attack when using this Adversarial-Traine sifier trained without adversarial samples raises question. Might it be that Adversarial Training defends not by filling out the space with more adversarial samples labeled correctly, but in the same way that Target Training does: by minimizing the terms of Minimization IP If the answer yes, would also explain the similarity of the results of Target Training and Adversarial Training against attacks that do not minimize perturbation.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul F. Christiano, John Schulman, and Dan Mané. Concrete problems AI safety. CoRR, abs/1606.06565 2016. URL lttt:://axxii
abs/1606. 06565
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. arXiv preprint arXiv:1707.07397 2017.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: 180.00180,.00420, 2018.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smndić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv:1712.04225: 2017
2017.
Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples. 2016.
arXiv preprint arXiv:1607.066
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14. ACM, 2017a.
Nicholas Carlini and David Wagner. Magnet and" efficient defenses against adversarial attacks" are not robust to adversarial examples. arXiv preprint 1eXiiv1711.0844 2017b.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium Security and Privacy (SP), pp. 39-57. IEBE, 2017c
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint 1902. 000000 2019.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjumpattack: query-efficient decision-based attack. arXiv preprint
3, 2019.
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop Artificial Intelligence and Security, pp. 15-26. ACM, 2017.
François Chollet et al. Keras. https://keras. 10. 2015.
Zhihua Cui, Fei Xue, Xingjuan Cai, Yang Cao, Gai-ge Wang, and Jinjun Chen. Detection of mali- cious code variants based on deep learning. IEEE Transactions on Industrial Informatics, 14(7): 3187-3196, 2018.
Guneet S Dhillon, Kamyar Arizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial de- fense. In International Conference on Learning Representations, 2018.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625-1634, 2018.
Oliver Faust, Yuki Hagiwara, Tan Jen Hong, Oh Shu Lih, and U Rajendra Acharya. Deep learning for healthcare applications based on physiological signals: a review. Computer methods and programs in biomedicine, 2018.
10

---

Under review as a conference paper at ICLR 2021
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using imput transformations. In International Conference on Learning Representations, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Shengyuan Hu, Tao Yu, Chuan Guo, Wei-Lun Chao, and Kilian Weinberger. A new defense against adversarial images: Turning a weakness into a strength. In Advances in Neural Informa- fion Processing Systems, pp. 1633-1644, 2019.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 and cifar-100 datasets. URI: https://www. cs. toronto. edu/kriz/cffar. html, 6, 2009.
Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236 2016.
Yann LeCun, Corinna Cortes, and Christopher JC Burges. The mnist database of handwritten digits, 1998. URL URL http://yann. lecun. com/exdb/nmist, 10:34, 1998.
Yingzhen Li, John Bradshaw, and Yash Sharma. Are generative classifiers more robust to adversarial attacks? In International Conference on Machine Learning. 2019.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In International Conference on Machine Learning, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: arXiv:1766006083, 2017.
Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 135-147. ACM, 2017.
Seyed-Mohsen Moosavi-Dezfooli: Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pp. 2574-2582, 2016.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1765-1773, 2017.
Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wis- tuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, and Ben Edwards. Adversarial robustness toolbox v1.0.1. CoRR, 1807.01069, 2018. URL pitpss//aarxn
01069
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Learning Representations, 2019.
Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross- entropy loss for adversarial robustness. In International Conference Learning Representattinss 2020.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEEF, 2016a.

---

Under review as a conference paper at ICLR 2021
Nicolas Papernot, Patrick D. McDaniel, and lan J. Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. CoRR, abs/1605.07277, 2016b URL http:://axiyiii org/ 2tos/1605.0000 07277
Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Ku- rakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang. Yi-Lin Juang. Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. Technical report on the cleverhans v2.1.0 adversarial examples library. arXiv preprint arXiv: 1610000088 2018.
Kevin Roth, Yannic Kilcher, and Thomas Hofmann. The odds are odd: A statistical test for detecting adversarial examples. arXiv preprint
2019.
Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J Fleet. Adversarial manipulation of deep representations. In International Conference Learning Representations, 2016.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan, Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Represen- tutions, 2018
Sanchari Sen, Balaraman Ravindran, and Anand Raghunathan. Empir: Ensembles of mixed preci- sion deep networks for increased robustness against adversarial attacks. In International Confer- ence on Machine Learning, 2020.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In Inter- national Conference on Learning Representations, 2018.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828 841, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2013.
Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint 2017a.
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv:1704.03453, 2017b.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002.08.. 083447 2020.
Gunjan Verma and Ananthram Swami. Error correcting output codes improve probability estimation and adversarial robustness of deep neural networks. In Advances in Neural Information Process- ing Systems, pp. 8643 8653, 2019.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference Learning Representations, 2018.
A APPENDIX
12

---

Under review as a conference paper at ICLR 2021
Algorithm 2 Adversarial Training of classifier N
Require: Batch size is m, number of dataset classes is k, untrained classifier with l output
classes, ADV_ATTACK is adversarial attack, TRAIN trains classifier on a batch and its ground truth
Ensure: Adversarially- -
Trained classifier N
while training not converged do
B {xr ..... 20
Get random batch Get batch ground truth Generate adv. samples from batch > New batch Duplicate ground truth Train classifier new batch and new ground truth
G ,.... "y",...,""
y
A = ADV AWVAATTACKI B)
B' = BUA= {x'
B'=BUAA=
xd adu F adv 8
G' = ty
TRAIN(N, B', G')
end while
Table 5: Architectures of Target Training classifiers for CIFARIO and MNIST datasets. For the convolutional layers, we use L2 kernel regularizer. Notice that the final Sofstmxx layers in both models have 20 output classes, twice the number of dataset classes. The default, unsecured classifiers have the same architectures, except the final layers have 10 output classes: 10.
CIFAR10
MNIST
Conv.EL 3x3x32
Conv.ReLU 3x3x32
BatchNorm
BatchNorm
Conv.ELU 3x3x32
Conv.ReLU 3x3x64
BatchNorm
BatchNorm
MaxPool 2x2
MaxPool 2x2
Dropout 0.2
Dropout 0.25
Conv.ELU 3x3x64
Dense 128
BatchNorm
Dropout 0.5
ELL 3x3x64
Dense Dense.Softmas
Softmax 20
BatchNorm
MaxPool 2x2
Dropout 0.3
Conv.ELU 3x3x128
BatchNorm
CLnv.ELU 3x3x128
BatchNorm
MaxPool 2x2
Dropout 0.4
Dense.Softmal 20
Algorithm 3 Target Training of classifter N using adversarial samples against attacks that do not minimize perturbation
Require: Batch size is m, number of dataset classes is k, untrained classifier N with 2k output classes, ADV ATTACK is an adversarial attack, TRAIN trains classifier on a batch and its ground truth
Ensure:
Classifier is Target- Trained against ADV ALTAAK
while training not converged do
B = {x² t
D> Get random batch Get batch ground truth > Generate adv. samples from batch > Assemble new batch from original batch and
G {y' ..... ym
A = ADV ACK, B)
B' BUA- {x
Th I
Pddy x adv
adversarial samples
G' (v), ..... ....,!" .y + k.,.., t + k} & Duplicate ground truth and increase duplicates by Train classifier new batch and new ground truth
TRAIN(N, B', G¹)
end while
13

---

Under review as a conference paper at ICLR 2021
Table 6: Expanded comparison of performance against attacks that not minimize perturbation shows that both Target Training and Adversarial Training can defend against attacks, the adversarial samples of which have not been used in training. Bold font has been used to highlight where such defense was effective. Both Target Training and Adversarial Training defend against some attacks that they have not been trained for, but not all. Unsecured classifier performance also provided as a performance baseline. Target Training appears be slightly better at defending against attacks, the samples of which have not been used in training.
CIFARIO (84.3%)
MNIST (99.1%)
Adv. samples in training
Adv. samples in testing
Target Train.
Advers. Train.
No
Target Train.
Adv.
No
Def.
Train.
Def.
CW-L2 (K- 40)
77.7 %
77.4%
8.5%
98.0%
98.0%
0.7%
CW-L2 (x 0)
71.3 %
12.3%
8.8%
97.4%
1.5%
8.8%
CW-T-2
DeepFool
75.8 %
13.2%
9.2%
97.6%
1.6%
1.3%
(conf=40)
PGD
10.0%
10.0%
32.7%
23.3%
2.9%
79.7%
FGSM(E 0.3)
10.6%
9.9%
11.8%
56.6% %
15.8%
10.0%
FGSM(E = 0.01)
48.9%
36.4%
40.4%
97.7%
97.8% %
98.6%
PGD
76.3 %
76.2% %
32.7%
92.3%
91.7% %
79.7%
CW-L2 (k 40)
7.3%
57.3%
8.5%
83.2%
98.4%
0.7%
PGD
CW-L2 (K 0)
12.8%
12.7%
8.8%
94.3%
22.7%
8.8%
DeepFool
15.0%
13.0%
9.2%
86.5%
4.7%
1.3%
FGSM(E 0.3)
10.7%
10.2%
11.8%
79.9%
95.4%
10.0%
FGSM(E = 0.01)
39.8%
41.5%
40.4%
98.2%
98.4% %
98.6%
FGSM(E = 0.3)
72.1 %
71.8% %
11.8%
98.0%
98.4%
10.0%
FGSM(c 0.01)
40.8%
42.1%
40.4%
98.5%
98.5
98.6%
FGSM
CW-L2 (K 40)
49.9%
74.2%
8.5%
58.8%
1.1%
0.7%
(6= 0.3)
CW-L2 (K = 0)
12.5%
12.7%
8.8%
51.8%
1.1%
8.8%
DeepFool
12.7%
12.8%
9.2%
48.3%
1.2%
1.3%
PGD
17.2%
1.2%
32.7%
72.6%
42.5 %
79.7%
Table 7: Here, we show Target Training performance on original, non-adversarial samples. When Target Training is not using adversarial samples against attacks with no perturbation, Target Training even exceeds unsecured classifier accuracy on non-adversarial samples (Adversarial Training is not applicable here because needs adversarial samples). When Target Training is using adversarial samples against attacks with perturbation, Target Training equals Adversarial Training performance.
CIFARIO (84.3%)
MNIST (99.1%)
Attack used for adv. samples in training
Target Training
Adversarial Training
No
Target Training
Adversarial Training
No
Defense
Defense
none (against min. perturb. attacks)
86.7%
NA
84.3%
98.6%
NA 99.1 %
CW-Lz (k 40)
77.7%
77.4%
84.3%
98.0%
98.0%
99.1 %
PGD
76.3%
76.9%
84.3%
98.3%
98.4%
99.1% %
FGSM(C 0.3)
77.6%
76.6%
84.3%
98.6%
98.6%
99.1% %
14

---

Under review as a conference paper at ICLR 2021
Table 8: Class output probabilities for Target Training on original, and adversarial samples from MNIST. Adversarial samples generated with CW-L2(1 0). Zero probability values and probabil-
ity values rounded to zero have been omitted.
Original images
O
2
3
4
S
8
9
Labels
0.51
0.44
0.62
0.78
0.75
0.62
0.65
0.61
0.52
9
0.43
0.49
11
0.57
12
0.38
13
0.22
14
0.25
15
0.38
16
0.35
17
0.39
18
0.48
0.57
Adversarial images
O
/
2
3
4
S
8
q
0.50
0.50
0.49
0.49
0.50
0.50
6
0.50
0.46
8
0.50
0.50
10
0.50
0.50
12
0.51
13
0.51
14
0.50
15
0.50
16
0.50
17
0.54
18
0.50
19
0.50
15

---

Under review as a conference paper at ICLR 2021
Table 9: Class output probabilities for Target Training on original, and adversarial samples from
CIFAR1O. Adversarial samples generated with (2- = 0). Zero probability values and proba- bility values rounded to zero have been omitted. The two highest class probabilities for each image are made bold. The deer (fifth image) appears to be misclassified as a horse.
Original images
Labels
air- plane
auto- mobile
bird
deer
dog
frog
horse
ship
truck
0
0.41
0.46
2
0.01
0.56
0.60
4
0,08
0.01
0.48
6
0.53
0.39
0.56
8
0.54
9
0.47
10
0.58
11
0.55
12
0.01
0.43
13
0.40
0.01
0.06
15
0.01
0.52
0.47
17
0.45
0.44
0.46
19
0.53
Adversarial images
Labels
air- plane 0.49
auto- mobile
bird
cat
deer
dog
frog
horse
ship
truck
0.01
0.54
0.01
0.49
0.47
0.08
0.01
0.51
0.48
0.44
0.49
0.46
0.50
10
0.49
0.46
12
0.01
0.50
0.53
0.01
14
0.05
0.01
0.49
16
0.52
17
0.41
0.51
18
0.54
19
0.50
16


Reviews:
[{"review_id": "g-Lt2EwNRtB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses the task of adversarial defense, particularly against untargeted attack. It starts from the observation that these attacks mostly minimize the perturbation and the classification loss, and proposes a new training strategy named Target Training. The method duplicate training examples with a special ground-truth label, to fool the adversarial attackers. Experiments are conducted on MNIST and CIFAR10 under several attacks.\n\n\\+ The proposed method is simple, and involves a little modification to the classifiers and small computation overheads.\n\n\\- In the bottom of page 5, authors claimed that \"adversary has complete knowledge of how the defense works\". Could the authors please elaborate that, and how are the attacks modified to handle the proposed defense? If the untargeted attack is conducted on a 20-way classifier (for MNIST) without modification, I am not sure this evaluation is valid. One can add an extra layer after the last layer of the classifier, mapping $(y_0,\\cdots, y_{2k-2})$ to $(y_0+y_k,\\cdots,y_{k-1}+y_{2k-1})$, and then perform attacks -- since this is how the classifier actually works.\n\n\\- In Equation Minimization 1, there is a tern $l$ which is the adversarial label. I am not sure it is appropriate here since we are doing untargeted attacks.\n\n\\- From Table 2, the proposed method seems to achieve almost the same performance compared with vanilla adversarial training.\n\n\\- In Section 4.3, authors say Target Training maintains performance on clean images, when trained with adversarial samples. This might be inaccurate -- we can say Target Training achieves same performance on clean images compared with vanilla adversarial training.\n\n---- Post rebuttal ---\nI appreciate the responses from the authors, which partially address the concerns I was having. However, I am still not fully convinced that the proposed method is significant enough. Thus I am increasing my rating from 4 to 5.", "extra_scores": {}, "extra_text": {}}, {"review_id": "TNukuwPmX33", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed an ad hoc defense mechanism against white-box attacks, by duplicating the training data with original samples or adversarial samples and the number of prediction classes. The authors claim that this method achieves better results compare to baseline methods.\n\nI admit that this method could potentially defend against gradient-based attacks like CW and PGD if the attacker have no knowledge of the defense mechanism. However, since this method is defending the white-box  threat model, I believe a simple attack could break it:\n\nTo generate adversarial sample for test data $x$ whose correct label is $y$ or $y+k$, run PGD algorithm with the objective function $\\hat{x}= argmax_{x} l(f(x),y)+l(f(x),y+k)$, where $\\hat{x}$ is the adversarial example, $k$ is the number of classes (before duplication) and $f$ is the network trained by target training. Namely, this is just maximize the loss for both the real class $y$ and the duplicated class $y+k$.\nI'm not sure why the authors did not include any adaptive attacks like this one in section 5.\n\nOther flaws:\n\n-The writing is confusing, a lot of details are omitted. For example, what is perturbation size for CIFAR10 under PGD attack?\n\n-In section 4.1, the authors say \"Thus, the undefeated Adversarial Training defense cannot be used as a baseline because\nit uses adversarial samples during training for all types of attack.\" I don't buy it. The choice of baseline method should be based on the threat model, not the algorithm or training data used.\n\n-The authors say \"Adversarial training assumes that the attack is known...\" I believe this is not true.\n\n-Black-box attack ZOO shows only 81.5% accuracy on unsecured classifier, which basically means this is not an effective black-box attack. How could the authors use an ineffective attack to demonstrate the effectiveness of their defense method? \n\nIn summary, given the execution of the experiment, I'm not convinced this is an effective defense against white-box attacks.", "extra_scores": {}, "extra_text": {}}, {"review_id": "KqOKh1MHzym", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposes target training to defend against adversarial attacks on machine learning models. Target training doubles the number of output classes, and aims to trick untargeted attacks into attacks that target at designated classes. Experimental results show that targeting training can achieve slightly better performance than adversarial training.\n\nPros:\n1. Target training applies simple changes to the structure of machine learning models, to defend against adversarial attacks.\n2. The idea of tricking adversarial attacks is novel.\n3. Target training can partially break transferability of adversarial samples.\n\nCons:\n1. What is the overhead of target training?\n2. Would target training degrade the performance on clean data?\n3. More discussion should be included on whether target training is effective against adaptive attacks.\n\n\nDetailed comments:\nWhile the idea of applying target training to defend against adversarial attacks is interesting, I have the following questions regarding the proposed method (performance, limitation, etc.).\n\n1. Target Training aims to convert “untargeted attacks to attacks targeted at designated classes”, but doesn’t Minimization 1 in Section 2.1 correspond to targeted attacks rather than untargeted attacks (l is the target label)?\n\n2. What is the overhead of target training, especially Algorithm 3? How does that compare with normal training and adversarial training?\n\n3. Adversarial training degrades the model’s performance on clean data. Does target training have the same limitation?\n\n4. Would Algorithm 1 (Algorithm 3) also be working against attacks that do not (do) minimize perturbations? How to choose between these two algorithms?\n\n5. Not effective against adaptive attacks is one of the main limitations of many existing defence method. It is unclear whether target training has the same limitation. Not using techniques that have been broken does not mean that target training is robust.", "extra_scores": {}, "extra_text": {}}, {"review_id": "KFotNUuNkCQ", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposed  a simple trick (doubling the number of output class) to foul the minimization, gradient-based adversarial attacks. The method is simple, powerful and requires nearly no changes to the current training infrastructure.  This paper provides a new direction how to design defense strategy.\n\nPros:\n(1) This idea is interesting and even requires no changes during the training process. Also, the author provides enough intuitions why they design this based on the current model of adversarial attack.\n(2) The experiment results are impressive and even exceeds unsecured classifier accuracy on non-adversarial samples.\n\nCons:\n(1) The paper is not well written especially for the intro part (no more information is provided compared to their abstract).  Also, some formula need more explanations to help better understanding, e.g., loss_f in the minimization 1. \n(2) The defense strategy is based on the minimization model of attacks which may restrict its generality.\n\nSome typos: \n\n(1) In the last paragraph of page 2, ...many gradient-based attacksSome.... This should be a typo.\n(2) What's the difference between the two optimization problems in page 2 and 3 ?  They are the same. Also, please label them for  \nbetter reference.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I thank the authors and reviewers for the lively discussions. Reviewers found the work to be interesting but some concerns were raised regarding the significance of the results. In particular, two reviewers mentioned that authors did not fully address their concerns in the rebuttal period. Given all, I think the paper still needs a bit of work before being accepted. I recommend authors to address comments raised by the reviewers to improve their work.

-AC 
Decision: Reject