LabelFool: A Trick in the Label Space
Authors: Yujia Liu
Tingting Jiang
Ming Jiang
Venue:  (2020)
Keywords: Adversarial attack
LabelFool
Imperceptibility
Label space

Abstract: It is widely known that well-designed perturbations can cause state-of-the-art machine learning classifiers to mis-label an image, with sufficiently small perturbations that are imperceptible to the human eyes. However, by detecting the inconsistency between the image and wrong label, the human observer would be alerted of the attack. In this paper, we aim to design attacks that not only make classifiers generate wrong labels, but also make the wrong labels imperceptible to human observers. To achieve this, we propose an algorithm called LabelFool which identifies a target label similar to the ground truth label and finds a perturbation of the image for this target label. We first find the target label for an input image by a probability model, then move the input in the feature space towards the target label. Subjective studies on ImageNet show that in the label space, our attack is much less recognizable by human observers, while objective experimental results on ImageNet show that we maintain similar performance in the image space as well as attack rates to state-of-the-art attack algorithms.

Under review as a conference paper at ICLR 2020
LABELFOOL:
A TRICK IN THE LABEL SPACE
Anonymous authors Paper under double-bling review
ABSTRACT
It is widely known that well-designed perturbations can cause state-of-the-art ma- chine learning classifiers mis-label an image, with sufficiently small perturba- tions that are imperceptible to the human eyes. However, by detecting the incon- sistency between the image and wrong label, the human observer would be alerted of the attack. In this paper, we aim design attacks that not only make classifiers generate wrong labels, but also make the wrong labels imperceptible observers. To achieve this, we propose an algorithm called LabelFool which iden- tifies a target label similar to ground truth label and finds a perturbation the image for this target label. We first find the target label for an input image by a probability model, then move the input in the feature space towards the target label. Subjective studies ImageNet show that in the label space, our attack is much less recognizable by human observers, while objective experimental results ImageNet show that maintain similar performance in the image space well as attack rates to state-of-the- attack algorithms.
human
INTRODUCTION
Deep neural networks are powerful learning models that achieve state-of-the-an pattern recogni- tion performance in classification tasks (Krizhevsky al. 2012b] LeCun et al.) 2010 He et al.)
2016). Nevertheless, it found that adding well-designed perturbations to original samples can make classifiers of deep neural networks fail (Szegedy et al. (013). These kinds of samples called adversarial samples. Techniques for generating adversarial samples are called attackers.
We think the ideal attacker should satisfy three levels of requirements. The first requirement is fooling networks which means making classifiers fail to classify an image successfully. For example, dog image be classified number of methods for achieving a high attack rate (Goodfellow et al. 2015 Carlini & Wagner, 2017
after added some well-designed perturbations. There are
Dong al. 2018).
atockert
large perturbations
attack fails
imperceptible May
apparent misdassiication
attack falls
clasifiar
imperceptible
monastary
impercaptible
successfully
motivation
tonminal
terminal Babel space)
limage space)
This graph illustrates the importance of the imperceptibility of adversarial samples in both image space and label space. Triangles are three attackers. Circles represent human observers.
Figure 1:
The second requirement for the ideal attacker is the imperceptibility in the image space. This means the magnitude of perturbations in the pixel level needs to be as tiny as possible so that it is imper- ceptible human eyes. For example, additive perturbations minimized with 1, norm generate

---

Under review as a conference paper at ICLR 2020
imperceptible adversarial samples SeyeddMMosser et al. 2016). only changing one or a few pixels (Su et al., 2019; Modas et al. 2019) can make classifiers fail. Moosavi-Deztooli et al. (2017) even show the existence of universal (image-agnostic) perturbations.
Extreme cases also exist where
The third requirement for the ideal attacker, which is newly proposed in this paper, is the imper- ceptibiliity of the error made by the classifier in the label space. It means making the classifier to mis-classify an image as the label which is similar to its ground truth, so that people won't 't notice the misclassification. For example, in Figure human user will probably ignore the mis-classification if an attacker caused a "church" to be mis-classified as a "monastery" as the third attacker does. However, human user will easily notice the mistake if an attacker caused a "church" to be mis- classified as a "dome" as the second attacker does or caused an apparent perturbation in the image space as the first attacker does. real applications, a human user will take defensive measures as soon as he notices the attack. Therefore making the whole attack process imperceptible is crucial for letting observers' guard down. Tiny perturbations in the image space but large perturbations in the label space can muddle through on the input terminal. But as soon as observers check on the output terminal and see the obviously-incorrec label for an input, they will realize that the classifier fail due to some attacks and take defensive measures immediately, just as Figure [ shows. This justifies the power of attacks which also confuse people in the label space. So the imperceptibility in the label space is quite important. However, to our best knowledge, few attackers have realized this point.
In this paper, we propose an untargeted-attack algorithm called LabelFool, to perturb image be mis-classified as the label which is similar to its ground truth, so that people won't notice the mis- classification. In the meantime, LabelFool also guarantees the imperceptibility in the image space as well as maintaining a high attack rate in fooling classifiers. There are two steps by which we accom- plish our goal. The first step is to choose a target label which is similar to the input image's ground truth. The second step is to perturb the input to be classified as this target label. The way is finding the classification boundary between the current label and the target label, and then moving the input towards this boundary until it is classified as the target label. We conduct subjective experiment on ImageNet (Deng et al., (209)) which shows that adversarial samples generated by our method are indeed much less recognizable in the label space by human observers than other attacks. We also perform objective experiments on ImageNet to demonstrate that adversarial samples generated by LabelFool still guarantee the imperceptibility in the image space well as maintaining a high attack rate in fooling classifiers.
2 RELATED WORK
The phenomenon that neural networks are sensitive to adversarial samples was proposed by SZegedy et al. (013)). Since then, many researchers have studied how to generat adversarial samples. FGSM (Goodfello et al., 2015) was proposed to maximize the classification error subject to norm based distortion constraints. CW attack (Carlini & Wagner 2017) generates adversarial samples by solving an optimization problem based on lo/12/1 constraint, and to CW attack is the first proposed method that can cause targeted misclassification on the ImageNet dataset, meaning that we can specify the label of adversarial samples. But this designation of the target label is arbitrary. Until this paper, there has no guide about how to choose a target label such that it is difficult for person to notice that the network has failed.
Besides achieving the goal of misclassification, many researchers realize the importance of imper- ceptibility in the image space (Xu et al. 2019). One-pixel attack (Su et al. 2019) and SparseFool (Modas et al.) 2019) attack networks in a scenario where perturbing only one/a few pixels can make a big difference. MMoosavi-eezfool. et al. (2017) show the existence of universal image-agnostic per- turbations for state-of-the-ar deep neural networks. DeepFool Meyee-MMhsen et al., (016)) seeks the minimum image-level distortion. And for generating adversarial samples, it directly moves the input sample to the nearest class in the feature space. This is the most closely related work to ours, because features extracted from classification models can reflect images' perceptual informa- tion and the classes which are close in the feature space are often perceptually similar. However, DeepFool approximates the multi-dimensional classification boundaries in two dimensions and this might make big errors on finding the nearest class. All these attacks generate adversarial samples by iteration and the algorithm stops soon as adversarial sample is born no matter what label it
2

---

Under review as a conference paper at ICLR 2020
belongs to. This will lead to apparent misclassification that observers will sound the defensive alarm quickly.
In this paper, we will compare our method with three attacks: FGSM, DeepFool and SparseFool to show the advantage of our method in the imperceptibility in the label space. We will also demon- strate that the performance gain in the label space not at the expense of the loss in the image space or attack rate.
LABELFOOL
In this section, we will introduce our method about how to choose a target label which is undetectable by human observers and how we can perturb the input image so that the classifier assigns this specific label. The whole pipeline is shown in Figure 2 All the symbols and notations used in this paper are summarized in Table [ We use the same notation i(i 1, 2....... for "class" and "label", because "class" and "label" are interchangeable in this paper. LabelFool contains two steps. The first step is to choose a target label for the input image which is similar to its ground truth. The second step is to perturb the imput image to be classified as this label. Inspired by DeepFool (Seyed-Mohas 2016), we make modifications at the feature level. We keep moving the input towards this chosen class at the feature level until it is classified the label we want.
al.,
3.1 CHOOSE A TARGET LABEL
The first step of our method is choosing the target label fo for an input image z. As we want the target label to be imperceptible in the label space to human observers, we need to find the most "similar" label to the input image's ground truth L, where the most "similar" means the nearest in the perceptual distance metric. However, lg is usually unknown when an input image is given. So important to estimate the probability distribution P of an input's ground truth lg based which, we can compute the distance between each class in the dataset and l,, then choose the nearest one as the target class. We propose weighted distance model to achieve this goal. Before introducing the model, there are some preparations.
Given two image 2, y, we choose pre-traiiee pre-trained image classification models to extract features Φ,., oy because these features can reflect some perceptual information. As we want to calculate the distance in the perceptual distance metric and cosine distance has been used measure perceptual similarity in many works (Lin et al. 2016; Wang et al. 2019), we compute the distance between æ and cos Pe yy), After having the distance between two images, we can compute the distance between classes. Each class is set of images. To measure the distance between two sets, we choose Hausdorff distance (Henrikson) 1999). The distance between class and class j is denoted as Di.j- Suppose dataset has n classes Then, can construct a matrix D E B=*"" by calculating the distance between all pairs of classes the dataset, and will be used the following probability model to provide the distance we need. After these preparations, we can start to decide the target label for an input image.
as d(z, 3)
As introduced before, need to estimate the probability distribution P of the ground truth l because we want to find the nearest label to 4. which is unknown in the beginning. When an image is put into classifier f, tfat--of-the-art machine learning classifiers usually output predicted label i, and probability vector á whose elements mean P(x E class i) P.: For simplicity, we suppose the elements in p are sorted in the descending order. Mcanwhile, p can be thought as P's approximation. Furthermore, we define a distance function between Le and the class i in n-classes dataset as D (L,). In order to choose the nearest label to lg as the target label tz, we need to estimate the expectation of Dz(Lz) which is denoted as Ei,r (D.((=)) 1...., n2. general, our target function is Eq. (I]
t. = arg min (D((r(
(1)
i=l....7
Specifically, when p1 is larger than some threshold 8, Maximum Likelihood Estimation (MLE) (Pfanzag] 2011) which means believe the classifier and take the predicted label i as
3

---

Under review as a conference paper at ICLR 2020
Table 1: Notations used in this paper.
Symbol
Meaning
a
Input image
The feature of image
D
The perceptual distance matrix where D represents the distance between class and class j Classifier
f
P
Probability vector (elements are in descending order)
L
The ground truth of the input image
The predicted class of the input image by the classifier
D(((2)
A function calculating the distance between class and the ground truth 4
bo
Target label for input
81, 82
Two thresholds, in this paper, 51 0.8, 62 = 0.01
M
The number of elements larger than 82 p. i.e. M max (n P3 > 62
Fj
The classification boundary between class lg and class j of an image
the ground truth l,, then choose the label (except (.) nearest to le as the target label ty. So in this circumstance, we assume ly = La and (P((,((((( = Dx... Therefore, l is
1 = min
Dzi if
(2)
When P1 is smaller than the threshold 61, we are not sure whether lu is equal to ig. Instead, we sample some labels and compute the weighted distance between each label i and these labels. We sample all labels whose probability are larger than a threshold 82 and we use M to represent the number of sampled labels. We think the input image might belong to of these M labels. The labels whose probability are smaller than 6g will be abandoned because we think the input image can hardly fall into these categories. The weight and the distance is provided by the vector P and matrix D respectively. So in this circumstance, as we are not sure which label is the ground truth, we want to find target label which has the minimum expected distance with all these possible labels.
M p. Di.j and the target label Therefore, the value of (D,,,) can approximated as E,
tu shown in Eq. 3). This can be explained by Importance Sampling (Owen & Zhou 2000) because it is hard to sample from the real probability distribution P. We can only use the probability
distribution p which an approximation of P to estimate value of (D,-
t, arg min
Σ ; i=
bi Das if <
In conclusion, the whole strategy for choosing the target label t. of an input image x computed as Eq. (4). The target label l, minimizes (D,()) just as Figure @egureeshowe shows.
arg min
Dai
if > 61
11....
t
M
arg min
Di. otherwise
3.2 GENERATE ADVERSARIAL SAMPLES
After having the target label, the second step is to attack the input image be mis-classified as this target label. It's easy to achieve by taking the target label as a parameter and putting into the targeted-attack algorithm such targeted-FGSM (Goodfellow et al. 2015) and targeted-CW
4

---

equals Iz,, the classification boundaries are the same as those before moving classification boundaries change. No matter whether the boundaries change or not, we repeatedly Under review as a conference paper at ICLR 2020
an image +- point
Dy = Hausdorff(clase 1, class f)
D11
D12
1
Pts
D22
D Lm
class set
class
D Dn.2
Dnn
probability vector
Pi
Yes
mearest to label
the Tabel
1
Σ
x Pyj
9.
Pz
2
D11
f,
desccnnding order
:
P1
argmin Ey fud..n
classifier
No
E
Pi*
Pn
Dun
Step
P(x E class 1) P
M
P. > 62)
F
Ditly by boundaries are Invariant
Itl,
b, boundaries change
F2
F
F
Ft
Step
Pipeline of our method. There two steps. In the first step, we first compute the distance Di,j between every two class j, in a n-classes dataset. Then we choose the target label l for input image by two strategies according the value of Ω1- The second step is attack the input into this target label. Solid lines are the real boundaries between the current label and the indicated label and dashed lines with notes F are the approximate two-dimensional boundaries. Red indicates the target label while blue indicates other labels. Our method moves the input towards the boundary Fe, until classified fz.
Figure 2:
(Carlini & Wagner 2017), but this operation may suffer huge loss in the image space because of large perturbations.
Inspired by DeepFool Seyed-Mohsen al. 2016). we propose method which an input image to be mis-classified as the target label successfully, but also ensure tiny perturbations in the image space. The mathematical derivation in this step similar DeepFool (Seyed-Mohsen et al., 2016) and the only difference is that, we have a target label chosen in the first step while DeepFool doesn't. As introduced in DeepFool (Seyed-Mohsen et al. 2016). a high dimensional classification boundary can be approximated by a line in two dimensions. As shown in Figure(] for an image No. Fj represents the approximated boundary between its current predicted class and class j and f is the target class we choose in the first step. In the first iteration, we move Tn towards Fr, and get new point I1. The direction of movement perpendicular to Fr,, The distance of the movement is the vertical distance from Zig to F,,- the predicted label l,, of the new point Otherwise, the
only attack
move the current point towards Fr, until it is classified as label ly or the maximum number of iterations has been reached. A pseudocode of the second step is shown Algorithm
Algorithm Generate Adversarial Samples
Input: image classifier f, target label fx Output: Adversarial image
[g] ||
ri
w
I: initialize xo t x, i +-
F++1 f- 2i + i
2: while i,, * ty and max iter do
7: end while
3: w < Vfi, (I:) V (x:)
8: return & = Z141
4: g fL. (z+) (z.)
5

---

Under review as a conference paper at ICLR 2020
4 EXPERIMENTS
In this paper, all experiments are conducted on ImageNet. ImageNet provides the CLS-LOC dataset for classification tasks. Its train split contains about 1300 thousand images. There are 50 thousand validation images and 100 thousand test images. Our experiments are conducted on the train split of CLS-LOC dataset which will be noted as ImageNet-traii split in the following part. We perform extensive experiments to show LabelFool can satisfy all three levels of requirements as an attacker. First we demonstrate the deceptiveness of samples generated by LabelFool to humans in the label space through subjective experiment. Then we calculate the perceptibility and image quality of adversarial samples to show there is not much loss in the image space even compared to DeepFool Becyed-Mossee et al. 2016), which is the state-of-the- art method in the image space. Finally, we conduct attacks on several models to prove the exceptional ability of our method on fooling neural networks which is the first requirement for ideal attack.
4.1 IMPERCEPTIRILITY IN THE LABEL SPACE
Setup. In this part, we will compare LabelFool with three attack methods: DeepFool, FGSM and SporseFool. We first sample 600 source images from ImageNet-train split randomly, and these images are in different classes. Each source image will derive four adversarial images and a base- line image, namely DeepFool-attacker
image, Labellool-attacker image, FGSM-attacker image, SparseFool-attacker image and clean image. Each adversarial image has its mis-classified label and each baseline image has the truth label. We then use term "puzzles" to describe the combination of an image and its label, for 5 x 600 3000 puzzles. A human observer needs to determine whether the label is correct for the image, answering "True" or "False" for each puzzle. To eliminate ob- servers' memory effects, we split 3000 puzzles into five groups, ensuring that 600 images in one group come from different source images. An interface presentation of our subjective experiment is shown in Appendix A We have 10 observers (3 females and 7 males, age between 20-29) to do our subjective experiment.
Evaluation. In this paper, we define index named Performance Gain (PG) the evaluation index. Human observers answer "Truc" or "False" for each puzzle, and the rate with which they answer incorrectly is called Confusion Rate (CR). So every observer has CR for each attack method or baseline. It is an absolute indicator demonstrating how much observers are confused by set of puzzles. But doing arithmetic on CR of different observers meaningless, as different observers have different baseline results. So we define a relative indicator called Performance Gain (PG), which demonstrates how much improvement in the confusion rate after attacking comparing with baseline. It is kind of normalization. The formula for PG is shown in Eq. (5). where CRA means the confusion rate of an attacker and CRB means the confusion rate of baseline. The higher PG an attacker has, the better it confuses people in the label space.
PGA CRA -CRB CRB
(A is an attacker, B Baseline)
(5)
Results. We report the average performance gain of 10 observers in total images in the left of Figure
3 As a whole, there is huge improvement compared with FGSM and SparseFool, about 25 percent improvement and 30 percent improvement respectively. Compared with DeepFool, the gap in PG a little smaller because DeepFool finds the nearest class in the feature level and features usually reflect images' perceptual information as we introduced in Section 2 But there is still 3 percent improvement performance gain.
As animal classes are more fine-grained, the effects of the imperceptibility in the label space become more pronounced. We attack an animal image so that its true animal label changed into a similar animal label, it is difficult for humans to notice that our attack is taking place. Mcanwhile, other attacks change the label into an obviously-incorreet label such as a non-animal category or another species (Some examples in Appendix B). In our subjective experiment, there are 247 animal images out of 600 source images. The right graph in Figure 3 shows the average performance gain of 10 observers in animal images. The accurate data for both graphs in Figure 3 is shown in Appendix [
As for animal images, the improvement is very obvious comparing with all three attack methods. There are about nearly 90 percent improvement in performance gain comparing with FGSM and

---

Under review as a conference paper at ICLR 2020
Total
Animal
1.80
4.25
performance Gain
performance Gain
4,00
1.50
3.50
1.40
3.25
2.75
1.30
2.50
DeepFool
LabelFool
FGSM
SparseFool
DeepFool
Labellool
FGSM
SparseFool
Figure 3: A line chart for average performance gain of 10 observers. The horizontal axis represents four attack methods. The vertical axis represents the mean value of 10 human observers' perfor- mance gain. The graph in the left is for total results, and the right is for animal images.
Fpol A significant improvement can also be seen when comparing with DeepFool, there are about 50 percent improvement in average performance gain.
4.2 IMPERCEPTIBILITY IN THE IMAGE SPACE
In this subsection, we will show our performance in image space to demonstrate that our improvement in the label space is not the cost of huge loss in the image space. We use three metrics to evaluate performance in the image space. One is perceptibility which is simi- lar the definition in previous works (Szegedy al. 2013 Seyed-Mohsen et al. 2016) .p W'
WN x HN Σ intensities (normalized in [0, IJ) of a pixel. The other two are perceptual similarity (Zhang et al,, 2018) and PieAPP (Prashnani et al., 2018). These similarity measures the perceptual distance between an image and its reference image while PieAPP measures the perceptual error. In this paper, the reference image is the clean image. And the smaller these three metrics are, the better the adversarial samples are.
|| where Ywrh a 3-dimensional vector representing the RGB h=1
HN
metrics for image quality. Perceptual
Perceptibility
Perceptual Similarity
PicAPP
10841
63940
ResMet-50 5D Resten-0 VGG, J9bn AleeNet
Resten 50
ResMet-34
VGG 196n
AleeNet
ResNen-50 50
Rechet-34
VGG_190n 188n
AleeNet
Figure 4: Mean value of perceptibility, perceptual similarity and PieAPP for adversarial samples generated by different attack methods on different models.
We randomly choose 1000 images from ImageNet and attack the classifier to generate 1000 adver- sarial samples. Then we compute mean value for these ad adversarial samples of three metrics. In this experiment, we test four classifiers: ResNet-34, ResNet-50, VGG-19 (with batch normalization) (Simonyan & Zisserman 2014) and AlexNet (Krizhevsky et al.) 2012a). The results are shown in Figure 4 whose original data are reported Appendix We can see although LabelFool is signifi- cantly better than FGSM and SparseFool, it is still little worse than DeepFool in all three metrics. However, visual results (Figure 5 indicate that human observers can not notice the difference be- tween LabelFool and DeepFool in the image space as the metric value is on such small scale.
4.3 FOOL NETWORKS
We will show attack rate in the last experiment which the most fundamental requirement for attacker. Results are shown in Table The results are the average value of three groups of

---

Under review as a conference paper at ICLR 2020
chickadee
Clean Image
DeepFool-Attacked
Labellool-Attacked
Clean Image
DerpFool-Attadded
Labelfool-Attacked
Perceptibility
3.2e-5
Perceptibilit
Perceptual Similartty
Perceptual Similarity PicAPP
PlcAPP
Figure 5: Two visual results of adversarial samples generated from AlexNet in the image space. In each result, the left image is the clean image, the middle one is DeepFool-attakked adversarial sample and the right one is LabelFool-attacke label/ the label after attacked. The three metrics adversarial samples are reported the table below them.
adversarial sample. Above the images the true
Table 2: Attack rate of different methods different models.
Model
DeepFool
LabelFool
FGSM
SparseFool
ResNet-34 ResNet-50
92.67%
97.50%
95.03%
92.60%
93.08%
97.88 %
95.09%
92.53%
VGG- 92.03%
97.48% %
94.59%
83.70%
AlexNet
90.35%
97.38%
96.44%
89.11%
experiments. Each group has 1000 original images from ImageNet, we use these original images to generate adversarial images for four models respectively. We surprisingly find that LabelFool has the highest attack rate on all models comparing with other methods. This might benefit from our probability model which is used to choose the target label. Because in our strategy, when Ộ1 S ò1, consider
do not use the predicted label the ground truth like other methods do. Instead, all labels whose probability are larger than 82 and choose the label nearest to these labels as the target label. This operation avoid some mistakes and improve the attack rate when the classifier doesn't 't give a correct classification result. An example is shown in Appendix D
CONCLUSION AND FURTHER DISCUSSION
Conclusion. In this study, we pay attention to tiny perturbations in the label space. To our best knowledge, are the first who points out the importance of the imperceptibilitty in the label space for adversarial samples. Furthermore, we explore feasible method named LabelFool to identify a target label "similar" with an input image's ground truth and perturb the input image to be mis-classified as this target label so that human observer will overlook the misclassification and lower the vigilance of defenses. Our experiments show that, while LabelFool a little behind DeepFool in the image space, it is much imperceptible in the label space to human observers. Since we adopt Importance Sampling instead MLE only traditional method, the success rate attack also get gains.
Further discussion. In this paper, we just propose a feasible way to generate adversarial sam- ples which can confuse people in the label space. However, there is room for improvement in our approach. Our results provide the following avenues for future rescarch.
The perceptual features can be optimized by well-designed loss function which can im- prove the accuracy rate in finding nearest label ulteriorly.
We only consider perceptual distance this paper, but semantic distance also has its signif- icance for reference of confusing people in the label space. We may take the semantic tree into consideration and make a trade off between perceptual distance and semantic distance in future research.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pp. 39-57. IEEE, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hicrarchical image database. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255. Icce, 2009.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9185-9193, 2018.
Ian Goodfellow, Shlens Jonathon, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference of Learning Representation, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Jeff Henrikson. Completeness and total boundedness of the hausdorff metric. MIT Undergraduate Journal of Mathematics, 1:69-80, 1999.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in Neural Information Processing Systems, pp. 1097-1105, 2012a.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in Neural Information Processing Systems, pp. 1097-1105, 2012b.
Yann LeCun, Koray Kavukcuoglu, and Clément Farabet. Convolutional networks and applications in vision. In Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp- 253-256. IEEE, 2010.
Tsung Yu Lin, Aruni Roychowdhury, and Subhransu Maji. Bilinear enn models for fine-grained visual recognition. In /EEE International Conference on Computer Vision, 2016.
Apostolos Modas, Moosavi-Dezfodd Scyed-Mohsen, Mohsen, and Ecole Polytechnique Federale de Lau- sanne Pascal Frossard. Sparsefool: a few pixels make a big difference. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2019.
Deoosvi-Dezzfooli, O. Fawzi Fawzi, and Frossard. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Art Owen and Yi Zhou. Safe and effective importance sampling. Journal of the American Statistical Association, 95(449):135-143 2000.
Johann Pfanzagl. Parametric statistical theory. Walter de Gruyter, 2011. ISBN 3-11-013863-8.
Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. PieAPP: Perceptual image-error assessment through pairwise preference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1808-1817, 2018.
Seyed-Mohsen, Moosavi-Dezoooli, Pascal Frossard A Alhussein Fawzi, and Fcole Polytechnique Fed- erale de Lausanne. DeepFool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 2019. doi: 10.1 109//VC. 2019.200 2890858.

---

Under review as a conference paper at ICLR 2020
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew R Scott. Multi-similarity loss with general pair weighting for deep metric learning. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 5022-5030, 2019.
Kaidi Xu, Sijia Liu, Pu Zhao, Pin Yu Chen, Huan Zhang, Quanfu Fan, Deniz Erdogmus, Yanzhi Wang, and Xue Lin. Structured adversarial attack: Towards general implementation and better interpretability. In International Conference of Learning Representation, 2019.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586-595, 2018.
A
AN INTERFACE PRESENTATION OF THE SUBJECTIVE EXPERIMENT
Figure shows the interface of our subjective experiments.
Note: whether given picture.
11800
spaghettl squash
you this labal, "Show Reforence Picturcs" button.
Show Releeaee Reterns
Figure 6: Interface presentation of our subjective experiment.
B EXAMPLES FOR ANIMAL CLASSES
Figure shows three examples for animal classes to demonstrate that LabelFool makes fine-grained changes but other methods make some ridiculous changes instead.
Image
Ground truth
DeepFool
LabelFool
FGSM
SparseFool
cock
partridge
hen
partridge
partridge
peacock
urchin
gallinule
wool
poncho
fire salamander
spotted salamander
sea slug
coral reef
courgette
Figure 7: Three examples for animal images. The first column shows the clean image. The second column shows the ground truth label and other columns show the label after attacked.
10

---

Under review as a conference paper at ICLR 2020
C
ORIGINAL DATA FOR FIGURE AND 4
Table 3 is the original data for Figure The original data of perceptibility, perceptual similarity,
PieAPP in Figure 4 reported in Table
6 respectively. is provided for the sake of convince if
anyone wants to rewrite Figure or
Table 3: Data for Figure
Index
DeepFool
LabelFool
FGSM
SparseFool
Total
Performance Gain
1.68
1.71
1.39
1.34
Animal
Performance Gain
3.55
4.04
3.11
3.00
Table 4: Perceptibility Data for Figure
Model
DeepFool
LabelFool
FGSM
SparseFool
ResNet-50
3.36E-05
4.75B-05
6.55B-04
2.56E-03
ResNet-34
3.35E-05
4.84E-05
6.59E-04
2.56E-03
VGG-19bn
3.18E-05
3.80F-05
6.44E-04
2.56E-03
AlexNet
3.59E-05
7.24E-05
6.68E-04
2.56E-03
Table 5: Perceptual Similarity Data for Figure
Model
DeepFool
LabelFool
FGSM
SparseFool
ResNet-50
1.48E-3
5.64E-3
0.20
0.42
ResNet-34
1.39E-3
5.91E-3
0.20
0.41
VGG-19bn
6.86E-4
2.54E-3
0.18
0.41
AlexNet
1.34E-2
3.50E-2
0.30
0.40
Table 6: PieAPP Data for Figure 4
Model
DeepFool
LabelFool
FGSM
SparseFool
ResNet-50
0.07
0.14
1.35
1.57
ResNet-34
0.07
0.15
1.31
1.46
VGG-19bn
0.05
0.08
1.27
1.59
AlexNet
0.07
0.21
1.28
1.21
D AN EXAMPLE FOR SECTION 4.3
This is example to illustrate why method has the highest attack rate. We only give example of DeepFool and LabelFool. SparseFool and FGSM have similar effects with DeepFool.
Figure 8 is example where the classifier fail give correct classification for the input image x. The ground truth x is class while the predicted class is class 3. this example, DeepFool takes class 3 as the true class. Then DeepFool finds the nearest class to class 3 in the feature space which class in this example, and moves the input image towards class 2. When the perturbed image classified as class 2 which is different from the predicted class, DeepFool considers the attack

---

Under review as a conference paper at ICLR 2020
succeed and stops the algorithm. However, it fails to attack actually because class 2 is the true class of æz
Different from DeepFool, LabelFool sample top classes in this example because their probabilities are larger than 0.01 and compute the expected distance between each class the dataset and these classes (3q. Finally, LabelFool choose class 394 the target class because it has the minimum expected distance with top classes. By moving æ towards class 394, LabelFool attacks successfully.
0.6746
Probabllity
0.35
Ground Truth Class: 2
0.18
0.0156
0.0050 0.0027 0.0016 0.0014
0.00
classifier: VGG_18 19
Class
(with batch normalization)
The line chart in the right part shows the top 10 elements in P, when image whose ground truth class is class 2 is given into the classifier (GG_19bn, The horizontal axis represents the class and the vertical axis represents the probability that belongs this class. The predicted class ly is class and means the classifier fail give correct classification.
Figure 8:
E
SUPPLEMENTARY EXAMPLES FOR LABELFOOL
Figure shows some supplementary examples to show what LabelFool actually does.
Image
Ground truth
DeepFool
LabelFool
FGSM
SparseFool
sea cucumber
tench
coho
barracouta
barracouta
whiptail lizard
banded gecko
alligator lizard
stole
damselfly
wolf spider
long-horned beetle
wolf spider
wolf spider
harvestman
beer bottle
bottle
pop bottle
pop bottle
wine bottle
chain saw
lumbermill
power drill
lumbermill
lumbermill
container
liner
dock
dock
dock
ship
Figure 9: Some other examples for illustrating what LabelFool does. The first column shows the clean image. The second column shows the ground truth label and other columns show the label after attacked.
12

---

Under review as a conference paper at ICLR 2020
F AN APPLICATION: FACE RECOGNITION
Figure is an application to show it is necessary to generate imperceptible adversarial examples in the label space even for image classification tasks. We take face recognition system for entrance as an example. In Figure I0 A is the person who is using the face system to go into the gate. LobelFool aims let the system misclassify and B who the one looks like A, but other untargeted attacks may let the system misclassify A an C who looks totally different from A. The attack is easy to be detected by the guard if the system misclassifies A and C, but hard detect the system misclassifies A and B. Letting fake B in will bring great potential risks to security and safety.
the person entering the gate
LabelFool's target id
Other methods' target
A
B
Figure 10: A is the person who is using face system to enter the gate. Green lines represent what LabelFool aims to do, that to miscalssify A and B who looks like A. Red lines represent what other untargeted attacks do. They misclassify A and C who looks totally different from A and this emror easy to be detected by the guard.
13


Reviews:
[{"review_id": "r1g-9ZHlsB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method to create adversarial perturbations whose target labels are similar to their ground truth. The target labels are selected using an existing perceptual similarity measure for images.  Perturbations are generated using a DeepFool-like algorithm. Human evaluation supports that the pair of the generated images and target labels are more natural to humans than prior attack algorithms.\n\nThis paper should be rejected due to the lack of motivation to create adversarial examples less detectable by humans automatically. Attackers can manually select target labels and apply targeted attacks. In the target label selection, attackers can choose less detectable labels if necessary. It is encouraged to provide some applications where attackers want to create less detectable adversarial examples in label space without manually assigning target labels.\n\n==========\nUpdate:\n\nAfter reading the authors' responses, the motivation of the paper became clearer. I will not get surprised if this paper is accepted. However, all reviewers still share concerns about the importance of the problem tackled. I think the paper needs to suggest more applications and emphasize the value of the goal in the main paper before being published.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HkgFLke2tr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes a technique for creating adversarial images where the added perturbations are not only imperceptible to machines, but also to human observers. The authors describe why this might be beneficial. The method works by finding labels that are not too far from the source image's ground-truth labels, and moving the source image in that direction. To find the target label, the authors use a threshold on the confidence of predicted ground-truth labels. The authors test their algorithm using a newly proposed metric of how much a method allows imperceptibility for a human observer. They show that their method creates images whose perturbations are more impercetible to humans, compared to other methods, but are also imperceptible to machines.\n\nMy concern is as follows: If the misclassification is between A and B, and they are related classes, is the attack so bad? And what are the scenarios in practice when a user simply wants to create an attack, without regards to the target label chosen? I imagine normally the attacker has a target label in mind, so the part of the paper that chooses a target label is not very useful; and this is the main element of novelty, since the rest of the method is from DeepFool, as the authors explain. Some specific use cases of this methods should be discussed. \n\nMinor suggestion: It would be useful to see examples like in Fig. 5 but with the classes (true/target) listed.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "B1g1QouqYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method for constructing adversarial attacks that are less detectable by humans, by changing the target class to be a class similar to the original class of the image. The resulting attack methodology is then studied in terms of its imperceptibility in label space, and shown to be less perceptible in label space to human observers, while not coming at a cost in image space.\n\nThe paper presents compelling evaluation of the method and does seem to succeed in proving that their proposed attack satisfies the stated goal. However, it appears as though this goal is somewhat counter to the main point of adversarial examples---indeed, if the label is reasonable to a human, then what makes the adversarial example adversarial? The main threat in adversarial examples research seems to be that it is possible to induce predictions that are arbitrarily different from humans' on natural-looking in puts. Thus, changing the label to something that a human actually agrees with would actually reduce the impact of the adversarial attack.\n\nIn order to improve the paper, I would suggest applying the same (or similar) methodologies to other areas of ML security where imperceptibility in label space is commonly desired---for example, in data poisoning attacks or backdoor attacks. In general, such attacks are much more likely to be \"inspected\" by humans, and so imperceptibility in both label and image space is very desirable. However, I suspect that this would require significant effort and changes to the paper, and so for now I recommend rejection.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Thanks for the discussion with reviewers, which improved our understanding of your paper significantly.
However, we concluded that this paper is still premature to be accepted to ICLR2020. We hope that the detailed comments by the reviewers help improve your paper for potential future submission.
Decision: Reject