Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration
AffiliationAddress
Abstract
Previous works have extensively studied the transferability of adversarial samplesin untargeted black-box scenarios.
However, it still remains challenging to craftthe targeted adversarial examples with higher transferability than non-targetedones.
Recent studies reveal that the traditional Cross-Entropy (CE) loss function isinsufficient to learn transferable targeted perturbations due to the issue of vanishinggradient.
In this work, we provide a comprehensive investigation of the CE functionand find that the logit margin between the targeted and non-targeted classes willquickly obtain saturated in CE, which largely limits the transferability.
Therefore,in this paper, we devote to the goal of enlarging logit margins and propose twosimple and effective logit calibration methods, which are achieved by downscalethe logits with a temperature factor and an adaptive margin, respectively.
Both ofthem can effectively encourage the optimization to produce larger logit margins andlead to higher transferability.
Besides, we show that minimizing the cosine distancebetween the adversarial examples and the targeted classifier can further improvethe transferability, which is benefited from downscale logits via L2-normalization.
Experiments conducted on the ImageNet dataset validate the effectiveness of theproposed methods, which outperforms the state-of-the-art methods in black-boxtargeted attacks.
The source code of our method is available at Link.
Introduction
In the past decade, deep neural networks (DNNs) have achieved remarkable success in variousfields, e.g. , image classification [24], image segmentation [19] and object detection [23].
However,
Goodfellow et al.
[5] reveal that the DNNs are vulnerable to adversarial attacks, in which addingimperceptible disturbances to the input can lead the DNNs to make an incorrect prediction.
Following,many approaches [3, 4, 1, 27, 29] have been proposed to construct more destructive adversarialsamples for investigating the vulnerability of the DNNs.
[5, 18] also show that the adversarialsamples are transferable across different networks, raising a more critical robustness threat under theblack-box scenarios.
Therefore, it is vital of exploring the vulnerability of the DNNs, which is veryuseful for designing robust DNNs.
Currently, most of the works [3, 29, 16, 10, 28, 6] have been devoted to the untargeted black-box attack,in which adversarial examples are crafted to fool unknown surrogate models to predict unspecifiedincorrect labels.
For example, [3, 29] leveraged input-level transformation or augmentation toimprove the non-targeted transferability.
[10] proposed a powerful intermediate feature-level attack.[28, 6] demonstrated that backpropagating more gradients through the skip-connections can increasethe transferability.
Despite the success in non-targeted cases, the targeted transferability remainschallenging, which requires eliciting the black-box models into a pre-defined target category label.
[-Top-1Top-2Top-3Margin (b) Po+Trip     300 Iterations
-]
For learning the transferable adversarial samples in untargeted cases, most methods have leveragedthe Cross-Entropy (CE) as the loss function.
However, [15, 30] recently showed that the CE loss isinsufficient for learning the adversarial perturbation in the targeted case due to the issue of vanishinggradient.
To deal with this issue, [15] adopt the Poincaré distance to increase the gradient magnitudeduring the optimization adaptively.
[30] demonstrated that an effortless logit loss equal to the negativevalue of the targeted logits could alleviate the gradient issue and achieve surprisingly strong targetedtransferability.
Besides, [30] also showed that optimizing with more iterations can significantlyincrease the targeted transferability.
In this study, we take a closer look at the vanishing gradient issue in the CE loss function and findthat the logit margin between the targeted and non-targeted classes will quickly get saturated duringthe optimization.
Moreover, this issue will influence the performance of the perturbations and thusessentially limit the transferability.
Specifically, along with the training iterations in CE, we observethat the logits of the targeted and non-targeted classes increase rapidly in the first few iterations.
However, after reaching the peak, the logit margin between the targeted and non-targeted classes willget saturated, and further training will decrease the logits simultaneously to maintain this margin.
This phenomenon is mainly due to the fact that the softmax function in CE will approximately outputthe probability of the target class to 1 when reaching the saturated margin ( e.g. , 10).
Thus, it raisesthe problem that the transferability will not be further increased even optimized with more iterations.
While in practice, we are encouraged to increase the transferability by maximizing both the logit forthe targeted class and its margin against other non-targeted classes to cross the decision boundaries ofother black-box models.
In this paper, we devote to enlarging logit margins to alleviate the above saturation issue in CE.
Inspired by the temperature-scaling used in the knowledge distillation [8], a higher temperature Twill produce a softer probability distribution over different classes.
We firstly leverage this scalingtechnique into the targeted adversarial attack to calibrate the logits.
Then the logits margin betweenthe targeted and non-targeted classes will not be saturated after only a few iterations and will keepimproving the transferability.
On the other aspect, instead of using a constant T , we further exploredan adaptive margin-based calibration by scaling the logits based on the logit margin of the targetclass and the highest non-target class.
In addition, we also investigate the effectiveness of calibratingthe targeted logit into the unit length feature space by L2-normalization, which is equivalent tominimizing the angle between the adversarial examples and the targeted classifier.
Finally, we conduct experiments on the ImageNet dataset to validate the effectiveness of the logitscalibration for crafting transferable targeted adversarial examples.
Experimental results demonstratedthat the calibration of the logits helps achieve a higher attack success rate than other state-of-the-artmethods.
Additionally, we tested the logit calibration in Generative Adversarial Networks (GANs)-based TTP method [21] to train the target-class-specific generators, which is also beneficial forincreasing the transferability in the resource-intensive method.
Related Works
In this section, we give a brief introduction of the related works from the following two aspects:untargeted black-box attacks and targeted attacks .
2.1 Untargetd Black-box Attacks
After [25] exposed the vulnerability of deep neural networks, many attack methods [29, 4] have beenproposed to craft highly transferable adversaries in the non-targeted scenario.
We first briefly reviewseveral seminal gradient-based attack methods that focus on enhancing the transferability againstblack-box models.
These methods also can be leveraged to improve the transferability in the targetedscenario.
Iterative-Fast Gradient Sign Method (I-FGSM)
[14] is an iterative version of FGSM [5], whichadds a small perturbation with a small step size α in the gradient direction iteratively:
[Equation]
where ˆ x ′ i denotes the intermediate adversarial image in the i th iteration, α = ϵ/T makes theadversaries be constrained by an upper-bound ϵ through the l p -norm, and T is number of iterations.
Momentum Iterative-FGSM (MI-FGSM) [3] integrates a momentum term into I-FGSM to accumulate the update direction of previous steps for achieving better transferablily. The iteration processcan be summarized as:
[Equation] where g i is the momentum term of gradient in the i th iteration and µ is a decay factor. Translation Invariant-FGSM (TI-FGSM) [4] is different from the above methods, which calculatethe average gradient on multiple translated images for the update.
This technique can prevent theattack from overfitting to the specific white-box model. To improve the computational efficiency, the TI-FGSM proposes to convolve the gradient of the untranslated image with a Gaussian kernel insteadof calculating the gradient on translated images, as follows: [Equation]
where W is a Gaussian kernel used for smoothing gradient and ∗ is the operator of convolution. Diverse Input-FGSM (DI-FGSM) [29] is conceptually similar to TI-FGSM, but it is designed basedon the input transformations with random resizing and padding operations rather than smoothinggradients. DI-FGSM can be represented as:
[Equation] where κ ∈ U (0 , 1) controls the using of the transformed image or the original one for gradientcalculation. T (ˆ x ′ i , p ) is implemented with the probability p at each iteration, which will bring morerandomness since augmentation parameters are not fixed during iterations.
2.2 Targeted Attacks
Targeted attacks are different from non-targeted attacks, which need to change the decision to aspecific target class.
[13] integrates the above non-targeted attack methods into targeted attacks tocraft targeted adversarial examples.
However, the performance is limited because it is insufficient tofool the black-box model only by maximizing the probability of the target class.
Po+Trip [15] proposes the Poincaré space as a metric space and utilizes Triplet loss to improvetargeted transferability by forcing adversarial example toward the target label and away from theground-truth label. The Poincaré distance can alleviate the vanishing gradient in the cross-entropyloss and adaptively adjust the magnitude of gradient, which can be described as: [Equation] where z ( x ′ ) is the output logits of the surrogate model, x ′ is the perturbed image, 1 t is the one-hotencoding related to the target class t and ξ is a small constant ( 0 . 0001 ).
For further improvingperformance, a new triplet loss is proposed to reduce the distance between the output of adversarialexample and the target label, and to increase the margin from the true label y , as below: [Equation]
[Equation] The overall loss function of Po+Trip is defined as L all = L Po ( x ′ , t ) + λ · L Trip ( t, z ( x ′ ) , y ) , whereλ controls the weight of the triplet loss. Logits [30] adopts a simple and straightforward idea by directly maximizing the target logit to pullthe adversarial examples close to the target class for addressing the vanishing gradient problem inexisting transferable attacks, which can be expressed as:
[Equation]
[Equation]
[Figure]
where z t ( · ) is the output logits of the target class.
[Figure]
Since targeted transferability is extremely difficult, many studies employ resource-intensive approaches, which train target class-specific models (auxiliary classifiers or generative models) onadditional large-scale data, as an auxiliary.
For example, the FDA methods [12, 11] use the intermediate feature distributions of CNNs to boost the targeted transferability by training class-specificauxiliary classifiers to model layer-wise feature distributions.
The GAP [22] trains a generative modelfor crafting targeted adversarial examples.
Subsequently, [20] uses a relativistic training objectiveto train the generative model for improving attack performance and cross-domain transferability.
Recently, the TTP [21] utilizes the global and local distribution matching for training target classspecific generators for obtaining high targeted transferability.
However, the TTP requires actual datasamples from the target class and brings expensive training costs.
Different from the above methods,we introduce two simple and effective logit calibration methods into the CE loss function, whichachieve competitive performance without additional data and training.
Method
Given a white-box surrogate model F s and an input x not from the targeted class t , our primary goalis to learn an imperceptible perturbation δ that can fool the F s to output the target t for ˆ x = x + δ .
Besides, the prediction of ˆ x will also be t when feeding to other unknown black-box surrogate models.
The l ∞ -norm is usually used to constrained the δ within an upper-bound ϵ , denoted as || δ || ∞ ≤ ϵ .
For the surrogate model F s , we denoted the feature for the final classification layer of the input x asϕ ( x ) .
The logit z i of a category i is computed by z i
=
W Ti ϕ
( x ) + b i , the W i and b
i are the classifierweights and bias.
The corresponding probability p i after the softmax is p i =
e zi (cid:80) e zj .
3.2 Logit Margin
When successfully attacked the F s , the logit z t of the target class will be higher than the logits z nt ofany other non-target class in the classification task.
Their logit margins can be computed by,
[Equation]
[15, 30] showed that it is insufficient to obtain transferable targeted adversarial samples that are onlyclose to the target class while not away from true class and other non-targeted classes.
Based on thisproperty, it encourages us to continually enlarge this logit margin to increase the separation betweenthe targeted and other non-targeted classes.
To have a better understanding of the relationship between the logit margins and the targetedtransferability, we visualize the average Top-3 logits (1 targeted class and other two non-targetedclasses) of 50 random adversarial samples trained for crafted ResNet50 by the CE, Po+Trip, and
the
Logit loss functions with MI, DI and TI following [30].
We also compute the average logit marginof the targeted class against the Top-20 non-targeted classes.
The logit and margin are shown in
Figure 1, and the transferability from ResNet50 to VGG16 is plot in Figure 2.
[Figure]
From Figure 1, we can observe that the logits of the targeted class and the Top-2 non-targeted classes increaserapidly in the first few iterations for the CE and Po+Triploss, as well as their logit margins.
When reaching thepeak, the margin is saturated, and the logits start to decrease simultaneously to maintain the saturated margin.
By comparing the CE and Po+Trip, the Po+Trip needsfewer more iterations to reach the saturated status and thusshows a marginal better transferability than CE, as shownin Figure 2.
In comparison, the Logit loss function willkeep increasing the logits of the targeted category and thelogit margin.
Therefore, the Logit loss function showsa much better targeted-attack success rate than CE and
Po+Trip.
On the other hand, the Logit loss also significantly increases the logits for other non-targeted classes.
To further analyze why the CE loss function saturated tothis logit margin and explore the effectiveness of increasing the margin during training, in thefollowing sections, we will revisit the cross-entropy loss function and introduce the logit calibrationto achieve this goal.
3.3 Revisiting the Cross-Entropy Loss
Firstly, our objective is to maximize the logit margin in Eq.
After computing the gradient w.r.t.
toϕ (ˆ x ) , we can get
[Equation]
This gradient indicates that the adversarial feature ϕ (ˆ x ) needs to move towards the target class whileapart from those non-target classes.
Next, we compute the gradient w.r.t.
to ϕ
(ˆ x ) in the Cross-Entropyloss function
[Equation]
and get the gradient ∂L ce ∂ϕ (ˆ x ) as
[Equation]
From Eq.
11, we actually can find the CE loss function is designed to adaptively optimize the ϕ (ˆ x )towards W t and away from other W i .
However, after being optimized for several iterations, the p i ofthe non-targeted class will quickly approximate to 0 and then significantly vanish the W t − W i .
Let’s consider the case only with 2 classes ( t and nt ), we have the probabilities p t and p nt as:
[Equation] [Equation]
As shown in Figure 3, the p t will get close to 1 when z t − z nt > 6 ( e.g. , p nt ≈ 2 e − 9 whenz t − z nt = 20 ).
In such a context, the gradient will significantly vanish.
Recall that, in the CE lossfunction (Figure 1 (a)), the logit margin between Top-1 and Top-2 logits first increases rapidly but willreach saturated status when approaching a certain value.
This further indicates that the optimizationof the CE loss function is largely restrained when the logit margin reaches a certain value.[-10 z t − z nt 0 .0 .0 .
-]
To this end, we raise the question if we explicitly enforce the optimization to enlarge thelogit margin ( z t − z nt ), could we get better transferable targeted adversarial samples? [Figure]
[Figure] To answer this, we propose to downscale the z t − z nt by afactor s in the CE and extent the informative optimizationfor more iterations.
Since in such circumstance, z t − z ntwill be enlarger by the factor s .
Specifically, suppose thatthe optimization will be saturated when z t − z nt reaches a 197 certain value v .
Using z t − z nt and z t − z nt s in the CE willboth approach the saturated value of v .
Then, it is easy toinfer that, for the latter case, z t − z nt will be v × s .
3.4 Calibrating the Logits
To downscale the z t − z nt during the optimization, we 202 investigate three different types of logit calibrations in thisstudy, i.e. , Temperature-based, Margin-based, and Anglebased.3.4.1 Temperature-based
Inspired by the Temperature-scaling used in the Knowledge distillation [8], our first logit calibration directly downscale the logits by a constant temperaturefactor T ,
[Equation]
After introducing the T , the probability distribution p will be more softer over different classes.
Thecorresponding gradient can be compute by:
[Equation]
The ˆ p i will not quickly approach to 0 after only a few iterations.
In Figure 4 (a)(b), we visualized the logits of using T = 5 and T = 20 .
We can find that targetedlogits and the logit margin will keep increasing as the same as the Logit in Figure 1.
Meanwhile, thetrend of T = 20 is very similar with the Logit [30] and we show that the Logit loss function can beconsidered as a special case of calibrating the logits with a large T in the supplementary.
[-Top-1Top-2Top-3Margin (a) T=      300 Iterations
-]
[-Top-1Top-2Top-3Margin (b) T=      300 Iterations
-]
3.4.2 Margin-based
The previous Temperature-based logit calibration contains a hype-parameter T , which could bedifferent for different surrogate model F s .
To migrate this issue, we further introduce an adaptivemargin-based logit calibration.
Specifically, we calibrate the logits by using the margin between the
Top-2 logits in each iteration, denoted as:
[Equation]
where ˆ z 1 and ˆ z 2 are the Top-1 and the Top-2 logit, respectively.
In this Margin-based logit calibration, we will enforce the p t and p ˆ1 of the Top-1 non-target class ateach iteration meet the following constraints:
[Equation]
[Equation]
Then, it can adaptively deal with the vanishing gradient issue in the original CE loss function.
Thelogits and the margin is shown in Figure 4 (c).
3.4.3 Angle-based
On the other aspect, different W t usually has a different norm.
To further alleviate the influence ofvarious norms, we calibrate the logit into the feature space with unit length by L2-normalization,
W Ti ϕ (ˆ x )+ b i
||
W i |||| ϕ ( x ) || .
If omit the b i , this calibration is compute the cos ( θ ) between ϕ (ˆ x ) and W i , and we termit as angle-based calibration.
Since, this angle-based calibration will bound each logit smaller thanone.
Instead of using the CE loss function, we directly minimize the angle between the ϕ (ˆ x ) and thetargeted W t .
The optimization loss function is:
[Equation]
The angle-based classifiers have been widely using in Face-Recognition task [17, 2].
In the experiments, we evaluate the performance of using different logit calibration methods and also evaluatetheir mutual benefits.
Experiments
4.1 Experimental Setup
In this section, we evaluate the effectiveness of logit calibration for improving transferable targetedadversarial attack.
Following the recent study [30], we conduct the experiments on the difficult ImageNet-Compatible Dataset 1 .
This dataset contains 1,000 images with 1,000 unique class labelscorresponded to the ImageNet dataset.
We implement our methods based on the source code 2 providedby the Logit [30].
The same four diverse CNN models are used for evaluation, i.e. , ResNet-50 [7],
DenseNet-121 [9], VGG-16 with Batch Normalization [24] and Inception-v3 [26].
The perturbationis bounded by L ∞ ≤ 16 .
The TI [4], MI [3] and DI [29] were used for all attacks, and || W || 1 = 5 is 246 set for TI.
The I-FSGM is adopted for optimization with the α = 2 .
The attacks were trained for 300iterations on a NVIDIA-2080 Ti GPU.
We run all the experiments for 5 times, and report the averagenumber of successfully attacked samples.
4.2 Comparison with other methods in single-model transfer
We first compare the proposed (temperature-based, margin-based and angle-based) logit calibrationmethods with the original CE, Po+Trip [15], and Logit [30] in the single-model transfer task.
In thistask, we take one model as the surrogate model for training, and test the targeted transferability inattacking other 3 models.
As shown in Table 1, the original CE loss function produces a worst performance than the Po+Tripand Logit.
But after performing the logit calibration in the CE loss function, we can find a significantperformance boost compared with the original CE.
All the calibration methods can outperform the
Logit, especially when using the ResNet50 and Dense121 as the surrogate.
These results indicate thatthe logit margin can significantly influence the performance of the targeted transferability.
On the examples/nips17_adversarial_competition/dataset
[Table] [Table] other aspect, we find that T
= 10 has better performance than T = 5 on the VGG-16, suggestingthat different models may need different T .
Instead of finding the best T for a different model, the
Margin-based calibration can solve the issue and reach the overall best transferability in all fourmodels.
However, we find that the Angle-based calibration is not working on the VGG16, whichneeds further investigation.
4.3 The influence of different T in CE
In this section, we evaluate the influence of using different T in the CE loss function.
The results arereported in Table 2.
From the Table, we can have the following observations.
The scaling factor
T has a significant influence on the targeted transferability.
Specifically, there is a large decrease inperformance when using a small T = 0 .
After increasing the T , we can observe the number ofsuccessfully attacked samples will increase.
The optimal T for different model is different.
Forexample, T = 5 can produce the overall best performance for ResNet50, Dense121, and Inceptionv3, while the VGG16 requires a large T to obtain better transferability.
The performance arecomparable when using T = 5 and T = 10 for ResNet50, Dense121, and Inception v3.
This isbecause that we use I-FSGM for optimization, which only considers the sign of the gradients.
Using a larger T , the performance will be similar as the Logit loss function (see Table 1).
We providea deep analysis of this phenomenon in the supplementary material.
4.4 The mutual benefits of different calibration methods
In this part, we evaluate the mutual benefits of combining different calibrations and can have thefollowing findings. 1) Combining the T=5 and Margin, there is no increase in performance compared
[Table]
[Table]
[Table]
with using one of them.
This is because that the gradient directions of these two methods are verysimilar.
Combining the T=5 and Angle, we can observe a further improvement when using ResNet50 and Dense121 as the surrogate model, e.g. , the transferable rate of “ResNet50 → Dense121”is increased to 823.8 with 300 iterations.
Since Angle has poor performance on ResNet 50, thetransferable rates of corresponding combinations are also low.
Combining the Margin and Angle,there are only slight improvements on ResNet50 and Dense121, while it can alleviate the negativeeffects caused by the angle-based calibration.
[-Finally, by jointly considering the results in Table 1, 2 and 4, we suggestusing T=
-]
4.5 Comparison with the TTP method
In this section, we further evaluate the proposed temperature-based logit calibration in the GAN-basedtargeted attacks.
Following the setting in TTP [21], we sampled 50K images from the ImageNettraining set and 50K images from the Painting dataset 3 , which are used to train the targeted generatorsfrom different source domains.
Instead of using the distribution matching and neighborhood similaritymatching loss [21], we only use the cross-entropy function for training the targeted generators whilekeeping other settings identical.
More training and evaluation details used by TTP can be referred to[21].
We used the ResNet50 as the surrogate model and reported the results in Table 4.
From Table 4, we make the following findings.
By using ImageNet as the training dataset, the TTPshows better transferability than the CE in attacking other black-box models.
The average targetedaccuracy of TTP is around 3% higher than that of CE.
After downscale the logit by 5 in the CE lossfunction (CE (T=5)), we can observe a significant boost of the Top-1 targeted accuracy for all models,reaching the average targeted accuracy of 91.98% (ImageNet).
For both ImageNet and Paintingas the training source, the CE (T=5) can surpass the TTP by a large margin (91.98% vs. 83.12% &89.70% vs. 81.33%).
These experimental results demonstrate that the proposed temperate-based logitcalibration is also effective in training generator-based targeted attackers.
Note that, compared to
TPP, our logit calibration has the benefit of without using any data from the target class.
Conclusion
In this study, we analyzed the logit margin in different loss functions for the transferable targetedattack, and find that the margin will quickly get saturated in the CE loss and thus limited thetransferablity.
To deal with this issue, we introduce to use logit calibrations in the CE loss function,including Temperature-based, Margin-based, and Angle-based.
Experimental results verified theeffectiveness of using the logit calibration in the CE loss function for crafting transferable targetedadversarial samples.
The proposed logit calibration methods are simple and easy to implement, whichcan achieve state-of-the-art performance in transferable targeted attack.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[No]
(c) Did you discuss any potential negative societal impacts of your work?
[No] No negative
effects
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See abstract.
(b) Did you specify all the training details ( e
g. , data splits, hyperparameters,
how they
were chosen)?
[Yes] See Section 4.1.
(c) Did you report error bars ( e
g. , with respect to the random seed after running experiments multiple times)?
[No] We report the average of running experiments 5times.
(d) Did you include the total amount of compute and the type of resources used ( e
g. , typeof GPUs, internal cluster, or cloud provider)?
[Yes] See URL in abstract
If you are using existing assets ( e
g. , code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] The code and data are public
(e) Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[No] No use of personally identifiable information orobjectionable content. [Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
[Table] (b) Did you describe any potential participant risks, with links to Institutional Review [Table] Board (IRB) approvals, if applicable?
[N/A](c)
Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
[Table]