On Success and Simplicity: A Second Look at Transferable Targeted Attacks

1 Introduction . Deep neural networks have achieved remarkable performance in various machine learning tasks , but are known to be vulnerable to adversarial attacks [ 1 ] . A key property of adversarial attacks that makes them critical in realistic , black-box scenarios is their transferability [ 2 , 3 ] . Current work on adversarial transferability has achieved great success for non-targeted attacks [ 4–12 ] , while several initial attempts [ 3 , 4 , 13 ] at targeted transferability have shown its extreme difficulty . Targeted transferability is known to be much more challenging and worth exploring since it can raise more critical concerns by fooling models into predicting a chosen , highly dangerous target class . However , so far state-of-the-art results can only be secured by resource-intensive transferable attacks [ 14–16 ] . Specifically , the FDA approach [ 14 , 15 ] is based on modeling layer-wise feature distributions by training target-class-specific auxiliary classifiers on large-scale labeled data , and then optimizing adversarial perturbations using these auxiliary classifiers from across the deep feature space . The TTP approach [ 16 ] is based on training target-class-specific Generative Adversarial Networks ( GANs ) through global and local distribution matching , and then using the trained generator to directly generate perturbations on any given input image . In this paper , we take a second , thorough look at current research on targeted transferability . Our main contribution is the finding that simple transferable attacks [ 4 , 6 , 8 ] that require neither model 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . training nor additional data can actually achieve surprisingly strong targeted transferability . We argue that this insight has been overlooked mainly because current research has unreasonably restricted the attack convergence by only using a small number of iterations ( see detailed discussion in Section 3 ) . Another key contribution of our work is , for the first time , demonstrating the general superiority of a very simple logit loss , which even outperforms the resource-intensive state of the art . In order to validate the general effectiveness of simple transferable attacks , in Section 4.1 , we conduct extensive experiments in a wide range of transfer scenarios . We test the commonly adopted single-model and ensemble transfer scenarios , but also introduce three new scenarios that are more challenging and realistic : an ensemble transfer scenario with little model similarity , a worse-case scenario with low-ranked target classes , and also a real-world attack on the Google Cloud Vision API . Experimental results in these new scenarios suggest that evaluation in only the commonly adopted , easy scenarios can not reveal the actual strength of different attacks , and may cause misleading comparative results . Additional experiments in Section 4.2 have shown the better performance of the simple transferable attacks than the state-of-the-art resource-intensive approaches . Finally , in Section 4.3 , inspired by the observation that the generated perturbations themselves reflect specific target semantics , we use the simple Logit attack to generate targeted Universal Adversarial Perturbations ( UAPs ) in a data-free manner . In contrast , recent advances in targeted UAPs [ 16–19 ] have inevitably relied on large-scale optimization over additional data . Overall , we hope our analysis of the weakness of commonly adopted attack settings and transfer scenarios will inspire a more meaningful evaluation on targeted transferability . 2 Related Work . In this section , we review existing simple transferable attacks ( Section 2.1 ) , and also recent resourceintensive transferable attacks ( Section 2.2 ) . Finally , we discuss related work on generating universal adversarial perturbations . 2.1 Simple Transferable Attacks . We refer to transferable attacks that require neither model training nor additional data , but only use iterative optimization on a single ( original ) image as simple transferable attacks . Simple transferable attacks have been extensively studied in the non-targeted case [ 4–12 ] , and also attempted in the targeted case [ 3 , 4 , 20 ] . These attacks are commonly built up on the well-known Iterative-Fast Gradient Sign Method ( I-FGSM ) [ 21 , 22 ] , which can be formulated as : x′0 = x , x ′ i+1 = x ′ i − α · sign ( ∇xJ ( x′i , yt ) ) , ( 1 ) where x′i denotes the perturbed image in the i-th iteration , and yt is the target class label . In order to ensure the imperceptibility , the added perturbations are restricted with respect to some Lp distance , i.e. , satisfying ‖x′ − x‖p ≤ . Current transferable attack methods have commonly adopted the L∞ distance , but can also be easily adapted to the L2 distance based on an L2 normalization [ 23 ] . For the loss function J ( · , · ) , most simple transferable attacks have adopted the Cross-Entropy ( CE ) loss . However , the CE loss has been recently shown to be insufficient in the targeted case due to its decreasing gradient problem [ 20 ] . To address this problem , the authors in [ 20 ] have proposed the Po+Trip loss , in which the Poincaré distance was used to adapt the gradients ’ magnitude : LPo = d ( u , v ) = arccosh ( 1 + δ ( u , v ) ) , δ ( u , v ) = 2 · ‖u− v‖22 ( 1− ‖u‖22 ) ( 1− ‖v‖22 ) , u = l ( x′ ) ‖l ( x′ ) ‖ , v = max { v − ξ , 0 } , ( 2 ) where u is the normalized logit vector and v is the one-hot vector with respect to the target class . ξ = 10−5 is a small constant to ensure numerical stability . The following triplet loss is also integrated for pushing the image away from the original class while pulling it into the target class : LTrip = [ D ( l ( x ′ ) , yt ) −D ( l ( x′ ) , yo ) + γ ] + , D ( l ( x′ ) , y ) = 1− ‖l ( x′ ) · y‖1 ‖l ( x′ ) ‖2‖y‖2 . ( 3 ) The overall loss function is then formulated as LPo+Trip = LPo + λLTrip . Note that in the original work , Po+Trip was evaluated in the commonly adopted , easy ensemble transfer scenario , which only involves models with similar architectures . In addition to devising new loss functions , there are other transfer methods [ 4 , 6 , 8 , 9 ] developed based on the assumption that preventing the attack optimization from overfitting to the specific source model can improve transferability . Such transfer methods can be easily plugged into different attacks without modifications , in contrast to the above methods that need to apply new attack loss functions . In this paper , we consider three [ 4 , 6 , 8 ] of such transfer methods that have been widely used in the literature , as described in the following text . Momentum Iterative-FGSM ( MI-FGSM ) [ 4 ] integrates a momentum term , which accumulates previous gradients in order to achieve more stable update directions . It can be expressed as : gi+1 = µ · gi + ∇xJ ( x′i , yt ) ‖∇xJ ( x′i , yt ) ‖1 , x′i+1 = x ′ i − α · sign ( gi ) , ( 4 ) where gi is the accumulated gradients at the i-th iteration , and µ is a decay factor . Another similar technique that instead uses the Nesterov accelerated gradient was explored in [ 9 ] . Translation Invariant-FGSM ( TI-FGSM ) [ 6 ] randomly translates the input image during attack optimization in order to prevent the attack from overfitting to the specific source model . This approach is inspired by the data augmentation techniques used for preventing overfitting in normal model training . Instead of calculating gradients for multiple translated images separately , the authors have proposed an approximate solution to accelerate the implementation . It is achieved by directly computing locally smoothed gradients on the original image via convolution with a kernel : x′i+1 = x ′ i − α · sign ( W ∗ ∇xJ ( x′i , yt ) ) , ( 5 ) where W is the convolution kernel used for smoothing . TI-FGSM was originally designed for boosting transferability with adversarially-trained models as target models and has been recently shown that a smaller kernel size should be used when transferring to normally-trained models [ 12 ] . Diverse Input-FGSM ( DI-FGSM ) [ 8 ] follows a similar idea to TI-FGSM , but applies random resizing and padding for data augmentation . Another important difference is that DI-FGSM randomizes augmentation parameters over iterations rather than fixing them as in TI-FGSM . The attack optimization of DI-FGSM can be formulated as : x′i+1 = x ′ i − α · sign ( ∇xJ ( T ( x′i , p ) , yt ) ) , ( 6 ) where the stochastic transformation T ( x′i , p ) is implemented with probability p at each iteration . In Section 4.1 , we demonstrate that simple transfer attacks with these three transfer methods can actually achieve surprisingly strong targeted transferability in a wide range of transfer scenarios . 2.2 Resource-Intensive Transferable Attacks . Due to the broad consensus that achieving targeted transferability is extremely difficult , recent researchers have resorted to resource-intensive approaches that require training target-class-specific models on large-scale additional data . Specifically , the Feature Distribution Attack ( FDA ) [ 14 ] follows the same attack pipeline as the above simple transferable attacks , but requires auxiliary classifiers that have been trained on additional labeled data as part of the source model . Each auxiliary classifier is a small , binary , one-versus-all classifier trained for a specific target class at a specific layer . That is to say , the number of auxiliary classifiers is the number of layers that are probed multiplied by the number of target classes that are required to model [ 14 ] . The attack loss function of FDA can be formulated as : LFDA = J ( Fl ( x′ ) , yt ) − η ‖Fl ( x′ ) −Fl ( x ) ‖2 ‖Fl ( x ) ‖2 , ( 7 ) where each auxiliary classifiers Fl ( · ) can model the probability that a feature map at layer l is from a specific target class yt . FDA ( N ) +xent [ 15 ] extends FDA by aggregating features from L layers and also incorporating the cross-entropy loss H ( · , · ) of the original network F ( · ) . The loss function of FDA ( N ) +xent can be expressed as : LFDA ( N ) +xent = ∑ l∈L λl ( LFDA + γH ( F ( x′ ) , yt ) ) , where ∑ l∈L λl = 1 . ( 8 ) Very recently , TTP [ 16 ] has achieved state-of-the-art targeted transferability by directly generating perturbations using target-class-specific GANs that have been trained via matching the distributions of perturbations and a specific target class both globally and locally . Specifically , the global distribution matching is achieved by minimizing the Kullback Leibler ( KL ) divergence , and the local distribution matching is by enforcing the neighbourhood similarity . In order to further boost the performance , data augmentation techniques , such as image rotation , crop resize , horizontal flip , color jittering and gray-scale transformation , have been applied during model training . We refer the readers to [ 16 ] for more technical details of TTP . These two transferable attacks , FDA ( N ) +xent and TTP , are resource intensive due to the use of largescale model training and additional data . However , in Section 4.2 , we show that simple transferable attacks , which require neither model training nor additional data , can actually achieve even better performance than them .

Summary: This work introduces techniques to improve the targeted transferability of adversarial examples. The authors mainly propose to use the simple logit loss for this purpose. The transferability is evaluated in three scenarios, including a real-world attack against the Google Cloud Vision API. The technique is also shown in the context of universal adversarial perturbations.