On Success and Simplicity: A Second Look at Transferable Targeted Attacks

1 Introduction . Deep neural networks have achieved remarkable performance in various machine learning tasks , but are known to be vulnerable to adversarial attacks [ 1 ] . A key property of adversarial attacks that makes them critical in realistic , black-box scenarios is their transferability [ 2 , 3 ] . Current work on adversarial transferability has achieved great success for non-targeted attacks [ 4–12 ] , while several initial attempts [ 3 , 4 , 13 ] at targeted transferability have shown its extreme difficulty . Targeted transferability is known to be much more challenging and worth exploring since it can raise more critical concerns by fooling models into predicting a chosen , highly dangerous target class . However , so far state-of-the-art results can only be secured by resource-intensive transferable attacks [ 14–16 ] . Specifically , the FDA approach [ 14 , 15 ] is based on modeling layer-wise feature distributions by training target-class-specific auxiliary classifiers on large-scale labeled data , and then optimizing adversarial perturbations using these auxiliary classifiers from across the deep feature space . The TTP approach [ 16 ] is based on training target-class-specific Generative Adversarial Networks ( GANs ) through global and local distribution matching , and then using the trained generator to directly generate perturbations on any given input image . In this paper , we take a second , thorough look at current research on targeted transferability . Our main contribution is the finding that simple transferable attacks [ 4 , 6 , 8 ] that require neither model 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . training nor additional data can actually achieve surprisingly strong targeted transferability . We argue that this insight has been overlooked mainly because current research has unreasonably restricted the attack convergence by only using a small number of iterations ( see detailed discussion in Section 3 ) . Another key contribution of our work is , for the first time , demonstrating the general superiority of a very simple logit loss , which even outperforms the resource-intensive state of the art . In order to validate the general effectiveness of simple transferable attacks , in Section 4.1 , we conduct extensive experiments in a wide range of transfer scenarios . We test the commonly adopted single-model and ensemble transfer scenarios , but also introduce three new scenarios that are more challenging and realistic : an ensemble transfer scenario with little model similarity , a worse-case scenario with low-ranked target classes , and also a real-world attack on the Google Cloud Vision API . Experimental results in these new scenarios suggest that evaluation in only the commonly adopted , easy scenarios can not reveal the actual strength of different attacks , and may cause misleading comparative results . Additional experiments in Section 4.2 have shown the better performance of the simple transferable attacks than the state-of-the-art resource-intensive approaches . Finally , in Section 4.3 , inspired by the observation that the generated perturbations themselves reflect specific target semantics , we use the simple Logit attack to generate targeted Universal Adversarial Perturbations ( UAPs ) in a data-free manner . In contrast , recent advances in targeted UAPs [ 16–19 ] have inevitably relied on large-scale optimization over additional data . Overall , we hope our analysis of the weakness of commonly adopted attack settings and transfer scenarios will inspire a more meaningful evaluation on targeted transferability . 2 Related Work . In this section , we review existing simple transferable attacks ( Section 2.1 ) , and also recent resourceintensive transferable attacks ( Section 2.2 ) . Finally , we discuss related work on generating universal adversarial perturbations . 2.1 Simple Transferable Attacks . We refer to transferable attacks that require neither model training nor additional data , but only use iterative optimization on a single ( original ) image as simple transferable attacks . Simple transferable attacks have been extensively studied in the non-targeted case [ 4–12 ] , and also attempted in the targeted case [ 3 , 4 , 20 ] . These attacks are commonly built up on the well-known Iterative-Fast Gradient Sign Method ( I-FGSM ) [ 21 , 22 ] , which can be formulated as : x′0 = x , x ′ i+1 = x ′ i − α · sign ( ∇xJ ( x′i , yt ) ) , ( 1 ) where x′i denotes the perturbed image in the i-th iteration , and yt is the target class label . In order to ensure the imperceptibility , the added perturbations are restricted with respect to some Lp distance , i.e. , satisfying ‖x′ − x‖p ≤ . Current transferable attack methods have commonly adopted the L∞ distance , but can also be easily adapted to the L2 distance based on an L2 normalization [ 23 ] . For the loss function J ( · , · ) , most simple transferable attacks have adopted the Cross-Entropy ( CE ) loss . However , the CE loss has been recently shown to be insufficient in the targeted case due to its decreasing gradient problem [ 20 ] . To address this problem , the authors in [ 20 ] have proposed the Po+Trip loss , in which the Poincaré distance was used to adapt the gradients ’ magnitude : LPo = d ( u , v ) = arccosh ( 1 + δ ( u , v ) ) , δ ( u , v ) = 2 · ‖u− v‖22 ( 1− ‖u‖22 ) ( 1− ‖v‖22 ) , u = l ( x′ ) ‖l ( x′ ) ‖ , v = max { v − ξ , 0 } , ( 2 ) where u is the normalized logit vector and v is the one-hot vector with respect to the target class . ξ = 10−5 is a small constant to ensure numerical stability . The following triplet loss is also integrated for pushing the image away from the original class while pulling it into the target class : LTrip = [ D ( l ( x ′ ) , yt ) −D ( l ( x′ ) , yo ) + γ ] + , D ( l ( x′ ) , y ) = 1− ‖l ( x′ ) · y‖1 ‖l ( x′ ) ‖2‖y‖2 . ( 3 ) The overall loss function is then formulated as LPo+Trip = LPo + λLTrip . Note that in the original work , Po+Trip was evaluated in the commonly adopted , easy ensemble transfer scenario , which only involves models with similar architectures . In addition to devising new loss functions , there are other transfer methods [ 4 , 6 , 8 , 9 ] developed based on the assumption that preventing the attack optimization from overfitting to the specific source model can improve transferability . Such transfer methods can be easily plugged into different attacks without modifications , in contrast to the above methods that need to apply new attack loss functions . In this paper , we consider three [ 4 , 6 , 8 ] of such transfer methods that have been widely used in the literature , as described in the following text . Momentum Iterative-FGSM ( MI-FGSM ) [ 4 ] integrates a momentum term , which accumulates previous gradients in order to achieve more stable update directions . It can be expressed as : gi+1 = µ · gi + ∇xJ ( x′i , yt ) ‖∇xJ ( x′i , yt ) ‖1 , x′i+1 = x ′ i − α · sign ( gi ) , ( 4 ) where gi is the accumulated gradients at the i-th iteration , and µ is a decay factor . Another similar technique that instead uses the Nesterov accelerated gradient was explored in [ 9 ] . Translation Invariant-FGSM ( TI-FGSM ) [ 6 ] randomly translates the input image during attack optimization in order to prevent the attack from overfitting to the specific source model . This approach is inspired by the data augmentation techniques used for preventing overfitting in normal model training . Instead of calculating gradients for multiple translated images separately , the authors have proposed an approximate solution to accelerate the implementation . It is achieved by directly computing locally smoothed gradients on the original image via convolution with a kernel : x′i+1 = x ′ i − α · sign ( W ∗ ∇xJ ( x′i , yt ) ) , ( 5 ) where W is the convolution kernel used for smoothing . TI-FGSM was originally designed for boosting transferability with adversarially-trained models as target models and has been recently shown that a smaller kernel size should be used when transferring to normally-trained models [ 12 ] . Diverse Input-FGSM ( DI-FGSM ) [ 8 ] follows a similar idea to TI-FGSM , but applies random resizing and padding for data augmentation . Another important difference is that DI-FGSM randomizes augmentation parameters over iterations rather than fixing them as in TI-FGSM . The attack optimization of DI-FGSM can be formulated as : x′i+1 = x ′ i − α · sign ( ∇xJ ( T ( x′i , p ) , yt ) ) , ( 6 ) where the stochastic transformation T ( x′i , p ) is implemented with probability p at each iteration . In Section 4.1 , we demonstrate that simple transfer attacks with these three transfer methods can actually achieve surprisingly strong targeted transferability in a wide range of transfer scenarios . 2.2 Resource-Intensive Transferable Attacks . Due to the broad consensus that achieving targeted transferability is extremely difficult , recent researchers have resorted to resource-intensive approaches that require training target-class-specific models on large-scale additional data . Specifically , the Feature Distribution Attack ( FDA ) [ 14 ] follows the same attack pipeline as the above simple transferable attacks , but requires auxiliary classifiers that have been trained on additional labeled data as part of the source model . Each auxiliary classifier is a small , binary , one-versus-all classifier trained for a specific target class at a specific layer . That is to say , the number of auxiliary classifiers is the number of layers that are probed multiplied by the number of target classes that are required to model [ 14 ] . The attack loss function of FDA can be formulated as : LFDA = J ( Fl ( x′ ) , yt ) − η ‖Fl ( x′ ) −Fl ( x ) ‖2 ‖Fl ( x ) ‖2 , ( 7 ) where each auxiliary classifiers Fl ( · ) can model the probability that a feature map at layer l is from a specific target class yt . FDA ( N ) +xent [ 15 ] extends FDA by aggregating features from L layers and also incorporating the cross-entropy loss H ( · , · ) of the original network F ( · ) . The loss function of FDA ( N ) +xent can be expressed as : LFDA ( N ) +xent = ∑ l∈L λl ( LFDA + γH ( F ( x′ ) , yt ) ) , where ∑ l∈L λl = 1 . ( 8 ) Very recently , TTP [ 16 ] has achieved state-of-the-art targeted transferability by directly generating perturbations using target-class-specific GANs that have been trained via matching the distributions of perturbations and a specific target class both globally and locally . Specifically , the global distribution matching is achieved by minimizing the Kullback Leibler ( KL ) divergence , and the local distribution matching is by enforcing the neighbourhood similarity . In order to further boost the performance , data augmentation techniques , such as image rotation , crop resize , horizontal flip , color jittering and gray-scale transformation , have been applied during model training . We refer the readers to [ 16 ] for more technical details of TTP . These two transferable attacks , FDA ( N ) +xent and TTP , are resource intensive due to the use of largescale model training and additional data . However , in Section 4.2 , we show that simple transferable attacks , which require neither model training nor additional data , can actually achieve even better performance than them .

Summary: This paper proposed a very interesting observation that the simple logit attack can consistently achieve higher targeted transferability than SOTA transferable attacks. The key insight is to allow the simple logit-based transferable attacks to converge. Extensive results presented well supported the claims.