Aug-ILA: More Transferable Intermediate Level Attacks with Augmented References

1 INTRODUCTION . Recent studies ( Szegedy et al. , 2013 ; Goodfellow et al. , 2015 ) showed that deep neural network ( DNN ) models are vulnerable to adversarial attacks , where perturbations are added to the clean data to fool the models in making erroneous classification . Such adversarial perturbations are usually crafted to be almost imperceptible by humans , yet causing apparent fluctuations in the model output . The effectiveness of adversarial attacks on deep learning models raises concerns in multiple fields , especially for security-sensitive applications . Another intriguing phenomenon is that adversarial attacks can transfer across different models ( Papernot et al. , 2016 ) . One explanation for this phenomenon is the overlapping decision boundaries shared by different models ( Liu et al. , 2017 ; Dong et al. , 2018 ) . In many black-box settings where the attacker has no access to the internal state of the model , such transferability can be exploited either to generate attacks from a surrogate model ( Zhou et al. , 2018 ) or to provide proper guidance to reduce the number of model queries ( Guo et al. , 2019 ) . While many methods set their goals to generate highly transferable attacks ( Moosavi-Dezfooli et al. , 2016 ; Wu et al. , 2020 ; Guo et al. , 2020 ) , some attempt to improve the transferability of a given adversarial example by fine-tuning it . Huang et al . ( 2019 ) proposed the Intermediate Level Attack ( ILA ) , a technique that takes an existing adversarial example as reference and boosts its black-box transferability by maximizing the projection of the intermediate feature map discrepancies between attacks . After being fine-tuned by ILA , a strong white-box attack can achieve remarkable black-box transferability , outperforming previous methods based on direct generation of transfer-based attacks ( Zhou et al. , 2018 ; Xie et al. , 2019 ) . More specifically , ILA takes three input images as references to fine-tune the attack , including a clean image , a reference attack of the clean image , and the updated image from the previous iteration . Both the clean image and reference attack remain unchanged throughout the fine-tuning process . If we view the fine-tuning process of ILA as a generalization task that attempts to make an attack effective for different models , a possible direction for improvement is to increase the diversity of the input references , such as through applying data augmentation . In this paper , we revisit ILA and explore possible room for improvement . A natural way to promote input diversity is to apply data augmentation techniques to the input references . We first evaluate the effect of typical image transformation operations in the context of ILA . Then , we look into transformations exploiting the adversarial perturbation , and apply them together with the best image transformation found previously . Specifically , we incorporate a mixture of augmentation operations : random image cropping and resizing on all input examples , reverse adversarial update on the clean image , and attack interpolation on the reference attack . Incorporating these adaptations to the original ILA , we propose the Augmented Intermediate Level Attack ( Aug-ILA ) to further enhance black-box transferability . By fine-tuning simple baseline attacks such as the Iterative Fast Gradient Sign Method ( I-FGSM ) , Aug-ILA not only outperforms ILA and its subsequent variants , but also state-of-the-art transfer-based attacks . Visualizations of the adversarial examples fine-tuned by Aug-ILA are illustrated in Figure 1 and Appendix N. Our main contributions can be summarized as follows : • We analyze the formulation of ILA and conduct extensive experiments to study the effect of some common image transformation operations as well as transformations exploiting the adversarial perturbation for augmenting the ILA references . • We propose Aug-ILA , a method that fine-tunes a given adversarial attack to reinforce its attack transferability . Aug-ILA follows the framework of ILA , but with effective augmentations introduced to its references . The resulting black-box transferability of Aug-ILA outperforms the original ILA by 18.97 % on average with perturbation budget = 0.03 ( ≈ 8/255 ) . • We identify factors affecting the transfer success rate and explain the roles of intermediate feature perturbation and augmentation in a transfer-based attack . 2 RELATED WORK . In the context of adversarial machine learning , the most common threat models are the white-box and black-box settings . The white-box setting assumes that one has access to the victim model ’ s internal state , including its gradient , parameters , training dataset , etc . The black-box setting , on the other hand , only allows querying the model with input but prohibits subsequent access to the model information . While more variations such as the grey-box and no-box settings exist , they are generally not considered in this work . Typical white-box attacks exploit the gradient of the model to generate adversarial examples . The Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2015 ) generates attacks by adding the signed gradient of the loss with respect to the image back to the image , obtaining a higher loss and possibly incorrect prediction . The Iterative Fast Gradient Sign Method ( I-FGSM , also known as BIM ) ( Kurakin et al. , 2017a ) performs FGSM iteratively while clipping the difference between the adversarial example and the original image within the perturbation budget . Projected Gradient Descent ( PGD ) ( Madry et al. , 2018 ) first initiates a random start within the perturbation budget and runs I-FGSM to update the attack . In the black-box setting , one can estimate the model information by repeatedly querying the model with varying inputs ( Chen et al. , 2017 ; Brendel et al. , 2018 ) . Recently , some novel methods have been proposed in order to reduce the number of queries ( Ilyas et al. , 2018 ; Bhagoji et al. , 2018 ; Li et al. , 2020a ) . However , the number of queries to the models is still abnormally large , making it easy to be detected by real-world systems . This turns the spotlight back to transfer-based attacks , where adversarial examples mainly rely on black-box transferability instead of consecutive attempts . 2.1 TRANSFER-BASED ATTACKS . Since strong gradient-based white-box attacks do not have high transferability by themselves ( Kurakin et al. , 2017b ) , a number of attempts have been made in the research community to improve transferability apart from increasing the attack strength . MI-FGSM . Dong et al . ( 2018 ) first introduced the Momentum Iterative Fast Gradient Sign Method ( MI-FGSM ) to incorporate momentum to stabilize the update direction in iterative attacks , resulting in more transferable attacks . DIM . The Diverse Input Method ( DI2-FGSM , or DIM ) ( Xie et al. , 2019 ) applies random resizing and zero-padding with a certain probability to the image in each iteration of I-FGSM . TIM . Dong et al . ( 2019 ) pointed out the discretion of attention map between the defended and undefended models and proposed the Translation-Invariant Method ( TIM ) by attacking an ensemble of translated images . To reduce the computational overhead , it was shown that kernel convolution can be applied to the gradient to achieve similar effects . SIM and NI-FGSM . Lin et al . ( 2020 ) introduced the scale-invariant property of DNN and proposed the Scale-Invariant Method ( SIM ) to attack an ensemble of images with a scaling factor of 0.5 on the pixel values . In the same paper , the Nesterov Iterative Fast Gradient Sign Method ( NI-FGSM ) was introduced , aiming to escape from poor local maxima with the Nesterov accelerated gradient . VMI-FGSM . Wang & He ( 2021 ) proposed variance tuning . For each iteration , the gradient is adjusted with an expectation of gradient sampled from the image ’ s neighborhood . Combining variance tuning with the composition of previous methods , denoted as the Composite Transformation Method ( CTM ) , they achieved one of the strongest transfer-based black-box attacks on defended models . A more recent work ( Wu et al. , 2021 ) replaced all input transformations with a DNN model trained to apply distortions that neutralize the adversarial noises . After preparing the denoising model , the attack is generated with the objective to sustain such distortions . All these methods suggested that adversarial transferability could be improved by augmenting the images or tuning the magnitude of gradient updates . Alternatively , some recently proposed methods exposed the surprising nature of CNN model architectures and utilized them to generate transferable attacks . Wu et al . ( 2020 ) found that attack transferability can be enhanced by scaling up the gradient in the skip connection and referred to the technique as Skip Gradient Method ( SGM ) . Guo et al . ( 2020 ) revised the linear property of DNN models and proposed LinBP to propagate backward without considering the non-linear layers . 2.2 INTERMEDIATE LEVEL ATTACK FOR FINE-TUNING ADVERSARIAL EXAMPLES . Consider an input image x , an existing adversarial attack x′ , a model F , and a function Fl that outputs the feature maps at layer l of the model . The original ILA projection loss proposed by Huang et al . ( 2019 ) minimizes the dot product : L ( x , x′ , x′′ ) = −∆y′′l ·∆y′l ( 1 ) where ∆y′l and ∆y′′l are two vectors defined as follows : ∆y′l = Fl ( x ′ ) − Fl ( x ) ( 2 ) ∆y′′l = Fl ( x ′′ ) − Fl ( x ) ( 3 ) This is equivalent to maximizing ∆y′′l ·∆y′l . The idea behind the formulation is to increase the norm without sacrificing the perturbation direction that causes misclassification . Due to the constraint of the perturbation budget , there is hardly room for fine-tuning in the image space . Instead , projection maximization is carried out in the feature space , specifically at layer l of the model . Based on the same framework , Li et al . ( 2020b ) proposed another formulation of ILA : max x′′ ( Fl ( x′′ ) − Fl ( x ) ) Tw * ( 4 ) where w * is a pre-computed parameter vector that directly maps the intermediate-level discrepancies to predict the adversarial loss , skipping the remaining model layers . One notable trick of the design is that the computation of w * involves every feature map discrepancy Fl ( x′t ) −Fl ( x ) in each iteration t during the generation of x′ . We refer to this attack as ILA++ in the rest of this paper . 3 OUR METHOD . Based on what we have observed from the previous work , adversarial attacks tend to exhibit higher transferability when more references are involved . The increase of references can be achieved by data augmentation and reusing the temporal information over multiple iterations . From Equation ( 1 ) , all three ILA input references x , x′ and x′′ are images , suggesting the possibility of applying image augmentation like that for representation learning to improve model generalization . One natural way to augment the images is to apply image transformation to the input before retrieving the intermediate feature maps . Consequently , the function Fl can be substituted by F ′l ( x ) = Fl ( T ( x ) ) ( 5 ) where T can be any image transformation function . However , unlike attacks that enforce a wrong prediction at the logit output , ILA aims to maximize the projection of feature map discrepancies . This requires perfect pixel alignment of x , x′ and x′′ , as a tiny shift in the image pixel induces misalignment in the feature map . Spatial transformations such as translation , cropping and rotation are required to be applied identically to all three ILA references . Otherwise , the optimization of ILA ends up being trivial in the sense of distorted feature discrepancies . Our claim is empirically verified in Appendix B , where the transformation T is not identical for the images . Different from augmentation that generalizes model representation , our task is to improve attack transferability , which can be interpreted as generalizing an adversarial example to succeed in attacking more models . Hence , apart from the common image processing methods , we may also explore any transformation associated with the adversarial perturbation . Intuitively , augmentation based on the adversarial perturbation can be viewed as an evaluation of multiple attacks before fine-tuning the final attack . This kind of pixel-wise augmentation can be applied to the images independently without conflicting with most spatial transformations . Algorithm 1 depicts the algorithmic details for applying these transformations together to ILA , where the clip function is to ensure that the first argument is within the range of the second and third arguments . Algorithm 1 Aug-ILA Require : x , x′ , Fl , perturbation budget , step size a , number of iterations n , a sequence of transformation functions T x′′ ← x′ for i← 1 to n do xrev ← 2x− x′ . reverse adversarial update Randomly initialize T . to ensure the image augmentations are aligned ∆y′l ← Fl ( T ( x ′ ) ) − Fl ( T ( xrev ) ) ∆y′′l ← Fl ( T ( x ′′ ) ) − Fl ( T ( xrev ) ) x′′ ← x′′ − a sign ( ∇x′′ ( −∆y′′l ·∆y ′ l ) ) x′′ ← clip ( x′′ , x− , x + ) x′′ ← clip ( x′′ , 0 , 1 ) α← ‖ ∆y′′l ‖2 ‖∆y′′l ‖2+‖∆y′l‖2 x′ ← αx′′ + ( 1− α ) x′ . attack interpolation end for return x′′

Summary: This paper introduces improvements over Transferable Intermediate-level Attacks (ILA), by incorporating data augmentation into the attack tuning stage. The data augmentations introduced include 3 kinds: simple transformations (cropping), reverse-adversarial update, and attack interpolation. An empirical study has been conducted on Imagenet, and 3 well-known architectures have been used as source models, and the attack transferability was evaluated onto 9 models using I-FGSM with $L_{inf}$ distance and 2 different perturbation sizes. Ablation studies have been conducted to demonstrate the effectiveness of hyperparameter choices, and different kinds of data augmentation used, demonstrating the effectiveness of the proposed approach compared to other ILA-based blackbox attacks.