Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets

1 INTRODUCTION . In deep neural networks ( DNNs ) , a skip connection builds a short-cut from a shallow layer to a deep layer by connecting the input of a convolutional block ( also known as the residual module ) directly to its output . While different layers of a neural network learn different “ levels ” of features , skip connections can help preserve low-level features and avoid performance degradation when adding more layers . This has been shown to be crucial for building very deep and powerful DNNs such as ResNet ( He et al. , 2016a ; b ) , WideResNet ( Zagoruyko & Komodakis , 2016 ) , DenseNet ( Huang et al. , 2017 ) and ResNeXt ( Xie et al. , 2017 ) . In the meantime , despite their superior performance , DNNs have been found extremely vulnerable to adversarial examples ( or attacks ) , which are input examples slightly perturbed with an intention to fool the network to make a wrong prediction ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Ma et al. , 2018 ; Bai et al. , 2019 ; Wang et al. , 2019 ; 2020 ) . Adversarial examples often appear imperceptible to human observers , and are transferable across different models ( Liu et al. , 2017 ) . This has raised security concerns on the deployment of DNNs in security critical scenarios , such as face recognition ( Sharif et al. , 2016 ) , autonomous driving ( Evtimov et al. , 2018 ) , video analysis ( Jiang et al. , 2019 ) and medical diagnosis ( Ma et al. , 2019 ) . Adversarial examples can be crafted following either a white-box setting ( the adversary has full access to the target model ) or a black-box setting ( the adversary has no information of the target model ) . White-box methods such as Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2014 ) , Basic Iterative Method ( BIM ) ( Kurakin et al. , 2016 ) , Projected Gradient Decent ( PGD ) ( Madry et al. , 2018 ) and Carlini and Wagner ( CW ) ( Carlini & Wagner , 2017 ) often suffer from low transferability †Correspondence to : Yisen Wang ( eewangyisen @ gmail.com ) in a black-box setting , thus posing only limited threats to DNN models which are usually kept secret in practice ( Dong et al. , 2018 ; Xie et al. , 2019 ) . Several techniques have been proposed to improve the transferability of black-box attacks crafted on a surrogate model , such as momentum boosting ( Dong et al. , 2018 ) , diverse input ( Xie et al. , 2019 ) and translation invariance ( Dong et al. , 2019 ) . Although these techniques are effective , they ( as well as white-box methods ) all treat the entire network ( either the target model or the surrogate model ) as a single component while ignore its inner architectural characteristics . The question of whether or not the DNN architecture itself can expose more transferability of adversarial attacks is an unexplored problem . In this paper , we identify one such weakness about the skip connections used by many state-of-theart DNNs . We first conduct a toy experiment with the BIM attack and ResNet-18 on the ImageNet validation dataset ( Deng et al. , 2009 ) to investigate how skip connections affect the adversarial strength of attacks crafted on the network . At each of the last 3 skip connections and residual modules of ResNet-18 , we illustrate the success rate of attacks crafted using gradients backpropagate through either the skip connection or the residual module in Figure 1 . As can be observed , the success rate drops more drastically whenever using gradients from a residual module instead of the skip connection . This implies that gradients from the skip connections are more vulnerable ( high success rate ) . In addition , we surprisingly find that skip connections expose more transferable information . For example , the black-box success rate was even improved from 52.52 % to 62.10 % when the attack skips the last two residual modules ( following the path in green color ) . Motivated by the above observations , in this paper , we propose the Skip Gradient Method ( SGM ) to generate adversarial examples using gradients more from the skip connections rather than the residual modules . In particular , SGM utilizes a decay factor to reduce gradients from the residual modules . We find that this simple adjustment on the gradient flow can generate highly transferable adversarial examples , and the more skip connections in a network , the more transferable are the crafted attacks . This is in sharp contrast to the design principles ( e.g. , “ going deeper ” with skip connections ) underpinning many modern DNNs . In particular , our main contributions are : • We identify one surprising property of skip connections in ResNet-like neural networks , i.e. , they allow an easy generation of highly transferable adversarial examples . • We propose the Skip Gradient Method ( SGM ) to craft adversarial examples using gradients more from the skip connections . Using a single decay factor on gradients , SGM is an appealingly simple and generic technique that can be used by any existing gradient-based attack methods . • We provide comprehensive transfer attack experiments , from different source models against 10 state-of-the-art DNNs , showing that SGM can greatly improve the transferability of crafted adversarial examples . When combined with existing transfer techniques , SGM improves the state-ofthe-art transferability benchmarks by a large margin . 2 RELATED WORK . Existing adversarial attacks can be categorized into two groups : 1 ) white-box attacks and 2 ) blackbox attacks . In the white-box setting , the adversary has full access to the parameters of the target model , while in the black-box setting , the target model is kept secret from the adversary . 2.1 WHITE-BOX ATTACKS . Given a clean example x with class label y and a target DNN model f , the goal of an adversary is to find an adversarial example xadv that fools the network into making an incorrect prediction ( eg . f ( xadv ) 6= y ) , while still remaining in the -ball centered at x ( eg . ‖xadv − x‖∞ ≤ ) . Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2014 ) . FGSM perturbs clean example x for one step by the amount of along the gradient direction : xadv = x + · sign ( ∇x ` ( f ( x ) , y ) ) . ( 1 ) The Basic Iterative Method ( BIM ) ( Kurakin et al. , 2016 ) is an iterative version of FGSM that perturbs for T steps with step size /T . Projected Gradient Descent ( PGD ) ( Madry et al. , 2018 ) . PGD perturbs normal example x for T steps with smaller step size . After each step of perturbation , PGD projects the adversarial example back onto the -ball of x , if it goes beyond the -ball : xt+1adv = Π ( xtadv + α · sign ( ∇x ` ( f ( xtadv ) , y ) ) ) , ( 2 ) where Π ( · ) is the projection operation . Different to BIM , PGD allows step size α > /T . There are also other types of white-box attacks including sparsity-based methods such as Jacobianbased Saliency Map Attack ( JSMA ) ( Papernot et al. , 2016 ) , sparse attack ( Modas et al. , 2019 ) , onepixel attack ( Su et al. , 2019 ) , and optimization-based methods such as Carlini and Wagner ( CW ) ( Carlini & Wagner , 2017 ) and elastic-net ( EAD ) ( Chen et al. , 2018 ) . 2.2 BLACK-BOX ATTACKS . Black-box attacks can be generated by either attacking a surrogate model or using gradient estimation methods in combination with queries to the target model . Gradient estimation methods estimate the gradients of the target model using black-box optimization methods such as Finite Differences ( FD ) ( Chen et al. , 2017 ; Bhagoji et al. , 2018 ) or Natural Evolution Strategies ( NES ) ( Ilyas et al. , 2018 ; Jiang et al. , 2019 ) . These methods all require a large number of queries to the target model , which not only reduces efficiency but also potentially exposes the attack . Alternatively , black-box adversarial examples can be crafted on a surrogate model then applied to attack the target model . Although the white-box methods can be directly applied on the surrogate model , they are far less effective in the black-box setting ( Dong et al. , 2018 ; Xie et al. , 2019 ) . Several transfer techniques have been proposed to improve the transferability of black-box attacks . Momentum Iterative boosting ( MI ) ( Dong et al. , 2018 ) . MI incorporates a momentum term into the gradient to boost the transferability : xt+1adv = Π ( xtadv + α · sign ( gt+1 ) ) , gt+1 = µ · gt + ∇x ` ( f ( x t adv ) , y ) ‖∇x ` ( f ( xtadv ) , y ) ‖1 , ( 3 ) where gt is the adversarial gradient at the t-th step , α = /T is the step size for a total of T steps , µ is a decay factor , and ‖ · ‖1 is the L1 norm . Diverse Input ( DI ) ( Xie et al. , 2019 ) . DI proposes to craft adversarial exampels using gradient with respect to the randomly-transformed input example : xt+1adv = Π ( xtadv + α · sign ( ∇x ` ( f ( H ( xtadv ; p ) ) , y ) ) ) , ( 4 ) where H ( xtadv ; p ) is a stochastic transformation function on x t adv for a given probability p. Translation Invariant ( TI ) ( Dong et al. , 2019 ) . TI targets to evade robustly trained DNNs by generating adversarial examples that are less sensitive to the discriminative regions of the surrogate model . More specifically , TI computes the gradients with respect to a set of translated versions of the original input : xt+1adv = Π ( xtadv + α · sign ( W ∗ ∇x ` ( f ( xtadv ) , y ) ) ) , ( 5 ) where W is a predefined kernel ( e.g. , uniform , linear , and Gaussian ) matrix of size ( 2k + 1 ) ( 2k + 1 ) ( k being the maximal number of pixels to shift ) . This kernel convolution is equivalent to the weighted sum of gradients over ( 2k + 1 ) 2 number of shifted input examples . Furthermore , there are other studies focusing on intermediate feature representations . For example , Activation Attack ( Inkawhich et al. , 2019 ) drives the activation of a specified layer on a given image towards the layer of a target image , to yield a highly transferable targeted example . Intermediate Level Attack ( Huang et al. , 2019 ) attempts to fine-tune an existing adversarial example for greater black-box transferability by increasing its perturbation on a pre-specified layer of the source model . Although the above transfer techniques are effective , they ( including white-box attacks ) either 1 ) treat the network ( either the surrogate model or the target model ) as a single component or 2 ) only use the intermediate layer output of the network . In other words , they do not directly consider the effects of different DNN architectural characteristics . Li et al . ( 2018 ) investigated the use of skip connections and dropout layers for sampling networks , which generates a huge set of ghost networks to perform an ensemble attack . Here , we focus on the architectural property of skip connections from the gradient view without modifying or generating any extra networks .

Summary: This paper proposes a modification to standard Projected Gradient Descent to improve transferability of adversarial examples, when the source model is a ResNet-like model containing skip connections. The method, Skip Gradient Method (SGM) modifies the backwards pass to scale down the gradient computed in each residual branch of the model, before these gradients are combined with the gradient from the skip connection. This thus upweights the gradients from the skip connections as opposed to residual modules. The paper demonstrates significant improvements in the single-model black-box transfer setting, against a variety of undefended and defended models.