Attacking deep networks with surrogate-based adversarial black-box methods is easy

1 INTRODUCTION . The paper that introduced adversarial examples in computer vision ( Szegedy et al. , 2014 ) also initiated the study of their transfer across models . Accordingly , the first papers to propose black-box attacks ( Papernot et al. , 2017 ; Liu et al. , 2017 ) did so within a transfer-based framework , typically using a simple “ fast-gradient ” optimisation with gradients supplied by a surrogate model . This allowed them to assume nothing about the attacked models except their input and output domains and , of course , that they would return the top predicted class for a given input image . This stood in contrast to white-box ( or “ first-order oracle ” ) approaches , e.g . Goodfellow et al . ( 2014 ) ; Moosavi-Dezfooli et al . ( 2016 ) ; Carlini & Wagner ( 2017 ) , which assumed full knowledge of the victim model ( notably its gradients ) while typically seeking to minimise the norm of the adversary and/or the compute time spent finding it . The transfer-based black-box methods instead assumed comparable knowledge of the surrogate , whether provided directly , trained on a similar data distribution , or constructed iteratively by matching the observed behaviour of the victim over repeated queries to it . While these methods demonstrated that some amount of transfer between different networks was feasible , the field widely acknowledged that they fell short of reliable success , and sought alternatives . Chen et al . ( 2017 ) and Bhagoji et al . ( 2018 ) introduced a modification of the threat model : they assumed the victim to provide not only the top class prediction , but also the scores/confidences of the candidate classes . This relaxation allowed for doing away with the need to acquire or construct surrogates of the victim , instead using numerical query-based methods to estimate the victim ’ s gradients directly . These approaches and their many descendants are called “ score-based ” attacks , operating within the “ zeroth-order oracle ” setting . The common problem they all face , noted in the seminal works above , is that direct numerical estimation of gradients is linear in the dimension of the input space . Depending on the input image resolution , this ranges from being costly to being infeasible . Thus , a core issue in score-based attacks is the need to limit the query count without compromising the quality of the gradient estimate : in Chen et al . ( 2017 ) alone , proposals included coordinate descent , basis transformation , coarse-to-fine optimisation , and priors on the gradient . The line of work which followed ( Ilyas et al. , 2018 ; 2019 ; Li et al. , 2019 ; Liu et al. , 2020 ; Guo et al. , 2019b ; Tu et al. , 2019 ) explored ways to improve the fundamental approach by experimenting with different optimisation methods and priors . Strikingly , and perhaps surprisingly , one of the most successful efforts was SimBA ( Guo et al. , 2019a ) , for “ Simple Black-Box Attack ” . Its name derives from the fact that it is an especially basic form of coordinate descent which achieves competitive results despite its deliberately simple approach : the greedy sequential assignment of signs to fixed-length steps along orthogonal basis directions . Though query-driven score-based black-box attacks were proposed as an alternative to transfer methods , the two approaches are not incompatible with one another . They are actually complementary : a query-based strategy can benefit from search directions that are more promising a priori , and a transfer-based strategy can benefit from flexible optimisation that allows it to dynamically correct approximation errors and consider alternative hypotheses . This was first suggested in contemporaneous works by Cheng et al . ( 2019 ) and Guo et al . ( 2019b ) , which were followed by Tashiro et al . ( 2020 ) and Yang et al . ( 2020 ) . In this paper , we argue that the core intuition of this branch of literature is sound . However , we also argue that when combining transfer- and query-based approaches , it is crucial to recognise that the failure of surrogate gradient transfer is generally overestimated , even by approaches which leverage it . Transfer generally succeeds : it merely requires an occasional source of appropriately chosen alternative hypotheses in order to avoid optimisation failure . It is this core insight that enables us to propose a simple but powerful new algorithm which achieves state-of-the-art results against the most modern competing methods on this problem , under their own experimental setup of minimising the number of queries required to identify ` 2-norm-bounded adversarial perturbations . We call our approach “ GFCS : Gradient First , Coimage Second ” . Our approach is straightforward in both principle and implementation . Each iteration , it attempts up to two fixed-length steps along a given direction , as in Guo et al . ( 2019a ) . So long as it yields a successful update , this candidate direction is simply the normalised adversarial loss gradient of a surrogate model . If the method is unable to accept any step in this direction , it instead samples randomly weighted sums of surrogate class-score gradients until one such direction yields a successful update , at which point it reverts to using surrogate adversarial loss gradients as before . All of the “ backup ” direction proposals are thus drawn from the row space of the surrogate ’ s Jacobian , which is called the “ coimage ” of the linearised surrogate . Intuitively , this represents the subspace of directions at the current point that the surrogate has any response to at all . As we demonstrate , GFCS is able to overwhelmingly rely on surrogate loss gradients , a fact which accounts for its extreme efficiency . The coimage search , on the other hand , plays a diversifying role which is necessary for achieving the method ’ s high success rate , but without imposing an excessive cost . This is possible because its dimension is bounded above by that of the network ’ s output space , and is thus generally drastically lower in dimension than the input image embedding space , e.g . 1000-D vs. 268203-D for the common ImageNet Inception-v3 implementation . Beyond the success of the attack in its own right , we note that existence of such a fast and effective attack which relies entirely on surrogate output gradients indicates that the studied networks are , in an important sense , very similar to one another . Given this fact and its implications for working with these networks under this threat model , we believe the community should carefully evaluate how it wants to approach this problem in the future . 2 RELATED WORK . Cheng et al . ( 2019 ) and Guo et al . ( 2019b ) were the first to , in their words , “ bridge the gap between the transfer-based attacks and the query-based ones ” in the score-based black-box context , noting that earlier methods “ often suffer from low attack success rates ( in the transfer case ) or poor query efficiency ( in the query-based case ) since it is non-trivial to estimate the gradient in a high-dimensional space with limited information ” . Guo et al . ( 2019b ) did this by supplying a bandits optimiser ( as used in Ilyas et al . ( 2019 ) ) with surrogate gradient estimates at each iteration , diversifying the proposed search directions through the use of test-time dropout and drop-layer . Cheng et al . ( 2019 ) presented a variant of random gradient-free ( RGF ) optimisation using the surrogate gradient as a biasing prior ( P-RGF ) and featuring a dynamically estimated bias parameter , albeit one whose optimal value itself depends on the unknown target gradient . The Output-Diversified Sampling ( ODS ) approach of Tashiro et al . ( 2020 ) sampled gradients of randomly weighted logit sums over multiple surrogate models in order to generate search directions for variations on SimBA , RGF , and some Boundary Attack ( Brendel et al. , 2018 ) variants , demonstrating improvement over all of the base methods . Yang et al . ( 2020 ) also proposed a surrogate-enhanced version of SimBA ( “ SimBA++ ” ) , sampling candidate pixels to perturb according to the distribution specified by the corresponding magnitudes of the surrogate gradient components , as well as making periodic attempts to directly transfer surrogate gradients estimated using smoothing and momentum . When combined with “ High Order Gradient Approximation ” , a method for dynamically updating the surrogate by matching the first- and zeroth-order victim model behaviour , it is dubbed “ Learnable Black-Box Attack ” ( LeBA ) . In addition to the above , we review relevant works on black-box attacks , particularly score-based methods that use ( possibly learned ) prior information and/or alternative optimisers . When they feature a relevant concept , we also include variations of the Boundary Attack method of Brendel et al . ( 2018 ) , which avoid requiring scores in exchange for much higher query counts : Frequency/scale priors : As noted by ( Chen et al. , 2017 ; Bhagoji et al. , 2018 ) , in order to be practical , any black-box method that relies on gradient estimation must have an approach to effectively reduce the intrinsic dimension of the estimate space . Guo et al . ( 2019a ; 2020 ) use low-frequency DCT dimensions for this purpose . The Boundary Attack variant of Brunner et al . ( 2019 ) uses Perlin noise similarly , and the Gaussian smoothing of the gradient in Yang et al . ( 2020 ) is conceptually related . Also closely related to the low-frequency prior is the use of spatial downsampling , whether applied to the image or its gradient , and whether implemented through pooling , interpolation , or striding : Tu et al . ( 2019 ) ; Ilyas et al . ( 2019 ) ; Li et al . ( 2019 ) ; Wang et al . ( 2021 ) all involve a version of it . Coarse-to-fine approaches , in which results at a coarser scale either solve the problem or initialise the optimiser of a finer one , appear in Moon et al . ( 2019 ) ; Al-Dujaili & O ’ Reilly ( 2020 ) . Data distribution modelling : Li et al . ( 2019 ) assumes the ability to parametrically model an adversarial distribution in the vicinity of the input point . Dolatabadi et al . ( 2020 ) replaces this parametric model with a normalising flow model trained on clean data . Tu et al . ( 2019 ) represents the data using an autoencoder and conducts the attack in its latent space . Huang & Zhang ( 2020 ) likewise conducts a latent-space attack , but in that of an encoder-decoder network trained to output adversaries on a source network . Ru et al . ( 2019 ) attempts to learn the latent-space dimension itself . Attention : Wang et al . ( 2021 ) uses the output of CAM ( Zhou et al. , 2016 ) on a proxy net as a map of pixels to attack . In Brunner et al . ( 2019 ) , a similar map is derived from the difference between the adversarial and original images , and used to weight the sampled perturbation elementwise . Gradient/feature priors : The use of gradient priors in Guo et al . ( 2019b ) ; Cheng et al . ( 2019 ) ; Tashiro et al . ( 2020 ) ; Yang et al . ( 2020 ) is discussed above , and the transfer-based approaches Papernot et al . ( 2017 ) ; Liu et al . ( 2017 ) are of course fundamentally based on this . Besides these , the method of Brunner et al . ( 2019 ) uses the gradient of a surrogate model to bias the orthogonal perturbation used in Boundary Attack , stating , “ even surrogate models that are too weak for direct transfer attacks can be used in our framework ” . Yan et al . ( 2021 ) , also a Boundary Attack variant , attempts to learn to mimic the gradients of a victim net using a customised pre-trained policy network . Huang & Zhang ( 2020 ) learns a latent space of transferable adversarial features from its surrogate . Andriushchenko et al . ( 2020 ) supplies a bespoke feature bank which essentially transfers empirical domain knowledge ( i.e . features determined to be likely to fool CNNs ) , both in the initialisation ( vertical stripes ) and the feature choice ( homogeneous squares in the ` ∞ case , pairs of opposite-signed “ decaying peaks ” in the ` 2 ) . Sahu et al . ( 2020 ) attempts to perform a “ black-box FGSM ” by learning correlations between components of the loss function gradient within a Gaussian Markov random field framework . The “ gradient priors ” of Ilyas et al . ( 2019 ) essentially amount to momentum and downsampling , and do not represent the sort of “ flexible gradient transfer ” we are otherwise discussing here . Optimisation algorithms : Again , a standard optimisation framework for black-box score-based attacks is ascent on gradients estimated via numerical derivatives in guessed directions , possibly using enhancements such as priors and/or simplifications such as coordinate ascent . We consider the aforementioned SimBA variants ( including the method we present ) to essentially be of this type . Variations on or alternatives to this approach have included bandits ( Ilyas et al. , 2019 ; Guo et al. , 2019b ) , Natural Evolution Strategies ( NES ) ( Li et al. , 2019 ; Huang & Zhang , 2020 ; Dolatabadi et al. , 2020 ) evolutionary algorithms ( Liu et al. , 2020 ; Wang et al. , 2021 ) , Bayesian optimisation ( Ru et al. , 2019 ; Shukla et al. , 2019 ) , and training of a policy network by REINFORCE ( Yan et al. , 2021 ) . Al-Dujaili & O ’ Reilly ( 2020 ) uses a custom “ flip/revert ” approach based on checking the overall effect of grouped sign changes to pixel perturbations . Assuming submodularity , Moon et al . ( 2019 ) uses the max-heap-based “ local search algorithm ” to alternate between greedily inserting and removing elements defining a partition between oppositely signed perturbation pixels . Shi et al . ( 2019 ) comprises a set of heuristics meant to reduce the norms of attacks based on transfer from surrogate models : many of these are in principle applicable to attack vectors obtained otherwise .

Summary: This work presents an adversarial attack in the black-box setting where an adversary has access to only logit outputs from the target model. The proposed approach combines the ideas from transfer-based attacks and zeroth-order optimization methods for score-based attack methods. Experiments on multiple architectures show that the proposed approach is query efficient while achieving similar success rates compared to recent approaches.