Attacking deep networks with surrogate-based adversarial black-box methods is easy

1 INTRODUCTION . The paper that introduced adversarial examples in computer vision ( Szegedy et al. , 2014 ) also initiated the study of their transfer across models . Accordingly , the first papers to propose black-box attacks ( Papernot et al. , 2017 ; Liu et al. , 2017 ) did so within a transfer-based framework , typically using a simple “ fast-gradient ” optimisation with gradients supplied by a surrogate model . This allowed them to assume nothing about the attacked models except their input and output domains and , of course , that they would return the top predicted class for a given input image . This stood in contrast to white-box ( or “ first-order oracle ” ) approaches , e.g . Goodfellow et al . ( 2014 ) ; Moosavi-Dezfooli et al . ( 2016 ) ; Carlini & Wagner ( 2017 ) , which assumed full knowledge of the victim model ( notably its gradients ) while typically seeking to minimise the norm of the adversary and/or the compute time spent finding it . The transfer-based black-box methods instead assumed comparable knowledge of the surrogate , whether provided directly , trained on a similar data distribution , or constructed iteratively by matching the observed behaviour of the victim over repeated queries to it . While these methods demonstrated that some amount of transfer between different networks was feasible , the field widely acknowledged that they fell short of reliable success , and sought alternatives . Chen et al . ( 2017 ) and Bhagoji et al . ( 2018 ) introduced a modification of the threat model : they assumed the victim to provide not only the top class prediction , but also the scores/confidences of the candidate classes . This relaxation allowed for doing away with the need to acquire or construct surrogates of the victim , instead using numerical query-based methods to estimate the victim ’ s gradients directly . These approaches and their many descendants are called “ score-based ” attacks , operating within the “ zeroth-order oracle ” setting . The common problem they all face , noted in the seminal works above , is that direct numerical estimation of gradients is linear in the dimension of the input space . Depending on the input image resolution , this ranges from being costly to being infeasible . Thus , a core issue in score-based attacks is the need to limit the query count without compromising the quality of the gradient estimate : in Chen et al . ( 2017 ) alone , proposals included coordinate descent , basis transformation , coarse-to-fine optimisation , and priors on the gradient . The line of work which followed ( Ilyas et al. , 2018 ; 2019 ; Li et al. , 2019 ; Liu et al. , 2020 ; Guo et al. , 2019b ; Tu et al. , 2019 ) explored ways to improve the fundamental approach by experimenting with different optimisation methods and priors . Strikingly , and perhaps surprisingly , one of the most successful efforts was SimBA ( Guo et al. , 2019a ) , for “ Simple Black-Box Attack ” . Its name derives from the fact that it is an especially basic form of coordinate descent which achieves competitive results despite its deliberately simple approach : the greedy sequential assignment of signs to fixed-length steps along orthogonal basis directions . Though query-driven score-based black-box attacks were proposed as an alternative to transfer methods , the two approaches are not incompatible with one another . They are actually complementary : a query-based strategy can benefit from search directions that are more promising a priori , and a transfer-based strategy can benefit from flexible optimisation that allows it to dynamically correct approximation errors and consider alternative hypotheses . This was first suggested in contemporaneous works by Cheng et al . ( 2019 ) and Guo et al . ( 2019b ) , which were followed by Tashiro et al . ( 2020 ) and Yang et al . ( 2020 ) . In this paper , we argue that the core intuition of this branch of literature is sound . However , we also argue that when combining transfer- and query-based approaches , it is crucial to recognise that the failure of surrogate gradient transfer is generally overestimated , even by approaches which leverage it . Transfer generally succeeds : it merely requires an occasional source of appropriately chosen alternative hypotheses in order to avoid optimisation failure . It is this core insight that enables us to propose a simple but powerful new algorithm which achieves state-of-the-art results against the most modern competing methods on this problem , under their own experimental setup of minimising the number of queries required to identify ` 2-norm-bounded adversarial perturbations . We call our approach “ GFCS : Gradient First , Coimage Second ” . Our approach is straightforward in both principle and implementation . Each iteration , it attempts up to two fixed-length steps along a given direction , as in Guo et al . ( 2019a ) . So long as it yields a successful update , this candidate direction is simply the normalised adversarial loss gradient of a surrogate model . If the method is unable to accept any step in this direction , it instead samples randomly weighted sums of surrogate class-score gradients until one such direction yields a successful update , at which point it reverts to using surrogate adversarial loss gradients as before . All of the “ backup ” direction proposals are thus drawn from the row space of the surrogate ’ s Jacobian , which is called the “ coimage ” of the linearised surrogate . Intuitively , this represents the subspace of directions at the current point that the surrogate has any response to at all . As we demonstrate , GFCS is able to overwhelmingly rely on surrogate loss gradients , a fact which accounts for its extreme efficiency . The coimage search , on the other hand , plays a diversifying role which is necessary for achieving the method ’ s high success rate , but without imposing an excessive cost . This is possible because its dimension is bounded above by that of the network ’ s output space , and is thus generally drastically lower in dimension than the input image embedding space , e.g . 1000-D vs. 268203-D for the common ImageNet Inception-v3 implementation . Beyond the success of the attack in its own right , we note that existence of such a fast and effective attack which relies entirely on surrogate output gradients indicates that the studied networks are , in an important sense , very similar to one another . Given this fact and its implications for working with these networks under this threat model , we believe the community should carefully evaluate how it wants to approach this problem in the future . 2 RELATED WORK . Cheng et al . ( 2019 ) and Guo et al . ( 2019b ) were the first to , in their words , “ bridge the gap between the transfer-based attacks and the query-based ones ” in the score-based black-box context , noting that earlier methods “ often suffer from low attack success rates ( in the transfer case ) or poor query efficiency ( in the query-based case ) since it is non-trivial to estimate the gradient in a high-dimensional space with limited information ” . Guo et al . ( 2019b ) did this by supplying a bandits optimiser ( as used in Ilyas et al . ( 2019 ) ) with surrogate gradient estimates at each iteration , diversifying the proposed search directions through the use of test-time dropout and drop-layer . Cheng et al . ( 2019 ) presented a variant of random gradient-free ( RGF ) optimisation using the surrogate gradient as a biasing prior ( P-RGF ) and featuring a dynamically estimated bias parameter , albeit one whose optimal value itself depends on the unknown target gradient . The Output-Diversified Sampling ( ODS ) approach of Tashiro et al . ( 2020 ) sampled gradients of randomly weighted logit sums over multiple surrogate models in order to generate search directions for variations on SimBA , RGF , and some Boundary Attack ( Brendel et al. , 2018 ) variants , demonstrating improvement over all of the base methods . Yang et al . ( 2020 ) also proposed a surrogate-enhanced version of SimBA ( “ SimBA++ ” ) , sampling candidate pixels to perturb according to the distribution specified by the corresponding magnitudes of the surrogate gradient components , as well as making periodic attempts to directly transfer surrogate gradients estimated using smoothing and momentum . When combined with “ High Order Gradient Approximation ” , a method for dynamically updating the surrogate by matching the first- and zeroth-order victim model behaviour , it is dubbed “ Learnable Black-Box Attack ” ( LeBA ) . In addition to the above , we review relevant works on black-box attacks , particularly score-based methods that use ( possibly learned ) prior information and/or alternative optimisers . When they feature a relevant concept , we also include variations of the Boundary Attack method of Brendel et al . ( 2018 ) , which avoid requiring scores in exchange for much higher query counts : Frequency/scale priors : As noted by ( Chen et al. , 2017 ; Bhagoji et al. , 2018 ) , in order to be practical , any black-box method that relies on gradient estimation must have an approach to effectively reduce the intrinsic dimension of the estimate space . Guo et al . ( 2019a ; 2020 ) use low-frequency DCT dimensions for this purpose . The Boundary Attack variant of Brunner et al . ( 2019 ) uses Perlin noise similarly , and the Gaussian smoothing of the gradient in Yang et al . ( 2020 ) is conceptually related . Also closely related to the low-frequency prior is the use of spatial downsampling , whether applied to the image or its gradient , and whether implemented through pooling , interpolation , or striding : Tu et al . ( 2019 ) ; Ilyas et al . ( 2019 ) ; Li et al . ( 2019 ) ; Wang et al . ( 2021 ) all involve a version of it . Coarse-to-fine approaches , in which results at a coarser scale either solve the problem or initialise the optimiser of a finer one , appear in Moon et al . ( 2019 ) ; Al-Dujaili & O ’ Reilly ( 2020 ) . Data distribution modelling : Li et al . ( 2019 ) assumes the ability to parametrically model an adversarial distribution in the vicinity of the input point . Dolatabadi et al . ( 2020 ) replaces this parametric model with a normalising flow model trained on clean data . Tu et al . ( 2019 ) represents the data using an autoencoder and conducts the attack in its latent space . Huang & Zhang ( 2020 ) likewise conducts a latent-space attack , but in that of an encoder-decoder network trained to output adversaries on a source network . Ru et al . ( 2019 ) attempts to learn the latent-space dimension itself . Attention : Wang et al . ( 2021 ) uses the output of CAM ( Zhou et al. , 2016 ) on a proxy net as a map of pixels to attack . In Brunner et al . ( 2019 ) , a similar map is derived from the difference between the adversarial and original images , and used to weight the sampled perturbation elementwise . Gradient/feature priors : The use of gradient priors in Guo et al . ( 2019b ) ; Cheng et al . ( 2019 ) ; Tashiro et al . ( 2020 ) ; Yang et al . ( 2020 ) is discussed above , and the transfer-based approaches Papernot et al . ( 2017 ) ; Liu et al . ( 2017 ) are of course fundamentally based on this . Besides these , the method of Brunner et al . ( 2019 ) uses the gradient of a surrogate model to bias the orthogonal perturbation used in Boundary Attack , stating , “ even surrogate models that are too weak for direct transfer attacks can be used in our framework ” . Yan et al . ( 2021 ) , also a Boundary Attack variant , attempts to learn to mimic the gradients of a victim net using a customised pre-trained policy network . Huang & Zhang ( 2020 ) learns a latent space of transferable adversarial features from its surrogate . Andriushchenko et al . ( 2020 ) supplies a bespoke feature bank which essentially transfers empirical domain knowledge ( i.e . features determined to be likely to fool CNNs ) , both in the initialisation ( vertical stripes ) and the feature choice ( homogeneous squares in the ` ∞ case , pairs of opposite-signed “ decaying peaks ” in the ` 2 ) . Sahu et al . ( 2020 ) attempts to perform a “ black-box FGSM ” by learning correlations between components of the loss function gradient within a Gaussian Markov random field framework . The “ gradient priors ” of Ilyas et al . ( 2019 ) essentially amount to momentum and downsampling , and do not represent the sort of “ flexible gradient transfer ” we are otherwise discussing here . Optimisation algorithms : Again , a standard optimisation framework for black-box score-based attacks is ascent on gradients estimated via numerical derivatives in guessed directions , possibly using enhancements such as priors and/or simplifications such as coordinate ascent . We consider the aforementioned SimBA variants ( including the method we present ) to essentially be of this type . Variations on or alternatives to this approach have included bandits ( Ilyas et al. , 2019 ; Guo et al. , 2019b ) , Natural Evolution Strategies ( NES ) ( Li et al. , 2019 ; Huang & Zhang , 2020 ; Dolatabadi et al. , 2020 ) evolutionary algorithms ( Liu et al. , 2020 ; Wang et al. , 2021 ) , Bayesian optimisation ( Ru et al. , 2019 ; Shukla et al. , 2019 ) , and training of a policy network by REINFORCE ( Yan et al. , 2021 ) . Al-Dujaili & O ’ Reilly ( 2020 ) uses a custom “ flip/revert ” approach based on checking the overall effect of grouped sign changes to pixel perturbations . Assuming submodularity , Moon et al . ( 2019 ) uses the max-heap-based “ local search algorithm ” to alternate between greedily inserting and removing elements defining a partition between oppositely signed perturbation pixels . Shi et al . ( 2019 ) comprises a set of heuristics meant to reduce the norms of attacks based on transfer from surrogate models : many of these are in principle applicable to attack vectors obtained otherwise .

Summary: This paper is devoted to the problem of generating adversarial examples to the predictions of machine learning models. That is, given an instance predicted by a target ML model one way, the paper aims at detecting another instance that is relatively close to the given one but such that the prediction of the model for the new instance is different. The paper proposes a novel adversarial attack algorithm, which behaves as twofold: (1) it tries to apply iterative ascent in the direction of the surrogate's loss gradient until it can and then (2) samples alternative directions from the coimage of the surrogate's local linearisation. The new approach is claimed to be a fast and practical attack as it significantly outperforms the competition in terms of the number of queries needed to generate adversarial examples successfully.