Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations

1 Introduction . Adversarial attacks are designed to expose vulnerabilities of Deep Neural Networks ( DNNs ) . With real-world applications of video classification based on DNNs emerging [ 1–3 ] , a key question that arises is “ what type of adversarial inputs can mislead , and thus render video classification networks vulnerable ? ” Designing such adversarial attacks not only helps expose security flaws of DNNs , but can also potentially stimulate the design of more robust video classification models . Adversarial attacks against image classification models have been studied in both white-box [ 4–8 ] and black-box [ 9–13 ] settings . In the white-box setting , an adversary has full access to the model under attack , including its parameters and training settings ( hyper-parameters , training data , etc . ) In the black-box setting , an adversary only has partial information about the victim model , such as the predicted labels of the model . In the case of video classification models , adversarial attacks in both white-box and black-box settings have garnered some interest [ 14–22 ] , although the body of work here is more limited than the case of image classification models . A common black-box attack paradigm is query-based , wherein the attacker can send queries to the victim model to collect the corresponding predicted labels , and thereby estimate the gradients needed ∗Equal contribution . Corresponding author : Shasha Li ( sli057 @ ucr.edu ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . for curating the adversarial examples . Unlike static images , videos naturally include additional information from the temporal dimension . This high dimensionality ( i.e. , sequence of frames instead of one image ) poses challenges to black-box adversarial attacks against video classification models ; in particular , significantly more queries are typically needed for estimating the gradients for crafting adversarial samples [ 19–22 ] . [ 19 ] reduces the number of queries by adding perturbations on the patch level instead of at the pixel level ; [ 20 , 21 ] propose to add perturbations only on key pixels . [ 22 ] considers the intrinsic differences between images and videos ( i.e. , the temporal dimension ) , and proposes to use the optical-flow of clean videos as the motion prior for adversarial video generation . Similar to [ 22 ] , we also explicitly consider the temporal dimension of video . However , rather than fixing the temporal search space using the motion prior of clean videos , we propose to parameterize the temporal structure of the space with geometric transformations . This results in a better structured and reduced search space , which allows us to generate successful attacks with much fewer queries in black-box settings than the state-of-the-art methods , including [ 22 ] . Contributions . In this paper , we propose a novel query-efficient black-box attack algorithm against video classification models . Due to the extra temporal dimension , generating video perturbations by searching for effective gradients remains a challenging task given the exceedingly large search space . These gradients are estimated by searching for ‘ directions ’ that maximize the probability of the victim model mis-classifying the crafted inputs . Our approach drastically reduces this large search space by defining this space with a small set of parameters that describe the geometric progression of gradients in the temporal dimension , resulting in a reduced and temporally structured search space . Conceptually , this parameterization of the temporal structure of the search space is performed using geometric transformations ( e.g . affine transformations ) . We refer to our algorithm as Geometrically TRAnsformed Perturbations , or GEO-TRAP . Despite this surprisingly simple strategy , GEO-TRAP outperforms existing black-box video adversarial attack methods by significant margins ( ∼ 1.8 % improvement in attack success rate with ∼ 73.55 % fewer queries for targeted attacks in comparison to the state-of-the-art [ 22 ] on the Jester dataset [ 23 ] ) . 2 Related Works . In this section , we review different black-box adversarial attacks strategies , and categorize our proposed method with respect to state-of-the-art black-box attacks designed for video classifiers . In most real-world attacks , the adversary only has partial information about the victim models , such as the predicted labels . In such black-box settings , the adversary can first attack a local surrogate model and then transfer these attacks to the target victim model [ 24 , 25 ] , formally called as transferabilitybased black-box attack . Alternatively , they may estimate the adversarial gradient with zero-order optimization methods such as Finite Differences ( FD ) or Natural Evolution Strategies ( NES ) by querying the victim model [ 12 , 13 , 26 ] , which is called query-based black-box attack . GEO-TRAP falls under the category of query-based black-box attacks ( designed for videos ) . Whilst several white-box attacks have been proposed for video classification models [ 14–18 ] , blackbox video attacks are relatively under explored . PATCHATTACK ( V-BAD ) [ 19 ] is the first to propose a black-box video attack framework which uses a hybrid attack strategy of first generating initial perturbations for each video frame by attacking a local image classifier , and then updating the perturbations by querying the victim model . Compared to PATCHATTACK [ 19 ] , GEO-TRAP does not require training a local classifier . PATCHATTACK [ 19 ] crafts video perturbations by treating each frame as a separate image , but reduces the search space of the gradient estimation by morphing the perturbations in patches/partitions . However , its attack performance has been shown to be inferior to that of a more recent approach [ 22 ] ( discussed below ) . HEURISTICATTACK [ 20 ] uses a querybased attack strategy , and reduces the search space by generating adversarial perturbations only on heuristically selected key frames and salient regions . SPARSEATTACK [ 21 ] reduces the search space by adding perturbations only on key frames using a reinforcement learning based framework . MOTION-SAMPLER ATTACK [ 22 ] proposed a query-based attack strategy that utilizes a motion excited sampler to obtain motion-aware perturbation prior by using the optical-flow of the clean video . This motion-aware prior reduces the search space for gradients resulting in fewer queries . Similar to [ 22 ] but different from [ 19–21 ] , GEO-TRAP explicitly considers the temporal dimension of video in order to search for effective gradients . However unlike [ 22 ] , GEO-TRAP does not fix the temporal structure of the search space using a pre-computed fixed motion prior , but parameterize it with simple geometric transformations . These black-box video attack methods are summarized in Table 1 . 3 Attacking via Geometrically TRAnsformed Perturbations ( GEO-TRAP ) . Notation . We denote the tuple of a video clip and its corresponding label as ( x , y ) , which represents a data-point in the distribution X . Each video sample x ∈ RT×H×W×C has T frames of H height , W width , and C channels . We denote the victim video classification model as fθ : X → Y , where θ represents the model ’ s parameters learned from the training subset of X , via a mapping to the label space Y . We further assume X consists of videos from |Y| = K categories . To make the perturbations imperceptible to humans , we impose the perturbation budget ρmax with the ‖ · ‖p norm . Throughout this paper , we consider ‖ · ‖∞ norm following [ 15 , 19 , 22 ] ( the method can be extended to p = 1 , 2 norms ) . To constrain ‖ · ‖∞ of perturbation below a budget ρmax , we use the clip ( · ) function to keep the perturbation pixel value in [ −ρmax , ρmax ] . The function sign ( · ) extracts the sign of given input variable . The superscript i , throughout the paper , denotes the iteration i . The subscript t denotes the frame index . For clarity , we represent vectors/tensors with the bold font and scalars with the regular font . Problem Statement . We consider the scenario of attacking a standard video classification model using a query-based paradigm under black-box settings ( assuming no access to θ nor the training subset of X ) . Specifically , we aim to craft perturbed videos xadv with imperceptible differences from x , in order to alter the decision of the target model fθ via multiple queries to guide the gradient estimation . This problem can be mathematically formulated as follows . argmin xadv L ( fθ ( xadv ) , y ) s.t . ‖xadv − x‖∞ ≤ ρmax ( 1 ) L ( fθ ( xadv ) , y ) is the objective function , capturing the similarity between the classifier ’ s output and the ground truth label y , and varies with different attack goals ( targeted or untargeted ) . The challenge is to obtain xadv with as few queries as possible by estimating gradient g ? = ∇xadvL ( fθ ( xadv ) , y ) , which is unknown in the considered black-box setting . Overview of GEO-TRAP . We propose a novel iterative video perturbation framework that follows the principle of the Basic Iterative Method [ 27 ] in order to fool fθ under ‖ · ‖∞ norm as follows . x ( 0 ) adv = x , x ( i ) adv = clip ( x ( i−1 ) adv − h sign ( g ( i ) ) ) ( 2 ) where h is a hyperparameter and g ( i ) is the gradient estimated by querying the black-box victim model at the ith iteration using our proposed GEO-TRAP algorithm . As shown in ( 2 ) , effective perturbations rely on the guidance of the gradient g ( i ) . Therefore , efficiently estimating g ( i ) is at the core of GEO-TRAP for successfully subverting video classifiers . We execute the following two steps in each iteration to estimate g ( i ) . 1 . For any input video x ( i ) , a random noise tensor rframe ∈ RH×W×C and a set of geometric transformation parameters Φwarp ∈ RT×D are chosen with each element sampled from a standard normal distribution . D represents the number of parameters needed for the geometric transformation of a single frame ( details are provided in Section 3.2 ) . In this setup , our search space for estimating g ( i ) consists of rframe and Φwarp . 2 . We then warp rframe with Φwarp to get the candidate direction π = [ r1 , r2 , · · · , rT ] ∈ RT×H×W×C ( see Algorithm 2.1 TRANS-WARP ) . π is then employed to compute a gradient estimator ∆ by querying the black-box victim model with a standard gradient estimation algorithm ( see Algorithm 2 GRAD-EST ) . The gradient estimator ∆ is then used to update g ( i ) . < latexit sha1_base64= '' 3Of2xn3T08TlGL32t2ebhz3oFkg= '' > AAACBXicbVC7SgNBFL3rM8ZX1NJmMAixMOwGUcuAjWUE84BkE2Ynk2TIzOwyMyuEZWt/wVZ7O7H1O2z9EifJFibxwIXDOfdyLieIONPGdb+dtfWNza3t3E5+d2//4LBwdNzQYawIrZOQh6oVYE05k7RumOG0FSmKRcBpMxjfTf3mE1WahfLRTCLqCzyUbMAINlbqdgKRDNNuUmKX3kXaKxTdsjsDWiVeRoqQodYr/HT6IYkFlYZwrHXbcyPjJ1gZRjhN851Y0wiTMR7StqUSC6r9ZPZ1is6t0keDUNmRBs3UvxcJFlpPRGA3BTYjvexNxf+8dmwGt37CZBQbKsk8aBBzZEI0rQD1maLE8IklmChmf0VkhBUmxha1kBKING9L8ZYrWCWNStm7LlcerorVSlZPDk7hDErgwQ1U4R5qUAcCCl7gFd6cZ+fd+XA+56trTnZzAgtwvn4BMM+Ymw== < /latexit > The overall attack strategy is summarized in Algorithm 1 and pictorially illustrated in Figure 1 . Since in Step 2 above , the gradient estimation ( i.e. , GRAD-EST ) procedure includes the geometric transformation strategy ( i.e. , TRANS-WARP ) , we will next describe GRAD-EST and then move on to TRANS-WARP . For the simplicity of exposition , we drop the superscript i and shorten the loss function to L ( fθ ( xadv ) , y ) to L ( as the model parameters θ remain unchanged ) in rest of this section . 3.1 GEO-TRAP Gradient Estimation ( GRAD-EST ) . Let g ? = ∇xL be the ideal value of the gradient of L at x , required to create xadv in ( 2 ) . To find an efficient estimator g for g ? , a ( new ) surrogate loss ` ( g ) = −〈g ? , g〉 is defined such that the estimator g has a sufficiently large inner product with the actual gradient g ? ( g is normalized to a unit vector ; we ignore the normalization operation for ease of explanation ) . The loss function definition and the algorithm to estimate g follow [ 13 ] . As g ? is unknown in the black-box setting , this surrogate loss function can be estimated as ` ( g ) = −〈g ? , g〉 = −〈∇xL , g〉 ≈ − L ( x+ g , y ) − L ( x , y ) . ( 3 ) To iteratively estimate g , we need to , in turn , estimate the gradient of ` ( g ) , i.e. , ∆ = ∇g ` ( g ) . With antithetic sampling [ 28 ] , ∆ can be estimated as ∆ = ` ( g + δπ ) − ` ( g − δπ ) δ π , ( 4 ) where δ is a small number adjusting the magnitude of the loss variation and π ∈ RT×H×W×C is a random candidate direction . Our core contribution lies in the fact that instead of randomly sampling π in the search space [ 13 ] , we reduce the search dimensionality by warping a randomly sampled tensor rframe ∈ RH×W×C with another randomly sampled geometric ( e.g. , affine ) transformation parameter tensor Φwarp ∈ RT×D to get π . The search space is then reduced from T ×H ×W ×C to ( H×W ×C ) + ( T ×D ) and D is a relatively small number , D H×W ×C . Withw1 = g+ δπ and w2 = g − δπ and combining ( 3 ) with ( 4 ) , we get ∆ = L ( x+ w2 , y ) − L ( x+ w1 , y ) δ π . ( 5 ) Note that by querying the victim model fθ with x + w1 , we are able to retrieve the value of L ( x+ w1 , y ) ; similarly we can obtain the value of L ( x+ w2 , y ) ( L ( · ) is defined following [ 17 ] . ) . In summary , we estimate ∆ with these two queries to the victim model . The resulting algorithm for estimating gradient of∇g ` or ∆ for consequently estimating g is shown in Algorithm 2 . Eventually at every iteration , we use ∆ to update g by applying a one-step gradient descent as g ← g − η∆ , where η is a hyperparameter to update g. This updated g is later used to obtain xadv using ( 2 ) . Algorithm 1 : GEO-TRAP : Query-based Iterative attack for Video Classifiers Input : video x , corresponding label y , step-size η for updating the gradient , step-size h for updating adversarial video . Output : adversarial video xadv 1 Initialize : x ( 0 ) = x , g ( 0 ) = 0 , i = 1 2 while argmax ( fθ ( x ( i ) ) ) = y do 3 ∆ = Grad-Est ( x ( i−1 ) , g ( i−1 ) , y ) / * Gradient Estimation * / 4 g ( i ) ← g ( i−1 ) − η∆ 5 x ( i ) ← clip ( x ( i−1 ) − hsign ( g ( i ) ) ) 6 i← i+ 1 7 end 8 return xadv = x ( i )

Summary: This paper studies adversarial attacks against black-box video classification models. By adopting geometric transformation, the proposed GEO-TRAP reduced the search space for attacking the models. The attack process against black-box models is similar as the extant methods, by estimating the gradient of the loss function. By conducting experiments, the proposed method could use fewer queries to achieve better attack performance.