Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations

1 Introduction . Adversarial attacks are designed to expose vulnerabilities of Deep Neural Networks ( DNNs ) . With real-world applications of video classification based on DNNs emerging [ 1–3 ] , a key question that arises is “ what type of adversarial inputs can mislead , and thus render video classification networks vulnerable ? ” Designing such adversarial attacks not only helps expose security flaws of DNNs , but can also potentially stimulate the design of more robust video classification models . Adversarial attacks against image classification models have been studied in both white-box [ 4–8 ] and black-box [ 9–13 ] settings . In the white-box setting , an adversary has full access to the model under attack , including its parameters and training settings ( hyper-parameters , training data , etc . ) In the black-box setting , an adversary only has partial information about the victim model , such as the predicted labels of the model . In the case of video classification models , adversarial attacks in both white-box and black-box settings have garnered some interest [ 14–22 ] , although the body of work here is more limited than the case of image classification models . A common black-box attack paradigm is query-based , wherein the attacker can send queries to the victim model to collect the corresponding predicted labels , and thereby estimate the gradients needed ∗Equal contribution . Corresponding author : Shasha Li ( sli057 @ ucr.edu ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . for curating the adversarial examples . Unlike static images , videos naturally include additional information from the temporal dimension . This high dimensionality ( i.e. , sequence of frames instead of one image ) poses challenges to black-box adversarial attacks against video classification models ; in particular , significantly more queries are typically needed for estimating the gradients for crafting adversarial samples [ 19–22 ] . [ 19 ] reduces the number of queries by adding perturbations on the patch level instead of at the pixel level ; [ 20 , 21 ] propose to add perturbations only on key pixels . [ 22 ] considers the intrinsic differences between images and videos ( i.e. , the temporal dimension ) , and proposes to use the optical-flow of clean videos as the motion prior for adversarial video generation . Similar to [ 22 ] , we also explicitly consider the temporal dimension of video . However , rather than fixing the temporal search space using the motion prior of clean videos , we propose to parameterize the temporal structure of the space with geometric transformations . This results in a better structured and reduced search space , which allows us to generate successful attacks with much fewer queries in black-box settings than the state-of-the-art methods , including [ 22 ] . Contributions . In this paper , we propose a novel query-efficient black-box attack algorithm against video classification models . Due to the extra temporal dimension , generating video perturbations by searching for effective gradients remains a challenging task given the exceedingly large search space . These gradients are estimated by searching for ‘ directions ’ that maximize the probability of the victim model mis-classifying the crafted inputs . Our approach drastically reduces this large search space by defining this space with a small set of parameters that describe the geometric progression of gradients in the temporal dimension , resulting in a reduced and temporally structured search space . Conceptually , this parameterization of the temporal structure of the search space is performed using geometric transformations ( e.g . affine transformations ) . We refer to our algorithm as Geometrically TRAnsformed Perturbations , or GEO-TRAP . Despite this surprisingly simple strategy , GEO-TRAP outperforms existing black-box video adversarial attack methods by significant margins ( ∼ 1.8 % improvement in attack success rate with ∼ 73.55 % fewer queries for targeted attacks in comparison to the state-of-the-art [ 22 ] on the Jester dataset [ 23 ] ) . 2 Related Works . In this section , we review different black-box adversarial attacks strategies , and categorize our proposed method with respect to state-of-the-art black-box attacks designed for video classifiers . In most real-world attacks , the adversary only has partial information about the victim models , such as the predicted labels . In such black-box settings , the adversary can first attack a local surrogate model and then transfer these attacks to the target victim model [ 24 , 25 ] , formally called as transferabilitybased black-box attack . Alternatively , they may estimate the adversarial gradient with zero-order optimization methods such as Finite Differences ( FD ) or Natural Evolution Strategies ( NES ) by querying the victim model [ 12 , 13 , 26 ] , which is called query-based black-box attack . GEO-TRAP falls under the category of query-based black-box attacks ( designed for videos ) . Whilst several white-box attacks have been proposed for video classification models [ 14–18 ] , blackbox video attacks are relatively under explored . PATCHATTACK ( V-BAD ) [ 19 ] is the first to propose a black-box video attack framework which uses a hybrid attack strategy of first generating initial perturbations for each video frame by attacking a local image classifier , and then updating the perturbations by querying the victim model . Compared to PATCHATTACK [ 19 ] , GEO-TRAP does not require training a local classifier . PATCHATTACK [ 19 ] crafts video perturbations by treating each frame as a separate image , but reduces the search space of the gradient estimation by morphing the perturbations in patches/partitions . However , its attack performance has been shown to be inferior to that of a more recent approach [ 22 ] ( discussed below ) . HEURISTICATTACK [ 20 ] uses a querybased attack strategy , and reduces the search space by generating adversarial perturbations only on heuristically selected key frames and salient regions . SPARSEATTACK [ 21 ] reduces the search space by adding perturbations only on key frames using a reinforcement learning based framework . MOTION-SAMPLER ATTACK [ 22 ] proposed a query-based attack strategy that utilizes a motion excited sampler to obtain motion-aware perturbation prior by using the optical-flow of the clean video . This motion-aware prior reduces the search space for gradients resulting in fewer queries . Similar to [ 22 ] but different from [ 19–21 ] , GEO-TRAP explicitly considers the temporal dimension of video in order to search for effective gradients . However unlike [ 22 ] , GEO-TRAP does not fix the temporal structure of the search space using a pre-computed fixed motion prior , but parameterize it with simple geometric transformations . These black-box video attack methods are summarized in Table 1 . 3 Attacking via Geometrically TRAnsformed Perturbations ( GEO-TRAP ) . Notation . We denote the tuple of a video clip and its corresponding label as ( x , y ) , which represents a data-point in the distribution X . Each video sample x ∈ RT×H×W×C has T frames of H height , W width , and C channels . We denote the victim video classification model as fθ : X → Y , where θ represents the model ’ s parameters learned from the training subset of X , via a mapping to the label space Y . We further assume X consists of videos from |Y| = K categories . To make the perturbations imperceptible to humans , we impose the perturbation budget ρmax with the ‖ · ‖p norm . Throughout this paper , we consider ‖ · ‖∞ norm following [ 15 , 19 , 22 ] ( the method can be extended to p = 1 , 2 norms ) . To constrain ‖ · ‖∞ of perturbation below a budget ρmax , we use the clip ( · ) function to keep the perturbation pixel value in [ −ρmax , ρmax ] . The function sign ( · ) extracts the sign of given input variable . The superscript i , throughout the paper , denotes the iteration i . The subscript t denotes the frame index . For clarity , we represent vectors/tensors with the bold font and scalars with the regular font . Problem Statement . We consider the scenario of attacking a standard video classification model using a query-based paradigm under black-box settings ( assuming no access to θ nor the training subset of X ) . Specifically , we aim to craft perturbed videos xadv with imperceptible differences from x , in order to alter the decision of the target model fθ via multiple queries to guide the gradient estimation . This problem can be mathematically formulated as follows . argmin xadv L ( fθ ( xadv ) , y ) s.t . ‖xadv − x‖∞ ≤ ρmax ( 1 ) L ( fθ ( xadv ) , y ) is the objective function , capturing the similarity between the classifier ’ s output and the ground truth label y , and varies with different attack goals ( targeted or untargeted ) . The challenge is to obtain xadv with as few queries as possible by estimating gradient g ? = ∇xadvL ( fθ ( xadv ) , y ) , which is unknown in the considered black-box setting . Overview of GEO-TRAP . We propose a novel iterative video perturbation framework that follows the principle of the Basic Iterative Method [ 27 ] in order to fool fθ under ‖ · ‖∞ norm as follows . x ( 0 ) adv = x , x ( i ) adv = clip ( x ( i−1 ) adv − h sign ( g ( i ) ) ) ( 2 ) where h is a hyperparameter and g ( i ) is the gradient estimated by querying the black-box victim model at the ith iteration using our proposed GEO-TRAP algorithm . As shown in ( 2 ) , effective perturbations rely on the guidance of the gradient g ( i ) . Therefore , efficiently estimating g ( i ) is at the core of GEO-TRAP for successfully subverting video classifiers . We execute the following two steps in each iteration to estimate g ( i ) . 1 . For any input video x ( i ) , a random noise tensor rframe ∈ RH×W×C and a set of geometric transformation parameters Φwarp ∈ RT×D are chosen with each element sampled from a standard normal distribution . D represents the number of parameters needed for the geometric transformation of a single frame ( details are provided in Section 3.2 ) . In this setup , our search space for estimating g ( i ) consists of rframe and Φwarp . 2 . We then warp rframe with Φwarp to get the candidate direction π = [ r1 , r2 , · · · , rT ] ∈ RT×H×W×C ( see Algorithm 2.1 TRANS-WARP ) . π is then employed to compute a gradient estimator ∆ by querying the black-box victim model with a standard gradient estimation algorithm ( see Algorithm 2 GRAD-EST ) . The gradient estimator ∆ is then used to update g ( i ) . < latexit sha1_base64= '' 3Of2xn3T08TlGL32t2ebhz3oFkg= '' > AAACBXicbVC7SgNBFL3rM8ZX1NJmMAixMOwGUcuAjWUE84BkE2Ynk2TIzOwyMyuEZWt/wVZ7O7H1O2z9EifJFibxwIXDOfdyLieIONPGdb+dtfWNza3t3E5+d2//4LBwdNzQYawIrZOQh6oVYE05k7RumOG0FSmKRcBpMxjfTf3mE1WahfLRTCLqCzyUbMAINlbqdgKRDNNuUmKX3kXaKxTdsjsDWiVeRoqQodYr/HT6IYkFlYZwrHXbcyPjJ1gZRjhN851Y0wiTMR7StqUSC6r9ZPZ1is6t0keDUNmRBs3UvxcJFlpPRGA3BTYjvexNxf+8dmwGt37CZBQbKsk8aBBzZEI0rQD1maLE8IklmChmf0VkhBUmxha1kBKING9L8ZYrWCWNStm7LlcerorVSlZPDk7hDErgwQ1U4R5qUAcCCl7gFd6cZ+fd+XA+56trTnZzAgtwvn4BMM+Ymw== < /latexit > The overall attack strategy is summarized in Algorithm 1 and pictorially illustrated in Figure 1 . Since in Step 2 above , the gradient estimation ( i.e. , GRAD-EST ) procedure includes the geometric transformation strategy ( i.e. , TRANS-WARP ) , we will next describe GRAD-EST and then move on to TRANS-WARP . For the simplicity of exposition , we drop the superscript i and shorten the loss function to L ( fθ ( xadv ) , y ) to L ( as the model parameters θ remain unchanged ) in rest of this section . 3.1 GEO-TRAP Gradient Estimation ( GRAD-EST ) . Let g ? = ∇xL be the ideal value of the gradient of L at x , required to create xadv in ( 2 ) . To find an efficient estimator g for g ? , a ( new ) surrogate loss ` ( g ) = −〈g ? , g〉 is defined such that the estimator g has a sufficiently large inner product with the actual gradient g ? ( g is normalized to a unit vector ; we ignore the normalization operation for ease of explanation ) . The loss function definition and the algorithm to estimate g follow [ 13 ] . As g ? is unknown in the black-box setting , this surrogate loss function can be estimated as ` ( g ) = −〈g ? , g〉 = −〈∇xL , g〉 ≈ − L ( x+ g , y ) − L ( x , y ) . ( 3 ) To iteratively estimate g , we need to , in turn , estimate the gradient of ` ( g ) , i.e. , ∆ = ∇g ` ( g ) . With antithetic sampling [ 28 ] , ∆ can be estimated as ∆ = ` ( g + δπ ) − ` ( g − δπ ) δ π , ( 4 ) where δ is a small number adjusting the magnitude of the loss variation and π ∈ RT×H×W×C is a random candidate direction . Our core contribution lies in the fact that instead of randomly sampling π in the search space [ 13 ] , we reduce the search dimensionality by warping a randomly sampled tensor rframe ∈ RH×W×C with another randomly sampled geometric ( e.g. , affine ) transformation parameter tensor Φwarp ∈ RT×D to get π . The search space is then reduced from T ×H ×W ×C to ( H×W ×C ) + ( T ×D ) and D is a relatively small number , D H×W ×C . Withw1 = g+ δπ and w2 = g − δπ and combining ( 3 ) with ( 4 ) , we get ∆ = L ( x+ w2 , y ) − L ( x+ w1 , y ) δ π . ( 5 ) Note that by querying the victim model fθ with x + w1 , we are able to retrieve the value of L ( x+ w1 , y ) ; similarly we can obtain the value of L ( x+ w2 , y ) ( L ( · ) is defined following [ 17 ] . ) . In summary , we estimate ∆ with these two queries to the victim model . The resulting algorithm for estimating gradient of∇g ` or ∆ for consequently estimating g is shown in Algorithm 2 . Eventually at every iteration , we use ∆ to update g by applying a one-step gradient descent as g ← g − η∆ , where η is a hyperparameter to update g. This updated g is later used to obtain xadv using ( 2 ) . Algorithm 1 : GEO-TRAP : Query-based Iterative attack for Video Classifiers Input : video x , corresponding label y , step-size η for updating the gradient , step-size h for updating adversarial video . Output : adversarial video xadv 1 Initialize : x ( 0 ) = x , g ( 0 ) = 0 , i = 1 2 while argmax ( fθ ( x ( i ) ) ) = y do 3 ∆ = Grad-Est ( x ( i−1 ) , g ( i−1 ) , y ) / * Gradient Estimation * / 4 g ( i ) ← g ( i−1 ) − η∆ 5 x ( i ) ← clip ( x ( i−1 ) − hsign ( g ( i ) ) ) 6 i← i+ 1 7 end 8 return xadv = x ( i )

Summary: This paper proposes a new method for constructing black-box adversarial attacks against video classifiers. Such attacks are usually hard to mount (moreso than attacks against image classifiers) since the dimension of the input is very large (T x C x H x W) instead of just (C x H x W), which increases the complexity of the corresponding gradient estimation problem. Previous work considered reducing this query complexity with various heuristics (e.g., sparsity/only attacking a few frames, or leveraging optical flow)---here, the authors propose an alternative prior. Namely, the proposed method learns a single C x H x W signal, as well as a parameterized (typically affine) transform for each frame (so another T x 6 parameters, for the affine case). This greatly reduces query complexity and is shown to improve upon previous more heuristic priors in practice.