Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations
Year: 2022

Abstract:
When compared to the image classification models, black-box adversarial attacks against video classification models have been largely understudied. This could be possible because, with video, the temporal dimension poses significant additional challenges in gradient estimation. Query-efficient black-box attacks rely on effectively estimated gradients towards maximizing the probability of misclassifying the target video. In this work, we demonstrate that such effective gradients can be searched for by parameterizing the temporal structure of the search space with geometric transformations. Specifically, we design a novel iterative algorithm Geometric TRAnsformed Perturbations (GEO-TRAP), for attacking video classification models. GEO-TRAP employs standard geometric transformation operations to reduce the search space for effective gradients by searching for a small group of parameters that define these operations. This group of parameters describes the geometric progression of gradients, resulting in a reduced and structured search space. Our algorithm inherently leads to successful perturbations with surprisingly few queries. For example, adversarial examples generated from GEO-TRAP have better attack success rates with ∼ 73.55% fewer queries compared to the state-of-the-art method for video adversarial attacks on the widely used Jester dataset. Overall, our algorithm exposes vulnerabilities of diverse video classification models and achieves new state-of-the-art results under black-box settings on two large datasets. Code is available here: https://github.com/sli057/Geo-TRAP

heading: 1 Introduction
text: Adversarial attacks are designed to expose vulnerabilities of Deep Neural Networks (DNNs). With real-world applications of video classification based on DNNs emerging [1–3], a key question that arises is “what type of adversarial inputs can mislead, and thus render video classification networks vulnerable?” Designing such adversarial attacks not only helps expose security flaws of DNNs, but can also potentially stimulate the design of more robust video classification models.
Adversarial attacks against image classification models have been studied in both white-box [4–8] and black-box [9–13] settings. In the white-box setting, an adversary has full access to the model under attack, including its parameters and training settings (hyper-parameters, training data, etc.) In the black-box setting, an adversary only has partial information about the victim model, such as the predicted labels of the model. In the case of video classification models, adversarial attacks in both white-box and black-box settings have garnered some interest [14–22], although the body of work here is more limited than the case of image classification models.
A common black-box attack paradigm is query-based, wherein the attacker can send queries to the victim model to collect the corresponding predicted labels, and thereby estimate the gradients needed
∗Equal contribution. Corresponding author: Shasha Li (sli057@ucr.edu)
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
for curating the adversarial examples. Unlike static images, videos naturally include additional information from the temporal dimension. This high dimensionality (i.e., sequence of frames instead of one image) poses challenges to black-box adversarial attacks against video classification models; in particular, significantly more queries are typically needed for estimating the gradients for crafting adversarial samples [19–22]. [19] reduces the number of queries by adding perturbations on the patch level instead of at the pixel level; [20, 21] propose to add perturbations only on key pixels. [22] considers the intrinsic differences between images and videos (i.e., the temporal dimension), and proposes to use the optical-flow of clean videos as the motion prior for adversarial video generation. Similar to [22], we also explicitly consider the temporal dimension of video. However, rather than fixing the temporal search space using the motion prior of clean videos, we propose to parameterize the temporal structure of the space with geometric transformations. This results in a better structured and reduced search space, which allows us to generate successful attacks with much fewer queries in black-box settings than the state-of-the-art methods, including [22].
Contributions. In this paper, we propose a novel query-efficient black-box attack algorithm against video classification models. Due to the extra temporal dimension, generating video perturbations by searching for effective gradients remains a challenging task given the exceedingly large search space. These gradients are estimated by searching for ‘directions’ that maximize the probability of the victim model mis-classifying the crafted inputs. Our approach drastically reduces this large search space by defining this space with a small set of parameters that describe the geometric progression of gradients in the temporal dimension, resulting in a reduced and temporally structured search space. Conceptually, this parameterization of the temporal structure of the search space is performed using geometric transformations (e.g. affine transformations). We refer to our algorithm as Geometrically TRAnsformed Perturbations, or GEO-TRAP. Despite this surprisingly simple strategy, GEO-TRAP outperforms existing black-box video adversarial attack methods by significant margins (∼ 1.8% improvement in attack success rate with ∼ 73.55% fewer queries for targeted attacks in comparison to the state-of-the-art [22] on the Jester dataset [23]).

heading: 2 Related Works
text: In this section, we review different black-box adversarial attacks strategies, and categorize our proposed method with respect to state-of-the-art black-box attacks designed for video classifiers.
In most real-world attacks, the adversary only has partial information about the victim models, such as the predicted labels. In such black-box settings, the adversary can first attack a local surrogate model and then transfer these attacks to the target victim model [24, 25], formally called as transferabilitybased black-box attack. Alternatively, they may estimate the adversarial gradient with zero-order optimization methods such as Finite Differences (FD) or Natural Evolution Strategies (NES) by querying the victim model [12, 13, 26], which is called query-based black-box attack. GEO-TRAP falls under the category of query-based black-box attacks (designed for videos).
Whilst several white-box attacks have been proposed for video classification models [14–18], blackbox video attacks are relatively under explored. PATCHATTACK (V-BAD)[19] is the first to propose a black-box video attack framework which uses a hybrid attack strategy of first generating initial perturbations for each video frame by attacking a local image classifier, and then updating the perturbations by querying the victim model. Compared to PATCHATTACK[19] , GEO-TRAP does not require training a local classifier. PATCHATTACK[19] crafts video perturbations by treating each frame as a separate image, but reduces the search space of the gradient estimation by morphing the
perturbations in patches/partitions. However, its attack performance has been shown to be inferior to that of a more recent approach [22] (discussed below). HEURISTICATTACK [20] uses a querybased attack strategy, and reduces the search space by generating adversarial perturbations only on heuristically selected key frames and salient regions. SPARSEATTACK [21] reduces the search space by adding perturbations only on key frames using a reinforcement learning based framework. MOTION-SAMPLER ATTACK [22] proposed a query-based attack strategy that utilizes a motion excited sampler to obtain motion-aware perturbation prior by using the optical-flow of the clean video. This motion-aware prior reduces the search space for gradients resulting in fewer queries. Similar to [22] but different from [19–21], GEO-TRAP explicitly considers the temporal dimension of video in order to search for effective gradients. However unlike [22], GEO-TRAP does not fix the temporal structure of the search space using a pre-computed fixed motion prior, but parameterize it with simple geometric transformations. These black-box video attack methods are summarized in Table 1.

heading: 3 Attacking via Geometrically TRAnsformed Perturbations (GEO-TRAP)
text: Notation. We denote the tuple of a video clip and its corresponding label as (x, y), which represents a data-point in the distribution X . Each video sample x ∈ RT×H×W×C has T frames of H height, W width, and C channels. We denote the victim video classification model as fθ : X → Y , where θ represents the model’s parameters learned from the training subset of X , via a mapping to the label space Y . We further assume X consists of videos from |Y| = K categories. To make the perturbations imperceptible to humans, we impose the perturbation budget ρmax with the ‖ · ‖p norm. Throughout this paper, we consider ‖ · ‖∞ norm following [15, 19, 22] (the method can be extended to p = 1, 2 norms). To constrain ‖ · ‖∞ of perturbation below a budget ρmax, we use the clip(·) function to keep the perturbation pixel value in [−ρmax, ρmax]. The function sign(·) extracts the sign of given input variable. The superscript i, throughout the paper, denotes the iteration i. The subscript t denotes the frame index. For clarity, we represent vectors/tensors with the bold font and scalars with the regular font.
Problem Statement. We consider the scenario of attacking a standard video classification model using a query-based paradigm under black-box settings (assuming no access to θ nor the training subset of X ). Specifically, we aim to craft perturbed videos xadv with imperceptible differences from x, in order to alter the decision of the target model fθ via multiple queries to guide the gradient estimation. This problem can be mathematically formulated as follows.
argmin xadv
L ( fθ(xadv), y ) s.t. ‖xadv − x‖∞ ≤ ρmax (1)
L ( fθ(xadv), y ) is the objective function, capturing the similarity between the classifier’s output and the ground truth label y, and varies with different attack goals (targeted or untargeted). The challenge is to obtain xadv with as few queries as possible by estimating gradient g? = ∇xadvL ( fθ(xadv), y ) , which is unknown in the considered black-box setting.
Overview of GEO-TRAP. We propose a novel iterative video perturbation framework that follows the principle of the Basic Iterative Method [27] in order to fool fθ under ‖ · ‖∞ norm as follows.
x (0) adv = x, x (i) adv = clip ( x (i−1) adv − h sign(g(i)) ) (2)
where h is a hyperparameter and g(i) is the gradient estimated by querying the black-box victim model at the ith iteration using our proposed GEO-TRAP algorithm. As shown in (2), effective perturbations rely on the guidance of the gradient g(i). Therefore, efficiently estimating g(i) is at the core of GEO-TRAP for successfully subverting video classifiers. We execute the following two steps in each iteration to estimate g(i).
1. For any input video x(i), a random noise tensor rframe ∈ RH×W×C and a set of geometric transformation parameters Φwarp ∈ RT×D are chosen with each element sampled from a standard normal distribution. D represents the number of parameters needed for the geometric transformation of a single frame (details are provided in Section 3.2). In this setup, our search space for estimating g(i) consists of rframe and Φwarp. 2. We then warp rframe with Φwarp to get the candidate direction π = [r1, r2, · · · , rT ] ∈ RT×H×W×C (see Algorithm 2.1 TRANS-WARP). π is then employed to compute a gradient estimator ∆ by querying the black-box victim model with a standard gradient estimation algorithm (see Algorithm 2 GRAD-EST). The gradient estimator ∆ is then used to update g(i).
<latexit sha1_base64="3Of2xn3T08TlGL32t2ebhz3oFkg=">AAACBXicbVC7SgNBFL3rM8ZX1NJmMAixMOwGUcuAjWUE84BkE2Ynk2TIzOwyMyuEZWt/wVZ7O7H1O2z9EifJFibxwIXDOfdyLieIONPGdb+dtfWNza3t3E5+d2//4LBwdNzQYawIrZOQh6oVYE05k7RumOG0FSmKRcBpMxjfTf3mE1WahfLRTCLqCzyUbMAINlbqdgKRDNNuUmKX3kXaKxTdsjsDWiVeRoqQodYr/HT6IYkFlYZwrHXbcyPjJ1gZRjhN851Y0wiTMR7StqUSC6r9ZPZ1is6t0keDUNmRBs3UvxcJFlpPRGA3BTYjvexNxf+8dmwGt37CZBQbKsk8aBBzZEI0rQD1maLE8IklmChmf0VkhBUmxha1kBKING9L8ZYrWCWNStm7LlcerorVSlZPDk7hDErgwQ1U4R5qUAcCCl7gFd6cZ+fd+XA+56trTnZzAgtwvn4BMM+Ymw==</latexit>
The overall attack strategy is summarized in Algorithm 1 and pictorially illustrated in Figure 1. Since in Step 2 above, the gradient estimation (i.e., GRAD-EST) procedure includes the geometric transformation strategy (i.e., TRANS-WARP), we will next describe GRAD-EST and then move on to TRANS-WARP. For the simplicity of exposition, we drop the superscript i and shorten the loss function to L(fθ(xadv), y) to L (as the model parameters θ remain unchanged) in rest of this section.

heading: 3.1 GEO-TRAP Gradient Estimation (GRAD-EST)
text: Let g? = ∇xL be the ideal value of the gradient of L at x, required to create xadv in (2). To find an efficient estimator g for g?, a (new) surrogate loss `(g) = −〈g?, g〉 is defined such that the estimator g has a sufficiently large inner product with the actual gradient g? (g is normalized to a unit vector; we ignore the normalization operation for ease of explanation). The loss function definition and the algorithm to estimate g follow [13].
As g? is unknown in the black-box setting, this surrogate loss function can be estimated as
`(g) = −〈g?, g〉 = −〈∇xL, g〉 ≈ − L(x+ g, y)− L(x, y) . (3)
To iteratively estimate g, we need to, in turn, estimate the gradient of `(g), i.e., ∆ = ∇g`(g). With antithetic sampling [28], ∆ can be estimated as
∆ = `(g + δπ)− `(g − δπ)
δ π, (4)
where δ is a small number adjusting the magnitude of the loss variation and π ∈ RT×H×W×C is a random candidate direction. Our core contribution lies in the fact that instead of randomly sampling π in the search space [13], we reduce the search dimensionality by warping a randomly sampled tensor rframe ∈ RH×W×C with another randomly sampled geometric (e.g., affine) transformation parameter tensor Φwarp ∈ RT×D to get π. The search space is then reduced from T ×H ×W ×C to (H×W ×C)+ (T ×D) and D is a relatively small number, D H×W ×C. Withw1 = g+ δπ and w2 = g − δπ and combining (3) with (4), we get
∆ = L(x+ w2, y)− L(x+ w1, y)
δ π. (5)
Note that by querying the victim model fθ with x + w1, we are able to retrieve the value of L(x+ w1, y); similarly we can obtain the value of L(x+ w2, y) (L(·) is defined following [17].). In summary, we estimate ∆ with these two queries to the victim model. The resulting algorithm for estimating gradient of∇g` or ∆ for consequently estimating g is shown in Algorithm 2. Eventually at every iteration, we use ∆ to update g by applying a one-step gradient descent as g ← g − η∆, where η is a hyperparameter to update g. This updated g is later used to obtain xadv using (2).
Algorithm 1: GEO-TRAP: Query-based Iterative attack for Video Classifiers Input :video x, corresponding label y, step-size η for updating the gradient, step-size h for updating adversarial video. Output :adversarial video xadv
1 Initialize: x(0) = x, g(0) = 0, i = 1 2 while argmax ( fθ(x (i)) ) = y do 3 ∆ = Grad-Est(x(i−1), g(i−1), y) /* Gradient Estimation */ 4 g(i) ← g(i−1) − η∆ 5 x(i) ← clip(x(i−1) − hsign(g(i))) 6 i← i+ 1 7 end 8 return xadv = x(i)

heading: 3.2 Noise Warping using Geometric Transformation (TRANS-WARP)
text: To tackle the challenge of the high-dimensionality of the search space, we propose to parameterize the search space with a single random noise tensor rframe ∈ RH×W×C and a sequence of geometric transformations Φwarp ∈ RT×D. Apart from the reduction of the search space of gradient estimation, our geometric transformation provides a temporal structure to π, which we discuss next.
At every iteration, π = [r1, r2, . . . , rT ] represents the candidate direction for ∆. These directions rt ∈ RH×W×C are used to compute ∆ in order to update gradient g. To obtain π, we use a sequence of transformation vectors Φwarp = [φ1,φ2, . . . ,φT ] where φt ∈ RD. The dimensionality D, chosen by the attacker, can vary depending on the transformation type that is populated from φt, e.g., D = 6 for affine transformation. We take affine transformation as an example to describe the warping process. We start by randomly sampling rframe and the sequence of φt along with initializing each element in the sequence of rt with zero in every iteration. TRANS-WARP then computes rt by warping rframe using the parameters in φt = [φt11, φ t 12, φ t 13, φ t 21, φ t 22, φ t 23] ∈ R6 of Φwarp ∈ RT×6 as follows. For all C channels, let (p, q) and (u, v) be the target and source coordinates in rt and rframe, respectively. rt (for all channels) is computed as
rt(p, q)← rframe(u, v), 1 ≤ p, u ≤ H, 1 ≤ q, v ≤W. (6) Location (p, q) is computed using the affine transform matrixMφt created with φt in homogeneous coordinates [29] as shown below. t is dropped for simplicity.pq
1  =Mφ uv 1  = φ11 φ12 φ13φ21 φ22 φ23 0 0 1 uv 1  (7) We compactly denote this warping operation in (6) and (7) with rt = T (rframe,φt). Affine transformation allows translation, rotation, scaling, and skew to be applied to rframe to get each rt. Therefore, the sequence of rt have affine geometric progression among its temporal dimension. Other examples of geometric transformations may be more constrained, such as the similarity transformationMSφ (that allows translation, dilation (uniform scale) and rotation with D = 4) and translation-dilation MTDφ (that allows translation and uniform dilation with D = 3) as shown below.
[φ11, φ12, φ13, φ23]→MSφ =  φ11 φ12 φ13−φ12 φ11 φ23 0 0 1  , [φ11, φ13, φ23]→MTDφ = φ11 0 φ130 φ11 φ23 0 0 1  (8)

heading: 4 What Makes GEO-TRAP Effective?
text: Potent iterative algorithms should rely on few queries for crafting successful perturbations for time efficiency. To minimize the number of queries, iterative algorithms need to find strong gradients in their early iterations. As discussed earlier, videos inherently incur a larger search space due to the temporal dimension and thus, pose challenges in searching for effective gradients. In this section, we provide empirical evidence to show that by parameterizing the temporal dimension, GEO-TRAP finds better gradients, in general, than previous works. We use three baselines in this analysis.
Algorithm 2: GRAD-EST(x(i−1), g(i−1) ∈ RT×H×W×C , y)→ Estimate ∆ = ∇g`(g) ∈ RT×H×W×C
Input :video x(i), label y, gradient estimator g(i−1), δ for loss variation, for approximation. Output :estimation of ∆ = ∇g`(g)
1 Sample rframe ∈ RH×W×C , Φwarp ∈ RT×D (each element from a normal distribution N (0, 1)) 2 π = Trans-Warp(rframe,Φwarp) /* Use Geometric Transformations */ 3 w1 = g
(i−1) + δπ, w2 = g(i−1) − δπ 4 L1 = L(x(i−1) + w2, y), L2 = L(x(i−1) + w1, y) /* Query victim model twice */ 5 ∆ = (L2 − L1)π/ δ 6 return ∆
• MULTI-NOISE ATTACK [13] which computes search directions rt separately for each frame by sampling each element of rt from a standard normal distribution, resulting in a search space dimension of T ×H ×W × C. It does not explicitly consider the temporal dimension; temporal progression in any arbitrary direction is possible between a sequence of perturbation frames.
• ONE-NOISE ATTACK which computes r1 by sampling each element from a standard normal distribution and applies the same r1 across all rt(t = 1, 2, · · · , T ). ONE-NOISE ATTACK reduces the search space but completely ignores the temporal dimension when generating the perturbation.
• MOTION-SAMPLER ATTACK [22] which uses the optical flow of the original video x to warp rframe to get each rt. It reduces the search space by using the motion prior of x as the temporal progression between perturbation frames. In contrast, rather than fixing the temporal search space using a motion prior, GEO-TRAP parameterizes the temporal structure of the space with Φwarp.
We measure the gradient estimation quality by calculating the cosine similarity between the ground truth g? and the estimated gradient g following [19] for the aforementioned baselines. For each attack, we average over 1000 randomly selected videos with their cosine similarity values in the first attack iteration. We choose the first iteration because the initial g? is the same for the different attack methods, ensuring a fair comparison. As shown in Figure 2(a), our proposed method for estimating the gradients, yields g of the best quality for both untargeted and targeted attacks among all evaluated approaches. This leads to faster loss convergence / few queries as shown in Figure 2(b). We validate such trends with different loss functions and more datasets in the Supplementary Material.
The empirical results validate that by carefully considering the temporal dimension and parameterizing the temporal structure of the gradient search space with geometric transformations, GEO-TRAP finds better gradients. GEO-TRAP and MOTION-SAMPLER ATTACK [22] are better than the other two; the reason could be that temporally structured perturbations are more likely to disrupt the motion context of videos. However, the gradients estimated by MOTION-SAMPLER ATTACK [22] are not as effective as our proposed approach; the reason could be that the motion-prior of the clean video does not necessarily represent the temporal behavior of effective video perturbations. By allowing flexibility of the temporal progression while maintaining only a minimally sufficient space through its geometric parameterization, GEO-TRAP generates effective temporally structured perturbations. Note that one could use other, potentially better ways to parameterize the temporal progression of the video perturbation; this is regarded as future works.
Algorithm 2.1: TRANS-WARP(rframe ∈ RH×W×C , Φwarp ∈ RT×D)→ Estimate π ∈ RT×H×W×C
Input :noise tensor rframe, warp tensors Φwarp, transformation operation Tφ(·). Output :candidate directions π = [r1, r2, · · · , rT ].
1 Initialize π = ∅ 2 for t = [1, 2, · · · , T ] do 3 φt = Φwarp[t] 4 rt = T (rframe,φt) /* Warping Operation */ 5 π ← append rt 6 end 7 return π

heading: 5 Experiments
text: Datasets. Following previous work like [15], we use the human action recognition dataset UCF-101 [30] and the hand gesture recognition dataset 20BN-JESTER (Jester) [23] to validate our attacks. UCF-101 includes 13320 videos from 101 human action categories (e.g., applying lipstick, biking, blow drying hair, cutting in the kitchen). Given the diversity it provides, we consider the dataset to validate the feasibility of our attacks on coarse-grained actions. Jester, on the other hand, includes hand gesture videos that are recorded by crowd-sourced workers performing 27 kinds of gestures (e.g., sliding hand left, sliding two fingers left, zooming in with full hand, zooming out with full hand). The appearance of different hand gestures is similar; it is the motion information that matters in the video classification. We use this dataset to validate our attack with regard to fine-grained actions.
Baselines. Among the four state-of-the-art black-box video attack methods [19–22] described in Section 2, we use [21, 22] as baselines for following reasons. Our first baseline is MOTION-SAMPLER ATTACK [22], which has been shown to outperform PATCHATTACK[19] , ONE-NOISE and MULTINOISE attacks (introduced in Section. 4). Our second baseline is HEURISTICATTACK [20]. We note that SPARSEATTACK [21] is not included in our analysis as we couldn’t replicate their results.
Attack Settings. We consider four state-of-the-art video classification models representing diverse methodologies of learning from videos, i.e., C3D [31], SlowFast [32], TPN [33] and I3D [34], as our black-box victim models to attack. More details about the four video models are provided in the Supplementary Materials. For UCF-101, we randomly select one video from each category following the setting in [19, 22]. For Jester, since the number of categories is small, we randomly select four videos from each category. All attacked videos are correctly classified by the black-box model. For targeted attack, a random target class is chosen for each video. The maximum noise value ρmax is 10 pixel values (out of 255) following [15, 35, 36]. We provide more results for different ρmax in Supplementary Material. Note that since the perturbation generated by HEURISTICATTACK [20] is sparse and thus more imperceptible, we do not impose a perturbation budget on it. We set the maximum query limit to Q = 60, 000 for untargeted attack and Q = 200, 000 for targeted attack. The other hyper-parameters, i.e., , δ, η, and h take the same values as mentioned in [22]. Unless otherwise specified, a translation-dilation transformation (with D = 3) is used for our attack method.
Metrics. Following [19, 22], we evaluate GEO-TRAP, in terms of (a) Success Rate (SR), i.e., the total success rate in attacking within query and perturbation budgets; and (b) Average Number of Queries (ANQ) i.e., the average total queries from attacks for all videos (including failed ones).

heading: 5.1 Comparison to State-of-the-Art
text: Untargeted Attack. We report the untargeted attack performance of our attack method and the baseline methods in Table 2. We observe that, in general, GEO-TRAP requires fewer average number of queries when attacking different black-box victim models: on average over 45 % fewer queries than MOTION-SAMPLER ATTACK [22]. At the same time, GEO-TRAP yields higher attack success rates: on average about 6% higher than HEURISTICATTACK [20]. When attacking SlowFast model on the Jester dataset, GEO-TRAP achieves 100% successful rate with only 521 queries while the baseline methods need at least 1906 queries. We also observe that the TPN model is more robust towards black-box attacks compared with the other three video recognition models.
Visualization. We show two visualizations of adversarial frames on Jester and UCF-101 in Fig. 3. We observe that the generated adversarial frames have little difference from the clean ones but can lead to a failed classification. Also, our attack method could lead to sparse perturbations in the spatial and temporal dimension as the perturbations are sometimes zoomed out (thus get very small), and sometimes are translated out of the sight with choice of geometric transformation. More examples are in the Supplementary Material.
Targeted Attack. We report the targeted attack performance of our method and the baseline methods in Table. 3. We observe that in some cases, HEURISTICATTACK [20] requires fewer number of queries than GEO-TRAP, but its attack success rates are pretty low in those cases. For example, when attacking the TPN model on the Jester dataset, although HEURISTICATTACK [20] requires only 12k average number of queries, its attack success rate is less than half of ours, 44.4% v.s. 92.6%. GEO-TRAP consistently yields higher attack success rates, on average over 30% higher than HEURISTICATTACK [20] and over 8% higher than MOTION-SAMPLER ATTACK [22]. In addition, in most cases, GEO-TRAP requires fewer average number of queries than the two baseline attacks, on average over 45 % fewer queries than MOTION-SAMPLER ATTACK [22]. The targeted attack performance further validates the effectiveness of our method.

heading: 5.2 Different Geometric Transformations in TRANS-WARP
text: As discussed in Section 3.2, different kinds of geometric transformations could be used in the TRANSWARP function. In addition to the translation-dilation transformation (MTDφ in (8), D = 3) employed throughout the paper, we report the performance of GEO-TRAP with two other different geometric
transformations, i.e., similarity transformation (MSφ in (8), D = 4) and affine transformation (Mφ in (7), D = 6). Figure 4 shows the untargeted attack performance on Jester with these different geometric transformations (more results are available in the Supplementary Material). We observe that the transformation with fewer degrees of freedom (DOF) ( translation-dilation transformation) tends to require fewer queries while having the same or higher attack success rates (the attack success rates are available in the Supplementary Material). We believe that D = 3 provides enough temporal flexibility to disrupt the motion context of the videos; additional degrees of freedom seemingly increase the search space unnecessarily, resulting in more queries.
6 Conclusion
Black-box adversarial attacks on video classifiers is a challenging problem that has been largely understudied. In this work, we demonstrate that searching for effectual gradients in a reduced but structured search space for crafting perturbations leads to highly successful attacks with fewer queries compared to state-of-the-art attack strategies. In particular, we propose a novel iterative algorithm that employs Geometric transformations to parameterize and reduce the search space, for estimating gradients that maximize the probability of mis-classification of the perturbed video. This simple and novel strategy exposes the vulnerability of widely used video classifiers. For instance, GEO-TRAP decreases average query numbers by 64.78%, 72.66% and 47.21% to attack C3D, SlowFast, and I3D, respectively, for almost 100% success rate in untargeted attacks.

heading: 7 Broader Impact
text: In this work, by leveraging geometric transformations for effective gradient estimations, we propose a highly query-efficient adversarial attack on video classification models which demonstrates state-ofthe-art results. As more and more safety-critical systems (e.g., perceptual modules in autonomous vehicles) nowadays rely on video models, we are hopeful that our work, in addition to future research (including designing sophisticated video generative models as in [37–39] for distribution-driven attacks [40, 41]), can eventually help build sufficiently robust video models to best avoid malicious sub-versions. On one hand, we believe our algorithm could allow further research in adversarial robustness and data augmentation strategies of deep vision models. It should also give a direction to researchers to design counter defense methodologies [42–46]. On the other hand, it highlights a key drawback of different video classifiers which will allow adversaries to design more sophisticated attacks, both in white-box and black-box settings. Addressing such fallacies in designing deep neural networks is of utmost importance before introducing them in real-world scenarios.
Acknowledgement. The authors would like to thank Dr. Cliff Wang of US Army Research Office for his extensive comments and input on this work. This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR00112090096. Approved for public release; distribution is unlimited.

Review summaries:
This paper studies adversarial attacks against black-box video classification models. By adopting geometric transformation, the proposed GEO-TRAP reduced the search space for attacking the models. The attack process against black-box models is similar as the extant methods, by estimating the gradient of the loss function. By conducting experiments, the proposed method could use fewer queries to achieve better attack performance.

The paper proposes a query-efficient score-based black-box adversarial attack on a video classification task. To address the problem of the high dimensionality, the paper introduces a simple method to reduce the dimension of the search space by applying random geometric transformations to a single random noise of size $H \times W \times C$ to generate a search direction candidate of size $T \times H \times W \times C$ . The paper empirically shows that the proposed method achieves a higher attack success rate with fewer queries than the baselines on various types of models and datasets.

This paper proposes a new method for constructing black-box adversarial attacks against video classifiers. Such attacks are usually hard to mount (moreso than attacks against image classifiers) since the dimension of the input is very large (T x C x H x W) instead of just (C x H x W), which increases the complexity of the corresponding gradient estimation problem. Previous work considered reducing this query complexity with various heuristics (e.g., sparsity/only attacking a few frames, or leveraging optical flow)---here, the authors propose an alternative prior. Namely, the proposed method learns a single C x H x W signal, as well as a parameterized (typically affine) transform for each frame (so another T x 6 parameters, for the affine case). This greatly reduces query complexity and is shown to improve upon previous more heuristic priors in practice.