Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations
Authors: Shasha Li
Abhishek Aich
Shitong Zhu
Salman Asif
Chengyu Song
Amit Roy-Chowdhury
Srikanth Krishnamurthy
Venue: NeurIPS 2021 Poster (2021)
Keywords: adversarial machine learning
black-box adversarial attack

Abstract: When compared to the image classification models, black-box adversarial attacks against video classification models have been largely understudied. This could be possible because, with video, the temporal dimension poses significant additional challenges in gradient estimation. Query-efficient black-box attacks rely on effectively estimated gradients towards maximizing the probability of misclassifying the target video. In this work, we demonstrate that such effective gradients can be searched for by parameterizing the temporal structure of the search space with geometric transformations. Specifically, we design a novel iterative algorithm GEOmetric TRAnsformed Perturbations (GEO-TRAP), for attacking video classification models. GEO-TRAP employs standard geometric transformation operations to reduce the search space for effective gradients into searching for a small group of parameters that define these operations. This group of parameters describes the geometric progression of gradients, resulting in a reduced and structured search space. Our algorithm inherently leads to successful perturbations with surprisingly few queries. For example, adversarial examples generated from GEO-TRAP have better attack success rates with ~73.55% fewer queries compared to the state-of-the-art method for video adversarial attacks on the widely used Jester dataset. Overall, our algorithm exposes vulnerabilities of diverse video classification models and achieves new state-of-the-art results under black-box settings on two large datasets.

Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations
Shasha Lí, Abhishek Aich', Shitong Zhu, M. Salman Asif, Chengyu Song, Srikanth V. Krishnamurthy, Amit K. Roy-Chowdhury University of California, Riverside, CA, USA
Abstract
When compared to the image classification models, black-box adversarial attacks against video classification models have been largely understudied. This could be possible because, with video, the temporal dimension poses significant additional challenges in gradient estimation. Query-efficient black-box attacks rely on effec- tively estimated gradients towards maximizing the probability of misclassifying the target video. In this work, we demonstrate that such effective gradients can be searched for by parameterizing the temporal structure of the search space with geometric transformations. Specifically, we design a novel iterative algorithm Geo- metric TRAnsformed Perturbations (GEO-TRAP), for attacking video classification models. GEO-TRAP employs standard geometric transformation operations to reduce the search space for effective gradients by searching for a small group of parameters that define these operations. This group of parameters describes the ge- ometric progression of gradients, resulting in a reduced and structured search space. Our algorithm inherently leads to successful perturbations with surprisingly few queries. For example, adversarial examples generated from GEO- TRAP have better attack success rates with 73.55% fewer queries compared to the -ate-off-hhe-att method for video adversarial attacks on the widely used Jester dataset. Overall, our algorithm exposes vulnerabilities of diverse video classification models and achieves new state-of-the-ar results under black-box settings on two large datasets. Code is available here: https: // h/gitthe gigghuu........ http://10o///GoTTTTT
Introduction
Adversarial attacks are designed expose vulnerabilities of Deep Neural Networks (DNNs). With real-world upplications of video classification based on DNNs emerging [1-3], key question that arises "what type of adversarial inputs can mislead, and thus render video classification networks vulnerable?" Designing such adversarial attacks not only helps expose security flaws of DNNs, but can also potentially stimulate the design of more robust video classification models.
Adversarial attacks against image classification models have been studied in both white-box [4-8] and black-box [9-13] settings. In the white-box setting, an adversary has full access to the model under attack, including its parameters and training settings syyper-parameters. training data, etc.) In the black-box setting, an adversary only has partial information about the victim model, such as the predicted labels of the model. In the case of video classification models, adversarial attacks in both white-box and black-box settings have garnered some interest [14-22], although the body of work here is more limited than the case of image classification models.
common black- box attack paradigm is query-based, wherein the attacker can send queries the victim model to collect the corresponding predicted labels, and thereby estimate the gradients needed
Equal contribution. Corresponding author: Shasha Li (s1i057@ucr.edl edu)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Table I: Comparison with state-of-the-art. GEO-TRAP, compared current black-box attack methods for videos, doesn't train a different network to craft perturbations, and parameterizes the temporal dimension of videos in searching for effective perturbation directions.
WITHOUT training "perturbation" network
CONSIDER temporal dimension?
PARAMETERIZE
Methods
lemporal dimennsonn?
PATCHATTACK[1]]
HEURISTICATTACK [20)
SPARSEATTACK [21]
MOTION-SAMPLEB ATTACK 1221
GEO-TRAP (Ours)
for curating the adversarial examples. Unlike static images, videos naturally include additional information from the temporal dimension. This high dimensionality (i.e., sequence of frames instead of one image) poses challenges to black-box adversarial attacks against video classification models; in particular, significantly more queries are typically needed for estimating the gradients for crafting adversarial samples [19-22]. [19] reduces the number of queries by adding perturbations on the patch level instead of at the pixel level; [20, 21] propose to add perturbations only on key pixels. [22] considers the intrinsic differences between images and videos (i.e., the temporal dimension), and proposes to use the optical-flow of clean videos the motion prior for adversarial video generation. Similar to [22], we also explicitly consider the temporal dimension of video. However, rather than fixing the temporal search space using the motion prior of clean videos, we propose to parameterize the temporal structure of the space with geometric transformations. This results a better structured and reduced search space, which allows black-box settings than the state-of-the-arr methods, including [22].
generate successful attacks with much fewer queries in
Contributions. In this paper, we propose a novel query-efficient black-box attack algorithm against video classification models. Due to the extra temporal dimension, generating video perturbations by searching for effective gradients remains a challenging task given the exceedingly large search space. These gradients are estimated by searching for 'directions' that maximize the probability of the victim model mis-classifying the crafted inputs. Our approach drastically reduces this large search space by defining this space with a small set of parameters that describe the geometric progression of gradients in the temporal dimension, resulting in a reduced and temporally structured search space. Conceptually, this parameterization of the temporal structure of the search space is performed using geometric transformations (c.g. affine transformations). We refer to our algorithm as Geometrically TRAnsformed Perturbations, or TROP. Despite this surprisingly simple strategy, GEO-TRAP outperforms existing black-box video adversarial attack methods by significant margins (~ 1.8% improvement attack success rate with 73.55% fewer queries for targeted attacks comparison to the state-of sate-of-tthe-at [22] on the Jester dataset (23]).
Related Works
In this section, we review different black-box adversarial attacks strategies, and categorize our proposed method with respect to state-of-the-art black-box attacks designed for video classifiers.
In most real-world attacks, the adversary only has partial information about the vietim models, such as the predicted labels. In such black-box settings, the adversary can first attack local surrogate model and then transfer these attacks to the target victim model [24, 25], formally called as transferabilityy based black-box attack. Alternatively, they may estimate the adversarial gradient with zero-order optimization methods such as Finite Differences (FD) or Natural Evolution Strategies (NES) by querying the victim model [12, 13, 26), which is called query-basen black-box attack. GEO-TRAS falls under the category of query-based black-bor box attacks (designed for videos).
Whilst several white-box attacks have been proposed for video classification models [14-18], black- box video attacks are relatively under explored. PATCHATTACK (V-BAD)[19] is the first to propose a black-box video attack framework which uses a hybrid attack strategy of first generating initial perturbations for each video frame by attacking a local image classifier, and then updating the perturbations by querying the victim model. Compared to PATCHATTACK[I9], GEO-TRAP does not require training local classifier. PATCHATTACK[19] crafts video perturbations by treating each frame as separate image, but reduces the search space of the gradient estimation by morphing the

---

perturbations in patches/partitions. However, its attack performance has been shown to be inferior to that of a more recent approach [22] (discussed below). HEURISTICATTACK [20] uses a query- based attack strategy, and reduces the search space by generating adversarial perturbations only on heuristically selected key frames and salient regions. SPARSEATTACK [21] reduces the search space by adding perturbations only on key frames using a reinforcement learning based framework. MOTION- SMMPLR ATTACK [22] proposed a query-based attack strategy that utilizes a motion excited sampler to obtain motion-aware perturbation prior by using the optical-flow of the clean video. This motion-aware prior reduces the search space for gradients resulting in fewer queries. Similar to [22] but different from [19-21], GEO-TRAP explicitly considers the temporal dimension of video in order to search for effective gradients. However unlike [22], GEO-TR TRAP does not fix the temporal structure of the search space using pre-computed fixed motion prior, but parameterize it with simple geometric transformations. These black-box video attack methods are summarized in Table 1.
3 Attacking via Geometrically TRAnsformed Perturbations (GEO-TRAP)
Notation. We denote the tuple of video clip and its corresponding label (æ.y), which represents a data-point in the distribution X. Each video sample z E H&T* has T frames of H height, W width, and C' channels. We denote the victim video classification model as fe : X + y. where 0 represents the model's parameters learned from the training subset of X, via a mapping to the label space V. We further assume X' consists of videos from |J| = K categories. To make the perturbations imperceptible to humans, we impose the perturbation budget Pmax with the || llp norm. Throughout this paper, we consider || Il norm following [15, 19, 22] (the method can be extended p = 1,2 2 norms). To constrain || |lee of perturbation below a budget Pmax. we use the clip(-) function to keep the perturbation pixel value in [ I-Pmax: Pmax]] The function sign(-) extracts the sign of given input variable. The superscript i, throughout the paper, denotes the iteration i. The subscript denotes the frame index. For clarity, we represent vectors/tensors with the bold font and scalars with the regular font.
Problem Statement. We consider the scenario of attacking standard video classification model using a query-based paradigm under black-box settings (assuming no access to nor the training subset of X). Specifically, we aim to craft perturbed videos adve with imperceptible differences from ar in order to alter the decision of the target model So via multiple queries to guide the gradient estimation. This problem can be mathematically formulated as follows.
argmin 2ath
(fo((ra)))))) v) S.I. |xand
011100 S Pmax
(I)
(fo (2 v) is the objective function, capturing the similarity between the classifier's output and the ground truth label y, and varies with different attack goals (targeted untargeted). The challenge
is to obtain aadv with as few queries as possible by estimating gradient g* Vnn. C(fo (Tadv) ), y).
which is unknown in the considered black-box setting.
Overview of GEO-TRAP. We propose novel iterative video perturbation framework that follows the principle of the Basic Iterative Method [27] in order to fool So under || ol norm as follows.
x (0) ady xr, adv
(8) clip(= (1-1) h (ig adv
where / is a hyperparameter and g is the gradient estimated by querying the black-box victim model at the 2dh iteration using our proposed GEO-TRAP algorithm. As shown in (2), effective perturbations rely the guidance of the gradient g (f)). Therefore, efficiently estimating (8) is at the core of GEO-TRAP for successfully subverting video classifiers. We execute the following two steps in each iteration to estimate gi))
1. For any input video (0)), a random noise tensor Firame E RXX and a set of geometric transfor- mation parameters & @warp E RTxD D are chosen with each element sampled from standard normal distribution. D represents the number of parameters needed for the geometric transformation of a single frame (details are provided in Section 3.2). In this setup, our search space for estimating (()) consists of T'rame and & warp.
2. We then warp Tirame with & warrp to get the candidate direction π [ri, T2, rr] E
RTxH XW X (see Algorithm 2.1 TRANS-WARP). π is then employed to compute a gradient estimator A by querying the black-box victim model with standard gradient estimation algorithm (see Algorithm 2 GRAD-EST). The gradient estimator A is then used to update (+))
3

---

التااو
GRAD-EST
A
(Algorithm 2)
+
y
-1
elementvise
'guory'
"prodtction'
g (1-1)
addition
X
Black box Classifier
elemennviise
multiplication
('tramee 4 warp
b, y hyper- parameters
TRANS-WARP (Algorihm 2.1)
goometrically transformed Piramn
i curvent
I.
clip(-)
Mar
g(e
A ostimaate of geadient
or [i-1)
Frame
h
sign(
label
Figure I: Overview GEO-TRAP. GEO-TRAB a black box attack algorithm guided by the key observation that strong gradients g( can be computed by finding better gradient search direction candidates We propose search each frame of the directions re by warping randomly sampled r'rame using geometric transformation Ma.; different r; are warped by the same Prime, thus have geometric progression among frames.
The overall attack strategy is summarized in Algorithm and pictorially illustrated in Figure 1. Since in Step above, the gradient estimation (i.e., GRAD-EST) procedure includes the geometric transformation strategy (i.e., TRANS- WARR)), will next describe GRAD-EST and then move on to TRANS- WARP. For the simplicity of exposition, we drop the superscript i and shorten the loss function to (3 y) L (as the model parameters O remain unchanged) in rest of this section.
3.1 GEO-TRAP Gradient Estimation (GRAD-EST)
Let g" = V C be the ideal value of the gradient of C at required to create Tadv in (2). To find an efficient estimator g for g". (new) surrogate loss E(g) [g" .) is defined such that the estimator has a sufficiently large inner product with the actual gradient g* (g is normalized to unit vector; we ignore the normalization operation for ease of explanation). The loss function definition and the algorithm to estimate follow [13].
As g" is unknown in the black box setting, this surrogate loss function can be estimated as
L(x cg. L(x,))
{(g) - (g (V
6
To iteratively estimate g, need to, in turn, estimate the gradient of ((g) i.e., A Vgl(g). With antithetic sampling [28]. A can estimated as
{(g 8T) fgg 8m) n
(4)
where 8 is small number adjusting the magnitude of the loss variation and E RTxHxWxC is random candidate direction. Our core contribution lies in the fact that instead randomly sampling in the search space [13], reduce the search dimensionality by warping randomly sampled tensor Pirame E H xW xC with another randomly sampled geometric (e.g., affine) transformation parameter tensor & warp RTXD get π. The search space is then reduced from T H W x C to
(H x W x C') + (T x D) and D is a relatively small number, D < H H x C. With w1 = g 8n
and w2 = g - 8ा and combining (3) with (4), we get
L(x + ew2, 1/) Lix + +EW1,
(5)
6d
Note that by querying the victim model fo with æ + cw1, we are able to retrieve the value of L(x + ew1, v); similarly we can obtain the value of L(x + ew2, y) (C(-) is defined following [17].).
In summary, we estimate A with these two queries to the victim model. The resulting algorithm for estimating gradient of V. or A for consequently estimating g is shown in Algorithm 2. Eventually every iteration, we use A to update g by applying a one-step gradient descent as g < 9 - 7/
where 7) is hyperparameter to update g. This updated is later used to obtain Tady using (2).

---

Algorithm 1: GEO-TRAP: Query-based Iterative attack for Video Classifiers
Input :video æz corresponding label y, step-size for updating the gradient, step-size h for updating adversarial video.
Output: adversarial video Tadv
Initialize: _0 = x, (0) 0, 0,i=1 ==
while arginax (fo (f(
do
A GRAD-EST (w (x(i-1) gl .y) 1 Gradient Estimation */
i ((-- 1
(+) + clip(xli-1) hg ())
i it1
7 end
return addy (1)
3.2 Noise Warping using Geometric Transformation STRANS-WARP.
To tackle the challenge of the high-dimensionality of the search space, propose parameterize the search space with single random noise tensor Thrame R W xC and a sequence geometric transformations 2 warp R x Apart from the reduction of the search space of gradient estimation, our geometric transformation provides a temporal structure to π, which we discuss next.
At every iteration, π [r1, [2,,72,... ..,.... TT] represents the candidate direction for A. These directions E R&x xC are used to compute A in order to update gradient g. To obtain we use sequence of transformation vectors & waarp = [Φ1 $2..... Φ2 where Φ; E RD. The dimensionality D, chosen vary depending
by the attacker, for affine transformation. We take affine transformation as an example to describe the warping process. We start by randomly sampling Fraame and the sequence of Φi along with initializing each element in the sequence of Tt with zero in every iteration. TRANS-WAR WARP then computes r; by warping Frame using the parameters in Φ [df 1/2 Φf.. &h 022. 0231 R of & RTx6 as follows. For all C' channels, let (p. q) and (u, v) be the target and source coordinates in re and Frrame- respectively. Tz (for all channels) is computed
the transformation type that is populated from Øi. e.g., D 6
(p(p.g) g) TTramm v), v),
S H, 15 W.
Location (p. q) is computed using the affine transform matrix Ma. created with de in homogeneous coordinates [29] as shown below. t is dropped for simplicity.
P ( 9
11
11
u
M
d22
2
22
1
1
We compactly denote this warping operation in (6) and (7) with r. - T(rrame Φt). Affine transfor-
mation allows translation, rotation, scaling, and skew be applied Frrame get each v.. Therefore, the sequence of r, have affine geometric progression among its temporal dimension. Other examples of geometric transformations may be more constrained, such as the similarity transformation M (that allows translation, dilation (uniform scale) and rotation with D 4) and translation-dilation MD (that allows translation and uniform dilation with D 3) as shown below.
Φn
d12
613
Φ
0
113
[d11, d12 013, {25) + S
212
Φ11
223
Io,, 013, 222) MdD
Φ11
223
(8)
0
4 What Makes GEO-TRAP Effective?
Potent iterative algorithms should rely on few queries for crafting successful perturbations for time efficiency. To minimize the number of queries, iterative algorithms need to find strong gradients in their early iterations. As discussed earlier, videos inherently incur a larger search space due to temporal dimension and thus, pose challenges in searching for effective gradients. In this section, we provide empirical evidence show that by parameterizing the temporal dimension, GEO-TRAP AP finds better gradicnts, in general, than previous works. We use three baselines in this analysis.
5

---

Algorithm 2: GRAD-EST(e" g(-11) E RRTxHWxX y) Estimate A Vo((g) E RTxH*Wxx
Input videeo (0) label y, gradient estimator g g(i 1) 5 for loss variation, e for approximation.
of A Vgl(g)
Sample Pitame E X xc T warp E RTTD (each element from a normal distribution (0, 1))
2 SLANS-WARD (rome T @werp)
1* Use Geometric Transformations */
3 W1 = g + 8TH, W2- = g (i-1) 8π
4 L1 C(x(i-1) + Ew2, v), L2 ((x(1-1) + tew1, y) 1* Query victim model twice */
A (Lz 11) //8
return A
Taugsted Attack
Untaggetel Attak
Gro-TRAP (Oura) MOTION-S Mcart NoisE ONE Noise
1.50
(9.9"3 (x300))
1,5
Loss Sunction C
0.00
3.2
0.5
3.1-
(ew.TinDaae)) Station Laraa
5000 20044
046 9606
0
2,000
4,000 6.000
Attack Methorks
Iterations
(a)
Figure 2: Gradient Analysis of GEO-TRAP. (a) GEO-TRAP's high query-efficiene, is direct implication of good quality gradient estimation (for both targeted and untargeted attack), shown here with higher cosine similarity with g compared to other methods. (b) Better quality of estimated gradients by GEO-Te AP results in a successful attack with fewer queries compared to other attacks.
MULTI-NOISE ATTACK [13] which computes search directions r2 separately for each frame by sampling each element of r; from a standard normal distribution, resulting in a search space dimension of T x H W x C. It does not explicitly consider the temporal dimension; temporal progression in any arbitrary direction is possible between a sequence of perturbation frames.
ONE-NOISE ATTACK which computes Tı by sampling each element from a standard normal distribution and applies the same rı across all rt = 1, ..... T). ONE-NOISE ATTACK reduces the search space but completely ignores the temporal dimension when generating the perturbation.
MOTION-SAMPLE ATTACK [22] which uses the optical flow of the original video æ to warp Trrame to get each Ts, It reduces the search space by using the motion prior of æ the temporal progression between perturbation frames. In contrast, rather than fixing the temporal search space using motion prior, GEO-TR: TRAP parameterizes the temporal structure of the space with $ @wapp.
We measure the gradient estimation quality by calculating the cosine similarity between the ground truth g* and the estimated gradient g following [19] for the aforementioned baselines. For each attack, average over 1000 randomly selected videos with their cosine similarity values in the first attack iteration. We choose the first iteration because the initial g* is the same for the different attack methods, ensuring fair comparison. As shown in Figure 2(a), proposed method for estimating the gradients, yields g of the best quality for both untargeted and targeted attacks among all evaluated approaches. This leads to faster loss convergence few queries shown in Figure 2(b). We validate such trends with different loss functions and more datasets in the Supplementary Material.
The empirical results validate that by carefully considering the temporal dimension and parameterizing the temporal structure of the gradient search space with geometric transformations, GEO- TRAP finds GEO-TRAP
better gradients. GEO- and MOTION- SAMPLER ATTACK [22] are better than the other two; the reason could be that temporally structured perturbations are more likely to disrupt the motion context of videos. However, the gradients estimated by MOTION- AMMLER ATTACK [22] are not as effective our proposed approach; the reason could be that the motion-prion of the clean video does not necessarily represent the temporal behavior of effective video perturbations. By allowing flexibility of the temporal progression while maintaining only a minimally sufficient space through its geometric parameterization, GEO-TRAP generates effective temporally structured perturbations. Note that one could use other, potentially better ways to parameterize the temporal progression of the video perturbation; this is regarded as future works.

---

Algorithm 2.1: TRANS- WARPS(rmmm RBxWxC @ warp
RT*D) Estimate RTxHxWx0
Input :noise tensor F'rame. warp tensors T Pwarp. transformation operation (+(-). Output:
: candidate directions π [1,, "2,"" PT]
Initialize π = 0
2 for t [1.2.-
do do
Φ = 1 warpl
r: (rrmme Φ.) 1 Warping Operation */
< append rt
end
return π
5 Experiments
Datasets. Following previous work like [15], we use the human action recognition dataset UCF-101 [30] and the hand gesture recognition dataset 20BN-JESTER (Jester) [23] to validate our attacks. UCF-101 includes 13320 videos from 101 human action categories (e.g., applying lipstick, biking, blow drying hair, cutting in the kitchen). Given the diversity provides, we consider the dataset to validate the feasibility of our attacks on coarse -rriind actions. Jester, on the other hand. includes hand gesture videos that are recorded by crowd-sourced workers performing 27 kinds of gestures (e.g., sliding hand left, sliding two fingers left, zooming in with full hand, zooming out with full hand). The appearance of different hand gestures similar; it the motion information that matters the video classification. We use this dataset to validate our attack with regard to iiteggained actions.
Baselines. Among the four state-of-the -frtt--rttttttttt black-box video attack methods [19-22] described in Section 2, we use [21, 22] as baselines for following reasons. Our first baseline is MOTION-S -SAN--ALELEE ATTACK [22], which has been shown to outperform PATCHATTACKX199] ONE-NOISE and MULTI- NOISE attacks (introduced in Section. 4). Our second baseline is HEURISTICATTACK [20]. We note that SPARSEATTACK [21] is not included in our analysis
couldn't 't replicate their results.
Attack Settings. We consider four state-of-the-ar video classification models representing diverse methodologies of learning from videos, i.e., C3D [31], SlowFast [32], TPN [33] and 13D [34], as our black-box victim models to attack. More details about the four video models are provided in the Supplementary Materials. For UCF-101, we randomly select one video from each category following the setting in [19, 22]. For Jester, since the number of categories is small, we randomly select four videos from each category. All attacked videos are correctly classified by the black-bor model. For targeted attack, a random target class is chosen for each video. The maximum noise value Pmax is 10 pixel values (out of 255) following [15, 35, 36]. We provide more results for different Pmax in Supplementary Material. Note that since the perturbation generated by HEURISTICATTACE [20] is sparse and thus more imperceptible, we do not impose a perturbation budget on it. We set the maximum query limit Q 60, 000 for untargeted attack and Q 200, 000 for targeted attack. The other eeper-parameters i.e., e, 5, 17. and take the same values as mentioned in [22]. Unless otherwise specified, translation-dilation transformation (with D 3) is used for attack method.
Metrics. Following [19, 22], we evaluate GEO-T TERP,, in terms of (a) Success Rate (SR), i.c., the total success rate in attacking within query and perturbation budgets; and (b) Average Number of Queries (ANQ) i.e., the average total queries from attacks for all videos (including failed ones).
5.1 Comparison to State-of-the-Art
Untargeted Attack. We report the untargeted attack performance of our attack method and the baseline methods in Table 2. We observe that, in general, GEO-TR. requires fewer average number of queries when attacking different black-box victim models: on average over 45 fewer queries than MOTION-SAMPLER ATTACK [22]. At the same time, GEO-TRA yields higher attack success rates: on average about 6% higher than HEURISTICATTACK [20]. When attacking SlowFast model on the Jester dataset, GEO-TRA AP achieves 100% successful rate with only 521 queries while the baseline methods need least 1906 queries. We also observe that the TPN model more robust towards black-box attacks compared with the other three video recognition models.
7

---

clean video, 3
UCF-101
Jester
Kadell (x:10)
GUO-TRAP Arak
Ja Fatel (x10)
MOTION SAMPLER
#udell (x100)
IEE IUSTIITIC Arauk
Figure 3: Visualization of Perturbations and Perturbed Video. We visualize the generated perturbations and perturbed video for GEO-TRAP and other baselines for UCF-101 (left) and Jester (right) datasets for untargeted attack against SlowFast classifier with Puaax 10/255.
Visualization. We show two visualizations of adversarial frames on Jester and UCF-101 Fig. We observe that the generated adversarial frames have little difference from the clean ones but can lead to failed classification. Also, our attack method could lead to sparse perturbations the spatial and temporal dimension the perturbations sometimes are translated out of the sight with choice of geometric transformation. More examples in the Supplementary Material.
sometimes zoomed out (thus get very small), and
Targeted Attack. We report the targeted attack performance of method and the baseline methods in Table. 3. We observe that in some cases, HEURISTICATTACK [20] requires fewer number of queries than GEO-TRAN but its attack success rates are pretty low in those cases. For example, when attacking the TPN model on the Jester dataset, although HEURISTICATTACK [20] requires only 12k average number of queries, its attack success rate is less than half of ours, 44.4% 92.6%. GEO-TRAP consistently yields higher attack success rates, on average over 30% higher than HEURISTICAATACK [20] and over 8% higher than MOTION SAMPLER ATTACK [22]. In addition, in most cases, GEO-TRAP requires fewer average number of queries than the two baseline attacks, average 45 % fewer queries than MOTION-SAMPLEN ATTACK [22]. The targeted attack performance further validates the effectiveness of our method.
5.2 Different Geometric Transformations in TRANS-WARP
As discussed in Section 3.2, different kinds of geometric transformations could be used in the TRANS- WARP function. In addition to the translation-dilaton transformation (MT) in (8), D 3) employed throughout the paper, we report the performance of GEO-TRAP with two other different geometric
Table 2: Untargeted Attacks. GEO-TRAP demonstrates highly successful untargeted attacks (high Success
Rate (SR) with fewer queries (low Average Number of Queries (ANQ))
Black-box Video Classifiers
Datasets
Methods
C3D
SlowFast
TPN
13D
ANQ
SR
ANQ
SR
ANQ
SR
ANQ (.)
SR (f)
HOURISTICATTACK MIOTION-SAPLLE AUTACK [22]
99.0%
3572
98.1%
4679
82.0%
4248
98.1%
Jester
99.0%
1906
100%
6269
91.3%
3029
99.4%
GEO-TRAP
1602
100%
521
100%
3315
92.4%
1599
100%
HIURTSTICATTACK [20)
3507
6539
71.8%
84.7%
[22]
97.2%
20369
75.8%
94.4%
GEO. TRAP (Ours)
1547
76.1%
4887
97.4%

---

Table 3: Targeted Attacks. GEO-TRAP demonstrates highly successful targeted attacks (high Success Rate
(SR)) with fewer queries (low Average Number Queries (ANQ))
Black-box Video Classifiers
Datasets
Methods
C3D
SlowFast
TPN
13D
ANQ (L)
SR (t)
ANQ (4)
(1)
ANQ(U)
(1)
ANQ (.)
(1)
HEURISTICATTACK (20)
15595
46.3%
30768
98.1%
12006
44.4%
31088
77.8%
Jester
MIOTION-SAMPLL ATTACK [22]
26704
98.2%
33087
100%
63721
80.9%
39037
90.7%
GEO-TRAP (Ours)
6198
100%
7788
100%
41294
92.6%
19542
98.2%
HFURISTICATTACK [20]
26741
29.0%
22152
61.4%
71828
36. 4%
92244
43.7%
UCF-101
MOTION-S ATTACK [22]
100467
71.1%
57126
86.0%
151409
31.6%
96498
59.6%
GEO-TRAP (Ours)
71820
85.8%
21878
98.0%
141629
40.0%
76708
74.6%
transformations, i.e., similarity transformation (Mş in (8), D = 4) and affine transformation (M& in (7), D = 6). Figure 4 shows the untargeted attack performance on Jester with these different geometric transformations (more results are available in the Supplementary Material). We observe that the transformation with fewer degrees of freedom (DOF) counslltion-dilatiin transformation) tends to require fewer queries while having the same or higher attack success rates (the attack success rates are available in the Supplementary Material), We believe that D 3 provides enough temporal flexibility to disrupt the motion context of the videos; additional degrees of freedom seemingly increase the search space unnecessarily, resulting in more queries.
Conclusion
Black-box adversarial attacks on video classifiers is a challenging problem that has been largely understudied. In this work, we demonstrate that searching for effec- tual gradients in a reduced but structured search space for crafting perturbations leads to highly successful at- tacks with fewer queries compared to state-of-the-ann attack strategies. In particular, we propose novel it- crative algorithm that employs Geometric transforma- tions to parameterize and reduce the search space, for estimating gradients that maximize the probability of mis mis-classine caassisssiiicttonn of the perturbed video. This simple and novel strategy exposes the vulnerability of widely used video classifiers. For instance, GEO-TRAP de- creases average query numbers by 64.78%, 72.66% and 47.21% to attack C3D, SlowFast, and I3D, respectively, for almost 100% success rate in untargeted attacks.
3,000
ANQ ANQ ())
2,000
1,000
11900
1,300
1,22
1,000
002
C3D
SlowFast
TPN
BD
Figure 4: Performance with different Ma. GEO-TRAP results in best performance when M& is set as translation-dilation operation.
Broader Impact
In this work, by leveraging geometric transformations for effective gradient estimations, we propose a highly query-efficient adversarial attack on video classification models which demonstrates state-of- the-art results. As more and more safety-critical systems (e.g.. perceptual modules in autonomous vehicles) nowadays rely on video models, we are hopeful that our work, in addition future research (including designing sophisticated video generative models as in [37-39] for distribution-dhrg driven attacks [40, 41J). can eventually help build sufficiently robust video models to best avoid malicious sub-versions. On one hand, we believe our algorithm could allow further research in adversarial robustness and data augmentation strategies of deep vision models. should also give a direction to researchers to design counter defense methodologies [42-46]. On the other hand, it highlights a key drawback of different video classifiers which will allow adversaries to design more sophisticated attacks, both in white-box and black-box settings. Addressing such fallacies in designing deep neural networks is of utmost importance before introducing them real-world scenarios.
Acknowledgement. The authors would like to thank Dr. Cliff Wang of US Army Research Office for his extensive comments and input on this work. This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR001 12090096. Approved for public release; distribution is unlimited.

---

References
[1] Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. 3d convolutional neural networks for automatic human action recognition, January 2013. US Patent 8,345,984.
[2] Wongun Choi, Yuanqing Lin, Yu Xiang, and Silvio Savarese. Subcategory-aware convolutional neural networks for object detection, May 8 2018. US Patent 9,965,719.
[3] Abhishek Aich, Meng Zheng, Srikrishna Karanam, Terrence Chen, Amit K Moy-Chowdhury, and Ziyan Wu. Spatio-temporea representation factorization for video-based person re-identification. arXiv preprini arXiv:2 1878, 2021.
[4] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing Properties of Neural Networks. arXiv preprint 112.v913 2013.
[5] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adver- sarial Examples. arXiv preprint 4rXiv:14/2.6572, 2014.
[6] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pages 39-57. IEEE, 2017.
[7] Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. Physical adversarial examples for object detectors. In 12th {USENIX] Workshop on Offensive Technologies ({ (WOOT) 18), 2018.
[8] Chaowei Xiao, Ruizhi Deng, Bo Li, Fisher Yu, Mingyan Liu, and Dawn Song. Characterizing adversarial examples based on spatial consistency information for semantic segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 217-234, 2018.
[9] Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learn- ing: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277.22 2016.
[10] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506-519, 2017.
[11] Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song. Practical black-box attacks on deep neural networks using efficient query mechanisms. In Proceedings of the European Conference on Computer Vision (ECCV), pages 154-169, 2018.
[12] Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 15-26,
2017.
[13] Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversar- ial attacks with bandits and priors. arXiv preprint arXiv:1807.07974 2018.
[14] Xingxing Wei, Jun Zhu, Sha Yuan, and Hang Su. Sparse adversarial perturbations for videos. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8973-8980, 2019.
[15] Shasha Li, Ajaya Neupane, Sujoy Paul, Chengyu Song, Srikanth V Krishnamurthy, Amit K Roy-Chowdhury, and Ananthram Swami. Stealthy adversarial perturbations against real-time video classification systems. In NDSS, 2019.
[16] Zhikai Chen, Lingxi Xie, Shanmin Pang, Yong He, and Qi Tian. Appending adversarial frames for universal video attack. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3199-3208, 2021.
[17] Roi Pony, Itay Nach, and Shie Mannor. Over-the-air adversarial flickering attacks against video recognition networks. arXiv preprint arXiv:2002.05123, 2020.
[18] Shao- Yuan Lo and Vishal M Patel. Multav: Multiplicative adversarial videos. arXiv preprint arXiv:2009.08053 2020.
[19] Linxi Jiang, Xingjun Ma, Shaoxiang Chen, James Bailey, and Yu-Gang Jiang. Black-box adversarial attacks on video recognition models. In Proceedings of the 27th ACM International Conference Multimedia, pages 864-872, 2019.
10

---

[20] Zhipeng Wei, Jingjing Chen. Xingxing Wei, Linxi Jiang. Tat-Seng Chua, Fengfeng Zhou, and lu-Gang Jiang. Heuristic black-box adversarial attacks on video recognition models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12338- 12345, 2020.
[21] Huanqian Yan, Xingxing Wei, and Bo Li. Sparse black-box video attack with reinforcement learning. arXiv preprint trXiv:2001.03754 2020.
[22] Hu Zhang, Linchao Zhu, Yi Zhu, and Yi Yang. Motion-excited sampler: Video adversarial attack with sparked prior. In European Conference on Computer Vision. Springer, 2020.
[23] Joanna Materzynska, Guillaume Berger, Ingo Bax, and Roland Memisevic. The jester dataset:
A large-scale video dataset of human gestures. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0-0, 2019.
[24] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. ICLR, 2017.
[25] Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing adversarial example transferability with intermediate level attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4733-4742, 2019.
[26] Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-bor adversarial attacks with limited queries and information. In International Conference on Machine Learning, pages 2137-2146. PMLR, 2018.
[27] Alexey Kurakin, Ian Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world, 2017.
[28] Hongyu Ren, Shengjia Zhao, and Stefano Ermon. Adaptive antithetic sampling for variance reduction. In International Conference on Machine Learning, pages 5420-5428. PMLR, 2019.
[29] R. Hartley and A. Zisserman. Multiple View Geometry Computer Vision. Cambridge University Press, ISBN: 0521540518, 18, second edition, 2004.
[30] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv: 12l..4422..... 2012.
[31] Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spa- tiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 4489-4497, 2015.
[32] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6202-6211. 2019.
[33] Ceyuan Yang. Yinghao Xu, Jianping Shi, Bo Dai. and Bolei Zhou. Temporal pyramid network for action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 591-600, 2020.
[34] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pages 6299-6308, 2017.
[35] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Univer- sal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1765-1773, 2017.
[36] Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and R Venkatesh Babu. Nag: Network for adversary generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 742-751, 2018
[37] Abhishek Aich, Akash Gupta, Rameswar Panda, Rakib Hyder, M Salman Asif, and Amit K Roy-Chowdhury. Non-adversarial video synthesis with learned priors. In Proceedings of the IEEE/CVT Conference on Computer Vision and Pattern Recognition, pages 6090-6099, 2020.
[38] Akash Gupta, Abhishek Aich, and Amit K. Roy-Chowdhury, Alanet: Adaptive latent attention network for joint video deblurring and interpolation. In Proceedings of the 28th ACM Interna- tional Conference on Multimedia, page 256-264, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450379885.

---

[39] Sergey Tulyakov, Ming- Yu Liu, Xiaodong Yang, and Jan Kautz. MoCoGAN: Decomposing motion and content for video generation. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1536-1535, 2018.
[40] Muzammal Naseer, Salman H Khan, Harris Khan, Fahad Shahbaz Khan, and Fatih Porikli, Cross-Domain Transferability of Adversarial Perturbations. arXiv preprint arXiv:1905. 1736,
2019.
[41] Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative Adversarial Per- turbations. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 4422-443 1,22,443,,. 8.
[42] Xuwang Yin, Soheil Kolouri, and Gustavo Rohde. Gat: Generative adversarial training for adversarial example detection and robust classification. In International Conference on Learning Representations, 2019.
[43] Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, volume 1, page 2, 2019.
[44] Shasha Li, Shitong Zhu, Sudipta Paul, Amit Roy-Chowdhury. Chengyu Song, Srikanth Krish- namurthy, Ananthram Swami, and Kevin S Chan. Connecting the dots: Detecting adversarial perturbations using context inconsistency. In European Conference on Computer Vision, pages 396-413. Springer, 2020.
[45] Xueping Wang. Shasha Li, Min Liu, Yaonan Wang, and Amit K Noy-Chowdhury, Multi-expert adversarial attack detection in person re-identification using context inconsistency. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
[46] Mingjun Yin, Shasha Li, Zikui Cai, Chengyu Song, M Salman Asif, Amit K Roy- Coy-Chowdhury. and Srikanth V Krishnamurthy. Exploiting multi-object relationships for detecting adversarial attacks in complex scenes. Proceedings of the IEEE/CVF International Conference Computer Vision, 2021.
12


Reviews:
[{"review_id": "rrRF7KSvu9F", "reviewer": "Reviewer_oH6x", "summary": "This paper proposed a block-box adversarial attack method on the video classification task. The proposed method requires fewer queries with high attack success rates compared to SOTA methods, by leveraging Geometric transformations to parameterize and reduce the search space. ", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is well written and organized. However, I have some concerns about the experimental results:\n\n1. from fig.3 the proposed method show perception video quality degradation compare with the clean video. what are the numerical differences (like PSNR) between x and xadv on different attack methods? Will this attack impact the video quality? \n\n2. from Table 2 and Table 3, why does [19] have less ANQ compare with the proposed method at the UCF-101 dataset on C3D and TPN classifier? \nIt would be better if the authors would provide detailed and insightful explanations. \n\n3. what's the exact time to finish an attack of the proposed method compared with the other SOTA methods instead of the number of query reduction?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "see the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "muHa2YWTJb6", "reviewer": "Reviewer_VM8Y", "summary": "This paper studies adversarial attacks against black-box video classification models. By adopting geometric transformation, the proposed GEO-TRAP reduced the search space for attacking the models. The attack process against black-box models is similar as the extant methods, by estimating the gradient of the loss function. By conducting experiments, the proposed method could use fewer queries to achieve better attack performance.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper proposed the GEO-TRAP method to attack black-box video classification models, but the idea seems trivial and it combines many methods from extant attack methods. The experimental results also need some further explanation.\n1)\tThe proposed method adds perturbation to each frame. How to guarantee the continuity of the generated perturbation in different frames?\n2)\tPlease explain the intuitive idea of adopting geometric transformations. How to generate such transformation and evaluate the effectiveness of the transformation by theoretical analysis;\n3)\tEstimating the gradient for attacking black-box models is commonly adopted in many extant methods. The novelty / main contribution is limited to the dimensionality reduction by randomly sampled geometric transformation parameter tensor (as Section 3.1 stated).\n4)\tThe paper does not state clearly whether the black-box model would output the confidence of the classification result (Section 3). If the model only outputs the classification result (without any confidence value), how to define the loss function? The model settings should be introduced in detail since there are still many different settings of black-box models;\n5)\tIn the experiment, the paper should also compare the added perturbation to the video, not only the success rate and the average number of queries;\n6)\tThe number of maximum queries of the method is set as 60,000 for untargeted attack and 200,0000 for targeted attack. How about the other algorithms? As shown in Table 2 and 3, the success rate of other algorithms is not high; would increasing the maximum number of queries achieves higher success rate?\n7)\tFrom the supplementary materials, the classification accuracy of the four black-box models on UCF-101 is lower than that on Jester. The attack success rate on UCF-101 is much higher that that on Jester. The high success rate may lie in low classification accuracy of the original models;\n8)\tThe proposed method adds perturbations to each frame. But such attack method cannot be adopted in real scenario. In many safety-critical systems such as the mentioned perceptual modules in autonomous vehicles, it is difficult to add perturbations on the video. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper could consider how to generate such adversarial attacks in real-world, which might be utilized to improve the current classification systems. As adversarial learning with generated adversarial examples might improve the robustness against some (may not exist) adversarial examples, but the accuracy of the original system would be reduced slightly.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}, {"review_id": "RDOgp63El-f", "reviewer": "Reviewer_AtrB", "summary": "This paper proposes a new method for constructing black-box adversarial attacks against video classifiers. Such attacks are usually hard to mount (moreso than attacks against image classifiers) since the dimension of the input is very large (T x C x H x W) instead of just (C x H x W), which increases the complexity of the corresponding gradient estimation problem. Previous work considered reducing this query complexity with various heuristics (e.g., sparsity/only attacking a few frames, or leveraging optical flow)---here, the authors propose an alternative prior. Namely, the proposed method learns a single C x H x W signal, as well as a parameterized (typically affine) transform for each frame (so another T x 6 parameters, for the affine case). This greatly reduces query complexity and is shown to improve upon previous more heuristic priors in practice.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The work was clearly written and I was able to understand the method rather easily given the text, which is not always the case with black-box attack papers. As far as I am aware, the idea of \"splitting\" the perturbation signal into a single frame and a temporal transformation is a new and creative one, and the results are enough to convince me that the method achieves a nontrivial improvement over prior work. My main comments are below:\n\n- I thought Figure 1 actually detracted from the readability of the work, particularly because it is called upon before the actual method is introduced in text; I spent a while trying to decipher the Figure when in fact the plain-text description of the algorithm is not so long and is (in my opinion) much easier to follow.\n\n- The paper hypothesizes that the key to GEO-TRAP's improvement is that it leverages the temporal continuity of videos. I think this claim could be strengthen via a quantitative experiment that compares some notion of the \"smoothness\" or continuity of a video with the query complexity required by each method to attack it. E.g., under the stated hypothesis, we would expect SINGLE-NOISE to do best on the smoothest videos (since it has fewer parameters), MULTI-NOISE to perform best on the least-continuous videos, and GEO-TRAP to perform best on the remaining/middling videos. It would be interesting to understand where the performances intersect and when it is best to use each method.\n\n- In Table 2, the paper should also present numbers of the form \"# of queries required to reach X% success rate\" for some value of X, or better yet, a graph that puts the query budget on the x axis and the success rate on the y axis. Without this graph it is hard to compare GEO-TRAP with other methods that have smaller query budgets but lower success rates. \n\n- The proposed method seems somewhat specific to L-infinity attacks, since you don't have to care about the interaction between different frames in terms of the perturbation budget. I don't think this is a fatal flaw of the paper, but it would be nice if the authors discussed how the method relies (or does not rely, if the above intuition is mistaken) on the considered threat model).\n\n- Very minor: in table 1, should \"perturbation model\" be \"substitute model\"?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors include a Broader Impact section discussing societal impact. As for limitations, as I mentioned above a discussion of the exact scope of the method (in terms of threat models) would strengthen the paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "H4c3cIfPU8f", "reviewer": "Reviewer_zixR", "summary": "The paper proposes a query-efficient score-based black-box adversarial attack on a video classification task. To address the problem of the high dimensionality, the paper introduces a simple method to reduce the dimension of the search space by applying random geometric transformations to a single random noise of size $H \\times W \\times C$ to generate a search direction candidate of size $T \\times H \\times W \\times C$ . The paper empirically shows that the proposed method achieves a higher attack success rate with fewer queries than the baselines on various types of models and datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strong Points\n\n- Clarity: The paper is well-written and easy to follow. The paper clearly states the threat model and attack settings and specifies the hyperparameters.\n- Simplicity: The paper proposes a simple method to generate the effective temporal structure of the search direction with geometric transformations that requires no additional parameters or learning process and can be easily implemented using only matrix multiplications.\n- Empirical analysis: The paper provides a thorough analysis of the gradient estimation quality of the proposed method (Figure 2 in Section 4) to validate its effectiveness.\n- Experiment: The paper conducts extensive experiments varying datasets, models, and perturbation budgets in both targeted and untargeted settings and shows that the proposed method outperforms the current state-of-the-art methods.\n\n### Weak Points\n\n- Novelty: The proposed algorithm is a simple modification of [1], replacing the motion prior by random geometric transformations. Although this change can significantly improve the attack performance, the novelty of the proposed method is incremental.\n- How it works: There is no clear theoretical or empirical explanation of how the proposed method leads to better-estimated gradients.\n- Baselines: It seems that the results of [1] are not reproduced properly. Based on the original paper, it can achieve a 98.02% attack success rate with a 4,748 average number of queries against I3D on the UCF-101 dataset, which is different from Table 2 in your paper. Note that this result is comparable to the performance of the proposed algorithm.\n\n### Final thoughts\n\n- The paper proposes a simple method to effectively finds search directions for estimating NES gradients. However, there are critical points on the novelty and reproducing the baseline. Therefore, I marginally accept this paper.\n\n### Reference\n[1] Hu Zhang, Linchao Zhu, Yi Zhu, and Yi Yang. Motion-excited sampler: Video adversarial attack with sparked prior. In ECCV 2020\n\n### Update\n\n- Thank you for the clarification. Most of my concerns has been addressed and I raise the score. Please include the results of ablation study (and its empirical analysis) that the performance gain comes mainly from the dilation operation to make the paper stronger.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "### Suggestion\n\n- Ablation study: Similar to the experiment in Section 5.2,  I suggest the authors conduct an ablation study that compares the performance of the proposed transformation (translation + dilation) with translation only or dilation only. The authors need to clarify why translation and dilation should be combined.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a black-box adversarial attack method for video classifiers. The challenge is to reduce the search space (T x C x H x W) for attack efficiency measured in number of queries. The idea is to employ geometrics transformations to parameterize and reduce the search space. The experiments demonstrate significant improvement in higher success rate at fewer queries compared to the baseline methods. I suggest the authors incorporate the suggestions raised by the reviewers in the camera ready version. Please include a) plot of query budget versus success rate in addition to the tables 2 & 3; b) the ablation study for the results on translation only and on dilation only.
Decision: Accept (Poster)