Temporal Relevance Analysis for Video Action Models
Authors: Quanfu Fan
Donghyun Kim
Chun-Fu Chen
Stan Sclaroff
Kate Saenko
Sarah Adel Bargal
Venue: Submitted to ICLR 2023 (2023)
Keywords: Temporal analysis
Frame relevance
Video data
Action recognition

Abstract: In this paper, we provide a deep analysis of temporal modeling for action recognition, an important but underexplored problem in
the literature. We first propose a new approach to quantify the temporal relationships between frames captured by CNN-based action models based on layer-wise relevance propagation. We then conduct comprehensive experiments and in-depth analysis to provide a better understanding of how temporal modeling is affected by various factors such as dataset, network architecture, and input frames. With this, we further study some important questions for action recognition that lead to interesting findings. Our analysis shows that there is no strong correlation between temporal relevance and model performance; and action models tend to capture local temporal information, but less long-range dependencies.

Under review as a conference paper at ICLR 2023
TEMPORAL RELEVANCE ANALYSIS FOR VIDEO ACTION MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we provide a deep analysis of temporal modeling for action recog- nition, an important but underexplored problem in the literature. We first propose a new approach to quantify the temporal relationships between frames captured by CNN-based action models based on layer-wise relevance propagation. We then conduct comprehensive experiments and in-depth analysis to provide a better understanding of how temporal modeling is affected by various factors such as dataset, network architecture, and input frames. With this, we further study some important questions for action recognition that lead to interesting findings. Our analysis shows that there is strong correlation between temporal relevance and model performance; and action models tend to capture local temporal information, but less long-range dependencies.
INTRODUCTION
State-of-the-ard action recognition systems are mostly based deep learning. Popular CNN-based approaches either model spatial and temporal information jointly by 3D convolutions (Carreira & Zisserman, 2017: Feichtenhofer, 2020; Tran et al., 2015) or separate spatial and temporal modeling by 2D convolutions (Fan et al., 2019; Lin et al., 2019) in a more efficient way.
One of the fundamental keys for action recognition is temporal modeling, which involves learning temporal relationships between frames. Despite the significant progress made on action recognition, our understanding of temporal modeling is still significantly lacking and some important questions remain unanswered. For example, how does an action model learn relationships between frames? Can we quantify the amount of temporal relationships learned by an model? Stronger backbones in general lead to better recognition accuracy (Chen et al., 2021: Zhu et al., 2020), but do they learn temporal information better? Do models capture long-range temporal information across frames?
In this paper, we provide deep analysis of temporal modeling for action recognition. Previous works focus on performance benchmark (Chen et al., 2021; Zhu et al., 2020), spatio-temporal feature visualization (Feichtenhofer et al., 2020; Selvaraju et al., 2017) salieny analysis (Bargal et al., 2018; Hiley et al., 2019b; Roy et al., 2019; Wang et al., 2018) to gain better understanding of action models. For example, comprehensive studies of CNN-based models have been conducted recently in (Chen et al., 2021; Zhu et al., 2020) to compare performance of different action models. Others (Monfort et al., 2019b; Selvaraju et al., 2017; Zhou et al., 2016) focus on visualizing the evidence used to make specific predictions, sometimes posed as understanding the relevance of each pixel on the recognition. In contrast, we aim to understand how temporal information captured by action models, ie.. temporal dependencies between frames or how frame relates to other frames in video clip.
In this work, we propose new approach to evaluate the effectiveness of temporal modeling based on layer-wise relevance propagation (Gu al., 2018; Montavon et al., 2019), a popular technique widely used for explaining deep learning models. Our approach studies temporal relationships between frames in an action model and quantify the amount temporal dependencies captured by the model, which is referred to as action temporal relevance (ATR) here (Sec. 3.2). Fig. illustrates our approach. We conduct comprehensive experiments on popular video benchmark datasets such as Kinetics400 (Kay et al.. 2017) and Something-Sometting (Goyal al., 2017) based on several representative CNN models including 13D (Carreira & Zisserman, 2017), TAM (Fan et al., 2019) and SlowFast (Feichtenhofer et al., 2018). Our experiments provide deep analysis of how

---

Under review as a conference paper at ICLR 2023
temporal relevance is affected by various factors including dataset, network architecture, network depth, and kernel size as well as the imput frames (Sec. 4.2). Finally, based on the performed analysis, we effort to deliver deep understanding of the important questions brought up above (Sec. 4.3). We exclusively focus on CNN-based approaches for action recognition. Nevertheless, are fully aware that the recently emerging transformer-bant
approaches such as (Amab al., 2021; Bertasius et al., 2021; Fan et al., 202 Liu et al., 2021) demonstrate comparable or better performance than CNN-based models, but studying transformers beyond the scope of this work. We summarize our contributions below:
Tool for Understanding Action Models. We present a new approach for better understanding of action modeling and develop means of evaluating the effectiveness of temporal modeling.
Temporal Relevance Analysis. We conduct comprehensive experiments to understand how tem- poral information in video is modeled under different settings.
Deep Understanding of Temporal Modeling. We study some fundamental questions in action recognition that leads to interesting findings: a) There no strong correlation between temporal relevance and model performance. Instead, temporal relevance is more related to architectures. b) Action models behave similarly on both temporal and static actions defined by human (Sevilla- Lara et al., 2019), and there is strong indication that temporal actions require stronger temporal dependencies learned by these models. c) As the number of input frames increases, action models capture more short-range temporal information (local contextual information), but less long-range dependencies. The better performance due to using more frames seems to be largely attributed to richer local contextual information, rather than global contextual information.
2 RELATED WORK
Action Recognition Models. Action recognition has achieved significant progress after the release of large-scale publicly available video datasets including Kinetics (Kay al., 2017), Something- Something V2 (Goyal et al., 2017), Sports M (Karpathy et al., 2014), Cooennts-In-Tinee (Monfort al.. 2019b), etc. Many models proposed different temporal modeling approaches handle the temporal dynamics of a video (Carreira & Zisserman, 2017; 7; Fan et al., 2019; Feichtenhofer, 2020; Feichtenhofer al., 2018: Hussein et al.. 2019; Lin et al., 2019; Luo & Yuille, 2019; Tran et al., 2015; 2019; Wang et al., 2020; 2016; 2018; Xie et al., 2018; Zhou et al., 2018). Chen et al. and Zhu et al. provided a comprehensive survey on how those and compare their model accuracy. Transformer-based models have also become popular after their introduction to the computer vision community (Dosovitskiy et al., 2021). addition, multiple recent eitention-based temporal modeling works have been proposed to enhance the transformer- based models, e.g., MViT (Fan et al., 2021a), TimeSformer (Bertasius et al., 2021), Video Swin (Liu et al., 2021), etc (Li et al., 2021; Neimark et al., 2021). The aforementioned works justify their capability of temporal modeling or the range of temporal modeling y validating the performance on benchmark datasets. In this work, we focus on quantifying the temporal relevance between each frame pair learned by a model to help understand the effects of different CNN-based approaches of temporal modeling for the task of action recognition.
models achieve temporal modeling
Model Analysis. There are a few works that have assessed the temporal importance in video, egg. Huang et al. proposed the method approaches to identify the crucial motion information in a video based on the C3D model and then used it to reduce sparse frames of the video without too much motion information; on the other hand, Sigurdsson by measuring the complexity at different levels, such as verb, object and motion complexity, and then composed those attributes to form the action class. Feichtenhofer et al. visualized the features learned from various models trained by optical flow to explain why the network fails in certain cases.
al. analyzed the action category
On the other hand, the receptive field is typically used to determine the range of network can theoretically see in both spatial and temporal dimensions. Luo al. thoroughly studied the spatial receptive field for image classification, and showed that the effective receptive field is much smaller than the theoretical one. In contrast, our work proposes an approach to quantify the learned temporal relevance between each frame pair, and uses this to understand how model architecture affects the temporal relevance.
Explainability. Another popular research direction is to explain the decision made by a model through visualization of the class activation map, egg., CAM (Zhou al., 2016), Grad-
2

---

Under review as a conference paper at ICLR 2023
Borkproppggte earh frome
Snow@oording video clip
Logits for each frome for each of the if classes
Rengoral relevance
Ayfonce vite
Action Model
f
CLAP Temporol Aeievance
fz
N M W xh
Action Model
fi
f2
fa
fz
Temporal relevaace with segateed diouglettion laper
Action Model
f4
fN
Wiy
(a) Inference
[b] Computing Relevance
Figure I: (a) A CNN takes a video with N frames and outputs the average (or ensemble) predictions from all frames the final prediction. (h) CLRP computes for input frame the relevance scores of the target class and other classes by LRP, respectively. The positive differences of the two relevance frame is related to other frames. Temporal relevance obtained to indicate the range of temporal dependencies of the frame on others.
represent how
CAM/GradCAMH+ (Selvaraju et al., 2017; Chattopadhay et al., 2018), and layer-wise relevance propagation (LRP) (Binder et al., 2016) approaches. Binder et al. proposed the layer-wise relevance propagation (LRP) show the relevance between each pixel and the predicted class; moreover, trastive LRP further considers the effects of non-predicted classes to enhance the relevance to the targeted class (Gu et al., 2018). Several works extend LRP for spatio temporal explainable meth- ods (Anders et al., 2019; Hiley et al., 2019;;b; Srinivasan et al., 2017). Hiley et al. surveyed a few frameworks that are used to explain action recognition models based on LRP or CAM approaches. Roy et al. (Nourani et al.. 2020; Roy et al., 2019) add a probabilistic model on top of the action model explain the meaning of each sub-action during the inference following Aakur et al.. Huang et al. study the effect of motion in action recognition by checking the accuracy drop from frames without using motion. Price & Damen explore frame-level contribution to the model output with El- ement Shapley Value. It should be noted that most of prior works try to find salient apatio-temport regions for action recognition while our work quantifies how a model learns temporal relevances and investigate framm- relationship based its architecture, opposed to why model makes a specific prediction.
METHODOLOGY
In this section we explain our approach quantify temporal information captured by action models. We use the terms temporal relevance the range of temporal dependencies for this, Our approach based on layer-wise relevance propagation (LRP) (Binder et al., 2016) and contrastive layer-wise relevance propagation (CLRP) (Gu et al., 2018), which we briefly describe below. We develop novel extension that enables CLRP to measure the temporal relevance between frames and apply to analyze CNN-based action models.
3.1 CONTRASTIVE LAYER-WISE RELEVANCE PROPAGATION
LRP highlights which spatial or temporal input features are relevant to the predictions. LRP back- propagates signals from the prediction layer with pre-defined propagation rules. Relevance scores at layer j (R2) are propagated to the previous layer (Rt) in neural network (employing a ReLU activation) following (Montavon al., 2019):
R, --E
agWiy R,, E a.w,
Σ
Eo,
(1)
where a; is the post-ReLU activation and wij is the weights connecting layer to layer j. LRP back- pook-propsgates the logit 4 of the target class l to the input X to get a relevance score R = hLRP(X, W, 4t). We positive weights (i.e., 2 rule (Gu et al., 2018)) for convolutional and linear layers to remove noise in the gradients. Similarly, we apply the z rule (Gu et al., 2018) to the input layer. We sum up all the pixel relevances in a frame compute the frame-level relevance.
3

---

Under review as a conference paper at ICLR 2023
As discussed in (Gu al., 2018), pixels with a high relevance value does not necessarily contribute to the target class only but to all the action classes. To mitigate the signals relevant to non-target classes, we extend contrastive layer-wise relevance propagation (CLRP) (Gu et al., 2018) for tem- poral analysis. Assume that we have weights W = [W' W ,....., W
W }. where W connects the (L- - 1)-th layer and the neuron for the class k. To offset signals for non-k classes, we construct W = {W*, W WL-1,
1 -1 * WE} by negating the sign of W and compute
R = hLRP X, W, .4). Finally, CLRP is then defined as follows:
heLRr = max (0. (R-R)).
3.2 ACTION TEMPORAL RELEVANCE (ATR)
It is generally believed that long-range temporal information is beneficial for action modeling. Ac- cordingly, the temporal receptive field of a model is expected to cover all the relevant frames. For 3D convolutions, the activation of each neuron for a specific frame depends on the activations of neu- rons from other frames. We define the range of this temporal dependency on other frames as action temporal relevance (ATR). ATR is closely related to the effective receptive field (ERF) discussed in (Luo et al., 2016), but in the temporal domain. The spatial receptive field is largely dependent on the network depth and the temporal kernel size, but pointed out in (Luo al., 2016), the actual/effective receptive fields of a unit in 2D-CNNs, covers only a fraction of the theoretical re- ceptive fields. In this work, we analyze the effective temporal receptive field for a video in temporal modules. Our approach also discovers the strength of temporal dependencies between frames. In ad- dition, should be noted that previous explainable methods on action modeling (e.g., (Bargal al., 2018; Ramanishka et al., 2017)) focus on finding spatially or temporally salient features while we aim to find the effective temporal range of a specific frame that important for action classification.
Temporal Relevance Computation. For a CNN-based action model, it takes N sampled frames from a video as input and ensembles (or averages) the predictions of the N frames as the final output (Fig. 1 (a)). We aim to obtain a temporal relevance matrix ANN computed from CLRP where ag represents the relevance of the j-th frame to the i-th frame. Note that the temporal relevance score indicates how strong the temporal dependence is between two frames. Higher Og (self-dependence) indicates shorter temporal dependence, where a model does not require other frames for its prediction.
{a:yli,j = ....N.
Given an action model, for each input frame ft, we compute the relevance score aij = hcLRP(]j, W,la) lax) between the frame i and j. where W is a set of model weights and lis is the logit of the frame f; for the class k. Note that the summation of 7th row of ANNN is the logit of the given frame fi for the action class kt i..., lik = Ej aij. because LRP is conservative. We com- pute the ATR r; of a frame ft as the length of the shortest continuous segment {fi.... ... ft. ft.-.. : f with the sum of relevance scores that exceeds a certain amount of the total relevance score of the video, iee, ri -1 i.e., 97.5% to cover relevant frames as many as possible.
1 ++ 1 and (Ef=( ()))/(/ a;j) 2 a. We empirically set to high value,
We further define two metrics at video level: (1) avg-ATR, the average of the ATRS of all the frames weighted by their logits (only positive logits 1,t considered). i.e,, avg-ATR= Di=N w; r; where w; = SSEN of and (2) AaX-ATR, the maximum ATR among all the frames, i.e., max-
ATR= (r. Note the logit-based normalization is necessary as it removes the effects of negative and small logits and makes ATR comparable across videos (see Fig. 2). To summarize, the relevance matrix provides frame-level ATR (i.e., effective temporal receptive field) while avg- ATR and max-ATR are video-level temporal relevance, which will be mostly used for our analysis. Moreover, we also the evance, which averages the video-level ATR accordingly. Higher ATR indicates higher temporal modeling capability of model.
ATTR/mx-ATRR for the dataset-level and class-level temporal rel-
4 EXPERIMENTAL SETUP AND ANALYSIS
4.1 DATASETS AND MODELS
Datasets. We mainly use Kinetics-400 (Kinetics) (Kay et al., 2017), Soathiing--omthingg V2 (SSV2) (Goyal et al., 2017) and MiT' (Monfort al., 2019a) in this analysis. Kinetics, arguably
4

---

Under review as a conference paper at ICLR 2023
the most popular benchmark dataset for action recognition, contains 400 activity classes; the other hand, SSV2 contains 174 classes for human-object interactions, and the videos are captured without strong background information. MiT contains 339 action classes including activity classes and human-object interactions. Most of our analysis in this work is based on models trained with the full Kinetics and SSV2 datasets. Due to the high computational costs for model training, also use mini-version of these datasets based on (Chen et al., 2021) for some experiments of this work, but only when necessary. In addition, we always compare models trained from full datasets or mini datasets for fairness.
CNN-based Models. In this work, we choose TAM2D (Fan et al., 2019) and I3D (Carreira & Zis- serman, 2017) as representative approach for 2D-CNNs and 3D-CNNs, respectively. While 13D is computationally inefficient, still serves as the basis for many recent action recognition approaches such as SlowFast (Feichtenhofer et al., 2018) and TPN (Yang et al., 2020), and surprisingly performs on par with them, as shown in (Chen et al., 2021). Similar to (Chen et al., 2021), I), we remove all the temporal pooling layers in I3D they hurt model performance. TAM2D is one of the efficient 2D- CNN models where temporal modeling is separated from spatial modeling. It is a generalized form of the popular TSM (Linn et al., 2019) and demonstrates strong performance the temporal aggregation module in TAM is indeed equivalent to 3DD-dephhwise convolution (Chen al., 2021), which has been adopted by SOTA approaches such X3D (Feichtenhofer, 2020) and CSN (Tran et al., 2019). We thus believe that studying these two representative video architectures can provide deep and helpful insight into how temporal dependencies are captured by action models. Additionally, we put an analysis of SlowFast in Appendix.
SSV2. Interestingly,
Training and Evaluation. We follow the training and evaluation protocols (Chen et al., 2021) for both full datasets and mini-datasets. We adopt the uniform sampling to sample the frames from video to model input. The uniform sampling first divides the whole video seqeunces into segments and then take one frame per segment to get F-frame input. In evaluation, we use the single-clip setting for performance evaluation. More details can be found in Appendix.
4.2 TEMPORAL RELEVANCE ANALYSIS
All the models in our study based on uniform sampling. Compared to dense sampling that sees only a small portion of video, uniform sampling covers significantly more of the video, so it is more suitable for studying long-range dependencies in temporal modeling. In Appendix, we provide a comparison of the two sampling strategies.
Our analysis contains set of models trained with different number of input frames (i.e., 8, 16, and 32) and backbones (i.e., ResNet18, 18, ResNet50, ResNet101 and Inception-Vi] A model denoted by X- X-Y-f[Z) where X, Y, and Z are the temporal module (i.e., I3D or TAM2D). backbone network and input frame number, respectively. For example, 13D-R50-f8 indicates an I3D model using ResNet50 as backbone and 8 frames as an input.
We compute the avg-ATR for video clips with correct predictions score > 0.5 and average the video- level avg-ATRs to obtain dataset-level ATRs. We first investigate how temporal modeling is affected by different datasets as well as various cebitecture-relatee factors including temporal module, back- bone, temporal kernel size, and network depth, and the number of input frames. Based on this, then focus on addressing several important questions of temporal modeling, which largely remain unclear in the field of video understanding.
Dataset. It is generally known that actions in SSV2 indicate strong tem- poral dependencies while Kinetics actions are scene-dependent and less temporal. Our results in Table I con- firm this nicely, consistently showing a larger average (and maximum) tem- poral relevance on SSV2 than on Ki-
Table 1: avg-ATR, max-ATR and model accuracy on different datasets. The red number indicates the different from Kinetics.
Model
Acc. (%)
avg-ATR
maX-ATR
Kineties
SSV2
Kineriex]
SSV2
Kinetics
SSV2
13D-R50-18
69.6
58.9
5.5=0.4
6.550 (+1.))
7,0+0.2 7.7+0.5 (+0.7)
JAM2D-R50-55
70.5
60.2
4.2-04
(
5.2+0.5
7.0+0.3 (+1.8)
13D-R50-116
72.5
62.2
6.6=0.5
8.2=0.5
(+1.6)
8.420.7
9.340.6 (+0.9)
TAMMI--R50-16
73.1
63.0
4.9204
7.320.6 (42.4)
6.1.0.88 8.310.7 (+2.2)
netics by I3D-R50. Fig. 2 (a) and (f) further illustrate the temporal relevance heatmaps of the two datasets, which suggest that a frame is temporally related to only a few neighboring frames (i.e., lo- cal dependencies). Nevertheless, the two datasets present striking differences in temporal modeling. Firstly, on Kinetics, all input frames make positive contributions to recognition, with more from the beginning and end frames of video. Conversely, SSV2, the frames in the middle play a more
5

---

Under review as a conference paper at ICLR 2023
(a) Kinetics
(b) Smoking hookah
Presenting
Jumping
(c) Singing
weather
pool
(g) Spilling (f) SSV2
(h) Approach
(i) Putting 1th
Sih colliding
to sth
cam
Ath
Figure 2: Heatmaps dataset-level and class-level avg-ATR for 13D-R50-f16. heatmap shows each frame is relevant to other frames, with lighter colors denoting higher relevance. Column demonstrates that models learn a larger avg-ATR SSV2 (I) than Kinetics (a). The first and second rows depict avg-ATR of individual classes from Kinetics (b)-(e) and SSV2 (g)-())
substantial role for recognition. Secondly, SSV2 shows significantly more relevances on the diago- nal, hinting that larger frame differences exist in the data than in Kinetics. Fig. 2 (b)-(c) and (g)-() show the heatmaps for a few different actions, which present noticeable variations between individ- ual classes. Similar results are observed with TAM2D. On mini-MiT, we observe that its ATR is slightly higher than that of Kinetics but lower than that of SSV2. Due to space limitations, the results of mini-MiT can be found in Appendix.
Temporal Module and Backbone. The key difference between 2D-CNNs and 3D-CNNs resides in the fact that 3D-CNNs jointly learn spatial and temporal information. The temporal modules action models are closely related to the backbones adopted by these models, and we investigate the effects of both jointly. Fig. 3 shows how the temporal relevance differs be- tween I3D and TAM2D using the same input frames and ResNet50 backbone. This clearly indicates that 13D yields a larger relevance than TAM2D, and the gap is more significant Kinetics. Since the temporal module of TAM2D is essentially depth-wise 1D temporal convolution, ex- changes temporal information along the same spatial location at the same channel only. Differently, I3D uses a 3D convolution that allows for interactions larger scope. thus understandable that TAM2D is not as effective as I3D in capturing temporal dependencies.
sug-a11 Diff
1.0
0.5
16
32
16
Number Frames
Figure 3: avg-ATR difference between I3D-R50 and TAM2D-R50, (avg-ATR(I3)) avg-ATR(TAM2)) different number of frames. presents larger tempo- ral relevances.
the other hand, stronger backbone net- works in general lead to better spatio- temporal representations and action recog- nition performance (Chen et al., 2021). What is still unclear whether a better backbone enables more temporal interac- tions for an action model.
mini-S9V2 mini Kinet
RLAN
16
DD-Roap
TARRD-R50
08-40
TAMSnU-i to
uD-Rdo
TARID-1
In Fig. we compare the avg-ATR cap- tured by models trained on mini datasets using different backbones including Inception-VI, ResNet18 and ResNet50, The results are sorted by avg-ATR from high to low. As seen from the figure, while the backbone networks present a similar order on two datasets, the order is somewhat random, and indicates strong correlations with the strength of the backbones. By closely examining the temporal modeling I3D and TAM2D, find that their differences could result in different number of temporal convolutions even with the same backbone, as shown by the numbers above the bars in Fig. 4. For example, 13D-R18 has 8 residual blocks, each of which has 2 3x3x3 convolutions, yielding total of 16
Figure Effects backbones on temporal relevance (16- frame models). The number on top of cach bar indicates the number of temporal convolutions in model while the num- her the side is the model accuracy. This shows that avg- ATR does not correlate strength of hackbones.

---

Under review as a conference paper at ICLR 2023
temporal convolutions. On the other hand, TAM2D-R18 has one temporal module following each residual block, giving a total of 8 temporal convolutions only. The number of temporal convolutions seems better explain the differences of temporal relevance between these models. Interestingly, 13D-Inception indicates the most effective temporal modeling capability with the largest temporal relevance. This is largely because of the 7x7x7 kernel used the beginning of the model, which greatly enhances 13D-Inception's 's ability to learn temporal information. As shown next, kernel size influences temporal modeling significantly.
To summarize, there is no strong correlation between ATR and model performance. We also observe that the SlowFast shows a similar trend. but with smaller avg-ATR than I3D and TAM2D due to architectural differences Sec. B.
8.5
8.0
7.5
7.0
13D-126
Network Depth. Fig. 5 shows that the avg-ATR is increased by network depth, but rather insignificant (< 0.5) consider- ing the notable growth in network depth (more precisely, the number temporal modules). We hypothesize that this has to do with the model ensemble effect discussed later in Sec. 4.3, which makes it less desired for a model to learn long-range temporal information.
6.5
130-132
T-M20-432
6.0
101
152
ResNet Depth
Figure 5: Effects of network depth SSV2. There is very marginal in- crease of avg-ATR by network depth.
Table 2: Effects of temporal kernel size on mini-SSV2.
Temporal Kernel Size. The convolutional kernel size of a temporal module directly controls how many frames are in- volved in temporal modeling. Intuitively, it should have a sig- nificant direct impact on temporal relevance, which already seen in I3D-Inception. To further validate this, we ex- periment with temporal kernels of 5, and 7 in TAM2D on mini-SSV2 in Table 2. As expected, a larger kernel size produces larger temporal relevance, and the increase in the case of 16 input frames is quite pronounced, compared to the impact of net- work depth on temporal relevance. Nevertheless, this table shows again that there is no correlation between temporal relevance and model accuracy.
Kemel size
TAM2D-R500 R50-
Ace. (%)
mvg-ATR max-ATR
have
6.3
7.4
5
64.1
7,8
8.0
7
8.0
8.0
Video dlip length
Input Frames. CNN-based action models usually require large number of input frames to achieve competitive performance (Carreira & Zisserman, 2017: Fan et al., 2019; Feichtenhofer et al., 2018; Chen et al., 102 It would be an intuitive and rea- sonable thought that a strong model would be able to capture long-range temporal dependencies better in such a case. As demonstrated in Fig. 6, when in- creasing the number of input frames, the avg-ATR is indeed increased accordingly, but at quite a slow pace. Quadrupling the input from frames to 32 frames only results in an increase of temporal rel- evance by ~2 frames. Nevertheless, when looking at the temporal relevance over the video clip length, a model using more input frames learns shorter- range temporal information (also see Fig. 2), which counter-intuitive. In what follows, we first vali- date our observations here, and then conduct further analysis of this in Sec. 4.3, wherein we relate this surprising finding to model ensembling. It should be noted that the long-range context we investigate here is within a single action and its temporal dependencies captured by models, not the long-term temporal information studied in (Wu et al., 2019) that is across actions in a video. We provide additional extensive and comprehensive analyses and results in the appendix.
avg- ATR:
32-frames
8.9
16-frames
aug ATR:8.0
8-frames
avg-ATR:
Figure 6: An illustration of the effect of input frames on temporal relevance of 13D-R50 trained Kinetics. More input frames lead larger temporal relevances (avg-ATR). i.e., the absolute length of the highlighted frames. However, a model captures range temporal dependen- cies. i.e. the relative length of the highlighted windows to the entire video clip length.
Additionally, we evaluate the captured ATR from our approach with a partial uniform sampling approach in Sec. A of the Appendix. The experiment also indicates that only a portion of entire input frames captured by ATR suffice for an action prediction (Fig. 12 in the Appendix).
7

---

Under review as a conference paper at ICLR 2023
4.3 WHAT ARE ACTION MODELS LEARNING?
We have provided an in-depth analysis in terms of how temporal information is learned by CNN models and how architecture-relate factors affect temporal modeling. Our approach is related to the effective receptive field (ERF) proposed in (Luo et al., 2016), in that the frame-level temporal relevance in our case can be considered as the ERF in the temporal domain. Our approach shows similar results to ERF with regards to kernel size, but our work focuses more video understanding. In this section, we further examine some important questions related to temporal modeling for video understanding based on our analysis above.
1) Do betfer-performing action models capture temporal dependencies better?
CNN-based approaches have made significant progress in action recognition and many of them claim to have achieved better spatiotem- poral representations for video understanding. However, the question remaining unanswered whether these approaches have actually enabled better temporal modeling. Based on the analysis above, our answer is prohably No. For example, 1) TAM2D has lower avg-ATR than I3D general, but its performance is on par with or better than I3D (Fig. 3); 2) 13D-Inception produces significantly larger temporal relevance than I3D-R50, but its performance is not as good 13D-R50 (Fig. 5); and 3) larger kemels lead to larger relevance, but not necessarily accuracy increase (Table 2). We further plot model accuracy vs. avg-ATR for 15 models on mini-Kinetics and mini-S5V2 in Fig. 7. The Pearson correlation coefficients are close to zero on both datasets, clearly indicating no correlation between accuracy and avg-ATR.
Yop-1 Accuracc (%
78
7.6
00.004
Coaft:
Tpp-1 Accuracy 2%6
69
67
Pearson Coeft
-2
74
65
663
61
59
57
10 avQ-ATR
12
8
10
12
aVg-ATR
(a) mini-Kineticc
(b) mini-SSV2
Figure 7: avg-ATR vs. accuracy among different mod- els based on mini datasets. The Peasron correlation efficient indicates that there is no correlation between accuracy and avg-ATR.
2) Do action models learn temporal informa- tion differently by temporal dynamics of ac- tions? The temporal dynamic of an action sug- gests how much temporal information is needed to correctly recognize Temporal and static datasets annotated by hu- mans from Kinetics and SSV2 were constructed for temporal dynamic analysis (Sevilla-Lara et al., 2019). The temporal dataset consists of classes where temporal information matters while the static dataset includes classes where temporal information is redundant.
9.0
mu 1o of cossess
6.-
8.5
E
a.o.
g7
action by human.
7.0
349 ATR 51d
6.5-
a0g-A11
aug-ATR shd
6.0
0
25 50
100 125
150
0.5
9.0
10.0
10.
(a) AVg-ATR
(b) max-ATR
Figure 8: (a) The distribution of avg-ATR and (b) the histogram of max-ATR on I3D-R50-f16 over classes for SSV2. Both ATRs do not show significant differences between classes.
It is then an interesting question understand whether an action model requires (or relies on) more temporal information to classify temporal actions. In Fig. 8, we show the per-class ATR for SSV2 classes (avg-ATR) in sorted order along with its standard deviation. For the majority of classes, their avg-ATRs are between 7.7 and 8.2, indicating no significant difference. Similarly, the difference of max-ATR is not greater than 1.0. This seems to suggest that an action model learns close amount of temporal information for most classes regardless of the temporal dynamics of actions. We inves- tigate this question further by analyzing the temporal dynamics of action classes based on temporal relevance, i.e., machine perception rather than human perception.
Specifically, we sort all the actions in dataset from high to low by tempo- ral relevance, and then pick the top-k and bottom-k classes as the most tem- poral classes and static classes, re- spectively.
Table 3: The class overlap of the temporal (T) and static (S) ac- tions identified by human and machine.
aTR
max- ATR
Model
S8V2
Kineties
SSV2
S
T
8
S
T
S
B5D-R50-F6
22.24
11.1%
15.6%
3.1%
11.1%
5.6%
12.5%
3.1%
TAM2D R50-416
38.9%
16.7%
18.8%
6.2%
27.8%
11.1%
21.9%
6.2%
Table 3 shows the overlap per- centages of the temporal and static
A elasses
18
32
32
18
18
32
32
8

---

Under review as a conference paper at ICLR 2023
datasets based on human and machine, respectively. Surprisingly, human and machine do not agree much with each other, especially on Kinetics and static classes.
We further plot the temporal rele- vance of the human-based tempo- ral and static classes in descending sorted order in Fig. 9. It can be seen that both temporal and static classes are scattered, indicating the tempo- rality of an action determined by human is not strong indicator of the temporal information needed by an action model for the action. In ad- dition, there are many static classes (blue) with large temporal relevance.
Teeeeo (22 cooen stane cumMe
Tempara (B disse)
8.6
Sall: 11 dasmi
6.8
0.4
6.6
0.2
6.2-
8.0
7.8
280
400
50
100
Classes
Classes
(a) Kinetics
(b) SSV2
Figure 9: Sorted avg-ATR of human-defined temporal and static actions (Sevilla-Lara et al., 2019) based on 13D-R50-f16. The scat- tered human-defined temporal (or static) actions suggest that the temporality of action is not good indicator of the temporal in- formation captured by a model.
Overall the large discrepancies from both datasets imply that the temporal information perceived by a human as useful for the recognition might not be the same as what an action model attempts to learn. There is strong indication that temporal actions require stronger temporal dependencies learned by a model (or static actions use less temporal information).
3) What temporal dependencies are learned by action models? Long-range temporal information in video data has long been considered crucial for action recognition. While many approaches (Wang al., 2018) have attempted to capture such global dependencies across frames, but there is no easy way to understand whether the improved performance of these approaches is actually attributed to better temporal modeling. Our method analyzes the temporal dependencies.
Our analysis above points to surprising find- ing that CNN-based approaches focus more
9.5
on learning local temporal information rather
9.0
8.5
1415
than capturing long-range dependencies, indi-
8.0-
55.0%
cated by a) the effect of input frames (Fig. 3); b) deeper models not resulting (notably) larger temporal relevances (Fig. 5); and c) the evaluation based on partial uniform sampling (Fig. 12). We conjecture that this model behav- ior has to do with the frame ensembling previ- ously shown in Fig. 1, which might be sufficient to provide long-range spatio-temporal features needed by an action model. We design sim- ple experiment to validate this. Specifically, we retrained few 16-frame models by only taking 1 frame or an average of evenly spaced 2, 4, and 8 frames as the model output. We expect as the number of frames used for ensembling is reduced, the model will resort to capturing more temporal interactions between frames to avoid dramatic performance degradation in such case. As seen in Fig. 10, the temporal relevance increases as expected when fewer frames are involved in the en- semble. This experiment indicates that the better performance of model using more input frames seems to be largely attributed to the ensemble of richer local contextual information learned by the model, rather than modeling global contextual information.
7.0
7.5
50.2%
41.9%
62.5%
6.5
Number of Frame Ensemble
Figure 10: Effects of frame ensemble on SSV2. Mod- els (TAM2D-R50-16)) trained using different num- frames for ensembling during prediction. The top-1 accuracy of corresponding models are annotated. The avg-ATR increases as fewer input frames are used for prediction, suggesting that model in such case capture more temporal information for recognition.
5 CONCLUSION
We have proposed an approach to quantify temporal modeling for action recognition. To the best of our knowledge, this is the first work that investigates the effective temporal receptive field, i.e., action temporal relevance (ATR), in action models. Based on this, we conduct an in-depth analysis of how temporal information is captured for action recognition, which provides better understanding of action modeling. Contrary to the common belief that long-range temporal information can be captured by state-of-the- the-ttt-rrtt action models, temporal information, and capture less long-range (global) temporal information. We also find that larger ATR does not necessarily lead to better accuracy.
observe that action models rely short-range (local)

---

Under review as a conference paper at ICLR 2023
Code of Ethics and Ethics statement. Our work is analyze and investigate the temporal rele- vances of widely used temporal action models. We believe that there are no ethical concerns related to this work.
Reproducibility. We provide additional training details and LRP rules in Sec. F and G of the Appendix. Our implementation is based on the captum library (cap). We will make code publicly available upon acceptance.
REFERENCES
https//github.com/yyytrtttiaiiet
Sathyanarayanan N Aakur, Fillipe DM de Souza, and Sudeep Sarkar. An inherently explainable model for video activity interpretation. In Workshops the Thirty-Second AAAI Conference Artificial Intelligence, 2018.
Christopher J Anders, Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. Understand- ing patch-based learning of video data by explaining predictions. In Explainable Al: Interpreting, Expluining and Visualizing Deep Leurning, pp. 297-309. Springer, 2019.
Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. Vivit: A video vision transformer, 2021.
Sarah Adel Bargal, Andrea Zunino, Donghyun Kim, Jianming Zhang, Vittorio Murino, and Stan Sclaroff. Excitation backprop for mns. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 1440-1449, 2018.
Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is Space- Tiace-Time Attention All You Need for Video Understanding? arXiv.org. February 2021.
Alexander Binder, Sebastian Bach, Gregoire Montavon, Klaus-Robert Müller, and Wojeiech Samek. Layer-wise relevance propagation for deep neural network architectures. In Kuinam J. Kim and Nikolai Joukov (eds.). Information Science and Applications (ICISA) 2016, pp. 913 922, Singa- pore, 2016. Springer Singapore. ISBN 978-981-100 -00-555--2.
Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In CVPR, pp. 0299-6308. 2017.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-
cam++: Generalized gradient-baad based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV), pp. 839-847. IEEE, 2018.
Chun-Fu Chen, Rameswar Panda, Kandan Ramakrishnan, Rogerio Feris, John Cohn, Aude Oliva, and Quanfu Fan. Deep analysis of enn-based spatio-tempon representations for action recogni- tion, June 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference on Learning Representations, 2021. URL https: //openreview. htte:/ooommidddxxooaoa
Facebook. Pyslowfast model zoo: http:///iiib.ooomccccooorrssbbbt
Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer. Multiscale vision transformers. 202
Quanfu Fan, Chun-Fu (Ricarhd) Chen, Hilde Kuehne, Marco Pistoia, and David Cox. More Is Less: Learning Efficient Video Representations by Temporal Aggregation Modules. In NewrIPS, 2019.
Quanfu Fan, Rameswar Panda, et al. An image classifier can suffice for video understanding. arXiv preprint arXiv:2 101.1106.41000 14104, 2021b.
10

---

Under review as a conference paper at ICLR 2023
Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition. In CVPR, June 2020.
Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. arXiv:18/2.03982,, 2018.
Christoph Feichtenhofer, Axel Pinz, Richard P. Wildes, and Andrew Zisserman. Deep Insights into Convolutional Networks for Video Recognition. International Journal of Computer Vision, 128 (2):420-437, 2020. ISSN 0920-5691.
doi: 10.
Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne West- phal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The" something something" video database for learning and evaluating visual common sense. In ICCV, 2017.
Jindong Gu, Yinchong Yang, and Volker Tresp. Understanding individual decisions of cnns via contrastive backpropagation. In Asian Conference Computer Vision, pp. 119-134. Springer, 2018.
Liam Hiley, Alun Preece, and Yulia Hicks. Explainable deep learning for video recognition tasks: A framework & recommendations. arXiv preprint arXiv:1909.05667 2019a.
Liam Hiley, Alun Preece, Yulia Hicks, David Marshall, and Harrison Taylor. Discriminating spatial and temporal relevance in deep taylor decompositions for explainable activity recognition. arXiv preprint arXiv:1908. 01536, 2019b.
De-An Huang, Vignesh Ramanathan, Dhruv Mahajan, Lorenzo Torresani, Manohar Paluri, Li Fei- Fei, and Juan Carlos Niebles. What makes video a video: Analyzing temporal information in video understanding models and datasets. In CVPR, pp. 7366-7375, 2018.
Noureldien Hussein, Efstratios Gavves, and Arnold W.M. Smeulders. Timeception for complex action recognition. In CVPR, June 2019.
Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei- Fei. Large-scale video classification with convolutional neural networks. In CVPR, pp. 1725- 1732, 2014.
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv: 1705.06950 2017.
Xinyu Li, Yanyi Zhang, Chunhui Liu, Bing Shuai, Zhu, Biagio Brattoli, Hao Chen, Ivan Marsic, and Joseph Tighe. VidTr: Video Transformer Without Convolutions. arXiv, 2021.
Lin, Chuang Gan, and Song Han. Temporal Shift Module for Efficient Video Understanding. In ICCV, 2019.
Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin trans- 13230.
former. CoRR, Abs/2106.132333 2021. URL httpt::t//// /ttttp://ariivv wow.g/iiioo....oooooooooooooo...
Chenxu Luo and Alan L. Yuille. Grouped spatial-temporal aggregation for efficient action recog- nition. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), October 2019.
Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. Proceedings of the 30th International Conference Neural Information Processing Systems, pp. 4905 4913, 2016.
Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan. Lisa Brown, Quanfu Fan, Dan Gutfreund, Carl Vondrick, et al. Moments in time dataset: one million videos for event understanding. IEEE transactions on pattern analysis and machine intelligence, 42(2):502-500 2019a.
11

---

Under review as a conference paper at ICLR 2023
Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Yan Yan, Lisa Brown, Quanfu Fan, Dan Gutfreund, Carl Vondrick, et al. Moments in time dataset: one million vidcos for event understanding. IEEE TPAMI, 2019b.
Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Rober Müller. Layer-wise relevance propagation: an overview. Explainable Al: interpreting, explaining and visualizing deep learning, pp. 193-209, 2019.
Daniel Neimark, Omri Bar, Maya Zohar, and Dotan Asselmann. Video transformer network, 2021.
Mahsan Nourami, Chiradeep Roy, Tahrima Rahman, Eric Ragan, Nicholas Ruozzi, and Vibhav Gogate. Don't explain without verifying veracity: An evaluation of explainable ai with video activity recognition. arXiv preprint arXiv:2005.02333, 2020.
Will Price and Dima Damen. Play fair: Frame attributions in video models. In Asian Conference on Computer Vision, pp. 480-497. Springer, 2020.
Vasili Ramanishka, Abir Das, Jianming Zhang, and Kate Saenko. Top-down visual saliency guided by captions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7206-7215, 2017.
Chiradeep Roy, Mahesh Shanbhag, Mahsan Nourani, Tahrima Rahman, Samia Kabir, Vibhav Gogate, Nicholas Ruozzi, and Eric D Ragan. Explainable activity recognition in videos. In IUI Workshops, 2019.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Laura Sevilla-Lara, Shengxin Zha, Zhicheng Yan, Vedanuj Goswami, Matt Feiszli, and Lorenzo Torresani. Only time can tell: Discovering temporal data for temporal modeling. CoRR, hts//1907.08400, 2019. URL //rxiv org/ abss/190 /et./11107004.....
Gunnar A Sigurdsson, Olga Russakovsky, and Abhinav Gupta. What actions are needed for under- standing human actions in videos? In ICCV, pp. 2137-2146, 2017.
Vignesh Srinivasan, Sebastian Lapuschkin, Cornelius Hellge, Klaus-Robert Müller, and Wojeiech Samek. Interpretable human action recognition in compressed domain. In 2017 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1692-1696. IEEE, 2017.
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning Spa- tiotemporal Features With 3D Convolutional Networks. ICCV, 2015.
Du Tran, Heng Wang, Lorenzo Torresani, and Matt Feiszli. Video classification with channel- separated convolutional networks. In ICCV, October 2019.
Heng Wang, Du Tran, Lorenzo Torresani, and Matt Feiszli. Video Modeling With Correlation Networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks: Towards good practices for deep action recognition. In ECCV. Springer, 2016.
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In CVPR, June 2018.
Chao- Yuan Wu. Christoph Feichtenhofer, Haoqi Fan, Kaiming He, Philipp Krahenbuhl, and Ross Girshick. Long-term feature banks for detailed video understanding. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 284-293, 2019.
12

---

Under review as a conference paper at ICLR 2023
Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy. Rethinking Spatiotem- poral Feature Learning: Speed-Accuracy Trade-offs in Video Classification. In ECCV, September 2018.
Ceyuan Yang, Yinghao Xu, Jianping Shi, Bo Dai, and Bolei Zhou. Temporal pyramid network for action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition (CVPR), June 2016.
Bolei Zhou, Alex Andonian, Aude Oliva, and Antonio Torralba. Temporal relational reasoning in videos. In ECCV, pp- 803-818, 2018
Yi Zhu, Xinyu Li, Chunhui Liu, Mohammadreza Zolfaghari, Yuanjun Xiong, Chongruo Wu, Zhi Zhang, Joseph Tighe, R Manmatha, and Mu Li. A comprehensive study of deep video action recognition. arXiv preprint Xiiv:2012.065667 2020.
13

---

Under review as a conference paper at ICLR 2023
duplication of
duplication
Input
f1
14
1
16
fi
16
Action Model
Logits
lus
Prediction for fy
Figure 11: illustration of partial uniform sampling. For a given frame g., .f) a window (e.g. fa
f11) from a set of uniformly sampled input frames. partial uniform sampling replaces the frames before W by the start frame of W (i.e., fa) and the frames after W by the end frame of W (i.e, f11). The modified input is fod into action model to obtain the prediction for f.
Appendix
In Appendix, we provide more details and additional results which are not included in the main paper due to the space limit. We first explore partial uniform sampling for model evaluation in Sec. A. In Sec. B, we investigate a different backbone (i.e., SlowFast) and dataset (i.e., mini-MiT) with our proposed approach (Sec. B). We also compare CLRP with LRP. Sec. C, we provide a comparison between dense and uniform sampling. We present more heatmap visualization in Sec. D and additional analysis on TAM2D in Sec. E. Finally, we give more details model training protocols in Sec. F and show how to set up propagation rules in LRP in Sec. G.
A MODEL EVALUATION BY PARTIAL UNIFORM SAMPLING
Our analysis above suggests that action models does not capture long-range dependencies as ex- pected when the number of input frames are sufficiently large 16). Fig. 2 strongly indicates that frame is only temporally related to a few neighboring frames. This implies that these few frames, rather than the entire input sequence, might suffice for the prediction from frame in model. In light of this, we propose sliding-window window method validate the temporal relevance results pre- sented above and help us understand temporal modeling better.
Recall that an action model usually generates a prediction for each frame separately and then en- sembles the predictions of all the frames by averaging to obtain the final prediction (Fig. 1 (a)). We thus develop a partial uniform sampling strategy, which limits the prediction of a particular frame to only a portion of the entire input frames. The idea is illustrated in Fig. 11.
Specifically, for a target frame fa residing in a window W of the input frames ranging between [L, (1 < S r). modify the input by replacing the frames before by and the frames after by fr. This modified input is then fed into the model to obtain the prediction from f.. Doing so only allows the prediction for fi receive information from partial portion of the input, i.e., frames in the window W. We repeat this for each frame by sliding a window over the entire input and taking the average of all the predictions to be the final model output. Note that our method only keeps the prediction of one frame at a time and the sliding window size can vary by frame. The frame-level ATR in Fig. 12 shows the accuracy of this process. We average frame-level ATRS of all videos to represent the window size. This distinguishes it from the widely used multi-clip evaluation that takes an average of the predictions from all imput frames clip.
We first experiment with fixed sliding window size in partial uniform sampling to evaluate I3D and TAM2D models with 16 and 32 frames on both datasets. As shown Fig. 12, the performance of all the models in the evaluation gets saturated quickly with around 8 frames on Kinetics, and 10 frames on SSV2, suggesting that frame-level prediction indeed does not use all the input. We further experiment with dynamically changing window size based ATR (i.e., each frame based on different window size defined by ATR), and the results (green and blue dots) match well with those based fixed window size. This confirms that measuring temporal relevance by proposed approach is reasonable. We also tried empty images in partial uniform sampling, but the results are worse than using frame image.
14

---

Under review as a conference paper at ICLR 2023
116 ATT
framedeeellA AtM
90
Top-5 Acc.
80
11
Top-5 Acc.
11
Accuracy (%)
Accuracy (96)
60
Tap-1
70
Top-1
11
40
11
60
Top 133--550-6
20
11
11
Top-1/30-655432
Top-1/8D-950-16
Top 5:30
Top 51305 R50
11
T3p.5/30-050482
40
2
10
16 18 22
30
32
2
4
8
10
12
14 16 18
22
26 30 32
(N)
Window (A) Stoe
(a) Kinetics
(b) SSV2
Figure 12: Model accuracy by window size using partial uniform sampling. The results show that frame-level ATR (blue and green dots) correctly indicates that only partial temporal information is needed for the prediction of a frame.
Table 4: avg-ATR, max-ATR and model accuracy on different datasets.
Single-clip Ace. (%)
avg-ATR
max-ATR
Model
Kinetics
SSV2
Kinetics
SSV2
Kinetics
SSV2
13D-R50-18
69.6
58.9
5.5++..4
6.5+0.4
7.0+0.2
7.7+0.5
TAM2D-R50-18
70.5
60.2
4.2++0.4
6.0+0.4
5.2+0.5
7.0+0.3
SlowFast-R500 -55-88880
68.9
3.740..7
5.410.8
13D-R50-f16
72.5
62.2
6.6+t..5
8.2++0.5
| 9.3+
TAM2D-R50-F16
73.1
63.0
4.9+0.4
7.3+0.6
6.1+0.8
8.3+0.7
SlowFast-R50- 150----------668
59.2
4.6++0.6
5.9++0.9
B ADDITIONAL RESULTS
SlowFast Results. The SlowFast video architecture (Feichtenhofer al., 2018) is dual-branch network with a slow branch processing frames at a low frame rate and a fast branch operating at a high frame rate. It is one of the best-performing models on Kinetics (Chen al., 2021). To apply CLPR to SlowFast, we first merge the prediction of the slow branch into the fast one. Specifically, we expand the logits of the slow branch to the same size as the fast branch while keeping the logits the slow frames unchanged and setting all others to O. We then add the expanded logits to the logits of the fast branch, and perform CLRP on top of them for each frame, which produces a X relevance matrix M where n is the number of input frames to the fast branch. Additionally, it's observed that the slow logits are dominant compared to the fast logits, suggesting that the fast branch only plays minor role in recognition. As result, the relevance matrix M is extremely uneven and sparse. To make our analysis meaningful, we sum up the relevance of every r X r block in M (Sec. 3.2) m matrix, where doing so, we mainly focus on the slow branch in our analysis, but without neglecting the small but complementary contribution from the fast branch.
to obtain
n/r r is the number of imput frames to the slow branch. By
With the changes above in our method, we compute the ATRs for two SlowFast models based on uni- form sampling, one model trained by ourselves on Kinetics using 8 slow frames and 32 fast frames SllowFastRR50-8x
and the other one trained by the authors of the original paper on SSV2 using 16 slow frames and 64 fast frames (Facebook). Table 4 lists the results of these two models and their I3D counterparts, and Fig 13 further illustrates their relevance heatmaps. Interestingly, while FlswFast is fed with much more input frames. its ATRs on both Kinetics and SSV2 are substantially smaller than those of I3D, suggesting that the fast branch do not strengthen model's 's temporal mod- eling ability as expected (Feichtenhofer et al., 2018). Closely examining the network architecture of Flsw reveals that its special design aiming at computational efficiency may weaken its temporal modeling ability. Both branches of SlowFast eliminate the 7x7x7 covolution in I3D, and the slow branch introduces temporal convolutions only after the third layer, As illustrated in (Feichtenhofer et al., 2018). Due to that, the slow branch primarily focuses on spatial modeling, not temporal mod- eling. In addition, the lightweight fast branch seems too narrow to effectively capture the temporal dependencies in the input frames. Our analysis provides an explanation of why SlowFast underper- forms on temporal datasets such as SSV2 (Table 4).
15

---

Under review as a conference paper at ICLR 2023
Kinetics: 13D-R50.0.
(b)
(c) SSV2: R50
(d) SSV2: SlowFast R50---t--50-166
Figure 13: Heatmaps of dataset-level avg-ATR for 13D-RS R50 and SlowFast- R500 Kinetics: (a) and (b); SSV2: (c) and (d).
Figure 14: avg-ATR of different backbones trained with 16 frames mini-MiT. We observe the similar trend in mini-MiT. 13D-R50 obtains lower avg-ATR ATR compared 13D-R18. time, TAM2D-R50 obtains higher avg-ATR com- pared to TAM2D-RI R18.
same
ATR
TAMGDALB
TAMIDARD
Results on mini-MiT, We provide additional results on mini-MiT (Monfort al., 2019b). In Fig. 14, we show avg-ATRs different models using frames. We observe a similar trend mini-MiT to the Fig.4 in the main paper. 13D-R50 obtains lower avg-ATR than I3D-R18 while TAM2D-R50 produces higher ATg-ATR than TAM2D-RA Table lists the avg-ATR and max-ATR of different models mini-MiT, showing higher ATRs than those of Kinetics but lower than those of SSV2. This probably because mini-MiT contains both human actions and human-object interactions, indicating intermediate temporality between Kínetics and SSV2.
LRP Results. Table 6 compares CLRP with LRP in terms of avg-ATR and max-ATR. In general, LRP obtains smaller avg-ATR and max-ATR than CLRP, indicating that by removing the signals related to non-target classes, the effective temporal range of input frames becomes longer in order reproduce the original prediction.
C DENSE SAMPLING VS. UNIFORM SAMPLING
comparison with uniform sampling where evenly-spaced frames across an video are extracted as the model input, dense sampling takes consecutive set of frames of given length from video as the input. is another sampling strategy widely used for action recognition, especially on the Kinetics benchmark. The densely sampled input frames highly redundant and only cover small portion of the video, thus practice, dense sampling relies on the ensemble the prediction results from multiple clips (10~30) achieve good results (multi-clip evaluation). In general, studying dense sampling for temporal analysis is less interesting. Here we provide comparison of these two sampling strategies based the Kinetics dataset.
Table 7 lists the avg-ATR and max-ATR of several I3D and TAM2D models based on uniform and dense sampling, respectively. We use the center clip of video dense sampling compute ATRs. Note that the model accuracies shown here are generated by the single-clip evaluation not the multi- clip evaluation generally used for dense sampling. As seen from the table, dense sampling tends to produce smaller averaged temporal relevance (avg-ATR) than uniform sampling, but larger max temporal relevance (max-ATR). We further show the temporal relevance heatmaps the models using 16 frames in Fig. 15. With uniform sampling, both I3D and TAM3D models demonstrate sig- nificant more contributions from the frames at the beginning and end video. This is reasonable in sequence of consecutive frames, the start and end frames more likely to present larger difference. In contrast, the models trained with uniform sampling indicate more contributions from the middle frames (Fig. 15-(a) and (b)).
16

---

Under review as a conference paper at ICLR 2023
Table 5: avg-ATR max max-ATR on mini-MiT.
avg-ATR MiT
max-ATR
Model
MiT
B3D-550f88
5.7:0.55
7.140.6
TAM2D-R50-88
5.5+0..5
7.0+0.44
13D-R50-f16
6.420.7
8.340.8
6.2.0.66
7.0=0.8
Table 6: Comparison of action temporal relevance (ATR) between CLRP and LRP.
CLRP avg-ATR
LRP avg-ATR
CLRP max-ATR
LRP ATR
Model
Kinetics
SSV2
Kinetics
SSV2
Kinetics
SSV2
Kinetics
SSV2
13D-R50-fIC
6.6
8.2
6.3 (-0.3)
7.8 (-0.4)
8.4
9.3
7.3 (-1.3)
8,9 (-1.0)
TAM2D-R50-F16
4.9
7.3
4.6 (-0.3)
6.6 (-0.7)
6.1
8.3
5.1 (-1.0)
(-1.)
D ADDITIONAL VISUALIZATION OF HEATMAP
Fig, 16 demonstrates the heatmaps of more videos from SSV2 and Kinetics. We also visualize the frames and their prediction logit scores shown by red color under each frame (the brighter the color, the higher the logit). Intuitively, predicting an action class in SSV2 (Fig. 16-(a))) requires more frames as the action cannot identified by very few frames. Conversely, in Kinetics, most actions can be easily identified even with few frames, as indicated by Fig. 16-(b). Especially, the examples of "doling lunadry" and "dancing ballet" do not present significant changes or movements objects.
E ADDITIONAL RESULTS FOR TAM2D
Model evaluation by partial uniform sampling on TAM2D. Fig. 17 shows the model evaluation with partial uniform sampling. The finding here is similar to that in the main paper (Sec. A), i.c the performance of the TAM models gets saturated quickly around 8 frames on Kinetics and 10 frames on SSV2.
ATR distributions over classes on TAM2D. Similar to Fig. 8 in the main paper, Fig. 18 shows the per-class ATR for TAM2D-R50-F166 We observe similar behavior that in Fig. 8 of the main paper that the majority of classes do not have significant differences in avg-ATR in (a). In (b), most of the max-ATRs for classes ranged from 9.0 to 10.0.
F TRAINING AND EVALUATION
Here elaborate more how train and evaluate models for our analysis. We follow the training and evaluation protocols in (Chen et al., 2021) for both full and mini mini-dataset. We adopt the uniform sampling sample the frames from video as the model input. The uniform sampling first divides the whole video sequences into F segments and then take one frame per segment to get F-frame input (random frame is used for training whole the center frame for inference). For the full datasets, the shorter side of video is randomly resized to the range of [256, 320] but keeps the aspect ratio, then random 224 x 224 spatial region are cropped along the time dimension. We trained the models with batch size of 1024 by 128 GPUs for Kinetics and batch of 128 by 16 GPUs for SSV2. More details can be found in (Chen et al., 2021). On the other hand, for the mini-dataset, we used the codes provided by (Chen et al., 2021) to train the models. During the evaluation, setting for performance evaluation.
the single-clip
G DETAILS ON PROPAGATION RULES
Our implementation is based on the captum code (cap). We use the 2+ rule (Gu et al., 2018) for convolutional layers and fully connected layers in CNN model, and the rule (Montavon al.,
17

---

Under review as a conference paper at ICLR 2023
Table 7: avg-ATR, max-ATR and model accuracy on Kinetics. The red number indicates different from
uniform sampling.
Model
Acc. (%)
avg-ATR
max-ATR
uniform
dense
uniform
dense
uniform
dense
RDD-R50-88
69.6
56.3
5.5+0.4
4,9+0.4 (-0.6)
7.0+0.2
7,3+0.5 (+0.3)
TAM2D-R50-8
70.5
56.3
4.240.4
4,34003 (+0.1)
5.24
6.2:1.88 (+1.0)
13D-R50-F16
72.5
60.0
16.++..5
6.240.5
(-0.)
8.4++077
8.7++0.8 (+0.3)
TAM2D-R50-11
73.1
4.940.4
4.8.0.5 (-0.1)
6.14...8
6.5+0.9 (+0.4)
aa) uniform (L3D-R50-IL)
(b) uniform (TAM2D R500-165
(c) dense (13D R50-116)
(d) dense (TAM2D-R50-IP))
Figure 15:
Heatmaps of mean avg-ATR Kinetics models based uniform and dense sampling. A heatmap
show how each frame is relevant to other frames, with lighter colors denoting higher relevance.
2019) (e -9) for max and average pooling layers. For batch normalization (BN), we apply the identity rule.
18

---

Under review as a conference paper at ICLR 2023
Approaching sth with your camera
Unfalding sth
Throwing sth
Move sth away from sth
(a) SSV2
Dribbling basketball
Checking tires
Doing laundry
Dancing ballet
(b) Kineties
Figure 16: Temporal relevance Heatmaps of actions in Kinetics and SSV2. The color bar under frame images represents the logit score of each frame for the predicted class. The brighter the color, the higher the score.
90
Top-5 Acc.
80
Top-5 Acc.
Accuraty (95)
Accuracy (%)
Top-1 Acc.
70
Top-1
11
40
Top-1 11---TTTA22--50-116
Top-1
20
Top-1:TAM2D R50
Top-5
Neg-S:TAM2D-R50-4
11
Tog-8-TAMM2--550-12
6
10 12
14
16
18
26
30
32
2
8
10
12
14
16
18
22
26 30 32
Window
Size
Window
(a) Kinetics
(b) SSV2
Figure 17: Model accuracy by window size using partial uniform sampling TAM2D. The results based frame-level ATR (blue and green dots) that only partial temporal information needed for the prediction of a frame.
19

---

Under review as a conference paper at ICLR 2023
6.5
number of classes
40
6.0
5.0
avo-ATR-
4.5
avg:ATR
10-
avg-ATR std
25
50
100 125 150 175
9.0
9.5
10.0
10.5
(a) avg-ATR
(b) max-ATR
Figure 18: (a) The distribution of avg-ATR and (b) the histogram of max-ATR over action classes on SSV2 produced by the model TAM2D-R50-f16. Both ATRs do not show significant differences between classes.
20


Reviews:
[{"review_id": "OnPXcnedeQT", "reviewer": "Reviewer_4NU1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an approach to measure the temporal relevance in video models and studies the relationship between the performance of video model and its temporal modeling capability. ", "summary_of_the_review": "An analysis without many concrete conclusions. It is hard to tell what we can learn from the paper and how the conclusions can benefit designing a better video model.", "strength_and_weaknesses": "### Pros:\n\n+  Understanding the temporal modeling for action recognition is an important task. \n\n### Cons:\n\n- The definitions of relevance scores are not clear.\n  - How is the relevance score $\\mathbf{\\mathit{R}}$ accumulated from $l_k$? Also, should it make more sense to be $\\mathbf{\\mathit{R}}_k$ since it is class specific?\n  - What is $z^+$ and $z^\\beta$ rule? Any intuitive explanation? \n  - The model to be analyzed employs a ReLU activation. What if other activations are used, eg. PReLU and GELU?\n  - It is easy to get confused between $a_i$ in Eq (1) and $a_{ij}$ in the Temporal Relevance Matrix.\n\n- ATR can be interpreted as a metric to measure the effective temporal receptive field. But the models that are studied in the paper have different *nominal* temporal receptive field (which can be directly obtained from the number of temporal convolutions being used along with their kernel size and stride). Therefore, does it make more sense to consider being normalized by the nominal temporal receptive field or consider effective and nominal temporal receptive field jointly?\n\n- It is an somewhat surprising observation that video models with more input frames learn shorter-range temporal information. But how come the model does this? I don't see any further explanation..\n\n- Many conclusions being drawn are either \"no strong correlation\" or \"no strong indication\". What can we learn from these conclusions? Or from an opposite perspective, should we conclude that the technique that is being used in the paper is not a good indicator to perform such temporal analysis??", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "GaPUOtxPXT4", "reviewer": "Reviewer_GyKL", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method to quantify the temporal relationships between frames to effectively analyze temporal relevance for video action models. And then, comprehensive experiments present the effects of various factors on temporal modeling. Finally, based on experimental analysis, this paper answers some important questions related to temporal modeling for action recognition.", "summary_of_the_review": "This work well extends existing methods for quantifying the temporal relationships between frames for action recognition. Comprehensive experiments are conducted to support the effectiveness of the proposed method. However, the innovation and generalization of the method need further consideration.", "strength_and_weaknesses": "Strengths\n1. This is the first work that investigates the effective temporal receptive field, i.e., action temporal relevance (ATR), in action models.\n2. This work well extends layer-wise relevance propagation (LRP) and contrastive layer-wise relevance propagation (CLRP) for quantifying the temporal relationships between frames for action recognition.\n3. This work implements elaborate experiments to present the effects of various factors on temporal modeling and answer some important questions related to temporal modeling for action recognition. Some findings seem surprising compared to our intuition.\n\nWeaknesses\n1. The method proposed in this paper is based on existing methods, LRP and CLRP. The innovativeness in terms of methods is relatively weak.\n2. Although this paper claims to focus on CNN-based methods, it is better to analysis the Transformer-based or pre-trained visual models. I wonder if it is able to apply the proposed techniques on the Transformer-based or Pre-trained models ?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "26hqAB2Ydp", "reviewer": "Reviewer_6sPt", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The work focuses on the temporal analysis of CNN-based action recognition models. The authors proposed a new computation metric to compare the effective relevance field along the temporal dimension. The paper delves into different aspects such as dataset effects and variations in network backbone architecture, and finally answers some of the questions pertaining to action models for video understanding.", "summary_of_the_review": "The extension of the metric already exists. Some of the experiment settings don’t look convincing to make the conclusion on those aspects. The paper still needs more work for certain sub-sections to have a conclusion on that analysis.", "strength_and_weaknesses": "**Strengths**\n\n- The authors have answered some unexplored questions about the temporal understanding of video action models. The analysis shows that backbone depth doesn’t have a significant impact, and that accuracy and avg ATR are inversely related most of the time. \n\n- Experiment with Input frames shows an increasing number of frames doesn’t directly translate to learning information from more frames.\n\n**Weakness**\n\n-  The extension of CLRP is not clear. In the original work [1], Section 4, the authors discussed the concept of two ways to model non-target class. CLRP2 equation looks exactly the same as mentioned in the proposed extension approach (Section 3.1). It appears that the original CLRP work is used as it is.\n\n- Temporal relevance Computation - Section 3.2 - The equation looks like only LRP is used for the ATR metric. Is it LRP or the CLRP?\n\n- Analysis:\n\n  - Selection of 3D CNN architecture - There could be a comparison table of performance with runtime and trainable parameters between a few architectures to justify the selection of I3D. There are several popular architecture which could have been used such as R3D and R21D are very popular architectures.\n\n  - Datasets: Difference in Batch size Training  - Why K400 and SSv2 are trained with different batch sizes 1028 vs 128? Won’t that impact the numbers shown in Table 1? \n\n  - Temporal Module - In Fig. 4, do different architectures have a different number of frames as input while training? If so, isn’t it unfair as in Figure 3, the number of frames matters, and the comparison is across different backbones here.\n  \n  - Fig. 5 - Missing values for R-152 backbones for I3D-f32 and TAM2D-f32.\n  \n  - Table 2 needs to be more detailed. It does not convey the whole story. What happens on the Kinetics dataset or a single dataset (like SSv2 in this case) different architecture?\n\n- Learning of Action Model - 3rd point - Can the authors clarify what do they mean by “retrained a few 16-frame models”? Don’t the models train already on the K400 and SSv2 datasets?\n\n- From all the analysis, could there be a recommendation to get the best performance out of the CNN-based temporal action model for both 3D and 2D?\n\n**References:**\n\n[1] Gu, Jindong et al. “Understanding Individual Decisions of CNNs via Contrastive Backpropagation.” ACCV (2018).\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject