Interpreting video features: a comparison of 3D convolutional networks and convolutional LSTM networks

1 INTRODUCTION . Two standard approaches to deep learning for sequential image data are 3D Convolutional Neural Networks ( 3D CNNs ) , e.g . the I3D model ( Carreira & Zisserman ( 2017 ) ) , and recurrent neural networks ( RNNs ) . Among the RNNs , the convolutional long short-term memory network ( C-LSTM ) ( Shi et al . ( 2015 ) ) is especially suited for sequences of images , since it learns both spatial and temporal dependencies simultaneously . Although both methods can capture aspects of the semantics pertaining to the temporal dependencies in a video clip , there is a fundamental difference in how 3D CNNs treat time compared to C-LSTMs . In 3D CNNs the time axis is treated just like a third spatial axis , whereas C-LSTMs only allow for information flow in the direction of increasing time , complying with the second law of thermodynamics . More concretely , C-LSTMs maintain a hidden state representing the current video frame when forward-traversing the input video sequence , and are able to model non-linear transitions in time . 3D CNNs instead convolve ( i.e . take a weighted average ) over both the temporal and spatial dimensions of the sequence . The question investigated in this paper is whether this difference has consequences for how the two models compute spatiotemporal features . We present a qualitative study of how 3D CNNs and CLSTMs respectively compute video features : what do they learn , and how do they differ from one another ? As outlined in Section 2 , there is a large body of work on evaluating video architectures on spatial and temporal correlations , but significantly fewer investigations of what parts of the data the networks have used and what semantics relating to the temporal dependencies they have extracted from it . Deep neural networks are known to be large computational models , whose inner workings are difficult to overview for a human . For video models , the number of parameters is typically significantly higher which makes their interpretability all the more pressing . We will evaluate these two types of models ( 3D CNN and C-LSTM ) on tasks where temporal order is crucial . The 20BN-Something-something-V2 dataset ( Mahdisoltani et al . ( 2018 ) , hereon Something-something ) will be central to our investigations ; it contains time-critical classes , agnostic to object appearance , such as move something from left to right or move something from right to left . We additionally evaluate the models on the smaller KTH actions dataset ( Schuldt et al . ( 2004 ) ) . Our contributions are listed as follows . • We present the first comparison of 3D CNNs and C-LSTMs in terms of temporal modeling abilities and highlight the essential difference between their assumptions concerning temporal dependencies in the data . • We extend the concept of meaningful perturbation introduced by Fong & Vedaldi ( 2017 ) to the temporal dimension to search for the most critical part of a sequence for classification . 2 RELATED WORK . The field of interpretability in the context of deep neural networks is still young but has made considerable progress for single image networks , owing to works such as Zeiler & Fergus ( 2013 ) , Simonyan et al . ( 2014 ) and Montavon et al . ( 2018 ) . One can distinguish between data centric and network centric methods for interpretability . Activity maximization , first coined by Erhan et al . ( 2009 ) , is network centric in the sense that specific units of the network are being studied . By casting the maximization of the activation of a certain unit as an optimization problem in terms of the input , one can compute the optimal input for that particular unit by gradient ascent . In data centric interpretability methods , the focus is instead on the input to the network , to reveal which patterns of the data that the network has discerned . Grad-CAM ( Selvaraju et al . ( 2017 ) ) and the meaningful perturbations explored in Fong & Vedaldi ( 2017 ) , which form the basis for our experiments , belong to the data centric category . These two methods are further explained in Section 3 . Layer-wise relevance propagation ( LRP ) ( Montavon et al . ( 2018 ) ) as well as Excitation backprop ( Zhang et al . ( 2016 ) ) are two other examples of data centric backpropagation techniques designed for interpretability , where the excitation backprop method follows from a simpler parameter setting of LRP . Building on excitation backprop by Zhang et al . ( 2016 ) , Adel Bargal et al . ( 2018 ) produce saliency maps for video RNNs without the use of gradients . Instead , products of forward weights and activations are normalized in order to be used as conditional probabilities , which are back-propagated . We have chosen to use Grad-CAM for our experiments , since it is one of the saliency methods in Adebayo et al . ( 2018 ) that passes the article ’ s sanity checks , as well as for its simplicity of implementation and wide usage . Limited works have been published with their focus on interpretability for video models ( Feichtenhofer et al . ( 2018 ) , Sigurdsson et al . ( 2017 ) , Huang et al . ( 2018 ) , Ghodrati et al . ( 2018 ) ) . Other works have treated it , but with less extensive experimentation ( Chattopadhyay et al . ( 2017 ) ) , while for example mainly presenting a new spatiotemporal architecture ( Dwibedi et al . ( 2018 ) , Zhou et al . ( 2018 ) ) . We build on the work by Ghodrati et al . ( 2018 ) , where the aim is to measure a network ’ s ability to model video time directly , instead of via the proxy task of action classification , which is most commonly seen . Three defining properties of video time are defined in the paper : temporal symmetry , temporal continuity and temporal causality , and are each presented accompanied by a measurable task . The third property is measured using the classification accuracy on the Somethingsomething dataset . An important contribution of ours with respect to this work is that we compare 3D CNNs and C-LSTMs , whereas Ghodrati et al . ( 2018 ) compare 3D CNNs to standard LSTMs . Their comparison can be argued as slightly unfair , as standard LSTM layers only take 1D input , and thus needs to collapse each image frame in the video to a vector , which removes some spatial dependencies in the pixel grid . Similar to our work , Dwibedi et al . ( 2018 ) investigate the temporal modeling capabilities of convolutional RNNs ( Convolutional Gated Recurrent Units ) trained on Something-something . The authors find that recurrent models perform well for the task , and a qualitative analysis of the learned hidden states of their trained model is presented . For each class of the dataset , they obtain the hidden states of the network corresponding to the frames of one clip and display its nearest neighbors from other clips ’ per-frame hidden state representations . These hidden states had encoded information about the relevant frame ordering for the classes . Sigurdsson et al . ( 2017 ) examined video architectures and datasets on a number of qualitative attributes . Huang et al . ( 2018 ) investigate how much the actual motion in a clip contributes the classification performance of a video architecture . To measure this , they perform classification experiments varying the number of sub-sampled frames used for a clip to examine how much the accuracy changes as a result . In a search-based precursor to our temporal mask experiments , Satkin & Hebert ( 2010 ) crop sequences temporally to obtain the most discriminative sub-sequence for a certain class . The cropping corresponding to the highest classification confidence is selected as being the most discriminative sub-sequence . Feichtenhofer et al . ( 2018 ) present the first network centric interpretability work for video models . The authors investigate spatiotemporal features using activity maximization . Zhou et al . ( 2018 ) introduce the Temporal Relational Network ( TRN ) which learns temporal dependencies between frames through sampling the semantically relevant frames for a particular action class . The TRN module is put on top of a convolutional layer and consists of a fully connected network between the sampled frame features and the output . Similar to Dwibedi et al . ( 2018 ) , they perform temporal alignment of clips from the same class , using the frames considered most representative for the clip by the network . They verify the conclusion previously made by Xie et al . ( 2017 ) , that temporal order is crucial on Something-something and show that their architecture is sensitive to that . They also investigate which classes of Something-something show the strongest sensitivity to temporal order . 3 APPROACH . 3.1 TEMPORAL MASKS . The proposed temporal mask method aims to extend the interpretability of deep networks into the temporal dimension , utilizing meaningful perturbation of the input , as shown effective in the spatial dimension by Fong & Vedaldi ( 2017 ) . When adopting this approach , it is necessary to define what constitutes a meaningful perturbation . In the mentioned paper , a mask that blurs the input as little as possible is learned for a single image , while still maximizing the decrease in class score . Our proposed method applies this concept of a learned mask to the temporal dimension . The perturbation , in this setting , is a noise mask approximating either a ’ freeze ’ operation , which removes motion data through time , or a ’ reverse ’ operation that inverses the sequential order of the frames . This way , we aim to identify which frames are potentially most critical for the network ’ s classification decision . The perturbing temporal mask is defined as a vector of values on the interval [ 0,1 ] with the same length as the input sequence . For the ’ freeze ’ type mask , a value of 1 for a frame at index t duplicates the value from the previous frame at t − 1 onto the input sequence at t. The pseudocode for this procedure is given below . for i in maskIndices do perturbedInputi ← ( 1−maski ) ∗ originalInputi +maski ∗ perturbedInputi−1 end for For the ’ reverse ’ mask type , all indices of the mask m that are activated are first identified ( threshold 0.1 ) . These indices are then looped through to find all contiguous sections , which are treated as sub-masks , mi . For each sub-mask , the frames at the active indices in the sub-mask are reversed . For example ( binary for clarity ) , an input indexed as t1:16 perturbed with a mask with the value [ 0 , 0 , 0 , 1 , 1 , 1 , 1 , 1 , 0 , 0 , 0 , 0 , 0 , 1 , 1 , 0 ] would result in the sequence with frame indices [ 1 , 2 , 3 , 8 , 7 , 6 , 5 , 4 , 9 , 10 , 11 , 12 , 13 , 15 , 14 , 16 ] . In order to learn the mask , we define a loss function ( Eq . 1 ) to be minimized using gradient descent , similar to the approach in Fong & Vedaldi ( 2017 ) . L = λ1‖m‖11 + λ2‖m‖ β β + Fc , ( 1 ) where m is the mask expressed as a vector m ∈ [ 0 , 1 ] t , ‖·‖11 is the L1 norm , ‖·‖ β β is the Total Variation ( TV ) norm , λ1,2 are weighting factors , and Fc is the class score given by the model for the perturbed input . The L1 norm punishes long masks , in order to identify only the most important frames in the sequence . The TV norm penalizes masks that are not contiguous . This approach allows our method to automatically learn masks that identify one or several contiguous sequences in the input . The mask is initialized centered at the middle of the sequence . To keep the perturbed input class score differentiable w.r.t . the mask , the optimizer operates on a mask vector that has values in R. A sigmoid function is applied to the mask before using it for the perturbing operation in order to keep its values in the [ 0,1 ] range . The ADAM optimizer is then used to learn the mask through 300 iterations of gradient descent . After the mask has converged , it is thresholded for visualisation purposes .

Summary: This paper presents a paradigm for generating saliency maps for video models, specifically, I3D (3D CNN) and C-LSTM. It extends Fong & Vedaldi, 2017 to generate a temporal mask and introduces two types of "meaningful perturbations" for videos: freezing and reversing frames; they use Grad-CAM (with no modifications) for generating spatial masks. The problem is well-motivated, as saliency maps have been extensively studied for image classification models, but rarely for video classification. Quantitatively, they demonstrate that frame-reversal is meaningful for the Something-something dataset but less for KTH because those actions rely more on spatial information than temporal (i.e., running, clapping). Qualitatively, they show their spatial and temporal masks on both datasets and suggest the a few insights: