Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper presents a paradigm for generating saliency maps for video models, specifically, I3D (3D CNN) and C-LSTM. It extends Fong & Vedaldi, 2017 to generate a temporal mask and introduces two types of "meaningful perturbations" for videos: freezing and reversing frames; they use Grad-CAM (with no modifications) for generating spatial masks. The problem is well-motivated, as saliency maps have been extensively studied for image classification models, but rarely for video classification. Quantitatively, they demonstrate that frame-reversal is meaningful for the Something-something dataset but less for KTH because those actions rely more on spatial information than temporal (i.e., running, clapping). Qualitatively, they show their spatial and temporal masks on both datasets and suggest the a few insights: 
* I3D's Grad-CAM visualizations show a center, default bias * I3D is less sensitive to the reverse perturbation * I3D temporal masks are typically shorter.

I currently rate this paper as a weak reject (though closer to borderline) paper for the following reasons (all of which can be improved in rebuttal):
1. Quality of technique
While the temporal masks are novel and qualitatively "make sense" (though this is subjective), the generation of the spatial masks is not novel, is unconnected to the temporal mask generation, and often doesn't make sense because of lack of temporal smoothness / cohesion, particularly for C-LSTM, where the visualizations when the mask is on appear quite "jumpy" (see Fig 2, Seq 1). It would be great to see more innovation on the temporal mask generation to address some of these issues (one natural approach that comes to mind would be learn spatial masks as done in Fong & Vedaldi, 2017, possibly with a temporal smoothness term between spatial masks and possibly combining temporal + spatial masks for freezing operation, i.e., only freeze spatial pixels) -- that said, I realize that this may be out of scope for a rebuttal.

2. Lack of support for qualitative claims
The paper makes 3 claims based on qualitative examples shown (see above bullets in summary); these claims could be easily substantiated qualitatively (by evaluating over the dataset or a subset of it).

3. Lack of discussion on limitations/benefits of technique + how to use/interpret technique
* There are no baseline comparisons for the proposed temporal mask generation. natural ones would be visualizing saliency methods (i.e., gradient, SmoothGrad, occlusion [Zeiler & Fergus, 2014], RISE [BMVC 2018], for instance, as examples of a few easy-to-implement, representative methods for backprop and perturbation methods) w.r.t. to temporal dimensions and then thresholding and applying those baseline temporal masks and demonstrating that the proposed temporal mask generation is.
* There's no discussion (or experiments) on the benefits & limitations of their approach; this is important as we've seen from papers like Mahendran & Vedaldi, ECCV 2016, Adebayo et al., NeurIPS 2018 and Kindermans et al., arXiv 2017 that some saliency methods fail to meet basic desirata (i.e., specificity to output class and model weights, etc.). Ideally, the authors would show that their temporal mask generation meets some desired criteria (as well as compare with baseline methods) to justify their approach.
* One limitation of Fong & Vedaldi 2017 is the difficulty of finding global optimum for the different hyper-parameter terms (Fong et al, ICCV 2019 addresses this, which might be of interest to the authors). There's no discussion about whether this problem persists for this work. Also -- it'd be interesting to see whether the reverse loss function (i.e., maximizing class score) yields similar results.
* More discussion can be added about how to interpret results / use the technique (i.e., what is this technique useful/not useful for? what do results mean?). For instance, the claim that I3D temporal mask is shorter suggests a complex phenomenon -- that the necessary temporal evidence is smaller. This is a bit surprising to me, as I3D performs better overall, so I would have expected it to encode more redundancy (this can be checked by exploring the classes in which I3D does perform better) -- however, this interpretation differs from that of the authors ("This is especially visible in the temporal mask of Sequence #3, where it is active specifically").

In addition to responding to the above with relevant text + preliminary experimental results, I also had the following questions/asks:
* For misclassified classes in Fig 2, are you optimizing w.r.t. the top predicted class or the ground truth class? Do they differ substantially when optimizing for different output classes?
* What were the lambda hyperparameters from Eq. 1 (and how were they chosen)? Are these relatively stable or are their instances of technique failure due to the difficulty in balancing these terms?
* Show Table 1 on only the classes that were focused on (i.e., the ones w comparable performance between the two models); I'm wondering if the impact of reversal on I3D is less than that on C-LSTM, as claimed by the authors (it's hard to tell when their baseline performance is different)

review_title: Official Blind Review #5

final_score: rating: 1: Reject
review_content: This work attempts to experimentally investigate the difference between Conv3Ds and ConvLSTMs with the aid of visualization. The visualization focuses on two aspects: (1) Temporal masking for identifying key frames and key segments and (2) GradCAM for spatial saliency. Both visualization techniques are illustrated on two public available video classification datasets.

This work has some major issues:
1. The quantitative results are not very relevant. It does not validate any of the following cases: (1) if Conv3Ds are more powerful, one should use the same number of parameters and compare classification results, or (2), in order to achieve the same level of accuracy, one of the two models is more parameter efficient. It is not clear which argument Table 1 is validating against if there is any as both parameters and accuracies vary. 
2. Qualitative results from Section 5.2 is not substantiated and underwhelming. For instance, it is hard to see why "Conv3Ds has a bias around the center while ConvLSTMs find relevant spatial features in multiple smaller areas". This is most likely due to visualization techniques other than the choice of models. 
3. The fundamental issue of this work is that it does not establish a hypothesis from the beginning and design experiments around it. Plenty of hand-wavy observations are made without further investigating the root of them, leaving readers unsatisfied, and quite often confused.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper shows a way to compare what is learned by two very different networks trained for a video classification task. The two architectures are state-of-the-art methods, one relying on 3d-CNNs (time= one dimension), the other on conv-LSTMs (time is treated sequentially, using hidden states to pass information). The idea of the authors is (i) to provide saliency maps for each of them, and (ii) to create interesting perturbations in order to measure the influence on the networks. The results indicate that these complex networks are usually focused on interesting features, and as we would imagine, LSTMs is more learning from temporal coherence than CNNs.

Overall, I think the paper is worth to be published at ICLR, even if I am not aware of the recent publications in this field. The contribution in terms of method is small, but I think that such careful studies can be very fruitful to the community.

Positive aspects:
- A significant effort has been put to creating meaningful perturbations for this particular task, i.e. temporal dependance and coherence.
- The effort to compare as best as possible two different approaches is fruitful, and very useful for the community as this is a real question to be raised.
- The experiments are made also with care, on 2 different datasets, and large efforts were made on explaining the different results.

Questions/remarks:
- As I was under Linux, I did not manage to view the videos, and I have not seen a link for downloading them. It would be best to add it as supplementary materials?
- What do you mean by 'The TV norm penalizes masks that are not coherent'? what is coherent?
- '...the mask is defined as a vector of values between [0,1]' : I understood that it was a real value between 0 and 1, but I think you meant more 'a binary vector'?
- Have you looked at the importance of the sub-sampling in the CNN framework? i.e., since the LSTM framework does not have that, maybe the differences in activation also depend on the length of the sequence. Maybe the large sequences, where CNN and LSTM would have the same number of frames, are less different?
- You are saying that the code will be made public, is it possible to make it public now?

Small remarks:
- '3D CNNs instead instead convolve'
- 'As outlined in section 2' when we are in the introduction
- '(Mahdisoltani et al.(2018) contains...'. parenthesis.
- Figure 1: the first figure is in too poor quality.

review_title: Official Blind Review #1

Meta-review: The paper addresses interpretability in the video data domain. The authors study and compare the saliency maps for 3D CNNs and convolutional LSTMs networks, analysing what they learn, and how do they differ from one another when capturing temporal information. To search for the most informative part in a video sequence, the authors propose to adapt the meaningful perturbations approach by Fong & Vedaldi (2017) to the video domain using temporal mask perturbations. 
While all reviewers and AC acknowledge the importance and potential usefulness of studying and comparing different generative models in continual learning, they raised several important concerns that place this paper below the acceptance bar: 
(1) in an empirical study paper, an in-depth analysis and insightful evaluations are required to better understand the benefits and shortcomings of the available and proposed models (R5 and R2). Specifically: 
(i) providing a baseline comparison to assess the benefits of the proposed approach -- please see R5’s suggestions on the baseline methods; 
(ii) analyzing how the proposed approach can elucidate meaningful differences between 3D CNNs and LSTMs (R5, R2). The authors discussed in their rebuttal some of these questions, but a more detailed analysis is required to fully understand the benefits of this study. 
(2) R5 and R2 raised an important concern that the temporal mask generation developed in this work is grounded on the generation of the spatial masks, which is counterintuitive when analysing the temporal dynamics of the NNs - see R5’s suggestions on how to improve. 
Also R5 has raised concerns regarding the qualitative analysis of the Grad-CAM visualizations. Happy to report that the authors have addressed these concerns in the rebuttal, namely reporting the results in Table 2 and providing an updated discussion. R1 has raised a concern about the importance of the sub-sampling in the CNN framework, which was partially addressed in the rebuttal. 
To conclude, the AC suggest that in its current state the manuscript is not ready for a publication and needs a major revision before submitting for another round of reviews. We hope the reviews are useful for improving and revising the paper.

Decision: Reject