Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes six quantitative metrics for evaluating post-hoc visual interpretation methods on time series. It demonstrates the efficacy of nine visual interpretation methods using these six metrics over some common neural network architectures. 
main_review: Strengths:
- The paper is well written overall
- Experiments are well designed where the six metrics can be compared for different tasks
- The paper demonstrates why it is necessary to evaluate all six metrics for every visual interpretation
- The paper compares 9 different interpretation methods
- The paper shows how these metrics can vary across datasets and for different models

Weaknesses:
- The metrics are derived from existing literature
- Figure 5 caption needs to be fixed

summary_of_the_review: The paper combines multiple interpretation approaches and evaluation metrics for those interpretation schemes. Thus, it demonstrates the need for multiple interpretation evaluation schemes to truly understand which ones are optimum. It also shows how these can vary across different datasets and models. So, although the paper is not novel, the experiments performed and the results obtained is very useful for researches looking for a way to perform post-hoc visualization. As a result, I consider it as a borderline paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This study deals with the issue of interpretability in the analysis of time series data using deep learning models. Six metrics for evaluating interpretation methods are introduced and nine existing interpretation methods are evaluated. One of these metrics is proposed by the author. One of the metrics is based on an experimental evaluation, which shows that none of the methods is superior in all the metrics.
main_review: The strength of this paper is that it provides a comprehensive evaluation of nine well-known interpretation methods for NNs with multiple architectures using multiple data sets. It is considered to be a very good survey paper to understand the characteristics of each method. The technical contribution of this study is mainly in the evaluation metric called localization, which is described in 3.6. This metric itself is based on a reasonable idea as an interpretive method. On the other hand, most of the considerations are derived from experimental results, which is a bit insufficient in terms of proposing new ideas based on deep technical insights.
summary_of_the_review: The paper is excellent as a survey paper focusing on performance evaluation, but somewhat inadequate as a research paper pursuing technical originality.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies deep neural network (DNN) explainability methods in the context of time-series data. Several metrics exist for evaluating the validity of DNN explainability methods on computer vision tasks. However, it is not clear whether these metrics are reliable when applied to DNN explainability methods on time-series data. This paper conducts experiments to compare 6 of those metrics on time-series classification and segmentation tasks. 
main_review: This paper studies deep neural network (DNN) explainability methods in the context of time-series data. Several metrics exist for evaluating the validity of DNN explainability methods on computer vision tasks. However, it is not clear whether these metrics are reliable for DNNs on time-series data. This paper conducts experiments to compare 6 different metrics on time-series classification and segmentation tasks. 


Strengths:
1. The paper is generally well organized. (ALTHOUGH there are minor typos scattered all over so I urge the authors to proof read their work). 
2. The work is really well motivated. It is important to validate explainability methods before deploying them, and to that end it is important to design good validation criteria. 
3. I found the experiments and the accompanying figures to be really well designed and insightful, thank you! In particular I liked figures 1, 4, and 5. The appendix also has insightful experiments and figures!

Weaknesses:
1. Can you explain how Fig. 2 is obtained? I think the observation in this figure is interesting in its own right. It is worrying that 6 different methods can have very low correlations. 
2. I think the discussion in section 4.3 is hard to read and not very well organized. But I also understand that there are a lot of variables (data, task, model, explainability method, and metrics) which makes it difficult to fully cover all of them in text. (I am not penalizing the paper for this). 

General questions (no need to run more experiments just wondering): 
1. Some metrics are exhibit high variability on particular datasets (e.g. Ford). I wonder what you think would happen if you tried transfer learning. For instance pretrain on a dataset where all metrics show relatively low variability, then finetune on FordA dataset?  
summary_of_the_review: The paper is well motivated, the experiments are well designed, and the insights are helpful to the community.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to use six evaluation metrics to evaluate saliency methods for time series classification. The experiments perform this evaluation for segmentation tasks, and find that few metrics achieve high scores with low variance.
main_review: 1) **Unclear objective / no novelty or insight**: The objective of this paper is very unclear, and there does not seem to be any novelty in the methods it introduces. Majority of the saliency evaluation methods it uses are known in literature, and these evaluation methods do not seem to change for time series classification. The paper simply proposes to use these metrics to evaluate saliency methods. Now, it is understandable that different saliency metrics are not aligned, but the paper proposes no solution for this. If each saliency method has different strengths and weakness, how can one use this "framework" to guide selection of saliency methods? This lack of a solid objective within this paper makes it unsuitable for publication at this time.
2) **Incorrect to use robustness of saliency map as a metric**: As a general remark on saliency metrics, please note that the only job of an evaluation metric should be to judge how well a saliency method reflects model behaviour. Metrics such as robustness or intra-class stability are thus contingent on such model behaviour. For example, if the underlying model is non-robust, there is no reason to expect robustness from saliency methods, and if a saliency method is indeed robust in such scenarios it does not reflect model behaviour and thus can be misleading. Thus usage of these metrics for evaluating saliency maps must be avoided.
3) **Incorrect to use localization of saliency map as a metric**: A similar critique holds for the localisation metric (including the pointing game and other metrics in literature). The implicit assumption here is that model holds the same parts of the input to be important as judged by a human evaluator. However models can be erroneous and can sometimes have spurious correlations and might assign a different portion of the input to be important. In such a case, we would want the saliency map to reflect modelling errors, which would necessarily perform poorly in the localisation metric. Hence localization is a not a good saliency evaluation metric.
summary_of_the_review: The paper does not present either a novel method or any novel insight. For this reason, I would recommend rejection.

The post-rebuttal discussions did not change my view of the paper, but I am improving my score in light of author updates.
review_title: null

Meta-review: This paper evaluates interpretation methods of neural networks on time series data. The reviewers find some values in this work, but were also consistently concerned with the main theme and novelty of this work. The authors have actively responded to reviewer comments, but the reviewers were not convinced with the major contributions and novelty. Thus the work is not ready for ICLR.
Decision: Reject