Paper: Explaining Time Series via Contrastive and Locally Sparse Perturbations (ICLR-2024)

Reviewer comment (Experiments):
Although the authors discuss the selection of positive and negative samples in Appendix B, the selection of positive and negative pairs in time series is quite controversial because the proximity of data samples does not guarantee similarities [1,2,…]. The method in Appendix B appears too naive and may pose a risk of incorrect sampling for time series pair selection.

Author response:
Thanks for your insights about the nuanced aspect of contrastive sample selection! The concept and strictness of data samples' proximity depend on the specific method used. Specifically, as you mentioned, proximal data means subseries within the stationary window in [1], and augmented subseries in [2]. Both of these are within the same instance and are relatively weakly constrained in terms of similarity. In contrast, our sampling employs an instance-wise comparison using a more stringent concept for similarity (Manhattan distance), and we have added performance comparisons of different similarity metrics in the latest version (see Table 8 in Appendix E.1). Our stricter criteria for similarity likely reduce the risk of noisy sampling. The core rationale behind this is that our aim in choosing contrastive samples is to craft perturbations rather than to learn a generalizable representation, which means a straightforward method is already effective. 

[1] Tonekaboni, et al.  Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding. In *ICLR*, 2021.

[2] Yue, et al.  TS2Vec: Towards Universal Representation of Time Series. In *AAAI*, 2022.

Reviewer comment (Theory):
Can you provide a more specific explanation of why the counterfactual of non-salient features is superior to ignoring that part (e.g., setting it to zero), as shown in Figure 1? Even though a zero value of $x$ does not affect the training to minimize the loss of prediction with weight $w$ as $wx$, using counterfactuals can have adverse effects.

Author response:
Thanks for your constructive comment. If it is an important feature, our goal is to have the counterfactual produce the opposite conclusion so that the loss $\mathcal{L}(f(x), f(x, m))$ increases, which leads to $m$ optimizing to $1$ and then minimizing the loss. For example, a regression task $y = f(x)$ where we need to perturb whether an observation $x=1$ is important or not. During training, our perturbation $x^r$ learns to a counterfactual $x^r=-3$ and a perturbation of other method is $x^{r \prime}=0$. If it is not salient, the change in $x$ is independent of $y$, so that $f(1)\approx f(-3)\approx f(0)$ and $m=0$. But if it is salient, $\mathcal{L}(f(1), f(-3)) > \mathcal{L}(f(1), f(0)) $ means that our $m$ is more likely to learn $1$, i.e, the loss changes from $\mathcal{L}(f(1), f(-3))$ to $\mathcal{L}(f(1), f(1))$. 
   
Moreover, we posit that perturbations in the function $f$ are not universally amenable to being substituted with zero; such substitutions may result in out of distribution outcomes within the explained model. Figure 1 illustrates the absence of zeros in the data domain during the intermediate period of time, while the $0$ and mean perturbations shift in shape. In instances where a trained model is confronted with previously unseen data, deleterious effects may occur [3], leading to erroneous judgments. Note that the counterfactual perturbation described in Eq. (2) is learned while concurrently constraining its extent using an $l_1$ norm term. This constraint serves to mitigate the generation of excessively anomalous counterfactual instances.
   

[3] Shen, et al. Towards out-of-distribution generalization: A survey. *arXiv:2108.13624*, 2021.

Reviewer comment (Novelty):
What is the difference in using the $\ell_0$ norm in Section 4.2 of your methods compared to previous methods that use the $\ell_0$ norm?

Author response:
Thanks for your constructive comment. ContraLSP distinguishes itself significantly from previous methods in that we do not use sparsity constraints such as the $l_1$ norm (Dynamask and Extrmask), $l_0$ norm [5, 6], $l_2$ norm [7] to select salient sub-features. Our mask is just skewed binary but not purely containing 0 and 1 due to the smooth constraint. We apply an $l_0$-like constraint, i.e., by injecting random noise into $\mu^\prime $ and representing the $l_0$ norm with a Gaussian error function. The expected regularization term is simply the sum of the probabilities (in Appendix A) that sparse gates are active or $\sum_{t,d}P(\mu^\prime[t,d]+\epsilon[t, d] > 0)$. This sum is equal to $\sum_{t,d}\Psi \left(\frac{\mu^\prime[t,d]}{\delta }\right)$, where $\Psi $ is the standard Gaussian CDF. The random noise $\epsilon$ of the model plays two important roles. Firstly, facilitates the training of weights in a binary model, specifically the gating network in Eq. (3). And secondly, it enables the re-evaluation of features that undergo sparsification during the early stages of training [8].

[5] Bastings, et al. Interpretable Neural Predictions with Differentiable Binary Variables. In *ACL*, 2019.

[6] Schlichtkrull, et al. Interpreting graph neural networks for nlp with differentiable edge masking. In *ICLR*, 2021.

[7] Yu, et al. Graph information bottleneck for subgraph recognition. In *ICLR*, 2022.

[8] Yang, et al. Locally sparse neural networks for tabular biomedical data. In *ICML*, 2022.

Reviewer comment (Novelty):
The advance feels somewhat incremental and the experiments performed are near-identical to the Dynamask paper (cited in text).

Author response:
We believe our approach significantly differs from the Dynamask method proposed in Crabbe et al. (2021), both in an important conceptual way, as well as in the details of the formulation, beyond the experimental datasets and metrics.
   
   The key conceptual difference is that Dynamask adapts image perturbations [1] to multivariate time series, where the perturbation operator $\Phi (x, m)$ accounts for this temporal information. Specifically, a perturbation is constructed for each instance, utilizing the feature values at adjacent time steps. This allows Dynamask to build meaningful perturbation. In contrast, our ContraLSP involves optimizing two components: the mask and the perturbation. It employs negative samples for constructing counterfactual perturbations by contrastive learning. Our masks integrate sample-specific sparse gates with the smooth constraint, producing binary-skewed masks on time series data. 
   
   In terms of the formulation details, we can simply express the mask optimization of Dynamask as: $\mathcal{R}(x_i, m_i) = \alpha_1 \sum ||\text{vecsort}(m_i)-r_a|| + \alpha_2\sum ||m_{t+1,i}-m_{t,i}||$, where the second term is the smoothing of masks. Dynamask uses area constraints in the first term, but it is not clear how to choose the hyperparameter $a$ on complex data. Differently, the mask optimization of ContraLSP is expressed as an $l_0$-like regularization as: $\mathcal{R}(x_i, m_i) = \alpha \sum_{t=1}^T\sum_{d=1}^D\left(\frac{1}{2}+\frac{1}{2} \text{erf}\left(\frac{\mu^\prime _i[t,d]}{\sqrt{2} \delta }\right)\right)$, where the smooth vectors $\mu^\prime_i$ restrict the penalty for jump saliency over time. By injecting a random noise into $\mu^\prime $ and representing the $l_0$ norm with a Gaussian error function (see Appendix A), it can facilitate the training of weights in a binary model, specifically the sparse gating network. Furthermore, we optimize learned counterfactuals in Eq. (2) instead of using the meaningful perturbation (using temporal Gaussian blur does not support optimization). It provides more uninformative information and thus improves explanation accuracy.
   
   The experimental datasets and metrics are consistent with Dynamask for fair comparisons, and most time series explanation methods use similar experiments. In addition, when explaining the regression model, we added different groups of rare experiments to validate the effect of local masks on the inter-sample correlation, which further distinguishes the other methods.
   

[1] Fong, et al. Interpretable explanations of black boxes by meaningful perturbation. In *CVPR*, 2017.

Reviewer comment (Writing):
The manuscript is hard to follow the text as the writing and motivation is not clear in a number of points. Terms are not always introduced in order and it is hard to appreciate the innovations is. One of my reservations about this manuscript is even if the algorithm is novel it will be hard for others to appreciate.

Author response:
Thanks for pointing this question out, we acknowledge the concerns raised regarding clarity and motivation. We are dedicated to revising the manuscript to enhance the organization, ensuring that terms are introduced in a logical order for better comprehension. The latest version has corrected inappropriate sentences, and a comprehensive review will be conducted for the final version.

Reviewer comment (Experiments):
The authors mention treatment of inter-sample correlations as an important component of the technique, but I do not see clear evidence of this.

Author response:
Thank you for the thoughtful comment. We want to be clear that ContraLSP is not handling inter-sample correlation, and we don't mention that. The inter-sample correlation of features often leads to notable generalization errors. To cope with it, we employ local sparse gates (as described in Eq. (3)) allowing samples to be independent of each other.  In addition, our investigation performs counterfactual perturbations using contrastive learning through inter-sample variation, which goes beyond the instance-level saliency methods by focusing on understanding both the overall and specific model's behavior across groups. Therefore, we designed different groups for white-box regressors to study it.

Reviewer comment (Writing):
Can you explain what in-domain and negative samples refer to “Other perturbations could be either not uninformative or not in-domain, while ours is counterfactual that is toward the distribution of negative samples“.

Author response:
Thanks for the nice question. As shown in Figure 1, the entire data domain encompasses two distinct sample categories delineated by the red and blue distributions. Perturbation of the original features (the red line) is necessary to find the salient sub-features. Some perturbation methods, such as mean perturbation (the blue line) and 0-value perturbation (the orange line), are not in the data domain for an intermediate period of time, thus we call it not "in-domain". In contrast, our perturbations learn counterfactual features (the black line) that categorize its differences from the original sample. We call the blue distribution the distribution of negative samples that are not consistent with the classification results of the original features. Consequently, our perturbations gravitate towards the distribution of negative samples through contrastive learning.

Reviewer comment (Writing):
“To cope with it, locally stochastic gates Yang et al. (2022) consider an instance-wise selector that employs heterogeneous samples. Lee et al. (2022) takes a self-supervised way with unlabeled samples to enhance stochastic gates that encourage the model explainability meanwhile.” The terms introduced here (stochastic gates, heterogeneous samples) are not defined. The writing is unclear as well.

Author response:
Thank you for your careful comment. The "local stochastic gates" is a gating structure used by Yang et al. [3] and originally proposed by Yamada et al. [4]. It injects a random noise into the mask and is used for sparsified feature selection. The mathematical form (see Eq. (3) in section 4.2) can be described as $m_i = \mu_i + \epsilon_i$, where $\epsilon \sim \mathcal{N}(0, \delta ^2)$. Heterogeneous samples occur when the samples have differences, where different black box models act on different samples thus eliciting differing salient features. We therefore designed different groups of experiments within the white-box regression simulation (see section 5.1) to validate the effectiveness of our method.

   We recognize the lack of description in this sentence and modify it to "To cope with it, local stochastic gates~\citep{yang22i} consider an instance-wise selector to heterogeneous samples, accommodating cases where salient features vary among samples. \citet{lee2021self} takes a self-supervised way to enhance stochastic gates that encourage the model's sparse explainability meanwhile." in the latest version. Thanks again for your careful reading.

[3] Yang, et al. Locally sparse neural networks for tabular biomedical data. In *ICML*, 2022.

[4] Yamada, et al. Feature selection using stochastic gates. In *ICML*, 2020.

Reviewer comment (Evaluation):
“our method significantly outperforms all other benchmarks.” I do not see any tests of significance.

Author response:
We evaluate the interpretability performance of various algorithms on four rare-experiments, presenting results in Table 1 in section 5.1. It shows that our method demonstrates significant outperformance across all metrics, except AUP, compared to other benchmarks. Notably, AUP is not effective as a performance discriminator in rare-experiment scenarios due to all algorithms being close to 1. Specifically, ContraLSP has an average 23% improvement in AUC, an average 49% improvement in Information, and an average 88% reduction in Entropy over the leading baseline across rare-experiments. It suggests that ContraLSP identifies really salient features in all experiments.

Reviewer comment (Presentation):
Figure 5 I found unclear.

Author response:
We appreciate the reviewer's feedback regarding the clarity of our illustrations, particularly Figure 5. We present a comparison between the perturbations generated by ContraLSP and Extrmask on the Rare-Observation (Diffgroups) experiment, as shown in Figure 5 (the updated version). The background color represents the mask value, with darker colors indicating higher values. We randomly select a sample in each of the two groups and sum all observations. ContraLSP provides counterfactual information, yet Extrmask's perturbation is close to 0. This suggests ContraLSP provides counterfactual perturbations that is more beneficial to non-salient areas while emphasizing the mask more distinctly in salient areas compared to Extrmask. This is consistent with the findings in Figure 1. Moreover, the performance of Extrmask is poor when samples exhibit heterogeneity among distinct groups, whereas ContraLSP remains relatively unaffected.

Reviewer comment (Reproducibility):
What is the $\alpha$ and $\beta$ values you use for each dataset? and how do you determine their values?

Author response:
Thanks for the nice question. In fact, we have given the specific parameters chosen for each dataset, see Table 6 in Appendix D.4. For your convenience, we list them as follows:

   | Parameter         | Rate-Time      | Rate-Observation        |     Switch-Feature     |     State      |     MIMIC-III      |
   | :-------------: | :-----------------: | :-----------------: | :---------------: | :---------------: | :---------------: |
   | $\alpha$ | 0.1       | 0.1              |   1.0   |  2.0  |   0.005   |
   | $\beta$    | 0.1       | 0.1              |   2.0   |  1.0  |   0.01   |

   Specifically, the four Rate experiments used a default value of 0.1 determined by experience and other mask-based methods. On this basis, we perform ablation experiments (See Tables 9 and 10 in Appendix E.2) around the default values and find the best parameters applicable to the Switch-Feature and State dataset. Note that since the proportion of salient features in the data is unknown, finding a subset of features is unstable for most of the interpretation methods by picking the regularization hyperparameter. We mentioned that it's in the limit: "However, an inherent limitation of our method is the selection of sparse parameters, especially when dealing with different datasets..."
   
   For the MIMIC-III dataset, we add the same ablation experiments in the latest version. We employ Accuracy and Cross-Entropy as metrics and show the average substitution in the following table. This Table shows that $\beta$ is best set to $0.01$ to learn counterfactual perturbations. Notably, lower values of $\alpha$ yield superior results, but excessively regularizing $m$ toward $0$ may prove disadvantageous [3]. Therefore, we select $\alpha=0.005$ and $\beta=0.01$ as deterministic parameters on the MIMIC-III dataset.
   

Table 11: Effects of $\alpha$ and $\beta$  on MIMIC-III mortality. We mask 20\% data and replace the masked data with the overall average over time for each feature. **bold** is the best and values are reported as Accuracy - Cross-Entropy.

|                |              $\alpha$= 0.001              |              $\alpha$= 0.005              |               $\alpha$= 0.01               |               $\alpha$= 0.1                |               $\alpha$= 1.0               |
| :------------: | :---------------------------------------: | :---------------------------------------: | :----------------------------------------: | :----------------------------------------: | :---------------------------------------: |
| $\beta$= 0.001 | ${0.982}{\pm0.003}$ - ${0.124}{\pm0.007}$ | ${0.983}{\pm0.003}$ - ${0.122}{\pm0.007}$ | ${0.984}{\pm0.002}$ - ${0.120}{\pm0.006}$  | ${0.993}{\pm0.001}$ - ${0.094}{\pm0.004}$  | ${0.997}{\pm0.001}$ - ${0.087}{\pm0.004}$ |
| $\beta$= 0.005 | ${0.981}{\pm0.002}$ - ${0.123}{\pm0.007}$ | ${0.984}{\pm0.002}$ - ${0.123}{\pm0.006}$ | ${0.984}{\pm0.003}$ - ${0.121}{\pm0.007}$  | ${0.993}{\pm0.002}$ - ${0.095}{\pm0.006}$  | ${0.996}{\pm0.001}$ - ${0.087}{\pm0.005}$ |
| $\beta$= 0.01  | ${0.980}{\pm0.003}$ - ${0.124}{\pm0.007}$ |  **0.980$\pm$0.002 -  0.127$\pm$0.007**   | ${0.984}{\pm0.002}$ - ${0.121}{\pm0.007}$  | ${0.994}{\pm0.002}$ - ${0.094}{\pm0.004}$  | ${0.996}{\pm0.001}$ - ${0.087}{\pm0.004}$ |
|  $\beta$= 0.1  | ${0.980}{\pm0.003}$ - ${0.127}{\pm0.007}$ | ${0.980}{\pm0.003}$ - ${0.127}{\pm0.007}$ | ${0.983}{\pm 0.003}$ - ${0.123}{\pm0.007}$ | ${0.992}{\pm0.002}$ - ${0.098}{\pm 0.006}$ | ${0.997}{\pm0.001}$ - ${0.087}{\pm0.005}$ |
|  $\beta$= 1.0  | ${0.981}{\pm0.002}$ - ${0.127}{\pm0.006}$ | ${0.981}{\pm0.003}$ - ${0.128}{\pm0.008}$ | ${0.983}{\pm0.002}$ - ${0.123}{\pm0.007}$  | ${0.989}{\pm0.002}$ - ${0.106}{\pm0.007}$  | ${0.996}{\pm0.001}$ - ${0.088}{\pm0.005}$ |

Reviewer comment (Experiments):
In section 4.1, I wonder why you choose Manhattan distance rather than more conventional Euclidean distance? Optionally, other metrics like cosine distance might work better for contrastive learning?

Author response:
We thank the reviewer for the insightful comment. Most explanation methods, such as Dynamask and Extrmask, construct the loss (mask or perturbation) using the $l_1$ norm. In Eq.(2), the first term encourages the original sample $x_i$ to differ from the perturbed sample $x^r_i$. The second term, akin to a regularization enforcing Manhattan distance from $0$ as $|x^r_i - 0|$, constrains the range of counterfactuals. We therefore intuitively used the Manhattan distance measure of similarity between samples.  
   
   To better relieve your concern, we add the effect of distance type in contrastive learning in the latest version (see Table 8 in Appendix E.1). We evaluate three typical distance metrics in Rare-Time and Rare-Observation datasets: Manhattan distance, Euclidean distance, and cosine similarity, and we list it as follows. The results indicate that the Manhattan distance is slightly better than the other examined losses.
   

Table 8: Performance of ContraLSP with different contrastive loss types on rare experiments.
| $\mathcal{L}_{cntr}$ | AUP    |        AUR        |     $I_m$      |     $S_m$     |
| :-------------: | :-----------------: | :-----------------: | :---------------: | :---------------: |
|  |  | Rare-Time |                |  |
| Manhattan distance | **1.00**$\pm$ **0.00** | **0.97**$\pm$ **0.01** |   19.51$\pm$ 0.30   |  **4.65** $\pm$ **0.71**  |
| Euclidean distance | **1.00**$\pm$ **0.00** |     0.97$\pm$ 0.02     | **19.67**$\pm$ **0.52** |     4.97 $\pm$ 0.55     |
| cosine similarity | **1.00**$\pm$ **0.00** | 0.96$\pm$ 0.02 | 18.41$\pm$ 0.64 | **5.87**$\pm$ **0.74** |
|  |  | Rare-Observation |  |  |
| Manhattan distance | **1.00**$\pm$ **0.00** | **1.00**$\pm$ **0.00** | 20.68$\pm$ 0.03 | **0.32** $\pm$ **0.16** |
| Euclidean distance | **1.00**$\pm$ **0.00** | 1.00$\pm$ 0.01 | **20.72**$\pm$ **0.06** | 0.69 $\pm$ 0.17 |
| cosine similarity | **1.00**$\pm$ **0.00** | 0.98$\pm$ 0.01 | 19.22$\pm$ 0.06 | 0.98 $\pm$ 0.23 |

Reviewer comment (Evaluation):
**Counterfactuals**: The paper generously uses counterfactuals in the text to indicate perturbations produced by their model, whereas this is a label free approach and the mask is learned to minimize the gap between the original and unperturbed samples. The contrasting objective is the only source for potential counterfactual generation, that too it is not guaranteed to do so -- this distinction should be made more explicit in the text, and reduce the usage of perturbations being called counterfactuals. The experiments mostly only measure the ability of ContraLSP on identifying salient features, so this claim should be tempered down.

Author response:
Thanks for pointing this question out. Regarding the use of counterfactuals in our paper, we acknowledge the need to clarify the distinction between our label-free perturbation and traditional counterfactual perturbation [1]. Traditional explanations select counterfactuals through different labels. In contrast, our model utilizes contrastive learning to minimize the difference between perturbed and negative samples, primarily driven by the distance to the current sample. We need to consider regression and classification tasks where the selection of counterfactuals is not satisfied by labeling, as using labeled data conflicts with explaining black-box models. Instead, we believe that the generation of negative samples can be tended to counterfactuals through unsupervised methods [2], such as top-k prediction and clustering. However, this is beyond the scope of this work. We have added it to the limitation in the new version and left it for future work.
   

[1] Goyal, et al. Counterfactual visual explanations. ICML, 2019.

[2] UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models. 2023.

Reviewer comment (Evaluation):
In this context, can the authors elaborate on the failures or weaknesses of ContraLSP? Specifically, when is it expected to fail, perhaps in comparison to techniques that work with labeled data?

Author response:
Thanks for pointing this question out. Regarding the use of counterfactuals in our paper, we acknowledge the need to clarify the distinction between our label-free perturbation and traditional counterfactual perturbation [1]. Traditional explanations select counterfactuals through different labels. In contrast, our model utilizes contrastive learning to minimize the difference between perturbed and negative samples, primarily driven by the distance to the current sample. We need to consider regression and classification tasks where the selection of counterfactuals is not satisfied by labeling, as using labeled data conflicts with explaining black-box models. Instead, we believe that the generation of negative samples can be tended to counterfactuals through unsupervised methods [2], such as top-k prediction and clustering. However, this is beyond the scope of this work. We have added it to the limitation in the new version and left it for future work.
   

[1] Goyal, et al. Counterfactual visual explanations. ICML, 2019.

[2] UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models. 2023.

Reviewer comment (Theory):
**The contrasting objective**: Since negatives are chosen at random, they are likely going to be weak negatives, and claiming these will be "more counterfactual" is probably not true. It must also be defined what "more counterfactual" means here -- more than what? How does the random negative selection ensure perturbations are crossing over class boundaries? Why is an L1 edit distance the right distance metric to do this?

Author response:
Thanks for your constructive comment. We apologize for the "more counterfactual" that is incorrect and have modified it in the latest version. We believe the selection of negative samples by random, rather than by the furthest distance because we need to consider the classification task.  By intuitively opting for random selection, we aim to diversify the negative samples across different categories, rather than solely focusing on those distantly positioned from the current category. Previous studies [3, 4] have also depended on the quality of randomly-selected negative samples. Of course, these samples often lack complexity, making them easily distinguishable and offering limited supervised signals [5]. How to select negative samples more rationally for perturbation remains an area requiring exploration, which we intend to address in future research endeavors.

[3] Huang, et al. Cross-sentence temporal and semantic relations in video activity localisation. In *ICCV*, 2021.

[4] Zhang, et al. Counterfactual contrastive learning for weakly-supervised vision-language grounding. In *NeurIPS*, 2020.

[5] Zheng M, et al. Weakly supervised temporal sentence grounding with gaussian-based contrastive proposal learning. In *CVPR*, 2022.

Reviewer comment (Theory):
**OOD Perturbations**: How does this objective _guarantee_ or at least ensure lack of OOD perturbations? Is the training of the perturbation function with contrastive loss sufficient to ensure this?

Author response:
Thank you for the thoughtful comment! To investigate whether the perturbed samples are within the original dataset's distribution, we add the analysis experiments of perturbation distribution in the latest version (see Appendix F). We first compute the distribution of the original samples by kernel density estimation (KDE). Subsequently, we assess the log-likelihood of each perturbed sample under the original distribution, called as KDE-score, where closer to $0$ indicates a higher likelihood of perturbed samples originating from the original distribution. Additionally, we quantify the KL-divergence between the distribution of perturbed samples and original samples, where a smaller KL means that the two distributions are closer. We conduct experiments on the Rare-Time and Rare-Observation datasets and the results are shown in Table 12. It demonstrates that our ContraLSP's perturbation is more akin to the original distribution compared to the zero and mean perturbation. Furthermore, Extrmask performs best because it generates perturbations only from current samples, and therefore the generated perturbations are not guaranteed to be uninformative. This conclusion aligns with the visualization depicted in Figure 1. If you have any additional questions or comments, we would be happy to have further discussions.

Table 12: Difference between the distribution of different perturbations and the original distribution.
|  | Rare-Time |        Rare-Time        |    Rare-Observation     |    Rare-Observation     |
| :-------------: | :-----------------: | :-----------------: | :---------------: | :---------------: |
| Perturbation type | KDE-score$\uparrow$  |     KL-divergence   $\downarrow$  | KDE-score $\uparrow$ | KL-divergence $\downarrow$ |
| Zero perturbation | -25.242 | 0.0523 | -23.377 |  0.0421|
| Mean perturbation | -30.805 |  0.0731 | -26.421 |  0.0589|
| Extrmask perturbation |-22.532 |  0.0219 | -19.102 | 0.0104 |
| ContraLSP perturbation |  -23.290| 0.0393 |  -22.732 | 0.0386|

Reviewer comment (Theory):
**Sparse gating and stochasticity**: Please define what the heavy-tailed nature of the sparse feature selection is, and how it is relevant to ContraLSP. The hard thresholding function in eqn (3) is only needed due to the random noise injected in the masking, since $\mu'$ is a sigmoid function already. Why is the noise needed in the first place?

Author response:
Thanks for your constructive suggestion. The process of feature selection and explanation necessitates stability in the chosen feature set. However, employing logistic masks introduces notable variance in the approximated Bernoulli variables due to their heavy-tailedness [6], which like Figure 1 Top left in [6], often resulting in inconsistencies within the selected feature set. Therefore, we utilize the Bernoulli continuous relaxation method based on Gaussian variables, represented as the $\ell_0$-like constraint in Eq. (5). ContraLSP incorporates sample-specific sparse gates as mask indicators that are optimized by the $\ell_0$-like constraint, producing binary-skewed masks specific to time series data. In addition, we ensure the consistency of the underlying time series patterns by considering temporal trends.
   

[6] Yamada, et al. Feature selection using stochastic gates. In *ICML*, 2020.

Reviewer comment (Writing):
The paper's writing is not easy to follow, and this makes it hard to assess the core contribution of the work more rigorously. There are a lot of vague statements which are not stated clearly. Some of these are listed below:  
 * ".. perturbation may not alleviate the distribution shift issue.." (in the abstract)  
 * ".. unregulated data distribution .." (in Sec 3)  
 * ".. allows perturbed features to perceive heterogeneous samples, thus increasing the impact of the perturbation.." (Sec 4)  
 * ".. counterfactual perturbations more distracting.." (Sec 4.1)  
 * ".. due to their heavy tailed nature.." (Sec 4.2)

Author response:
Thanks for the nice question, but it's not right. To prevent temporal abruptness and jump saliency, we adopt the sigmoid-weighted unit that $\mu^\prime=\mu*\sigma(\tau*\mu)$ to smooth the mask. Thus, $\mu^\prime\in \mathbb{R}$ where is learnable, it is just smoothed by the sigmoid function. The reason why we inject a Gaussian noise on $mu^\prime$ is that the $\ell_0$-like constraint is used instead of the traditional $\ell_1$ regularization in Eq. (5), which makes the mask more binary-skewed [6] (i.e., biased towards 0 or 1 but smoothness).

Reviewer comment (Presentation):
**Illustrations**: I recognize time series explanations are more challenging than visual data like imagery. However, the current set of illustrations are not very clear. For e.g. in Fig 5 why is the sum of salient observations shown? What is the inference from this figure? It is very unclear; please make the key observations more explicit, perhaps with the help of a simpler dataset or time series and more consistent with Fig 1, which is easier to follow.

Author response:
We appreciate the reviewer's feedback regarding the clarity of our illustrations, particularly Figure 5. We present a comparison between the perturbations generated by ContraLSP and Extrmask on the Rare-Observation (Diffgroups) experiment, as shown in Figure 5 (the updated version). We randomly select a sample in each of the two groups and sum all observations. ContraLSP provides counterfactual information, yet Extrmask's perturbation is close to 0. This suggests ContraLSP provides counterfactual perturbations that are more beneficial to non-salient areas while emphasizing the mask more distinctly in salient areas compared to Extrmask. This is consistent with the findings in Figure 1. In addition, we show the perturbation of summed observations, leveraging statistical features to understand this dataset. This is because there are 50 observations in this experiment, but only 5 salient observations are included. 

We thank you again for your constructive comments and for your efforts to improve the quality of our paper.