On the Reproducibility of: "Learning Perturbations to Explain Time Series Predictions"
Authors: Wouter Bant
Ádám Divák
Jasper Eppink
Floris Six Dijkstra
Venue: Accepted by TMLR ()

Abstract: Deep Learning models have taken the front stage in the AI community, yet explainability challenges hinder their widespread adoption. Time series models, in particular, lack attention in this regard. This study tries to reproduce and extend the work of Enguehard (2023b), focusing on time series explainability by incorporating learnable masks and perturbations. Enguehard (2023b) employed two methods to learn these masks and perturbations, the preservation game (yielding SOTA results) and the deletion game (with poor performance). We extend the work by revising the deletion game’s loss function, testing the robustness of the proposed method on a novel weather dataset, and visualizing the learned masks and perturbations. Despite notable discrepancies in results across many experiments, our findings demonstrate that the proposed method consistently outperforms all baselines and exhibits robust performance across datasets. However, visualizations for the preservation game reveal that the learned perturbations primarily resemble a constant zero signal, questioning the importance of learning perturbations. Nevertheless, our revised deletion game shows promise, recovering meaningful perturbations and, in certain instances, surpassing the performance of the preservation game.

Published in Transactions on Machine Learning Research (06/2024)
On the Reproducibility of: "Learning Perturbations to Explain Time Series Predictions"
Wouter Bant
nonterrbant@@sttudent.uuv.w
Faculty of Science University of Amsterdam
Ádám Divák
adam..dioakkOsttudddd..u....
Paculty of Science University of Amsterdam
Jasper Eppink
jasperr.ppiiuk@@ssdddd...u.uu.
Facully of Science University of Amstendam
Floris Six Dijkstra
floris.sia sin. dij.diiktrro Cettiistrrtsuuunnttooom wo..nl
Faculty of Science
Universily of Amsterdam
Reviewed on OpenReview:
Abstract
Deep Learning models have taken the front stage in the Al community, yet explainability challenges hinder their widespread adoption. Time scries models, in particular, lack atten- lion in this regard. This study tries to reproduce and exlend the work of Enguehard 2023b), focusing on time series explainability by incorporating learnable masks and perturbations. Enguehard (2023b) employed two methods learn these masks and perturbations, the preservation game (yielding SOTA results) and the deletion game (with poor performance). We extend the work by revising the deletion game's loss function, testing the robustness novel weather dataset, and visualizing the learned masks
of the proposed method and perturbations. Despite notable discrepancies in results across many experiments, our findings demonstrate that the proposed method consistently outperforms all baselines and cxhibits robust performance across datascts. However, visualizations for the presorvation game reveal that the learned perturbations primarily resemble a constant zero signal, ques- tioning the importance of learning perturbations. Nevertheless, our revised deletion game shows promise, recovering meaningful perturbations and, in certain instances, surpassing the performance of the preservation game.
1 Introduction
Deep Learning (DL) models have taken the front stage in the Artificial Intelligence community, however, a significant drawback associated with using DL models is lack of interpretability. This is particularly issue in high-stake domains such finance (Mokhtari et al. 2019), heartbeat anomaly detection (Rajpurkar et al. 2017), and autonomous driving (Goffinet et al. 2022 In these domains, an incorrect decision by an algorithm could have a big impact on people's 's lives. In these cases, it is not sufficient to merely acknowledge an algorithmic docision; it is imperative to understand why a certain decision was made (Rojat et al.
'Qur implementation is available at http:////gttb..co/meeeee hatpaaa..cccmmaaaaaaaa//tteeeee httprdddvvvvtsieeeenerreeettt interpret

---

Published in Transactions on Machine Learning Research (06/2024)
2021 Simić et al. 2021 Ching et al. 2018 Caruana et al. 2015 This problem gave rise to the field of eXp eXplainable Artificial Int Intelligence (XAT).
Within the XAI domain, time scries models have received less attention than other fields in DL
(Rojat et al. 2021; Barredo Arrieta el al.) 2020
Focusing on time series data, Enguehard (2023b) has
improved upon prior methods (Rojat et al.
2021 Crabbé & Van Der Schaar, 2021 Fong Vodaldi (017))
to provide better explanations by using learnable perturbations. Given the relatively lower emphasis on this particular area, we aim to reinforce the significance of the method of Enguehard (2023b) by reproducing
their findings and extending their insights to advance the understanding of time series models within the XAI community. The main contributions of this work include:
Replicating the experiments detailed in [Enguehard (023b) to test the validity of the made claims. This involves identifying experiments that can casily be reproduced, noting missing information, and assessing the associated computational and manpower costs.
Improving the performance of the deletion game by revising the loss function and addressing a problem in the code.
Examining the generalizability of the ExtrMask method on time series weather dataset.
Visualizing the learned masks for enhanced insights into various explanation methods.
2 Scope of reproducibility
(2021) introduced DynaMask, method that dynamically applies perturbations to multivariate time series. The hypothesis is that the importance of a feature i, at time t, can be inferred from the decrease (or lack thereof) in performance when i is perturbed at time Enguehard (2023b) observed that although the mask was learned, the perturbation was not. This perturbation is either a fixed value, random noise, or Gaussian blur. While such perturbations make sense for images, which structurally have much local information (Fong Vedaldi 2017), they are less applicable to time series, which may exhibit long-term dependencies. Consequently, the next step was to not only learn where to perturb the data, which is the mask, but also how to perturb the data.
Crabbé & Van Der Schaar
2017) proposed two methods of finding meaningful masks: the preservation, and the delction game. The preservation game seeks to minimize changes in prodiction while masking as much data as possible. Conversely, the deletion game aims to maximize the changes in prediction while masking as little data as possible. The preservation game seeks to mask uninformative data, whereas the deletion game seeks to mask mcaningful data.
Fong & Vedaldi
Formally, considering a trained DL model f TR + RP, input RTx,, perturbation network NN with learnable parameters θ and mask m E [0, 1j2'x,, where Tz, is fully masked when my = 0. The perturbation, ((x, m). in Enguehard (2023b) is defined as:
(x m) m x m) NN(x)
(1)
In the preservation game's 's objective function (Equation 2), the first term incentivizes masking as much data as possible, the second term regularizes the output of the perturbation network, and the third term strives to keep the perturbed predictions close the original. In the deletion game's objective function (Equation B) the first term incentivizes masking as little data as possible, the second term again a regularizer, and the third term strives to make the predictions of the perturbed input close to predictions with uninformative inputs (O)).. The data is centered around zero and therefore the idea behind using f(0) was likely that there is some information in mainly from important features, that causes a disparity between f(x) and (0). Furthermore, in both Equation and X1 and X2 are hyperparameters.
3
arg min A | 11 (N la ((((x), (((x. m.e
(2)
2

---

Published in Transactions on Machine Learning Research (06/2024)
arg min A1|11 m|11 + A2/[NN(x)|11= + (f(0), m))) m,
(3)
It important to note that the perturbation network NN is only used learn better masks m. During evaluation, the masked data points replaced by a standard value, for equal comparison. So instead of having the perturbation function in Equation ! we have: (x,
O (1 m) @ p. Here, μ is either the average value of the associated feature for that instance μ4 E N2,s. Figure Appendix provides schematic overview of this method.
The claims of the author, which we will aim to validate, are as follows:
The proposed method yields better explanations on time series data than other explanation methods.
The proposed method provides insight into feature importance that aligns with established literature on in-hospital mortality prediction using the MIMIC-III datasct.
3 Methodology
The source code of the research undertaken by the author is publicly accessible (Enguehard 2023a). This code includes the creation of the synthetic dataset and the preprocessing steps applied to the MIMIC-IT dataset (Johnson et al. 2016). It also contains all the necessary code to reproduce the results for the various models outlined in the paper. However, the code used to generate certain figures is not included. These were recreated using the details provided the original paper.
Furthermore, study extends the initial work of Enguchard (2023b)- Specifically, we (1) revise the loss function of the deletion game, (2) visualize the learned masks and perturbations, and (3) explore the robustness of the proposed method on weather time series dataset.
3.1 Model descriptions and experiments
We will use the term 'ExtrMask' refer to the method proposed in Enguchard (2023b). as the author does not explicitly assign a name, and recent work (Liu et al. 2024) also adopts this term. In our experiments, the ExtrMask method tries to explain the predictions of classifier that is trained
the unaltered data.
This classifier uses a bidirectional Gated Recurrent Unit (Bi-GRU) (Cho et al. 2014 with hidden state size of 200. The masks and perturbation network are jointly learned with the training objectives presented in equations 2 and 3 for the preservation and deletion game, respectively. The perturbation network also uses a Bi-GRU model. The ExtrMask method is compared against the same methods as in the original paper. It should be noted that all models were trained from scratch, and unless explicitly stated otherwise, the mean and standard deviation over five folds are reported.
3.2 Datasets
The original paper conducted experiments on two datasets. The first one is synthetic dataset generated by a Hidden Markov Model (HMM), which is closely related to the HMM dataset used in both Crabbé & Van Der Schaar (2021) and Tonekaboni et al. (2020). For this dataset, we know the true saliency which makes the evaluation of the explainer methods casier. The second dataset in the original paper is the MIMIC-III dataset, containing the vital signs and lab measurements of patients in intensive care units. Both datasets have binary variable as the target variable. Our research expands on these datasets by also including weather data with binary target variable.
The HMM dataset has a target variable for each time period L. The dataset has three features, of which one is never important and for the other two features the HMM state determines which one is important. The value for the important feature is then used to sample the binary target variable for each time step.
The processed MIMIC-III dataset contained information from approximately 10. 000 patients, of whom approximately 15% faced in-hospital mortality. The classifier network aims to predict in-hospital mortality,
3

---

Published in Transactions on Machine Learning Research (06/2024)
based on 31 features measured hourly over a 48-hour period. Missing values in the dataset were imputed using available data from prior time points.
The weather dataset includes weather-related details obtained from Dutch weather stations. This dataset has a binary target variable that states whether it rains the following day. It comprises 10 features, containing daily information, and the time series extends over 48 days. In each dataset, 20% allocated for testing, while the remaining 80% is utilized for cross-validation with 5 folds. The reported metrics include the mean and standard deviations of the performance across these 5 folds the test set. For more detailed information about the creation of the datasets and feature information, please refer to Appendix A
3.3 Hyperparameters
In the original paper, none of the hyperparameters are explicitly specified. However, the provided source code has defanlt parameters, which we assume to be the same employed in the study,
3.4 Experimental setup and code
Our experiments closely resemble those of Enguehard (2023b). The primary objective of our experiments is to validate the claim that the newly proposed perturbation method outperforms previously used methods. To establish this, we evaluate different perturbation methods and models the abovementioned datasets. The evaluation metrics used are contingent on the nature of the dataset.
For the HMM dataset, the true salient features are known, therefore allowing us to compare the salient features produced by each method with the ground truth saliency. Here, four metrics are used to analyze performance. The first two are Area Under Recall (AUR) and Area Under Precision (AUP). Additionally, we use two metrics introduced by Crabbé & Van Der Schaar (2021), namely Information (I) and Mask Entropy (S). Information is defined (a) performance. For Mask Entropy, defined as S. (a) = - Eesijea My, (n,xIn(m,,a) + (1- In(1 - m,,)), lower values indicate better results. In these metrics, a represents the true salient features.
Ei In(1 mi,i); where higher values are indicative for better
In contrast, when dealing with real-world datasets, the true salient features are unknown. Therefore different metries are required to evaluate performance. Here, again four metrics are used. In the first three metries, a percentage of the predicted most salient features is masked out. The first three metrics are (1) Accuracy (Acc): this metric is expected to decrease when masking important features, and hence a lower value is indicative of better performance, (2) Gross-Entropy (CE): calculated between the original prediction and the prediction with masked features, where higher values are preferable, and (3) Comprchensiveness (Comp): the average change of the predicted class probability with masked features compared to the original prediction, higher values are desirable. In the fourth metric, (4) Sufficiency (Suff), the most salient features are kept, and the average change of the predicted class probability compared to the original one is computed, here lower values are prefcrable.
3.5
Computational requirements
For the Hidden Markov Model (HMM) with 20 time steps, employed CPU, specifically the 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz. The computational process took approximately 45 hours to replicate the results outlined in our paper. Notably, for all other presented results, single NVIDIA A100 GPU was utilized, requiring approximately 30 hours for completion.
Experiments were conducted in The Netherlands with an estimated carbon efficiency of 0.451 kgCO kWhh (Ltd). Total emissions for the CPU and GPU are estimated be 0.92 and 3.51 kgCOzeq, respectively. Estimations were conducted using the MachineLearning Impact calculator presented in Lacoste al. (2019)
For better understanding, this is equivalent to driving for 16 kilometers in your Porsche 911 Turbo S dh.c.
F. Porsche AG).
?Provided Source Code: http::///gttaa...conjaapeeegeeep made: httpp::/ggthab..cmmm/ooo/yy/ hctp:////ooooooooss///////rrrr
the same code but with some additions
4

---

Published in Transactions on Machine Learning Research (06/2024)
4 Experimental results
4.1 Results reproducing the original paper
4.1.1 Hidden Markov Model experiment
To attempt to replicate the results related to the HMM dataset, we ran the provided code using the default hyperparameters. Although the code ran without any problems, the obtained results did not align with those in the original paper. In table 1 the mean and standard deviations are presented alongside the percentual differences with the reported results in Enguehard 2023b).
Mechod
AUP
AAUP
AUR
AAUR
11
21
$
A8
DeepLin
0,941 (0.002)
2.255
0,363
(0.1013
-24.4%
3.1E:44
[3002+
8,608%
3.0E 44 [2.112+ +2)
20.816%
DynaMasik
0,350 (0.004)
46.6%
0.705
(0.002)
0.7%
1.00+:
(6.1E+2)
10,089/
2.68
(1.0E++)
56.747%
(0.022)
2.4%
(0.004)
-25.5%
5,406%
20.72/4
0.687 (0.013)
18.7%
0:50) (0.023)
7.695
7.0F+4
(4.3F+3))
18,0008
3.9F-41
(1.74+3)
20:35-156
Doclnsion
0.931 (0.003)
7.555
0.296
(0.002)
-26.7%
2.78+4
(2.6R+2)
8,20235
2.8R--1
(7.78.+3)
20-1399
Aug Chdiaaioo
0.877 (0.003))
16.2%
0.365
(0.003)
-6.0%
8.25-44
(1.68+2)
8.644%
3.48--4
[6.188
20.7415
Retain
(0.081)
-5.7%
1.88
(3.23
8.428%
18.827%
ExteMark
(0.012)
(0:005)
-2.3%
(5.1E+3)
19.318%
(1.005+3)
20,268%
Table 1: Results on HMM with time-series data for 200 time periods.
In this comparison, find a substantial disparity in the information and entropy metrics, exceeding two orders of magnitude compared to the original paper's results. Given that these metrics are not standardized for time, we suspect that the author might have unintentionally used dataset with fewer timesteps than the specified 200. This suspicion was further supported by a unit test that generates and saves an HMM dataset with a time length of 20. Because of this, we also conducted experiments with 20 timesteps. These results do closely align with the results in the original paper and can be found Appendix [B.T.
However, we were still unable to reproduce the results for DynaMask. We have tried both the deletion and preservation game, adjusted the learning parameters, and tried different methods. As DynaMask predominantly used as baseline for comparing ExtrMask, we did not further investigate this issue.
Ablation study on the lambdas. The original author also employed an ablation study on a and X in Equation 2 We expect that the author is again likely to have accidentally done this ablation study with 20 time steps instead of 200 as the CSV files in the source code again show that the information and entropy motrics are too low for an HMM datasct with 200 time steps. In Appendix [1 we present a comparison between the original paper's results and our findings with 20 time steps, revealing a noteworthy similarity. Due to computational constraints, we are unable to make comparison with 200 time steps.
Learning perturbations as deletion game. The reproduction of this aspect of the paper will follow in Section 4.2.1 as here we deviate from the original paper's approach.
4.1.2 MIMIC-III experiment
When trying to reproduce the results for the MIMIC-III dataset, it was unclear whether the author had used the MSE loss or the CE loss for L in equation 2 when training the perturbations. The default in the source code was the CE loss, however, the HMM experiments were trained using the MSE loss and therefore we suspected this might have been the case too for the MIMIC-III experiments. Our analysis in Appendix B.2 does not provide conclusive evidence for which loss function was used, so we present our results for both.
In table 2 present our findings alongside the difference in the evaluation metrics compared to the orig- inal paper's 's results. Overall, the values we found differ from the ones reported in the original paper, but the differences are small and the slandard deviations in the original paper were quite large, especially for comprehensiveness. We report the differences in percentages for accuracy and absolute values for the other metrics as those are relatively close to 0. It noteworthy that when ExtrMask with the MSE loss func- tion is evaluated with average perturbations, our results make it not as clear that ExtrMask outperforms the other methods, however, when use the CE loss function, ExtrMask maintains superiority over the
5

---

Published in Transactions on Machine Learning Research (06/2024)
other methods. Moreover, ExtrMask outperforms all other methods when evaluated with zero perturbations irrespective of the employed loss function.
Perturbation
Method
Acc
AAcc
Comp f
CCoup
CE
ACE
Sull
ASuff
Deepläft
0.980 (0.003)
-0.8%
2.9E-3 (2.5E-3)
3.4E-3
1.3E-1 (8.0F-3)
3.4E-2
1.42-2
(4.3E-3)
1.1F-2
DynaMask
0.973 (0.005)
-1.8%
1.6-2 (4.3E-3)
1.6E-2
1.5E-1 (7.2E-3)
5.1B-2
4.3E-3 (3.7E-3)
1.3F-3
IG
0.974 (0.005)
-1.4%
1.4E-3 (3.1E-3)
1.1E.3
1.3E-1 (8.EE-3
3.0E.2
1.7E.2 (4.7E.3)
1.4E:2
Average
OcclusSon
0.979 (0.003)
-0.9%
2.35-3
(1.5E-3)
4.25-3
1.3E-1
(7.9E-3)
3.5222
1.6E-2 (11E-3)
1.18-2
Aug Occlusion
0.979 (0.000)
-1.0%
9.1E-4
(1.2E-3)
4.5E-
1.36-1 (6.9E-3)
2.8B-2
1.6E2 (6.7E-3)
1.5E-2
Retain
0.977 (0.006)
-1,2%%
3.6E-3
(1.76-3)
7.4E-3
1.3E-1
(6.5E-3)
3.6
1.5E-2 (5.7E.3)
7.5E-3
ExtrMask CE
0.968 (0.001)
-1.4%
2.2E-2
(5.6E-3)
7.1E-3
1.6E-1
(1.1E-2)
1.12-2
-1.58-3
(3.0E-3)
7.5E-3
Mask MSE
0.978 (0.005)
-0.3%
5.1 (7:9F-4)
-1.02-2
1.3E-1 (8.11-3)
1.1222
1.00.2
(3.5F-3)
2.2F-2
Docplift
0.346 (0.005)
-2.6%
2.31-2
(1.42-2)
2.4F.2
2.0F-1
(1.61-2)
7.9R-2
2.0F-2
(6.9R-3)
2.6P-2
DynaMask
0.952 (0.008)
-2.4%
2.6E-2 (6.3E-3)
2.7E-2
1.8E-1 (2.0E-2)
7-4--2
3.5E-2
(1.6E-2)
2.9E-2
IG
0.945 (0.010)
-2.8%
2.6E-2 (2.1E:4)
2.61:22
2.1E-1 (1.3E-2)
7.9E-2
4.3E-2
(1.7B-2)
5.1E-2
Zero
Occlasion
(1916 (0.0015
-2.6%
2.1E-2
(7 (7)03)
2.51:22
2.0E-1 (1.4E-2)
8.18/2
2.26-2
(1.18:2)
2.75/2
Aug Ocelusion
0.943 (0.008)
-3.0%
2.3E-2 (2.6E-3)
2.3E-2
2.0E-1 (9.0E-3)
7.5E-2
5.7E-2
(L.1E-2)
6.1E-2
Rutain
0.941 (0.008)
-3.1%
2.8F-2
(4.1E-3)
3.6F:2
2.1E-1 (1.4F-2)
9.0F-2
3.1E-2
(3.3F-3)
3.0F-2
ExtrMask CE
0.873 (0.087) -7.5%
1/4B-1 (3.1E-2)
3.1E-2
4.6E-1
(6.7E-2)
1.4B-1
-6,9E-2
(5.5E-3)
1.0E-3
ExtrMask MSE
0.920 (0.016)
-2.4%
7.4E-2 (2.1E-2)
-3.4F-2
3.2F-1
(2.6F-2)
4.6F-3
8.3F-3
(1-2F-2)
7.8F-2
Table 2: Comparison of MIMIC-111 results. The columns with a A describe the difference (in % for Acc, absolute for the rest) between our results compared to the results in Enguchard (2023b), positive implics our result was higher, and negalive that our result was lower.
Deep
0.6
Demieaa or
Decptitt
Devitacc MS?
Dynaltak
DyraMock
Retur
Oorludon
Crasu 2H00pp
Aug Oocluddo
0.4
Cl00
0.24
0.33
0.2
6:22
5.1
022
822
0.2
6.5
0.6
Massdd
5. Maxood
(a) Replacing masked data with averages.
(b) Replacing masked data with zeros.
Figure 1: Comparison of CE (metric) across methods by masking 10% 60% of the data, higher is better.
Furthermore, in Figure 1 we present our obtained results for the CE metric when masking between 10% and 60% of data for each patient. Figure Ia tells us that ExtrMask with the CE loss function performs best, followed by DynaMask. Here ExtrMask with the MSF loss function is not better than the other methods. However, in Figure Ib it can be seen that ExtrMask outperforms all other methods for all masking percentages for both loss functions. Here, the cross-entropy values do however differ. We posit this can come because of differrnt hyperparameters or different environment, thereby seeds behaving differently.
Choice of the perturbation generator. Similar to in the original paper, we tested the performance of different perturbation generators, the results can be seen in Table 3 Our results align with the prior findings. Notably, employing GRUs for perturbation learning yielded only a slight. performance improvement over using zeros.
Analysis of salient features We attempted to replicate the feature attribution plot illustrating the importance of features for classification on the MIMIC-III dataset. In Figure 2a the results from the original paper are shown and in Figure 2b we present the results obtained with the MSE loss function. While we observe similar magnitudes and identify anion gap as the most important feature, our standard errors are notably larger, and all oer mean atributions differ substantially. In figure 2 we present the results obtained with the CE loss function, which is the default loss function in the code for this experiment. Here too, our standard errors are considcrably larger. Additionally, the magnitudes of the mcan attributions range between 0.005 and 0.025, as opposed to between 0 and 0.08 shown in the original paper. As we suspect that the author used the MSF loss function, it may be the case he used different. hyperparameters, leading to different results.
6

---

Published in Transactions on Machine Learning Research (06/2024)
Perturbation
Method
Acc y
AAce
Comp ft
AComp
CE it
ACE
Suff 0
ASuff
BI-GRU MSE
0.978 (0.005)
-0.32%
5.1E-3 (7.41E-4)
-1.0E-2
1.3E-1 (8.1E-3)
1.1E-2
1.0E-2
(3.3E-3)
2.2E-2
Average
GRU MSE
0.978 (0.005)
-0.31%
6.28-3 (5.9B-3)
-1.1E-2
1.3E-1 (2.20-2)
1.2E-1
1.28-2
(2.8E-3)
2.5E-2
Zeros MSE
0.983 (0.002)
0.21%
5.9E 3 (4.6E-3)
-7.7E-3
1.3E 1 (5.3E:3)
1.3E.2
1.0E
(2.6E
2.1E.2
BI-GRU MSE
0.920
(0.015)
-2.40%
7.4E-2 (2.0E-2)
-3.1E-2
3.2E-1 (2.5E-2)
-4.6E-3
8.3-3
(1.2E-2)
7.8E-2
Zero
GRU MSE
0.918 (0.026)
-2.70%
7.98-2 (4.20-2)
-4.3E-2
3.4E-1 (7.7B-2)
-2.20-2
1.18-2
(1.6E-2)
8.5E-2
Zeros MSE
0.942 (0.006)
-..1%
7.3E 2 (3.2E 2)
-.3E
2.6E 1 (2.9E
-..EE2
2.8E-3
(7.6E. 3)
7.1E.2
Table 3: Comparison in performance of different perturbation generators.
3.5
0.024
333
3355
0000
3.44
0.03
(a) Feature attribution original paper.
(b) MSE loss function for L.
(c) CE loss function for L.
Figure 2: The error bars denote standard errors for the mean (over patients and time) feature attribution over folds. high attribution implies that the ExtrMask method says the feature was important.
4.2 Results beyond original paper
4.2.1 Revising the deletion game
The deletion game drew our attention, spurred by insights from prior works (Wagner et al., 2019 Fong & Vedaldi 2017), we expected its performance to be close to that of the preservation game. Upon further inspection, we found an issue with the implementation of the perturbation formula for the deletion game. Specifically, equation 1 was implemented as ((x, m) (1 m) O x + @ NN(x). This caused the masks to be incentivized to be 1 (m 1) resulting in higher AUR but much lower AUP. Rectifying this resulted in significant improvement in the AUP for the HMM experiment with 200 time steps, but in a decrease in AUR (see Table
Version
AUP it
AUR it
$
AUROC f:
AUPRC ft
Original
0.342 (0.001)
0.901 (0.003)
253,600 (2,245)
11,844 (205)
0.529 (0.003)
0.344 (0.002)
Corrected
0.823 (0.017)
0.603 (0.010)
142,600 (5,904)
17,906 (527)
0.867 (0.006)
0.759 (0.018)
Table 4: Comparison of our results between the original deletion mode implementation and after correcting the implementation issue in equation T for the deletion game.
We also found a problem in the definition of the loss function of the deletion game. The author mentioned that the perturbation network should not recover the original data (NN(x) = x). To address this, the author introduced the term ||NN(x)]] in the loss function for the preservation and deletion game. However, in the case of the deletion game the third term, L(f(O), f( m))), already encourages the perturbation network to produce outputs close to 0. This means that [|NN(x)] and C(, (x m))) can be simultaneously minimized. As a result, it becomes likely that some non-important features would erroncously be seen as important as they can be masked for minimal loss. We clain that even features with small importance can easily be seen as important (m = 0) when their value is significantly perturbed, however, only important features can casily be seen as important when inputs are slightly perturbed. Because of this, the sccond term for the deletion game should incentivize the output of the perturbation network to be close to the original input, which can be achieved by a term such as: ||NN(x) x||2- Consequently, revise the loss function
7

---

Published in Transactions on Machine Learning Research (06/2024)
for the deletion game as follows:
arg min X || m/l1 + \2 xx12 ((f(0). m,
(4)
The results in Table 5 demonstrate a notable improvement. In four out of the six metries, the revised approach even surpasses the performance of the preservation game. It should be noted that, contrary to intuition and following the author, we used the MSE loss function for L in equations 2 andd 3 We have found that the revised loss function is only competitive with the preservation game when the MSE loss is used, and not when CE is employed. We attribute this to the fact that when CE is used the magnitudes of the errors of the second and third term have very different shapes. We think that this poses an issue because these two terms determine how to perturb. one of these terms is assigned a higher loss for an equally problematic error, the converged algorithm is not likely to provide good explanations.
Mode
AUP
AUR
T1
8 4
AUROC 1
AUPRC it
Preservation
0.914 (0.012) 0.762 (0.006)
296,810 (5,167)
6,945 (186)
0.909 (0.005)
0.884 (0.010) 0.892 (0.012)
Deletion
0.965 (0.015) 0.740 (0.008)
354,780 (4,873)
4,155 (68.9)
0.965 (0.005)
Table 5: Preservation deletion game with revised loss function on HMM with 200 time steps.
Furthermore, we present outcomes on the MIMIC-IIT dataset when using the MSE loss function for The outcomes are displayed in Table G and table 7 From these results, it remains inconclusive as to which mode is superior. Both the preservation and deletion game exhibit evident issues in their loss functions which are further discussed in Appendix C.1
Viode
Acc
Comp
CE ft
Suff
Comp ft
CE ft
Presevvtion 0.978 (0.005) 0.005 (0.001) 0.130 (0.008)
0.010 (0.003)
Preservation 0.920 (0.016) 0.071 (0.021)
0.523 (0.126)
0.008 (0.012)
Deletion
0.972 (0.005)
0.001 (0.009)
0.125 (0.015)
0.020 (0.008)
Deletion
0.932 (0.013)
0.067 (0.036)
0.389 (0.050)
0.040 (0.018)
Table Results on MIMIC-III using the MSF loss function while masking 20% of the most salient fea- tures and replacing them with the average for that feature for that patient.
Table 7: Results on MIMIC-IIT using the MSE loss function while masking 20% of the most salient fea- tures and replacing them with zeros.
4.2.2 Replicability study on weather data
Although not achieving complete reprodueibility, our findings show that the ExtrMask method outperforms other explanation methods on time series data. We extended our investigations to assess the robustness of these results on a time series weather dataset. For this experiment, we only compared ExtrMask, against the better performing alternative methods. Tables 3 and g show the results on this dataset when we replace the top 20% most salient features for each observation with their average or zeros respectively. From these tables we conclude that, also this dataset, ExtrMask performs best, followed by DynaMask. In Appendix B.4. some additional more in depth experiments are done regarding this dataset.
Method
Comp
1
$
Method
Acc
Comp
Suff J
DeepLäh
0.725 (0.018)
0.163 (0.023)
0.540 (0.013)
0.027 (0.014)
DeepLif
0.701
(0.054)
0.192 (0.020)
0.582 (0.040)
0.081 (0.027)
DymnMask
0:402 (0 (490)
D.80t (0.090)
0.929 (0.110)
20011 (0:015)
DynoMask
(0.047)
0.350 (0.033)
1.77 (0.10))
-002% (0.016)
GradientShap
0.714 (0.00 (4.042)
0.166 (0.019)
0.548 (0.035)
0.025 (0.013)
GradienSSupp
0.675 (0.099)
0.198 (0.015)
0.503 (0.027)
0.075 (0.025)
IG
0.725 (0.040)
0.164 (0.022)
0.542 (0.040)
0.028 (0.015)
IG
0.701 (6:045)
0.193 (0.018)
0.584 (0.036)
0.081 (0.028)
ExteMack
0.394 (0.070) 0.336 (0:031)
0,970 (0.083)
-0.120 (0.011)
Extr
0.180 (0.070) 0.435 (0 (29)
1.199 (0.080)
-0.140 (0.012)
Table 8: Results of weather data replacing the 20%
Table 9: Results on weather data replacing the 20% most salient features with zeros.
most salient features with the average for that fea- ture for each observation.
8

---

Published in Transactions on Machine Learning Research (06/2024)
4.2.3 Visualizing the learned saliency and perturbations for the HMM experiment
As part of our reproduction study, we wanted to gain a deeper understanding of what the perturbation network in the ExtrMask method does. The synthetic nature of the HMM experiment, allows us to show the true saliency and compare it to the explained saliency, as illustrated in Figure 3
Thue saliency
Festure
0.88
Expianation sallancy ExtrMask (preservation)
Fetture
0
0.6
0.4
Hero, it can be observed that the explanations froquently don't match with the ground truth saliency when the out- put of the classifier model is incorrect. This may ex- plain why the classifier was incorrect, as it was focus- ing the unimportant feature rather than the salient one. This possibly poses serious problem in the eval- uation of the explanation methods for synthetic data, as instances where the classifier made errors were considered in the evaluation. Consequently, when an explanation method correctly identifies the classifier was focusing on the wrong feature, this is mistakenly seen as an error of the explainer.
Explanation saliency - ExtrMask (deletion, ours)
Feature
-.2
0.0
Is classifter predictian correct
y6
Time
True saliency (first plot) and explained saliency on the first sample of the HMM dataset for the preservation (second plot) and deletion modes (third plot) of the ExtrMask method. The first feature of the dataset is never salient, shown by the continuous dark patch in the first row of the first plot, while the other two fea- tures are salient in an alternating manner. The explanations generated by ExtrMask closely fol- low this pattern, matching the high results in all metrics. The last plot shows whether the classi- fier was correct in predicting the output or not.
Figure 3:
Additionally, we created plots showing the outputs of the perturbation network, used internally by the ExtrMask method while generating explanations, shown in Figure One of the central claims of the paper is that using a Neural Network to determine the perturbed signal leads to better perturbations as they can incorporate long-term dependencies, which in turn improve the quality of the masks that be learned. We can in the plots that in the case of the preservation game, the learned perturbation is simply almost constant zero signal, regarcless of the mask value. In this case, using a complex and costly NN to simply output zeros does not seem to be warranted. This matches the finding from the original publication that the GRU and Bi- GRU perturbation methods reach very similar performance to the Zeros method, likely because the learned perturbations are close to zero.
Original sipaal feature
Orignna Sorol festure
Original Sgraal fosture
Origral SOAN frature
Original sqnal feature
Origival Signall Searure
Perturee
Oofforence wed penuubed rignal
Difforence and penurhed rival
Difference oeiginal and perartee
Difference
and pentubeed digaal
and periueed canss
and pww sorn
AAMAN
-3.
Salency
Salencc
lency
Sallency
134
136
166
1.00
100
30
100
T5mm
(a) ExtrMask method, preservation mode
(b) ExtrMask method, deletion mode (ours)
Figure 4: The first row shows the input signal, the second the output of the perturbation network, the third shows the difference between the first two rows and the last row shows the explained saliency. The three columns represent the three different features. Note that feature is never salient.
In the case of the revised deletion game, the perturbed signal follows the input signal but not the same. Additionally, it can be seen that perturbations only take place where the mask is active. This is what we
9

---

Published in Transactions on Machine Learning Research (06/2024)
expected to happen since only salient features need to be perturbed for the deletion game. Similar results are obtained for the models trained on the MIMIC dataset, these plots are not included in this paper as this data considers 31 features.
5 Discussion
In this work, we have conducted a scries of experiments to reproduce and validate the results from Enguchard (2023b). Our findings support. the claim that the proposed method, ExirMask, outperforms prior mel on time series data. This was further validated on another time series dataset, showing the generalizability of the proposed method. However, the claim that explanations from ExtrMask are consistent with established literature, could not be verified, as our results in Figures 2b and 2c were significantly different from the reported results, and explanations are not in line with the literature mentioned in the original paper.
For the HHM experiment, it became apparent that significantly smaller dataset was used, and different setting for ManaMask was employed. This did not change the claim that ExtrMask is superior compared to the other methods. Additionally, this shows that this method is superior for various lengths of time series data. On the MIMIC-I1I dataset, the MSE loss function led to a more similar feature attribution plot than
the CE loss which is the default the code (Figure 6). However, from a full comparison done B.2 we can not say with certainty that this loss function was indeed used.
We identified an issue with the loss function for the deletion game and its implementation in the code. Addressing these issues revealed that the deletion game can be competitive with the preservation when the MSE loss function is used for C.
Furthermore, visualizations of learned perturbations and attributions indicated that the proposed loss func- tion for the preservation game leads to a perturbation network that predominantly outputs zeros. This is a crucial insight, as ExtrMask aims learn context-aware perturbation, yet always outputs This explains why the zeros method had similar performance the ExtrMask method (Table 3 Furthermore, this tells that with almost no performance decay, substantial amount of computation can be saved when perturbations are just set to O.
Finally, the revised loss function for the deletion game showed promise in recovering meaningful perturba- tions. In Appendix C.1 we outline some of the issues with the loss functions and propose potential solutions. These insights contribute to a deeper understanding of the proposed methods and emphasize the need for refinement in certain aspects of the approach.
5.1 What was easy?
The great majority of the code was very well-written, clear, and well-documented,, iacilitating a smooth execution of the experiments. Also, for the HMM experiments, CSV files were provided which gave direct access to the author's reported results. Additionally, getting credentialed for the MIMIC-III dataset was swift process.
5.2 What was difficult?
There was large amount of files and code in the repository, much of which was unrelated the experiments in the original paper. This caused some difficulty in locating the relevant code. Additionally, did not find code to reproduce Figure 2 which could have helped to explain why the presented figure differs so much from our results. Furthermore, the original paper did not explicitly specify hyperparameters, lcaving us to believe that the source code's default values were used. However, in almost all experiments, these values did not yield results consistent with the reported ones. Despite experimenting with different hyperparameters, we could not achieve results more closcly aligned with the reported results.
5.3 Contact with the authors
We docmed it unnecessary to establish direct communicatinn with the original author.
10

---

Published in Transactions on Machine Learning Research (06/2024)
References
Alejandro Barredo Arricta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Al- berto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Pusion, 58:82 115, 2020. ISSN 1566-2535. doi: https:
/ (/dbi.org/10.1016/// httf:///...01./......oooooo wwffus..019.12.... URL $1566253519309103
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference Knowledge Discovery and Data Mining, KDD '15, pp. 1721-1730, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450336642. doi: 10. 10.1145/2783258 2788611.. URL
2788613
Travers Ching, Daniel S. Himmelstein, Brett K. Beaulieu-Jones, Alexandr A. Kalinin, Brian T. Do, Gre- gory P. Way, Enrico Ferrero, Paul-Michael Agapow, Michael Zietz, Michael M. Hoffman, Wei Xie, Gail L. Rosen, Benjamin Lengerich, Johnny Israeli, Jack Lanchantin, Stephen Woloszynek, Anne E. Carpen- ter, Avanti Shrikumar, Jinbo Xu, Evan M. Cofer, Christopher Lavender, Srinivas C. Turaga, Amr M. Alexandari, Zhiyong Lu, David J. Harris, Dave DeCaprio, Yanjun Qi, Anshul Kundaje, Yifan Peng, Laura K. Wiley, Marwin H.S. Segler, Simina M. Boca, S. Joshua Swamidass, Austin Huang, Anthony Git- ter, and Casey S. Greene. Opportunities and obstacles for deep learning in biology and medicine. BioRriv, 2018. doi: 10.1101/127700 URL https://wwi www.:/wirxiv..omm www.xii.org//aoooeoeooooooo
KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. CoRR, 1259,, 2014. URL http:// arxiv. aariv.org//ate wrgg/bs//////////////////// 1259
Jonathan Crabbé and Mihaela Van Der Schaar. Explaining time series predictions with dynamic masks. International Conference on Machine Learning, pp. 2166-2177. PMLR, 2021.
J. Enguehard. time interpret. MIT-1-ov-fin file 2023a.
interpret?tab http:///tttaaacoooooooogggaaaaa. aotp://elaaad.mmmeeeeetteeetaaaa
Joseph Enguchard. Learning perturbations to explain time series predictions. In International Conference on Machine Learning, pp. 9329 9342. PMLR, 2023b.
Ruth C. Fong and Andrea, Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3449 3457, 2017. doi: 10.1109/ ICCV.201 2017.777.
Etienne Goffinet, Mustapha Lebbah, Hanane Azzag, Giraldi Loic, and Anthony Coutant. Functional non- parametric latent block model: multivariate time series clustering approach for autonomous driving
validation. Computational Statistics & Data Analysis, 176:107565, 2022. ISSN 0167-9473. doi: https:
2020 // URL http:p///ww....aaaaaacccc.oo/oor
50167947322001451
Dr. Ing. h.c F. Porsche AG. porsche.com international fuel-consumption. URL. https::/www.ppntt
com
Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. Mimic-iii, a freely accessible critical care database. Scientific Data, 3(1):160035, May 2016. ISSN 2052-4463, doi: 10.1038/sdata.2011 35. URL
1038/ wtt.//.....////aaaaaaooooo.. 2016.
Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. Quantifying the carbon emissions of machine learning. arXiv preprint Xi11:1910.0000 0.00000 2019.
11

---

Published in Transactions on Machine Learning Research (06/2024)
Zichuan Liu, Yingying Zhang, Tianchun Wang, Zefan Wang, Dongsheng Luo, Mengnan Du, Min Wu,
Yi Wang, Chunlin Chen, Lunting Fan, and Qingsong Wen. Explaining time series via contrastive and locally sparse perturbations, 2024.
Carbon Footprint Ltd.
Carbonfootprint.com home of carbon footprinting. URL https://wwee
carbonfootprint.ccmm
Karim El Mokhtari, Ben Peachey Higdon, and Ayşe Başar. Interpreting financial time series with shap values. CASCON '19, pp. 166-172, USA, 2019. IBM Corp.
Pranav Rajpurkar, Awni Y. Hanmun, Masoumeh Haghpanahi, Codie Bourn, and Andrew Y. Ng. Cardiologist- level arrhythmia detection with convolutional neural networks. CoRR, abs/1707.01836 2017. URL http:
//arxiv. www.xii.rrg//bba.e wttg///////////////////////////t 01836
Thomas Rojat, Raphaël Puget, David Filliat, Javier Del Ser, Rodolphe Gelin, and Natalia Díaz-Rodriguex. Explainable artificial intelligence (xai) on timeseries data: A survey, 2021.
Iijaa Simić, Vedran Sabol, and Eduardo Veas. Xai methods for neural time series classification: A brief review. Xi preprint arXiv:2108. 000:02008008099, 2021.
Sana Tonekaboni, Shalmali Joshi, David Duvenaud, and Anna Goldenberg. What went wrong and when? instance-wise feature importance for time-series models. CoRR, abs/2003.02521, 2020. URL https:
//arxiv.org/abby wbwgg//gab///oo/ 2002112201120001
Jorg Wagner, Jan Mathias Kohler, Tobias Gindele, Leon Hetzel, Jakob Thaddaus Wiedemer, and Sven
Behnke. Interpretable and fine-grad grained visual explanations for convolutional neural networks. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
12

---

Published in Transactions on Machine Learning Research (06/2024)
A Additional explanations of the method
A.1 Schematic overview of training ExtrMask
Predictions without masking Teatures
Time series data
RNN Based classifier
Evaluation
Predictions masking most salient feature
series data
ExtrMask
Mask
Top-k
Figure 5: A flowchart overview the training and evaluation ExtrMask. The blue process represents the training of the classifier. For this classifier, ExtrMask will try to determine which features were important for its predictions. In the yellow process, these predictions are used in the objective function of either the preservation game (Equation 2 the deletion game (Fquation 4.2.1p. This objective function minimized in the ExtrMask block to derive perturbations that aid in determining improved masks, indicating the importance of each feature. As the ground truth saliency is not known, the black process involves comparing the original predictions with predictions obtained by masking a "Top-k' percentage of either the predicted most salient features or the predicted least salient features for each observation. These masks consist of zeros or the average value of the particular feature for that observation.
In figure schematic overview of the training and evaluation of ExtrMask is depicted. The training, the minimization of Equation 2 for the preservation game or Equation g for the deletion game, happens inside the ExtrMask block. The black process only applies to data for which the ground truth salieney is not known. If this information is available, the black process consists of comparing the ground truth saliency with the learned masks.
A.2 General information about the datasets
Dataset
Train Samples
Test Samples
Time Series Length
URL
Bimary classificution
step
Link
Binary classification end time period
MIMIC-III
80%
20%
Link
Binary classification at end of time period
Woather data
800
200
48
Link
Table 10: The datasets used for empirical evaluations across two tasks.
Table 10 presents general information about the thrce datasets used. In each dataset, 20% is allocated for testing, while the remaining 80% is utilized for validatiio with 5 folds. The reported metrics include the mean and standard deviations of the performance across these 5 folds the test set..
13

---

Published in Transactions on Machine Learning Research (06/2024)
A.2.1 Hidden Markov Model
The HMM specified as follows. The hidden state 8, can be either 0 or 1. For each sample, we generate 200 states (1 E [1, 200)), where the hidden stales are generated based on an initial state distributiin 0..5 and the following transition matrix:
0.1 0.9
C 0.1 0.9
At each time step, given hidden state, the imput vector x comprises three features sampled from (us, Es.). Where pi (0.1, 1.6, 0.5) and μi (-0.1, 0.4, -1.5) and the following covariance matrices:
0.8
0.8
0.01
0
Eg
0
0.8
0.01
0.01
0.8
0
0
0.01
0.8
0
0
0.8
Now the target variable is sampled from Bernoulli distribution:
Bernoulli ((1 exp(x2,))) if 8:
yt ~
Bernoulli ((1 exp(23,)))
if s
This means that 8; 0 the only salient feature is feature and when 8t the only salient feature is feature 3. Furthermore, it be seen that feature 1 is never important.
A.2.2 MIMIC-III
The MIMIC-I1I clinical database (version 1.4) was used for conducting this research. This dataset is a real- world dataset consisting vital signs and lab test results of patients in an intensive-care unit. Due to the sensitive nature of the information in the dataset, not publicly accessible. For research purposes, access can be requested.
Since we are dealing with private, sensitive information, are unable to provide feature statistics. A total of 31 features were employed in predicting in-hospital mortality for a given patient. Hourly data was used over a period of 48 hours. Missing values are imputed using previously available oncs. When no previous data was available for imputing, the values were imputed using a standard value.
To give the reader some context about the distribution of the target variable of the dataset, we will give some rough estimates. The dataset contains information from roughly 10.000 patients. The classifier network aims to predict in in-hospital mortality based on 31 features. Features measured hourly over 48-hour period. Approximately 15% of patients within the dataset experienced mortality during their hospitalization period.
A.2.3 Weather data
The meteorological data is sourced from Dutch weather stations and is publicly available through the Royal Dutch Meteorological Institute, which falls under the Ministry of Infrastructure and Water Management.
For the purpose of our investigation, data from 2020 to 2023 was downloaded Within this dataset, we selected the following Len features.
DDVEC: The average wind direction in degrees (360=north, 90=cast, 180=south, 270=west, O=calm/variable))
FG: The daily average wind speed (in m/s)
TG: The daily average temperature (in 0.1 degree Celsius)
SQ: The daily sunshine duration (in 0.1 hours) calculated from the global radiation (-1 for < 0.05 hours)
14

---

Published in Transactions on Machine Learning Research (06/2024)
DR: The daily rain duration (in 0.1 hours)
RH: The daily sum of precipitation (in 0.1 mm) (-1 for < 0.05 mm)
PG: The daily average air pressure reduced to sea level (in 0.1 hPa)
VVN: The daily minimum visibility occurred; 0: <100 m, 1:100-200 m, 2:200-300 m..., 49:4900-5000 m, 50:5-6 km, 56:6-7 km, 57:7-8 km,... 79:29-30 km, 80:30-35 km, 81:35-40 km,.., 89: >70 km
NG: The daily average cloud cover (coverage of the upper sky in eighths, 9-upper sky invisible)
UG: The daily average relative humidity (in percent)
Within the datasct, there was daily data from 50 different weather stations. All weather stations with missing data for one of the fealures were excluded from our research, leaving us with the data of 14 weather stations. To construct our usable dataset, we employed a sampling approach. We randomly selected a weather station, followed by the random selection of starting date within the station's records. Next, data spanning the selected day and the subsequent 49 were collected to form single data point. This sampling procedure was iterated until 1000 data points were obtained. The final step in preprocessing was to remove the final observation within each data point and use this to create a binary target variable indicating the occurrence of rainfall. The goal of the classifier therefore became predicting the occurrence of rainfall the following day, using the features from the 48 days prior.
In our sampled dataset, the target variable indicated rainfall 51.5% of the time. there were no missing values. It should be noted that the utilization of this dataset was primarily motivated by the need to evaluate the robustness of the ExtrMask perturbation method. This is done using a dataset that may be deemed somewhat unconventional.
B Additional results
B.1 HMM results with 20 time steps
As mentioned previously, we suspect that the author accidentally reported his results in his original paper with 20 time steps. In one of the unit tests, saw that much smaller dataset with only 20 time steps is generated which could explain why the reported information and entropy metrics much lower than the ones we obtain. To support this suspicion we also ran the experiment with 20 time steps.
Method
AUP
AAUP
AUR ft
AAUR
If
A1
A8
DeepLift
0,900 (0.007)
-2.2%
0.408
(0.002)
2.9%
365.0 (3.7)
1.8%
145.5 (1.8)
0.2%
DynaMask
(0.022)
-48.8%
0.788 (0.008)
3.3%
(18.7)
41.7%
124.6
(4:2)
175.1%
Fir
0.458 (0.014)
15.9%
0.584 (0.026)
6.4%
424.1 (20.3)
-2.8%
168.0
(3.3)
2.7%
Gradiot Shop
(0.013)
-5.3%
0.423 (0.002)
2.2%6
339.7 (10.1)
1.3%
138.1
(1.9)
0.45
IG
0.897 (0.007)
-2.2%
01469 (0.002)
3.3%
368.2 (4.3)
2.5%
145.0
(1.6)
-0.4%
Lime
(0.002)
353.3 (3.2)
1.8%
Occlusion
0.872 (0.008)
0.6%
0.117 (0.008)
6.4%
357.8 (6.5)
4.9%
155.9 (1.4)
1.7%
Ooclusion
358.2 (6.1)
3.95
Rotain
0.730 (0.061)
13.2%
0.382 (0.028)
14.4%
253.7 (23.9)
23.2%%
145.3
(7.3)
4.85
ExuMaack MSE
0.904 (0.015)
2.1%
0.773 (0.006)
-0.9%
1574.6 (35.7)
2.5%
$3.9 (1.2)
-0.3%
Table 11: Comparison of HMM results. The columns with a A describe the difference between our results
compared to the results in Enguehard (2023b). positive implies our result was higher, and negative that our result was lower.
Table shows that, with the smaller dataset with only 20 time steps, all methods, except DynaMask, yield results comparable to those reported in Enguehard (023b)). Furthermore, when comparing the results of our ablation study (Table 12) with those presented in the paper (Table 13). we observe mostly similar results, except for the combination ê1 1 and X 0.1. This suggests that the author did, indeed, use a dataset with only 20 time steps, as opposed to the 200 time steps reported in the original paper.
15

---

Published in Transactions on Machine Learning Research (06/2024)
0.01
0.1
0.01
0.1
10
0.01
0.500--0.22
0.789-0.500
0.617-0.112
0.355-0.178
0.355-0.154
0.01
0.509-0.815
0,755-0.441
0.752-0.050
0.355-0.177
0,386 -.16--0.155
0.1
0.510 0.525
0.691
0.657
0.634 0.107
0.539 0.178
0.386 0.189
0.1
0.512 0.912
0.646 0.626
0.354 0.151
0.316 0.150
0.368 0.202
0.210-0.000 0.505
0.688 0.861
0,905 0,774
0.358 0.200
0,352 0.157
A2
1
0.515 0.803
0.030 0.881
0,822 0.745
01303 0.164
0,350 00.17
0.506-0.500
0.656-0.644
0,906-0.776
0,858-0.30
0.3000
0.410-0.188
10
0.479-0.800
0.653-0.229
0.892
0,900
0.414-0.186
0.501
0,900 0.776
0.409 0.166
100
0.187 0.595
0.571 0.374
Table 12: Our ablation study with 20 time steps.
Table 13: Ablation study from the original paper.
B.2 Cross-Entropy vs Mean Squared Error
In the original paper, the author did not specify the loss functions used for L in equations 2 and 3 For the HMM experiment, the default in the code is the unintuitive choice of MSE, considering is a binary classification problem. For MIMIC-III, the default is CE. Because of this confusion, we will compare the results for both loss functions on both datasets with the results reported in the original paper.
AUP
AUR
Original paper
0.885 (0.930)
0.781 (0.012)
1,536 (79.0)
31 (3,70)
CE
0.916 (0.013)
0.657 (0.011)
15,0620
(6.690)
15,850 (318)
MSE
0:914 (U.012)
0.762 (0.006)
296,840 (6.167)
6,946 (1866
Results on ExtrMask and of change in loss function tested on the HMM dataset. Only our best results were emphasized by making them bolt. Mcan and std are reported over 5 folds.
Table 14:
In table the the results for Extrmask of the original paper are shown and below are our results when applying the CE or the MSE loss. It should be noted that our results cannot really be compared with the results of the original paper. The reason for this the empirical evidence shown previously that the author likely accidentally ran his experiments a dataset with a time period of 20 instead of 200. Results with the MSE loss function do somewhat align more closely with the reported results in the original paper. This could suggest that the author might have employed this loss function, however, the likely accident that the author made makes us not want to draw this conclusion. Interestingly, the CE loss function performs worse than the MSE loss function. This is surprising as in binary classification problems, the cross-cntropy loss typically is the default choice.
Perturbation
Acc 4
Comp ft
CE
Suff
Original paper
0.981 (0.004)
0.015 (0.004)
(0.008)
-0.012 (0.004)
Average
CE
0.968 (0.005)
0.022 (0.006)
0.159 (0.011)
.0.004 (0.003;
MSE
0.978 (0.005)
0.005 (0.001)
0.130 (0.008)
0.010 (0.004)
Original paper
0.943 (0.008)
0.109 (0.025)
0.315 (0.057)
.0.099 (0.006;
Zaro
CR
0.873 (0.037)
0.138 (0.031)
0.463 (0.067)
-0.069 (0.005)
MSE
(0.016)
0.074 (0.021) 0.323 (0.026)
0.008 (0.012)
Table 15: Results on ExtrMask and change in loss function tested on the MIMICIII dataset. In this experiment, 20% of data is masked. The loss which comes closest to the original paper is highlighter in bolt. Both lambdas are set to 1. Mean and std are reported over 5 folds.
In table 15
we are able to compare our results to that of the original paper as we have not found any big differences in results or flaws in the original execution. In the table, the numbers highlighted in bolt are closest to that of the original paper. In total the CE loss is closest four times and the MSE loss also closest four times. Again it can therefore from this table alone not be concluded which loss the author used.
From table 15 is that the CE loss gives a significant jump in performance, as it outperforms MSE in all metrics. In our research we have not explored this difference in performance any further, however, this could be interesting for further research.
can not deduce which loss the author used, however, something that can be deduced
Lastly, we also compared feature performance using the MSE loss and the CE loss. In figure this comparison can be seen. From this figure, we can with some certainty deduct that the author likely used the MSE loss. We believe this because the scale of the attribution axis only matches the MSE loss. Our figure for the MSF
16

---

Published in Transactions on Machine Learning Research (06/2024)
642
1.991
6.0
1009
(a) Feature attribution original paper
(b) MSE loss function for L.
(c) CE loss function for L.
Figure 6: The error bars denote standard errors for the mean (over patients and time) feature attribution over 5 folds. A high attribution implics that the ExtrMask method says the feature was important for the prediction of the binary classifier.
loss does not completely match that of the original paper, however, our MSE results are relatively close and much closer than that of the CE loss figure. These findings suggest that the author likely used the MSE loss function.
B.3 Additional results MIMIC-III
As in the original paper, we will now present the results for comprehenssvenees (Figure) 7 accuracy (Figure 8 and sufficiency ( when mask between 10% and 60% of the data for each patient. We show this for various methods where ExtrMask refers to the preservation game with the CE loss function, ExtrMask MSE refers to the preservation game with the MSE loss function, and ExtrMask MSE Del refers to the revised deletion game (see Section 4.2.1 with the MSE loss function.
For all metrics, across all percentages masked, we observe that the ExtrMask method with the CE loss function perforrsmms all other methods. The values this method obtains are, discussed in the previous section (Appendix B.2), way better than the ones reported in the original paper. However, all baselines also obtain better values than reported in the paper.
DespLitt
DeesLit
6.53
Ext Mask
Dot Mark MCE Del
Ext Muak MSE
Ocdus
Occlusto on
DynaHask
Compehunneerrr
0.15
DynaWark
Octus
Retain
Aug Cecluiio
Aug
0.03
0.30
-0.23
0.00
0.1
0.2
0.5
0.6
63
0.2
0.5
0.3
(a) The masked data is replaced with the lime average of that feature for each patient.
(b) The masked data is replaced with zeros.
Figure 7: Comparison of comprehensiveness across methods by masking 10% 60% of the data, higher is better.
17

---

Published in Transactions on Machine Learning Research (06/2024)
Deeti in
DeesLit
0.15
Exp Mock Ocl
Eee Maak
0.90
Occlaeon
Dynakork
0.52
DynaHask
Oechus
0.955
0.00
Avp Deciusion
Retah
0.470
0.99
0.54
0.40
Maxoed
Muskel
(a) By masking between 10% and 60% of the data for each patient and replacing the masked data with the average value for that feature over time for each patient.
(b) By masking between 10% and 60% of the data for each observation and replacing the masked data with zeros.
Figure 8: Comparison of accuracy across methods by masking 10% 60% of the data, lower better.
Douplit
Deeplit
Dantase
ExiMtest MDT Del
Eartask MGE
Cecuabor
0.01
DyraNast
0.000
Retain
Aug
0.02
0.00
Aug Oodia lon
Decuslor
Retain
2.08
-0 02
0.50
-3.055
99.100
0.1
0.2
0.3
0.4
0.5
0.6
5.3
02
0.4
0.5
0.6
Manked
(a) By masking between 10% and 60% of the data for each patient and replacing the masked data with the average value for that feature over time for each patient.
(b) By masking between 10% and 60% of the data for each observation and replacing the masked data with zeros.
Figure 9: Comparison of sufficiency across methods by masking 10% 60% of the data lower is better.
B.4 Additional results weather data
As for the MIMIC-III dataset, we will now present the results for cross-entropy (Figure 10), comprehensive- ness (Figure I1], accuracy (Figure 12). and sufficiency (Figure [3) when mask between 10% and 60% of the data for each patient. We show this for various methods where ExtrMask refers to the preservation game with the CE loss function
We observe that when masking more than 30%, DynaMask exhibits better metrics than ExtrMask. A plausible explanation could be that ExtrMask is better detecting the most important features but struggles identify the lesser yet still relevant features. However, it is important note that these results are specific to this dataset and do not hold for the MIMIC-III dataset the synthetic HMM dataset.
18

---

Published in Transactions on Machine Learning Research (06/2024)
Deepl
Deepl in
12
GradicmsShap
Occlusion
ExtMask
122
Occlusion
Extrbakk
@nddentiig
Aug oxcusion
Aug Occlusion
1.0
DynaMask
Dyratlak
Balain
Pelam
0.8
0.8
0.4
Macked
Massed
(a) By masking between 10% and 60% of the data each observation and replacing the masked data with the aver- age value for that feature over time for each observation.
(b) By masking between 10% and 60% of the data for each observation and replacing the masked data with zeros.
Figure 10: Comparison of cross-entropy for different methods, higher is better.
Deeplit
Extntack
Eximtakk
Graderthhp
Occudin
0.4
GradientSiap
Dcclusion
Aug Occlusion
DynaMask
0.25
0.3
Comprereeeeeee
Rebrin
0.29
0.15
0.1
(b) By masking between 10% and 60% of the data for each observation and replacing the masked data with the observation and replacing the masked data with zeros.
(a) By masking between 10% and 60% of the data for each
age value for that feature over time for each observation.
Figure 11: Comparison of comprehensiveness for different methods, higher is better.
0.9
Deepiit
0.
Despuit
ExtrMask
ExtiNaak
Gradientshup
0.8
Gradenttine
Grilaton
Aug Occusio
0.7
Aug Oeclusion
DymsMark
DynaWask
0.7
Retain
Rebain
0.6
033
0.4
0.
6.1
0.1
0.2
0.3
0.4
0.5
0.5
0.1
0.2
0.3
0.4
0.5
0.6
Munked
Maxiced
(a) By masking between 10% and 60% of the data for each observation and replacing the masked data with aver- age value for that feature time for each observation.
(b) By masking between 10% and 60% of the data for each observation and replacing the masked data with zeros.
Figure 12: Comparison of accuracy for different methods, lower is better.
19

---

Published in Transactions on Machine Learning Research (06/2024)
Deeplat
0.25
Despurt ExtMask
GrazlentShap
0.20
Godershhpp
0.10
Aug
0:35
Aug
Suncweyy
0.05
Sufficnnz
0.00
-0.55
-0.30
-0.35
0.3
0.2
0.3
0.4
6.5
0.6
0.1
6.2
0.3
0.4
0.5
0.6
% Masked
% Masked
(a) By masking between 10% and 60% of the data for each observation and replacing the masked data with the aver- age value for that feature over time for each observation.
(b) By masking between 10% and 60% of the data for each observation and replacing the masked data with zeros.
Figure 13: Comparison of sulficiency for different methods, lower is better.
B.5 Visualizations of the learned perturbations and masks
Visualizations of the explanations for each method in Figure 15 reinforce the conclusions drawn from the numeric results, showing that ExtrMask performs much better than any of the other methods this data set. DynaMask performs really poorly on this data set, and we could not find the reason for this, even though we tried executing it with a wide variety of hyperparaneters. All other methods seem to work surprisingly similar - they all capture the information that one feature is never salient, and they vaguely follow the true saliency, but have a very high variation between successive time steps and can not point out the single important feature with high accuracy.
Visualizations of the perturbations for the MIMIC dataset show similar patterns what we have seen the HMM dataset- the learned perturbation is 0 for the preservation game and has meaningful difference compared to the original signal for the deletion game at the time steps where saliency is high.
C Suggestions for further research
C.1 Possible improvements for the loss functions of ExtrMask
The preservation game's objective function given by:
arg min X Ai|[m||ı + \2|[NN(x) |1 ((((()))
m_e
Here, the problematic term is ||N[()]]]- The purpose of this term is to punish the perturbation network when it outputs something close to the original input. However, the term currently punishes outputs that are far from 0, so when the original input is 0 and the neural network recovers it, there is no loss incurred. A theoretically better term would be max(r - [[NN(x) - x|ş, a), where T, a, and p are hyperparameters that determine the maximum, minimum, and shape of the loss, respectively. However, because of computational constraints, we were not able to conduct an elaborate hyperparameter search and have not found settings that come close to the performance obtained with the originally proposed loss function.
The revised loss function for the deletion game is given by:
arg min A || - m|]1 + \2 - x12 CL(()), m))) m,NENN
For the deletion game, the primary concern lies in the last term, (()()), m))). This term should penalize perturbations that do not change the output of f, the model to be explained. However, when the original input was 0 and the perturbation network learns to recover this, the output will not change and there will no loss be incurred, Here, a theoretically better term would be max(r -((((x). f(ø(x, m))), a). Also here we were limited by computational resources.
20

---

Published in Transactions on Machine Learning Research (06/2024)
True saliency
Feature
Explanation saliency DeepLift
Feature
Explanation saliency DynaMask
Feature
0.8
Explanation saliency ExtrMask (preservation)
Feature
Explanation saliency ExtrMask (deletion, ours)
Feature
0.6
Saliency (mask) values
Explanation saliency GradientShap
Feature
Explanation saliency IG
Feature
0.4
Explanation sallency LIME
Feature
Explanation saliency Aug Occ
0.2
Feature
Explanation saliency Occlusion
Feature
Explanation saliency Retain
Feature
Is classifier prediction correct
yes
175
Time
Figure 14: ExtrMask method, deletion mode (ours)
Figure 15: True saliency (first plot) and explained saliency on the first sample of the HMM dataset for all methods. The last subplot shows whether the classifier correct in predicting the output not. This figure is expanded version of Figure 3
21

---

Published in Transactions on Machine Learning Research (06/2024)
Orgin syull b
Orghelagnee- featrr
Doginal f
Organe 221w Intere?
Drighil sign Narurr
Or sgnd festre
nikt Sayal
hetatedd cipst
Intubalioge
Notubee appul
Inturaaligss
Wenvtee sipul
Petere atphal ard
Offernce stgina
Drteerte orghe pertrtedd
à Piaaace atginal perzatdd
Unheere orgge
Difearnc of anihha
Digiaal
cedgeriatel Appll
pritelusll sandl
seperated sisd
ar centated sopell
Selercy
Salancy
Seargy
Salancy
Searg
Salircy
Orghulaage 22
O5g sgrll Nettor
Oriine Jgge
Oigind Aa
Orghelagna Jelore -alles
15
Oégiaal dy -hattrr 37
Petuteel larel
briatedspp
Peturbst
Nriatei sipul
Potureed
Ieiuated sipul
D'ttures onginal
Crfessa
Eiteeras erighnl
Ofternnc atginal
Irteers ergen
Ditiancss of anighaal
5ikro
Schoxy
1.-
62-
Dognd upst belac
Digadd
Orgeaiiged Inderr
Depad Appl
Petuited ugral
hotabed upsi
Faturacagge
Peruuted sigul
D'vessse csstnd cndperatcl
Otbaesccl stgaa sottteeeess
Blean custel
Diaanse atynn
Noovese crgea
Blesonc aleeenu
Salercy
salency
Sallrey
Orghnliagga farun
Origial
Orginil 21
Orgrainggal Nastrr
Dégiaa dyral-
Peturbod
ard pa tutel lipal
sed senubbe gaM
paruradd
and penateee Gynal
patuisdd liga
als pensttd ogal
Salercy
Salence
tykeoy
Scleee
Spleey
Soborey
Figure 16: Visualization of the learned perturbations using the ExtrMask method deletion game (ours) on the MTVIC dataset. This is similar to what is reported for the HMM dataset in Figure 4
22


Reviews:
[{"review_id": "jLqiteD8J1", "reviewer": "Reviewer_6CZD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n- Regarding the original paper on the deletion game, I believe that modifying the loss function due to its poor performance will greatly benefit those who intended to use the ExtrMask method as it is.\n\n- The paper is well-written and structured in a way that even those with limited background knowledge can easily follow along.\n\n## Weaknesses\n\n- While the main text is well-composed, the abstract does not seem to summarize the content of the paper effectively and contains parts that feel awkward to read. For example, the phrase \"In this work\" is used twice, and both the second and fourth sentences lack substantial information.\n\n- The description of Table 4 is insufficient. Specifically, it is unclear what the \"corrected\" version refers to. It seems that there is an explanation at the bottom of page 6, but it appears to be inadequate.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- Mainly, they conduct the experiments detailed in \"Learning Perturbations to Explain Time Series Predictions\" to verify the accuracy of the claims made.\n\n- They enhance the performance of the deletion game by modifying the loss function.\n\n- They extended their examination of the ExtrMask method's superior performance on time series data to assess its robustness using a weather dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It would be beneficial to revise the Abstract to make it read more naturally.\n\n- It would be beneficial to provide additional explanations related to Table 4, especially details about \"an issue with the implementation of the perturbation formula for the deletion game\"", "broader_impact_concerns": "There is no concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pfAFJZHG7d", "reviewer": "Reviewer_kdaA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The authors, for the most part, provide a comprehensive overview on preservation and deletion methods. \n2. The authors comprehensively reproduce the experiments of the original paper and point out some discrepancies and why they might be occurring\n3. The authors identified an issue with the loss function for the deletion game and its implementation in the code. They addressed these issues and their proposed deletion game is now competitive with the preservation when the MSE loss function.\n\nWeaknesses:\n1. I believe the objective function for the deletion method in Eq 3 is not necessarily correct and needs to be explained better. Why is f(0) deemed as uninformative? Is it assumed that f(0) = 0? Suppose f is a binary classifier so that f maps vectors x to the interval [0,1]. Then if we have a new sample x for which f(x) = 0, then minimizing L(f(0), f(x*m + (1-m)*NN(x))) does not make sense. In the example I just gave, shouldn't the goal be to instead minimize L(1, f(x*m + (1-m)*NN(x))), i.e., find the features in x that when masked, flip/ruin the original prediction of f(x) = 0? If this could be clarified I would appreciate it!\n2. Could the authors provide a more in depth explanation as to why Eq 4 (their proposed method) is better? In particular, the sentence \"The intuition is that every feature with small importance can be seen as important when its value is significantly perturbed, however, only important features will be seen as important when the input is slightly perturbed\" does not really make sense to me and I would appreciate a more in-depth explanation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work focuses on reproducing and extending the work of Enguehard (2023b), which uses preservation and deletion explanation methods for time series data. The authors \n\n1. Reproduce the reported results of Enguehard (2023b).\n\n2. Extend the work by revising the deletion game's loss function. They test the robustness of their proposed method on a new dataset and find that the proposed method indeed outperforms all baselines and is robust. Furthermore, in many experiments, the authors obtain notably different results. The author's revised deletion game shows promise, recovering meaningful perturbations and, in some cases, performs better than the preservation game.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As mentioned in the weaknesses, if the authors could address the following points\n\n1. Clarify my questions on the original deletion game objective function.\n2. Provide a more in depth explanation as to why Eq. 4 is better.\n\nThat would be great!", "broader_impact_concerns": "There are no concerns on the ethical implications of this work that would require adding a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "iMFoineMjS", "reviewer": "Reviewer_eFYk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper is enjoyable to read: very clearly written with plenty of illustrations\n2. The empirical results are complete and cover an additional dataset\n3. The authors propose an improvement to the original paper to further boost the results\n\nOverall, this is a great initiative and I would like to warmheartedly thank the authors for sharing their valuable findings with the research community. \n\n**Weaknesses**\n\n1. It is not clear whether what is the final conclusion of the comparison in terms of the masking strategy\nWhile comparing the plots of Figure 1, I noticed that they are different from Figures 3 and 4 in the original paper. The authors mentioned that indeed Figure 3 is different from Figure 1 (left), yet they do not comment on it further. Is this because this particular setup (masking with the average) is never used? Or does it mean that Enguehard (2023) chose this setup to have a more favorable setup for the comparison? It seems that in the original paper, both masking strategies led to the same results with ExtrMask being more efficient.\n\n2. It seems rather odd that ExtrMask outputs zero perturbations, yet it still provides better results than DynaMask. Is there any reason for this? Does it mean that learnable masks are more important than learnable perturbations?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a reproducibility study of a recent paper \"Learning Perturbations to Explain Time Series Predictions\" by Joseph Enguehard published at ICML’23. The main goal of the study is to verify to which extent the presented results of the latter paper can be reproduced. The authors do a good job at presenting the overall context of this study as well as a high-level introduction to the task of explainable time-series predictions. The authors discover that the results of Enguehard (2023) are reproducible to a large extent but with several important caveats. The study further evaluates this work on a different dataset and proposes an improvement to it that boosts the results obtained by Joseph Enguehard even further.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Minor:\n1. Some links at the end of Section 3.2 are broken (Table and Appendix)\n2. I would be grateful if the authors could reply to my comments on the weaknesses part", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree that the presentation is balanced, interesting, and contributes to the better understanding of the method in Enguehard (2023b), as well as proposing new modifications and improvements. The authors have successfully addressed the reviewers comments, and there are no other remaining questions.
Decision: Accept (poster)