FreqRISE: Explaining time series using frequency masking

Venue: NLDL_org_2025_Conference
Published: 2024-08-27 16:40:46

Abstract: Time series data is fundamentally important for many critical domains such as healthcare, finance, and climate, where explainable models are necessary for safe automated decision-making. To develop explainable artificial intelligence in these domains therefore implies explaining salient information in the time series. Current methods for obtaining saliency maps assumes localized information in the raw input space. In this paper, we argue that the salient information of a number of time series is more likely to be localized in the frequency domain. We propose FreqRISE, which uses masking based methods to produce explanations in the frequency and time-frequency domain, and outperforms strong baselines across a number of tasks.

Main paper: 
 masking based explanations: 
 Here, we present a new method for transforming masks between domains (Section 2.1) and use this𝑔 𝑔 !" 𝑓 … … … 𝑿 # 𝑿 $ 𝑴 # 𝑿 𝑺 ⊙ 𝑴 # 𝑿 % 0.34 0.97 0.42 𝑹 % = 1 𝑁 ⋅ 𝔼[𝑴] . 𝒚 0 & 𝑴 & ' &() … 𝒚 0 ⊙ Figure 1.The FreqRISE framework shown when, as an example, using the Short-Time Discrete Fourier Transform (STDFT) as the invertible mapping, g. g maps X T into the domain of interest, g : X T → X S . Here, we sample N masks and produce N masked versions of X S . Using the inverse of g, we map back to the time domain, obtain ŷ, and compute the relevance map as a weighted average over the masks.new method to create FreqRISE (Section 2.2), the, to our knowledge, first masking-based XAI method operating in the frequency domain of time series.While many time series models are trained directly on the raw time series data, the time domain is often insufficient for providing complete explanations [11] .This is due to most XAI methods being built on the assumption that the explanations are localized and sparse. However, if two classes are characterized by e.g. their frequency content, this information is neither localized nor sparse in the time domain.We, therefore, propose transforming the signals to a domain where the information is assumed to be localized and sparse and to provide the explanations in this domain.Our aim is to explain the black box model f (•) that outputs the class probabilities y ∈ R C from a time series input X ∈ R V ×T , where V is the number of input variables and T is the length of the time series. Masking-based methods, such as RISE [9] and RELAX [18] , use masks M ∈ [0, 1] V ×T , to mask out features in the input space through elementwise multiplication, X = X ⊙ M and observe the resulting changes in the model output:EQUATION)The result is therefore a relevance map over the input features in the input domain.Instead, assume an invertible mapping from the time domain, T , into the domain of interest, S, g : X T → X S . We can formulate an alternative masking-based approach, where the masks are applied in the new domain after which the input is transformed back into the time domain. One example of such mapping is the discrete Fourier transform (DFT), which maps the signal into the frequency domain, X S ∈ C V ×F . We can then apply masks M S ∈ [0, 1] V ×F in the frequency domain and obtain the masked input in the time domain via the inverse 133 mapping:134 X = g -1 g (X) ⊙ M S .(2) 135We can then use (1) to obtain the changes in the 136 model output resulting from the mask.In this paper, we focus on the frequency domain 138 and the time-frequency domain, obtained through 139 the DFT and the short-time DFT (STDFT). How-140 ever, the formulation can be extended to other in-141 vertible mappings. A prominent masking-based framework is RISE [9] . 144 RISE assumes that the masks M are sampled from a 145 distribution D. RISE then estimates the relevance, 146 R c,v,t , of class c for a point in input space, X v,t , 147 as the expected value of ŷc (obtained using (1)) 148 conditioned on M v,t = 1, i.e. that the point is 149 observed:150 R = E M [ŷ • M ].(3) 151For a high accuracy classifier, we expect ŷ to be high 152 when important features are not masked out. In 153 practice, we can estimate the relevance using Monte 154 Carlo sampling. We produce N masks and estimate 155 the expected value as a weighted sum, normalized 156 by the expectation over the masks:157 R = 1 N • E[M ] N i ŷi • M i .(4) 158In this work, we deal with univariate time series, i.e. 159 V = 1, however the methods can be extended to 160 multivariate cases.We combine the RISE framework with our pro-162 posed frequency and time-frequency masking and 163 call our method FreqRISE. The FreqRISE frame-164 work is shown in Figure 1 when using STDFT as 165 the invertible mapping.  
experimental setup 167: 
 We conduct experiments on two datasets: the syn-168 thetic dataset presented in [12] and AudioMNIST 169 [21] . Both datasets have been shown to have salient domain [12] . We use the same synthetic dataset as in [12] . The EQUATION)180We set all a j = 1, randomly sample the phase ψ j ∼ for each of the 60 speakers (12 female/48 male) [21] . We follow [21] and downsample all signals to 8kHz and zero-pad all windows to 8000 samples.We use the same 1D convolutional architecture as in [21] on the raw time series and train two versions: one trained for predicting the spoken digit and one trained to predict the gender of the speaker. We expect the salient information for the gender and digit task to be localized in the frequency and timefrequency domain, respectively.We use FreqRISE to compute relevance maps in the frequency and time-frequency domains and standard RISE in the time domain. We use 1000 data points from the gender and digit test sets respectively for testing the explanation methods. All relevance maps for AudioMNIST are estimated using N = 20, 000 samples. All masks are sampled from a Bernoulli distribution with p = 0.5 on a lower dimensional grid and linearly interpolated to create smooth masks. When computing the (Freq)RISE relevance maps, we use the logits prior to the softmax activation, since these show higher sensitivity to changes in the input.In the time-frequency domain, we use the onesided STDFT with a Hanning window of size 455 and an overlap of 420 samples between subsequent windows following [21] . In STDFT domain, the binary masks are sampled as grids of size 25 × 25.In the time and frequency domains, we use grids of size 200.As baselines, we use Integrated Gradients (IG) [22] and Layer-wise Relevance Propagation (LRP) [7] . When computing relevance maps in the frequency and time-frequency domains, we follow [12] and equip the models with virtual inspection layers, which map the input into the relevant domains. Due to relevance conservation, we are restricted to using rectangular, non-overlapping windows for the STDFT. We therefore use rectangular windows of size 455 with no overlap.Due to the lack of ground truth explanations [23] , quantitative evaluation for XAI is performed by measuring desirable properties [24] . Below we describe three such desirable properties that we use to quantify the quality of the relevance maps.Localization: Localization scores are widely used to evaluate if explanations are co-located with a region of interest [13, 18] . For the synthetic data, we know the ground truth explanations and can therefore use localization metrics to evaluate the methods. We use the relevance rank accuracy [25] . Assuming a ground truth mask, GT , of size K and a relevance map R, we take the K points with the Faithfulness: Faithfulness measures to what degree an explanation follows the predictive behaviour of the model and is a widely used measure for quantifying quality of explanations [12, 16] . We follow prior works [12, 13] and compute faithfulness as follows. Given a relevance map, R, we iteratively remove the 5%, 10%, . . . , 95% most important features by setting the signal value to 0. We then evaluate the model performance as the mean probability of the true class and plot it to produce deletion plots.Finally, we compute the area-under-the-curve (AUC) to produce our final faithfulness metric. A low AUC means that the explanation is faithful to the model.Complexity: Finally, we evaluate the complexity of the explanation as an estimate for the informativeness [26] . The complexity is estimated as the Shannon entropy of the relevance maps. 
synthetic data results: 
 The two models trained on the synthetic dataset both achieve an accuracy of 100%. Table 1 shows Figure 3 . AudioMNIST: Deletion plots for both tasks in frequency and time-frequency domains. We delete features according to the importance and measure model outputs. FreqRISE outperforms the baselines in both domains on the digit task and the frequency domain on the gender task.The model trained on the digit classification task 303 achieves and accuracy of 96.9%, while the model 304 trained on the gender tasks achieves an accuracy of 305 98.6%.Following the procedure described in Section 3.4, 307 we initially compute the faithfulness results. Fig- 308 ure 3 (top) shows the deletion plots for the digit clas-309 sification task in the frequency and time-frequency 310 domain. We have also included two additional base-311 lines, namely randomly deleting features (Rand.) 312 and deleting features based on their amplitude 313 (Amp.). In both domains, the mean true class prob-314 ability quickly drops. After dropping only 10% of 315 the features, FreqRISE has a mean true class prob-316 ability of 0.139 in the time-frequency domain and 317 0.248 in the frequency domain. After this, the value 318 continues to drop with features being removed. In Table 2 (top), the faithfulness scores are shown 329 for both models in all three domains. We notice that 330 in the time domain, RISE performs either worse 331 or equivalently compared to LRP. However, when 332 monics. The fundamental frequency is known to be discriminative of gender [27] .Figure 5 shows the relevance map computed using FreqRISE in the time-frequency domain for the digit task. The spoken and predicted digit is 9. The STDFT reveals that the signal starts around t = 0.3s, where the relevance map puts most of the relevance. However, around t = 0.5s, there are two distinct patterns emerging along the frequency axis. The fact that the method is allowed to distribute the relevance differently across frequencies depending on the time shows the benefit of having both the time and frequency component when computing relevance maps for the digit task. 
discussion and conclusion: 
 Time series data is inherently difficult to interpret, due to the complex patterns of the signals [10] . Therefore, providing relevance maps in a domain where information is sparser and more easily interpretable is desirable. We therefore proposed FreqRISE, which computes relevance maps in the frequency and time-frequency domains using modelagnostic masking methods. Vielhaben et al. [12] previously produced relevance maps in these domains using gradient based methods, which are modeldependent, limiting the usability in cases where model gradients are not available. RISE and FreqRISE consistently yield high complexity scores. While LRP and IG are able to assign zero relevance, this is generally not the case for Freq(RISE). This is, however, not necessarily indicative of the perceptual quality of the relevance maps. The entropy could e.g. be reduced through postprocessing of relevance maps.We used a synthetic dataset with known salient features in the frequency domain to compute localization scores. In the low noise setting, FreqRISE gave slightly higher accuracy in identifying the correct frequency components. However, when testing in the high noise setting, FreqRISE outperformed  
: 
 titude of domains [9, 13, 14] . Masking-based approaches evolve around learning or estimating the relevance maps by iteratively applying binary masks to the input and measuring the resulting change in the output. Especially optimization-based masking approaches have shown great promise for time series data [13, 15, 16] . In optimization-based masking approaches, the relevance is learned by posing an objective that maximizes the number of masked out features in the input, while simultaneously minimizing the change in the model output [17] . The mask is optimized via gradient descent through the model and the mask generating function. These methods can be difficult to use due to hyper-parameter tuning. Additionally, they suffer from the significant drawback of requiring access to the model gradients.Alternatively, model-agnostic methods require only access to the input and output of the network and are easily adaptable to a multitude of network architectures. Prominent model-agnostic methods are also based on masking and involve estimating the relevance via Monte Carlo sampling. This category includes RISE [9] and RELAX [18] . The sampling based methods have a few clear advantages: they do not require any complex optimization techniques, and we can directly constrain the sampling space to ensure compatibility with our data. While these methods have been successfully applied for time series [19, 20] , no previous work in this category has focused on providing relevances in another domain than the input domain.In this paper, we take inspiration from RISE and propose the first approach for masking-based XAI in the frequency domain of time series, FreqRISE. We:• Propose FreqRISE, the first masking-based approach for providing relevance maps in the frequency and time-frequency domain.• Provide a comprehensive evaluation of our proposed approach across two datasets and four tasks.• Show that masking-based relevance maps in the frequency and time-frequency domain outperform competing methods across several metrics. 
 
 Appendix: As with other masking-based approaches, computing the relevance maps for FreqRISE involves choosing a number of hyper-parameters, i.e. the size of the grid in which to sample binary masks, the probability p with which is bin is chosen, and the number of masks. As of now, there is no principled way to choose either and we have heuristically chosen the hyper-parameters based on qualitative assessment on a few validation samples. A more systematic approach, would be to tune the parameters on a validation set by choosing a few metrics, such as faithfulness and complexity.For AudioMNIST, FreqRISE needs a large number of masks to converge (N = 20, 000). We found that when using fewer masks, the method still finds the relevant bits, but sorting out the irrelevant bits of the signal requires more masks. Wickstrøm et al. [18] presented theoretical results on computing the number of masks and found that N = 3, 000 should yield results with a low error. It would be interesting to investigate convergence properties for our datasets in light of the theoretical results.Additionally, Mercier et al. [20] propose TimeREISE which uses a multiple mask sizes and sampling probabilities to compute the relevance maps for each time series. Finally, Cooper et al. [28] apply RISE to images and propose a hierarchical systematic mapping to reduce the number of masks. Both of these avenues could be interesting to explore when masking in the frequency domain.Additional relevance maps in all domains on the digit and gender task for AudioMNIST. shows that LRP seems to highlight most of the STDFT as important, seemingly ranked by the amplitude of the actual signal. This is not very useful for explanations. The time domain plots show that the time relevance of LRP seems to be centered around the middle of the signal. While for RISE, the time relevance is still scattered. In the frequency domain, the information seems more localized for both LRP and FreqRISE. In the STDFT domain, we see that FreqRISE struggles to identify relevant information, whereas LRP distributes relevance on the three lowest frequency components. This indicates that the frequency domain is sufficient for explaining the gender task.  


Decision: The paper presents a new way to do XAI in the frequency domain of models which classify time-series.

The idea is potentially novel but the presentation is significantly unclear.
The basic definition given in equation 3 is mathematically quite unclear,
- even if one tries to match this against the RISE paper, https://arxiv.org/pdf/1806.07421

Given the constraints of time it is quite challenging to ascertain the correctness with confidence,
but I tend to believe that with a significant overhaul of the writing maybe the ambiguities can be resolved.

Reviews: review: title: title: Interesting proposal, but lacking crucial information to establish its significance
paper_summary: paper_summary: The authors propose an explainable model to highlight the importance of frequencies in signals. By masking the time-frequency representation, the authors can output a relevance map of the frequencies by time, providing additional information compared to simply masking on either the time or frequency domains alone. 

The proposal is tested against two baselines with two datasets (one synthetic and one real) for three tasks.
main_review: main_review: Thank you to the authors for addressing my comments and engaging in an interesting exchange. I find the proposal compelling, and the results are promising. However, this version of the paper would benefit from additional comparative analysis and, above all, greater clarity to better establish the proposal's relevance. Additionally, the datasets used may not capture the full range of potential applications, making it challenging to generalize the performance findings presented.

Other reviewers also noted insufficient discussion of the results and a lack of guidance on how to use the proposal (especially select the different hyperparameters). Together, these points contribute to my recommendation to reject this paper in its current form.
strength_weakness: strength_weakness: The paper lacks details and misses important baselines. It needs rewriting to make the content clearer. Additionally, the story should be, in my opinion, revised from time series to signal as nothing guarantee that it can be applied to usual time series.
title: title: Promising concept but critical gaps in analysis and discussion
paper_summary: paper_summary: The authors introduce FreqRISE, an extension of the RISE explainability method applied to time series data by transforming it into the frequency and time-frequency domains. FreqRISE outperforms baseline methods in identifying relevant features, particularly in noisy environments and tasks where information is sparse. However, it consistently produces high complexity explanations, which may hinder interpretability and raises concerns about the correctness of the relevance maps, as they may include irrelevant information.
main_review: main_review: I recommend accepting this paper, as it introduces a valuable approach to XAI for time series in the frequency domain. FreqRISE offers advantages in noisy data environments, demonstrating potential in challenging applications. While high complexity in relevance maps remains an issue, especially in low-noise contexts, the preliminary post-processing solutions are encouraging steps towards improved interpretability. Additionally, while guidance for selecting between frequency and time-frequency domains is only preliminary, this paper does provide a good basis for further development. Overall, this work is a promising addition to time series explainability and lays groundwork for future research in frequency-based interpretability.
strength_weakness: strength_weakness: - **Insufficient discussion of performance results**
    - In the synthetic dataset’s low-noise case FreqRISE performs the same as baseline methods, yet there is little discussion of what this means. Similarly, the fact that FreqRISE significantly outperforms in high-noise settings but not in low-noise ones raises questions about the method's behaviour and its reliance on noise levels. The implications of these results need to be explored further.
    - In the AudioMNIST task FreqRISE performs better in the frequency domain, but not in the time-frequency domain. The paper does not provide sufficient explanation of why FreqRISE’s performance varies between these domains. The authors should address what this discrepancy implies for different types of tasks and whether FreqRISE is generally more suited to purely frequency-based data.
- **High complexity of explanations**: FreqRISE consistently produces explanations with high complexity scores. This complexity can make the relevance maps difficult to interpret, limiting their practical usefulness. The paper doesn’t adequately discuss the consequences of this high complexity. The authors should explore whether the added accuracy is worth the trade-off in interpretability and whether post-processing could reduce this complexity.
- **Unclear argument for superiority**: The high complexity and mixed results make it difficult to conclude that FreqRISE is a superior method to the baselines. The lack of a detailed discussion around these limitations weakens the paper's overall argument. Without addressing how FreqRISE’s complex explanations might reduce its practical value, the claim of superiority over baseline methods is questionable.
- **Minor issues**: There are minor issues such as typos (line 102, “build”; line 177, “allows to”) and insufficient detail in the figure captions (figs. 3, 4, 5). Standalone figure descriptions would improve the clarity of the results and help readers better understand the key findings.