Contimask: Explaining Irregular Time Series via Perturbations in Continuous Time
Authors: Max Moebus
Björn Braun
Christian Holz
Venue: NeurIPS 2025 poster ()
Keywords: Irregular Time Series
Explainability
Interpretability
Explanations
Perturbations
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: Contimask is the first post-hoc explainer for irregular time series models capable of explaining models that besides the value of observed data factor in missingness or time intensity.

Abstract: Explaining black-box models for time series data is critical for the wide-scale adoption of deep learning techniques across domains such as healthcare. Recently, explainability methods for deep time series models have seen significant progress by adopting saliency methods that perturb masked segments of time series to uncover their importance towards the prediction of black-box models. Thus far, such methods have been largely restricted to regular time series. Irregular time series, however, sampled at irregular time intervals and potentially with missing values, are the dominant form of time series in various critical domains (e.g., hospital records). In this paper, we conduct the first evaluation of saliency methods for the interpretation of irregular time series models. We first translate techniques for regular time series into the continuous time realm of irregular time series and show under which circumstances such techniques are still applicable. However, existing perturbation techniques neglect the timing and structure of observed data, e.g., informative missingness when data is not missing at random. Thus, we propose Contimask, a simple framework to also apply non-differentiable perturbations, such as simulating that parts of the data had not been observed using NeuroEvolution. Doing so, we successfully detect how structural differences in the data can bias irregular time series models on a real-world sepsis prediction task where 90% of the data is missing. Source code is available on GitHub.

Contimask: Explaining Irregular Time Series Models via Perturbations in Continuous Time
Max Moebus, Björn Braun, and Christian Holz
Department of Computer Science, ETH Zurich
Zurich, Switzerland
(max. Imax.moebus); (bjoern...... braun); ; {christian holz) nolzh.inf.eehh....... ch
Abstract
Explaining black-box models for time series data is critical for the wide-scale adoption deep learning techniques across domains such as healthcare. Recently, explainability methods for deep time series models have seen significant progress by adopting saliency methods that perturb masked segments of time series uncover their importance towards the prediction of black-box models. Thus far, such methods have been largely restricted to regular time series. Irregular time series, however, sampled at irregular time intervals and potentially with missing values, are the dominant form of time series in various critical domains (e.g.. hospital records). In this paper, for the interpretation of irregular time series models. We first translate techniques for regular time series into the continuous time realm of irregular time series and show under which circumstances such techniques are still applicable. However, existing perturbation techniques neglect the timing and structure of observed data, e.g., informative missingness when data is not missing at random. Thus, we propose Contimask, simple framework perturbations, such as simulating that parts of the data had not been observed using NeuroEvolution. Doing so, we successfully detect how structural differences in the data can bias irregular time series models on real-world sepsis prediction task where 90% of the data is missing. Source code is available on GitHub
conduct the first evaluation of saliency methods
also apply non-differentiable
1 Introduction
Deep learning promises to change the analysis of time series data across various domains, including healthcare, economics, ecology, or physics. To enable the broad-scale application of deep learning models in these domains, model internals have to be verifiable. Suggestions made by models must become explainable for humans using them, which has become the challenge of the field of explainable AI (X AI). Especially in fields such as healthcare, where models could be used to assist patient treatment, human-level explainability is vital enable impactful broad-scale application [9.
Explainability methods for deep learning initially surged for image models [6] 38 31. and have seen some recent progress for time series [13 [4 22]. Methods for explaining time series models initially translated masking and perturbation techniques from the image domain to the realm of time series and extended them to create more realistic perturbations that more naturally imitate time series 14 So far. these methods have only focused on regularly sampled time series, i.e., samples arrive at equally spaced intervals and without missing values.
However, in many domains, time series data arrive at irregularly sampled intervals since data is often only generated in response and the state of the patient's 's health determine when data is collected so that hospital resources wasted 9 Similarly, mobile and wearable computing, signals might only be partially
specific event. For example, in healthcare, the physician
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

al regular time series
b) irregular time series
what if different velue had been observed?
what if different value had been abserved?
what this point had been observed earlier/late later?
what adcitional data had been observed?
what if this point had not been observed?
0.0
0.1
0.2
0.3
0.4 0.5
0.7
0.8
0.9
1.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
09
1.0
apply perturbation
apply perturbation
Figure I: a) for regular time series, the timing of observations is fixed. Counterfactual examples thus simulate different values for existing timestamps. b) for irregular time series, counterfactual examples take many different forms. They might simulate different values for existing timestamps, change the timing observations, remove observations all together or simulate additional observations. Contimask adopts methods from regular time series to simulate different values of observed data. We further introduce a perturbation to simulate that data had not been observed, which helps to uncover the importance of time intensity and missing data, which is neglected by current methodologies.
In such scenarios, data is not missing at random, and missing data and the time interval between data collection can be informative. Irregular time series models are capable of not only considering observed data, but also informative missingness (also called time intensity) []] These models showed particular success for modeling hospital and general practitioner (GP) records, where up to 90% of data might missing, and data is generally sparse
measurable 16 18
33
In scenarios such as the ones described above, it is of particular importance to verify model internals. Take the example of hospital records: does the model simply predict that a patient will have certain disease because a physician ordered a related test? In this case, the model would be of little use in practice. However, information collected way before the treating physician's 's diagnosis is already contributing towards the model's prediction, the model might be a useful aid to improve patient care.
Logically, since the timing of observations is fixed for regularly sampled time series, existing masking and perturbation techniques to explain time series models ignore the timing of observations (see Figure This poses issue when trying to explain models for irregular time series when data is not missing at random, such as in the cases described above.
In this paper, we investigate techniques to calculate saliency maps for irregular time series models. We identify scenarios where adopting perturbation techniques for regular time series work well for irregular time series, and conditions under which they fail to recover the true saliency. We then propose Contimask: a framework applying non -on-differentiable perturbations to irregular time series data to uncover also saliency connected to the timing/minn misiiiggesssssssss of data. Contimask operates in the continuous time realm of irregular time series and utilizes NeuroEvolution to train masks that, besides altering observed data points, simulate that data had not been observed.
Contributions To the best of our knowledge, Contimask is the first method calculate saliency maps for irregular time series models without access to model internals and to apply non-differentiable perturbations that alter the timing and structure of data. Doing so, are able explain also models where missingness influences predictions-comeeee common for irregular time series. In summary, we
propose a novel perturbation for irregular time series that simulates that data has not been observed,
propose a novel method that uses NeuroEvolution to apply also non-differentiable perturbations,
show that small networks in continuous time are more effective to calculate saliency maps for irregular time series-eee esseecallyy if one incorporates Fourier feature activations, and
showcase on real-world prediction task how irregular time series models pick structural biases.

---

2 Background & Related Works
2.1 Irregular Time Series
Irregular time series differ from regular time series in that observations are not collected at uniform time intervals, and not all channels of the time series have be observed all times, i.e., missingness. This format is prevalent in many real-world applications such as healthcare, where data is collected in response events rather than a fixed schedule, and not always the same information is collected.
Understanding and modeling these gaps is critical for downstream tasks. Consequently, any frame- work for interpreting predicting from irregular time series must account not only for the observed values but also for the timing and structure of data (and the structure of missing data).
Notation To formalize the setting of irregular time series, we unify well-established notation [8] that ultimately represents the th element of dataset D a triplet (4,, J., d,).. We assume supervised setting. where we have some black-box function ff that maps elements of 'D to labels y- Therefore, D {s Vn) n I...., N) is a dataset with N elements. An individual element has a label yn. and a sparse and irregularly sampled multivariate time series with C' channels, say Sn.
The C channels of the multivariate time series 8n might be sampled different times, leading to varying numbers of total observations per channel: Lea- Each channel e of time series 8n can thus be noted as = (fen: Ten). ). where ten = [ton..... ......... than] and Idn = [r'm ..... 2nnn") are the list of time points and corresponding observations of channel of element {sn: Mn E D. Ultimately, to transform the time series into triplet of tensors (t, d),, = (tn, In, dn). we lct = - ten E | contain the sorted unique time stamps across all channels. We let xr, E R|r,/xC combine all observed values across all C' channels, and dm be binary data mask indicating channel c has been observed:
if if (0)
= {
if j such that t2, = (1)
In (1, c) = nan otherwise
otherwise
Model Architectures Almost all models for irregular time series involve some level of smoothing or interpolation on raw data or derived hidden representations. These models are designed to handle the non-uniform sampling and potential missingness inherent in irregular time series. Earlier techniques used Gaussian Process Adapters [12), or learned both high- and low low-frequency smoothing functions (26). Oftentimes, such approaches would later discretize the time axis again and simply extract representations interpolated in continuous time at some fixed reference points [12]. Currently, the majority of irregular time series models fall into two main architectural families:
Continuous-time recurrent units: These include Neural Ordinary, Controlled, and Stochastic Differential Equations (NODEs (2). NCDEs ( and NSDEs 20 ). These approaches generalize discrete recurrent neural networks (RNNs) and model hidden states in continuous time using differential equations, or using continuous-time Kalman filter [4
Multi-time attention (mtan 28 leverages attention mechanisms that operate across irregular timestamps. mtan (28), and adoptions thereof, have been particularly successful at interpolating irregular time series
Encoder-decoder models with multi-time attention:
27
2.2 Post-hoc Instance-Level Explanation
In this paper, we attempt what has been termed "Post-hoc Instance-level Explanation" [1]. Originally concerned with image classification, such methods have the goal of explaining the prediction of an arbitrary black-box model f for a single input: e.g., explaining why the provided image of a dog is indeed classified dog [29]. The output of such methods is often described as a saliency map (38] Such maps highlight the parts of the input that are deemed important towards the prediction [3].
The earliest explainability methods tracked representations and/or gradients through the consecutive hidden layers of the network [38 29 31), many requiring access to intermediate layers or architectural modifications (6. This was followed by approaches that modified parts of the input, observing changes in the prediction of f to detect areas of importance ( Many of the current explainability methods for regular time series focus the objective introduced by Fong and Vedaldi [6]:

---

For simplicity, we will ignore the previously defined irregular time series triplet for now, and note any model inputs as z, i.c. we aim to explain some black-box f X - y. Further, let m E [0, 1 1]dlm(=) be a soft mask and (xx m) perturbation of Then, Fong and Vedaldi [6] optimize the following objective (excluding some further penalty terms on m))
min (I) m))) + Al| mi[11:
(I)
where L is some loss function encouraging large perturbations that only minimally affect (xx).
2.2.1 Explanations for Regular Time Series
Most works time series explainability are based the objective defined by Fong and Vedaldi ( where m is learned via gradient descent due to the problem being differentiable (given that f is itself differentiable). Earlier works are FIT (36) and DynaMask [4), who also introduced crucial artificial benchmark problems. While FIT tracks the predictive distribution of f over time to estimate feature importance, DynaMask translates the objective defined in Eq. to the realm of time series. Motivated by Fong and Vedaldi [6], Crabbé and Van Der Schaar [ introduce a penalty on the total variation of m, encouraging smoothness and optimizing for m to cover a set proportion a E (0, 1) of
T-1 C ΣΣ 1=
min Cly, f(f(xx + A vecsort(m) rall2 + X2 mE 10
m me,sl (2)
where x E RT'CC is fully observed time series of length T with C channels, m [0, 1 ro is a sorted vector with the same number of entries as m with the desired proportion being equal to 1 (otherwise 0), which encourages sharp masks that cover proportion of xr
Dynamask proposes two main perturbations for time series: a Gaussian blur CCB and fading to a moving average FMA, Both perturbations applied independently for each channel
FMA (x, m)cc -m1ee Fi, + (1 m,,c) Pt.e:
Σi=1 It' 9o(m)) 1!)
$
GB (z, m)icc
(4)
Bo 1!)
where (t) cxp
with or (m) Omax (1 -m)m Mi,e 2W+1 W Fe and W e
2x
is a sliding window size.
Works like ExtremalMask 5 TimeX++ 13, and ContraLSP 14] have replaced closed-form
perturbations by small RNNs to learn perturbations ((x, m, 0), where 0 parametrizes RNN. This led to perturbations imitating the original data more closely and improved the resulting saliency maps.
2.3 Explainability Methods for Irregular Time Series
Currently, there exists no method for post-hoc instance-level irregular time series explanation. Even though TimeX (22). a self-supervised pre- pre-training strategy to detect explanations via consistency constraints in the latent space, can be adopted to work for methods such as mtan, requires access to model internals. Further, it does not work for architectures such NCDE due to the CDE solve.
2.4 NeuroEvolution
Gradient descent is the predominant optimization technique 'learn' the parameters of neural networks, say f. Techniques such as Adam [10] Stochastic Gradient Descent (SGD) 23 require be differentiable, and the objective f trying optimize using to be differentiable also. NeuroEvolution on the other hand, does not require f or the objective to be differentiable and thereby capable of solving a much larger set of problems 77. While NeuroEEvllttonn is evidently slower, and at times, less reliable than gradient descent, recent work demonstrated state-of-the-art MNIST30K performance using NeuroEvolution [I]. In comparison to Neuroevolution, Adam and SGD operate at light speed. However, they require the objective to supply gradients, i.c., to be differentiable.

---

3 Methods
Our approach Contimask makes step towards post-hoc instance-level explanations for irregular time series models to also detect saliency dependent on missingness and time intensity. Such methods alter, i.e., perturb, the input based on some learned mask m and observe changes in model output. Loosely speaking, if part of the input can be altered greatly without changing the model output, such a region is considered uninformative (i.e., non-salient) towards the prediction of the model. Conversely, if small changes in some regions have a large effect on model output, such regions would be deemed informative (i.e., salient) towards the prediction of the model. All current approaches for explaining time series models only change the value of observed data and ignore the timing/presence of data as possible explanation. For irregular time series, the action space of producing viable counterfactual examples is much larger than for regular time series (see Figure T. Since for regular time series the timing of data is fixed (and thereby the fact that data was observed), only the value of observed data can be altered. A counterfactual example of irregular time series might include different timing of data points, different values of observed data points, data might be missing altogether, or additional data might have been observed. We strive that our approach generalizes to any black box function and does not require access to or specific knowledge of the model internals.
Finding a saliency map in continuous time Following notation from 6 51. we optimize the following objective for some black-box model that takes as input a triplet (t,,, Tn, dn) D as defined in Section 2. Assume we are trying to explain f(tn, Zn, dy), where |t, = T, and In E TTxC is a time series with C channels, and therefore d,, E [0, 1] where Iy might include missing values that are marked with a O in d,, non-missing entries are denoted by 1 in d,. In contrast to prior work, we define m : - [o. 1] as function operating in continuous time. Provided some perturbation + and a loss function L, we optimize:
min L(I( du); [[nn Fn, da)) + A LL (1 (1- (i-m(u)))due A2
C E Im(u)/lu,
(5)
where (In, Rn, dr)) = ((,,, Fn, dn): m). Similarly to Eq. we are trying to maximize the area covered by m while ensuring is somewhat smooth and minimizing changes in model output.
We adopt @CB and FMA as used in DynaMask to the setting of irregular time series, i.e., where m is a function in continuous time and includes missing values as indicated by the binary mask d:
-(ts
[m(i) ( Tie + (1 (1- (1)me)) pi.o Ii
if di = 1,
FMA ((t, d.,, m)aa = ts.
if d2. = 0, d1.c)
(6)
T -1=1 BVe (t(m(e))((tt))
if die = 1,
n GB ((1, d), m); li
Σl Et 9o(m(t)))) -t)
die
(7)
Ti.e
if di = 0,
where (z(1) = exp
,2 20²
with (m) = Fmax (1 - m and Pi.e 2W-1 Σ Fy where
W N is a sliding window size. Note that and d remain unchanged, and only the value of the observed data (z) is altered. Please find visual examples of these perturbations in Appendix[A.. A.
A perturbation that alters the structure of observed data We further introduce simple, yet effective, data perturbation that removes data points, i.e., simulates that data points had not been observed. We refer to this perturbation simply the Deletion aeetiinneetuuaattonnn---o Del in short. This forms the first perturbation to also affect d of our triplet (and potentially f):
[1a Ti.e: di.e) Del (t, m),
if Ber(m(())) =0
1.c (t,, nan, 0) if Ber(m(())) =
(8)
Here, Ber(p) denotes the sample from Bernoulli distribution with probability of success equal to p. If Ber(m(i).. 1, Del therefore alters and d. Effectively, in case there are time points for which all data is being removed, t is also being altered. We ignore this scenario in the above notation.
5

---

Non-differentiabiity of perturbations that alter the structure of data Without knowledge of the specific model internals, simulating that data had not been observed is non-differentiable w.r.t. the applied perturbation mask m. since this requires mi to be hard mask (i.e., a binary or boolean mask). This holds true for all perturbations that affect the structure of the data, i.e, that affect not just æ, but also d, or the order of f. With access to and exact knowledge of model internals, deletion of data can be simulated for mtan models or, partially, also for models such as NCDE (see Appendix (A.2). Perturbations that alter the timing of data or insert new data points (see Fig. Therefore, Contimask relies on gradient-free optimization to learn mask m..
[
will face similar issues.
We learn m using NeuroEvolution and employ the PGPE algorithm 22 7 as implemented in EvoTorch [35] using the ClipUp optimizer [34]. While NeuroEvolution gets around the issue of non-differentiabilty. it trains much slower, which we discuss more extensively in Appendix A.3
Perturbation masks as functions in continuous time We define saliency, and the mask we are trying to construct in continuous time. Rather than optimizing for a tensor m E RTxxC we learn m(t): : R- + [0, 1] as small feedforward neural network. We show that this reduces the required number of parameters to accurately define m and, particularly when employing NeuroEvolution, leads to better performance and faster training. To learn m, we evaluated various small network archi- tectures. Beyond a 3 3-layer MLP with ReLu activations, we evaluated architectures that incorporated feature transformations to create sharper masks. In particular, we tested sinusoidal representations (SIRENs) 30 Haar functions 21] and Fourier feature transformations 32 We ultimately opted for Fourier feature transformations given performance on initial baselines (Table T and proven success on images 155. In this paper, MT implies m E T whereas MLP implies m(t) IR + [0, 1/5.
MLPs with and without Fourier feature transformations We term the small MLP with Fourier transformations MLP-F. Given a scalar input t, MLP-F applies fixed encoding using L exponentially spaced Fourier frequencies, transforming the input into 2 2--dimensional feature vector composed of sine and cosine terms. If not stated otherwise, we set T. 10, such that:
L-1 (t) [sin(2* Rt), cos xt) k=
These features, 7t), are then passed through a MLP with three linear layers and ReLU activations, mapping to an output of dimension C. When we refer simply to MLP, we feed a scalar input straight into three linear layers with ReLU activations. We use a final sigmoid layer such that m(l) E [0, 1]c.
4 Evaluation
We consider 5 problem settings. We first convert two commonly used synthetic scenarios for regular time series explanations into the continuous time setting. Termed 'Rare Time' and 'Rare Feature' by Crabbé and Van Der Schaar 44, they depend on low number of salient times and a low number of salient features, respectively, and form scenarios that saliency methods have traditionally struggled with (22 13 14]. The original versions of these two artificial problems depend on the value of observed data. We will refer to this setting as value-based, i.c., the function that are trying to explain depends on the element x of our triplet defined in Section 2. We then adapt these two scenarios and create two problem settings, where the value of observations (i.e., x)) does not impact the output of the function we are trying to explain, but only the timing of data, i.e., t. We refer to this setting as temp-based. We finish by explaining model trained common problem for irregular time series models: sepsis prediction from hospital records based on [8]
Rare Time& & Rare Feature In these two artificial problems, create white-box regressors that only depend on a salient area, say A = AT x Ax, where Ax C [1 C7. While in the regular time series settings, Ay C [0: 7J, we define A. C [0, 1] again in continuous time.
In the Rare Time scenario, we consider 100 randomly sampled time points in the interval [0, and 3 channels. At each iteration of the experiment, we sample random location for each of the three channels to define our salient area. The salient area covers 20% of the first channel, 10% of the second, and 45% of the third. In the Rare Feature scenario, we sample again 100 time points but simulate time series with 50 channels. For each iteration, we randomly sample 5 consecutive features that are the only salient ones, and we set AT [0.25, 0.75) in all cases.

---

For both scenarios, as per 4 the output of the white-box regressor is Σ EA z3.0--
Temp-based settings We adapt the previously defined scenarios and change the white-bor regressor to Euooe 1, i.e. we count all data points in A such that the output is only dependent on L.
Sepsis prediction We train a NCDE and mtan model on the sepsis prediction task as implemented by Kidger et al. [8] We then take these models, and apply the Del, FMA, and GB perturbations to train MLP-F masks to cover 10% of the observed data to explain two cases: one where the model is initially confident that patient will die of sepsis within the first 72 hours in the hospital; and one where the model is initially confident of survival. We only explain cases on the test set (5.4% mortality). In addition to static variables (i.e. fixed over time), 34 features were sampled irregularly over 72 hours at hour resolution with R 90% missing values.
Since the NCDE model trained by Kidger et al. [8] takes as input cubic spline coefficients rather than the irregular time series triplet (S.ction[2.1).. we first have to reconstruct the test set from the spline coefficients. At each iteration, we then apply our perturbations on the reconstructed data, calculate the new cubic spline coefficients, and feed those into the model to observe the change in prediction. Recalculating the coefficients for single evaluation is very slow (30s on H200 GPU). Therefore, we train smaller MLP-F masks with a hidden dimension of 16, and L = 12 Fourier features for only 200 epochs using PGPE and limit ourselves to two participants with low and high probability of sepsis. While the mtan model operates directly on the irregular time series and is therefore much faster to explain, stick to above settings for fair comparability. vram
Metrics For the Rare Time & Rare Feature settings, ground truth saliency maps are available. We calculate the Fl score (F1), Precision (Prec), and Recall (Rec) for correctly identifying these maps, using a threshold of 0.5 to binarize masks. However, masks are usually already binary, as also indicated by very low S.n values, especially for MLP-F masks. Further, calculate metrics introduced in A Jm (A) = E(sojE In(1 - ([(1-)(()a)) to capture the information content of mask m over the salient area A (higher is better), and Sm
- L(ie)e m m(i)e) In(1 m(i)c) measure the sharpness of the mask over A (lower is better).
In m()). + (1
For sepsis prediction, we calculate the average change in log-odds as the main metric. To allow for a fair comparison between Del, which removes data, and FMA and GB, which only alter the value of observed data, we calculate two metrics for each mask m. First, we apply Del based on the output of m observe the change in predictions, which we term 'Del odds change'. Second, we impute 0 for all data points suggested by the trained mask, similar to 4 5], which we term 'Imp odds change'. Note that per (8) all features are normalized, i.e., imputing inserts the mean feature.
Experiment setup
We run all experiments using an H200 GPU needing at most 8GB of VRAM.
5 Results
In Table [ we show how gradient descent and gradient- free training via NeuroEvolution compare for the value-based Rare Feature problem when applying the FMA perturbation. Here, we compare different mask parameterizations, all with similar parameter count close to 5,000. We set λι 0.01, A2 = 0.001 and train for 16,000 epochs using an Adam optimizer with a learning rate of 0.01, or 2000 iterations using the PGPE optimizer with a population size of 100. For PGPE, we initialize with radius of 3, and center learning rate of 0.5 (+0.3). While MLP and MLP-F have not yet meaningfully converged after 16,000 iterations, MT achieves an almost perfect Fl score using gradient descent. However, when training using NeuroEvolution, MLP and MLP-F outperform MT, even outperforming the gradient descent training. We provide a more detailed comparison between the different training strategies, and how parameter count influences convergence and computation cost in Appendix A.3] Since MLP-F consistently outperforms MLP, we chose MLP-F in all subsequent problems with hidden layer size 32 and L 10, resulting in parameter count of 3,400.
In Table 2 we compare different perturbations to train masks parameterized by MLP-F using Neu- roEvolution for the basue and temp-based problem. We use the same settings for the PGPE algorithm as before, we identify the optimal values for X and X2 (Eq. 5) based on broad-sacle grid s.t. A₁ = 100g or d 100Ag. The optimal parameters are reported in Appendix A.4 While
7

---

Gradient Decent
NeuroEvolution
Mask
F1 1
Prec ?
Rec 1
I1
S 4
F1 f
Pree 1
Rec 1
It
S 4
MLP
0.192
0.419
0.131
189
48.235
0,534
0.522
0.582
2387
2.050
MLP-F
0.250
0.796
0.167
268
79.126
0.622
1.000
0.452
1877
0.218
MT
0.978
1.000
0.957
335
238.789
0.219
0.147
0.436
449
134.712
Table 1: Comparison of different mask parameterizations and whether they are learned using gradient
descent or NeuroEvolution. Performance is compared on the Rare Feature Dataset using differentiable perturbations as proposed in 44, averaged across 10 runs. The mask tensor (MT) has 5,000 entries (50 features x 100 time points). The hidden dimensions of MLP and MLP-F are scaled to approx. the same number of parameters, resulting in 4,914 and 4,898 parameters, respectively.
all three perturbations perform similarly for the value-based setting, Del, unsurprisingly, clearly outperforms FMA and GB for the temp-based setting, being the only perturbation to detect saliency that is solely based on L. Similarly, in Table we observed comparable performance for perturbation FMA and Del for the value-based setting, yet a complete failure to detect saliency for perturbations FMA and GB in the temp-based setting. We generally observe lower FI scores for the Rare Time setting than the Rare Feature Problem, indicating slightly higher difficulty.
Value-based
Temp-based
Perturbation
F1 f
Prec f
Rec f
If
S +
F1
1 Prec 1
Rec t
If
S $
GB
0.556
0.973
0.430
1786
0.004
0.000
0.000
0.000
0
0.000
FMA
0.638
0.774
0.557
2311
0.066
0.000
0.000
0.000
0
0.000
Del (ours)
0.891
0.967
0.851
3533
0.004
0.982
0.967
1.000
4152
0.004
Table 2:
Method comparison on the Rare Feature Dataset across 10 runs using MLP-F masks.
Examples of fitted masks are included in Appendix|A. A.7
In Table we compare the explanations of MLP-F masks for the real-world sepsis prediction task. We trained NCDE (8) and mtan (28 model for the sepsis task proposed in [8]. Both models which achieves a binary AUC of roughly 0.90 on a held-out test set (the same 20% split as per (8). We compare the change in log-odds when perturbing = 10% of the observed data points that are indicated as important by the respective MLP-F mask. We train masks using an adapted version of Eq. 5 where we penalize not the total area covered by the mask but the deviation from the 10% of observed data (see Appendix AA33 for details). Since returned masks usually cover a larger area than 10%, we randomly sample them down to the desired 10%. We again start a grid search for different optimal penalty weights (Λ₃ and Ae in Eq. 5). starting with very high values of a (we set X2 = 0.1A₁ or A2 = ).. We iteratively reduce A and A2 by factor of 10 until the resulting mask perturbs at least 10% of the observed data. Selected hyperparameters are outlined in Appendix A.S An ablation study shows the effect of different values for \ and X2 for the sepsis prediction task using the mtan model in Appendix [..6) Since evaluations of the NCDE model require calculating spline coefficients for every new perturbation, explaining NCDE models is very slow. The mtan model operates on 'raw' data without requiring additional calculations, and evaluations are much faster. When explaining prediction made by either model, we observed a difference in speed of around 70x. For the NCDE model, we train the PGPE optimizer for only 200 iterations with population size of 50, which brings a single run down to roughly 2 hours using an H200 GPU, The same process takes less than 2 minutes for the mtan model. We discuss computational speed in Appendix A.3 and A.8
A.6
For the NCDE model, we explain the predictions for 50 participants of the test set (25 who were predicted to become septic and 25 who were predicted not to become septic). For the mtan model, we explain total of 200 predictions for the 100 participants with the highest and lowest probability of becoming septic, respectively. We find that Del clearly outperforms FMA and GB for patients who are predicted to develop sepsis. Generally, all perturbations perform much better explaining Pred: Sepsis cases, i.e., where the model predicts sepsis. For cases that are predicted to not develop sepsis, value altering perturbations outperform Del. For predictions made using the mtan model, we investigate the explanations made using Contimask more closely in Section 5.1
8

---

Value-basee
Temp-baser
Perturbation
F1 f
Prec f
Rec f
If
S 4
F1
Prec
Rec f
If
GB
0.336
0.794
0,283
396
0,003
0,000
0,000
0,000
0
0,000
FMA
0.747
0.997
0.600
840
0.001
0.000
0.000
0.000
0
0,000
Del (ours)
0.692
1.000
0.531
744
0.001
0.248
1.000
0.147
206
0.001
Table 3: Method comparison the Rare Time Dataset across 10 runs using MLP-F masks. Examples of fitted masks are included in Appendix A.7
A.7
mTAN
NCDE
Pred: Sepsis
Pred: No Sepsis
Pred: Sepsis
Pred: No Sepsis
Perturbation
Del
Imp 1
Del t
Imp
Del f
Imp
Del f
Imp t
GB
4.41
10.45
6.29
11.80
0.25
0.26
4.01
4.05
FMA
4.16
3.41
7.26
7.19
0.26
0.23
5.15
5.16
Del (ours)
7.88
2.26
4.57
5.90
4.97
0.21
2.72
3.94
Table 4: Comparison of perturbations using Contimask for Sepsis prediction from ICU data [8] using
an MI.P-F mask. We train m'TAN (28) and NCDE [8 model for the sepsis prediction task and
explain predictions on the left-out test set. For mTAA, we investigate the 100 cases with the highest and lowest predicted probability of developing sepsis. For NCDE, we only investigate 25 cases each.
5.1 Case Study
Table 5: Retention and sepsis-related imbal-
ance for the 6 features removed most often by Del-perturbation for mtan sepsis predictions
For the Sepsis prediction task using the mtan model, we further investigated the results of fitted masks. We display 4 examples in Figure 2 Table [3] showcases the 6 features that were removed proportionally most often for participant that was predicted to develop sepsis by the mtan model. Interestingly, these are mainly static features (age, height, ICUTType), or fea- tures that are up to 52% more likely to have been recorded for a patient that later becomes septic (Im- balance). We include a full table in Appendix [A.9 Table 13 Retention indicates how often feature
Feature
Retention (%)
Imbalance
Age
79.12
1.00
Height
79.82
1.00
AST
80.63
1.52
ICUType
81.96
1.00
ALP
82.04
1.44
Cholesterol
83.00
1.43
retained after applying the learned Deletion mask. Since not all signals are recorded for all patients, there is the possibility of structural differences in the data between patients who develop sepsis and patients who do not. Examples of this might include blood markers that are only relevant to monitor if the treating physician believes that the patient the Imbalance column indicates how much more likely a patient who later developed sepsis is to have any observation of that specific feature, highlighting the potential of bias related to this particular feature. A value greater than indicates that this feature is recorded more often for patients who later become septic. Troponin I, bloodmarker often used test for sepsis, has the highest Imbalance with value of 2.6. AST, ALP, and Cholesterol are among the next most imbalanced features, and are indicated to strongly influence predictions by the mtan model. This indicates that the mtan highly influenced by decisions of treating physicians about whether to monitor certain features, which might be highly indicative of whether a patient is at risk of becoming septic. If this is indeed the case, such a model would be of little use in practice as it simply confirms the decision of a treating physician. The capability irregular time series models to incorporate the structure the data into prediction can therefore also become issue and introduce bias.
high risk of developing sepsis. The value in
Discussion
We show that to explain irregular time series models, existing techniques for regular time series are only partially sufficient. We propose the Contimask framework, where we learn masks parameterized small feed-forward networks with Fourier feature transformations using NeuroEvolution. Within

---

0
10
10
10
10
20
20
20
time (hours)
30
30
30
1
40
40
40
50
50
50
50
60
60
60
70
70
70
70
20
40
20 features
40
0
20
40
0
20
40
Teatures
features
features
Figure 2: Example masks fitted to 4 participants predicted to develop sepsis. Observed features that are not affected by the perturbation are marked in yellow. Green dots are removed by the fitted mask.
this framework, we are able uncover saliency that is based on the structure of the data, such as time intensity informative missingness, via our novel Deletion perturbation, where we simulate that data had not been observed. While our Deletion perturbation is cin-differentiable (as any perturbation that alters the structure of the observed data), it clearly outperforms differentiable perturbations on toy datasets and sepsis predictions on real-world data. In particular, existing techniques for regular the value of observed data, which Contimask
time series fail if the saliency is not dependent using Deletion on is able to detect. As indicated by the strong performance of Contimask using the Deletion perturbation when explaining model for sepsis prediction, missingness and the structure of data are important to explain irregular time series models operating on hospital records. We find particular evidence of irregular time series models picking up structural differences in the data introduced by decision of the treating physicians, which might introduce dangerous biases.
The applicabiliity of Contimask is limited based on the model family that is being explained: while some architectures (e.g., mtan) are much quicker explain, explaining NCDE models is particularly challenging given the repeated fitting of cubic splines. Further, Contimask is limited in its ability to incorporate learned perturbations similar to 15 13), where more fine-tuned perturbations improve explanations, since NeuroEvolution does not scale well gradient descent. In practice, we have found that the slower speed of NeuroEvolution makes it more difficult to obtain stable results. Thus, improved NeuroEvolution strategies might lead to improvements in speed and also accuracy.
We thereby conclude that the development of novel post-hoc explanation tools for irregular time series is crucial to foster more broad-scale application of irregular time series in critical domains such as healthcare, where data is not missing at random. In such scenarios, it is critical assess if the model is indeed useful in practice or has picked up biases that might also arise from informative missingness. Existing techniques for regular time series are not sufficient, and perturbations truly imitating irregular time series are needed for better model explanations. However, one of two advancements is necessary such explainability methods: Either, 1) advancements on differentiable
to increase the applicability perturbations that uncover similar patterns such our introduced Deletion perturbation, advancements of existing NeuroEvolution methods to speed up the lcarning masks such as the ones used in Contimask are needed. Since the latest generative models for irregular time series not capable of altering the timing and structure (i.c., missingness) of data [19], generating more diverse counterfactual examples in a differentiable manner still remains very difficult problem. Tailoring NeuroEvolution techniques to train masks for 2D problems more efficiently might a more promising route for future work.
2)
Further work needed to combine value-altering and structure-altering perturbations into a single framework. Existing techniques for regular time series might be able to pick up saliency that Contimask using the Deletion perturbation not, as also indicated by our results for sepsis-negative cases where valuc-altering perturbations performed best.
10

---

References
[1] B. Braun, R. Armani, M. Meier, M. Moebus, and C. Holz. egoppg: Heart rate estimation from eye-tracking cameras in egocentric systems to benefit downstream vision tasks. arXiv preprint arXiv:2502.20873 2025.
[2] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[3] Y. Chen, K. Ren, Y. Wang, Y. Fang, W. Sun, and D. Li. Contiformer: Continuous-tine transformer for irregular time series modeling. Advances in Neural Information Processing 3y:tems,,6:4777 111111444775555 2023.
[4] J. Crabbé and M. Van Der Schaar. Explaining time series predictions with dynamic masks, In International conference on machine learning. PMLR, 2021.
[5] J. Enguchard. Learning perturbations to explain time series predictions. In A. Krause, E. Brun- skill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning. PMLR, 2023.
[6] R. C. Fong and A. Vedaldi. Interpretable explanations black boxes by meaningful perturbation. In Proceedings of the IEEE international conference on computer vision, pages 3429-3437, 2017.
[7] D. Ha. Reinforcement learning for improving agent design. Artificial life, 25(4):352-365. 2019.
[8] P. Kidger, J. Morrill, J. Foster, and T. Lyons. Neural controlled differential equations for irregular time series. Advances in neural information processing systems, 2020.
[9] R. King, T. Yang, and B. J. Mortazavi. Multimodal pretraining of medical time series and notes. In Machine Learning for Health (MI4H), pages 244-255. PMLR, 2023.
[10] D. P. Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: arXiv:1412.666 1412.6980 2014.
[11] K. Lenc, E. Elsen, T. Schaul, and K. Simonyan. Non-diffeernniable supervised learning with evolution strategies and hybrid methods. arXiv preprint arXiv:1906.03// 2019.
[12] S. C.-X. Li and B. M. Marlin. scalable end-to-end gaussian process adapter for irregularly sampled time series classification. Advances in neural information processing systems, 29, 2016.
[13] Z. Liu, T. Wang, J. Shi, X. Zheng, Z. Chen, L. Song, W. Dong, J. Obeysekera, F. Shirani, and D. Luo. Timex++: Learning time-series explanations with information bottleneck. In International Conference on Machine Learning, pages 32062-32082. PMLR, 2024.
[14] Z. Liu, Y. Zhang, T. Wang, Z. Wang, D. Luo, M. Du, M. Wu, Y. Wang, C. Chen, L. Fan, and Q. Wen. Explaining time series via contrastive and locally sparse perturbations. Proceedings of the 12th International Conference on Learning Representations, 2024.
[15] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1))99-106, 2021.
[16] M. Moebus, S. Gashi, M. Hilty, P. Oldrati, and C. Holz. Meaningful digital biomarkers derived from wearable sensors to prediet daily fatigue in multiple sclerosis patients and healthy controls. Iscience, 27(2), 2024.
[17) M. Moebus, M. Hilty, P. Oldrati, I.. Barrios, P. A. Consortium, and C. Holz. Assessing the role of the autonomic nervous system as driver of sleep quality in patients with multiple sclerosis: Observation study. JMIR Neurotechnology. 2024.
[18] M. Moebus, L. Hauptmann, N. Kopp, B. U. Demirel, B. Braun, and C. Holz. Nightbeat: Heart rate estimation from wrist-worn accelerometer during sleep. IEFE Journal of Biomedical and Health Informatics, 29(2):870-877, 2025. doi: 10.1109/BHLL2222333444
11

---

[19] I. Naiman, N. B. Erichson, P. Ren, M. W. Mahoney, and O. Azencot. Generative modeling of regular and irregular time series data via koopman vaes. In The Twelfth International Conference on Learning Representations, 2024.
[20] Y. Oh, D. Lim, and Kim. Stable neural stochastic differential equations analyzing irregular time series data. In The Twelfth International Conference on Learning Representations, 2024.
[21] Y. Pang, X. Li, Yuan, Tao, and J. Pan. Fast haar transform based feature extraction for face representation and recognition. IEEE Transactions on Information Forensics and Security, 4(3):411-455 2009.
[22] O. Queen, T. Hartvigsen, T. Koker, H. He, T. Tsiligkaridiss and M. Zitnik. Encoding time- series explanations through selervuue model behavior consistency. Advances in Neural Information Processing Systems, 36:32129-3211 2023.
[23] H. Robbins and S. Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.
[24] M. Schirmer, M. Eltayeb, S. Lessmann, and M. Rudolph. Modeling irregular time series with continuous recurrent units. In International conference on machine learning, pages 19388- 19405. PMLR, 2022.
[25] F. Sehnke, C. Osendorfer, T. Rückstießß A. Graves, J. Peters, and J. Schmidhuber. Policy gradi- ents with parameter- based exploration for control. In Artificial Neural Netorks-ICANN 2008: 18th International Conference, Prague, Czech Republic, September 3-6, 2008, Proceedings, Part 18, pages 387-396 Springer, 2008
[26] S. N. Shukla and B. Marlin. interpolation-predicihh networks for irregularly sampled time series. In International Conference on Learning Representations, 2019. URL /ttpps:// openreview www.:n//tooommidieieiieeee
[27] S. N. Shukla and B. M. Marlin. Heteroscedastic temporal variational autoencoder for irregularly sampled time series. arXiv preprint arXiv:2107.11350, 2021.
[28] S. N. Shukla and B. M. Marlin. Multi-time attention networks for irregularly sampled time series. arXiv preprint arXiv:2101.10318. 2021.
[29] K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 2013.
[30] V. Sitzmann. J. Martel, A. Bergman, D. Lindell, and G. Wetzstein. Implicit neural representations with periodic activation functions. Advances in neural information processing systems, 33: 7462-7473, 2020.
[31] M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319 -328. PMLR, 2017.
[32] M. Tancik, Srinivasan, B. Mildenhall, Fridovich- Keil,, N. Raghavan, U. Singhal, R. Ra- mamoorthi, J. Barron, and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances neural information processing systems, 33:7537-7547, 2020.
[33] S. Tipimeni and C. K. Reddy. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series. ACM Transactions on Knowledge Discovery from Data (TKDD), 16(6):1-17) 2022.
[34] N. E. Toklu, P. Liskowski, and R. K. Srivastava. Clipup: A simple and powerful optimizer for distribution-basel policy evolution. In International Conference on Parallel Problem Solving from Nature, pages 515-527. Springer, 2020.
[35] N. F. Toklu, Atkinson, V. Micka, Liskowski, and R. K. Srivastava. EvoTorch: Scalable evolutionary computation in Python. arXiv preprint, 2023. https://arii.orr/b//330......
12

---

[36] S. Tonekaboni, S. Joshi, K. Campbell, D. K. Duvenaud, and A. Goldenberg. What went wrong and when? instance-wise feature importance for time-series black-box models. Advances in Neural Information Processing Systems, 33:799-809, 2020.
[37] A. Y. Yıldız, E. Koç, and A. Koç. Multivariate time series imputation with transformers. IEEE Signal Processing Letters, 29:2517-255 2022.
[38] M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In Computer ECS 2014 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Purt I 13, pages 818-833. Springer, 2014.
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] pprovided proper justification is given (e.g., "error bars are not reported because would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our contributions are accurately listed.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We clearly communicate the limitations of our work terms of compute and required tuning of hyperparameters.
14

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
We do not provide theoretical proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our Methods and Evaluation Sections allow for clear reproducibility. Even without our provided code, implementing method is straightforward.
Guidelines:
The answer NA means that the paper does not include experiments.
15

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibilityy which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We publicly share our code and all data is either synthetically created as part of the code we provide or publicly available online and we provide the download and processing scripts.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https:/nnips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 7/nips. wwwpcccccccc......c...cco
wpwplicgguides/////pppppoo guwdge/eee/eeee (oodeSubisssii sssssnooiiiiyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We clearly outline the tests conduct, and how we tuned hyperparameters. Even without our code, our paper includes all those details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
Where applicable, we include standard deviations across 10 test runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split. initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer:[Yes
We outline the specs of the server we used and how much memory was being used since much smaller server would have been sufficient.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: To the best of our knowledge, we fully conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work does not have direct implications society.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
18

---

Our works evaluates broadly used models either on small completely artificial toy datasets and one commonly used publicly available dataset hosted via PhysioNet. Guidelines:
Justification:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: To the best of our knowledge, we owners of assets used the paper are properly credited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, papprssii thcodee com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We did not collect new data.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
We did not collect new data.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs were not involved in any stage this research.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our L.MM policy huttps://neurbb ce / for what should or should be described.
20

---

A
Appendix
A.1 Examples Perturbation In Continuous Time
Figure 3 shows examples of the three perturbations applied to the same single-channel irregular time series. All perturbations are applied to the same area of that time series.
perturbation
parturbation
perturbation
Katnge
X
10
10
19
Tine
Time
(a) FMA
(b) GB
(c) Del
Figure 3: Examples the three perturbations FMA, GB, and Del applied to same single channel irregular time series in the gray-shaded area. FMA insert the Moving Average, GB applies a Gaussian Blur, and Del deletes the points that inside the gray-shaded area.
A.2 Simulating The Deletion Of Data In Differentiable Manner
Without knowing how exactly model processes the data it receives input, it is not possible to simulate that data had not been observed a way that differentiable with respect to the mask m, where highlights which data points are being deleted. In such case, needs to be converted to hard mask. Essentially, at some point during the process, m needs to function as an index. This means it has to be converted to integer or boolean data type, which breaks the automatic differentiation graph in frameworks such as PyTorch.
For NCDE models, for instance, the way in which data is processed internally allows to (partially) simulate that data had not been observed. NCDE models model irregular data using differential equations. Via integration (i.e., the numerical ODE solver of one's choice), the hidden states of hidden
model are derived continuous time. Given the nature of differential equations, states will therefore only update if the data changes. One can therefore delete the data observed a certain timepoint by constructing forward (or indeed backward) fill that overwrites said timepoint to duplicate addition, we are still able to backpropagate w.r.t. m. However, note that this includes overwriting alter that
of the neighboring timepoints. Since this only involves multiplication and
the time intensity channels, and also the entry in t. Thereby, observed at the chosen timepoint. This means simulating that no data had been observed at all at that timepoint-ase across all channels.
data had been
possible simulate the deletion of data points in a way that is differ- entiable with respect to the deletion mask m for models such mtan if one has access to model internals. Since mtan relies on attention modules to deal with missing data, nipulate the attention mask to simulate that some further datapoints had not been observed. To do so, one requires access to the attention module. mtan uses scaled dot-product attention for some reference points (set somewhat arbitrarily to project the irregular data into fixed dimension) and the timings of observed data points. The points one trying to 'delete' can then be subtracted from the resulting attention mask or the corresponding part is simply multiplied by zero during the dot-product operation. These points then no longer influence the state of the model at the reference points, and thereby longer have influence the model output. Since this operation does not require hard mask, and only involves multiplication and subtraction, it allows to simulate that data had not been observed if one is willing access model internals.
Going one step further,
possible ma-
Please note that we have not tested the ideas described above practice. One would need to verify that they behave exactly intended and indeed produce gradients useful for learning. While the
21

---

above strategies for mtan and NCDE do not break gradients, it is not guaranteed that they indeed feedback gradients that enable stable learning. If one is using an objective similar to Eq. 5 the gradients related to the penalty terms will not be zero, but gradients related to the error in will likely oftentimes be zero which might still make learning difficult.
A.3 NeuroEvolution: performance, parameter counts, and runtime
In Table O we compare the runtime and performance when trainingMLP-F masks using different parameter counts. While performance varies based on parameter count, the time required for training remains relatively constant. All experiments were performed on H200 GPU, where the used VRAM never exceeded 8 GB. In Table the performance is averaged across the 10 runs, while the runtime is for all 10 experiments together. The results show that much smaller mask with only 1906 parameters achieves similar performance to a mask close to 5000 parameters.
Training using NeuroEvolution takes 4-5 times longer than using gradient descent. As discussed in Section we observe that masks parameterized as a Tensor of the same shape as X and d train in much fewer iterations using gradient descent, but require more iterations using NeuroEvolution. We' ve trained for 2,000 iterations using NeuroEvolution, and 32,000 iterations using gradient descent. Masks parameterized as MLP-F converge much better using NeuroEvolution but worse using gradient descent. We can only speculate about the reasons for this. Potentially, they are related to the assumptions algorithms such as PGPE make about the distributions of the parameters they are trying to optimize.
Model Size
NeuroEvolution
Gradient Descent
Dim
L
Count
F1
Prec
Rec
Time
F1
Prec
Rec
Time
MLP-F
34
4914
0.622
1.000
0.452
2:18
0.250
0.796
0.167
0:27
MLP-F
32
32
4786
0.690
0.961
0.556
2:18
0.170
0.372
0.115
0:26
MLP-F
32
24
4274
0.638
0.774
0.557
2:08
0.170
0.569
0.112
0:26
MLP-F
16
24
1906
0.670
0.897
0.560
2:17
0.218
0.560
0.143
0:26
MLP-F
16
12
1522
0.584
0.835
0.490
2:18
0.157
0.377
0.108
0:27
MLP-F
8
24
914
0.539
0.717
0.515
2:17
0.107
0.381
0.068
0:26
MLP-F
12
722
0.497
0.806
0.429
2:13
0.106
0.353
0.070
0:27
Table 6: Model size, performance, and runtime averaged across 10 runs for the balue-based Rare Feature problem when training MLP-F masks using the FMA perturbation across 10 runs. The performance is averaged across 10 runs, the runtime (Time) is given for all 10 runs in total in (h:mm), ie., training an MLP-F mask with 4914 14 parameters 10 times takes 2 hours and 18 minutes using NeuroEvolution and only 27 minutes using gradient descent.
A.4 Regularizing Parameters
Table 7 [ndd g display the chosen values for A₁ and A2 for the Rare Feature and Rare Time problems for each of the settings. We started with values for X that penalize the area covered by the mask too heavily and then decreased Ar until the mask returned reasonable results, where initially we restricted A such that \ 10A₂ A 100 Once value for X set, optimized further for X2 also allowing for smaller values of X2.
Temp-based
Perturbation
À
A
A2
GB
0.01
0.001
0.1
0.000001
FMA
0.01
0.001
0.1
0,000001
Del
0.1
0.001
0.1
0.000001
Table 7: Values for X and Azfor the Rare Feature Problem reported in Table
22

---

Value-based
Temp-based
Perturbation
A
Ag
A
A2
GB
0.1
0,0001
0,00001
0.000001
FMA
0.1
0.0001
0.00001
0.000001
Del
0.1
0.0001
0.00001
0.000001
Table 8: Values for λ and X2 for the Rare Time Problem reported in Table
A.5
Updated Objective Function For Real-world Sepsis Task
For the sepsis prediction task, we adapted Equation from:
5 such that the loss term involving A changes
A ΣΣ E///0 (1
(1-(u)))
mm()))) du
(9)
0
to:
TS - 11-E// EL/ (m(u)c) A
(10)
where TS is the target size, i.e. in our case 100% 10% = 90%.
For the sepsis task, the goal is to maximize the divergence in prediction while the mask should only alter previously defined target size TSS. for example, 10% of all observed time points. The goal for the previous tasks on synthetic data was keep the prediction unchanged while altering as many data points possible.
The full objective for the sepsis task then becomes:
min -()(- dn), (In, Tn, dn)) A1 TS TS-EA OS-E/,
E// (m(())) du
A2 Σ/ |m(u)'d du.
(11)
A.6 Ablation Study for A and A22
We conducted an ablation study on the hyperparameters X and X2 for the Del, FMA, and GB attribution methods on the sepsis prediction task using mTAAN The results are reported for 10 cases predicted as sepsis-positive. Each cell contains two values: the first corresponds to the Del odds change, and the second to the Val odds change. Note that these results can be higher than those reported for the full set of 100 sepsis-positive cases, as the top 10 predictions are typically easier to explain-mon more confident predictions tend to exhibit more pronounced feature relevance.
Table Ablation on and A2 for Del. Each cell shows Del odds change Val odds change.
A1
X2
A A A1/1110 110
A \
0.01
6.24 7.06
10.34 7.94
5.95 8.27
0.1
1.44
9,9117.88
6.51 6.82
0.5
10.75
9.05 6.89
8.09 7.42
1.0
12.92
18.42
13.6417.19
10.99 8.48
10.0
11.2416.88
8.50 17.55
8.69 7.30
A.7 Examples of Fitted Masks
To visualize how metrics such as F1-Score, Precision, and Recall represent the quality of fitted masks, Figure 4 shows examples of MLP-F masks fitted based on GB, FMA, and Del perturbations using NeuroEvolution. The masks are fitted for the value-based Rare-Feature problem and correspond to what is displayed in Table 2 (100 time points and 50 features). Averaged across the 10 runs, the
23

---

Table 10: Ablation on X and A2 for FMA. Each cell shows Del odds change
Val odds change.
A1
A
X1/100
A \
0.01
6.76
7.76
6.01 17.81
10.08 77.48
0.1
9.59
9.68 99.31
0.5
7.16
10.4718.17
12.63 9.44
1.0
6.95
10.24
11.83 7.15
10.0
17.04
11.63 6.56
11.43 7.71
Table 11: Ablation on A and A2 for GB. Each cell shows Del odds change
Val odds change.
X1
A 0
A
\1 /100
A2
X1/10 10
0.01
5.86
| 15.78
5.54 15.58
8.91
15.16
0.1
4.64
15.22
5.85
14.84
9.14 16.04
0.5
6.05
16.92
9.27
11.02
8.59 15.48
1.0
7.04
16.15
8.31
116.82
9.30 14.62
10.0
7.95
8.31
10.99
11.16
9.86 11.93
masks based on GB correspond to Fl Score of 0.556, compared to 0.638 for masks based on FMA and 0.891 for masks based on Del. The ground truth saliency changes every run (i.e. changes row by row) but is the same for all perturbations (i.e., only one ground truth saliency per row).
A.8 Details On The Sepsis Prediction Task
We train model according to [8] for sepsis prediction on real-world hospital data. Kidger et al. [ feed cubic spline coefficients to the model. Since the computation these coefficients takes long if there missing data in X, they compute and save the coefficients once and simply load them during iterative training and testing. Since we alter the underlying data using our perturbations, we have recompute the cubic spline coefficients every iteration, which is very costly. Computing the coefficients according to Kidger et al. S takes approximately 30 seconds for a single triplet (4, X, d). While this is largely parallelized, this significantly increases the time needed to explain single prediction by the model. As per Table 10 runs to using NeuroEvolution take around 2 hours and 18 minutes. This is roughly the time it takes to conduct one run for the real-world sepsis prediction task. Note that we have already decreased the iterations for PGPE from 2000 to 200. While the reduction 200 iterations cuts computation time, it leads to poorer fitted saliency maps. We have not been able to quantify this drop in performance.
For other models such mtan, where the raw data is fed into the models, perturbing data is much more straight forward and such experiments will be much closer to the Rare Feature and Rare Time examples terms of required compute.
Table 12 shows the A and A2 values for the real-world sepsis prediction task using the NCDE model. Similar to before, we started with too large values of ê and set \ 10A₂ or a 100 We then iteratively reduced A until we reached reasonable performance and then further finetuned X2- Generally, we did not consider alternative values for λ₁ and 22 other than multiplies of 10" for some positive or negative integer π.
For the mtan model, we simply set \ = 1, and X2 = 0 (ablation study above).
Pred: No Sepsis
Pred: Sepsis
Perturbation
X
A
A2
GB
1
0.01
0.01
FMA
0.00001
0,000001
0.00001
0.000001
Del
0.01
10
0.1
Table 12: Regularization parameters for the real-world sepsis prediction task using the NCDE model.
24

---

GB 1
FMA
DEL
True
BFAR SPA9
40
GB 2
FMA
DEL
True
40
3
FMA 3
DEL
True
40
DEL
20
9.8
20
20
0.
7
FMA
DEL
True
20
GB 8
FMA 6
DEL
True
40
GB 9
FMA 9
DEL
True
190
80
GB 10
FMA 10
DEL 10
Truc
40
40
80
20
40
80
20
60
Figure 4: Examples fitted masks using GB, FMA, and for the value-based Rare Feature problem (MLP-F masks trained using NeuroEvolution). The masks correspond to the results is displayed in
Table 2
A.9 Mask Evaluation For Sepsis Prediction Task Using mtan Model
evaluated for every feature of the data often was removed during the real-world sepsis task based the mtan model. If feature removed very oftenn, i.c., retention is low, it is estimated
25

---

to have large effect on the prediction. In Table 13, we ordered all features by retention ratio, i.e., the most important features are listed first. It is more extensive version of T In the Sepsis Imbalance column, we ve calculated how much more often this feature was present for a patient who later becomes septic compared to patient who does not become septic. High values indicate that the shear presence of particular feature might be strong indicator of patient later developing sepsis. This corresponds to decisions of the treating physicians who might decide to only collect certain information if patient is at high risk of developing sepsis.
Feature
Retention Ratio
Sepsis Imbalance
Description
Age
0.7912
1.0000
Patient age (y)
Height
0,7982
1,0000
Height (cm)
AST
0.8063
1.5256
Aspartate transaminase
ICUType
0.8196
1,0000
ICU unit type
ALP
0.8204
1.4433
Alkaline phosphatase
Cholesterol
0,8300
1,4276
Serum cholesterol
RespRate
0,8360
0,5092
Respiratory rate (bpm)
Creatinine
0.8422
1.0123
Serum creatinine
0,8589
1,0100
Potassium (mmol/L.
Mech
0.8597
1.1880
Mechanical ventilation
Na
0,8610
1,0100
Sodium (mmol/L)
PH
0.8656
1.1483
Arterial pH
NISySABP
0.8741
1.0183
Non-invasive systolic BP
Gender
0.8790
1.0000
Sex (F=0, M=l)
MAP
0,8793
1.0734
Mean arterial pressure
DiasABP
0.8846
1.0689
Diastolic BP
Weight
0,8852
1,0000
Weight (kg)
PaO2
0.8902
1.1642
Arterial oxygen (mmHg)
SysABP
0.8935
1.0689
Systolic BP
Temp
0.8946
1.0102
Temperature (?C)
Platelets
0.8956
1.0138
Platelet count
TroponinTT
0.8998
1.4946
Cardiac troponin
HR
0.9038
1.0102
Heart rate (bpm)
GCS
0.9057
1.0102
Glasgow Coma Scale
Urine
0.9058
0.9904
Urine output (mL)
Lactate
0,9093
1,3661
Blood lactate
ALT
0.9104
1.5256
Alanine transaminase
Glucose
0.9114
1.0191
Blood glucose
HCT
0.9174
1.0123
Hematocrit (%)
TroponinI
0,9186
2,6003
Cardiac troponin I
SaO2
0.9190
1.0734
O2 saturation (%)
NIDiasABP
0.9236
1.0200
Non-invasive diastolic BP
FiO2
0.9239
1,2318
Inspired O2 fraction (%
NIMAP
0.9337
1.0200
Non-invasive MAP
HС03
0.9370
1.0115
Bicarbonate (mmol/L)
BUN
0.9421
1.0123
Blood urea nitrogen
Bilirubin
0.9458
1.4518
Total bilirubin
PaCO2
0.9475
1.1642
Arterial CO2 (mmHg)
WBC
0.9551
1.0168
White blood cells
Albumin 0.9648
1.3987
Serum albumin
Mg
0.9721
1.0191
Magnesium (mmol/L.)
Table 13: Features included in the real-world sepsis prediction task ordered by their estimation importance (most important features first) including the retention ratio, indicator of structural bias (Imbalance) of this feature, and short description.
26


Reviews:
[{"review_id": "WasdeZFLbx", "reviewer": "Reviewer_tzvM", "summary": "The paper presents Continask, a novel approach to address the black-box interpretation problem of irregular time series data, specifically focusing on the healthcare, where up to 90% of data may be missing. The authors propose an innovative Deletion perturbation method to quantify missingness importance, and to resolve the non-differentiability issue of perturbation, this paper employs a NeuroEvolution strategy. The approach is validated on synthetic and real-world sepsis prediction tasks.", "questions": "1. Are there other specific methods or optimizations that could be explored to improve the computational efficiency of NeuroEvolution? \n2. Would it be possible to benchmark the performance of MLP-F by comparing it to TimeX (with the same perturbation setup, i.e., modifying only the observations)? A direct comparison could highlight the competitive advantages or limitations of the proposed approach.\n3. How do different hyperparameters affect the quality of the time mask? An ablation study focused on hyperparameters would help clarify which factors are most influential in model performance.\n4. Could the sepsis prediction case study be extended to 20 instances? Increasing the sample size would help strengthen the statistical significance and provide a more robust validation of the method.", "limitations": "1.The slower computational speed of NeuroEvolution compared to gradient descent significantly limits the scalability of the model, particularly in large-scale datasets. This is a key area for improvement in future work.\n2.The impact of Deletion perturbation in scenarios with high missing data is not fully explored. It would be helpful to investigate how this method behaves in such conditions, especially with regard to the realism of the missing samples.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper introduces a post-hoc instance-level explanation framework for irregular time series, which is a valuable contribution, particularly in the healthcare field where interpretability is crucial.\n2. The innovative NeuroEvolution approach effectively addresses the non-differentiability issue in optimization problems, a novel solution compared to traditional gradient descent methods.\n\nWeaknesses:\n1. While NeuroEvolution shows promise, its computational efficiency is a limitation. It is significantly slower than traditional gradient descent methods, making it impractical for larger datasets. This limitation confines the sepsis analysis to only two cases, which could be a major drawback for real-world applications.\n2. The discussion on the reliability of the Deletion perturbation method in high-missing-data scenarios could be expanded. It is unclear how the method performs when realistic samples are missing, which is common in real-world healthcare data.", "was_revised": true, "final_justification": "Most of my concerns have been addressed and the contribution of this paper is clear.", "rebuttal": "We'd like to thank the reviewer for their time and thoughtful comments. We were delighted to hear they found our paper 'innovative' and a 'valuable contribution'. In the below, we hope to answer all raised questions.\n\n**Q** Alternative optimizers\n\n**R**: As several reviewers raised concerns about computational speed, we investigated potential bottlenecks in our code more thoroughly. We found that the main bottleneck for the sepsis task lies in the Neural CDE (NCDE) needing to recalculate cubic spline coefficients at every model interaction. This is particularly expensive for irregular data (i.e., data with gaps), which limits parallelization.\n\nWhile we noted this as a potential issue in the original manuscript, we had underestimated its impact. To better understand this, we trained a multi-time attention network (mTAN) on the same sepsis task. Since mTAN processes raw time series directly, model interactions are significantly faster. This resulted in a 70× speed-up, reducing a process that previously required ~2 hours with the NCDE model to under 2 minutes with mTAN. Importantly, we only changed the model we are explaining, not Contimask. Hence, the slow speeds were related to the NCDE model, not Contimask. This generally showcases that Contimask can explain predictions within a much more practical time frame. Any method that uses perturbations on raw time series (especially with gaps, such as irregular time series) will experience a severe slow-down when explaining NCDE models given the expensive recomputation of the subic spline coefficients.\n\nBelow, we report the average change in model output for 100 sepsis-positive and 100 sepsis-negative cases as predicted by mTAN, rather than the NCDE model (higher is better). While the Del perturbation outperforms FMA and GB at explaining cases that are predicted as sepsis-positive, GB performs best explaining cases predicted as sepsis-negative. We will include this table in the final version of the manuscript.\n\n#### Table 1: sepsis mTAN\n\n|       | **Sepsis**      |                | **No-sepsis**    |                |\n|-------|-----------------|----------------|------------------|----------------|\n|       | Del Odds Change | Imp Odds Change| Del Odds Change  | Imp Odds Change|\n| GB    | 4.41            | 10.45          | 6.29             | 11.80          |\n| FMA   | 4.16            | 3.41           | 7.26             | 7.19           |\n| Del   | 7.88            | 2.26           | 4.57             | 5.90           |\n\nWe also extended the evaluation of the sepsis task with the NCDE model to 25 sepsis-positive and 25 sepsis-negative cases. Please find the results in the Table below, which show the same trend: while Del performs strongest for cases predicted to develop sepsis, it performs weakest on cases predicted to not develop sepsis.\n\n#### Table 2: sepsis NCDE\n|       | **Sepsis**      |                | **No-sepsis**    |                |\n|-------|-----------------|----------------|------------------|----------------|\n|       | Del Odds Change | Imp Odds Change| Del Odds Change  | Imp Odds Change|\n| GB    | 0.25            | 0.26           | 4.01             | 4.05           |\n| FMA   | 0.26            | 0.23           | 5.15             | 5.16           |\n| Del   | 4.97            | 0.21           | 2.72             | 3.94           |\n\nAll models were trained with λ1=1, and λ2=0 (ablation study below).\n\nThis difference in performance between Del and FMA/GB for sepsis-positive and -negative cases highlights Contimask's unique capabilities. As raised also by reviewr ykqd, we conducted a qualitative analysis into the returned masks. Using the Del perturbation, Contimask oftentimes removes entire sensor streams (i.e. channels), which drastically reduces the probability (according to mTAN and NCDE) that a patient becomes septic.\n\nInterestingly, there seems to be a structural difference in the data between patients who later develop sepsis, and those who don't: patients who later become septic are up to 2.6 times more likely to have observations in the ALT, AST, and Troponin channel (blood markers used to identify/monitor sepsis). Using the Del perturbation, Contimask reveals that both mTAN and NCDE are both heavily influenced by data being observed in these channels and use this as a main discriminator of whether a patient will become septic. Of course, whether these channels are observed is decided by the treating physicians who decide which information to monitor. Thereby, the models do not actually provide the clincians with any meaningful information but oftentimes just use the clinicians judgement as a discriminator.\n\nFor cases prediced as septic-negative, removing data does not increase the predicted probability of developing sepsis as strongly and value altering perturbations (i.e., FMA and GB) produce better results.\n\nWe believe these findings highlight Contimask's unique capabilites to uncover the reliance of irregular time series on whether data was observed at all, especially if using the Del perturbation. In clinical settings, this can help to reveal that a trained model does not actually uncovered meaningful patterns to help the treating physicians.\n\nFurther optimizations are possible in future work. Although we ultimately selected the EvoTorch library after an extensive comparison of open-source alternatives, it remains sub-optimal because a significant portion of the PGPE optimizer’s computation still runs on the CPU.\n\nAnother alternative would be to revert to using a single Tensor over a small feedforward network and apply genetic algorithms as a gradient-free optimization framework. Here the Tensor would be binary. While this approach can be efficient for short time series with few channels, it becomes computationally expensive for longer time series with many channels, as the parameter count would grow linearly with both length and channel number.\n\n**Q:** Benchmark against TimeX\n\n**R:** Unfortunately, this is not directly possible. While TimeX, a baseline primarily developed for regular time series, could in principle be used to explain models such as mTAN, we were unable to reproduce its results with the provided repository. The original implementation relies on a graph-neural-network–based model for irregular time series, but links for data download are currently broken, and it is unclear how to train the model or integrate a model like mTAN with the explainer. In addition, TimeX does not perturb data in the classical sense: it requires access to model internals and inspects the latent space. For models such as NCDE, this is likely tricky given the CDE solve. Since Contimask only queries the model and does not require access to model internals, a direct comparison would not be entirely fair, but we agree that it would be interesting to see how access to model internals improves explainability.\n\n\n**Q:** Hyperparameter ablation study\n\n**R:** We have included an ablation study on λ1 and λ2 in the Tables below for Del, FMA, and GB for the sepsis task using mTAN. These are computed for 10 cases predicted as sepsis-positive. The first value corresponds to Del odds change, and the second to Val odds change. Please note that results are at times higher than what is reported across 100 sepsis-positive cases since the top 10 predictions are easier to explain than the top 100 predictions (more confident predictions usually have more pronounced features).\n\n#### Table 3: D\n\n| λ1      | λ2 = 0        | λ2 = λ1 / 100 | λ2 = λ1 / 10 |\n|---------|---------------|---------------|--------------|\n| **0.01** | 6.24 \\| 7.06  | 10.34 \\| 7.94 | 5.95 \\| 8.27 |\n| **0.1**  | 9.13 \\| 8.44  | 9.91 \\| 7.81  | 6.51 \\| 6.82 |\n| **0.5**  | 10.75 \\| 8.83 | 9.05 \\| 6.89  | 8.09 \\| 7.42 |\n| **1.0**  | 12.92 \\| 8.42 | 13.64 \\| 7.19 | 10.99 \\| 8.48 |\n| **10.0** | 11.24 \\| 6.81 | 8.50 \\| 7.55  | 8.69 \\| 7.30 |\n\n\n#### Table 4: FMA\n\n| λ1      | λ2 = 0        | λ2 = λ1 / 100 | λ2 = λ1 / 10 |\n|---------|---------------|---------------|--------------|\n| **0.01** | 6.76 \\| 7.76  | 6.01 \\| 7.81  | 10.08 \\| 7.48 |\n| **0.1**  | 7.78 \\| 8.15  | 9.50 \\| 8.95  | 9.68 \\| 9.31  |\n| **0.5**  | 7.16 \\| 6.83  | 10.47 \\| 8.17 | 12.63 \\| 9.49 |\n| **1.0**  | 8.10 \\| 6.95  | 10.24 \\| 9.15  | 11.83 \\| 7.15 |\n| **10.0** | 11.74 \\| 7.04 | 11.63 \\| 6.56 | 11.43 \\| 7.71 |\n\n\n#### Table 5: GB\n\n| λ1      | λ2 = 0        | λ2 = λ1 / 100 | λ2 = λ1 / 10 |\n|---------|---------------|---------------|--------------|\n| **0.01** | 5.86 \\| 15.78 | 5.54 \\| 15.58 | 8.91 \\| 15.16 |\n| **0.1**  | 4.64 \\| 15.22 | 5.85 \\| 14.84 | 9.14 \\| 16.04 |\n| **0.5**  | 6.05 \\| 16.92 | 9.27 \\| 17.02 | 8.59 \\| 15.48 |\n| **1.0**  | 7.04 \\| 16.15 | 8.31 \\| 16.82 | 9.30 \\| 14.62 |\n| **10.0** | 7.95 \\| 8.31  | 10.99 \\| 11.16| 9.86 \\| 11.93 |\n\nContiMask is generally robust to moderate hyper-parameter changes.\n\nIn the sepsis prediction task, the mask must cover a fixed input proportion. Very small λ1 and λ2  values cause the area constraint to be ignored, while very large values slow convergence since the changes in model output are overpowered. For the sepsis prediction task, the model output is in the range of -15 to 15, and λ1=1 forms a good starting point. Adjusting λ1 and λ2  one to two orders of magnitude below the threshold yields little additional change.\n\n**Q:** The impact of Deletion perturbation in scenarios with high missing data\n\n**R:** The sepsis prediction task is actually a scenario with a high ratio of missing data. As outlined above, while certain channels, such as heart rate, are almost continuously observed, others (e.g., bloodtests such as AST, ALT, Troponin) are missing for a large fraction of patients. It is in these scenarios that Contimask with the Del perturbation is particularly effective at detecting when the mere presence of data influences the model’s predictions as shown by our findings related to the channels AST, ALT, Troponin in the sepsis task (see above)."}, {"review_id": "VHCNkCQARJ", "reviewer": "Reviewer_AND9", "summary": "This paper addresses the challenge of explaining black-box models for irregular time series (ITS) data, which are prevalent in domains like healthcare, where observations occur at non-uniform intervals with high missingness rates. The authors identify limitations in existing saliency methods designed for regular time series, which fail to account for the timing/structure of observations in ITS. They propose ​Contimask, a framework that:\n\n- Translates perturbation techniques (e.g., Gaussian blur, moving average) to the continuous-time domain of ITS.\n- Introduces a novel ​Deletion perturbation​ (Del) that simulates unobserved data points via Bernoulli sampling, enabling the detection of saliency related to missingness and time intensity.\n- Utilizes ​NeuroEvolution​ (PGPE algorithm) to optimize non-differentiable perturbations, parameterizing masks as MLPs with Fourier features (MLP-F) for efficiency.", "questions": "Q1. Contimask was tested on only two sepsis cases due to computational costs. How would it perform on larger datasets (e.g., 100+ patients)? Can the authors provide evidence of scalability (e.g., parallelization, approximations) or runtime metrics (e.g., avg. time per mask)?\n\nQ2. Del removes data points via Bernoulli sampling, potentially creating unrealistic \"gaps\" if adjacent timestamps are masked. How does this affect fidelity in clinical contexts (e.g., where missingness patterns are structured)?\n\nQ3. λ₁ and λ₂ in Eq. 5 critically affect mask sparsity/smoothness. How were they optimized for sepsis tasks? Can the authors show ablation studies on synthetic data?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Quality:​​\n\n- Strengths: The problem formulation is well-motivated, especially for healthcare applications. The integration of NeuroEvolution to handle non-differentiable perturbations is innovative. Experiments on synthetic data validate Del’s ability to detect time-based saliency (Table 2-3).\n\n- ​Weaknesses: Experimental validation is ​severely limited. The sepsis task evaluates only two cases due to computational constraints (Sec. 4), undermining statistical significance. Results in Table 4 show marginal improvements, but lack rigorous comparison across multiple instances or baselines (e.g., TimeX++ for ITS). The claim of \"significant improvements\" is overstated given the tiny sample size.\n\n### Clarity:​​\n\n- ​Strengths: The paper is generally well-structured, with a clear background on ITS models (Sec. 2.2) and method details (Sec. 3).\n\n- Weaknesses: Critical details are ambiguous. For example: (i) How hyperparameters (λ₁, λ₂) were tuned for sepsis tasks is not specified (only mentioned as \"adapted version of Eq. 5\"); (ii) The computational overhead of NeuroEvolution (vs. gradient-based methods) is discussed but not quantified.\n\n### Significance:​​\n\n- Strengths: The focus on ITS explainability fills a genuine gap, especially for clinical applications. The Deletion perturbation is a novel concept for capturing missingness importance.\n\n- ​Weaknesses: Practical utility is ​questionable. Contimask’s reliance on NeuroEvolution makes it prohibitively slow (2 hours per run for sepsis cases), limiting real-world adoption. No comparison with SOTA ITS explainers (e.g., TimeX) weakens impact claims.\n\n### Originality:​​\n\n- Strengths: The extension of perturbations to continuous time and the Del operator are non-trivial contributions. MLP-F masks offer parameter efficiency (Table 1).\n\n- ​Weaknesses: NeuroEvolution for explainability is not new (e.g., evolution strategies in RL). The translation of FMA/GB perturbations to ITS (Sec. 3) feels incremental, lacking fundamental innovation beyond prior work (DynaMask).", "was_revised": true, "final_justification": "Most of my concerns have been well addressed by the author's thorough responses. Therefore, I have improved my score.", "rebuttal": "We'd like to thank the reviewer for their time and thoughtful comments, which we have incorporated in our revision to refine and strengthen our paper. We were delighted to hear they found our paper 'well-motivated' and 'innovative' (AND9). In the below, we hope to answer all raised questions.\n\n**Q:** Quantification of computational overhead of NeuroEvolution\n\n**R:** We quantified the computational overhead in terms of runtime in Table 5 in the Supplementary Material. In a differentiable set-up based on the FMA perturbation, we found NeuroEvolution to take around 4 times longer than gradient-based methods as implemented for instance in DynaMask across 10 test runs. Interestingly, gradient-based optimization not always converge in these experiements, however.\n\n**Q:** Computational cost of Contimask for sepsis task and evluation on larger dataset\n\n**R:** As several reviewers raised concerns about computational speed, we investigated potential bottlenecks in our code more thoroughly. We identified that the main bottleneck for the sepsis task lies in the design of the Neural CDE (NCDE) model. This model requires cubic spline coefficients as input rather than the raw time series. Consequently, at every model interaction, cubic spline coefficients must be recalculated, which is computationally expensive—particularly in the presence of missing data, which limits parallelization.\n\nWhile we noted this as a potential issue in the original manuscript, we had underestimated its impact. To better understand this, we trained a multi-time attention network (mTAN) on the same sepsis task. Since mTAN processes raw time series directly, model interactions are significantly faster. This resulted in a 70× speed-up, reducing a process that previously required ~2 hours with the NCDE model to under 2 minutes with mTAN. Importantly, we only changed the model we are explaining. Hence, the slow speeds were related to the NCDE model, not Contimask.\n\nBelow, we report the average change in model output for 100 sepsis-positive and 100 sepsis-negative cases predicted by mTAN, rather than the NCDE model. While the Del perturbation outperforms FMA and GB for sepsis-positive cases, GB performs best for sepsis-negative cases. We will include this table in the final version of the manuscript.\n|       | **Sepsis**      |                | **No-sepsis**    |                |\n|-------|-----------------|----------------|------------------|----------------|\n|       | Del Odds Change | Imp Odds Change| Del Odds Change  | Imp Odds Change|\n| GB    | 4.41            | 10.45          | 6.29             | 11.80          |\n| FMA   | 4.16            | 3.41           | 7.26             | 7.19           |\n| Del   | 7.88            | 2.26           | 4.57             | 5.90           |\n\nFurther, we have also extended the sepsis task based on the NCDE model to 25 cases each: 25 cases that were intially predicted to develop sepsis, and 25 cases that were intially not predicted to develop sepsis. The below table confirms the results depicted in the original paper and observed also in the Table above. The Del perturbation works best at explaining cases predicted to be sepsis-positive, while FMA/GB work better for cases predicted not to develop sepsis. All models were trained with λ1=1, and λ2=0 (ablation study below).\n\n|       | **Sepsis**      |                | **No-sepsis**    |                |\n|-------|-----------------|----------------|------------------|----------------|\n|       | Del Odds Change | Imp Odds Change| Del Odds Change  | Imp Odds Change|\n| GB    | 0.25            | 0.26           | 4.01             | 4.05           |\n| FMA   | 0.26            | 0.23           | 5.15             | 5.16           |\n| Del   | 4.97            | 0.21           | 2.72             | 3.94           |\n\n\nWhy the Del perurbation works better for cases predicted to be sepsis-positive relates actually relates to the question raised below. Please see our response below.\n\n**Q:** Gaps created by Bernoulli sampling and fidelity in clinical contexts (where missingness patterns might be structured)\n\n**R**: To remove patches of data, i.e. to create larger gaps, and potentially remove an entire sensor stream was in fact one of our motivations to use Bernoulli sampling in the first place. Related to the review by yKqd03, we investiagted returned masks, which showcased the strength of Contimask and the Deletion perturbation.\n\nIn our analysis of the mTAN model, we observed for many sepsis-positive cases that Contimasks using the Deletion perturbation would remove entire sensor streams (mainly blood markers highly predictive of sepsis such as ALT, AST, Troponin), which led to drastically lower probabilities of sepsis (accroding to the models). Interestingly, sepsis-positive patients are up to 2.6 times more likely to have data for the aforementioned sensor streams than sepsis-negative patients. This suggests that clinicians tend to adapt monitoring setups based on patient condition and patients at higher risk of sepsis are monitored more closely.\n\nUsing the Del perturbation, Contimask reveals that both mTAN and NCDE are both heavily influenced by data being observed in these channels and use this as a main discriminator of whether a patient will become septic. Of course, whether these channels are observed is decided by the treating physicians who decide which information to monitor. Thereby, the models do not actually provide the clincians with any meaningful information but oftentimes just use the clinicians judgement as a discriminator.\n\nFor cases prediced as septic-negative, removing data does not increase the predicted probability of developing sepsis as strongly and value altering perturbations (i.e., FMA and GB) produce better results.\n\nWe believe these findings highlight Contimask's unique capabilites to uncover the reliance of irregular time series on whether data was observed at all, especially if using the Del perturbation. In clinical settings, this can help to reveal that a trained model does not actually uncovered meaningful patterns to help the treating physicians.\n\nRemoving entire patches of data is crucial for spotting this.\n\n**Q:** Tunig of λ₁ and λ₂ in Eq. 5 for the sepsis tasks\n\n**R:** We thank the reviewer for raising this. We adapted equation 5 such that the loss term involving λ1 changes from:\n\n$\\lambda_1 \\sum_C \\int_0^T(1-m(u)_c) du$\n\nto:\n\n$\\lambda_1 (TS - \\sum_C \\int_0^T( m(u)_c ) du)$.\n\nFor the sepsis task, the goal is to maximize the divergence in prediction while the mask should only alter a previously defined target size $TS$, say 10% of all observed timepoints. The goal for the previous tasks on synthetic data was to keep the predition unchanged while altering as many datapoints as possible.\n\nThe full obective for the sepsis task then becomes:\n$\\min_m -\\mathcal{L}(f(t_n, x_n, d_n), (\\tilde{t}_n, \\tilde{x}_n, \\tilde{d}_n)) + \\lambda_1 (TS - \\sum_C \\int_0^T( m(u)_c ) du) + \\lambda_2 \\sum_C \\int_0^T \\vert m(u)'_c \\vert du$.\nWe will add this in the final version of the manuscript. Intitially, we removed it to save space.\n\nTuning 𝜆1 and 𝜆2 is generally straightforward. If no target mask size is enforced, setting λ1 too high can dominate the model output and collapse the mask. Even with a target size, overly large values may still slow convergence or reduce performance. In the sepsis task, where model outputs ranged from −15 to 15, λ1=1 produced reasonable results, so we conducted a focused search around this value, with 𝜆2 set to 0, λ/10, or λ1/100.\n\nThe tables below show an ablation study for 10 cases predicted to bevome septic by mTAN. Each cell reports Del odds change and Imp odds change. Note that performance can appear higher than in the full evaluation (100 cases), as these top-ranked predictions are typically easier to explain due to more pronounced features.\n\n#### Table 1: D\n\n| λ1      | λ2 = 0        | λ2 = λ1 / 100 | λ2 = λ1 / 10 |\n|---------|---------------|---------------|--------------|\n| **0.01** | 6.24 \\| 7.06  | 10.34 \\| 7.94 | 5.95 \\| 8.27 |\n| **0.1**  | 9.13 \\| 8.44  | 9.91 \\| 7.81  | 6.51 \\| 6.82 |\n| **0.5**  | 10.75 \\| 8.83 | 9.05 \\| 6.89  | 8.09 \\| 7.42 |\n| **1.0**  | 12.92 \\| 8.42 | 13.64 \\| 7.19 | 10.99 \\| 8.48 |\n| **10.0** | 11.24 \\| 6.81 | 8.50 \\| 7.55  | 8.69 \\| 7.30 |\n\n\n#### Table 2: FMA\n\n| λ1      | λ2 = 0        | λ2 = λ1 / 100 | λ2 = λ1 / 10 |\n|---------|---------------|---------------|--------------|\n| **0.01** | 6.76 \\| 7.76  | 6.01 \\| 7.81  | 10.08 \\| 7.48 |\n| **0.1**  | 7.78 \\| 8.15  | 9.50 \\| 8.95  | 9.68 \\| 9.31  |\n| **0.5**  | 7.16 \\| 6.83  | 10.47 \\| 8.17 | 12.63 \\| 9.49 |\n| **1.0**  | 8.10 \\| 6.95  | 10.24 \\| 9.15  | 11.83 \\| 7.15 |\n| **10.0** | 11.74 \\| 7.04 | 11.63 \\| 6.56 | 11.43 \\| 7.71 |\n\n\n#### Table 3: GB\n\n| λ1      | λ2 = 0        | λ2 = λ1 / 100 | λ2 = λ1 / 10 |\n|---------|---------------|---------------|--------------|\n| **0.01** | 5.86 \\| 15.78 | 5.54 \\| 15.58 | 8.91 \\| 15.16 |\n| **0.1**  | 4.64 \\| 15.22 | 5.85 \\| 14.84 | 9.14 \\| 16.04 |\n| **0.5**  | 6.05 \\| 16.92 | 9.27 \\| 17.02 | 8.59 \\| 15.48 |\n| **1.0**  | 7.04 \\| 16.15 | 8.31 \\| 16.82 | 9.30 \\| 14.62 |\n| **10.0** | 7.95 \\| 8.31  | 10.99 \\| 11.16| 9.86 \\| 11.93 |\n\nGenerally, if λ1 is not too large, decreasing it by 1 or 2 orders of magnitude (and λ2 in accordance) does not have a huge effect on the returned masks. Slower convergence could also be compensated for by increased iterations. In this ablation study though we fixed iterations at 200 as in the general sepsis task. Outside of the range λ1=10 and λ1=0.01, we noted a performance drop off when keeping iterations fixed at 200.\n\nWe will add this to our manuscript as well."}, {"review_id": "QCHiEORkJS", "reviewer": "Reviewer_yKqd", "summary": "The authors provide a novel interpretability method, Contimask, for irregular time-series data which is sampled irregularly and can having missing values. The algorithm works by perturbing the data with a perturbation function f to find which changes in input have the largest effect on a model's prediction. While machine learning currently focuses on gradident descent, this assumes access to a differentiable function f, which the authors don't assume access to so they leverage a genetic algorithm. This allows them to define novel perturbation strategies that are not differentiable but effective. They test their algorithm on 3 datasets with 2 baselines.", "questions": "1. Can you provide a qualitative display of the interpretability enabled by your method? This seems important for a method in the interpretability literature.\n2. Can you provide more baselines?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "quality. The paper is clear and high-quality.\n\nclarity: the paper is clear. more could be see about how much computation and time is needed for running experiments given that speed is a limitation of the method.\n\nsignificance: the paper could do more comparisons of baselines. 2 does not seem sufficient. I appreciate the ablations on the method between gradient descent and neuroevolution approaches.  \n\noriginality: this is the first method for irregular time-series.", "was_revised": true, "final_justification": "They have added more baselines and promised to add the kind of visualization I think is important for this kind of method.", "rebuttal": "We'd like to thank the reviewer for their time and thoughtful comments, which we have incorporated in our revision to refine and strengthen our paper. We were delighted to hear they found our paper 'clear and high-quality'. In the below, we hope to answer all raised questions.\n\n**Q:** Qualitative display of interpretability\n\n**R:** We thank the reviewer for this suggestion. While we previously included visualizations of fitted masks on synthetic data, we agree that mask visualizations for the sepsis prediction task would also be of great value.\n\nAs several reviewers raised concerns about computational speed, we investigated potential bottlenecks in our code more thoroughly. We identified that the main bottleneck for the sepsis task lies in the computation of cubic spline coefficients as part of the Neural CDE (NCDE) model. While we noted this as a potential issue in the original manuscript, we had underestimated its impact. To better understand this, we trained a multi-time attention network (mTAN) on the same sepsis task. Since mTAN processes raw time series directly, model interactions are significantly faster. This resulted in a 70× speed-up, reducing a process that previously required ~2 hours with the NCDE model to under 2 minutes with mTAN. We have run a much more extensive evaluation using the mTAN model and conducted also a qualitative analsys.\n\nIn our qualitative analysis of the mTAn-model sepsis task, we observed particularly interesting cases where entire sensor streams (e.g., blood markers highly predictive of sepsis such as ALT, AST, Troponin) were removed by Contimask leading to drastically lower probabilities of sepsis. Interestingly, sepsis-positive patients are up to 2.6 times more likely to have data for the aforementioned sensor streams than sepsis-negative patients. This suggests that clinicians tend to adapt monitoring setups based on patient condition and patients at higher risk of sepsis are monitored more closely.\n\nContiMask often removes these sensor streams for sepsis-positive cases, leading to dramatic changes in the model’s predictions. This reveals that the trained mTAN and NCDE model may rely heavily on patterns related to clinician monitoring decisions, rather than intrinsic patient data, which limits its practical utility. The model therefore provides little additional value for a treating clinician since it uses a decision made by a clinician as its main discriminator. The ability to spot this is something unique to Contimask and the Deletion perturbation.\n\nFor sepsis-negative cases, data removal is less effective in altering predictions because these patients are typically monitored less extensively. In such cases, value-altering perturbations are more effective than the Deletion perturbation at changing the model’s predictions.\n\nWe will include these visualizations and a more detailed discussion in the final version of the manuscript.\n\n\n**Q:** baselines\n\n**R:** We extended our experiments to also explain a multi-time attention network (mTAN) in addition to the Neural CDE. TimeX would most liekly struggle to explain the NCDE model given the CDE solve. While TimeX could, in principle, explain models such as mTAN, we were unable to reproduce its results using the provided repository. The original implementation relies on a graph-neural-network–based model for irregular time series, but links for data download are broken, and it is unclear how to train the model or integrate a model like mTAN with the explainer.\n\nMoreover, TimeX requires access to model internals and inspects the latent space, whereas Contimask only queries the model, making a direct comparison less fair. Nonetheless, this would be an interesting direction for future work. At present, no immediately applicable baseline exists that performs post-hoc instance explanation for time series models without the need to access model internals."}]

Author rebuttal:

Meta-review: This paper study explainability for black-box models for irregular time series data. The authors propose a method for perturbing input time series while incorporating irregularity-specific features into the saliency map. They also propose a genetic algorithm to learn the saliency maps without access to model gradients, potentially increasing the applicability. In the experiments, the proposed method outperforms two baselines on three datasets. The paper was overall received well by reviewers, who found the problem important and new, the solution clever and interesting, and overall found the results convincing. However, they also noted some key limitations. The main drawback from each reviewer was that the proposed method is extremely slow. However, during the discussion phase, this was somewhat resolved as the slow runtimes were tied to the NCDE model. Using a faster model, the authors expanded the experiments significantly. This should be included in the revised paper. Reviewers also noted that some experimental details were lacking and the work needed a hyperparameter study, which the authors included during the discussion period. Finally, reviewers noted there were limited explainability baselines. This was also resolved during the discussion period, as there aren’t methods to directly apply in this case. It would also strengthen the paper to put some examples in the main paper and potentially conduct a small human study to further verify the quality of the saliency maps. Overall, the paper studies an interesting and understudied problem with potential for the community to build on it.
Decision: Accept (poster)