Inherently Interpretable Time Series Classification via Multiple Instance Learning
Authors: Joseph Early
Gavin Cheung
Kurt Cutajar
Hanting Xie
Jas Kandola
Niall Twomey
Venue: ICLR 2024 spotlight (2024)
Keywords: Multiple Instance Learning
Time Series Classification
Interpretability
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Conventional Time Series Classification (TSC) methods are often black boxes that obscure inherent interpretation of their decision-making processes. In this work, we leverage Multiple Instance Learning (MIL) to overcome this issue, and propose a new framework called MILLET: Multiple Instance Learning for Locally Explainable Time series classification. We apply MILLET to existing deep learning TSC models and show how they become inherently interpretable without compromising (and in some cases, even improving) predictive performance. We evaluate MILLET on 85 UCR TSC datasets and also present a novel synthetic dataset that is specially designed to facilitate interpretability evaluation. On these datasets, we show MILLET produces sparse explanations quickly that are of higher quality than other well-known interpretability methods. To the best of our knowledge, our work with MILLET is the first to develop general MIL methods for TSC and apply them to an extensive variety of domains.

Published as a conference paper at ICLR 2024
INHERENTLY INTERPRETABLE TIME SERIES CLASSIFICATION VIA MULTIPLE INSTANCE LEARNING
Joseph Early; Gavin KC Cheung, Kurt Cutajar, Hanting Xie! Jas Kandola & Niall Twomey?
Corresponding authors: J.A. Earl 1 1 soton. ac. uk; n wittwmee@eee @amaaeyy@aaaaaee zon. Co. .uk
ABSTRACT
Conventional Time Series Classification (TSC) methods are often black boxes that obscure inherent interpretation of their decision-making processes. In this work, we leverage Multiple Instance Learning (MIL) to overcome this issue, and pro- pose new framework called MILLET: Multiple Instance Learning for Locally Explainable Time series classification. We apply MILLET to existing deep learn- ing TSC models and show how they become inherently interpretable without com- promising (and in some cases, even improving) predictive performance. We eval- uate MILLET on 85 UCR TSC datasets and also present a novel synthetic dataset that is specially designed to facilitate interpretability evaluation. On these datasets, we show MI MILLET produces sparse explanations quickly that are of higher quality than other well-known interpretability methods. To the best of our knowledge our work with MILLET, which is available on GitHub is the first develop general MIL methods for TSC and apply them to an extensive variety of domains.
Example Time Series Spikes
Serios
Conventional TSC
MILLET: Multiple Instance Learning for Locally Explainable Time Series Classification
Probesily
Poobasilyy
Hastang
Time
Class
Class
Figure I: Conventional TSC techniques (left) usually only provide class-level predictive probabili- ties. In addition, our proposed method (MILLET, right) also highlights class-conditional discrimi- natory motifs that influence the predicted class. In the heatmap, green regions indicate support for the predicted class, red regions refute the predicted class, and darker regions are more influential.
INTRODUCTION
Time Series Classification (TSC) is the process of assigning labels to sequences of data, and oc- curs in wide range of settings examples from the popular UCR collection of datasets include predicting heart failure from electrocardiogram data, and identifying household electric appliance usage from electricity data (Dau et al., 2019). Each of these domains have their own set of class- conditional discriminatory motifs (the signatures that determine the class of a time series). Deep Learning (DL) methods have emerged as a popular family of approaches for solving TSC problems. However, we identify two trawbacks with these conventional supervised learning approaches: 1) representations are learnt for each time point in a time series, but these representations are then lost through an aggregation process that weights all time points equally, and 2) these methods are black boxes that provide no inherent explanations for their decision making, i.e. they cannot localise the cass-conditional diseriminatory motifs. These drawbacks not only limit predictive performance, but also introduce barriers to their adoption in practice as the models are not transparent.
To mitigate these shortcomings, take alternative view of DL for TSC, approaching it Multiple Instance Learning (MIL.) problem. MIL. is a weakly supervised learning paradigm in which collection (bag) of elements (MIL. instances) all share the same label. In the context of TSC, bag is a time series of data over contiguous interval. In the MIL setting, the learning objective is to assign class labels unlabelled bags of time series data whilst also discovering the salient motifs
* Unnierriity of Southampton, UK work completed during internship at Amazon Prime Video, UK. 1 Amazon Prime Video, UK httpps:/ // thub. com/ JAKa 1 //MLI/MI wwwiimeeeeeeaeeeee factiion

---

Published as a conference paper at ICLR 2024
within the time series that explain the reasons for the predicted class. As explore in this work, MIL is well-suited to overcome the drawbacks identified above, leading to inherent interpretability without compromising predictive performance (even improving it in some cases). We propose new general framework applying MIL. to TSC called MILLET: Multiple Instance Learning for Locally Explainable Time series classification. Demonstrative MILLE" model outputs are depicted in Fig. 1.
MIL is well-suited to this problem setting since it was developed for weakly supervised contexts, can be learnt in an end-to-end framework, and boasts many successes across several domains. Fur- thermore, MIL has the same label specificity as TSC: labels are given at the bag level, but not at the MIL instance level. To explore the intersection of these two areas, we propose plug-and-pe cepts that are udapted from MIL and applied to existing TSC approaches (in this work, DL models?), Furthermore, to aid in our evaluation of the interpretability of these new methods, we introduce a new synthetic TSC dataset, WebTraffic, where the location of the class-conditional discriminatory motifs within time series are known. The time series shown in Fig. 1 is sampled from this dataset.
con-
Our key contributions are follows:
1. We propose MILLET, a TSC framework that utilises MIL to provide inherent interpretabil- ity without compromising predictive performance (even improving it in some cases).
2. We design plug-and-ple MIL methods for TSC within MILLET.
3. We propose a new method of MIL. aggregation, Con joncttitii ive pool ing, that outper- forms existing pooling methods in our TSC experiments.
4. We propose and evaluate 12 novel MILLET models on 85 univariate datasets from the UCR TSC Archive (Dau et al., 2019), as well as novel synthetic dataset that facilitates better evaluation of TSC interpretability.
BACKGROUND AND RELATED WORK
Time Series Classification While a range of TSC methods exist, in this work we apply MII to DL. TSC approaches. Methods in this family are effective and widely used (Ismail Fawaz et al., 2019; Foumani et al., 2023); popular methods include Fully Convolutional Networks (FCN), Residual Networks (ResNet), and InceptionTine deed, a recent TSC survey, Bake Off Redux (Middlehurst et al., 2023), found InceptionTing to be competitive with SOTA approaches such as the ensemble method HIVE-COTE 2 (HC2; Middle- hurst et al., 2021) and the hybrid dictionary-convolutiee Dempster et al., 2023). Although the application of Matrix Profile for TSC also yields inherent in- terpretability (Yeh et al., 2017; Guidotti & D' O'Onofrio, 2021), we choose to focus on DL approaches due to their popularity, strong performance, and scope for improvement (Middlchurst et al., 2023).
(Wang et al., 2017: Ismail Fawaz et al., 2020). In-
method Hydra-MultiRocker (Hydra-MP)
Multiple Instance Learning In its standard assumption, MIL is binary classification problem: bag is positive if and only if at least one of its instances is positive (Dietterich et al., 1997). As we are do not constrain it to any specific MIL assumption except that there are temporal relationships, i.c. the order of instances within bags matters (Early et al., 2022: Wang al., 2020). As we explore in Sec. 3.4, this allows us to use positional encodings in our MI LIET methods. Although the application of MIL to TSC has been explored prior to this study, earlier work focused on domain -ppeifc problems such as intensive care in medicine and human activity recognition (Dennis et al., 2018; Janakiraman, 2018; Poyiadzi et al., 201 8; Poyiadzis et al., 2019; Shanmugam et al., 2019). Furthermore, existing work considers MII. as its own unique approach separate from existing TSC methods. The work most closely related MIL TSC framework specifically
designing MILLET to be general and widely applicable TSC approach,
to ours is Zhu et al. (2021), which proposes an uncertainty- designed for long time series (marine vessel tracking), but without the generality and flug-and--lac nature of MILLET. Therefore, to the best of our knowledge, our work with MILLET is the first to apply MIL to TSC in more general sense and to do across an extensive variety of domains.
Interpretability TSC interpretability methods can be grouped into several categories (Theissler et al., 2022)- in this work we focus on class-wise time point attribution (saliency maps), i.c. identi- fying the discriminatory time points in time series that support and refute different classes. This is a form of local interpretation, where model drcision-making is explained for individual time series
While we focus on DI. TSC in this work, envision that our M LET framework can be applied to other TSC approaches in the future, such as the ROCKET family of methods (Dempster al., 2020; 2023).
2

---

Published as a conference paper at ICLR 2024
(Molnar, 2022). It also aligns with MIL interpretability proposed by Early et al. (2021): which are the key MIL instances in a bag, and what outcomes do they support/refute? MILLET facilitates interpretabilitty by inherently enhancing existing TSC approaches such that they provide interpre- tations alongside their predictions with a single forward pass of the model. This is in contrast to perturbation methods such as LIME (Ribeiro al., 2016), SHAP (Lundherg & Lee, 2017), Oc- clusion Sensitivity (Zeiler & Fergus, 2014), and MILLI (Early et al., 2021), which are much more expensive run (often requiring 100+ forward passes per interpretation). An interpretability ap- proach that can be run with single forward pass is Class Activation Mapping (CAM) (Zhou et al., 2016; Wang al., 2017). It uses the model's s weights to identify discriminatory time points, and serves as benchmark in this work. For more details existing TSC interpretability methods and their evaluation metrics, see App. A.I, Theissler et al. (2022), and Simić et al. (2021).
METHODOLOGY
To apply MIL to TSC, propose the broad framework MILLET: Multiple Instance Learning for Locally Explainable Time series classification. We advocate for the use of MIL in TSC as it natural fit that provides inherent interpretability (explanations for free) without requiring any additional labelling beyond that already provided by existing TSC datasets (we further discuss our motivation for using MIL in App. A).
3.1
THE MILLET FRAMEWORK
A TSC model within our MILLET framework has to satisfy three requirements:
Time Series as MIL Bags Input data consists of time series, X₁, where a time and is the sample index.² Each time step is a c-dimensional vector, where e is the number of channels in the time series in this work we focus on univariate time series (c = 1) and assume all time series a dataset have the same length. We consider each time series as a MIL bag. meaning each time point is MIL instance.* 4 A time series bag can be denoted as X; E exe and each bag has an associated bag-level label Y, which is the original time series label. There is also the concept of MIL instance labels {u/ 8 ..... vf }. but these are not provided for most MIL datasets (like the absence of time point labels in TSC). Framing TSC MIL problem allows obtain interpretability by imposing the next requirement.
Requirement 1:
series formed of time points: X; = {xt, ...................... xf}
........'
Requirement Time Point Predictions To facilitate interpretability our framework, we spec- ify that models must provide time point predictions along with their time series predictions. Further- more, the time point predictions should inherent the model this makes it possible to identify which time points support and refute different classes without having use post-hoc methods.
Requirement 3: Temporal Ordering TSC is sequential learning problem so impose a fur- ther requirement that the framework must respect the ordering of time points. This is in contrast to classical MIL methods that assume MIL instances are iid.
3.2 MILLET FOR DL TSC: RETHINKING POOLING
To demonstrate the use of MILLT,, we apply it to DL TSC methods. Existing DL TSC architectures (e.g. FCN, ResNet, and InceptionI CemptionTime imm) mainly consist of two modules: a feature extractor UFF (we refer to these
backbones) and a classifier dCCLF. For an input univariate time series X1, thE produces a set of d-dimensional feature embeddings Z; E Rxxd = z z
z These
embeddings are consolidated via aggregation with Global Average Pooling (GAP) to give a single feature vector of length d. This is then passed to VOLF to produce predictions for the time series:
( [
Feature Extraction: Z; = = (X:); GAP + Classification: Y; = VCLF
(1)
FFollowing convention from MIL, we use uppercase variables to denote MIL bag / time series data and lowercase variables to denote MIL instance / time point data.
"There an overlap in TSC and MIL terminology: both use the term 'instance' but in different ways. In MIL it denotes an element in a bag. and in TSC it refers to an entire time series (e.g. Finstance-based explanations" from Theissler et al., 2022). To avoid confusion, we use 'time series' to refer to entire time series TSC instance) and 'time point' to refer value for particular step time series (a MII. instance).
3

---

Published as a conference paper at ICLR 2024
The specification of dh E naturally satisfies Req. 1 from our MILLET framework discriminatory information is extracted on a time point level. Req. is satisfied as long as the DL architecture makes of layers that respect the ordering of the time series such as convolutional or recurrent layers. In the MIL. domain, the GAP + Classification process (Fqn. 1) is known as mean Embedd i ng pooling, used in methods such MI-Net (Wang al., 2018). However, this aggregation step does not inherently produce time point class predictions, and consequently does not fit Req. 2.
To upgrade existing DL TSC methods into the MILLET framework and satisfy Req. 2, we explore four MIL pooling methods for replacing GAP. Attention, Instance, and Addit sive are inspired by existing MIL approaches, while Con junctive is proposed in this work. Replacing GAP in this way is pllug-ndd-layy i.c. any TSC method using GAP or similar pooling can easily be upgraded one of these methods and meet the requirements for MILLET.
Attention pooling (Ilse et al., 2018) does weighted averaging via an attention head BATTN:
a E [0, VATTN 2!):
Y; = VOLF
(2)
j=1
Instance pooling (Wang al., 2018) makes prediction for each time point:
s E RR = voLF 2
(3)
Additive pooling (Javed et al., 2022) is combination of Attention and Instance:
df E E00 WATTN (
y = veLr (1/cz)(
Y,
(4)
Conjunctive ive pooling is our proposed novel pooling approach, where attention and classifi- cation are independently applied to the time point embeddings, after which the attention values used to scale the time point predictions. This is expected to benefit performance as the attention and classifier heads are trained in parallel rather than sequentially, i.c. the classifier cannot rely on the attention head to alter the time point embeddings prior to classification, making it more robust. We use the term Con jonjunee junet ive to emphasise that, from an interpretability perspective, a discrimina- tory time point must be considered important by both the attention head and the classification head. Formally, Conjunctio ive is described as:
df E [0 WATTN 2
8 =
VOLF =
(2)):
(5)
Fig. 2 shows schematic representation comparing these pooling approaches with Embedding (GAP). Note there are other variations of these MIL pooling methods, such as replacing mean with max in Embedding and Instance, but these alternative approaches are not explored in this work.
3.3 MILLET DL INTERPRETABILITY
As a result of Requirement in Sec. 3.1, we expect the models to be inherently interpretable. For DL methods, this is achieved through the MIL pooling methods given in Sec. 3.2 differ- ent MIL pooling approaches provide alternative forms of inherent interpretability. Inst ance performs classification before pooling (see Eqn. 3), it produces set of time point predictions 91 E Rx | .f.....ff. .f]. Additive and Conjunctive also make time point predictions, but include attention. To combine these two outputs, weight the time point predictions by the attention scores: y E Rxxe = [a ozy?.... afy!]- ..?!...!!!..! Note that y is used to signify the attention weighting of the original time point predictions y:-
On the other hand, Attention is inherently interpretable through its attention weights a; E [0, 1] [a], az ....,, ag], which can be interpreted a measure of importance for each time point. Note, unlike Instance Additive, and Conjunct ive, the interpretability output for Attent i on not class specific (but only general measure of importance across all classes).
4

---

Published as a conference paper at ICLR 2024
(a) Global Average Emmeedding) Pooling
(b) Attention Poolling
Instance Pooling
Point Embodddings
Embodddings
Time Point
Embeddings
*
Global Pastiig
Time Point Adtentionn
Alintice Roal
Cauuler
Time Series
Showioo Mogited Pooling
Time Point Predictions
Mocr (+f)
Embbecking
Time Scries Endoalding
Chouster
Awarge Posking
Clanpler
tour
Time Series Prediction
Poco (21)
Prodiction
Prediction
(d) Additive
(c) Conjunctive Pooling
Time Point Emtheklings
Time Point Embeddings
Time Point Attention
Hooo
Time Point
Time Prodictions
Pezzr (x))
Atrasion Hiightog
Weighted Point Farhoddings
Weighted Time Point Prodictions
afsicur (ef)
Point Pradictions
Show (+ta))
Average Poolox
Anorgee Pooiing
Mnze (otkz)
Time Serics Prediction
(ddttcco(4))
Prodiction
Figure 2: The five different MII. pooling methods used in this work. Each takes the same input: a bag of time point embeddings Z₁ E gxxd = [z) 2, .2......8 .2]. While they all produce the same overall output (a time series prediction), they produce different interpretability outputs.
3.4 MILLET DL MODEL DESIGN
We design three MILLET DL models by adapting existing backbone models that use GAP: FCN, ResNet, and Incept ionTimei While extensions of these methods and other DL approaches exist (see Foumani al., 2023), we do not explore these as none have been shown outperform Incept in ime (Middlehurst et al., 2023). Nevertheless, the MILLET framework can be applied to any generic DL TSC approach that uses GAP follows the high-level structure Eqn.
Replacing GAP with one of the four pooling methods in Sec. 3.2 yields total of 12 new models. In
each case, the backbone models produce feature embeddings of length 128 (Z, E 121122)) The models are trained end-to-end in the same manner as the original backhone methods we discuss additional options for training in Sec. We introduce three further enhancements:
1. Positional Encoding: As time point classification and attention are applied each time point independently, the position of a time point within the times series can be utilised (with GAP, positional encoding would lost through averaging) this allows for further expressivity of the ordering of time points and enforces Req. 3 of our MILLET framework. We inject fixed positional encodings (Vaswani al., 2017) after feature extraction.
2. Replicate padding: Zero padding is used in the convolutional layers of the backbone chitectures. However, in our interpretability experiments, we found this biased the models towards the start and end of the time series padding with zeros creating false signal in the time series. As such, we replaced zero padding with replicate padding (padding with the boundary value) which alleviated the start/end bias. However, we note that particular problems may benefit from other padding strategies.
3. Dropout: To mitigate overfitting in the new pooling methods, we apply dropout after in- jecting the positional encodings (p 0.1). No dropout was used in the original backbones.
The original Incept Time approach is an ensemble of five identical network architectures trained from different initialisations, where the overall output is the mean output of the five networks. To facilitate a fair comparison, we use the same approach for FCN, ResNet, and MILLET. See Sec. for implementation details, and App. B for model, training, and hyperparameter details.
5

---

Published as a conference paper at ICLR 2024
4 INITIAL CASE STUDY
4.1 SYNTHETIC DATASET
In order explore our MILLET concept, and evaluate the inherent interpretability of our models, we propose a new synthetic dataset called WebTraffic. By demonstrating daily and weekly season- ality, it is designed to mimic trends observed in streaming and e-commerce platforms. We inject different signatures into collection of synthetic time series to create ten different classes: ze- roth normal class and nine signature classes. The signatures are partially inspired by the synthetic anomaly types proposed in Goswami al. (2023). The discriminatory time points are known as we are able to control the location of the injected signatures. Therefore, we are able to evaluate models can identify both the signature and the location of the discriminatory time points - both can be achieved inherently by our MILLET models. Each time series is length = 1008, and the training and test set contain 500 time series see Fig. 3 for examples and App. C.1 for more details.
Class O: None
Class 1: Spikes
Class 2: Flip
Class 3: Skew
Class 4: Noise
Value
humm
M
M
Class Cutoff
Class 6: Average
Class Wander
Class Peak
Class 9: Trough
Value
500
1000
500
1000
500
1000
500
1000
500
1000
Time
Original Time Series
Signaturn Time Series
Discriminatory Region
Figure 3: An example for each class of our WebTraffic dataset. Signatures are injected in a single random window, with the exception of Class (Spikes), which uses random individual time points. 4.2 WebTraffic RESULTS
We compare the four proposed MIL pooling approaches for MILLET with GAP on our WebTraffic dataset. Each pooling method is applied to the FCN, ResNet, and InceptionTl backbones. We conduct five training repeats of each model, starting from different network initialisations, and then ensemble them to a single model. We find that MILLET improves interpretability without being detrimental to predictive performance. In actuality, MILLET improves accuracy averaged across all backbones from 0.850 to 0.874, with a maximum accuracy of 0.940 for Conjunct, ive Incept i con.imm? 5 In Fig. 4 we give example interpretations for this best-performing model. The model is able to identify the correct discriminatory regions for the different classes, but focuses on certain parts of the different signatures. For example, for the Spikes class, the model identifies regions surrounding the individual spike time points, and for the Cutoff class, the model mainly identifies the start and end of the discriminatory region. This demonstrates how our interpretability outputs are not only able to convey where the discriminatory regions are located, but also provide insight into the model's 's decision makinn process, providing transparency.
To quantitatively evaluate interpretability Early et al. (2021). This approach uses ranking metrics, i.e. looking at the predicted importance order rather than actual interpretation values. The two metrics used are Area Over The Perturbation Curve to Random (AOPCR) and Normalised Discounted Cumulative Gain at n (NDCG@n). The former is used to evaluate without time point labels, and the latter is used to evaluate with time point labels. 6 In this evaluation, we compare to baselines CAM (applied to the original GAP models) and SHAP (applied all models, see App. B.3). CAM is lightweight post-hoe interpretability method (but not intrinsically part of the model output unlike our MILLET interpretations). SHAP is much more expensive run than CAM or MILLET as it has to make repeated forward passes of the model. In this case, we use SHAP with 500 samples, meaning it is 500 times more expensive than MILLET. In actuality, we find MILLET over 800 times faster than SHAP (see App. E.4).
WebTraffic dataset, we use the same process as
$ For complete results, see App. D.2.
'For more details on both metrics, see App. D.I. Note that NDCG@n can be used for this dataset as we know the locations of discriminatory time points (where the signatures were injected). However, for the UCR datasets used Sec. 5, the discriminatory time points are unknown, therefore only AOPCR can be used.

---

Published as a conference paper at ICLR 2024
Spikes
Noise
Cutoff
Peak
Time
1000
600
800
200
200
600
1000
Intreettala
200
400
600
BOD
1000
200
600
000
500
1000
200
400
600
800
1000
200
400
000
800
1000
1000
Tlmo
Figure 4: Interpretations for Conj. Incept ieeepiioiime on our WebTraffic dataset. Top: Time
series with the known discriminatory time points highlighted. Middle: Interpretability scores for each time point with respect to the target class. Bottom: Interpretability scores heatmap as in Fig. 1.
As shown in Table 1, MILLET provides better interpretability performance than CAM SHAP. SHAP performs particularly poorly, especially considering it is so much more expensive to run. Due to the exponential number of possible coalitions, SHAP struggles with the large number of time points. In some cases, it even has a negative AOPCR score, meaning its explanations are than random. For each backbone, MILLET has the best AOPCR and NDCG@n performance. The exception to this is NDCG@n n for Incept ionTime, where CAM is better (despite MILLET having better AOPCR score). This is likely due the sparsity of MILLET explanations as shown for the Cutoff and Peak examples in Fig. 4, MI LIET produces explanations that may not achieve full coverage of the discriminatory regions. While sparsity is beneficial for AOPCR (fewer time points need to be removed to decay the prediction), it can reduce NDCG @ n as some discriminatory time points may be identified (for example those the middle of the Cutoff region).
Table Interpretability performance (AOPCR NDCG@n) on WebTraffic. For SHAP and MILLET, results are given for the best performing pooling method. For complete results, see App. D.2.
FCN
ResNet
InceptionTime
Mean
CAM
12.780 / 0.532
20.995 0.582
12.470 0.707
15.415 / 0.607
SHAP
1.977 / 0.283
-0.035 / 0.257
-4.020 / 0.259
-0.692 / 0.266
MILLET
14.522 0.540
24.880 0.591
13.192 / 0.704
17.531 / 0.612
UCR RESULTS
We evaluate MILLET on the UCR TSC Archive (Dau et al., 2019) wwidely acknowledged as definitive TSC benchmark spanning diverse domains across 85 univariate datasets (see App. C.2). Below are results for predictive performance, followed by results for interpretability.
5.1 PREDICTIVE PERFORMANCE
For the three backbones, we compare the performance of the four MIL pooling methods with that of GAP. This is used to evaluate the change in predictive performance when using MI UILLET in a wide variety of different domains, and determine which of the pooling methods proposed in Sec. 3.2 is best. Averaged across all backbones, mance, with an accuracy improvement from 0,841 + 0.009 to 0.846 + 0.009 when compared to has the highest average accuracy of 0.856 + 0.015 (see App. D.3 for all results). Given this result, we then compare the performance of the three MILLET Con junct ive approaches with current SOTA methods, which intended provide better con- text for the performance of our newly proposed models. We select the top performing method from seven families of TSC approaches as outlined by Middlehurst al. (2023).
find Con junct ve gives the best perfor-
GAP. Conjun Conjunctive Anception"s

---

Published as a conference paper at ICLR 2024
As HC2 is very computationally pensive meta-ensemmle of multiple dif- ferent classifiers, we do not consider it to be an equal comparison to our meth- ods. In Fig. 5 give critical dif- ference (CD) diagram (Demšar, 2006) for balanced accuracy. We find that 1) using Con junctivie improves perfor- mance in all cases, and 2) Con junctive Incept i in is comparable to (even slightly better than) the SOTA of HC2 and Hydra-MP We provide further re- sults in Table 2. While Conjunctive InceptionTime is the best approach on balanced accuracy (outperforming the HC2 and Hydra-MR SOTA methods), it is not quite as strong on the other metrics. However, it remains competitive, and for each backbone using MILLET improves performance across all metrics.
1.838
Bal. Acc. Rank
13
12
11
10
8
7
6
5
3
PF 2.118
5.182 Conj-IT
FCN FreshPRINCE H.294
[T
5.632 HC2
RSTSF 8.291
5,929
Hydra-MR
Conj-PCN RDST 8.97)
Conj-RN
6,741 RN
WEASEL-D
Figure 5: Critical difference diagram comparing Conjunctive MILLET methods with SOTA.
Table 2: Performance on 85 UCR datasets in the form: mean / rank / number of wins. HC2 is included for reference but not compared to as it is a meta-ensemee of several other TSC approaches.
Method
Accuracy 1
Bal. Accuracy
AUROC t
NLL +
Hydra-MR
0.857 5.306 19
0.831 / 5.929 17
0,875 10.682
0.953 / 6.965 /
FCN
0.828 / 9.088 7
0.804 / 8.494 8
0.929 / 7.653 / 13 1.038 / 7.176
Conj. FCN
0.838 / 7
0.814 / 7.3300 7
0.934 / 6.288 / 20
0.973 6.247 /
ResNet
/843/7.288
0.819 / 6.741 /
0.937 6.035 / 18 1.091 7.788 /
Conj. ResNet.
0.845 / 7.200 / 13
0.822 / 6.618 14
0.939 / 5.512/ 16
1.035 7.447 /2
ITime
0.853 / 6.112/ 19
0.832 5,494 / 19
0.939 5.453 / 26
1,078 / 7.118 9
Conj. ITime
0.856 5.606
19
0.834
5.182/ 23
0.939
5.276 / 26
1.085
7.341 /
HCC2
0.860/4.953 21
0.830
5.612 17
0.950 3 441 /43
0.607/
/.9441 14
5.2
INTERPRETABILITY PERFORMANCE
In order to understand the interpretability of our MILLET methods on wide variety of datasets, we evaluate their interpretability on the same 85 UCR datasets used in Sec. 5.1.
As the UCR datasets do not have time point la- bels, we can only evaluate model interpretability using AOPCR. Averaged across all backbones, we find that MILLET has best AOPCR 6.00, compared to 5.71 achieved by GAP. Of the indi-
Predictive Performance (Bal. Acc.)
0.835
0.830
0.825
vidual pooling methods within MILLET, we find
0.820
that Con junctive has the best interpretability performance. Attentio on performs poorly this is expected as it does not create class-specific interpretations, but only general measures im- portance; also identified in general MIL inter-
0.815
FCN MILLET GAP
0.810
RN MILLET GAP
IT MILLET GAP
0.805
Pareto Front
0.800
pretability by Early et al. (2021). In Fig. we observe a trade-off between interpretability and prediction, something we believe is insightful for model selection in practice. As backbone complexity increases, predictive performance in- creases while interpretability decreases.? ? The Pareto front shows MILLET dominates GAP for FCN and InceptionTing MILLET gives better interpretability than GAP for individual ResNet models, but struggles with the ensemble ResNet. models. For complete results and further discussion, see App. D.3.
4.0
4.5
5.5
6.0
6.5
7.0
Interpretability Performance (AOPCR)
Figure 6: The interpretabiliiy- prreeeiii perfor- mance trade-off. Fach cross indicates a different pooling method (Attent i on omitted).
but not ResNet.
neept i on'ii is the most complex backbone, followed by ReaNet, and then PCN the simplest.
8

---

Published as a conference paper at ICLR 2024
Time Series
SHAP
MILLET
Perturbation Curves
Washhng Machtae
10
20
440
2441
480
720
240
480
720
0
240
480
720
240
480
720
100%
75%
50%
Tumble Dryur
Prod.
240
480
720
240
480
720
240
480
720
240
480
720
100%
75%
50%
10
Dishwosber
Pameey
Pweee
5
480
50%
Time
Pertarbation
CAM
SHAP
MILLET
Random
Figure 7: Comparison of interpretations on the LargeKit chwnhpplinncee UCR dataset. Left: Original time series. Middle: Interpretability scores heatmap for CAM, SHAP, and MILLET as Fig. 1. Right:
Perturbation curves showing the rate at which the model prediction decays when time points are removed following the orderings proposed by the different interpretability methods.
Fig. shows interpretations for CAM, SHAP, and MILLET on LargeKit chenAppphhhepppppc anness; UCR dataset for identifying household electric appliances (Washing Machine, Tumble Dryer, or Dishwasher) from electricity usage. From the MILLET interpretability outputs, we identify that the model has learnt different motifs for each class: long periods of usage just above Washing Machine, spikes above five indicate Tumble Dryer, and prolonged usage at just below five indicates Dishwasher. The Washing Machine example contains short spikes above five but MILLET identifies these refuting the prediction, suggesting the model does not relate these spikes with the Washing Machine class. SHAP provides very noisy interpretations and does not show strong performance on the perturbation curves. Similar to our findings for WebTraffic (Sec. 4.2), MILLET provides sparser explanations than CAM, i.e. focusing on smaller regions and returning fewer dis- criminatory time points, which is helpful when explaining longer time series. We provide further analysis in the Appendix: seaa-s (App. D.3), false negatives (App. E.1), dataset variance (App. E.2), model variance (App. E.3), run time (App. E.4), and ablation study (App. E.5).
indicate
CONCLUSION
The MILLET framework presented this work the first comprehensive analysis MII. for TSC. Its positive value is demonstrated across the 85 UCR datasets and the WebTraffic dataset proposed in this work, MILLET provides inherent mechanisms to localise, interpret, and explain influences on model behaviour, and improves predictive accuracy in most cases. Through the transparent decision- making gained from MILLET, practitioners can improve their understanding without the need for expensive (and often ineffective) post-hoc explainability methods. In addition, MILLET explanations are sparse they distill the salient signatures of classes to a small number of relevant sub-sequences, which is especially important for long time series. We believe this work lays firm foundations for increased development of MII. methods TSC, and facilitates future work:
model dynamics
Extension to more datasets: This could include the full set of 142 datasets used in Bake Off Redux (Middlchurst et al., 2023), multivariate datasets, and variable length time series. Variable lengths would not require any methodological changes (contrary several other methods), but multivariate settings would require interpretability to consider the input channel (Hsieh et al., 2021).
Application to other models: We have demonstrated the use of MILLET for DL TSC models. Fu- ture work could extend use to other types of TSC models, e.g. the ROCKET (convolutional) fam- ily of methods (Dempster et al., 2020), which includes Hydra-MR. Our proposed pooling method, Con junct ve, also applicable general MII. problems beyond TSC.
Pre-training/ boaiiiinggiine-unniin tuning: While we trained MILLET models in an end-to-end manner, an alter- native approach take a pre-trained GAP model, replace the GAP layers with one of the proposed MIL pooling methods, and then fine-tune the network (facilitating faster training).

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
The code for this project was implemented in Python 3.8, with PyTorch as the main library for ma- chine learning. A standalone code release is available at: https: /////tittt github. com/ JAEarly/ MILTimeSeriesClase ff fication. This includes our synthetic dataset and the ability to use our plug-and-play MILLET models.
Model training was performed using an NVIDIA Tesla V100 GPU with 16GB of VRAM and CUDA v12.0 to enable GPU support. For reproducibility, all experiments with a stochastic nature (e.g. model training, synthetic data generation, and sample selection) used pscudo-random fixed seeds. A list of all required libraries is given in the code release mentioned above.
REFERENCES
João Bento, Pedro Saleiro, André Cruz. Mário AT Figueiredo, and Pedro Bizarro. TimeSHAP: Explaining recurrent models through sequence perturbations. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2565-2573, 2021.
Jonathan Crabbé and Mihaela Van Der Schaar. Explaining time series predictions with dynamic masks. In International Conference on Machine Learning, pp. 2166-2177, PMLR, 2021.
Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, (0001123--105, 2019.
Angus Dempster, François Petitjean, and Geoffrey I. Webb. ROCKET: Exceptionally fast and accurate time series classification using random convolutional kemels.. Data Mining and Knowledge Discovery. 34(5): 1454-1495, 2020.
Angus Dempster, Daniel F. Schmidt, and Geoffrey I. Webb. Hydra: Competing convolutional kemnels for fast and accurate time series classification. Data Mining and Knowledge Discovery, pp. 1-27, 2023.
Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1-30, 2006.
Don Dennis, Chirag Pabbaraju. Harsha Vardhan Simhadri, and Prateek Jain. Multiple instance learning efficient sequential data classification on resource-comsttriined devices. Advances in Neural Information Processing Systems, 31, 2018.
Thomas G. Dietterich, Richard H. Lathrop, and Tomás Lozano-Péree Solving the multiple instance problem with axis axis-parallel rectangles. Artificial Intelligence, 89(1-2):31127 1997.
Joseph Early, Christine Evers, and Sarvapali Ramchurn. Model agnostic interpretability for multiple instance learning. In International Conference on Learning Representations, 2021.
Joseph Early, Tom Bewley, Christine Evers, and Sarvapali Ramchurn. Non-markovian reward modelling from trajectory labels via interpretable multiple instance learning. Advances in Neural Information Processing Systems, 35:27652 27663, 2022.
Navid Mohammadi Foumani, Lynn Miller, Chang Wei Tan, Geoffrey I. Webb, Germain Forestier, and Mahsa Salehi. Deep leamning for time series classification and extrinsic regression: A current survey, arXiv preprint arXiv:2302.0251/1 2023.
Pedro J Freire, Sasipim Srivallapanondh, Antonio Napoli, Jaroslaw E Prilepsky, and Sergei K Turitsyn. Computational complexity evaluation of neural network applications signal processing. arXiv preprint 1rXiv:200..12191, 2022.
Mononito Goswami, Cristian Challu, Laurent Callot, Lenon Minorics, and Andrey Kan. Unsupervised model selection for time-series anomaly detection. In International Conference on Learning Representations, 2023.
Riccardo Guidotti and Mattco D' D'Onofrio. Matrix profile-based interpretable time series classifier. Frontiers in Artificial Intelligence, 4:699448, 2021.
Tsung- Yu Hsieh. Suhang Wang, Yiwei Sun, and Vasant Honavar. Explainable multivariate time series clas- sification: deep neural network which leams to attend to important variables as well as time intervals. In Proceedings of the 14th ACM International Conference Web Search and Data Mining, pp. 607-615,
2021.
10

---

Published as a conference paper at ICLR 2024
Maximilian Ilse, Jakub Tomezak, and Max Welling. Attention-bsedd deep multiple instance learning. In International Conference on Machine Learning, pp. 2127-2136. PMLR, 2018.
Aya Abdelsalam Ismail, Mohamed Gunady, Hector Corrada Bravo, and Soheil Feizi. Benchmarking deep learning interpretability in time series predictions. Advances in neural information processing systems, 33: 6441-6452, 2020.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weher, Lhassane Idoumghar, and Pierre-Alain Muller. Deep learning for time series classification; a review. Data Mining and Knowledge Discovery, 33(4):917-963, 2019.
Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey Webb, Lhassane Idoumghar, Pierre-Alaie Muller, and François Petitjcan. InceptionTime: Finding AlexNet for time series classification. Data Mining and Knowledge Discovery, 34(6):1936-11 1962, 2020.
Vijay Manikandan Janakiraman, Explaining aviation safety incidents using deep temporal multiple instance learning. In Proceedings of the 24th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 406-415, 2018.
Syed Ashar Javed. Dinkar Juyal, Harshith Padigela. Amaro Taylor- Weiner, Limin Yu, and Aaditya Prakash. Additive MIL: Intrinsically interpretable multiple instance learning for pathology. Advances in Neural In- formation Processing Systems, 35:20689-20700 207022 2022.
Kin Kwan Leung, Clayton Rooke, Jonathan Smith, Saba Zuberi, and Maksims Volkovs. Temporal dependencies in feature importance for time series prediction. In The Eleventh International Conference Learning Representations, 2022.
Scott M. Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances Neural Information Processing Systems, 30, 2017.
Matthew Middlehurst, James Large, Michael Flynn, Jason Lines, Aaron Bostrom, and Anthony Bagnall. HIVE- COTE 2.0: a new meta ensemble for time series classification. Machine Learning, 110(11-12)3211-333 2021.
Matthew Middlehurst, Patrick Schäfer, and Anthony Bagnall, Bake off redux: review and experimental evaluation recent time series classification algorithms. arXiv preprint arXiv:2304. 13029, 2023.
Christoph Molnar. Interpretable machine learning. edition, 2022. lenooic/inng github..c//ntee.e... http:///ceneerrtaalenll...
URL https::/ccrissttpptteeest//
Amin Nayebi, Sindhu Tipimeni. Chandan K Reddy, Brandon Foreman, and Vignesh Subbian. WindowSHAP: efficient framework for explaining time-sories classifiers based on Shapley values. Journal of Biomedical Informatics, 144: 14438,,, 2023.
Rafael Poyiadzi, Raul Santos- CantosRRorriguez and Niall Twomey. Lahel propagation for learning with lahel propor- tions. 2018 IEFE 28th International Workshop on Machine Learning for Signal Processing (MISP), pp. 1-6. IEEE, 2018.
Rafael Poyiadzis. Raul Santos- Rodriguez, and Niall Twomey. Active learning with label proportions. In ICASSP 2019-2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 0097-3101 IEBE, 2019.
Owen Queen, Thomas Hartvigsen, Teddy Koker, Huan He, Thcodoros Tsiligkaridis, and Marinka Zitnik. Encoding time-series explanations through self-supervised model behavior consistency. arXiv preprint 200. 2023.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Bhoomi Shah and Hetal Bhavsar. Time complexity in deep learning models. Procedia Computer Science, 215: 202-210, 2022.
Divya Shanmugam, Davis Blalock, and John Guttag. Multiple instance learning for ECG risk stratification. In Machine Learning for Healthcare Conference, pp. 124-139. PMLR, 2019.
Ilija Simié, Vedran Sabol, and Eduardo Veas. XAT Methods for Neural Time Series Classification: A Brief Review. arXiv preprint arXiv: 2exxv:2208.000 008.08009,0000.00 2021.

---

Published as a conference paper at ICLR 2024
Torty Sivill and Peter Flach. Limesegment: Meaningful, realistic time series explanations. In International Conference on Artificial Intelligence and Statistics, pp. 3418 3433. PMLR, 2022.
Andreas Theissler, Francesco Spinnato. Udo Schlegel. and Riccardo Guidotti. Explainable AI for time series classification: review, taxonomy and research directions. IEEE Access, 10: 10:100700-100724, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances Neural Information Processing Systems, 30, 2017.
Kaili Wang. Jose Oramas, and Tinne Tuytelaars. defense LSTMs addressing multiple instance learning problems. Proceedings of the Asian Conference on Computer Vision, 2020.
Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai, and Wenyu Liu. Revisiting multiple instance neural networks. Pattern Recognition, 74:15-24 2018.
Zhiguang Wang, Weizhong Yan, Tim Oates. Time series classification from scratch with deep neural networks: A strong baseline. 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1578-1585, IEEE, 2017.
Chin-Chia Michael Yeh, Nickolas Kavantzas, and Eamonn Keogh. Matrix profile IV: using weakly labeled time series to predict outcomes. Proceedings of the VLDB Endowment, 10(12):1802-1112 2017.
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 2014, Proceedings, Part I 13, pp. 818 833. Springer, 2014.
Bolci Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 22299, 2016.
Yuansheng Zhu, Weishi Shi, Deep Shankar Pandey, Yang Liu, Xiaofan Que, Daniel E. Krutz, and Qi Yu. Uncertainty aware multiple instance learning from large-scale long time series data, In 202/ IEEE Interna- tional Conference on Big Data (Big Data), pp. 1772-1778. IEEE, 2021.
12

---

Published as a conference paper at ICLR 2024
A
WHY MIL?
To answer the question of "why MIL", we first consider the requirements for interpreting TSC mod- els. Our underlying assumption is that, for classifier to predict certain class for time series, there must be one or more underlying motifs (a single time point or collection of time points) in the time series that the model has identified as being indicative of the predicted class. In other words, only certain time points within a time series are considered discriminatory by the model (those that form the motifs), and the other time points are ignored (background time points or noise). This could also be extended to class refutation, where the model has learnt that certain motif's indicate that a time series does not belong to particular class. To this end, the objective of TSC interpretability is then to uncover these supporting and refuting motifs, and present them as an explanation as to why a particular class prediction has been made.
We next consider how we could train model to find these motifs if we already knew where and what they were, i.e. conventional supervised learning sense where the motifs are labelled. In this case, we would be able to train a model to predict a motif label from an input motif - in the simplest case this could be predicting whether a motif is discriminatory or non-discriminatory. This model could then be applied to unlabelled time series and used to identify its motifs. Effectively, this hypothetical motif model is making time point level predictions.
Unfortunately, it is very difficult to train models in the above manner. The vast majority of time series datasets are only labelled at the time series level - no labels are provided at the time point level, therefore the motifs and their locations are unknown. While there are several successful deep learning models that are able to learn to make time series level predictions from time series level labels (e-g FCN, ResNet, Incept ion Time,, they are black boxes they provide no inherent interpretation in the form of supporting/refutin motifs via time point level predictions. Without time point level labels, conventional supervised learning is unable to develop models that inherently make time point level predictions. While post-hoc interpretability methods could be used to uncover the motifs, they can be expensive (in the case of LIME and SHAP), or as we show in this work, inferior to inherent interpretability. As such, we can draw a set of requirements for new and improved interpretable TSC approach:
1. Inherent interpretability: The model should provide time point predictions (i.e. motif identification) as part of its time series prediction process. This means interpretations are gained effectively for free as a natural byproduct of time series prediction. It also means expensive or ineffective post-hoc interpretability approaches are not required.
2. Learn from time series labels: As stated above, time series classification datasets rarely provide time point-level labels. Therefore, the model must be able to learn something insightful at the time point level given only time series labels.
3. Provide a unified framework: As there are a diverse range of existing TSC methods of different families, an effective TSC interpretability approach should be as widely applicable as possible to facilitate continued research in these different areas. This would also mean existing bodies of research can be applied in conjunction with the framework.
Given these requirements, we advocate for MIL as an appropriate method for an inherently inter- pretable TSC framework. MIL is designed for settings with bags of instances, where only the bags are labelled, not the instances. In TSC setting. this equates to having time series labels without time point labels, which exactly what we outlined above (and describes the vast majority of TSC datasets). Furthermore, certain types of existing MIL. approaches, for example Instance and Additive, work by first making a prediction for every time point, and then aggregating over these predictions to make a time series prediction. This is inherently interpretable and facilitates motif identification. Finally, the over-arching concept of MIL for TSC, i.e. learning from time series la- bels but making both time point and time series predictions, is not tied to any one family of machine learning approach.
We also consider answers potential questions about alternative solutions:
1. Q: Why not label the motifs/time points to allow for supervised learning? A: As discussed above, is rare for a time series dataset to have time point labels. It could be possible label the motifs, for example by having medical practitioners identify
13

---

Published as a conference paper at ICLR 2024
discriminatory irregular patterns in ECG data. However, this labelling process is very time- consuming, and in some cases would require domain expertise. which is challenging and expensive to acquire. Furthermore, it would then require anyone interested in applying such supervised techniques to their own data to fully label everything at the time point level, increasing the cost and time of developing new datasets.
2. Q: Why not label some of the motifs/time points and use semi-supervised approach? A: While it might be possible to train a semi-supervised model that only requires some of the time points to be labelled, the model no longer end-to-end. As the model only learns to predict at the time point level, it does not provide time series level predictions itself. Rather, some additional process required to take the time point predictions and transform them into a time series prediction. Furthermore, a semi-supervised model has no method for leveraging both the time series labels and the partial time point labels.
3. Q: What does MIL achieve beyond methods such as attention? A: While attention has been utilised previously in TSC, it does not provide class-specifie interpretations, only general measures of importance across all classes. So while attention might identify motifs, it does not state which class these motif's belong to, nor whether they are supporting or refuting. Furthermore, attention is far less explicit than time point predictions there is no guarantee that attention actually reflects the underlying motifs of decision- Decision-making, whereas in MIL the time point predictions directly determine the time series prediction.
A.1 RELATED WORK (CONTINUED)
In this section, we further discuss existing TSC interpretability approaches and contrast them with approach. MILLET is time point-based explanation approach (following the taxonomy of Theissler et al., 2022), meaning it produces interpretations for each individual time point in a time series. This is in contrast to other types of methods such as subsequence-basee provide interpretations at a lower granularity. As such, we choose to compare against interpretability methods with the same level of granularity, i.e. those that also make time point-based interpretations (such as CAM and SHAP).
techniques, which
While alternative time series-specific SHAP approaches such as TimeSHAP (Bento al., 2021) or WindowSHAP (Nayebi et al., 2023) could have been used, these both reduce the exponential sampling problem of SHAP by grouping time points together. As such, they do not have the same granularity as MILLET and do not meet our requirements. Two similar methods to SHAP are LIME (Ribeiro et al., 2016) and MILLI (Early et al., 2021), but these require careful tuning of hyperpa- rameters (which SHAP does not), so are infeasible to run over the large number of datasets used in this work. LIMESegements (Sivill & Flach, 2022), an extension of LIME for TSC, has the same issue as TimeSHAP and WindowSHAP mentioned above: it groups time points into subsequences so does not produce interpretations at the same granularity as MILLET. Alternative approaches for TSC interpretability have similar granularity issues to the cost of using MILLET or CAM). Such methods include Dynamask (Crabbé & Van Der Schaar, 2021), WinIT (Leung et al., 2022), and TimeX (Queen et al., 2023). If MILLET were to be extended to multivariate TSC problems, would be compatible with Temporal Saliency Rescaling (TSR; Is- mail al., 2020), which has been shown to improve time point-based interpretability methods in multivariate settings.
often very expensive to compute (relative
While our choice of interpretability evaluation metrics are based those used by Barly al. (2021), different metrics are used in existing TSC interpretability works, but these come with their own set of disadvantages. Metrics such as area under the precision curve (AUP) and area under the recall curve (AUR) are useful in that they separate whether the identified time points are indeed discrimina- tory (precision) from whether all discriminatory time points are identified (recall/coverage). While this would be beneficial in evaluating the effect of MI LLET sparsity, it does not account for the order- ing of time points in the interpretation, i.c. we want to reward the interpretation method for placing discriminatory time points earlier in the ordering (and punish it for placing non-discriminatory time points earlier on); this is something that NDCG@n achieves. The mean rank metric (Leung et al., 2022) also suffers from this issue - it is effectively an unweighted version of NDCG@n. Further- more, AUP and AUR metrics require time point labels and sometimes need access to the underlying data generation process for resampling; AOPCR does not need either.
14

---

Published as a conference paper at ICLR 2024
B MODEL DETAILS
B.I MILLET MODEL ARCHITECTURES
In the following section, we provide details on our MILLET models. For conciseness, we omit details on the backbone architectures. See Wang et al. (2017) for details on FCN and ResNet, and Ismail Fawaz et al. (2020) for details on inceptionTinee tractor backbones used in this work produce fixed-size time point embeddings of length 128: Zi E 12 | 2 ....., .2...., ",], where t is the input time series length. This is the initial Fea- ture Extraction phase and uses unchanged versions of the original backbones.
Each of the three feature ex-
A breakdown of the general model structure is given in Eqn. A.I. Note how Positional En- coding only applied after Feature Extraction. Furthermore, Positional Encoding and Dropout only applied in MILLET models, i.e. when using Instance, Attention Additive, or Con jonctttiinn ivee pooling.
Feature Extraction + Positional Encoding + Dropout - MIL Mooling
(A.1)
Below detail the Positional Encoding processes, and then give architectures for each of the MIL Pooling approaches. These are kept unchanged across backbones.
B.1.1 POSITIONAL ENCODING
Our approach for Positional Encoding uses fixed positional encodings (Vaswani et al., 2017);
PEtpos 28) sin sin(pos/10000 dmmodel). )
ppo = cos(pos/ (00000 25/d //mdddl)
(A.2)
where pos [1, [1..... f]] is the position in the time series, dmoode 128 is the size of the time point embeddings, and i = [I.... dmodea/2). As such, the Positional Encoding output is the same shape the input time point embeddings (R(x128) The positional encodings are then simply added the time point embeddings.
In cases where time points are removed from the bag, e.g. when calculating AOPCR (see App. D.I), we ensure the positional encodings remain the same for the time points that are still in the bag. For example, if the first 20 time points are removed from a time series, the 21xt time point will still have positional encoding ( not (E
B.1.2 MIL POOLING ARCHITECTURES
In Tables A.I to A.5 we provide architectures for the different MIL pooling methods used this work (see Fig. 2 for illustrations). Each row describes layer in the pooling architecture. In each case, the input is a bag time point embeddings (potentially with positional encodings and dropout already applied, which does not change the input shape; App. I.I. The input is batched with batch size of b, and each time series is assumed to have the same length 1. Therefore, the input is four dimensional: batch size X number of channels time series length x embedding size. However, in this work, as we are using univariate time series, the number of channels is always one. The problem has e classes, and the pooling methods produce logit outputs softmax is later applied as necessary.
Table A.1: MIL Pooling: Embedding (GAP).
Process
Layer
Input
Output
Pooling
Mean
bx x x 128 (Time Point Embs.)
b x
x x x 128 (TS Emb.)
Classifier
Linear
bx1x1x x 128 (Time Series Emb.)
bx 1x1x1xx (TS Pred.)
15

---

Published as a conference paper at ICLR 2024
Table A.2: MII. Pooling: Attention We internal dimension of 8 in the attention head, and apply sigmoid rather than softmax due to the possibility of long time series. Attention weighting scales the time point embeddings by their respective attention scores.
Process
Layer
Input
Output
Attention
Linear tanh x x t 128 (Time Point Embs.) b x 1 x Linear + sigmoid b x x t x 8
b x 1 x t x (Attn. Scores)
Pooling
Attn. Weighting
x X t x 128 (Time Point Embs.)
b 1 X t x 128
Mean
b t 128
b X x 128 (TS Emb.)
Classifier Linear
xx x 1 x 128 (Time Series Emb.)
X 1 x x c (TS Pred.)
Table A.3: MIL Pooling: Instance.
Process
Layer
Input
Output
Classifier
Linear
x 1 x x 128 (Time Point Embs.)
bx x t c (TP Preds.)
Pooling
Mean
x 1 x x (Time Point Preds.)
bx 1 x x e (TS Pred.)
Table A.4: MIL MIL Pooling: Additive.
Process
Layer
Input
Output
Attention
Linear + tanh
b x x x 128 (Time Point Embs.) b x 1 x 8 Linear + sigmoid b x x t x 8
bx 1 x t x (Attn. Scores)
Classifier Attn. Weighting
b X X t x 128 (Time Point Embs.)
b 1 X t x 128
Linear
b t 128
b 1 t (TP Preds.)
Pooling Mean
b x x t x e (Time Point Preds.)
x 1 x x c (TS Pred.)
Table A.5: MIL Pooling: Conjunctive In this case, attention weighting scales the time point predictions by their respective attention scores, rather than scaling the embeddings.
Process Layer
Input
Output
Attention Linear + tanh
b x x 128 (Time Point Embs.) b x L Linear + sigmoid b x t x 8
b x 1 x (Attn. Scores)
Classifier Linear
b I 128 (Time Point Embs.) bx 1 L (TP Preds.)
Pooling
Attn. Weighting
x t e (Time Point Preds.) bx xxxx
bx x L e bx 1 x 1 x c (TS Pred.)
Mean
B.2 TRAINING AND HYPERPARAMETERS
In this work, all models were trained in the same manner. We used the Adam optimiser with fixed learning rate of 0.001 for 1500 epochs, and trained to minimise cross entropy loss. Training was performed in an end-to-end manner, i.e. all parts of the networks (including the backbone feature extraction layers) were trained together, and no pre-training or fine-tuning was used. Dropout (if used) was set to 0.1, and batch size was set to min [16, jum training time seriex/10 /1]] to account for datasets with small training set sizes. For example, dataset contains only 100 training time series, the batch size is set to 10.
No tuning of hyperparameters was used values were set based on the those used for training the original backbone models. As the existing DL TSC methods used fixed hyperparameters, our de- also has the benefit of
cision not to tune the hyperparameters facilitates a fairer comparison. providing robust set of default values for use in derivative works. However, better performance could be achieved by tuning hyperparameters for individual datasets. We would expect hyperparam- eter tuning for MT LLET to have greater impact than doing so for the GAP versions of the models as
16

---

Published as a conference paper at ICLR 2024
MILLET provides more scope for tuning (e.g. dropout, attention-head size, and whether to include positional encodings). As greater flexibility is achieved by having the ability to add/remove/tune the additional elements of MILLET, this would lead to more specialised models (and larger perfor- mance improvements) for each dataset. For example, in our ablation study (App. E.5), we found beneficial for some datasets but not others.
that positional encoding
No validation datasets were used during training or evaluation Instead, the final model weights were selected based on the epoch that provides the lowest training loss. As such, training was terminated early a loss of zero was reached (which was a very rare occurrence, but did happen). Models started with random weight initialisations, but pseudo-random fixed seeds were used to enable reproducibility. For repeat training, the seeds were different for each repeat (i.e. starting from different random initialisations), but these were consistent across models and datasets.
B.3 SHAP DETAILS
In our SHAP implementation we used random sampling of coalitions. Guided sampling (selecting coalitions maximise the SHAP kernel) would have proved too expensive: the first coalitions sampled would be all the single time point coalitions and all the t - length coalitions (for a time series of length t), which results in 2/ coalitions and thus 2/ calls the model. In the case of WebTraffic, this would be 2016 samples, rather than the 500 we used with random sampling (which still took far longer to run than MILLET, see App. E.4). Furthermore, Early et al. (2021) showed random sampling to be equal to or better than guided sampling in some cases.
C DATASET DETAILS
C.I SYNTHETIC DATASET DETAILS (WebTraffic)
In our synthetic time series dataset, WebTraffic, each time series is week long with a sample rate of 10 minutes (60 * 24 7/10- 1008 time points). The training and test set are independently generated using fixed seeds facilitate reproducibility, and both balanced with 50 time series per class (500 total time series for each dataset).
To explain the synthetic time series generation process, we first introduce new function:
W WarpedSin(a,, s, sin sin(x') + b, where x' = 2 p).
a
(A.3)
Parameters o, b, p. and s control amplitude, bias (intercept), phase, and skew respectively. W WarpedSin is used to generate daily seasonality in the following way:
Sample b,p,8,0.j) = Di
(A.4)
RateDay(ap (atteDay(a..bp.33 b,p. b.p. W WarpedSin(ap. 6,p 0.55, s/11444)
(A.5)
where j [1...., 1008) is the time index and or is parameter that controls the amount of noise cre- ated when sampling (via normal distribution). The daily rates provide daily seasonality (i.e. peaks in the evening and troughs in the morning). However, to take this further we also add weekly sea- sonality (i.e. more traffic at the weekends than early in the week). To do so, we further utilise W WarpedSin:
MateWeek((we = W arrpedSimaaw 1, 0.6, 2.j,2.,/1008)
(A.6)
To add this weekly seasonality, we multiply the daily sampled values by the weekly rate. Therefore, to produce base time series, we arrive at formula with six parameters:
17

---

Published as a conference paper at ICLR 2024
CempleWeek(apg) aw, b,p,,,,,,,a s, b. * RateWeck(aw:])
for E [1...., 1008). (A.7)
To generate collection of n time series, sample the following parameter distributions times (i.e. once for every time series we want to generate):
Amplitude daily: ap (2,
Amplitude weekly aw ~ Up (0.8, 1.2)
Bias b ~ (2(,.5,5)
Phase p ~ ~~ 0.05)
Skew ~ (1,
Noise 22 (2,
We use ~ UR (,, c) to denote uniform random sampling between and c, where E R. Below, we also use uniform random integer sampling a' (z, c), where a' E Z.
The above generation process results in a collection of n time series, but currently they are all class zero (Class O: None) as they have had no signatures injected. We describe how we inject each the nine signature types below. Aside from the Spikes signature (Class 1). all signatures are injected in random windows of length 1 Uz (36, 288) starting position p ~ (0,
1). The minimum window size of 36 corresponds to 0.25 days, and the maximum size of 288 corresponds to 2 days. In all cases, values are clipped to be non-negative, i.c. all time points following signature injection are 2 O. These methods are inspired by, but not identical to, the work of Goswami et al. (2023). We provide an overview of the entire synthetic dataset generation process in Fig. A.1. Exact details on the injected signatures are given below, along with focused examples in Fig. A.2.
Class 1: Spikes Spikes are injected at random time points throughout the time series, with proba-
bility p = 0.01 for each time point. The magnitude of a spike is drawn from (3.00, 2.0), and then added to subtracted from the original time point value with equal probability.
Class 2: Flip The randomly selected window is flipped in the time dimension.
Class 3: Skew A skew is applied the time points in the randomly selected window. A skew amount is first sampled from (0.255, 0.45). which is then add to or subtracted from 0.5 with equal probability. This gives new skew value E [0.05, 0.25) U [0.75, 0.95). The random window is then interpolated such that the value at the midpoint now located at time point [w /] within the window, i.e. stretching the time series on one side and compressing on the other.
Class 4: Noise Noise added to the random window. The amount of noise is first sampled from Noise (0.5, Then, for each time point in the selected window, noise is added according to (0, Noise).
Class 5: Cutoff A cutoff value is sampled from (g(0000, 0.2) The values in the random window are then set to (c,, 0.1).
Class 6: Average This signature is the opposite of noise injection, i.e. applying smoothing to the values in the random window. This is achieved through applying a moving average with window size sampled from Uz(5, 10).
Class 7: Wander A linear trend is applied values in the random window. The trend linearly transitions from to 22 (2.0, 3.0), and is then added to or subtracted from the values in the window with equal probability.
18

---

Published as a conference paper at ICLR 2024
Daily Rate
10
Daily Sample
5
Weekly Rate
1.5
1.0
0.5
Daily Sample Weekly Rate
5
Final Time Series
200
400
600
800
1000
Time
Figure A.1: An overview of our synthetic dataset generation. From top to bottom: daily scasonality rate, daily seasonality with sampled noise, weekly seasonality rate, base time series with daily and weakly seasonality, and final time series with signatures injected into the base time series.
Class None
Class Spakes
Class 2: Flip
Class 3: Skow
Class Nouse
CutaiT
Average
Class Wander
Ponk
Closs Trough
100
200
203
200
285
1.00
2003
2008
1100
2000
2018
3000
200
288
Figure A.2: Examples of injected signatures. Each example focuses on the window in which the signatures are injected (i.c. omitting the rest of the time scries), and windows are set to a fixed length of 288 (the maximum length when selecting random windows) to aid visualisation.
19

---

Published as a conference paper at ICLR 2024
Class 8: Peak A smooth peak is created from the probability density function (PDF) of ( 1) from -5 to 5, and then the values are multiplied by scalar sampled from (x(1.5.2.5) Values in the random window are then multiplied by the values of the peak, creating smooth transition from the existing time series.
The same method to generate the Peak signatures used to generate a trough, but the PDF values are instead multiplied by (R(-2.5, -1.5) (same scalar sample range but negative).
Class Trough
C.2 UCR DATASET DETAILS
For the UCR datasets, used the original train/test splits provided from the archive source." n-nomalisation was applied to datasets that were not already normalised. The exhaustive list of univariate UCR datasets used in this work is:
Adiac, ArrowHead, Beef, BeetleFly, BirdChicken, Car, CBF, Chlorinecoocenttratinnn CinCECGTorso, Coffee, Computers, CricketX, CricketY, Cricket2, DiatonSizeReduction,, DistalPhalammottiieeeeeerruu DiscalPhalaaxottnnnCcrreett DistalPhalanxIM, Earthquakes, ECG200, ECG5000, ECGFiveDays, ElectricDevices, FaceAll, FaceFour, FacesUCR, FiftyWords, Fish, FordA, FordB, GunPoint, Ham, Handoutlines, Haptics, Herring, InlineSkate, InsettVingeestSound, ItalyPowerDemand, largekiicchenkppllancee,, Lightning2, Lightning7, Mallat, Meat, Medicallimages, oidueePhaaaaoooinssgggeeeooo,o,
SualleehhiaanxottiiecCorrrrrsr Middle@PhalanxI7 oa MoteStrain, Nontnvasiiverrrttakcccchhoerrl
0live0i1, O Ince PwtitrrssttthhPhmmmeeeeeeee
hrgph//hhe....toooooooooooenoaar Phoneme, Plane,
ortpapahhhhaaaaaannneeeeeeaeiern Screerryee, ShapeletSim, o
Sttp://uurrressstteeeeeeeeeeee ottpr//raeeettttaaaaaooooreerre $cnyAT3ORkoottsrrfacce,, starLightCurves,
Strawberry, SwedishLeaf,
Symbols, SyntheticControl,, ToeSegmentationl, ToeSegmentation2, Trace, TwoLeadECG, TwoPatterns, UhaveGesturetiibraayAllll
WavveGestureL.Draryy.. UWaveGesturetibraryy
baveeessture:bbrary...
Wafer, Wine, WordSynonyms, Worms, WormsTwoClass, Yoga.
D ADDITIONAL RESULTS
D.1 INTERPRETABILITY METRICS
Below we provide more details on the metrics used to evaluate interpretability, which are based on the process proposed by Early et al. (2021).
Evaluation without time point labels When time point labels present, the model can be evaluated via perturbation analysis. The underlying intuition is that, given a correct ordering of time point importance in time series, iteratively removing the most important (discriminatory) time points should cause the model prediction to rapidly decrease. Conversely, a random or incorrect ordering will lead much slower decrease in prediction. Formally, when evaluating the interpre- tations generated by a classifier F, for time series X₁ = {x] x² ..... xf} with respect to class e, we first re-order the time series according the importance scores (with the most important time points first): O {o)
AOPCR:
)f). .f....,,.ff .......0 The perturbation metric is then calculated by:
O t-1 Σ
(X,oRF(X; Oi,;;)));
(A.8)
where MoRF Oi. MoRF 0 Oze,j 1) and MoRF O,.,0) === XX,-
{o))
herpss///ww..ccuurrreeeeeeeeeee series www.eeeeeeeeea_aaeeeeeee dataa_atta___________aa______
20

---

Published as a conference paper at ICLR 2024
MoRF is used to signify the ordering is Most Relevant First. In Eqn. A.8, the perturbation curve is calculated by removing individual time points and continues until all but one time point (the least important as assessed by the model) is left. This is expensive to compute, be made for each perturbation. To improve the efficiency of this calculation, we group time points together into blocks equal 5% of the total time series length, and only perturb the time series until 50% of the time points have been removed. As such, we only need to make 10 calls to the model per time series evaluation.
call to the model must
To facilitate better comparison between models, we normalise by comparing to random ordering. To compensate for the stochastic nature of using random orderings, we average over three different random orderings, where R. the rl repeat random ordering:
AOPC X1 Oi.)) - AO,C(X, R.))
AOPCR(X, O1.e)
(A.9)
NDCG@n: Evaluation with time point labels the time point labels are known, perfect ordering of time point importance would have every discriminatory time point occurring at the start. there are n discriminatory time points, we would expect to see these in the first n places in the ordered interpretability output. The fewer true discriminatory time points there are in the first places, the worse the interpretability output. Furthermore, we want to reward the model for placing discriminatory time points earlier in the ordering (and punish it for placing non non-disrrimintory time points earlier on). This can be achieved by placing a higher weight on the start of the ordering. Formally,
rel(Oi..s)))
NDCG@n(Or..) = IDCG
(A.10)
log2(j + +1)
where IDCG
log2(j 1)
if of is discriminatory time point, otherwise.
and ree( rel((1,,e.))
D.2 WebTraffic ADDITIONAL RESULTS
We first provide a complete set of results for predictive performance on WebTraffic, comparing the GAP with MILLET. Tables A.6, A.7, A.8 give results on accuracy, AUROC, and loss respectively.
Table A.6: WebTraffic Accuracy.
Table A.7: WebTraffic AUROC.
FCN ResNet ITime Mean
FCN ResNet
ITime Mean
GAP
0.756 0.860
0.934
0.850
GAP
0.961 0.982
0.997
0.980
Attention
0.820
0.866
0.936
0.874
Attention
0.973
0.984
0.997
0.984
Instance
0.782
0.862
0.938
0.861
Instance
0.962
0.982
0.997
0.980
Additive
0.814
0.858
0.940
0.871
Additive
0.973
0.984
0.997
0.984
Con Conjunctive
0.818
0.850
0.940
0.869
Conjunctive
0.973
0.984
0.996
0.985
Table A.8: WebTraffiic Loss.
FCN
ResNet
ITime
Mean
GAP
0.939
0.633
0.268
0.614
Attention
0.863
0.701
0.279
0.614
Instance
0.871
0.709
0.257
0.612
Additive
0.882
0.678
0.252
0.604
Conjunctive
0.866
0.638
0.277
0.594
21

---

Published as a conference paper at ICLR 2024
Table A.9 shows the complete interpretability results. Best performaaee in each comes from one of MILLET Inst Addit iive or Con jonjuiiivv. The exception is NDCG@n for CAM on Inception ionTime, which, discussed in Section 4.2, is likely due to the sparsity of MILLET explanations. We also note that SHAP performs very poorly across all pooling methods, and that Attention on is also worse than the other methods (as it does not make class-specific explanations).
Table A.9; Interpretability performance (AOPCR / NDCG@1) on our WebTruffic dataset. Results generated using the ensembled versions of the models. This is expanded version of Table 1.
FCN
ResNet
InceptionTime Mean
CAM
12.780 / 0.532
20.995 0.582
12.470 / 0.707
15.415 / 0.607
SHAP Attention
1.507 / 0.271
-3.293 / 0.250
-5.376 0.244
-2.387 / 0.257
SHAP Instance
1.977 / 0.283
-0.035 / 0.257
-4.020 0.259
-0.692 0.266
SHAP Additive
0,900 / 0.270
-1.077 / 0.250
-5.952/0.00
0.244
-2.043 / 0.256
SHAP- Conjunct
0.987 0.267
-0.552 0.225
-5.359
/0.246
-1.641 0.257
SHAP Best
1.977 / 0.283
-0,035 0.257
-4.020 0.255
-0.692 0.266
MILLET Attention
4.780 / 0.425
6.382 / 0.380
-1.597 0.420
3.188 / 0.408
MILLET - Instance
12.841 0.540
23.090 / 0.584
13.192 / 0.704
16.374 / 0.609
MILLET Additive
14.522
0.532
24.880
0.589
10.274 / 0.684
16.559 / 0.602
MILLET - Con joncutcivve
13.221
0.539
24.597
0.591
11.100 / 0.694
16.306
0.608
MILLET Best
14.522 0.540 24.880
0.591
13.192 / 0.704
17.531
0.612
D.3 UCR ADDITIONAL RESULTS
In this section we give extended results on the UCR datasets. First, Table A. compares predic- tive performance of the MILLET methods we find that Conjunctive gives the best average performance. In Table A.11 I we give MILLET interpretability results on the UCR datasets for both individual and ensemble models. Interestingly. we find that MILLET performs well on all cases except the ensemble ResNet models. In this case, its performance drops significantly compared to the individual ResNet performance something that is not observed for the other backbones. We ob- serve something similar in our ablation study, see App. E.5, We then compare MI MILLET performance with that of six SOTA methods in Table A.
Table A.10: MILLET predictive performance (accuracy balanced accuracy) 85 UCR datasets.
FCN
ResNet
InceptionIie
Mean
GAP
0.828 / 0.804
0.843 / 0.819
0.853 0.832
0,841 / 0.818
Attention
0.781 0.754
0.846 / 0.823
0.855 0.832
0.827 0.803
Instance
0,829 0.804
0.842 0.8118
0,855 0.833
0.842 / 0.819
Additive
0.835 0.810
0.845/ 0.822
0.855 0.832
0.845 0.822
Con Conjttiive
0.838 0.814
0.845 0.822
0.856 0.834
0.846 0.823
Table A.11: MILLET AOPCR interpretability performance (individual ensemble) 85 UCR datasets. Best results over all MILLET models is given for reference.
FCN
ResNet
InceptionTin
Mean
GAP
6,518 6,534
6.341 / 6.445
3.392 4.144
5,417 5.707
Attention
-0.023 0.474
0.620 1.513
-0.909 -0.936
-0.104 / 0.351
Instance
6.868 6.925
6.338 5,903
4.260 / 3.973
5.822 5,600
Additive
6.443 6.298
6.526 5.871
4.963
5.083
5.977 5.751
Conjunctive
6.361 / 6.498
6,438 / 6,006
4.553 4.764
5.784 / 5.756
MILLET Best
6.868 / 6.925
6.526/6.00
4.963 5,083
SOTA results obtained from Bake Off Redux using column zero of the results files (original train/test split). We did not use the FCN, ResNet, i'T' results we trained our own versions of these models.
22

---

Published as a conference paper at ICLR 2024
Table A.12:
Results for MILLET against baselines 85 UCR datasets. Results given in the form mean/ / rank/ / number of wins. HC2 is included for reference but not directly compared to.
Method
Accuracy
Bal. Accuracy
AUROC
NLL
Bydra-Mi
0,857 8.688 19 0.831 /9.994 17 0,875 / 18.647 / 0.953 / 11.194 /
FreshPRINCE
0.833 12.841 14 0.801 13.824
13 0.944 10.206
0.714/ 10.800
PF
0.821 / 14.729 8 0.795 15.000 6 0.924/115 12.141 / 14 0.709 10.906
RDST
0.850 / 10.729 / 13 0.822/ 11.529/ 11 0.870/ 19.165 /6 0.997 / 12.612/9
RSTSF
0.842 / 12.382 12 0.810/ 13.835/ 10 0.945/9.835 22 0.723 / 11.212/8
WEASEL-D
0.850 / 10.376 17 0.823/ 11.53/
13 0.871/19.559/
0.999 12.288 /7
FCN
0,828 / 15.053
0.804 14.512/6
19.9/13.818/ 13 1.038 11.529 /3
Attn. FCN
0.781 / 14.929/6
0.754 14.547/5
12 1.194 13.082 / 0.927/33.029
Ins. FCN
0,829 / /7
0880//4.3766
0.930/13.353 18 1.023 10.412/3
Add. FCN
0.835 / 5
0.810/13.100/5
0.933/11.594 14 0.994 / 10.247 /3
Conj. FCN
0.838 12.624/6
0.814/12.247/6
0.934/11.012/16 16 0.973 9.635 /3
ResNet
0.843 11.7111
0.819 11.27/1
0.937/10.55/ 18 1.091 13.024 /
Attn. ResNet
0.846 11.400 / 0.823
10.671/9
0.939/9.888
13 1.051 12.188 /
Ins. ResNet 0.842 / 11.918/ 10 0.818
11.653/9
0.936/ 0.936/10.394/ 10.394 17 1.071 12.553 /2
Add. ResNet 0.845 / 11.147 0,822
0.936/102882 10.500/9
14 1.073 13.082
Conj. ResNet
0.845 11.33/
10 0.822
10.806 / 10 0.939/9.299 / 15
1.035 / 12.176/1
ITime
0.853 /.832/
0.832 8.965
0.939 9.500 24
1.078 11.571 /7
Attn. ITime
0.855 99.512 10
0.832 0802/9 2.08//1
0.940 8.876 / 20
1.069 / 11.618 /3
Ins. ITime
0.855 99.471
0.833
9.053
0.940
8.406 25
1.050 / 10.929
Add. ITime
0.855 / 9.239
0.832
8.729 12
0.940 / 8.394
1,067 / 11,488 /
Conj. ITime
0.856 / 8.976 15
0.834/
8.482
17
0,939
9.0006 / 22
1.085 / 12.065
HC2
0.860 / 7.988 / 20
0.830
9.353
0.950 / 6.006 42
0.607 / 8.388
We perform a further direct comparison against the best two SOTA results, HC2 and Myydra-MR allowing us to evaluate how many and on which datasets MILLET performs better. As shown in Fig. A.3, we find that our best- perfforeig MILLET approach (Conjunct Inceptiontl wins draws 48/85 (56.5%), 49/85 (57.7%), and 52/85 (61.2%) UCR datasets against Inception HC2, and Hydra Mondraa-MRii respectively.
1.0
MILLET Better
1.0
MILLET Better
MILLET Better
Coni, lnceptioo Time Bal. Acc.
0.8
0.8
0.8
BeetleFly
Lightning2
x -30"
Jede:
0.1
0.4
0.8
CCCCooo
Harming
0.0
39/9/33
MILLET Worse
0.0
39/
/336
36
MILLET Worse
Worse
0.0
46/6/33
MILLET Worse
0.8
0.4
1.0
0.0
Inception Time Dal. Acc.
IIC2 Bal. Acc.
Bydra MR Dal. Acc.
Figure A.3: Direct comparison of our best MILLET method against SOTA. Numbers in the bottom left indicate the number of wins draws losses for Con junctivve InceptionTine Datasets with the greatest positive and negative differences indicated.
23

---

Published as a conference paper at ICLR 2024
E ADDITIONAL EXPERIMENTS
E INVESTIGATION OF FALSE NEGATIVES
Our experiments showed Conjunctive pooling gives better predictive and interpretability per- formance than other pooling methods. To further investigate why this is the case, we analysed its performance on our proposed WebTraffic dataset. We found its accuracy was between 0.86 and 1.0 for nine of the ten classes, showing it had consistently strong performance in identifying most sig- natures. However, for class (Peak), its accuracy dropped to 0.76 rather large drop relative to the other classes. Class O (None) was the prediction that was made for the majority of the incorrect predictions for class 8, i.e. the model failed to identify peak and did not find any other class signatures, predicted the None class. We used the interpretability facilitated by MILLET in- vestigate what was happening with these incorrect predictions. We found that, in some cases, the model was able to identify the correct region (positive predictions for class 8 the location of the peak) despite its final prediction being incorrect. Examples are shown in Fig. A.4 - observe how the middle example shows support for class (Peak) in the correct region despite the model getting the overall prediction incorrect. This identification of incorrect predictions and ability to analyse the model's precision-making in more detail further highlights how MILLET can be useful in production.
Incorrect Prediction but Region Found
Incorrect Prediction and Region Not Found
Correct, Prediction
Interpretation Onig. vs Sign.
Original
W
Class 8 (Peak) Class o (None)
Interpretation
1000
1000 200
Figure A.4: False negative investigation for Conjunctive ive inceptionTime on the WebTraffic dataset. For the final example, the deviation from the original time series is very small and centred near existing peak, which makes correct identification more difficult. Top: The signature time series for an example from class (Peak), where the original time series and signature region are shown. Middle: The model's 's interpretation for class 0 (None). Bottom: The model's interpretations for class 8 (Peak).
E.2 PERFORMANCE BY DATASET PROPERTIES
As discussed in Sec. 5.1, Con junct. ve Incept i conTime me had the best performance on balanced accuracy when evaluated over 85 UCR datasets (outperforming SOTA methods such HC2 and Hydra-MR). To investigate whether this improvement is due to better performance on imbalanced datasets, we assessed balanced aceuracy with respect test dataset imbalance. To measure the imbalance of dataset, we use normalised Shannon entropy:
Dataset Balance
log
(A.11)
log
1=1
where c is the number of classes, c; is the number time series for class i, and n is the total number of time series in the dataset [ - G: n). This gives of dataset balance between
24

---

Published as a conference paper at ICLR 2024
O and 1, where is perfectly balanced (equal number of time series per class) and values close to indicate high levels of imbalance. We used a threshold of 0.9 to identify imbalanced datasets, and provide comparisons of Con junctive InceptionTinn with GAP Incept CereptionTmmn HC2, and Hydra-Mi MR on these datasets in Fig. A.5.
1.0
Belanced
0.8
0.4
Time
('Imee
HC2
Balanced Accuracy
0.10
MELET
SILLET Betser
MILLET (-800)
Differncc
0.05
0.00
Wicme
Dutuset Imballnce (Normalised Shamnon Entropy)
Figure A.5: The effect of dataset imbalance on MILLET (Con (oonttiiivv InceptionTime) compared with GAP inceptionTime (left), HC2 (middle), and Bydra-ME (right). Top: Bal- anced accuracy on the imbalanced datasets for MILLET and each comparative method. Bottom: The gain in balanced accuracy on the imbalanced datasets when using MILLET. Numbers indicate MILLET's win/draw/loss and total improvement in balanced accuracy on these datasets.
From these results, we identify that MILLET has better performance than the SOTA methods when there is high (test) dataset imbalance. It wins on 8/13, 9/13, and 9/13 datasets against GAP Incepti onT ime, HC2, and Hydra-MR respectively, and improves balanced accuracy by up to 7.4%. This demonstrates MILLET is more robust to dataset imbalance than the other methods, po- tentially due to the fact that is has make timestep predictions. Note this is without any specific focus on optimising for class imbalance, e.g. weighted cross entropy loss could be used during training further improve performance imbalanced datasets.
Using the UCR results, now compare performance across time series length and the number of training time series. Figure Fig. A.6 shows the average balanced accuracy rank on different partitions of the UCR datasets for HC2, Hydra-Ma -R,, Incept LonTime, and jonjunciive Incept ioption"? ime. The results are relatively consistent across different time series lengths. How- for the number of training time series, than GAP Incepti inmimm for smaller datasets, but excels on the larger datasets.
that Con junctive ive Inception's ime is worse
3.25
3.00
Average Bank
2.50
2.25
2.00
1.75
[0.
[100, 300)
[300,
60)
[60,
[500,
Time Series Leagth
Number of Training Time Series
.....:
HC2
Hydra. MR
InceptionTimo
Conjunctive InceptionTime
Figure A.6: Comparison of Con junct iieetivve Incept i on ime against GAP, HC2, and Hydra-MR for two dataset properties. Bin ranges are chosen to give approx. equal bin sizes.
25

---

Published as a conference paper at ICLR 2024
E.3 MODEL VARIANCE STUDY
As noted by Middlehurst et al. (2023), while InceptionTing performs well overall, it often performs terribly on certain datasets, i.e. it high variance in its predictive performance. In Fig. A.7, we show that MILLET does aid in reducing variance while improving overall performance. Notably, Conjunctive InceptionTin has lower variance than HC2 and Hydra-Man
Modian Accuracy
Devtation from
0.0
-0.1
-0.2
Conj.
InceptionTmme
HC2
Hydra-MR
Conj. ITmme
Conj.
Figure A.7: Evaluation of model variance with respect to median accuracy. Models are ordered from left right by total variance.
E.4 RUN TIME ANALYSIS
Below we analyse how MILLET increases the complexity of the original backbone models. We first compare the number of model parameters (App. E.4.1) for different backbone and pooling combina- tions applied to the UCR Fish dataset, which is chosen as is relatively central in the distribution of dataset statistics (175 training time series, 463 time points per time series, and 7 classes). We then compare the training and inference times on this same dataset (App. E.4.2). Finally, we calculate the time complexity of the differnet pooling approaches and assess how they scale with respect to the number of timesteps and the number classes (App. E.4.3).
E.4.1 NUMBER OF PARAMETERS
In Table 1.13, we detail the number of model parameters for the different backbones and aggregation approaches on Fish. Instance has the same number of parameters as GAP as the only change in the aggregation process is swap the order in which pooling and classification are applied. Similarly, Atten attentiin ion, Addit ttditii vive and Conjunct sunntiii ve all have the same number of parameters as each other, as they all include the same attention head (just applied different ways). Including this attention head only leads to an increase of approximately 0.4% in the number of parameters. Note these exact values will change for datasets with different numbers of classes, but the number of additional parameters for the attention head will remain the same.
Table A.13: Number of model parameters for Fish. Percentages indicate the relative increase in parameters from the GAP model.
Pooling
FCN
ResNet
Inception' Time
GAP
265.6K
504.9K
422.3K
Attention
266.6K (+0.4%)
505.9K (+0.2%)
423.3K (+0.2%)
Instance
265.6K (+0.0%)
504.9K (+0.0%)
422.3K (+0.0%)
Additive
266.6K (+0.4%)
505.9K (+0.2%)
423.3K (+0.2%)
Conjunctive
266.6K (+0.4%)
505.9K (+0.2%)
423.3K (+0.2%)
E.4.2 TRAINING//NFERENCE TIME
In Table A.14, compare model training and inference times Fish,, and also include how long SHAP takes to run for these models. Due to the additional complexity of these methods (e.g. making time point predictions, applying attention, and using positional embeddings). the training times increase by up to 6%. Similarly, inference time increases by up to 7.5%. For SHAP, generating single explanation takes 6+ seconds compared the MILLET explanations which are generated as part of the inference step. Using Con Conjunttion ive as example, SHAP is over 800 times slower than MILLET.
26

---

Published as a conference paper at ICLR 2024
Table A.14: Run time (wall clock) analysis results using Incept on me UCR Fish. Per- centages following the MILLET methods give the increase in time relative to the backbone GAP model. Note inference time is given in milliseconds but SHAP is given in seconds.
Model
Train (seconds)
Inference (milliseconds)
SHAP (seconds)
GAP
565 + 2
7.50 = 0.02
6.21 + 0,03
Attention
597 + (+5.7%)
8.02 = 0.02 (+6.9%)
6.53 = 0,01 (+5.2%)
Instance
582 + I (43.0%)
7.75 + 0,01 (+3.3%)
6,38 + 0,02 (+2.6%)
Additive
599 (+6.0%)
8.06 0.01 (+7.5%)
6.51 = 0.02 (+4.8%)
Conjunctive
599 + (+6.0%)
8.05 = 0.01 (+7.3%)
6.51 = 0.02 (+4.8%)
E.4.3 TIME COMPLEXITY ANALYSIS
To provide more thorough theoretical analysis, we give results for the number of real multiplications in the pooling methods (Freire et al., 2022). We focus solely on the pooling methods, omitting the computation of the backbone (as it is independent of the choice of pooling method). We also omit the computation activation functions and other non-linear operations. Results are given for single inputs (no batching) and ignore possible underlying first define the number of real multiplications for common functions of the pooling methods:
pttpiiatiiooppaallllstttoooooo of functions. We
Feed-forward layer: t + d o, where is the number of timesteps, d is the input size (128 in this work), and o the output size (based on Shah & Bhavsar, 2022).
Attention head consisting of two layers: t * * a + & o, where a is the size of the hidden layer the attention head (8 in this work). This is consistent for the architectures that use attention (Attentic ion, Additive, and Con junnuuttiinn ive); see Tables A.2, A.4 and A.5. where is the number of tensors and is the
Weight average list of tensors: length of each tensor.
Given these definitions, calculate the overall number of multiplications for each pooling method.
We provide results Table A.15 and visualisation in Fig. A.8. We make several observations:
Attention ion has the best scaling with respect to and c. However, it remains a poor choice overall due to its poor interpretability performance compared to other methods.
GAP CAM and Instance are the next fastest pooling methods. Instance is more efficient than GAP + CAM when 1 * e < d * (t+ c). In this work, as 128, Instance is marginally quicker than GAP CAM.
Additive and Conjunct iive ive are the two slowest methods due to the additional over- head of applying attention and also making instance predictions, but the margin between them and GAP CAM / Instance is not as large as one might expect. This is due to the small hidden layer size in the attention head (a = 8). Con junet ve is more efficient than Addit siditivee ve when e < d, which is true in all cases in this work as d = 128.
In general, Con Jonjunctive than predictive performance. If faster computation is required, or if the number of timesteps number of classes very large, Instance becomes better choice.
the best choice when computational cost is less important
Table A.15: Time complexity analysis (number of real multiplications) different MII pooling ap- proaches. For GAP, we include the calculation of CAM this is required to produce interpretations (the other pooling methods produce interpretations inherently). We also state how the number of real multiplications scale with respect to the number of timesteps and the number of classes c.
Pooling Method Number of Real Multiplications
Scale w.n.f
Scale w.n.t
GAP CAM
d+t*c+
d+c+d
d+t
Attention Instance
d+t (2d+
d d+c+c
d d+t+t
d*++c+i*c
Additive
dxtxctdxtsa-(atdd
dxc+
d+t+t
Conjunctive
d+t*c+d#iva- (
2c
d ++t
27

---

Published as a conference paper at ICLR 2024
Classes
Classes
Claseee 10
Clasees
Clossee 50
Compleatty
10
260
500
750
1000
00
270
100
700
1000
60
250
100
790
1030
210
100
710
1030
210
100
710
1020
Time Series Length
2007 Length
Length 250
Length = 500
Length 750
Length 1000
T1me Complecity
60
30
20 Number of Classes
50
10
20
60
90
10
GAP + CAM
Attencion
Instanoe
Additive
Conjunctive
UCK Dataver Mean
Figure A.8: Visualisation of time complexity (number of real multiplications) for the different pool- ing methods used in this work. The mean time series length / number of classes across the 85 UCR datasets used in this work shown (vertical dashed lines). Top: Time complexity as time series length increases for varying number of classes. Bottom: Time complexity as the number of classes increases for various time series lengths.
E.5 ABLATION STUDY
Our MILLET models make several improvements over the backbones, adding MIL pooling, po- sitional encodings, replicate padding. and dropout (Sec. 3.4). understand where the gains in performance over the backbone models come from, conduct ablation study. To do so, additional model training rums, starting with the original backbone models and incrementally adding MILLET components in the order: MIL pooling, positional encoding, replicate padding, dropout, and ensembling. The final stage represents the full MILLET implementation. To reduce overheads in model training time and compute resources, conduct the study on the three UCR datasets where MI LLET shows the biggest increase in balanced accuracy the backbone: Beet e aFll. Li Liihhtnigg) ng7, and FaceAll.
focus Con junctive Incepti innT conTime ime and
In Table A.16 we provide results of the ablation study for balanced accuracy (predictive perfor- mance). We note that, on average, each component of MILLET improves performance, and the complete implementation (Step 6) has the best performance, For these datasets, the use of MIL pooling always improves performance over GAP. Additional components then change the perfor- mance differently for the different datasets. For example, positional encoding is very important for Beet eFly, but replicate padding most important for Fa FaceAll. Interestingly, replicate padding gives the biggest average performance increase across these datasets. However, these re- sults are confounded by the order of implementation, i.c. replicate padding only applied after MIL pooling and positional encoding have been applied. As such, further studies are required untangle the contribution of each component, but that is beyond the scope of this work.
Table A.16: Ablation study on balanced accuracy. We begin with the original backbonc model (without ensembling). We then incrementally add MILLET components until reach the complete MILLET model. Results are given as balanced accuracy improvement, where improvement the difference in balanced accuracy relative the prior row.
Model Config
Beet leFly
Lightning7
FaceAll
Mean
1. GAP (Single)
0.870
0.819
0.910
0.867
2. MIL
0.870 +0.000
0.844 +0.024
0.912 +0.002
0.875 +0.009
3. Pos Enc
0,900 +0.030
0.832
-0.012
1911//0.000
0.881 / +0.006
Replicate
0.900 +0.000
0.840
+0.008
0.967 / +0.057
0.903 / +0.022
5. Dropout
0.920 +0.020
+0.008 0.848
0.970 +0.003
0.913 +0.010
6. Ensemble
0.950 +0.030
0.863 +0.015
0.974 +0.003
0.929 +0.016
28

---

Published as a conference paper at ICLR 2024
In Table A.17 provide results of the ablation study for AOPCR (interpretability performance). Interesting, the addition of MIL pooling and positional encoding is detrimental to interpretability in these examples, despite improving predictive performance. However, interpretability improves once replicate padding and dropout are included. Further work is required to understand if these interpretability increases would also occur if replicate padding and dropout were applied the GAP backbones, or if they improve performance when applied in conjunction with MIL pooling, Finally, observe that interpretability decreases when ensembling the models. This is somewhat intuitive, as the interpretations are now explaining the decision-making of five models working in conjunction it would be interesting explore the difference in interpretations when analysing each model in the ensemble separately rather than together.
Table A.17: Ablation study on AOPCR. Results are given as AOPCR / improvement.
Model Config
BeetleFly
Lightning7
FaceAll
Mean
1. GAP (Single)
0.122
4.085
0.552
1.586
2. + MIL
-0.736 -0.858
3.985 -0.100
0.859 / +0.306
1.369 / -0.217
3. + Pos Enc
-1.978 -1.241
3.863 / -0.121
0.633 / -0.225
0.840 / -0.529
4. + Replicate
-1.926 +0.055
4.093 40.230
4.023 +3.390
2.064 +1.224
5. + Dropout
1.242 / +3.168
4.253 / 45.159
3,997 / -0.026
3.164 / +1.101
6. + Ensemble
0.787 -0.456
4.190 / -0.063
3.585 / -0.412
2.854 -0.310
29


Reviews:
[{"review_id": "K6OGLddFsF", "reviewer": "Reviewer_NeUd", "summary": "This paper introduces a framework based on Multiple Instance Learning to enhance the interpretability of time series classification models. The framework, MILLET, proposes passing the extracted feature embeddings from any backbone (e.g. FCN, ResNet) through a positional encoding, dropout, and a final pooling layer. Depending on the pooling structure, the method can make the underlying model more interpretable in some settings while also improving performance in others. The authors propose a new pooling method, conjunctive pooling, specifically for time series. MILLET is evaluated through the construction of 12 new models on 85 UCR datasets and a newly introduced synthetic dataset for interpretability evaluation. This approach represents a novel application of MIL to TSC and offers improved interpretability in various domains.", "questions": "These questions will help clarify my understanding of the paper. Some of these could benefit from additional analysis in the paper itself:\n\n1/ What is the author’s intuition for the added performance of conjunctive pooling over other pooling methods?\n2/ In Figure 6, are the x’s referring to different pooling methods for MILLET or multiple runs of the Conjunctive pooling model?\n3/ One of the most interesting things about the WebTraffic dataset is the different signatures grounded in real-world patterns. The authors note that the Conjunctive InceptionTime model identifies regions around Spikes and only the start and end of Cutoffs. Does classifier or pooling selection change how the interpretability functions or performs across these various class types? Can this tell us anything more about how the Conjunctive Pooling functions or why it performs better?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Thanks to the authors for their submission: it contains useful research that shows good research practices while explaining an interesting and novel idea within multivariate time series classification and interpretability. The results of this work will be informative to other researchers and are significant in improving our understanding of applying deep learning methods with time series. Some specific strengths of this research:\n\n- The Multiple Instance Learning presented in this work has more general applications within time series classification than previous work and provides a more robust evaluation of the benefits and drawbacks across a range of tasks, both synthetic and real.\n- MILLET model design adds very little complexity to existing models while contributing improved interpretability. It is flexible enough to work with any backbone model (FCN, ResNet, InceptionTime, and more) while maintaining performance. \n- The proposed synthetic dataset, WebTraffic, provides a helpful contribution to the task of benchmarking time series interpretability. With the ability to scale up to large sizes and the replication of a common time-series use-case it could be a helpful foundation to build-on in the future.\n- Performance of Conjunctive Pooling shows improvements across the class of neural network models for time series classification.", "weaknesses": "While the results and paper are generally strong, there are a few areas for improvement particularly as regards to interpretability claims:\n\n- Lack of comparison against previous benchmarks for saliency maps for feature attribution in time series classification from TSR [1], DynaMask [2], WinIT [3], and TimeX [4]. All of these works provide additional synthetic datasets for evaluating interpretability methods and show performance against more general methods like Feature Occlusion and Integrated Gradients. While MILLET seems like to improve on such methods due to the better computational efficiency, it is not thoroughly evaluated in the paper, except in counting the number of forward passes in the difference between SHAP, CAM, and MILLET.\n\n- Interpretability evaluation metrics. It is not clear that AOPCR and NDCG@n can be strictly ported over to the time-series setting. For example, as pointed out, with NDCG@n the time points in the middle of a region of missing data may be considered important by the ground truth, but may not be highlighted by the interpretability method, instead the beginning or end may be highlighted. These scores may be weighted differently for similar outcomes. More discussion around the impact of this is relevant to researchers.\n \n[1] https://arxiv.org/pdf/2010.13924.pdf \n[2] https://arxiv.org/pdf/2106.05303.pdf \n[3] https://arxiv.org/pdf/2107.14317.pdf \n[4] https://arxiv.org/pdf/2306.02109.pdf", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4UnilFr9aX", "reviewer": "Reviewer_pSU6", "summary": "The paper introduces a novel intrinsically interpretable deep learning model. The authors framed time series classification as a Multiple Instance Learning (MIL) which can highlight the most influential time points in the outcome of the model. This method employs a various techniques such as attention, instance pooling, additive pooling, and conjunctive pooling across an ensemble of deep methods where each method offer different interpretability.", "questions": "Overall, this paper could be a significant algorithmic contribution and I think the authors done amazing job on presenting it. I wonder if the method can be applied to other domains.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Nicely written. Well evaluated. Novelty.", "weaknesses": "I was not able to identify any weakness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "thBfylEXdt", "reviewer": "Reviewer_uQF7", "summary": "The paper presents MILLET a model for Multiple Instance Learning for Locally Explainable Time series classification.", "questions": "Questions can be derived from the weaknesses part.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper is well written and all the choices are justified and carefully explained\n+ The experimentation is deep\n+ The proposal is novel (to the best of my knowledge)\n+ The references are updated", "weaknesses": "- I would have appreciated a comparison with LIME or with LIMESegments\n- I would have appreciated a comparison against ROCKET or MiniROCKET at least as competitor for the TSC task. Further usage of MILLETS also for ROCKET will completely fulfill the purpose of proposing this approach as a model-agnostic one.\n- To fully understand the paper the reader is constrained to refer to the Supplementary Material. A suggestion is to save some space and anticipate in the main paper some of the details of the Supplementary Material.\n- Experiments with the synthetic dataset should have been performed by varying the number of records, time stamps, classes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ebJKsudcJ3", "reviewer": "Reviewer_Yc2u", "summary": "This paper introduces a new framework that leverages Multiple Instance Learning to make deep learning TSC models inherently interpretable without compromising predictive performance. The authors evaluate MILLET on 85 UCR TSC datasets and show that it produces sparse explanations quickly and of higher quality than other interpretability methods.", "questions": "1.\tHow does MILLET perform with other TSC models?\n2.\tHow does MILLET compare with other TSC interpretability methods?\n3.\tCan you provide a more detailed analysis of the algorithm complexity?\n4.\tHave you considered the potential impact of the choice of hyperparameters on the performance of MILLET?\n5.\tHave you considered the potential impact of class imbalance on the performance of MILLET?\n\nAfter reading the author's rebuttal and discussions I am more incline to accept the paper.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The strengths of this paper include:\n\n1.\tIntroducing a new MILLET framework that makes deep learning TSC models inherently interpretable by leveraging the MIL approach without compromising predictive performance. In particular, the authors proposed exploring four MIL (attention, instance, additive and conjunctive) pooling methods to increase interpretability while replacing GAP. Moreover, including positional encoding ensures the modelling of time series constraints.\n2.\tProposing a new synthetic dataset called WebTraffic to explore the MILLET concept and evaluate the inherent interpretability of their models. The authors compared the four proposed MIL pooling approaches for MILLET with GAP on their WebTraffic dataset. Each pooling method is applied to the FCN, ResNet, and InceptionTime backbones.\n3.\tEvaluating MILLET on 85 UCR datasets and showing that it produces sparse explanations quickly and of higher quality than other interpretability methods. The authors found that \nwhile Conjunctive InceptionTime is the best approach for balanced accuracy (outperforming the HC2 and Hydra-MR SOTA methods), it is not quite as strong on the other metrics. However, it remains competitive, and for each backbone, using MILLET improves performance across all metrics. Moreover, the authors found that the Conjunctive has the best interpretability performance.", "weaknesses": "The major weaknesses are summarized below:\n\n1.\tThe paper does not provide a detailed comparison of MILLET with other state-of-the-art TSC models. Although the authors claim that “We design three MILLET DL models by adapting existing backbone models that use GAP: FCN, ResNet, and InceptionTime. While extensions of these methods and other DL approaches exist (see Foumani et al., 2023), we do not explore these as none have been shown to outperform InceptionTime (Middlehurst et al., 2023).” the application of MILLET to other models and further comparisons with other state of the art TSC is missing and is relevant better to measure the effectiveness and generalizability of the proposed approach.\n\n2.\tThe paper does not provide a detailed comparison with other TSC interpretability methods. (e.g.LIME)\n\n3.\tThe paper does not provide a detailed algorithm complexity analysis. While the authors provide information on the run time of MILLET (see E.3), a more detailed analysis of the algorithm complexity would help establish its scalability and feasibility in large-scale applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)