Improving Counterfactual Explanations for Time Series Classifications Models in Healthcare Settings
Anonymous Author(s)
AffiliationAddress email
Abstract
Explanations of machine learning models’ decisions can help build trust between amodel and a user, as well as identify and isolate unexpected model behavior.
Timeseries data, abundant in medical applications, and their associated classifiers pose aparticularly difficult explainability problem due to the inherent feature dependencythat results in complex modeling decisions and assumptions.
Counterfactual explanations for a given time series tells the user how the input to the model needs tochange in order to receive a different class prediction from the classifier.
While afew methods for generating counterfactual explanations for time series have beenproposed, the needs of simplicity and plausibilty have been overlooked.
In thispaper, we propose an easily understood method to generate realistic counterfactualexplanations for any black box time series model.
Our method, Shapelet-Guided Realistic Counterfactual Explanation Generation for Black-Box Time Series Classifiers (SGRCEG), grounds the search for counterfactual explanations in shapelets,which are discriminatory subsequences in time series.
SGRCEG greedily constructs counterfactual explanations based on shapelets.
Additionally, SGRCEGalso employs a realism check, so the likelihood of producing a counterfactual that isnot plausible is minimized.
Using SGRCEG, model developers as well as medicalpractitioners can better understand the decisions of their models.
Time series classification has widespread applications in the medical field including anomaly detectionand assisting in diagnosis.
Especially in such high stakes situations, there has been growing awarenessthat model building and the ability to explain model decisions must co-exist.
The dependent natureof time series features as well as the complexity of the models used to learn patterns from them,however, make it particularly difficult to build digestible explanations.
For tabular data, counterfactualexplanations offer an intuitive way of explaining classifier decisions.
Counterfactual explanations tellthe user how a given input to the model needs to change in order to receive a different class predictionfrom the classifier [13].
For the counterfactual explanation to be useful, it needs to be close to the original input and realistic (i.e., in distribution).
Counterfactual explanations for time series data and their respective classifiers are more complicated.
While the definition of counterfactual explanations for time series is identical, the goal is to find a similar time series to the input time series but one that is different enough to be assigned a differentlabel by the classifier, time series counterfactual explanations are inherently more complex due to thedependent structure of their features.
While a few methods for generating counterfactual explanationsfor time series have been proposed, our goal is to offer an improved counterfactual explanation generation method so the results are tailored to medical practitioners’ needs.
To answer the specific challenges posed by time series and their associated models, we propose a two step method that first grounds the search for counterfactual explanations in key subsequences of a given set of time series, creating a common language with which to discuss these explanations, and then second, generates realistic counterfactual explanations for any black box model using these key subsequences.
The method, Shapelet-Guided Realistic Counterfactual Explanation Generation for
Black-Box Time Series Classifiers (SGRCEG, pronounced “sugar keg”), uses shapelet classificationto identify subsequences that are globally representative of a model’s classification decision.
Figure 1shows one time series from each class from the ECG200 data set [4] with shapelet sequenceshighlighted.
Looking at Figure 1, a practitioner can focus on which windows, defined by shapelets,the model is using to decide to which class a time series belongs.
Once the key subsequences are identified, given a time series of interest and a dictionary of the start ofshapelet sequences and their lengths, SGRCEG sequentially replaces the window defined by shapeletsequence in that time series with the aligned sequence from a “close” time series with the desiredpredicted label until the label changes.
SGRCEG also employs a realism check, so the likelihood ofproducing a counterfactual that is not plausible is minimized.
Using SGRCEG, a practitioner couldexamine multiple time series, and by focusing on counterfactual explanations generated within thesekey subsequences within each time series, they can quickly gain a better understanding of modelbehavior. Furthermore, SGRCEG can provide actionable direction with respect to the robustness ofthe model.
For example, if counterfactual explanations are generated with a short substitution, thismay be an indication the model is too sensitive, and the model may require tuning or an expandedtraining set.
Finally, SGRCEG is suited for any univariate time series classification model and doesnot assume any knowledge of the true labels or model type.
In the following sections, we first cover related methods for finding time series classification counterfactual explanations.
We then introduce our method and show how it can be used to generatecounterfactual explanations for healthcare time series.
Finally, we outline areas for future research.
Related work
While there have been a few methods proposed to generate counterfactual explanations for timeseries [8, 9], we focus on the two most relevant frameworks to our work, CoMTE, a method proposedby [1], and an instance-based method put forth by [6], for the sake of brevity.
CoMTE is a modelagnostic method that works with multivariate time series.
CoMTE greedily replaces entire univariatesequences of the the time series of interest with univariate sequences from a “distractor,” a samplefrom the training set which is as similar as possible to the queried time series but receives a differentpredicted label.
This method does not use information about influential subsequences.
In theunivariate case, CoMTE collapses to find the closest time series in the other class.
While this doesguarantee a plausible time series, it may not produce the closest counterfactual explanation, and itdoes not offer succinct insight into what the key differences are between the two time series.
A similar method proposed by [6], which we will call the Native Guide method, works on univariatetime series.
They find the “native guide” or “nearest unlike neighbor” (NUN) which is equivalent tothe distractor from [1], then use it to replace parts of the queried time series until the label changes.
While their experiments use a neural network and associated class activation maps, they note theirapproach is model agnostic and can be used in combination with any vector-based feature importancemethod (e.g., Shap values).
Using local feature importance methods to identify key subsequencesadds a layer of complexity to their framework that is not practical for a setting like healthcare where efficiency is key.
The authors of [6] raise concerns about the interpretability of shapelets. However [5],[7], and [14] show shapelets help practitioners understand model behavior.
Additionally, shapeletsoffer a global view of important subsequences, which we believe could help ground analysis forpractitioners.
They offer insight into what subsequences are actually informative and thus assistin ignoring random noise.
Shapelets also have the added advantage of being easy for layman tounderstand, as their goal, finding “shapes” that are present in one class and not in another, is intuitive. A reviewer alerted us to the parallel work of [2] published in August of 2022.One key differencebetween SGRCEG, which was developed primarily during summer of 2021 while authors Hanand Akbarian were interns at Cognitive Scale, and SETS is SETS is for multivariate time serieswhile SGRCEG is focused on univariate time series. Additionally, SGRCEG is more flexible inthat it requires fewer predefined parameters, particularly in the generation of the shapelet dictionary. SETS uses a shapelet algorithm that requires predetermining the length of the shapelets, whereaswe specifically chose an algorithm which does not.
The shapelet algorithm choice may limitinterpretability, since the length of important sequences is likely unknown.
[2] also includes anadditional step which eliminates shapelet candidates based on a distance threshold, requiring anotherpredetermined parameter.
To find a nearest unlike neighbor, SETS uses KNN, so the user must alsodetermine an appropriate k , and lacks the flexibility that SGRCEG offers with the ability to use anysimilarity measure for finding NUNs.
Finally, SETS uses three metrics to determine plausibility:isolation forest (IF), one class support vector machine (OC-SVM), and local outlier factor (LOF)similar to [6] in a post-hoc fashion rather than being built into the algorithm.
SGRCEG uses LOF inthe algorithm itself, and could be changed to another outlier detection method.
Method
Let T i
= { t 1 , t 2 , ..., t m } , i = 1 , ..., n , be a univariate time series of length m with associated labely i
= l , l = 1 , ..., k , where k is the number of classes.
1 Denote D =
{ T 1 , T 2 , ..., T n } as a set oftime series, and D tr = { T k } n tr k =1 and and D te =
{ T k } n te k =1 are the training and test sets, respectively,where D tr ∩ D te = ∅ and D tr ∪ D te =
D.
Suppose we have a queried time series, T q ∈ D , for which we want to find a counterfactualexplanation, and a classifier C such that C : T q (cid:55)→ ˆ y q where ˆ y q is the predicted class label given to
T q by C .
Then the counterfactual explanation, T cf , is a time series such that C :
T cf (cid:55)→ ˆ y ′ , where ˆ y ′is the desired class label.
Ideally, T cf is similar to T q so that the change is as minimal as possiblewhile still being realistic.
Algorithm 1 Counterfactual Explanations Generation Method
Require: T q ∈ D, D
Tr = { T 1 , T 2 , ..., T n } , classifier C , shapelet set S
T cf =
None
T NUN =
min d ( X q , X t ) , X t ∈ D
Tr where C ( T NUN ) ̸
= C ( T q ) for ( s start , s length ) in S do for i = 1 : s length do
T c =
( T q [: s start ] , T NUN
[ s start : s start + i ] , T q [ s start + i :])
▷ Candidate explanation if C ( T q ) ̸
= C ( T c ) then
T cf = T c
Return T cf if T cf is None then
Return T NUN
The first step of the SGRCEG framework is to identify global subsequence locations that areinformative of the classifier’s predictions and present these to the user.
Given a classifier C and timeseries set D , we extract these subsequence locations using shapelet analysis. Once the classifier isbuilt, we save the classifier’s labels for each time series. Then using those labels build a library ofshapelets.
Shapelets are subsequences that are present in one class, but not other classes. We use [10] to extract a library of shapelets, which does not have the user predetermine the shapelet length.
We sort the extracted shapelets by their respective information gain, and construct S, a sorted list of tuples ( s start , s length ) of the starting index of the shapelet sequence and the of thesequence.
SGRCEG then presents the shapelets to the user so they are familiar with where the counterfactualexplanations will be generated, but due to space constraints, we cannot show all of them.
Figure 1 demonstrates a condensed version of our idea.
It shows one time series per each class fromthe ECG200 data set [4] with a shapelet sequence highlighted.
Practitioners can examine thesesubsequences to gain an understanding of which windows are important to the classifier.
Having identified where in the time series sequences to look for counterfactual explanations, the usercan then query SGRCEG for counterfactual explanations.
The counterfactual explanation generationprocess is detailed in Algorithm 1.
For a given queried time series T q , SGRCEG first finds thenearest unlike neighbor (NUN), T NUN where T NUN is closest under some distance d ( · , · ) where
C ( T NUN ) ̸ = C ( T q ) .
We tested dynamic time warping (DTW) and Euclidean distance for oursimilarity measure, but any appropriate measure or algorithm can be used.
DTW is resistant to the“stretching”/“warping” of time series that other measures (e.g., Euclidean) can be sensitive to [12].
Since there is not one single measure that is the best for all time series data, SGRCEG includes thisflexibility to account for one’s prior knowledge of the data.
Then, beginning at position s start for the shapelet with the highest information gain, SGRCEGsubstitutes values from T NUN into T q for a maximum subsequence of length s length .
At eachsubstitution we checked if the classifier assigns the perturbed version of T q , called T c , the desiredlabel.
If T c ’s prediction does not change after the end of the shapelet sequence is reached, SGRCEGrepeats the process with the shapelet with the next highest information gain, and so on until C ( T c ) =
y ′ or have exhausted all shapelets.
If all shapelets are exhausted and no plausible counterfactualexplanation is found, we use the NUN as the counterfactual explanation.
Finally, SGRCEG checks if the counterfactual explanation candidate is in distribution with a localoutlier factor model (LOF), which is an algorithm that uses density-based outlier detection to givea score to an observation indicating if it is an outlier [3].
We chose LOF as our plausibility check,but this could be replaced with another outlier detection method. SGRCEG rejects a counterfactualcandidate if it is not in distribution. Thus, it makes sense to use a less sensitive method for outlierdetection, which ideally strikes a balance between plausibility and sparseness.
Electrocardiogram Case Study
To explore the effectiveness of SGRCEG, we use the ECG200 dataset from [4], which contains 200electrocardiogram time series of length 96, and has two classes: a normal heartbeat and a myocardialinfarction event.
We use the predefined training and test set splits that each contain 100 time series.
We train a k-nearest neighbors classifier from [11] with Euclidean distance using the predefinedtraining set.
We compare SGRCEG with the Native Guide method proposed by [6] where the featureimportance vectors are Shap values from the NUNs. [Table]
The results of these experiments are summarized in Table 1.
We used two versions of SGRCEG: onewhere we used any shapelet associated with any class (SGRCEG) and one where we used shapeletsof the opposite class to inform substitution (Separate SGRCEG).
Overall, SGRCEG outperformsthe Native Guide method and Separated SGRCEG, successfully finding more valid counterfactualexplanations that are more succinct.
Native Guide (Shap) does return counterfactual explanations thatare marginally more realistic on average (i.e., higher LOF mean), but this is not surprising given the
NUN is returned more frequently, which are by definition more realistic.
Figure 2 shows a queried time series that was predicted to have a myocardial infarction (pink),where SGRCEG looks for counterfactual explanations (blue stars), and the generated SGRCEGcounterfactual explanation (blue dots).
Looking at these results, a user can first identify which timewindow is important for receiving this prediction (i.e., the shapelet sequence) and how the time seriesneeds to change to receive a different prediction.
Since the explanations are succinct and groundedwithin a window the user expects, SGRCEG allows a user to quickly examine multiple queried timeseries in order to gain understanding about model behavior.
Conclusions & Future Work
We have presented a novel way to generate plausible and interpretable counterfactual explanationsfor time series data.
In a real world example, we found SGRCEG typically produces counterfactualexplanations that are more succinct and realistic when compared to a competing method.
As this is anew area of research, there are still many paths for future work.
It would be interesting to optimizewhere to find the ideal index at which to start the substitution, and it is also worth exploring differentshapelet methods.
There are also other methods for determining whether or not a sample may be in distribution, whichstill needs to be explored in SGRCEG.
Another interesting research direction would be to align timeseries or match time stamps so SGRCEG could work with time series of different lengths.