Paper: MIRAGE: Modelling Interpretable Multivariate Time Series Forecasts with Actionable Ground Explanations (ICLR-2024)

Reviewer comment (Novelty):
What is the exact novelty of Mirage over T-DPSOM paper? What is the motivation behind DMM architecture?

Author response:
Dear Reviewer, 

Thank you for your valuable feedback which has helped us reshape our paper.  We would like to firstly address your primary comment on what is the exact novelty of MIRAGE and purpose of  DMM.
Transformers based Multi-variate Time Series (MTS) forecasting has made significant progress (with very negligible errors). However, with the advances in AI architectures, model interpretability and explainability have not received much attention (partially also because these terms have not yet been fully understood). Also, existing knowledge has well established exclusivity between these three objectives - prediction accuracy, interpretability and explainability and often the prior objective is more sought after.  However, in critical situations like predicting a death in an ICU or sudden gaming overindulgence affecting ones mental well-being; an accurate prediction (forecast) without a contributing evidence (explanation) is irrelevant. 
 In our paper, we refer to the only available SOTA network focussed on interpretability - SOM-VAE (ICLR 2019) and its refined variant T-DPSOM. But here, authors make an assumption on data linearity over time. We quantify that our non-linear, a-periodic time series data of players’ gaming indulgence violates these constraints and warrants a relook at the solution( Section Motivation).  MIRAGE comes up with a novel formulation based on the fundamental intuition that the latent state of a process helps infer if the immediate next data point is going to be linear or non-linear.  MIRAGE incorporates a gradient based method of learning a DMM ( Discrete Markov Model ) as a part of the T-DPSOM model.  However, yet it becomes hard to guarantee a smooth temporal trajectory and how MIRAGE trains a damping factor from the DMM output to supplement the interpretability is MIRAGE. We now provide additional quantitative metrics to validate MIRAGE driven interpretability for a non-linear player dataset. We use interpretability to also identify (classify) future player trajectories as overindulgent or healthy. We report improved accuracy (precision/recall) with MIRAGE vs the T-DPSOM model in the Evaluation Section

Reviewer comment (Evaluation):
In Table 1, Mirage underperforms Crossformer over MSE/MAE score for all public data.

Author response:
- Comment regarding under-performance of MIRAGE w.r.t. Crossformer
Paper states that MIRAGE MSE results do not compete with any of the Transformer based models, which are the SOTA models for predictions. MIRAGE does better than LSTNet (Lai et al. , 2017) (which is one of the best industry standard benchmarks after the Transformer based models) on ETTh1 and ETTm1 datasets (quoted in Crossformer - ICLR 2023). LSTNet specifically focuses on MTS forecasting and uses a CNN to extract cross-dimension dependencies and LSTM for cross-time dependencies. We have corrected our bold markings of figures which were intended to highlight against LSTnet and had mentioned so, in the writeup.  Our humble apologies the confusion.

Reviewer comment (Presentation):
In Table 1, authors bolded the values of Mirage in MSE, where clearly Crossformer has the lowest scores. Is this mistake done to create misinterpretation for the reviewers or just a type error?

Author response:
- Comment regarding under-performance of MIRAGE w.r.t. Crossformer
Paper states that MIRAGE MSE results do not compete with any of the Transformer based models, which are the SOTA models for predictions. MIRAGE does better than LSTNet (Lai et al. , 2017) (which is one of the best industry standard benchmarks after the Transformer based models) on ETTh1 and ETTm1 datasets (quoted in Crossformer - ICLR 2023). LSTNet specifically focuses on MTS forecasting and uses a CNN to extract cross-dimension dependencies and LSTM for cross-time dependencies. We have corrected our bold markings of figures which were intended to highlight against LSTnet and had mentioned so, in the writeup.  Our humble apologies the confusion.

Reviewer comment (Writing):
Overall the paper is very hard to conceive, especially Section 4 Evaluation.

Author response:
- For comments related to lack of clarity etc.
We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, and figure overcrowding.  We have re-arranged our writing to clearly explain our motivation and the differentiation we add. We have refined our description of the model and architecture adding much more clarity and re-discussed some of the evaluation findings.

Reviewer comment (Experiments):
For someone not a medical/health data expert, the real-world medical data needs a bit more explanation. What do the time series represent, length of the sequence, condition, etc.? Is bilirubin level a time-series feature for a patient?

Author response:
- For Medical Data
eICU data is a time series data for patients' vital signs  (blood and other body parameters) out of ICU, from the eICU Collaborative Research Database ( Goldberger et al.). We have added more details on how the heat maps and graphs were generated in the new revision of our paper. To summarise, colour of a cell in the heat map (2-D SOM map of the centroids) represents the mean of the metric under evaluation ( -e.g Apache health score or Players overindulgence score) of the points which map to it. Each cell represents SOM cluster and scores were normalised [0-1] before taking mean.

Reviewer comment (Presentation):
It is unclear how the trajectory steps are being tracked and how the heatmap is being generated. Does the heatmap represent the attention weights?

Author response:
- For Medical Data
eICU data is a time series data for patients' vital signs  (blood and other body parameters) out of ICU, from the eICU Collaborative Research Database ( Goldberger et al.). We have added more details on how the heat maps and graphs were generated in the new revision of our paper. To summarise, colour of a cell in the heat map (2-D SOM map of the centroids) represents the mean of the metric under evaluation ( -e.g Apache health score or Players overindulgence score) of the points which map to it. Each cell represents SOM cluster and scores were normalised [0-1] before taking mean.

Reviewer comment (Writing):
The paper uses many abbreviations without terminology, e.g., acf.

Author response:
- For comments related to lack of clarity etc.
We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, and figure overcrowding.  We have re-arranged our writing to clearly explain our motivation and the differentiation we add. We have refined our description of the model and architecture adding much more clarity and re-discussed some of the evaluation findings.

Reviewer comment (Presentation):
Table and figure captions are not self-explanatory.

Author response:
- For comments related to lack of clarity etc.
We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, and figure overcrowding.  We have re-arranged our writing to clearly explain our motivation and the differentiation we add. We have refined our description of the model and architecture adding much more clarity and re-discussed some of the evaluation findings.

Reviewer comment (Presentation):
The reviewer had to go back and forth multiple times to understand the entire loss function. Using proper equation labels would have helped a lot.

Author response:
- For comments related to lack of clarity etc.
We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, and figure overcrowding.  We have re-arranged our writing to clearly explain our motivation and the differentiation we add. We have refined our description of the model and architecture adding much more clarity and re-discussed some of the evaluation findings.

Reviewer comment (Presentation):
Evaluation analysis explanations are hard to conceive. Authors first start with Fig. 5 then suddenly jump back to Fig. 3.

Author response:
- For comments related to lack of clarity etc.
We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, and figure overcrowding.  We have re-arranged our writing to clearly explain our motivation and the differentiation we add. We have refined our description of the model and architecture adding much more clarity and re-discussed some of the evaluation findings.

Reviewer comment (Presentation):
The reviewer is not clear on how the figures (3–5) are generated.

Author response:
- For Medical Data
eICU data is a time series data for patients' vital signs  (blood and other body parameters) out of ICU, from the eICU Collaborative Research Database ( Goldberger et al.). We have added more details on how the heat maps and graphs were generated in the new revision of our paper. To summarise, colour of a cell in the heat map (2-D SOM map of the centroids) represents the mean of the metric under evaluation ( -e.g Apache health score or Players overindulgence score) of the points which map to it. Each cell represents SOM cluster and scores were normalised [0-1] before taking mean.

Reviewer comment (Novelty):
Overall, the Mirage results show underperforming Crossformer on both public and real-world data. Then what is the usefulness of the Mirage architecture?

Author response:
- Overall MIRAGE MSE results outperform the only available SOTA benchmarks for interpretability - SOM-VAE and T-DPSOM and LSTnet which is a pure time series forecasting model. MIRAGE underperforms to Crossformer and its variants, which however do not address interpretability and explainability 
- MIRAGE shows impressive explanations for player overindulgence and  ICU re-admissions or healthy discharge cases. 
- MIRAGE extends SOTA networks for interpretability on non-linear datasets and quantifies the results (newly added).
- For the open real world datasets (ETTh and others) we contend that MIRAGE is probably ahead in time, to reveal interesting facts on which covariates in the data gives an early indication of rise in electricity demand in a city with no ground truth available to validate.
- Lastly, we believe that open sourcing MIRAGE will significantly elevate the state of preventive health care with proactive causal and actionable explanations.

Reviewer comment (Writing):
Not sure I understand the premise of interpretability/explainability used in paragraph 1 of the introduction.

Author response:
We also discuss  interpretability vs. explainability and how they differ.

Reviewer comment (Presentation):
Figure 1 is overcrowded and confusing. Most elements are undefined. Poor quality (delineation of underlying elements, poor alignment). Figure 2: what is CCE loss? What is shift loss?

Author response:
Thank you for your valuable feedback which has helped us reshape our paper.  We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, undefined notations and figure overcrowding, hyphenation and parentheses issues, to the best of our ability.

Reviewer comment (Presentation):
Figures in the experimental results section are generally illegible with little or no labeling. Fig 4, for example: what are alternate features? What do the medical variables correspond to, and how does this correlate to medical insight? There is no legend for what dashed/solid lines correspond to.

Author response:
Thank you for your valuable feedback which has helped us reshape our paper.  We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, undefined notations and figure overcrowding, hyphenation and parentheses issues, to the best of our ability.

Reviewer comment (Writing):
A lot of notation is undefined. e.g. what is index $w$? Difference between $Z$ and $z$?

Author response:
Thank you for your valuable feedback which has helped us reshape our paper.  We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, undefined notations and figure overcrowding, hyphenation and parentheses issues, to the best of our ability.

Reviewer comment (Writing):
Definition of abbreviations (many in abstract!).

Author response:
Thank you for your valuable feedback which has helped us reshape our paper.  We have now updated our pdf handling all the mentioned labelling, missing data, lack of clarity, undefined notations and figure overcrowding, hyphenation and parentheses issues, to the best of our ability.

Reviewer comment (Writing):
Language is overcomplicated (see abstract again) with many undefined or unclear ideas: “proactive comprehension of trajectory to an extremity”, “observations are competitively mapped”, “part of its learning stride”, “collaboratively trained”, “results are assuring”, “Recollect that”, “agrressive” typo, “movement to criticalities on temporally chaotic datasets”, “Providing Ground explanations” etc.

Author response:
Apologies for the confusion with the terminologies. We have much simplified our writing now.

Reviewer comment (Experiments):
Why use an LSTM and not a transformer as prediction architecture?

Author response:
- Why not Transformer instead of LSTM :  Agreed, very appropriate suggestion, especially to the prediction variability question we pose in the conclusion section of the paper. Transformers have proven to provide the best results so far, with negligible errors. In our comparison studies we do mention that if prediction accuracy is the end goal, Transformers based extensions (Crossformer or Scaleformer - ICLR 2023) is the way to go.  Exclusivity of training objectives between prediction accuracy, interpretability and explainability has led the researchers to choose only one objective at a time.  Transformers, due to complex mapping from features to intermediate representations, to the best of our understanding there is not much exposure on explanations or interpretability so far, except for the Temporal Fusion Transformer (more details in response to Reviewer 1). 

MIRAGE, attempts all three objectives - prediction, interpretability and explainability, at once. It trains an attention layer, to expose explainability with a definite time reference for a proactive action. This time window cannot be generalised (unlike the Temporal Fusion Transformers) but has to be very specific to a user, e.g a particular player who is going to be overindulgent or a patient going for ICU readmission in the coming future.  So, replacing LSTM by a Transformer would need a detailed study on preserving the explainability across its intricate mappings from feature to the latent space.

Reviewer comment (Reproducibility):
How is $C$ determined?

Author response:
- How is c derived - c is the total number of discrete Markov states DMM could predict. They are derived empirically through experimentation. We leveraged Optuna framework for this.  We have now presented empirical results on varying c on open datasets in the Appendix (Section C, Table 8).

Reviewer comment (Evaluation):
Why is the MSE of MIRAGE on eICU data (Table 2) not reported? Perhaps authors could report reconstruction error in addition to forecasting performance.

Author response:
- MSE on eICU data for MIRAGE: Apologies, MSE was reported in the wrong column. We did not intend to report MSE on Crossformer as we already see it's much better. Also, as our focus is on interpretable predictions. Comparisons on MSE with Crossformer and LSTnet are reported for the open real world datasets as these are the standard benchmarks for those.

Reviewer comment (Evaluation):
How do authors determine that “interpretations appear quite smooth” in Fig 7?

Author response:
- We perhaps missed to mention that MIRAGE handles both linear (like eICU and ETTh/m) as well as non-smooth (player dataset) time series. The DMM (Discrete Markov Model) is capable of predicting if the next transition is coming from the same condition or a different one. For instance in Figure 7 (ETTm dataset) the heat map trajectory generated by MIRAGE is quite smooth. By smooth, we mean that in the consecutive time step the SOM centroid either remains the same or moves to the neighbouring centroid (SOM centroid in the left, right, up or down) which is not the case for the player (non-linear) data (Fig 6) which shows long jumps. But both these trajectories have been generated by MIRAGE.  We have also performed active validation on trajectory based prediction for about 3K players and we have reported much improved precision/recall numbers in the Evaluation Section over the state of the art. Definitely, your suggestion on human based assessment is valid and we will consider.

Reviewer comment (Evaluation):
Could authors provide numerical results that support the interpretability/correctness of latent trajectories, beyond the few qualitative examples proposed?

Author response:
- Interpretability: We know, interpretability is vaguely defined in the AI systems as of today. In this paper we further extend the first of its kind established notion of interpretability in SOM-VAE (ICLR 2019) paper. Unfortunately, to the best of our knowledge there no standard metrics for its rigorous evaluation. SOM-VAE primarily, establishes its claim by visual trajectories on eICU data. We have published alike visual trajectories for all types of datasets and more in the Appendix.  However, based on your feedback, we now report a metric based on Pearson's correlation to represent correlation between a linearity of MIRAGE trajectory for non-linear temporal progression and the corresponding transition in the discrete state of the process which generated it (Section Evaluation). Its quantification (numerical results) on the overall player data is also added in the Appendix C.2.  We use interpretability to also identify (classify) future player trajectories as overindulgent or healthy. We report improved accuracy (precision/recall) with MIRAGE vs  the T-DPSOM model in the Evaluation Section

Reviewer comment (Evaluation):
The analysis for the other datasets is somewhat limited. Since the paper primarily revolves around model interpretability, a more in-depth examination of how the features in these datasets are employed in making predictions would enhance the comprehensiveness of the research.

Author response:
With regards to your concern on the in-depth analysis - 
Forecasting Accuracy - We present MSE results on prediction accuracy for our own player dataset, as well as eICU and few other real-world datasets and compare them with SOTA benchmarks. We have open sourced our tool, the player dataset as well our notebooks. 
 
Interpretability - As we know, interpretability has been vaguely defined in the AI systems as of today. In this paper we further extend the first of its kind established notion of interpretability in the SOM-VAE paper. Unfortunately, to the best of our knowledge there are no standard metrics for its rigorous evaluation. SOM-VAE primarily establishes its claim by visual trajectories on eICU data.  We present similar visual trajectories. However, based on your feedback we report a metric based on Pearson's correlation to validate interpretability (Section Evaluation). General quantification on the overall player data is also added in the Appendix C.2. We use interpretability to identify (classify) future player trajectories as overindulgent or healthy. We report improved accuracy (precision/recall) with MIRAGE vs the T-DPSOM model.

Explainability - Explainability highlights features in the input space that caused a particular trajectory.  To our understanding providing a comprehensive validation is difficult unless the insights are at a generic level. In eICU and the player use case, each trajectory is unique and hence generic explanations would not be of much value.  We provide interesting examples with attentive features on each of these datasets on how MIRAGE is able to pinpoint the exact causal feature (especially for eventualities), most of which have been validated either with the available ground truth or with a rigorous manual evaluation. 
There are no studies yet on the features and their dependencies/influence on the response variable for the real world datasets (like ETTh). One may say that for these datasets MIRAGE is probably ahead in time or rather by using our model, the research community can start exploring much deeper explanations. For instance, how can one prevent the electricity demand from rising by probably controlling one or more  attentive covariate features? Patients' health data (eICU), due to high levels of privacy, lacked any ground truth on what caused deaths for certain patients.  MIRAGE revealed intuitive explanations (drop in bilirubin for a liver diseased patient, indicated recovery). We believe that open sourcing MIRAGE will significantly elevate the state of predictive and preventive health care with proactively actionable explanations. Additionally, we provide feature importance maps for all datasets, which can be used as a guide to associate a trajectory with the corresponding attentive features (Appendix C.1 and C.3 ). More explainability examples have also been provided in the Appendix C.

Reviewer comment (Presentation):
There are labeling issues in Table 6/1.

Author response:
Thank you for your valuable feedback which has helped us to reshape our paper.  We have now updated our pdf handling all the mentioned labeling, missing data, lack of clarity issues.

Reviewer comment (Presentation):
Figure 2 isn’t labeled as such.

Author response:
Thank you for your valuable feedback which has helped us to reshape our paper.  We have now updated our pdf handling all the mentioned labeling, missing data, lack of clarity issues.

Reviewer comment (Presentation):
The text within the figures should be presented in a larger font size to improve readability, ensuring that readers can easily interpret the visual content.

Author response:
Thank you for your valuable feedback which has helped us to reshape our paper.  We have now updated our pdf handling all the mentioned labeling, missing data, lack of clarity issues.

Reviewer comment (Writing):
The description of the MIRAGE model is difficult to follow, which may pose a barrier to understanding its functionality.

Author response:
Thank you for your valuable feedback which has helped us to reshape our paper.  We have now updated our pdf handling all the mentioned labeling, missing data, lack of clarity issues.  We have rearranged our writing (pls refer to the new paper pdf) to clearly explain our motivation and the differentiation we add. We have refined our model description and architecture and re-discussed some of the evaluation findings with few additions.

Reviewer comment (Experiments):
Have you considered comparing your model with architectures based on the Temporal Fusion Transformer?

Author response:
Temporal Fusion Transformer:  We found this work quite promising and relevant as it tried to provide explanations for forecasts. Few differentiating factors between TFT and MIRAGE to best of our understanding:
TFT have used interpretability and explainability interchangeably but we’ve used interpretability to answer the “why” of the prediction output by visualising the forecast on SOM map at each interval and used explainability to answer the “how” of the prediction output by obtaining the timestamp and associated feature responsible for the future prediction.
TFT paper has not published MSE metric to compare with the latest SOTA transformers models in MTS forecasting like Crossformer, Scaleformer (ICLR'23) etc..
TFT primarily discusses the extraction of overarching trends and patterns from the data, rather than delving into the specific causality of predictions on a case-by-case basis. For instance, in our study, we demonstrated how the model attends to changes in Hct levels (eICU) to predict the likelihood of a patient not surviving. We have referenced TFT work in our related work section.