Paper: Towards Transparent Time Series Forecasting (ICLR-2024)

Reviewer comment (Reproducibility):
No specific code references were found.

Author response:
## Code
As we say in the reproducibility statement,
> The code to reproduce the results and for the visualization tool can be found in supplementary materials.

Reviewer comment (Novelty):
The focus on static variables seems to limit the framework's utility in true time-series forecasting. Given that you claim to forecast time-dependent variables based solely on fixed attributes, isn't this more akin to regression analysis? This approach seems niche.

Author response:
# 3. Limitations
## Clear scope of the work
We want to highlight that we clearly describe the scope of our work (Section 3, Section 8, Appendix E.6). Although the conceptual framework in Section 2 is agnostic to the type of input, we focus on settings with static features to develop formalism and demonstrate a practical implementation.
## Reasoning behind the scope
The main goal of our work is to extend the notion of transparency to a novel class of outputs (trajectories), going beyond standard regression and classification algorithms (see Section 2 for more details). Thus, it provides a concrete answer to the question: What does it mean to understand the change in the output in time series forecasting? Of course, to ensure transparency, we also need to understand changes in the input. To provide a clear exposition, we focus on settings where this is straightforward, and we hope future work will extend it to more complex input types. We believe there is sufficient novelty without this extension.
## Real-life settings with static features
As we describe in Section 3, there are numerous scenarios that include predicting a trajectory from static features. Many of them include applications to medicine and pharmacology. Static features include the baseline patient's covariates (such as age, weight, or genetic information) and the information about the type of treatment or the dose of the drug. The trajectory of interest can be a tumor volume over time or drug plasma concentration. If necessary, one or a few initial observations at pre-specified times can also be considered to be static features. More examples of such scenarios can be found in finance (predicting stock values from the company's static data), time-to-event problems (predicting the survival or the hazard function), or physics (modeling some dynamical systems from their initial conditions). In some scientific or engineering domains, time can be even replaced by other continuous variables. For instance, when modeling stress-strain or current-voltage curves.
## Terminology
As there are many conflicting terminologies used throughout the literature, we specify the exact setting in Section 2.1, where we contrast time series forecasting with regression and classification models. We further describe the specific problem formulation in Section 3. We identified one place where we used the term "time series setting" and updated it to "time series forecasting" to make it consistent with the rest of the paper.

Reviewer comment (Presentation):
In order to avoid the two variables limit, have you considered visualizing the contour plot using techniques like t-SNE, PCA, UMAP, or similar?

Author response:
## Dimensionality reductions techniques
Although dimensionality reduction is not necessary for answering the questions we consider in our work, it undoubtedly can provide additional information about the model. This is exactly what we describe in "Applications" in Section 8. Adhering to our framework provides additional output next to the standard forecasted trajectory: the current composition and the coordinates of the transition points. Traditional XAI techniques for regression and classification can be applied to these additional outputs instead of individual trajectory points to gain more meaningful explanations. Thus, techniques such as dimensionality reduction, feature importance methods, local surrogates, and counterfactual explanations can now be extended to time series forecasting. This opens up numerous potential extensions and applications of our approach. We have added an additional section in Appendix E to discuss these kinds of applications.

Reviewer comment (Presentation):
Since transparency is the primary goal of your paper, more space should be devoted to proving this claim. For instance, I had difficulty understanding Figure 4.

Author response:
We acknowledge your comment regarding Figure 4. We have now added additional snapshots of the interface to Appendix E.4 which should help in understanding how our model can visualize the effect of perturbing two features at the same time.

Reviewer comment (Theory):
The framework appears limiting in applications where time series variability is an important aspect of the interpretation.

Author response:
## Variability
TIMEVIEW does not model the noise introduced by the measurement process. It assumes that there is a true underlying trajectory $y_*$ that we want to approximate (see Section 3). However, if the variability you describe refers to a deterministic property of the predicted trajectory, for instance, seasonality, then this can be accommodated by our framework! We could model seasonal changes as sine function where the frequency, magnitude, and offset are determined by the neural network (the same way as the B-Spline coefficients). Our model would be defined as:
$$ g(\mathbf{x})(t) = \sum_{k=1}^B h(\mathbf{x})_b \phi_b(t) + A(\mathbf{x}) \sin(\omega(\mathbf{x}) t + \psi(\mathbf{x}))$$
where $A:\mathbb{R}^M\rightarrow \mathbb{R}$, $\omega:\mathbb{R}^M\rightarrow \mathbb{R}$, $\psi:\mathbb{R}^M\rightarrow \mathbb{R}$  are either separate neural networks or they share weights with the encoder $\mathbf{h}$.

Then, we can treat the compositions in the same way as the current implementation of TIMEVIEW but with three additional properties to monitor (in addition to transition points). This allows us to answer questions like "How would the variability increase if I decrease this particular feature?". We have now added a discussion of this extension to Appendix E.

Reviewer comment (Experiments):
Are there alternatives to the cubic spline setup when using your framework?

Author response:
## Alternatives to cubic splines
As we describe in Section 5, we chose cubic splines because we needed a space of trajectories that satisfies these two criteria:
> 1. The set of dynamical motifs $\mathcal{S}$ is compatible with the class of predicted trajectories $\hat{\mathcal{Y}}$.
> 2. For every $\hat{y} \in \hat{\mathcal{Y}}$ we can calculate its composition $\mathcal{C}_{\mathcal{S}}[\hat{y}]$ 

We described above how cubic splines can be extended by adding a sine function. We could use many other basis functions, such as a sine basis or some types of wavelets. They could satisfy the first condition, but the composition would likely need to be calculated numerically and not analytically as it is done now.

In case you are interested in a different set of motifs (please see Appendix A.4 for examples), other choices may also be possible. For instance, for range motifs or monotonic motifs, Haar wavelets may be a good choice, and we suspect the compositions can be calculated efficiently. We have now included a discussion about alternatives to cubic splines in Appendix E.

Reviewer comment (Theory):
Are there ways to incorporate domain knowledge/priors into this framework?

Author response:
## Domain knowledge/priors
One way to incorporate priors is to choose the number of knots for the cubic splines. The bigger the number of knots, the more complex trajectories can be fit. More generally, as mentioned above, different basis functions (possibly for different motifs) can be chosen to approximate the trajectory.

Reviewer comment (Experiments):
Although the authors already include a few examples of real-world problems where the tooling can be applied, I think that a more fleshed out experiment that runs throughout the whole paper can be especially helpful for conveying the contributions in a clearer manner (Appendix E accomplishes this very well).

Author response:
We agree with your suggestion regarding the integration of a continuous example throughout the paper. While the detailed discussion in Appendix E.1 exemplifies our contributions effectively, we understand the impact of a coherent narrative in the main text. To address this, we have further developed the tumor volume example introduced in the beginning. We have integrated this theme more consistently across the paper, replacing Figure 1 with a more illustrative snapshot of the tumor volume interface. We have also ensured that this example is referenced in Section 2, further tying together the paper's various sections. Please let us know if you find that these revisions improves the clarity and exposition.

Reviewer comment (Presentation):
The comparison to other methods towards the end of the paper addresses a question I had regarding the trade-offs between interpretability and performance, but this constraint could also be featured more prominently throughout the paper, such that the proposed methodology is consistently presented in light of this compromise.

Author response:
Thank you for your suggestion about a more prominent exposition of the trade-off between interpretability and performance. To address this, we have expanded our discussion on this aspect, now featuring it more prominently in the introduction (Contributions) and Section 5 (Implementation).

**Actions taken**
- Exchanged Figure 1 for a different snapshot of the interface.
- Referred to the tumor example in Section 2.
- Added discussion about the trade-off between interpretability and performance in the introduction and Section 5.

Reviewer comment (Evaluation):
I would be interested in learning more about instances where the proposed simulation is *not* suitable, and what properties of time series one would need to inspect before deciding to apply this technique (beyond the requirement for static features).

Author response:
We have already touched upon it in Appendix E.6 (Limitations and future works). Although it is not a limitation of our conceptual framework, TIMEVIEW may struggle to predict trajectories that cannot be well approximated using a cubic spline. For instance, curves with sharp changes in the derivative or discontinuities. This could be addressed by considering motifs that are not dependent on any smoothness constraints (for instance, monotonic motifs as in Example 3 in Appendix A.4) and by using a different space of predicted trajectories $\hat{\mathcal{Y}}$.

Reviewer comment (Writing):
In my opinion, the writing style is not adapted and will confuse or have readers get lost. There are too many references to future sections/definitions/details, which complexify the reading. For instance, Authors introduce some concept, but do not give further explanation (even simplified one), but rather referred to specific sections, which makes readers go back and forth through the paper.

Author response:
We appreciate your remark about further explaining some of the terms when they are first introduced. We have now improved the description of *trends* and *properties* in the introduction, hopefully making the paper more straightforward.

Reviewer comment (Writing):
There are too many claims that are not referenced or backed-up in this paper, which diminishes its credibility.

Author response:
Thank you for your question about the proof of our claim that certain models
> often satisfy bi-level transparency but have poor performance.

It allows us to better join this sentence with the two sentences directly following it that already provide explanations and point to experimental evidence.
> For instance, all trajectories predicted by linear regression and GAMs are parallel; thus, they cannot model different trends (Section 7). Decision Trees capture non-additive interactions, enabling flexible forecasting models. However, they require many splits to approximate the ground truth, leading to poor performance or incomprehensibility (Section 7).

Similarly, we appreciate your question about evidence for our statement 
> these methods are not directly applicable to time series forecasting—when we want to predict a whole trajectory rather than a single label.

It allows us to connect it better with the sentence directly preceding it that explains that these models cannot be used *directly* for time series forecasting as, by definition, they issue static predictions (single labels), not whole trajectories. 

We hope these changes improve our sections' cohesiveness and make it easier to find corresponding evidence for our claims.

Reviewer comment (Novelty):
Where is the explainability? If I understand correctly, the proposal can help determine the impact of varying given static parameters (figure 4) on the output of the model. However, it does not infer why it has this specific impact (the hidden relation in the model). In my opinion, the proposal is not different from black-box models, as it does not give a clear relation between input characteristics that generate a given output. It looks more like an exploratory platform than an XAI model.

Author response:
We recognize that your primary concern about our work is that TIMEVIEW does not allow us to "understand the reason" for the impact on the algorithm's output. We interpret your statement as referring to the fact that our method is not *simulatable*. Below, we describe what simulatability is, why it is challenging to define for time series forecasting, and why it may not be very useful in practice.
## Simulatability
Although our framework allows for answering meaningful counterfactual questions, it does not show the exact process by which the values of the trajectory are calculated from the inputs. In other words, it does not satisfy the very strict requirement of simulatability (Lipton, 2018) described as "for a model to be fully understood, a human should be able to take the input data together with the parameters of the model and in reasonable time step through every calculation required to produce a prediction". Although useful for *static predictions*, directly applying this definition to time series forecasting is challenging. Does "produce a prediction" refer to evaluating the values of the trajectory at some pre-specified points? How many points? Which points? Note that this corresponds to a bottom-up approach to understanding a trajectory and, as explained earlier, may not be very useful in practice. Is the only way to be simulatable to have a closed-form mathematical expression (like in symbolic regression)? In particular, we want to highlight the following.
1. The issue of simulatability for time series forecasting has yet to be investigated and needs to be defined. (Barredo Arrieta et al., 2020)
2. Whether it is possible to have a flexible and simulatable model for time series forecasting is an open problem.
3. As simulatability is closely linked to bottom-up understanding, it is not clear if it has clear advantages in practice (disadvantages of bottom-up understanding discussed in Section 2.2 and earlier in this response).

Thus, we introduce a different notion of transparency and highlight its advantages in Section 2.3. Let us explain with a concrete example how TIMEVIEW makes understanding the change in the output easier than other bottom-up approaches.

## TIMEVIEW makes understanding the change in the trajectory easier: a concrete example 

Let $y:[0,1] \rightarrow \mathbb{R}$ be a trajectory given by $y(t) = t^2$. Consider some change of the input such that the new trajectory is now described by $y(t)=-t^2$ . We contrast a bottom-up description with a top-down description.
### Bottom-up description 
Bottom-up approach focuses on the change at individual time points. That would amount to: 
- For $t=0.0$ the value stays the same; 
- for $t=0.2$ the value decreases by $0.08$;
- for $t=0.4$ the value decreases by $0.32$;
- for $t=0.6$; the value decreases by $0.72$;
- for $t=0.8$ the value decreases by $1.28$;
- for $t=1.0$ the value decreases by $2.0$.

We claim that it is often not the most useful way to represent the change as it is difficult to comprehend all these individual changes to piece together a more holistic understanding. It is also restricted only to a few time points. 
### Top-down description
Using top-down approach, we would say: "The trajectory changes from increasing and convex starting at $0.0$ and ending at $1.0$ to a decreasing concave trajectory starting at $0.0$ and ending at $-1.0$". We argue that this description is easier to comprehend and already includes the essential aspects of this change.

A similar example can be constructed for understanding a change in a particular property with the trend remaining the same. For instance, a change from $y(t) = \sin(\pi t)$ to $y(t) = 2sin(\pi t) + 1$, where the property of interest is the maximum of the trajectory $y(t_{max})$ . A top-down description would say, "The trend of the trajectory remains the same, but the maximum $y(t_{max})$ changes from 1.0 to 3.0". Note that **this may not be captured by a bottom-up approach if the maximum is not obtained at one of the pre-specified time points** (e.g., $t_{max} \notin \{0.0,0.2,0.4,0.6,0.8\}$). It also **cannot be captured if $t_{max}$ changes as well**.

The information about the change in the trajectory can be read from the left part of our interface (the colorful bands or 2D contour plots), whereas the information about the properties can be read from the right part of the interface. **Please note that it does not require moving the sliders back and forth.** This information is readily available from a snapshot of the interface (as shown in Appendix E.1). 

These examples highlight how a top-down approach to understanding trajectory change is more intuitive and useful than bottom-up approaches.

Reviewer comment (Evaluation):
Why is the proposal more comprehensible than other baselines? Someone could set up a platform to visualize the impact of changing the input of a linear model on the produced output, but would it be a good XAI model?

Author response:
Please note that it does not require moving the sliders back and forth. This information is readily available from a snapshot of the interface (as shown in Appendix E.1). These examples highlight how a top-down approach to understanding trajectory change is more intuitive and useful than bottom-up approaches.

Reviewer comment (Writing):
The bottom-up approach mentioned by the Authors is rather a “bottom” approach in my understanding, meaning that models use raw data (usually regularly spaced) to do TSF. Authors need to clarify what is the “up” part, especially for readers to understand the difference with the proposed approach.

Author response:
We first want to clarify that "bottom-up" and "top-down" refers to how the trajectory is comprehended, not how the prediction is generated. We use these terms to differentiate between XAI techniques rather than contrast model architectures. Your questions made us realize that this distinction can be made even more explicit in our paper, so we added a footnote in Section 2 to prevent any future confusion.

Current XAI methods usually produce explanations for individual time points ("bottom"). These, in turn, have to be pieced together by a human to understand changes in trends ("up") from changes at individual time steps. Thus we call it "bottom-up". This is challenging, and we discuss the limitations of this approach in Section 2.2 and then demonstrate them experimentally in Figures 11-13 in Appendix E.3. As we describe, these explanations do not allow for answering essential questions such as "Would the predicted tumor volume keep decreasing if we adjusted the treatment?". In contrast, our top-down approach (explained in Section 2.3), where we understand the trend of the trajectory ("top") and then its properties ("down"), allows for answering such questions. We show it in Appendix E.1, where we review several questions and show how our interface allows us to answer them.

Reviewer comment (Writing):
Authors need to clarify whether *Bottom-up* means from time points predicting time points to define a trajectory, and if understanding trajectory from time points is not natural for humans, why not determine trends, min, max, and other statistics (with sliding window) to make understanding simpler? Why is the top-down approach better?

Author response:
We first want to clarify that "bottom-up" and "top-down" refers to how the trajectory is comprehended, not how the prediction is generated. We use these terms to differentiate between XAI techniques rather than contrast model architectures. Your questions made us realize that this distinction can be made even more explicit in our paper, so we added a footnote in Section 2 to prevent any future confusion.

Current XAI methods usually produce explanations for individual time points ("bottom"). These, in turn, have to be pieced together by a human to understand changes in trends ("up") from changes at individual time steps. Thus we call it "bottom-up". This is challenging, and we discuss the limitations of this approach in Section 2.2 and then demonstrate them experimentally in Figures 11-13 in Appendix E.3. As we describe, these explanations do not allow for answering essential questions such as "Would the predicted tumor volume keep decreasing if we adjusted the treatment?". In contrast, our top-down approach (explained in Section 2.3), where we understand the trend of the trajectory ("top") and then its properties ("down"), allows for answering such questions. We show it in Appendix E.1, where we review several questions and show how our interface allows us to answer them.

Reviewer comment (Writing):
Authors need to clarify whether *Top-down* means defining a trajectory from time points, then predicting a trajectory by first looking at trend and properties, and then time points, or some other process.

Author response:
We first want to clarify that "bottom-up" and "top-down" refers to how the trajectory is comprehended, not how the prediction is generated. We use these terms to differentiate between XAI techniques rather than contrast model architectures. Your questions made us realize that this distinction can be made even more explicit in our paper, so we added a footnote in Section 2 to prevent any future confusion.

Current XAI methods usually produce explanations for individual time points ("bottom"). These, in turn, have to be pieced together by a human to understand changes in trends ("up") from changes at individual time steps. Thus we call it "bottom-up". This is challenging, and we discuss the limitations of this approach in Section 2.2 and then demonstrate them experimentally in Figures 11-13 in Appendix E.3. As we describe, these explanations do not allow for answering essential questions such as "Would the predicted tumor volume keep decreasing if we adjusted the treatment?". In contrast, our top-down approach (explained in Section 2.3), where we understand the trend of the trajectory ("top") and then its properties ("down"), allows for answering such questions. We show it in Appendix E.1, where we review several questions and show how our interface allows us to answer them.

Reviewer comment (Theory):
Authors never demonstrated the issue with the bottom-up approach and why bottom-up approaches are not bi-level transparent.

Author response:
This is challenging, and we discuss the limitations of this approach in Section 2.2 and then demonstrate them experimentally in Figures 11-13 in Appendix E.3.

Reviewer comment (Writing):
In the example box in the introduction, the definition of "feature" and "instance" is not clear and may confuse readers. In usual TSF, feature (or variate) means other time series that support the prediction of the target (exogenous features). But it looks like a feature in the introduction example is a characteristic or a specific time step of the time series. Authors need to clarify these points.

Author response:
The questions in the box in the introduction are only prototypical examples from question classes identified by Liao et al. (2020). Concrete questions are shown at the bottom of the box. We adjusted the terms in the box to avoid possible confusion. We specify the exact problem formulation in Section 3.

Reviewer comment (Writing):
Authors need to clarify whether "instance" means an input sample of the model.

Author response:
The questions in the box in the introduction are only prototypical examples from question classes identified by Liao et al. (2020). Concrete questions are shown at the bottom of the box. We adjusted the terms in the box to avoid possible confusion. We specify the exact problem formulation in Section 3.

Reviewer comment (Writing):
In my opinion, the term "__time series setting__" is not appropriate; it should be "multistep forecasting" or something similar.

Author response:
As there are many conflicting terminologies used throughout the literature, we specify the exact setting in Section 2.1, where we contrast time series forecasting with regression and classification models. We further describe the specific problem formulation in Section 3. We identified one place where we used the term "time series setting" and updated it to "time series forecasting" to make it consistent with the rest of the paper.

Reviewer comment (Presentation):
The definition of knots should not be in the appendix.

Author response:
We have added a short explanation of knots in Section 5.

Reviewer comment (Theory):
The statement “Thus, understanding a (continuous) trajectory by individual values is unnatural for humans” is problematic because time series are never continuous, even at low frequency sampling — they are still a set of points that can be approximated by a function/trajectory.

Author response:
Thank you for your question about continuous time series. It allows us to improve further our explanation of the setup, which is already included in the sentence directly following the one you quote:
> Of course, in practice, we only observe discrete samples of y, which may be irregular and noisy.

By providing a stronger link between these two sentences, we hope to prevent future confusion about this issue. Modeling a phenomenon as a continuous system is an established practice in sciences and engineering. That includes settings mentioned in our paper, such as disease modeling or drug concentration in blood. It is also well represented in the machine learning literature (Brunton et al., 2016; Chen et al., 2018). We have also added a discussion to Appendix E about scenarios where the phenomenon cannot be studied as a continuous system. However, our conceptual framework in Section 2 still applies even in these settings.

Reviewer comment (Writing):
The statement “there are numerous ways a function can change” is imprecise. For each time step, the function will either increase, decrease, or remain constant, same as regression. Possibly, Authors mean the function (trajectory) in the prediction window may take different shapes depending on outputs.

Author response:
In your comment, you raise the following interesting point:
> I disagree for each time steps the function will either increase, decrease or remain constant, same as regression.

This is exactly what we refer to as a "bottom-up" understanding of the trajectory exercised by most of the XAI approaches, and we devote Section 2.2 to exploring its limitations. Based on your remark, we believe that a more in-depth discussion addressing your statement would significantly improve the accessibility of this section and improve the overall clarity. Thus, we have added a dedicated section to Appendix E and refer to it at the end of Section 2.1.

Let us consider a set of time points $\mathcal{T} = \\{1,2,\ldots,9,10\\}$. As you state, the function can increase, decrease or remain constant for each time step. Thus, the function described by these ten values can change in $3^{10} = 59049$ ways (which is much bigger than $3$—the number of ways an output of a regression model can change). This is what we describe in the paragraph on "increased cognitive load" in Section 2.2. It is infeasible for a human to reason about all these different possibilities, which motivates our proposal of a top-down approach.

Reviewer comment (Experiments):
The claim “often satisfy bi-level transparency but have poor performance” lacks proof. Which models from the experiments are bottom-up approaches?

Author response:
Thank you for your question about the proof of our claim that certain models
> often satisfy bi-level transparency but have poor performance.

It allows us to better join this sentence with the two sentences directly following it that already provide explanations and point to experimental evidence.
> For instance, all trajectories predicted by linear regression and GAMs are parallel; thus, they cannot model different trends (Section 7). Decision Trees capture non-additive interactions, enabling flexible forecasting models. However, they require many splits to approximate the ground truth, leading to poor performance or incomprehensibility (Section 7).

Reviewer comment (Novelty):
The statement “However, these methods are not directly applicable to time series forecasting” should be supported by a reference. Linear regression can be applicable to time series forecasting, and prior work has shown linear models can be competitive with Transformer-based models.

Author response:
Similarly, we appreciate your question about evidence for our statement 
> these methods are not directly applicable to time series forecasting—when we want to predict a whole trajectory rather than a single label.

It allows us to connect it better with the sentence directly preceding it that explains that these models cannot be used *directly* for time series forecasting as, by definition, they issue static predictions (single labels), not whole trajectories.

Reviewer comment (Theory):
The paper does not explain why decomposition methods [1] and [2] (trend + seasonality decomposition) would be less explainable than the proposed approach.

Author response:
We appreciate their novel architecture that decomposes time series into a trend and seasonality, but it is not clear to us how that model structure aids in answering the meaningful questions described in our work.

Reviewer comment (Writing):
The paper needs proof-reading; for example: “different prediction” has wrong opening brackets, and smoothing parameter "s" and motif "s" should be renamed to avoid confusion.

Author response:
We have now fixed some quotation marks that evaded our proofreading efforts. Thank you for finding them.

### Smoothing parameter
We note that the "smoothing parameter s" appears only in Appendix C.3, where we refer to a function `UnivariateSpline` from the `scipy` library, and we use the letter from its original documentation to denote its argument. Although "smoothing parameter s" is not used in any other place in the paper, we added a footnote in Appendix C.3 to avoid confusion. Table 4 in Appendix A.1 lists all essential symbols used in the paper.