TILDE-Q: a Transformation Invariant Loss Function for Time-Series Forecasting
Authors: Hyunwook Lee
Chunggi Lee
Hongkyu Lim
Sungahn Ko
Venue: Submitted to ICLR 2023 (2023)
Keywords: Time-Series Forecasting
Deep Learning
Loss functions
Time-series similarity

Abstract: Time-series forecasting has caught increasing attention in the AI research field due to its importance in solving real-world problems across different domains, such as energy, weather, traffic, and economy. As shown in various types of data, it has been a must-see issue to deal with drastic changes, temporal patterns, and shapes in sequential data that previous models are weak in prediction. This is because most cases in time-series forecasting aim to minimize $L_p$ norm distances as loss functions, such as mean absolute error (MAE) or mean square error (MSE). These loss functions are vulnerable to not only considering temporal dynamics modeling but also capturing the shape of signals. In addition, these functions often make models misbehave and return uncorrelated results to the original time-series. To become an effective loss function, it has to be invariant to the set of distortions between two time-series data instead of just comparing exact values. In this paper, we propose a novel loss function, called TILDE-Q (Transformation Invariant Loss function with Distance EQuilibrium), that not only considers the distortions in amplitude and phase but also allows models to capture the shape of time-series sequences. In addition, TILDE-Q supports modeling periodic and non-periodic temporal dynamics at the same time. We evaluate the effectiveness of TILDE-Q by conducting extensive experiments with respect to periodic and non-periodic conditions of data, from naive models to state-of-the-art models. The experiment results indicate that the models trained with TILDE-Q outperform those trained with other training metrics (e.g., MSE, dynamic time warping (DTW), temporal distortion index (TDI), and longest common subsequence (LCSS)).

Under review as a conference paper at ICLR 2023
TILDE-Q: A TRANSFORMATION INVARIANT Loss FUNCTION FOR SIME-SERIES FORECASTING
Anonymous authors Paper under double-blind review
ABSTRACT
Time-series forecasting has caught increasing attention in the AI research field due to its importance in solving real world problems across different domains, such as energy, weather, traffic, and economy. As shown in various types of data, has been must-see issue to deal with drastic changes, temporal patterns, and shapes in sequential data that previous models are weak in prediction. This because most cases in time-series forecasting aims to minimize I,p norm distances as loss functions, such as mean absolute error (MAE) or mean square error (MSE). These loss functions are vulnerable to not only consider temporal dynamics modeling, but also capture the shape of signals. In addition, these functions often make models misbehave and return uncorrelated results the original time-series. To become an effective loss function, it has to be invariant to the set of distortions between two time-series data instead of just comparing exact values. In this paper, we propose a novel loss function, called TILDE-Q (Transformation Invariant Loss function with Distance EQuilibrium), that not only considers the distortions in amplitude and phase but also allows models to capture the shape of time-series sequences. addition, TILDE-Q supports modeling periodic and non-perinn non-periodic temporal dynamics at the same time. We evaluate the effectiveness of TILDE-Q by conducting extensive experiments with respect to periodic and non-periodic conditions of data, from naive models to state-of-the-art models. The experiment results indicate that the models trained with TILDE-Q outperforms those trained with other training metrics (e.g.. MSE, dynamic time warping (DTW), temporal distortion index (TDI), and longest common subsequence (LCSS)).
INTRODUCTION
Time-series forecasting has been core problem across various domains, including traffic domain (Li eet al. 2018 Lee et al. 2020), economy (Zhu & Shasha 2002), and disease propagation analysis (Mat- subara et al., 2014). The crucial part of the time-series forecasting is modeling of the complex temporal dynamics (e.g.. non-stationary signal, periodicity). Temporal dynamics, intuitively, shape, has always been one of the most attention attention-getting getting keywords in time-series domains, such as rush hour of traffic data abnormal usage of the electricity (Keogh et al., 20033] (Bkkshi & Stephanopoulos! 1994; Weigend & Gershenfeld 1994] Wu et al.) 2021][Zhuu et al. (022). Deep learning methods are one of the appealing solutions model complex non-linear temporal dependencies and non- stationary signals, but recent work reveals that even deep learning is often insufficient to model temporal dynamics. To properly model the temporal dynamics, Wu et al. (202T); Zhou et (2022) have proposed a novel deep learning approaches with input sequence decomposition. Le Guen &
Thome (2019) try model sudden changes timely and accurately with dynamic time warping (DTW).
Bica et al. ( (2020) adopts domain adversarial training to learn balanced representations, which is
treatment invariant representations over time. Wu et al. (202T): Zhou et al. (2022) have less attention
to the essence of the problem: shape, in other words, temporal dynamics. Le Guen & Thome (2019);
Bica et al. (220)) try to capture the shape but still have some limitations like Fig. (c).
A shape is a part of patterns in time-series data with given time interval that could give valuable information, such as rise, drop, trough, peak, and plateau. We call the prediction is informative when it could properly consider the shape. In real-world applications like economies, such informative prediction is crucial to make decisions. To gain informative forecasting, the model should consider the shape rather than only aim to forecast accurate value for each time step. However, existing models do

---

Under review as a conference paper at ICLR 2023
V
TILDE-Q
Deep Learning Model (Informer)
MSE
DTW-based
Figure 1: Ground-truth and forecasting results of Informer model with three training metrics (top) TILDE-Q. (middle) MSF, and (bottom) DTWW-based loss function. (middle) MSE tends generate non-informative forecasting results, similar to an average value of data and (bottom) DTW often produces misaligned results. Red dotted box contains three training metrics.
not consider learning shape (Wu et al. 2021| Zhou et al. 2022 Bica et al. 2020 Le Guen & Thome 2019), so the forecasting results are often inaccurtee and uninformative, because deep learning model tends to learn in easy way (Karras et al. 2019). Fig. shows three real forecasting results with same model, different training metrics. When we utilize mean squared error (MSE) as an objectives, the model only aims reduce gap between prediction and ground truth for each time-step. As results, the model generates relatively easy prediction regardless of temporal dynamics (Fig. (b)). rarely gives information about original time-series. In contrast, if consider both gap and shape of prediction and ground truth, the model could achieve both accuracy and temporal dynamics, as shown in Fig. (a).
In this work, we aim to design a novel objective function that guides models to improve forecasting performance by learning the shapes in time-series data. To design such shape-awar loss function. we review existing literature (Esling & Agon 2012 Bakshi & Stephanopoulos, 1994 Keogh 2003)
and investigate the notions of shapes and distortions that interrupt measurement for recognizing similarity of two time-series data in terms of shapes (Sec. 3.1, Sec. [.2] and Sec. 3.3). Based on the investigation, we newly propose required conditions for constructing an objective function for shape-aware time-series forecasting (Sec. 3.4). We then present novel loss function, TILDE-Q (Transformation Invariant Loss function with Distance EQualibrium), that enables shape-aware representation learning with three different loss terms, which are invariant to the distortions (Sec. 4 For evaluation, we conduct extensive experiments with state-of- state-of-the-art deep learning models for time-series forecasting with TILDE-Q. The results indicate that TILDE-Q model-agnostic and could improve accuracy of existing models, compared to MSE and DILATE.
Contributions We make the following contributions: (1) To understand shape- awarenesssss and distortion invariances in time-series forecasting, we investigate existing distortions in amplitude and phase; (2) we implement TILDE-Q that has invariances to many existing distortions and achieves shape-awareness and informative forecasting in timely manner; and (3) we show that the proposed TILDE-Q allows models to have higher accuracy compared to those with existing metrics such as DTW, TDI, and LCSS on average.
2 RELATED WORK
2.1 TIME-SERIES FORECASTING
There are many methods for time-series forecasting from traditional ones, such ARIMA model (Box ctt al. 2015) and hidden markov model (Pesaran et al. 2004) to recent deep learning models. this section, briefly describe the recent deep learning models for time-series forecasting, Starting with the huge success of the recurrent neural networks (RNNs) (Clevert et al. 2016 Li et al. 2018, Yu et al. (017). researchers have developed novel deep learning architectures, improving forecasting performance. To effectively capture long-ten term dependency, which is weakness of RNNs,
Stoller et al. (2020) have proposed convolutional neural networks (CNNs). However, it is required
2

---

Under review as a conference paper at ICLR 2023
to stack lots of the same CNNs to capture long-term dependency (Zhou et al. 2021 Attention- based approaches have been another popular research direction in time-series forecasting, including Transformer (Vaswani al. 2017) and Informer (Zhou al. 202I). Although the attention- basodddddddddd-- models effectively capture temporal dependencies, they require high computational cost and often struggle to find proper temporal information (Wu al. (02T). To cope with the problem, /Wu et al.) (2021); Zhou et al. (2022) utilize the input decomposition method that helps models better encode appropriate information. The other state-of-the-an models adopt neural memory networks (Kaiser et al. 2017 Sukhbaatar et al. 2015 Madotto et al. 2018 Lee et al. 22022). which refer to historical
data stored memory to generate meaningful representation.
2.2 TRAINING METRICS
Conventionally, mean squared error (MSE), Lp norm and its variants are the mainstream to optimize forecasting models. However, they are not the best metric to train forecasting models (Esling & Agon 2012) because time-series is temporally continuous. Additionally, Lp norm gives less information about temporal correlation among tme--series data. To better model temporal dynamics in time- series data, researchers have used differentiable, approximated dynamic time warping (DTW), alternative metric of MSE (Cuturi & Blondel| 2017] Abid & Zou 2018 Mensch & Blondel| 2018). However, using DTW as loss function results in ignoring temporal localization of changes. Recently, Le Guen & Thome (2019) suggests DILATE, a training metric to timely catch sudden changes of non-stationary signals with smooth approximation of DTW and penalized temporal distortion index (TDI). To guarantee to work in timely manner, Le Guen & Thome (019) introduce loss function that gives harsh penalty when predictions show high temporal distortion. However, TDI relies on the DTW path, and DTW often shows misalignment because of its noise- and scale-sensitive. Thus, DILATE often loses its advantage with complex data, showing disadvantages at the beginning of the training. In this work, we discuss distortions and transformation invariances and design new loss function that allows models to learn shapes in the data and produce noise-robust forecasting results.
PRELIMINARY
In this section, we aim to investigate common distortions without losing the goal of time-series forecasting (i.c., modeling temporal dynamics and accurate forecasting). To help understand the concepts, we first define notations and terms (Sec. B.I.p We then discuss common distortions in time-series in transformation perspectives that need to be considered for building shape-aware loss function (Sec. 3.2) and describe how other loss functions (e.g.. DTW and TDI) handle shapes during learning (Sec. 3.3 Last, we explain the conditions for effective time-series forecasting (Sec. 3.4
3.1 NOTATIONS AND DEFINITIONS
Let X, denote a data point at a time step t. Then, we can define a time-series forecasting problem as:
Definition 1. Given T T-length historical time-series X [X, X₁ X, RF at time and corresponding T' -lenghh future time-series Y [V++,.... YHTr, RC, time-series forecasting foreccain aims to learn mapping function f T + R XC
To distinguish the label (i.e., ground-truthh and prediction time- data, we note the label data as Y and prediction data Y. Next, we set up two goals for time-series forecasting, which require not only precise, but also informative forecasting Wu et al. (202T): Zhou et al. (2022); Le Guen & Thome (2019) as follow:
Mapping function should be learnt to point-wisely reduce distance between Y and Y; and
The output Y should have similar temporal dynamics with Y.
Temporal dynamics are informative patterns in time-series, such rise, drop, trough, peak, and plateau. The optimization for the point-wise distance is the conventional methods utilized in deep learning domain, which could be obtained by using MAE or MSE. But in the real-world problem, for example, the traffic speed prediction or the economics, accurate forecasting of such "temporal dynamics." |Esling & Agon| 2012) also emphasized the importance of measuring temporal dynamics,
3

---

Under review as a conference paper at ICLR 2023
Shifting
Uniform Scaling
Dynamic Scaling
Amplitude
Time
Mn
Figure 2: Example of the six distortions on the amplitude axis (top) and temporal axis (bottom).
as "...allowing the recognition of perceptually similar objects even though they are not mathematically identical." In this paper, we define the temporal dynamics as follows:
Definition 2. Temporal dynamics (or shapes) are the informative periodic and non-periodic patterns in time-series data.
In this work, we aim to design a shape-aware loss function that satisfies both goals. To this end, we first discuss distortions that two time- semesseress with similar shapes can have.
Definition Given two time-series F and G having similar shape but not being mathematically identical, F could be formulated by transformation HGG). Then, can call time-series F and G have a distortion, which could be represented by transformation H.
Distortion generally occurs in different aspects. Distortions are defined as temporal distortion (i.e., warping) and amplitude distortion (i.c.,scaling) with respect to its relevance of dimension, time and amplitude. Existing distortion in data leads to misbehavior of the model, as measurements are interrupted by the distortion. For example, if we have two time-series and G = + k, which have similar shape but different means, G could represent many temporal dynamics of F. However, measurements often evaluate F and G are different (e.g., measuring with MSE) and causes misguidance of the model in training. As such, it is important to have measurements that consider similar shape invariant to distortion. We define a measurement for a distortion as follow:
Definition 4. Let transformation H. represents distortion H. Then, we call measurement 'D invariant to H. if 35 0: D(T, H(T)) 8 for any time-series T.
3.2 SEMEISERIES DISTORTIONS IN TRANSFORMATION PERSPECTIVES
Distortion, a gap between two similar Lime-seriss affects on capturing shapes in time-series data. As such, it is important investigate different distortions and their impact on representation learning aspects. There are six common time-series distortions that models encounter during learning (Esling Agon 2012 Batista et al.) 2014 Berkhin 2006;; Warren Liao 2005 Kerr et al. -AppitAmpli Shifting, Phase Shifting, Uniform Amplification, Uniform Time Scaling, Dynamic Amplification, and Dynamic Time Scaling. Next, we explain each common time- series distortion in terms of transformation with n-length time-series (t) (t1). (t.)....., [t,)],, where t= [t, ty...., tn].
Fig. 2 presents example distortions, categorized by amplitude and time dimensions.
Amplitude Shifting describes how much time-series shifts against another time-series. This can be described with two time-series and the degree of shifting (k): G(t) = F(t) k- [f()) + h...... ff....f((tt) (t.,. + k), where & E IR is constant.
Phase Shifting is the same type of transformation (i.e., translation) amplitude shifting, but it occurs along with the temporal dimension. This distortion can be represented with two time- series functions with the degree of shift (k): G(t) F(t k) [ta + k)..... (t, + k)) where l E R is constant. Cross-correlatie (Paparrizos Gravano 2015 Vlachos et al. 2005) is the most popular measure method that invariant this distortion.
4

---

Under review as a conference paper at ICLR 2023
Uniform Amplification is transformation that changes the amplitude by multiplication of & E R. This distortion can be described with two functions and a multiplication factor (k): G(t) = k Ftt) = 1k f f(t))...... k. ft..)].
Uniform Time Scaling means a uniformly shortened or lengthened F(t) on the temporal axis. This distortion can be represented as G(t) = [ (0l.....m(mmm]] ((m)) where g(4;) = (((+-)) and & E R Although Keogh et al. (2004) propose uniform time warping methods to handle this distortion, it still remains one of the difficult distortion types to measure, due to the difficulty in finding the scaling factor k without testing all possible cases (Keogh 2003).
Dynamic Amplification can be interpreted as any distortion occurred by non-zero mul- tiplication the amplitude dimension. This distortion can be described as follows: G(t)
......,,(ttt) [t,,)] with function h(t) such that VIeT, h(t) + Local amplification a representative distortion of this type of distortions, which still remains challenging to solve.
H(t) F(t) = [h(tz)
Dynamic Time Scaling means any transformation that dynamically lengthens or shortens signals on the temporal dimension including local time scaling (Batista et al., 2014) and occlusion (Batista et al. [2014] Vlachos et al., 2003). It can be represented as follows: Gtt) = (h(()) = [f(((tt))..... f(h(t.))) where lu(t) a positive, strictly increasing function. Dynamic time warping (DTW) (Bellman & Kalaba [959 Berndt & Clifford
Dass 1994 Keogh& & Ratanamahatana 2005) is the most popular technique on this distortion, Das et al. (197]p also introduce the longest common subsequence (LCSS) algorithm to tackle occlusion, noise, and outliers in this distortion.
There are several studies on shape-aware clustering (Bellman & Kalaba 1959 Batista et al. 2014 Paparrizos Gravano 2015; Berkhin| 2006 Warren Liao| 2005| Kerr et al. 2008) and classifica- tion (Xi et al. [2066] Batista et al.] 2014, Srisai & Ratanamahatana 2009) tasks with the consideration of shapes. On the other hand, only a few studies exist for time-series forecasting tasks, including Le Guen & Thome (2019) that utilizes dynamic time warping (DTW) and temporal distortion index (TDI) for modeling temporal dynamics. Next we describe mean square error (MSE) and DILATE, proposed by Le Guen & Thome (2019), and discuss their invariance to the distortions.
3.3 DISTORTION HANDLING IN CURRENT TIME-SERIES FORECASTING OBJECTIVES
Many measurement metries have been used in the time-series forecasting domain, and those based on the Lp distance, including Euclidean distance, are widely used to handle time-series data. However, such metrics do not have invariance to the aforementioned distortions (Ding al. 2008 Le Guen & Thome 2019) due to its point-wise mapping. Specifically, since L.p distance compares the values per time step, it cannot handle temporal distortions appropriately and vulnerable scaling of the data. Le Guen & Thome (2019) propose a loss function, called DILATE, to overcome the inadequate characteristic in the Lp distance metries by recognizing temporal dynamics with DTW and TDI. In terms of transformation, DILATE handles dynamic time scaling, especially, local time scaling with DTW, and phase shifting with penalized TDI, defined follows:
EDILATE((..0) = -y log (
Σ exp ( AEA.I
(A, (A, vi) (1- +(1-a)00)))
where A, 8(9i, wz). S2 are the warping path, cost matrix, and squared penalization matrix, respectively,
While DILATE shows better performance than existing methods, there is a missing point in invariance point of view. DILATE highly depends on DTW, which allows dynamic aligmment of the time- series for the predefined window. In such windows, DTW could align the signal regardless of their information, for example, periodicity. As a result, the model makes misbehavior that could cheat DTW within the window, as shown in Fig. [ bottom. DTW's scale and noise sensitivity are also problematic. Basically, DTW computes the Euclidean distance of two time-series after its temporal alignment in dynamic programming and the alignment relies on the distance function. Consequently, the dynamic alignment of the DTW can be properly achieved only when two series have the same range (Esling & Agon| 2012 Bellman & Kalaba 1959). That means, it hardly achieves invariance on amplitude distortion without appropriate pre- processccssngg Gong & Chen (2017) also show that DTW poorly matches the prediction and target (i.e., ground-truth) time-series with amplitude shifting. Even
5

---

Under review as a conference paper at ICLR 2023
when the target time-series is aligned with normalization, we cannot guarantee that the predicted and target time-series are properly aligned due to DTW's ' high sensitivity to noise. As a result, DILATE can generate poor alignment results that cause wrong optimization of TDI, which produces instability during optimization steps and incorrect results. To design an effective shape-aware loss function, have to understand measures and when the measures have transformation invariances. In the next section, we discuss how we interpret transformations in time-series forecasting point of view and which types of transformations should be considered in objective function design.
3.4 TRANSFORMATION INVARIANCES IN TIME-SERIES FORECASTING
In the time-series domain, data often have various distortions so measurements are needed to satisfy a number of transformation invariances for meaningfully modeling temporal dynamics. As discussed in Sec. 3.1 we set the goal of time-series forecasting as (1) point-wisely reducing the gap between prediction and target time-series and (2) preserving temporal dynamics of the target time-series. To satisfy both of them, we have consider (1) the method that should not have negative impact on the traditional goal of accurate time-series forecasting and (2) the distortions that play a crucial role in capturing the temporal dynamics of the target time-series. In this section, we review all six distortions whether it is feasible loss function or not, discuss their benefits and trade-offs, and find appropriate distortions to be considered in time-series forecasting.
Amplitude Shifting In wide range of situations, it is beneficial to capture the the trends of time- series sequence in spite of shifts in terms of amplitude. Thus, being invariant to amplitude shifting in a loss function takes many advantages in time-series forecasting: (1) shape-awareness invariant to amplitude shifting. (2) accurate deviation of values in modeling, and (3) effective on-time prediction of the peak sudden changes. To guarantee the amplitude shifting invariant in the optimization stage, the loss function should induce an equal gap between prediction and ground truth data in each step. Formally speaking, the loss function with consideration of the amplitude shifting should satisfy;
C(Y, Y) = 0 + Wsen,.....) d(yi, 0:) = k,
(1)
where E IR is an arbitrary and equal gap, and d(ys. (i) is a signed distance with a boundary y: > ÿt- By allowing tolerance between prediction and target time-series, models can follow trends in time-series instead of tending to predict exact values in point-wise. In short, unlike existing loss functions that handle only point-wise distance (e.g., DTW), we should deal with both the point-wise distance and its relational distance values to guarantee amplitude shifting.
Phase Shifting There are forecasting tasks, whose main objectives concern accurate forecasting of peaks and periodicity in time-series (c.g.. heart beat data and stock price data). For such tasks, phase shifting invariance is one of the best solutions for (1) modeling periodicity, regardless of translation on temporal axis and (2) having precise statistics with shapes, such peak and plateau values. If loss function is to be invariant to phase shifting, the function should satisfy:
C(Y, Y) = 0 e Y and Y have the same dominant frequency.
(2)
Note Eq. 2 allows a similar shape as target time-series in forecasting, not exactly same shape (e.g.. sin (z) and 2 sin(x + zo) with the same dominant frequency).
Uniform Amplification This proposition will be useful in case of sparse data that contains significant number of zeros. By adopting the uniform amplification invariance, models are able to focus non-zero sequences, whereas this proposition allows models to receive less penalty in zero sequences. Since guarantees shape- awareness with a multiplication factor in a timely manner as Fig. [ invariance for uniform amplification fits well. To have model trained with the uniform amplification invariance, the loss function should satisfy:
C(Y, Y) = 0 + Vie
y1
k() + 0).
(3)
y
Uniform Time Scaling, Dynamic Amplification, and Dynamic Time Scaling After careful consideration, we conclude that uniform time scaling, dynamic amplification, and dynamic time scaling are incompatible for optimization. We describe the reason below.

---

Under review as a conference paper at ICLR 2023
To achieve invariance for the uniform time scaling, the loss function should satisfy:
C(Y, Y) = 0 + Bc zz {o yt Des) {clyet Ji [0, ....,.........
This proposition will influence negatively original temporal dynamics, considering that it gives the tolerance of mispredicting periodicity (e.g., daily periodic signals) and even cannot catch events (e.g., abrupt changing values) in timely manner. In summary, it hinders models from capturing shape and corrupts periodic information.
For both dynamic amplification and dynamic time scaling, loss functions always are zero for all pairs when we do not set the limit of tolerance. For example, if we do not limit tolerance, the proposition for dynamic amplification invariance is as follow:
C(Y, Y) = 0 + VG E : c.tss
If a loss function satisfies the proposition, it is always zero because there always exists C2 yv/ 9: except ÿJ = O. Therefore, it is not able to give any information because all random values could be an optimal solution. The same situation happens with the dynamic time scaling if do not limit the window. Consequently, all of the uniform time scaling, dynamic amplification, and dynamic time scaling are unsuitable be objectives in time-series forecasting.
METHODS
In this section, we describe a novel loss function TILDE-Q (a Transformation Invariant Loss function with Distance EQuilibrium), which allows models to perform shape-aware time-series forecasting based on the three distortion invariances. To build transformation invariant loss function, we have to design loss function that satisfies the proposition for amplitude shifting invariance (Eq. [ phase shifting invariance (Eq. 2). and uniform amplification shifting invariance (Eq. 3). as discussed in Sec. 3.4 We select them for our loss function because they help models capture the shape and do not harm the goal of the traditional time- series forecasting (i.e., minimize gap between prediction and target time-series). Not only the loss function should satisfy these propositions, but also it should consider correlations between the whole sequence of outputs and ground truths rather than point- wisely optimizing the model. It is not achieved by other loss functions, such as MSE or DILATE. To handle all three distortions and the whole sequence of correlations, sa.shift, phase, and amp losses) that achieve one or more invariance by utilizing softmax, Fourier coefficient, and nuto-correlation design loss function.
build three objective functions
Amplitude Shifting Invariance with Softmax (Amplitude Shifting) To strengthen amplitude shifting invariance, we design a loss function that satisfies Eq. 1 This means, d(yi, 9s) needs to be the same value for all i. To satisfy the condition, we utilize the softmax function:
Sa (Y, Y) T" Σ9 ] Softmax(d(j)) (.))). coftmax(d(yja 0:))
(4)
Σj=1 (lyy.
where T", Softmax, and .(..)) are the length of sequence, softmax function, and signed distance function, respectively. Because the Softmax produces the proportion of each value, it only reaches to the optimal solution when it satisfies Eq. I Also, if we utilize Softmax, there is no need to know arbitrary equal gap k.
Invariances with Fourier Coefficients (Phase Shifting) As we discussed in Sec. 3.4 one candidate method to obtain phase shifting invariance is to Fourier coefficients. As described in prior studies (NG & GOLDBERGER (007), we can reconstruct original time-series only with dominant frequencies. In this way, utilize the norm of dominant Fourier coefficient of ground truth and prediction sequences as our additional objective function, achieving phase shifting invariance. When comes the other frequencies, we denote the norm of prediction sequence to reduce the value of Fourier coefficient. Consequently, with the help of our loss function, this loss function allows model to be noise robustness because the Fourier coefficients of white noises in original time-series are relatively small. Simply, we optimize the distance between Fourier coefficients of two time-series as:
||FFYY - F(Y) (|
dominant frequency
Lphase (Y, ))
(5)
otherwise
7

---

Under review as a conference paper at ICLR 2023
where || 1lp is the Ly norm. To obtain the dominant frequency terms, calculate the norm of the Fourier coefficient for each frequency and filter them with the squared root of sequence length, vTV. We also guarantee the minimum number of dominant frequencies vT². This loss function obtains uniform amplification invariance by utilizing a normalization technique to Fourier coefficients. For example, sin and sin z have the same Fourier coefficients if properly normalized. In summary. by Eq. [ we could obtain (1) invariance for phase shifting, (2) invariance for uniform amplification, and (3) robustness to noise.
Invariances with auto-correlation (Uniform Amplification) Although Fourier coefficients can be considered as a reasonable solution to catch the periodicity of the target time-series, it is not fully invariant to phase shifting for three reasons-(I) the statisties (e.g., mean and variance) in data keep changing, (2) such changing statistics also cause the changes of Fourier coefficients even in the same frequency, and (3) objectives only with a norm of them cannot fully represent the original time-series. Thus, we introduce an objective based on normalized cross-correlation, which satisfies Eq. 2 for periodic signal:
Lamp
= || Y) - R(Y, Y)Il:
(6)
where .): is normalized cross correlation function. This loss function helps predicted sequences mimic label sequences by calculating difference between the auto-correlation of the label sequences and cross-correlation between label and predicted sequences. Therefore, the label and prediction have similar temporal dynamics regardless of phase shifting and uniform amplification.
In summary. we introduce TILDE-Q (Transformation Invariant Loss Function with Distance Equilib- rium), combining Eq. Eq. and Eq. o follows:
LTILDES Y. Y) aha.shif 8) + (1 Cphase (Y, 8) + Camp (Y, 8),
(7)
where a E [0, 1] and is hyperparameter.
5 EXPERIMENTS
In this section, we present the results of our comprehensive experiments, demonstrating the effective- ness of TILDE-Q and importance of transformation invariance.
Experimental Setup
We conduct the experiments with four state-of-the-art models, including
Informer (Zhou et al. 202TD N-Beats (Oreshkin et al. 2020), Autoformer (Wu et al. 202T), and FEDformer Zhou al.) 2022 and one simple sequence-to- to-seuueeccee gated recurrent unit (GRU) model. We use seven real-world datasets ECCee000, Traffic, ETTh2, ETTm2, ECL, Exchange, and Weather and one synthetic Syythhe-S for model training. We repeat each experiment with a model and dataset 10 times in combinations with three different objective functions. Appendix A provides detailed explanations on the datasets, hyperparameter setting, and model architectures.
Table 1: Experimental results of short-term time-series forecasting on the three datasets with sequence- io-sequence GRU model.
Methods
GRU + MSE
GRU + DII ATE
GRU
TILDE-Q
Eval
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
Symthtic
0.0107
3,5080
1.0922
0.3525
0.0130
3,4005
1.1242
0.3825
0.0119
3,2873
1.1564
0.3811
ECG5010
0.2152
1.9718
0.8442
0.7743
0,8270
3,9579
2.0281
0.4356
0,2141
1.9575
0.7714
0.7773
Traffic
0.0070
1.4628
0.2343
0.7209
0.0095
1.6929
0.2814
0.6806
0.0072
1.4600
0.2276
0.7220
5.1 EXPERIMENT RESULTS
Evaluation Metrics In the experiment, we evaluate TILDE-Q with four evaluation metrics: mean squared error (MSE), dynamic time warping (DTW), its corresponding temporal distortion index (TDI) all of which are used in Le Guen & Thome (2019). As DTW is sensitive to noise and generates incorrect paths when one of the time-series data is noisy (as discussed in Sec. 3.3). we additionally the longest common subsequence (LCCS) for comparison, which is more robust outliers and
8

---

Under review as a conference paper at ICLR 2023
Table 2: Experimental results six real-world datasets (four cases) with four SOTA models and three training metrics. For all experiment, wC set input sequence length T
96
Medel
N-Boors
Inforner
Auroformer
HiMoorner DILATE
Methock
MSE
DELAATE
TILD6-Q
MSE
DILATE
VILDE-O
MSE
DELATE
TILDE-Q
MSE
TILLE-Q
Mesic
MSE
LCSS
MSE
LCSS
MSE
LC35
MSE
LCSS
MSE
LCSS
MSE
LCSS
MSE
LC55|
LCSS
MSE
96
10.157
00406
00.100
01487
16155
0.566
1.663
0.51
8:37%
0.537
[0.55
00.231
0.:31
06631
11.91
0.135
192
5,239
0.450
0,6.8
0.463
0.123
0.581
0.281
0.425
0.405
0.489
0.243
0/431
0.597
0.681
0.252
0.533
0.207
0.598
0.181
0663
0.369
0.526
0.199
0,612
356
6.289
0.454
1.140
0.458
0.213
0.537
0.308
0.643
0.416
0.506
0.295
0.16
0.339
0.595
0.375
0.525
0.236
0.597
0.230
0.605
0.351
0.509
0.258
0.504
00.168
0.418
1,01
0.457
113914
0.528
0.287
6.607
0.427
0.481
0.315
0.416
0.285
0.517
0.479
0
0.237
0.579
11991
0.50
(1267
0.161
637h2
192
[80779
0.022
0,672
0.57h
0.205
0.437
0.510
0.125
0.616
0.115
0.623
0.254
0.526
0:551
0.686
0.834
0.611
0.185
0.963
0.550
0.125
0.681
0.098
0.734
0.185
0.539
6.897
0.756
350
(0102
0.45%
0.250
01:11
0.178
0.619
11:86
0.636
(.250)
0.502
8:176
0,655
0.158
0.603
0.200
0.517
1.354
0661
01133
01607
1.349
0.505
0.327
0,632
[0.257
02,40
0.417
01585
11233
0.717
11216
08576
0.377
0.478
0.306
0.566
0.19
0/606
0.256
0.500
08388
06627
01196
01006
11251
0.061
6.182
0.6%6
0:450
01621
0.457
0.335
0.718
0.645
0.307
0.733
01:31
0.61
0.665
356
0.519
0,.5%
1.246
0.509
0.30
0.711
0.338
0.723
1.233
0,50
0.337
0.714
0.462
0,653
0.759
0,609
0.063
0.6
0.331
0.721
1.097
0,089
0.339
0.730
720
[0.524
0,571
1.306
0.533
11454
0.096
01641
0.455
1.300
0.550
8:467
0.429
0.500
0.618
0.855
0.607
0.504
0.641
0.396
0.995
01774
0.640
0.394
0.701
Traliic Eschargg
0,442
0.43
0.275
0.336
0.468
(0526
1.216
1,568
0.435
0.965
0.4%6
0.443
356
1.453
0:415
3,67%
0:557
1841
0.460
1.571
01468
1.873
0.443
1.691
0.488
0.548
0.118
0.805
0.511
0.51
0.332
01491
0.414
[1253
0.39)
0.590
0.134
7:0
1.856
0.407
3.001
0.540
2.89
0.462
1.764
0.468
1,820
0.529
1.915
0.510
1.362
0.216
1.404
01230
1.19
1122
12:2
01354
1:511
0.376
1,170
0.398
[6:234
0,830
2,332
0.525
0.229
0.837
0.261
2.961
0.731
6:228
0.549
00156
0.345
0.792
2,450
0.399
0.359
0.517
720
9.430
0,796
2.352
0,518
[1445
0.509
013447
0.515
2,655
0.587
0.356
0,755
0.372
0,871
1,073
0,818
0.234
0.56
0.229
0,892
3.003
0.558
0.351
0,373
Weathes
[000M
0.006
0.421
0:407
0,003
0.33
0.876
[UN]]
1.517
11814
15.55
111077
0.004
0.516
8,812
8:000
0.540
01017
0.4N2
0/002
0.511
[UH1
11544
11.97
11/017
0.001
8.579
0,013
0.507
0.567
0.000
0.418
00.15
0.998
0.508
(41044
0.514
9.805
0.569
(.508
0.414
(L,12
(1510
0.005
0.002
0.574
Coust
8
0
0
0
16
24
9
4
14
17
20
17
16
noise (Esling& & Agon 2012). The longer the length of matched subsequences is achieved, the better performance LCSS shows in modeling the shapes. For the ofae-oof-the-art models, we reports the MSE and LCSS. For the detailed results including DTW and TDI, please refer to Appendix B
Results and Analysis Table I shows the results of short-term forecasting performance of gated recurrent unit (GRU) optimized with MSE, DILATE, and TILDE-Q metrics. Synthetic, ECG5000, and Traffic datasets are used for the experiment. With the Synthetic dataset, every used metric shows its own benefits. This result indicates that similarity of the shape and MSE measures have clear advantage when a model is trained and evaluated with themselves. Also, since the model evaluated with real-world datasets, it is revealed TILDE-Q outperforms other objective functions in most evaluation metrics. These results indicate our approach for learning shapes in time-series data works better than existing methods for forecasting. DILATE does not show impressive performance with ECG5000 due to its high sensitiveness to noise, as discussed in Sec. 3.3
Table 2 summarize the experiment results with four state-of-the- models, N-Beats, Informer, Autoformer, and FEDformer. The models make predictions for both short-term (L=96) and long-term (T. up to 720), that we can investigate their performances with different forecasting difficulties. In most of datasets, the models with TILDE-Q outperform those with other training metrics. Especially for long-term forecasting, we observe that for N-Beats and Informer with TILDE-Q significantly improve the performance with the other metrics. We provide some visual examples in Appendix B and more detailed analysis, qualitative experiments with example visualizations, ablation study results in Appendix B This result implies that TILDE-Q improves performances of the models in learning temporal dynamics, including LCSS of N-Beats (improved over 10%)
6 CONCLUSION AND FUTURE WORK
We propose TILDE-Q, a transformation invariant loss function with distance equilibrium, which allows shape-aware time-series forecasting in a timely manner. To design TILDE-Q, we review existing transformations in time-series data and discuss the conditions that ensure transformation invariances during optimization tasks. The designed TILDE-Q ensures model to be invariant to the amplitude shifting, phase shifting, and uniform amplification so that a model better captures the shape in time-series data. To prove the effectiveness of TILDE-Q, we conduct comprehensive experiments with state-of-the. models and real-world datasets. The results indicate that the model trained with TILDE-Q generates more timely, robust, accurate, and shape-aware forecasting in both short-term to long-term forecasting tasks. We conjecture that this work can facilitate future research on transformation invariances and shape-aware forecasting.
9

---

Under review as a conference paper at ICLR 2023
REFERENCES
Abubakar Abid and James Zou. Learning a warping distance from unlabeled time series using sequence autoencoders, In Advances in Neural Information Processing Systems, volume 31, pp. 10568-10578, 2018.
B.R. Bakshi and G. Stephanopoulos. Representation of process trends-iv, induction of real-time patterns from operating data for diagnosis and supervisory control. Computers Chemical Engineering. 38(4):30--332 1994.
Gustavo E. A. P. A. Batista, Eamonn J. Keogh, Oben Moses Tataw, and Vinícius M. A. de Souza. CID: an efficient complexity-invarian distance for time series. Data Mining and Knowledge Discovery, 28(3):634-60) 2014. doi: 10.10077/1001---13--0333
R. Bellman and R. Kalaba. On adaptive control processes. IRE Transactions on Automatic Control, 4 (2):1 9, 1959.
Pavel Berkhin. survey of clustering data mining techniques. In Grouping Multidimensional Data Recent Advances in Clustering, pp. 25-71. Springer, 2006.
Donald J. Berndt and James Clifford. Using dynamic time warping to find patterns in time series. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. AAAIWS 94, pp. 359-370. AAAI Press, 1994.
Ioana Bica, Ahmed M Alaa, James Jordon, and Mihaela van der Schaar. Estimating counterfactual treatment outcomes over time through adversarially balanced representations. In International Conference on Learning Representations, 2020.
George E. P. Box, Gwilym M. Jenkins, Gregory C. Reinsel, and Greta M. Ljung, Time series analysis: forecasting and control. John Wiley, 2015.
Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). In Proceedings of the International Conference on Learning Representations, 2016.
Marco Cuturi and Mathieu Blondel. Soft-dtw: A differentiable loss function for time-series. In Proceedings of the 34th International Conference on Machine Learning, ICML' 17, pp. 894-903, 2017.
Gautam Das, Dimitrios Gunopulos, and Heikki Mannila. Finding similar time series. In Principles of Data Mining and Knowledge Discovery, pp. 88-100, 1997.
Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The uer time series archive. IEEE/CAA Journal of Automatica Sinica, 1(93: 1305, 2019.
Hui Ding. Goce Trajcevski, Peter Scheuermann, Xiaoyue Wang, and Eamonn Keogh. Querying and mining of time series data: Experimental comparison of representations and distance measures. Proceedings of the VLDB Endowment, 1(2):1542 14515555,,,2, 2008.
Philippe Esling and Carlos Agon. Time-series data mining. ACM Computing Surveys, 45(1), 2012.
Zhichen Gong and Huanhuan Chen. Dynamic state warping. CoRR, abs/ 10//1703.01110 141, 2017.
Lukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. Learning to remember rare events. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Eamonn J. Keogh. Efficiently finding arbitrarily scaled patterss in massive time series databases. In Knowledge Discovery in Databases: PKDD 2003, volume 2838 of Lecture Notes in Computer Science, pp. 253-265, 2003.
10

---

Under review as a conference paper at ICLR 2023
Eamonn J. Keogh and Chotirat (Ann) Ratanamahatana, Exact indexing of dynamic time warping. Knowledge and Information Systems, 7(3):55--38) 386, 2005.
Eamonn J. Keogh, Jessica Lin, and Wagner Truppel. Clustering of time series subsequences is mean- ingless: Implications for previous and future research. In Proceedings of the IEEE International Conference Data Mining, pp. 115-122. IEEE Computer Society, 2003.
Eamonn Keogh, Themis Palpanas, Victor B. Zordan, Dimitrios Gunopulos, and Mare Cardle. Indexing large human-motion databases. In Proceedings of the International Conference Very Large Data Bases, pp. 780-791, 2004.
G. Kerr, H.J. Ruskin, M. Crane, and P. Doolan. Techniques for clustering gene expression data. Computers in Biology and Medicine, 38(3):283-293, 2008
Vincent Le Guen and Nicolas Thome. Shape and time distortion loss for training deep time series forecasting models. In Advances Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Chunggi Lee, Yeonjun Kim, Seungmin Jin, Dongmin Kim, Ross Maciejewski, David Ebert, and Sungahn Ko. A visual analytics system for exploring, monitoring, and forecasting road traffic congestion. IEEE Transactions on Visualization and Computer Graphics, 26(11):1133-3146 2020. doi: 10.1 10111//TC..2191992222222
Hyunwook Lee, Seungmin Jin, Hyeshin Chu, Hongkyu Lim, and Sungahn Ko. Learning to remember patterns: Pattern matching memory networks for traffic forecasting. In International Conference on Learning Representations, 2022.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In Proceedings of the International Conference on Learning Representations. OpenReview.net, 2018.
Andrea Madotto, Chien- Sheng Wu, and Pascale Fung. Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pp. 1468-18 1478, 2018.
Yasuko Matsubara, Yasushi Sakurai, Willem G. van Panhuis, and Christos Faloutsos. FUNNEL: au- tomatic mining of spatially coevolving epidemics. In The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 105-114. ACM, 2014.
Arthur Mensch and Mathieu Blondel. Differentiable dynamic programming for structured predic- tion and attention. In Proceedings of the 35th International Conference on Machine Learning. volume 80 of Proceedings of Machine Learning Research, pp. 3462-3471, 2018.
JASON NG and JEFFREY GOLDBERGER. Understanding and interpreting dominant frequency analysis of af electrograms. Journal of Cardiovascular Electrophysiology, 18(6):680-685 685, 2007.
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference Learning Representations, 2020.
John Paparrizos and Luis Gravano. K-shape: Efficient and accurate clustering of time series. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD '15 pp. 1855-1870, 2015. doi: 10. 10.1145/2223372.777777
M.H. Pesaran, D. Pettenuzzo, and A. Timmermann. Forecasting time series subject to multiple structural breaks. Cambridge Working Papers in Economics 0433, Faculty of Economics, University of Cambridge, 2004.
Dararat Srisai and Chotirat Ann Ratanamahatana. Efficient time series classification under template matching using time warping alignment. In Proceedings of the International Conference on Computer Sciences and Convergence Information Technology, pp. 685-690, 2009.

---

Under review as a conference paper at ICLR 2023
Daniel Stoller, Mi Tian, Sebastian Ewert, and Simon Dixon. Seq-u-net: A one-dimensional causal u-net for efficient sequence modelling. Christian Bessiere (ed.), Proceedings of the International Joint Conference on Artificial Intelligence. pp. 2893-2900. ijcai.org, 2020.
Sainbayar Sukhbaatar, arthur szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In Advances in Neural Information Processing Systems, volume 28, 2015.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnttt (eds.), Advances in Neural Information Processing Systems, pp. 5998-6008, 2017.
Michail Vlachos, Marios Hadjieleftheriou, Dimitrios Gunopulos, and Eamonn Keogh. Indexing multi-dimensional time-series with support for multiple distance measures. In Proceedings the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '03, pp. 216-225, 2003.
Michail Vlachos, Philip S. Yu, and Vittorio Castelli. On periodicity detection and structural periodic similarity. In Proceedings of the SIAM International Conference on Data Mining, pp. 449 460, 2005.
T. Warren Liao. Clustering of time series data-a survey. Pattern Recognition, 38(1 38(11))1857-1874, 2005.
Andreas S. Weigend and Neil A. Gershenfeld. Time Series Prediction: Forecasting the Future and Understanding the Past. Aadison-Wesleyy 1994. ISBN 1-201-62201-2.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transform- ers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems, volume 34, pp. 22419-22430, 2021.
Xiaopeng Xi, Eamonn Keogh, Christian Shelton, Li Wei, and Chotirat Ann Ratanamahatana, Fast time series classification using numerosity reduction. Proceedings of the International Conference on Machine Learning, ICML '06, pp. 1033-1040. Association for Computing Machinery, 2006.
Fisher Yu, Vladlen Koltun, and Thomas A. Funkhouser. Dilated residual networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 636 666.444 IEEE Computer Society, 2017.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, Proceedings of the AAAI Conference on Artificial Intelligence, 35(12):11106 11115, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 27268 272286 2022.
Yunyue Zhu and Dennis Shasha. Statstream: Statistical monitoring of thousands of data streams in real time. In Proceedings of the International Conference on Very Large Databases, pp. 358-369. Morgan Kaufmann, 2002.
12

---

Under review as a conference paper at ICLR 2023
A DETAILED EXPERIMENT SETUP
Dataset In our experiment, we utilize six datasets - Synthetic, ECG5000, and Traffic dataset for the simple model (i.e., gequence-to-Sqquenee Gated Recurrent Unit) and ETTh2, ETTm2, and Electricity for the state-of-the of-e-of-the-art model (i.e., Informer and N-Beats). For each dataset, we describe some metadata them and experimental setting, including the input length n and prediction window L.
Synthetic: As Le Guen Thome (2019) describe, the Synthetic dataset is an artificial dataset for measuring model performance on sudden changes (step functions) with input signal composed of two peaks. The amplitude and temporal position of the two peaks are randomly selected. Then the selected position and amplitude of the step determined by a peak position and amplitude. We use 500 time-series for training, 500 for validation and 500 for testing. For the Synthetic dataset, we set input length as 20 and prediction window L 40. The generation code is provided in DILATE Githut
ECG5000; This dataset is originally a 20-hour long BCG (Blectrocardiogram), downloaded from Physione and archived in UCR Time Series Classification Archive (Dau et al. 2019). The data is split by each heartbeat and processed be in equal lengths (140). In the training, we use 500 for training, 500 for validation, and 4000 for testing. We take first = 84 steps as input and predict last 56 steps.
Traffic dataset is collection of 48 months (2015-2016) hourly road occupancy rate (between 0 to 1) data from the California Department of Transportation As Guen & Thome (2019) do, we utilize univariate series of the first sensor, total of 17544 data points. We set our problem as forecasting
Traffic:
future occupancy rates with n 168 historical data (past week). We use 60% of the data for training, 20% for validation, and the rest for evaluation.
ETT: The ETT (Flectricity Transformer Temperature) dataset, published by [Zhou et al. (2021), ) is 2-year data collected from two separated counties in China, including ETTh2 and ETTm2 datasets. Each data point has a target value of "oil temperature" and other 6 power load features. ETTh2 and ETTm2 datasets have 1-hour and 15 15-minute intervals, respectively. As As/Zhou et al. (2021) do, split them into 12/4/4 months for the uraining/allidation/eeiinng Detailed settings, such as the input and output length and hyperparameter setting, are based the information at Informer Githut
ECL: The ECL (Electricity Consuming Load) is a dataset recorded in kWh every 15-minutes from 2012 to 2014, for 321 clients. In experiment, split them into 15/3/4 months for the vrain/validationneest, as Zhou et al. (2021) do. Note that we use the same hyperparameter settings in the ETTh2 dataset.
Deep Learning Model Architectures We perform experiments with three different model architec- tures, including pruuence-to-equenee GRU, Informer, and N-Beats. To induce models to predict future time-series in timely manner, we set a 0.5 and 0.01 for TILDE-Q. Other training metries, including MSE and DILATE, are used as described in their original papers. All models are trained with Early Stopping and ADAM optimizer.
Sequence-to-Sequence GRU To evaluate TILDE-Q in simple model, we utilize one layer Sequence- to-Sequence GRU model. For the training of the GRU model, we set learning rate of le 3, hidden size of 128, trained by maximum 1000 epochs with Early Stopping and ADAM optimizer.
Informer When we train Informer with ETTh2, ETTm2, and ECL dataset, we utilize the official code and hyperparameter setting. the case of BCL dataset, as author answered their official code we utilize same hyperparameter and dataset splitting criteria ETTh2 dataset.
N-Beats For N-Beats, we utilize two generic blocks with the hidden size of 128. Additionally, we set the learning rate le for all three datasets.
httas://phvsionntt.orr
http://pems.dot.. htte:://gihhbbbcoo///o/ooo///
13

---

Under review as a conference paper at ICLR 2023
Autoformer For Autoforme 1
we use the official code and hyperparameter setting. For the ETTh2 dataset, we utilize hyperparameter settings described in the official code of FEDFormer"
FEDformer For FEDformet
R use the official code and hyperparameter setting.
B
ADDITIONAL EVALUATIONS
B.1
DETAILED EXPERIMENT RESULTS AND ANALYSIS
Table 3: Detailed experimental results on six real-world datasets (four cases) with N-Beats.
Methods
N-Beats
MSE
N-Beats
DILATE
N-Bcats
TILDE-Q
Metric
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
96
0.1869
7.2379
2.3787
0,4688
0.3105
6,5849
3.6490
0,4879
0,1557
5,1011
1.3240
0,5862
ET1m2 BT
192
0,2385
11,5667
4.9153
0.4505
0.6186
9.7254
7.0831
0.4637
0,1738
7.6334
2,4122
0.5819
336
0,2889
16.5255
11.5207
0.4544
1.1406
13.7328
14.6986
0.4584
0.2132
11.3351
5,3556
0,5373
720
0.3881
24.1570
18.8462
0.4381
1.6713
19.4392
23.7028
0.4575
0.3044
17.6006
9.6636
0.5287
0.0790
3,9685
2.0436
0.6721
0.1524
7,9302
5.5597
0.4379
0.0952
4,0110
2.1939
0.6902
192
0.1224
6.8695
3,2834
0,5762
0.2055
10.0393
8.5602
0,5107
0,1286
6.3556
4.9798
0,6160
336
0,1824
12.1438
8,5915
0.4587
0.2501
12.6342
16.1473
0.4819
0.1705
8,9377
8,3539
0.6195
720
0,2370
22.8676
17.8458
0.4929
0.4170
17.7764
24.6877
0,5836
0.2336
14,2715
19.0883
0,7070
96
0.3666
3,5207
0.2989
0.6589
1.1156
5.1430
2.6613
0.5074
0,3183
2.9707
0.4844
0.7229
BCL
192
01.4307
5.7578
0.4253
0.6212
1.1859
7.3406
2.8488
0.4973
0,3383
4.1817
0.4229
0.7187
336
0.51999
8.5563
0.5384
0.5965
1.2460
9.5096
3.0517
0.5091
0,3831
5.6643
0.3024
0,7112
720
0.6240
13,9436
0.6510
0,5717
1.3061
13,1928
3,7279
0,5337
0.4540
8,9997
0.3251
0.6960
Exchange
96
0,4496
8.6395
4.3197
0,4424
0.3945
8,9661
4.3286
0,4316
0.2748
7.9744
5.2964
0,4467
192
1.2161
12.1857
10,5166
0,4157
1.5684
13.0560
9,2434
0,4061
1.6629
11.5557
8.7896
0.4348
336
1.4529
14.7085
19.0407
0,4130
3,6784
17.5189
17.2512
0,3871
1.8432
12.5648
20,8871
0.4603
18563
21.7347
50.6751
0.4073
3,9008
26.7020
74.0546
0.3400
2.8487
19.1588
53.8069
0.4619
0,2349
2,1046
0.0216
0.8303
2.3325
3,9657
1,2052
0.5250
0.2286
2.0699
0.0207
0.8371
Weather Traflic
192
0.3014
3,4040
0.0142
0.7916
2,5627
5,4169
1.1355
0,5515
0,3352
3,2559
0.0119
0,8028
336
0.3455
4.6409
0.0088
0.7918
2.4599
8,2828
1.3377
0.5208
0.3990
4,2622
0.0066
0.8206
720
0.4298
7.0561
0.0045
0.7958
2.3522
12.6258
0.9967
0,5177
0.4480
6.7344
0.0034
0,8085
0.0042
9.3228
5.9134
0.4072
0.0023
8,9289
5.0617
0.4256
0.0010
6.5198
6.0450
0.5168
192
0.0056
10.9682
11,9549
0.4212
0.0030
12.8164
10.6858
0.4307
0.0017
8.8391
9.0867
0.5076
336
0.0058
13.3578
14.6572
0.4243
0.0087
20.4895
23.7903
0.3579
0.0026
11.8682
9.6758
0.5074
720
0.0068
18.5861
1432
0.4315
0.1534
28.6021
47.4488
0,3982 0.0029
17.1895
19.1942
0.5078
At first, we observe that the model optimized with TILDE-Q outperforms the same model optimized with other objective functions in both short- and long-term forecasting tasks. An interesting point in the results is the large increased errors of TDI and DTW with long-term forecasting. For example, TDI of Informer with DILATE shows dramatically increased error with ECL dataset, as the forecasting window increases, while LCSS does not produce such large increased error. We attribute this to the weakness of DTW-based loss functions, which have weakness due to high sensitiveness on noise. In contrast, TILDE-Q does not show such large performance drop and even achieves better performance in the long-term forecasting (e.g.. Table 4 ETTh2). Additionally, we can find that Informer with TILDE-Q on ECL data and N-Beats with TILDE-Q on all three datasets show significant improvements. It indicates that TILDE-Q success model shape, but other metrics could not. We provide additional qualitative results below.
Next, we present qualitative analysis of the results. Fig. Bshows how the model with different training metrics forecast with different datasets. From the figure, we have noticed that TILDE-Q allows the model to generate more robust, shape-aware forecasting, regardless of the amplitude shifting, phase shifting, and uniform amplification. For example, in the case of N-Beats (Fig. (b) bottom), TILDE-Q generate forecasting results, which are more robust, shape-aware prediction compared to other metrics. We also see the strength in the Informer case (Fig. (b). top). Even when the model has not enough ability to capture shape, TILDE-Q tries to retrieve the shape. We provide additional gualitative results with visualization below. When the model have enough ability to capture shape (i.c., except ETTh2, Informer of T E [192, 336, 720)). TILDE-Q shown its noise-robust.
httpss//ggrthobb.cco/////igggggp
14

---

Under review as a conference paper at ICLR 2023
Table 4: Detailed experimental results on six real-world datasets (four cases) with Informer.
Methods
Informer+ + MSE
Informer
DILATE
Informer
TILDE-Q
Metric
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
96
0,2466
6,9254
3.6676
0,4633
0.3284
6.3109
3.5838
0.5037
0.1768
5.8437
1.6734
0.5379
BTTm2 BrTh2
192
0,2818
10.2654
11,1580
0,4254
0.4086
8.8262
7.1780
0.4893
0,2433
10.2134
9,9865
0.4317
336
0.3089
12.1822
18,7014
0.4434
0.4164
10.3779
13.2580
0.5062
0.2958
13,5586
20.2850
0.4165
720
0.2877
17.6369
38.4617
0.4425
0.4229
14.1196
23.9403
0.4815
0.3157
18.4617
43.3238
0.4262
0..8889
3.4007
1.5719
0,7386
0.1263
6.0144
2.7757
0.5129
0.0871
3.1354
1.3474
0.7817
192
0.1157
5,7964
2.8128
0,6705
0.2340
9,7004
7,8354
0.5266
0.1317
5.7093
2,9129
0.6983
336
0,1866
8,9971
6.7970
0.6365
0.2805
11,7889
13.3861
0,5025
0.1767
9.0866
7,4023
0.6555
720
0.2165
14,7685
24.6694
0.5768
0.3745
16.7734
29.2783
0.4747
0.2063
15.3057
24.1959
0.5860
96
0.2709
2.8067
0.1720
0.7032
0,9856
3.6394
1.4794
0.6324
0.2800
2,9466
0.2473
0.7275
BCL
0.2793
4,1193
0.1508
0,7060
1.1209
5.2289
2.1749
0.6053
0.3077
4,2693
0.2978
0.7336
336
0.3203
5,9533
0,1642
0.7222
1.2331
7,8470
3.0415
0.5694
0.3271
5,8090
0.1984
0.7143
720
0.6414
15.8561
4,4284
0.4564
1.3706
12.5981
5.6720
0.5506
0.4676
11.4027
0.7107
0.6298
Exchange
96
0.3534
8.0965
4.8843
0,4689
0.3260
7,7370
5.6336
0,4678
0,5264
7,9866
6,5120
0,4553
192
0,9682
11,0843
11,3110
0.4647
0.9737
10,8894
15.6770
0,4584
1.2845
10,4358
10.7009
0,4959
336
1.3710
12.8076
18,5937
0.4676
1.6735
12.7034
29.2013
0.4428
1.6912
12.2349
18.2197
0.4932
720
1.7586
22.6852
59.4243
0,4681
1.8292
16,0093
56,8687
0,5293
1.9130
24.0510
62.8152
0,5104
0.2606
2,0994
0,0208
0,8329
2.9612
2.3355
0,9646
0.7312
0.2284
2,0027
0.0194
0,8490
Weather Tratfic
192
0,2920
3,2573
0.0126
0,8158
2.9978
3,5451
0.8429
0.7394
0,2753
3.1721
0.0125
0,8248
336
0.3109
4,6581
0.0078
0.8115
2.9696
4,9879
1.2672
0.7117
0,2993
4.4715
0.0077
0.8170
720
0.3472
6.7989
0.0040
0,8146
2.6845
10.7450
3,4514
0,5874
0.3859
7.5424
0.0051
0.7752
96
0.0043
8,2890
5.4604
0.4556
0.0069
6,5571
4.7505
0,5159
0.0021
5.5412
4.5012
0.5602
01.0311
10.7993
9.2928
0.4523
0.0041
10.5645
9.4713
0,4704
0,0028
8,2535
5,8289
0.5516
336
0.0051
13.8721
22.2699
0,4451
0.0055
12.0586
16.4933
0,4884
0.0039
10.8802
10.5220
0.5668
720
0.0061
21.7720
41,5877
0.4476
0.0737
16.8378
29.8112
0,5142
0.0447
13,9934
20.9991
0.5689
Table 5: Detailed experimental results on six real-world datasets (four cases) with Autoformer.
Methods
Autoformer
MSE
Autoformer
DILATE
Autoformer + TILDE-Q
Metric
MSH
1YTW
TDI
LCSS
MSE
DTW
TDI
LCCS
MSE
DTW
TDI
LCCS
96
0.1538
5,2227
2.1865
0.6187
0.2211
6.0453
2.5345
0.5315
0.1494
5.1060
1.9752
0.6317
ETTm2 Er
192
0.1974
7,8730
3,3382
0.6019
0,2825
8.6696
5.6671
0.5335
0,2079
7.8917
3.7532
0.5984
336
0,2393
10.8002
7,3141
0.5954
0,3759
11.0335
13.1347
0.5257
0,2360
10,7212
7.0085
0,5971
720
0.2859
16.3502
15.9233
0,5772
0.4296
15.9819
22.2173
0.4924
0.2378
16.0002
13.7906
0.5795
96
0.0990
4.3498
2.5052
0.6756
0.1135
5.3197
2.2211
0.5996
0.0940
3,9078
2,2587
0.7075
192
0.1344
6.3207
3.3676
0.6512
0.1854
8.5209
3,7894
0.5506
0,1259
6.0979
2.9278
0.6810
336
0.1587
9,4374
6,9205
0.6036
0.2001
12.0265
8.8305
0.5370
0.1548
9.5223
7.2875
0.6169
720
0.1999
14.8332
11.9655
0.6064
0.2665
17.8025
17.4114
0.5001
0,1885
14.5844
9.9918
0.6277
96
0.4209
3,5957
0,2461
0.6487
0.6813
3.6490
0.4780
0.6253
0,3515
3.2173
0.2298
0.6912
BCL
192
0,4206
4.9924
0.3416
0.6574
0.7319
5.5324
0.2775
0.6118
0.4032
4,8581
0,3301
0.6680
336
0.4621
6.6888
0.2795
0.6535
0.7895
7.5665
0.2503
0.6091
0.4637
6.7335
0.3923
0.6429
720
0.5005
10.8571
0.2383
0.6183
0.8630
12.1416
0.1877
0.6074
0.5049
9.8492
0.2525
0.6420
Exchange
96
0.2472
8. 8,2957
5.8340
0.4577
0.1921
8,4651
5.6328
0.4646
0.1730
8.2044
5.1165
0.4577
192
0.3255
11.4212
17.0909
0.4319
0,4732
82.8599
19.0164
0.4124
0.2955
11.3655
15.4372
0.4433
336
0.5483
15.1853
44.4975
0.3277
0.8035
18.0948
57.5819
0.3114
0.5331
16.7350
45 8166
0.3321
720
1.3620
24.6397
145.3080
0.2357
1.4936
27.7069
15166611
0.2302
1.1993
19,5296
121.8509
0.2233
96
0.2562
1,9689
0.0178
0.8761
0,4835
1.9044
0.0392
0.8521
0.2275
1.8778
0.0168
0,8879
Weather Traffic
192
0.2604
2,89222
0,0091
0.8780
015653
3.0466
0.11343
118187
0.2497
2,8793
0.0116
0,8817
336
0.2474
4.0026
0,0051
0.8797
0,8155
4,2637
0.0327
0.8047
0,2422
3,9469
0.0059
0.8760
720
0.2720
6.4371
0,0030
0,8710
1.0729
6.0776
0.0217
0.8176
0,2836
6.1751
0.0034
0.8674
96
0.0168
7,4658
6.0336
0.4818
0,0019
5.9775
4,9688
0.5306
0.0015
5.9829
4.8957
0.5461
192
0.0069
10.6173
7.4506
0.4941
0.0026
8,3686
5.4565
0.5423
0.0017
7.7799
6.1032
0.5355
336
0.0052
12.5224
13.2607
0.4898
0.0030
12.4524
1811
0.4854
0.0020
10.3144
9.0025
0.5252
720
0.0078
18.5079
25.8063
0.4744
0.0115
20.1354
36.7754
0.4721
0.0023
15.2563
18.3133
0.5102
smooth forecasting with correctly modeled temporal dynamics. In the most of N-Beats results and some of Informer results, TILDE-Q reveals that these models have enough ability to capture the temporal dynamics with proper loss function. In summary, TILDE-Q proves that it is agnesiig, noise-robust, and able to capture the shape.
15

---

Under review as a conference paper at ICLR 2023
Table 6: Detailed experimental results on six real-world datasets (four cases) with FRDformer.
Methods]
FEDformer
MSE
FEDformer
DILATE
FEDfomer
TILDE-Q
Metric
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
MSE
DTW
TDI
LCSS
96
0.1299
4.7265
1.2607
0.6690
0.1906
6.2294
1.8228
0.5261
0.1381
4.7578
1.3560
0.6621
FT7h2
192
0.1819
7.6178
2,6979
0.6229
0.2688
8.8422
4.8043
0.5261
0.1988
7.6174
2.7712
0.6124
0.2305
10.5860
6.7027
0.6050
0.3506
11.4834
12.8408
0.5091
0,2382
10.4108
6.6218
0.6039
720
0.2776
15.7013
14.7466
0.5911
0.4327
14,0692
20.6266
0.5091
0.2871
15.3120
16.4059
0.5808
96
0.0682
3.0962
1.3862
0.7868
0.1147
4.6648
2.2981
0.6325
0,0669
3.0328
1.3556
0.7918
E11m2
192
0.0976
5,2417
2,0295
0.7340
0. 1848
8.0678
4.4893
0.5391
0.0971
5.1508
2.1782
0.7384
336
0.1326
8.3151
5,4619
0.6667
0.2493
13.6349
11.7563
0.5049
0.1279
8,3010
4.5488
0.6828
720
0.1957
14.2579
11.8328
0.6262
0.2913
17.4636
41,9434
0.4806
0.1822
14.1131
10.4778
0.6361
0.2531
2.6402
0.1436
0.7322
0,4794
2.8685
0.2482
0.6943
0.2638
2.6594
0.1614
0.7265
ECI.
192
0.2945
3,8647
0.1831
0.7306
0.5485
4.3313
0.1732
0.6813
0,2821
3.7830
0.1277
0.7340
336
0,3313
5,2789
0.1078
0.7207
0.6967
5.7911
0,1985
0.6892
0,3385
5.1763
0.1229
0.7290
720
0.3956
8,5881
0.0632
0.6961
0.7741
10.1163
0.8837
0.6403
0,3939
8.5665
0.0784
0.7013
Taffic Exchange
0.11477
8,5595
4,4258
0.4347
0,3884
8.9178
7.0385
0.4439
0.1215
8,0591
6.1979
0.4704
0.2694
12.5168
12.3117
0,4202
0.5912
13.1929
15,4207
0.4187
0.2956
11.5607
12.1302
0,4474
336
0.4916
16.4673
27.0756
0.4140
0.7520
18.1381
33,7878
0.3969
0,5896
15.9267
24.8808
0,4342
720
1,2115
25.6243
108.0500
0.3838
1,5110
26.7031
91.6313
0.3760
1.1700
24.6190
71.8783
0.3935
96
0.2074
1.9132
0.0165
0.8824
0,3533
1.8617
0.0291
0.8609
0.1867
1,8386
0.0152
0.8983
0.2051
2.7761
0.0085
0.8951
1.4682
2.7545
0,1312
0.8591
0.1961
2.7.80
0.0083
0.8975
336
0.2140
3,7583
0.0047
0.9023
2.9741
3.7440
0.1779
0.8519
0,2059
3.7834
0.0050
0.8947
720
0.2291
5.9735
0.0026
0.8919
3.0829
5.8173
0,0949
0.8580
0.2312
6.1080
0.0022
0,8735
96
0.0070
6.1550
4,7039
0.5264
10.0017
5.9507
4,5891
0,5535
0.0014
5.5205
4.1085
0.5792
Weather
192
0.0063
7.6906
6,2940
0.5417
0.0020
8.6593
4.6333
0.6000
0.0017
6.9706
5.0003
0,5863
0.0046
10.3501
9,4691
0.5261
0.0053
8249
9.6239
0.4801
0,0018
9.3016
7,4610
0.5784
720
0.0060
15.2429
24.5379
0.4911
0.0029
16.3929
20.0782
0.4892
0.0024
13.7936
15.7668
0.5737
Infiemer.
GRU-DILATE
GRU-TILDE-Q
ETT22 N-lizata 2.- 220
1h1
Figure 3: Qualitative results with simple sequence- -o-encece---suuence GRU model (a) and state-of-ther -he
model (b).
B.2 ADDITIONAL QUALITATIVE EXAMPLES
16

---

Under review as a conference paper at ICLR 2023
GRU-DELTT
GRU-TIUBES
GRUIMS
GRU-DILATT
GRU-TTUEES
GRI MISI
GRLL DILATI
GIEA TIL DESS
MM
Synthetic
ECG5000
Traflic
Figure 4: Qualitative results with simple sequence-to-sequenne GRU model
ETTIA, Informer, 96
ETTEZ, obents,
ETINE, Auteformer,
KTT12, Informer, 150
PTTRO phents 150
VITEOO Anteffrrerr 192
Figure Qualitative results with ETTh2 in short-term forecasting
Figure 6: Qualitative results with ETTh2 in long-term forecasting
17

---

Under review as a conference paper at ICLR 2023
ETTm2, Informer.
ETTmZ, nbeots,
ETTo2, Autoformer.
-4-
.05
-27
+2.00
Leae
-1.0
1.34
11180
comate
Informer,
ETTmZ, Autoformer,
1.53
1.3
+.20
01.00
TEDRO
710000
Figure 7: Qualitative results with ETTm2 in short-term forecasting
ETTmZ, Informer, 336
ETTa2, nbeats 336
ETTm2, Antoformer. 336
-025
4.25
20
ETTm2, Informer: 720
ETTm2 nbeats, 720
ETTm2 Autoformer: 720
-23
Figure 8: Qualitative results with ETTm2 in long-term forecasting
18

---

Under review as a conference paper at ICLR 2023
ECL Informer, 96
ECL obents,, 26
ECL, Astefermer 96
RCL Iodortmess 192
ECL abbutt
BCL Aut Autoformee 152
Figure 9: Qualitative results with ECL in short-term forecasting
Figure 10: Qualitative results with ECL in long-term forecasting
19

---

Under review as a conference paper at ICLR 2023
B.3 ABLATION STUDY
To evaluate the effect of the o, 7, and measure the effect of each loss function, we conduct a set of experiment with ETTh2 dataset and N-Beats the long-term forecasting problem. As can see in the Fig. [T] the model tends to predict amplification-free forecasting when a increases. This results indicate our motivation, "ho.shift will return the forecasting results with same standard deviation with timely manner but without consideration of proper average value."
Furthermore, in the top of Fig. 12 we can observe three things: (1) if utilize La.shift only, we intended, it have different average (-1.19 vs. 0.11) but relatively similar standard deviation (0.408 vs. 0.299): (2) the case of Lphuse only, they can capture dominant frequency and produce relatively less-noisy forecasting: (3) Lamp have relatively similar average value (-1.195 vs. -0,319), but it has far different standard deviation (0.408 vs. 8.592). In contrast, forecasting results of the model trained with MSE is very noisy and hard to interpret (Fig. 12 bottom). Note that normalized the results in Fig. 12 because of the scale issue.
In Table 7 we provide how model performances vary with respect to hyperparameters of TILDE-Q For the default setting, we utilized a = 0.5, ry = 0.01. Because the design of TILDE-Q mainly focuses on shape modeling, hyperparameter. But their trade-offs are revealed decrease a,
see that DTW and LCSS not critically changing for the
the MSE and TDI. For example, when we observe TDI increases. It indicates the trade-offs of phase shifting invariance, which has tolerance for non-timely forecasting. Also, we can see that increasing a or affects the MSE. When have a we have no Cphase and less penalty for the statistical differences, and its absence causes the high MSE, as we can see in Fig. II also affects the MSE, but Lphase reduces Lamp's 's side effect.
Table 7: Ablation study on with ETTh2, L 720, and N-Beats
Metrie
Default
1 = 0.1
- = 0.5
2 1.0
a 0.0
0.1
0.8
1.0)
Casnsyi only
Cphase only
Comp only
MSE
0.3005
0.2968
0.3083
0.3168
0.3075
0.3161
0.2872
1.1752
1.5123
0.3391
1.8453
DTW
17,5154
17.5265
17.5649
17.7302
17.7564
17.5931
17.6508
17.6886
17.7261
17.848
18.0261
TDI
9.2197
9.1303
9.2261
9.4366
10.3550
9.8957
8.4725
8.7118
10.2519
12.8568
10,5602
LCSS
0.5382
0.5366
0.5277
0.5137
0.5050
0.5137
0.5584
0.5445
0.5341
0.4920
0.5086
ETTh2, abeals, 720
-15
1.00
382
900
12
Figure 11: Ablation study result visualization with different a on ETTh2 dataset
20

---

Under review as a conference paper at ICLR 2023
label, mean -1,195, std 0.408
Amplitude only,
0.299
Phase only, mean -0.056, std 0.031
Amolification only, mean -0 00.319. std 8.592
100
200
300
400
500
600
700
-0.5 0.50
-.00
-12
-150
-1.75
Label, mean -1.195, std 0.408
MSE, mean -0.967, stdl 0.097
-2.00
100
200
300
400
500
600
700
Figure 12: Ablation study result visualization of three proposed loss function on ETTm2 dataset
21


Reviews:
[{"review_id": "iGE4w9Vqe1", "reviewer": "Reviewer_bzqM", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work studied distortion handling in timer series forecasting problems. In particular, a novel loss function is proposed to guide models to consider amplitude, phase, and uniform amplification shifting invariance. ", "summary_of_the_review": "This paper is well-written and provides a high-level of readability to most readers who work in the field of time series forecasting. The authors thoroughly explain why point-wise metrics could mislead the model and how models could benefit from learning shape-aware loss functions. The experiment shows the significant improvement the proposed loss function could bring, and a detailed experiment analysis is provided. ", "strength_and_weaknesses": "Strengths\n1. The related work is comprehensive and shows a thorough analysis of different time series distortions.\n2. The problem studied in this paper is important for the applications related to time series forecasting.\n3. The paper is well-organized and easy to read.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "LDcL_e428W", "reviewer": "Reviewer_szr1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates distortions in amplitude and phase to understand shape-awareness and distortion invariances in time-series forecasting. This paper uses TILDE-Q that has invariances to many existing distortions and achieves shape-awareness and informative forecasting in a timely manner. Experimental results show the efficacy of proposed method.", "summary_of_the_review": "This paper investigates distortions in amplitude and phase to understand shape-awareness and distortion invariances in time-series forecasting. This paper uses TILDE-Q that has invariances to many existing distortions and achieves shape-awareness and informative forecasting in a timely manner. Experimental results show the efficacy of proposed method.\n\nHowever,  the experiments are not compared with the state-of-the art methods, such as FedFormer, NHITS and etc. Also it's not sure that whether this method is general enough for other Transformer-based models, such as FedFormer and etc.", "strength_and_weaknesses": "Strengths:\n\n1. It's novel to investigates distortions in amplitude and phase to understand shape-awareness and distortion invariances\n2. The formulation on time series distortions in transformation perspective is novel and interesting. \n\nWeaknesses:\n\n1. The experiments are not compared with the state-of-the art methods, such as FedFormer, NHITS and etc.\n2. It's not sure that whether this method is general enough for other Transformer-based models, such as FedFormer and etc.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KpuzPHsxkD", "reviewer": "Reviewer_4GPm", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a new loss function (transformation invariant loss function with distance equilibrium) for time series forecasting. This new loss function helps capture the shape of time-series sequences. The authors tested the effectiveness of the new loss function on multiple datasets and models.", "summary_of_the_review": "The paper assembles too many strong assumptions without very good justification. It is a clear rejection.\n\n-----------\nI have read the authors' responses and would like to keep the same score.", "strength_and_weaknesses": "Strength:\n1. The authors have done a thorough literature review on the SOTA methods, models, and metrics on time series forecasting.\n2. The authors also have experimental results to compare the proposed loss function and the other two metrics with 4 SOTA models on 5 datasets.\n\nWeaknesses:\n1. 'The output Y_hat should have similar temporal dynamics with Y.'\nThis is a strong assumption without much justification. It will be helpful to elaborate on why temporal dynamics are important and give examples of how they benefit time series forecasting in reality.\n2. The definition of temporal dynamics, and distortion is quite fuzzy. It is necessary to give mathematical definitions for these two concepts.\n3. The authors criticized that DILATE handles time scaling with DTW and phase shifting with TDI but did not explain the link between DILATE and the new loss function. It will be good to have a separate section to clearly explain the relationship between DILATE and TILDE-Q. Is it beneficial to merge these metrics into one?\n4. The Fourier transform is one of the classical ways to analyze time series data and the authors used the Fourier coefficients for phase shifting. However, it is not clear how to define dominant frequency.\n6. Ablation study is not complete.\nIt is not clear how the authors tuned hyperparameters alpha and gamma to generate results in Tables 1-4 which is critical to justify the effectiveness of the new metric. Thoroughly tuning these parameters can help get better results than roughly tuning compared loss functions like DILATE. It is encouraging that the authors showed some results of tuning alpha in the Appendix but still no result w.r.t. gamma is available.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CDMXh1opxIu", "reviewer": "Reviewer_wdgV", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new loss function for time-series called TILDE-Q, which has 3 components. Each loss component is invariant to different distortions, including amplitude shifting, phase shifting, and uniform amplification. The paper starts with formal definitions of common time-series distortions and then designs 3 loss components that are invariant to 3 types of distortion. The experiment results suggest that the new loss function is model-agnostic. It can improve the accuracy of different existing time-series models, even on common existing metrics such as MSE or DTW.\n\n", "summary_of_the_review": "The paper gives a good systematic review of common time-series distortions. The paper is well-presented with good support materials. The designs of the 3 loss components are elegant and effective. The experiment results are strong, which shows that TILDE-Q, the weighted sum of the 3 loss components, can improve the accuracy of different existing time-series models, even on common existing metrics such as MSE or DTW.", "strength_and_weaknesses": "Strength:\n  - The paper is self-contained. It provides a systematic study of different time-series distortions and builds up invariant loss components from there.\n  - The paper is well-presented, with clear notations and figures. The appendix is also very useful with detailed experiment setup and qualitative results.\n  - The new loss function yields significant accuracy improvements across different time-series models on various datasets.\n\nWeaknesses:\n  - The paper does not study the loss components' weights $\\alpha$ and $\\gamma$. How sensitive are those hyper-params? What happens if we change them, qualitatively and quantitively?\n  - TILDE-Q itself is not invariant to any of the amplitude shifting, phase shifting, and uniform amplification distortions.\n  - Maybe it is worth studying what happens if we add MSE to TILDE-Q loss.\n  - The paper claims that \"time-series forecasting models should aim to both accurately forecast the value for each time-step and the prediction should have similar shapes as those in original time-series\". This is not necessarily true. In practice, time-series forecasts deliver value by allowing future planning. Therefore, the cost of a specific error or distortion in the forecast depends strongly on the kind of planning we use the forecast for.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "5NIan9XmNFG", "reviewer": "Reviewer_iGk7", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose the tilDEq loss function that combines phase, amplitude, and shift sub-losses and claims to outperform MSE. The paper needs to improve its motivation and experiments.", "summary_of_the_review": "Strong reject.\nThe importance of the study of loss functions tailored for the forecasting task starkly contrasts with this submission's quality.", "strength_and_weaknesses": "Strengths\nL-norm minimization for machine learning and in particular forecasting, is a very important topic.\nThe authors identify some challenges of L-norm minimization within the forecasting task.\n\nWeaknesses\n1. Experimentation is not extensive; the authors are cutting by half the number of datasets from AutoFormer (Wu, 2021) experiments that used ETT, ECL, Exchange, Traffic, and Weather. Is it possible for only the datasets that align with the paper's research to be reported? Please report the results on all the missing datasets.\n2. Motivation of the paper is lacking, see comments of Figure 1.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject