TILDE-Q: a Transformation Invariant Loss Function for Time-Series Forecasting

## Abstract

Abstract Time-series forecasting has gained increasing attention in the field of artificial intelligence due to its potential to address real-world problems across various domains, including energy, weather, traffic, and economy. While time-series forecasting is a well-researched field, predicting complex temporal patterns such as sudden changes in sequential data still poses a challenge with current models. This difficulty stems from minimizing L p subscript 𝐿 𝑝 L_{p} italic_L start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT norm distances as loss functions, such as mean absolute error (MAE) or mean square error (MSE), which are susceptible to both intricate temporal dynamics modeling and signal shape capturing. Furthermore, these functions often cause models to behave aberrantly and generate uncorrelated results with the original time-series. Consequently, the development of a shape-aware loss function that goes beyond mere point-wise comparison is essential. In this paper, we examine the definition of shape and distortions, which are crucial for shape-awareness in time-series forecasting, and provide a design rationale for the shape-aware loss function. Based on our design rationale, we propose a novel, compact loss function called TILDE-Q (Transformation Invariant Loss function with Distance EQuilibrium) that considers not only amplitude and phase distortions but also allows models to capture the shape of time-series sequences. Furthermore, TILDE-Q supports the simultaneous modeling of periodic and nonperiodic temporal dynamics. We evaluate the efficacy of TILDE-Q by conducting extensive experiments under both periodic and nonperiodic conditions with various models ranging from naive to state-of-the-art. The experimental results show that the models trained with TILDE-Q surpass those trained with other metrics, such as MSE and DILATE, in various real-world applications, including electricity, traffic, illness, economics, weather, and electricity transformer temperature (ETT). Official codes are available at https://github.com/HyunWookL/TILDE-Q

## 1 Introduction

Time-series forecasting has been a core problem across various domains, including traffic domain (Li et al., ; Lee et al., ), economy (Zhu & Shasha, ), and disease propagation analysis (Matsubara et al., ). One of the key challenges in time-series forecasting is the modeling of complex temporal dynamics (e.g., non-stationary signal and periodicity). Temporal dynamics, intuitively, shape, is the most emphasized keywords in time-series domains, such as rush hour of traffic data or abnormal usage of electricity (Keogh et al., ; Bakshi & Stephanopoulos, ; Weigend & Gershenfeld, ; Wu et al., ; Zhou et al., ).

Although deep learning methods are an appealing solution to model complex non-linear temporal dependencies and nonstationary signals, recent studies have revealed that even deep learning is often inadequate to model temporal dynamics. To properly model temporal dynamics, novel deep learning approaches, such as Autoformer (Wu et al., ) and FEDFormer (Zhou et al., ), have proposed input sequence decomposition. Still, they are trained with $L_{p}$ norm-based loss function, which could not properly model the temporal dynamics, as shown in Fig. 1, (top). On the other hand, Le Guen & Thome () attempt to model sudden changes in a timely and accurate manner with dynamic time warping (DTW), and Bica et al. () adopt domain adversarial training to learn balanced representations, which is a treatment invariant representations over time. Le Guen & Thome (); Bica et al. () try to capture the shape but still have some limitations, as depicted in Fig. 1 (middle), implying the need for further investigation of the shape.

Figure: Figure 1: Ground-truth and forecasting results of Informer model with three training metrics, as shown in the blue box: (top) MSE, (middle) DTW-based, and (bottom) TILDE-Q loss function. (top, middle) The blue boxes indicates the original intention of loss function (desired) and misbehaviors.
Refer to caption: x1.png

The identification of shape, denoting the pattern in time-series data within a given time interval, plays an important role in addressing aforementioned limitation in time-series forecasting problem. It can provide valuable information, such as rise, drop, trough, peak, and plateau. We refer to the prediction as informative when it can appropriately model the shape. In real-world applications, including economics, informative prediction is invaluable for decision-making. To achieve such informative forecasting, a model should account for shape instead of solely aiming to forecast accurate value for each time step. However, existing methods inadequately consider the shape (Wu et al., ; Zhou et al., ; Bica et al., ; Le Guen & Thome, ). Moreover, deep learning model tends to opt for an easy learning path (Karras et al., ), yielding inaccurate and uninformative forecasting results disregarding the characteristics of time-series data. Fig. 1 illustrates three real forecasting results obtained with Informer (Zhou et al., ) and different training metrics. When the mean squared error (MSE) is used as an objective, the model aims to reduce the gap between prediction and ground truth for each time-step. This “point-wise” distance-based optimization has less ability to model shape, resulting in generating uninformative predictions regardless of temporal dynamics (Fig. 1 (top)); the model rarely provides information about the time-series. In contrast, if both gap and shape of the prediction and ground truth are taken into account, the model can achieve high accuracy with proper temporal dynamics, as shown in Fig. 1 (bottom). Consequently, time-series forecasting requires a loss function that consider both point-wise distance (i.e., traditional goal) and shape.

In this work, we aim to design a novel objective function that guides models in improving forecasting performance by learning shapes in time-series data. To design a shape-aware loss function, we review existing literature (Esling & Agon, ; Bakshi & Stephanopoulos, ; Keogh, ) and explore the concepts of shapes and distortions that impede appropriate measurement of similarity between two time-series data in terms of shapes (Sec. 3.1, Sec. 3.2, and Sec. 3.3). Based on our investigation, we propose the necessary conditions for constructing an objective function for shape-aware time-series forecasting (Sec. 4.1). Subsequently, we present a novel loss function, TILDE-Q (Transformation Invariant Loss function with Distance EQualibrium), which enables shape-aware representation learning by utilizing three loss terms that are invariant to distortions (Sec. 4.2). For evaluation, we conduct extensive experiments with state-of-the-art deep learning models with TILDE-Q. The experimental results indicate that TILDE-Q is model-agnostic and outperforms MSE and DILATE in MSE and shape-related metrics.

#### Contributions

In summary, our study makes the following contributions. (1) We delve into the concept of shape awareness and distortion invariances in the context of time-series forecasting. By thoroughly investigating these distortions, we enhance our understanding of their impact on time-series forecasting problems. (2) We propose and implement TILDE-Q, which has invariances to three distortions and achieves shape-awareness, empowering informative forecasting in a timely manner. (3) We empirically demonstrate that the proposed TILDE-Q allows models to have higher accuracy compared to the models trained with other existing metrics, such as MSE and DILATE.

## 2 Related Work

### 2.1 Time-Series Forecasting

Many time-series forecasting methods are available, ranging from traditional models, such as ARIMA model (Box et al., ) and hidden Markov model (Pesaran et al., ), to recent deep learning models. In this section, we briefly describe the recent deep learning models for time-series forecasting. Motivated by the huge success of recurrent neural networks (RNNs) (Clevert et al., ; Li et al., ; Yu et al., ), many novel deep learning architectures have been developed for improving forecasting performance. To effectively capture long-term dependency, which is a limitation of RNNs, Stoller et al. () have proposed convolutional neural networks (CNNs). However, it is required to stack lots of the same CNNs to capture long-term dependency (Zhou et al., ). Attention-based models, including Transformer (Vaswani et al., ) and Informer (Zhou et al., ), have been another popular research direction in time-series forecasting. Although these models effectively capture temporal dependencies, they incur high computational costs and often struggle to obtain appropriate temporal information (Wu et al., ). To cope with the problem, Wu et al. (); Zhou et al. () have adopted the input decomposition method, which helps models better encode appropriate information. Other state-of-the-art models adopt neural memory networks (Kaiser et al., ; Sukhbaatar et al., ; Madotto et al., ; Lee et al., ), which refer to historical data stored in the memory to generate meaningful representation.

### 2.2 Training Metrics

Conventionally, mean squared error (MSE), $L_{p}$ norm and its variants are mainstream metrics used to optimize forecasting models. However, they are not optimal for training forecasting models (Esling & Agon, ) because the time-series is temporally continuous. Moreover, the $L_{p}$ norm provides less information about temporal correlation among time-series data. To better model temporal dynamics in time-series data, researchers have used differentiable, approximated dynamic time warping (DTW) as an alternative metric of MSE (Cuturi & Blondel, ; Abid & Zou, ; Mensch & Blondel, ). However, using DTW as a loss function results in temporal localization of changes being ignored. Recently, Le Guen & Thome () have suggested DILATE, a training metric to catch sudden changes of nonstationary signals in a timely manner with smooth approximation of DTW and penalized temporal distortion index (TDI). To guarantee DILATE’s operation in a timely manner, penalized TDI issues a harsh penalty when predictions showed high temporal distortion. However, the TDI relies on the DTW path, and DTW often showed misalignment because of noise and scale sensitivity. Thus, DILATE often loses its advantage with complex data, showing disadvantages at the training. In this paper, we discuss distortions and transformation invariances and design a new loss function that enables models to learn shapes in the data and produce noise-robust forecasting results.

## 3 Preliminary

In this section, we investigate common distortions focusing on the goal of time-series forecasting (i.e., modeling temporal dynamics and accurate forecasting). To clarify the concepts of time-series forecasting and related terms, we first define the notations and terms used (Sec. 3.1). We then discuss common distortions in time-series from the transformation perspective that need to be considered for building a shape-aware loss function (Sec. 3.2) and describe how other loss functions (e.g., dynamic time warping (DTW) and temporal distortion index (TDI)) handle shapes during learning (Sec. 3.3). We will discuss the conditions for effective time-series forecasting in the next session (Sec. 4.1).

Figure: Figure 2: Example of the six distortions on the amplitude axis (top) and temporal axis (bottom).
Refer to caption: x2.png

### 3.1 Notations and Definitions

Let $X_{t}$ denote a data point at a time step $t$. We define a time-series forecasting problem as follows:

###### Definition 3.1 .

Given $T$-length historical time-series $\mathbf{X}=[X_{t-T+1},\dots,X_{t}],X_{i}\in\mathbb{R}^{F}$ at time $i$ and a corresponding $T^{\prime}$-length future time-series $\mathbf{Y}=[Y_{t+1},\dots,Y_{t+T^{\prime}}],Y_{i}\in\mathbb{R}^{C}$, time-series forecasting aims to learn the mapping function $f:\mathbb{R}^{T\times F}\rightarrow\mathbb{R}^{T^{\prime}\times C}$.

To distinguish between the label (i.e., ground truth) and prediction time-series data, we note the label data as $\mathbf{Y}$ and prediction data as $\mathbf{\hat{Y}}$. Next, we set up two goals for time-series forecasting, which require not only precise but also informative forecasting (Wu et al., ; Zhou et al., ; Le Guen & Thome, ) as follows:

- •
The mapping function $f$ should be learnt to point-wisely reduce distance between $\mathbf{\hat{Y}}$ and $\mathbf{Y}$;
- •
The output $\mathbf{\hat{Y}}$ should have similar temporal dynamics with $\mathbf{Y}$.

Temporal dynamics are informative patterns in a time-series, such as rise, drop, peak, and plateau. The optimization for point-wise distance reduction is a conventional method used in the deep learning domain, which can be obtained using the MAE or MSE. However, in a real-world problem, such as traffic speed or stock market prediction, accurate forecasting of temporal dynamics is required. Esling & Agon () also emphasized the measurement of temporal dynamics, as “…allowing the recognition of perceptually similar objects even though they are not mathematically identical.”
In this paper, we define temporal dynamics as follows:

###### Definition 3.2 .

Temporal dynamics (or shapes) are informative periodic and nonperiodic patterns in time-series data.

In this work, we aim to design a shape-aware loss function that satisfies both goals.
To this end, we first discuss distortions that two time-series with similar shapes can have.

###### Definition 3.3 .

Given two time-series $\mathbf{F}$ and $\mathbf{G}$ having similar shapes but not being mathematically identical, let $\mathcal{H}$ is transformation that satisfies $\mathbf{F}=\mathcal{H}(\mathbf{G})$. Then, the time-series $\mathbf{F}$ and $\mathbf{G}$ are considered to have a distortion, which can be represented by the transformation $\mathcal{H}$.

A distortion can generally be classified as a temporal distortion (i.e., warping) or an amplitude distortion (i.e., scaling) depending on its dimension–time and amplitude.
Existing distortions in the data lead to misbehavior of the model, as they distort the measurements to be inaccurate.
For example, if we have two time-series $\mathbf{F}$ and $\mathbf{G}=\mathbf{F}+k$, which have similar shapes but different means, $\mathbf{G}$ could represent many temporal dynamics of $\mathbf{F}$.
However, measurements often evaluate $\mathbf{F}$ and $\mathbf{G}$ as completely different signals and cause misguidance of the model in training (e.g., measuring the distance of $\mathbf{F}$ and $\mathbf{G}$ with MSE).
As such, it is important to have measurements that consider a similar shape invariant to distortion.
We define a measurement for distortion as:

###### Definition 3.4 .

Let transformation $\mathcal{H}$ represent a distortion $H$. Then, we call measurement $\mathcal{D}$ invariant to $\mathcal{H}$ if $\exists\delta>0:\mathcal{D}(\mathbf{T},\mathcal{H}(\mathbf{T}))<\delta$ for any time-series $\mathbf{T}$.

###### Definition 3.1 .

###### Definition 3.2 .

###### Definition 3.3 .

###### Definition 3.4 .

### 3.2 Time-Series Distortions in Transformation Perspectives

Distortion, a gap between two similar time-series, affects shape capturing in time-series data. Thus, it is important to investigate different distortions and their impacts on representation learning aspects.
There are six common time-series distortions that models encounter during learning (Esling & Agon, ; Batista et al., ; Berkhin, ; Warren Liao, ; Kerr et al., )–Amplitude Shifting, Phase Shifting, Uniform Amplification, Uniform Time Scaling, Dynamic Amplification, and Dynamic Time Scaling.
Next, we explain each common time-series distortion in terms of transformation with an $n$-length time-series $\mathbf{F}(t)=[f(t_{1}),f(t_{2}),\dots,f(t_{n})]$, where t = $[t_{1},t_{2},\dots,t_{n}]$. Fig. 2 presents example distortions, categorized by amplitude and time dimensions.

- •
Amplitude Shifting describes how much a time-series shifts against another time-series. This can be described with two time-series and the degree of shifting $k$: $\mathbf{G}(t)=\mathbf{F}(t)+k=[f(t_{1})+k,\dots,f(t_{n})+k]$, where $k\in\mathbb{R}$ is constant.
- •
Phase Shifting is the same type of transformation (i.e., translation) as amplitude shifting, but it occurs along the temporal dimension. This distortion can be represented by two time-series functions with the degree of shift $k$: $\mathbf{G}(t)=\textbf{F}(t+k)=[f(t_{1}+k),\dots,f(t_{n}+k)]$, where $k\in\mathbb{R}$ is constant.
Cross-correlation (Paparrizos & Gravano, ; Vlachos et al., ) is the most popular measure method that is invariant to this distortion.
- •
Uniform Amplification is a transformation that changes the amplitude by multiplication of $k\in\mathbb{R}$. This distortion can be described with two functions and a multiplication factor $k$: $\mathbf{G}(t)=k\cdot\mathbf{F}(t)=[k\cdot f(t_{1}),\dots,k\cdot f(t_{n})]$.
- •
Uniform Time Scaling refers to a uniformly shortened or lengthened $\mathbf{F}(t)$ on the temporal axis.
This distortion can be represented as $\mathbf{G}(t)=[g(t_{1}),\dots,g(t_{m})]$, where $g(t_{i})=f(t_{\lceil k\cdot i\rceil})$ and $k\in\mathbb{R}^{+}$. Although Keogh et al. () have proposed uniform time warping methods to handle this distortion, it still remains a challenging distortion type to measure because of the difficulty in identifying the scaling factor $k$ without testing all possible cases (Keogh, ).
- •
Dynamic Amplification is any distortion that occurs through non-zero multiplication along the amplitude dimension. This distortion can be described as follows: $\mathbf{G}(t)=\mathbf{H}(t)\cdot\mathbf{F}(t)=[h(t_{1})\cdot f(t_{1}),\dots,h(
t_{n})\cdot f(t_{n})]$ with function $h(t)$, such that $\forall_{t\in\mathbb{T}},h(t)\neq 0$. Local amplification is representative of such distortions, which still remains challenging to solve.
- •
Dynamic Time Scaling refers to any transformation that dynamically lengthens or shortens signals along the temporal dimension, including local time scaling (Batista et al., ) and occlusion (Batista et al., ; Vlachos et al., ).
It can be represented as follows: $\mathbf{G}(t)=\mathbf{F}(h(t))=[f(h(t_{1})),\dots,f(h(t_{n}))]$, where $h(t)$ is a positive, strictly increasing function. DTW (Bellman & Kalaba, ; Berndt & Clifford, ; Keogh & Ratanamahatana, ) is the most popular technique invariant to this distortion. Das et al. () have also introduced the longest common subsequence (LCSS) algorithm to tackle occlusion, noise, and outliers in this distortion.

Shape-aware clustering (Bellman & Kalaba, ; Batista et al., ; Paparrizos & Gravano, ; Berkhin, ; Warren Liao, ; Kerr et al., ) and classification (Xi et al., ; Batista et al., ; Srisai & Ratanamahatana, ) tasks that consider shapes have been extensively studied.
However, only a few studies exist for time-series forecasting tasks, including Le Guen & Thome () that utilize DTW and TDI for modeling temporal dynamics.
Next, we describe the MSE and DILATE, proposed by Le Guen & Thome (), and discuss their invariance to distortions.

### 3.3 Distortion Handling in Current Time-Series Forecasting Objectives

Many measurement metrics have been used in the time-series forecasting domain, and those based on the $L_{p}$ distance, including Euclidean distance, are widely used to handle time-series data.
However, such metrics are not invariant to the aforementioned distortions (Ding et al., ; Le Guen & Thome, ) because of their point-wise mapping. In particular, since $L_{p}$ distance compares the values per time step, it cannot handle temporal distortions appropriately and is vulnerable to data scaling. Le Guen & Thome () have proposed a loss function called DILATE to overcome the inadequate characteristic in the $L_{p}$ distance metric by recognizing temporal dynamics with DTW and TDI.
In terms of transformation, DILATE handles dynamic time scaling, especially local time scaling with DTW, and phase shifting with penalized TDI, defined as follows:

$$ $\mathcal{L}_{{dilate}}(\hat{y}_{i},y_{i}):=-\gamma\log\Big{(}\sum_{\mathbf{A} \in\mathcal{A}_{k,k}}e^{-\frac{\langle\mathbf{A},\alpha\Delta(\hat{y}_{i},y_{i })+(1-\alpha)\mathbf{\Omega}\rangle}{\gamma}}\Big{)},$ $$

where $\mathbf{A}$, $\Delta(\hat{y}_{i},y_{i})$, $\mathbf{\Omega}$ are the warping path, cost matrix, and penalization matrix, respectively.

While DILATE performs better than existing methods, it has a limitation from the perspective of invariance. DILATE highly depends on DTW, which allows for the dynamic alignment of the time-series for a predefined window. In such windows, DTW can align the signal regardless of its information (e.g., periodicity). As a result, the model creates misbehavior that can cheat DTW within the window, as shown in Fig. 1 middle. DTW’s scale and noise sensitivity are also problematic. DTW computes the Euclidean distance of two time-series after its temporal alignment in dynamic programming, and the alignment relies on the distance function. Consequently, the dynamic alignment of DTW can be properly achieved only when the two time-series have the same range (Esling & Agon, ; Bellman & Kalaba, ). This means that it hardly achieves invariance to amplitude distortion without appropriate pre-processing. Gong & Chen () also show that DTW poorly matches the prediction and target (i.e., ground truth) time-series with amplitude shifting. Even when the target time-series is aligned with normalization, the appropriate alignment of the prediction and target time-series cannot be guaranteed because of DTW’s high sensitivity to noise. As a result, DILATE can generate poor alignment results, which can cause wrong TDI optimization, producing incorrect results and instability during the optimization steps. To design an effective shape-aware loss function, we must understand the measures and in which cases they have transformation invariances. In the next section, we interpret transformations from a time-series forecasting viewpoint and discuss the types of transformations that should be considered in objective function design.

## 4 Methods

In this section, we discuss and propose the design rationale for the shape-aware loss function (Sec. 4.1). Based on the design rationale, we implement a novel loss function, TILDE-Q (a Transformation Invariant Loss function with Distance EQuilibrium), which allows models to perform shape-aware time-series forecasting based on three distortion invariances.

### 4.1 Transformation Invariances in Time-Series Forecasting

In the time-series domain, data often have various distortions; thus, measurements need to satisfy numerous transformation invariances for meaningfully modeling temporal dynamics. As discussed in Sec. 3.1, we set the goals of time-series forecasting as (1) point-wisely reducing the gap between the prediction and target time-series and (2) preserving the temporal dynamics of the target time-series. To satisfy both of them, we have to consider (1) a method that does not negatively impact on the traditional goal of accurate time-series forecasting and (2) distortions that play a crucial role in capturing the temporal dynamics of the target time-series. In this section, we review all six distortions based on whether their corresponding invariance is feasible to be a loss function for time-series forecasting, discuss the loss function’s benefits and trade-offs, and identify appropriate distortions to be considered in time-series forecasting.

#### Amplitude Shifting

In a wide range of situations, it is beneficial to capture the trends of time-series sequences despite shifts in amplitude.
Thus, being invariant to amplitude shifting in a loss function is highly advantageous in time-series forecasting: (1) shape awareness invariant to amplitude shifting, (2) accurate deviation of values in modeling, and (3) effective on-time prediction of the peak or sudden changes. To guarantee an amplitude shifting invariance in the optimization stage, the loss function should induce an equal gap $k$ between the prediction and ground truth data in each step.
Specifically, the loss function considering amplitude shifting should satisfy:

$$ $\mathcal{L}(\mathbf{Y},\mathbf{\hat{Y}})=0\Leftrightarrow\forall_{i\in[1,\dots ,n]},d(y_{i},\hat{y}_{i})=k,$ (1) $$

where $k\in\mathbb{R}$ is an arbitrary and equal gap, and $d(y_{i},\hat{y}_{i})$ is a signed distance with a boundary $y_{i}>\hat{y}_{i}$. By allowing tolerance between the prediction and target time-series, models can follow trends in time-series instead of predicting exact values point-wisely. In short, unlike existing loss functions, which handle only point-wise distance (e.g., DTW), we should deal with both point-wise distance and its relational distance values to guarantee amplitude shifting.

#### Phase Shifting

There are some forecasting tasks whose main objectives concern accurate forecasting of peaks and periodicity in time-series (e.g., heartbeat data and stock price data). For such tasks, phase shifting invariance is an optimal solution for (1) modeling periodicity, regardless of the translation on the temporal axis, and (2) having precise statistics with shapes, such as peak and plateau values. To be invariant to phase shifting, the loss function should satisfy

$$ $\mathcal{L}(\mathbf{Y},\mathbf{\hat{Y}})=0\Leftrightarrow\mathbf{Y},\mathbf{ \hat{Y}}\textrm{ have the same dominant frequency}.$ (2) $$

Note that Eq. 2 allows a similar shape as the target time-series in forecasting, not exactly the same shape (e.g., $\sin(x)$ and $2\sin(x+x_{0})$ with the same dominant frequency).

#### Uniform Amplification

This proposition can be utilized in the case of sparse data that contains a significant number of zeros.
By adopting uniform amplification invariance, models are able to focus on non-zero sequences, whereas this proposition allows models to receive less penalty in zero sequences. Since it guarantees shape awareness with a multiplication factor in a timely manner, as shown in Fig. 2, invariance for uniform amplification fits well. To have a model trained with uniform amplification invariance, the loss function should satisfy the following proposition:

$$ $\mathcal{L}(\mathbf{Y},\mathbf{\hat{Y}})=0\Leftrightarrow\forall_{i\in[1,\dots ,n]},\frac{y_{i}}{\hat{y}_{i}}=k(\hat{y}_{i}\neq 0).$ (3) $$

#### Uniform Time Scaling, Dynamic Amplification, and Dynamic Time Scaling

After careful consideration, we conclude that uniform time scaling, dynamic amplification, and dynamic time scaling are incompatible for optimization. the reasons are described below.

To achieve invariance for uniform time scaling, the loss function should satisfy below:

$$ $\mathcal{L}(\mathbf{Y},\mathbf{\hat{Y}})=0\Leftrightarrow\exists c\in\mathbb{Z }^{+}\text{, where }\\ \{c|y_{i}=\hat{y}_{ci}\}\cup\{c|y_{ci}=\hat{y}_{i}\}\forall i\in[0,1,\dots,T^{ \prime}].$ (4) $$

This proposition will negatively influence the original temporal dynamics, considering that it creates the tolerance for mispredicting periodicity (e.g., daily periodic signals) and cannot identify events (e.g., abrupt changing values) in a timely manner. In summary, it hinders models from capturing shapes and corrupts periodic information.

For both dynamic amplification and time scaling, the loss functions are zero for all pairs when there is no limit for tolerance. Formally, the proposition for dynamic amplification invariance is as follows:

$$ $\displaystyle\mathcal{L}(\mathbf{Y},\mathbf{\hat{Y}})=0\Leftrightarrow\forall c _{i}\in\mathbb{R}:y_{i}=c_{i}\hat{y}_{i},$ $$

If a loss function satisfies this proposition without bound for $c_{i}$, it is always zero because there always exists $c_{i}=y_{i}/\hat{y}_{i}$, except $\hat{y}_{i}=0$. Therefore, it is not able to provide any information because all random values could be an optimal solution. The same situation happens for the dynamic time scaling if we do not limit the window. Consequently, all three objectives–uniform time scaling, dynamic amplification, and dynamic time scaling are unsuitable to be objectives in time-series forecasting.

### 4.2 TILDE-Q: Transformation Invariant Loss function with Distance EQuilibrium

To build a transformation invariant loss function, we need to design a loss function that satisfies the proposition for amplitude shifting (Eq. 1), phase shifting (Eq. 2), and uniform amplification shifting invariance (Eq. 3), as discussed in Sec. 4.1. Furthermore, the loss function should guarantee a small $L_{p}$ norm between prediction and label, which is the traditional goal of forecasting. Both conditions are hard to simultaneously satisfy by existing loss functions, such as the MSE or DILATE. To handle all three distortions while considering traditional goal, we build three objective functions (a.shift, phase, and amp losses) that can achieve one or more invariance by using softmax, Fourier coefficient, and autocorrelation to design a loss function.

#### Amplitude Shifting Invariance with Softmax (Amplitude Shifting)

To strengthen amplitude shifting invariance, we design a loss function that satisfies Eq. 1.
This means that $d(y_{i},\hat{y}_{i})$ must have the same value for all $i$.
To satisfy this condition, we utilize the softmax function:

$$ $\mathcal{L}_{a.shift}(\mathbf{Y},\mathbf{\hat{Y}})=T^{\prime}\sum_{i=1}^{T^{ \prime}}|\frac{1}{T^{\prime}}-\textrm{Softmax}(d(y_{i},\hat{y}_{i}))|,$ (5) $$

where $T^{\prime}$, Softmax, and $d(\cdot,\cdot)$ are the sequence length, softmax function, and signed distance function, respectively. Because softmax produces the proportion of each value, it can obtain the optimal solution only when it satisfies Eq. 1. Since Softmax outputs the relative values, it could handle any gap $k$.

#### Invariances with Fourier Coefficients (Phase Shifting)

As discussed in Sec. 4.1, a potential method that can be used to obtain phase shifting invariance is the use of Fourier coefficients. According to the literature (NG & GOLDBERGER, ), the original time-series can be reconstructed with a few dominant frequencies. Thus, we utilize the gap between dominant Fourier coefficients of ground truth and prediction as our objective function for achieving phase shifting invariance. For the other frequencies, we use the norm of the prediction sequence to reduce the value of the Fourier coefficient. Consequently, this loss function keeps the temporal dynamics of the original time series (i.e., dominant frequencies) and enables noise robustness by reducing white noises in non-dominant frequencies. We achieve phase shifting invariance by optimizing the following loss function:

$$ $\mathcal{L}_{phase}(\mathbf{Y},\mathbf{\hat{Y}})=\begin{cases}||\mathcal{F}( \mathbf{Y})-\mathcal{F}(\mathbf{\hat{Y}})||_{p},&\text{dominant freq.}\\ ||\mathcal{F}(\mathbf{\hat{Y}})||_{p},&\text{otherwise}\end{cases}$ (6) $$

where $||\cdot||_{p}$ is the $L_{p}$ norm. To obtain the dominant frequency terms, we calculate the norm of the Fourier coefficient for each frequency and filter them with the squared root of sequence length, $\sqrt{T^{\prime}}$. We also guarantee the minimum number of dominant frequencies as $\sqrt{T^{\prime}}$. This loss function obtains uniform amplification invariance through the application of a normalization technique to Fourier coefficients. For example, $\sin x$ and $c\cdot\sin x$ have the same Fourier coefficients if appropriately normalized. In summary, from Eq. 6, we can obtain (1) invariance for phase shifting, (2) invariance for uniform amplification, and (3) robustness to noise.

#### Invariances with Autocorrelation (Uniform Amplification)

Although Fourier coefficients can be considered a reasonable solution to determine the periodicity of the target time-series, they are not completely invariant to phase shifting for three reasons: (1) the data statistics (e.g., mean and variance) keep changing, (2) such changing statistics also cause changes in Fourier coefficients even at the same frequency, and (3) objectives only with a norm of Fourier coefficient cannot fully represent the original time-series.
Thus, we introduce an objective based on normalized cross-correlation, which satisfies Eq. 2 for a periodic signal:

$$ $\mathcal{L}_{amp}(\mathbf{Y},\mathbf{\hat{Y}})=||R(\mathbf{Y},\mathbf{Y})-R( \mathbf{Y},\mathbf{\hat{Y}})||_{p},$ (7) $$

where $R(\cdot,\cdot)$ is a normalized cross-correlation function. This loss function helps predicted sequences mimic label sequences by calculating the difference between the autocorrelation of the label sequences and the cross-correlation between the label and predicted sequences. Therefore, the label and prediction have similar temporal dynamics, regardless of phase shifting or uniform amplification.

In summary, we introduce TILDE-Q, combining Eq. 5, Eq. 6, and Eq. 7 as follows:

$$ $\mathcal{L}(\mathbf{Y},\mathbf{\hat{Y}})=\alpha\mathcal{L}_{a.shift}(\mathbf{Y },\mathbf{\hat{Y}})\\ +(1-\alpha)\mathcal{L}_{phase}(\mathbf{Y},\mathbf{\hat{Y}})+\gamma\mathcal{L}_ {amp}(\mathbf{Y},\mathbf{\hat{Y}}),$ (8) $$

where $\alpha\in[0,1]$ and $\gamma$ are hyperparameters.

**Table 1: Experimental results on six real-world datasets in multivariate time-series forecasting setting with prediction lengths $T^{\prime}=\{24,36,48,60\}$ for ILI and $T^{\prime}=\{96,192,336,720\}$ for others. The results are averaged from all prediction lengths. We set input sequence length $T=96$ except ILI dataset. For ILI dataset, we set input sequence length $T=36$. Avg. Improved means the average improvements of TILDE-Q over the model trained with MSE. We have colored the best training metric in red.**
| Model | iTransformer | PatchTST | Crossformer | TimesNet | DLinear | FEDformer | NSformer | Autoformer |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |
| ETT* | 0.408 | 0.412 | 0.403 | 0.405 | 0.387 | 0.400 | 0.387 | 0.397 | 0.535 | 0.521 | 0.427 | 0.439 | 0.415 | 0.419 | 0.401 | 0.411 | 0.404 | 0.408 | 0.401 | 0.400 | 0.461 | 0.459 | 0.438 | 0.451 | 0.533 | 0.470 | 0.512 | 0.465 | 0.586 | 0.516 | 0.541 | 0.492 |
| Electricity | 0.179 | 0.269 | 0.175 | 0.266 | 0.204 | 0.291 | 0.203 | 0.282 | 0.188 | 0.284 | 0.181 | 0.278 | 0.195 | 0.295 | 0.192 | 0.292 | 0.225 | 0.319 | 0.212 | 0.294 | 0.227 | 0.337 | 0.224 | 0.333 | 0.196 | 0.295 | 0.194 | 0.293 | 0.250 | 0.351 | 0.232 | 0.338 |
| Traffic | 0.428 | 0.282 | 0.426 | 0.281 | 0.555 | 0.362 | 0.514 | 0.335 | 0.550 | 0.304 | 0.540 | 0.296 | 0.620 | 0.336 | 0.600 | 0.328 | 0.672 | 0.418 | 0.667 | 0.399 | 0.610 | 0.378 | 0.606 | 0.376 | 0.644 | 0.355 | 0.637 | 0.351 | 0.637 | 0.395 | 0.619 | 0.386 |
| Weather | 0.260 | 0.281 | 0.257 | 0.274 | 0.262 | 0.281 | 0.258 | 0.280 | 0.259 | 0.315 | 0.248 | 0.301 | 0.259 | 0.287 | 0.256 | 0.282 | 0.268 | 0.317 | 0.266 | 0.306 | 0.309 | 0.360 | 0.302 | 0.342 | 0.312 | 0.323 | 0.310 | 0.317 | 0.366 | 0.396 | 0.346 | 0.376 |
| Exchange | 0.389 | 0.421 | 0.379 | 0.415 | 0.365 | 0.410 | 0.364 | 0.403 | 0.943 | 0.711 | 0.833 | 0.660 | 0.403 | 0.436 | 0.407 | 0.438 | 0.323 | 0.392 | 0.301 | 0.379 | 0.554 | 0.515 | 0.524 | 0.499 | 0.546 | 0.488 | 0.474 | 0.457 | 0.532 | 0.518 | 0.494 | 0.493 |
| ILI | 2.333 | 0.984 | 2.220 | 0.949 | 2.253 | 0.933 | 2.143 | 0.903 | 3.724 | 1.281 | 3.297 | 1.202 | 2.346 | 0.963 | 2.083 | 0.899 | 2.815 | 1.150 | 2.475 | 1.071 | 3.307 | 1.276 | 3.113 | 1.225 | 2.613 | 1.024 | 2.032 | 0.921 | 3.327 | 1.261 | 3.199 | 1.241 |
| Avg. Improved | - | - | 2.08% | 1.77% | - | - | 2.43% | 2.76% | - | - | 8.85% | 6.38% | - | - | 3.25% | 2.21% | - | - | 4.48% | 4.67% | - | - | 3.42% | 2.59% | - | - | 7.02% | 3.5% | - | - | 5.69% | 3.68% |

**Table 2: Experimental results of short-term time-series forecasting on the three datasets with sequence-to-sequence GRU model. We have colored the best training metric in red and the second best underlined.**
| Methods | GRU + MSE | GRU + DILATE | GRU + TILDE-Q |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Eval | MSE | DTW | TDI | LCSS | MSE | DTW | TDI | LCSS | MSE | DTW | TDI | LCSS |
| Synthetic | 0.0107 | 3.5080 | 1.0392 | 0.3523 | 0.0130 | 3.4005 | 1.1242 | 0.3825 | 0.0119 | 3.2873 | 1.1564 | 0.3811 |
| ECG5000 | 0.2152 | 1.9718 | 0.8442 | 0.7743 | 0.8270 | 3.9579 | 2.0281 | 0.4356 | 0.2141 | 1.9575 | 0.7714 | 0.7773 |
| Traffic | 0.0070 | 1.4628 | 0.2343 | 0.7209 | 0.0095 | 1.6929 | 0.2814 | 0.6806 | 0.0072 | 1.4600 | 0.2276 | 0.7220 |

## 5 Experiments

In this section, we present the results of our comprehensive experiments, demonstrating the effectiveness of TILDE-Q and the importance of transformation invariance.

#### Experimental Setup

We conduct the experiments with eight state-of-the-art models–Autoformer (Wu et al., ), FEDformer (Zhou et al., ), NSformer (Liu et al., ), DLinear (Zeng et al., ), TimesNet (Wu et al., ), Crossformer (Zhang & Yan, ), PatchTST (Nie et al., ), and iTransformer (Liu et al., ) and one basic GRU model. For model training, we use seven real-world datasets–ECL, ETT, Electricity, Traffic, Weather, Exchange, and Weather–and one synthetic dataset, Synthetic. We repeat each experiment with a model and dataset 10 times in combination with two different objective functions. Appendix B provides detailed explanations of the datasets, hyperparameter settings, model, and source code. We also provide additional qualitative results in the Appendix.

#### Evaluation Metrics

In this experiment, we evaluate TILDE-Q with three evaluation metrics: mean absolute error (MAE), mean squared error (MSE), dynamic time warping (DTW), and its corresponding temporal distortion index (TDI), all of which are referred from Le Guen & Thome (). As DTW is sensitive to noise and generates incorrect paths when one of the time-series data is noisy (as discussed in Sec. 3.3), we additionally use the longest common subsequence (LCSS) for comparison, which is more robust to outliers and noise (Esling & Agon, ). The longer the matched subsequences, the higher the LCSS score will be achieved in modeling the shapes. For state-of-the-art models, we report the MAE and MSE. For detailed results, including forecasting results for different prediction lengths, please refer to Appendix C.

#### Experimental Results and Analysis

Table 2 shows the results of the short-term forecasting performance of the GRU model optimized with the MSE, DILATE, and TILDE-Q metrics. With the Synthetic dataset, each metric used shows its own benefits. This result indicates that loss functions with shape similarity or MSE have their specialty for shape and exact value, respectively. It also means a better MSE does not guarantee a better solution for temporal dynamics. Moreover, since the model is evaluated with real-world datasets, it is revealed that TILDE-Q outperforms other objective functions in most evaluation metrics. These results indicate that our approach to learning shapes in time-series data achieves better results than existing methods for forecasting. DILATE does not show impressive performance with ECG5000 due to its high sensitivity to noise, as discussed in Sec. 3.3. Table 1 summarizes the comprehensive experimental results obtained with the eight state-of-the-art models. Compared to MSE baseline, TILDE-Q makes particularly better prediction among all the models, even for the DLinear (Zeng et al., ), which consists of two one-layer linear layers. For the Autoformer and NSformer, TILDE-Q makes significant improvement around 5%, making the models recognize additional shape-related information beyond the frequency-based terms. For the recent models (i.e., iTransformer and PatchTST) that interpret input signals with embedding or patching, TILDE-Q is less beneficial than the other models. Crossformer makes the most impressive results with 8.85% performance improvement. This improvement is caused by Crossformer’s design, which particularly focuses on resolving inter-domain dependency among multivariate time series. TILDE-Q is able for Crossformer to recognize the cross-time dependency (i.e., shape and temporal changes) better, indicating the importance of both temporal and inter-domain behaviors. This insight reveals possible future research investigating loss function related to causality and inter-domain dependency.

## 6 Conclusion and Future Work

We propose TILDE-Q that allows shape-aware time-series forecasting in a timely manner. To design TILDE-Q, we review existing transformations in time-series data and discuss the conditions that ensure transformation invariance during optimization tasks. The designed TILDE-Q is invariant to amplitude shifting, phase shifting, and uniform amplification, ensuring a model better captures shapes in time-series data. To prove the effectiveness of TILDE-Q, we conduct comprehensive experiments with state-of-the-art models and real-world datasets. The results indicate that the model trained with TILDE-Q generates more timely, robust, accurate, and shape-aware forecasting in both short-term and long-term forecasting tasks. We conjecture that this work can facilitate future research on transformation invariances and shape-aware forecasting.

## 7 Impact Statements

This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

## Appendix A Implementation Details of TILDE-Q Sublosses

As we discussed in Sec. 4.2, TILDE-Q consists of three sublosses: $L_{a.shift}$, $L_{phase}$, and $L_{amp}$. Our design rationale for selecting these sublosses is described in Sec. 4.1. In this section, we describe the detailed connection between the sublosses and the design rationale (Eqs. 1, 2, and 3).

#### Amplitude Shifting

Given two sets of points with the same length $T$, $X,X^{\prime}\in\mathbb{R}^{T}$, let us define their distance using the signed distance function $g:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}$. Then, for each point $x,x^{\prime}$ in set $X,X^{\prime}$, we can define a point-wise distance set $D$ with $g$ as below:

$$ $D=[g(x_{1},x^{\prime}_{1}),\dots,g(x_{T},x^{\prime}_{T})]=[d_{1},\dots,d_{T}].$ $$

When we design $L_{a.shift}$, we have one main question: given an arbitrary $X,X^{\prime},$ and $g$, how do we build a loss function that is invariant to any arbitrary gap $k$. In this work, we have reformulated this task from ensuring equal gaps between all points into making uniform distribution of the gaps (i.e., $\sum_{i}{p_{d_{i}}\log p_{d_{i}}}$ on the interval $[1,T]$). Please note that we convert gaps into relative values since an absolute domain requires information for $k$ for each data sample. Without loss of generality, we can say that this problem is equivalent to the problem of entropy maximization. Let us suppose that we convert the distance set $D$ into the probability distribution by $Softmax$ function, $p_{d_{i}}=Softmax(d_{i})$. In this case, we can say that our optimization problem is maximize the entropy as below:

$$ $\displaystyle\text{maximize }L$ $\displaystyle=\sum_{i=1}^{T}{p_{d_{i}}\log p_{d_{i}}},$ $$

which is well-known to have its global optima with $\forall_{i\in[1,T]}p_{d_{i}}=\frac{1}{T}$. Therefore, we formulate $L_{a.shift}$ as Eq. 5, which satisfies Eq. 1. Please note that the noise robustness of $L_{a.shift}$ relies on that of the signed distance function, $g$. Since $L_{a.shift}$ requires computation of $g$ and $Softmax$, it takes $O(n)$ time for its computation.

#### Phase Shifting

To discuss phase shifting and periodicity of a time-series, Fourier transform is an inevitable factor. However, in the real-world dataset, a few problems arise: 1) we are unaware of the frequencies and periodicity of the data itself, and 2) a direct use of Fourier coefficients may be biased by noise. During the design phase, we aim to solve these problems with $L_{phase}$. To extract the main flow of time-series data (i.e., the dominant periodicity or frequencies), we first define the dominant frequencies based on their statistical significance. Let $X\in\mathbb{R}^{T}$ as an input signal. In the machine learning domain, researchers commonly suppose the input signal follows normal distribution $X\sim\mathcal{N}(0,I)$. Accordingly, its Fourier coefficients on frequency $k$ is:

$$ $\displaystyle\mathcal{F}(X)$ $\displaystyle=\sum_{n=1}^{T}x_{n}e^{-i2\pi kn/T}\sim\mathcal{N}(0,T).$ $$

After Fourier transform, we define $k$ as a dominant frequency if $k$ is greater than $\sqrt{T}$, which indicates statistical significance. However, in some cases, we have only a short sample to represent signals or a noisy signal that has no periodicity, which does not yield a statistically significant $k$. To prevent such cases, in $L_{phase}$, we guarantee that at least $\sqrt{T^{\prime}}$ number of frequencies are selected as dominant frequencies. $L_{phase}$ requires $O(n\log n)$ time for its computation, which is inherited from complexity of Fast Fourier Transform.

#### Uniform Amplification

Although effective, $L_{phase}$ has two limitations: 1) it is not perfectly phase shifting invariant as it is optimized with Fourier coefficients, and 2) aforementioned two subloss terms still make no consideration for uniform amplification invariance. Inspired by Paparrizos & Gravano (), we utilize normalized correlation for the uniform amplification. Specifically, we normalize correlation $R$ as follows:

$$ $\displaystyle R(\mathbf{X},\mathbf{Y})$ $\displaystyle=\frac{Corr(\mathbf{X},\mathbf{Y})}{\sqrt{Corr(\mathbf{X},\mathbf {X})\cdot Corr(\mathbf{Y},\mathbf{Y})}},$ $$

where $Corr$ is cross-correlation or auto-correlation, and $R$ is normalized correlation. By using this term, we have uniform amplification invariant measure. We utilize $L_{amp}$ as the subcomponent with small $\gamma$, since tolerance for the multiplication factor (i.e., uniform amplification) has greater influence than addition or phase shifting. As $L_{phase}$, by using fast Fourier transform, $L_{amp}$ takes $O(n\log n)$ time.

#### TILDE-Q Design Rationale: α 𝛼 \alpha italic_α and γ 𝛾 \gamma italic_γ

$L_{a}.shift$ is built for amplitude shifting and designed to be effective with both periodic and nonperiodic signals. In contrast, $L_{p}hase$ handles uniform amplification and is tailored to perform optimally with periodic signals. Since $L_{a}.shift$ and $L_{p}hase$ complement each other, we set $\alpha$ to balance them. For example, a large $\alpha$ value will work well for nonperiodic signals and will have less penalty for amplitude shifting. Additionally, we utilize $L_{a}mp$ as a subcomponent to calibrate the results (e.g., $gamma=0.01$). With this design, while preserving the shape-awareness of TILDE-Q, users can control specific invariances or conditions. For example, users can increase the value of $\alpha$ to emphasize nonperiodic modeling when a dataset has no particular periodicity. This user-oriented objective setting is one of the strengths of TILDE-Q and increases its utility.

## Appendix B Detailed Experiment Setup

#### Dataset

In our experiment, we utilize eight datasets – Synthetic, ECG5000, and Traffic dataset for the simple model (i.e., Sequence-to-Sequence Gated Recurrent Unit) and ETTm1, Electricity (i.e., ECL), Traffic, Weather, Exchange, and ILI for the eight recent time-series forecasting models.
For each dataset, we describe some metadata of them and the experimental setting, including the input length $n$ and prediction window $L$. Our implementation could be found in Github(^1^11[https://github.com/HyunwookL/TILDE-Q](https://github.com/HyunwookL/TILDE-Q)).

Synthetic: As Le Guen & Thome () describe, the Synthetic dataset is an artificial dataset for measuring model performance on sudden changes (step functions) with an input signal composed of two peaks.
The amplitude and temporal position of the two peaks are randomly selected. Then the selected position and amplitude of the step are determined by a peak position and amplitude.
We use 500 time-series for training, 500 for validation, and 500 for testing.
For the Synthetic dataset, we set the input length as $n=20$ and the prediction window as $L=40$.
The generation code is provided in DILATE Github(^2^22[https://github.com/vincent-leguen/DILATE](https://github.com/vincent-leguen/DILATE)).

ECG5000: This dataset is originally a 20-hour long ECG (Electrocardiogram), downloaded from Physionet(^3^33[https://physionet.org/](https://physionet.org/)) and archived in UCR Time Series Classification Archive (Dau et al., ).
The data is split by each heartbeat and processed in equal lengths (140).
In the training, we use 500 for training, 500 for validation, and 4000 for testing.
We take the first $n=84$ steps as input and predict the last $L=56$ steps.

Traffic: Traffic dataset is a collection of 48 months (2015-2016) hourly road occupancy rate (between 0 to 1) data from the California Department of Transportation(^4^44[http://pems.dot.ca.gov](http://pems.dot.ca.gov)).
For the GRU model, we utilize univariate series of the first sensor, a total of 17544 data points as Le Guen & Thome () do.
We set our problem as forecasting $L=24$ future occupancy rates with $n=168$ historical data (past week). We use 60% of the data for training, 20% for validation, and the rest for evaluation. For recent time-series forecasting models, we conduct multivaraite time-series forecasting and adopt hyperparameter settings from Time-Series-Library Github(^5^55[https://github.com/thuml/Time-Series-Library](https://github.com/thuml/Time-Series-Library)).

ETT: The ETT (Electricity TraNSformer Temperature) dataset, published by Zhou et al. (), is 2-year data collected from two separate counties in China, including ETTm1 dataset.
Each data point has a target value of “oil temperature” and other 6 power load features.
As Zhou et al. () do, we split them into 12/4/4 months for the training/validation/testing. Detailed settings, such as the input and output length and hyperparameter setting, are based on the information at Time-Series-Library Github 5.

ECL: The ECL (Electricity Consuming Load) is a dataset recorded in kWh every 15 minutes from 2012 to 2014, for 321 clients.
In our experiment, we split them into 15/3/4 months for the train/validation/test, as Zhou et al. () do.
Note that we use the same hyperparameter settings in the ETTm1 dataset.

Weather: Weather dataset contains 21 meteorological indicators from Weather Station of the Max Planck Biogeochemistry Institute in 2020 with 10-minute interval.

Exchange: Exchange (Wu et al., ) collects the panel data of daily exchange rates from 8 countries from 1990 to 2016. We follow the hyperparameter settings from author’s official codes.

Illness: Illness dataset contains the influenza-like illness patients in the United states in a weekly frequency.

Deep Learning Model Architectures We perform experiments with three different model architectures, including Sequence-to-Sequence GRU, Informer, and N-Beats.
To induce models to predict future time-series in a timely manner, we set $\alpha=0.5$ and $\gamma=0.01$ for TILDE-Q.
Other training metrics, including MSE and DILATE, are used as described in their original papers.
All models are trained with Early Stopping and ADAM optimizer. To evaluate TILDE-Q in simple model, we utilize one layer Sequence-to-Sequence GRU model. For the training of the GRU model, we set a learning rate of $1e-3$, hidden size of 128, trained by maximum 1000 epochs with Early Stopping and ADAM optimizer. In addition to the basic GRU model, we conduct experiments with eight state-of-the-arts models as follows: 1) Autoformer (Wu et al., ), the first model that utilize frequency-domain information with autocorrelation module, 2) FEDformer (Zhou et al., ), an advanced version of Autoformer by introducing frequency-enhanced decomposition module, 3) nonstationary transformer (NSformer in this paper), which introduces series stationarization and de-stationary attention to resolve non-stationary characteristics of real-world time-series, 4) DLinear (Zeng et al., ), a simple linear model with input decomposition, tackling the importance of understanding of time-series, 5) TimesNet (Wu et al., ), a model to discover multiple periods and capture temporal 2D-variations with TimesBlock module, 6) Crossformer (Zhang & Yan, ), a first model to handle cross-dimension dependency for MTS forecasting, 7) PatchTST, a model patching time-series data for the better embedding, and 8) iTransformer (Liu et al., ), embed single time-series to the vector to preserving its temporal information.

## Appendix C Additional Evaluations

### C.1 Detailed Experiment Results and Analysis

**Table 3: Experimental results on six real-world datasets. We compared TILDE-Q and MSE by conducting extensive experiments with eight models. For all baselines, we set input sequence length $T=96$ except ILI dataset. For ILI dataset, we set input sequence length $T=36$. Avg. means the average results from all four prediction lengths. We have colored the best training metric in red.**
| Model | iTransformer | PatchTST | Crossformer | TimesNet | DLinear | FEDformer | NSformer | Autoformer |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q | MSE | TILDE-Q |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |  |
| ETT* | 96 | 0.344 | 0.378 | 0.334 | 0.365 | 0.323 | 0.363 | 0.318 | 0.351 | 0.361 | 0.407 | 0.344 | 0.380 | 0.338 | 0.379 | 0.343 | 0.377 | 0.346 | 0.373 | 0.336 | 0.360 | 0.384 | 0.420 | 0.364 | 0.410 | 0.420 | 0.416 | 0.404 | 0.407 | 0.506 | 0.482 | 0.452 | 0.447 |
| 192 | 0.381 | 0.394 | 0.380 | 0.390 | 0.368 | 0.389 | 0.372 | 0.389 | 0.423 | 0.449 | 0.395 | 0.413 | 0.399 | 0.407 | 0.389 | 0.400 | 0.382 | 0.392 | 0.380 | 0.386 | 0.444 | 0.449 | 0.425 | 0.442 | 0.488 | 0.446 | 0.474 | 0.439 | 0.609 | 0.518 | 0.596 | 0.513 |  |
| 336 | 0.418 | 0.418 | 0.416 | 0.415 | 0.398 | 0.405 | 0.397 | 0.404 | 0.612 | 0.575 | 0.436 | 0.447 | 0.423 | 0.425 | 0.411 | 0.419 | 0.414 | 0.413 | 0.415 | 0.410 | 0.495 | 0.476 | 0.468 | 0.468 | 0.553 | 0.484 | 0.533 | 0.478 | 0.635 | 0.534 | 0.553 | 0.499 |  |
| 720 | 0.488 | 0.457 | 0.481 | 0.450 | 0.458 | 0.444 | 0.461 | 0.443 | 0.742 | 0.652 | 0.533 | 0.517 | 0.498 | 0.464 | 0.462 | 0.447 | 0.475 | 0.453 | 0.473 | 0.445 | 0.519 | 0.490 | 0.496 | 0.482 | 0.669 | 0.535 | 0.638 | 0.530 | 0.593 | 0.528 | 0.562 | 0.509 |  |
| Avg. | 0.408 | 0.412 | 0.403 | 0.405 | 0.387 | 0.400 | 0.387 | 0.397 | 0.535 | 0.521 | 0.427 | 0.439 | 0.415 | 0.419 | 0.401 | 0.411 | 0.404 | 0.408 | 0.401 | 0.400 | 0.461 | 0.459 | 0.438 | 0.451 | 0.533 | 0.470 | 0.512 | 0.465 | 0.586 | 0.516 | 0.541 | 0.492 |  |
| Electricity | 96 | 0.148 | 0.240 | 0.146 | 0.238 | 0.181 | 0.270 | 0.181 | 0.261 | 0.147 | 0.248 | 0.146 | 0.245 | 0.167 | 0.270 | 0.166 | 0.269 | 0.210 | 0.302 | 0.199 | 0.278 | 0.196 | 0.310 | 0.197 | 0.308 | 0.166 | 0.269 | 0.172 | 0.274 | 0.201 | 0.316 | 0.200 | 0.313 |
| 192 | 0.166 | 0.255 | 0.164 | 0.255 | 0.187 | 0.276 | 0.186 | 0.267 | 0.162 | 0.261 | 0.161 | 0.260 | 0.185 | 0.286 | 0.184 | 0.284 | 0.210 | 0.305 | 0.197 | 0.279 | 0.211 | 0.323 | 0.210 | 0.319 | 0.188 | 0.288 | 0.185 | 0.286 | 0.224 | 0.335 | 0.224 | 0.331 |  |
| 336 | 0.178 | 0.271 | 0.176 | 0.270 | 0.203 | 0.291 | 0.201 | 0.282 | 0.193 | 0.291 | 0.181 | 0.282 | 0.204 | 0.305 | 0.200 | 0.300 | 0.223 | 0.319 | 0.208 | 0.292 | 0.237 | 0.348 | 0.228 | 0.339 | 0.202 | 0.302 | 0.195 | 0.297 | 0.252 | 0.355 | 0.242 | 0.346 |  |
| 720 | 0.225 | 0.310 | 0.214 | 0.301 | 0.245 | 0.325 | 0.242 | 0.316 | 0.250 | 0.334 | 0.230 | 0.324 | 0.224 | 0.320 | 0.216 | 0.314 | 0.258 | 0.350 | 0.244 | 0.325 | 0.263 | 0.367 | 0.260 | 0.364 | 0.227 | 0.322 | 0.224 | 0.316 | 0.324 | 0.396 | 0.263 | 0.363 |  |
| Avg. | 0.179 | 0.269 | 0.175 | 0.266 | 0.204 | 0.291 | 0.203 | 0.282 | 0.188 | 0.284 | 0.181 | 0.278 | 0.195 | 0.295 | 0.192 | 0.292 | 0.225 | 0.319 | 0.212 | 0.294 | 0.227 | 0.337 | 0.224 | 0.333 | 0.196 | 0.295 | 0.194 | 0.293 | 0.250 | 0.351 | 0.232 | 0.338 |  |
| Traffic | 96 | 0.395 | 0.268 | 0.401 | 0.270 | 0.544 | 0.359 | 0.499 | 0.323 | 0.522 | 0.290 | 0.515 | 0.280 | 0.593 | 0.321 | 0.576 | 0.311 | 0.696 | 0.429 | 0.709 | 0.420 | 0.587 | 0.366 | 0.595 | 0.372 | 0.622 | 0.346 | 0.614 | 0.340 | 0.629 | 0.383 | 0.597 | 0.372 |
| 192 | 0.417 | 0.276 | 0.421 | 0.278 | 0.540 | 0.354 | 0.493 | 0.319 | 0.530 | 0.293 | 0.534 | 0.290 | 0.617 | 0.336 | 0.582 | 0.325 | 0.647 | 0.408 | 0.636 | 0.386 | 0.606 | 0.379 | 0.598 | 0.375 | 0.643 | 0.356 | 0.628 | 0.349 | 0.637 | 0.403 | 0.622 | 0.392 |  |
| 336 | 0.433 | 0.283 | 0.427 | 0.280 | 0.551 | 0.358 | 0.510 | 0.337 | 0.558 | 0.305 | 0.550 | 0.298 | 0.629 | 0.336 | 0.615 | 0.333 | 0.657 | 0.410 | 0.641 | 0.387 | 0.621 | 0.383 | 0.613 | 0.379 | 0.645 | 0.354 | 0.642 | 0.353 | 0.623 | 0.391 | 0.614 | 0.383 |  |
| 720 | 0.467 | 0.302 | 0.454 | 0.294 | 0.586 | 0.375 | 0.552 | 0.362 | 0.589 | 0.328 | 0.562 | 0.316 | 0.640 | 0.350 | 0.627 | 0.341 | 0.689 | 0.424 | 0.680 | 0.406 | 0.626 | 0.382 | 0.619 | 0.379 | 0.667 | 0.365 | 0.662 | 0.362 | 0.658 | 0.404 | 0.644 | 0.397 |  |
| Avg. | 0.428 | 0.282 | 0.426 | 0.281 | 0.555 | 0.362 | 0.514 | 0.335 | 0.550 | 0.304 | 0.540 | 0.296 | 0.620 | 0.336 | 0.600 | 0.328 | 0.672 | 0.418 | 0.667 | 0.399 | 0.610 | 0.378 | 0.606 | 0.376 | 0.644 | 0.355 | 0.637 | 0.351 | 0.637 | 0.395 | 0.619 | 0.386 |  |
| Weather | 96 | 0.177 | 0.218 | 0.174 | 0.213 | 0.177 | 0.216 | 0.174 | 0.215 | 0.158 | 0.230 | 0.157 | 0.226 | 0.172 | 0.220 | 0.171 | 0.217 | 0.200 | 0.256 | 0.196 | 0.243 | 0.217 | 0.296 | 0.216 | 0.285 | 0.189 | 0.238 | 0.179 | 0.224 | 0.274 | 0.336 | 0.259 | 0.310 |
| 192 | 0.223 | 0.256 | 0.220 | 0.244 | 0.225 | 0.258 | 0.221 | 0.257 | 0.206 | 0.277 | 0.200 | 0.260 | 0.219 | 0.261 | 0.210 | 0.255 | 0.240 | 0.295 | 0.237 | 0.284 | 0.276 | 0.336 | 0.267 | 0.311 | 0.268 | 0.296 | 0.263 | 0.293 | 0.330 | 0.378 | 0.323 | 0.364 |  |
| 336 | 0.280 | 0.299 | 0.277 | 0.293 | 0.285 | 0.301 | 0.279 | 0.298 | 0.272 | 0.335 | 0.270 | 0.319 | 0.280 | 0.306 | 0.284 | 0.300 | 0.285 | 0.332 | 0.282 | 0.324 | 0.339 | 0.380 | 0.330 | 0.365 | 0.351 | 0.338 | 0.367 | 0.350 | 0.389 | 0.410 | 0.355 | 0.382 |  |
| 720 | 0.359 | 0.350 | 0.356 | 0.347 | 0.360 | 0.350 | 0.357 | 0.349 | 0.398 | 0.418 | 0.363 | 0.399 | 0.365 | 0.359 | 0.359 | 0.354 | 0.348 | 0.384 | 0.347 | 0.372 | 0.403 | 0.428 | 0.395 | 0.405 | 0.441 | 0.418 | 0.433 | 0.402 | 0.469 | 0.458 | 0.448 | 0.448 |  |
| Avg. | 0.260 | 0.281 | 0.257 | 0.274 | 0.262 | 0.281 | 0.258 | 0.280 | 0.259 | 0.315 | 0.248 | 0.301 | 0.259 | 0.287 | 0.256 | 0.282 | 0.268 | 0.317 | 0.266 | 0.306 | 0.309 | 0.360 | 0.302 | 0.342 | 0.312 | 0.323 | 0.310 | 0.317 | 0.366 | 0.396 | 0.346 | 0.376 |  |
| Exchange | 96 | 0.092 | 0.214 | 0.089 | 0.209 | 0.086 | 0.204 | 0.083 | 0.196 | 0.271 | 0.377 | 0.236 | 0.351 | 0.105 | 0.234 | 0.104 | 0.233 | 0.078 | 0.200 | 0.077 | 0.194 | 0.160 | 0.290 | 0.152 | 0.279 | 0.144 | 0.265 | 0.133 | 0.258 | 0.172 | 0.302 | 0.154 | 0.279 |
| 192 | 0.185 | 0.308 | 0.184 | 0.308 | 0.181 | 0.302 | 0.177 | 0.296 | 0.515 | 0.533 | 0.451 | 0.495 | 0.214 | 0.337 | 0.202 | 0.328 | 0.166 | 0.301 | 0.163 | 0.295 | 0.282 | 0.385 | 0.277 | 0.378 | 0.268 | 0.370 | 0.236 | 0.350 | 0.318 | 0.413 | 0.271 | 0.378 |  |
| 336 | 0.366 | 0.438 | 0.345 | 0.426 | 0.329 | 0.416 | 0.337 | 0.422 | 1.239 | 0.873 | 1.037 | 0.779 | 0.365 | 0.439 | 0.360 | 0.430 | 0.298 | 0.410 | 0.259 | 0.384 | 0.484 | 0.512 | 0.457 | 0.497 | 0.470 | 0.507 | 0.450 | 0.485 | 0.513 | 0.534 | 0.488 | 0.516 |  |
| 720 | 0.914 | 0.724 | 0.895 | 0.715 | 0.864 | 0.718 | 0.857 | 0.698 | 1.745 | 1.060 | 1.606 | 1.014 | 0.926 | 0.733 | 0.962 | 0.759 | 0.749 | 0.655 | 0.703 | 0.643 | 1.288 | 0.873 | 1.209 | 0.844 | 1.302 | 0.811 | 1.078 | 0.733 | 1.126 | 0.822 | 1.061 | 0.799 |  |
| Avg. | 0.389 | 0.421 | 0.379 | 0.415 | 0.365 | 0.410 | 0.364 | 0.403 | 0.943 | 0.711 | 0.833 | 0.660 | 0.403 | 0.436 | 0.407 | 0.438 | 0.323 | 0.392 | 0.301 | 0.379 | 0.554 | 0.515 | 0.524 | 0.499 | 0.546 | 0.488 | 0.474 | 0.457 | 0.532 | 0.518 | 0.494 | 0.493 |  |
| ILI | 24 | 2.551 | 1.023 | 2.352 | 0.956 | 2.308 | 0.954 | 2.257 | 0.896 | 3.541 | 1.249 | 3.300 | 1.208 | 2.007 | 0.942 | 1.753 | 0.876 | 2.858 | 1.160 | 2.771 | 1.134 | 3.567 | 1.355 | 3.177 | 1.253 | 2.453 | 0.981 | 1.708 | 0.860 | 3.597 | 1.324 | 3.577 | 1.358 |
| 36 | 2.237 | 0.964 | 2.206 | 0.950 | 2.333 | 0.926 | 2.280 | 0.929 | 3.615 | 1.251 | 3.170 | 1.168 | 2.752 | 1.005 | 2.447 | 0.942 | 2.689 | 1.110 | 2.526 | 1.089 | 3.553 | 1.331 | 3.259 | 1.248 | 2.959 | 1.069 | 2.220 | 0.957 | 3.389 | 1.274 | 3.315 | 1.269 |  |
| 48 | 2.290 | 0.975 | 2.173 | 0.935 | 2.307 | 0.935 | 2.080 | 0.890 | 3.698 | 1.278 | 3.187 | 1.180 | 2.533 | 0.962 | 2.239 | 0.880 | 2.814 | 1.151 | 2.273 | 1.016 | 2.979 | 1.196 | 2.963 | 1.176 | 2.592 | 1.025 | 2.122 | 0.923 | 3.160 | 1.234 | 2.973 | 1.159 |  |
| 60 | 2.255 | 0.975 | 2.147 | 0.954 | 2.064 | 0.917 | 1.954 | 0.896 | 4.043 | 1.344 | 3.529 | 1.253 | 2.092 | 0.942 | 1.892 | 0.898 | 2.900 | 1.178 | 2.331 | 1.043 | 3.127 | 1.222 | 3.054 | 1.222 | 2.446 | 1.021 | 2.078 | 0.942 | 3.163 | 1.210 | 2.930 | 1.179 |  |
| Avg. | 2.333 | 0.984 | 2.220 | 0.949 | 2.253 | 0.933 | 2.143 | 0.903 | 3.724 | 1.281 | 3.297 | 1.202 | 2.346 | 0.963 | 2.083 | 0.899 | 2.815 | 1.150 | 2.475 | 1.071 | 3.307 | 1.276 | 3.113 | 1.225 | 2.613 | 1.024 | 2.032 | 0.921 | 3.327 | 1.261 | 3.199 | 1.241 |  |

Figure: Figure 3: Qualitative results with simple sequence-to-sequence GRU model (a) and state-of-the-art model (b).
Refer to caption: x3.png

At first, we observe that the model optimized with TILDE-Q outperforms the same model optimized with other objective functions in both short- and long-term forecasting tasks. However, each model has a wide range of TILDE-Q’s performance improvement caused by their design. In this section, we describe our findings related to both model design and the difficulty of the tasks. In the case of Autoformer and NSformer, we could observe that their performance improvements are significantly higher than that of FEDformer or TimesNet. In design, Autoformer makes predictions with autocorrelation, which has a limitation in being aware of frequency-domain features than that of FEDformer or TimesNet. In the case of the NSformer, it utilizes series stationarization, which could be helpful to remove possible side effects from TILDE-Q, such as an equal gap caused by $L_{a.shift}$. This difference caused by design indicates two facts: 1) TILDE-Q could help the unseen feature modeling with its shape-awareness, as in Autoformer’s case, and 2) the model design influences the loss function, as NSformer helps the optimization with TILDE-Q. Crossformer is one good example with 8.85% improvement. For the Crossformer, which aims to resolve inter-domain dependency but has less attention to the temporal dependencies, TILDE-Q improves Crossformer’s temporal dependency modeling, and Crossformer’s inter-domain modeling supports TILDE-Q’s limitation for the inter-domain modeling.

In case of the dataset, Electricity or Traffic datasets makes the lowest improvements among all datasets. This is caused by task difficulty–for example, in case of the Autoformer, Electricity tasks with prediction lengths $T^{\prime}=\{96,192,336\}$, Autoformer can solve the tasks without support for the shape-awareness. However, in case of the Electricity with prediction length 720, a relatively challenging task, Autoformer with MSE struggles to properly solve the task. This problem is also observable in Crossformer, which has relatively limited ability for temporal dependency modeling than the other models.

Next, we present a qualitative analysis of the results.
Fig. 3 shows how the model with different training metrics forecasts with different datasets.
From the figure, we have noticed that TILDE-Q allows the model to generate more robust, shape-aware forecasting, regardless of amplitude shifting, phase shifting, and uniform amplification.
For example, in the case of Autoformer (Fig. 3 (b) top), TILDE-Q helps the model to handle multiple periodicity, which is not achieved with MSE (blue line). In contrast, Autoformer trained with MSE predicts only a single periodicity, indicating the limitation of MSE on shape-awareness.
The strength of TILDE-Q is also observable in the iTransformer (Fig. 3 (b), bottom). Even when the model could make multiple periodicity modeling, TILDE-Q makes it more precise.
In summary, TILDE-Q proves that it is model-agnostic, noise-robust, and shape-aware loss function and is far beneficial for the time-series forecasting.

Figure: (a) iTransformer
Refer to caption: x4.png

Figure: (a) iTransformer
Refer to caption: x10.png

Figure: (a) iTransformer
Refer to caption: x16.png

Figure: (a) iTransformer
Refer to caption: x22.png

### C.2 Qualitative Results with Visualization

To provide a clear comparison for MSE and TILDE-Q among different datasets and models, we list supplementary forecasting results of four representative datasets in Fig 4– 7. We provide qualitative results with six models–Autoformer (Wu et al., ), DLinear (Zeng et al., ), TimesNet (Wu et al., ), Crossformer (Zhang & Yan, ), PatchTST (Nie et al., ), and iTransformer (Liu et al., ). Among various models and datasets, TILDE-Q shows its superior performance than MSE baseline.

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. Theoretical Properties and Convergence of TILDE-Q Loss

Could you provide more insights into the theoretical properties of TILDE-Q, particularly in terms of convergence and robustness to noise? Specifically, how does TILDE-Q’s invariance to amplitude and phase shifts impact the stability of the training process? An explanation or analysis of these properties could clarify whether TILDE-Q may introduce any trade-offs or require certain conditions for effective convergence.

2. Hyperparameter Selection and Sensitivity

The hyperparameters 𝛼 and  𝛾 play a key role in balancing TILDE-Q’s distortion invariances. Could you provide guidance on how these hyperparameters were chosen for the experiments and any best practices for tuning them? Additionally, it would be helpful to know if TILDE-Q’s performance is sensitive to the values of these hyperparameters, as this may impact practical implementation across various datasets.

3. Broader Comparison with Other Shape-Aware Metrics

TILDE-Q is compared mainly against MSE and DILATE. Could you clarify why other shape-aware metrics, like CID-DTW or MSM, were not included? Such a comparison might offer additional context for understanding TILDE-Q’s unique strengths. If computational constraints were a factor, do you have qualitative or preliminary findings on how TILDE-Q may perform relative to these methods?
Relevance of Fourier Coefficients for Phase Shifting Invariance

The paper suggests Fourier coefficients to achieve phase shifting invariance. Could you elaborate on why Fourier coefficients are preferred over other potential methods for phase handling? Additionally, are there specific scenarios where this choice may limit TILDE-Q’s effectiveness in capturing certain phase-shifted patterns?

4. Applicability to Non-Deep Learning Models

TILDE-Q is shown to work well with Transformer-based and GRU models. Do you foresee any challenges in applying TILDE-Q to more traditional forecasting models, such as ARIMA or SARIMA? This information could be useful for practitioners interested in testing TILDE-Q with non-deep learning approaches.

5. Computational Complexity and Efficiency

Since TILDE-Q combines several components (e.g., Fourier coefficients, softmax, cross-correlation), could you comment on its computational complexity compared to traditional loss functions like MSE? Additionally, are there potential optimizations or trade-offs that would make TILDE-Q more computationally feasible for large datasets or real-time applications?
Suggestions for Improvement

6. Include Hyperparameter Sensitivity Analysis

Adding a sensitivity analysis for the hyperparameters 𝛼 and 𝛾 would help readers understand TILDE-Q’s robustness and provide guidance for practitioners applying it to new datasets.

7. Explore Real-World Case Studies or Long-Term Forecasting Scenarios

Providing a case study or a long-term forecasting scenario where shape preservation is crucial (e.g., equipment monitoring or anomaly detection) would highlight TILDE-Q’s practical relevance and make its benefits more tangible.

8. Clarify the Limitations or Conditions for Optimal Use

While TILDE-Q shows impressive results, it would be helpful to outline any known limitations or conditions where TILDE-Q may be less effective. For instance, clarifying situations where high noise or extreme non-periodicity may reduce TILDE-Q’s effectiveness could guide users on when to apply it and what adjustments may be necessary.
soundness: 3
strengths: Originality

The paper introduces a novel loss function, TILDE-Q (Transformation Invariant Loss function with Distance Equilibrium), which directly addresses a long-standing challenge in time-series forecasting: capturing shape distortions. This approach is innovative in two main ways:

1. It expands beyond conventional Lp norm loss functions by targeting invariance to transformations such as amplitude shifting, phase shifting, and uniform amplification.

2. TILDE-Q creatively combines Fourier coefficients and normalized cross-correlation in its loss function formulation, allowing models to learn temporal dynamics and shape in time-series data more effectively. This novel formulation is highly relevant, as shape-aware forecasting remains underexplored, with traditional metrics often failing in practical scenarios where pattern preservation is essential.

Quality

The paper demonstrates technical depth, grounding the TILDE-Q loss function in rigorous theoretical justifications and thoroughly detailing the transformation invariances it aims to achieve. Additionally, the experiments are carefully conducted, testing the new loss function across multiple state-of-the-art forecasting models and a range of datasets (e.g., ECL, Electricity, Traffic). The authors have thoughtfully incorporated comparative analysis with other prominent loss functions, such as MSE and DILATE, and have shown measurable improvements in both short-term and long-term forecasting. Furthermore, the empirical results support TILDE-Q’s robustness in both periodic and non-periodic settings, establishing the method’s versatility and reliability across different temporal dynamics.

Clarity

The paper is well-organized and clearly written, making its technical contributions accessible without oversimplification. The mathematical formulations are precise, with key concepts like transformation invariance and shape-awareness explained in sufficient detail. Visual aids and examples (e.g., showing model outputs with different loss functions) help illustrate the paper's motivation and results effectively. The authors also contextualize their approach within prior literature, comparing TILDE-Q to existing methods and addressing the limitations of conventional loss functions (e.g., MSE’s inability to capture temporal distortions).

Significance

The proposed TILDE-Q loss function offers substantial benefits to time-series forecasting, particularly in domains where shape preservation is crucial, such as traffic analysis, electricity usage, and economic forecasting. By improving models' ability to capture and retain shape characteristics, TILDE-Q offers significant potential for enhancing predictive accuracy and reliability in real-world applications. The work is especially relevant given the increasing demand for accurate forecasting in complex, non-stationary datasets where existing methods struggle. The approach is model-agnostic, broadening its applicability across various forecasting architectures, which adds further value to the broader AI and forecasting communities.
summary: This paper introduces a novel loss function for time-series forecasting. Traditional loss functions like Mean Squared Error (MSE) or Dynamic Time Warping (DTW) are insufficient for complex temporal patterns and often fail to capture shape distortions in time-series data accurately. This paper addresses these limitations by proposing TILDE-Q (Transformation Invariant Loss function with Distance Equilibrium), which is designed to be invariant to amplitude shifts, phase shifts, and uniform amplifications.

The main contributions of the paper include:

1. A comprehensive exploration of shape awareness and distortion invariances in time-series data, enhancing understanding of their impact on forecasting accuracy.

2. The design of TILDE-Q, a loss function that achieves shape-aware modeling by accommodating amplitude, phase, and amplification distortions.

3. Empirical evaluations showing that models trained with TILDE-Q outperform those using standard metrics (e.g., MSE, DILATE) in various real-world applications, demonstrating higher forecasting accuracy and robustness across diverse domains like traffic, electricity, and weather forecasting.

This loss function is particularly beneficial for tasks requiring shape preservation in forecasts, providing more reliable and informative results by focusing on temporal dynamics rather than solely point-wise accuracy
weaknesses: 1. Limited Theoretical Analysis of Loss Function Properties

While TILDE-Q’s design is explained in detail, the paper could benefit from a deeper theoretical analysis of the properties and potential trade-offs of the loss function, particularly concerning convergence behavior and sensitivity to noise. For example:

Sensitivity Analysis: TILDE-Q is designed to be invariant to amplitude and phase shifts, but there is limited theoretical discussion on how these invariances impact convergence or stability during training, especially in noisy datasets. A formal exploration or proof of these characteristics would strengthen confidence in the loss function’s robustness and may reveal cases where TILDE-Q could be fine-tuned.

Generalization Properties: Including a theoretical analysis of the loss function’s generalization capabilities for complex time-series data could add value. For example, exploring how well TILDE-Q balances the trade-off between capturing shape and maintaining point-wise accuracy across different domains would support its broader applicability.

2. Limited Comparisons with Alternative Loss Functions Beyond MSE and DILATE

Although the experiments demonstrate TILDE-Q’s effectiveness against MSE and DILATE, it would be beneficial to include comparisons with additional shape-aware loss functions or distance measures. Some alternatives worth considering include:

CID (Complexity Invariant Distance) or CID-DTW: These metrics are known for handling shape similarity in time-series data. While TILDE-Q shows clear advantages over DILATE, CID-based methods are also shape-sensitive, so they could provide further insights into TILDE-Q’s comparative strengths.

MSM (Move-Split-Merge Distance): Another distance measure often used for shape-based comparisons in time-series forecasting. Incorporating MSM into experiments would offer a more comprehensive picture of TILDE-Q’s relative advantages in modeling shape.

3. Limited Analysis on Hyperparameter Sensitivity and Selection

The paper introduces hyperparameters 𝛼 and  𝛾 in the TILDE-Q formulation, which weigh the importance of each distortion invariance. However, there is minimal guidance on how these should be chosen, and the paper does not explore the sensitivity of TILDE-Q to variations in these values:

Hyperparameter Sensitivity Study: Conducting a hyperparameter sensitivity analysis across various datasets and tasks would provide insight into how to best set or tune these values for optimal results.

Recommendations for Specific Domains: Providing specific recommendations or heuristics for setting 𝛼 and  𝛾 based on the dataset characteristics (e.g., high periodicity, non-stationary) would make the paper more practical for practitioners implementing TILDE-Q in different applications.

4. Model-Agnostic Results Could Be Strengthened with Broader Model Comparisons

The paper tests TILDE-Q across several state-of-the-art models, but it focuses heavily on Transformer-based architectures. To strengthen claims of model-agnosticism, it would be helpful to test TILDE-Q on additional, fundamentally different models, such as:

Traditional Statistical Models: Testing on models like ARIMA could illustrate how TILDE-Q performs in a more conventional setting, providing insights into whether TILDE-Q’s shape-aware capabilities are beneficial even in non-deep learning contexts.
Recurrent Models: While the paper includes some experiments with a GRU, additional recurrent models such as LSTM or BiLSTM could highlight whether TILDE-Q’s shape-awareness is generally beneficial across sequential deep learning architectures.

5. Limited Real-World Application Case Studies

While the experiments include several datasets, the paper could further validate TILDE-Q’s practical value by demonstrating its application in a real-world forecasting scenario where shape-preserving forecasts are crucial. Such a case study would:

Illustrate Practical Relevance: Applying TILDE-Q in a specific domain with high stakes on shape preservation (e.g., predicting anomalies in sensor data for equipment monitoring) could make its benefits more concrete and relatable to practitioners. Long-Term Forecasting: Exploring a real-world scenario with longer forecast horizons and showing how TILDE-Q maintains shape fidelity over extended time frames could further highlight its advantages over traditional metrics.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. For the experiments in Table 1 and Table 2, are the hyperparameters the same (including neural architectures, learning rate, etc.) exactly the same when training with MSE or TILDE-Q? How are the hyperparameter searched?

2. Since the TILDE-Q is invariant to amplitude shifting, phase shifting, and uniform amplification, why can it typically improve MSE? Scenarios may happen where the prediction has these transformations, which does not affect the TILDE-Q, but increases the MSE.
soundness: 3
strengths: Significance: The paper addresses the important problem of capturing the shape (temporal dynamics) in time series forecasting, and introduces a loss function that can better account for it. I think the significance is good.

Originality: I think the originality is reasonable.

Quality: Overall, the paper has good quality. The experiments is sound and extensive.

Clarity: The paper is written in a clear way.
summary: This paper introduces a Transformation Invariant Loss function with Distance EQuilibrium (TILDE-Q) for time series forecasting. The paper discusses six distortions that usually appear in time series, and proposes a loss function (consisting of three terms) which are invariant to amplitude shifting, phase shifting, and uniform amplification. Experiments are performed which validates that the proposed loss function generally improves the metrics (e.g., MSE, MAE) and can better capture the shape.
weaknesses: There are a few aspects that could improve the presentation and soundness of the paper:

1. Perform ablation study that contain one or two terms of the constituting terms in TILDE-Q, and evaluate the results using different metrics similar to Table 2

2. Put example visualizations (e.g. Appendix C.2) in the main text, to give a more vivid illustration of the benefits of the proposed loss function.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - Please refer to the weaknesses section.
soundness: 3
strengths: - The paper is clear and well-written. The main ideas are well exposed.
- Code is provided, making reproducibility easier.
- The presented results demonstrate the performance of the proposed approach. Table 1 demonstrates that the method outperforms regular training objectives quite consistently.
- The authors provide an extensive evaluation of distortion/augmentation methods, hence there is little ambiguity that they have covered the search space extenstively.
summary: This paper introduces TILDE-Q, a novel shape-aware loss function for time-series forecasting that addresses limitations of traditional distance-based objectives. By focusing on "shape" in time-series data, the proposed loss function enhances the model's ability to generate informative predictions that capture temporal dynamics, such as peaks and troughs, rather than just reducing point-wise errors. TILDE-Q is model-agnostic and demonstrates robustness to various distortions, outperforming traditional metrics like MSE and DILATE in both accuracy and shape-related evaluations. The approach offers improved forecasting performance across diverse applications.
weaknesses: - The main weakness I find in this paper is that, essentially, it demonstrates that data augmentation (via input transformations) leads to improved learning performance. This is both true, and interesting, but also expected based on (1) the fact that similar augmentations are key components of similar methods in computer vision and other fields (e.g. the whole field of contrastive learning [1]), and (2) have already been shown to have a strong impact in time-series representation learning [2, 3]. Comparisons to other objectives ([2, 3]) that also use augmentations sound particularly warranted.




References
[1] Learning Representations by Maximizing Mutual Information Across Views, P Bachman et al.
[2] Yue, Zhihan, et al. "Ts2vec: Towards universal representation of time series." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 8. 2022.
[3] Woo, Gerald, et al. "Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting." arXiv preprint arXiv:2202.01575 (2022).

---

confidence: 4
contribution: 3
initial_rating: 3
presentation: 4
questions: 1. In Equation 1: The choice of k is questionable. If k is the same for all time points, it may not always align with real-world scenarios where the deviation between predicted and true values varies over time. 
2. The idea of "shape awareness invariant to amplitude shifting" implies that the model will capture the overall shape or pattern regardless of vertical shifts. So, how this is achieved through a fixed gap k is unclear, as the shape might still be affected if the deviation varies.

3. Equation 2 seems to be oversimplified. Having the same dominant frequency does not necessarily mean that the two time-series samples are similar enough. Two time-series with the same dominant frequency (as in your example; sin(x) and 2 sin(x+x0)) can still have substantial variations in amplitude, phase, and waveform shape.
4. I also find that the claim “Eq. 2 allows a similar shape as the target time-series in forecasting, not exactly the same shape” is vague. It doesn’t clarify what level of similarity is acceptable, and it leaves open questions about how such similarity is quantified.

5. How is constant “k” from Eq. 1 is still applied in Eq. 3?
6. What if \hat{y}_i in Eq. 3 is not 0?
7. The authors claim that softmax produces relative values, hence it can handle any gap k. Softmax outputs are bounded between 0 and 1, and the sum of these values over all elements equals 1. My concern is that this normalization might not effectively capture a consistent gap across all values. 
8. How do the authors chose the values of \alpha and \beta in Eq. 8? Do we need to tune their values per dataset?
9. For Figure 1, the blue boxes before the arrows are not clear what do they mean. What do the left and right subfigures mean?
soundness: 2
strengths: The authors clearly described the problem and their journey toward the proposed loss function, so a big thanks for the clear writing and illustration, which made the review easier.
summary: This paper proposes a shape-aware loss function (TILDE-Q) to be used to train time series forecasting models. This loss function learns the shape of the time series besides considering amplitude and phase distortions.
weaknesses: While the paper is well-presented, it still has some weaknesses that need to be addressed.

1. I have some concerns about the main equations, that I list in the "Questions" section.

2. The experiments section does not cover many of the expected questions. For example, I expected to find: 
    - An ablation study that investigates the contributions of each component of TILDE-Q. 
    - An experiment that demonstrates the effectiveness of TILDE-Q under controlled conditions where the transformations (amplitude shifting, phase shifting, and uniform amplification) are systematically introduced. 
    -  A sensitivity analysis to the effect of choosing hyperparameters \alpha and \beta and how they influence performance. 
    - An assessment of the computational complexity added by TILDE-Q, compared with simpler MSE loss.

Meta-review: This paper introduces TILDE-Q, a novel shape-aware loss function for time-series forecasting that captures amplitude and phase distortions while modeling both periodic and non-periodic dynamics. By addressing the limitations of traditional loss functions like MSE and MAE, TILDE-Q enables better prediction of complex temporal patterns and outperforms existing metrics across diverse real-world applications.

All the reviewers agreed on the merits of the paper, which is clearly written and  addresses a significant question. The proposed methodology is original and efficient on the considered benchmark datasets. The major weaknesses highlighted by reviewers are in general a limited experimental part in terms of ablations or comparisons with related methods (CID-DTW, MSM), or in describing some theoretical properties of their loss. In this light, I am recommending a reject decision, and I encourage the authors to further strengthen their work on the questions raised by reviewers.

Decision: Reject