AdaWaveNet: Adaptive Wavelet Network for Time Series Analysis
Authors: Han Yu
Peikun Guo
Akane Sano
Venue: Accepted by TMLR ()

Abstract: Time series data analysis is a critical component in various domains such as finance, healthcare, and meteorology. Despite the progress in deep learning for time series analysis, there remains a challenge in addressing the non-stationary nature of time series data. Most of the existing models, which are built on the assumption of constant statistical properties over time, often struggle to capture the temporal dynamics in realistic time series and result in bias and error in time series analysis. This paper introduces the Adaptive Wavelet Network (AdaWaveNet), a novel approach that employs Adaptive Wavelet Transformation for multi-scale analysis of non-stationary time series data. AdaWaveNet designed a lifting scheme-based wavelet decomposition and construction mechanism for adaptive and learnable wavelet transforms, which offers enhanced flexibility and robustness in analysis. We conduct extensive experiments on 10 datasets across 3 different tasks, including forecasting, imputation, and a newly established super-resolution task. The evaluations demonstrate the effectiveness of AdaWaveNet over existing methods in all three tasks, which illustrates its potential in various real-world applications.

Published in Transactions on Machine Learning Research (11/2024)
AdaWaveNet: Adaptive Wavelet Network for Time Series Analysis
Han Yu
hy29@rice.edd edu
Deportment of Electrical and Computer Engineering
Rice University
Peikun Guo
p934 Orice. edu
Department of Computer Science Rice University
Akane Sano
akane.sano@ricee.c edu.
Department of Electricol and Computer Engineering
Rice University
Reviewed on OpenReview: /t wptpr//pppereee....om net/ forum? id= m4bE999F1X
Abstract
Time series data analysis is critical component in various domains such finance, healthcare, and meteorology. Despite the progress in deep learning for time series analysis, there remains a challenge in addressing the non-stationary nature of time series data. Most of the existing models, which are built on the assumption of constant statistical propertics over time, often struggle to capture the temporal dynamics realistic time series and result in bias and error time series analysis. This paper introduces the Adaptive Wavelet Network (Ada WoueNet), novel approach that employs Adaptive Wavelet Transformation for multi- scale analysis of non-stationary time series data. Ada WaveNet designed a lifting schemn- based wavelet decomposition and construction mechanism for adaptive and learnable wavelet transforms, which offers enhanced flexibility and robustness in analysis. We conduct extensive experiments on 10 datasets across different tasks, including forecasting, imputation, and a newly established super-resolution task. The evaluation ions demonstrate the effectiveness of Ada WaveNet over existing methods in all three tasks, which illustrates its potential in various real-world applications. The code implemented for the Ada WaveNel is available at https://githin www////o////oo////////o/eoeeeeeo wewpo:/wwllloolldleeeeel-- Attp///Waaaaeee...
1 Introduction
Time series data, extensively encountered in various domains, including finance, healthcare, and meteorology, require effective analytical methodologies (Esling & Agon 2012). Therefore, understanding and analyzing time series data has triggered substantial interest in various real-world applications. Recently, the rapid development of deep learning has significantly transformed the landscape of time series analysis. These advancements have triggered breakthroughs various applications, including forocasting (Zhou et al. 2021
2022b
Wu al. )021). imputation (Wu et al. 2022 Xu al. 2022) and anomaly detection (Blázquez-Gare Li & Jung 2023)
et al. 2021
However, even with the promising performances of the aforementioned methods, notable limitation in this research area is the inadequate focus on the non-slationary nature of time series data. Non-stationarity, with its evolving statistical properties and time-dependent patterns, poses a significant challenge for traditional deep learning models (Hyndman Athanasopoulos [2018] Shumway et al. 2017). The constantly shifting nature of time series data can make it difficult for the aforementioned methods fully capture its dynamic patterns, which palentially leads to inaccurate analysis.

---

Published in Transactions on Machine Learning Research (11/2024)
To address the non-stationary challenge, recent efforts have aimed to adapt deep learning methods for temporal dynamic analysis (Liu et al., 2022b; 2023bp. However, due to the designing basis, such instance- wise normalization and Fourier transform, these methods may lack the adaptability to process multi-scale features and capture the changing temporal dynamics across different signals. Thus, despite the advancements in modeling non-stationary time series data by these methods, there remains a critical need for an approach that combines multi-scale analysis with efficient computational strategies and adaptability.
In response to these challenges, we propose Ada WaveNet, novel architecture that employs adaptive wavelet transformations within an efficient, multi-scale framework. Unlike existing wavelet- based methods such FEDformer (Zhou et al., (022b),, which relies on manually tuned wavelet paramcters and a fixed Transformer architecture, Ada WoweNee leverages the lifting scheme (Sweldens 1998) to enable fully learnable, adaptive wavelet transformations. Also, expanded from the existing time-series decompositon, we proposed a grouped linear module to enhance the dependencics among varying channels. This approach dynamically adjusts the data's temporal properties, which offers superior flexibility and robustness in handling non-stationary time series. By combining adaptive multi-scale analysis with an efficient computational design, Ada WaveNel addresses critical gaps in current methods for analyzing complex time series data.
Our contribution can be summarized as:
We introduce Ada WaveNet, a novel architecture designed to address the challenges posed by non-stationary time series data through adaptíve, multi-scale approach. Differing from prior decomposition-based or fixed-parameter methods, Ada WaveNet leverages a learnable wavelet trans- formation via the lifting scheme, which enables dynamic adaptation to evolving statistical properties within the data. While we leverage time-series decomposition module that separates trend and seasonal components following previous methods, the adaptability is further enhanced by a grouped linear module, which uniquely captures channel-specific dependencies.
We establish new benchmark for super-rosolution in the field of time series data. This benchmark aims to enhance the quality of data obtained from under-sampled sequences and improve the overall efficiency and effectiveness of time series data monitoring and analysis.
Our extensive evaluations demonstrate that Ada WaveNet outperforms existing methods in forecasting and super-resolution tasks. These results suggest the capability of the proposed method for diverse real-world applications.
2
Related Work
2.1 Time Series Analysis with Deep Learning
'he evolution of deep learning has significantly impacted temporal modeling and time series analysis. Recurrent neural networks (RNNs), such as those based Long Short-Term Memory (LSTM) (Hochreiter)
& Schmidhuber 1997 Siami-Namini et al. 2019). are designed to capture temporal dependencies through
internal states. Multi-L Layer Perceptrons (MLPs) [Zen et al. 2023 Li et al. 2023 have shown effectiveness in temporal modeling by processing woint-wise projections of sequences. Convolutional Neural Networks (CNNs) LLea et al. 2017 Liu et al.) 2022a) excel in extracting hierarchical features and detecting complex patterns, leveraging their strength spatial and temporal data processing. Moreover, the Transformer variants have demonstrated remarkable results in time scries applications by capturing long-range dependencics and processing entire sequences efficiently (Zhou et al., 2022b Liu et al.) 2023a; 2022bb Zhhang & Yan 2022p. Nevertheless, in real-world applications, these well-proven structures may struggle with non-stationarity in time series data because their learned patterns and dependencies are based on the assumption of consistent statistical properties. On the other hand, most of the realistic temporal data, which is non-stationary with dynamics over time, violates the consistent assumption.
These methods have been developed and applied to various tasks, including forccasting (Zhou et al. 2021 2022b. Wu al. 2021). imputation (Wu et al. 2022 Xu et al.) 2022). and anomaly detection (Báázquez Garcia et al. 2021 Li & Jung. 2023 However, the field of super-resolution in time series analysis remains relatively
2

---

Published in Transactions on Machine Learning Research (11/2024)
unexplored. This technique, crucial for enhancing signal quality and detail, can significantly benefit sensing applications. For example, in wearable sensors, super-resolution can extend battery life and reduce storage needs by enabling post-processing enhancement of data resolution instead of continuous high-frequency sampling. This approach not only conserves resources but also provides detailed insights for precise tasks such as health monitoring.
2.1.1 Decomposition Based Methods
Decomposition--assed methods have emerged promising techniques for time series analysis by separating data into interpretable components such as trend and residual terms, which enhances the ability to capture both short-term and long-term patterns (Wu et al.] [021]] Wang et al. 2024 Zhu et al. 2023 Cao et al.,
2023
Hu et al. 2023). For instance, DRCNN (Zhu et al. 2023) usee a multi-rcsolution approach with
convolutional kernels to improve forecasting accuracy across different time scales. Similarly, MICN (Wang lt al. 2023 combines multi-scale local and global context modeling through convolution and isometric convolutions, achioving significant improvements long-term forccasting tasks. TEMPO (Cao et al.) 2023 exlends Transformer-basee models by introducing a prompt-based generalive pre-training architecture that decomposes time series into trend, scasonal, and residual components and establishes foundational model for time series forecasting.
Despite the success of the aforementioned decomposition--asse methods, they have limitations in handling the dynamic and evolving nature of non-stationary time series data. For instance, DRCNN's focus on fixed decomposition into residual and trend terms may fail to capture complex temporal shifts over time, while MICN, though addressing both short-term and long-term patterns, relies on pre-defined convolutional kernels that lack adaptability to dynamically changing signals. To leverage the benefits of decomposition while incorporating adaptability, our proposed Ada WaveNet introduces a novel adaptive wavelet-based lifting scheme that dynamically learns wavelet coelficients through end-to-end training.
2.2 Non-Stationarity-Ennaee Ennanced Models
Recent developments in time series analysis have started addressing non-stationarity issues (Liu et al., 2022b; Zhou et al. 2022b Liu et al. 2023b|c). For instance, Liu et al. (202bb) introduced Non-stationary Transformers with strategies like Series Stationarization and De-slalionary Allention to slandardize signal statistics over time. Zhou et all (2022b) employed frequency domain-enhanced attentions in Transformers, incorporating Fourier and wavelet transform-basse techniques. Additionally, Liu et al. (2023b) integrated Koopman operator theory for analyzing non-linear dynamical systems by transforming signals into a linear, high-dimensional space. While these methods effectively model stat ionarity, they exhibit. limitations such as the need for manually tuned filters in FEDformer and extensive computations for long-term signals or high-dimensional projections.
Recent ad advanceeents in adaptive wavelet-based models, such the Adaptive Multi-Scale Wavelet Neural Network (AMSW-NN) (Ouyang et al. 2021), have demonstrated the effectiveness of combining multi-scale convolutional neural networks with depthwise convolutions. However, AMSW-NN primarily focuses on generating candidate frequency decompositions without explicitly considering inter-channel relationships or dynamic adaptability at different scales. To address these gaps, we propose the Adaptive Wavelet Network (Ada Wave:Ne)). Ada WaveNet utilizes a novel lifting scheme that learns adaptive wavelet coefficients through end-to-en tod training and incorporates a grouped linear module for efficient trend processing. This dual-residual approach dynamically captures both fine-gra fine-grained and broad patterns, which enhances adaptability and performance across diverse time scries datascts.
3 Background
Non-stationary time series data often contain transient behaviors and changing frequencies, which makes traditional mothods that assume constant statistical propertics less effective. Wavelet analysis addresses this by enabling multi-resolution decomposition, where each scale handles different frequency bands over time. For instance, high-frequency components can capture rapid changes or anomalies that occur over a short.
3

---

Published in Transactions on Machine Learning Research (11/2024)
duration; whereas low-frequency components capture long-term trends or patterns that evolve more gradually. Thus, wavelet transforms provide a natural way to represent non-stationary data by focusing on both time-localized features and frequency dynamics, which makes them particularly useful for time series where the underlying statistical properties are not consistent.
This section reviews the wavelet transform and the lifting scheme (Sweldens) 1998) concepts to provide foundational knowledge essential for understanding the proposed method.
3.1 Wavelet Transform
The wavelet transform is a versatile tool for analyzing signals across multiple scales, simultaneously capturing both frequency and temporal information. Unlike the Fourier transform, which focuses exclusively on the frequency domain, the wavelet transform enables a time-frequency analysis by decomposing a signal into components that reveal its structure at different resolutions. This ability is crucial for handling non- stationary time series, which often contain short-term fluctuations (high-frequency) alongside long-term trends (low-frequency). Given a signal f. the discrete wavelet transform decomposes it into the following form:
f(x) = (f = Σ Wi.Veis())
(1)
1
1.j
In the equation, (),(() v (2 represents the wavelets at different scales i and translations j, with w (f,((())) denoling the wavelet coefficients.
The wavelet transformation provides an extensive capability for analyzing the changing dynamics and non-stationarity in time series data, which shows advances compared to the Fourier transform's 's frequency- centric approach. Traditional wavelet bases, such as Haar (Haar 1909). Daubechies (Daubechies) 1988). and Biorthogonal (Cohen et al., 1992) wavelets, have been widely used in various real-world applications. However, the selection of an optimal wavelet basis remains challenge, the effectiveness of different bases can vary considerably depending the specific characteristies and strueture of the real-world time series data being analyzed.
3.2 Lifting Scheme
The lifting scheme, also known the second-generation wavelet approach, was introduced by Sweldens (1998) to enhance the flexibility and adaptability of wavelet transforms. Differing from traditional wavelets built from dilations and translations of single function, the lifting scheme construcis wavelets in the spatial domain using a series of simple operations. This approach preserves essential wavelet properties while offering greater adaptability to specific signal charactoristics.
The scheme processes an input signal x to segregale it into approximation (c) and detail (d) sub-bands, which achieves multi-resolution analysis through three main stages: split, update, and predict. This method enables the crcation of custom wavelets that can be more efficiently computed and better adapted to irregular data structures or non-standard sampling grids. By doing so, the lifting scheme provides an efficient route to achieve similar mathematical properties and practical results as traditional wavelets, which makes it particularly valuable for various signal processing applications.
The procedures of the lifting scheme can be considered as:
Split: The input signal is divided into two non-overlapping components: the even (xx) and odd (x.) components, denoted as [n[n] xr [2[m] and [m] =22m+11]
+ 1].
Update: This stage separates the signal in the frequency domain to generate the approximation c. An update operator U.) is applicd to sequence of neighboring odd polyphase samples, yiclding c(n) [x[n]] + UU= [n]).
Predict: Given the correlation between Ia and To a predictor P(.) is developed for one partition based on the other. The detail sub-band d, is computed as the prediction residual d[n] = xo[n] - P(c (m]).
The lifting scheme improves the flexibility of wavelet transformations by allowing for data-driven adaptation of wavelet coefficients, which makes more suitable for analyzing non-periodic and intricate signals frequently
4

---

Published in Transactions on Machine Learning Research (11/2024)
encountered in real-world applications. Also, in the proposed Ada WaveNet architecture (Section 4 the lifting scheme is crucial for enabling a hierarchical decomposition of the time series data, where the signal is split into coarser approximations and finer details at each level. This allows the model to capture both local fluctuations and global trends in the data and model the non-stationary characteristics of time scries.
4 Adaptive Wavelet Network
We propose an Adaptive Wavelet Network (Ada WaveNed), which comprises time series decomposition module, stacked adaptive wavelet (Ada Ware) blocks based the lifting scheme (Sweldens 1998 and a grouped linear module. Figure | illustrates the overall framework of our method. We denote the input sequence as Tinput E RCxL and the target model output as Tyred E RCXL, The target output can represent various terms depending on the task, such as future sequences for the forecasting task or completed signals for the imputation task. The decomposition module processes the time series data into seasonal (x,) and trend (Itrend) components. The Ada Wave blocks then transform 2s into low-rank approximation sr and wavelet coelficients ej at different levels 1. The channel-wise attention layer models the intermediate x² across channels predict the targeted low-rank approximation at. We reconstruct the predicted seasonal phase i, from C and i' using inverse adaptive wavelet (InvAda Wave) blocks. The trend component Ftrend often exhibits alignment issues and discrepancies across variates, and we employ grouped linear module that applies distinct linear heads to different channel groups, to enhance the quality of trend phase predictions. The network's s final output the sum of f, and Itrend.
Ada WaveNet offers several advantages, including multi-scale processing to mitigate non-stationary issues and data-driven approach to learn wavelet coefficients through the lifting scheme adaptively. This adaptability is a key aspect of our proposed method. Additionally, the Ada Wave and InvAda Wave blocks, based convolutional layers, provide ionpulational elficiency compared to the prior sell-atiention-baet implementation of the wavelet transform (Zhou et al.) 2022b). The grouped linear module further improves the modeling of the trend component, which tackles the discrepancies across channels or signal variates.
In the following subsections, we detail the proposed blocks, including the time series decomposition, Ada WaveNet, and Grouped Linear Module.
4.1 Time Series Decomposition
We use the additive time series decompasition method (Hamilton) 2020: Wu et al. 2021 to separate the time scries sequences into their scasonal and trend components:
Tinput = Is + Ftrend
(2)
In this equation, the trend component (Itrend) shows the overall direction and long- aan-term movements of the data over time. It is often calculated using moving average, which smooths out short-term fluctuations to highlight longer-term trends. The seasonal component (Ix)) captures repeating patterns that occur within a specific period and helps identify systematic variations in the data.
To perform the decomposition, we first apply a moving average to the time series to estimate the trend component. Once the trend is identified, we subtract it from the original time series to isolate the seasonal component. By separating the data in this way, we can better understand and analyze the different scales of variation present in the time series. This decomposition is the first step in our multi-scale analysis framework.
4.2 Adaptive Wavelet Block
Given the seasonal component 3s of a time series, we apply Lifting Wavelet Transform (LWT) using Convolutional Neural Networks (CNNs) to refine features at various levels of gramularity iteratively. These processes transform the seasonal component at level 1 - into a more refined level 1, denoted as ai and generate the corresponding detail coefficients, c'.
Splitting Step: The input 2 (initially zQ. = =x)) is split into odd and even indexed components:
él = dd= [2 WEN
(3)
5

---

Published in Transactions on Machine Learning Research (11/2024)
Linear
pped
Clustering
Linear
Steend
Mtrand
Grouped
Linear Module
Trand
Decomp
Seasonal
Zinget
Spsit
CWA
<N> AdeWave Blocks
<N> InvedaWave Blocks
Decamp
Decomposition
Time
Splöt
Index Split
CWAA
Attention Layer
Channel-wise
CoNsy
Convolutional Layers
Cone
Transposed Convolutional
Layers
Figure 1: Illustration of the Ada WaweNel framework for time series analysis. The input sequence Timput undergoes decomposition into trend (Itrena) and seasonal (xx,) components. The trend component is processed through a clustcring algorithm followed by grouped linear modulc produce a refined trend prediction ftrnnd- Concurrently, the seasonal component is processed through stacked Ada Wove blocks, employing index splitting, convolutional layers, and channel-wise attention layer to capture multi-scale features and generate low-rank approximation $/. This is followed by inverse Ada Wave blocks that reconstruct the seasonal prediction &.. The final predicted output Tpred is obtained by summing the predicted seasonal and trend components.
d d = (2i ViEN
(4)
Convolutional Kernel-based Prediction and Update Steps: Inspired by Huang & Fang (002)). we employ convolution operations as the wavelet filters in each split subset to extract approximations and coefficients. The learnable 1D convolution kernels are considered the ideal basis of the lifting scheme in study. This operation can be represented
e = (W e' + 1()
(5)
0² = o of + b.))
(6)
where * denotes the convolution operation, W and W are the convolutional filter weights, and by and 6/ are the biases for the even and odd components level 1, respectively.
In the prediction step, because, in many signals, there is smooth transition between consecutive samples. The prediction step aims to estimate the finer details of the signal (odd indexed components) using smoothed version (even indexed components) to capture the high-frequency variations by computing the detail coefficients c!:
the even-indexed components to estimate the odd-indexed components. This is
d - f=d-oo(W) * b,,))
Then, the update step utilizes these detail coefficients to refine the even indexed components:
cd = el + to(W! df + bb))
These steps iteratively improve the signal representation in capturing the overall trend and the intricate details within the data.

---

Published in Transactions on Machine Learning Research (11/2024)
4.3 Channel-wise Attention for Approximation Projection
To enhance the feature representation of the seasonal component 2 specifically at the final level of the Ada Wave blocks, self-attention (SA) mechanism is employed, inspired by Liu et al. (2023a). The self- attention structure is applicd to x²" where N denotes the final level of decomposition. This computation after the last layer of decomposition ensures efficient and focused refinement of the feature map of the low-rank approximation the seasonal component, as the length of the sequences after N blocks of Ada Wave blocks becomes (L/(2*)). The channel- wise attention mechanism operates on the channels of x to refine its approximation, to project the processed seasonal component onto the targeted sequences as a. Importantly, during this process, the detail coefficients (c*) remain unchanged.
Focusing the whannel-wise attention mechanism at the final decomposition layer is both computationally efficient and effective in capturing the essential characteristies of the time series. It allows the model to emphasize the global contextual information, which is crucial for the accurate representations of the seasonal components in complex time series data.
4.4 Inverse Adaptive Wavelet Blocks
To reconstruct the original seasonal component. 4, from its refined representation s obtained after predicted approximation &N, we utilize an inverse process facilitated by Convolutional Transpose Networks. This inverse procedure employs transposed convolutional layers to upscale the feature maps and merge the detail coefficients with the upsampled seasonal components iteratively.
The inverse of the combining step involves an element-wise subtraction of the detail coefficients from the seasonal component at the current. level l:
c'=
Inverse Update and Prediction Steps: The transposed convolution operations are applied to refine the split components and to estimate the original even indexed components:
e' = el ow d +d++
(10)
The original odd indexed components are reconstructed by adding the predicted detail coefficients:
d= d' + o é + b)T))
(11)
where WT. and WLT denote the transposed convolutional filter weights, and b and 6/.T are the corre- sponding biases for the inverse update and prediction operations, respectively.
Finally, the odd and even indexed components are interleaved to reconstruct the seasonal component at the previous level - 1:
- =e' WiN
(12)
$ = VieN
(13)
4.5 Grouped Linear Module
We process the trend component. Itrend using two-step approach: clustering the channels and applying distinct linear projections based on the clustering labels. This method effectively captures and models the long-term progression of the trend component.
In the first step, we group the channels of Strend using a clustering algorithm, specifically K-means. This clustering step helps to identify patterns and group similar channels together. For example, the diagram as in Figure [] channels are clustered into different groups according to their characteristics. This step is crucial for understanding the temporal patterns within the trend data that might indicate different states across variates.
7

---

Published in Transactions on Machine Learning Research (11/2024)
AdaWoveNet Forecasting (MSE)
iTansformer
0.331
TimesNet
DLinear
PutchTST
Non-Stationary Transformer
FEDformer
0,500
0.200 0.200
0.071
0.071
Super-resclution (MSE)
Imputation (MSE)
Figure 2: Comparison of model performances in forecasting, imputation, and super-resolution.
Following the clustering, each group of channels is connected with a linear projection layer. This step ensures that the model can apply specific linear projections to different clusters of the trend data. This design aims to help the model accurately capture different patterns within the trend data.
This dual approach of clustering followed by grouped linear transformations particularly effective in dealing with complex time series data. By initially clustering the trend components into different groups, the model can recognize and adapt different underlying patterns. Subsequent application of distinct linear projections to each cluster further enhances the model's 's ability to represent and forecast the trend component accurately.
5 Experiments
We conduct experiments on different time series analysis tasks to evaluate the proposed Ada WaveNet, including forecasting, imputation, and the newly proposed super-resolution tasks.
Ada WaveNet extensively benchmarked against established models from recent literature. For models related to wavelet and frequency domain enhancements, comparisons include FreTS (Yi et al. 2023). FiLM (Zoou et al. 2022a), TimesNet (Wu et al.) 2022), and FFDformer (Zhou et al. 2022b). Additionally, models previously recognized for state-of-tho-art (SOTA) performances, such as iTransformer (Liu et al. 2023a), DLinear [Zeng et al. 2023), and PatchTST (Nie et al. 2022), are included as experimental baselines. The Non-stationary Transformer (Stationary) (Liu et al. 2022b), known for addressing non-stationary issues in time scries, also featured for comparison.
Figure 2 presents the aggregated results across the forecasting, imputation, and super-resolution tasks. The results indicate that our proposed Ada WaveNet method achieves SOTA performance in all three areas of time series analysis.
5.1 Time Series Forecasting
Forecasting essential time series applications, such as weather, traffic, exchange rate, etc. In this section, we conduct extensive experiments to evaluate the forecasting performance of the proposed Ada WaweNet on varying-domain datascts. Following prior studics such as iTransformer (Liu al.) 2023a and TimesNet (Wu et al., 2022), we adopt a long-term forecasting setting with datasets including traffic, electricity (ECL), exchange rate, weather, solar energy, and electricity transformer temperature (ETT), For each dataset, we set the predicting length Lp with (96, 192, 336, 720) with the inputting observation length equal to the predicting length.
The performances evaluations are shown in Table ! The proposed Ada WaveNet achieves promising performances both MSE and MAE across various datasets. In particular, compared to the prior frequency domain-enhanced methods, Ada WaveNet outperforms 7.7% in MSE and 6.2% in MAE when compared the best performances in all these methods. For the SOTA methods such iTransformer (Lu et. al. 2023a and
8

---

Published in Transactions on Machine Learning Research (11/2024)
Table 1: Forecasting task. The prediction lengths for all datasets are established at {96, 192, 336, 720], with the past sequence length matching the prediction lengths. Evaluation metrics include Mean Squared Error (MSE) and Mean Absolute Error (MAE). The lowest MSE is indicated in bold red, while the second lowest is underlined in blue.
Model (Year)
Ada Wa (Ours)
Prandormer
Timas
DLinear (2025
PatchTSS (2022
Stacionary 26222h
FILM
(2025a)
(2025)
2022
21022
(20225)
Pred Length
MSB
MAB
MES
MAA
M13B
MAA
M3B
M.A
MBS
M.A.
MSE
MAE
MSE
MAB
MSB MAA
MSB
MAB
86
0.146
0.248
0:1.51
0.977
0. 1.165
0.208
0. 0.265
0.167
0.209
0.193
0.306
192
0:158
0.260
0.156
0.164
0.261
0.185
0.248
0:174
0.275
0.158
0.254
0.183
0.295
262
0.361
0.205
0.326
BCL
950
0.171
0.268
0.177
0.274
0.202
0.280
0.196
0.301
0.176
0.278
0.176
0.278
0.207
0.310
0. 0.281
0.577
0.224
0.330
720
0.196
0.202
0.201
0. 0.00
0.226
0.322
0.215
0.317
0.201
0.300
0.207
0.310
0.214
0.317
0.289
0.386
0.242
0.357
Avg.
0.108
0.267
0.171
0. 0.208
0.180
0.267
0. 0.100
0.254
0.100
0.280
0.181
0.378
0.100
0.298
0.309
0.394
0:217
0.333
86
0.100
0.215
0.177
0.173
0:231
0.172
0.220
0.196
0.254
0.176
0.317
0.107
0.238
0.163
0.234
0:210
0.295
192
0.203
0.245
0.313
0.50
0.301
0.268
0.222 0.266
0.327
0. 8.286
0.212
0 2.255
0.261
0.298
0.230
0. 0.266
0:265
0.341
Weather
336
0.248
0.286
0.255
D 1.5
0.255
0.313
0. 0.286 0.314
0 0.362
0.313
0.236
0 0.2
0.316
0.330
0.265
0. 285
0.316
0.350
720
0.913
0.588
0.325
0.345
02320
0.356
0.375 0.373
0.915
0.599
0.321
0. 0.339
0.490
0.378
0.922
0. 0.558
0.372
0.403
Aeg.
0.288
0.271
0.249
0.276
0.288
0.202
0. 0.264 0.293
0.261
0. 0.303
0.241
0. 0.27
0.294
0.311
0.268
0.22 0.258
0.253
0.356
56
4.7
20
0:300
0:271
0.325
0:333
0.002 0.310
0.052
0.397
0.544
0 369
0.010
0.341
0.647 0. 353
0.565
0.363
102
0 401
2.
0.400
0. 0.280
0.514
0.329
0 595
0.321
0 622
0.354
0.547
0. 361
0.621
0.348
0. 462
0. 302
0.586
0.366
Traffic
336
D-407
0.281
0.415
956
0.527
0.341
0.611
0.330
D 624
0 0.355
0-555
0. 368
0.628
0.346
0 847
0. 3.305
0.507
0.368
720
0:433
0:207
0.428
0.901
0.339 0.546
0.344 0.026
0.064
0.408
0.608
0.393
0.647
0.384
0.485
0.521
0.015
0.390
Avg.
0.410
288
0.528
0.841
0.329
0.641
0.579
0,502
0. 0.370
0.027
0.354
6.610 0.525
0.000
0.372
86
0.086
0:201
0.087
0.088 0:214
0.113 0.265
6 055
0:227
0.093
0-212
0.150
0.265
0.166 0.307
0:147
0:278
102
188
0.197
0.347
0.361
0.189
0.188
0.312
0.261
0.375
0.224
0.340
0:200
Exchange
336
0.395
0.523
0.391
0.477
0-415 0.328
0.581
0.467
0.502
Avg.
0.613
0
0.065
0.608
0.455 0.498
0.524
192
0.207
0.216
0.251
0
0.
0
0.244
0.346
Soloc
336
0.216
0.200
0.220
0.297
0.255
0
1.325
0.214
373
0.220
0.372
0.251
0.352
720
0:214
0.2
0.223
0.255
0.306
24245 0.257
2.2
0.276
291
282
0.252
0.355
Avg.
0,209
0.262
0.218
0. 2727
0.254
0.294
0.244 0.250
0.264
0.581
0.211
0.2
0.228
0.208
0.291
0.502
0.248
0.349
86
0.396
D.120
0. 131
0.413
0.400
0.420
D:3K5
0.451
10
0.358
0.505
0.387
389
0.377
0.118
192
0.437
0.431
0.463
0.466
0.467
0.401
0. 0.064 0.526
0.430
0.448
0.426
0 425
0.608
0.676
0.437
0.430
0.422
0.461
ETT51
88G
D 4.4
0.441
0.480
0.475
0.501
0.403
0. 0.009
0.495
0.437
0.403
0.450
0.456
0.064
0.668
0.469
0.45G
0.451
0.453
730
0.510
0-407
0.600
0. 565
0.602
0.573
0. 0.708 0.615
0.492
0.510
0.505
0.470
0.713
0.615
0. 009
0.501
0.497
0.490
Avg.
0.644
0.441
0.495
0.46.
0.496
0.489
0.545
0.514
0,436
0.439
0.444
0. 0.439
0.659
0.868
0.445
0.446
0.437
0.455
56
D.326
0.366
0.354
0.381
0.343
0.376
0. 0.330
0.371
0.345
0.371
0.334
0 3666
0.423
0.415
0.351
0.370
0.379
0.418
192
D-3335
0.370
0.355
0.384
0.361
0.394
0.386
0.106
0:342
0.368
0.338
0. 0.368
0-166
0.416
0.388
0-101
0.415
0.128
ETTal
330
0.375
0.334
0.884
0.406
0.393
0.400
0.403
0.422
0:370
0. 3.3
0. 0.307
0.302
0.556
0.426
0.400
0. 0.420
0.417
0.491
720
D. 459
425
0. 4.4
0. 4.4
0.466
0.457
0 1.406
0.456
0.420
0.422
0.456
0 0.447
0.616
0.531
0. 0.447
0. 0.488
0.484
0.478
Avrg
D.369
0.385
0.405
0.392
0.409
0. 0.596
0.414
0.369
0.387
0.374
0. 0.394
0.472 0.515
0. 1.597
0.408
0.424
0.439
Tst. Cosnnt
18
21
5
0
5
3
5
2
0
PStchTST (Nic et al. 2022), the evaluation results also suggest that our proposed method outperforms them in the forecasting task.
5.2 Time Series Imputation
During time series data collection, various factors can disrupt continuous observation and monitoring and lead to missing values within the dataset. For example, factors such as the malfunction of the devices and interference of signals can all trigger data quality issues, e.g., missing observation. Therefore, in this study, we also investigate the capability of the proposed method in time series imputation tasks. Following the evaluation strategy as Wu et al. (2022), we conduct extensive experiments on the datascts with controlled masking rales under a random missing setting. We further extend the experimental strategy by introducing the extended missingness. In (Wu et al. 2022), the missingness was crafted by randomly masking timestamps with a certain ratio that simulates sporadic data loss; whereas the extended missingness masks contiguous subsequences of the original signals across all channels, which emulates the prolonged periods of missing data. Refer to A Appendix| A.2 for the detailed descriptions of two masking methods. In this experiment, we examine the proposed method and the baselines on the weather and electricity data. Also, considering the common missingness of sensing data of the wearable sensors Xu et al. (2022). the experiments cover biobchavioral datasets, including 12-lead electrocardiogram (ECG) from the PTB-XI, dataset Wagner et. al. (2020), and cletroenccophalogram (EEG) from the SDeep-EDFE datasct Kemp et al. (2000).
Table 2 shows the imputation performance for each dataset. The observed results show the two different masking strategies create different levels of challenges for the imputation tasks, The extended masking task introduces larger errors for all the models. For both imputations, our proposed method shows competitive performances on the PTB-XL and Sleep-EDFE datasets, where the sequence lengths are siguificantly longer than the ECL and weather datasets. TimesNet (Wu et al. 2022) outperforms all the other models in ECL and Weather random imputation tasks; whereas Ada WaveNet shows substantial improvement compared to the other basclines on ECL and Weather data for the random masked imputation task. Visualization of imputed examples can be found in Appendix C.2
9

---

Published in Transactions on Machine Learning Research (11/2024)
Table 2: Impulation lask. Experiments are conducted on two lypes of imputation - random and extended. In each case, we mask 11.,5% 25%, 37.5%, 50%} of time points or segments randomly from the original sequences. For the ECL and Weather datascts, sequence lengths are set to 96, while for the PTB-XL and Sleep-EDFF datasets, the lengths are 1000 and 3000, respectively. Evaluation metrics include Mean Squared Error (MSE) and Mean Absolute Error (MAE). The lowest MSE is marked bold red, and the second lowest. is underlined in blue.
Model
Nev
Timanres
StAtIGnary
(Year)
(Quza)
XTAE
0.093
0.085
0,375
0.108
0.124
0.256
0
0.068 0:213
0:141 0.279
0.110
0.236
0.102
2.22
0.118
0.241
0.186
0.266
0.5
0.122
0.234
0.152
0.286
0.147
0
0.102
0:173
0.303
0.114
0.240
0.108
0.228
0.135
0.255
0.158
0.284
0.100
0.215
0.117
0.245
0 230
0.095 0.212
0.136
0.313
0.000
0.221
0.100
0.119
0.240
0.180
0.260
0.125
0.083
0.039
0.029
0.082
0.023
0 044
0.025 0.045
0.089
0.084
0.025
0.086
0.027
0.061
0.081
0.068
0.044
0,110
C/t Sloep- Slep-EDFF EDDE PTB X1 West Randors
0.375
0.26
0.041
0.050
0.077
0.098
0.046
0.055
0.088
0.105
0.058
0.044
0 0.120
0.080
0.0331 0.057 7
0.020 0.052
0.048
0.057
0.109
0.117
0. 0:051
0. 0.040
0.079
0.086
0.053
0.020
0.058
0.62
0.042
0.055
0.080
0.111
0. 0.107
0.062
0.160
0.231
0.5
0.052
0.129
0.068
0.136
0.072
0.142
0.0354
0.082
0.086
0.158
0 0.058
0.107
0.057
0.00
0.073
0.135
0.183
0.311
Ave.
0.047
0.091
0.050
0.097
0.030
0 000
0.035
0.054
0.053
0.110
0 0.044
0.057
0,002
0.659
0,050
0.100
0. 0.009
0.205
0.125
0:017
0.023
0.2
0.014
0.020
0 040
0.025
0.2
0.000
021
0.03M
0.33
0.028
00000
0 DMM
0.0M
0.25
0.024
0 0.087
0.084
0.055
0.048
0 052
0.030
0.041
0.054
0.071
0 03555
0.046
0:033
0.045
0.0:0
0.052
0 0.0
0.054
0.576
0.029
0. 0.099
0.000
0.089
0.052
0 0.068
0:04
0.067
0.088
0.058
0.045
0.089
0.049
0.062
0.074
0.089
0.5
0.044
0.058
0.077
0.085
0.083
0.077
0.041
0.050
0.973
0.087
0.067
0.099
0.057
0.000
0.063
0.075
0. 0.091
0.103
Aeg.
0.
0.0G3
0.068
0.046
0.060
0.052
0.039
1.001
0.057 0.045
0. 0.077
0:038
0.375
0.037
0.048
0.062
002
0.050
0.067
0.056
0.071
0
0.070
064
0.054
0.065
0
0.084
0.5
0.013
0.055
0.070
0.058
0 073
0.052
0.074
0.068
0.085
05057
0.069
0.081
095
0.063
0.075
0 082
0.102
Avy.
0.034
0.045
0.047
0.059
0 056
0.077
0.065
0.051
0.056
0.045
0.055
0.064
079
0.049
0.063
0.062
0.080
0.125
0.098
0.207
0.108
0.217
0.130
0. 0.217
0.114
0.227
0.168
0.276
0.150
0.228
0.115
236
0.126
0.218
0.152
0.234
0.25
0.104
0.207
0.138
0.215
0.128
0. 0.222
0.117
0:225
0.138
0.250
0.133
0.251
0.124
232
0.127
0.228
0.155
0.261
0,376
0.121
0.228
0.138
0.241
0.160
0. 0.2b7
0.159
0.245
0.194
0.292
0. 0.165
0.283
0.154
0.245
0.151
0.255
0. 0.181
0.270
0.5
0.126
0.230
0.143
0.241
0.165
0 262
0.140
0.246
0.686
0.266
0 0.172 0.263
0.1.37
0.156
0.256
0. 0.177
0.293
Avg.
0.112
0.218
0.127
9.999
0.146
D 2410
0.128
0.235
0.173
0.266
0.147 0.207
0.128
237
0.100
0-938
0. 161
0.370
Slapp- Slxp-EDFF EDFE PTB Xt Weat her
0.125
0.26
0.084
0.0BY
0.127
0.091
B.0Y4
0.062
0.115
1.1
0.072
0.000
0 0.112
0. 185
0.216
0.056 0.118
0:257
0.072
0.007
0.149
0.200
0 0.082
0.070
0.108
0.118
0.039
0.101
0.222
0.101
0.097
0.143
0:135
0.195
0.164
0.10Y
0.218
Extended
0.375
0.5
0.307
0.088
0.154
0.170
0.102
0.119
0.139
135
0.115
0.127
0.
0. 171
0.113
0.101 0.142
0.157
0.135
0.121
0.184
0:202
0.090
0:1:2 0.153
0.188
0.113
0.125
0.150
0.191
0.116
0.128
0:181
0.167
0. 0.177
0. 1h6
0.231
0.229
Avg.
0.089
0.138
0.094
0 132
0.101
0 180
0.129
0 0.169
0.160
0 090022
0.229
0.108
192
0.110
0.157
0.
0.219
0-002
0.050
0.6
0.050
0.122
0.135
0. 0.186
0.141
0.110
0,126
0.105
0.132
0.118
0.147
0.009
0.088
0.088
0.107
0.078
106
0.076
0.100
0.004
0.123
0.086
0.104
0.075
0.0
0.083
0.105
0.
0.121
0.125
0.088
0.087
0.082
0.112
0.070
0. 080
0.083
0.104
0.102
0.154
0.079
0.096
0.115
0.149
0,089
0.115
0.103
0.140
0.25
0.082
0.105
0.114
0.146
0.085
0.1:0
0.106
0.133
0.117
0.158
0.096
0.120
0.125
0.164
0.105
0.158
0.116
0.155
0,376
0.104
0.137
0.119
0.164
0.110
0.142
0.124
0.155
0.139
0.188
0.104
0.182
0.133
0.177
0.122
0.160
0.142
0.197
0.5
0.109
0.141
0.131
0.188
0.121
0 1.1
0.142
0:191
0.155
0.207
0.197
0.163
0:147
0.196
0.134
0.180
0.139
0.195
Ave..
0.090
0.118
0.114
0.150
0.097
191
0.114
0.146
0.126
0.172
0.102
0.126
0.130
0.172
0.112
0.148
0.125
0.172
Table 3: Super-resolution task. Super-resolutio upsampling ratios are set at {2, 5, 10). Experimentally, sequence lengths are fixed at 200 for ETTml, ETThl, and Traffic datasets, and at 1000, 3000, and 960 for PTB-XL, Sleep-EDFE, and CLAS datasets, respectively. Evaluation metrics include Mean Squared Error (MSE) and Mean Absolute Error (MAE), with all results being averages over 4 masking ratios. The lowest MSE is highlighted in bold red, while the second lowest is underlined in blue.
Modal
Aaveaee
Hrmnstormer (2025a)
FreTa (2028
TimocsNnt (022)
DLinese (2925)
FutebT5T (2022]
Stationacy (20226)
FILM 2022
PElllormer
(Year)
(Oues)
20220
51t Hosto
MAK
MAR
MAB
MARK
MAR
MEE.
MAR
MCB
STAR
MEP
KEAB.
MAR
MSR
MAR
MSD.
MAR
0.016
0.088
0.021
0.097
0.112
0.102
0-027
0.000
0.044
0.125
0.033
0.1.1
0.053
0.104
0.051
0.136
GAUST
0.109
ETTml
10
0.035
0.058
0.161
0.101
0.086
0.036
0.173
0.110
0.077
0.040
0.179
0.118
0.054
0.0:9
0.112
0.147
0.070
0.087
0.160
0.104
0.077
0.057
0.182
0.148
0.062
0.098
0.106
0.117
0.047
0.070
0.322
0.172
0.081
0.058
0.158
0.100
Avg.
0.036
0.112
0.041
0.127
0.047
0.
0.067
0.159
9.056
0:146
0:044
0.129
0.056
1-155
0.008
0.152
0.038
011
0.040
0.125
0.043
0.116
0.051
0.123
0.003
0.134
0.064
0.128
0.061
0.150
0:045
0.12
0:03Y
0.100
RTTH:
0:178
0.093
0.108
0.270
0.107
0:202
0.291
0.210
0.
0. 190
000
0.200
0.287
0.102
0.187
0. 202
0.287
0.128
0:202
0. 0.233
0.110
0.196
0:292
0.217
0.168
0.000
0.266
164
0.189
0.100
2.2
2924
0.107
0.2001
0. 2.20
297
Avg.
0.109
0.192
0.118
0.200
0.111
0
0. 2.205
0.151
0.120
0.212
0.108
197
0.112
202
0.115
0. 20.
0.107
0.090
0.127
0.110
0.135
0.125
0.141
136
0.124 0.11
0.157
0.150
0.102
1.14
0.156
Tratllo
0.210
0.322
0.281
0.197
0.320
0.208
0.290
0.109
0.
0.226 0.218
0.307
0.809
0.241
0.221
0.922
0.350
0.230
0.337
0.219 0.197
0.205
0.358
0.346
0.250
0.401
0.277
262
3777
0.256
0.863
0.223
0.258
0.007
0.016
0.007 0.015
0.000
0.015
0.011
0.014 0.020
0.012
0.025
0.008
0.010
0.022
PTB-XL
0.048
0.017
0.048
0.015
0.042
0.042
0.039
0.0ST
0.050
0.031
0.020
0.026
0.002
0.022
0.034
0.011
0.133
0.0
0.150
0.031
0.020 0.0
0.021
0.1G5
SDep-EDER
0.022
0.141
0.036
0.168
0
T6E
0.035
0.170
0.035
0.162
0.045
0.184
0.039
0.3
0.049
0.179
0.027
0.150
0.177
0 036
170
0.044
0.181
0.048
190
0.194
0 0.064
0.212
0.053
0.20
0.061
0. 2.292
0.020
0.144
0.031
0.162
0 122
0 164
0.635
0.173
0.055
0.1
0.087
0.177
0.047
0.187
0.0SY
0.5
0.049
0.186
0.014
0.048
0.057
0. 0.023
0.055
0.637
0.083
0.022
65
0.038
0.074
0.014 0.081
0.630
0.071
0.028
0.062
CLAS
10
0.034
0.061
0.079
0.101
0.017
0.006
0.081
0.120
0.041
0.062
0.111
0.082
0.045
0.000
0.107
0.089
0.05
0.000
0 0R8
0.114
0.072
0.051
0.096
0.130
0.046
0.066
0.117
0.082
0.052
0.078
0. 103
0.144
0.039
0.062
0.116
0.084
Avg.
0.084
0.076
0.046
0.087
0.0 0.042
0.083
0.086 0.648
0.030
0. 0.065
0.058
0.104
0.002
0.093
0.033
0.100
0.043
0.087
10

---

Published in Transactions on Machine Learning Research (11/2024)
Table 4: The averaged results of model ablation with mean squared error as the evaluation metric. "GL' denotes the Grouped Linear module, 'CWA" indicates Channel-wise Attention, and "AWB' signifies the Ada Ware component. Tasks marked (F) for lorecasting and (T) for extended imputation. The highest. MSE is highlighted in bold red, while the second highest is underlined in blue. Refer to Table in the Appendix for comprehensive results.
w/o
GL
RevIN
CWA
AWB
Weather (F)
0.233
0.240
0.246
0.256
0.255
Traffic (F)
0.415
0.425
0.422
0.519
0.473
ECL (1)
0.112
0.115
0.117
0.122
0.131
PTB-XL (I)
0.069
0.080
0.070
0.069
0.082
5.3 Time Series Super-resolution
This study introduces a benchmark for super-reslution in time series data, which can be crucial for enhancing the detail and quality of data, particularly when dealing with under-sampled or low-resolutinn datasets. The objective of the super-resolttin task is to reconstruct the original high-resolution signal from the down-sampled version and increase the temporal resolution by reconstructing the data at finer granularities. The super-resolution task differs from traditional imputation in that it focuses on upsampling the temporal resolution rather than simply filling in missing values. Specifically, requires the model to infer the higher- frequency components that are missing due to down-sampling. For instance, reconstructing a 100 Hz ECG signal from 10, 20, 50 Hz inputs requires generating fine-grained details that are not directly observable in the low-resolution input.
In this context, we conduct super-resolution experiments on the ETT, traffic, ECG (PTB-XL), EEG (Sleep- EDFE), and electrodermal activity (EDA) datasets from the CLAS study (Markova et al. 2019). We evaluate the performance varions models, including Ada WaveNet, by upscaling the temporal resolution at ratios of 2, 5, 10. Our results demonstrate that Ada WaveNet excels at reconstructing high-resolution signals, capturing both low- and high-froo fregufrre components, which leads superior performance compared to other baseline models.
Table shows the evaluation results for the super-resolution task. The proposed method shows competitive results compared to the baseline methods, especially on datasets such as Traffic, EDFE, and CLAS, We credit the observed improvement to the multi-scale analysis inside the model architecture. In this experiment, we find the frequency domain enhanced methods including FreTS, TimesNet, and FEDformer generally provide better results in strongly periodic data. For example, the first three places of averaged super-resolution reconstruction performances on ECG signals in the PTB-XL dataset are TimesNet, Ada WaveNet, and FreTS, respectively. The channel-wisc attention also brings ad advantages models, such as Ada WaweNet and (Transformer, in reconstructing the nigh-dimensiooal traffic data.
6
Discussion
In this section, we discuss the model ablations and efficiency. Also, we showcase an example of the multi-scale analysis performed by the proposed method, Further, we discuss the limitations of this work.
6.1 Ablation Study
An ablation study was conducted to assess the contribution of individual components within the proposed Ada WaveNet framework to its overall performance. This involved evaluating the model on forecasting tasks and extended imputation tasks with the omission of specific components: the Grouped Lincar Module, reversible instance normalization (RevIn) as described by (Kim el al. 2021), channel-wise Attention inspired by Liu et al. (2023a), and the Ada WaveNet and InvAda WaveNet blocks. The results are presented in Table which indicates that the AWB component significantly enhances performance on the ECL and PTB-XL datasets for imputation tasks. Meanwhile, CWA shows notable efficacy in forecasting tasks. The GL module also demonstrates improved results on the PTB-XL dataset during imputation.
11

---

Published in Transactions on Machine Learning Research (11/2024)
0.4
ETTh1
Memory Fooiprint
0.5
Traffic
Memory Footprint
Stationary 2 43
MSE
GB
5
20
FEDformer 3.9 G6, 318.8 mg
PatchTST 37. GR, 1469-3 mM
Stationary 2. me
0.325-
DLinear mm
0.4
TimesNet G2,
AdaWaveNet 1.5 06, 11
Fr Gn, 1
Transtermer
PatchTST 188 32.mm
ITransformer 75..G 1407
1.96 m.
TimesNet GB 947.2
FEDforeer
AdaWaveNet
02 22,,0 228.0
0.25
100
300
500
700
900
500
1500
2500
3500
4500
Training Time (ms / iter)
Training Time (ms / iter)
Comparison of model elliciency the ETThl and Traffic datasets. The y-axis represents the averaged mean squared error performances across the forecasting and super-resolution tasks; whereas the x-axis represents the model training time for each iteration. The input length is 192 with a batch size of 16.
Figure 3:
6.2 Efficiency Analysis
Efficiency is one of the important concerns in time series analysis. Therefore, we evaluate the efficiency of the proposed method and compare the results with all the baseline models we used in the main experiments. Controlling the input length of the sequences as 96 time points, we select the data samples from the ETThl and Traffic sets, as they have the least (7) and the most (862) numbers of variates. The environment used in this evaluation is AWS g5 instances with Nvidia A10 GPUs. The batch size is fixed as 16. Following the previous studies (Wu al. 2022 Liu et al. 2023a). we evaluate the efficiency of the models aspects of performances (MSE), training speed, and memory footprints.
The comparison results shown in Figure illustrate that Ada WaveNet achieves state-of-the-art performance while maintaining reasonable model efficiency. The DLinear model exhibits the smallest memory footprint and fastest training time among the evaluated models. Ada WaveNet demonstrates comparable training speed and memory usage to the iTransformer; while achieving significant savings in memory and training time compared to established approaches such as PatchTST.
6.3 Wavelet Decomposition
Figure 4 illlustrates the wavelet decomposition process employed by Ada WaveNel in a time series forecasting task. The decomposition splits the seasonal component into layers approximations and detail coefficients. A channel-wisc attention mechanism then applied to forecast future sequence approximations. Each decomposition level incorporates residual connection that integrates the reconstructed signals from the IndAda Wave blocks. This example illustrates the multi-scale analytical capability of the proposed method, which enables the model to extract features across various gramularities. Moreover, the flexibility introduced in the learnable CNN kernels empowers the Ada Ware and InvAda Wave blocks to dynamically adapt to varying input data.
6.4 Limitations
6.4.1
Model Complexity
Ada WaveNel incorporates effective components, such as the grouped linear module and cross-channel attention mechanisms, to model dependencies across similar variates in the trend phase. It also introduces multi-scale capabilities through the Ada Wave blocks. Despite its promising performance, Ada WaveNet is less efficient. compared simpler MLP-based models such DLinear Zeng al. (2023). which potentially limits its applicability in environments where computational resources are constrained real-time analysis is required.
12

---

Published in Transactions on Machine Learning Research (11/2024)
GroundTruth Prediction
25
50
75
100
125
150
175
200
Linear
0
AdvWave Block
InvAdaWave Block
3
Linear
AdaWave Block
InvAdaWave Block
W
Projection
Figure 4: An example of the wavelet decomposition and projection in time series forecasting on ETTh1 data with two layers of Ada Wave and InvAda Wave blocks. z and & represents the the approximation at wavelet level the input and target sequences, respectively. e is the wavelet coefficients decomposed by the Ada Wave blocks.
6.4.2 Generalization to Different Signal Types
Ada WaveNet demonstrates robust performance in forecasting tasks with data types such as weather and solar, as well as extended imputation lasks with electricity and EEG data. However, its capacity generalize across the full spectrum of time series data and various tasks has room be further improved. For example, in tasks involving random imputation of electricity and weather datasets, TimesNet exhibits superior performance. Similarly, iTransformer outperforms Ada WoveNet in traffic forecasting tasks.
7 Conclusion and Future Work
This paper presented Ada WaveNet, novel and efficient. architecture for addressing the challenges of non- stationarity in time series data analysis. Through the integration of adaptive wavelet transformations, Ada WaveNel demonstrates the advantages of modeling multi-scale data representations in time series data. Our extensive experiments are conducted on 10 datasets for forecasting, imputation, and the newly established super-resolution tasks, and the results indicate the effectiveness of our approach.
Future eflorts will focus on exploring and expanding Ada WaveNet and its multi-scale analysis framework to a broader range of tasks, such as classification and anomaly detection. Also, we aim to explore the possibility of using multi-scale analysis in largo-scale pro-training models.
Acknowledgments
This work was supported by National Science Foundation (# 2047296) and National Institute Health (# R01DA059925)
References
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Ane Garela, Angel Conde, Usue Mori, and Jose A Lozano. A review on outlier/anomaly detection in time series data. ACM Computing Surveys (CSUR), 54(3):1 33, 2021.
Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt- based generative pro-trained transformer for time series forecasting. arXiv preprint S133:231000444.
2023.
13

---

Published in Transactions on Machine Learning Research (11/2024)
Albert Cohen, Ingrid Daubechies, and J-C Feauveau. Biorthogonal bases of compactly supported wavelets. Communications on pure and applied mathemalics, 45(5):485-55) 560, 1992.
Ingrid Daubechies. Orthonormal bases of compactly supported wavelets. Communications on pure and applied mathematics, 41(7)900 996, 1988.
Philippe Esling and Carlos Agon. Time-series data mining. ACM Computing Surveys (CSUR), 45(1):1 34, 2012.
Alfred Haar. Zur theorie der orthogonalen funktioonessstemm.. Georg-Angust-Univeess Gottingen., 1909.
James D Hamilton. Time series analysis. Princeton university press, 2020.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 988):1735-1780, 1997.
Jie Hu, Zhanao Hu, Tianrui Li, and Shengdong Du. A contrastive learning based universal representation for time series forecasting. Information Sciences, 635:86 98, 2023.
Hao Huang and Yi Fang. Adaptive wavelet transformer network for 3d shape representation learning. International Conference on Learning Representations, 2021.
Rob Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018.
Bob Kemp, Acilko H Zwinderman, Bert Tuk, Hilbert AC Kamphuisen, and Josefien JL Oberye. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the eeg. JEFE Transactions on Biomedical Engineering, 47(9):1185 1194, 2000.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference Learning Representations, 2021.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Tin. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 95 104, 2018
Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks for action segmentation and detection. In proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 156-165, 2017.
Gen Li and Jason J Jung. Deep lcarning for anomaly detection in multivariate time scries: Approaches, applications, and challenges. Information Pusion, 91:93 102, 2023.
Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.1072) 2023.
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816 5828, 2022a.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881-966 2022b.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long, itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.00 0.6625,, 2023a.
Yong Liu, Chenyu Li, Jianmin Wang, and Mingsheng Long. Koopa: Learning non-stationary time series dynamics with koopman predictors. arXio preprint Xrx::22905. 18808, 2023b.
14

---

Published in Transactions on Machine Learning Research (11/2024)
Zhiding Liu, Mingyue Cheng, Zhi Li, Zhenya Huang, Qi Liu, Yanhu Xie, and Enhong Chen. Adaptive normalization for non-stationary time series forecasting: A temporal slice perspective. In Thirly-seventh Conference on Neural Information Processing Systems, 2023c.
Valentina Markova, Todor Ganchev, and Kalin Kalinkov. Clas: A database for cognitive load, affect and stress recognition. In 2019 International Conference on Biomedical Innovations and Applications (BIA),
pp. 1-4. IEEE, 2019.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arX1::2211.14790, 2022.
Kewei Ouyang, Yi Hou, Shilin Zhou, and Ye Zhang. Adaptive multi-scale wavelet neural network for time series classification. Information, 12(6):252, 2021.
Robert H Shumway, David Stoffer, and David Stoffer. Time series analysis and its applications, volume 3.
Springer, 2017.
Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. The performance of lstm and bilstm in forecasting time series. In 2019 IEEE International conference on big data (Big Dala), pp. 3285-33-33 3292. JEEE, 2019.
Akara Supratak and Yike Guo. Tinyslcepnet: An efficient deep learning model for sleep stage scoring based on raw single-channel eeg. In 2020 42nd Arnual International Conference of the IEEE Engineering in Medicine & Biology Sociely (EMBC), pp. 641-644. IEEE, 2020.
Wim Sweldens. The lifting scheme: A construction of second generation wavelets. SIAM journal on mathematical analysis, 29(2):511-544) 1998.
Patrick Wagner, Nils Strodthoff, Ralf-Diete Bousseljot, Dieter Kreiseler, Fatima Lunze, Wojeiech Samek, and Tobias Schaeffter. Ptb-xl, large publicly available cloctrocardiograply datasct. Scientific data, 7(1): 154, 2020.
Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifci Xiao. Mien: Multi-scale local and global context modeling for long-lerm series forecasting. In The eleventh international conference on learning representations, 2023.
Shiyu Wang, Haixu WI, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting, arXio preprint 2r05 14616, 2024.
Wetterstation. Weather, 2020.
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34: 22419 22430, 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arv preprint or orXiv:2210.02186 2022.
Maxwell Xu, Alexander Morcno, Supriya Nagesh, Varol Aydemir, David Wetter, Santosh Kumar, and James Rehg. Pulseimpule: A novel benchmark task for pulsative physiological signal imputation. Advances Neural Information Processing Systems, 35:26874-26888, 2022.
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang. Pengyang Wang, Hui He, Defu Lian, Ning An, Longbing Cao, and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. arXiv preprinl ar 2rx1:2231..000 1.00114,, 2023.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AATT conference artificial intelligence, volume 37, pp. 11121-11128, 2023.
15

---

Published in Transactions on Machine Learning Research (11/2024)
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multi- variate time series forecasting. In The Eleventh International Conference on Learning Represenlations, 2022.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-scries forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106 11115, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, et al. Film: Frequency improved legendre memory model for long-term time series forecasting, Advances in Neural Information Processing Systems, 35:12677-
2022a.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting, In International Conference on Machine Learning, pp. 172688272287 PMLR, 2022b.
Yuzhen Zhu, Shaojie Luo, Di Huang, Weiyan Zheng, Fang Su, and Beiping Hou. Drenn: decomposing residual convolutional neural networks for time series forecasting, Scientific Reports, 13(1):15901, 2023.
Appendix
A Implementation Details
In this section, we cover the implementation details of the methods and experiments, including datasets, algorithm details, hyperparamcters, and basclines.
A.1 Datasets
There are 10 datasets used in this study, and the overall summary of the datasets information can be seen Table Also, the descriptions of each set are introduced in this section.
ETT (Electricity Transformer Temperature): The ETT dataset (Zhou et al. 2021 includes detailed records of electricity transformer temperatures and corresponding electricity consumption data. This high- resolution dataset, sampled every 15 minutes, covers two years. It provides insights into the relationship between transformer temperatures and electricity demand, crucial for maintaining efficient and safe operations of power systems. In our research, we utilize the ETT dataset to forecast short-term electricity demand and transformer temperatures.
Electricity Load Diagrams: The Electricity Load Diagrams dataset, sourced from the UCT Machine Learn- ing Repository (Asuncion & Newman 2007 comprises electricity consumption data recorded at 15-minute
Table 5: Details of datasets used in the experiments. Data Split means the number of samples split into the train, validation, and test sets.
Dataset
Dimentions
Frequency
Data Split
Forecasting Imputation
SR
ETTml
7
15 mins
(34465, 11521, 11521)
FTTh1
7
Hourly
(8545, 2881, 2881)
V
ECL
321
Hourly
(18317, 2633, 5261)
Traffic
862
Hourly
(12185, 1757, 3509)
Weather
21
10 mins
(36792, 5291, 10540)
Exchange
8
Daily
(5120, 665, 1422)
Solar
137
10 mins
(36601, 5161, 10417)
PTB-XL
12
500 Hz
(14771, 1493, 1652)
Sleep-EDD EDFE
1
100 Hz
(22212, 9519, 10577)
CLAS
100 Hz
(993, O, 359)
16

---

Published in Transactions on Machine Learning Research (11/2024)
intervals from multiple customers. The dataset represents diverse range of electricity consumers, including both individual households and industrial customers. The data spans from 2011 to 2014, encompassing various consumption patterns and seasonal effects. For our study, we focus on forecasting electricity demand on an hourly basis.
Traffic (PeMS: Performance Measurement System): The PeMS dataset, provided by the California Department of Transportation, is a comprehensive collection of traffic flow data. It contains real-time traffic speed and volume information collected from over 39,000 individual sensors across the freeway system of California, These sensors report data every 30 seconds, offering exceptionally detailed view of traffic patterns. Our study utilizes this datasct to forecast traffic flow and congestion levels. The data spans several years, but for our analysis, we focus on a one-year period ensuring a mix of workdays and weekends to capture varying traffic behaviors.
Weather (Global Surface Summary of the Day): The weather datasct from NOAA's National Climatic Data Center (Wetterstation 2020) offers daily weather summaries from a wide array of weather stations around the world. This dataset includes essential meteorological parameters such temperature, humidity, precipitation, wind speed, and atmospheric pressure. The data, spanning over several decades, provides a rich source for analyzing and forecasting weather patterns. For our research, we select subset of the dataset encompassing ten years of data from stations across different. climatic zones. The goal is to develop models capable of predicting weather conditions such as temperature and precipitation.
The Foreign Exchange Rates dataset (Lai et al., 2018 encompasses daily exchange rates of various currencies against. the US dollar from the year 2000 to 2019. It is a comprehensive datase) that includes the exchange rates of over 50 currencics and offers detailed view of global financial dynamics. Our study aims to forecast the daily exchange rates of major currencies. The dataset's span allows for the analysis of long-term trends as well as responses to major global events. For preprocessing, we ensure data continuity by addressing any missing values and then normalize the data to account for different scales across currencies.
Exchange Rates:
Solar Energy Prediction: The Solar Energy Prediction dataset from the UCI Machine Learning Repository (Asuncion & Newman 2007) contains solar power generation data alongside various meteorological variables. The dataset, collected over one year, includes measurements such solar irradiance, temperature, humidity, and wind speed, sampled at 10-minute intervals. This comprchensive dataset enables the development of models for predicting solar energy output, a key factor in managing renewable energy resources. In our analysis, we focus on forecasting daily solar energy data.
PTB-XL: The PTB-XL (Wagner et al. 2020 dataset is a large dataset containing 21,837 clinical 12-lead electrocardiogram (ECG) records from 18,885 patients of 10-second length, where 52% are male and 48% are female with ages ranging from to 95 years (median 62 and interquartile range of 22). There are two sampling rates: 100 Hz and 500 Hz, available in the dataset, but in our experiments, only data sampled at 100 Hz are used. The raw ECG data are annotated by two cardiologists into five major categories, including normal ECG (NORM), myocardial infarction (MI), ST/T Change (STTC), Conduction Disturbance (CD), and Hypertrophy (HYP). The dataset contains a comprehensive collection of various co-occurring pathologies and a large proportion of healthy control samples. We experimented with imputation and super-resolution tasks. Further, to ensure a fair comparison of machine learning algorithms trained on the dataset, we follow the recommended splits of training and test sets, which results a training/ meriiing//sstingg ttsting ratio of 8/1.
The FDDep-FDF (expanded) (Kemp et al. 2000p dataset contains whole-night. sleep recordings from 822 subjects with physiological signals and sleep stages that were annotated manually by well-trained technicians. In this dataset, the physiological signals, including Ppz-Cz/Pz-OO electroenccphalogram (EEG), electrooculogram (FOG), and chin electromyogran (EMG), were sampled at 100 Hz. To model the relationship between the sleep patterns and physiological data, we split the whole-night recordings into 30-second Fpz-Cz Cz EEG segments as in SSpratak & Guo (2020), which resulted in a total of 42308 EEG and sleep pattern pairs. We divided 25% of the samples into a testing set according to the order of the subject IDs.
Sleep-EDFE:
The CLAS dataset Markova et al. (2019) aims to support rescarch on the automated asscssment of certain states of mind and emotional conditions using physiological data, The dataset cousists of synchronized recordings of ECG, photopletysmogram (PPG), electrodermal activity (EDA), and acceleration (ACC) signals.
CLAS:
17

---

Published in Transactions on Machine Learning Research (11/2024)
Random Masking
Extended Masking
Figure 5: Comparison of masking methods for time series imputation. The left column depicts random masking, where individual data points are randomly concealed throughout the scries, simulating sporadic data loss. The right column illustrates exlended masking, where contiguous segments are masked to emulate prolonged periods of missing data. Each row represents an independent time series sample subject to the rospective masking method.
Sixty-two healthy subjects participated and were involved in three interactive lasks and two perceptive tasks. The perceptive tasks, which leveraged the images and andio-video stimuli, were purposely selected to evoke emotions in the four quadrants of arousal-valence space. In this study, our goal was to use the EDA signal detect binary high/low stress states that are annotated in arousal-valence space. We processed the raw EDA data with lowpass Butterworth filter with cutoff frequency of 0.2 Hz, then split the sequences into 10-second segments. We divided the train/tee /test set in subject-independer manner and utilized the data from 17 subjects the test set according to subject ids (> 45).
A.2 Imputation Settings
In the context of time scries imputation, masking refers to the process of artificially removing conccaling parts of the data to simulate missing values, which the model then attempts to impute. Two distinct masking approaches are commonly employed: random masking and extended masking.
This method involves randomly selecting points or segments within the time series data and setting them as missing or masked, The randomness this approach is intended to simulate data missingness that occurs sporadically and without a specific pattern, which a common scenario in real-world datasets. However, this method may not adequately represent scenarios where data is missing for extended periods, which is also practical occurrence.
Random Masking:
To address the limitations of random masking, we introduce the extended masking approach. This technique creates artificial gaps by masking contiguous segments of the time series. Extended masking is designed to simulate scenarios where data might be missing due to prolonged outages or systematic issues, providing a more challenging and realistic task for the imputation model to tackle.
Extended Masking:
Both methods are essential for testing the robustness and versatility of imputation models, ensuring they can handle various types of missing data patterns.
18

---

Published in Transactions on Machine Learning Research (11/2024)
Table 6: Comparison of performances of using different numbers of clusters in grouped linear module.
Datasct number of clusters
k =
1
k = 2
k =
3
k = 4
k =
5
ECL
0.175
0.173
0.170
0.168
0.169
Weather
0.240
0.233
0.236
0.235
0.235
Exchange
0.481
0.484
0.482
0.481
0.482
Solar
0.209
0.214
0.210
0.212
0.214
ETTh1
0.462
0.457
0.449
0.444
0.446
Table 7: The hyperparameter settings of Ada WaveNel for each dataset.
hyperparam.
lifting kernel size
lifting Levels n clusters
learning rate
ETT
4
0.0005
ECL
3
0.0005
Traffic
1
0.001
Exchange Rate
4
0.0001
Weather
7
4
0.0005
Solar
0.0005
PTB-XL
5
0.001
Sleep-EDFF EDFF
16
5
0.001
CLAS
5
0.001
A.3
Implementation Details
In this section, we introduce the experimental environments and the hyperparameter used for each dataset.
A.3.1
Experimental Environments
All the modules are implemented in Py Torch 1.11 and a Python version of 3.10. To speed up the experiments, GPU instances are utilized. We use AWS G5 instances, which are equipped with Nvidia A10 24GB GPUs, in training and inferring the models.
A.3.2 Hyperparameter Setting
In this section, we list some essential hyperparameters used in this study. For the design of Ada Wave blocks, we adjust the depth of transformations in a range of to 5; whereas the kernel size of the utilized convolutional kernels is adjusted based on the datasets. The number clusters used in the grouped linear model varies from 1 to 9 depending on the channels of signals. Also, we adjust the learning rate for each set of experiments achieve better convergent performances. Further, determine the number of clusters in the proposod group linear module, we conduct extensive experiments as shown in Table 6 The detailed hyperparameters of each dataset are listed in Table 7
A.4 AdaWaveNet Ablation Study
Table shows the full results of the ablations of the model.
B Case Study: Analyses of Non-stationary Signals with AdaWaveNet
In this case study, we present a comprehensive approach to synthesizing non-stationary time scries data, which combines low-frequency sinusoidal components, high-frequency transients, linear trends, and Gaussian noise create a basic non-stationary signal. This represented by the equation:
<(1) sin fit) + sin fat)e a(t-(o)² + Bl e(l)
(14)
19

---

Published in Transactions on Machine Learning Research (11/2024)
Table 8: The full results of model ablation with mean squared error as the evaluation metric. The highest MSE is highlighted in bold, while the sccond highest is underlined. Refer to Table in the Appendix for comprehensive results.
w/o
Grouped
Linear
Itein
Channel
Attention
AdaWave Block
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.169
0.215
0.174
0.226
0.180
0.231
0.185
0.239
0.177
0.231
192
0.203
0.245
0.209
0.254
0.216
0.260
0.227
0.266
0.222
0.257
Weather
336
0.248
0.286
0.255
0.299
0.262
0.311
0.272
0.328
0.276
0.337
Forecasting
720
0.313
0.336
0.321
0.348
0.325
0.356
0.341
0.375
0.345
0.382
Avg.
0.233
0.271
0.240
0.282
0.246
0.290
0.256
0.302
0.255
0.302
96
0.417
0.291
0.429
0.303
0.418
0.293
0.513
0.338
0.472
0.314
192
0.401
0.281
0.415
0.297
0.409
0. 0.288
0.499
0.331
0.466
0.312
Traffic
336
0.407
0.284
0,416
0.297
0.417
0.292
0.520
0.349
0.470
0.306
720
0.433
0.297
0.439
0.314
0.442
0.314
0.545
0.382
0.483
0.317
Avg.
0.415
0.288
0.425
0.303
0.422
0.297
0.519
0.350
0.473
0.312
0.125
0.098
0.207
0.099
0.208
0,103
0.212
0,108
0,214
0.112
0.219
0.25
0.104
0.207
0.106
0.212
0.106
0.214
0.112
0.215
0.121
0.227
ECL
0.375
0.121
0.228
0.124
0.233
0.129
0,240
0.131
0,232
0.140
0.240
Imputation
0.5
0.126
0.230
0,131
0.237
0,130
0,235
0.137
0,243
0.149
0.250
Avg.
0.112
0.218
0,115
0.223
0.117
0,225
0.122
0,226
0.131
0.234
0.125
0.049
0.062
0.058
0.071
0.052
0.065
0.053
0.064
0.063
0.084
0.25
0.063
0.080
0.072
0.089
0.067
0.088
0.062
0.078
0.072
0.091
PTB-XL
0.375
0.074
0.089
0.085
0.114
0.071
0.082
0.077
0.089
0.090
0.110
0.5
0.089
0.122
0.103
0.136
0.091
0.121
0.085
0.117
0.104
0.127
Avg.
0.069
0.088
0.080
0.103
0.070
0.089
0.069
0.087
0.082
0.103
where fi and f2 are the low and high frequencies respectively, a controls the transient decay, 8 represents the linear trend, and E(t) is Gaussian noise. This formulation captures fundamental non-stationary behaviors while maintaining interpretability. We first generate signals with a regular low-frequency component (5 Hz) and a high-frequent component (50 Hz) as the simple signal. Further, for increased complexity, we introduce traffic-like signals that incorporate daily and weekly cyclical patterns to mimic real-world traffic fluctuations:
Straffie (t) = sin(2 + 0.5 sin(4424f) + sin
+ 0.5t + +(()
(15)
Similarly, electricity consumption-minic mimic signals simulate seasonal and daily variations typical of power usage data:
Selectricity (1) sin + 2 sin(4 + (2x3266))) + 21 + =())
(16)
Both complex models include trend components and noise to simulate realistic non stationary data. Fur- thermore, we implement optional variance shifts and step changes to test model robustness against abrupt signal changes. The variance shift is modeled as an increase in the amplitude of <(t) at a specific time point, while the step change represented by a sudden additive constant to the signal. Based on the three settings including simple, traffic, and electricity we further adjust (1) the variance shift values to be 1 and 2, for moderate and large shifts, (2) the step changes as -0.5 or 0.5 to indicate the negative and positive changos. The synthetic signals are visualized in Figure 6 In total, for each signal, 1024 data points generated, where the first 512 timestamps are used in training models. In the evaluation stage, we compare model predictions with denoised signal, i.e., sin 56) + sin(2 506) -00tt-0.)) + tt. Also, the variance shifts and step changes are only applied in the test phases.
The performance of various models, including Ada WaweNet, DLinear, TimosNet, FreTS, PStchTST, and iTransformer, was evaluated on this synthetic non-stationary time series. Table 9 summarizes the results of this case study. Further, we showcase two examples, including Simple and Traffic with variance shift and step changes, in Figure 7 and g With the lowest MSE and MAE among all models, Ada WaveNett's predictions closcly align with the ground truth, effectively capturing both the low-frequene sinusoidal component and the high-frequency transient component. In contrast, other models struggle to varying degrees in accurately predicting the non-stationary signal. DLinear shows significant deviation from the ground truth, particularly in capturing the high-frequency components, which results high MSE and MAE. FreTS and TTranfoormrr demonstrate moderate performance but fail capture the finer details of the signal, particularly the high-frequency components.
20

---

Published in Transactions on Machine Learning Research (11/2024)
Simple
Simple
Himplee-Modeiee Moderate Shilt, Negative Step
Simple Large Shift, Positive Step
Amploude
Amglonth
0.0
0.5
1.0
0.0
0.2
3.4
0.6
0.01
1.0
0.0
0.2
01.4
0..5
0.80
1.0
Time
Time
Traffic
Traffic Moderate Shirt, Negative Stop
Traffic Lange Shift, Positive Stap
My
M MM
Ampiltut
Ampltutt
Anp
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
2.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
Time
Time
Electricitt
Electricity Moderate Shift, Negative Step
Electricity Large Shift, Positive Step
10.0
10.0
7.5
7.5-
5.0
5.0
5.0
2.5
Amat. 180
2.5
0.0
-25
-5.0
-5.0
-7.
-10.0
-10.9
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
3.4
0.8
0.8
0.0
02
0.4
0.6
0.8
1.0
Time
Time
Time
Figure 6: Synthetic non-stationary time series data generated using three complexity levels (Simple, Traffic, and Electricity) and three parameter settings (No shift/step, Moderate shift with negative step, and Targe shift with positive stop). Each row represents a different complexity level, while cach column shows the conditions of variance shift and step changes.
Table 9: The averaged results of the model forecasting on synthetic non-stationary signals. Models are trained on 512 data samples, whereas the test samples are the later 512 points. Both the lookback length and prediction length are fixed at 96. The evaluation metrics are mean squared error (MSE) and mean absolute error (MAF)
Simple
Traffic
Electricity
Models
MSE
MAE
MSE
MAE
MSE
MAE
Transformer
0.454
0.525
1.268
0.904
2.677
2.564
DLinear
0.599
0.692
1.032
0.858
1.885
1.679
TimesNet
0.397
0.492
1.622
1.027
2.571
2.488
FreTS
0.676
0.707
2.081
1.169
1.701
1.502
PatchTST
0.330
0.453
1.168
0.866
1.550
1.351
Ada WaveNet
0.215
0.367
0.478
0.576
1.299
1.193
21

---

Published in Transactions on Machine Learning Research (11/2024)
Original Non-Stationary Signal
Ground Truth (Dencised Prediction Period)
Orogra Sigal
w
Grean Trubh
Ampluue
Time
Time
Truth
Dalvar Fraditto
Ampludd
06
24
1.
TimesNet Prediction Ground Truth
FreTS Prediction Ground Truth
Ground Truth Fradetts
Amplnute
Ground Turn Tiredhet Predittio
Time
Time
PatchTST Prediction vs Ground Truth
Mannsfrmer Prediction vs Ground Truth
Ground
Ground Tuah
Amol tudi
Time
Time
Figure Comparison of model predictions on a synthetic non-stationary time series [Simple]. Top row: Original non-stationary signal (left) and ground truth for the prediction period (right). Subsequent rows: Predictions from Ada WaveNet, DLinear, TimesNet, FreTS, PStchTST, and iTransformer models compared against the ground truth.
22

---

Published in Transactions on Machine Learning Research (11/2024)
Original Non-Stationary Signal
Ground Truth (Dencised Prediction Period)
Orogna Sigal
Truth
Time
Time
Adult of teet
Dlavour Pradiction
02
0.
TimesNet Prediction Ground Truth
FreT's Prediction Ground Truth
Ground Torn
Ground Tean
Toneddert Presiction
Time
Time
PatchTST Prediction vs Ground Truth
Transformer Prediction vs Ground Truth
Ground Trush
Time
Time
Figure Comparison of model prodictions on a synthetic non-stationary time scries [Traffic- moderate variance shift & negative step change]. Top row: Original non-stationary signal (left) and ground truth for the prediction period (right). Subsequent rows: Predictions from Ada WaveNet, DLinear, TimesNet, FreTS, PSt and iTransformer models compared against the ground truth.
23

---

Published in Transactions on Machine Learning Research (11/2024)
(e) Original Sta-Staat Signal
130
Aneluute
255
2.0
-7.5
02
1.0
Seasonal Component (Decomposition)
(c) Thend (Decomposition
0.25
0.00
-0.25
Approximation
3.5
Amplbade
3.2
-2.5
1.0
exmmaaio Seasonal (Level
Detail Confficionts (Level
0.50
0.2
Ampiincee
0.1
0.0-
-0.22
-0.1-
-0.50
0.0
0.2
0.s
1.0
02
(d-3) Approximation of Seasonal (Level 2)
(e-3) Detail Coe Coefficients
Sea sonal LLevel
M M
Amsioode
-01
-32
-3.4
0.0
0.2
0.6
1.0
02
0.6
0.8
Time
(d-4) Appraxdmation of Seasonal (Level 3)
3.2
-32
-34
0.0
0.2
0.4
0.6
0.6
1.0
Figure 9: Multi-level decomposition of non-stationary time series using Ada WaveNet. The top plot shows the input signal, followed by four levels of approximations (left column) and their corresponding wavelet. coefficients (right column). Each level represents a coarser approximation of the original signal, with the coefficients capturing the details lost between successive approximation levels.
24

---

Published in Transactions on Machine Learning Research (11/2024)
Prekition
2.5
Fesketin
15-
2.0-
2.3
05
-35
-2.
175
220
PatchTST
TimesNett
DLincar
Goouddruth Precktion
Groutil-th
GroundFivtt
2.0
2.0-
Prosicson
1.-
1.0
25-
0.0
3.0
-0.5
25
100
125
150
175
250
1.00
325
350
175
200
100
125
1.50
200
Nonstaationary Transformer
FLL
FEDformer
Groundhuuhh Predition
GroundTr-th Ptediction
Ground
2.0-
2.5
Fretiction
15
3.5
3.5-
05-
0.5
3.5
30-
-2.5
-35-
200
123
120
175
220
125
220
172
120
135
200
Figure 10: Visualization of forecasting task on traffic dataset. The length of both the input sequence and forecasting sequence 96.
Figure illustrates the multi-scale decomposition capabilities of Ada WaveNet when processing the generated non-stationary time series. The approximation each level represents progressively coarser version of the original signal, which effectively separates low-frequency trends from high-frequency details. Concurrently, the wavelet coefficients encode the fine-grained information lost between successive approximation levels. This hierarchical decomposition allows Ada WaveNet to adapt to the signal's changing characteristics at multiple scales. For instance, the Level 0 approximation retains much of the original signal's structure, while Levels 1, 2, and focus on broader trends. This adaptive multi-scale approach enables the proposed model to learn both local fluctuations and overarching patterns in non-stationary time series.
C
Showcases
In this section, showcase some examples of forecasting, imputation, and super-resolution tasks.
C.1
Forecasting
Figure 10 presents a comparative example of forecasting future traffic volumes using various models. The figure reveals a notable disparity between past sequences and predicted sequences for this particular variate. The visual results indicate that Ada WaveNet yields the most accurate forecast in this instance. Additionally, the iTransformer model also performs commendably, which suggests that its channel-wise attention mechanism is particularly useful for analyzing past traffic patterns for prediction purposes.
C.2 Imputation
The examples of imputation, including the random imputation and extended imputation are demonstrated in this section.
25

---

Published in Transactions on Machine Learning Research (11/2024)
2.3
Preditiion
2.0
Redictioon
2.0
Fredetion
1.0
0.5
-0.
-03
-1.
-1.
-1.9
PatchTST
TimesNet
Diiinear
GmmndTuu Pradkation
GenansTTuth
GruudtTrmm Trudicion
1.3
3.3
0.5
0.5
0.0
0.0
6.3
-0.
-1.9
Nonstationayy.
Tannsfomer
FILM
FEDrermer
Groundruzh Preditiion
borth
GroancTruth Rediitii
GroundFruth Fredittin lnn
2.0
1.5
1.0
3.5
0.0
-05
0.0
0.
-1.0
-0.5
8
-10
-15
Figure 11: Visualization of a random imputation task on ETTh1 dataset. The sequence length is 96 and the masked ratio is 0.25.
C.2.1 Random Masking
We provide an example of imputation with the random masking method. The proposed Ada WaveNet and all the other baseline methods. As shown in Figure [1] Ada AdaWaveNet, alongside baseline methods such as PatchTST, TimesNet, and Monstationary-Traasfooo Transformer, is capable of capturing the temporal dynamics. Notably, AdveaveNet shows superior performance imputing fine-grained details, effectively handling both the seasonality during peak phases and the underlying trends in flatter regions.
C.2.2 Extended Masking
We provide an example of imputation with the extended masking method. The proposed Ada WaveNet and all the other baseline methods. Shown as Figure 12 the proposed method exhibits a close approximation the ground truth, which indicates higher predictive accuracy within this interval. The consistency across the models outside the masked region implies shared ability to capture the temporal dynamics in non-masked intervals; while the differences within the masked region highlight the distinct predictive capabilities and potential overfitting issues of the individual models.
C.3 Super-resolution
The visualization presents the results of super-resolution task time series data, specifically forecasting traffic volume. Each subplot represents the performance of a different model: Ada WaveNet, iTransformer, FreTS, TimesNet, DLinear, PatchTST, Non-stationary Transformer, FiLM, and FEDformer. In each plot, three lines are denoted as the Ground Truth (blue line), which is the actual high-resolution data; the Prediction (orange line), which the model's predicted high-resolution data; and the Low-resolution Input (gray line), which serves as the model's input data and represents the downsampled or coarse version of the Ground Truth.
26

---

Published in Transactions on Machine Learning Research (11/2024)
Predittion
1.3
1.3
1.9
1.7
Wedd
Prediction
0.5
PatchTST
TimesNet
Dlineer
Nww
1.2
GroundTuua
3.3
GroundTiar TATT
0.5
0.5
0.5
20
Nonstationary. Tanssfommr
FILM
FEDfermer
Groundruzh Preditionn
2.0
2.0
Geamecrruhh Prodiction
0.5
0.0
StoundTuun
-0.5
Pradiction
Figure 12: Visualization of an extended imputation task on ETThl dataset. The sequence length is 96 and the masked ratio is 0.25.
The predictions of Ada WaveNet closely follow the Ground Truth across the entire sequence. The fidelity of Ada WaveNef's prediction the Ground Truth, particularly in capturing the peaks and troughs of the traffic volume, showcases the model's capability in the super-resolution task. The granularity of details in the prediction suggests that Ada WaweNet effectively upsamples the low-resolution input and reconstructs nuanced and accurate traffic data.
27

---

Published in Transactions on Machine Learning Research (11/2024)
Ada Wheellet
Transformet
FretS
Groundruut
Groundhuth
Draunshuuh
12-
-2
32
300
230
200
200
PatchTST
TimesNot
Diiinar
25
Greundinuss Presktion Uie reso Your
25
Grounddlitth
Ground Tluth
Prediction
Prediction
20
20-
reneiittinn input
30-
Emae Ena-reociaton Inger
15
15.
10
10
25
S0
25
100
125
150
175
200
25
50
75
100
125
159
175.
200
25
50
100
125
150
175
200
Innary
Fwstition
Prediction
Frediction Lonvnackattnn trgor.
hour.
Lownosdittnn rput
35.
15
Figure 13: Visualization of a super-resolution task traffic data with input length of 200. The super- resolution ratio is 5.
28


Reviews:
[{"review_id": "n5XxmwbxE2", "reviewer": "Reviewer_72rq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Innovative Approach: Introduces a novel adaptive wavelet network for handling non-stationary time series data.\n\n2. Comprehensive Evaluation: Provides extensive experiments across multiple datasets and tasks, demonstrating superior performance.\n\n3. Multi-Scale Analysis: Effectively incorporates multi-scale analysis, improving flexibility and robustness.\n\n4. New Benchmark: Establishes a benchmark for super-resolution tasks in time series analysis.\n\nWeaknesses:\n\n1. Lack of Related Works: The paper does not adequately cover existing decomposition-based methods, other than recent FedFormer and CrossFormer discussed already, there are still papers like DRCNN[1], MICN[2], and TEMPO[3] nor does it sufficiently discuss related works in wavelet analysis, which has been used on time series analysis for years.\n\n[1] Zhu, Y., Luo, S., Huang, D., Zheng, W., Su, F., & Hou, B. (2023). DRCNN: decomposing residual convolutional neural networks for time series forecasting. Scientific Reports, 13(1), 15901.\n\n[2] Wang, H., Peng, J., Huang, F., Wang, J., Chen, J., & Xiao, Y. (2023). Micn: Multi-scale local and global context modeling for long-term series forecasting. In The eleventh international conference on learning representations.\n\n[3] Cao, D., Jia, F., Arik, S. O., Pfister, T., Zheng, Y., Ye, W., & Liu, Y. (2023). Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:2310.04948.\n\n2. Contribution Clarity: The paper's contributions are weakened by the lack of distinction from previous related works.\n\n3. Non-Stationary Analysis: Although targeting non-stationary time series is highlighted, the paper lacks experimental analysis explaining how and why the model effectively addresses non-stationarity.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces the Adaptive Wavelet Network (AdaWaveNet), a novel architecture for time series analysis, addressing the challenge of non-stationary data. The contributions include:\n\n1. Novel Architecture: AdaWaveNet offers an adaptive, multi-scale approach to enhance accuracy and reliability in analyzing non-stationary time series data.\n\n2. Super-Resolution Benchmark: Establishes a new benchmark for super-resolution in time series, improving data quality from under-sampled sequences.\n\n3. Performance Evaluation: Extensive experiments demonstrate AdaWaveNet's superiority over existing methods in forecasting, imputation, and super-resolution tasks, indicating its potential for diverse real-world applications.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Expand Related Work: It's critical to include a more comprehensive discussion of existing decomposition-based methods, as well as other wavelet analysis works. This will strengthen the context and position of the paper within the field.\n\n2. Clarify Contributions: Clearly differentiate the paper's contributions from existing works. This is important to ensure the novelty and impact of the research are well understood. \n\n3. Non-Stationary Analysis: Conduct and include experimental analysis on how the model addresses non-stationarity. This would secure my recommendation for acceptance by providing evidence of the model's effectiveness in this area.\n\n4. Table 1's score doesn't seem to match PatchTST's original results. However, can the authors clarify where the difference comes from, and can we align it with the PatchTST paper?", "broader_impact_concerns": "The authors discussed the broader impact in section 7.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rCgMvjH1ha", "reviewer": "Reviewer_xiVg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n-\tThis paper is well-written and clear.\n-\tThe proposed method is reasonable, where adopting wavelet analysis is a good idea.\n-\tExtensive experiments on diverse tasks are included.\n-\tThe showcase visualization in the Appendix is intuitive and appreciated.\n\n## Weaknesses\n\n1.\tI think this paper does not provide a clear connection between wavelet analysis and non-stationary time series. Why using wavelet analysis can help the model tackle non-stationary problems? More discussions are required.\n2.\tTo some extent, the super-resolution task is a special imputation problem. Specifically, it can be viewed as imputing the interval between low-frequency sampled time points. More descriptions of the unique challenges in super-resolution are expected.\n3.\tIn Section 4.1, the authors mentioned “It is often calculated using a moving average, which smooths out short-term fluctuations to highlight longer-term trends.” I think a citation to Autoformer (Wu et al., 2021) is needed since the moving average decomposition is proposed in that paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents the AdaWaveNet by utilizing the decomposition and wavelet network for time series analyses. AdaWaveNet focuses on the non-stationary problem in time series, which is widely evaluated in three different tasks (a new super-resolution task) and compared with extensive baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "They need to clarify the relation between wavelet analysis and non-stationary time series and the special challenges in super-resolution task.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "WqkD9W3MYz", "reviewer": "Reviewer_Du9K", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghs: \n- The approach seems interesting. \n- Results looks impressive. \n\nWeaknesses: \n- Description of method and its motivation could be improved. For example, from sections 3 and 4, I can understand their idea in somewhat in high level, but it is hard to get deeper understanding about the approach. Maybe it is my lack of understanding related to lifting scheme which is new to me, but somehow I find the description is confusing and it is not clear to me that why the results architecture is “Wavelet” or motivated by it. See below. \n- Results section is not very well organized. The content is some what randomly placed between main body and appendix. I would say that a general rule about appendices is that you should be able to read the manuscript without it and still get essential parts of what has been done in the research. This does not clearly hold with this manuscript as understanding of experiments requires lots of jumping back and worth between the main body and appendix. And all necessary details are not even referred in the main body, so you cannot find those without reading through the whole thing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The manuscript considers time serial analysis, specifically cases in which signal are not stationary. They propose a new architecture to model such time series based on Wavelets and Lifting scheme. They apply their proposed approach to several non-stationary time series estimation tasks including forecasting, imputation and super-resolution tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Presentation should be improved as mentioned above. They could consider the following comments/questions:\n- Wavelet is described in (1) as a continuous signal (f is the function/signal and x is the variable), but this seems to change in section 3.1 where x is a discrete signal. Also cij in (1) are Wavelet coefficients i.e. scalars, but in section 3.2 c (“approximation”) seems to be signal. Are c’s in (1) and section 3.2 still somehow related, so is some kind of form of Wavelet coefficients?\n- Organize the experiments section as mentioned above\n- I find it very odd that limitations of the method are placed in appendix. \n\nMinor changes:\n- Citations has mistake, I think in some places \\citep should be used instead \\cite and vice versa. \n- Font in tables is very small, very hard to read if the manuscript is printed (may be general issue in latex template, the font overall is very small)\n- Table captions say that lowest MSE is highlighted with red, but also MAE is highlighted. Why blue is underlined, but red is not?", "broader_impact_concerns": "Not applicable", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a novel method and gave a clear analysis for its utility, where all reviewers agreed about the novelty of the method and clarity of the analysis. The major improvements needed were in organization and better differentiating the work from prior work. It has room for improvement in describing its motivation but none-the-less fits should be accepted.
Decision: Accept (poster)