TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
Authors: Haixu Wu
Tengge Hu
Yong Liu
Hang Zhou
Jianmin Wang
Mingsheng Long
Venue: ICLR 2023 poster (2023)
Keywords: Time Series Analysis
Deep Learning

Abstract: Time series analysis is of immense importance in extensive applications, such as weather forecasting, anomaly detection, and action recognition. This paper focuses on temporal variation modeling, which is the common key problem of extensive analysis tasks. Previous methods attempt to accomplish this directly from the 1D time series, which is extremely challenging due to the intricate temporal patterns. Based on the observation of multi-periodicity in time series, we ravel out the complex temporal variations into the multiple intraperiod- and interperiod-variations. To tackle the limitations of 1D time series in representation capability, we extend the analysis of temporal variations into the 2D space by transforming the 1D time series into a set of 2D tensors based on multiple periods. This transformation can embed the intraperiod- and interperiod-variations into the columns and rows of the 2D tensors respectively, making the 2D-variations to be easily modeled by 2D kernels. Technically, we propose the TimesNet with TimesBlock as a task-general backbone for time series analysis. TimesBlock can discover the multi-periodicity adaptively and extract the complex temporal variations from transformed 2D tensors by a parameter-efficient inception block. Our proposed TimesNet achieves consistent state-of-the-art in five mainstream time series analysis tasks, including short- and long-term forecasting, imputation, classification, and anomaly detection. Code is available at this repository: https://github.com/thuml/TimesNet.

Published as a conference paper at ICLR 2023
TIMESNET: TEMPORAL 2D- 2D-VARIATION MODELING FOR GENERAL TIME SERIES ANALYSIS
Haixu Wu,' Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long School of Software, BNRist, Tsinghua University, Beijing 100084, China {whx20, fuyong21, htg21, h 1o-zhou11 @mam 1s ingungguu.... edu.cn {1 limwanng mi ingghnng)hggnggn @ nggua. edu.cn cn
ABSTRACT
Time series analysis is of immense importance in extensive applications, such weather forecasting, anomaly detection, and action recognition. This paper focuses temporal variation modeling, which is the common key problem of extensive analysis tasks. Previous methods attempt to accomplish this directly from the ID time series, which extremely challenging due the intricate temporal patterns. Based on the observation of multi-periodicity in time series, we ravel out the com- plex temporal variations into the multiple intraperiod- and interperiod intrrperiod-variaations. variations.
To tackle the limitations of ID time series in representation capability. we extend the analysis of temporal variations into the space by transforming the time series into a set of 2D tensors based on multiple periods. This transformation can embed the intraperiod- and inerperiod-variatiin the 2D tensors respectively, making the 2D-variations to be easily modeled by 2D kernels. Technically, we propose the TimesNet with TimesBlock as a task -ekerel backbone for time series analysis. TimesBlock discover the multi- periodiiii adaptively and extract the complex temporal variations from transformed 2D ten- sors by a parameter-efficient inception block. Our proposed TimesNet achieves consistent state-of-the-art in five mainstream time series analysis tasks, including short- and long-term forecasting, imputation, classification, and anomaly detection. Code is available at this repository: https://github.com. wwtpgghhhu....tttuum.eeeeeo httttthummmmmmmeeee
into the columns and rows
INTRODUCTION
Time series analysis is widely used in extensive real-world applications, such the forecasting of meteorological factors for weather prediction (Wu et al., 2021). imputation of missing data for data mining (Friedman) 1962). anomaly detection of monitoring data for industrial maintenance (Xu et al.) 2021) and classification of trajectories for action recognition (Franceschi et al. 2019). Because of its immense practical value, time series analysis has received great interest (Lim & Zohren 2021).
Different from other types of sequential data, such as language or video, time series is recorded continuously and each time point only saves some scalars. Since one single time point usually cannot provide sufficient semantic information for analysis, many works focus the temporal variation, which is more informative and can reflect the inherent properties of time series, such as the continuity, periodicity, trend and etc. However, the variations of real-world time series always involve intricate temporal patterns, where multiple variations (c.g. rising, falling, fluctuation and etc.) mix and overlap with each other, making the temporal variation modeling extremely challenging.
Especially in the deep learning communities, benefiting from the powerful non-linear modeling capacity of deep models, many works have been proposed to capture the complex temporal variations in real-world time series. One category of methods adopts recurrent neural networks (RNN) to model the successive time points based the Markov assumption (Hochreiter & Schmidhuber 1997: Lai et al. 2018 Shen et al. 2020). However, these methods usually fail in capturing the long- term dependencies and their efficiency suffers from the sequential computation paradigm. Another category of methods utilizes the convolutional neural network along the temporal dimension (TCN)
"Equal Contribution

---

Published as a conference paper at ICLR 2023
interperiod-variene
Period 3
Period
hyd Typ
Period
Introperiod- variation
Volue
Period
ma
Myw
und
Period
Time points at sams phese of different periods
Interpariode sinraiiiddvrriaiionn Intraperiodvvriaation
Figure I: Multi- periodicity and temporal 2D- of time series. Each period involves the ciruaperiod-varittie and interperiod-variation We transform the original ID time series into a set of 2D tensors based on multiple periods, which can unify the intraperiod- and interperiod-variitions..
to extract the variation information (Franceschi al. 2019 He & Zhao 2019). Also, because of the locality property of the me--imeension convolution kernels, they can only model the variations among adjacent time points, thereby still failing in long-term dependencies. Recently, Transformers with attention mechanism have been widely used in sequential modeling (Brown et al.) 2020, Dosovitskiy et al. [021] Liu al., 2021b). In time series analysis, many Transformer-based models adopt the attention mechanism or its variants to capture the pair-wisc temporal dependencies among time points (Li et al., 2019 Kitaev al. 2020 Zhou et al., [021] (022). But it hard for attention mechanism to find out reliable dependencies directly from scattered time points, since the temporal dependencies can be obscured deeply in intricate temporal patterns (Wu et al., 202T).
In this paper, to tackle the intricate temporal variations, we analyze the time series from new dimension of multi- nultt--priodiicity Firstly, we observe that real-world time series usually present multi-periodicity, such as daily and yearly variations for weather observations, weekly and quarterly variations for electricity consumption. These multiple periods overlap and interact with each other, making the variation modeling intractable. Secondly, for each period, we find out that the variation of each time point is not only affected by the temporal pattern of its adjacent area but also highly related to the variations of its adjacent periods. For clearness, name these two types of temporal variations and interperiod-variattee temporal patterns within period. The latter can reflect long-term trends of consecutive different periods. Note that for the time series without clear periodicity, the variations will be dominated by the intraperiod-variation and is equivalent to the ones with infinite period length.
as varaperiod-variatiin
respectively. The former indicates short-term
Since different periods will lead to different intraperiod- and interperiod-varian variaaiion,,, the multi- periodicity can naturally derive modular architecture for temporal variation modeling, where we can capture the variations derived by certain period in one module. Besides, this design makes the intricate temporal patterns disentangled, benefiting the temporal variation modeling. However, it is notable that the ID time series is hard to explicitly present two different types of variations simultaneously. To tackle this obstacle, we extend the analysis of temporal variations into the 2D space. Concretely, as shown in Figure [. we can reshape the 1D time series into a 2D tensor, where each column contains the time points within a period and each row involves the time points at the same phase among different periods. Thus, by transforming ID time series into set of 2D tensors, we can break the bottleneck of representation capability in the original ID space and successfully unify the intraperiod- and interperiod variatioon in 2D space, obtaining the temporal 2D-vvriattions.
Technically, based on above motivations, we go beyond previous backbones and propose the TimesNer as new fask-general model for time series analysis. Empowering by TimesBlock, TimesNet can discover the multi-periodicity of time series and capture the corresponding temporal variations in a modular architecture. Concretely, TimesBlock can adaptively transform the 1D time series into set of 2D tensors based on learned periods and further capture intraperiod- and interperiod-varian variations in the 2D space by a parameter-efficient inception block. Experimentally, TimesNet achieves the consistent state-of-the-arr in five mainstream analysis tasks, including short- and long-term term forecasting, imputation, elassification and anomaly detection. Our contributions are summarized in three folds:
Motivated by find out a modular way for temporal variation modeling. By transforming the ID time series into 2D space, we can present the intraperiod- and interperiod- varitations simultaneously.
and complex interactions within and between periods,
2

---

Published as a conference paper at ICLR 2023
We propose the TimesNet with TimesBlock to discover multiple periods and capture temporal 2D-variade -vrriationss from the transformed 2D tensors by a parameter- inception block.
As a task-general foundation model, TimesNet achieves the consistent state-of-the-art in five mainstream time series analysis tasks. Detailed and insightful visualizations are included.
RELATED WORK
As key problem of time series analysis, temporal variation modeling has been well explored.
Many classical methods assume that the temporal variations follow the pre-defined patterns, such as ARIMA (Anderson & Kendall 1976), Wolt-rinter (Hyndman & Athanasopoulos 2018) and Prophet (Taylor & Lethamm 2018). However, the variations of real-world time series are usually too complex to be covered by pre-defined patterns, limiting the practical applicability of these classical methods.
In recent years, many deep models have been proposed for temporal modeling, such MLP, TCN,
RNN-based models (Hochreiter & Schmidhuber [997] Lai et al. 2018 Franceschi et al. 2019)
Technically, MLP-based methods (Oreshkin et al. 2019: Challu et al. 2022 Zeng et al. 2023 Zhang
ctt al., 2022) adopt the MLP along the temporal dimension and encode the temporal dependencies into the fixed parameter of MLP layers. The TCN-based (2019) methods capture the temporal variations by convolutional kernels that slide along the temporal dimension. The MANN-based methods
(Hochreiter & Schmidhuber 1997; Lai al., 2018 Gu et al. 2022) utilize the recurrent structure and capture temporal variations implicitly by state transitions among time steps. Note that none of these methods consider the temporal 2D-variations derived by periodicity, which is proposed in this paper.
Besides, Transformers have shown great performance in time series forecasting (Zhou et al. 2021
Liu et al. 2021a Wu et al. 2021 Zhou et al. 2022). With attention mechanism, they can discover the temporal dependencies among time points. Especially, Wu et al. present the Autoformer with Auto- Correlation mechanism to capture the series-wise temporal dependencies based on the learned periods. In addition, to tackle the intricate temporal patterns, Autoformer also presents a deep decomposition architecture to obtain the seasonal and trend parts of input series. Afterward, FRDformer (Zhou design to enhance the seasonal-trend decomposition and
et al. 2022) employs the mixture-of-exe presents sparse attention within the frequency domain. Unlike previous methods, we ravel out the intricate temporal patterns by exploring the nulti-periodicity of time series and capture the temporal 2D-variations in 2D space by well-acknowledget computer vision backbones for the first time.
It is also notable that, different from previous methods, we no longer limit to specific analysis task and attempt to propose a task-general foundation model for time series analysis.
3 TIMESNET
As aforementioned, based on the multi-periodicity of time series, propose the TimesNet with modular architecture to capture the temporal patterns derived from different periods. For each period, to capture the corresponding intraperiod- and cntrperiod-variatiionn
design a TimesBlock within the TimesNet, which can transform the ID time series into 2D space and simultancously model the two types of variations by parameter-efficient inception block.
3.1 TRANSFORM ID-VARIATIONS INTO 2D- VA-IIIIATONNS
As shown in Figure I each time point involves two types of temporal variations with its adjacent area and with the same phase among different periods simultaneously, namely intraperiod- and interperiod- variations. However, this original 1D structure of time series can only present the variations among adjacent time points. To tackle this limitation, we explore the two-dimension structure for temporal variations, which can explicitly present variations within and between periods, thereby with more advantages in representation capability and benefiting the subsequent representation learning.
Concretely, for the length-T time series with C recorded variates, the original ID organization is XID E RxC To represent the interperiodvvariatinn we need discover periods first. Technically, we analyze the time series in the frequency domain by Fast Fourier Transform (FFT) as follows:
A -Avg [Amp (FFT(XI)))) (fi...., ,fx) = arg Topk (A), = f.e(1.-. .FD)
T
i E (1, ,.)
fr
(1)
3

---

Published as a conference paper at ICLR 2023
ID-variations
Frequency
Voke
Amplaude
Frequency
Capture Temporal 2D-variations by Kernels
Frequency
Time
Frequency
Fraquency
Reshape
Temporal 2D-variations
Reshape
Myw www
Frequency 2
Reshaps
Reshape
Paried
m
Intraperiad variation
ww
Penind 2
Frequency
Period 3
Interperiodwvariation
Figure 2: A univariate example to illustrate 2D structure in time series. By discovering the periodicity, can transform the original ID time series into structured 2D tensors, which can be processed by 2D kernels conveniently. By conducting the same reshape operation to all variates of time series, we extend the above process to multivariate time series.
Here, FFT(.) and Amp(-) denote the FFT and the calculation of amplitude values. A E IRT represents the calculated amplitude of each frequency, which is averaged from C' dimensions by Avg(-). Note that the j-th value A, represents the intensity of the frequency-j periodic basis function, corresponding to the period length 17 Considering the sparsity of frequency domain and to avoid the noises brought by meaningless high frequencies (Chatfield 1981 Zhou et al. (022), we only select the top-k amplitude values and obtain the most significant frequencies (/,,... fu] with the unnormalized amplitudes {A f....... AAxh ,A
where k is the hyper-parameter These selected frequencies also correspond period lengths {pI.- ... .pk Due the conjugacy of frequency domain, we only consider the frequencies within {I, 12... We summarize Equation
I as follows:
A, -.. (ri, P&3 = Period(XIn).
Based on the selected frequencies (fi.... f&} and corresponding period lengths {p1,-.. pk), we can reshape the ID time series XID E RTxC into multiple 2D tensors by the following equations:
X Reshapep...t (Padding(X1p)),
[1,....A)..
where Padding(-) is to extend the time series by zeros along temporal dimension make it compati- ble for Reshapep.. (-), where p; and fo represent the number of rows and columns of the transformed 2D tensors respectively. Note that X E R denotes the i-th reshaped time series based on frequency-fi. ft. whose columns and rows represent the intraperiod- vvriatton and interperiod-vaaiien variation under the corresponding period length p; respectively. Eventually, as shown in Figure 2 based on the
selected frequencies and estimated periods, we obtain a set of 2D tensors (X ... X which indicates different temporal 2D-variations derived by different periods.
It is also notable that, this transformation brings two types of localities to the transformed 2D tensors, that is localities among adjacent time points (columns, intraperiod-variane (rows, cnerperiod-variattonn Thus, the temporal 2D-variations can be easily processed by 2D kernels.
and adjacent periods
3.2 TIMESBLLOCK
As shown in Figure we organize the TimesBlock in a residual way (He et al. 2016). Concretely, for the length-T ID input time series XID E '' we project the raw inputs into the deep features X E RRT 72'xx by the embedding layer X EmbeddX at the very beginning. For the l-th layer of TimesNet, the input is X X E RR x and the process can be formalized as:
X iD TimesBlock (x(=") + X
(4)
As shown in Figure 3 for the I-th TimesBlock, the whole process involves two successive parts: capturing temporal 2D-variations and adaptively aggregating representations from different periods.
Capturing temporal 2D-variations Similar to Equation [ we can estimate period lengths for deep features | by Period(-). Based estimated period lengths, transform the 1D time series
4

---

Published as a conference paper at ICLR 2023
XiD
TimesBlock
Adaptive Aggregation
(X
TimesBlock
Space
Poramater-fficient Inception block
(X
X
Inception Block
TimesBlock
Reshape
Reshope
Bock
Space
M P
lay Weo
TimesBlock
FFT for Pariods
Softmax
ID Space
Time
Figure 3: Overall architecture of TimesNet. TimesNet is stacked by TimesBlocks in residual way. TloesBlocks can capture various temporal 2D-variations from different reshaped tensors by parameter-efficien inception block in 2D space and fuse them based normalized amplitude values.
into 2D space and obtain a set of 2D tensors, from which we can obtain informative representations by parameter-efficeen inception block conveniently. The process is formalized follows:
A 4-1, {fi... fich, (pi. fPi,
Pkj Period (x
x L 2D
Reshapep..ft (Padding(X (x iE (1,
X
Inception
x
iE -1, ,
X ID
Trunc [Reshaper,drs x
T (
iE {1,.... .k),
where X E IRP.Xfexdam is the i-th transformed 2D tensor. After the transformation, we process the 2D tensor by a parameter-efficien inception block (Szegedy et al., 2015) as Inception(-).. which vision backbones. Then
involves multi-scale 2D kernels and is one of the most well-acknowleddee we transform the learned 2D representations X back to ID space X E Tx''dd for aggregation, where we employ Trunc(-) to truncate the padded series with length (p; x fi) into original length T.
Note that benefiting from the transformation of ID time series, the 2D kernels in the inception block can aggregate the multi-scale intraperiod saaperriod-vriittooo variation (columns) and interperiod-varte varrreriod-vaiaationn (rows) simultancously, covering both adjacent time points and adjacent periods. Besides, adopt shared inception block for different reshaped 2D tensors { (x/1) x!A J improve parameter efficiency, which can make the model size invariant to the selection of hyperr-prameele k.
Adaptive aggregation Finally, we need to fuse k different 1D ID-representations (X ID for the next layer. Inspired by Auto-Correlation (Wu al., 2021). the amplitudes A can reflect the relative importance of selected frequencies and periods, thereby corresponding to the importance of each transformed lensor. Thus, we aggregate the ID-representations based on the amplitudes:
ID L
A
Softmax (A
A/A
b Σ AL-1 x 8Li ID.
(6)
X ID
Since the variations within and between periods are already involved in multiple highly-structured 2D tensors, TimesBlock can fully capture multi-scale temporal 2D-variations simultaneously. Thus, TimesNet can achieve more effective representation learning than directly from 1D time series.
Generality in 2D vision backbones Benefiting from the transformation of ID time series into temporal 2D-variations, we can choose various computer vision backbones replace the inception block for representation learning, such as the widely-used ResNet (He et al.] (016)) and ResNeXt (Xie et al. (017),, advanced ConvNeXt (Liu et al. 2022b) and attention-based models (Liu et al. 2021 Thus, our temporal 2D-variation design also bridges the ID time series to the booming 2D vision backbones, making the time series analysis take advantage of the development of computer vision community. In general, more powerful 2D backbones for representation learning will bring better performance. Considering both performance and efficiency (Figure right). we conduct the main experiments based on the parameter-efficient inception block shown in Equation
5

---

Published as a conference paper at ICLR 2023
4 EXPERIMENTS
To verify the generality of TimesNet, we extensively experiment on five mainstream analysis tasks, including short- and long-term forecasting, imputation, classification and anomaly detection.
Implementation Table I is summary of benchmarks. More details about the dataset, experiment implementation and model configuration can be found in Appendix A
Table 1: Summary of experiment benchmarks.
Tasks
Benchmarks
Metries
Series Length
Long-term:
ETT (4 subsets), Electricity.
96~720
MSE, MAE
Forecasting
Truffic, Weather, Exchange, ILI
(ILI: 24--60)
Short-term:
M4 subsets)
SMAPE, MASE, OWA
6~44
Imputation
ETT (4 subsets), Electricity, Weather
MSE, MAE
96
Classification
UEA (10 subsets)
Accuracy
19511751
Anomaly Detection
SMD, MSI, SMAP, SWaT, PSM
Precision, Recall, F1-Socre
100
Baselines Since we attempt to propose foundation model for time series analysis, extensively compare the well-acknowledded and advanced models in all five tasks, including the RNN-based mod-
els: LSTM (197)), LSTNet (2018) and LSSL. (2022); CNN-based Model: TCN (2019); MLP-based models: LightTS (2022) and DLinear (2023): Transformer-basee models: Reformer (022). Informer (2021) Pyraformer (2021a). Autoformer (2021). FEDformer (2022), Non-stationary Transformer (022a) and ETSformer (2022). Besides, we also compare the state-of-the-ar models for each specific task, such as N-HiTS (2022) and N-BEATS (2019) for short-term forecasting, Anomaly Transformer (202T) for anomaly detection, Rocket (2020) and Flowformer (2022) for classification and etc. Overall, more than 15 baselines are included for a comprehensive comparison.
Forecoring (MSE)
Anonaly Detection
ConeNNat
Reshant
Sharterm Foreccaing (SMAPE)
Classfcarion (Accrooy)
KIVAL
Temester john)
U/ileme 2002)
Iueefer
Incaption (ndeppadeet))
SwisBlick
Uyelt (203))
Dameer (10))
1DAume (2002)
Anomaly Detecion
Iepulation (MSE)
(#18cor))
Salome (2033)
Training Time (iwe/rea)
Figure 4: Model performance comparison (left) and generality in different vision backbones (right).
4.1
MAIN RESULTS
As foundation model, TimesNet achieves consistent state-of- tht-hhh-rtt art performance on five mainstream analysis tasks compared with other customized models (Figure left). The full efficiency comparison is provided in Table of Appendix. Besides, by replacing the inception block with more powerful vision backbones, we can further promote the performance of TimesNet (Figure rightt, confirming that our design can make time series analysis take advantage of booming vision backbones.
4.2 SHORT- AND LONG-TERM FORECASTING
Setups Time series forecasting is essential in weather forecasting, traffic and energy consumption planning. To fully evaluate the model performance forecasting, we adopt two types of benchmarks, including long-term and short-term forecasting. Especially for the long-term setting, we follow the benchmarks used in Autoformer (202I), including ETT (Zhou et al. 2021). Electricity (UCT), Traffic (PeMS), Weather (Wetterstation), Exchange (Lai et al. 2018) and ILI (CDC), covering five real-world applications. For the short-term dataset, we adopt the M4 (Spyros Makridakis 2018). which contains the yearly, quarterly and monthly collected univariate marketing data. Note that each dataset in the long-term setting only contains one continuous time series, where we obtain samples by sliding window, while M4 inyolves 100,000 different time series collected in different frequencies.

---

Published as a conference paper at ICLR 2023
Table 2: Long-term forecasting task. The past sequence length is 36 for ILI and 96 for the others. All the results are averaged from 4 different prediction lengths, that is {24, 36, 48, 60) for ILI and {96, 192, 336, 720) for the others. See Table 13 Appendix for the full results.
Models
TimesNet
ETSformer
LighTTS
DLincar
FEDformer
Stationary
Autoformer Pyraformer
Informer
LogTrans
Reformer
(Ours)
2022
2022
2023
2022
2022n
2021
2021a
20211
2019
2020
Metric
MSE MAE
MSE MAE
MSE MAE
MSE MAE
MSE MAE
MSB MAE
MSE MAE MSE MAE
MSE MAE
MSE MAE
MSE MAE
ETTml
0.400 0.406
0.429 0.425
0.435 0.810044407000000444444440000000
0.45
0.5000 0.500..88...55111111
0.60
0.929
0.671 1.5550.00
ETTm2
(.291 0.333
0.869
0.810
1.479
ET1bl
0.44500.5504..
0.75
ETTh2
0.427
1.729/2.686 1.494 6.736 2.191
Blectricity
0.19 0.295
0,442
Traflic
0.620
0.422
Weather 0.259 0.287
0.271
0.656
Exchange
0.441...445
0.410 htt.::40.00.........000000000000
0.998|1.422 0.968 1,280 0.932
2. 0:933 2.497 1.004/7.382 2.666/66666616.. 1.090|2.847 1.144 2.077 0.914 2.0.035....051111 4.839 1,485 1.445
Table 3: Short-term forecasting task on M4. The prediction lengths are in [6, 48] and results are weighted averaged from several datasets under different sample intervals. See Table 14 for full results.
Models TimesNet N-IETS N-BEATS ETSformer LighTTS DLinear FEDformer Stationary Autoformer Pyrnformer Informer LogTrans Reformer
(Ours)
2022
2019
2022
2022
2023
2022
B0222
2021
2021a
202T
2019
2020
SMAPE
11.829
11.851
14.086
MASE
1,599
2.095
OWA
0.851
0.861
0.855
1,172
1.051
1.051
0.918
0.930
0.939
1,480
1.230
1.378
1.775
Results TimesNet shows great performance in both long-term and short-term settings (Table 3) Concretely, TimesNet achieves state-of-the-art in more than 80% of cases in long-term forecasting (T3) For the M4 dataset, since the time series are collected from different sources, the temporal variations can be quite diverse, making forecasting much more challenging. Our model still performs best in this task, surpassing extensive advanced MLP-based and Transformer-based models.
4.3 IMPUTATION
Real-world systems always work continuously and are monitored by automatic observation equipment. However, due to malfunctions, the collected time series can be partially missing, making the downstream analysis difficult. Thus, imputation is widely-used in practical applications. this paper, we select the datasets from the electricity and weather scenarios ETT (Zhou et al. 2021), Electricity (UCT) and Weather (Wetterstation), where the data-missing problem happens commonly. To compare the model capacity under different proportions of missing data, we randomly mask the time points in the ratio of (1.55% 25%, 37. 37.5% 50%
Setups
benchmarks, including
Results Due to the missing time points, the imputation task requires the model to discover underly- ing temporal patterns from the irregular and partially observed time series, As shown in Table 4 our proposed TimesNet still achieves the consistent ofate-of-the-art in this difficult task, verifying the model capacity in capturing temporal variation from extremely complicated time series.
4.4 CLASSIFICATION
Setups Time series classification can be used in recognition and medical diagnosis (Moody et al.) 2011). We adopt the sequence-level classification to verify the model capacity in high-level represen- tation learning. Concretely, we select 10 multivariate datasets from UEA Time Series Classification Archive (Bagnall et al.) 2018), covering the gesture, action and audio recognition, medical diagnosis by heartbeat monitoring and other practical tasks. Then, we pre- pre-process the datasets following the descriptions in (Zerveas al. 2021 where different subsets have different sequence lengths.
7

---

Published as a conference paper at ICLR 2023
Table Imputation task. We randomly mask (12.5%, 33.537.5% 50%) time points in length-96 time series. The results are averaged from 4 different mask ratios. See Table [ for full results.
Models
TimesNet
ETSformer
LighTTS
DLinear
FEDiormer
Stationary Autoformer Pyraformer
Informer
Log Trams
Reformer
(Ours)
2022
2022
2023
2022
202A4
2021
2021a
2021
2019
2020
Metric
MSE MAE
MSE MAD MSE MAE MSE MAE MSE MAI
MSE MAE MSE MAE MSE MAE MSE MAD MSE MAE MSE MAE
ETTml
00270.107000
000.000000000000000... 15 570 www. 0.s.s/0.0000.54400055..
ETTm2
0.022
0.165 http://wwww..o....ooooooooooooaa
ETTh1
ETTh2
0.049 010.00000....0000000000000000000 ottp://..............o.....oorr.
0
Weather f.0.00.ssppo777..77700000......0 htgp://0.e................ca.aaa http://ww..............oooooooo.
0.057
Results As shown in Figure 5
Classcaal Marhods
TCNiBosed
MLPilosse
TimesNet achieves the best perfor-
RNN44ared
Tranaforma--aued
Ours
mance with an average accuracy of 73.6%, surpassing the previous ofate-of-the-art classical method Rocket (72.5%) and deep model Flowformer 7. is also no- table that the MLP-based model DLinear fails in this classification task (67.5%) which performs well in some time series forecasting datasets. This is because DLinear only adopts one-layer MLP model on the temporal dimension, which might be suitable for some autore- gressive tasks with fixed temporal dependencies but will degenerate lot in learning high-level representations. In contrast, TimesNet unifies the temporal 2D-variation in 2D space, which is convenient to learn informative representation by 2D kernels, thereby benefiting the classification task that requires hierarchical representations.
Average Accurasy (%)
72.5
71.8
789
71.
70.
7277
70.7
67.5
glowel
lightt
Figure 5: Model comparison classification. "*." in the Transform- ers indicates the name of =former. The results are averaged from 10 subsets of UEA. See Table 17 Appendix for full results.
4.5 ANOMALY DETECTION
Setups Detecting anomalies from monitoring data is vital to industrial maintenance. Since the anomalies are usually hidden in the large-scale data, making the data labeling hard, we focus on unsupervised time series anomaly detection, which is to detect the abnormal time points. We compare models on five widely-used anomaly detection benchmarks: SMD (Su et al. 2019). MSL (Hundman) et al. 2018), SMAP (Hundman et al. 2018), SWaT (Mathur & Tippenhauer 2016 PSM (Abdulaal ct al. 2021), covering service monitoring, space & earth exploration, and water treatment applications. Following the pre- pre-processing methods in Anomaly Transformer (22II) we split the dataset into consecutive non-overlapping segments by sliding window. In previous works, the reconstruction is a classical task for unsupervised point-wiss, wise representation learning. where the reconstruction error is natural anomaly criterion. For fair comparison, we only change the base models for reconstruction and use the classical reconstruction error as the shared anomaly criterion for all experiments.
Results Table demonstrates that TimesNet still achieves the best performance in anomaly de- tection, outperforming the advanced Transformer-ban -based models FEDformer (2022) and Autoformer (2021). The canonical Transformer performs worse in this task (averaged Fi-score 76.88%). This may come from that anomaly detection requires the model to find out the rare abnormal temporal patterns (Lai et al.) 2021), while the vanilla attention mechanism calculates the similarity between each pair of time points, which can be distracted by the dominant normal time points. Besides, by taking the periodicity into consideration, TimesNet, FEDformer and Autoformer all achieve great performance. Thus, these results also demonstrate the importance of periodicity analysis, which can highlight variations that violate the periodicity implicitly, further benefiting the anomaly detection.
8

---

Published as a conference paper at ICLR 2023
Table 5: Anomaly detection task. We calculate the F1-score (as %) for each dataset. means the *former. A higher value of Fl-score indicates better performance. See Table 15 for full results.
Models
TimesNet
TimesNet
ETS.
FED.
LighTTS
DLineur
Stationury
Auto.
Pyrı.
Anomaly
In.
Re.
LogTrans
Trans.
(ResNeXt) (Inception)
2022
20122
A022
2023
A0220
2021
A121a
2021
2021
2020
2019
2017
SMD
85.81
85.12
83.13
85.08
82.53
77.10
84.72
85.11
83.04
85.49
81.65
75.32
76.21
79.56
84.18
78.95
77.50
79.05
84.86
83.31
79.57
78.68
SMAP
71.52
70.85
69.50
70.76
69:21
69.26
71.09
71.12
71.09
71.18
69.92
70.40
69.97
69.70
SWaT
91.74
92.10
84.91
93.19
93.33
87.52
79.88
92.74
91.78
83.10
81.43
82.80
80.52
80.37
PSM
97.47
95.21
91.76
97.23
97.15
93.55
97.29
93.29
82.08
79.40
77.10
73.61
76.74
76.07
Avg Fl
86.34
85,49
82.87
84,97
84,23
82:46
82,08
84.26
82.57
80.50
78.83
77.31
76.60
76.88
We replace the joint criterion in Anomaly Transformer (2021)
with reconstruction error for fair comparison.
4.6
MODEL ANALYSIS
Representation analysis We attempt to explain model performance from the representation learning aspect. From Figure 6 we can find that the better performance in forecasting and anomaly detection corresponds to the higher CKA similarity (2019), which is opposite to the imputation and classification tasks. Note that the lower CKA similarity means that the representations different layers, namely hierarchical representations. Thus, these results also indicate the property of representations that each task requires. As shown in Figure [6] TimesNet can learn appropriate representations for different tasks, such as low-level representations for forecasting and reconstruction in anomaly detection, and hierarchical representations for imputation and classification. FEDformer (2022) performs well in forecasting and anomaly detection tasks but fails in learning hierarchical representations, resulting poor performance in imputation and classification. These results also verify the lask-genrrality of our proposed TimesNet as foundation model.
distinguishing among
contrast,
0.20
0.30
Ascurcoy
E
0.13
0.00
CA Snnlery
CVA Sialurty (b) Imputation [Eechrick Mask 37.5%)
CMA Send-dy
CA Sondery
(c) Forecosting (Weather hputt6prredie3366
[e] Classiication (FEMS-SF)
[d] Asomaly Datection (SMD)
Figure 6: Representation analysis in four tasks. For each model, we calculate the centered kernel alignment (CKA) similarity (2019) between representations from the first and the last layers. A higher CKA similarity indicates more similar representations. TimesNet is marked by red stars.
Temporal 2D-variations We provide a case study of temporal 2D-variations in Figure 7 We can find that TimesNet can capture the cellipperiodiicties pre- cisely. Besides, the transformed 2D tensor is highly structured and informative, where the columns and rows can reflect the localities between time points and periods respectively, supporting our motivation in adopting 2D kernels for representation learning. See Appendix for more visualizations.
Electricity
4.0
230
Frequency
-1.0
Value
Bar
Frequency~9
Period =72
Fraquency 15
Period =40
Period
-24
CONCLUSION AND FUTURE WORK
Figure 7: A case of temporal 2D -D-vaiiations.
This paper presents the TimesNet as a task-general foundation model for time series analysis. Mo- tivated by the multi-periodicity. TimesNet can ravel out intricate temporal variations by a modular architecture and capture intraperiod- and interperiod-variine variations in 2D space by a parameter-efficient inception block. Experimentally, TimesNet shows great generality and performance in five main- stream analysis tasks. In the future, we will further explore large-scale pre- pre-traine meeitraining methods in time series, which utilize TimesNet as the backbone and can generally benefit extensive downstream tasks.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported by the National Key Research and Development Plan (2020AAA0109201) National Natural Science Foundation of China (62022050 and 62021002), Civil Aircraft Research Project (MZJ3-2N21) Beijing Nova Program (Z201 100006820041), and CCF-Ant Group Green Computing Fund.
REFERENCES
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. KDD, 2021.
O. Anderson and M. Kendall. Time-series. 2nd edn. R. Stat. Soc. (Series D), 1976.
Anthony J. Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron George Bostrom, Paul Southam, and Eamonn J. Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint arXiv: 11111111005555 2018.
Donald Berndt and James Clifford. Using dynamic time warping to find patterns in time series, In KDD Workshop, 1994.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurlPS, 2020.
CDC. html
Illness.
Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza, Max Mergenthaler, and Artur Dubrawski. N-hits: Neural hierarchical interpolation for time series forecasting. arXiv preprint
arXiv:2201.12886 2022.
Chris Chatfield. The analysis of time series: an introduction. 1981.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. KDD, 2016.
Angus Dempster, Franccois Petitjean, and Geoffrey Webb. Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Min. Knowl. Discov,, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition scale. In ICLR, 2021.
Yean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. In NeurIPS, 2019.
Milton Friedman. The interpolation of time series by related series. J. Amer. Statist. Assoc, 1962.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In ICLR, 2022.
Kaiming He, Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016.
Yangdong He and Jiabao Zhao. Temporal convolutional networks for anomaly detection in time series. J. Phys. Conf. Ser, 2019.
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 1997.
10

---

Published as a conference paper at ICLR 2023
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Söderström. Detecting spacecraft anomalies using Istms and nonparametric dynamic thresholding. KDD, 2018.
Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization. In ICI.R, 2015.
URL http://arxiv org/ abs / 6980.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In ICLR.
2020.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neural network representations revisited. ICML, 2019.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In SIGIR, 2018.
Kwei-Heng Lai, D. Zha, Junjie Xu, and Yue Zhao. Revisiting time series outlier detection: Definitions and benchmarks. In NeurlPS Dataset and Benchmark Track, 2021.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer time series forecasting. In NeurlPS, 2019.
Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Philos. Trans. Royal Soc. A, 2021.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex Liu, and Schahram Dust- dar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In ICLR, 2021a.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Rethinking the stationarity in time series forecasting. In NeurlPS, 2022a.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021b.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.
A convnet for the 2020s. CVPR, 2022b.
Aditya P. Mathur and Nils Ole Tippenhauer. Swat: a water treatment testbed for research and training on ICS security. In CySWATER, 2016.
George B. Moody, Roger G. Mark, and Ary L. Goldberger. Physionet: Physiologic signals, time series and related open source software for basic, clinical, and applied research. EMBC, 2011.
Boris Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. ICLR, 2019.
Adam Paszke, S. Gross, Francisco Massa, A. Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Z. Lin, N. Gimelshein, L. Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Vito Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-perffrmance deep learning library. In NeurIPS, 2019.
PeMS. Traffic.
dot ca. /ov/
Lifeng Shen, Zhuocong Li, and James T. Kwok. Timeseries anomaly detection using temporal hierarchical one-class network. In NearIPS, 2020.
Spyros Makridakis. M4 dataset, 2018. URL httpss//gittub..cc//aaooottttttp M4 hat-meehhdd//teeeeaatttteeeeee
Ya Su, Y. Zhao, Chenhao Niu, Rong Liu, W. Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. KDD, 2019.
11

---

Published as a conference paper at ICLR 2023
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, D. Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. CVPR, 2015.
Sean J Taylor and Benjamin Letham. Forecasting scale. Am, Stat., 2018.
UCL
Electricity.
uci http://c........cemmmmmedaaeaae. edd.emm///m///ttsstt/////
E EleetricatyLlobbb.aaraa2222.1111
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurlPS, 2017.
Wetterstation. Weather. https: http://wwwwbb.m bgc- www.wbgc-jena..ooom www.aaaa.a.....................a de/setter/
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven C. H. Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting. arXiv preprint urXiv:2202.01381 2022.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer:
Decomposition transformers
with Auto-Correlation for long-term series forecasting, In NeurIPS, 2021.
Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Flowformer: Linearizing transformers with conservation flows. ICML, 2022.
Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. CVPR, 2017.
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In ICLR, 2021.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? 2023.
G. Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. KDD, 2021.
Zhang, Yizhuo Zhang, Wei Cao. Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprint arXiv:2207.01 186, 2022.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML, 2022.
A IMPLEMENTATION DETAILS
We provide the dataset deseriptions and experiment configurations in Table and 7 All experi- ments are repeated three times, implemented in PyTorch (Paszke et al. 2019) and conducted on a single NVIDIA TITAN RTX 24GB GPU. To make the model handle various dimensions of in- put series from different datasets, we select the dmodel based on the input series dimension C' by min max ( 2 C dmin), }, dmux) ) (see Table 7 for details of dmin and d. This protocol can make the model powerful enough for multiple variates and also keep the model size compact.
All the baselines that we reproduced are implemented based configurations of the original paper or official code. It is also notable that none of the previous methods are proposed for general time series analysis. For fair comparison, we keep the input embedding and the final projection layer the same among different base models and only evaluate the capability of base models. Especially for the forecasting task,
MLP on temporal dimension get the initialization of predicted future. Since we focus on the temporal variation modeling, we also adopt the Series Stationarization from Non-stationary Transformer (Liu et al. 2022a) eliminate the affect the distribution shift.
12

---

Published as a conference paper at ICLR 2023
For the metries, adopt the mean square (MSE) and mean absolute enror (MAE) for long-term forecasting and imputations. For anomaly detection, we adopt the Fl-score, which is the harmonic mean of precision and recall. For the short-term forecasting, following the N-BEATS (Oreshkin et al.) 2019), we adopt the symmetric mean absolute percentage error (SMAPE), mean absolute scaled error (MASE) and overall weighted average (OWA) the metrics, where OWA is special metric used in M4 competition. These metrics can be calculated as follows:
200
H
X;
X,
100
X; X
SMAPE
MAPE H
H
+ X
X; Zj=m+1
X,1
SMAPE
MASE
MASE
OWA
X j--ml
SMAPENaive2
MASENaive2
m
where m is the periodicity of the data. X, "x are the ground truth and prediction results of the future with H time pints and C' dimensions. X, means the i-th future time point.
B HYPER-PARAMER PARAMETER SENSITIVITY
We introduce hyper- papamm-rrreeter Equation to select the most significant frequencies. We provide the sensitivity analysis for these two syper--arameters in Figure 8 We can find that our proposed TimesNet can present a stable performance under different choices all four tasks.
(x10
(%)
F1-Szore
ETTKI
ETim2
ETTR
Bechicky
Weather
SMD
SWAT
64
Equation (1) Imputation Task
Equation (1] for Anomaly Detection Tosk
13.0
(Weighted Average)
95
12.5
Accuracy
90
SMAPE
85
Spolensrrbbrdight
12.0
80
FoceDanecion
70
in Equation (1) for Classification Task
Equation [1] for Shortterm Forecasting Task
Figure Sensitivity analysis of byper-paramedt k imputation and anomaly detection task. Especially, we select the 37.5% mask ratio for choose the two largest subsets: SpokenArobieDDgiie and Feteletention for evaluation. For short-term forecasting, we adopt the weighted average for sensitivity analysis.
experiments. For classification, we
Besides, from Figure g
also find out the following observations:
For the low-level modeling tasks, such as forecasting and anomaly detection, the selection will affect the final performance more. This may come from that & will directly affect the amount of information in deep representations.
13

---

Published as a conference paper at ICLR 2023
Table 6: Dataset descriptions. The dataset size is organized in (Train, Validation, Test).
Tasks
Dataset
Dim
Series Length
Dataset Size
Information (Frequency)
ETTml, ETTm2
7
[96. 192, 336. 720)
(34465, 11521, 11521)
Electricity (15 mins)
ETThl, ETTh2
7
[96. 192, 336. 720)
(8545, 2881)
Electricity (15 mins)
Forecasting
Electricity
321
(96, 192, 336, 720
(18317, 2633, 5261)
Electricity (Hourly)
(Long-term)
Traffic
862
(96, 192, 336, 720)
(12185, 1757, 3509)
Transportation (Hourly)
Weather
21
(96, 192, 336, 720)
(36792, 5271, 10540)
Weather (10 mins)
Exchange
8
(96, 192, 336, 720}
(5120, 665, 1422)
Exchange rate (Daily)
ILI
|
7
(24, 36,48, 60)
(617, 74, 170)
Illness (Weekly)
M4- Yearly
(23000, 23000)
Demographic
M4-Quarterly
1
8
(24000, 0, 24000)
Finance
Forecasting
M4-Monthly
18
(48000, 0, 48000)
Industry
(short-term)
M4-Weakly
13
(359, 0, 359)
Macro
M4-Daily
(4227, O, 4227)
Micro
M4-Hourly
48
(414, 0, 414)
Other
ETTmI,ETTm2
7
96
(34465, 11521, 11521)
Electricity (15 mins)
ETThi, ETTh2
7
96
(8545, 2881, 288 I)
Electricity (15 mins)
Imputation
Electricity
321
96
(18317, 2633, 5261)
Electricity (15 mins)
Weather
96
(36792, 5271, 10540)
Weather (10 mins)
EdhanssCommeetraation
3
1751
(261, O, 263)
Alcobol Industry
FaceDetection
144
62
(5890, 0, 3524)
Face (250Hlz)
Handwriting
3
152
(150, O, 850)
Handwriting
Heartheat
405
(204, 0, 205)
Heart Beat
Classification
JapeneseVowels
12
29
(270, 370)
Voice
(UEA)
PEMS-SF
9663
144
(267, O, 173)
Transportation (Daily)
SeliRegulationSCPI
6
896
(268, 0, 293)
Health (256Hz)
SelfRegulationSCP2
7
1152
(200, O, 180)
Health (256Hz)
SpokenAraabiDiigiss
13
93
(6599, 2199)
(1155HH))
UWaveGestureLibrayy
3
315
(120, 0, 320)
Gesture
SMD
38
100
(566724, 141681, 708420)
Server Machine
Anomaly
MSI.
55
100
(44653, 1 11664, 73729)
Spacecraft
Detection
SMAP
25
100
(108146, 27037, 427617)
Spacecraft
SWaT
51
100
(396000, 99000, 449919)
Infmstructure
PSM
25
100
(105984, 26497, 87841)
Server Machine
Table 7: Experiment configuration of TimesNet. All the experiments use the ADAM (015) optimizer with the default hyperparameter configuration for (81, 32) (0.9, 0.999).
Model Hyper-parameter
Training Process
Tasks / Configurations
k (Equ.
Layers
dmi
dmxx
LR"
Loss
Batch Size
Epochs
Long-term Forecasting
2
32
512
10
MSE
10
Short-term Forecasting
2
16
64
10-3 3
SMAPE
16
10
Imputation
|
2
64
128
10-3
MSE
10
Classification
3
|
2
32
64
10
3
Cross Entropy
16
30
Anomaly Detection
3
3
32
128
10
4
MSE
128
10
dmodet = min minnmax/ (2 [log C1, dmin }, dmax), where C' input series dimension.
LR means the initial learning rate.
14

---

Published as a conference paper at ICLR 2023
For the high-level modeling tasks, such as classification and imputation, the model perfor- mance will be more robust to the selection of k, since the key to these tasks is to extract hierarchical representations.
Giving consideration to both efficiency and performance, we set kt = 3 for imputation, classification and anomaly detection and k 5 for short-term forecasting.
C ABLATION STUDIES
To elaborate the property of our proposed TimesNet, we conduct detailed ablations on the representa- tion leaning in 2D space, model architecture and adaptive aggregation.
2D space As shown in Table 8 replacing the inception block with more powerful blocks will bring further performance promotion, such as ResNeXt (Xie et al. 2017). Swin Transformer (Liu et al. b021bb and ConvNeXt (Liu et al. 2022b). It is also notable that using the independent parameters will also bring improvement, while this will cause the model size related to the selection of hyper- parameter k. Considering the efficiency and model performance, we choose the parameter-efficient inception block as our final solution. These results also verify that our design bridges the 1D time series analysis with 2D computer vision backbones.
Table 8: Ablations on the representation leaning 2D space, where we replace the parameter-efficient inception with other well-acknowledged vision backbones. See Figure for efficiency comparison.
Datasets
SMD
MSL
SMAP
SWaT
PSM
Avg Fl
Metries
R
F1
F1
P
F1
P
F1
P
F1
(%)
ResNNet
87.64 80.33 83.82
83.99
85.42
84.70
92.09
57.90
71.10
86.59
95.81 90.97
98.24
89.12
93.45
84.81
ResNeXt
88.66 83.14 85.81
83,92
86.42
85.15
92.52
58.29
71.52
86.76
97.32 91.74
98.19
96.76
97.47
86.34
Swin Transformer
88.51
82.22 85.25
87.36
86.93 87.14
92.12
57.60
70.88
89.02
95.81 92.29
98.45
91.80 95.01
86.11
ConyNext
87.89 84.67 86.25
87.31
86.93 87.12
92.42
59.19
72.16
89.05
95.81 92.31
97.99
95.28 96.62
86.89
Inception (Ind")
87.54 81.04 84.17
87.44
86.93
87.18 18
91.92
57.69
70.89
28.98
96.00 92.36
98.11
89.13
93.41
85.60
Inception (Shared") )
87.76 82.63 85.12
82,97
85.42 84.18 91.50 57.80 70.85 88.31 96.24
92.10
98.22
92.21 95.21
85.49
* this paper, we adopt a parameter-efficient design that uses the same parameters different transformed 2D tensors, namely Shared. For comparison, we also compare with the independent design, that uses different parameters for different transformed 2D tensors, namely Ind.
Model architecture We also conduct experiments different architectures. Surprisingly, as shown in T we find that combining with the deep decomposition architecture in Autoformer (Wu et al.) 2021) cannot bring further promotion. These results may come from that in the that the input series already present clear periodicity, our design can capture the -ariiiiat effectively. As for the case that they are without clear periodicity, the model will learn the most significant frequency as 1, where the trend of time series be covered by the intraperiod-varian vvriitaoonoo modeling. These results also verify that our proposed TitesNet handle the analysis for time series without clear periodicity.
Besides, in this paper, to take advantages of the deep representations, we place the transformation from ID-variations to 2D- varriiaaaions in every TimesBlock. Here, we compare our design with the case that only conducts the transformation on the raw input series. From Table 9 we can find that the performance of Transform raw data degenerates a lot (Avg F1-score: 85.49% 84.85%), indicating the advantages of our design.
Table 9: Ablations on model architecture. + decomposition is to combine the deep decomposition architecture proposed by Autoformer (Wu et al.) 2021) with TimesNet. Transform raw data refers conducting the transformation on the original time series, instead of deep features.
Datasets
SMD
MSL
SMAP
SWaT
PSM
Avg Fl
Metrics
P
R
F1
P
R
F1
P
R
Fl
P
R
F1
P
R
Fl
(%)
TimesNet
87.76 82.63 85.12 82,97
85.42
84.18
91.50
$7.80
70.85
88.31
96.24
90
98.22
92.21
95.21
85.49
+ decomposition
87.44 78.49 82.72
83.48
86.47
84.95
91.64
57.34
70.54
89.68
95.60
92.54
98.42
93.12
95.69
85.29
Transform raw data
86.83 79.17 82.82
85.23
86.47
85.84
91.92
57.60
70.82
87.68
95.81
91.57
97.64
89.14
93.20
84.85
15

---

Published as a conference paper at ICLR 2023
Adaptive aggregation As shown in Equation following the design in Autoformer (2021). we adopt the amplitudes after the Softmax function as aggregation weights of processed tensors (X ID X(.) Here we include two variants for comparison. The first is directly-sum, namely
E X The second is remove the Softmax function, that is E A E1. Table 10
demonstrates that our design in adaptive aggregation performs the best.
Table 10: Ablations on adaptive aggregation.
Datasets
SMD
MSL.
SMAP
SWaT
PSM
Avg FI
Metrics
R
F1
R
FI
R
F1
P
R
F1
P
F1
(%)
TimesNet
87.76 82.63 85.12
82.97
85.42
84.18
91.50
57.80
70.85
88.31
96.24
92.10
98 22
92.21
95.21
85,49
Directly-sum
87.10 78.93
82.81
85.81
85,42
85.62
91.35
57.58
70.64
87.28
96.00
91.43
98.13
87.22
92.35
84.57
Removing-Soffne Softmax
87.27 79.31 83.10
83,91
86.47
85.17
91.33
58.57
71.55
87.13
95.81
91.27
98.00
92.48
95.16
85.25
D MORE REPRESENTATION ANALYSIS
To give an intuitive understanding of 2D-variations, we visualize the transformed 2D tensors in Figure From the visualization, can obtain the following observations:
The interperiod-variitionn can present the long-term trends of time series. For example, in the first case of Exchange, the values in each row decrease from left to right, indicating the downtrend of the raw series. And for the ETThl dataset, the values in each row are similar to each other, reflecting the global stable variation of the raw series.
For the time series without clear periodicity, the temporal 2D-variations can still present informative 2D structure. If the frequency one, the crraperroo-vvriiatinn is just the original variation of series. Besides, the ceopeeroodvvaiiatinn can also present the long-term trend, benefiting the temporal variation modeling.
The transformed 2D-variations demonstrate two types of localities. Firstly, for each column (intraperiod intraperiod-variiation)) the adjacent values are close to each other, presenting the locality among adjacent time points. Secondly, for each row cnoeperrodd-vaiaaiionn the adjacent values are also close, corresponding to the locality among adjacent periods. Note that the non-adjacent periods trend, such as the case from the Exchange dataset. These observations of localities also motivate adopt the 2D kernel for representation learning.
be quite different from each other, which can be caused by global
domain
150
30
2
Pariod
Frequency domain
Fraq- 5,30 30.45
Frequency
Bar
Exchange
25
Frequency domain
50-
14
top.1 3.4.8
-1X
Figure 9: More showcases for temporal 2D-variations
16

---

Published as a conference paper at ICLR 2023
E MULTI-PERIODICITY OF TIME SERIES
As shown in Figure 10 we calculate the density of each period length for different datasets. We can find that real-world time series present multi cenliiipeiooicity to extent. For example, the Electricity dataset contains the periods with length-12 12 and length-24.
(a) Electricity
(b) ETTh1
[c] Exchange
(d) Weather
Figure 10: Statistics of period length in experimental datasets. We conduct FFT to the raw data and select the top-6 signifieant frequencies for each length-96 segment. Then, record the corresponding period lengths and plot the normalized density for each period length.
SHOWCASES
To provide clear comparison among different models, we provide showcases the regression tasks, including the imputation (Figure [.] long-term forecasting (Figure [2] and short-term forecasting (Figure 13). Especially in the imputation task, the MLP-based models degenerate lot. This because the input series has been randomly masked. However, the MLP-based models adopt the fixed model parameter to model the temporal dependencies among time points, thereby failing this task.
TimesNat
Stationary
Autoformer
-044
-0.5
-9.6
m
-07
-37
-03%
-08
-0.0
DLinear
LighTTS
FEDformer
Oroan Sivert
-0.4
Cooan Strvet
FIANO
-05
-300
-10
-3.0
Figure 11: Visualization of ETTml imputation results given by models under the 50% mask ratio setting. The black lines stand for ground truth and the orange lines stand for predicted values.
G MODEL. EFFICIENCY ANALYSIS
To summarize the model performance and efficiency, calculate the relative performance rankings for comparing baselines. The rankings are compared from the common models used in all five tasks: LSTM (199)) and LSSL (022)); TCN (2019); LightTS (2022) and DLinear (2023); Reformer (2020). Informer ().2I). Pyraformer (2021a). Autoformer (2021). FEDformer (2022), Non-stationary Transformer (2022a), Sformer (2022) and our proposed NetesNet, namely 13 models in total.
17

---

Published as a conference paper at ICLR 2023
TimesNet
Stationary
Autoformer
0.75
0.80
0.00
0.00
8.00
9.00
-0.00
-350
-3.00
-0.77
-0.2
-0.7
Oroan Mert
Occan Whrch
CTrch
2.0
203
302
600
230
202
500
400
323
200
506
400
DLinear
LightTS
FEDformer
0.00
0.:40
0.04
0.00
-0.0
59
-.77
-.72
-375
330
Figure 12: Visualization of ETTm2 predictions by different models under the input- 1uput-96-predict-333 predictt-3t36 setting. The black lines stand for the ground truth and the orange lines stand for predicted values.
TimesNet
NHIT
N-BEATS
MM
M
M
M
M
MM
MM
200e-
20
40
10
30
50
DLinear
Autoformer
FEDformer
100.0
504.8
Mh
M
M
Figure 13: Visualization of M4 predictions by different models. The black lines stand for the ground truth and the orange lines stand for predicted values.
As shown Table II our proposed TimesNet achieves the best performance all five tasks. Among the top three models, TimesNet also achieves the greatest efficiency, Compared to MLP-based models, our proposed TimesNet shows a significant advantage in performance. And benefiting from the utilization of 2D kemnels and parameter- efficieent design, the parameter size is invariant when the input series changes. Compared to Transformer bassoormmee--eddddddde models, TimesNet with great efficiency in GPU memory, which is essential in long sequence modeling.
18

---

Published as a conference paper at ICLR 2023
Table 11: Model efficiency comparison and their rankings in five tasks. The efficiency measurements are recorded on the imputation task of ETTh1 dataset. The rankings are organized in the order of long- and short-term forecasting, imputation, classification and anomaly detection. indicates the out-of-memory situation. A smaller ranking means better performance.
Models
Parameter
GPU Memory
Running Time
Ranking
Series Length
(MB)
(MiB)
iter)
Five tasks
Avg Ranking
384
0.067
1245
0.024
TimesNet
768
0.067
1585
0,040
(1111,111)
1.0
(ours)
1536
0.067
2491
0.045
3072
0.067
2353
0.073
384
1.884
2321
0.046
Non-stationary
768
1,910
4927
0.118 /
2,,
3.4
Transformer
1536
1.961
3072
384
1.848
2101
0,070
768
1,848
3209
0.071
Autoformer
(7,4,3,5,3)
4.4
1536
1.848
5395
0.129
3072
10043
0,255
384
2,901
5977
0.807
768
2.901
7111
1.055
FEDformer
6, 9,
4.8
1536
2.901
9173
1,482
3072
2.901
384
0.163
1055
0.009
768
0.614
1077
0.013
LightTS
(6. 5, 4, 10, 4)
5.8
1536
2.403
1127
0.015
3072
9.534
1311
0.030
384
0.296
1057
0.006
768
1.181
1093
0,006
DLincar
6, 12,
6.4
1536
4.722
1159
0.007
3072
18.881
1433
0.026
384
1.123
1831
0,042
768
1.123
2565
0.047
ETSformer
65,9,9,6,5)
6.8
15366
1.123
4081
0.072
3072
1.123
7065
0.143
384
1,903
1577
0.044
768
1.903
2125
0.047
Informer
(10, 8,
7.6
1536
1.903
3153
0.088
3072
1.903
5194
0.165
384
1.157
1681
0.030
768
1.157
2301
0,046
Reformer
(11, 11, 11,11, II)
8.8
1536
1.157
5793
0.102
3072
1.157
384
1.308
2047
0.046
768
1.996
6077
0.119 /
Pyraformer
(9, 10. 12, 8, 6)
9.0
1536
3.372
3072
384
0.121
1135
0.010
768
0.220
1139
0.011
LSSL
(8, 12, 10, 7, 13)
10.0
1536
0.417
1147
0.013
3072
0.812
1197
0.032
384
0.372
1195
0.020
768
0.372
1333
0.020
TCN
(12, 7, 11, 11, 12)
10.6
1536
0.372
1533
0.025
3072
0.372
1983
0.061
384
0,268
1201
0.064
768
0.268
1323
0.122
LSTM
(13, 13, 13, 13, 10)
12.4
1536
0.268
1539
0.229
3072
0.268
2017
0.452
19

---

Published as a conference paper at ICLR 2023
H MODEL PERFORMANCE IN MIXED DATASET
To verify the model capacity in large-scale pre-training, we evaluate the model performance when it is trained from a mixed dataset. Concretely, we mixed the hourly-collected ETThI, ETTh2 and the 15-minute collected ETTml, ETTm2 as the mixed dataset. Note that this mixed dataset contains diverse temporal patterns and periodicities in different data instances, making the unified training challenging. From Table 12 we can find that TimesNet can handle this mixed dataset well and generally promote the model performance in four independent subsets.
Besides, we can also find that except TimesNet, for other baselines, the mixed training may decrease the model performance in some subsets, indicating that other baselines cannot handle the complex periodicities in the mixed dataset. These results also verify the potential of TimesNet in performing as the general- purpose backbone for large-scale pre-training in time series.
Table 12: Comparison between unified training and independent training for imputation task.
Datasets
ETTml
ETTm2
ETTh1
ETT12
Mask Ratio
12.5%
25%
37.5%
50%
12.5%
25%
37.5%
50%
12.5%
25%
37.5%
50%
12.5%
25%
37.5%
50%
Autoformes
Unified
MSE
0.034
0.048
0.078
0.027
0.034
0.066
0.086
0.114
0.133
0.055
0.065
MAE
0.122
0.146
0.163
0.185
0.091
0.102
0.109
0.117
0.174
0.200
0,229
0.247
0.135
0.147
0.157
0.171
MSE
0.034
0.046
0.067
0.026
0.035
0.074
0.137
0.060
Independent
MAF
0.124
0.144
0.161
0.174
0.092
0.101
0.108
0.119
0.182
0.203
0,222 0.248
0.138
0.149
0.163 0.173
FEDtormer
Unified
MSE
0.057
0.099
0.089
0.172
0.101
0.164
0.125
0.161
MAE
0.143
0.169
0.192
0.224
0,166
0,205
0.244
0,287
0.196
0,228
0,258
0,289
0.204
0.246
0.283 0.326
MSE
0.035
0.052
0.089
0.056
0.080
0.156
0.070
0.106
0.124
0.165
0.137
0.187
Independent
MAE
0.135
0.166
0.191
0.218
0.159
0.195
0.231
0.276
0.190
0.236
0,258
0.299
0.212
0.258
0.304
0.341
MSE
0.019
0.023
0.037
0.020
0.025
0.035
0.046
0.075
0.036
0.040
TimesNet
Unified
MAE
0.091
0.099
0.109
0.123
0.075
0.081
0.086
0.095
0.126
0.144
0.159
0.181
0.112
0.119
0.129
0.140
Independent
MSE
0.019
0.023
0.037
0.026
0.069
0.102
0.046
0.052
0.060
MAE
0.092
0.101
0,111
0.124
0,080
0.085
0.091
0.098
0.159
0.178
0.196
0.215
0.130
0.141
0.151
0.162
I FULL RESULTS
Due to the space limitation of the main text, we place the full results of all experiments in the
following: long-term forecasting in Table 13, short-term forecasting in Table 14 imputation in Table 16 classification in Table [7] and anomaly detection in T
20

---

Published as a conference paper at ICLR 2023
Table 13: Full results for the long-term forecasting task. We compare extensive competitive models under different prediction lengths. The input sequence length is set to 36 for the ILI dataset and 96 for the others. Aug is averaged from all four prediction lengths.
Models
TimesNet
ET
LighTTS"
DLincat"
PEDiomer
Stationary
Autolormer
Pynaiomme
Informer
LogTrams
Reformer
LSSL
LSTM
(Ours)
2722
2022
2022
2021
2021
2019
1997
Metric
MSE MAE MSE MAE MSE MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE MAE MSE MAE
MSE MAE
MSE
MAB
96
11.33
0.375 0 375 o 300 oello... 374 07..0.000 0.400l0.345
3.35
0.372
0,379
0.419/0.386 o 0.398 3.5 0:475
1.543
0.510
0.571 6.0000 0.546 a 0.528/0.450
0.477 0.664
192
0,374 0,387 0-408 0.410/0.400 0,407
0.3W1
389
0.426
0.441 0.459 0,444 0.553 0
0.557 0.537
0.795 0.66900877
o 0.700 0.592 0,469 0.592
0,481 1.113 0.776
336
0.410
0.411 0.135 0.128 0,138
0,435
0:413
0:445
0.459 0.495
0.464 (.621 0.537
0.751
0.655
1,212 0.871 1.124 0,832 0.898 0.72) (0.883 0.574 1,267 0.832
720
[0.478
06.40)
0.499 0.42
0,527
0.502
01.47
0:453
0.543
0,400
0.585
0.585
0.516 0:561
(0.00
0.724 1.166 0.823
1.153 0.820 1. 1.102 0.841
(06632 0.596
1.324
Avg
|0.400
0,406 0.429 0.425 0.435 0.437
0.437
0.402
0.448
0.452
0.481
0.456|0.588 0.517 (0.991
0.607/0.961 0.734/0.929 0.525 (0.999 0.67] (0.533 0.532)
1.14 0.782
96
10.187
0.267 0.189 0.280 O 0.209 0.30s|o.193
0.292/0.203
0.287 0.192 0.274 (.255
0.339
(0.135
0.507
0.365 0.453/0.768
0.642 J0.658 0.619/0.243
0.342|2.001 1.073
192
00.44
0.309 0 253 0.3 011 0.382
0,362
0.
328
0,250
0.330
0.281 0.340
0.3400..300
0,730
0.673
0.563.0.999 0.757 1 1,078 0.827 0.82770.592 0, 2.344 1.112
336
0.322
0.351 0.314 o 3.3 0.442 0,466
0:427 0.325 0.366 0.334
0.361
(.339 0.372 1.201
0.845 0.845
1.363
0.887 1.334
0.872
| 1,549 0.972 (1,932 0.724
0.97220.332
0.724 2,568 1,238
0.408
0.403
0:111 0:112 0.675 0.587
0.55:
0.522
0:421 0.115 0.417 0.413
0.433
0.433
0.132
$.625
3.451
3.379
1.336
1.336
2.018 1.328 2.63822633333
1.242 1.372 0.879 0.879 2.720 1.287
Avg
[0.291 0.333
299
0 342 0.409 0.436|0.350
0:401 0.
o.3 0.349 0.36 0.347
0.371
1.495
0 01.56/1 1,410 0.810
10000011.5555 1.535 o 0.900) |147 0.915/0.7355
0.598/ 1.177
96
0.384
0.102
0.191
0.479
0.424
0.432
0.400 0.376
0.119
0.491
[0.199
0.155
(0661
0.612
0.713/0.878
0.740
0.728/0.548
0.526|
Losd 0,773
192
0,436
0.429 0.538
0.504 0.475 0.462
0.437
0.432 0.420 0.448 0.534
0.504
0.500
0.482
0,790
0.681 1.008 L.792 0.824 0.923 0.766 0.526
1,217 0.832
336
004491
0,469 0.574
0.521 0,518 0.485 0.481
0.459 0.459 o 465
0.588
0.535
0:496
0,891
0.738
0.738 1.107 0.809
1.238 0.933 1 0.007 0.835 1.29%
0.942
1,259 0.841
0.521 0.500 0.500 0.535 0.533 0.519
0.516 0.506
0.507
0.643 0.616
0.51:
0.512
0.782
0.865
1.135 0.852
| 0.889
0.659
1.271 0.838
| Avg
[0.458 01.450 11445000.54220311000000 0.479
0:452 0.440 0.400 0.537/
0:487
(0.277
0.705)
0.795]
0.837
0.29 0.505 (.666]]
1.18 0.871
96
10,3:0
0.333
0,387 0,358
0.358
0.3970..476
0.476
0.45s
0,3466
0,388
1.665
0.597
2.5
1.197
1.317
1.616
1.036] 2.522 1,278
192
0.402 0.414 0.130 0.439
0.520
0.50s
0.477
0.476
0.429
0.439
0.512
0.493
0.452
0.788
5.682
1.931
4.315
1.635
11.12
2.979
2,083
1.197 3,312 1.384
1.197
0.452
0.452 0.485 0.626 0.559
0.541
0.496
0.487 0.552 0.551 0.482
0.486
0.247
4.721
1.835
L.124
1604
9.323
2:769
2,970
1.49
3.439
3.291 1.388
00.42
0.468
0.500 0.197 0.863 0.672
0.657
0.463
0.474
0.562 0.560 0.511
0.963
0.783
1.625
1.540
1.697
1.363 3.257 1.357
Avg
(0.414
0.439
0,452 0.452 0.543
0:515
0.437 0.44% 0.5.0005600000 0.516//.40 0.459
(0836
0.836.0.7054.4411
1.720
1.494
1.250/1 2.095 1.352
Electricity
96
10.168
0.272
0.187
0.304/0.207
0.507/o.197
0.282/0.
1.19 0.308 0.109
0.100...255....... 0.275/..201
0.317 0.336
0.449
0.368
0.357
0.332
0.402
0.39210.375 0.437
192
0.181
0,289 0.199
0,315 0.213
0.316
0.285
0.201 0,315 0.182
0.222
0.330
0.330
0.378
0.249
0.386
0.366
0.266
0.368
0.348
0.433
0.390 0.173
0.198
0,300
0.212
0.329 0.230
0.333 0.209
0.301
0.214
0.229
0.200
0,304
0.231
0.376
0.443
0.390
0.380
0.433
0.311 0.403 0.473
720
0.220
0.320
0:233
0:345
0.265 0.3600 0.333
0.246
0.355
0.222
0.321
0.254
0.361
0.376
0.445
0.377 0.439
0.376
Avg
[0.192 0.296 0.208
0.323 0.22 0.329 0.300 0.214
0.227
0.192 0.296
0.296...22
0.227
0.338
(.379
0.311
0.397
0 0.000 0.428
96
10,593
0.321 0.007
0.392
0.615
0,391
00.60
0.396
0.587
0.300
0.260
0.612
0.33)
0.388
(0.867
0.469
0.391 |0.68
0.384
3323
0.423
w.466.8...3...... 0.436|0.843 0.453
Traffic
192
16.617
0.336 0.621
0.309 0.001 0.382
0.598 0.370 0.373 0.613
0.616
0.382
0.467
0.4677.666
0.69
0.379
0.679
0.685
0.390
0.420
0.481 00:47 0.453
0.629
0.336 0.622 0.366 0.613 0.386
0.605
0.373 0.621
0,283
0.622
0.767
0.734
0.742
0.420
0.476 0.455
720
0.640
0.350
0.632 0.39
0.688
0.407
0.645
0.394
0.626
0.382
0.653
0,355
0.660
0.408
0.896
0.473
0.472
0.296 0.423
0.489
1.500 0.805
Ava
[0.620 0.336 0.621 0.396
0.392
0.383 0.610
0.376
0.3400.622
0.379
0.469/0.764
0.416/0.705
0.395
0.422|0.832
0.471 1.011 0.541
96
lo.172
0.220/0.197 0.197 0.281
0.220
0.182
0.242]0.196
0.255
0.217
0.29
0.223
0.266
0.336
(0.622
0.556l0.300
0.38:|0.458
0.490l0.699
0.996
0.252|0.369 0.406
Weather
192
0.219
0.261 0.233
0:312 222
0.287
0.237
0.296
0.276
0.336
0.245
0,285
0.307
0.362
0.739
0.598
0.658
0.589 0.2
0.638
00.288
0.313 0.438
336
00.800 0.306 0.2 0.353 0.282 0.334
0.285
0.335
03939
3.32
0.338
0.395
1.004
0.753
0.578
0.523
n.652 0.399 0.596
0.355
720
0.365
0.359 0.352
0.288
9,352
0.386
0.345
0.381
0.403
0.428
0.414
0.419
0.428
1.420 0.934
1,059
0.741
0.869
0,675 1.130 0.792
0.415 0.520
Avg
[0.259 0.287 0.271 0.334 0.266
0.265
0.317 0.309 0.360
0,314
0.382 0.338
(0.91
0.717
0.546
0.602
(0.803
0.656/0.271
0.33+|00.44
10.107
0.234
0.085
0.204
116
0.262
1.08
0.218
D. 148
0.278
0.111
0.237/0.197
11197
0.123
1.748
1.105|0.84
0.752/0.968
0,812
01820
0.474) 1.43 1.049
Exchunge
192
0.226
0.344 0.182
0,303
0.215
0.359
0.176
0.315
0.271
0.380
0.219
0.335
0.300
0.369
1.151
1.204
0.895
1.040 0.851
1.188 0.776 0.698
1.946 1.179
336
00.667
0.448 0.348
0.428
0.377
0,466
0.31.
0.427
0.460
0.500
0.421
0.509
0.524
1.943
1.172
1.172
1.672
1.036
1.659
1.081
1.357
0:976
1.029
0.797
0.797
2.186 1.231
720
0.994
0.746
1,0255000
0.774
03831
0.830
0.695
1.195
0.841
1.002
0.760
1.447
0:941
p.085
1,2066
1,310
1994 1.127
1.016
2,283
1,232 2,99 1.437
Avz
[0.416
0.427 0,385
0.447/0.354
0.511
0.500/0.61
0,454
0.539
0.530
1.913
1.150|1.550
0.998/1
1.402
0.968
0.932
1.121
0.798/2.105 1.221
2.317
0.934
0.000'8
3.13
2,144
2.398
1.040)
1.228
1.360
2.294
(1945)
55.48
1.287
17.994
2.012/5.764
1.677)4.480
1.444
4.00
1.383/4.581
1.425/5.914 1.734
0.920
1.007/6
1.902
2.646
1.088)
2.679
0,848
7.551
2.031
4.755
1.467 4.799
1.467
4.783
4,444
1,41666631
1.845
2,238
0.940
2.359
0.9/2
7.299
1.982
2.614
1.086
2.622
1.078
2,010
2.669
1.085
7,662
2.0574.763
4.469
1.468
4,832
4.459
1.4366.736 1.857
60
2.027
0.928
2.487
1.016
7.283
1.985
2.804
1.146)
2.857
1.157
2.17%
0.963
2.770
1.125
7,931
2.100
1, 1.268 1.560
882
1.48344651
6.474 1,879
Avg
13139
0.931
2.497 1.004 7.382 2.616
2.616 1.090 2.847 1.144
2.077 0.914
1.161
|1.355 2.050/5.137 1.544/4839 1.485 (4.224 1.45|44508 1440|66.388 1.829
Count
means that there are some mismatches between our ouputt setting and their papers. We adopt their official codes and only change the length of input and output sequences for a fair comparison.
21

---

Published as a conference paper at ICLR 2023
Table 14: Full results for the short-term forecasting task in the M4 dataset. in the Transformers indicates the name of sformer. Stationary means the Non-stationary Transformer.
Models
TimesNet N-HITS DEATS
ETS.
LightTS
DLinear
FED,
Stationay
Auto.
Pyra.
In.
LogTrans
Rc.
LSTM
TCN
LSSL
(Ours)
2022
2019
2022
2022
2023
2022]
2022n
2021
2021a
2021
2019
2020
1997
2019
2022
Yearly
SMAPE
13.387
13:418
13.436
18.009
14.247
16.965
13.728
13.717
13.974
15.530
14.727
17.107
16.169
176.040
14,920
61.675
MASE
2.996
3.045
3.043
4.487
3.109
4.283
3.048
3.078
3.134
3.711
3.418
4.177
3.800
31.033
3.364
19.953
OWA
0.786
0.793
0.794
1.115
0.827
1.058
0.803
0.807
0,822
0.942
0.881
1.049
0.973
9.290
0,880
4.397
Monthly Quarterly
SMAPE
10.100
10.202
10.124
13.376
11.364
12.145
10.792
10.958
11.338
15.449
11.360
13:207
13.313
172
11.122
65.999
MASE
1.182
1.194
1.169
1,906
1.328
1,520
1,283
1.325
1.365
2,350
1.401
1.827
1.775
19.753
1.360
17.662
OWA
0.890
0.899
0.886
1,302
1.000
1.106
0.958
0.981
1.012
1,558
1.027
1,266
1.252
15.049
1.001
9.436
SMAPE
12.670
12.791
12.677
14.588
14.014
13.514
14,260
13.917
13.958
17.642
14.062
16.149
20.128
143.237
15.626
64.664
MASE
0.933
0.969
0.937
1,368
1.053
1.037
1.102
1.097
1.103
1.913
1.141
1.660
2.614
16.551
1.274
16.245
OWA
0.878
0.899
0.880
1.149
0
0.956
1.012
0.998
1.002
1,511
1.024
1.340
1.927
12.747
1.141
9.879
Others
SMAPE
4.891
5.061
4.925
7.267
1.5.880
6.709
4.954
6.302
5.485
24.786
24.460
23.236
32.491
186.282
7.186
121.844
MASE
3.302
3,216
3.391
5,240
11,434
4.953
3.264
4,064
3.865
18.581
20,960
16.288
33.355
119.294
4.677
91.650
OWA
1.035
1.040
1.053
1.591
3.474
1.036
1.304
1.187
5.538
5,879
5,013
8.679
38.411 1.494
27.273
Weighted Average
SMAPF
118229
11.927
11.851
14:718
13.525
13.639
12.840
12,780
12,909
16.987
14.086
16.018
18. 18,200
160.031
13.961
67.156
MASE
1.613
1.599
2,408
2.111
2.095
1.701
1.756
1.771
3.265
2.718
3.010
4.223
25. 78,788
1.945
21.208
OWA
0.851
0.861
0.855
1.172
1.051
1.051
0.918
0.930
0.939
1.480
1,230 1.378
1.775 12.642 1.023 8.021
adopts a special ensemble method to promote the performance. For fair comparisons, we remove the ensemble and only compare the pure forecasting models.
The original paper N-BEATS (Oreshkin al.
2019)
Table 15: Full results for the anomaly detection task. The P. R and Fl represent the precision, recall and Fl-score (%) respectively. Fi-score is the harmonic mean of precision and recall. A higher value of P, R and Fl indicates better performance.
Datasets
SMD
MSL
SMAP
SWaT
PSM
Avg Fl
Metrics
R
F1
R
P
F1
R
F1
R
(%)
LSTM
1997
78.52
65.47
71.41
78.04
86.22
81.93
91.06
57.49
70.48
78.06
91.72
84.34
69.24
99.53
81.67
77.97
Transformer
2017
83.58
76.
79.56
71.57
87.37
78.68
89.37
57.12
69.70
68.84 96.53
80.37
62.75
96.56
76.07
76.88
LogTrame
2019
83.46
70.13 76.21
73.05
87.37
79.57
89
57.59
69.97
68.67
97.32 80.52
63.06
98.00
76.74
76.60
TCN
2019
84.06
79.07 81.49
75.11
82.44
78.60
86.90
59.23 70.45
76.59
95.71 85.09
54.59
99.77
70.57
77.24
Reformer
2020
82,58
69.24 75.32
85.51
83.31 84.40
90.91
57.44 70.40
72.50 96.53 82.80
59.93
95.38
73.61
77.31
Informer
2021
86.60
77.23 81.65
81.77
86.48
84.06
90.11
57.13 69.92
70.29 96.75 81.43
64.27
96.33 77.10
78.83
Anomaly"
88,91
82.23 85.49
79.61
87.37
91.85
58.11 71.18
72.51 97.32 83.
68.35
94.72 79.40
80.50
Pyraformer
2021a
85.61
80.61 83.04
83.81
85.93
84.86
92.54
57.71 71.09
87.92 96.00 91.78 71.67
96.02 82.08
82.57
Autoformer
2021
88.06
82.35 85.11
77.27
80.92 79.05
90.40
58.62 71.12
89.85 95.81 92.74
99.08
88.15 93.29
84.26
LSSL
2022
78.51
65.32 71.31
77.55
88.18
82.53
89.43
53.43 66.90
79.05 93.72 85.76
66.02
92.93 77.20
76.74
Stationary
2022a
88.33
81.21 84.62
68.55
89.14 77.50
89.37
59.02 71.09
68.03 96.75 79.88
97.82
96.76 97.29
82.08
DLinear
2023
83.62
71.52 70
84.34
85.42 84.88
92.32
55.41 69.26
80.91
95.30 87.52
98.28
89.26 93.55
82.46
ETSformer
2022
87
79.23 83.13
83.13
84.93 85.03
92.25
55.75 69.50
90.02
80.36 84.91
99.31
85.28 91.76
82.87
LighTTS
2022
87.10
78.42 82.53
82.40
75.78 78.95
92.58
55.27 69.21
91.
91.98
94,72 93.33
98.37
95.97 97.15
84,23
FEDformer
2022
87.95
82.39 85.08
77.14
80.07 78.57
90.47
58.10 70.76
90.17
96.42 93. 19
97.31 97.16 97.23
84.97
TimesNet
(Inception)
87.76
82.63 85.
82.97
85.42 84.18 91.50 57.80 70.85
88.31
96.24 92. 10 98.22
92.21 95.21
85.49
TimesNet
(ResNeXt)
88.66
83.14 85.81
83,92 86.42 85.15 92.52 58.29 71.52
86.76 97.32 91.74 98.19 96.76 97.47 86.34
The original paper of Anomaly Transformer (Xu al. 202T adopts the temporal association and recon- struction error as a joint anomaly criterion. For fair comparisons, we only use reconstruction error here.
22

---

Published as a conference paper at ICLR 2023
Table 16: Full results for the imputation task. We randomly mask 12.5%, 25% 37.5% and 50% time points to compare the model performance under different missing degrees. in the Transformers indicates the name of =former.
Models
TimesNet
LighTS
DLincar"
Strionary
Pyra
LogTrams
LSTM
TCN
LSSL 2023
(Ours)
2022
2043
209
2021
2741
2019
20.00
1997]
2019
Mask Ratio MSE MAG
MAT
MSE MAG
MSE
MAT
MSE
MSE MAE MAF
MAE
MAE
MAE
12.5%
0.019
0.190009200000
0.139 0,073
0.150 0,058 162 0.035 0.135
1026
0.007
0.034
0.400.000 0.543 0.047 0.04 0.141 0012 0.120
0.5300..500
10101 0.231
25%
0.023 0.101
0..9
0.229 0.205 0.080 O. 195 0.052
1666
0.119
0.046
0.144
0.553 pits o. 0800 o 0.44
0.146 14661.00 1,02 0.807
0.807 0.518 0,500 032
37.5%
0.0290.111
0.133
0.271 0.113 0.231 0.103 0.209
0.069
0.191
0.131
0.057
0.166
0,237
0.58] 0.079 0.200 o 0.066 o. 0.182
0.796 0, 0.216
50%
0.1.6...144..886
0.036
0.323 0.134 0,255 0.152 0.245 0.099 0.218
0. 4.145
0.067
0.174 0.710 0.505 (.099 0.215 o. 0.072 0.200
0. 0.763 0,1.199994446.00
Arg
0.027 0.100.....000000 000.0000000000000 0.25300..00000 /04 0.218 0.093 0. 0.006.00.....0 o. 0.17
0.126
0.051 0.
0.717 0.077 0.185 150.0.0.05....005555....000.00
0.166
0.969
ETTu2
12.5%
0.018 000000000000000000000000000 0.239....00444
0.000.0000...00
0.127 0.062 O. 166 0.056. 02
0.0e
0.033
00.033..0000
0.000/0.304
1.013 000.00.8000 0.805 0.440000000.440010000000
25%
0.00000000...00
0,044 0.143 0.085 0. 0.080 0.19
1124
0.0
0.006
0.101 0.100:444 0,431
0.260
1,019
0.814 2.336604422000 0.
37.5%
0.09)
(32 0.051 0.051 0.159 0. 0.6 0.
0.027
0 103
0.000
0.106
0.521
0.293
0.175
0,917
50%
0.026
0.0%
0.421 0,059 0.174 0.247 0.16
00000
0.5
0.119
0.560
0.329
01.4000.83355 0.2.6
Asg
0.022 0.08%/ 208 0. 00027
05151 0.096 208 0.215
0
0.500..
0000000000000000
12.5%
0.057
0.159
0.263
0.240......4444
0,345 0.151 0.267 0.100.
0.165
0.144
0.13clos
0.857
0.
0.234
0.229
0.194]
0.896
0,554
25%
0.069
0.16
0.304
0.200
0.364
0.180
292
0000 189
0.050
0.2000000000
0,82
0.325
0.227
1,202 0.883
0.8%3 0.610 0.6.100.6666000.000 0.577
37.5%
0.084
0.166
0.347
0.20 0. 0.215
0:21
0.109
0.675
0.230.000.000
0.261
1.200
0.967
50%
0.102
0.400 0.440
0.25
0.2150.325022000
04600
0.00
0.540
Avg
0.078 0.880.1870300 022.8870222222 0.329
0.201 0.306
0.306 011.0..7.0000.20000000000
0.303
0.214
0.682
0.245
0.873
0.571 0.424
Eh
12.5%
0.040
0.100..1300.18877
0.219/0.101
0.231
0.100
0.110
0.754
0.43) 0.133
0.289/2.000
1.000
0.120004100..0000
0.555
25%
0.046 0.14
0.279
0.190
0.246
0.127
0.147
0,000
0.149
1.077
0.322 0...2220.444.111 011.4444400010100111100
0,531
2.00 1.105
0.4199.49000.40
37.5%
0.0520.111
0,465
0.060
0.165
0.800
0.370
2.033
1.111
50%
0.060
0.00 0.572
0.268
0.332.03111
0.153
1,193
0.838
1.119
0ww.::6660..29....o 0.529
Avg
0.49
0.055
156/
0.792 w.3.07.....475.....
1. 1.114 40000
Electricity
12.5%
0.085
0.202
0.331
0.092
0:210
0.366 0.000021....002277.00...
25%
0.089
0.333
0.096
0.220
0.244
0.219
0.31
37.5%
0.094
0.21
0,3419
0.273
0,1
0.276
0.1
0.266
0
0:220
0.104
0.229
0.381
0.222
0.328
0.35
0.315
0.364 0,5877 0.23
50%
0.100
0.357
0.1 005 0...
0.113
0.2300..29
0.383
p.228
0.331
w11...00000....000000000.0000... 0.56 0...
0,597
Avg
0.092 0...2.10000.340333.....1
131 0.262
0.1
130.0.25
0.225|0.297
0.367
0.328
1750.303
0.333 027.).7..030 0.366jo..00.22...
Weather
12.5%
0.025
0.007 0.141
0.084
0.041
0.107
0.05
10.066
0.017
14
0.22000.033
0.093.00377
0.07200.011
0.07e
0.379 0,267.0.000066
25%
0.029
0.052 0.065
0.052
0.111
0,048
0.064
01163
0.03000000
0.05-0.1447
0.222 0.042
0,100 0.024
0.082
0.40% 0.293 0.042....22..00
37.5%
0.131
0.057 0.081
0.087
0.158 0.121 0.057 0.1 1. 0.23
o
0.032
0.0000.156
0.000
0.240 0.111 110099
0.070
0.091
0,406
0.463
0.
0.291 0.047
0.
50%
0.134
0.02 0.207 0.065 0.153 0.066 0.1 p...11.........211l..1
0.06s
0.037
(.007
164 0. 0.249
0.
0.04
0.08 0.099
o.ow
0.443 195 0.330 128
Asg
0.030 0.054 0.171 0.171 0.117 0.052 0.1.522.11110000.. 119.10000009990000 0.205 0.009
0.039/..031
1..07/1..122 0.233
1.:045
0.104 00039
0.038
0.087/0.36
n.434/0.113 0.291 0.045 1018
1" Count
0
0
Table 17: Full results for the classification task. *. in the Transformers indicates the name of #former. We report the classification accuracy (%) as the result. The standard deviation within 0.1%.
Classical methods
RNN
TCN
Transformers
MLP
Datasets Models
ITW OCI Rusket LSTM
LSTNS
LSS1.
TON
Trans.
Re.
In.
Pyra.
Auto, Station. FED.
ETS. Flow. DJ DlLinearLLghhTTS Sim.a...iggTS...Ti...i
1994
2016
20220
1997)
2018
20197
2017
2030
2021
2021a
[021]
2022a
2022
2022
20022
21223
2023
(Ours)
BhunoCConcentraiio
32.3
43.7
45.2
32.3
39.9
31.1
28.9
32.7
31.9
31.6
30.8
31.6
32.7
31.2
28.1
33.8
32.6
29.7
35.7
FiceDetection
52.9
63.3
64.7
57.7
65.7
66.7
52.8
67.3
68.6
67.0
65.7
68.4
68.0
66.0
66.3
67.6
68.0
67.5
68.6
Handwriting
28.6
15.8
58.8
15.2
25.8
24.6
53.3
32.0
27.4
32.8
29.4
36.7
31.6
28.0
32,5
33.8
27.0
26.1
32.1
Hearteett
71.7
73.2
75.6
72.
77.1
72.7
75.6
76.1
77.1
80.5
75.6
74.6
73.7
73.7
71.2
77.6
75.1
75.1
78.0
JapaneseVowels
94.9
86.5
96.2
79.7
98.1
98.4
98.9
97.8
99.2
98.4
98.9
96.2
96.2
98.4
PEMS-SF
71.1
98.3
75.1
39.9
86.7
86.1
68.8
82.1
82.7
81.5
83.2
82.7
87.3
80.9
86.0
83.8
75.1
88.4
89.6
SeltRegalzionSCPi
77.7
84.6
90.8
68.9
84.0
90.8
84.6
92.2
90.4
90.1
88.1
84.0
89.4
88.7
89.6
92.5
87.3
89.8
91.8
SellRegassioSCC22
53.9
48.9
53.3
46.6
52.8
52.2
55.6
53.9
56.7
53.3
53.3
50.6
57.2
54.4
55.0
56,1
50.5
51.1
57.2
DioionAraabidiiiia
96.3
69.6
71.2
31.9
100.0
95.6
97.0
100.0
100.0
100.0
100.0
98.8
81.4
100.0
99.0
UWwseGestureLibbrry
90.3
75.9
94.4
41.2
87.8
85.9
88.4
85.6
85.6
85.6
83.4
85.9
87.5
85.3
85.0
86.6
82.1
80.3
85.3
Average Accuracy|
67.0
66.0
72.5
48.6
71.8
70.9
70.3
71.9
71.5
72.1
70.8
71.1
72.7
70.7
71.0
73.0
67.5
70.4
73.6
23


Reviews:
[{"review_id": "ZeRqG4OEr_Y", "reviewer": "Reviewer_Re8p", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on modeling the temporal variations in the time series data. Specifically, based on the observation that there are often multiple periodicities in time series data, the paper proposes to divide a single multi-dimensional time series into several sets of smaller pieces of the time series, each with a different periodicity. Then, the inner-period variance and intra-period variance could be modeled together with a CNN backbone, like inception as used in this paper. Empirical evaluations are conducted with multiple datasets on forecasting, imputation, classification, and anomaly detection tasks.", "summary_of_the_review": "Overall, I think this paper proposes a reasonable idea to tackle the long-term dependency issue in time series data modeling tasks and the solution is also sound. The empirical evaluations also show impressive results.", "strength_and_weaknesses": "Strengths:\n\n- The idea of tackling the long-term dependency by utilizing the multiple periodicities in time series is reasonable and the use of FFT to extract the period from the frequency domain also makes sense.\n- The empirical evaluation is comprehensive and promising results are shown.\n\nWeaknesses:\n\n- The periodicity in time series is an important assumption in this paper. Some time series data are not strictly periodic, but instead, they are composed of several repeating segments. In such cases, it is not clear if the proposed method still works.\n- It is not clear if the proposed model can be applied to a dataset containing multiple datasets but with different period lengths for each data sample. For example, the time series data for patients in hospitals.\n\n\nA minor detail: In the line before Eq. (1), \"Fast Fourier Transformer (FFT)\" should be \"Fast Fourier Transform (FFT)\".", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "R-nhOtJ99z", "reviewer": "Reviewer_v92e", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors argue that 1D time series data are composed of multiple intraperiod-variations and interperiod-variations. Since there is a limit to capturing these variations as 1D time series representation, the authors transform the 1D time series into a set of 2D tensors based on multiple periods so that the analysis can be performed in a 2D space. The proposed method can adaptively discover multi-periodicity and extract complex temporal changes from 2D tensors. Experimental results show state-of-the-art performance in five mainstream time series analysis tasks, including short-term prediction, long-term prediction, imputation, classification, and anomaly detection.", "summary_of_the_review": "The proposed method provides a new approach for 1D time series data by transform the 1D time series into a set of 2D tensors. It enables to use 2D kernels for analysis of both intraperiod- and interperiod-variations simultaneously. The inception block can be easily replaced by other existing models so that its flexibility is also high.", "strength_and_weaknesses": "Strength:\n1. The proposed method solves the bottleneck of representation capability in 1D space and enables simultaneous representation of intra-period and inter-period variation in 2D space. While 1D time series could only represent variations between temporally adjacent time points, the proposed method offers the possibility of representing variations of the same phase in different periods. As the result, the proposed method allows the 2D kernel to simultaneously handle changes in adjacent time and changes between adjacent periods.\n\n2. Good design for flexibility. The inception block can be replaced by any model of the type that uses a 2D kernel. In fact, the authors investigated its performance by replacing the inception block by other models such as ResNet and ResNeXt, etc. \n\n3. The proposed method provides SOTA performance in five main tasks in time-series data analysis. While the models of previous studies differ in their strengths in each task, the proposed method almost consistently performs the best.\n\n\nConcerns:\n1. What does the dimension “C” of time series data correspond to in this study?\n\n2. According to the appendix, the top-k was experimentally decided to be three (k=3). As far as I see the Fig.8, the performance was not so changed in the range between 1 and five. How carefully should the parameter k be decided? If a larger k is selected, is there any possibility to increase/decrease the performance?\n\n3. Small concern. I found a paper, which has already submitted in Arxiv. The contents of both papers are almost same except for author area: one is anonymized, the other shows the author list. I hope this doesn't violate the double blind rule.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "BW9z4ZuDLNd", "reviewer": "Reviewer_gBbZ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThis paper focuses on temporal variation modeling and proposes a representation method intended for incorporating multiple intraperiod- and interperiod-variations.\nFor each selected frequency, the proposed method generates a two-dimensional representation of the original time series and applies a convolutional network to the representation.\nThe proposed method is evaluated on several time series tasks including long/short-term forecasting, imputation, classification, and anomaly detection.  ", "summary_of_the_review": "\nThis paper focuses on temporal variation modeling and proposes a representation method intended for incorporating multiple intraperiod- and interperiod-variations. The proposed method is based on some existing ideas. It is evaluated on several time series tasks including long/short-term forecasting, imputation, classification, and anomaly detection, and shows competitive performance. The main concern is about the novelty and soundness of the proposed method, and probably a more appropriate positioning of the work. ", "strength_and_weaknesses": "Strength:\n\nThe proposed method takes into account the multiple frequencies that could exist in multi-variate time series. \nThe experimental evaluation is extensive on several different time series tasks and shows competitive performance. \n\nWeakness:\n\nIncorporating different component information from time series into neural network-based models is not a new idea, for instance, one of the very recent works is [1]. Meanwhile, the following issues would affect the assessment of the paper. \n\n(a) Some math notation would be misleading. e.g., $X_{1D}$ is two-dimensional to represent multivariate time series. \n\n(b) Figure 2 is confusing. According to Eq. (1), the frequency analysis is applied to the multivariate time series, while Figure 2 gives the impression that it is to the univariate time series. \n\nBased on Eq. (5), the Inception is applied to a matrix of dimension $ (f_i * p_i) * C$, while the illustration of 2D kernel seems to show that one dimension of the input to the Inception CNN is the number of frequencies. \n\n(c) Given the frequencies selected based on the magnitude, the efficacy of adaptive aggregation Eq. (6) is unclear. It would be good to have some ablation study on this. \n\n(d) During the training, the frequency analysis is applied to individual data instances or the batch of instances? Would the different granularities of this operation affect the performance?\n\n(e) The truncate operation in Eq.(5) looks a bit unconvincing, given that it could blindly get rid of useful information depending on the dimension size relative to the dimension T. \n\n(f)  In Table 11, the result of the baseline DLinear seems to be worse than that in the original paper [2]. It would be good to elaborate on it.\n\n[1] Woo, Gerald, et al. \"ETSformer: Exponential Smoothing Transformers for Time-series Forecasting.\" arXiv preprint arXiv:2202.01381 (2022).\n\n[2] Zeng, Ailing, et al. \"Are Transformers Effective for Time Series Forecasting?.\" arXiv preprint arXiv:2205.13504 (2022).\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster