Title: FITS: Modeling Time Series with 10k Parameters
Venue: NeurIPS 2023
Accepted: no
Keywords: Time series analysis

Time series forecasting

Complex-valued neural network
Abstract: In this paper, we introduce FITS, a lightweight yet powerful model for time series analysis. Unlike existing models that directly process raw time-domain data, FITS operates on the principle that time series can be manipulated through interpolation in the complex frequency domain. By discarding high-frequency components with negligible impact on time series data, FITS achieves performance comparable to state-of-the-art models for time series forecasting and anomaly detection tasks, while having a remarkably compact size of only approximately $10k$ parameters. Such a lightweight model can be easily trained and deployed in edge devices, creating opportunities for various applications.
The anonymous code repo is available in: \url{https://anonymous.4open.science/r/FITS}

1

Abstract

In this paper, we introduce FITS, a lightweight yet powerful model for time series analysis. Unlike existing models that directly process raw time-domain data, FITS operates on the principle that time series can be manipulated through interpolation in the complex frequency domain. By discarding high-frequency components with negligible impact on time series data, FITS achieves performance comparable to state-of-the-art models for time series forecasting and anomaly detection tasks, while having a remarkably compact size of only approximately 10 k parameters. Such a lightweight model can be easily trained and deployed in edge devices, creating opportunities for various applications. The anonymous code repo is available in: https://anonymous.4open.science/r/FITS

1 Introduction 11

Time series analysis plays a crucial role in numerous domains, including finance, energy, weather 12 forecasting, and signal processing, where understanding and predicting temporal patterns are essential. 13 Existing time series analysis methods primarily focus on extracting features in the time domain (Zhou 14 et al., 2021; Liu et al., 2022; Zeng et al., 2022; Nie et al., 2023; Zhang et al., 2022). However, due to 15 the inherent complexity and dynamic nature of time series data, the information contained in the time 16 domain tends to be sparse and dispersed. Consequently, researchers design intricate methodologies 17 and complex models to capture and exploit this information, often relying on approaches such as 18 transformer architectures (Zhou et al., 2021; Wu et al., 2021; Zhou et al., 2022a). However, these 19 sophisticated techniques often lead to the proliferation of large-scale and computationally demanding 20 models, posing challenges in terms of efficiency and scalability. 21

Conversely, the frequency domain representation of time series data offers a more concise and 22 compact representation of its underlying information. Recognizing this potential, previous studies 23

have explored the utilization of frequency domain information in time series analysis. For instance, 24

FEDformer (Zhou et al., 2022a) incorporates spectral information as a supplementary feature, en25

hancing the modeling capabilities of transformer-based time series models. Another approach, 26

FNet (Lee-Thorp et al., 2022), leverages frequency domain multiplication to replace convolution 27

operations, thereby reducing computational overhead. Moreover, LTSF-Linear (Zeng et al., 2022) 28

has demonstrated that highly accurate predictions can be achieved by solely learning the dominant 29

periodicity. Similarly, methods like TimesNet (Wu et al., 2023) segment the time series based on 30

frequencies with high amplitude and employ CNNs for multi-periodicity feature extraction. 31

However, existing methodologies often overlook the fundamental nature of the frequency domain 32

representation, which utilizes complex numbers to express both amplitude and phase information. 33

Motivated by the fact that longer time series segments provide a higher-resolution frequency rep34

resentation, we propose FITS ( F requency I nterpolation T ime S eries Analysis Baseline). The core 35

component of FITS is a complex-valued linear layer that can explicitly learn amplitude scaling and 36

phase shift to perform interpolation in the complex frequency domain. Although FITS conducts 37

interpolation in the frequency domain, it remains an end-to-end time domain model incorporating 38

the rFFT (Brigham & Morrow, 1967). Specifically, we project the input segment to the complex 39 frequency domain for frequency interpolation using rFFT. We then project the interpolated frequency 40 representation back to the time domain as a longer segment for supervision. This end-to-end design 41 enables FITS to adapt to various downstream tasks with commonly-used time domain supervision, 42 such as forecasting and reconstruction. 43

44

45

46

47

48

49

50

Additionally, FITS incorporates a low-pass filter to obtain a compact representation with minimal information loss, resulting in small model volume and minimal computational overhead while maintaining state-of-the-art (SOTA) performance. Notably, under most settings, FITS achieves SOTA performance with under 10k parameters , which is 50 times smaller than the lightweight temporal linear model DLinear (Zeng et al., 2022) and approximately 10,000 times smaller than other mainstream models. The low memory and computation overhead make FITS suitable for deploying or even training on edge devices for forecasting or anomaly detection.

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

To summarize, our contributions are twofold:

We introduce FITS, a lightweight model containing merely 5k ∼ 10k parameters for time series analysis. Despite its compact size which is several orders of magnitude smaller than mainstream models, FITS delivers exceptional performance in various tasks, including long-term forecasting and anomaly detection, achieving state-of-the-art performance in several datasets.

FITS employs the complex-valued neural network for time series analysis, which provides a novel perspective that simultaneously captures amplitude and phase information, leading to more comprehensive and efficient modeling of time series data.

2 Related Work and Motivation

2.1 Frequency-aware Time Series Analysis Models

Recent advancements in time series analysis have witnessed the utilization of frequency domain information to capture and interpret underlying patterns. FNet (Lee-Thorp et al., 2022) leverages a pure attention-based architecture to efficiently capture temporal dependencies and patterns solely in the frequency domain, eliminating the need for convolutional or recurrent layers. On the other hand, FEDFormer (Zhou et al., 2022a) and FiLM (Zhou et al., 2022b) incorporate frequency information as supplementary features to enhance the model's capability in capturing long-term periodic patterns and speed up computation.

The other line of work aims to capture the periodicity inherent in the data. For instance, DLinear (Zeng et al., 2022) adopts a single linear layer to extract the dominant periodicity from the temporal domain and surpasses a range of deep feature extraction-based methods. More recently, TimesNet (Wu et al., 2023) achieves state-of-the-art results by identifying several dominant frequencies instead of relying on a single dominant periodicity. Specifically, they use the Fast Fourier Transform (FFT) to find the frequencies with the largest energy and reshape the original 1D time series into 2D images according to their periods.

However, these approaches still rely on feature engineering to identify the dominant period set. Selecting this set based on energy may only consider the dominant period and its harmonics, limiting the information captured. Moreover, these methodologies are still considered inefficient and prone to overfitting.

2.2 Divide and Conquer the Frequency Components

Treating a time series as a signal allows us to break it down into a linear combination of sinusoidal components without any information loss. Each component possesses a unique frequency, initial phase, and amplitude. Forecasting directly on the original time series can be challenging, but forecasting each frequency component is comparatively straightforward, as we only need to apply a phase bias to the sinusoidal wave based on the time shift. Subsequently, we linearly combine these shifted sinusoidal waves to obtain the forecasting result.

Specifically, the resulting forecasted values maintain the frequency features of the original time series 89 with a reasonable time shift, ensuring that semantic consistency is maintained. 90

However, forecasting each sinusoidal component in the time domain can be cumbersome, as the 91 sinusoidal components are treated as a sequence of data points. To address this, we propose conducting 92 this manipulation in the complex frequency domain, which offers a more compact and information93 rich representation, as described below. 94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

3 Method

3.1 Preliminary: FFT and Complex Frequency Domain

The Fast Fourier Transform (FFT, (Brigham & Morrow, 1967)) is a widely used algorithm for efficiently computing the Discrete Fourier Transform (DFT) of a sequence of complex numbers. The DFT is a mathematical operation that converts a discrete-time signal from the time domain to the complex frequency domain. In cases where the input signal is real, such as in time series analysis, the Real FFT (rFFT) is commonly used to obtain a compact representation. With an input of N real numbers, the rFFT produces a sequence of N/ 2 + 1 complex numbers that represent the signal in the complex frequency domain.

Complex Frequency Domain

In Fourier analysis, the complex frequency domain is a representation of a signal in which each frequency component is characterized by a complex number. This complex number captures both the amplitude and phase of the component, providing a comprehensive description. The amplitude of a frequency component represents the magnitude or strength of that component in the original time-domain signal. In contrast, the phase represents the temporal shift or delay introduced by that component. Mathematically, the complex number associated with a frequency component can be represented as a complex exponential element with a given amplitude and phase:

where X ( f ) is the complex number associated with the frequency component at frequency f , | X ( f ) | is the amplitude of the component, and θ ( f ) is the phase of the component. As shown in Fig. 1(a), in the complex plane, the complex exponential element can be visualized as a vector with a length equal to the amplitude and angle equal to the phase: Imaginary 𝑋 ᇱ = ( 𝑋 ଵ ⋅ |𝑋 ଶ |)𝑒 ௝(ఏ భ ାఏ మ )

௝ఏభ

Therefore, the complex number in the complex frequency domain provides a concise and elegant 116 means of representing the amplitude and phase of each frequency component in the Fourier transform. 117 Real 𝜃ଵ 𝜃ଶ

Imaginary

Real

𝑋 =

𝑋 𝑐𝑜𝑠𝜃 + 𝑗 𝑋 𝑠𝑖𝑛𝜃|

= 𝑋 𝑒 ௝ఏ

𝜃

𝑋 𝑐𝑜𝑠𝜃

𝑋 𝑠𝑖𝑛𝜃

|𝑋|

Imaginary

Real

𝑋ଵ = |𝑋 ଵ |𝑒

௝ఏభ

𝑋 ᇱ

= ( 𝑋 ଵ

⋅ |𝑋

ଶ |)𝑒

௝(ఏ భ ାఏ మ )

𝜃ଵ

𝜃ଶ

(a) Complex number on the complex plane

Imaginary

(b) Complex number multiplication

= 𝑋 𝑒 ௝ఏ

Real 𝜃 𝑋 𝑐𝑜𝑠𝜃 𝑋 𝑠𝑖𝑛𝜃 |𝑋| Time Shift and Phase Shift . The time shift of a signal corresponds to the phase shift in the frequency 118 domain. Especially in the complex frequency domain, we can express such phase shift by multiplying 119 a unit complex exponential element with the corresponding phase. Mathematically, if we shift a 120 signal x ( t ) forward in time by a constant amount τ , resulting in the signal x ( t -τ ) , the Fourier 121 transform is given by: 122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

The shifted signal still has an amplitude of | X ( f ) | , while the phase θ τ ( f ) = θ ( f ) -2 πfτ shows a shift which is linear to the time shift.

In summary, the amplitude scaling and phase shifting can be simultaneously expressed as the multiplication of complex numbers, as shown in Fig. 1(b).

3.2 FITS Pipeline

Motivated by the fact that a longer time series provides a higher frequency resolution in its frequency representation, we train FITS to generate an extended time series segment by interpolating the frequency representation of the input time series segment. We use a complex-valued linear layer to learn such interpolation. According to the fact that the amplitude scaling and phase shifting can be conveniently expressed as the multiplication of complex numbers, such complex linear combination allows FITS to effectively incorporate both the amplitude scaling and phase shift of frequency components during the interpolation process. As shown in Fig. 2, we use rFFT to project time series segments to the complex frequency domain. After the interpolation, the frequency representation is projected back with inverse rFFT (irFFT).

RIN

𝑋 ෨

𝑋′

෡

𝑥

Value

Time

0

2 nd Harmo.

𝑋 ෨ ′

COF

0

0

0

𝑋 ෠

𝑥 ො ௟௕௪

Value

Time

𝑥 ො ௛

rFFT

LPF

iRIN

irFFT

0 Padding

Complex-valued

Linear Layer

139

frequency representation. To eliminate the 0-frequency component, we pass it through reversible

140

Time instance-wise normalization (RIN) (Kim et al., 2022) to obtain a zero-mean instance. As a result,

Furthermore, we incorporate a low-pass filter (LPF) into the FITS model to further reduce its size. 143 The LPF removes high-frequency components above a specified cutoff frequency, resulting in a more 144 compact model representation while retaining the important information of the time series. The 145 rationale behind this design will be elaborated in the subsequent section. Despite operating in the 146 frequency domain, FITS is supervised in the time domain using common loss functions such as Mean 147 Squared Error (MSE) after the irFFT, allowing for diverse supervision tailored to different time series 148 downstream tasks. 149

150

151

152

153

In the case of forecasting tasks, we generate the look-back window along with the horizon as shown in Fig. 2. This allows us to provide supervision for forecasting and backcasting, where the model is encouraged to accurately reconstruct the look-back window. Our ablation study reveals that combining backcast and forecast supervision can yield improved performance in certain scenarios.

For reconstruction tasks, we downsample the original time series segment based on a specific 154 downsampling rate. Subsequently, FITS is employed to perform frequency interpolation, enabling 155 the reconstruction of the downsampled segment back to its original form. Thus, direct supervision 156 is applied using reconstruction loss to ensure faithful reconstruction. The reconstruction tasks also 157 follow the pipeline in Fig. 2 with the supervision replaced with reconstruction loss. 158

Time

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

3.3 Key Mechanisms of FITS

Complex Frequency Linear Interpolation. To control the output length of the model, we introduce an interpolation rate denoted as η , which represents the ratio of the model's output length L o to its corresponding input length L i .

It is worth noting that frequency interpolation operates on the normalized complex frequency representation, which has half the length of the original time series. Importantly, this interpolation rate can also be applied to the frequency domain, as indicated by the equation:

Based on this formula, with an arbitrary frequency f , the frequency band 1 ∼ f in the original signal is linearly projected to the frequency band 1 ∼ ηf in the output signal. As a result, we define the input length of our complex-valued linear layer as L and the interpolated output length as ηL . Notably, when applying the Low Pass Filter (LPF), the value of L corresponds to the cutoff frequency (COF) of the LPF. After performing frequency interpolation, the complex frequency representation is zero-padded to a length of L o / 2 , where L o represents the desired output length. Prior to applying the irFFT, an additional zero is introduced as the representation's zero-frequency component.

Low Pass Filter (LPF). The primary objective of incorporating the LPF within FITS is to compress the model's volume while preserving essential information. The LPF achieves this by discarding frequency components above a specified cutoff frequency (COF), resulting in a more concise frequency domain representation. The LPF retains the relevant information in the time series while discarding components beyond the model's learning capability. This ensures that a significant portion of the original time series' meaningful content is preserved. As demonstrated in Fig. 3, the filtered waveform exhibits minimal distortion even when only preserving a quarter of the original frequency domain representation. Furthermore, the high-frequency components filtered out by the LPF typically comprise noise and trends, which are inherently irrelevant for effective time series modeling.

0

200

400

15

20

25

30

0

100

200

0

200

400

No Filter MSE: 0.0000

(a) Original

0

200

400

15

20

25

30

0

100

200

0

200

400

COF

COF: 120 MSE: 0.0727

(b) COF at 6

th

harmonic

0

200

400

15

20

25

30

0

100

200

0

200

400

COF

COF: 60 MSE: 0.1660

(c) COF at 3

rd

harmonic

0

200

400

15

20

25

30

0

100

200

0

200

400

COF

COF: 40 MSE: 0.4296

(d) COF at 2

nd

harmonic

Selecting an appropriate cutoff frequency (COF) remains a nontrivial challenge. To address this, we propose a method based on the harmonic content of the dominant frequency. Harmonics, which are integer multiples of the dominant frequency, play a significant role in shaping the waveform of a time series. By aligning the cutoff frequency with these harmonics, we keep relevant frequency components associated with the signal's structure and periodicity. This approach leverages the inherent relationship between frequencies to extract meaningful information while suppressing noise and irrelevant high-frequency components. The impact of COF on different harmonics' waveforms is shown in Fig. 3. We further elaborate on the impact of COF in our experimental results.

190

191

4 Experiments for Forecasting

4.1 Forecasting as Frequency Interpolation

Typically, the forecasting horizon is shorter than the given look-back window, rendering direct 192 interpolation unsuitable. Instead, we formulate the forecasting task as the interpolation of a look-back 193 window, with length L , to a combination of the look-back window and forecasting horizon, with 194 length L + H . This design enables us to provide more supervision during training. With this approach, 195 we can supervise not only the forecasting horizon but also the backcast task on the look-back window. 196 Our experimental results demonstrate that this unique training strategy contributes to the improved 197 performance of FITS. The interpolation rate of the forecasting task is calculated by: 198

where L represents the length of the look-back window and H represents the length of the forecasting horizon.

4.2 Experiment Settings

Datasets. All datasets used in our experiments are widely-used and publicly available real-world datasets, including, Traffic, Electricity, Weather, ETT (Zhou et al., 2021). We summarize the characteristics of these datasets in Tab. 1. Apart from these datasets for long-term time series forecasting, we also use the M4 dataset to test the short-term forecasting performance.

Baselines . To evaluate the performance of FITS in comparison to state-of-the-art time series forecasting models, including PatchTST (Nie et al., 2023), TimesNet (Wu et al., 2023), FEDFormer (Zhou et al., 2022a), FiLM (Zhou et al., 2022b) and LTSF-Linear (Zeng et al., 2023), we directly refer to the reported results in the original papers under the same settings. We report the comparison with other transformer-based methods in the appendix.

Evaluation metrics . We follow the previous works (Zhou et al., 2022a; Zeng et al., 2022; Zhang et al., 2022) to compare forecasting performance using Mean Squared Error (MSE) as the core metrics. Moreover, to evaluate the short-term forecasting, we symmetric Mean Absolute Percentage Error (SMAPE) following TimesNet (Wu et al., 2023).

Implementation details . Following the settings of LTSF-Linear (Zeng et al., 2023), we set the look-back window of FITS as 720 for any forecasting horizon. Further experiments also show that a longer look-back window can result in better performance. To avoid information leakage, We choose the hyper-parameter based on the performance of the validation set.

4.3 Comparisons with SOTAs

Competitive Performance with High Efficiency

We present the results of our experiments on long-term forecasting in Tab. 2 and Tab. 3. The results for short-term forecasting on the M4 dataset are provided in the Appendix. Remarkably, our FITS consistently achieves comparable or even superior performance across all experiments.

Tab. 4 presents the number of trainable parameters for various TSF models using a look-back window of 96 and a forecasting horizon of 720 on the Electricity dataset. The table clearly demonstrates the exceptional efficiency of FITS compared to other models.

Among the listed models, the parameter counts range from millions down to thousands. Notably, large models such as TimesNet and Pyraformer require a staggering number of parameters, with

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

300.6M and 241.4M, respectively. Similarly, popular models like Transformer, Informer, Autoformer, and FEDformer have parameter counts in the range of 13.61M to 20.68M. Even the lightweight yet state-of-the-art model PatchTST has a parameter count of over 1 million.

In contrast, FITS stands out as a highly efficient model with an impressively low parameter count. With only 4.5K to 16K parameters, FITS achieves comparable or even superior performance compared to these larger models. It is worth highlighting that FITS requires significantly fewer parameters compared to the next smallest model, Dlinear, which has 139.7K parameters. For instance, when considering a 720 look-back window and a 720 forecasting horizon, the Dlinear model requires over 1 million parameters, whereas FITS achieves similar performance with only 10k-50k parameters.

This analysis showcases the remarkable efficiency of FITS. Despite its small size, FITS consistently achieves competitive results, making it an attractive option for time series analysis tasks. FITS demonstrates that achieving state-of-

the-art or close to state-of-the-art performance with a considerably reduced parameter footprint is possible, making it an ideal choice for resource-constrained environments.

Case Study on ETTh2 Dataset

We conduct a comprehensive case study on the performance of FITS using the ETTh2 dataset, which further highlights the impact of the look-back window and cutoff frequency on model performance. We provide a case study on other datasets in the Appendix. In our experiments, we observe that increasing the look-back window generally leads to improved performance, while the effect of increasing the cutoff frequency is minor.

Tab. 5 showcases the performance results obtained with different look-back window sizes and cutoff frequencies. Larger look-back windows tend to yield better performance across the board. On the other hand, increasing the cutoff frequency only results in marginal performance improvements. However, it is important to note that higher cutoff frequencies come at the expense of increased computational resources, as illustrated in Tab. 6.

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

Considering these observations, we find utilizing a longer look-back window in combination with a low cutoff frequency to achieve near state-of-the-art performance with minimal computational cost. For instance, FITS surpasses other methods when employing a 720 look-back window and setting the cutoff frequency to the second harmonic. Remarkably, FITS achieves state-of-the-art performance with a parameter count of only around 10k. Moreover, by reducing the look-back window to 360, FITS already achieves close-to-state-of-the-art performance by setting the cutoff frequency to the second harmonic, resulting in a further reduction of the model's parameter count to under 5k (as shown in Tab. 6).

These results emphasize the lightweight nature of FITS, making it highly suitable for deployment and training on edge devices with limited

computational resources. By carefully selecting the look-back window and cutoff frequency, FITS can achieve excellent performance while maintaining computational efficiency, making it an appealing choice for real-world applications.

5 Experiment for Anomaly Detection

5.1 Reconstruction as Frequency Interpolation

As discussed before, we tackle the anomaly detection tasks in the self-supervised reconstructing approach. Specifically, we make a N time down-sampling on the input and train a FITS network with an interpolation rate of η Rec = N to up-sample it.

5.2 Experiment Settings

Datasets . We use five commonly used benchmark datasets: SMD (Server Machine Dataset (Su et al., 2019)), PSM (Polled Server Metrics (Abdulaal et al., 2021)), SWaT (Secure Water Treatment (Mathur &Tippenhauer, 2016)), MSL (Mars Science Laboratory rover), and SMAP (Soil Moisture Active Passive satellite) (Hundman et al., 2018).

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

Baselines . We compare FITS with models such as TimesNet (Wu et al., 2023), Anomaly Transformer (Xu et al., 2022), THOC (Shen et al., 2020), Omnianomaly (Su et al., 2019). Following TimesNet (Wu et al., 2023), we also compare the anomaly detection performance with other models (Zeng et al., 2023; Zhang et al., 2022; Woo et al., 2022; Zhou et al., 2022a).

Evaluation metrics . Following the previous works (Xu et al., 2022; Shen et al., 2020; Wu et al., 2023), we use Precision, Recall, and F1-score as metrics.

Implementation details . We use a window size of 200 and downsample the time series segment by a factor of 4 to match the original segment during training with the FITS model. Anomaly detection follows the methodology of the Anomaly Transformer (Xu et al., 2022), where time points exceeding a certain reconstruction loss threshold are classified as anomalies. The threshold is selected based on the highest F1 score achieved on the validation set. To handle consecutive abnormal segments, we adopt a widely-used adjustment strategy (Su et al., 2019; Xu et al., 2018; Shen et al., 2020), considering all anomalies within a specific successive abnormal segment as correctly detected when one anomalous time point is identified. This approach aligns with real-world applications, where an abnormal time point often triggers the attention to the entire segment.

5.3 Comparisons with SOTAs

As shown in Tab. 7, FITS achieves remarkable results on several datasets. Notably, on the SMD and SWaT datasets, FITS exhibits exceptional performance with F1-scores almost reaching perfection at around 99.95% and 98.9%, respectively. This demonstrates FITS' ability to accurately detect anomalies and classify them correctly. In comparison, other models, such as TimesNet, Anomaly Transformer, and Stationary Transformer, struggle to match FITS' performance on these datasets.

However, FITS shows comparatively lower performance on the SMAP and MSL datasets. These datasets present a challenge due to their binary event data nature, which may not be effectively captured by FITS' frequency domain representation. While models specifically designed for anomaly detection, such as THOC and Omni Anomaly, achieve higher F1-scores on these datasets.

For a more comprehensive evaluation, waveform visualizations and detailed analysis can be found in the appendix, providing deeper insights into FITS' strengths and limitations in different anomaly detection scenarios. It is important to note that the reported results are achieved with a parameter range of 1-4K and MACs (Multiply-Accumulate Operations) of 10-137K, which will be further detailed in the appendix.

6 Conclusions and Discussion

In this paper, we propose FITS for time series analysis, a low-cost model with 10 k parameters that can achieve performance comparable to state-of-the-art models that are often several orders of magnitude larger. As a frequency-domain modeling technique, FITS has difficulty handling binary-valued time series and time series with missing data. For the former category, time-domain modeling is preferable as the raw data format is sufficiently compact. For the latter category, we could first employ simple yet effective time-domain imputation techniques and then apply FITS for efficient analysis.

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

References

Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery; Data Mining , KDD '21, pp. 2485-2494, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325. doi: 10.1145/3447548.3467174. URL https://doi.org/10.1145/3447548.3467174 .

O. Brigham and R. E. Morrow. The fast fourier transform. IEEE Spectrum , 4(12):63-70, 1967. doi: 10.1109/MSPEC.1967.5217220.

Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderstrom. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &amp Data Mining . ACM, jul 2018. doi: 10.1145/3219819.3219845. URL https: //doi.org/10.11452F3219819.3219845 .

Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations , 2022. URL https://openreview.net/ forum?id=cGDAkQo1C0p .

James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. Fnet: Mixing tokens with fourier transforms, 2022.

Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. In Advances in Neural Information Processing Systems , 2022.

Aditya P. Mathur and Nils Ole Tippenhauer. Swat: a water treatment testbed for research and training on ics security. In 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater) , pp. 31-36, 2016. doi: 10.1109/CySWater.2016.7469060.

Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations , 2023.

Lifeng Shen, Zhuocong Li, and James Kwok. Timeseries anomaly detection using temporal hierarchical one-class network. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 13016-13026. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/ 2020/file/97e401a02082021fd24957f852e0e475-Paper.pdf .

Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery; Data Mining , KDD '19, pp. 2828-2837, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450362016. doi: 10.1145/3292500.3330672. URL https://doi.org/10.1145/3292500. 3330672 .

Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting, 2022.

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems , 34:22419-22430, 2021.

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations , 2023.

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

Haowen Xu, Yang Feng, Jie Chen, Zhaogang Wang, Honglin Qiao, Wenxiao Chen, Nengwen Zhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, Youjian Zhao, and Dan Pei. Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 . ACM Press, 2018. doi: 10.1145/3178876.3185996. URL https://doi.org/10.1145/2F3178876.3185996 .

Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy, 2022.

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504 , 2022.

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? 2023.

Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprint arXiv:2207.01186 , 2022.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pp. 11106-11115, 2021.

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning , 2022a.

Tian Zhou, Ziqing Ma, xue wang, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, and Rong Jin. FiLM: Frequency improved legendre memory model for long-term time series forecasting. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems , 2022b. URL https://openreview.net/forum?id= zTQdHSQUQWc .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce FITS, a lightweight model for time series analysis. Unlike existing models that process raw time-domain data directly, FITS operates on the principle of manipulating time series through interpolation in the complex frequency domain. By discarding high-frequency components that have a negligible impact on the time series data, FITS achieves performance comparable to state-of-the-art models for time series forecasting and anomaly detection tasks. Additionally, FITS has a remarkably compact size, consisting of only approximately 10k parameters. This lightweight model is mainly constituted by a simple Complex-valued Linear Layer functioning on frequency domain.\n\nSTRENGTHS:\n- About the presentation: Straight-forward to follow, simple and direct sentences.\n- About the contribution: The proposed FITS framework is very light-weighted, since its learning parameters mainly coming from 1 dense layer. This contribution make its great application for actual real-world scenarios, applying on edge devices. The natural idea about *Low Pass Filter* is not new, but is used with good reasoning.\n- About the experiment: Authors illustrate the effectivenesses of FITS with two main time series tasks: Forecasting and detecting anomaly. While these experiments are not extensive, it effectively support the contributions the authors claim: Comparative performance and much more lightweight compared to existing SOTAs.\n\nWEAKNESSES:\n- About the presentation: Figure 1 has some too dim colors and hard to see; Figure 2 is also quite small, the font size should be increased.\n- About the contribution:\n    - Details about input or output dimensions are not discussed (only discuss about the temporal axis). It is not clear whether the framework can be applied to multivariate series? How to choose cutoff frequency in case of multivariate series, when the harmonics are likely to be different over different variables?\n    - While the different components constituting FITS are used with clear intentions, these techniques or algorithms (rFFT, RIN, low-pass filter, …) are not new and even well established.\n    - The experiments for both time series and anomaly detection tasks suggest the framework has a great variance when evaluating on different datasets. In general, FITS cannot achieve SOTA performance in many scenarios, which may be unsuitable for performance-critical applications.\n\nQUESTIONS:\n- Are there any plans to enhance the visibility of the figure, perhaps by adjusting the colors, the font size, or using a different visualization technique?\n- Can the proposed framework be extended to handle multivariate time series? If so, how do you choose the cutoff frequency when the harmonics are likely to differ across different variables?\n\nLIMITATIONS:\nAuthors recognize following weaknesses:\n- FITS struggles with binary-valued time series and time series with missing data.\n- Binary-valued time series are better suited for time-domain modeling due to their compact raw data format.\n- For time series with missing data, a two-step approach is suggested: apply simple time-domain imputation techniques before utilizing FITS for analysis.\n\nThe authors should consider amend the weaknesses mentioned above if possible or make some modifications to the manuscripts to rebuttal the comments.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper builds a model for time series learning in the frequency domain. The key idea is to discard high-frequency components to reduce the model size. This leads to a simple model under 10k parameters, 50x smaller than DLinear. Experimental results suggest the model can achieve comparable performance to the state-of-the-art methods on long-term forecasting tasks and several anomaly detection tasks.\n\nSTRENGTHS:\n1. Transforming time series into a frequency domain and then training a model is an interesting idea.\n2. The model is extremely small, even 50X smaller than DLinear, which is already very small.\n3. The experiments on long-term forecasting are strong.\n\nWEAKNESSES:\n1. It is unclear whether the approach has practical value. DLinear could be already sufficiently small to fit in a normal edge device. Thus, it is not persuading to have an even smaller model.\n2. Training efficiency is not reported.\n3. The experiments on anomaly detection are not strong. Firstly, all the baselines are neural networks. However, traditional methods like OCSVM and IForest can have strong performance [1]. Secondly, the existing anomaly detection datasets could be flawed. Thus, I am particularly interested to know how the proposed method performs in the synthetic data provided in [1]. The pattern-wise outliers in [1] are synthesized by modifying the sinusoidal waves. Thus, the proposed method seems to well align with the design of this dataset. I am curious how it performs on this dataset.\n\n[1] Revisiting Time Series Outlier Detection: Definitions and Benchmarks\n\nQUESTIONS:\n1. How does the proposed method perform on the synthetic anomaly dataset [1]?\n2. Can the proposed method outperform classical anomaly detection methods, such as OCSVM and IForest?\n3. Why do we need such a small model, as DLinear may already be small enough for most of the edge devices?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a novel method on time-series forecasting and anomaly detection. The solution relies on the frequency-domain feature of the given time series and proposes to use a linear model to interpolate the data in the frequency domain. After that, the model utilizes the inverse FFT operation to transform the frequency domain data into time domain. The interpolated time series is longer than the original one. Thus, the forecasting has been conducted based on the augmented time series sequence.\n\nSTRENGTHS:\n1. The idea is novel. The interpolation of the time series in the frequency domain provides some insights for this community.\n\n2. The model efficiency is quite amazing, only 10K parameters have provided the comparable even better performance for the time-series prediction task.\n\nWEAKNESSES:\n1. The experiments are not promising. What's the number of the random runs and did you control the random seeds? I suggest to report the mean and std values of the results.\n\n2. As this is the new architecture of the time-series model, more analysis should be conducted such as more prediction tasks, more ablation studies for the key components of the method, etc.\n\n3. The reason behind the effectiveness of the proposed method remains unclear to me.\n\nQUESTIONS:\nN/A.\n\nLIMITATIONS:\nSee those in weakness part. \n\nMoreover, the augmented time series is not appropriate for the forecasting tasks. Forecasting is the extension for one given time series, however, simply interpolating for augmenting the original time series is not intuitively correct, because it has modified the original data piece rather than prediction upon the given information (in time domain).\nAnd the experimental results show that the performance is not comparable to the baseline methods such as PatchTST. I suggest the authors to further improve the method and figure out the reason of its effectiveness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

comments: SUMMARY:
This paper proposes an impressive compact model, named FITS, for time series tasks, including forecasting and anomaly detection. FITS achieves manipulation to time series through interpolation in the frequency domain. The whole framework is quite simple and has remarkably few parameters. FITS achieves competitive performance to SOTA baselines on both forecasting and anomaly detection with about 50 times fewer parameters. With such impressive performance, the proposed model would have a certain impact on the community.

STRENGTHS:
1. The idea of implementing time series forecasting and anomaly detection through interpolation in frequency domain is interesting and technically sound.
2. Detailed designs, including LPF and utilization of RevIN, are well motivated and described, making the whole framework reasonable and easy to follow.
3. The proposed model achieves impressive performance on both forecasting and anomaly detection with a remarkably compact size.

WEAKNESSES:
1. Lack of time consumption analysis. Due to rFFT and irFFT in the model, time efficiency is the main concern. 
2. The coverage of related works is barely satisfactory. Adding some preliminary about manipulation in frequency domain to the manuscript would be helpful to understand the model.
3. Typos. For example, duplicated citations in line 383 and line 385.

QUESTIONS:
1. It is appreciated to further list time consumption of the proposed model under different settings and compare time efficiency with baselines.
2. For forecasting task, what if the model is supervised in frequency domain? 
3. Did authors try other interpolation methods, e.g., convolution?

LIMITATIONS:
Not applicable.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a transformer model for forecasting and anomaly detection. The model performs both forecasting and reconstruction in the frequency domain with a fraction of parameters compared to state-of-the-art transformers. The model demonstrates impressive performance on anomaly detection and forecasting tasks.\n\nSTRENGTHS:\n1. This is an interesting work and novel to the best of my knowledge. \n2. I strongly believe that the lightweight nature of the model is an asset. \n3. The model demonstrates near state-of-the-art performance. \n4. The paper is well-written.\n\nWEAKNESSES:\nMajor: Most of these are not deal breakers, but the following would make the evaluation of the model stronger:\n1. Baselines for forecasting : The authors compare with only transformer based models. I would encourage the authors to compare with advanced non-transformer based models such as N-HiTS [1] and N-BEATS [2]. \n2. Handling of multi-variate time-series data (see Q1)\n3. Baselines for anomaly detection: I would again encourage the authors to compare with state-of-the-art time-series anomaly detection e.g., DGHL [3]. \n4. Evaluation for anomaly detection detection: I would encourage the authors to use standard evaluation metrics for anomaly detection like adjusted best F1 [3, 4], Average Precision [4, 5], and Volume Under Surface (VUS) [6]. The datasets used also have known flaws [7]. \n\nMinor:\n1. \"To avoid information leakage, We choose\" --> \"To avoid information leakage, we choose\"\n\nReferences:\n\n1. Challu, Cristian, et al. \"NHITS: Neural Hierarchical Interpolation for Time Series Forecasting.\" Proceedings of the AAAI Conference on Artifi\n2. Oreshkin, Boris N., et al. \"N-BEATS: Neural basis expansion analysis for interpretable time series forecasting.\" arXiv preprint arXiv:1905.10437 (2019).\n3. Challu, Cristian I., et al. \"Deep generative model with hierarchical latent factors for time series anomaly detection.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2022.\n4. Goswami, Mononito, et al. \"Unsupervised model selection for time-series anomaly detection.\" arXiv preprint arXiv:2210.01078 (2022).\n5. Schmidl, Sebastian, Phillip Wenig, and Thorsten Papenbrock. \"Anomaly detection in time series: a comprehensive evaluation.\" Proceedings of the VLDB Endowment 15.9 (2022): 1779-1797.\n6. Paparrizos, John, et al. \"Volume under the surface: a new accuracy evaluation measure for time-series anomaly detection.\" Proceedings of the VLDB Endowment 15.11 (2022): 2774-2787.\n7. Wu, Renjie, and Eamonn Keogh. \"Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress.\" IEEE Transactions on Knowledge and Data Engineering (2021).\n\nQUESTIONS:\n1. How does the model handle multi-variate time-series data? Does the model capture cross-channel information? \n2. What are MACs (Multiply-Accumulate Operations)? Neither the paper nor the supplementary material talks about it. \n3. I would like to know exactly how the reconstruction works, I am not sure if I understand it with the given information.\n\nLIMITATIONS:\nThe authors discuss some limitations of your work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}