Title: Frequency-aware Generative Models for Multivariate Time Series Imputation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Time series; Time series Imputation; Generative Models; Frequency domain
Abstract: Missing data in multivariate time series are common issues that can affect the analysis and downstream applications.
Although multivariate time series data generally consist of the trend, seasonal and residual terms, existing works mainly focus on optimizing the modeling for the first two items. However, we find that the residual term is more crucial for getting accurate fillings, since it is more related to the diverse changes of data and the biggest component of imputation errors.
Therefore, in this study, we introduce frequency-domain information and design Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI). Specifically, FGTI employs a high-frequency filter to boost the residual term imputation, supplemented by a dominant-frequency filter for the trend and seasonal imputation. Cross-domain representation learning module then fuses frequency-domain insights with deep representations.
Experiments over various datasets with real-world missing values show that FGTI achieves superiority in both data imputation and downstream applications.

Frequency-aware Generative Models for Multivariate Time Series Imputation

Abstract

Missing data in multivariate time series are common issues that can affect the analysis and downstream applications. Although multivariate time series data generally consist of the trend, seasonal and residual terms, existing works mainly focus on optimizing the modeling for the first two items. However, we find that the residual term is more crucial for getting accurate fillings, since it is more related to the diverse changes of data and the biggest component of imputation errors. Therefore, in this study, we introduce frequency-domain information and design F requency-aware G enerative Models for Multivariate T ime Series I mputation ( FGTI ). Specifically, FGTI employs a high-frequency filter to boost the residual term imputation, supplemented by a dominant-frequency filter for the trend and seasonal imputation. Cross-domain representation learning module then fuses frequency-domain insights with deep representations. Experiments over various datasets with real-world missing values show that FGTI achieves superiority in both data imputation and downstream applications.

1 Introduction

Missing data are commonly observed in the multivariate time series due to diverse reasons [25], which would encumber subsequent analysis and applications [17]. It is not surprising that more accurate missing data imputation generally leads to better performance in downstream applications. 1

Existing techniques [13; 28] have revealed that time series data can be decomposed into three distinct terms, i.e., trend, seasonal, and residual, and try to compute an imputation by modeling the first two items as accurately as possible. Figure 1 reports a survey for the imputation accuracy of the three terms by representative imputation methods over the pre-decomposed KDD [6] dataset with 10% missing values. 2 The dataset comprises 8,034 consecutive readings of meteorological and air quality data taken over a year in Beijing. In this dataset, the trend term may reflect long-term changes in climate or air quality conditions, and the seasonal term might capture patterns associated with different seasons. Moreover, the residual term may consist of short-term, irregular, and high-frequency changes. As shown in Figure 1, the imputation error is mainly caused by the residual term, which has not been well studied, unfortunately.

Recent studies indicate that the high-frequency components are intricately related to the residual [46; 58; 26] and contain critical information for imputing the residual term. Unfortunately, deep learning architectures cannot generalize well in modeling high-frequency components. [38; 43]. To

(a)

(b)

meet this challenge, we design F requency-aware G enerative Models for Multivariate T ime Series I mputation ( FGTI ), which can extract frequency-domain information and use two cross-domain (i.e., time-domain and frequency-domain) representation learning modules to guide the generation process of deep models. Specifically, we start with the high-frequency filter designed to extract the high-frequency information essential for guiding the accurate imputation of the residual term. This choice is consistent with recognizing the critical role of high-frequency information in imputation accuracy. Then, we introduce the dominant-frequency filter to address the potential challenge posed by high-frequency information for imputing trend and seasonal terms [4]. Furthermore, our cross-domain representation learning frameworks combine frequency-domain information with deep representations in the time-domain, enabling seamlessly intertwining frequency-domain information with time and attribute dependencies modeling.

We design a frequency-aware generative model FGTI with frequency-domain information integrated by the high-frequency filter and the dominant-frequency filter, to enhance the awareness of the frequency-domain.

We introduce two cross-domain representation learning modules that provide models with prior knowledge of intricate frequency-related patterns for missing data imputation.

We evaluate FGTI on three time series datasets with real-world missing values, which demonstrates the superiority of FGTI in both imputation accuracy and downstream applications.

2 Related work

Traditional imputation methods usually employ the statistics, such as mean value [23], median value [16], or last observed value [3], to impute missing data for multivariate time series. It is not surprising that such traditional signals cannot make full use of the valuable semantics of available data. BTMF [9] and TIDER [28] employ the low-rank matrix factorization to impute missing data. Unfortunately, due to the matrix capacity's limitation, it is still challenging to accurately match imputation values with the underlying complex relationships and dependencies.

Many studies have shown that deep learning based imputation methods are effective to fill multivariate time series data, such as BRITS [6], TST [57], SAITS [14], STCPA [54], TimesNet [52]. According to [48], forecasting models [50; 55] can also be applied to imputation task. Additionally, GRIN [12] and DAMR [39] use graph neural networks to incorporate known relationships between attributes. However, these methods with fixed model outputs cannot capture the uncertainty and variability of missing values. Recently, researchers have attempted to utilize large language models (LLMs) as the backbone for time series analysis [59]. Since there are significant differences between time series data and natural language, it still has a lot of room for improvement.

To capture the uncertainty and variability, researchers introduce generative models [8] into the missing data imputation by learning the implicit distribution of missing values. In the early stage, researchers mainly use variational Autoencoders (VAE) or generative adversarial networks (GAN) to generate new samples that match the distribution of the training dataset and impute missing values with the generated samples. VAE-based methods learn data distributions by optimizing the reconstruction error and regularizing in the latent space [31; 15; 35; 36; 11]. However, they may not capture the complex variability of missing data well and may produce inaccurate results, especially when the latent spaces are not well aligned. GAN-based approaches use the adversarial training technique of the generator and discriminator to improve the imputation results [56; 29; 30; 34]. Unfortunately, they may face convergence difficulties that can affect the imputation accuracy.

Diffusion models have been introduced into the imputation task recently, considering the success in various fields [40; 18; 24]. To impute time series data, CSDI [44] designs conditional score-based diffusion models with the conditions only on observed values, and SSSD [2] utilizes both conditional diffusion models and structured state space models. Additionally, MIDM [49] develops the noise sampling, addition, and denoising mechanisms, and PriSTI [27] further studies the enhanced prior modeling by extracting spatio-temporal dependencies as contextual conditions for spatio-temporal data imputation. However, as analyzed in the introduction, existing studies underestimate the importance of accurately modeling the residual term for missing data imputation.

For the time series imputation methods in frequency domain, mvLSWimpute [51] utilizes wavelet transforms to guide imputation, APDNet [60] uses the Fourier Temporal and Fourier Variable Interaction modules to model dependencies. In addition, the frequency domain time series forecasting methods FEDformer [58], FreTS [55] can also be applied to imputation task. However, they did not consider using frequency domain information to model the missing data's residual terms accurately, which is critical for boosting the overall imputation performance.

In contrast, our FGTI captures high-frequency information and dominant-frequency information to get a more accurate modeling of the residual term, while assisting in describing trend and seasonal terms.

3 Frequency-aware Generative Models

In this paper, we focus on the incomplete multivariate time series imputation problem. The input multivariate time series X = ( X 1 , . . . , X D ) ∈ R D × L is a set of D attribute values recorded at L consecutive timestamps, where each attribute series X d ∈ R L . Each element x ij in X is the observation of the i -th attribute at the j -th timestamp, which is probably missing. We use the binary mask matrix M ∈ { 0 , 1 } D × L to represent the missing status of observations in X , where m ij = 1 in M denotes that x ij is complete, otherwise x ij = 0 . In our context, we refer to the imputation target as ˆ X . During the training of the imputation model, we choose some observations as the imputation target. When we impute missing values, we treat all the missing values as the imputation target.

In Figure 1, it is evident that the main obstacle to improving multivariate time series imputation is the residual term. Considering a recognized fact that the residual term often contains high-frequency components from the perspective of Fourier analysis [46; 58; 26], introducing prior knowledge of frequency-domain information can be a feasible approach to enhancing model performance.

As a class of superior data imputation models, deep generative models treat the time series imputation task as calculating the conditional imputation target probability distribution q ( ˆ X 0 | C ) , where q ( ˆ X 0 ) is the clean data distribution and existing deep generative imputation models [56; 29; 44] use the observed values X in the time-domain as the condition C for probability distribution calculation. Note that we denote the complete or the imputed imputation target as the clean imputation target ˆ X 0 .

However, frequency principal [38; 43] reveals that deep models cannot generalize well to highfrequency information. As a result, it may not accurately impute the residual term [13] inherent in the time series dataset that cannot be trivialized in the imputation task [28].

To tackle the above challenge, we incorporate frequency-domain information into condition C to enhance the performance of generative models. Our FGTI implements the condition C that contains time-domain observation condition X C , as well as the frequency-domain conditions C H and C D .

High-frequency Filter

𝐗

𝐂

𝐇

FFT

IFFT

Dominant-frequency Filter

𝐗

𝐂

𝐃

FFT

IFFT

3.1 Frequency-domain Condition Filter

Our frequency-domain condition includes the nonlinear transformation of two parts: the highfrequency condition that guides the residual term and the dominant-frequency condition that contains background structure information to impute the trend and seasonal terms, as shown in Figure 2.

3.1.1 High-frequency Filter

The high-frequency filter extracts high-frequency information C H from the time-domain observations, which is used to guide the imputation of residual terms in time series. Since different attributes in a multivariate time series can be heterogeneous, we consider extracting high-frequency information separately for each attribute series X d ∈ R L , where d = 1 , . . . D .

We first interpolate X d , then obtain the amplitude vector A ∈ R ⌊ ( L +1) / 2 ⌋ for X d over sample frequency components F = { 1 L , . . . , 1 L ⌊ ( L +1) / 2 ⌋} by the Fast Fourier Transform ( FFT ):

To get the high-frequency condition, we discard the frequency components below a cutoff threshold F and map the remaining components to time-domain by the Inverse Fast Fourier Transform ( IFFT ),

where ⊙ denotes the Hadamard product.

Finally, we concatenate the corresponding high-frequency information vectors C H d for each attribute sequence to form the high-frequency condition C H ∈ R D × L ,

For the whole input time series X , the time complexity of performing the high-frequency filtering is O ( DL log L ) . Since the time complexity of performing FFT for each attribute series is O ( L log L ) , selecting high-frequency components in the frequency domain has a time complexity of O ( L ) , and performing IFFT costs O ( L log L ) time.

3.1.2 Dominant-frequency Filter

The conditions extracted by the dominant-frequency filter from the time-domain observations not only provide the background structure information for generative models to guide the imputation of the trend and seasonal terms, but also mitigate the interference of the high-frequency condition on the imputation of the trend and seasonal terms 3 .

The dominant-frequency information is mainly composed of frequency components with large amplitudes. If we have obtained the representation ( A , F ) of X d in the frequency-domain from Equation 1, we can find the topκ frequency components with the largest amplitude according to A ,

Diffusion Step

𝑡~Uniform 1, … , 𝑇

Noise imputation

target

𝐗 "

!

Observation

condition

𝐗

𝐂

High-frequency

Filter

Dominant-frequency

Filter

𝜖~𝒩 𝐎,𝐈

Concat

Concat

Encoder

Projector

Time-frequency

Representation

Learning

Attribute-frequency

Representation

Learning

Minimize

ℒ

!

ϵ

#

Imputation target

Projector

Denoising Network

High-frequency

condition

𝐂

𝐇

Dominant-frequency

condition

𝐂

𝐃

Incomplete

time series

𝐗

Then we project these κ frequencies to the time-domain via the Inverse Fast Fourier Transform,

Finally, we compose the dominant-frequency condition C D ∈ R D × L by concatenating all C D d ,

Similar to the high-frequency filter, the time complexity of performing dominant-frequency filtering for the whole input X is O ( DL log L ) .

3.2 Cross-domain Representation Learning

With the aim of integrating frequency-domain conditions into deep generative models, we first use an encoder to map the conditions to representation C F ∈ R D × L × K in the latent space,

where K is the channel number of the latent space. In this paper, we implement the Encoder( · ) with the well-acknowledged transformer [47] backbone, since it can self-adaptively extract critical information in C H and C D by the self-attention mechanism.

To accurately capture time and attribute dependencies guided by frequency-domain information, we design two frameworks: Time-frequency representation learning and Attribute-frequency representation learning. They integrate the current intermediate hidden representations R in ∈ R D × L × K in the time-domain of deep generative models with the frequency-domain representation C F .

Specifically, we use the cross-attention mechanism that can efficiently learn the various input modalities [21; 40] for representation fusion.

3.2.1 Time-frequency Representation Learning

To capture time dependencies with the aid of frequency-domain information, we divide input hidden representation R in = { R in d ∈ R L × K } D d =1 and frequency information C F = { C F d ∈ R L × K } D d =1 into D segments according to attributes. For each pair of latent representation segment ( R in d , C F d ) ,

the learning process to obtain time-frequency representation of each attribute R t d ∈ R L × K is

where Q d = C F d · W Q t , K d = C F d · W K t , V d = R in d · W V t , W Q t , W K t , W V t ∈ R K × K are learnable weight matrices.

Then we concatenate R t l of all attributes to obtain the representation R t ∈ R D × L × K ,

3.2.2 Attribute-frequency Representation Learning

To capture dependencies between different attributes based on frequency-domain information, we divide the latent time-frequency representation R t = { R t l ∈ R D × K } L l =1 and C F = { C F l ∈ R D × K } L l =1 into L segments, according to timestamps. The learning process to obtain attribute-frequency representation of each timestamp R a l ∈ R L × K is as follows:

where Q l = C F l · W Q a , K l = C F l · W K a , V l = R tf l · W V a . To get the updated representation R a , we need to concatenate all R a l according to timestamps,

3.3 Frequency-aware Diffusion Model

Recently, diffusion generative models have demonstrated remarkable proficiency and have emerged as the leading generative models in numerous fields [24; 18]. Thus, we take the diffusion model as an example to introduce how to use frequency-domain conditions to boost missing data imputation.

Specifically, we implement FGTI by the frequency-aware diffusion model. Our frequency-aware diffusion model fuses with frequency-domain conditions to learn the conditional imputation target distribution q ( ˆ X 0 | X , C H , C D ) , through two Markov chain processes of diffusion step T , i.e., the diffusion forward process and the diffusion reverse process.

The diffusion forward process involves gradually adding Gaussian noise into the imputation target,

where q ( ˆ X t | ˆ X t -1 ) = N ( √ α t ˆ X t -1 , β t I ) , β t ∈ (0 , 1) , is a hyperparameter satisfying β t < β t +1 for t = 1 , . . . , T -1 . In addition, α t = 1 -β t and q ( ˆ X 0 ) is the complete data distribution.

According to DDPM [20], ˆ X t has a closed-form solution,

where ϵ ∼ N ( 0 , I ) , α t = ∏ t i =1 α i . Therefore, we can directly obtain ˆ X t from ˆ X 0 . The details for deriving the closed-form solution can be found in Appendix A.2.1. Note that when T is large enough, q ( ˆ X T | ˆ X 0 ) ≈ q ( ˆ X T ) , q ( ˆ X T ) ≈ N ( 0 , I ) .

Our diffusion reverse process gradually removes Gaussian noises added to the imputation target based on the time-domain observation condition X C , the high-frequency condition C H and the dominant-frequency condition C D , which can be formalized as the Markov chain,

We next show that introducing the high-frequency condition C H and dominant-frequency condition C D can reduce the uncertainty of the diffusion reverse process, improving the imputation accuracy.

Proposition 3.1. The conditional entropy

with additional high-frequency condition C H and dominant-frequency condition C D in the diffusion reverse process.

It can be proved by the chain rule of the conditional entropy, as detailed in Appendix A.1.

Based on the classifier-free guidance diffusion model [19; 20], p θ ( ˆ X t -1 | ˆ X t , X C , C H , C D ) can be

[ σ t -1 ] 2 = (1 -α t -1 ) β t 1 -α t , where ϵ θ ( · ) is the denoising network with learnable parameter set θ as present in Appendix A.3.1. The mathematical details are presented in Appendix A.2.2.

As shown in Figure 3, the denoising network incorporates frequency-domain information into modeling time dependencies and attribute dependencies, through the time-frequency representation learning module and attribute-frequency representation learning module to guide the denoising.

For training the denoising network, we randomly select some observed values as the imputation target ˆ X and use the remaining observations as the observation condition X C for each update step, since the ground truth of missing values is unknown. We train the denoising network by minimizing the following objective function L θ ,

where t ∼ Uniform { 1 , . . . , T } , ˆ X 0 ∼ q ( ˆ X 0 ) , ϵ ∼ N ( 0 , I ) .

For data imputation, we treat all missing values as the imputation target and all the observed values as the observation condition, i.e., ˆ X = X ⊙ (1 -M ) , X C = X . We start from ˆ X T ∼ N ( 0 , I ) and perform the T -step diffusion reverse process following Equation 14, to obtain final imputation values ˆ X 0 . Please see the detailed training and imputation algorithms in Appendix A.3.2.

4 Experiment

This section experimentally evaluates both the imputation effectiveness and the improvement of real downstream applications for our FGTI, against various competing methods. All experiments are performed on a machine with Intel Core 3.0GHz i9 CPU, NVIDIA GeForce RTX 3090 24GB GPU, and 64GB RAM. The source code and datasets are available online [1].

4.1 Experimental Setup

4.1.1 Datasets

We employ three real time series datasets with real-world missing values. KDD [6] collects 8,034 meteorological and air quality readings of nine stations from January 30, 2017 to January 31, 2018 in Beijing, with 4.46% real missing values. This dataset is collected every one hour and eleven sensor readings are recorded at each station. Guangzhou [10] records traffic speeds per ten minutes on 214 anonymous roads in Guangzhou from August 1, 2016 to September 30, 2016. There are 1.29% real missing values in the dataset. PhysioNet [42] contains 37 measurement readings from 11,988 patients within 48 hours of the ICU admission. 79.71% measurements are missing in the dataset, and 1,707 patients died after 48 hours of the ICU admission. Following existing studies [29; 30], since the ground truth is unavailable, we ignore these missing values when evaluating the imputation accuracy in comparative experiments and model analysis, but consider them in the application study.

4.1.2 Criteria

Following previous studies [33; 27], we employ RMSE [22] and MAE [7] to evaluate the imputation accuracy. For both, the smaller the value is, the more effective the imputation will be. For the air quality prediction application over the KDD dataset in Section 4.5, we also use RMSE as the metric. In addition, the AUC score [32] is used to measure the patient mortality forecasting application over the PhysioNet dataset. The large the value is, the better the forecasting result will be.

4.1.3 Baselines

We compare with fifteen widely adopted time series imputation methods, including statistics-based Mean [41], matrix factorization based BTMF [9] and TIDER [28], deep learning based BRITS [6], TST [57], SAITS [14], TimesNet [52], LaST [50] and FreTS [55], GNN-based GRIN [12], VAE-based TimeCIB [11]. GAN-based GAIN [56], Diffusion-based CSDI [44], SSSD [2], and PriSTI [27]. For methods such as GRIN and PriSTI that require an adjacency matrix as input to show relationships between attributes, we use the identity matrix by default, where every attribute has dependencies with others in the time series. Since LaST and FreTS only focus on the time series forecasting task, we adapt them to the imputation task based on the seting of TimesNet. For methods in which the authors recommend parameters such as SAITS, MIWAE, GPVAE, CSDI and PRiSTI, we use these parameters as suggested. The other methods are also configured in a best-effort fashion by iteratively choosing good parameters.

4.2 Comparative Experiments

We first explore the imputation performance of different methods over real datasets with different missing rates. The observed values with various missing rates are randomly removed under the missing completely at random (MCAR) mechanism [5] to form the imputation target. Each experiment is repeated five times with different generated missing values and random seeds, and the average result is reported in Table 1. Note that RMSE reflects the absolute difference between the imputation value

MCAR

MAR

MNAR

0.0

0.5

1.0

RMSE

(a)

MEAN

BTMF

TIDER

BRITS

TST

SAITS

TimesNet

LaST

FreTS

TimeCIB

GAIN

CSDI

SSSD

PriSTI

FGTI

MCAR

MAR

MNAR

0.0

0.5

MAE

(b)

and ground truth in the context of the data scale, and is not bounded by a specific range. The missing rates in the table are with respect to the observed values.

We can find that our method achieves the best imputation accuracy under various missing rates. When there is more missing data, deep learning based imputation models are less accurate due to the lack of observation condition information. Nevertheless, our approach uses frequency domain information and achieves superior results. Our FGTI model surpasses the state-of-the-art generative imputation models in various cases, thanks to the incorporation of high-frequency and dominant-frequency condition information.

Moreover, since missing data are usually associated with the environment in reality, we consider two additional typical missing data injection mechanisms following the same line of the existing study [33], i.e., missing at random (MAR) [53] and missing not at random (MNAR) [45]. Specifically, the probability of missing data in MAR is higher when the temperature reading is low in the KDD dataset. On the other hand, in the MNAR scenario, there is a higher probability of missing data during the periods when the reading of each feature is lower. Figure 4 and Figures 8-9 in Appendix A.4.1 show the corresponding imputation results. One can find that the imputation results under different missing mechanisms are relatively similar, and our FGTI achieves optimal performance consistently. This result demonstrates that FGTI can handle missing data in various missing scenarios.

4.3 Ablation Analysis

We explore the effect of different elements in our FGTI on imputation performance through the following four ablation scenarios. (1) w/o Cross-domain : No extra condition representation is provided for cross-domain representation learning frameworks, where the fusion processes degrade to the standard self-attention. This scenario is used to validate the role of cross-domain representation for imputation. (2) w/o Frequency condition : In this scenario, the frequency-domain information is absent, and only the observations in the time-domain are utilized as the condition, to investigate the impact of the frequency-domain information. (3) w/o Dominant-frequency filter : Remove the dominant-frequency filter from the structure to observe its impact on data imputation performance. (4) w/o High-frequency filter : This case exemplifies how crucial the high-frequency filter is, by removing it from the pipeline.

Based on the results presented in Table 2, it is evident that cross-domain representation learning frameworks and the two types of frequency-domain information are crucial in the process of imputation, which verifies the necessity of each component in our FGTI.

4.4 Resource Consumption

We present the resource consumption results of different methods in Figure 5. It can be observed that the running time of FGTI is roughly at the same level as other diffusion-based methods. The overall

0

10000

20000

MiB

(a)

MEAN

BTMF

TIDER

BRITS

TST

SAITS

TimesNet

LaST

FreTS

TimeCIB

GAIN

CSDI

SSSD

PriSTI

FGTI

Running Time

0

5000

10000

Second

Air quality prediction

0.5

0.6

0.7

RMSE

(a)

Incomplete

MEAN

BTMF

TIDER

BRITS

TST

SAITS

TimesNet

LaST

FreTS

TimeCIB

GAIN

CSDI

SSSD

PriSTI

FGTI

Mortality forecast

0.60

0.65

0.70

AUC

0.80

0.85

(b)

resource consumption of FGTI, implemented based on the diffusion model, is slightly higher than that of CSDI, due to the inclusion of high-frequency information and dominant-frequency information. However, since FGTI can achieve better imputation results than other methods, as shown in Table 1, we argue that it is acceptable to incur such an extra resource consumption.

4.5 Application Study

To validate the effectiveness of applying imputation in real-world downstream applications, we consider air quality prediction and mortality forecasting tasks. For the air quality prediction application, we first impute real-world missing data in the KDD dataset by various imputation methods. Then, we analyze the records in the previous twelve hours and use the AdaBoost regressor [37] to estimate the average PM2.5 concentration for the upcoming six hours. Figure 6(a) shows that our method achieves the highest improvement in the air quality prediction task. Figure 6(b) reports the mortality forecast performance over the PhysioNet dataset. We train the MLP classifier [37] to forecast the mortality on the data without/with imputation. As shown, FGTI achieves the best performance again, which verifies the applicability of our work. Notably, various imputation methods provide a noteworthy and favorable impact on the forecast task, which demonstrates the necessity of imputation.

5 Conclusion

In this paper, we study imputing incomplete multivariate time series data, through reducing the imputation error in the residual term. By effectively incorporating frequency-domain insights into the generative framework, our FGTI surpasses existing models by capturing high-frequency information and dominant-frequency information. The introduced cross-domain representation learning frameworks further enhance its capability to handle time and attribute dependencies. Comprehensive experimental evaluations over real-world incomplete datasets demonstrate the superiority of FGTI in both the imputation accuracy and the improvement of downstream applications.

References

https://github.com/FGTI2024/FGTI24 .

Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. TMLR , 2023.

Mehran Amiri and Richard Jensen. Missing data imputation using fuzzy-rough methods. Neurocomputing , 205:152-164, 2016.

Kasun Bandara, Rob J Hyndman, and Christoph Bergmeir. Mstl: A seasonal-trend decomposition algorithm for time series with multiple seasonal patterns. abs/2107.13462.

Philip Bohannon, Michael Flaster, Wenfei Fan, and Rajeev Rastogi. A cost-based model and effective heuristic for repairing constraints by value modification. In SIGMOD , 2005.

Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. BRITS: bidirectional recurrent imputation for time series. In NeurIPS , 2018.

Tianfeng Chai and Roland R Draxler. Root mean square error (rmse) or mean absolute error (mae)?-arguments against avoiding rmse in the literature. Geoscientific model development , 7 (3):1247-1250, 2014.

Zhengping Che, Sanjay Purushotham, Max Guangyu Li, Bo Jiang, and Yan Liu. Hierarchical deep generative models for multi-rate multivariate time series. In ICML , 2018.

Xinyu Chen and Lijun Sun. Bayesian temporal factorization for multidimensional time series prediction. IEEE TPAMI , 44(9):4659-4673, 2022.

Xinyu Chen, Yixian Chen, and Zhaocheng He. Urban traffic speed dataset of guangzhou, china, 2018. URL https://doi.org/10.5281/zenodo.1205229 .

MinGyu Choi and Changhee Lee. Conditional information bottleneck approach for time series imputation. In ICLR , 2024.

Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the g_ap_s: Multivariate time series imputation by graph neural networks. In ICLR , 2022.

Robert B Cleveland, William S Cleveland, Jean E McRae, and Irma Terpenning. Stl: A seasonal-trend decomposition. J. Off. Stat , 6(1):3-73, 1990.

Wenjie Du, David Côté, and Yan Liu. SAITS: self-attention-based imputation for time series. ESWA , 219:119619, 2023.

Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. GP-VAE: deep probabilistic time series imputation. In AISTATS , 2020.

David S Fung. Methods for the estimation of missing values in time series. 2006.

Md Kamrul Hasan, Md Ashraful Alam, Shidhartho Roy, Aishwariya Dutta, Md Tasnim Jawad, and Sunanda Das. Missing value imputation affects the performance of machine learning: A review and analysis of the literature (2010-2021). Informatics in Medicine Unlocked , 27: 100799, 2021.

Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross-attention control. In ICLR , 2023.

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR , abs/2207.12598.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS , 2020.

Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and João Carreira. Perceiver: General perception with iterative attention. In ICML , 2021.

Shawn R. Jeffery, Minos N. Garofalakis, and Michael J. Franklin. Adaptive cleaning for RFID data streams. In VLDB , 2006.

Mehmed Kantardzic. Data mining: concepts, models, methods, and algorithms . John Wiley & Sons, 2011.

Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In ICLR , 2021.

Roderick JA Little and Donald B Rubin. Statistical analysis with missing data , volume 793. John Wiley & Sons, 2019.

Bing Liu and Huanhuan Cheng. Financial time series classification method based on lowfrequency approximate representation. Engineering Reports , page e12739, 2023.

Mingzhe Liu, Han Huang, Hao Feng, Leilei Sun, Bowen Du, and Yanjie Fu. Pristi: A conditional diffusion framework for spatiotemporal imputation. In ICDE , 2023.

Shuai Liu, Xiucheng Li, Gao Cong, Yile Chen, and Yue Jiang. Multivariate time-series imputation with disentangled temporal representations. In ICLR , 2023.

Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, and Xiaojie Yuan. Multivariate time series imputation with generative adversarial networks. In NeurIPS , 2018.

Yonghong Luo, Ying Zhang, Xiangrui Cai, and Xiaojie Yuan. E 2 gan: End-to-end generative adversarial network for multivariate time series imputation. In IJCAI , 2019.

Pierre-Alexandre Mattei and Jes Frellsen. MIWAE: deep generative modelling and imputation of incomplete data sets. In ICML , 2019.

Donna Katzman McClish. Analyzing a portion of the roc curve. Medical decision making , 9(3): 190-195, 1989.

Xiaoye Miao, Yangyang Wu, Lu Chen, Yunjun Gao, Jun Wang, and Jianwei Yin. Efficient and effective data imputation with influence functions. In VLDB , 2021.

Xiaoye Miao, Yangyang Wu, Jun Wang, Yunjun Gao, Xudong Mao, and Jianwei Yin. Generative semi-supervised learning for multivariate time series imputation. In AAAI , 2021.

Ahmad Wisnu Mulyadi, Eunji Jun, and Heung-Il Suk. Uncertainty-aware variational-recurrent imputation network for clinical time series. IEEE TCYB , 52(9):9684-9694, 2022.

Alfredo Nazábal, Pablo M. Olmos, Zoubin Ghahramani, and Isabel Valera. Handling incomplete heterogeneous data using vaes. PR , 107:107501, 2020.

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay. Scikit-learn: Machine learning in python. JMLR , 12:2825-2830, 2011.

Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron C. Courville. On the spectral bias of neural networks. In ICML , 2019.

Xiaobin Ren, Kaiqi Zhao, Patricia J. Riddle, Katerina Taskova, Qingyi Pan, and Lianyan Li. DAMR: dynamic adjacency matrix representation learning for multivariate time series imputation. In SIGMOD , 2023.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR , 2022.

Alex Rubinsteyn and Sergey Feldman. fancyimpute: An imputation library for python, 2016. URL https://github.com/iskandr/fancyimpute .

Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predicting inhospital mortality of icu patients: The physionet/computing in cardiology challenge 2012. In 2012 Computing in Cardiology , 2012.

Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In NeurIPS , 2020.

Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. CSDI: conditional score-based diffusion models for probabilistic time series imputation. In NeurIPS , 2021.

Bhekisipho Twala. An empirical comparison of techniques for handling incomplete data using decision trees. AAI , 23(5):373-405, 2009.

Joram van Driel, Christian NL Olivers, and Johannes J Fahrenfort. High-pass filtering artifacts in multivariate classification of neural time series data. Journal of Neuroscience Methods , 352: 109080, 2021.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS , 2017.

Jun Wang, Wenjie Du, Wei Cao, Keli Zhang, Wenjia Wang, Yuxuan Liang, and Qingsong Wen. Deep learning for multivariate time series imputation: A survey. abs/2402.04059.

Xu Wang, Hongbo Zhang, Pengkun Wang, Yudong Zhang, Binwu Wang, Zhengyang Zhou, and Yang Wang. An observed value consistent diffusion model for imputing missing values in multivariate time series. In KDD , 2023.

Zhiyuan Wang, Xovee Xu, Weifeng Zhang, Goce Trajcevski, Ting Zhong, and Fan Zhou. Learning latent seasonal-trend representations for time series forecasting. In NeurIPS , 2022.

Rebecca E. Wilson, Idris A. Eckley, Matthew A. Nunes, and Timothy Park. A wavelet-based approach for imputation in nonstationary multivariate time series. Stat. Comput. , 2021.

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR , 2023.

Jing Xia, Shengyu Zhang, Guolong Cai, Li Li, Qing Pan, Jing Yan, and Gangmin Ning. Adjusted weight voting algorithm for random forests in handling missing values. PR , 69:52-60, 2017.

Qianxiong Xu, Sijie Ruan, Cheng Long, Liang Yu, and Chen Zhang. Traffic speed imputation with attentions and cycle-perceptual training. In CIKM , 2022.

Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mlps are more effective learners in time series forecasting. In NeurIPS , 2023.

Jinsung Yoon, James Jordon, and Mihaela van der Schaar. GAIN: missing data imputation using generative adversarial nets. In ICML , 2018.

George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. In KDD , 2021.

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML , 2022.

Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In NeurIPS , 2023.

Wei Zhuang, Jili Fan, Jiayu Fang, Wenxuan Fang, and Min Xia. Rethinking general time series analysis from a frequency domain perspective. KBS , 2024.

A Appendix

A.1 Proof of Proposition 3.1

We use the conditional entropy in information theory to reflect the amount of uncertainty. For the reverse process in [20], the imputation target ˆ X t is specified as the condition, thus we use

to model the uncertainty of the reverse process of DDPM.

Similarly, we use H ( ˆ X t -1 | ˆ X t , X C ) to model the uncertainty of the reverse process of the CSDI [44] model, which only uses the observations as the condition. For our FGTI, we utilize H ( ˆ X t -1 | ˆ X t , X C , C H , C D ) .

According to the property of the conditional entropy, we first have

Using the definition of mutual information, we have

From Equation 12, we know that I ( ˆ X t -1 ; ˆ X t ) > 0 , we thus have

According to the chain rule of the entropy, we have

Then we can get

According to the Equation 12, ˆ X t -1 adds the noise one less time than ˆ X t , which indicating that ˆ X t -1 is closer to the observations. Thus, we have

By substituting this, we can obtain

Following the same line, we can also derive that

This result implies that adding C H and C D to the condition can simplify the distribution that the diffusion models need to learn by reducing the entropy. This simplification can lead to more efficient learning and improve the model's imputation performance by narrowing down the scope of the target distribution's randomness and making its outcomes more predictable.

A.2 Mathematical details of FGTI

A.2.1 Details of Equation 13

If the complete imputation target distribution q ( ˆ X 0 ) is known, we can first get a sampled complete imputation target ˆ X 0 ∼ q ( ˆ X 0 ) . Following Equation 12, we can obtain ˆ X t by

Similarly, we can also obtain ˆ X t -1 by

Combining the above two equations, we get

As ϵ t -1 ∼ N ( 0 , I ) , ϵ t ∼ N ( 0 , I ) , we can infer that √ α t (1 -α t -1 ) ϵ t -1 ∼ N ( 0 , [ α t (1 -α t -1 )] I ) , √ 1 -α t ϵ t ∼ N ( 0 , (1 -α t ) I ) . Therefore, we can get

where ϵ ∼ N ( 0 , I ) .

A.2.2 Details of the Parameterization

Combining the Bayes' theorem, we start by

According to Equation 12, the expected p θ ( ˆ X t | ˆ X t -1 , X C , C H , C D ) is

Furthermore, by incorporating Equation 13, we can obtain

where ˆ X 0 θ is a virtual result that is expected to be obtained through Equation 14. Combining Equation 12, we can parameterize it by ˆ X t = √ α t ˆ X 0 θ + √ 1 -α t ϵ θ ( t, ˆ X t , X C , C H , C D ) , where ϵ θ ( · ) outputs the predicted added noise to ˆ X t -1 based on the parameter set θ . In other words, ˆ X 0 θ = 1 √ α t [ ˆ X t -√ 1 -α t ϵ θ ( t, ˆ X t , X C , C H , C D )] .

By merging the above three distributions, we can derive

where ( ˆ X t -1 ) 2 denotes the inner product of ˆ X t -1 .

On the other hand, from the probability density function of the Gaussian distribution, we can also obtain

Therefore, we can get

We can first obtain the following result from the first equation as α t = 1 -β t

After that, we take it into the second equation

As the virtual result ˆ X 0 θ = 1 √ α t [ ˆ X t -√ 1 -α t ϵ θ ( t, ˆ X t , X C , C H , C D )] , we have

Concat

MLP

Projector

𝐕

!

Cross Attention

Layer

Embedding

𝐐

!

𝐊

!

𝐕

"

Cross Attention

Layer

𝐐

"

𝐊

"

Noise imputation

target

𝐗 %

#

Observation

condition

𝐗

𝐂

𝐑

𝒕

𝐑

𝒂

MoE

Module

𝐂

𝐅

Position

Encoding

High-frequency

condition

𝐂

𝐇

Dominant-frequency

condition

𝐂

𝐃

Concat

Transformer

Timestamp

length

Attribute

Number

Embedding

Embedding

Concat

MLP

Side Infor

MLP

MLP

MLP

ReLU

MLP

ϵ

*

Diffusion Step

𝑡~Uniform 1, … , 𝑇

Denoising Network

A.3 Implementation Details

A.3.1 Detailed Architecture of the Denoising Network

In this section, we present the detailed architecture of the denoising network ϵ θ ( · ) in FGTI model. As shown in Figure 7, the input projector of the noise imputation target and the observation condition is an MLP layer, and the output projector is a 2-layer MLP with ReLU activation function. For the encoder of the frequency-domain information, it is implemented by a transformer backbone consisting of the position encoding layer and the transformer encoder layer.

A.3.2 Algorithms

Algorithm 1 Training process of FGTI implemented by the diffusion model

Input:

Incomplete time series X , the number of diffusion step T

Output:

Optimized denoising network ϵ θ ( · )

1: repeat

2: ˆ X 0 ← select observed values in X

3: t ∼ Uniform { 1 , . . . , T }

4: ϵ ∼ N ( 0 , I )

5: ˆ X t ← √ α t ˆ X 0 + √ 1 -α t ϵ

6: Perform Gradient Descent by ∇L θ = ∇ θ ∥ ∥ ∥ ϵ -ϵ θ ( t, ˆ X t , X C , C H , C D )∥ ∥ ∥ 2

7: until converged

In this section, we provide the detailed training process of our proposed FGTI model implemented by the diffusion model in Algorithm 1, and the imputation process in Algorithm 2.

A.4 Supplemental Experiments

A.4.1 Missing Mechanisms

In this section, we explore the imputation performance of the missing at random (MAR) [53] and missing not at random (MNAR) [45] missing mechanisms over the Guangzhou dataset and the PhysioNet dataset.

Algorithm 2 Imputation process of FGTI implemented by the diffusion model

Input: A incomplete time series sample X , the number of diffusion step T , the optimized denoising network ϵ θ ( · )

Output: Filled missing values ˆ X 0

As shown in Figure 8 and Figure 9, FGTI consistently achieves optimal performance, demonstrating its ability to handle missing data in various scenarios.

MCAR

MAR

MNAR

0.0

0.5

RMSE

(a)

MEAN

BTMF

TIDER

BRITS

TST

SAITS

TimesNet

LaST

FreTS

TimeCIB

GAIN

CSDI

SSSD

PriSTI

FGTI

MCAR

MAR

MNAR

0.00

0.25

0.50

MAE

(b)

MCAR

MAR

MNAR

0.0

0.5

1.0

RMSE

(a)

MEAN

BTMF

TIDER

BRITS

TST

SAITS

TimesNet

LaST

FreTS

TimeCIB

GAIN

CSDI

SSSD

PriSTI

FGTI

MCAR

MAR

MNAR

0.0

0.5

MAE

(b)

A.4.2 Hyperparameter Evaluation

In this section, we perform parameter sensitivity experiments on two critical hyperparameters: the cutoff frequency of the high-frequency filter and the maximum magnitude frequency number of the dominant-frequency filter.

Effect of the cutoff frequency F . Figure 10 shows the imputation results with various cutoff frequencies F of the high-frequency filter. It can be found that if F is too small, the model may not be able to accurately capture the high-frequency information necessary for guiding the imputation of the time series residual term. The reason is that the high-frequency filter output may include too much low-frequency information. Conversely, if F is too large, the model cannot obtain enough high-frequency information to guide the imputation of the residual term.

Effect of the maximum magnitude frequency number κ . We investigate the imputation results when adjusting the maximum amplitude frequency number κ used for the dominant-frequency filter in Figure 11. As shown in the figure, the imputation model cannot perform well with a small κ . This is because the dominant-frequency filter cannot obtain enough smoothing information, which causes

0

0.1

0.2

0.3

0.4

0.23

0.24

0.40

0.42

0.6

0.7

(a)

0

0.1

0.2

0.3

0.4

0.15

0.16

MAE

0.39

0.28

(b)

KDD

Guangzhou

PhysioNet

RMSE

1

5

10

15

20

0.23

0.24

0.40

0.42

0.6

0.7

(a)

1

5

10

15

20

0.15

0.16

MAE

0.35

0.25

(b)

KDD

Guangzhou

PhysioNet

κ

RMSE

κ

high-frequency signals to interfere with the imputation of the trend and seasonal terms of the time series. On the contrary, if κ is too large, it can cause some high-frequency information to mix with the output condition of the dominant-frequency filter. This can prevent the model from effectively obtaining background structure information needed for imputing the trend and seasonal terms. As a result, the imputation result for extreme cases may not be optimal.

Based on the experimental results, we set the cutoff frequency F of the high-frequency filter to 0.3 and set the number of maximum magnitude frequency κ of the dominant-frequency filter to 10. In addition, for other settings related to the diffusion model, we adopt hyperparameters recommended by the existing well-established models [44; 24]. These models have demonstrated strong performance in similar tasks, and their hyperparameters have been extensively validated in the paper.

A.4.3 Imputation Target Select Strategies

For training the denoising network, we randomly select some observed values as the imputation target. In this process, the mask ratio and mask pattern to get the imputation target directly determine the effectiveness of training. Thus, in this section, we explore the performance of FGTI with different mask ratios and mask patterns.

Effect of Mask Ratio We first mask different ratios of observations as the imputation target for training, the performance of FGTI with different mask ratios is shown in Table 3. In addition, we consider a special case where observations with different ratios are randomly masked as imputation targets at each training step, instead of using a fixed masking ratio.

We can find that since the random ratio mask strategy can increase the learning complexity and enhance the modeling ability of the diffusion model, the random ratio mask strategy achieves optimal or sub-optimal performance in most cases. So we use the random ratio mask strategy by default.

Effect of Mask Pattern Then we explore the performance when using different mask patterns. Following CSDI [44] and PriSTI [27], we consider three mask pattern strategies: (1) Block missing (2) Mix missing (3) Random missing. The results is shown in Table 4

It can be found that Block missing or Mix missing strategy is not comparable to Random missing in most cases due to the possibility that the mask pattern may not correspond to the actual missing scenario. So we use the Random missing mask pattern by default.

A.5 Comparative Experiments of Generative Baselines

To compare the imputation performance of FGTI with probabilistic generative baselines in more detail, we adopt CRPS [44] to evaluate the gap between the learned and ground truth distributions for different probabilistic generative methods following [44; 27].

First, we inject different rates of missing values by the MCAR mechanism, and report the CRPS performance with different missing rates in Table 5. Then we report the CRPS by varying the missing mechanism with 10% missing values in Table 6.

We can find that our method outperforms other probabilistic generative methods for all cases due to the introduction of frequency-domain conditions, thus providing empirical evidence for Proposition 3.1. It can be also found that the variations of CRPS are basically the same as RMSE and MAE for a specific model with different settings.

A.5.1 Case Study

In order to verify the role of the high-frequency condition and the dominant-frequency condition, in this section we conduct a case study of FGTI for trend, seasonal, residual term over the predecomposed KDD dataset.

We first perform STL decomposition of the KDD dataset into Trend, Seasonal and Residual terms. Then we select 10% observations of the original KDD dataset as the mask positions by MCAR, and then mask the corresponding positions of the three terms. Finally we imputation the missing values in the three terms separately and report the performances. Note that this setup is the same as the survey experiment shown in Figure 1 in Section 1.

To study the role of high-frequency information, dominant-frequency information, and frequencydomain information, we consider the three ablation scenarios (1)w/o Dominant-frequency filter (2) w/o High-frequency filter and (3) w/o Frequency condition in Section 4.3.

We report the imputation results of different scenarios over different terms in Table 7

We can find that for the trend term, retaining the dominant-frequency condition gives the best results, while the high-frequency condition may interfere with the imputation. For the seasonal term, the results are similar to the trend term, but the dominant-frequency information contributes less to the imputation for the seasonal term than for the trend term. This suggests that the seasonl term mainly corresponds to the dominant-frequency information, but also contains some of the high-frequency information. In contrast, the results of the experiments on the residual term show that the residual term mainly corresponds to high-frequency condition. Since we choose the transformer as the encoder in Cross-domain Representation Learning and utilize cross-attention as the fusion mechanism of the two frequency-domain conditions in time-frequency representation learning and attribute-frequency representation learning modules, our method can self-adaptively adjust the weights of the highfrequency information and the dominant-frequency information for different timestamps. Thus our method can outperform existing methods for datasets with multiple circumstances in most cases, as illustrated in Table 1.

A.5.2 Visualizations

To showcase the imputation results of our FGTI model, we visualize the results with the state-of-theart imputation methods CSDI and PriSTI in Figure 12 and Figure 13. We can find that the CSDI imputation results are not quite accurate for some fast-changing points due to the lack of sufficient condition guidance. On the other hand, both FGTI and PriSTI produced more accurate imputation results because they both used additional conditions. However, as shown in Tabel 1, FGTI still yields better results than PriSTI, suggesting that the high-frequency information and the dominant-frequency information we use are more superior to the interpolation information used by PriSTI.

A.6 Societal Impact Statement

The development our FGTI imputation model could have a significant positive impact on various sectors including healthcare, finance, and environmental monitoring. In healthcare, improved imputation models can lead to more accurate health monitoring systems, enabling early detection and treatment

150

160

170

180

190

Timestamp

-1.5

-1.0

-0.5

0.0

Value

Attribute 6

observation

ground truth values

CSDI

FGTI

150

160

170

180

190

Timestamp

-2

-1

0

1

2

Value

Attribute 66

150

160

170

180

190

Timestamp

-1

0

1

Value

Attribute 96

1590

1600

1610

1620

1630

Timestamp

-0.5

0.0

0.5

1.0

Value

Attribute 6

1590

1600

1610

1620

1630

Timestamp

-0.5

0.0

0.5

Value

Attribute 66

1590

1600

1610

1620

1630

Timestamp

-1

0

1

Value

Attribute 96

3170

3180

3190

3200

3210

Timestamp

0.0

0.5

1.0

1.5

2.0

Value

Attribute 6

3170

3180

3190

3200

3210

Timestamp

0.0

0.5

1.0

Value

Attribute 66

3170

3180

3190

3200

3210

Timestamp

-0.5

0.0

0.5

1.0

Value

Attribute 96

4760

4770

4780

4790

4800

Timestamp

-0.5

0.0

0.5

1.0

Value

Attribute 6

4760

4770

4780

4790

4800

Timestamp

-1

0

1

2

Value

Attribute 66

4760

4770

4780

4790

4800

Timestamp

-1

0

1

Value

Attribute 96

of conditions by filling gaps in patient data. This can ultimately improve patient outcomes and reduce healthcare costs. In the financial sector, enhanced time series imputation can provide better forecasts and risk assessments, aiding in more informed decision-making and potentially stabilizing markets by decreasing uncertainty. Environmentally, better data imputation can improve weather prediction models and climate monitoring systems, aiding in disaster readiness and enhancing our ability to address climate change.

However, the deployment of advanced imputation models also raises certain concerns. If used in sensitive areas like surveillance, these models could lead to privacy invasions by reconstructing

150

160

170

180

190

Timestamp

-1.4

-1.2

-1.0

-0.8

Value

Attribute 6

observation

ground truth values

PriSTI

FGTI

150

160

170

180

190

Timestamp

-1

0

1

2

Value

Attribute 66

150

160

170

180

190

Timestamp

-1

0

1

Value

Attribute 96

1590

1600

1610

1620

1630

Timestamp

-0.5

0.0

0.5

1.0

Value

Attribute 6

1590

1600

1610

1620

1630

Timestamp

-0.5

0.0

0.5

Value

Attribute 66

1590

1600

1610

1620

1630

Timestamp

-1

0

1

Value

Attribute 96

3170

3180

3190

3200

3210

Timestamp

1.00

1.25

1.50

1.75

Value

Attribute 6

3170

3180

3190

3200

3210

Timestamp

0.0

0.5

1.0

Value

Attribute 66

3170

3180

3190

3200

3210

Timestamp

0.0

0.5

1.0

Value

Attribute 96

4760

4770

4780

4790

4800

Timestamp

0.0

0.5

1.0

Value

Attribute 6

4760

4770

4780

4790

4800

Timestamp

-1

0

1

2

Value

Attribute 66

4760

4770

4780

4790

4800

Timestamp

-1

0

1

Value

Attribute 96

missing or incomplete data to track individuals without consent. In financial markets, sophisticated imputation methods could also exacerbate inequality by disproportionately benefiting institutions with the resources to leverage state-of-the-art technology, potentially leading to greater market dominance. Additionally, reliance on automated data imputation may result in complacency, where errors in imputation models propagate unnoticed, leading to decisions based on inaccurate or misleading data.

To mitigate negative impacts of our imputation model, implement stringent data privacy laws, and ethical guidelines, provide equal access to technology resources across entities, and establish rigorous validation processes to ensure accuracy and fairness. Regular auditing and transparency in algorithm deployment can also play a critical role.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of this work in terms of resource consumption in Section 4.4.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We include the theoretical proof in Appendix A.1.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We include the detailed experimental settings in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The anonymous source code and datasets are available online [1].

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We include the detailed experimental settings in Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report the results averaged from five experiments with different random seeds.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We report the necessary computer resources.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: In every respect in the paper, we follow the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We include the impact statement in Appendix A.6.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All data, models, and code in the paper respect the license.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper proposes FGTI that uses frequency-domain information with high-frequency and dominant-frequency filters for accurate imputation of residual, trend, and seasonal components. 
Experimental results demonstrate FGTI's effectiveness in improving both data imputation accuracy and downstream applications.

STRENGTHS:
1. The authors leverage high-frequency and dominant-frequency features to enhance time series imputation.

2. They introduce time-domain and frequency-domain representation learning modules to comprehensively capture relevant information.

3. Extensive experiments are conducted to demonstrate the method's effectiveness.

WEAKNESSES:
1. The formulation and writing of this paper should be improved, with attention to correcting several typos.

2. The authors claim that the residual of a time series mainly comprises high-frequency components. However, in many time series, there may be multiple scales of seasonality and trend features, potentially resulting in a final residual that is too noisy to contain useful information.

3. It appears that the proposed method utilizes frequency features of time series as conditions for the diffusion model, which seems straightforward and may limit its contribution.

QUESTIONS:
1. In Table 2, the absence of the High-frequency filter appears to impact performance less on KDD and Guangzhou datasets. However, on PhysioNet, the situation is reversed. What factors might explain this discrepancy?

2. How were the hyper-parameters for the experiments chosen? Could you elaborate on the methodology used for their selection?

3. What is the purpose of Proposition 3.1? It seems unnecessary and almost self-evident.

LIMITATIONS:
None
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a generative model, Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), for multivariate time series imputation. The proposed model is designed to enhance the imputation performance by modeling the residual component of time series data. To this end, the paper leverages the observation that the residual components are usually high-frequent. It utilizes two filters in the frequency domain to extract high-frequency and dominant-frequency information from the time series using FFT and IFFT. The frequency information is then input into a denoising network to generate the missing data conditioned on the observed part of the time series. Experiments are performed using three real-world datasets and a number of recent baselines are benchmarked.\n\nSTRENGTHS:\n- The idea of using frequency filters to model the residual component of time-series data is natural and reasonable.\n- The empirical evaluation is systematic and the experiment results show consistent improvement compared with existing methods.\n\nWEAKNESSES:\n- It is not clear if the modeling of temporal dependency is sufficient in the time-frequency and attribute-frequency representation learning module. From Eq. (8-9), it seems that only two attention layers are used without considering the time stamp.\n- The model architecture and specification used for Projectors in the denoising network is not described.\n- There is a gap between Sections 3.2 and 3.3. The time-frequency representation and attribute-frequency representation learning are described and the final output $\\mathbf{R}^a$ is defined. However, it is not clear how this representation is used in the diffusion model. It seems from Eq. (14) and Appendix A.2.1 that $\\mathbf{R}^a$ is not involved in the parameterization $\\epsilon_\\theta$.\n\nQUESTIONS:\nPlease kindly see and clarify my comments in the Weaknesses section.\n\nLIMITATIONS:\nLimitations of high demand of computational resources are discusses with relevant experiment results analyzed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new model, called FGTI, to address the issue of missing data in multivariate time series extracting frequency-domain information. The authors argue that existing methods, in general, neglect the residual term, which is the most significant contributor to imputation errors. FGTI incorporates frequency-domain information using high-frequency and dominant-frequency filters to improve the imputation of the residual, trend, and seasonal terms. The model is evaluated on three real-world datasets and demonstrates superior performance in both imputation accuracy and downstream tasks compared to existing methods.\n\nSTRENGTHS:\n- The paper is well-written and well-motivated.\n- The authors tackle a very important problem of time-series imputation, focusing on fully utilizing frequency-domain information, which has not been considered enough.\n- The experiments are thorough, including ablation studies and different missing scenarios. The method is compared with cutting-edge baselines.\n\nWEAKNESSES:\n- The experiments are thorough and include multiple state-of-the-art imputation methods. However, the paper misses two important time-series imputation methods: one with a VAE-based approach [A] and one applying trend-seasonality decomposition [B]. While it's understandable that the authors might not have had enough time to include these papers since they are from ICLR 2024, technical comparisons with these methods would be necessary before acceptance.\n\n- While the authors provided experiments with various missing scenarios and missing rates, it is not clear why the proposed method outperforms the conventional methods and under what circumstances. For instance, extracting high-frequency information could be significantly distorted if missingness occurs with a certain frequency (which seems realistic considering periodic noises). The paper would be more convincing if the authors provided experiments that assess the effect of the trend and seasonality components of the datasets (or using synthetic datasets) and missing patterns.\n\n- The experiments would be more appealing if the authors provided a breakdown (seasonality and trend) of the assessments on the imputed time-series.\n\nMinor comment:\n\n- Regarding Figure 1, since the RMSE and MAE comparison essentially conveys similar information about the population-level importance of the trend-seasonality decomposition, replacing one with specific time-series examples where the trend and seasonality of the missing values are correctly imputed would better illustrate the motivation. Also, information about how Figure 1 is reported is missing.\n\nReferences:\n\n[A] Choi and Lee, \"Conditional Information Bottleneck Approach for Time Series Imputation, ICLR 2024.\n[B] Liu et al., \"Multivariate Time-series Imputation with Disentangled Temporal Representations,\" ICLR 2024.\n\nQUESTIONS:\n- Regarding Weakness 2: Please describe why the proposed method works better than conventional methods and under what circumstances. Do the seasonality and trend profiles of the given time-series dataset matter, and if so, how?\n\n- It seems that how to mask the input (to learn the underlying diffusion model) is very important. How does the model's performance change depending on the masking ratio and/or masking patterns? This is crucial as masking may distort the trend and seasonality components of the given time-series.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), a model which addresses the challenge of missing data in multivariate time series by focusing on the often-overlooked residual term. The paper also incorporates frequency-domain information to enhance imputation performance. Experiments show that this outperforms many existing time series imputation baselines on three real-world datasets.\n\nSTRENGTHS:\nS1. The work proposes an approach for time series imputation, which is an important topic for a wide range of applications.\n\nS2. The authors aim to address the influence of frequency-domain information, w.r.t, both high-frequency condition and dominant-domain condition.\n\nS3. Extensive experiments have been carried out.\n\nWEAKNESSES:\nW1. In Introduction and Section 3.1.2, it shows less explanation on why frequency components with large amplitudes could guide both the imputation of trend and seasonal terms. Did it address the intricately entangled trend-seasonal representations, which is highly important in current approaches?\n\nW2. In the section of Related Work, I did not find discussions regarding existing research work and this study in frequency domain for time series tasks, this would be hard to clearly justify the main difference/novelty of this paper compared with current approaches.\n\nW3. In the evaluation, only RMSE and MAE are used as metrics. However, it would be better to include additional metrics such as CRPS.\n\nW4. It would be fairer if the authors compared their approach with a time series model using frequency domain information, which also addresses the trend, seasonal, and residual terms.\n\nW5. Both CSDI and PriSTI address the issue of missing strategies. From the results of Figure 4, 7 and 8, the improvement of MAR and MNAR was much lower than that of MCAR on the Guangzhou and Physio datasets. Was it due to differences in the data distribution, or was it due to the setting of the hyper-parameters?\n\n[1] Learning Latent Seasonal-Trend Representations for Time Series Forecasting.\n[2] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting\n[3] CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation\n[4] PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation\n\nQUESTIONS:\nPlease see Weaknesses.\n\nLIMITATIONS:\n1.\tMy main concern with the work is that the contribution of the proposed method might be incremental. Compared with CSDI, frequency domain filters are utilized to incorporate frequency domain information, and further cross-domain representation learning and frequency-aware diffusion framework are natural implementations, which lack additional novelty.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-21"}