Frequency-aware Generative Models for Multivariate Time Series Imputation

Authors: xinyu  yang
yu  sun
xiaojie  yuan
xinyang  chen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 11:15:09

Abstract: Missing data in multivariate time series are common issues that can affect the analysis and downstream applications.
Although multivariate time series data generally consist of the trend, seasonal and residual terms, existing works mainly focus on optimizing the modeling for the first two items. However, we find that the residual term is more crucial for getting accurate fillings, since it is more related to the diverse changes of data and the biggest component of imputation errors.
Therefore, in this study, we introduce frequency-domain information and design Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI). Specifically, FGTI employs a high-frequency filter to boost the residual term imputation, supplemented by a dominant-frequency filter for the trend and seasonal imputation. Cross-domain representation learning module then fuses frequency-domain insights with deep representations.
Experiments over various datasets with real-world missing values show that FGTI achieves superiority in both data imputation and downstream applications.

Main paper: 
 introduction: 
 Missing data are commonly observed in the multivariate time series due to diverse reasons [25] , which would encumber subsequent analysis and applications [17] . It is not surprising that more accurate missing data imputation generally leads to better performance in downstream applications. 1 Existing techniques [13; 28] have revealed that time series data can be decomposed into three distinct terms, i.e., trend, seasonal, and residual, and try to compute an imputation by modeling the first two items as accurately as possible. Figure 1 reports a survey for the imputation accuracy of the three terms by representative imputation methods over the pre-decomposed KDD [6] dataset with 10% missing values. 2 The dataset comprises 8,034 consecutive readings of meteorological and air quality data taken over a year in Beijing. In this dataset, the trend term may reflect long-term changes in climate or air quality conditions, and the seasonal term might capture patterns associated with different seasons. Moreover, the residual term may consist of short-term, irregular, and high-frequency changes. As shown in Figure 1 , the imputation error is mainly caused by the residual term, which has not been well studied, unfortunately.Recent studies indicate that the high-frequency components are intricately related to the residual [46; 58; 26] and contain critical information for imputing the residual term. Unfortunately, deep learning architectures cannot generalize well in modeling high-frequency components. [38; 43] . To meet this challenge, we design Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), which can extract frequency-domain information and use two cross-domain (i.e., time-domain and frequency-domain) representation learning modules to guide the generation process of deep models. Specifically, we start with the high-frequency filter designed to extract the high-frequency information essential for guiding the accurate imputation of the residual term. This choice is consistent with recognizing the critical role of high-frequency information in imputation accuracy. Then, we introduce the dominant-frequency filter to address the potential challenge posed by high-frequency information for imputing trend and seasonal terms [4] . Furthermore, our cross-domain representation learning frameworks combine frequency-domain information with deep representations in the time-domain, enabling seamlessly intertwining frequency-domain information with time and attribute dependencies modeling.Our research makes several notable contributions:• We design a frequency-aware generative model FGTI with frequency-domain information integrated by the high-frequency filter and the dominant-frequency filter, to enhance the awareness of the frequency-domain.• We introduce two cross-domain representation learning modules that provide models with prior knowledge of intricate frequency-related patterns for missing data imputation.• We evaluate FGTI on three time series datasets with real-world missing values, which demonstrates the superiority of FGTI in both imputation accuracy and downstream applications.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract and introduction accurately reflect the paper's contributions and scope.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 Traditional imputation methods usually employ the statistics, such as mean value [23] , median value [16] , or last observed value [3] , to impute missing data for multivariate time series. It is not surprising that such traditional signals cannot make full use of the valuable semantics of available data. BTMF [9] and TIDER [28] employ the low-rank matrix factorization to impute missing data. Unfortunately, due to the matrix capacity's limitation, it is still challenging to accurately match imputation values with the underlying complex relationships and dependencies.Many studies have shown that deep learning based imputation methods are effective to fill multivariate time series data, such as BRITS [6] , TST [57] , SAITS [14] , STCPA [54] , TimesNet [52] . According to [48] , forecasting models [50; 55] can also be applied to imputation task. Additionally, GRIN [12] and DAMR [39] use graph neural networks to incorporate known relationships between attributes. However, these methods with fixed model outputs cannot capture the uncertainty and variability of missing values. Recently, researchers have attempted to utilize large language models (LLMs) as the backbone for time series analysis [59] . Since there are significant differences between time series data and natural language, it still has a lot of room for improvement.To capture the uncertainty and variability, researchers introduce generative models [8] into the missing data imputation by learning the implicit distribution of missing values. In the early stage, researchers mainly use variational Autoencoders (VAE) or generative adversarial networks (GAN) to generate new samples that match the distribution of the training dataset and impute missing values with the generated samples. VAE-based methods learn data distributions by optimizing the reconstruction error and regularizing in the latent space [31; 15; 35; 36; 11] . However, they may not capture the complex variability of missing data well and may produce inaccurate results, especially when the latent spaces are not well aligned. GAN-based approaches use the adversarial training technique of the generator and discriminator to improve the imputation results [56; 29; 30; 34] . Unfortunately, they may face convergence difficulties that can affect the imputation accuracy.Diffusion models have been introduced into the imputation task recently, considering the success in various fields [40; 18; 24] . To impute time series data, CSDI [44] designs conditional score-based diffusion models with the conditions only on observed values, and SSSD [2] utilizes both conditional diffusion models and structured state space models. Additionally, MIDM [49] develops the noise sampling, addition, and denoising mechanisms, and PriSTI [27] further studies the enhanced prior modeling by extracting spatio-temporal dependencies as contextual conditions for spatio-temporal data imputation. However, as analyzed in the introduction, existing studies underestimate the importance of accurately modeling the residual term for missing data imputation.For the time series imputation methods in frequency domain, mvLSWimpute [51] utilizes wavelet transforms to guide imputation, APDNet [60] uses the Fourier Temporal and Fourier Variable Interaction modules to model dependencies. In addition, the frequency domain time series forecasting methods FEDformer [58] , FreTS [55] can also be applied to imputation task. However, they did not consider using frequency domain information to model the missing data's residual terms accurately, which is critical for boosting the overall imputation performance.In contrast, our FGTI captures high-frequency information and dominant-frequency information to get a more accurate modeling of the residual term, while assisting in describing trend and seasonal terms.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations of this work in terms of resource consumption in Section 4.4.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
frequency-aware generative models: 
 In this paper, we focus on the incomplete multivariate time series imputation problem. The input multivariate time series X = (X 1 , . . . , X D ) ∈ R D×L is a set of D attribute values recorded at L consecutive timestamps, where each attribute series X d ∈ R L . Each element x ij in X is the observation of the i-th attribute at the j-th timestamp, which is probably missing. We use the binary mask matrix M ∈ {0, 1} D×L to represent the missing status of observations in X, where m ij = 1 in M denotes that x ij is complete, otherwise x ij = 0. In our context, we refer to the imputation target as X. During the training of the imputation model, we choose some observations as the imputation target. When we impute missing values, we treat all the missing values as the imputation target.In Figure 1 , it is evident that the main obstacle to improving multivariate time series imputation is the residual term. Considering a recognized fact that the residual term often contains high-frequency components from the perspective of Fourier analysis [46; 58; 26] , introducing prior knowledge of frequency-domain information can be a feasible approach to enhancing model performance.As a class of superior data imputation models, deep generative models treat the time series imputation task as calculating the conditional imputation target probability distribution q( X0 |C), where q( X0 ) is the clean data distribution and existing deep generative imputation models [56; 29; 44] use the observed values X in the time-domain as the condition C for probability distribution calculation. Note that we denote the complete or the imputed imputation target as the clean imputation target X0 . However, frequency principal [38; 43] reveals that deep models cannot generalize well to highfrequency information. As a result, it may not accurately impute the residual term [13] inherent in the time series dataset that cannot be trivialized in the imputation task [28] .To tackle the above challenge, we incorporate frequency-domain information into condition C to enhance the performance of generative models. Our FGTI implements the condition C that contains time-domain observation condition X C , as well as the frequency-domain conditions C H and C D . Our frequency-domain condition includes the nonlinear transformation of two parts: the highfrequency condition that guides the residual term and the dominant-frequency condition that contains background structure information to impute the trend and seasonal terms, as shown in Figure 2 .The high-frequency filter extracts high-frequency information C H from the time-domain observations, which is used to guide the imputation of residual terms in time series. Since different attributes in a multivariate time series can be heterogeneous, we consider extracting high-frequency information separately for each attribute seriesX d ∈ R L , where d = 1, . . . D.We first interpolate X d , then obtain the amplitude vector A ∈ R ⌊(L+1)/2⌋ for X d over sample frequency components F = { 1 L , . . . , 1 L ⌊(L + 1)/2⌋} by the Fast Fourier Transform (FFT):EQUATIONTo get the high-frequency condition, we discard the frequency components below a cutoff threshold F and map the remaining components to time-domain by the Inverse Fast Fourier Transform (IFFT),EQUATIONwhere ⊙ denotes the Hadamard product.Finally, we concatenate the corresponding high-frequency information vectors C H d for each attribute sequence to form the high-frequency conditionEQUATIONFor the whole input time series X, the time complexity of performing the high-frequency filtering is O(DL log L). Since the time complexity of performing FFT for each attribute series is O(L log L), selecting high-frequency components in the frequency domain has a time complexity of O(L), and performing IFFT costs O(L log L) time.The conditions extracted by the dominant-frequency filter from the time-domain observations not only provide the background structure information for generative models to guide the imputation of the trend and seasonal terms, but also mitigate the interference of the high-frequency condition on the imputation of the trend and seasonal terms 3 .The dominant-frequency information is mainly composed of frequency components with large amplitudes. If we have obtained the representation (A, F) of X d in the frequency-domain from Equation 1, we can find the top-κ frequency components with the largest amplitude according to A, Then we project these κ frequencies to the time-domain via the Inverse Fast Fourier Transform,EQUATIONEQUATIONFinally, we compose the dominant-frequency condition C D ∈ R D×L by concatenating all C D d ,EQUATIONSimilar to the high-frequency filter, the time complexity of performing dominant-frequency filtering for the whole input X is O(DL log L).With the aim of integrating frequency-domain conditions into deep generative models, we first use an encoder to map the conditions to representation C F ∈ R D×L×K in the latent space,EQUATION)where K is the channel number of the latent space. In this paper, we implement the Encoder(•) with the well-acknowledged transformer [47] backbone, since it can self-adaptively extract critical information in C H and C D by the self-attention mechanism.To accurately capture time and attribute dependencies guided by frequency-domain information, we design two frameworks: Time-frequency representation learning and Attribute-frequency representation learning. They integrate the current intermediate hidden representations R in ∈ R D×L×K in the time-domain of deep generative models with the frequency-domain representation C F . Specifically, we use the cross-attention mechanism that can efficiently learn the various input modalities [21; 40] for representation fusion.To capture time dependencies with the aid of frequency-domain information, we divide input hidden representationR in = {R in d ∈ R L×K } D d=1 and frequency information C F = {C F d ∈ R L×K } D d=1into D segments according to attributes. For each pair of latent representation segment (R in d , C F d ), the learning process to obtain time-frequency representation of each attributeEQUATIONwhereQ d = C F d • W Q t , K d = C F d • W K t , V d = R in d • W V t , W Q t , W K t , W V t ∈ R K×K are learnable weight matrices.Then we concatenate R t l of all attributes to obtain the representation R t ∈ R D×L×K ,EQUATIONTo capture dependencies between different attributes based on frequency-domain information, we divide the latent time-frequency representationR t = {R t l ∈ R D×K } L l=1 and C F = {C F l ∈ R D×K } L l=1into L segments, according to timestamps. The learning process to obtain attribute-frequency representation of each timestamp R a l ∈ R L×K is as follows:EQUATIONwhereQ l = C F l • W Q a , K l = C F l • W K a , V l = R tf l • W V a .To get the updated representation R a , we need to concatenate all R a l according to timestamps,EQUATIONRecently, diffusion generative models have demonstrated remarkable proficiency and have emerged as the leading generative models in numerous fields [24; 18] . Thus, we take the diffusion model as an example to introduce how to use frequency-domain conditions to boost missing data imputation.Specifically, we implement FGTI by the frequency-aware diffusion model. Our frequency-aware diffusion model fuses with frequency-domain conditions to learn the conditional imputation target distribution q( X0 | X, C H , C D ), through two Markov chain processes of diffusion step T , i.e., the diffusion forward process and the diffusion reverse process.The diffusion forward process involves gradually adding Gaussian noise into the imputation target,EQUATIONwhereq( Xt | Xt-1 ) = N ( √ α t Xt-1 , β t I), β t ∈ (0, 1), is a hyperparameter satisfying β t < β t+1 for t = 1, . . . , T -1. In addition, α t = 1 -β t and q( X0 ) is the complete data distribution.According to DDPM [20] , Xt has a closed-form solution,EQUATIONwhere ϵ ∼ N (0, I), α t = t i=1 α i . Therefore, we can directly obtain Xt from X0 . The details for deriving the closed-form solution can be found in Appendix A.2.1. Note that when T is large enough, q( XT | X0 ) ≈ q( XT ), q( XT ) ≈ N (0, I).Our diffusion reverse process gradually removes Gaussian noises added to the imputation target based on the time-domain observation condition X C , the high-frequency condition C H and the dominant-frequency condition C D , which can be formalized as the Markov chain,EQUATIONwherep θ ( Xt-1 | Xt , X C , C H , C D ) = N µ θ Xt-1 | Xt , X C , C H , C D , σ t-1 2 .We next show that introducing the high-frequency condition C H and dominant-frequency condition C D can reduce the uncertainty of the diffusion reverse process, improving the imputation accuracy.Proposition 3.1. The conditional entropyH Xt-1 | Xt , X C , C H , C D < H Xt-1 | Xt , X C ,with additional high-frequency condition C H and dominant-frequency condition C D in the diffusion reverse process.It can be proved by the chain rule of the conditional entropy, as detailed in Appendix A.1.Based on the classifier-free guidance diffusion model [19; 20] ,p θ ( Xt-1 | Xt , X C , C H , C D ) can be parameterized as µ θ Xt-1 | Xt , X C , C H , C D = 1 √ α t Xt -β t √ 1-α t ϵ θ t, Xt , X C , C H , C D , σ t-1 2 = (1-α t-1 )β t 1-α t, where ϵ θ (•) is the denoising network with learnable parameter set θ as present in Appendix A.3.1. The mathematical details are presented in Appendix A.2.2.As shown in Figure 3 , the denoising network incorporates frequency-domain information into modeling time dependencies and attribute dependencies, through the time-frequency representation learning module and attribute-frequency representation learning module to guide the denoising.For training the denoising network, we randomly select some observed values as the imputation target X and use the remaining observations as the observation condition X C for each update step, since the ground truth of missing values is unknown. We train the denoising network by minimizing the following objective function L θ ,EQUATIONwhere t ∼ Uniform {1, . . . , T } , X0 ∼ q( X0 ), ϵ ∼ N (0, I).For data imputation, we treat all missing values as the imputation target and all the observed values as the observation condition, i.e., X = X ⊙ (1 -M), X C = X. We start from XT ∼ N (0, I) and perform the T -step diffusion reverse process following Equation 14, to obtain final imputation values X0 . Please see the detailed training and imputation algorithms in Appendix A.3.2.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiment: 
 This section experimentally evaluates both the imputation effectiveness and the improvement of real downstream applications for our FGTI, against various competing methods. All experiments are performed on a machine with Intel Core 3.0GHz i9 CPU, NVIDIA GeForce RTX 3090 24GB GPU, and 64GB RAM. The source code and datasets are available online [1].We employ three real time series datasets with real-world missing values. KDD [6] Following previous studies [33; 27] , we employ RMSE [22] and MAE [7] to evaluate the imputation accuracy. For both, the smaller the value is, the more effective the imputation will be. For the air quality prediction application over the KDD dataset in Section 4.5, we also use RMSE as the metric.In addition, the AUC score [32] is used to measure the patient mortality forecasting application over the PhysioNet dataset. The large the value is, the better the forecasting result will be.We compare with fifteen widely adopted time series imputation methods, including statistics-based Mean [41] , matrix factorization based BTMF [9] and TIDER [28] , deep learning based BRITS [6] , TST [57] , SAITS [14] , TimesNet [52] , LaST [50] and FreTS [55] , GNN-based GRIN [12] , VAE-based TimeCIB [11] . GAN-based GAIN [56] , Diffusion-based CSDI [44] , SSSD [2] , and PriSTI [27] . For methods such as GRIN and PriSTI that require an adjacency matrix as input to show relationships between attributes, we use the identity matrix by default, where every attribute has dependencies with others in the time series. Since LaST and FreTS only focus on the time series forecasting task, we adapt them to the imputation task based on the seting of TimesNet. For methods in which the authors recommend parameters such as SAITS, MIWAE, GPVAE, CSDI and PRiSTI, we use these parameters as suggested. The other methods are also configured in a best-effort fashion by iteratively choosing good parameters.We first explore the imputation performance of different methods over real datasets with different missing rates. The observed values with various missing rates are randomly removed under the missing completely at random (MCAR) mechanism [5] to form the imputation target. We can find that our method achieves the best imputation accuracy under various missing rates. When there is more missing data, deep learning based imputation models are less accurate due to the lack of observation condition information. Nevertheless, our approach uses frequency domain information and achieves superior results. Our FGTI model surpasses the state-of-the-art generative imputation models in various cases, thanks to the incorporation of high-frequency and dominant-frequency condition information.Moreover, since missing data are usually associated with the environment in reality, we consider two additional typical missing data injection mechanisms following the same line of the existing study [33] , i.e., missing at random (MAR) [53] and missing not at random (MNAR) [45] . Specifically, the probability of missing data in MAR is higher when the temperature reading is low in the KDD dataset.On the other hand, in the MNAR scenario, there is a higher probability of missing data during the periods when the reading of each feature is lower. Figure 4 and Figures 8-9 in Appendix A.4.1 show the corresponding imputation results. One can find that the imputation results under different missing mechanisms are relatively similar, and our FGTI achieves optimal performance consistently. This result demonstrates that FGTI can handle missing data in various missing scenarios.We explore the effect of different elements in our FGTI on imputation performance through the following four ablation scenarios. Based on the results presented in Table 2 , it is evident that cross-domain representation learning frameworks and the two types of frequency-domain information are crucial in the process of imputation, which verifies the necessity of each component in our FGTI.We present the resource consumption results of different methods in resource consumption of FGTI, implemented based on the diffusion model, is slightly higher than that of CSDI, due to the inclusion of high-frequency information and dominant-frequency information. However, since FGTI can achieve better imputation results than other methods, as shown in Table 1 , we argue that it is acceptable to incur such an extra resource consumption.To validate the effectiveness of applying imputation in real-world downstream applications, we consider air quality prediction and mortality forecasting tasks. For the air quality prediction application, we first impute real-world missing data in the KDD dataset by various imputation methods. Then, we analyze the records in the previous twelve hours and use the AdaBoost regressor [37] to estimate the average PM2.5 concentration for the upcoming six hours. Figure 6 (a) shows that our method achieves the highest improvement in the air quality prediction task. Figure 6 (b) reports the mortality forecast performance over the PhysioNet dataset. We train the MLP classifier [37] to forecast the mortality on the data without/with imputation. As shown, FGTI achieves the best performance again, which verifies the applicability of our work. Notably, various imputation methods provide a noteworthy and favorable impact on the forecast task, which demonstrates the necessity of imputation. 
conclusion: 
 In this paper, we study imputing incomplete multivariate time series data, through reducing the imputation error in the residual term. By effectively incorporating frequency-domain insights into the generative framework, our FGTI surpasses existing models by capturing high-frequency information and dominant-frequency information. The introduced cross-domain representation learning frameworks further enhance its capability to handle time and attribute dependencies. Comprehensive experimental evaluations over real-world incomplete datasets demonstrate the superiority of FGTI in both the imputation accuracy and the improvement of downstream applications. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We include the detailed experimental settings in Section 4.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We report the results averaged from five experiments with different random seeds.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: We include the impact statement in Appendix A.6. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: All data, models, and code in the paper respect the license.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A.1 Proof of Proposition 3.1We use the conditional entropy in information theory to reflect the amount of uncertainty. For the reverse process in [20] , the imputation target Xt is specified as the condition, thus we useH Xt-1 | Xt = -p θ Xt-1 , Xt log p θ Xt-1 | Xt d Xt-1to model the uncertainty of the reverse process of DDPM.Similarly, we use H Xt-1 | Xt , X C to model the uncertainty of the reverse process of the CSDI [44] model, which only uses the observations as the condition. For our FGTI, we utilizeH Xt-1 | Xt , X C , C H , C D .According to the property of the conditional entropy, we first haveH Xt-1 | Xt ≤ H Xt-1 .Using the definition of mutual information, we haveI Xt-1 ; Xt = H Xt-1 -H Xt-1 | Xt .From Equation 12, we know that I Xt-1 ; Xt > 0, we thus haveH Xt-1 | Xt < H Xt-1 .According to the chain rule of the entropy, we haveH Xt-1 , Xt , X C = H Xt-1 | Xt , X C + H Xt , X C .Then we can getH Xt-1 | Xt , X C = H X C | Xt-1 , Xt + H Xt-1 , Xt -H X C | Xt -H Xt = H Xt-1 | Xt + H X C | Xt-1 , Xt -H X C | Xt .According to the Equation 12, Xt-1 adds the noise one less time than Xt , which indicating that Xt-1 is closer to the observations. Thus, we haveH X C | Xt-1 , Xt < H X C | Xt .By substituting this, we can obtainH Xt-1 | Xt , X C < H Xt-1 | Xt .Following the same line, we can also derive thatH Xt-1 | Xt , X C , C H , C D < H Xt-1 | Xt , X C .This result implies that adding C H and C D to the condition can simplify the distribution that the diffusion models need to learn by reducing the entropy. This simplification can lead to more efficient learning and improve the model's imputation performance by narrowing down the scope of the target distribution's randomness and making its outcomes more predictable.A.2.1 Details of Equation 13If the complete imputation target distribution q( X0 ) is known, we can first get a sampled complete imputation target X0 ∼ q( X0 ). Following Equation 12, we can obtain Xt byXt = √ α t Xt-1 + √ 1 -α t ϵ t , ϵ t ∼ N (0, I).Similarly, we can also obtain Xt-1 byXt-1 = √ α t-2 Xt-2 + 1 -α t-1 ϵ t-1 , ϵ t-1 ∼ N (0, I).Combining the above two equations, we getXt = √ α t √ α t-1 Xt-2 + 1 -α t-1 ϵ t-1 + √ 1 -α t ϵ t = √ α t α t-1 Xt-2 + α t (1 -α t-1 )ϵ t-1 + √ 1 -α t ϵ t .As ϵ t-1 ∼ N (0, I), ϵ t ∼ N (0, I), we can infer that α t (1-α t-1 )ϵ t-1 ∼ N (0, [α t (1 -α t-1 )]I), √ 1 -α t ϵ t ∼ N (0, (1 -α t )I). Therefore, we can get Xt = √ α t α t-1 Xt-2 + 1 -α t α t-1 ϵ = √ α t α t-1 α t-2 Xt-3 + 1 -α t α t-1 α t-1 ϵ = . . . = t i=1 α i X0 + 1 - t i=1 α i ϵ,where ϵ ∼ N (0, I).Combining the Bayes' theorem, we start byp θ ( Xt-1 | Xt , X C , C H , C D ) = p θ ( Xt | Xt-1 , X C , C H , C D ) p θ ( Xt-1 | X C , C H , C D ) p θ ( Xt | X C , C H , C D ) .According to Equation 12, the expectedp θ ( Xt | Xt-1 , X C , C H , C D ) is p θ ( Xt | Xt-1 , X C , C H , C D ) ∼ N ( √ α t Xt-1 , (1 -α t )I).Furthermore, by incorporating Equation 13, we can obtainp θ ( Xt-1 | X C , C H , C D ) ∼ N ( α t-1 X0 θ , (1 -α t-1 )I), p θ ( Xt | X C , C H , C D ) ∼ N ( α t X0 θ , (1 -α t )I),where X0 θ is a virtual result that is expected to be obtained through Equation 14. Combining Equation 12, we can parameterize it by Xt = α t X0 θ + 1 -α t ϵ θ t, Xt , X C , C H , C D , where ϵ θ (•) outputs the predicted added noise to Xt-1 based on the parameter set θ. In other words, In this section, we present the detailed architecture of the denoising network ϵ θ (•) in FGTI model. As shown in Figure 7 , the input projector of the noise imputation target and the observation condition is an MLP layer, and the output projector is a 2-layer MLP with ReLU activation function. For the encoder of the frequency-domain information, it is implemented by a transformer backbone consisting of the position encoding layer and the transformer encoder layer.X0 θ = 1 √ α t Xt -1 -α t ϵ θ t, Xt , X C , C H , C D .Algorithm 1 Training process of FGTI implemented by the diffusion model Input: Incomplete time series X, the number of diffusion step T Output: Optimized denoising network ϵ θ (•) 1: repeat 2: X0 ← select observed values in X 3: t ∼ Uniform {1, . . . , T } 4: ϵ ∼ N (0, I) 5: Xt ← α t X0 + 1 -α t ϵ 6: Perform Gradient Descent by ∇L θ = ∇ θ ϵ -ϵ θ t, Xt , X C , C H , C DIn this section, we explore the imputation performance of the missing at random (MAR) [53] and missing not at random (MNAR) [45] missing mechanisms over the Guangzhou dataset and the PhysioNet dataset. Xt-1 ← 1 √ α t Xt -β t √ 1-α t ϵ θ t, Xt , X, C H , C D + (1-α t-1 )β t 1-α tIn this section, we perform parameter sensitivity experiments on two critical hyperparameters: the cutoff frequency of the high-frequency filter and the maximum magnitude frequency number of the dominant-frequency filter.Effect of the cutoff frequency F. Figure 10 shows the imputation results with various cutoff frequencies F of the high-frequency filter. It can be found that if F is too small, the model may not be able to accurately capture the high-frequency information necessary for guiding the imputation of the time series residual term. The reason is that the high-frequency filter output may include too much low-frequency information. Conversely, if F is too large, the model cannot obtain enough high-frequency information to guide the imputation of the residual term.Effect of the maximum magnitude frequency number κ. We investigate the imputation results when adjusting the maximum amplitude frequency number κ used for the dominant-frequency filter in Figure 11 . As shown in the figure, the imputation model cannot perform well with a small κ. This is because the dominant-frequency filter cannot obtain enough smoothing information, which causes On the contrary, if κ is too large, it can cause some high-frequency information to mix with the output condition of the dominant-frequency filter. This can prevent the model from effectively obtaining background structure information needed for imputing the trend and seasonal terms. As a result, the imputation result for extreme cases may not be optimal.Based on the experimental results, we set the cutoff frequency F of the high-frequency filter to 0.3 and set the number of maximum magnitude frequency κ of the dominant-frequency filter to 10. In addition, for other settings related to the diffusion model, we adopt hyperparameters recommended by the existing well-established models [44; 24] . These models have demonstrated strong performance in similar tasks, and their hyperparameters have been extensively validated in the paper.For training the denoising network, we randomly select some observed values as the imputation target. In this process, the mask ratio and mask pattern to get the imputation target directly determine the effectiveness of training. Thus, in this section, we explore the performance of FGTI with different mask ratios and mask patterns.We first mask different ratios of observations as the imputation target for training, the performance of FGTI with different mask ratios is shown in Table 3 . In addition, we consider a special case where observations with different ratios are randomly masked as imputation targets at each training step, instead of using a fixed masking ratio.We can find that since the random ratio mask strategy can increase the learning complexity and enhance the modeling ability of the diffusion model, the random ratio mask strategy achieves optimal or sub-optimal performance in most cases. So we use the random ratio mask strategy by default.Then we explore the performance when using different mask patterns.Following CSDI [44] and PriSTI [27] , we consider three mask pattern strategies: (1) Block missing (2) Mix missing (3) Random missing. The results is shown in It can be found that Block missing or Mix missing strategy is not comparable to Random missing in most cases due to the possibility that the mask pattern may not correspond to the actual missing scenario. So we use the Random missing mask pattern by default.To compare the imputation performance of FGTI with probabilistic generative baselines in more detail, we adopt CRPS [44] to evaluate the gap between the learned and ground truth distributions for different probabilistic generative methods following [44; 27] .First, we inject different rates of missing values by the MCAR mechanism, and report the CRPS performance with different missing rates in Table 5 . Then we report the CRPS by varying the missing mechanism with 10% missing values in Table 6 .We can find that our method outperforms other probabilistic generative methods for all cases due to the introduction of frequency-domain conditions, thus providing empirical evidence for Proposition 3.1. It can be also found that the variations of CRPS are basically the same as RMSE and MAE for a specific model with different settings.In order to verify the role of the high-frequency condition and the dominant-frequency condition, in this section we conduct a case study of FGTI for trend, seasonal, residual term over the predecomposed KDD dataset.We first perform STL decomposition of the KDD dataset into Trend, Seasonal and Residual terms.Then we select 10% observations of the original KDD dataset as the mask positions by MCAR, and then mask the corresponding positions of the three terms. Finally we imputation the missing values in the three terms separately and report the performances. Note that this setup is the same as the survey experiment shown in Figure 1 in Section 1.To study the role of high-frequency information, dominant-frequency information, and frequencydomain information, we consider the three ablation scenarios (1)w/o Dominant-frequency filter (2) w/o High-frequency filter and (3) w/o Frequency condition in Section 4.3.We report the imputation results of different scenarios over different terms in We can find that for the trend term, retaining the dominant-frequency condition gives the best results, while the high-frequency condition may interfere with the imputation. For the seasonal term, the results are similar to the trend term, but the dominant-frequency information contributes less to the imputation for the seasonal term than for the trend term. This suggests that the seasonl term mainly corresponds to the dominant-frequency information, but also contains some of the high-frequency information. In contrast, the results of the experiments on the residual term show that the residual term mainly corresponds to high-frequency condition. Since we choose the transformer as the encoder in Cross-domain Representation Learning and utilize cross-attention as the fusion mechanism of the two frequency-domain conditions in time-frequency representation learning and attribute-frequency representation learning modules, our method can self-adaptively adjust the weights of the highfrequency information and the dominant-frequency information for different timestamps. Thus our method can outperform existing methods for datasets with multiple circumstances in most cases, as illustrated in Table 1 .To showcase the imputation results of our FGTI model, we visualize the results with the state-of-theart imputation methods CSDI and PriSTI in Figure 12 and Figure 13 . We can find that the CSDI imputation results are not quite accurate for some fast-changing points due to the lack of sufficient condition guidance. On the other hand, both FGTI and PriSTI produced more accurate imputation results because they both used additional conditions. However, as shown in Tabel 1, FGTI still yields better results than PriSTI, suggesting that the high-frequency information and the dominant-frequency information we use are more superior to the interpolation information used by PriSTI.The development our FGTI imputation model could have a significant positive impact on various sectors including healthcare, finance, and environmental monitoring. In healthcare, improved imputation models can lead to more accurate health monitoring systems, enabling early detection and treatment of conditions by filling gaps in patient data. This can ultimately improve patient outcomes and reduce healthcare costs. In the financial sector, enhanced time series imputation can provide better forecasts and risk assessments, aiding in more informed decision-making and potentially stabilizing markets by decreasing uncertainty. Environmentally, better data imputation can improve weather prediction models and climate monitoring systems, aiding in disaster readiness and enhancing our ability to address climate change.However, the deployment of advanced imputation models also raises certain concerns. If used in sensitive areas like surveillance, these models could lead to privacy invasions by reconstructing missing or incomplete data to track individuals without consent. In financial markets, sophisticated imputation methods could also exacerbate inequality by disproportionately benefiting institutions with the resources to leverage state-of-the-art technology, potentially leading to greater market dominance.Additionally, reliance on automated data imputation may result in complacency, where errors in imputation models propagate unnoticed, leading to decisions based on inaccurate or misleading data.To mitigate negative impacts of our imputation model, implement stringent data privacy laws, and ethical guidelines, provide equal access to technology resources across entities, and establish rigorous validation processes to ensure accuracy and fairness. Regular auditing and transparency in algorithm deployment can also play a critical role.NeurIPS Paper ChecklistPlease see an empirical study in Section A.5.1. 


Decision: This paper proposes a method called Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI). The approach aims to more carefully deal with high-frequency residual components when performing time series imputation tasks. The paper has uniformly positive reviews. The reviewers indicate that the paper is generally well-written and well-motivated. The experiments are extensive with a large and representative set of prior approaches and strong results. Some additional baselines and ablations were suggested and the authors provided them during the rebuttal phase. The reviewers who checked in during the author response indicated that their concerns were addressed by the author response.

Reviews: review: paper_summary: paper_summary: The authors present Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), a model which addresses the challenge of missing data in multivariate time series by focusing on the often-overlooked residual term. The paper also incorporates frequency-domain information to enhance imputation performance. Experiments show that this outperforms many existing time series imputation baselines on three real-world datasets.
strength_weakness: strength_weakness: W1. In Introduction and Section 3.1.2, it shows less explanation on why frequency components with large amplitudes could guide both the imputation of trend and seasonal terms. Did it address the intricately entangled trend-seasonal representations, which is highly important in current approaches?

W2. In the section of Related Work, I did not find discussions regarding existing research work and this study in frequency domain for time series tasks, this would be hard to clearly justify the main difference/novelty of this paper compared with current approaches.

W3. In the evaluation, only RMSE and MAE are used as metrics. However, it would be better to include additional metrics such as CRPS.

W4. It would be fairer if the authors compared their approach with a time series model using frequency domain information, which also addresses the trend, seasonal, and residual terms.

W5. Both CSDI and PriSTI address the issue of missing strategies. From the results of Figure 4, 7 and 8, the improvement of MAR and MNAR was much lower than that of MCAR on the Guangzhou and Physio datasets. Was it due to differences in the data distribution, or was it due to the setting of the hyper-parameters?

[1] Learning Latent Seasonal-Trend Representations for Time Series Forecasting.
[2] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting
[3] CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation
[4] PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation
questions: questions: Please see Weaknesses.
limitations: limitations: 1.	My main concern with the work is that the contribution of the proposed method might be incremental. Compared with CSDI, frequency domain filters are utilized to incorporate frequency domain information, and further cross-domain representation learning and frequency-aware diffusion framework are natural implementations, which lack additional novelty.
paper_summary: paper_summary: The paper proposes a new model, called FGTI, to address the issue of missing data in multivariate time series extracting frequency-domain information. The authors argue that existing methods, in general, neglect the residual term, which is the most significant contributor to imputation errors. FGTI incorporates frequency-domain information using high-frequency and dominant-frequency filters to improve the imputation of the residual, trend, and seasonal terms. The model is evaluated on three real-world datasets and demonstrates superior performance in both imputation accuracy and downstream tasks compared to existing methods.
strength_weakness: strength_weakness: - The experiments are thorough and include multiple state-of-the-art imputation methods. However, the paper misses two important time-series imputation methods: one with a VAE-based approach [A] and one applying trend-seasonality decomposition [B]. While it's understandable that the authors might not have had enough time to include these papers since they are from ICLR 2024, technical comparisons with these methods would be necessary before acceptance.

- While the authors provided experiments with various missing scenarios and missing rates, it is not clear why the proposed method outperforms the conventional methods and under what circumstances. For instance, extracting high-frequency information could be significantly distorted if missingness occurs with a certain frequency (which seems realistic considering periodic noises). The paper would be more convincing if the authors provided experiments that assess the effect of the trend and seasonality components of the datasets (or using synthetic datasets) and missing patterns.

- The experiments would be more appealing if the authors provided a breakdown (seasonality and trend) of the assessments on the imputed time-series.

Minor comment:

- Regarding Figure 1, since the RMSE and MAE comparison essentially conveys similar information about the population-level importance of the trend-seasonality decomposition, replacing one with specific time-series examples where the trend and seasonality of the missing values are correctly imputed would better illustrate the motivation. Also, information about how Figure 1 is reported is missing.

References:

[A] Choi and Lee, "Conditional Information Bottleneck Approach for Time Series Imputation, ICLR 2024.
[B] Liu et al., "Multivariate Time-series Imputation with Disentangled Temporal Representations," ICLR 2024.
questions: questions: - Regarding Weakness 2: Please describe why the proposed method works better than conventional methods and under what circumstances. Do the seasonality and trend profiles of the given time-series dataset matter, and if so, how?

- It seems that how to mask the input (to learn the underlying diffusion model) is very important. How does the model's performance change depending on the masking ratio and/or masking patterns? This is crucial as masking may distort the trend and seasonality components of the given time-series.
limitations: limitations: N/A
paper_summary: paper_summary: This paper proposes a generative model, Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), for multivariate time series imputation. The proposed model is designed to enhance the imputation performance by modeling the residual component of time series data. To this end, the paper leverages the observation that the residual components are usually high-frequent. It utilizes two filters in the frequency domain to extract high-frequency and dominant-frequency information from the time series using FFT and IFFT. The frequency information is then input into a denoising network to generate the missing data conditioned on the observed part of the time series. Experiments are performed using three real-world datasets and a number of recent baselines are benchmarked.
strength_weakness: strength_weakness: - It is not clear if the modeling of temporal dependency is sufficient in the time-frequency and attribute-frequency representation learning module. From Eq. (8-9), it seems that only two attention layers are used without considering the time stamp.
- The model architecture and specification used for Projectors in the denoising network is not described.
- There is a gap between Sections 3.2 and 3.3. The time-frequency representation and attribute-frequency representation learning are described and the final output $\mathbf{R}^a$ is defined. However, it is not clear how this representation is used in the diffusion model. It seems from Eq. (14) and Appendix A.2.1 that $\mathbf{R}^a$ is not involved in the parameterization $\epsilon_\theta$.
questions: questions: Please kindly see and clarify my comments in the Weaknesses section.
limitations: limitations: Limitations of high demand of computational resources are discusses with relevant experiment results analyzed.
paper_summary: paper_summary: This paper proposes FGTI that uses frequency-domain information with high-frequency and dominant-frequency filters for accurate imputation of residual, trend, and seasonal components. 
Experimental results demonstrate FGTI's effectiveness in improving both data imputation accuracy and downstream applications.
strength_weakness: strength_weakness: 1. The formulation and writing of this paper should be improved, with attention to correcting several typos.

2. The authors claim that the residual of a time series mainly comprises high-frequency components. However, in many time series, there may be multiple scales of seasonality and trend features, potentially resulting in a final residual that is too noisy to contain useful information.

3. It appears that the proposed method utilizes frequency features of time series as conditions for the diffusion model, which seems straightforward and may limit its contribution.
questions: questions: 1. In Table 2, the absence of the High-frequency filter appears to impact performance less on KDD and Guangzhou datasets. However, on PhysioNet, the situation is reversed. What factors might explain this discrepancy?

2. How were the hyper-parameters for the experiments chosen? Could you elaborate on the methodology used for their selection?

3. What is the purpose of Proposition 3.1? It seems unnecessary and almost self-evident.
limitations: limitations: None

Comments: title: Thanks for the response
Thanks for the response
Thank you for your response
Continued response for W3, C1, Q2
Continued response for W2, Q1
Continued response for W5, L1
Continued response for W4
Continued response for W3
comment: Thank you for appreciating our detailed rebuttals and experimental efforts. We are delighted to know you find our paper interesting with solid motivation and thorough experiments. 

**Given these positive remarks and to strengthen our position for final acceptance, may we kindly request you to reconsider your score and potentially upgrade it?**  This would greatly enhance our chances of contributing our findings to a broader audience.

Best regards,

Authors
I have read all the rebuttals and I fully appreciate the experiments the authors made. This is an interesting paper with good motivation and thorough exepriments. I tend to keep my initial decision, which is "Weak Accept".
Thank you for your thoughtful evaluation and the feedback provided in your review. We are pleased to hear that the additional figure and our responses have helped in addressing your concerns.

- We will certainly revise Equation (7) and Fig.3 to replace Encoder() with Transformer() to more explicitly indicate that time information can be captured via Transformer models, as you've suggested.

- Additionally, we will incorporate detailed descriptions of the input projector and output projector in the main text,
as detailed in the Rebuttal of response W2,
to avoid any ambiguities regarding their architecture and specifications.

- We also acknowledge your satisfaction with the clarification provided through the revised figure regarding the use of 
Ra and Rt. We will ensure that similar clarity is maintained throughout the manuscript to aid the understanding of our model components.

Thank you again for your invaluable feedback and guidance. If you have any questions, please send them to us, we look forward to discussing with you to further improve our work.

Best regards,

Authors
I appreaciate the response from the authors and the additional figure to clarify the model architecture. My concerns have been addressed. Specifically,
- The time information can indeed by captured by Transformer models. I would suggest authors consider spelling this out by revising Eq. (7) from $\text{Encoder}(\cdot)$ to $\text{Transformer}(\cdot)$ to enhance clarity.
- The architecture and specifications for the Projectors are clarified. Please add the relevant descriptions in the main text.
- The use of $\mathbf{R}^a$ and $\mathbf{R}^t$ are clarified with the revised architecture figure.

Given that my concerns are addressed, I increase my overall rating from 4 to 5.
**3. (W3) Provided a breakdown (seasonality and trend) of the assessments**

Thank you for your suggestions. 
The pipeline of our method does not perform a direct STL decomposition, but instead uses the high-frequency and dominant-frequency information to potentially guide the imputation of the three terms.
Due to the effect of missing values, we will get different decomposition results before and after imputation, so for each term the ground truth will be lacking for the evaluation.
Therefore, we first perform STL decomposition of the original KDD dataset, then inject the missing values for the three terms respectively to verify the imputation results, and the results are shown in Fig. 1 of Section 1.
In addition, we also add a Case study about the trend, seasonal and residual terms using the pre-decomposed KDD dataset, and the results are shown in the above table. 
The results show that dominant-frequency information can mainly help the imptuation of the Trend and Seasonal terms, and the high-frequency information is mainly help to the Residual term.

**4. (C1) Replace RMSE or MAE with examples, and clarify Figure 1 reporting details**

Thank you for your valuable suggestions for boosting our manuscript.
Regarding the setting for the study of Fig. 1, we first perform STL decomposition of the KDD dataset into Trend, Seasonal and Residual terms.
Then we select 10% observations of the original KDD dataset as the mask positions by MCAR, and mask the corresponding positions of the three terms.
Finally, we impute the missing values of the three terms by different methods separately.
Following your valuable suggestion, **we add the examples of imputations on trend, seasonal and residual terms in Fig. 1, which is uploaded in the supplementary pdf file at the top of the page**.
and add the setup for this survey study.
The results also indicate that the SOTA time-series imputation methods are inaccurate in imputing the residual term.

**5. (Q2) Explain how masking ratio/patterns affect model performance**

Thank you for your comments. We agree that mask ratio and mask pattern are critical for the learning process.
We randomly mask different ratios of observations as the imputation target at each training step, instead of using a fixed mask ratio, following CSDI [43].
The performance of FGTI with different mask ratios is shown in the following table:
| Mask Ratio | KDD     |         | Guangzhou |         | PhysioNet |         |
|------------|---------|---------|-----------|---------|-----------|---------|
|            | RMSE    | MAE     | RMSE      | MAE     | RMSE      | MAE     |
| 10%        | 0.4372  | 0.1925  | 0.2388    | 0.1647  | 0.5992    | 0.3235  |
| 20%        | 0.4143  | 0.1700  | **0.2312**    | **0.1576**  | 0.5853    | **0.2893**  |
| 30%        | 0.4257  | 0.1707  | 0.2335    | 0.1600  | 0.5836    | 0.2800  |
| 40%        | 0.4185  | 0.1697  | 0.2372    | 0.1634  | 0.6181    | 0.3105  |
| 50%        | 0.4183  | 0.1714  | 0.2343    | 0.1578  | 0.6308    | 0.3138  |
| Random     | **0.4057**  | **0.1489**  | 0.2325    | 0.1584  | **0.5801**    | 0.2856  |

We can find that since the random mask strategy can increase the learning complexity and enhance the modelling ability of the diffusion model, thus our masking strategy achieves optimal or sub-optimal performance in most cases.

Then, we explore the performance when using different mask patterns.
Following CSDI [43] and PriSTI [26], we consider (1) Block missing (2) Mix missing strategy (3) Random missing mask pattern，the results is shown in the following table:
| Mask Pattern | KDD     |         | Guangzhou |         | PhysioNet |         |
|--------------|---------|---------|-----------|---------|-----------|---------|
|              | RMSE    | MAE     | RMSE      | MAE     | RMSE      | MAE     |
| Block        | 0.4187  | 0.1778  | 0.2387    | 0.1596  | 0.6224    | 0.3384  |
| Mix          | 0.4193  | 0.1792  | **0.2325**    | **0.1531**  | 0.6034    | 0.3208  |
| Random       | **0.4057**  | **0.1489**  | **0.2325**    | 0.1584  | **0.5801**    | **0.2856**  |

It can be found that Block missing or Mix missing is not comparable to Random missing in most cases due to the possibility that the mask pattern may not correspond to the actual missing scenario.
So, we use the Random missing mask pattern by default.

Reference:

[a] Conditional Information Bottleneck Approach for Time Series Imputation
**2. (W2,Q1) Clarify why your method outperforms conventional ones, considering trend, seasonality, and missing patterns in datasets**

 Thank you for your valuable suggestions for boosting our manuscript.
We conduct a case study of FGTI for trend, seasonal, and residual terms over the pre-decomposed KDD dataset.
We first perform STL decomposition of the KDD dataset into Trend, Seasonal and Residual terms.
Then we select 10% observations of the original KDD dataset as the mask positions by MCAR, 
and then mask the corresponding positions of the three terms.
To study the role of high-frequency information, dominant-frequency information, 
and frequency-domain information, 
we consider the three ablation scenarios: (1) w/o Dominant-frequency filter
(2) w/o High-frequency filter and (3) w/o Frequency condition in Section 4.3.
We report the imputation results of different scenarios over different terms in following table:
| Component                     | Trend      |            | Seasonal   |            | Residual   |            |
|-------------------------------|------------|------------|------------|------------|------------|------------|
|                               | RMSE       | MAE        | RMSE       | MAE        | RMSE       | MAE        |
| w/o Frequency condition       | 0.048      | 0.0155     | 0.0572     | 0.0364     | 0.5132     | 0.2975     |
| w/o Dominant-frequency filter | 0.0482     | 0.0157     | 0.0533     | 0.0334     | **0.4956** | **0.2814** |
| w/o High-frequency filter     | **0.0409** | **0.0143** | **0.0485** | **0.0301** | 0.5129     | 0.2912     |
| FGTI                          | 0.0448     | 0.0159     | 0.0523     | 0.0325     | 0.5068     | 0.2885     |

We can find that for the Trend term, retaining the dominant-frequency information gives the best results, 
while the-high frequency information may interfere the imputation.
For the Seasonal term, the results are similar to the Trend term,
but the dominant-frequency information contributes less to the imputation for the Seasonal term than for the Trend term.
This suggests that the Seasonl term mainly corresponds to the dominant-frequency information, but also contains some of the high-frequency information.
In contrast, the results of the experiments on the Residual term show that it mainly corresponds to high-frequency information.
Since we choose Transformer as the encoder in Cross-domain Representation Learning and utilize cross-attention as the fusion mechanism of the two frequency domain information in Time-frequency Representation Learning and Attribute-frequency Representation Learning, our method can self-adaptively adjust the weights of the high-frequency information and the dominant-frequency information for different timestamps.
Thus our method can outperform conventional methods for datasets with any circumstances in most cases, as illustrated in Table 1.

In addition, our method's pipeline does not perform STL decomposition of the time series, but extracts the frequency domain informations to guide imputation, so we do not need to know the trend and seasonality of the dataset.
Thanks to the encoder structure and cross-attention based fusion mechanism in Cross-domain Representation Learning, we can adaptively adjust the weights of the two frequency domain information on datasets dominated by different terms.

Furthermore, we consider three missing mechanisms, MCAR, MAR, and MNAR, in our comparative experiments. MCAR implies that each attribute could be missing at any frequency, while MAR potentially implies that all attributes' missing status depends on the frequency of a particular attribute (e.g., high temperatures could cause sensor failure).
For MNAR, the missing status of all attributes depends on the frequency of that attribute.
As shown in Fig 4, Fig 7 and Fig 8 in the manuscript, our method still outperforms conventional methods
with different missing mechanisms (patterns).
**5. (W5) What cause lower MAR and MNAR improvements compared to MCAR for Guangzhou and PhysioNet**

Thank you for your question.
MCAR, MAR and MNAR reflect different missing scenarios in reality.
For MCAR, the missing value is not dependent on other attributes.
For MAR, the probability of missingness depends only on available information (e.g. missing records of traffic flow data may be related to rush hour, activities in public places).
For MNAR, the missing observation depends on othe the attribute itself (e.g. reliability of recording devices).
The effect of the models on different mechanisms depends on the temporal and attribute relations they capured in the dataset.
Under the MAR and MNAR mechanisms, the critical temporal or attribute dependencies are more likely to be missing compared to MCAR, and therefore models may perform not well for MAR and MNAR.
This also reflects the importance of introducing frequency domain information to guide the imputation.

**6.  (L1) The contribution of the proposed method might be incremental**

Thank you for your comments.
**As far as we know, this is the first work to recognize the insight of paying special attention to residual term when impute time series, and introduce frequency-domain information to guide generative models.**
We choose to implement FGTI through the diffusion model because diffusion models are currently demonstrating excellent performance in several fields [17,23].
The frequency domain filters and the cross-domain representation learning module can be flexibly ported to other generative models.
We think this insight can shed light into potential future directions in time series imputation.
Therefore, we argue that our FGTI is not an incremental work.

References:

[a] MSTL: A Seasonal-Trend Decomposition Algorithm for Time Series with Multiple Seasonal Patterns

[b] A wavelet-based approach for imputation in nonstationary multivariate time series

[c] Rethinking general time series analysis from a frequency domain perspective

[d] FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

[e] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting

[f] Learning Latent Seasonal-Trend Representations for Time Series Forecasting
**4. (W4) Compared with the frequency domain method, and the method address the three terms**

Thank you for the valuable suggestions. 
We add LaST [f] and FreTS [e] as baselines for comparison.
Since they focus on the time series forecasting task,  we adapt them to the imputation task based on the TimesNet [48] setting.
Furthermore, we have considered the TIDER model which takes into account trend, seasonal and residual terms in the MF process. 
We first report the part of updated Table 1 for the added baselines:
| 　     | Miss. Rate | Metric |  LaST  |  FreTS |    FGTI    |
|--------|------------|--------|:------:|:------:|:----------:|
| KDD    | 10%        | RMSE   | 0.473  | 0.630  | **0.406** |
|        | 　         | MAE    | 0.287  | 0.412  | **0.149** |
|        | 20%        | RMSE   | 0.532  | 0.741  | **0.451** |
|        | 　         | MAE    | 0.310  | 0.489  | **0.161** |
|        | 30%        | RMSE   | 0.574  | 0.796  | **0.448** |
|        | 　         | MAE    | 0.350  | 0.546  | **0.176** |
|        | 40%        | RMSE   | 0.634  | 0.850  | **0.478** |
| 　     | 　         | MAE    | 0.393  | 0.591  | **0.205** |
| Guang. | 10%        | RMSE   | 0.347  | 0.456  | **0.230** |
|        | 　         | MAE    | 0.244  | 0.340  | **0.170** |
|        | 20%        | RMSE   | 0.440  | 0.602  | **0.258** |
|        | 　         | MAE    | 0.312  | 0.460  | **0.176** |
|        | 30%        | RMSE   | 0.545  | 0.709  | **0.291** |
|        | 　         | MAE    | 0.388  | 0.547  | **0.202** |
|        | 40%        | RMSE   | 0.637  | 0.787  | **0.356** |
| 　     | 　         | MAE    | 0.458  | 0.611  | **0.254** |
| Phy.   | 10%        | RMSE   | 0.768  | 0.804  | **0.580** |
|        | 　         | MAE    | 0.516  | 0.540  | **0.286** |
|        | 20%        | RMSE   | 0.786  | 0.825  | **0.577** |
|        | 　         | MAE    | 0.550  | 0.576  | **0.309** |
|        | 30%        | RMSE   | 0.825  | 0.861  | **0.624** |
|        | 　         | MAE    | 0.578  | 0.603  | **0.336** |
|        | 40%        | RMSE   | 0.850  | 0.883  | **0.669** |
| 　     | 　         | MAE    | 0.603  | 0.626  | **0.376** |

We then report the results for varying missing mechanisms with 10% missing values in the following table, and update Fig.7, Fig.8 and Fig.9:
| 　     | Miss. Mech. | Metric | LaST   | FreTS  | FGTI       |
|--------|-------------|--------|--------|--------|------------|
| KDD    | MCAR        | RMSE   | 0.473  | 0.630  | **0.406** |
|        | 　          | MAE    | 0.287  | 0.412  | **0.149** |
|        | MAR         | RMSE   | 0.473  | 0.630  | **0.406** |
|        | 　          | MAE    | 0.287  | 0.412  | **0.149** |
|        | MNAR        | RMSE   | 0.619  | 0.809  | **0.499** |
|        | 　          | MAE    | 0.326  | 0.473  | **0.174** |
| Guang. | MCAR        | RMSE   | 0.347  | 0.456  | **0.230** |
|        | 　          | MAE    | 0.244  | 0.340  | **0.170** |
|        | MAR         | RMSE   | 0.327  | 0.465  | **0.218** |
|        | 　          | MAE    | 0.234  | 0.356  | **0.150** |
|        | MNAR        | RMSE   | 0.309  | 0.441  | **0.200** |
|        | 　          | MAE    | 0.227  | 0.337  | **0.140** |
| Phy.   | MCAR        | RMSE   | 0.768  | 0.804  | **0.580** |
|        | 　          | MAE    | 0.516  | 0.540  | **0.286** |
|        | MAR         | RMSE   | 0.309  | 0.441  | **0.200** |
|        | 　          | MAE    | 0.227  | 0.337  | **0.140** |
|        | MNAR        | RMSE   | 0.768  | 0.804  | **0.580** |
|        | 　          | MAE    | 0.516  | 0.540  | **0.286** |

After this, we report the resource consumption over KDD dataset with 10% missing values in the following table,
and update Fig.5:
|          | GPU Consumption Usage (MiB) | Running Time (s) |
|----------|-----------------------------|------------------|
| LaST     | 444                         | 4.63             |
| FreTS    | 1118                        | 9.09             |

Finally we report the updated downstream application experiments in the following table, and update Fig.6:
|            | Air quality prediction (RMSE) | Mortality forecast (AUC) |
|------------|-------------------------------|--------------------------|
| LaST      | 0.64                          | 0.80                     |
| FreTS     | 0.65                          | 0.80                     |
| FGTI       | **0.59**                      | **0.86**                 |

We can find that our FGTI still outperforms all baselines due to the guidance of high-frequency information and dominant-frequency information, as well as the generating ability of the diffusion model.
**3. (W3) Include additional metrics such as CRPS**

Thank you for the thoughtful comments and valuable suggestions.
Since the CRPS is used to evaluate probabilistic imputation baselines, we report the CRPS for all probabilistic imputation baselines  according to the experimental settings in CSDI [43]and PriSTI [26].
First, we report the CRPS performance with different missing rates in the following table：
| Dataset | Miss. Rate | MIWAE  | GPVAE  | TimeCIB | GAIN   | CSDI   | SSSD   | PriSTI | FGTI   |
|---------|------------|--------|--------|---------|--------|--------|--------|--------|--------|
| KDD     | 10%        | 0.524  | 0.443  | 0.466   | 0.709  | 0.224  | 0.352  | 0.232  | **0.158**  |
|         | 20%        | 0.526  | 0.445  | 0.467   | 0.718  | 0.245  | 0.370  | 0.248  | **0.170**  |
|         | 30%        | 0.532  | 0.447  | 0.469   | 0.729  | 0.259  | 0.374  | 0.268  | **0.186**  |
|         | 40%        | 0.530  | 0.457  | 0.471   | 0.746  | 0.278  | 0.401  | 0.301  | **0.216**  |
| Guang.  | 10%        | 0.312  | 0.333  | 0.360   | 0.692  | 0.265  | 0.316  | 0.209  | **0.155**  |
|         | 20%        | 0.312  | 0.330  | 0.357   | 0.694  | 0.277  | 0.299  | 0.244  | **0.168**  |
|         | 30%        | 0.312  | 0.335  | 0.356   | 0.695  | 0.292  | 0.353  | 0.310  | **0.193**  |
|         | 40%        | 0.312  | 0.333  | 0.358   | 0.697  | 0.324  | 0.382  | 0.362  | **0.243**  |
| Phy.    | 10%        | 0.689  | 0.659  | 0.466   | 0.739  | 0.544  | 0.617  | 0.444  | **0.343**  |
|         | 20%        | 0.717  | 0.665  | 0.467   | 0.761  | 0.589  | 0.665  | 0.457  | **0.369**  |
|         | 30%        | 0.750  | 0.674  | 0.469   | 0.787  | 0.627  | 0.630  | 0.467  | **0.389**  |
|         | 40%        | 0.779  | 0.680  | 0.471   | 0.814  | 0.671  | 0.676  | 0.491  | **0.441**  |

Then we report the CRPS by varying the missing mechanism with 10% missing values in the following table：
| Dataset | Miss. Mech. | MIWAE  | GPVAE  | TimeCIB | GAIN   | CSDI   | SSSD   | PriSTI | FGTI   |
|---------|-------------|--------|--------|---------|--------|--------|--------|--------|--------|
| KDD     | MCAR        | 0.524  | 0.443  | 0.466   | 0.709  | 0.224  | 0.352  | 0.232  | **0.158**  |
|         | MAR         | 0.539  | 0.437  | 0.470   | 0.710  | 0.229  | 0.489  | 0.239  | **0.164**  |
|         | MNAR        | 0.615  | 0.435  | 0.490   | 0.715  | 0.244  | 0.456  | 0.252  | **0.174**  |
| Guang.  | MCAR        | 0.312  | 0.333  | 0.360   | 0.692  | 0.265  | 0.316  | 0.209  | **0.155**  |
|         | MAR         | 0.258  | 0.334  | 0.298   | 0.692  | 0.252  | 0.367  | 0.208  | **0.148**  |
|         | MNAR        | 0.241  | 0.340  | 0.294   | 0.693  | 0.251  | 0.267  | 0.210  | **0.144**  |
| Phy.    | MCAR        | 0.689  | 0.659  | 0.466   | 0.739  | 0.544  | 0.617  | 0.444  | **0.343**  |
|         | MAR         | 0.679  | 0.660  | 0.593   | 0.739  | 0.550  | 0.724  | 0.454  | **0.356**  |
|         | MNAR        | 0.726  | 0.655  | 0.608   | 0.743  | 0.566  | 0.715  | 0.476  | **0.366**  |

Combining the results in Table 1 and Fig.4, Fig.7 and Fig.8,  it can be found that the variations of CRPS are basically the same as RMSE and MAE for a specific model with different settings.
Thanks again to the authors for their valuable suggestions. We will add this part of the experiment in the Appendix.