Frequency-aware Generative Models for Multivariate Time Series Imputation
Authors: Xinyu Yang
Yu Sun
Xiaojie Yuan
Xinyang Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: Time series; Time series Imputation; Generative Models; Frequency domain
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: This paper proposes a frequency-aware generative model (FGTI) for multivariate time series imputation, which integrates frequency-domain information and uses cross-domain representation learning modules to enhance imputation accuracy.

Abstract: Missing data in multivariate time series are common issues that can affect the analysis and downstream applications.
Although multivariate time series data generally consist of the trend, seasonal and residual terms, existing works mainly focus on optimizing the modeling for the first two items. However, we find that the residual term is more crucial for getting accurate fillings, since it is more related to the diverse changes of data and the biggest component of imputation errors.
Therefore, in this study, we introduce frequency-domain information and design Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI). Specifically, FGTI employs a high-frequency filter to boost the residual term imputation, supplemented by a dominant-frequency filter for the trend and seasonal imputation. Cross-domain representation learning module then fuses frequency-domain insights with deep representations.
Experiments over various datasets with real-world missing values show that FGTI achieves superiority in both data imputation and downstream applications.

Frequency-awarc Generative Models for Multivariate Time Series Imputation
Xinyu Yang', Yu Sun Xiaojie Yuan', Xinyang Chen
College of Computer Science, DISSec, Nankai University, China (yangxinyu@dbis. sunyuê, ,aniyuu,,uuu yuannxjjjaaaja))))a @anaaai.. edu Schooo of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China chenxinyang @ hit.edu.cn
Abstract
Missing data in multivariate time series are common issues that can affect the analysis and downstream applications. Although multivariate time series data generally consist of the trend, seasonal and residual terms, existing works mainly focus on optimizing the modeling for the first two items. However, we find that the residual term is more crucial for getting accurate fillings, since it is more related the diverse changes of data and the biggest component of imputation errors. Therefore, in this study, we introduce rrequencyddomal information and design Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTT). Specifically. FGTI employs high-frequeng, filter to boost the residual term imputation, supplemented by a meminant-ffrquenn filter for the trend and seasonal imputation. Cross-domain representation learning module then fuses frequency-doma insights with deep representations. Experiments over various datasets with real-world missing values show that FGTI achieves superiority in both data imputation and downstream applications.
Introduction
Missing data are commonly observed in the multivariate time series due to diverse reasons [25]. which would encumber subsequent analysis and applications [17]. not surprising that more accurate missing data imputation generally leads to better performance in downstream applications [T]
Existing techniques [3] 28 have revealed that time series data can be decomposed into three distinct terms, i.e., trend, seasonal, and residual, and try to compute an imputation by modeling the first two items accurately possible. Figure I reports a survey for the imputation accuracy of the three terms by representative imputation methods over the pre-decomposed KDD [6) dataset with 10% missing values The dataset comprises 8,034 consecutive readings of meteorological and air quality data taken over year in Beijing. In this dataset, the trend term may reflect long- long-term changes climate or air quality conditions, and the seasonal term might capture patterns associated with different seasons. Morcover, the residual term may consist of short-term, irregular, and high-frequenge the imputation error is mainly caused by the residual term, which has
changes. As shown in Figure [ not been well studied, unfortunately.
Recent studies indicate that the high- fegh-freeuency components intricately related to the residual [4, 58 (6) and contain critical information for imputing the residual term. Unfortunately, deep learning architectures cannot generalize well in modeling high-frequency components. [38 43 To
"Corresponding authors.
Please see an empirical study in Section 4.5
Pleasse Section 2 for brief survey, see Appendix A.5.1 for detailed experiment setup.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

BTMF SAITS
TimesNet
TimeCIB
PriSTI PGTI
observation
ground truth values
GRIN
PriSTI
PGTT
Trond of Attribute
Value
0.7
150
160
170
100
190
Soasonal of Attribute
6
0.6-
Value
RMSE
-0.2
150
160
170
180
190
Residual of Attribute 6
0.5
Value
0.21
0.0
0.0
Trond
Seasonal
Residual
0.5
150
1600
170
180
190
(a)
(b)
Figure 1: Improving the imputation accuracy of the residual term is the key to boosting the imputation performance of the model.
meet this challenge, we design Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), which can extract frequency-domain information and use two cross-domain (i.e., time-domain and frequency-domaiio representation learning modules to guide the generation process of deep models. Specifically, we start with the high-frequency filter designed to extract the high-frequency information essential for guiding the accurate imputation of the residual term. This choice is consistent with recognizing the critical role of high-frequency information in imputation accuracy. Then, we introduce the dominant-frequenc: filter to address the potential challenge posed by high-frequency information for imputing trend and seasonal terms 4 Furthermore, our cross-domain representation learning frameworks combine frequency-domen information with deep representations in the time-domain, enabling seamlessly intertwining frequency-domen information with time and attribute dependencies modeling.
Our research makes several notable contributions:
We design frequency-awarr generative model FGTI with ciqquency-domate information integrated by the high-frequency filter and the dominant dominant-freqquency frequency filter, to enhance the awareness of the siqueeccy-domiin
We introduce two cross-domain representation learning modules that provide models with prior knowledge of intricate frequency-related patterns for missing data imputation.
We evaluate FGTI on three time series datasets with real-world missing values, which demonstrates the superiority of FGTI in both imputation accuracy and downstream applica- tions.
2 Related work
Traditional imputation methods usually employ the statistics, such as mean value 23 median value [16, or last observed value (3], to impute missing data for multivariate time series. It is not surprising that such traditional signals cannot make full use of the valuable semantics of available data. BTMF [9] and TIDER (88) employ the low-rank matrix factorization to impute missing data. Unfortunately. due to the matrix capacity's 's limitation, is still challenging to accurately match imputation values with the underlying complex relationships and dependencies.
Many studies have shown that deep learning based imputation methods are effective fill multivariate time series data, such as BRITS 66, TST 57]. SAITS [14), STCPA [54), TimesNet [22 According 488, forecasting models 550 (55 can also applied to imputation task. Additionally, GRIN [12
and DAMR 39 use graph neural networks to incorporate known relationships between attributes. However, these methods with fixed model outputs cannot capture the uncertainty and variability of missing values. Recently, researchers have attempted to utilize large language models (LLMs) as the backbone for time series analysis [59]. Since there are significant differences between time series data and natural language, it still has a lot of room for improvement.

---

To capture the uncertainty and variability, researchers introduce generative models [8] into the missing data imputation by learning the implicit distribution of missing values. In the early stage, researchers mainly use variational Autoencoders (VAE) or generative adversarial networks (GAN) to generate new samples that match the distribution of the training dataset and impute missing values with the generated samples. VAE-based methods learn data distributions by optimizing the reconstruction error and regularizing in the latent space [3T, IS; 35; 36 [T]. However, they may not capture the complex variability of missing data well and may produce inaccurate results, especially when the latent spaces are not well aligned. GAN-based approaches use the adversarial training technique of the generator and discriminator to improve the imputation results [56] 29 30 34]
Unfortunately,
they may face convergence difficulties that can affect the imputation accuracy.
Diffusion models have been introduced into the imputation task recently, considering the success in various fields EC 18 24]. To impute time series data, CSDI [44] designs conditional score-based diffusion models with the conditions only on observed values, and SSSD (2) utilizes both conditional diffusion models and structured state space models. Additionally, MIDM
49 develops the noise sampling. addition, and denoising mechanisms, and PriSTI (27 further studies the enhanced prior modeling by extracting spatio-temporal dependencies as contextual conditions for spatio-temporal data imputation. However, as analyzed in the introduction, existing studies underestimate the importance of accurately modeling the residual term for missing data imputation.
For the time series imputation methods in frequency domain, myLSWimpute [51] utilizes wavelet transforms to guide imputation, APDNet [60] uses the Fourier Temporal and Fourier Variable Interaction modules to model dependencies. In addition, the frequency domain time series forecasting methods FEDformmr [58), FreTT (55) can also be applied to imputation task. However, they did not consider using frequency domain information to model the missing data's 's residual terms accurately, which is critical for boosting the overall imputation performance.
In contrast, our FGTI captures high-frequency information and dominant-frequency information to get a more accurate modeling of the residual term, while assisting in describing trend and seasonal terms.
3 Frequency-aware Generative Models
In this paper, we focus on the incomplete multivariate time series imputation problem. The input multivariate time series X = (X...... Xp) E Dx is a set of D attribute values recorded at I consecutive timestamps, where each attribute series X, E R_. Each element Tij in X is the observation of the i-th attribute the j-th timestamp, which is probably missing. We use the binary mask matrix M E {0, 13 DxL to represent the missing status of observations in X, where mij = 1 in M denotes that Fay is complete, otherwise Zi = 0. In our context, we refer to the imputation target as X. During the training of the imputation model, we choose some observations as the imputation target. When we impute missing values, we treat all the missing values as the imputation target.
In Figure I it is evident that the main obstacle to improving multivariate time series imputation is the residual term. Considering a recognized fact that the residual term often contains high-frequency components from the perspective of Fourier analysis [46] 58 26), introducing prior knowledge of domaan information can be feasible approach to enhancing model performance.
As class of superior data imputation models, deep generative models treat the time series imputation task as calculating the conditional imputation target probability distribution g( [ C), where q( ()
is the clean data distribution and existing deep generative imputation models [56, 29 44) use the observed values X in the time-domain the condition C for probability distribution calculation. Note that we denote the complete or the imputed imputation target as the clean imputation target X
reveals that deep models cannot generalize well to high- frequency information. As result, it may not accurately impute the residual term [ inherent in the time series dataset that cannot be trivialized in the imputation task (28].
However, frequency principal 38 43
To tackle the above challenge, we incorporate frequency-domaio information into condition C to enhance the performance of generative models. Our FGT1 implements the condition C that contains time-domain observation condition x well as the Eeqquency-domain conditions CH and CD.
3

---

IFFT
CH
High-frequency Filter
littlad
IFFT
X
Dominant-frequency Filler
C
Figure 2: The high-frequency filter with FF = 0.3 and the dominant dominant-frequens, frequency filter with 3.
3.1
Frequency-domain Condition Filter
Our frequency-domain condition includes the nonlinear transformation of two parts: the high- frequency condition that guides the residual term and the cominant-frequeee condition that contains background structure information to impute the trend and seasonal terms, as shown in Figure 2
3.1.1 High-frequency Filter
The high- freu-frr filter extracts high- geh-frrquency information CH from the time-domain observations, which is used to guide the imputation of residual terms in time series. Since different attributes in multivariate time series can be heterogeneous, we consider extracting high-frequency information separately for each attribute series X RL,, where d 1,.... D.
We first interpolate X then obtain the amplitude vector A E R((++1//2) for X, over sample frequency components =12- ..:. [L 1)/2]} } by the Fast Fourier Transform (FFT):
(A. F) FFT (X+).
To get the high-frequenyy condition, discard the frequency components below cutoff threshold F and map the remaining components to time-domain by the Inverse Fast Fourier Transform (IFFT),
CH = IFFT A O (F>
where denotes the Hadamard product.
Finally, concatenate the corresponding high-frequency information vectors CH for each attribute sequence form the high-frequency condition CH T DxL
HH
Concat
(CH)) D )ml)
For the whole input time series X, the time complexity of performing the high-frequency filtering is O(DL log L). Since the time complexity of performing FFT for each attribute series is O log L). selecting high-frequency components in the frequency domain has a time complexity of OL), and performing IFFT costs O(L log L) time.
3.1.2 Dominant-frequenc Filter
The conditions extracted by the comminant-frequence filter from the time-domain observations only provide the background structure information for generative models to guide the imputation of the trend and seasonal terms, but also mitigate the interference of the high-frequency condition the imputation of the trend and seasonal terms")
The dominant-frequenc information is mainly composed of frequency components with large amplitudes. we have obtained the representation (A, F) of X. in the frequency-doma from Equation I we can find the top-k frequency components with the largest amplitude according to A, {f...., fx} arg topx(A).
44)
'Please empirical study in Section A.5.I

---

Imputation targel
Minimize Ly
Diffusion Step
Uniforn
Noie impulation Sarget X
Concal
Projeder
Time-frequency
Alrrbute-frequerrcy Representation Learning
Projector
Representation
Learning
Incomo
MV
Encoder
Observation
Concal
Denoising Network
Higuufreyeency Filter
Dominant-frequency
Domnertt-noquungg
condition
Figure 3: The pipeline of FGTI implemented by the frequency-aware diffusion model. FGTI incorporates high- frequeeyy representations to guide the residual term and compensates for the trend and seasonal terms with the domminat--requenccy frequency representations. With cross-domain representation learning, our FGTI includes frequency-done domain information into time and attribute dependencies modeling to estimate the diffusion noise.
Then we project these frequencies to the time-domain via the Inverse Fast Fourier Transform,
= IFFT A @ (F E ((i,...,
(5)
Finally, we compose the dominant-
fominant-frequency frequency condition C E Dx by concatenating all C
CD Concal
(1c))
Similar to the high-frequency filter, the time complexity of performing forinent-frequuecy filtering for the whole input X OoD T.).
Cross-domain Representation Learning
With the aim of integrating frequency-domain conditions into deep generative models, we first use an encoder to map the conditions to representation C E Dx in the latent space,
F Encoder [Concat (CH, cP)].
(7)
where K is the channel number of the latent space. In this paper, implement the Encoder(-) with the well-acknowledded transformer [47] backbone, since can self-adaptively extract critical information in CH and C by the self-attention mechanism.
To accurately capture time and attribute dependencies guided by frequency- information, we design two frameworks: Time-frequency representation learning and Attribute-freque representa- tion learning- They integrate the current intermediate hidden representations in E DDLLX in the time-domain of deep generative models with the frequency-domat representation CF,
Specifically, use the cross-attention mechanism that can efficiently learn the various input modali- ties [1 for representation fusion.
3.2.1 Time-frequency Representation Learning
To capture time dependencies with aid of frequency-domain information, we divide input hidden representation R {Ri" R.. y2 ) and frequency information CF (CT R f.x K)D Jd=1
into D segments according to attributes. For each pair of latent representation segment (R. CP)
5

---

the learning process to obtain timeffrequency representation of each attribute R E Rx is
QaKa
R = Softmax
Vd.
(8)
VK
where Q4 C WP, K learnable weight matrices.
C₁ WK. V.
R
W. W WP, WY E R
Then we concatenate R/ of all attributes to obtain the representation R E DXXXX
(R)))_))
= Concat
(9)
3.2.2 Attribute-frequency Representation Learning
To capture dependencies between different attributes based on frequency-domain information, we divide the latent time-frequency representation R {R} E ) and C
{CF E RDDx into segments, according to timestamps. The learning process to obtain attribute-rrequency representation of each timestamp R, RRLKK is as follows:
QiKi T VK
R
Softmax
Vi,
(10)
where Q = CF w.. = CF WK V, = R WV
To get the updated representation R need to concatenate all R, according to timestamps,
Concat (1R1)
R
(11)
3.3 Frequency-aware Diffusion Model
Recently, diffusion generative models have demonstrated remarkable proficiency and have emerged as the leading generative models in numerous fields (24 18. Thus, we take the diffusion model as an example introduce how to use frequency-domat conditions to boost missing data imputation.
Specifically, we implement FGTT by the frequency-awan diffusion model. Our frequency-awan diffusion model fuses with frequency-domain conditions learn the conditional imputation target distribution g( 9 X, CH CP), through two Markov chain processes of diffusion step T,, i.e., the diffusion forward process and the diffusion reverse process.
The diffusion forward process involves gradually adding Gaussian noise into the imputation target,
(x 1:7
x
II
(12)
where qX' X N\Va'X IT,, 8t (0. 1), is a hyperparameter satisfying 8 ge+1 fort I,.... , T 1. In addition, a' = 1 - 3e and (Xx)
) is the complete data distribution.
According to DDPM 20 X has closed-form solution,
-1-
(13)
where a'' a Therefore, deriving the closed-form form solution can be found Appendix A.2.1 Note that when T is large enough, xXTT X))= N ((T),, ( (XT)) RV I).
directly obtain X from X The details for
Our diffusion reverse process gradually removes Gaussian noises added to the imputation target based on the time-domain observation condition x the high-frequency condition CH and the cominant-frqquene condition CD, which can be formalized as the Markov chain,
T
Po X T-110
x
II (X
X C CH CP),
(14)

---

where po (X X X.; CH CC) =N (N)==((H)) X X X CH CD) 19)
We next show that introducing the high-frequenoe condition CH and dominant-frquenny condition CD can reduce the uncertainty of the diffusion reverse process, improving the imputation accuracy. Proposition 3.1. The conditional entropy
H
CH. CH
with additional hrege-fry condition CH and dominant-frequency condition CD in the diffusion reverse process.
It can be proved by the chain rule of the conditional entropy, as detailed in Appendix
A.I
Based on the classifier-free guidance diffusion model 19
20
Pa
t-1
X
CH
CD) can be
parameterized as 148
X x
X XC
CH CD
X
3
X
XC
CH.,
CD)
VI-a"
[o'-1]2- (1 (1-a--1)8 where c.(.) is the denoising network with learnable parameter set as present
in Appendix A.3. The mathematical details are presented in Appendix A.2.2
As shown in Figure the denoising network incorporates requuencydomaaie information into modeling time dependencies and attribute dependencies. through the time- freeffreyenccy representation learning module and attribute-freet frequeenyyyyy representation learning module to guide the denoising.
For training the denoising network, we randomly select some observed values as the imputation target X and use the remaining observations as the observation condition X for each update step, since the ground truth of missing values is unknown. We train the denoising network by minimizing the following objective function Ca.
Lo 6 60 (1,X*, XXXXXO XC cH, "D)
(15)
where L ~ Uniform (1,..., T.....,T]. 7}, X
((x)), )e (O,I).
For data imputation, we treat all missing values as the imputation target and all the observed values as the observation condition, i.e., X X (1 M), x X. We start from X (O, and perform the T Ttspep diffusion reverse process following Equation 14 to obtain final imputation values X Please see the detailed training and imputation algorithms in Appendix A.3.2
4 Experiment
This section experimentally evaluates both the imputation effectiveness and the improvement of real downstream applications for our FGTI, against various competing methods. All experiments are performed on machine with Intel Core 3.0GHz 19 CPU, NVIDIA GeForce RTX 3090 24GB GPU, and 64GB RAM. The source code and datasets are available online [I.
4.1
Experimental Setup
4.1.1 Datasets
We employ three real time series datasets with real-world missing values. KDD 6 collects 8,034 meteorological and air quality readings of nine stations from January 30, 2017 to January 31, 2018 in Beijing, with 4.46% real missing values. This dataset is collected every one hour and eleven sensor readings are recorded at each station. Guangzhou [10 records traffic speeds per ten minutes on 214 anonymous roads in Guangzhou from August 1, 2016 September 30, 2016. There are 1.29%
real missing values in the dataset. PhysioNet [42] contains 37 measurement readings from 1,988 patients within hours of the ICU admission. 79.71% measurements are missing in the dataset, and 1,707 patients died after 48 hours of the ICU admission. Following existing studies (29 30) since the ground truth is unavailable, we ignore these missing values when evaluating the imputation accuracy in comparative experiments and model analysis, but consider them in the application study.
7

---

Table 1: Imputation performance of various methods over real datasets with different missing rates
Dataset Miss. Metric Mean BTMF TIDER BRITS TST SATTS
TimesNet Fr GRIN TimeCIB GAIN CSDI SSSD PGSTI
KDD 10% RMSE 0,993 0.529
0.777
0.700 0.594 0.542
0.484
0.473
0.630 0.565
0.589
0,864 0.459 0.697 0.472 0.406
MAE 0,718 0.285
0.527
0.407 0,360 0.304
0.313
0.287 0.412 0.322
0.367
0.607 0.177 0.397 0.169 0.149
20%
RMSE 1.077 0.554
0.797
0.729
0.740 0.575
0,542
0.532
0.741 0.607
0.613
0.877
0.500 0.701 0.534 0.451
MAE 0.286
0.531
0.416
0.310
0.307
0.310 0.489 0.339
0.369
0.606
0.187 0.392 0.180 0.161
30% RMSE 0.997 0.541
0.783
0.720
0.642 0.574
0.578
0.574
0.796 0.617
0.603
0.870
0.519 0.717 0.547 0.448
MAE 0,717 0.286
0.528
0.420
0,376
0.319
0.357
0.350
0.546 0.360
0.370
0.612 0.199 0,413 0.195 0.176
40% RMSE 1.001 0.548
0.790
0.734
0,702 0.593
0.648
0.634
0.850 0.650
0.611
0,883
0.569 0,747 0.581 0.478
MAE 0,718 0.287 0.532 0.428
0,387 0.332
0.418
0.393
0.591 0.387
0.372
0.623
0.220 0.435 0.217 0.205
Guang. 10% RMSE 0.799 0.384
0.549
0.481
0.368 0.417
0.400
0.347
0.456 0.466
0.451
0.804
0.306 0.434 0.242 0,230
MAF 0.592 0.252 0.392 0.299
0.249 0.264
0.270
0.244
0.340 0.354
0.300
0.550
0.210 0.293 0.170 0.170
20% RMSE 0,799 0.384
0.537
0.481
0.398 0,415
0.433
0,440
0.602 0.501
0,448
0,804
0.324 0,460 0.324 0.258
MAE 0,592 0.252
0.382
0.300 0,275 0.264
0.303
0.312
0,460
0.385
0.298
0,550 0.220 0,315 0.197 0.176
30% RMSE 0.799 0.384
0.536
0.485
0.442 0,420
0.481
0.545
0.709 0.542
0.448
0.805
0.364 0.545 0,510 0.291
MAE 0.592 0.252
0.382
0.301
0.312 0.267
0.348
0.388
0.547 0.419
0.298
0.551
0.242 0,384 0.271 0.202
40%
RMSE 0,800 0.385 0,541
0.491
0.540 0.422
0.542
0.637
0.787 0.584
0.449
0.807
0.439 0.622 0.650 0.356
MAE 0.592 0.253
0.387 0.306
0.397 0.270
0.401
0.458
0.611 0.455
0.299
0.554 0.283 0.444 0.381 0.254
Phy.
10% RMSE 0.932 0.630
0.879 0.732
0.632 0.645
0.776
0.768
0.804
0.682
0.697
1.006
0.619 0.875 0.652 0,580
MAE 0.678 0.348
0.605
0.446
0.389 0.371
0.525
0.516
0.540 0.424
0.450
0.747
0.369 0.286
20% RMSE 0.935 0.627
0.889
0.718
0.640 0.641
0.806
0.786
0.825 0.670
0.683
0.988
0.664 0.834 0.638 0.577
MAE 0.675 0.362
0.624
0.451
0.417 0.384
0.569
0.550
0.576 0.434
0.455
0.740
0.335 0.507 0.376 0.309
30% RMSF 0,934 0.658 0.911
0.734
0.688 0.670
0.849
0.825
0.861 0.695
0.697
0,,995
0.805 0,882 0.661 0.624
MAF 0.676 0,382
0.638
0.457
0.452 0.404
0.600
0.578
0.603
0.446
0.459
0,738 0.360 0,545 0.387 0.336
40% RMSE 0.932 0.677 0.935
0.739
0.732
0.688
0.872
0.850
0.883
0.708
0.698
0,983
0.705
0,904 0.679 0.669
MAE 0.677 0.412 0.658
0.466
0.493 0.431
0.623
0.603
0.626
0.464
0.466
0.729
0.395
0.555
0.406 0.376
4.1.2 Criteria
we employ RMSE 22 and MAE 7 to evaluate the imputation Following previous studies 33
27
accuracy. For both, the smaller the value is, the more effective the imputation will be. For the air quality prediction application over the KDD dataset in Section 4.5] we also use RMSE as the metric. In addition, the AUC score [2) is used to measure the patient mortality forecasting application over the PhysioNet dataset. The large the value is, the better the forecasting result will be.
4.1.3 Baselines
We compare with fifteen widely adopted time series imputation methods, including statistics-
Mean 41), matrix factorization based BTMF 99] and TIDER (28) deep learning based BRITS
66. TST (57), SAITS (4), TimesNet (52] LaST (50) and FreTS [55], GNN-based GRIN T2].
TAE-based TimeCIB [I].. GAN-based GAIN (56). Diffusion- based CSDI 44) SSSD [2]. and
PriSTI (27). For methods such as GRIN and PriSTI that require an adjacency matrix as input to show relationships between attributes, we use the identity matrix by default, where every attribute has dependencies with others in the time series. Since LaST and FreTS only focus on the time series forecasting task, we adapt them to the imputation task based on the seting of TimesNet. For methods in which the authors recommend parameters such as SAITS, MIWAE, GPVAE, CSDI and PRiSTI, we use these parameters as suggested. The other methods are also configured in best-effort fashion by iteratively choosing good parameters.
4.2 Comparative Experiments
We first explore the imputation performance of different methods over real datasets with different missing rates. The observed values with various missing rates are randomly removed under the miss- ing completely at random (MCAR) mechanism (5) to form the imputation target. Each experiment is repeated five times with different generated missing values and random seeds, and the average result is reported in Table I Note that RMSE reflects the absolute difference between the imputation value
8

---

TimesNet
TimeCIB
PriSTI
BRITS (a)
SAITS
LaST
CRIN
SSSD
FCTT
(b)
1.0
RMSE
MAE
0.5
0.5
0.0
0.0
MCAR
MAR
MINAR
MCAR
MAR
MNAR
Figure 4: Varying the missing mechanism over KDD dataset with 10% missing values
Table 2: Ablation analysis of FGTI with 10% missing values
Method
KDD
Guungzhou
PhysioNet
RMSE
MAE
RMSE
MAE
RMSE
MAE
Cross-domain
0,4210
0.1603
0.2365
0.1607
0.6288
0.3748
w/o Frequency condition
0,4151
0.1505
0.2402
0.1635
0.6165
0,3719
w/o soniinatt-freqencyy filter
0.4151
0,1518
0,2383
0.1625
0.6540
0,3799
figh-fre
0.1493
0.2367
0.1611
0.7294
0.3654
FGTI
0.40157
0.1489
0.2325
0.1584
0,5801
0.2856
and ground truth the context of the data scale, and is not bounded by a specific range. The missing rates in the table are with respect to the observed values.
We can find that our method achieves the best imputation accuracy under various missing rates. When there is more missing data, deep learning based imputation models are less accurate due to the lack of observation condition information. Nevertheless, our approach uses frequency domain information and achieves superior results. Our FGTI model surpasses the state-of-the-ar generative imputation models in various cases, thanks to the incorporation of high-frequency and dominant- freeuuucy condition information.
Moreover, since missing data are usually associated with the environment in reality, consider two additional typical missing data injection mechanisms following the same line of the existing study 33], i.e., missing at random (MAR) 53 and missing not at random (MNAR) 45 Specifically, the probability of missing data in MAR is higher when the temperature reading is low in the KDD dataset. the other hand, in the MNAR scenario, there a higher probability of missing data during the periods when the reading of each feature is lower. Figure and Figures the corresponding imputation results. One can find that the imputation results under different missing mechanisms are relatively similar, and our FGTI achieves optimal performance consistently. This result demonstrates that FGTI can handle missing data in various missing scenarios.
g in Appendix A.4.1 show
4.3 Ablation Analysis
We explore the effect of different elements in our FGTI on imputation performance through the following four ablation scenarios. (1) w/o Cross-domain: No extra condition representation is provided for cross-domain representation learning frameworks, where the fusion processes degrade to the standard self-attention. This scenario is used to validate the role of cross-domain representation for imputation. (2) w/o Frequency condition: In this scenario, the frequency -oominn information absent, and only the observations in the time-domain are utilized as the condition, to investigate the impact of the requency-domaal dominant- dominant-frequency filter from the structure to observe its impact on data imputation performance. (4) w/o fighefryyuency filter: This case exemplifies how crucial the high- seghffreyuency filter is, by removing from the pipeline.
information. (3) w/o cominant-frequene
filter: Remove the
Based on the results presented in Table it is evident that cross-domale representation learning frame- works and the two types of siquenccy-domain information are crucial in the process of imputation, which verifies the necessity of each component in our FGTI.
4.4 Resource Consumption
We present the resource consumption results of different methods in Figure 5 It can be observed that the running time of FGTI is roughly at the same level as other inffusiio-based methods. The overall

---

TimeCIB
BTMF
SAITS
LaST
CRIN
GAIN
(a)
20000
Second
10000
M1B
10000
5000
GPU Memory Usage
Running Time
Figure 5: Resource consumption over KDD dataset with 10% missing values
Incomplete
TIDER
NN TST
TimesNet LaST
277
FreIs GRIN
TimeCIB
CSDI
PriSTI
MEAN
BRITS
SAITS
GAIN
SSSD
B2C
FGTI
BTMF
(a)
(b)
0.85
RMSE
0.7
0.80
0.6
0.70
0.65
0.5
0.60
Air quality prediction
Mortality forecast
Figure 6: Application results of air quality prediction over KDD dataset and mortality forecast over Physionet dataset
resource consumption of FGTI, implemented based on the diffusion model, is slightly higher than that of CSDI, due the inclusion of high-frequency information and dominant-frequency information. However, since FGTI can achieve better imputation results than other methods, as shown in Table argue that it is acceptable incur such extra resource consumption.
4.5 Application Study
To validate the effectiveness of applying imputation in real-world downstream applications, we con- sider air quality prediction and mortality forecasting tasks. For the air quality prediction application, we first impute real-world missing data in the KDD dataset by various imputation methods. Then, we analyze the records in the previous twelve hours and the AdaBoost regressor
(37] to estimate the average PM2.5 concentration for the upcoming six hours. Figure ((a) shows that our method achieves the highest improvement in the air quality prediction task. Figure reports the mortality forecast performance over the PhysioNet dataset. We train the MLP classifier (37 to forecast the mortality on the data without/with imputation. As shown, FGTI achieves the best performance again, which verifies the applicability of our work. Notably, various imputation methods provide a noteworthy and favorable impact on the forecast task, which demonstrates the necessity imputation.
5 Conclusion
In this paper, we study imputing incomplete multivariate time series data, through reducing the imputation error in the residual term. By effectively incorporating frequency-domain insights into the generative framework, our FGTI surpasses existing models by capturing gigh-freeuency infor- mation and dominant- frreuuunttfrrreccyyyy information. The introduced cross-domain representation learning frameworks further enhance its capability to handle time and attribute dependencies. Comprchensive experimental evaluations over real-world incomplete datasets demonstrate the superiority of FGTI in both the imputation accuracy and the improvement of downstream applications.
Acknowledgements
This work is supported in part by the Fundamental Research Funds for the Central Universities. Nankai University (63231 147), the National Natural Science Foundation of China (62302241, 62306085, 62372252, 72342017), Shenzhen College Stability Support Plan (GXWD20231 1300022022011111111290000000
10

---

References
[2] Juan Miguel Lopez. Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. TMIR, 2023.
[3] Mehran Amiri and Richard Jensen. Missing data imputation using fuzzy-rough methods. Neurocomputing, 205:152-164. 2016.
[4] Kasun Bandara, Rob Hyndman, and Christoph Bergmeir. Mstl: A seasonal-trend decomposi- tion algorithm for time series with multiple seasonal patterns. abs/2107.1345
[5] Philip Bohannon, Michael Flaster, Wenfei Fan, and Rajeev Rastogi. A cost-based model and effective heuristic for repairing constraints by value modification. In SIGMOD, 2005.
[6] Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. BRITS: bidirectional recurrent imputation for time series. In NeurlPS, 2018.
[7] Tianfeng Chai and Roland R Draxler. Root square error (rmse) mean absolute error (raguetas against avoiding rmse in the literature. Geoscientific model development, 7 (3):1247-1250, 2014.
[8] Zhengping Che, Sanjay Purushotham, Max Guangyu Li, Bo Jiang, and Yan Liu. Hierarchical deep generative models for multi-rate multivariate time series. In ICML, 2018.
[9] Xinyu Chen and Lijun Sun. Bayesian temporal factorization for multidimensional time series prediction. IEEE TPAMI, (469))4659944733 2022.
[10] Xinyu Chen, Yixian Chen, and Zhaocheng He. Urban traffic speed dataset of guangzhou, china, ctteooddd..12522200 1205229
2018. UAL/attpss//oo...rg//00...//2//3
[11] MinGyu Choi and Changhee Lee. Conditional information bottleneck approach for time series imputation. ICLR, 2024.
[12] Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the g_ap_s: Multivariate time series imputation by graph neural networks. In ICLR, 2022.
[13] Robert B Cleveland, William S Cleveland, Jean McRae, and Irma Terpenning. Stl: A seasonal-trend decomposition. Off. Stat, 6(1):3-73, 1990.
[14] Wenjie Du, David Côté, and Yan Liu. SAITS: self-attention-buet based imputation for time series. ESWA, 219:119619 2023.
[15] Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. GP-VAE: deep probabilistic time series imputation. In AISTATS, 2020.
[16] David S Fung. Methods for the estimation of missing values in time series. 2006.
[17] Md Kamrul Hasan, Md Ashraful Alam, Shidhartho Roy, Aishwariya Dutta, Md Tasnim Jawad, and Sunanda Das. Missing value imputation affects the performance of machine learning: review and analysis of the literature 22010-2021). Informatics in Medicine Unlocked, 27: 100799, 2021.
[18] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross-attention control. In ICLR, 2023.
[19] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abs/2207.12598.
[20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[21] Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and João Carreira. Perceiver: General perception with iterative attention. In ICML, 2021.

---

[22] Shawn R. Jeffery, Minos N. Garofalakis, and Michael J. Franklin. Adaptive cleaning for RFID data streams. In VLDB, 2006.
[23] Mehmed Kantardzic. Data mining: concepts, models, methods, and algorithms. John Wiley & Sons, 2011.
[24] Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In ICLR, 2021.
[25] Roderick JA Little and Donald B Rubin. Statistical analysis with missing data, volume 793. John Wiley & Sons, 2019.
[26] Bing Liu and Huanhuan Cheng. Financial time series classification method based on low- frequency approximate representation. Engineering Reports, page e12739, 2023.
[27] Mingzhe Liu, Han Huang, Hao Feng, Leilei Sun, Bowen Du, and Yanjie Fu. Pristi: A conditional diffusion framework for spatiotemporal imputation. In ICDE, 2023.
[28] Shuai Liu, Xiucheng Li, Gao Cong, Yile Chen, and Yue Jiang. Multivariate time-series imputation with disentangled temporal representations. In ICLR, 2023.
[29] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, and Xiaojie Yuan. Multivariate time series imputation with generative adversarial networks. In NeurlPS, 2018.
[30] Yonghong Luo, Ying Zhang, Xiangrui Cai, and Xiaojie Yuan. E²gan: End-to-end generative adversarial network for multivariate time series imputation. In IJCAI, 2019.
[31] Pierre-Alexandee Mattei and Jes Frellsen. MIWAE: deep generative modelling and imputation of incomplete data sets. In ICML,
[32] Donna Katzman McClish. Analyzing portion of the curve. Medical decision making, 9()): 190-195, 1989.
[33] Xiaoye Miao, Yangyang Wu, Lu Chen, Yunjun Gao, Jun Wang, and Jianwei Yin. Efficient and effective data imputation with influence functions. VLDB, 2021.
[34] Xiaoye Miao, Yangyang Wu, Jun Wang, Yunjun Gao, Xudong Mao, and Jianwei Yin. Generative semi-supervised learning for multivariate time series imputation. In AAAI, 2021.
[35] Ahmad Wisnu Mulyadi, Eunji Jun, and Heung-Il Il Suk. Uncertainty- imputation network for clinical time series. IEEE TCYB, 52(9):9884-699 2022.
[36] Alfredo Nazábal, Pablo M. Olmos, Zoubin Ghahramani, and Isabel Valera. Handling incomplete heterogeneous data using vaes. PR, 107:107501, 2020.
[37] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vander- Plas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay, Scikit-learn Machine learning in python. JMLR, 12:2825-2830 2011.
[38] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron C. Courville. On the spectral bias of neural networks. In ICMI., 2019.
[39] Xiaobin Ren, Kaiqi Zhao, Patricia J. Riddle, Katerina Taskova, Qingyi Pan, and Lianyan Li. DAMR: dynamic adjacency matrix representation learning for multivariate time series imputation. In SIGMOD, 2023.
[40] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
[41] Alex Rubinsteyn and Sergey Feldman. fancyimpute: An imputation library for python, 2016. URL http:: commiskanddr/Eaccyinnann
12

---

[42] Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predicting in- hospital mortality of icu patients: The physionet/computnng in cardiology challenge 2012. In 2012 Computing in Cardiology, 2012.
[43] Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keiil. Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. In NeurIPS, 2020.
[44] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. CSDI: conditional score-based diffusion models for probabilistic time series imputation. In NeurlPS, 2021.
[45] Bhekisipho Twala. An empirical comparison of techniques for handling incomplete data using decision trees. AAI, 23(5):333-405, 2009.
[46] Joram van Driel, Christian NI. Olivers, and Johannes Fahrenfort. High-pass filtering artifacts in multivariate classification of neural time series data. Journal of Neuroscience Methods, 352: 109080, 2021.
[47] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
[48] Jun Wang, Wenjie Du, Wei Cao, Keli Zhang, Wenjia Wang, Yuxuan Liang, and Qingsong Wen. Deep learning for multivariate time series imputation: A survey. abs/2402.04059.
[49] Xu Wang, Hongbo Zhang, Pengkun Wang, Yudong Zhang, Binwu Wang, Zhengyang Zhou, and Yang Wang. An observed value consistent diffusion model for imputing missing values in multivariate time series. In KDD, 2023.
[50] Zhiyuan Wang, Xovee Xu, Weifeng Zhang, Goce Trajcevski, Ting Zhong, and Fan Zhou. Learning latent seasonal-trend representations for time series forecasting. In NeurIPS, 2022.
[51] Rebecca E. Wilson, Idris A. Eckley, Matthew A. Nunes, and Timothy Park. A wavelet-based approach for imputation in nonstationary multivariate time series. Stat. Comput., 2021.
[52] Haixu WU, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In ICLR, 2023.
[53] Jing Xia, Shengyu Zhang, Guolong Cai, Li Li, Qing Pan, Jing Yan, and Gangmin Ning. Adjusted weight voting algorithm for random forests in handling missing values. PR. 69:52-60, 60, 2017.
[54] Qianxiong Xu, Sijie Ruan, Cheng Long, Liang Yu, and Chen Zhang. Traffic speed imputation with attentions and cycle-perceptual training. In CIKM, 2022.
[55] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He. Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. In NeurlPS, 2023.
[56) Jinsung Yoon, James Jordon, and Mihaela van der Schaar. GAIN: missing data imputation using generative adversarial nets. In ICMI. 2018.
[57] George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-base framework for multivariate time series representation learning. In KDD, 2021.
[58] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. ICML, 2022.
[59] Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In NeurlPS, 2023.
[60] Wei Zhuang, Jili Fan, Jiayu Fang, Wenxuan Fang, and Min Xia. Rethinking general time series analysis from frequency domain perspective. KBS, 2024.
13

---

A Appendix
A.1 Proof of Proposition 3.1
We use the conditional entropy in information theory to reflect the amount of uncertainty. For the reverse process in 20), the imputation target X k is specified as the condition, thus we
H X - -
po x
x
log po
[X-1 X² ax
to model the uncertainty of the reverse process of DDPM.
Similarly, we use H [ X xc) model the uncertainty of the reverse process of the CSDI 44 model, which only the observations the condition. For our FGTI, utilize H X -- X x CH, CD)
According to the property of the conditional entropy, we first have
H (X X S H 1
Using the definition of mutual information, we have
(x--1;;1) = H. X (x--1) (X1-1 (x)).
From Equation 12 know that I (11-1, x²) 0, we thus have
H (x x) < H (
According to the chain rule of the entropy, we have
H {x-- X x0)
H (X-- X²
1X xc) +11 (x: xo).
Then can get
H x Xt x
= c 8t-1 x:) + H Xt-1 x1) H (xc (x)) (x:)
= x*) + H X ,,- - HI (xo |x))
According the Equation 12 X 1 adds the noise one less time than X1, which indicating that X closer to the observations. Thus, we have
H ( xt) < (xc 1x:).
By substituting this, we can obtain
H (X'-11 1-1 X 1X < H1 (X (x:)
Following the same line, we can also derive that
H x
X x
CH C
< (111 X x²).
This result implies that adding CH and CD to the condition can simplify the distribution that the diffusion models need to learn by reducing the entropy. This simplification can lead to more efficient learning and improve the model's imputation performance by narrowing down the scope of the target distribution'ss 's randomness and making its outcomes more predictable.
14

---

A.2 Mathematical details of FGTI
A.2.1 Details of Equation 13
If the complete imputation target distribution (X ) is known, we can first get sampled complete
imputation target X g(X"). Following Equation 12
we can obtain X by
X = Va'x 1- + vI - a ~N(11)).
Similarly, we can also obtain X by
X 2 v a 1 NO..I).
Combining the above two equations, we get
X'= Va (Va'-Ix'-2 vI - a1-66-1)-1) v1 - a'e'
1a-' (Va'(1- at a- 1- v1 - ald)
Ase- ~ V(0,I), e ~ NOO, I). we can infer that Va - a ~ (0,0 a²-1))1),
vI- ~10,, -1--aa a²) I). Therefore, can get
X Va'at-ix'- +
aa'a--lat-22 2x + V1 a'aa''''''
x +
Il a'c
where e ~~ 10,
A.2.2 Details of the Parameterization
Combining the Bayes' theorem, we start by
C. CD) = po X X1-1 x CH CD) Po X
x CH, CH,CP)
Po PX
X c
=o
Po X XC,CH, CP)
According to Equation
12 the expected pa (X - x CH, CP) is
pe x --1 _ C CP) N(Va' a I')
Furthermore, by incorporating Equation
13 obtain
(x C C. CP) NV 20. Var-i)
( c CH CP) ~ ~N<VV Va)1)
where X is a virtual result that is expected to be obtained through Equation 14 Combining Equation we can parameterize it by X x + V1- - (t,'',
CH CP) where
Eo(.) outputs the predicted added noise ( based the parameter set 0. In other words, tt, X. X' X CH CD)
X
x v1 1/a''co
15

---

By merging the above three distributions, we can derive
Po (x X' x. CD & exp
1
X Va!Xi-122
(X vor-1x0)2
2
1 a
1 - a
(X' 18--Ve -aaxzg)2
1 a
1 a
1
(Xc1-1²²
exp
2
3
1 xt-1
2V
2Va'-130
81-1 *-1+.....
1|
+
8
1 -
where (X'-1)² denotes the inner product of X
On the other hand, from the probability density function of the Gaussian distribution, we can also obtain
1 (1-1
X
X X
CH CD
n1
Pp(X (-1 X X CHH CD) x exp
2
(al-1)2
22 2
exp
2
10
21
2
11-
X x CH CD
+
|o|-1)2
Therefore, get
a'
1
[ot-1]3
Br
1 1-
2p18
X
X
x
CH
CD]
2
a 2V
2
X!
c
ge
1 - x-11
We can first obtain the following result from the first equation as at
3r
(1
aa-1)1 ge
1 a
After that, we take it into the second equation
pg
X
X C C. CD
a1-
(1-x
a x9 30
(1 11-a-11)p
(1-a--1)1 1)8
As the virtual result X
1 X
v1 - (1, (1,8 x CH, cD)] ].
we have
V
Va -1--
aT-IBt gt
x
CH X
CD]
X
110
l-a¹
1- al
Va--
Vat-Igh gt
d
(,,X', CH, CH. cP)
1-a¹
Va--1
Xt
g
CH cB ( cD)
V
v1 - σt
16

---

Eirbedding
Denoising Network
puistion target
Darcill
MLP Pooedrr
Cross Attention Layer
Cross Amention Layer
MoE Midii
MUP
ReLu
Obsaroation condiion
MLP
Embetding
Tmmedamp length
Tranddornnt Enoozier Layer
High-rocuency conditoo C*
Concat
Posttio Erooding
Side Infor
ML.P
Concall
Embediing
Atmoote Number
Figure 7: Architecture of the Denoising Network cx(+) in FGTI
A.3
Implementation Details
A.3.1 Detailed Architecture of the Denoising Network
In this section, we present the detailed architecture the denoising network cØ(-) in FGTI model. As shown in Figure the input projector of the noise imputation target and the observation condition an MLP layer, and the output projector is a 2-layer MLP with ReLU activation function. For the encoder of the frequency-dome domain information, it is implemented by transformer backbone consisting of the position encoding layer and the transformer encoder layer.
A.3.2 Algorithms
Algorithm Training process FGTI implemented by the diffusion model
Input: Incomplete time series X, the number of diffusion step T
Output: Optimized denoising network (-)
1: repeat
2: X select observed values in X
3: to Uniform {1, ,,...,T)
4: ((O.I
5: X' X V1-ac
6: Perform Gradient Descent by VLg Vo volle- colle-eo 6o
XC CH
x
|
7:
until converged
In this section, provide the detailed training process of proposed FGTI model implemented by and the imputation process in Algorithm
the diffusion model Algorithm [
A.4
Supplemental Experiments
A.4.1 Missing Mechanisms
In this section, explore the imputation performance of the missing at random (MAR) 53 and missing not at random (MNAR) 45 missing mechanisms over the Guangzhou dataset and the PhysioNet dataset.
17

---

Algorithm 2 Imputation process of FGTI implemented by the diffusion model
Input: A incomplete time series sampleX, the number of diffusion step T, the optimized denoising network co(-)
Output: Filled missing values X
1: X < missing values X
2: X" ~ (0,1)
3: for t = , do
4:
ift then
5:
c~N(0,1)
6: else
7:
E+-
8:
end if
9:
X
Va
X 8¹ م'=e0
Vi-ai
(1,X',x, CH CH, co)
11- 1-a"
10: end for
As shown in Figure and Figure 9 FGTI consistently achieves optimal performance, demonstrating its ability to handle missing data in various scenarios.
MEAN
TIDER BRITS
TST SAITS
TimesNel
FreTS GRIN
TimeCIB GAIN
CSDI
PriSTI
BIMF
LaST
SSSD
HP FCTI
(a)
0.50
RMSE
0.5-
MAE
0.25
0.0
0.00
MCAR
MAR
MNAR
MCAR
MAR
MNAR
Figure Varying the missing mechanism over Guangzhou dataset with 10% missing values
MEAN
IIDER BRITS (a)
N
TST SAITS
TimesNet
FreTs
TimeCBB
FA
CSDI
INT
PriSTI
BTMF
LaST
GRIN
GAIN
SSSD
LEL FCTI
1.0
RMSE
MAE
0.5
0.5
0.0
0.0
MNAR Figure 9: Varying the missing mechanism PhysioNet dataset with 10% missing values
MCAR
MAR
MNAR
MCAR
A.4.2
Hyperparameter Evaluation
In this section, we perform parameter sensitivity experiments on critical hyperparameters: the cutoff frequency of the dominant- fonquent-frequency filter.
filter and the maximum magnitude frequency number of the
Effect of the cutoff frequency F. Figure 10 shows the imputation results with various cutoff frequencies F of the high-frequency filter. It can found that if F is too small, the model may not be able to accurately capture the figh-freuuency information necessary for guiding the imputation of the time series residual term. The reason that the high-frequency filter output may include too much low-frequency information. Conversely, if F is large, the model cannot obtain enough high-frequency information to guide the imputation of the residual term.
Effeet of the maximum magnitude frequency number k. We investigate the imputation results when adjusting the maximum amplitude frequency number x used for the dominant-frequeng, filter Figure shown in the figure, the imputation model cannot perform well with a small s. This because the dominant-frequeet filter cannot obtain enough smoothing information, which causes
18

---

KDD
+ Guangzhou
PhysioNet
(b)
0.7
0.39
RMSE
0.28
0.42
0.16
0.40
0.24
0.15
0.23
0.1
0.2
0.3
0.4
0.1
0.2
0.3
0.4
Figure 10: Varying the cutoff frequency F of the high high-frrquency filter with 10% missing values
KDD
Guangzhou
PhysioNet
(a)
(b)
0.7
0.6
0.35
RMSE
0.25
0.42
0.16
0.40
0.24
0.15
0.23
5
10
15
20
10
15
20
K
Figure 11: Varying the number of maximum magnitude frequency x of the dominant-frequente filter with 10% missing values
high-frequency signals interfere with the imputation of the trend and seasonal terms of the time series. On the contrary, if is too large, it can cause some high-frequency information to mix with the output condition of the cominant-frequent obtaining background structure information needed for imputing the trend and seasonal terms. As result, the imputation result for extreme cases may not be optimal.
filter. This can prevent the model from effectively
Based on the experimental results, we set the cutoff frequency F of the figh-frequency filter to 0.3 and set the number of maximum magnitude frequency к of the dominant -rrequent-freuueccy filter 10. In addition, for other settings related to the diffusion model, we adopt hyperparameters recommended by the existing establssbl models 44 24]. These models have demonstrated strong performance in similar tasks, and their hyperparameters have been extensively validated in the paper.
A.4.3 Imputation Target Select Strategies
For training the denoising network, randomly select some observed values the imputation target. In this process, the mask ratio and mask pattern to get the imputation target directly determine the effectiveness of training. Thus, in this section, explore the performance of FGTI with different mask ratios and mask patterns.
Effect of Mask Ratio We first mask different ratios of observations as the imputation target for training, the performance of FGTI with different mask ratios is shown in Table 3 In addition, we consider a special case where observations with different ratios are randomly masked as imputation targets at each training step, instead of using a fixed masking ratio.
We can find that since the random ratio mask strategy can increase the learning complexity and enhance the modeling ability of the diffusion model, the random ratio mask strategy achieves optimal or sub-optimal performance in cases. So use the random ratio mask strategy by default.
Effeect of Mask Pattern Then we explore the performance when using different mask patterns. Following CSDI [4) and PriSTI (27). we consider three mask pattern strategies: (1) Block missing (2) Mix missing (3) Random missing. The results is shown in Table 4
19

---

Table 3: Varying the mask ratio of the imputation target when training the denoising network with 10% missing values
Mask Ratio
KDD
Guangzhou
PhysioNet
RMSE
MAR
RMSF
MAE
RMSE
MAE
10%
0.4372
0.1925
0.2388
0.1647
0.5992
0.3235
20%
0.4143
0.1700
0.2312
0.1576
0.5853
0.2893
30%
0.4257
0.1707
0.2335
0.1600
0.5836
0.2800
40%
0.4185
0.1697
0.2372
0.1634
0.6181
0.3105
50%
0.4183
0.1714
0.2343
0.1578
0.6308
0,3138
Random Ratio
0.4057
0.1489
0.2325
0.1584
0.5801
0.2856
Table 4: Varying the mask pattern of the imputation target when training the denoising network with 10% missing values
Mask Pattern
KDD
Guangzhou
PhysioNet
RMSE
MAR
RMSE
MAE
RMSE
MAF
Block missing
0.4187
0.1778
0.2387
0.1596
0.6224
0.3384
Mix missing
0.4193
0.1792
0.2325
0.1531
0.6034
0.3208
Random missing
0,4057
0.1489
0.2325
0.1584
0,5801
0.2856
It can be found that Block missing or Mix missing strategy is not comparable to Random missing in most cases due to the possibility that the mask pattern may not correspond to the actual missing scenario. So we use the Random missing mask pattern by default.
A.5 Comparative Experiments of Generative Baselines
To compare the imputation performance of FGTI with probabilistic generative baselines in more detail, we adopt CRPS 44 to evaluate the gap between the learned and ground truth distributions for different probabilistic generative methods following [4] 27).
First, we inject different rates of missing values by the MCAR mechanism, and report the CRPS
performance with different missing rates in Table 5 Then we report the CRPS by varying the missing
mechanism with 10% missing values in Table 6
We can find that our method outperforms other probabilistic generative methods for all cases due to the introduction of frequency-domit conditions, thus providing empirical evidence for Proposition 3.1 It can be also found that the variations of CRPS are basically the same as RMSE and MAE for a specific model with different settings.
A.5.1 Case Study
In order to verify the role of the high-frequency condition and the dominant-frqquency condition, in this section we conduct case study of FGTI for trend, seasonal, residual term over the pre- decomposed KDD dataset.
We first perform STL decomposition of the KDD dataset into Trend, Scasonal and Residual terms. Then we select 10% observations of the original KDD dataset as the mask positions by MCAR, and then mask the corresponding positions of the three terms. Finally we imputation the missing values in the three terms separately and report the performances. Note that this setup is the same as the survey experiment shown in Figure D in Section
To study the role of high-frequee frequency information, dominant- freeuunn-- information, and frequency- domain information, we consider the three ablation scenarios (1)w/o Dominant-frequeng filter (2) w/o High-frequency filter and (3) w/o Frequency condition in Section 4.3
We report the imputation results of different scenarios over different terms in Table
20

---

Table 5: CRPS of various probabilistic generative methods with different missing rates
Dataset
Missing Rate
TimeCIB
GAIN
CSDI
SSSD
PriSTI
FGT1
KDD
10%
0.466
0.709
0.224
0.352
0.232
0.158
20%
0.467
0.718
0.245
0.370
0.248
0.170
30%
0.469
0.729
0.259
0.374
0.268
0.186
40%
0.471
0.746
0.278
0.401
0.301
0.216
Guang.
10%
0.360
0.692
0.265
0,316
0.209
0.155
20%
0.357
0.694
0.277
0.299
0.244
0.168
30%
0.356
0.695
0.292
0.353
0.310
0.193
40%
0.358
0.697
0.324
0,382
0.362
0.243
Phy.
10%
0.466
0.739
0.544
0.617
0.444
0.343
20%
0.467
0.761
0.589
0.665
0.457
0.369
30%
0.469
0.787
0.627
0.630
0.467
0,389
40%
0.471
0.814
0.671
0.676
0.491
0.441
Table 6: CRPS of various probabilistic generative methods with different missing mechanisms
Dataset
Miss mechanism
TimeCIB
GAIN
CSD1
SSSD
PriS11
FGI1
KDD
MCAR
0.466
0.709
0.224
0.352
0.232
0.158
MAR
0.470
0.710
0.229
0.489
0.239
0.164
MNAR
0.490
0.715
0.244
0.456
0.252
0.174
Guang.
MCAR
0.360
0.692
0.265
0.316
0.209
0.155
MAR
0.298
0.692
0.252
0.367
0.208
0.148
MNAR
0.294
0.693
0.251
0,267
0.210
0.144
Phy.
MCAR
0.466
0.739
0.544
0.617
0.444
0.343
MAR
0.593
0.739
0.550
0.724
0.454
0.356
MNAR
0.608
0.743
0.566
0.715
0.476
0.366
We can find that for the trend term, retaining the dominant-frequency condition gives the best results, while the high-frequency condition may interfere with the imputation. For the seasonal term, the results are similar the trend term, but the dominant-frequency information contributes less to the imputation for the seasonal term than for the trend term. This suggests that the seasonl term mainly corresponds the dominant- domquent--requeeey information, but also contains some of the high-frequency information. In contrast, the results of the experiments on the residual term show that the residual in
term mainly corresponds gigh-freqeency condition. Since choose the transformer the encoder
Representation Learning and utilize cross-attention as the fusion mechanism of the two reeqeencydooain conditions in time-frequency representation learning and aatribute-frequene representation learning modules, our method can self-adaptively adjust the weights of the high- frequency information and the cominant-frequent information for different timestamps. Thus method can outperform existing methods for datasets with multiple circumstances in most cases, as illustrated in Table
A.5.2 Visualizations
To showcase the imputation results of our FGTI model, we visualize the results with the state-of-the art imputation methods CSDI and PriSTI in Figure 12 and Figure 13 We can find that the CSDI imputation results are not quite accurate for some fast-changing points due to the lack of sufficient condition guidance. On the other hand, both FGTI and PriSTI produced more accurate imputation results because they both used additional conditions. However, as shown in Tabel [ FGTI still yields better results than PriSTI, suggesting that the high-frequency information and the dominant-frean dominant-frequency information we use are more superior to the interpolation information used by PriSTI.
A.6 Societal Impact Statement
The development our FGTI imputation model could have significant positive impact various sec- tors including healthcare, finance, and environmental monitoring. In healthcare, improved imputation models can lead to more accurate health monitoring systems, enabling early detection and treatment
21

---

Table 7: Imputation results for the trend, seasonal and residual terms of KDD dataset with 10% missing values
Component
Trend
Seasonal
Residual
RMSE
MAE
RMSE
MAE
RMSE
MAE
w/o Frequency condition
0.0480
0.0155
0.0572
0.0364
0.5132
0.2975
Dominant- frequency filter
Domquant-frqquecyy
0.0482
0.0157
0.0533
0.0334
0.4956
0.2814
w/o High- sigh-frequency filter
0.0409
0.0143
0.0485
0.0301
0.5129
0.2912
FGTI
0.0448
0.0159
0.0523
0.0325
0.5068
0.2885
observation
ground truth values
CSDI
FGTI
Attribute
Attribute
Value
0.5
Valuo
Valuo
-1.0
160
180
150
160
170
190
170
180
190
Timestamp
Timestamp
Timestamp
Attribute 6
Attribute 66
Attribute 96
0.3
Valce
0.5
Valuue
Valuee
0.0
-0.5
1590
1600
1610
1620
1630
1590
1600
1610
1620
1630
1590
1600
1610
1620
1630
Timestamp
Timestamp
Timestamp
Attribute
Attribute 66
Attribute 96
2.0
1.5
1.0
1.0-
Value
1.0
0.5
0.5
0.0
0.0
-0.5
3170
3180
3190
3200
3210
3170
3180
3190
3200
3210
3170
3180
3190
3200
3210
Timestamp
Timestamp
Timestamp
Attribute
Attribute
Attribute
1.0
0.5
Valtee
Vallue
-0.5
4760
4770
4780
4790
4800
4770
4780
4790
4800
4760
4780
4790
4800
Timestamp
Timestamp
Timestamp
Figure 12: Imputation results visualization compared with CSDI for KDD dataset with 10% missing values.
of conditions by filling gaps in patient data. This can ultimately improve patient outcomes and reduce healthcare costs. In the financial sector, enhanced time series imputation provide better forecasts and risk assessments, aiding in more informed decision- malssiinmmakingg.. and potentially stabilizing markets by decreasing uncertainty. Environmentally, better data imputation improve weather prediction models and climate monitoring systems, aiding in disaster readiness and enhancing our ability to address climate change.
However, the deployment of advanced imputation models also raises certain concerns. If used in sensitive areas like surveillance, these models could lead to privacy invasions by reconstructing
22

---

observation
ground truth values
PriSTI
FGTI
Attribute
Attribute 66
Attribute 96
1.0
Valion
-1.2
150
160
170 Timestamp
180
190
150
160
170 Timeslamp
180
190
150
160
170
180
190
Timestamp
Attribute
Attribute
Attribute
1.0
Value
0.5
Vallue
Vallue
-0.5
1600
1620
1590
1600
1610
1620
1630
1590
1600
1620
1630
Timestamp
Timestamp
Timestamp
Attribute
Attribute
Attribute 96
1.0
1.0-
Vallua
0.0
W
0.0
3190 Timestamp
3210
3200
3200
3210
Timestamp
Timestamp
Attribute 6
Attribute 66
Attribute 96
Value
Maluus
Vicuo
0.5
0.0
4760
4770
4780 Timestamp
4790
4800
4760
4770
4760
4790
4800
4760
4770
4780
4790 4800
Timestamp
Timestamp
Figure 13: Imputation results visualization compared with PriSTI for KDD dataset with 10% missing values.
missing incomplete data track individuals without consent. In financial markets, sophisticated imputation methods could also exacerbate inequality by disproportionately benefiting institutions with the resources to leverage state-of-the-an technology, potentially leading greater market dominance. Additionally, reliance on automated data imputation may result in complacency, where errors in imputation models propagate unnoticed, leading to decisions based on inaccurate misleading data.
To mitigate negative impacts of our imputation model, implement stringent data privacy laws, and ethical guidelines, provide equal access to technology resources across entities, and establish rigorous validation processes to ensure accuracy and fairness. Regular auditing and transparency in algorithm deployment also play critical role.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of this work in terms of resource consumption in Section 4.4
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model cell-speciiiiatinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
24

---

Answer: [Yes]
Justification:
We include the theoretical proof in Appendix A.1
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We include the detailed experimental settings in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
25

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The anonymous source code and datasets are available online 1.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ips.cc//pill wttpcccpppppppppeeeeeeeeeeeeeeee httpa///guddeee/eeeeabiiissssss /Cc////oddiiisssseeeeeee ssiiiiiiccyyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We include the detailed experimental settings in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: seeds.
We report the results averaged from five experiments with different random
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
26

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the necessary computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: In every respect in the paper, we follow the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include the impact statement in Appendix A.6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All data, models, and code in the paper respect the license.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "7LmUCl1XoB", "reviewer": "Reviewer_WWjA", "summary": "The authors present Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), a model which addresses the challenge of missing data in multivariate time series by focusing on the often-overlooked residual term. The paper also incorporates frequency-domain information to enhance imputation performance. Experiments show that this outperforms many existing time series imputation baselines on three real-world datasets.", "questions": "Please see Weaknesses.", "limitations": "1.\tMy main concern with the work is that the contribution of the proposed method might be incremental. Compared with CSDI, frequency domain filters are utilized to incorporate frequency domain information, and further cross-domain representation learning and frequency-aware diffusion framework are natural implementations, which lack additional novelty.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The work proposes an approach for time series imputation, which is an important topic for a wide range of applications.\n\nS2. The authors aim to address the influence of frequency-domain information, w.r.t, both high-frequency condition and dominant-domain condition.\n\nS3. Extensive experiments have been carried out.", "weaknesses": "W1. In Introduction and Section 3.1.2, it shows less explanation on why frequency components with large amplitudes could guide both the imputation of trend and seasonal terms. Did it address the intricately entangled trend-seasonal representations, which is highly important in current approaches?\n\nW2. In the section of Related Work, I did not find discussions regarding existing research work and this study in frequency domain for time series tasks, this would be hard to clearly justify the main difference/novelty of this paper compared with current approaches.\n\nW3. In the evaluation, only RMSE and MAE are used as metrics. However, it would be better to include additional metrics such as CRPS.\n\nW4. It would be fairer if the authors compared their approach with a time series model using frequency domain information, which also addresses the trend, seasonal, and residual terms.\n\nW5. Both CSDI and PriSTI address the issue of missing strategies. From the results of Figure 4, 7 and 8, the improvement of MAR and MNAR was much lower than that of MCAR on the Guangzhou and Physio datasets. Was it due to differences in the data distribution, or was it due to the setting of the hyper-parameters?\n\n[1] Learning Latent Seasonal-Trend Representations for Time Series Forecasting.\n[2] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting\n[3] CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation\n[4] PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and valuable suggestions. Below, we provide our response to the questions and concerns.\n\n**1. (W1) Why large amplitude frequency components guide trend and seasonal term imputation**\n\nThank you for your comments.\nWe recognise that our current expression is insufficient.\nIn STL decomposition, the dominant frequency components(with large amlitude) typically correspond to the trend and seasonal terms, while the high-frequency information is often associated with the residual term [a].\nThis is because trend captures long-term movements, and seasonal captures major period patterns.\nWe also add an empirical evidence on the pre-decomposed KDD dataset in following table:\n| Component                     | Trend      |            | Seasonal   |            | Residual   |            |\n|-------------------------------|------------|------------|------------|------------|------------|------------|\n|                               | RMSE       | MAE        | RMSE       | MAE        | RMSE       | MAE        |\n| w/o Frequency condition       | 0.048      | 0.0155     | 0.0572     | 0.0364     | 0.5132     | 0.2975     |\n| w/o Dominant-frequency filter （Preserving high-frequency information） | 0.0482     | 0.0157     | 0.0533     | 0.0334     | **0.4956** | **0.2814** |\n| w/o High-frequency filter （Preserving dominant-frequency information）    | **0.0409** | **0.0143** | **0.0485** | **0.0301** | 0.5129     | 0.2912     |\n| FGTI                          | 0.0448     | 0.0159     | 0.0523     | 0.0325     | 0.5068     | 0.2885     |\n\nThis suggests that the Trend term mainly corresponds to the dominant-frequency information, the Seasonl term mainly corresponds to the dominant-frequency information, but also contains some of the high-frequency information, the Residual term mainly corresponds to high-frequency information.\nSince we choose transformer as the encoder and utilize cross-attention as the fusion mechanism of the two frequency domain information, our method can self-adaptively adjust the weights of the high-frequency information and the dominant-frequency information for the characteristics of the three terms in different datasets.\n\nAnd we will add the content of the above expressions in Section.1 and Section 3.1.2.\n\n**2. (W2) Discuss existing frequency domain research for time series tasks**\n\nThank you for your valuable suggestions for improving the quality of the manuscript.\nOur current Related work section mainly focuses on the currently state-of-the-art time series imputaiton methods.\nFor the time series imputation methods in frequency domain, mvLSWimpute [b] utilizes wavelet transforms to guide imputation, APDNet [c] uses the Fourier Temporal and Fourier Variable Interaction modules to model dependencies.\nIn addition, the frequency domain time series forecasting methods FEDformer [d], FreTS [e] can also be applied to imputation task.\nHowever, they did not consider how to use frequency domain information to accurately model the residual terms of the missing data,\nwhich is critical for boosting the overall imputation performance.\nIn contrast, our FGTI captures high-frequency information and dominant-frequency information to get a more accurate modeling of the residual term, while assisting in describing trend and seasonal terms.\n\nWe will add to the discussion on imputation methods in frequency domain to the related work section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LNmtIHXvnc", "reviewer": "Reviewer_qcDA", "summary": "The paper proposes a new model, called FGTI, to address the issue of missing data in multivariate time series extracting frequency-domain information. The authors argue that existing methods, in general, neglect the residual term, which is the most significant contributor to imputation errors. FGTI incorporates frequency-domain information using high-frequency and dominant-frequency filters to improve the imputation of the residual, trend, and seasonal terms. The model is evaluated on three real-world datasets and demonstrates superior performance in both imputation accuracy and downstream tasks compared to existing methods.", "questions": "- Regarding Weakness 2: Please describe why the proposed method works better than conventional methods and under what circumstances. Do the seasonality and trend profiles of the given time-series dataset matter, and if so, how?\n\n- It seems that how to mask the input (to learn the underlying diffusion model) is very important. How does the model's performance change depending on the masking ratio and/or masking patterns? This is crucial as masking may distort the trend and seasonality components of the given time-series.", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-written and well-motivated.\n- The authors tackle a very important problem of time-series imputation, focusing on fully utilizing frequency-domain information, which has not been considered enough.\n- The experiments are thorough, including ablation studies and different missing scenarios. The method is compared with cutting-edge baselines.", "weaknesses": "- The experiments are thorough and include multiple state-of-the-art imputation methods. However, the paper misses two important time-series imputation methods: one with a VAE-based approach [A] and one applying trend-seasonality decomposition [B]. While it's understandable that the authors might not have had enough time to include these papers since they are from ICLR 2024, technical comparisons with these methods would be necessary before acceptance.\n\n- While the authors provided experiments with various missing scenarios and missing rates, it is not clear why the proposed method outperforms the conventional methods and under what circumstances. For instance, extracting high-frequency information could be significantly distorted if missingness occurs with a certain frequency (which seems realistic considering periodic noises). The paper would be more convincing if the authors provided experiments that assess the effect of the trend and seasonality components of the datasets (or using synthetic datasets) and missing patterns.\n\n- The experiments would be more appealing if the authors provided a breakdown (seasonality and trend) of the assessments on the imputed time-series.\n\nMinor comment:\n\n- Regarding Figure 1, since the RMSE and MAE comparison essentially conveys similar information about the population-level importance of the trend-seasonality decomposition, replacing one with specific time-series examples where the trend and seasonality of the missing values are correctly imputed would better illustrate the motivation. Also, information about how Figure 1 is reported is missing.\n\nReferences:\n\n[A] Choi and Lee, \"Conditional Information Bottleneck Approach for Time Series Imputation, ICLR 2024.\n[B] Liu et al., \"Multivariate Time-series Imputation with Disentangled Temporal Representations,\" ICLR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the constructive comments and suggestions for further experiments. We will try our best to incorporate the suggestions in the revised version.\nBelow, we provide our response to the questions and concerns.\n\n**1. (W1)  Compared with the most recent baselines**\n\nThank you for the valuable suggestions. \nTimeCIB [a] is a very competitive VAE-based imputation method, so we incorporate TimeCIB into the baselines for comparative experiments.\nIn addition, we have already considered TIDER [27] in our experiments.\n\nWe first report the part of updated Table 1 for the added baseline:\n| 　     | Miss. Rate | Metric |  FreTS |    FGTI    |\n|--------|------------|--------|:------:|:----------:|\n| KDD    | 10%        | RMSE   | 0.630  | **0.406** |\n|        | 　         | MAE    | 0.412  | **0.149** |\n|        | 20%        | RMSE   | 0.741  | **0.451** |\n|        | 　         | MAE    | 0.489  | **0.161** |\n|        | 30%        | RMSE   | 0.796  | **0.448** |\n|        | 　         | MAE    | 0.546  | **0.176** |\n|        | 40%        | RMSE   | 0.850  | **0.478** |\n| 　     | 　         | MAE    | 0.591  | **0.205** |\n| Guang. | 10%        | RMSE   | 0.456  | **0.230** |\n|        | 　         | MAE    | 0.340  | **0.170** |\n|        | 20%        | RMSE   | 0.602  | **0.258** |\n|        | 　         | MAE    | 0.460  | **0.176** |\n|        | 30%        | RMSE   | 0.709  | **0.291** |\n|        | 　         | MAE    | 0.547  | **0.202** |\n|        | 40%        | RMSE   | 0.787  | **0.356** |\n| 　     | 　         | MAE    | 0.611  | **0.254** |\n| Phy.   | 10%        | RMSE   | 0.804  | **0.580** |\n|        | 　         | MAE    | 0.540  | **0.286** |\n|        | 20%        | RMSE   | 0.825  | **0.577** |\n|        | 　         | MAE    | 0.576  | **0.309** |\n|        | 30%        | RMSE   | 0.861  | **0.624** |\n|        | 　         | MAE    | 0.603  | **0.336** |\n|        | 40%        | RMSE   | 0.883  | **0.669** |\n| 　     | 　         | MAE    | 0.626  | **0.376** |\n\nWe then report the imputation results for varying missing mechanism with 10% missing values in the following table, and update Fig.7, Fig.8 and Fig.9:\n| 　     | Miss. Mech. | Metric | TimeCIB | FGTI       |\n|--------|-------------|--------|---------|------------|\n| KDD    | MCAR        | RMSE   | 0.589   | **0.406** |\n|        | 　          | MAE    | 0.367   | **0.149** |\n|        | MAR         | RMSE   | 0.589   | **0.406** |\n|        | 　          | MAE    | 0.367   | **0.149** |\n|        | MNAR        | RMSE   | 0.693   | **0.499** |\n|        | 　          | MAE    | 0.408   | **0.174** |\n| Guang. | MCAR        | RMSE   | 0.451   | **0.230** |\n|        | 　          | MAE    | 0.300   | **0.170** |\n|        | MAR         | RMSE   | 0.376   | **0.218** |\n|        | 　          | MAE    | 0.245   | **0.150** |\n|        | MNAR        | RMSE   | 0.327   | **0.200** |\n|        | 　          | MAE    | 0.230   | **0.140** |\n| Phy.   | MCAR        | RMSE   | 0.697   | **0.580** |\n|        | 　          | MAE    | 0.450   | **0.286** |\n|        | MAR         | RMSE   | 0.327   | **0.200** |\n|        | 　          | MAE    | 0.230   | **0.140** |\n|        | MNAR        | RMSE   | 0.697   | **0.580** |\n|        | 　          | MAE    | 0.450   | **0.286** |\n\nAfter this, we report the updated resource consumption over the KDD dataset with 10% missing values in the following table, and update Fig.5:\n|          | GPU Consumption Usage (MiB) | Running Time (s) |\n|----------|-----------------------------|------------------|\n| TimeCIB  | 930                         | 6.52             |\n| FGTI     | 9103                        | 3874.30          |\n\nFinally, we report on the updated downstream application experiments in the following table,\nand update Fig.6:\n|            | Air quality prediction (RMSE) | Mortality forecast (AUC) |\n|------------|-------------------------------|--------------------------|\n| TimeCIB    | 0.63                          | 0.82                     |\n| FGTI       | **0.59**                      | **0.86**                 |\n\nWe can find that FGTI still outperforms all baselines due to the guidance of high-frequency information and dominant-frequency information, as well as the generating ability of the diffusion model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qloQAUhZMB", "reviewer": "Reviewer_Q2t8", "summary": "This paper proposes a generative model, Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI), for multivariate time series imputation. The proposed model is designed to enhance the imputation performance by modeling the residual component of time series data. To this end, the paper leverages the observation that the residual components are usually high-frequent. It utilizes two filters in the frequency domain to extract high-frequency and dominant-frequency information from the time series using FFT and IFFT. The frequency information is then input into a denoising network to generate the missing data conditioned on the observed part of the time series. Experiments are performed using three real-world datasets and a number of recent baselines are benchmarked.", "questions": "Please kindly see and clarify my comments in the Weaknesses section.", "limitations": "Limitations of high demand of computational resources are discusses with relevant experiment results analyzed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The idea of using frequency filters to model the residual component of time-series data is natural and reasonable.\n- The empirical evaluation is systematic and the experiment results show consistent improvement compared with existing methods.", "weaknesses": "- It is not clear if the modeling of temporal dependency is sufficient in the time-frequency and attribute-frequency representation learning module. From Eq. (8-9), it seems that only two attention layers are used without considering the time stamp.\n- The model architecture and specification used for Projectors in the denoising network is not described.\n- There is a gap between Sections 3.2 and 3.3. The time-frequency representation and attribute-frequency representation learning are described and the final output $\\mathbf{R}^a$ is defined. However, it is not clear how this representation is used in the diffusion model. It seems from Eq. (14) and Appendix A.2.1 that $\\mathbf{R}^a$ is not involved in the parameterization $\\epsilon_\\theta$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your thoughtful comments and valuable suggestions. We have incorporated the suggestions in the revised paper. Below, we provide our response to the concerns.\n\n**1. (W1) Clarify if temporal dependency modelling is sufficient**\n\nWe would really appreciate your feedback.\nWe apologize for the inadequate illustrations and explanations of the time-frequency and attribute-frequency representation learning modules\nIn line 144, we state that the Encoder(.) is implemented by the transformer backbone.\nSince a position encoding layer exists in this encoder, we can add the timestamp information for the corresponding Query and Key in the two representation learning modules.\nTo improve the presentation, **we add a figure illustrating the detailed architecture of the denoising network, which is uploaded in the supplementary pdf file at the top of the page**.\nAnd we will add this figure to Section 3.3.\nIn addition, the setup where modelling temporal dependencies than modelling attribute dependencies by two attention layers is common in time series imputation methods [26,43].\nConsidering that our method outperforms other imputation methods for different missing rates and different missing mechanisms in Table 1, Fig. 4, Fig. 7 and Fig.8, the modelling of the time dependence is sufficient.\n\n**2. (W2) Describe the model architecture and specifications used for Projectors in the denoising network**\n\nThank you for your suggestions.\nWe apologize for the inadequate description and explanation of the detailed structure of the denoising network.\nAs shown in Fig.2, which is uploaded in the supplemental pdf, the input projector is an MLP layer, and the output projector is a 2-layer MLP with a ReLU activation function.\nThis setup is consistent with the other SOTA imputation methods [26,43], which have been extensively validated in those literature.\n\n**3. (W3) Clarify how time-frequency and attribute-frequency representations are used in the diffusion model**\n\nThank you for your valuable suggestions for boosting our manuscript.\nWe apologise for not having a very clear representation of the denoising network architecture of the diffusion model.\nThe Fig.2 in supplemental pdf illustrating the detailed architecture of the denoising network will add in the Section 3.3.\nTo achieve the imputation of generative models that are helped by the guidance of frequency-domain information, we should incorporate the representation learning modules into the generative model architecture.\nAs shown in the figure, $\\mathbf{R}^{a}$ is a hidden vector in the denoising neural network of the diffusion model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R7wJqFGqXB", "reviewer": "Reviewer_SUMs", "summary": "This paper proposes FGTI that uses frequency-domain information with high-frequency and dominant-frequency filters for accurate imputation of residual, trend, and seasonal components. \nExperimental results demonstrate FGTI's effectiveness in improving both data imputation accuracy and downstream applications.", "questions": "1. In Table 2, the absence of the High-frequency filter appears to impact performance less on KDD and Guangzhou datasets. However, on PhysioNet, the situation is reversed. What factors might explain this discrepancy?\n\n2. How were the hyper-parameters for the experiments chosen? Could you elaborate on the methodology used for their selection?\n\n3. What is the purpose of Proposition 3.1? It seems unnecessary and almost self-evident.", "limitations": "None", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The authors leverage high-frequency and dominant-frequency features to enhance time series imputation.\n\n2. They introduce time-domain and frequency-domain representation learning modules to comprehensively capture relevant information.\n\n3. Extensive experiments are conducted to demonstrate the method's effectiveness.", "weaknesses": "1. The formulation and writing of this paper should be improved, with attention to correcting several typos.\n\n2. The authors claim that the residual of a time series mainly comprises high-frequency components. However, in many time series, there may be multiple scales of seasonality and trend features, potentially resulting in a final residual that is too noisy to contain useful information.\n\n3. It appears that the proposed method utilizes frequency features of time series as conditions for the diffusion model, which seems straightforward and may limit its contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and valuable suggestions. We have tried our best to incorporate the suggestions in the revised version. Below, we provide our response to the questions and concerns.\n\n**1. (W1) The formulation and writing of this paper should be improved**\n\nWe appreciate your feedback on improving clarity and grammatical accuracy. We have meticulously revised the manuscript, correcting errors such as \"Recently, researchers attempt to utilize large language models\" to \"Recently, researchers have attempted to utilize large language models\" (line 62), and ensuring all sentences are structurally sound and clearly presented.\n\n**2. （W2）Explain how multiple scales of seasonality and trend features in time series affect the residual’s usefulness, which may be too noisy**\n\nThank you for your comments.\nSince our method does not explicitly perform STL decomposition, it is not affected by the multiple scales of trends and seasonality.\nOur method leverages Fourier transform to analyze both dominant-frequency and high-frequency information, which inherently addresses multiple scales of trend and seasonality.\nBy our dominant-frequency filter, it is possible to capture multiple scales of trend and seasonal feature information.\nIn addition, even when the high-frequency information that mainly contributes to the residual is noisy, we can still use the dominant-frequency information and adjust the weights of the high-frequency condition by attention layers for accurate imputation.\nAs shown in Table 1, our method always outperforms other methods in different scenarios.\n\n\n**3. （W3）Straightforward and may limit its contribution**\n\nOur core insight is that existing methods cannot accurately imputation the residual term, therefore introducing the corresponding frequency-domain information.\n**The insight of modelling high-frequency signals for time series imputation has not yet been recognized by existing studies.**\nOur proposed filters and representation learning modules are simple and effective for imputation, and can shed light on further works.\nIn addition, our proposed filters and representation learning modules can also be flexibly ported to other generative models and are not limited to applying diffusion models.\n\n**4. （Q1）Explain why the absence of the High-frequency filter impacts performance differently on KDD, Guangzhou, and PhysioNet datasets**\n\nThank you for your question.\nThe difference in impact of dominant-frequency and high-frequency information depends on the characteristics of datasets.\nFor the PhysioNet dataset, the obtained dominant-frequency information may be biased due to it contains at least 79.71% missing values.\nFor the KDD and Guangzhou datasets, the dominant-frequency information will play a larger role because of the lower missing rates.\nIn these cases, our method can adaptively adjust the weight of high-frequency information and dominant-frequency information to help the imputation model.\n\n**5. （Q2）Hyper-parameters for the experiments chosen**\n\nFor the two critical hyperparameters of the high-frequency filter and the dominant-frequency filter, we conduct empirical search to identify the optimal hyperparameters, as shown in Fig. 9 and Fig. 10 in Appendix A.4.2.\nFor other settings related to diffusion model, we adopt hyperparameters recommended by the existing well-established models [23,43]. These models have demonstrated strong performance in similar tasks, and their hyperparameters have been extensively validated in those literature.\nTo improve the presentation of the hyperparameters, we will add the above description of the parameters for the diffusion model to Appendix A.4.2.\n\n**6. （Q3）Purpose of Proposition**\n\nThe purpose of Proposition 3.1 is to illustrate the mechanism behind introducing the high-frequency condition and the dominant frequency condition for facilitating generative models to impute missing values.\nThis Proposition shows that with the introduction of the two conditions, the entropy of the target distribution will be reduced, the randomness of the target distribution will be narrowed, so that the generative models will more accurately impute missing values.\nTo add empirical evidence for this proposition, we also add experiments using CRPS to evaluate the gap between the learned and ground truth distributions for different generative baselines in the following table:\n\n| Dataset | Miss. Rate | MIWAE  | GPVAE  | TimeCIB | GAIN   | CSDI   | SSSD   | PriSTI | FGTI   |\n|---------|------------|--------|--------|---------|--------|--------|--------|--------|--------|\n| KDD     | 10%        | 0.524  | 0.443  | 0.466   | 0.709  | 0.224  | 0.352  | 0.232  | **0.158**  |\n|         | 20%        | 0.526  | 0.445  | 0.467   | 0.718  | 0.245  | 0.370  | 0.248  | **0.170**  |\n|         | 30%        | 0.532  | 0.447  | 0.469   | 0.729  | 0.259  | 0.374  | 0.268  | **0.186**  |\n|         | 40%        | 0.530  | 0.457  | 0.471   | 0.746  | 0.278  | 0.401  | 0.301  | **0.216**  |\n| Guang.  | 10%        | 0.312  | 0.333  | 0.360   | 0.692  | 0.265  | 0.316  | 0.209  | **0.155**  |\n|         | 20%        | 0.312  | 0.330  | 0.357   | 0.694  | 0.277  | 0.299  | 0.244  | **0.168**  |\n|         | 30%        | 0.312  | 0.335  | 0.356   | 0.695  | 0.292  | 0.353  | 0.310  | **0.193**  |\n|         | 40%        | 0.312  | 0.333  | 0.358   | 0.697  | 0.324  | 0.382  | 0.362  | **0.243**  |\n| Phy.    | 10%        | 0.689  | 0.659  | 0.466   | 0.739  | 0.544  | 0.617  | 0.444  | **0.343**  |\n|         | 20%        | 0.717  | 0.665  | 0.467   | 0.761  | 0.589  | 0.665  | 0.457  | **0.369**  |\n|         | 30%        | 0.750  | 0.674  | 0.469   | 0.787  | 0.627  | 0.630  | 0.467  | **0.389**  |\n|         | 40%        | 0.779  | 0.680  | 0.471   | 0.814  | 0.671  | 0.676  | 0.491  | **0.441**  |\n\nAs shown in the above table and Table 1, our method outperforms other methods for all metrics due to the introduction of frequency domain conditions, thus echoing the proposition.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their thoughtful and constructive comments. We are greatly encouraged that they found our idea and contributions to be significant (Reviewer WWjA, qcDA, Q2t8 and SUMs), and technical sound (Reviewer WWjA, qcDA, SUMs). We are grateful that they identified our method to be effective (Reviewer WWjA, qcDA, Q2t8 and SUMs) and our paper to be well-written (Reviewer WWjA, qcDA, Q2t8). However, we believe that there are still several questions and concerns mentioned in the reviews that need to be addressed.
Meanwhile, we also revised the paper and the appendix according to the reviewers' valuable suggestions. The main changes are as follows:
- We add several competitive imputation methods (i.e. LaST, FreTS, TimeCIB) recommended by reviewer WWjA and qcDA in our experiments.
- Thanks to the comments from WWjA and SUMs，we add a section about experimental results of CRPS metric as suggested by reviewer .
- We also add a case study exploring the role of high-frequency condition and dominant-frequency condition in the Appendix, relying on the comments of Reviewers WWjA and qcDA.
- We provide the experiment analysis of the mask ratio and mask pattern in the Appendix as recommended by Reviewer qcDA.
- We check the full manuscript thoroughly and improve the presentation and grammar as recommended by Reviewer qcDA and Q2t8.
- As shown in the attached pdf file, we update Fig. 1 according to Reviewer qcDA's suggestion and add a figure presenting the detailed architecture of the denoising network according to Reviewer Q2t8's comment.

Best,

Authors

Meta-review: This paper proposes a method called Frequency-aware Generative Models for Multivariate Time Series Imputation (FGTI). The approach aims to more carefully deal with high-frequency residual components when performing time series imputation tasks. The paper has uniformly positive reviews. The reviewers indicate that the paper is generally well-written and well-motivated. The experiments are extensive with a large and representative set of prior approaches and strong results. Some additional baselines and ablations were suggested and the authors provided them during the rebuttal phase. The reviewers who checked in during the author response indicated that their concerns were addressed by the author response.
Decision: Accept (poster)