Generative Time Series Forecasting with Diffusion, Denoise and Disentanglement
Anonymous Author(s)
AffiliationAddress email
Abstract
Time series forecasting has been a widely explored task that is of great importancein many applications.
However, it is common that real-world time series dataare recorded in a short time period, which results in a big gap between the deepmodel and the limited and noisy time series.
In this work, we propose to addressthe time series forecasting problem with generative modeling.
and propose abidirectional variational auto-encoder (BVAE) equipped with diffusion, denoise anddisentanglement, namely D 3 VAE.
Specifically, a coupled diffusion probabilisticmodel is proposed to augment the time series data without increasing the aleatoricuncertainty contributed to the data.
To ensure the generated series move towardsthe true target, we further propose to adapt and integrate the multiscale denoisingscore matching into the diffusion process for time series forecasting.
In addition,to enhance the interpretability and stability of the prediction, we treat the latentvariable in a multivariate manner and disentangle them on top of minimizing totalcorrelation.
Extensive experiments on both synthetic data and real-world data showthat D 3 VAE outperforms competitive algorithms with remarkable margins.
Introduction
Time series forecasting is of great importance for risk averse and decision making.
Traditional
RNN-based methods capture temporal dependencies of the time series to predict the future.
LSTMsand GRUs [52, 16, 15, 41] introduce the gate functions into cell structure to handle long-termdependencies effectively.
CNN-based models capture the complex inner patterns of the time seriesthrough the convolutional operations [28, 4, 3].
Recently, the former-based models have shown greatperformance in time series forecasting [51, 53, 25, 29] with the help of multi-head self-attention.
However, one big issue of the neural networks in time series forecasting is the uncertainty [14, 1]resulting from the properties of the deep structure.
The models based on vector autoregression(VAR) [5, 10, 23] try to model the distribution of the time series from the hidden states which couldprovide more reliability to the prediction.
Yet the performance of VAR is not satisfactory [27].
Interpretable representation learning is another merit for time series forecasting.
Variational AutoEncoders (VAEs) have shown not only the superiority in modeling latent distributions of the data andreducing the gradient noise [37, 24, 30, 45] but also the interpretability on time series forecasting [11,12].
However, the interpretability of VAEs might be inferior due to the entangled latent variables.
There have been efforts to learn representation disentangling [22, 2, 18], which show that the welldisentangled representation can improve the performance and robustness of the algorithm.
Moreover, real-world time series are often noisy and recorded in a short time period, which may resultin overfitting and poor generalization issues [13, 42, 54, 49] 1 .
To this end, we address the time seriesforecasting problem with generative modeling.
Specifically, we propose a bidirectional variationalauto-encoder (BVAE) equipped with diffusion, denoise and disentanglement, namely D 3 VAE.
Morespecifically, we first adapt the Nouveau VAE [45] to the time series forecasting task and develop a
BVAE as the backbone.
Then, we propose a coupled diffusion probabilistic model to remedy thelimitation of time series data by augmenting the input time series, as well as the output time series,inspired by the diffusion models [19, 35, 36].
Though the merit of generalizability is helpful, thediffused samples might be corrupted, which results in a generative model moving towards the noisytarget.
Therefore, we further develop a scaled denoising score matching network for cleaning diffusedtarget time series.
In addition, we disentangle the latent variables of the time series by assuming thatdifferent disentangled dimensions of the latent variables correspond to different temporal patterns(such as trend, seasonal and so on).
Our contributions can be summarized as:
• We propose a coupled diffusion probabilistic model aiming to reduce the aleatoric uncertainty
of the time series
and improve the generalization capability of the generative model.
• We integrate the multiscale denoising score matching into the coupled diffusion process to
improve the accuracy of the generated results.
• We disentangle the latent variables of the generative model to improve the interpretability for time series forecasting.
• Extensive experiments on both synthetic datasets and real-world datasets demonstrate thatD 3 VAE outperforms competitive baselines with satisfactory margins.
Methodology
2.1 Generative Time Series Forecasting
Problem Formulation.
Given an input multivariate time series X = { x 1 , x 2 , ..., x n | x i ∈ R d }and the corresponding target time series Y = { y n +1 , y n +2 , ..., y n + m | y j ∈ R d ′ } ( d ′ ≤ d ).
Weassume that Y can be generated from latent variables Z ∈ Ω Z that can be drawn from the Gaussiandistribution Z ∼ p
( Z | X ) .
The latent distribution can be further formulated as p ϕ
( Z | X ) =
g ϕ ( X )where g ϕ denotes a nonlinear function.
Then, the data-density of the target series is given by:
[Equation]
where f θ denotes a parameterized function.
The target time series can be obtained directly bysampling from p θ ( Y ) .
Bidirectional Variational Auto-Encoder.
In this work, we employ the bidirectional variationalauto-encoder (BVAE) [45] as the backbone generative model which is described in Fig. 1, where Z is treated in a multivariate fashion Z = [ z 1 , · · · , z n ] ( z i ∈ R m ) and z i +1 ∼ p
( z i +1 |
z i , X ) .
Then, n isdetermined in accordance with the number of residual blocks in the encoder, as well as the decoder. In our problem settings, time series forecasting is to learn the representation Z that captures usefulsignals of X , and map the low dimensional X to the high dimensional latent space which is suitablefor factor disentangling. However, many real-world time series data are recorded in a short timeperiod with inevitable noises, which may hinder the neural networks being applied directly. Hence,large amount of data are needed to ensure the generalization of neural network (see Appendix G).
Before diving into the detailed techniques, we first introduce a preliminary proposition.
Proposition 1.
Given a time series X and its inherent noise ϵ X , we have the decomposition:
X = ⟨ X r , ϵ X ⟩ , where X r is the ideal time series data without noise.
X r and ϵ X are independentfrom each other.
Let p ϕ
( Z | X ) =
p ϕ ( Z | X r , ϵ X ) , the estimated target series (cid:98) Y can be generatedwith the distribution p θ ( (cid:98) Y | Z )
=
p θ ( (cid:98) Y r | Z ) · p θ ( ϵ (cid:98) Y | Z ) where (cid:98) Y r is the ideal part of (cid:98) Y and ϵ (cid:98) Y isthe estimation noise.
Without loss of generality, (cid:98) Y r can be fully captured by the model.
That is,∥ Y r − (cid:98) Y r ∥ −→ 0 where Y r is the ideal part of the ground truth target series Y .
In addition, Y can bedecomposed as Y = ⟨ (cid:98) Y r , ϵ Y ⟩ ( ϵ Y denotes the noise of Y ).
Therefore, the error between the groundtruth and the prediction, i.e. ∥ Y − (cid:98) Y ∥ =
∥
ϵ Y − ϵ (cid:98) Y ∥ > 0 , can be deemed as the combination ofaleatoric uncertainty and epistemic uncertainty.
2.2 Coupled Diffusion Probabilistic Model
The diffusion process proposed by [19] is fixed to a Markov chain that gradually adds Gaussian noiseto the data (see Appendix E for illustrative examples of the diffusion process on time series data).
Specifically, given the input X = X
(0) ∼ q ( X (0) ) , the approximate posterior q ( X (1: T ) | X (0) ) canbe obtained as
[Equation]
where a monotonically increasing (the value difference is the same among all adjacent pair of points)variance schedule β
= { β 1 , · · · , β T | β t ∈ [0 , 1) } is employed to control the level of noise to beadded.
Then, let α t = 1 − β t
and ¯ α t = (cid:81) ts =1 α s , we have
[Equation] [Equation]
Furthermore, according to Proposition 1 and Eq.
(3), we have X (0) =
⟨ X r , ϵ X ⟩ and X ( t ) ∼q
( X ( t ) | X (0) ) .
Consequently, X ( t ) can be decomposed as follows:
[Equation]
(4) where δ X denotes the standard Gaussian noise of X .
As α can be determined when the varianceschedule β is initialized, the ideal part is also determined in the diffusion process.
Let (cid:98) X ( t ) r = √
¯ α t X rand δ ( t ) (cid:98)
X =
√ ¯
α t ϵ X
+ (1 − ¯ α t ) δ X , then, according to Proposition 1, we have
[Equation]
where δ ( t ) (cid:98) Y denotes the generated noise contributed to (cid:98) Y ( t ) .
To relieve the effect of the aleatoricuncertainty resulted from time series data, we further apply the diffusion process to the target series
Y = Y (0) ∼ q ( Y (0) ) .
In particular, a scale parameter ω is adopted, such that β ′ t =
ωβ t , α ′ t = 1 − β ′ tand
¯ α ′ t = (cid:81) ts =1 α ′ s .
Then, according to Proposition 1, we have the following decomposition:
[Equation]
Consequently, we have q ( Y ( t ) )
=
q ( (cid:101) Y ( t ) r ) q ( δ ( t ) (cid:101) Y ) .
Afterward, with Proposition 1 and Eqs.
(5) and (6)we can draw following conclusions.
Lemma 1.
∀ ε > 0 , there exists a probabilistic model f ϕ,θ :
= ( p ϕ , p θ ) to guarantee that
D KL ( q ( (cid:101) Y ( t ) r ) ||
p
θ ( (cid:98) Y ( t ) r ))
< ε , where (cid:98) Y ( t ) r = f ϕ,θ
( X ( t ) ) .
Lemma 2.
With the coupled diffusion probabilistic model, lim t →∞ D KL ( q ( δ ( t ) (cid:101) Y ) || p θ ( δ ( t ) (cid:98) Y ) | z ( t ) ) <
D KL ( q ( ϵ Y ) ||
p θ ( ϵ (cid:98) Y )) .
Proof.
According to Proposition 1, let ϵ Y =
⟨ ϵ (cid:98) Y , δ Y ⟩ , then q ( ϵ Y ) =
q ( ϵ (cid:98) Y ) q ( δ Y ) .
Let ∆ =
D KL ( q ( ϵ Y ) ||
p θ ( ϵ (cid:98) Y ))
− D KL ( q ( ϵ (cid:98) Y )
||
p θ ( ϵ (cid:98) Y )) , we have
[Equation]
)) > 0 , which leads to D KL ( q ( ϵ Y )
||
p θ ( ϵ (cid:98) Y ))
> D KL ( q ( ϵ (cid:98) Y )
||
p θ ( ϵ (cid:98) Y ) > 0 .
Moreover, both δ ( t ) (cid:101) Y and δ ( t ) (cid:98) Yare Gaussian noises, when t → ∞ , ∃ ε ′ > 0 , we have D KL ( q ( δ ( t ) (cid:101) Y )
||
p θ ( δ ( t ) (cid:98) Y | Z ( t ) ))
≤ ε ′ <
D KL ( q ( ϵ Y ) ||
p θ ( ϵ (cid:98) Y )) .
Therefore, the uncertainty contributed to the generative model and the inherent data noise can bereduced through the coupled diffusion probabilistic model.
In addition, the diffusion process augmentsthe input series, as well as the target series, simultaneously, which can improve the generalizationcapability for (esp.
short) time series forecasting.
2.3 Scaled Denoising Score Matching for Diffused Time Series Cleaning
Although the time series data can be augmented with the aforementioned coupled diffusion probabilistic model, the generative distribution p θ ( (cid:98) Y ( t ) ) tends to move towards the diffused target series
Y ( t ) which have been corrupted [32, 43].
To further “clean” the diffused target series, we employ the Denoising Score Matching (DSM) to accelerate the de-uncertainty process without sacrificing themodel flexibility.
DSM [46, 32] was proposed to link Denoising Auto-Encoder (DAE) [47] to Score
Matching (SM) [20].
Let (cid:101) Y denote the diffused target series, then we have the objective
[Equation]
(7) where p σ 0 ( (cid:101) Y , Y ) is the joint density of pairs of clean and corrupted samples ( Y, (cid:101) Y ) ,∇ (cid:101) Y log( q σ 0 ( (cid:101) Y | Y )) is the derivative of log density of the single noise kernel which is dedicated to replace the Parzen density estimator: p σ 0 ( (cid:101) Y )
=
(cid:82) q σ 0 ( (cid:101) Y | Y ) p ( Y ) dY in score matching, and E ( (cid:101) Y ; ζ ) isthe energy function.
In the particular case of Gaussian noise, log( q σ 0 ( (cid:101) Y | Y ))
=
− ( (cid:101) Y − Y ) 2 / 2 σ 02 + C .
Thus, we have
[Equation]
Then, for the diffused target series at step t , we can obtain
[Equation]
To scale the noise of different levels [32], a monotonically decreasing series of fixed σ values{ σ 1 , · · · , σ T | σ t = 1 − ¯ α t } (refer to the aforementioned variance schedule β in Section 2.2) isadopted.
Therefore, the objective for multi-scaled DSM is
[Equation]
where σ ∈ { σ 1 , · · · , σ T } and l ( σ t ) = σ t .
With Eq.
(10), we can ensure that the gradient has the rightmagnitude by setting σ 0 .
In the generative time series forecasting setting, the generated samples will be tested without applyingthe diffusion process.
To further denoise the generated target series (cid:98) Y , we apply a single step gradientdenoising jump [40]:
[Equation]
The generated results tend to possess bigger distribution space than the true target, and the noisy termin Eq.
(11) approximates the noise between the generated target series and the “cleaned” target series.
Therefore, σ 20 ∇ (cid:98) Y E ( (cid:98) Y ; ζ ) can be treated as the estimated uncertainty of the prediction.
2.4 Disentangling Latent Variables for Interpretation
The interpretability of the time series forecasting model is of great importance for many downstreamtasks [44, 17, 21].
With disentangling the latent variables of the generative model, not only theinterpretability but also the reliance of the prediction can be further enhanced [31].
To disentangle the latent variable Z
= [ z 1 , · · · , z n ] , we attempt to minimize the Total Correlation(TC)
[50, 22], which is a popular metric to measure the dependence among multiple random variables,
[Equation]
where m denotes the number of factors of z i that need to be disentangled.
In general, lower TC meansbetter disentanglement if the latent variables preserve useful information.
However, a very low TCcan still be obtained when the latent variables carry no meaningful signals.
Through the bidirectionalstructure of BVAE, such issues can be tackled without too many efforts.
As shown in Fig.
1, thesignals are disseminated in both encoder and decoder, such that rich semantics are aggregated intothe latent variables.
Furthermore, to alleviate the effect of potential irregular values, we average thetotal correlations of z 1: n , then the loss w.r.t.
the TC score of BVAE can be obtained:
[Equation]
Optimization.
To reduce the effect of uncertainty, the coupled diffusion equipped with the denoisingnetwork is proposed without sacrificing too much generalizability.
Then we disentangle the latentvariables of the generative model by minimizing the TC of the latent variables.
Finally, we reconstructthe loss with several trade off parameters, and with Eqs.
(10), (11) and (13) we have
[Equation]
where L mse is the MSE between (cid:98) Y ( t ) and Y ( t ) .
We minimize the above loss function to learn thegenerative model.
Experiments
3.1 Experiment Settings
Datasets.
We generate two synthetic datasets suggested by [9],
[Equation]
where w t ∈ R 2 and 0 ≤ w t, 1 , w t, 2 ≤ 1 ( t = 1 , 2 , 3 ), F ∈ R 2 × k ∼ U
[ − 1 , 1] , k denotes thedimensionality and N is the number of time points, a, b are two constants.
We set a = 0 .
9 , b =0 .
2 , k = 20 to generate D 1 , and a = 0 .
5 , b = 0 .
5 , k = 40 for D 2 , and N = 800 for both D 1 and D 2 .
Six real-world datasets with diverse spatio-temporal dynamics are selected, including Traffic [27],
Electricity 2 , Weather 3 , WP (Wind Power) 4 and ETTs [53] (ETTm1 and ETTh1).
To highlight theuncertainty in short time series scenarios, for each dataset, we slice a subset from the starting point tomake sure that each sliced dataset containing at most 1000 time points.
Subsequently, we obtain 5% -
Traffic, 3% -Electricity, 2% -Weather, 2% -WP, 1% -ETTm1 and 5% -ETTh1.
The statistical descriptionsof the real-world datasets can be found in Appendix F.
All datasets are split chronologically andadopt the same train/validation/test ratios, i.e
We collected this dataset and will publish it upon acceptance
[Table]
Baselines.
We compare D 3 VAE with one GP (Gaussian Process) based method (GP-copula [38]),two auto-regressive methods (DeepAR [39] and TimeGrad [36]), and four VAE-based methods, i.e.vanilla VAE, NVAE [45], factor-VAE (f-VAE for short) [22] and β -TCVAE [6].
Implementation Details.
An input- l x -predict- l y window is applied to roll the train, validation andtest sets with stride one time step, respectively, and this setting is adopted for all datasets.
Hereinafter,the last dimension of the multivariate time series is selected as the default target variable.
We use Adam optimizer with initial learning rate 5 e − 4 .
The batch size is 16 and the training is setto 20 epochs at most equipped with early stopping.
The number of disentanglement factors is chosenfrom { 4 , 8 } , and β T ∈ β is set to range from 0 .
01 to 0 .
1 with different diffusion steps T ∈ [100 , 1000] ,then ω is set to 0 .
The trade-off hyperparameters are set as ψ = 0 .
05 , λ = 0 .
1 , γ = 0 .
001 for ETTs,and ψ = 0 .
5 , λ = 1 . 0 , γ = 0 .
01 for others.
All the experiments were carried out on a Linux machinewith a single NVIDIA P40 GPU.
The experiments are repeated 5 times and the average and varianceof the predictions are reported.
We use the Continuous Ranked Probability Score (CRPS) [34] and
Mean Squared Error (MSE) as the evaluation metrics, for both metrics, the lower the better.
Inparticular, CRPS is used to evaluate the similarity of two distributions and is equivalent to Mean
Absolute Error (MAE) when two distributions are discrete.
3.2 Main Results
Two different prediction lengths, i.e. l y ∈ { 8 , 16 } and l x = l y , are evaluated.
The results of longerprediction lengths are available in supplementary materials (see Appendix D).
Toy Datasets.
In Table 1, we can observe that D 3 VAE achieves SOTA performance most of the time,and achieves competitive CRPS in D 2 for prediction length 16.
Besides, VAEs outperform VARs and
GP on D 1 , but VARs achieve better performance on D 2 , which demonstrate the advantage of VARsin learning complex temporal dependencies.
Real-World Datasets.
As for the experiments on real-world data, D 3 VAE achieves consistent
SOTA performance except for the prediction length 16 on Wind dataset (Table 2).
Particularly, underthe input-8-predict-8 setting, D 3 VAE can provide remarkable improvements on Traffic, Electricity,
Wind, ETTm1, ETTh1 and Weather w.r.t.
MSE reduction (90%, 71%, 48%, 43%, 40% and 28%).
Regarding the CRPS reduction, D 3 VAE achieves 73% reduction on Traffic, 31% on Wind and 27%on Electricity under the input-8-predict-8 setting, and 70% reduction on Traffic, 18% on electricityand 7% on Weather under the input-16-predict-16 setting.
Overall, D 3 VAE gains 43% MSE reductionand 23% CRPS reduction among above settings.
More results under the longer prediction setting andfull dataset setting can be found in supplementary materials.
Uncertainty Estimation.
The uncertainty can be assessed by estimating the noise of the outcomeseries when doing the prediction (see Section 2.3).
Through scale parameter ω , the generated distribution space can be adjusted accordingly (results on the effect of ω can be found in Appendix D.3).
Theshowcases in Fig.
2 demonstrate the uncertainty estimation of the yielded series in Traffic dataset, [Table] where the last six dimensions are treated as the target variables.
We can find that the noise estimationcan quantify the uncertainty effectively.
For example, the uncertainty grows rapidly when extremevalues are encountered.
Disentanglement Evaluation.
It is difficult to label disentangled factors by hand for time series forecasting, thus we take different dimensions of Z as the factors to be disentangled:
Z = [ z 1 , · · · , z n ] ,then an instance consisting of factor and label ( z i,j , j ) is constructed.
We shuffle these n examplesand attempt to classify them with a discriminator, then the disentanglement can be evaluated bymeasuring the loss of discriminator.
Due to the space limit, the evaluation of disentanglement withdifferent factors can be seen in supplementary materials (Appendix C).
3.3 Model Analysis
Ablation Study of the Coupled Diffusion and Denoising Network.
To evaluate the effectivenessof the coupled diffusion model (CDM), we compare the full versioned D 3 VAE with its three variants:i) D
3 VAE − (cid:101) Y , i.e. D 3 VAE without diffused Y , ii)
D 3 VAE − (cid:98) X , i.e. D 3 VAE without diffused X , andiii) D 3 VAE − CDM , i.e. D 3 VAE without any diffusion.
Besides, the performance of D 3 VAE withoutthe denoising network (i.e. DSM) is also reported when the target series is not diffused, which aredenoted as D 3 VAE − (cid:101)
Y − DSM and D 3 VAE − CDM − DSM .
The ablation study is carried out on Trafficand Electricity datasets under the settings of input-16-predict-16 and input-32-predict-32. In Table 3,we can find that the diffusion process is effective in augmenting the input or the target. Moreover,
[Table]
varying T on the Electricity dataset.
when the target is not diffused, the denoising network would be deficient for the noise level of thetarget cannot be estimated by then.
Variance Schedule β
and The Number of Diffusion Steps T .
In order to reduce the effect of theuncertainty while preserving the informative temporal patterns, the extent of the diffusion shouldbe configured properly.
Too small variance schedule or inadequate diffusion steps will lead to ameaningless diffusion process, on the contrary, the diffusion will be out of control.
Here we analyzethe effect of the variance schedule β and the number of diffusion steps T .
We set β 1 = 0 and changethe value of β T in the range of [0 . 01 , 0 . 1] , and T ranges from 100 to 4000 .
As shown in Fig.
3, wecan see that the prediction performance can be improved if proper β T and T are adopted.
Discussion
Sampling for Generative Time Series Forecasting
The Langevin dynamics has been widely applied to the sampling of the energy based mod- els (EBMs) [8],
[Equation]
(15) where k ∈ { 0 , · · · , K } , K denotes the number of sampling steps, and ρ is a constant.
With K and ρbeing properly configured, the samples with high quality can be generated.
The Langevin dynamicshas been successfully applied to image reconstruction and language processing [26, 7].
In this work, we employ a single-step gradient denoising jump to generate the target series.
Theexperiments carried out demonstrate the effectiveness of such single step sampling.
To investigatewhether it is worthy to take more sampling steps for further performance improvement of timeseries forecasting, we conduct additional empirical study.
We showcase the prediction results underdifferent sampling strategies in Fig. 4.
By omitting the additive noise in Langevin dynamics, weemploy the multi-step denoising for D 3 VAE to generate the target series and plot the generated resultsin Fig. 4a.
Then, with the standard Langevin dynamics we can implement a generative procedureinstead of denoising and compare the generated target series with different ρ (see Figs. 4b to 4d).
Wecan observe that, more sampling steps might not be helpful in improving prediction performancefor generative time series forecasting (Fig. 4a).
Besides, larger sampling steps would lead to highcomputational complexity.
On the other hand, different configurations of Langevin dynamics (withvarying ρ ) cannot bring indispensable benefits for time series forecasting (Figs. 4b to 4d).
[Figure]
Limitations.
With the coupled diffusion probabilistic model, the aleatoric uncertainty of time series can be reducedthough, new bias is brought into the series to mimic the distribution of the input and target.
However,as a common issue in VAEs that any introduced bias in the input will result in bias in the generatedoutput [48], the diffusion steps and variance schedule need to be chosen cautiously, such that thismodel can be applied to different time series tasks smoothly.
And the proposed model is devised forgeneral time series forecasting, it should be used properly to avoid the potential negative societalimpacts, such as any kind of illegal applications.
In time series predictive analysis, disentanglement of the latent variables has been of great importancefor interpreting the prediction to provide more reliance.
Due to the lack of prior knowledge ofthe entangled factors in generative time series forecasting, only the unsupervised disentanglementlearning can be done, which has been proven theoretically feasible for time series [31].
Despite this,for boarder applications of disentanglement and better performance, it is still worth of exploring theways of labeling the factors of time series in the future.
Moreover, because of the uniqueness of timeseries data, it is also an promising direction to explore more generative and sampling methods fortime series generation task.
Conclusion
In this work, we propose a generative model with the bidirectional VAE as the backbone.
To furtherimprove the generalizability, we devise a coupled diffusion probabilistic model for time seriesforecasting.
Then a scaled denoising network is developed to guarantee the prediction accuracy.
Afterward, the latent variables are further disentangled for better model interpretability.
Extensiveexperiments on synthetic data and real-world data validate that our proposed generative modelachieves SOTA performance comparing to existing competitive generative models.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 4 for details.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] WediscussÅ the potential negative societal impacts in Section 4(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Our problem formulation and theoretical proof are described in Section 2, and all the theoreticalresults are with reasonable and necessary assumptions.
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] We use six real-world datasets, for each of them is provided by URL or citation in Section 3.1.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 3 and Appendix F
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] All the experimental results are repeated five times, the
mean and standard deviation of the results are provided in Section
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 3.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? Yes] The
datasets and baseline models that used in this paper are
all cited in Section 3.1.
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The newly introduced dataset Wind is provided in the supplementary materials and will be released later.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] The datasets used in this paper are all public datasets and we didn’t use/curate data from other people.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] The data we use doesn’t contain any identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]