FIDE: Frequency-Inflated Conditional Diffusion Model for Extreme-Aware Time Series Generation

Authors: asadullah hill galib
pang-ning  tan
lifeng  luo
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 22:50:37

Abstract: Time series generation is a crucial aspect of data analysis, playing a pivotal role in learning the temporal patterns and their underlying dynamics across diverse fields. Conventional time series generation methods often struggle to capture extreme values adequately, diminishing their value in critical applications such as scenario planning and management for healthcare, finance, climate change adaptation, and beyond. In this paper, we introduce a conditional diffusion model called FIDE to address the challenge of preserving the distribution of extreme values in generative modeling for time series. FIDE employs a novel high-frequency inflation strategy in the frequency domain, preventing premature fade-out of the extreme value. It also extends traditional diffusion-based model, enabling the generation of samples conditioned on the block maxima, thereby enhancing the model's capacity to capture extreme events. Additionally, the FIDE framework incorporates the Generalized Extreme Value (GEV) distribution within its generative modeling framework, ensuring fidelity to both block maxima and overall data distribution. Experimental results on real-world and synthetic data showcase the efficacy of FIDE over baseline methods, highlighting its potential in advancing Generative AI for time series analysis, specifically in accurately modeling extreme events.

Main paper: 
 introduction: 
 Generative models [18, 10, 13] have revolutionized the AI landscape, demonstrating their broad applicability across diverse domains, including computer vision and natural language processing. Such models are designed to learn the underlying data distribution and exhibit resilience to overfitting while promoting automatic feature extraction. Diffusion-based models [12, 19] , in particular, have emerged as a popular generative AI method due to their capability to generate realistic, high-quality data. This paper examines the application of diffusion-based models for time series generation. In particular, we investigate the following issue: How well do existing diffusion models preserve the fidelity of extreme values (i.e., tail distribution) of the original time series?The modeling of extreme values in time series is essential for informed decision-making across diverse applications, including weather forecasting, earthquake prediction, and disease outbreak detection. Effective generative modeling of these extremes is important as it aids in learning the underlying data distribution, facilitating data augmentation, and improving uncertainty estimation, all of which are crucial for developing robust risk management strategies and enhancing disaster preparedness measures. While there has been growing research on applying diffusion models for time series [20, 2] , their ability to preserve the distribution of extreme values remains largely underexplored. In this study, we examine how effectively diffusion models preserve extreme values in the form of block maxima [4] , defined as the peak value within a specified time window.To illustrate the difficulty of modeling the distribution of block maxima, Figure 1 shows the result of applying the Denoising Diffusion Probabilistic Model (DDPM) [12] to a synthetic AR(1) dataset. Figure 1 : Comparing the distributions of all values and block maxima values for real and generated samples using DDPM [12] when applied to the synthetic AR(1) dataset.While DDPM shows proficiency in generating samples that closely align with the overall data distribution (left diagram), it struggles to preserve the distribution of block maxima values (right diagram) when the generated time series is partitioned into disjoint time windows.In this paper, we identify the key shortcomings of existing diffusion models that hamper their ability to accurately model block maxima values. We then present a novel framework to overcome this limitation. Our key observation is that unusually large block maxima values, often linked to abrupt temporal changes, are strongly associated with high-frequency components of the time series. As the diffusion-based generative model gradually introduces noise with a linearly increasing variance schedule, it slowly diminishes the long-term trends (low-frequency components) of the time series while quickly attenuating the high-frequency components. These high-frequency components are crucial for reproducing extreme block maxima values. This limitation hampers the accurate representation of the block maxima, necessitating the development of new techniques.To address this challenge, we propose an end-to-end diffusion model framework termed FIDE. First, to mitigate the rapid dissipation of high-frequency components in the diffusion model, we introduce a novel high-frequency inflation strategy within the frequency domain. This strategic augmentation ensures the sustained emphasis on block maxima, preventing their premature fadeout. We further employ a conditional diffusion-based generative modeling approach to guide the time series generation by conditioning on their block maxima. To enhance the preservation of the block maxima distribution while learning the overall data distribution, we extend the conventional framework with a regularization term in the loss function based on the negative log-likelihood of the Generalized Extreme Value (GEV) distribution. Using these strategies, we empirically show that our approach effectively addresses the challenges of learning the overall data distribution while simultaneously preserving the block maxima distribution. 
preliminaries: 
 Consider a time series dataset D = {x m,0 } M m=1 comprising of M samples, where each sample x m,0 = (x 1 m,0 , x 2 m,0 , • • • , x T m,0 ) is a univariate time series of finite length T . Let f m,0 ∈ R T be the Fourier coefficients, whose k-th frequency component is obtained by applying the following discrete Fourier transform on x m,0 :EQUATIONThe time series can be recovered from its Fourier coefficients using the following inverse discrete Fourier transform:EQUATIONFor brevity, we will drop the sample subscript m when it is clear from the context. Let, ω k = 2πk T be the k-th frequency in Fourier transform.Given a sample x 0 , let y 0 be its corresponding block maxima value, where y 0 = max τ ∈{1,••• ,T } x τ 0 . The distribution of the block maxima values is governed by the Generalized Extreme Value (GEV) Figure 2 : Removal of high-frequency components from daily temperature time series significantly alters the magnitude of its block maxima value (at time step 20), as evidenced by its high residual. distribution, whose cumulative distribution function is given as follows [4] :EQUATIONwhere µ (location), σ (scale), and ξ (shape) are the distribution parameters. Given M independent block maxima values, denoted as {y 1,0 , y 2,0 , • • • , y M,0 }, with the cumulative distribution function given by Equation (3), the distribution parameters can be estimated using the maximum likelihood approach by minimizing the following negative log-likelihood function:-log L GEV (µ, σ, ξ) = M log σ + 1 ξ + 1 M i=1 log 1 + ξ y i,0 -µ σ + M i=1 1 + ξ y i,0 -µ σ -1/ξ (4)Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: See 6. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
on the rapid dissipation of block maxima in diffusion models: 
 While diffusion models have demonstrated remarkable capabilities in learning complex data distributions, a significant challenge arises in accurately capturing the distribution of block maxima values, as evidenced by Figure 1 . Addressing this shortcoming is crucial for enhancing the performance and applicability of these models across various domains. In this section, we delve into the root cause of this phenomenon and present insightful observations that shed light on the underlying issue.Our first key observation reveals a connection between block maxima with abrupt changes and the high-frequency components of many real-world time series. Block maxima, often characterized by their rarity and abrupt temporal changes, are intrinsically linked to the high-frequency components of the data. This relationship is observed in many real-world datasets, where the block maxima values do not typically evolve smoothly but rather emerge through large deviations from their adjacent values.To illustrate this, consider the real-world temperature time series depicted in Figure 2 . In this plot, we first transform the time series into its Fourier domain, obtaining its frequency components, and selectively zeroing out its top-5 highest frequency components. We then reconstruct the time series via its inverse Fourier transform and compute the difference between the original and reconstructed time series. The recovered signal exhibits a notable distortion around the block maxima value, as evidenced by the larger residual at time step 20, where the block maxima value occurs. This suggests that the removal of high-frequency components of a time series has a significant impact on the Our second key observation unveils a concerning behavior of diffusion models: the addition of noise diminishes high frequency components, i.e., block maxima, at a faster rate compared to other values in the signal. As diffusion-based generative models gradually introduce noise characterized by a linearly increasing variance scheduler, they inadvertently attenuate the signals associated with high-frequency components. These components, as established in our first observation, are crucial for accurately reproducing block maxima. Concurrently, the models effectively capture the long-term trends and low-frequency components, which are conducive to learning the overall data distribution. However, the high frequency components dissipate more rapidly, hindering the model's ability to adeptly learn the distribution of the block maxima values.Figure 3a illustrates this phenomenon. By tracking the evolution of residuals, or the differences between the original and perturbed time series generated by DDPM, we observe a discernible pattern: block maxima dissipate at a faster rate compared to other values, as evidenced by the higher residuals associated with these extreme points. Notably, in the early iterations highlighted by the green circle, the substantially higher residual suggests that the block maxima signal is rapidly transformed into noise, outpacing the dissipation rate of other values. This behavior poses a formidable challenge for diffusion models in effectively capturing the distributions of the block maxima values.To substantiate our observations, the theorem below offers a rigorous justification for the rapid dissipation of block maxima during the forward process of the diffusion model (see Appendix C for proof and details). Let x 0 be an input sample and x n be the perturbed sample after n iterations of the forward process, wherex t n = x t n-1 + ϵ t n and ϵ t n ∼ N(0, σ 2 ϵ t n) is Gaussian noise. Due to the linearity of the Fourier transform operator F, we have:EQUATIONTheorem 1. Under certain mild assumptions (see Appendix C), the ratio of high-frequency and low-frequency components after perturbation during the forward process of the diffusion model is:lim k→kmax |f k n | 2 lim k→0 |f k n | 2 = δ ≪ 1 (6)where k max is the index of the maximum frequency and δ = f kmax n , which is generally close to 0.In short, our findings shed light on a fundamental limitation of diffusion models while modeling block maxima and underscore the need for a more tailored approach to preserve its distribution.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]Justification: See C.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
proposed framework: fide: 
 In this section, we present the detailed methodology of our proposed approach, addressing the challenges associated with capturing extreme values of time series within diffusion-based generative models. Figure 4 provides an overview of the FIDE framework. In order to counteract the rapid decay of high-frequency components in the frequency domain while adding noise in the forward process of DDPM, we present a strategy for high-frequency inflation. Let f 0 = FFT (x 0 ) denote the vector of Fourier coefficients resulting from applying the discrete Fourier transform to the time series x 0 . These coefficients are arranged in ascending order from lowest to highest frequency. Consequently, the last κ elements of f 0 correspond to the coefficients associated with the κ highest frequencies. Our goal is to inflate the top-κ frequency components of f 0 as follows:Γ i = 1, if i ≤ κ γ, if i > T -κ and f 0 = Γ ⊙ f 0where γ > 1 is the inflation weight and ⊙ denotes the element-wise multiplication.With the modified coefficients f 0 , the inverse Fourier transform (IFFT) is applied to get the modified time series, x 0 = IFFT f 0 , containing the inflated high-frequency components. Here, the highfrequency components are inflated by γ > 1. The following theorem shows how this inflation strategy helps the high-frequency components (block maxima) diminish less rapidly in the diffusion forward process compared to before (see Appendix C for proof and details).Theorem 2. Let f k n be the Fourier coefficient after inflating high-frequency components with a factor of γ such that γ > 1. Let δ = f kmax n be the Fourier coefficient of the maximum frequency before inflation, and δ ′ = δ • γ = f kmax n be the Fourier coefficient of the maximum frequency after inflation. Then, using Lemma 1 and under certain mild assumptions (see Appendix C), the ratio of high-frequency and low-frequency components after inflation and perturbations is:EQUATIONThus, by applying high-frequency inflation, the high-frequency components including abrupt block maxima will be preserved by a factor of γ compared to the previous case. We can see the effects of this inflation empirically as well. Figure 3b shows how inflating the high-frequency components helps in preserving the block maxima values for longer iterations of the diffusion model. This enables the block maxima after high-frequency inflation to dissipate at a similar rate compared to other values in the earlier iterations. The diffusion model will have more iterations to capture the block maxima signal.We use the inflated time series x 0 as input time series to be perturbed during the forward process instead of x 0 . By adopting x 0 as the reference for the unperturbed sample, we ensure that the denoising diffusion process takes advantage of the enhanced representation provided by the inflated high-frequency components. This nuanced adjustment contributes to the efficacy of our proposed framework in capturing and preserving essential information during the diffusion process.To enable the generation of samples conditioned on block maxima, we extend the conventional diffusion model to a conditional model. Here, the reverse process is conditioned on block maxima y 0 . Grounded in extreme value theory [4] , the block maxima values {y 0 } are governed by the Generalized Extreme Value (GEV) distribution, distinctly diverging from the distribution of all values x 0 ∼ p θ (x 0 ). This mandates a strategic shift in our learning objective. Rather than marginally targeting p θ (x 0 ), our objective now extends to mastering the joint distribution p θ (x 0 , y 0 ), driven by a nuanced understanding of the unique characteristics inherent in extreme events and their crucial impact on the overall distribution. We formally extend the diffusion model's marginal distribution to a joint distribution in the following theorem (see Appendix C for proof and details). Theorem 3. Consider an extension of the conventional diffusion model from learning a marginal distribution p θ (x 0 ) to a joint distribution p θ (x 0 , y 0 ) conditioned on block maxima y 0 . In this context, the variational lower bound can be formulated as follows:-log p θ (x 0 , y 0 ) ≤ Eq log q(x 1:N |x 0 , y 0 )EQUATIONFirst, we adopt x 0 as the reference for the unperturbed sample x 0 as discussed in the previous subsection. After reparameterization and ignoring the weighting term, as suggested by [12] , the first term of the variation lower bound can be expressed as:L DDPM = E xn,ϵn,n,y0 ∥ ϵ n (x n , n, y 0 ) -ϵ n ∥ 2 2(9) Additionally, considering a Generalized Extreme Value (GEV) distribution for block maxima, the second term is simplified as log L GEV (µ, σ, ξ), as defined in Eq. 4.The preceding theorem establishes a clear link between the variational lower bound and an interpretable objective loss function:-EQUATIONHere, L DDPM represents the expected reconstruction error between actual and estimated noise, and log L GEV (µ, σ, ξ) captures the negative log-likelihood of the block maxima governed by the GEV distribution.To enforce fidelity on both the block maxima and overall data distribution, we incorporate the Generalized Extreme Value (GEV) distribution within the DDPM framework following Theorem 3. We first fit a GEV distribution using maximum log-likelihood estimation with all the block maxima (y 0 ) values in the training data. The fitted GEV distribution is parameterized by µ, σ, and ξ, denoted as θ gev = {µ, σ, ξ}. Using the conditional diffusion process, the estimated noise is given by ϵ n (x n , n, y 0 ). Consequently, the estimated denoised sample can be obtained as:x 0 = x n -ϵ n .Then, utilizing the fitted GEV distribution, the log-likelihood of the estimated denoised block maxima, y 0 = max τ ∈{1,••• ,T } x τ 0 , is calculated. This negative log-likelihood, -log L GEV (µ, σ, ξ, y 0 ), is finally incorporated into the loss function of training.Algorithm 1 summarizes the pseudocode for training and Algorithm 2 summarizes the pseudocode for the sampling step of FIDE. The overall loss function L FIDE is constructed by combining two key components: the DDPM loss L DDPM and the negative log-likelihood of the Generalized Extreme Value (GEV) distribution -L GEV . The formulation is expressed as follows:EQUATION)where λ is a hyperparameter controlling the influence of the GEV distribution on the loss.In this context, L DDPM evaluates the mean squared difference between the estimated noise term ϵ n and the true noise term ϵ n within the conditional diffusion process. Its purpose is to guide the generative model towards effectively capturing the underlying data distribution. The second element, -log L GEV (µ, σ, ξ, y 0 ), encapsulates the negative log-likelihood of the GEV distribution. This component assesses how well the fitted GEV distribution aligns with the estimated block maxima values y 0 derived from the denoised samples. Here, the log-likelihood has a negative sign to indicate a minimization objective, aligning with the overall goal of minimizing the loss function.Trainingrepeat x 0 ∼ q(x 0 ) where x 0 = (x 1 0 , x 2 0 , • • • , x T 0 ) f 0 = FFT(x 0 ) f 0 = γ ⊙ f 0 x 0 = IFFT(f 0 ) n ∼ Uniform({1, • • • , N }) ϵ n ∼ N (0, I) y 0 = max(x 0 ) x n = √ α n x 0 + √ 1 -α n ϵ n y 0 = max(x n -ϵ n (x n , n, y 0 )) Take the gradient step on ∇ θ || ϵ n (x n , n, y 0 )) -ϵ n || 2 2 -λ • log L GEV (µ, σ, ξ, y 0 ) until convergedInput: Block maxima ŷ0 ∼ GEV(y 0 ) and Trained Model ϵ Output: Generate time series, x0 .xN ∼ N (0, σ 2 ) for n = N, • • • , 1 do z ∼ N (0, I) xn-1 = 1 √ αn (x n -1-αn √ 1-αn ϵ n (x n , n, ŷ0 )) +σ n z end for return x0Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: See 5. 
experimental evaluation: 
 We have performed extensive experiments to evaluate the performance of our FIDE framework. All the code and datasets used in this paper are available at https://github.com/galib19/FIDE. The datasets used are described in Appendix D.We compared our proposed framework against various generative models: (1) GAN-based: We utilize two GAN-based approaches as our baselines. The first approach is Conditional GAN (cGAN [15] ), which introduces conditional information to the training process, enabling targeted generation based on specified conditions. The second baseline is TimeGAN [21] , which is a generative model designed specifically for time-series generation. (2) VAE-based: We employ beta-VAE [11] , conditional beta-VAE [16] , and TimeVAE [6] as baseline methods for comparison. Both beta-VAE and conditional [7] and Fourier-Flows [1] as our baseline methods. (4) Diffusion-based: We consider two baselines for comparison, namely, the denoising diffusion probabilistic model (DDPM) [12] and time series diffusion model called Diffusion-TS [22] .We partitioned each dataset into training, validation, and testing, according to a 8:1:1 ratio. We repeated the experiments 5 times. Prior to applying the various algorithms, the time series data is standardized to have zero mean and unit variance. The encoder component of our framework employs a 3-layer transformer architecture, accompanied by fully connected layers. The training was facilitated using the Adam optimizer. For all the methods, we perform extensive hyperparameter tuning on the length of the embedding vector, the number of hidden layers, the number of nodes, the learning rate, and the batch size. The optimal hyperparameters were determined using the Ray Tune framework, integrating an Asynchronous Successive Halving Algorithm (ASHA) scheduler to enable early stopping. All experiments were conducted on NVIDIA T4 GPU.To assess the effectiveness of the proposed framework, we utilize four metrics: Jensen-Shannon (JS) Divergence, KL Divergence, CRPS (Continuous Rank Probability Score), and Predictive Score. The first three metrics examine how well the generated samples fit the original data distribution. The Figure 5 : Comparison of block maxima distribution and all values distribution for real and generated samples using the proposed FIDE model and DDPM [12] when applied to the synthetic AR(1) dataset.fourth metric, Predictive Score [21] , evaluates the generative model's ability to replicate the temporal characteristics of the original data. This is done by training an LSTM-based sequence model for time series forecasting using the synthetic samples produced by each generative model. The model's performance is measured by its mean absolute error (MAE) on the original test data, providing insight into how well the generative model preserves the temporal patterns of the data. In short, the evaluation focuses on forecasting block maxima on the test dataset using the model trained on generated data.Table 1 compares the performance of FIDE against the various baselines in terms of their ability to capture the block maxima distribution for 5 diverse datasets (AR1, Stock, Energy, Temperature, and ECG). In terms of the distribution metrics (JS divergence, KL divergence, and CRPS), FIDE consistently achieves the best results, providing evidence of FIDE's superior performance in preserving the block maxima distribution. For the Predictive Score metric, FIDE achieves the best results in 3 out of 5 datasets and ranks second in the remaining 2 datasets. To further illustrate FIDE's capabilities, Figures 5-(a ) and (b) compare the distribution of block maxima values generated by DDPM [12] and FIDE for the AR(1) dataset. Note that, while DDPM struggles to capture the block maxima distribution accurately, FIDE generates samples that more faithfully preserve the fidelity of the distribution. This improvement is particularly noticeable in the upper tail behavior, which is critical for applications that require precise modeling of extreme block maxima values. This superior performance is not surprising as it directly results from our method's emphasis on block maxima distribution, achieved through the introduction of frequency inflation, conditional generation based on block maxima, and incorporation of the GEV distribution into the generative modeling framework.As FIDE prioritizes the accurate modeling of block maxima, we have also evaluated its efficacy in capturing the distribution of all (block maxima and non-block maxima) values in time series. The results are shown in Tables 2 and 4 (in Appendix E). Note that FIDE achieves comparable performance to state-of-the-art methods like DDPM [12] and Diffusion-TS [22] . This is further illustrated by the distribution plots of all values for DDPM and FIDE given in Figure 5-(c ) and (d). The results in Table 2 also show that FIDE consistently outperforms VAE-based, GAN-based, and Flow-based alternatives.For Predictive Score, while TimeGAN and TimeVAE show marginally better results, FIDE maintains competitive performance against other baseline methods. These results suggest minimal performance degradation when applying FIDE to time series data. Despite its emphasis on block maxima values, this does not significantly compromise its ability to model the overall distribution. This positions FIDE as a robust and versatile generative model for capturing extreme values in time series.In our ablation study depicted in Table 3 , we systematically assessed the individual contributions of each component within our proposed framework. By selectively deactivating elements such as the GEV loss, conditional block maxima input, and high-frequency inflation module, we observed consistent performance degradation across all scenarios. Notably, the absence of the conditional block maxima input significantly impacted the Jenson-Shannon Divergence and KL Divergence metrics, while the lack of the GEV loss had the most pronounced effect on the CRPS metric. Surprisingly, the predictive score remained relatively resilient to the deactivation of any single component, suggesting a degree of redundancy or compensatory mechanisms among the remaining components. Overall, our ablation study highlights the indispensable role of each component in achieving optimal performance in our model. In summary, our findings underscore the holistic importance of the individual components, with their synergistic interplay contributing to the overall effectiveness of FIDE. 
conclusions: 
 This framework examines the challenges of applying diffusion models to capture extreme values in time series. Through a comprehensive exploration of the constraints within current diffusion-based models, the proposed FIDE framework addresses these limitations by introducing a novel strategy to maintain high-frequency components of the time series. FIDE extends conventional diffusion models to enable conditional generation of block maxima by integrating a loss function based on the generalized extreme value (GEV) distribution. The superiority of the framework over various baseline methods is validated through rigorous experiments on both synthetic and real-world data. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: See 5. In all Tables, we report the standard deviation with 5 runs.Guidelines:• The answer NA means that the paper does not include experiments. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: It conforms to the code of ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: It does not use user data and it verifies all the information provided. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: NA Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [NA] Justification: NA Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Time series generation has been a subject of extensive research, leveraging a variety of statistical [23] and machine-learning [16, 21] techniques to capture temporal dependencies and complexities within data. Generative methods, including Generative Adversarial Networks (GANs) [10] , Variational Autoencoders (VAEs) [13] , normalizing flows [18] , and diffusion-based approaches [12, 19] , have demonstrated efficacy in time series generation and garnered interest due to their ability to learn underlying data distributions for data generation. Normalizing flows are constrained by their computational complexity, limited expressiveness, and suboptimal sample quality, thereby restricting their capacity for effective modeling. Numerous works have delved into enhancing GANs, introducing variations like RcGAN [15] and TimeGAN [21] , which have demonstrated improvements in generating realistic time series data. TimeGAN [21] specifically adopts a GAN architecture to generate time-series data, employing an encoder and decoder to transform a time-series sample into latent vectors. However, GAN-based generative models are susceptible to issues like mode collapse and unstable behavior during training. While VAEs have not been extensively applied to synthetic time series generation, their effectiveness in addressing related challenges, such as time series imputation [8] , suggests their potential utility in this domain. Diffusion-based models are also gaining traction for their ability to generate high-quality data such as images and videos, bypassing the challenges associated with discriminator networks in GANs and avoiding the artifact-prone lower-dimensional latent spaces of VAEs. There are a couple of diffusion-based works [20, 2] that have been employed for time series, but they are specifically designed for discriminative tasks.While generative AI for time series offers numerous advantages, it has not been extensively explored, especially in terms of modeling extreme values. The difficulty of modeling extremes using generative models such as normalizing flows [14] This section presents the relationship between abrupt block maxima and high frequency components of a time series. Definition 1 (Abrupt Block Maxima). Let, x 0 ∈ R T be a time series of length T and y 0 ≡ x τ 0 = max t∈1,...,T x t 0 be its block maxima, where τ is the time step of the block maxima. Then, x τ 0 is considered an abrupt block maxima if dx dt | t=τ > ρ, where ρ is a threshold. We argue that the block maxima values in real-world time series often exhibit an abrupt change behavior compared to the non-block maxima values. To substantiate this, we conduct an empirical analysis across five distinct datasets, wherein we assess the percentile distribution of the first-order derivatives associated with the block maxima values, as depicted in Figure 6 . Specifically, given a time series, we first partition it into a set of disjoint time windows and compute the block maxima value within each window along with the first-order difference, ∆x t = x t 0 -x t-1 0 , for each time step t. We then compute the percentile of ∆x τ associated with the block maxima x τ 0 of the window and plot its distribution using a boxplot as shown in Figure 6 . Our findings affirm the conjecture that the block maxima tends to exhibit an elevated value for its time derivative (with a median larger than 70% of all first-order differences), thereby indicating a notable association between block maxima occurrences and the abrupt changes in a time series.More importantly, the abrupt changes are strongly influenced by the high frequency components of the time series. This can be observed by differentiating the inverse Fourier transform shown in Equation 2 and decomposing the derivative into low and high frequency components:dx t m,0 dt = 1 T T k=1 f k m,0 d dt e i2πtk/T = 1 T T k=1 i2πk T f k m,0 e i2πtk/T = 1 T T k=1 iω k f k m,0 e iω k t ,where ω k = 2πk T . Let κ be the threshold index for dividing the frequencies into low (k ≤ κ) and high (k > κ) frequency components. Then, we have:EQUATIONTo illustrate the impact of high frequency components on the abrupt changes, we compute the value of the second term in Equation ( 7) for time steps with abrupt 1 (A) and non-abrupt (S) changes for various datasets. The results shown in Fig. 7 suggest that the sum of high frequency terms for abrupt changes is consistently higher than the sum of high frequency terms for non-abrupt changes. In essence, the abrupt block maxima values often manifest as high-frequency components in the Fourier domain as they introduce sharp transitions in the time domain signal. This explains the high residual shown in Fig. 2 for the block maxima when the high frequency components are zeroed out.Let ϵ t n be the Gaussian noise added in the diffusion step n during the forward process of the diffusion model and x t 0 be the original value in the time series. The perturbed signal at diffusion step n can be expressed as: x t n = x t n-1 + ϵ t n . Let σ 2 n and σ 2 ϵn be the variances of perturbed time series and noise respectively at diffusion step n while σ 2 n-1 is the variance of the time series at diffusion step n -1.Note that σ 2 ϵn increases linearly according to a linear noise scheduler as the diffusion step increases. Let S f n (ω k ) and S E n (ω k ) denote the power spectral density (PSD) for the k-th frequency component of the perturbed time series and noise respectively for the diffusion step n, where ω k = 2πk T . Our theoretical analysis is based on the following assumptions: Assumption 1. An abrupt block maxima is linked to the high-frequency components of the time series. Assumption 2. The noise ϵ t n is a stationary random process with a constant power spectral density (PSD) for diffusion step n, i.e., ∀k : S E n (ω k ) ≈ σ 2 ϵn . Assumption 3. The power spectral density (PSD) of the perturbed time series for diffusion step n can be modeled using the following generalized Gaussian function:S f n (ω k ) = σ 2 n • exp(-α n |ω k | βn ) (13)where α n is a scaling factor, and β is a shape parameter. Remark 1. The rationale and supporting evidence for Assumption 1 is presented in Section 3. Remark 2. Assumption 2 is intuitive as the Gaussian noise used in the diffusion model has approximately constant PSD over the frequency range of interest. Remark 3. Assumption 3 is reasonable as for most real-world time series, the energy spectrum is localized at the lower frequency (ω k ≈ 0), also known as the fundamental frequency, which quickly decays with increasing frequency (ω k → ω max ) [5] . Therefore, we use the generalized Gaussian function to model this decaying behavior. Note that, the exponential decay behavior will eventually transition into a uniform distribution according to the diffusion model's forward process. Consequently, the shape β n of the distribution function (Eqn 13) is not constant; instead, it evolves with the diffusion step n to remain consistent with the diffusion model's forward process. Initially, when n is small, 1 ≤ β n ≤ 2, representing an exponential decay. However, as n → N , where N is the final diffusion step, β n → ∞, and the PSD transitions to a uniform distribution. This transformation occurs because, according to the forward process of the diffusion model, as n increases, Gaussian noise with linearly increasing variance is added to the perturbed time series, making the signal increasingly noise-like until it becomes white noise at step N . Lemma 1. The difference between the variance of the perturbed time series x t n and the variance of the noise ϵ t n is equal to a constant ζ such that:EQUATIONwhereζ = σ 2 0 + n-1 i=1 σ 2 ϵiProof. First, we have the following:EQUATIONApplying the variance operator to both sides of Equation 15 and utilizing the property that the variance of a sum of independent random variables is the sum of their individual variances, we obtain:EQUATION)where σ 2 ϵn denotes the variance of the noise at step n. Recursively applying Equation 16, we can express the variance of the perturbed time series at step n as:EQUATION)Subtracting σ 2 ϵn from both sides of Equation 17, we get:EQUATIONTherefore, the difference between the variance of the perturbed time series and the variance of the noise at step n is equal to the constant ζ = n-1 i=1 σ 2 ϵi , which completes the proof.The lemma establishes a crucial bound on the difference between the variances of the perturbed time series and the noise, which is leveraged in the subsequent theorem to analyze the behavior of the Fourier transform of the perturbed time series at low and high frequencies.We now provide the proof for Theorem 1 in the main paper.Proof for Theorem 1 Using Lemma 1 and Assumptions 1, 2, and 3, we can prove the theorem as follows: For low frequencies, i.e., ω k → 0, taking the limit as ω k → 0 on the expression for the signal power spectral density S f n (ω), we have:lim k→0 S f n (ω k ) = lim k→0 σ 2 n • exp(-α n |ω k | βn ) = σ 2 n since exp(-α n |0| βn ) = exp(0) = 1. Therefore, as k → 0, S f n (ω k ) = |f k n | 2 approaches the variance σ 2n . So, we can write:lim k→0 S f n (ω k ) = lim k→0 |f k n | 2 = σ 2 n = ζ + σ 2 ϵn = σ 2 0 + n-1 i=1 σ 2 ϵi + σ 2 ϵn = σ 2 0 + n i=1 σ 2 ϵiSimilarly, for high frequencies, i.e., k → k max , taking the limit as ω → ω max on the expression for the signal power spectral density S f n (ω k ) (Eq. ( 13)), we have:lim k→kmax S f n (ω k ) = lim k→kmax σ 2 n • exp(-α n |ω| βn ) → σ 2 n • δ where δ = f kmax n ≪ 1. As k → k max , S f n (ω k ) approaches σ 2 n • δ.We can also write:lim k→kmax |f k n | 2 = δ • ζ + σ 2 ϵn = δ • σ 2 0 + n i=1 σ 2 ϵiTaking the ratio of high-frequency and low-frequency components after perturbations yields:lim k→kmax |f k n | 2 lim k→0 |f k n | 2 = δThereby, high-frequency components or abrupt block maxima dissipate rapidly compared to lowfrequency components or smooth changes. Our findings shed light on a fundamental limitation of diffusion models while modeling block maxima and underscore the need for tailored approaches to preserve the block maxima consistent with the other values and to address the accurate representation of block maxima distributions.We now provide the proof for Theorem 2 in the main paper.Proof. Using Lemma 1 and Assumptions 1, 2, and 3, we can prove the theorem as follows: For low frequencies, i.e., ω k → 0, taking the limit as ω k → 0 on the expression for the signal power spectral density S f n (ω), we have:lim k→0 S f n (ω k ) = lim k→0 σ 2 n • exp(-α n |ω k | βn ) = σ 2 n since exp(-α n |0| βn ) = exp(0) = 1. Therefore, as k → 0, S f n (ω k ) = |f k n | 2 approaches the variance σ 2n . So, we can write:lim k→0 S f n (ω k ) = lim k→0 |f k n | 2 = σ 2 n = ζ + σ 2 ϵn = σ 2 0 + n-1 i=1 σ 2 ϵi + σ 2 ϵn = σ 2 0 + n i=1 σ 2 ϵiSimilarly, for high frequencies, i.e., k → k max , taking the limit as ω → ω max on the expression for the signal power spectral density S f n (ω k ) (Eq. ( 13)), we have:lim k→kmax S f n (ω k ) = lim k→kmax σ 2 n • exp(-α n |ω| βn ) → σ 2 n • δ ′ where δ ′ = δ • γ As k → k max , S f n (ω k ) approaches σ 2 n • δ • γ.We can also write:lim k→kmax |f k n | 2 = δ • γ • ζ + σ 2 ϵn = δ • γ • σ 2 0 + n i=1 σ 2 ϵiTaking the ratio of high-frequency and low-frequency components after perturbations yields:lim k→kmax |f k n | 2 lim k→0 |f k n | 2 = δ • γProof for Theorem 3Proof. The proof begins by expressing the negative log-likelihood of the joint distribution -log p θ (x 0 , y 0 ) in terms of conditional probabilities:EQUATIONWe performed our experiments using the following datasets. (1) Synthetic Data (AR2): AR(2) dataset comprises synthetic time series data generated using an autoregressive model of order 2.(2) Financial Data (Stocks): It features continuous-valued and aperiodic sequences, such as daily historical Google stocks data spanning from 2004 to 2019. We consider the adjusted closing price data for this work. (3) Energy Data (Appliance Energy): The UCI Appliances energy prediction dataset [3] encompasses multivariate, continuous-valued measurements. We consider appliance energy data for analysis. The original dataset [9] for "ECG5000" originates from a 20-hour long electrocardiogram (ECG) obtained from the Physionet database. Specifically, it is derived from the BIDMC Congestive Heart Failure Database (chfdb), with the record labeled as "chf07." The processed data encompasses 5,000 heartbeats randomly selected from the original dataset.Table 4 reports the evaluation of all values in time series using JS Divergence and predictive score, comparing the performance of our proposed method against baseline methods. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• The full details can be provided either with the code, in appendix, or as supplemental material.This research is supported by the U.S. National Science Foundation under grant IIS-2006633. Any use of trade, firm, or product names are for descriptive purposes only and do not imply endorsement by the U.S. Government.A time step has abrupt change if its dx dt is in the top 90%. 


Decision: This paper introduces FIDE (Frequency Inflated Diffusion Estimation), a novel approach for generating time series data that better captures extreme values. The method combines high-frequency inflation, conditional diffusion modeling, and integration of the Generalized Extreme Value (GEV) distribution.

Strengths highlighted by the reviewers:
- Addresses a significant and underexplored problem in time series generation.
- Innovative combination of techniques: high-frequency inflation, conditional diffusion modeling, and GEV distribution integration.
- Comprehensive experiments across various datasets demonstrating improved performance.
- Clear presentation and mathematical formulation.

Weaknesses:
- Some questions about comparisons with recent state-of-the-art diffusion models.
- Potential trade-off between capturing overall distribution and extreme values could be further explored.

The paper presents a novel and important contribution to the field of time series generation, with a focus on preserving extreme values. The majority of reviewers recognize its significance and technical soundness. While some concerns were raised, the authors have adequately addressed most of them in their rebuttal.

Reviews: review: paper_summary: paper_summary: The paper presents a novel generative model designed to better capture extreme values in time series data. FIDE introduces a high-frequency inflation strategy to prevent the loss of extreme values, integrates conditional diffusion modeling to condition on block maxima, and incorporates the Generalized Extreme Value (GEV) distribution to ensure the accuracy of extreme value representation. Empirical results show that FIDE outperforms existing methods in maintaining the distribution of extreme events across various datasets, making it a practical application in the generative modeling of time series data.
strength_weakness: strength_weakness: 1. Although inflating high-frequency signals is intuitive, the possible influence on the fidelity of generated time series is not discussed.

2. The compared baselines, especially diffusion-based approaches, are not enough. For example, there are several works developing diffusion models for general time series generation (Ref-1), or domain-specific time series generation (Ref-2,3). Also, I think those diffusion-based probabilistic time series forcasting approaches can also be used for generation.

Ref-1 Yuan, X., & Qiao, Y. (2024). Diffusion-ts: Interpretable diffusion for general time series generation. ICLR24

Ref-2 Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2020). Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761.

Ref-3 Zhou, Z., Ding, J., Liu, Y., Jin, D., & Li, Y. (2023, November). Towards generative modeling of urban flow through knowledge-enhanced denoising diffusion. Sigspatial23.
questions: questions: Please answer my listed weaknesses above.
limitations: limitations: None.
paper_summary: paper_summary: The article introduces FIDE (Frequency Inflated Diffusion Estimation), which is geared towards better capturing
extreme values when generating time series through diffusion models, which the authors stress as crucial in domains like climate
science and disaster preparedness.

The approach involves inflating high-frequency components and conditionally generating samples based on block maxima, integrating
the GEV distribution to ensure fidelity in extreme event representation.

The authors run experiments comparing their approach to GANs and VAEs across a diverse set of datasets (AR1, Stock, Energy, Temperature, ECG).
They compare both the overall data distribution and extreme values and find their approach to show promising performance.
strength_weakness: strength_weakness: It is not clear to me from the presented results whether there is a trade-off between being able to capture the overall distribution of values well and capturing extreme values of the distribution. The experiments seem to suggests that that might be the case, but it could partially just be a matter of picking a different loss-function during selection of the hyperparameter for the GEV loss. It would be good to get more clarity on this aspect.
questions: questions: What would be a model or situation where Assumption 1 is violated? I.e. the statement is that block maxima are related to high frequency components in _many_ real-world time series. Are there any notable exceptions and would one need a completely different modeling approach to capture those, or would your method still allow one to sensibly generate those time series even though the Assumption is violated?
limitations: limitations: The paper adequately addresses its limitations and does not appear to have negative societal impact.
paper_summary: paper_summary: This article discusses the shortcomings of the insufficient ability to focus on maximum values when applying DDPM in the field of time series generation, and proposes a new framework to overcome this problem by introducing a high-frequency expansion strategy in the frequency domain to ensure the emphasis on high-frequency values. At the same time, the article also proposes a generative modeling method based on conditional diffusion.
strength_weakness: strength_weakness: 1. The article does not set up an ablation experiment to explore the effects of the improved strategy.

2. The article has a small amount of experiments and incomplete research.
questions: questions: 1. In Chapter 3, the article mentioned that adding linear noise to the diffusion model will reduce high-frequency components faster. There are many ways to add noise in the diffusion model, such as the Noise Schedule in iDDPM. Is there any review on this part? Discussion and experimentation?

2. Should the Transformer in Figure 4 in Chapter 4 be a Transformer Encoder?

3. In the field of time series generation, the conditional generation of diffusion models has been studied in the article DiffusionTS. What is the difference between the two?

4. In the selection of comparative algorithms, there have been some time series generation methods based on flow methods, eg: Fourier Flow. Do articles based on this method have good performance in the distribution of block maximum values?

5. The article has mostly discussed the issue of block maximum value distribution in time series generation. Should we discuss the performance of the data generated by enhancing the block maximum value distribution in downstream tasks?
limitations: limitations: NA
paper_summary: paper_summary: The problem statement, solution and its mathematical formulation is nicely presented in the draft. 

Authors have identified a problem in the time series generation process which is under explored and they propose a new algorithm Frequency-Inflated Conditional Diffusion Model (FiDE) to address this. The problem lies in the modelling of block maxima which are related to extreme events in time series data where traditional diffusion models fail or does not take into account properly. 

Authors demonstrated that these extreme values correspond to high frequency fourier components in time series and proposed the algorithm to inflate the weights of high frequency fourier components. The inflation of high-frequency components in the Fourier domain to prevent their premature dissipation during the diffusion process. This approach enables the model to retain significant extreme values that are typically lost in traditional modeling techniques. 

The paper presents detailed performance comparison with several classes of models, e.g. VAE, diffusion, and GAN using datasets from several domains. The comparison is extended to 4 metrics explaining the motivation to use them.
strength_weakness: strength_weakness: The results [Fig 5] show the comparison of block maxima values for only one dataset. More details on the dataset and more comparison would be helpful to confirm the generality of the proposed algorithm. 

Figure 5 shows the final results. It shows the improvement in the block maxima modelling but at the same time it does show degradation in modelling the bulk. An additional comparison of ALL - block maxima with a quantitative estimate of degradation in the performance is needed to understand the full impact of FIDE. 

The table shows the comparison of metrics for block maxima because this is the main focus of this paper. Understood! But at the same time to make it a fair comparison the quantification on the ALL - maxima block is also needed to fully evaluate the model. With the present table and figures it is challenging to quantify if the model is doing "more good" than "bad". 

Authors do mention in Results section L 257-261 that DDPM has slightly better performance but it is not quantified.
questions: questions: The paper is generally well-written, but a few important points remain unanswered, which piques my curiosity. They are as follows:

- When defining the block maximum what is the time window? What criteria defined this time window? Is there a thumb rule or model to choose it dynamically? 

- A discussion on error (unless missed) is needed. How are errors estimated on the metric reported in table 1. What are these errors (statistical/systematic or something else)?
limitations: limitations: - More experimental comparison like figure 5 are really needed in the paper draft for further clarity. 
- What is the effect of FIDE on all samples (not just block maxima), a number would be useful to quote similar to block maxima.

Comments: comment: **Regarding Q3:** 

We thank the reviewer for suggesting the Wilcoxon-Holm test to further validate the performance improvement of our proposed method relative to DiffusionTS. Although we typically use t-test statistics, we have conducted the Wilcoxon-Holm test as recommended. This analysis was performed on results from three datasets across four performance metrics for both our method and DiffusionTS, which serves as the second-best baseline in most cases. Out of 12 comparisons (3 datasets × 4 metrics), our method demonstrated statistically significant performance improvements over DiffusionTS in 10 cases, with p-values less than 0.05. For the other two cases, the p-values were 0.09 and 0.154. 

DiffusionTS struggles to effectively capture extreme values due to its primary focus on generating and reconstructing entire time series, rather than specifically preserving the distribution of extreme values. According to the objective function (Eq. 10) of DiffusinTS, it aims to minimize mean squared error in the time domain and frequency domain. This approach leads the model to prioritize the central tendency of the data, resulting in a strong focus on predicting the conditional expectation. However, this focus tends to underrepresent extreme values, which are often located in the tail of the distribution. Our proposed method addresses this limitation by incorporating both mean squared error and a Generalized Extreme Value (GEV) loss (see Eq. 11 in our paper). This combination allows our model to simultaneously maintain overall accuracy and accurately capture the distribution of extreme values, which is critical for applications that rely on modeling rare events.

Furthermore, DiffusionTS's objective function (Eq. 10) and methodology do not adequately account for the preservation of high-frequency components, which are crucially linked to extreme values, as demonstrated both empirically and theoretically in our paper. The mean squared error applied to Fourier coefficients in DiffusionTS treats low and high frequency components equally. However, time series are typically dominated by low-frequency components, with most of the energy or power spectral density concentrated in these regions. High-frequency components, despite their importance for extreme values, contain much less energy. Consequently, this equal treatment often results in overlooking or underemphasizing these critical high-frequency components. Our proposed method addresses this limitation by introducing a targeted strategy to inflate high-frequency components. This approach ensures that these components do not prematurely fade out during the diffusion model's noising process, allowing them to dissipate at a rate comparable to low frequency components, thus better preserving the characteristics of extreme values.

**Regarding Q4:** 

We employed comparable experimental setups for DiffusionTS, Fourier-Flows, and RealNVP as we did for our proposed method and other baseline approaches. To ensure a fair comparison, we carefully tuned the general hyperparameters (number of epochs, learning rate, batch size) for all methods under evaluation. To account for variability and ensure fairness in our comparisons, we conducted 5 independent runs for each method. We then reported the mean and standard deviation of all performance metrics across these runs.

**Regarding the last comment:** 

We appreciate the reviewer's comment about the challenges of data scarcity in time series generation tasks. However, it's important to clarify that our research addresses a distinct problem: preserving extreme values in time series generation. This preservation of extreme values remains crucial regardless of whether the underlying context involves small-scale or large-scale data. We acknowledge the reviewer's suggestion to verify our method's performance on small-scale datasets. While this is indeed an interesting direction for further investigation, given the limited time remaining for authors' response, we were unable to conduct such an analysis at this time. Finally, as shown in the paper (figure 5), while current diffusion models can effectively capture the general pattern of a time series in the context of large scale data, they fail to capture the distribution of extreme values. Effective generative modeling of the extreme values is significant for developing robust risk management strategies and enhancing disaster preparedness measures, as noted in the introduction.
Thanks. I am not satisfied with the author's rebuttal. there are still a lot of issues that need to be clarified here.

Q1: The motivation for using this strategy requires further analysis and validation.

Q3: Based on the numerical experimental results, it is difficult to determine whether the performance improvement of the proposed method compared to diffusionTS is due to capturing extreme values. The differences between the two methods need further explanation. Why can't diffusionTS capture extreme values, but your method can? What is the motivation and principle behind this? You should use Wilcoxon-Holm analysis the results in Table 1 to demonstrate the advantages and disadvantages of the proposed method.

Q4: What were the experimental setups for diffusionTS, Fourier-Flows, and RealNVP? How is the fairness of the comparison ensured?

W1: Wilcoxon-Holm analysis should be used to demonstrate that the results in Table 2 indeed show a significant performance improvement. The mean values alone do not sufficiently indicate that your results are superior. I also do not think it is a good idea to place the ablation experiments in the supplementary materials.

The core challenge in time series generation tasks is the scarcity of data, yet diffusion models require large amounts of data for training. Please verify the performance of the proposed method on small-scale datasets, such as 10% of the stock dataset. I am certain that diffusionTS cannot handle this; please provide an explanation and validation. What is the significance of generating time series in the context of large-scale data?