FIDE: Frequency-Inflated Conditional Diffusion Model for Extreme-Aware Time Series Generation
Authors: Asadullah Hill Galib
Pang-Ning Tan
Lifeng Luo
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion model
time series
extreme values
Primary area: machine_learning_for_physical_sciences
TL;DR: A frequency-inflated conditional diffusion model that enhances time series generation by preserving extreme values distribution

Abstract: Time series generation is a crucial aspect of data analysis, playing a pivotal role in learning the temporal patterns and their underlying dynamics across diverse fields. Conventional time series generation methods often struggle to capture extreme values adequately, diminishing their value in critical applications such as scenario planning and management for healthcare, finance, climate change adaptation, and beyond. In this paper, we introduce a conditional diffusion model called FIDE to address the challenge of preserving the distribution of extreme values in generative modeling for time series. FIDE employs a novel high-frequency inflation strategy in the frequency domain, preventing premature fade-out of the extreme value. It also extends traditional diffusion-based model, enabling the generation of samples conditioned on the block maxima, thereby enhancing the model's capacity to capture extreme events. Additionally, the FIDE framework incorporates the Generalized Extreme Value (GEV) distribution within its generative modeling framework, ensuring fidelity to both block maxima and overall data distribution. Experimental results on real-world and synthetic data showcase the efficacy of FIDE over baseline methods, highlighting its potential in advancing Generative AI for time series analysis, specifically in accurately modeling extreme events.

FIDE: Frequency-Infatted Conditional Diffusion Model for Extreme-Aware Time Series Generation
Asadullah Hill Galib, Pang-Ning Tan, and Lifeng Luo Michigan State University
Emails: (galibasa, ptan, lluo] @msuuedu
Abstract
Time series generation is crucial aspect of data analysis, playing pivotal role in learning the temporal patterns and their underlying dynamics across diverse fields. Conventional time series generation methods often struggle capture extreme values adequately, diminishing their value in critical applications such as scenario planning and risk management for healthcare, finance, climate change adaptation, and beyond. In this paper, we introduce a conditional diffusion model called FIDE address the challenge of preserving the distribution of extreme values in gen- erative modeling for time series. FIDE employs novel fegg-frryeeccy inflation strategy the frequency domain, preventing premature fade-out of the extreme values. It also extends the traditional diffusion-based model, enabling the genera- tion of samples conditioned on the block maxima, thereby enhancing the model's capacity capture extreme events. Additionally, the FIDE framework incorporates the Generalized Extreme Value (GEV) distribution within its generative modeling framework, ensuring fidelity to both block maxima and overall data distribution. Experimental results on real-world and synthetic data showcase the efficacy FIDE over baseline methods, highlighting its potential in advancing Generative Al for time series analysis, specifically in accurately modeling extreme events.
Introduction
have revolutionized the AI landscape, demonstrating their broad applicability across diverse domains, including computer vision and natural language processing. Such models are designed to learn the underlying data distribution and exhibit resilience to overfitting while promoting automatic feature extraction. Diffusion- based models [2 19). in particular, have emerged as popular generative AI method due to their capability to generate realistic, high-quality data. This paper examines the application of diffusion-based models for time series generation. In particular, we investigate the following issue: How well do existing diffusion models preserve the fidelity of extreme values (i.e., tail distribution) of the original time series?
Generative models 18
10 13
The modeling of extreme values time series essential for informed decision-making across diverse applications, including weather forecasting, earthquake prediction, and disease outbreak detection. Effective generative modeling of these extremes is important as it aids in learning the underlying data distribution, facilitating data augmentation, and improving uncertainty estimation, all of which are crucial for developing robust risk management strategics and enhancing disaster preparedness measures. While there has been growing research on applying diffusion models for time series (20 2 their ability to preserve the distribution extreme values remains largely underexplored. In this study, we examine how effectively diffusion models preserve extreme values in the form of block maxima [4), defined as the peak value within a specified time window.
To illustrate the difficulty of modeling the distribution of block maxima, Figure I shows the result of applying the Denoising Diffusion Probabilistic Model (DDPM) 12
synthetic AR(1) dataset.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Real Generated
Real Senerated
04
00
(a) All Values Distribution
(b) Block Maxima Distribution
Figure I: Comparing the distributions of all values and block maxima values for real and generated
samples using DDPM 12 when applied to the synthetic AR(I) dataset.
While DDPM shows proficiency in generating samples that closely align with the overall data distribution (left diagram), it struggles to preserve the distribution of block maxima values (right diagram) when the generated time series is partitioned into disjoint time windows.
In this paper, we identify the key shortcomings of existing diffusion models that hamper their ability to accurately model block maxima values. We then present a novel framework to overcome this limitation. Our key observation that unusually large block maxima values, often linked to abrupt temporal changes, are strongly associated with high-frequency components of the time series. As the feffuusion-basd generative model gradually introduces noise with a linearly increasing variance schedule, it slowly diminishes the long-term trends fow-freyyency components) of the time series while quickly attenuating the gigh-frequency components. These sigh-freuencyy components crucial for reproducing extreme block maxima values. This limitation hampers the accurate representation of the block maxima, necessitating the development new techniques.
To address this challenge, we propose an end-to-end diffusion model framework termed FIDE. First, mitigate the rapid dissipation of geghffreuennyy components the diffusion model, we introduce novel high-frequency inflation strategy within the frequency domain. This strategic augmentation ensures the sustained emphasis on block maxima, preventing their premature fade- out. We further employ conditional diffusion-based generative modeling approach to guide the time series generation by conditioning on their block maxima. To enhance the preservation of block maxima distribution while learning the overall data distribution, we extend the conventional framework with a regularization term in the loss function based on the negative log-likelihood of the Generalized Extreme Value (GEV) distribution. Using these strategies, empirically show that our approach effectively addresses the challenges of learning the overall data distribution while simultaneously preserving the block maxima distribution.
2 Preliminaries
Consider a time series dataset D = [xm.o)mm m=1 comprising of M samples, where each sample
Xm.0 = (x1 "ym_o o: 2m..rr... ... =m" is a univariate time series of finite length T. Let fm.0 E R²" be the Fourier coefficients, whose k-th frequency component is obtained by applying the following discrete Fourier transform on Xm
27tk//T
Σ
x,moo cos
12tt
x sin
2ntk T
)
(1)
The time series can be recovered from its Fourier coefficients using the following inverse discrete Fourier transform:
1
22tkk/T
Σ T
1
fhh
(2ithh + i fh sin
(2rth
(2)
T
1=1
k=1
For brevity, we will drop the sample subscript m when is clear from the context. Let, W/. the k-th frequency in Fourier transform.
2nk be 7
Given a sample Xg, let yo corresponding block maxima value, where yo max .T) 2. The distribution of the block maxima values is governed by the Generalized Extreme Value (GEV)

---

(b)
[b]
Original
FFT
20
20
30
40
Stop
3
Zero- Cero-out high frequencyy
Residual
[Original Reconstructed]
IFFT
60
2.5
0.3
0.5
94
05
(d)
[G]
Figure 2: Removal of high-freegg freu-frreuency components from daily temperature time series significantly alters the magnitude of its block maxima value (at time step 20), evidenced by its high residual.
distribution, whose cumulative distribution function is given as follows
G(y) = exp { 1 E(
1 11
1/5
σ
where y (location), a (scale), and & (shape) are the distribution parameters. Given M independent block maxima values, denoted {y,,0, "01,0, 1/2,0.-' )M,,), with the cumulative distribution function given by Equation (3), the distribution parameters can be estimated using the maximum likelihood approach by minimizing the following negative log-likelihood function:
- log Lavv (,, a, -M log a +
(
+1
M Σ 1=1
log 1 +
91.0
1
M
1
31.0
P
-1/4
t=1
3 On the Rapid Dissipation of Block Maxima in Diffusion Models
While diffusion models have demonstrated remarkable capabilities in learning complex data distribu- tions, significant challenge arises in accurately capturing the distribution of block maxima values, as evidenced by Figure Addressing this shortcoming crucial for enhancing the performance and applicability of these models across various domains. In this section, delve into the root cause of this phenomenon and present insightful observations that shed light on the underlying issue.
Our first key observation reveals connection between block maxima with abrupt changes and the high-frequency components of many real-world time series. Block maxima, often characterized by their rarity and abrupt temporal changes, are intrinsically linked to the high-frequency components of the data. This relationship is observed in many real-world datasets, where the block maxima values do not typically evolve smoothly but rather emerge through large deviations from their adjacent values.
To illustrate this, consider the real-world temperature time series depicted in Figure In this plot, we first transform the time series into its Fourier domain, obtaining its frequency components, and selectively zeroing out its top-5 highest frequency components. We then reconstruct the time series via inverse Fourier transform and compute the difference between the original and reconstructed time series. The recovered signal exhibits a notable distortion around the block maxima value, as evidenced by the larger residual time step 20, where the block maxima value occurs. This suggests that the removal of high figh-freeuency components of time series has significant impact the

---

Evolution
Iteration
Residuals Evolution
1.2
0.8
Volues
0.8
Values
Resieual
0.6
0.4
Origina Perturbbed
0.2
Block Maxima
Block
Medtan UMI
Perturbedd
Median
20
40
100
200
20
40
100 tteratio
200
Time step
Iteration
Time step
(a) Attenuation of Block Maxima by DDPM
(b) Effect of High Frequency Inflation
Figure 3: A comparison of the effects of noise addition by existing DDPM versus high-frequency inflation on the block maxima of generated samples.
accurate representation of block maxima values. A more detailed analysis supporting this argument given Appendix B
Our second key observation unveils a concerning behavior of diffusion models: the addition of noise diminishes high frequency components, i.e., block maxima, at a faster rate compared to other values in the signal. As diffusion-based generative models gradually introduce noise characterized by a linearly increasing variance scheduler, they inadvertently attenuate the signals associated with high-frequency components. These components, as established in our first observation, for accurately reproducing block maxima. Concurrently, the models effectively capture the long-term trends and low-frequency components, which conducive learning the overall data distribution. However, the high frequency components dissipate more rapidly, hindering the model's ' ability to adeptly learn the distribution of the block maxima values.
crucial
Figure 3a illustrates this phenomenon. By tracking the evolution of residuals, or the differences between the original and perturbed time series generated by DDPM, we observe discernible pattern: block maxima dissipate at a faster rate compared to other values, as evidenced by the higher residuals associated with these extreme points. Notably, in the early iterations highlighted by the green circle, the substantially higher residual suggests that the block maxima signal is rapidly transformed into noise, outpacing the dissipation rate of other values. This behavior poses formidable challenge for diffusion models in effectively capturing the distributions the block maxima values.
To substantiate our observations, the theorem below offers a rigorous justification for the rapid dissipation of block maxima during the forward process of the diffusion model (see Appendix [f proof and details). Let Xo be an input sample and be the perturbed sample after iterations of the forward process, where st, = -hh11 + e and e ~ N(0, o2. is Gaussian noise. Due to the linearity of the Fourier transform operator F, we have:
F(x.)= =(xnn))( F(Gn)
= Eh
Theorem Under certain mild ussumptions (see Appendix C. the rutio of high-frequency and low-frequency components after perturbation during the forward process of the diffusion model is:
limp + kmns 1f812
8
(6)
lim f/
where kmax the index of the maximum frequency and 8 pkimax, which is generally close to O.
In short, our findings shed light fundamental limitation of diffusion models while modeling block maxima and underscore the need for a more tailored approach to preserve distribution.
4 Proposed Framework: FIDE
In this section, we present the detailed methodology of our proposed approach, addressing the challenges associated with capturing extreme values of time series within diffusion-based generative models. Figure provides overview of the FIDE framework.

---

High Frequency Components Infiation
Denoising Diffusion Probabilistic Model
P 1n-11
Top frequency components x where A>
Forward Process
4 Fip
High Frequency Infiation
Fo
X
IFFT
1n-
X
N
Reverse Process
Nale
Network
Conditional Reverse Process Network
Transformer
Fositional Embedding
N
Add & norm
Estimating
Denoised Sample
Xn
Linear
Xx
Linear
50 max
Embedding
GEV Distribution Enforcement
X0
-[(0|mxx(X))
(ax(
Fitting GEV
gev
Negative Log Likelihood
max
(X)
g
Sampling using Conditional Reverse Process Network
8
8g
Eg (8 Mo
90
Fitted GEV distribution all block
8gen
Stampting
Figure 4: Proposed FIDE framework for generating time series with extreme events
4.1 High Frequency Components Inflation
In order to counteract the rapid decay of high-frequency components in the frequency domain while adding noise the forward process DDPM, we present a strategy for high-frequency inflation. Let fo = FFT (xo) denote the vector of Fourier coefficients resulting from applying the discrete Fourier transform to the time series Xg- These coefficients are arranged ascending order from lowest to highest frequency. Consequently, the last elements of f, correspond the coefficients associated with the x highest frequencies. Our goal is to inflate the top-k frequency components of fo as follows:
1, if S
I
ifi> T and Fo
O fo
7
where 7 > 1 is the inflation weight and @ denotes the element-wise multiplication.
With the modified coefficients fo. the inverse Fourier transform (IFFT) applied to get the modified time series, Xn = IFFT (fo). containing the inflated high-freyuency components. Here, the high- frequency components are inflated by 1. The following theorem shows how this inflation strategy helps the high- high-frequency components (block maxima) diminish less rapidly in the diffusion forward process compared to before (see Appendix C for proof and details).
5

---

Theorem 2. Let fn be the Fourier coefficient after inflating freuueryy components with factor of such that > 1. Let 5 11 be the Fourier coefficient of the maximum frequency before inflation, and 8' 6. 7 fk be the Fourier coefficien of the maximum frequency after inflation. Then, using Lemma 1 and under certain mild assumptions (see Appendix C the ratio of frequecyy and low- freqerryy components after inflation and perturbations is:
limg-tkma 1
6 6.Y
lim
Thus, by applying high-free gigh-freueenyy inflation, the fegghfrr components including abrupt block maxima will be preserved by factor of compared to the previous case. We can see the effects of this inflation empirically as well. (3b]soww how inflating the high-frequency components helps in preserving the block maxima values for longer iterations of the diffusion model. This enables the block maxima after high-frequency inflation to dissipate at similar rate compared to other values in the earlier iterations. The diffusion model will have more iterations to capture the block maxima signal.
4.2 Forward Process
We use the inflated time series Xy as input time series to be perturbed during the forward process instead of Xo- By adopting Xo the reference for the unperturbed sample, we ensure that the denoising diffusion process takes advantage of the enhanced representation provided by the inflated high-frequency components. This nuanced adjustment contributes the efficacy of our proposed framework in capturing and preserving essential information during the diffusion process.
4.3
Conditional Reverse Diffusion Process
To enable the generation of samples conditioned on block maxima, we extend the conventional diffusion model to a conditional model. Here, the reverse process conditioned on block maxima Yo. Grounded in extreme value theory [4) the block maxima values {wo} are governed by the Generalized Extreme Value (GEV) distribution, distinctly diverging from the distribution all values Xo P9 (xo). This mandates strategic shift in learning objective. Rather than marginally targeting py (xo). our objective now extends to mastering the joint distribution pe (Xo. yo), driven by nuanced understanding of the unique characteristics inherent in extreme events and their crucial impact on the overall distribution. We formally extend the diffusion model's 's marginal distribution to joint distribution in the following theorem (see Appendix for proof and details).
Theorem 3. Consider an extension of the conventional diffusion model from learning a marginal distribution po(xo) to joint distribution pe (xo, yo) conditioned on block maxima yo. In this context, the variational lower bound can be formulated as follows:
log pe (Xo, yoo S IEg log 9 No, Mo) log pe (30)
(8)
(xo:N)
First, adopt Xo as the reference for the unperturbed sample xo as discussed in the previous subsection. After reparameterization and ignoring the weighting term, as suggested by [12), the first term of the variation lower bound can be expressed as:
CDDPM = m. IX,,, n, yo) - u,
Additionally, considering a Generalized Extreme Value (GEV) distribution for block maxima, the second term is simplified as log Laev (H4, a, E), as defined in Eq. 4
The preceding theorem establishes clear link between the variational lower bound and inter- pretable objective loss function:
-log pe (Xo, yo) S CDDPM - Alog Co CoE (, := L
(10)
Here, LDDPM represents the expected reconstruction error between actual and estimated noise, and log LGEV (l, a, captures the negative logelikelihood of the block maxima governed by the GEV distribution.

---

4.4 GEV Distribution Enforcement Module
To enforce fidelity on both the block maxima and overall data distribution, we incorporate the Generalized Extreme Value (GEV) distribution within the DDPM framework following Theorem We first fit GEV distribution using maximum log- likeliikloood estimation with all the block maxima (y0) values in the training data. The fitted GEV distribution is parameterized by 14, and &. denoted as gee = {1u, E. Using the conditional diffusion process, the estimated noise given by (X,, n. Vo). Consequently, the estimated denoised sample can be obtained as: X -En Then, utilizing the fitted GEV distribution, the log-likelihood of the estimated denoised block maxima, Vo (1, ..} X is calculated. This negative log-likelihood, llog CGEV(M, o, E,3o), is finally incorporated into the loss function of training.
B
4.5 Optimization
Algorithm T summarizes the pseudocode for training and Algorithm 2 summarizes the pseudocode
for the sampling step of FIDE. The overall loss function CFIDE is constructed by combining two key components: the DDPM loss CDDPM and the negative log-likelihood of the Generalized Extreme Value (GEV) distribution CGEV. The formulation is expressed follows:
LFIdE m || (Xn, n, vo) 1 X log Larv(u, o, E, 3o)
(II)
where À is hyperparameter controlling the influence of the GEV distribution on the loss.
In this context, LDDPM evaluates the mean squared difference between the estimated noise term En and the true noise term e, within the conditional diffusion process. Its purpose is to guide the generative model towards effectively capturing the underlying data distribution, The second element, - log (GEV(/4 a. Bo), encapsulates the negative likelikll of the GEV distribution. This component assesses how well the fitted GEV distribution aligns with the estimated block maxima values 7/ derived from the denoised samples. Here, the log-likelihood has negative sign to indicate a minimization objective, aligning with the overall goal of minimizing the loss function.
Algorithm 1 Training
Algorithm 2 Sampling
repeat
Input:
Block maxima ÿo
GEV(U6) and
X (xx)) where X = 2p. ... .x,))
Trained Model
fo FFF(xo)
Output: Generate time series, Sn-
fo @ fo
8 ~ NN(0, o²)
Xy IFFT((.)
for N.... do
Uniform( (1. ,J) En ~ I(.
z ~ (0,
8 Van (Sn, 1- En (Rn, n, (o))
Do mxx(xo)
VI-an
X = yan No
U2 En (Rn, yo))
tonz
Yo max
end for return %0
Take the gradient step on
Val VollEn (X., 110) - -
-X log LGEV 0,
until converged
Experimental Evaluation
We have performed extensive experiments to evaluate the performance of our FIDE framework. All the code and datasets used in this paper are available at htttp:://github com/ /on///abbb9//IIE The datasets used are described in Appendix D
We compared our proposed framework against various generative models: (1) GAN-based: We utilize two GAN-based approaches as our baselines. The first approach is Conditional GAN (cGAN [15).. which introduces conditional information to the training process, enabling targeted generation based on specified conditions. The second baseline is TimeGAN (21] which is generative model designed specifically for time-series generation. (2) VAE-based: We employ beta-VAF []], conditional beta- VAE [16), and Time
(6] as bascline methods for comparison. Both beta- VAB and conditional
7

---

Table Comparison of generated samples' block maxima distribution metrics and predictive score
using the various methods. Bold and Underlined entries denote the best and second-bess result
Metries
Methods
ARI
Stock
Energy
Temperature
ECG
beta-VAE
0.0211=0.0187
0.110510.0188
0.0722-10.0095
0.0140:10.0125
0.1210=10.0214
c-beta- VAE
0.0190=0.0125
0.1011=0.0152
0.0710==0.0888
0.0109-:000998
0.1120=0.0352
TAm
0.0015-000003
0.105410.0071
0,0795
0.0096.1000002
0.0985
TimeGAN
0.0840=0.0199
0.141110.1585
0.0950=0.0089
0.0112110.012
0.1620=1.0021
Js
cGAN
0.0690=0.0091
0.1211=0.0205
0.0890=0.0093
0.0091=0.0008
0.1440=000211
Divergence
RealNVP
0.0754=0.0121
0.1185=00.008
0.0905=0.0084
0.0089=+0.0007
0.1411=0.0116
Fourier-Flows
0.0612=000045
0.1108++00015
0.0820=0.0044
0.0078++00010
0.1392=0.0202
DDPM
0.0010-0.0007
0.0752+000082 0.0912+00.002
0.0082++00009
0.1041++..0122
Diflusion-TS
0.0011-0.0008
0.085410.0045
0.0712.10.071
0.00771000000
1.0001
FIDE (Ours)
0.000400.0001
0.0000-0.0061
0.0090=0.0092
0.0001
0.09000.00082
beta- VAR
0.000
0.1947+0.0184
0.1210
0.0410
0.20000000048
c-beta- c-bela-VAE
0.0091-0.0012
0.174410.0105
0.1160
0.0360110.0114
0.1880
Time VAE
0.0105=0.0007
0.251411.0152
0.1625110.005
0.0490.10.00 0006
0.225410.0068
TimeGAN
0.1920=0.0156
0.2425=0.0251
0.1590=0.0198
0.0550=0.0145
0.2540=0.0255 0.00255
KL
cGAN
0.1240=0.012
0.2001
0.1510=0.0021
0.2210=0.0184
Divergence
RealNVP
0.1298-0.0215
0.229511.0014
0,1605
0.0512-10.0108
0.2305110.015
Fourier-Flows
0.1235=0.0104
0.2045
0.1448=000035
0.0013
0.2254
DDPM
0.0062-000008
0.1915=0.0125
0.1120=000108
0.0326:10.0090
0.0004
Diffusion-TS
0.0054-0.0077
0.1889=000018
0.1089=000115
0.0311=0.0078
0.0891
FIDE (Ours)
0.0030- 0.0009
0.0504
0.0950 0.0098
0.0029+ 0.0008
11810-0.0084
beta- VAE
0.1247=0.0189
0.314920.0348
0.2410=00029%
0.155420.0214
0.305
e-beta-VAB VAE
0.1154=00155
0.2698=0.0211
0.2574=0.0241
0.1420=000311
0.0150=0.0414
Time
0.1511-0.0081
0.2547+000155
0.2853-0.1082
0.0847+00007T
0.0252+0.0204
TimeGAN
0.1858-000214
0.0425
0.2685
0.2
0.0401
cGAN
0.1224-0.0157
0.268910.0301
0,238510.0187
0.109010.00144
0.298510.0311
CRPS
RealNVP
0.1325=0.0144
0.2545110.0258
0.2041=0.0214
0.2014110.0344
0.2
Fouricr-Flows
0.1305=0.0254
0.2589=0.0214
0.2015=0.0211
0.1975=0002511
0.2884
DDPM
0.0422-000084
0.242210.0188
0,2199
0.1516110.0211
0.0088
Diffusion-TS
0.0398-000092
0.205811000211
0.2125-10.054
0.1025
0.0415
FIDE (Ours)
0.0310-0.098
0.2115=0.0152
0.2085_0.0985
0.001
0.2245=0.0204
beta- VAB
0.0201
0.9528 10.0314
0.0110
0.681410.0108
0.942010.0142
c-beta-VAE
0.6240=0.0145
0.9226
0.7317-0.0163
0.67181100002 0.25
0,9310-0.0214
Time
0.6150=0.0104
0.914010.0216
0.7325=0.0195
0.672311000366
0,9150-0.0112
TimeGAN
0.6050=0.0104
0.8950=0.0198
0.7280=000187
0.6718:200047
0.800
Predictive
cGAN
0.6120=0.0044
0.9354++00210
0.7310=0.0147
0.6847=000041
0.9290=0.0191
Score
RealNVP
0.6884-0.0011
0.9988+0.0354.
0.7898-0.0254
0.7852+0.0017
0.9730+0.0215
Fourier-
0.6925=000031
0.9844
0.795510.0008
0.7871 0.0021
0,9655.1.0222
DDPM
0.6148-0.008
0.89971100111
0.0000
0.6708 10.009
0.112110.0121
Diffusion-TS
0.6105=0.0044
0.8912+000105
0.7055=000084
0.67081100010
0.908920.0095
FIDE (Ours)
0.6081 000098
0.8171
0.7240
1.6694:000082
0.9040#0.0012
beta- VAE incorporate a specific disentanglement objective to encourage the model to learn more interpretable and factorized representations while VAE [6] promotes interpretability. (3) Flow- based: We use normalizing flows-based approaches such as RealNVP 7 and Fourier-Flows M as our baseline methods. (4) iuffussionbasede We consider two baselines for comparison, namely, the denoising diffusion probabilistic model (DDPM) [12 and time series diffusion model called
Diffusion-T TS 22
5.1 Experimental Settings
We partitioned each dataset into training, validation, and testing, according to a 8:1:1 ratio. We repeated the experiments 5 times. Prior to applying the various algorithms, the time series data is standardized to have zero mean and unit variance. The encoder component of our framework employs 3-layer transformer architecture, accompanied by fully connected layers. The training was facilitated using the Adam optimizer. For all the methods, we perform extensive hyperparameter tuning on the length of the embedding vector, the number of hidden layers, the number of nodes, the learning rate, and the batch size. The optimal hyperparameters were determined using the Ray Tune framework, integrating Asynchronous Successive Halving Algorithm (ASHA) scheduler to enable early stopping. All experiments were conducted on NVIDIA T4 GPU.
To assess the effectiveness of the proposed framework, we utilize four metrics: Jensen-Shannon (JS) Divergence, KI. Divergence, CRPS (Continuous Rank Probability Score), and Predictive Score. The first three metrics examine how well the generated samples fit the original data distribution. The
8

---

Table 2 Comparison of the generated sample distribution for all values using KL divergence and CRPS metrics. Bold and Underlined entries denote the best and second-best result. For the JS Divergence and Predictive Score metrics, the results are given in Appendix E
Metries
Methods
ARI
Stuck 0.0188=0.0016
Energy
Temperature
ECG
beta-VAF
0.0020=0.0003
0.0181=000015
0.0025+000003
0.0041=0.0004
e-beta-VAF
0.0017-0.0004
0.0178+0.0019
0.0177+0.0017
0.0022+0.0004
0.0028+0.0004
TimeVAE
0.0016-0.0003
0.0169110.0015
0.0159-100021
0.001811.0002
0.0026-1..0003
TimeGAN
0.0025=0.0003
0.018210.0025
0.0061-0.0016
0.00211000005
0.0034110.005
KL
cGAN
0.0018=0.0003
0.0178=000018
0.0169:0.0016
0.0015::.0000
0.0029=000003
Divergence
RcalNVP
0.0023=0.004
0.0185=000019
0.0185=0.0017
0.0019=0.0004
0.0036=0.0002
Pouricr-Flows
0.0019=0.0033
0.0173+0.0021
0.0165=0.0015
0.0017=000003
0.0028=000003
DDPM
0.0010-0.001
0.0117+0.0011
0.0114+0.0010
0.0009+0.000
0.0019+0.0003
Diffusion-TS
0.0011-0.0001
0.011411..012
0.0016..0.009
0.001011.0002
0.0003
FIDE (Ours)
0.0012=000001
0.012110.0015
0.010910.0009
0.00111:00002
0.0021110.004
beta- VAE
0.0201-0.0011
0.4955+0:0125
000002
0.0914+0.0010
000029
c-beta-VAr
0.1984=0.0022
0.4205=0.0148
0.5514=0.2100
0.0899==00000
0.1388=000006
TAm
0.1848=000038
0.4841=0.085
0.4815=0.0189
0.0889+0.0009
0.1422=000077
TimeGAN
0.2412-0.0019
0.3941+000115
0.4415-0.0171
0.091 1.0911+0000 1+0000088
0. 0.220++000062
cGAN
0.1974-0.0012
0.4451+0.0201
0.0914=0.0211
0.0903
0.1298 0056
CRPS
RealNVP
0.2511-0.0019
0.425410.0194
0.5125
0.0919110.00 0007
O. 0.1405 100003
Fourier-Flows
0.2214=0.0024
0.381411001144
0.5514-0.0123
0.09 1.001220.0000
1281=000007
DDPM
0.1595=0.0018
0.2054=00.0144
0.3115=0.0154
0.0875=000000
0.00000000062
Diffusion-TS
0.156500.0016
0.2985 1474
0.0285=0.0149
0.0863=
0.1018
FIDE (Ours)
0.3001+0.010
0.3251
0.0893+000007
0.1061 +0.0054
Mei Genenned
Beck Medina
Block Fan
wikes
Uhi RDPM
69 Ding DOPM [k]
Figure 5: Comparison of block maxima distribution and all values distribution for real and generated samples using the proposed FIDE model and DDPM [2] when applied the synthetic AR(1) dataset.
fourth metric, Predictive Score (21), evaluates the generative model's ability to replicate the temporal characteristics of the original data. This done by training an LSTM-based sequence model for time series forecasting using the synthetic samples produced by each generative model. The model's performance is measured by its mean absolute error (MAE) on the original test data, providing insight into how well the generative model preserves the temporal patterns of the data. In short, the evaluation focuses on forecasting block maxima on the test dataset using the model trained on generated data.
5.2 Experimental Results
Table compares the performance of FIDE against the various baselines in terms of their ability to capture the block maxima distribution for 5 diverse datasets (AR1, 1, Stock, Energy, Temperature. and ECG). In terms of the distribution metrics (JS divergence, KL divergence, and CRPS), FIDE consistently achieves the best results, providing evidence of FIDE's superior performance in preserv- ing the block maxima distribution. For the Predictive Score metric, FIDE achieves the best results in 3 out of 5 datasets and ranks second in the remaining 2 datasets. To further illustrate FIDE's capabilities, Figures (a) (a) and (b) compare the distribution of block maxima values generated by DDPM 12 and FIDE for the AR(1) dataset. Note that, while DDPM struggles to capture the block maxima distribution accurately, FIDE generates samples that more faithfully preserve the fidelity of the distribution. This improvement is particularly noticeable in the upper tail behavior, which is critical for applications that require precise modeling of extreme block maxima values. This superior performance is not surprising it directly results from method's emphasis on block maxima distribution, achieved through the introduction of frequency inflation, conditional generation based on block maxima, and incorporation of the GEV distribution into the generative modeling framework.
As FIDE prioritizes the accurate modeling of block maxima, we have also evaluated efficacy in capturing the distribution of all (block maxima and non-block maxima) values in time series. The

---

Table 3 Ablation Study of generated samples' block maxima distribution metrics and predictive score
using the proposed FIDE model and without individual component of the model
Metrics
Methods
Stock
Energy
Temperature
FCG
PIDE frequency inflation
0.0006+0.000
0.089840.0054
0.082220.0084
0.00090000000
0.098400.0058
Jensen-
FIDE conditional
0.0007+0.0000
0.105440.0089
0.0941+0.0098
0.0010+0.0000
0.0
Shannon
FIDE GEV Loss
0,0005+0.0000
0.0813+0.0035
0.0715+0.0041
0.0008+0.0001
0.0922-0.0056
Divergence
FIDE
0.00000000000
0.0680+40.0099 0.0700+0.NM61
0.000700.0001
0,093000.0002
FIDE frequency inflation
0.0042+0.0008
0.155940.0161
0.1054+0.0049
0.0036+0.0006
0.18540.0006
KL
FIDE conditional
0.005 0010
16894000210
0. 10894000095
70041+0.0010
0. 1901 40,0063
Divergence
FIDE
GEV Loss
01001900000
0. 14000188
0. 1000
0,00320000009
0. 01822-011992
FIDE
0.0030+0.00009
0.150440.0122
0.0950+0.0009
0.0029+00000K
0.0800
PIDE frequency inflation
0.0391+0.0008
0.217240.0158
0.2152+0.0791
0.0649+0.0088
0.2372-0.0181
FIDE conditional
0.0335+0.0089
0.2165+0.0132
0.2082-0.0768
0.0651+0.0047
0,2382-000184
CRPS
FIDE GEV
0.0415+0.007
0:223240:0203
0.218950.084
0.0815+0.0102
0.2456-0.0399
FIDE
0.0310+0.0098
0.2115+0.0152
0.208560.09885
0.0517+0.0082
0.2344+0.0244
FIDE frequency inflation
0.607700.00122
0.894240.0158
0.726440.0069
0.6711+0.0091
0.9081 +0.0155
Predictive
FIDE conditional
0.609540.0079
0.8901+0.0141
0.7261+0.0081
0.6715+0.0078
0.9059-0.0122
Score
FIDE GEV Loss
0.608960.0889
0.889140:0122
0.726960.00774
0.6712+0.0009
0.906-40.0058
FIDE
1..081+0.0098
0.8871+0.0104
0.7240+0.0087
0.6694+0.0082
0.9040=0.0112
results are shown in Vables[annde 4 (in Appendix Eb. Note that FIDE achieves comparable performance tfte-of-thhe-at methods like DDPM 12] and Diffusion-TS 22 This is further illustrated by the distribution plots of all values for DDPM and FIDE given in Figure (c) and (d). The results in Table alss show that FIDE consistently outperforms VAE-based, GAN-based, and Flow-based alternatives. For Predictive Score, while TimeGAN and VAE show marginally better results, FIDE maintains competitive performance against other baseline methods. These results suggest minimal performance degradation when applying FIDE to time series data. Despite its emphasis on block maxima values, this does not significantly compromise its ability to model the overall distribution. This positions FIDE robust and versatile generative model for capturing extreme values in time series.
5.3 Ablation Study
In ablation study depicted in Table 3 systematically assessed the individual contributions of each component within our proposed framework. By selectively deactivating elements such as the GEV loss, conditional block maxima input, and high-frequency inflation module, we observed consistent performance degradation across all scenarios. Notably, the absence of the conditional block maxima input significantly impacted the Jenson-Shannon Divergence and KL Divergence metrics, while the lack of the GEV loss had the most pronounced effect on the CRPS metric. Surprisingly. the predictive score remained relatively resilient to the deactivation of any single component, suggesting a degree of redundancy or compensatory mechanisms among the remaining components. Overall, our ablation study highlights the indispensable role of each component in achieving optimal performance in our model. In summary, our findings underscore the holistic importance of the individual components, with their synergistic interplay contributing to the overall effectiveness of FIDE.
Conclusions
This framework examines the challenges of applying diffusion models to capture extreme values in time series. Through a comprehensive exploration of the constraints within current diffusion based models, the proposed FIDE framework addresses these limitations by introducing novel strategy to maintain high-frequency components of the time series. FIDE extends conventional diffusion models to enable conditional generation of block maxima by integrating loss function based on the generalized extreme value (GEV) distribution. The superiority of the framework over various baseline methods validated through rigorous experiments on both synthetic and real-world data.
7 Acknowledgment
This research is supported by the U.S. National Science Foundation under grant I1S-2006633. Any use of trade, firm, or product names are for descriptive purposes only and not imply endorsement by the U.S. Government.
10

---

References
[1] Ahmed Alaa, Alex James Chan, and Mihaela van der Schaar. 2021. Generative Time-series Modeling with Fourier Flows. In International Conference Learning Representations.
[2] Marin Biloš, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, and Stephan Günnemann. 2023. Modeling Temporal Data as Continuous Functions with Stochastic Process Diffusion. In International Conference on Machine Learning. PMLR, 2452-2470.
[3] Luis Candanedo. 2017. Appliances Energy Prediction. UCI Machine Learning Repository. DOI:
[4] Stuart Coles, Joanna Bawa, Lesley Trenner, and Pat Dorazio. 2001. An infroduction to Statistical Modeling of Extreme Values. Vol. 208. Springer.
[5] Jonathan Crabbé, Nicolas Huynh, Jan Stanczuk, and Mihaela van der Schaar. 2024. Time Series Diffusion in the Frequency Domain, arXiv preprint arXiv:2402.0544 0033033333333 (2024).
[6] Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. 2021. VAE A Variational Auto-Encoder for Multivariate Time Series Generation. arXiv preprint arXiv:2111.08095 (2021).
[7] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016. Density Estimation using Real NVP. arXiv preprint arXiv:1605. 08803 (2016).
[8] Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. 2020. GP-VAE: Deep Probabilistic Time Series Imputation. In International conference artificial intelligence and statistics. PMLR, 1651-1661.
[9] Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph Mietus, George B Moody, Chung-Kang Peng. and H Eugene Stanley. 2000. PhysioBank, Physio Tookko, and PhysioNet: components of a new research resource for complex physiologic signals. circulation 101, 23 (2000), e215-e222
[10] Ian Goodfellow, Jean Pouget-Ahadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. Advances in neural information processing systems 27 (2014).
[11) Trina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016. beta-VAE: Learning Basic Visual Concepts with Constrained Variational Framework. In International conference on learning representa- tions.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. Advances in neural information processing systems 33 (2020), 6840-6851.
[13] Diederik Kingma and Max Welling. 2013. Auto-encoding Variational Bayes. arXiv preprint arXiv:1312.0 14 (2013).
[14] Andrew McDonald, Pang-Ning Tan, and Lifeng Luo. 2022. COMET Flows: Towards Generative Modeling Multivariate Extremes and Tail Dependence. In Proceedings of the Thirty-First International Joint Conference Artificial Intelligence, 1JCAI-22.
[15] Olof Mogren. 2016. C-RNN-GAN: Continuous Recurrent Neural Networks with Adversarial Training, arXiv preprint
(2016).
[16] Alexander Nikitin, Letizia lannucci, and Samuel Kaski. 2023. TSGM: A Flexible Framework for Generative Modeling Synthetic Time Series. arXiv preprint arXiv:2305. (202))))
[17] Australian Bureau of Meteorology. 2016. pnprrmaaiim/ddaily -aiimmmmiimmmmmmm mwnimmm-temperattuutime temperatures atuures
https://ww..kgglee.com www.:/gggle...omtasatttttttttet.
[18] Danilo Rezende and Shakir Mohamed. 2015. Variational Inference with Normalizing Flows. In International conference machine learning. PMLR, 1530-1538.

---

[19] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2020. Score- based Generative Modeling through Stochastic Differential Equations. arXiv preprini arXiv:201 1. 13456 (2020).
[20] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. 2021. CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation. Advances in Neural Information Processing Systems 34 (2021), 14804-24814
[21] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. 2019. Time-series Generative Adversarial Networks. Advances in neural information processing systems 32 (2019).
[22] Xinyu Yuan and Yan Qiao. 2024. ISffusiin-TS: Interpretable Diffusion for General Time Series Generation. arXiv preprint arXiv:2403.01742 (2024).
[23] Mingda Zhang. 2018. Time Series: Autoregressive models AR, MA, ARMA, ARIMA. Univer- sity of Pittsburgh (2018).
12

---

A Related Works
Time series generation has been a subject of extensive research, leveraging a variety of statistical
23
and machine-learning T6 21 techniques capture temporal dependencies and complexities within data. Generative methods, including Generative Adversarial Networks (GANs) [TO), Variational Autoencoders (VAEs) [33 normalizing flows [18], and diffusion- -asedd approaches [22] 19 have
demonstrated efficacy in time series generation and garnered interest due to their ability to learn under- lying data distributions for data generation. Normalizing flows are constrained by their computational complexity, limited expressiveness, and suboptimal sample quality, thereby restricting their capacity for effective modeling. Numerous works have delved into enhancing GANs, introducing variations like ReGAN [[S] and TimeGAN [2I] which have demonstrated improvements in generating realistic time series data. TimeGAN [21 specifically adopts a GAN architecture to generate time-series data, employing encoder and decoder to transform time-series sample into latent vectors. However, GAN-based generative models are susceptible to issues like mode collapse and unstable behavior during training. While VAEs have not been extensively applied to synthetic time series generation, their effectiveness addressing related challenges, such as time series imputation [8], suggests their potential utility in this domain. Diffusion-basser to generate high-quality data such as images and videos, bypassing the challenges associated with discriminator networks in GANs and avoiding the artifact-prone lower-dimensional latent spaces of VAEs. There are couple of diffusion- works (20) [ that have been employed for time series, but they are specifically designed for discriminative tasks.
models are also gaining traction for their ability
While generative AI for time series offers numerous advantages, has not been extensively explored, especially in terms of modeling extreme values. The difficulty of modeling extremes using generative models such as normalizing flows [4] has been recognized in previous research. Studies by Wiese et al. (2019) and Jaini et al. (2020) highlight the inability of normalizing flows accurately capture heavy-tailed marginal distributions. Specifically, these studies show that any attempt to map heavy-tailed distributions light-tailed distributions (e.g., Gaussian) cannot maintain Lipschitz- boundedness. However, this challenge remains largely unexplored within the realm of diffusion models.
B Relationship between Abrupt Block Maxima and High Frequency Components
This section presents the relationship between abrupt block maxima and high frequency components of time series.
Box Plot Percentiles for Different Data Sources
100
80
Porcentics
000 00000000000
20
Temperature
Energy Ueta Sources
Figure 6: Percentile distribution of first order derivatives for the block maxima values in different time series datasets. Observe that the derivatives tend to exhibit elevated percentile values.
Definition 1 (Abrupt Block Maxima). Let, X, R be time series of length T and yo = x max ....,, 2 be its block maxima, where T is the time step of the block maxima. Then, z is
considered an abrupt block maxima di t=y > p. where p is threshold.
du
13

---

Box Plot High Frequency Percentiles
1,75
Denbratves Percentil
1.50
1.25
0.75
0.50
0.00
Temperature Temperature
Stock (A)
Stock (5)
Energy (A)
Energy (5)
ECG
ECG
AR1
AR1
(A)
(5)
[A]
[5)
(A)
(5)
Figure 7: The summation of high frequency terms for abrupt (A) changes is consistently higher than for smooth (S) changes. (A) denotes abrupt changes, (S) denotes smooth changes.
We argue that the block maxima values in real-world time series often exhibit an abrupt change behavior compared the non-block maxima values. To substantiate this, conduct empirical analysis across five distinct datasets, wherein we assess the percentile distribution of the first-order derivatives associated with the block maxima values, depicted in Figure 6 Specifically, given time series, we first partition into a set of disjoint time windows and compute the block maxima value within each window along with the first-order difference, Ar* = zá =6 for each time step We then compute the percentile of ArT associated with the block maxima zū of the window and plot its distribution using a boxplot as shown in Figure 6 Our findings affirm the conjecture that the block maxima tends to exhibit an elevated value for its time derivative (with median larger than 70% of all first-order differences), thereby indicating a notable association between block maxima occurrences and the abrupt changes in time series.
More importantly, the abrupt changes are strongly influenced by the high frequency components of the time series. This can be observed by differentiating the inverse Fourier transform shown in Equation and decomposing the derivative into low and high frequency components:
da m dl.
1 T
T
d 12ttt//T
T
12T T
T
pl
iww pl
m. funt
m,O
dl.
k=1
where Wi T
Let be the threshold index for dividing the frequencies into low (k S 1t) and high (k > k) frequency components. Then, we have:
dx dt
1
jun iur
twns
Σ
iwh maammmm.mee ewwe
(12)
T
T
To illustrate the impact of high frequency components on the abrupt changes, we compute the value of the second term in Equation 7 for time steps with abrup (T (A) and non-abrupt (S) changes for various datasets. The results shown in Fig. suggest that the sum of high frequency terms for abrupt changes is consistently higher than the sum of high frequency terms for non-abrupt changes. In essence, the abrupt block maxima values often manifest as high-frequency components in the Fourier domain as they introduce sharp transitions in the time domain signal. This explains the high residual shown in Fig. for the block maxima when the high frequency components are zeroed out.
C Theoretical Analysis
Let ef, be the Gaussian noise added in the diffusion step during the forward process of the diffusion model and wn be the original value in the time series. The perturbed signal at diffusion step n can be expressed as: =/, - < Let of and o² be the variances of perturbed time series and noise respectively at diffusion step n while o is the variance of the time series diffusion step 1.
'A time step has abrupt change if its dx is in the top 90%.
de
14

---

Note that o²,, increases linearly according to linear noise scheduler as the diffusion step increases. Let S (wi)) and S (wx) denote the power spectral density (PSD) for the k-th frequency component of the perturbed time series and noise respectively for the diffusion step n, where W/ T
2
Our theoretical analysis is based on the following assumptions:
Assumption 1. An abrupt block maxima is linked to the high-frequency components of the time series.
Assumption 2. The noise e, is a stationary random process with a constant power spectral density (PSD) for diffusion step n, i.e., vk S (wx) R o²...
Assumption 3. The power spectral density (PSD) of the perturbed time series for diffusion step can be modeled using the following generalized Gaussian function:
S (wx) o cxp(-ax (wk (n))
(13)
where n is scaling factor, and 8n, is a shape parameter. Remark 1. The rationale and supporting evidence for Assumption presented in Section 3
Remark Assumption 2 intuitive as the Gaussian noise used the diffusion model has approxi- mately constant PSD over the frequency range of interest.
Remark 3. Assumption 3 is reasonable as for most real-world time series, the energy spectrum is localized at the lower frequency (w), 22 0), also known as the fundamental frequency, which quickly decays with increasing frequency (wi + Wmax) [3]. Therefore, we use the generalized Gaussian function to model this decaying behavior. Note that, the exponential decay behavior will eventually transition into uniform distribution according to the diffusion model's forward process. Consequently, the shape in of the distribution function (Eqn [3] is not constant; instead, evolves with the diffusion step n. to remain consistent with the diffusion model's s forward process. exponential decay. However,
Initially, when n is small, 1 < Bn I 2, representing where N the final diffussion step. Bn - 20, and the PSD transitions a uniform distribution. This transformation occurs because, according to the forward process of the diffusion model, as increases, Gaussian noise with linearly increasing variance is added to the perturbed time series, making the signal increasingly noise-like until it becomes white noise at step N.
- N,
Lemma 1. The difference between the variance of the perturbed time series zt, and the variance of the noise é", is equal to constant Ç such that:
on
o²2 =
(14)
where Ç = = E1= 02
Proof. First, we have the following:
c
(15)
Applying the variance operator to both sides of Equation 15 and utilizing the property that the variance of sum of independent random variables is the sum of their individual variances, we obtain:
0 o o2
(16)
where o² denotes the variance of the noise at step n. Recursively applying Equation 16 we can express the variance of the perturbed time series at step as:
(17)
Subtracting of from both sides of Equation 17 we get:
n-1
o -2 = Σ
02
(18)
i=1
Therefore, the difference between the variance of the perturbed time series and the variance of the which completes the proof.
noise at step n is equal to the constant < = Σーー oz
11
Li
The lemma establishes a crucial bound on the difference between the variances of the perturbed time series and the noise, which is leveraged in the subsequent theorem analyze the behavior of the Fourier transform of the perturbed time series at low and high frequencies.
We now provide the proof for Theorem in the main paper.
15

---

Proof for Theorem 1
Using Lemma 1 and Assumptions 1 and we can prove the theorem
follows: For low frequencies, i.e., we > 0, taking the limit as W/ signal power spectral density S (w), we have:
on the expression for the
lim S (wx)
lim o exp(-annwuy ww (mn oz k-0
8=0
since exp(-ax exp(-an|0)sm) exp(0) 1. Therefore, as k - 0, S. o So, we can write:
|
approaches the variance
n-1
lim S% k-00
lim | k-0
on = = 0 = o
0 Σ
+ 6n
0
+
d
1=1
Similarly, for high frequencies, i.e., h - kimax, taking the limit as w + Wmmx on the expression for the signal power spectral density S (wx) (Eq. (33), have:
(Wk) lim on sepp(-anlul + o hinc
lim
where 8 flams 1.
As k - kmax, S/ (wx) approaches o d. We can also write:
lim | = 8. (5+ = kmak
00
i=1
Taking the ratio of high-frequency and low-frequency components after perturbations yields:
limg->kaa |
lim
f
Thereby, high-frequency components or abrupt block maxima dissipate rapidly compared to low- frequency components smooth changes. Our findings shed light fundamental limitation of diffusion models while modeling block maxima and underscore the need for tailored approaches to preserve the block maxima consistent with the other values and address the accurate representation of block maxima distributions.
We now provide the proof for Theorem 2 in the main paper.
Proof for Theorem
Proof. Using Lemma I and Assumptions 2 and 3 we can prove the theorem as follows: For low
frequencies, i.e., Wk 0, taking the limit as wi the expression for the signal power spectral density 8 (w:), have:
lim s (wx) k=0
lim of exp(-an/wa! oz k00
since exp(- exp(-a,10)%%) cxp(O) 1. Therefore, as k 0, S
|f approaches the variance
o. So, we can write:
lim S (wx) lim fh oz = 02 = = Σ k=+0
02
+
02_ og +
k-0
1=1
Similarly, for high frequencies, i.e., k + kmax, taking the limit - Wmax on the expression for the signal power spectral density St. (wx) (Eq. (3)), ). we have:
lim S.. Je-
(wx) (w(ww)
lim of exp((aan o k-
16

---

where 8' = 8..9
As + kmax. Sh(wx)
approaches of 8- 6%% We can also write:
n
lim J
f 8. 8.7. (5+ = -..7.
0
Taking the ratio of high-frequency and low-frequency components after perturbations yields:
limg-tk... 1/ = 8 6
lim 1/
Proof for Theorem
Proof. The proof begins by expressing the negative log-likelihood of the joint distribution - log poppo (Xo, yo) in terms of conditional probabilities:
log pø (xo, yo)
log pe (xoo vo) pe() log (ogpo(xollo)) log pe (yo)
S DxL (g(( No: yn) (o))lpp((111)) /x0. vo))
log po (xo vo) log (wo))
E,y log ((x1:N vo)
log po vo)
(19)
po p=(Xo:N)
log po
((X1N//X0 Mo) px(x0.N)
Egy log
log Pe(yo)
D Data
We performed our experiments using the following datasets. Synthetic Data (AR2): AR(2) dataset comprises synthetic time series data generated using an autoregressive model of order 2. (2) Financial Data (Stocks): features continuous-valued and aperiodic sequences, such as daily historical Google stocks data spanning from 2004 to 2019. We consider the adjusted closing price data for this work. (3) Energy Data (Appliance Energy): The UCI Appliances energy prediction dataset continuous-valued measurements. We consider appliance energy data for analysis. (4) Weather/Climate Data (Daily Minimum Temperature). This dataset [I7] comprises daily minimum temperatures in Melbourne, Australia, from 1981 to 1990. (5) Medical
3 encompasses multivariate,
Data (ECG5000: Congestive Heart Failure): The original dataset 9 for "ECG5000" originates from 20-hour long electrocardiogram (ECG) obtained from the Physionet database. Specifically, is derived from the BIDMC Congestive Heart Failure Database (chfdb), with the record labeled "cht07." " The processed data encompasses 5,000 heartbeats randomly selected from the original dataset.
E Experimental Results
Table Sable/rrepe reports reports the evaluation of all values in time series using JS Divergence and predictive score, comparing the performance of our proposed method against baseline methods.
17

---

Table Comparison of generated samples' (all values) JS Divergence and Predictive Score using the baselines methods. Bold and Underlined entries denote the best and second-best result.
Metries
Methods
AR1
Stock
Energy
Temperature
ECG
beta-VAE
0.0013=0.0004
0.00911100011
0.0091=0.0008
0.0012:10.002
0.001510.0000
c-beta- VAE
0.0011=0.0004
0.0088==00012
0.0089=0.0007
0.0010:1.0000
0.0014=0.0002
TAm
0.0010=0.0003
0.0086++00008
0.0081=0.0011
0.0009=000000
0.0014=0.0001
TimeGAN
0.0015-0.0004
0.0092+000005
0.0022+0.0009
0.001 ++0.0003
0.0018+0.0003
Js
cGAN
0.0012=0.0002
0.0091=000010
0.0085=0.0008
0.0008=00.001
0.0015=0.0002
Divergence
RealNVP
0.0014=0.0003
0.0093=000011
0.0092=0.0008
0.0010=+0.0002
0.0017=000001
Fourier-Flows
0.0013-0.0004
0.0087++0.001
0.0083=000007
0.0009+00.0001
0.0015=0.0002
DDPM
0.0007-0.000
0.0061+0.0007
0.0058+0.0005
0.0005
0.0010+0.0001
Diflusion-TS
0.000
0.005711.0008
0.000110.0005
0..000
0.009-10.0001
FIDE (Ours)
0.0008-000001
0.006810:0010
0.0066
0.0006-10.000 0002
0.001110.0001
beta-
0.8121-0:0410
1.0915+000215
0.8515
0.0021+0.0109
000033
c-bela- VAE
0.7551=0.0555
1.08411100021
0.8442.1.00100
0.7958:10.0089
0.991110.015
TimeVAE
0.7714-0.0345
1.0662::00211
0.0399-1.0089
0.0121
0.982210.0101
TimeGAN
0.7514=0.0451
1.0621=0.0198
0.015
0.7856 0098
0.0822
Predictive Score
cGAN
0.7694=0.0354
1.0721=0.0188
0.8433=0.0181
0.7905=0.0122
0.9874=000151
RealNVP
0.8011-0.038
1.0914=0.0178
0.8533=0.0154
0.8033=+000135
0.0001
Fourier-Flows
0.7985=0.0324
1.1008
0.8501 1
0.79881100014
0.995410.0188
DDPM
0.7711=0.044|
1.0751
0.8488=0.0133
0.7912=:00122
0.9925=0.0167
Diffusion-TS
0.7684=0.0405
1.0722+000189
0.00125
0.7889=0.0022
0.9910=0.0155
FIDE (Ours)
0.7651-0.0488
1 0.669++0.0192
0.8458-0.0151
0.7895
0.9852+0.0158
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
18

---

The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
19

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is new model (e.g., , a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
See
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/ public/guides. wwweeoomiiissoooooooooooooo
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https cttt:ps.ccppublic/// hwwbbl/cc/ggides//ooooooo
wtw.eeeSuubmissiiooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
20

---

The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: See
In all Tables, we report the standard deviation with 5 runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics httpps://euurips.ccm http:///ww.p..ccp../c///cccclaaa
didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification:
It conforms to the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
21

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: does not use user data and it verifies all the information provided.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
22

---

Answer: [Yes]
Justification:
See and A
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's ' creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: See
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
23

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
24


Reviews:
[{"review_id": "2s4hETwZkp", "reviewer": "Reviewer_ggGX", "summary": "The paper presents a novel generative model designed to better capture extreme values in time series data. FIDE introduces a high-frequency inflation strategy to prevent the loss of extreme values, integrates conditional diffusion modeling to condition on block maxima, and incorporates the Generalized Extreme Value (GEV) distribution to ensure the accuracy of extreme value representation. Empirical results show that FIDE outperforms existing methods in maintaining the distribution of extreme events across various datasets, making it a practical application in the generative modeling of time series data.", "questions": "Please answer my listed weaknesses above.", "limitations": "None.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Maintaining extreme value in generated time series data is an important but challenging problem. The proposed method can potentially satisfy this urgent need in real applications.\n\n2. The problem is well motivated in Section 3, and the proposed method is reasonable and explained clearly.\n\n3. The experimental results demonstrate that the proposed method can indeed preserve statistical information of extreme values.", "weaknesses": "1. Although inflating high-frequency signals is intuitive, the possible influence on the fidelity of generated time series is not discussed.\n\n2. The compared baselines, especially diffusion-based approaches, are not enough. For example, there are several works developing diffusion models for general time series generation (Ref-1), or domain-specific time series generation (Ref-2,3). Also, I think those diffusion-based probabilistic time series forcasting approaches can also be used for generation.\n\nRef-1 Yuan, X., & Qiao, Y. (2024). Diffusion-ts: Interpretable diffusion for general time series generation. ICLR24\n\nRef-2 Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2020). Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761.\n\nRef-3 Zhou, Z., Ding, J., Liu, Y., Jin, D., & Li, Y. (2023, November). Towards generative modeling of urban flow through knowledge-enhanced denoising diffusion. Sigspatial23.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to Question 1:**\n\nWe appreciate the reviewer's concern regarding the fidelity of generated time series. We have investigated this issue in our experiments. First, Figure 5 demonstrates the tradeoff between preserving the overall distribution and the distribution of block maxima. The results suggest  that our model excels in capturing block maxima, with minimal alteration to the overall series. In contrast, the baseline DDPM approach, which is capable of replicating the overall distribution, significantly underestimates the distribution of block maxima values. The analysis and discussion provided in Lines 257-261 highlights this fidelity aspect. While we did not provide the full quantitative results (similar to *Table 1*) for the entire time series due to space constraints, we have included them in the attached *Table A2* (provided in the general rebuttal PDF) and will include them in the appendix of the revised paper. \n\nFinally, the predictive score metric shown in *Table 1* also shows the fidelity of the generated time series by utilizing them for downstream time series forecasting tasks. Following a similar approach as in (Yoon et al, 2019), the downstream prediction task here corresponds to a multi-step time series forecasting task, which includes both predicting the block maxima and non-maxima values in a forecast window.  The results suggest that our method achieves the best performance on three datasets and the second-best on two others, underscoring the model’s ability to accurately reproduce the temporal properties of the time series. \n\n**Response to Question 2:**\n \nWe acknowledge the reviewer's valid point regarding the additional baseline methods. While our initial selection encompassed various generative model types, we concur that incorporating additional diffusion-based baselines is warranted, given our proposed method's foundation in diffusion models. In response to this insightful suggestion, we have expanded our comparative analysis to include the recent Diffusion-TS model, as recommended by the reviewer.  \n\nThe results of this extended comparison are summarized in attached *Table A1* (provided in the general rebuttal PDF). Our findings conclusively demonstrate that our proposed method continues to outperform all baseline methods, including these new additions. Notably, Diffusion-TS emerges as the second-best baseline in most scenarios, underscoring the efficacy of diffusion-based approaches in this domain.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hLFk8SCPQv", "reviewer": "Reviewer_Nq9n", "summary": "The article introduces FIDE (Frequency Inflated Diffusion Estimation), which is geared towards better capturing\nextreme values when generating time series through diffusion models, which the authors stress as crucial in domains like climate\nscience and disaster preparedness.\n\nThe approach involves inflating high-frequency components and conditionally generating samples based on block maxima, integrating\nthe GEV distribution to ensure fidelity in extreme event representation.\n\nThe authors run experiments comparing their approach to GANs and VAEs across a diverse set of datasets (AR1, Stock, Energy, Temperature, ECG).\nThey compare both the overall data distribution and extreme values and find their approach to show promising performance.", "questions": "What would be a model or situation where Assumption 1 is violated? I.e. the statement is that block maxima are related to high frequency components in _many_ real-world time series. Are there any notable exceptions and would one need a completely different modeling approach to capture those, or would your method still allow one to sensibly generate those time series even though the Assumption is violated?", "limitations": "The paper adequately addresses its limitations and does not appear to have negative societal impact.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem statement and proposed solution of the paper are clearly laid out and mathematically described. The authors also propose a meaningful experiment setup to empirically test their approach against sensible baselines.", "weaknesses": "It is not clear to me from the presented results whether there is a trade-off between being able to capture the overall distribution of values well and capturing extreme values of the distribution. The experiments seem to suggests that that might be the case, but it could partially just be a matter of picking a different loss-function during selection of the hyperparameter for the GEV loss. It would be good to get more clarity on this aspect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to Questions:**\n \nWe thank the reviewer for the question. While Assumption 1 holds for numerous real-world time series, it may not be applicable to slowly varying time series without abrupt changes, where block maxima build up smoothly. \nIn such instances, a customized diffusion model like ours may not be necessary, as the block maxima should dissipate at a rate similar to other values during the noise addition process. To demonstrate this, we modified the example time series shown in Figure 3 of the paper by applying a moving average to smooth the data. We then applied the forward pass of diffusion model to the smoothed time series. As illustrated in the attached Figure 1 (see the attached PDF in the general rebuttal), both the block maxima and other (non-maxima) values of the smoothed time series dissipated at a similar rate, making Assumption 1 no longer valid. In this situation, the slow evolution of the block maxima is part of the general pattern of the time series. Thus, existing diffusion models should be sufficient to capture the block maxima without the need for our model. \n \n**Response to Weakness:**\n  \nIndeed, our results reveal a trade-off between accurately capturing the overall distribution and effectively modeling the extreme values. This is evident in Figure 5 which shows that our proposed method excels in capturing the block maxima distribution while slightly underperforms in capturing the distribution of all values. \nTo further validate this observation, we conducted an experiment using temperature data, evaluating the trade-off with different inflation weights (1.0, 1.15, 1.3) applied to high-frequency components. An inflation weight of 1.0 indicates no inflation, while 1.3 denotes inflating the high-frequency components by a factor of 1.3. We observed that decreasing the inflation weight from 1.3 to 1.15 and then to 1.0 generally resulted in decreased performance in capturing the block maxima distribution while improving the performance in capturing the overall distribution.\nOur ablation study (Table 2, Appendix) further supports this finding, showing that removing the GEV loss leads to lower performance compared to our proposed framework.  In short, these experiments demonstrate that the task of balancing the trade-off between capturing the overall distribution against the extreme values can be achieved by selecting the appropriate inflation weights (through cross-validation) and incorporating a GEV loss into the objective function.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oxe4ZuxMRt", "reviewer": "Reviewer_4b7m", "summary": "This article discusses the shortcomings of the insufficient ability to focus on maximum values when applying DDPM in the field of time series generation, and proposes a new framework to overcome this problem by introducing a high-frequency expansion strategy in the frequency domain to ensure the emphasis on high-frequency values. At the same time, the article also proposes a generative modeling method based on conditional diffusion.", "questions": "1. In Chapter 3, the article mentioned that adding linear noise to the diffusion model will reduce high-frequency components faster. There are many ways to add noise in the diffusion model, such as the Noise Schedule in iDDPM. Is there any review on this part? Discussion and experimentation?\n\n2. Should the Transformer in Figure 4 in Chapter 4 be a Transformer Encoder?\n\n3. In the field of time series generation, the conditional generation of diffusion models has been studied in the article DiffusionTS. What is the difference between the two?\n\n4. In the selection of comparative algorithms, there have been some time series generation methods based on flow methods, eg: Fourier Flow. Do articles based on this method have good performance in the distribution of block maximum values?\n\n5. The article has mostly discussed the issue of block maximum value distribution in time series generation. Should we discuss the performance of the data generated by enhancing the block maximum value distribution in downstream tasks?", "limitations": "NA", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The logic of the article is relatively clear, the language is relatively clear and precise, and it is easy to understand.\n\n2. The experimental settings of the article are relatively clear and the results are well organized.", "weaknesses": "1. The article does not set up an ablation experiment to explore the effects of the improved strategy.\n\n2. The article has a small amount of experiments and incomplete research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to Question 1:**\n \nWe initially evaluated linear, sqrt, and sigmoid noise schedulers. We chose the linear scheduler as it provides a more gradual perturbation compared to the sqrt and sigmoid, which alter data more rapidly in initial iterations. We appreciate the reviewer's suggestion regarding iDDPM's noise schedule. Investigating the impact of alternative schedulers, such as the cosine scheduler in iDDPM, on high-frequency component dissipation is indeed an intriguing avenue for research. While beyond the scope of the current study, we acknowledge this as a valuable direction for future work and will explore it in subsequent investigations.\n\n**Response to Question 2:**\n\n Yes, it is a Transformer Encoder.  We will change the notation/figure in the revised manuscript accordingly. \n\n**Response to Question 3:** \n\nOur proposed framework is designed to improve the generation of extreme values (i.e., block maxima) in time series whereas DiffusionTS enhances time series generation by emphasizing interpretability and the realistic representation of the generated data. Due to the tradeoff between preserving the overall distribution and accurately representing extreme values, DiffusionTS is insufficient to reproduce the extreme value distribution. To support this, we have included experiments comparing DiffusionTS to our method in the attached general rebuttal PDF (See *Table A1*). The results indicate that while DiffusionTS effectively captures the general distribution, it does not adequately preserve the extreme values. \n\n**Response to Question 4:**\n\n We appreciate the reviewer's suggestion to include flow-based methods in our comparative analysis. In response, we have incorporated two such baselines: Fourier-Flows and RealNVP. The results of this extended comparison are presented in the attached *Table A1* (general rebuttal PDF). Our findings suggest that the proposed method consistently outperforms both flow-based baselines. Moreover, while flow-based methods perform comparably to the VAE- and GAN-based methods, they generally fall short when compared to diffusion-based approaches. This comparison further validates the efficacy of our proposed method across a broader spectrum of state-of-the-art approaches.\n\n**Response to Question 5:** \n\nThe Predictive Score metric used in (Yoon et al., 2019) in *Table 1* directly addresses the performance of generated data in a predictive downstream task. This metric evaluates how well a predictive model trained on generated data performs when tested on real data, effectively assessing the fidelity of our generated time series in practical downstream applications. Our method achieves top performance on three datasets and second-best on two others. This underscores the practical utility of our approach beyond mere distribution matching, showing its value in generating data that preserves important predictive characteristics of the original time series.\n\n**Regarding Weakness 1:**\n\n Indeed, we have provided an ablation experiment (See Appendix *Table 2*) exploring the effects of different strategies/modules of our proposed framework. This analysis specifically illustrates how different components of our approach contribute to the overall performance. These results provide valuable insights into the relative importance of each element in our model", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iI5yozDwKn", "reviewer": "Reviewer_8C1A", "summary": "The problem statement, solution and its mathematical formulation is nicely presented in the draft. \n\nAuthors have identified a problem in the time series generation process which is under explored and they propose a new algorithm Frequency-Inflated Conditional Diffusion Model (FiDE) to address this. The problem lies in the modelling of block maxima which are related to extreme events in time series data where traditional diffusion models fail or does not take into account properly. \n\nAuthors demonstrated that these extreme values correspond to high frequency fourier components in time series and proposed the algorithm to inflate the weights of high frequency fourier components. The inflation of high-frequency components in the Fourier domain to prevent their premature dissipation during the diffusion process. This approach enables the model to retain significant extreme values that are typically lost in traditional modeling techniques. \n\nThe paper presents detailed performance comparison with several classes of models, e.g. VAE, diffusion, and GAN using datasets from several domains. The comparison is extended to 4 metrics explaining the motivation to use them.", "questions": "The paper is generally well-written, but a few important points remain unanswered, which piques my curiosity. They are as follows:\n\n- When defining the block maximum what is the time window? What criteria defined this time window? Is there a thumb rule or model to choose it dynamically? \n\n- A discussion on error (unless missed) is needed. How are errors estimated on the metric reported in table 1. What are these errors (statistical/systematic or something else)?", "limitations": "- More experimental comparison like figure 5 are really needed in the paper draft for further clarity. \n- What is the effect of FIDE on all samples (not just block maxima), a number would be useful to quote similar to block maxima.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The problem statement and its novel solution are the strength of the research presented. It is of great importance to model the tails of time series distribution which are generally not very well modeled. \n - correctly identifying that the tail is mainly due to high frequency Fourier components and authors model it well using the algorithm proposed. Strength lie in the fact that they found the root cause and then proposed the solution.", "weaknesses": "The results [Fig 5] show the comparison of block maxima values for only one dataset. More details on the dataset and more comparison would be helpful to confirm the generality of the proposed algorithm. \n\nFigure 5 shows the final results. It shows the improvement in the block maxima modelling but at the same time it does show degradation in modelling the bulk. An additional comparison of ALL - block maxima with a quantitative estimate of degradation in the performance is needed to understand the full impact of FIDE. \n\nThe table shows the comparison of metrics for block maxima because this is the main focus of this paper. Understood! But at the same time to make it a fair comparison the quantification on the ALL - maxima block is also needed to fully evaluate the model. With the present table and figures it is challenging to quantify if the model is doing \"more good\" than \"bad\". \n\nAuthors do mention in Results section L 257-261 that DDPM has slightly better performance but it is not quantified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Response to Question 1:**\n\nThe choice of time window for defining block maxima is domain-dependent and application-specific, driven by the temporal characteristics of the data and the phenomena of interest. For climate or energy data, monthly block maxima (30-day or 90-day windows) are often relevant, capturing sub-seasonal to seasonal extremes. ECG data may require hourly or smaller windows to detect critical cardiac events, while financial data typically uses monthly or quarterly windows, aligning with reporting periods and long-term trends. The optimal window size should be determined based on the specific research questions, data resolution, and relevant time scales of the studied phenomena. \n\n**Response to Question 2:**\n\nWe thank the reviewer for the question. The errors are estimated statistically. Each experiment was repeated five times, with the standard deviation across these iterations serving as our error estimate. This approach accounts for variability in model performance due to stochastic elements, such as random initialization of the network in training. We will explicitly detail this methodology in the revised manuscript to enhance clarity and reproducibility.\n\n**Response to Weaknesses:**\n \nAlthough our focus is on block maxima, our model generates the entire time series. We provided a comparison of the distributions for all values in Figure 5. However, due to space constraints, we were unable to include comparative figures for all baseline methods. Nevertheless, we have now included a table comparing the performance for all values (similar to *Table 1*). This table (*Table A2*) is provided in the attached PDF (please see the general rebuttal), which shows the metrics comparing the distribution and predictive performance for all values. The results indicate that our method is quite comparable to DDPM, though slightly worse, due to the tradeoff between preserving the overall vs extreme value distribution. However, it is still better than other VAE-based, GAN-based, and flow-based methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you all for your careful and valuable suggestions. In response to your insightful feedback, we have expanded our comparative analysis to include the latest Diffusion-TS model and two additional flow-based baselines (Fourier-Flows and RealNVP) as recommended by the reviewers. This ensures a more comprehensive evaluation. We have also addressed your other suggestions and clarified the points you raised in the individual rebuttal.

Please find the attached PDF that contains 1 figure and 2 tables to address reviewers' comments.

Meta-review: This paper introduces FIDE (Frequency Inflated Diffusion Estimation), a novel approach for generating time series data that better captures extreme values. The method combines high-frequency inflation, conditional diffusion modeling, and integration of the Generalized Extreme Value (GEV) distribution.

Strengths highlighted by the reviewers:
- Addresses a significant and underexplored problem in time series generation.
- Innovative combination of techniques: high-frequency inflation, conditional diffusion modeling, and GEV distribution integration.
- Comprehensive experiments across various datasets demonstrating improved performance.
- Clear presentation and mathematical formulation.

Weaknesses:
- Some questions about comparisons with recent state-of-the-art diffusion models.
- Potential trade-off between capturing overall distribution and extreme values could be further explored.

The paper presents a novel and important contribution to the field of time series generation, with a focus on preserving extreme values. The majority of reviewers recognize its significance and technical soundness. While some concerns were raised, the authors have adequately addressed most of them in their rebuttal.
Decision: Accept (poster)