Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach
Authors: Tal Gonen
Itai Pemper
Ilan Naiman
Nimrod Berman
Omri Azencot
Venue: NeurIPS 2025 poster ()
Keywords: time series analysis
time series generation
few shot generation
unified training
Primary area: deep_learning
TL;DR: We tackle the challenge of few-shot time series generation by proposing a unified pretrained model that outperforms state-of-the-art baselines across diverse domains.

Abstract: Generative modeling of time series is a central challenge in time series analysis, particularly under data-scarce conditions. Despite recent advances in generative modeling, a comprehensive understanding of how state-of-the-art generative models perform under limited supervision remains lacking. In this work, we conduct the first large-scale study evaluating leading generative models in data-scarce settings, revealing a substantial performance gap between full-data and data-scarce regimes. To close this gap, we propose a unified diffusion-based generative framework that can synthesize high-fidelity time series across diverse domains using just a few examples. Our model is pretrained on a large, heterogeneous collection of time series datasets, enabling it to learn generalizable temporal representations. It further incorporates architectural innovations such as dynamic convolutional layers for flexible channel adaptation and dataset token conditioning for domain-aware generation. Without requiring abundant supervision, our unified model achieves state-of-the-art performance in few-shot settings—outperforming domain-specific baselines across a wide range of subset sizes. Remarkably, it also surpasses all baselines even when tested on full datasets benchmarks, highlighting the strength of pretraining and cross-domain generalization. We hope this work encourages the community to revisit few-shot generative modeling as a key problem in time series research and pursue unified solutions that scale efficiently across domains.  Code is available at https://github.com/azencot-group/ImagenFew.

Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach
Tal Gonen" Itai Pemper" Ilan Naiman Nimrod Berman Omri Azencot Faculty of Computer and Information Science
Ben-Gurion University of The Negev
(talgon, itaipem, naimani, bermann)@poss htt.e/pptttttuu.eeeoooo azencot@bgu. ac.il
Abstract
Generative modeling of time series is a central challenge in time series analysis, particularly under data-scarce conditions. Despite recent advances in generative modeling, a comprehensive understanding of how state-of-the-ar generative models perform under limited supervision remains lacking. In this work, we conduct the first large-scale study evaluating leading generative models data-scarce settings, revealing substantial performance gap between full-data and data-scarce regimes. generative framework
To close this gap, we propose a unified diffusion-bar that can synthesize high-fidelity time series across diverse domains using just few examples. Our model is pre-trained on large, heterogeneous collection time series datasets, enabling it to learn generalizable temporal representations. It further incorporates architectural innovations such as dynamic convolutional layers for flexible channel adaptation and dataset token conditioning for domain-aware generation. Without requiring abundant supervision, our unified model achieves state-of-the-art performance in few-shot Bettings-outperfoonn baselines across a wide range of subset sizes. Remarkably, it also surpasses all baselines even when tested on full datasets benchmarks. highlighting the strength of pre- pre-training and cross-domain generalization. We hope this work encourages the community to revisit few-shot generative modeling as a key problem in time series research and pursue unified solutions that scale efficiently across domains. Code is available at htttt::////t thubb.ttt..u.... com/ azencot- wowoottgrrupp/oo ImagenFew
domain-specific
1 Introduction
Many engineering and scientific domains face challenges in collecting high-qualing time series data due to cost, privacy, and other barriers. In seismology, earthquake recordings are sparse and geographically limited [57]: climate research requires expensive, long-term sensor deployments [30]:
and biomedical data like ECGs often suffer from rndeeeeeprssnnnttioon and privacy constraints [ 49
These limitations hinder the development of robust machine learning models, which typically rely on large, diverse datasets. promising alternative large-scale time series collection the training of generation models 773 51 50 74). These models aim to capture both the distribution of features at each time step and the complex temporal dynamics across time. Once trained to approximate the underlying data distribution, such models enable the generation of novel, reliable data samples related to the original dataset. While generative modeling offers a promising way to alleviate data scarcity, existing models are often designed and evaluated under the assumption of abundant training data. In particular, despite the growing interest in generative approaches, their performance low-data settings, which reflect many real-world scientific applications, remains largely unexplored.
Equal Contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this context, our first main contribution is a systematic evaluation of state-of fatte-ofttee -herttr- time series generation models under low-data conditions. We introduce a novel benchmark that brings together datasets from diverse real-world domains, including finance, climate, and biomedicine, and simulate data-scarce scenarios by limiting training to a small fraction of each dataset (e.g.. 5%) [1]. This benchmark allows us to assess the resilience of generative models when data is limited, mirroring the constraints commonly faced in practical applications. Our results reveal a consistent drop in generation quality under such conditions, highlighting that current approaches struggle to maintain good performance with limited data. This finding underscores the pressing need for more robust methods that can operate effectively in low-data regimes.
Existing time series generation models are typically trained separately for each dataset or problem domain (73 33 [74 B41, a paradigm that proves restrictive in scenarios characterized by limited data availability. Inspired by the success of unified training approaches for vision [9 language [8] and -ongeeneraaive tasks in time series [24] 70, 17]. we extend this paradigm to generative modeling under data-scarce conditions. Rather than training separate models for each dataset, we aim to build a single generative model exposed to wide range of domains, including energy, finance, climate, and traffic, to encourage cross-domain generalization and few-shot generation. We hypothesize that such diversity enables the model to learn transferable temporal structures, improving its ability to generate high-quality samples even when only a few training examples are available.
To realize this goal, our second main contribution is the development of a unified generative model for time series data, trained across a heterogeneous set of domains. We first pre-train our model on large, diverse time series corpus, and then fine-tune it on downstream generative tasks with limited data. Our architecture builds on the Imagen'imme framework [50], which maps time series to invertible image representations, enabling the use of diffusion- based generation in the image domain. To seamlessly handle datasets with varying channel dimensions, we propose a Dynamic Convolution (DyConv) layer that interpolates the weights of the channel dimension, ensuring unified architecture even when test-time inputs differ in channels structure. Finally, to enable domain-specific sampling in our multi-domain trained model, we introduce dataset loken mechanism that conditions the diffusion process on domain-specific information.
Our third main contribution is the empirical demonstration of significant improvements over existing methods. Our approach maintains strong performance when trained on only 15% 10%, or even 5% of the original dataset. We further evaluate our model under extreme low-resource conditions, where only 10, 25 and 50 training examples are available, and observe that continues to produce high-quality samples. Specifically, our unified model achieves, average, over 47.65% and 23. 23.94% performance gains on Discriminative Score and contextFID, respectively, compared to state-of-the-art baselines across multiple datasets and data-limited scenarios. Beyond performance metrics, we conduct in-depth analyses, including scaling law experiments, to better understand the relationship between data volume, model capacity, and generalization. We believe that our model, extensive empirical evaluation, and the proposed benchmark, can serve as a foundation for further research and drive progress in time series generation under constranntt environments.
2 Related Work
Generative modeling of time series. There are three major existing paradigms for generative modeling of time series data. The first is based on Generative Adversarial Networks (GANs) (26). which have been applied to capture temporal dynamics through adversarial learning 77 34], A
second line of work employs Variational Autoencoders (VAEs) (8), leveraging probabilistic latent
representations to model sequential structure [5T
42
191 More recently, the success of diffusion
models in domains such as image 28
62
61
63
4
36
6) audio generation 440 44 and other
domains (29] 5 [66] has sparked growing interest in their adaptation to time series [6 74, 50 22]. Despite these advances, existing models have largely been developed and evaluated under the assumption of abundant training data. None of these approaches has systematically investigated performance under severe data scarcity, a condition common in many real-world domains such as healthcare, seismology, and climate science. In contrast, our work demonstrates the feasibility of time series generation under extreme low-data conditions, highlighting its practical relevance and robustness in real-world deployment scenarios.
Unified training of time series. Recent advances in foundation models, such as large language models 88 67 65 and vision transformers [39 45), demonstrate the power of training on large-scale,

---

diverse datasets to enable broad generalization across tasks with minimal fine-tuning tuning 77. Motivated by this success, researchers have begun exploring analogous approaches in time series analysis, aiming to develop models with similar general-purpose capabilities. One stream of research investigates how
to adapt pre-trained language models for time series tasks [77 27
10
35 13. Another stream of
work focuses on designing specialized architectures that can handle the heterogeneity of time series
data [17 71 24 [T]. For example, TimesNet [7]] extracts frequency-awarr representations using Fourier-based transformations to better capture multi-scale temporal patterns. UniTS [24] employs modified transformer block to capture universal time series representations, enabling transferability
from a heterogeneous, multi-domann pre- treiniii dataset, and TimesFM [17] is based on pre-training a decoder-style attention model using a large time series corpus. In spite of the significant progress in time series tasks such as forecasting, classification, and anomaly detection, generative modeling for time series are typically trained on and adapted to individual datasets. In this work, we take a step toward unifying generative modeling through pre-training, introducing unified model for time series generation. Our goal is to develop single, versatile model that can generalize across domains and perform effectively on extremely small datasets, requiring only minimal fine-tuning.
3 Background
Problem formulation. Let time series be defined as a sequence of real-valued vectors observed at discrete time steps, X1.7 = 1 "2,,,,,,,,, IT), where each 2,, E Rd. We are given a small dataset >N of samples drawn from unknown distribution p(x1:r). When N is small enough that generative models struggle to learn meaningful temporal structure, we define it as the few-shot setting. This typically corresponds datasets containing tens of examples, or less than one order of magnitude below the size commonly used to train deep time series models, The goal is to learn generative model po (xI:T) such that po (XI:) R p(XI.T). ), despite the limited size of Danta
Diam (x (x
Imagen Timee [50] is a diffusion- based framework for time series generation that leverages advances in high-resolution image synthesis. It first maps time series into 2D images using, e-g.. delay embedding transformation. In delay embedding, a univariate or multivariate time series is mapped into an image by extracting local patches over the temporal axis and organizing them spatially. Mapping time series to the image domain enables the use of powerful vision diffusion models for generative modeling. After generation in the image space, the samples are mapped back to the time series domain via inverse delay embedding. This approach bypasses architectural constraints of temporal models and achieves strong performance in both unconditional and conditional generation tasks. In our work, adopt imagen'Time the generative backbone due to its high sample fidelity and compatibility with few-shot adaptation.
4 Benchmarking Few-Shot Capabilities of Generative Models
We study state-of-the-arr time series generators in the few-shot regime, where only a limited collection J.v is available, with small N (from tens to hundreds of examples). Our comparison
Danta = [x(1) {xi:T]== spans the three major generative paradigms: VAEs, GANs, and diffusion models. We evaluate four strong recent representatives of these approaches: TimeGAN (73,, KoVAE [51]. DiffusionTS (74), and Imagen Time (50). All four have achieved impressive results on well-established benchmarks when trained abundant data; here, we test how well those results transfer to data-scarce settings.
Benchmark datasets and evaluation metrics. To evaluate their performance, we collect a suite of 12 real-world and synthetic datasets covering a wide spectrum of domains, temporal dynamics. and channel dimensionalities: MuJoCo (66), ETTml, ETTm2, ETTh2
(75). Sine, Weather [69],
ILI [1). Saugeen River Flow [47) ECG200 53 Sel/RegulationSCPI 21. AirQuality (72), and StarLightCurves (56). We provide detailed statistics and preprocessing steps in A.2 To simulate data scarcity, we subsample each dataset using two complementary strategies: percentage- sampling, retaining 5%, 10%, or 15% of the sequences; and fixed-count sampling, limiting the training set to (#10, #25, #50) sequences. Unlike genuinely tiny datasets, this design lets us compare generated samples with held-out real data from the same distribution. We assess generation quality with three widely adopted measures for time series: Discriminative Score (73), Predictive Score [73], and contextFID
33 These metries respectively quantify (i) the distinguishability of generated from
3

---

Ours
ImagenTime
DIfFTS
KoVAE
TimeGAN
A 0.447
8.5
0.359
5.2
mcan Disc.
0.271
contextFID
3.9
0.183
2.6
0.095
1.3-
#10
#25
450
100%
#10
#25
#50
100%
Train subset size
Train subset size
Figure 1: We compare five models a few-shot scenario using Discriminative Score (A) and ContextFID (B), where in both, lower-is-better. Our model consistently outperforms others across all subset sizes, with clear gains on smaller subsets. TimeGAN consistently reports contextFID scores above 6.5 across all subset sizes, which is why it is not visible in the visualization shown in Fig. (B).
real samples, (ii) the preservation of temporal dependencies important for prediction, and (iii) the proximity of contextual embeddings between synthetic and real sequences. Formal definitions and implementation details are given in App. B.3
Results. We train all models across all setups (one model per subset) and present the partial results in Fig. [ The plots show that state-of-the-ar (SOTA) models experience sigmificant performance degradation in data-scarce settings. More comprehensive results are provided in Tab. [T These findings confirm that few-shot generation remains major challenge for current SOTA models. A commonly proposed approach to mitigate data scarcity is data augmentation [37 60 However, in the context of time series, augmentation techniques may harm performance when the underlying distributional biases are not aligned [68 52.. Instead, we draw inspiration from the promising performance of
unified training frameworks in zero- and few-shot time-series forecasting [24] 11 70 [7]. Building
on these advances, we extend the unified training paradigm to the generative setting and develop a model pe (x1.T (Dataa that can effectively adapt to the few-shot target distribution and approximate p(X1.T) for novel time series generation tasks.
Our Unified Generative Time Series Model
Existing methods for time series generation typically train a dedicated model for each specific dataset or domain 77 51 50
74 34). While effective when abundant training data is available, such strategies become impractical in low-data settings, where the low number of examples limits the model's ability to produce high-quality samples (Sec. 4 To address this, we propose unified generative modeling framework that trains single model across a heterogeneous collection of time series datasets spanning multiple domains, including finance, energy, climate, traffic, and biomedical signals, see App. [I The key idea is that exposure to a broad range of temporal structures and representations
statistical properties during pre-training enables the model to learn of time series dynamics. This, in turn, improves the model's ability to generate high-quality samples even when only few examples from new target domain are available. Analogous to the way foundation models in vision and language benefit from large-scale, diverse pre-training [55] [54], we hypothesize that unified time series generative model can generalize more effectively under few-shot conditions by leveraging prior knowledge acquired from diverse sources.
Modeling. Similarly to Imagen [50], our approach builds upon an image-based framework [36]. We adopt this image-based generative backbone for several key reasons: (1) it achieves state-of-the-art results in unconditional and conditional time series generation; (2) it exhibits strong generalization capabilities across a wide range of sequence lengths, including very long horizons; and (3) it benefits directly from rapid progress in the vision diffusion community, allowing continual improvements in generative quality through better backbone models. We illustrate the core components of our generative modeling framework in Fig. 2 For each sample in the train set, X1.7 (I1, 22, .., Fr), where each Dt Rd and d is the channel number, we convert it through delay embedding

---

Chanrats
No
Infarance
DyConv
UNet
DyConv
I
Padding
Lata lokan
Figure 2: In our architecture, time series data from a wide range of domains are first transformed into image representations. Then, their noisy version and their data tokens are processed by a neural network equipped with dynamic convolutions (DyConvs), which accommodate varying channel sizes. Finally, only during inference, the generated images are transformed back into time series data.
transformation, T, into an image (X1:T) zime RCxHXx We additionally pad the resulting image square shape. We follow EDM [36] and ImagenTime (50) frameworks, adding noise è ~ (O, o² to y.ll ling' Ting = ing + e, where a is the noise schedule. Then, the objective of the model is to clean the noise via the following loss function:
(M(o o (4 σ) -iie l121
C C=
where A(o) is a weighting function and Ie is defined as follows:
To To 0) (axip(()=== + Comr No ja (o)
where N₂ is a neural network. Cskip modulates the skip connection, Gin and Cout scale the input and output magnitudes, and Cnoise maps noise level into conditioning input for Ng. For robust training, we use the same values as in EDM's preconditioning [36]. We feed the network with an additional optional dataset token input y. explained next. Unlike [50, apply a dynamic binary mask to the padded input sequences. The mask is adjusted at runtime based on the actual sequence length, marking valid time steps and excluding padded regions. This allows the model to distinguish meaningful data from padding and ensures that, during training, it learns to focus exclusively on valid regions while effectively ignoring padded values. This mechanism enables the model to be pre-trained on sequences of a fixed length and subsequently fine-tuned on sequences different lengths, supporting time series generation of variable-length inputs without architectural modifications or additional pre-training. In contrast to traditional approaches (50) 73 74). which typically rely on fixed-length inputs require retraining for each length, our method naturally supports cross-temporal resolution transfer. Additional implementation details of the masking mechanism are provided in Appendix [. This capability is empirically validated by our experiments in Section 6.2 which show that models pre-traine pre-trained on one sequence length maintain strong performance when tuned on different lengths.
DyConv. Unlike prior generative time series models, our model must flexibly handle varying input and output channel sizes. A common workaround involves setting maximum number of channels and padding all samples match this size- similar how we standardized the time axis. While simple, this approach (1) restricts the few-shot training regime since the input channel is constrained to a fixed size, and (2) wastes computation on datasets with fewer channels. We address this by DyCon, a dynamic convolutional layer for adaptive channel handling. Inspired by DyLinear 24), we use a single learnable canonical kernel of shape [K, K, Co. C1). where K is the kernel size and Co. Ci are fixed reference dimensions. At runtime, the kernel is resized via bicubic interpolation to match each dataset's actual channel dimensions. Formally, let the input be x E H H and the target channel size Cou-- DyConv constructs kernel W interp E IR K KXXX, X XCa via bicubic interpolation over the channel dimensions of a canonical kernel W:
DyConv(z; W) = Conv2D (z, Interp Cins Cour))
with a similar interpolation applied to the bias. Unlike standard convolutions, DyConv allows a single parameter set to generalize across datasets with varying input/output channels. In our UNet, DyConv used both to map inputs to a shared embedding space and to project outputs back to the original dimensionality. This enables handling of multivariate time series with differing variable
5

---

we show that training with DyConv significantly outperforms naive channel padding. offering improved efficiency and avoiding the constraints imposed by padding. Further details and ablation results are provided in App. B.land and App. C.T
counts, without architecture changes. In Sec. 6.5
Dataset token. To enable multi-domain sampling, we introduce a dataset token mechanism that conditions the diffusion process on domain information. Each dataset is assigned a unique token that acts as an identifier of its source domain. During both training and generation, this token is mapped to learnable embedding and injected into the denoising network via the adaptive group normalization (AdaGN) module [0]]. This embedding is incorporated into the intermediate features of the diffusion model, allowing it modulate its behavior based on the dataset context. Through this mechanism, the model captures dataset-specific characteristics while still leveraging shared temporal patterns across domains. At inference time, the dataset token continues to guide the generative process, ensuring that samples are drawn from the correct target distribution corresponding to the desired domain. Additionally, we experiment with training and fine-tuning without the data token. Interestingly, while pre- pre-training without the dataset token is still effective when fine-tuning occurs, we cannot reliably sample from the pre-traines model without a fine-tuning stage, see Sec. 6.5
Pre-training. We pre-train our model across wide collection of time series datasets. This unified pre-training exposes the model to a broad range of temporal structures and input dimensionalities, encouraging the learning of transferable representations that useful in data-scarce settings. Specifi- cally, we utilize the datasets: Stocks, Energy (9), ETThl (75), Exchange [41], MSL [32]. SMAP (2),
PSM M. SMD [64), ECG5000 25 Fenl 1 (59) SelfRegulation [2]
Blink 14, ElectricDevices 4 Trace [58]. FordB [8] U'aaveGistturelLbraaa
[46). EMOPain [1]]
Chinatown 18. and SharePricelncrease 48 These datasets combine commonly used collections
from various tasks in UniTS (24] with datasets from generative modeling research [50, (74). In our
approach, all of these datasets are jointly employed during the pre-training phase, encompassing approximately 300,000 time series in total. During pre-training, we utilize the full corpus over 1.000
epochs with learning rate of 10-4, conducted in distributed setup across two NVIDIA RTX 4090 GPUs, requiring roughly 4 hours of training time.
Fine-tuning for few-shot generation. To address data scarcity in novel domains, we fine-tune the model with a dedicated dataset token for each new dataset encountered. This token is mapped to learnable embedding that represents the identity of the new domain. The embedding is initialized randomly and optimized jointly with the rest of the model during fine-tuning. It serves as a signal to distinguish the target dataset from those seen during pre- trainingg thereby guiding the model to generate samples that reflect the unique characteristics of the new distribution. Fine-tuning is performed on a small subset of the target dataset, in accordance with our few-shot benchmark protocol. This enables the model to quickly adapt its generative behavior the new domain, supporting strong generalization in low-resource scenarios. We also experimented with other fine-tuning strategies, such as freezing all weights except the biases or applying Low-Rank Adaptation (LoRA) (31]. However, we observe poor results compared to our fine-tuning, see a detailed analysis in App.
C.2
6 Experiments
In this section, we empirically evaluate the performance and key aspects of our model. We start by evaluating its performance on the novel few-shot generation benchmark against the SOTA baselines (Sec. Then, we also show our robustness when evaluated on varying-length data (Sec. 6.2). Next, we explore the effects of model scale and its generalization abilities longer sequences (Sec. 6.3 Finally, we perform ablation studies on the main properties of our method (Sec. 6.5)
6.1 Fhw-Shot Benchmark for Time Series Generation
To assess the effectiveness of our unified generative model in low-data regimes, we conduct
comprehensive few-shot generation study using the same target datasets introduced in Sec. Our pre-trained model is fine-tuned on each dataset and evaluated against the baselines from Sec. B
ImagenTime [50], DiffusionTS 74), KoVAE [5I), and TimeGAN 73.. We report the averaged
Discriminative Score (Disc.), Predictive Score (Pred.), and contextFID (c-FID) in Tab. Across all
subset sizes and evaluation metrics, our method consistently outperforms the baselines, demonstrating
strong performance in both percentage-based and count-based few-shot settings. Remarkably, our

---

Table 1: Full and few-shot results across subset scales. We report Discriminative Score (Disc.), Predictive Score (Pred.), and contextFID (c-FID) For each subset size, values reflect averages across all evaluation datasets. Our method, Imagenfew (pre-trained then fine-tuned on sequence length 24), consistently outperforms all baselines. Bold marks best, underline second-best.
Subset
Metric
ImagenFew
ImagenTime
DifTTS
KoVAE
VAE
TimeGAN
Improvement
Disc. 0.027
0.113
0.203
0.356
0.185
0.379
76.1%
100%
Pred.
0.448
0.452
0.457
0.482
0.455
0.520
0.88%
c-FID
0.866
1.390
1.78
5.351
3.068
7.912
37.69%
Disc,
0.110
0.321
0.248
0.389
0.239
0.388
53.97%
5%
Pred.
0.458
0.469
0.479
0.485
0.467
0.499
1.93%
c-FID
0.674
3.464
2.087
5.232
1.747
11.055
61.42%
Disc.
0.083
0.248
0.233
0.375
0.232
0.402
64.22%
10%
Pred. 0.452
0.458
0.469
0.491
0.466
0.494
1.31%
e-FID.
0.578
2.757
1.944
5.326
1.692
14.057
65.84%
Disc, 0.066
0,236
0.229
0.363
0.215
0.384
69.30%
15%
Pred. 0.451
0.458
0,464
0.484
0.465
0.486
1,52%
c-FID,
1.086
2.211
2.064
5.644
1,303
8.290
16.66%
Disc. 0.259
0.357
0,362
0.427
0.312
0.382
16.99%
#10
Pred. 0.489
0.492
0,525
0.514
0,510
0.467
-4.71%
c-FID,
3,800
5.393
4.984
6.166
2,383
28.572
-59.46%
Disc.
0.190
0.350
0.338
0.407
0.260
0.376
43.78%
#25
Pred.
0.467
0.469
0.498
0.515
0.473
0.538
0.42%
e-FID.
1,582
4,599
3,980
5.934
1.874
7.277
15.58%
Disc. 0.149
0.342
0.313
0.397
0.239
0.367
37.66%
#50
Pred. 0,460
0.466
0.493
0.502
0.475
0.472
1.28%
c-FID. 0.987
3.639
3.626
5.816
1.75
9.744
43.60%
unified model, even when fine-tuned on only 5% of the data, outperforms ImagenTime [50) trained on the full dataset and achieves average performance gains of 55.72% and 54.25% in discriminative and contextFID scores, respectively, over all competing models. Beyond few-shot scenarios, our approach also excels in data-rich settings. tine-tuning our pre-trained trained model on large datasets yields further performance gains, highlighting the benefits of pre- training across diverse domains. This suggests that our model acquires robust inductive bias that enables superior generalization compared to models trained from scratch. These findings underscore the versatility and strength of our unified generative framework, particularly in few-shot generation. However, some datasets, such as Weather [69] and ECG200 [2). remain challenging under limited data conditions, seen in the full results table provided in App. C.6
6.2 Pre-training with Varying Sequence Lengths
We examine how the pre-training sequence length influences downstream fine-tuning performance. We pre-trained four models using fixed sequence lengths of 12, 24, 36, and 64 time steps, following the protocol described in Sec. 55) Each model was fine-tuned and evaluated on six datasets: ECG200, ETTh2, ETTm1, ETTm2, ILI, and synthetic sine wave, using sequence lengths of 12, 24, 36, and
Sequence length comparison
GFLOPs comparison
A
B
Podding
0.35
0.14
DyConv
mean Disc.
0.30
GFLOPs
Our(12)
0.12
Our(24)
0.25
Our(36)
0.10
0.20
0.15
12
24
36
64
200
400
600
800
1000
Sequence length
Pre-train max. channels
Figure 3: (A) Length generalization capabilities comparing five models, including four pre-trained on fixed sequence lengths and one without pre-training (w/o PT). Our(t) models were pre-trained exclusively on sequences with & steps. (B) Computational complexity measured in GFLOPs.
7

---

64. For comparison, we include models trained from scratch (w/o PT baseline). Performance is reported as the average discriminative score under few-shot generation setup using 5%, 15%, #25, and #50 of the available samples. Results are shown in Fig. 3A.. On sequence lengths of 12, 24, and 36, all pre-trained models achieve consistently strong performance, significantly outperforming the baseline. As sequence length increases and thus, making the task more difficult, the baseline performance degrades markedly, whereas pre-trained models remain robust. At length 64, the model pre-trained on that specific length substantially outperforms those pre-trained on shorter sequences. These results demonstrate that our method not only generalizes well in low-data regimes, but also transfers effectively across different temporal resolutions, essential property for real-world time series applications.
6.3 Pre-training Impact Across Model Scales
We now explore the influence of model size on fine-tuning performance, focusing on the more challenging setup introduced in the previous section, where we observed strong results on generating time series of length 64. To this end, we pre-train four models with increasing parameter counts: Base (6M), Medium (15M), Large (26M), and XL (40M), using an identical pre-training protocol. Each model is subsequently fine-tuned and evaluated across all downstream tasks, with the generation target fixed to 64 time steps. For comparison, we also include models trained without pre-training ("wwo PT"). We report Discriminative Score (Disc.), Predictive Score (Pred.), and contextFID (c-FID), with all results averaged over datasets and subset-size conditions to reveal general performance trends. This setup enables us to assess how model capacity and pre-training interact in the few-shot regime.
Tab. Sob.@ssun size on fine-tuning performance for 64-step time series generation in the few-shot setting. Several key trends emerge: (1) Fine-tuned models consis- tently outperform their w/o PT counter- parts across all scales. (2) Discrimina- tive Score improves with model size un-
summarizes the effect of model
Table 2: Model size ablation comparing fine-tuned (FT)
models with those trained without pre-training (w/o PT).
Base
Medium
Large
XL
XL-+S Det.
Pred. c-FID Disc.
FT
0.15
0.16
0.13
0.13
-14.02%
w/o PT
0.35
0.35
0.26
0.27
-23.29%
4.85
5.03
4.56
4.51
-7.37%
der both training modes, with the fine-
w/o PT
15.87
15.36
8.12
8.15
-48.62%
tuned Large model achieving the best score (0.130). Interestingly, the XI.
0.4993
0.4974
0.4959
0.4965
-0.57%
w/o PT
0.5043
0.5036
0,5007
0.5023
-0.40%
model performs slightly worse than the Large model, suggesting diminishing re- turns at higher capacity. (3) c-FID follows a similar trend, with pre-training yielding substantial gains, especially for smaller models. (4) Predictive Scores remain relatively stable across scales, with slight improvement for larger pre-trained models. (5) Pre-training also reduces the sensitivity of performance to scale, shown by the XL-S Det. column. For example, c-FID in w/o PT deteriorates by 48.62% when moving from XL to Base, while fine-tuned models show much smaller gaps. Overall, pre- pre-training allows smaller models to approach the performance of larger ones.
Table 3: Out- -fdoma-n fine- tuning. The models are tested on a sequence length of 24.
Disc:
c-FID 1
Tasascet
Method
10%
108
5%
15%
Stocks
ImagenFoow
0.018
0.017
0.018
0.138
0.054
0.132
0.090
0.854
0.430
0.183
Imagen'Tmme
0.201
0.505
0.316
0.275
0.206
0.247
0.846
0.742
0.991
3.267
2.16
1.845
Exchange
ImagenFew ImagenTime
0,046
0,036
0,044
0.352
0.279
0.178
0.137
0.119
0.124
2.574
2.222
0.833
0,470
0,470
0.433
0.496
0.176
0.191
3,552
3.366
3,285
8.132
5.901
4.639
6.4 Out-of-domain Generalization
We evaluate our model's ability to generalize to unseen domains. To achieve this, we intentionally excluded financial datasets (Stocks and Exchange) from the pre-training phase. Subsequently, we fine-tuned our model, ImagenFew, separately on each of these oft-of-domain datasets. The results, compared against the ImagenTime baseline, are presented in Table In this setting, we find that ImagenFew significantly outperforms Imagen'Time Time across all metrics and experimental setups on both the Stocks and Exchange datasets. This strong performance is particularly noteworthy given that our model had no prior exposure to financial data during pre-training. This suggests our unified approach a promising step toward cross-domain generalization, where knowledge from source domains can be effectively transferred to new, unseen one.
8

---

Table 4: Ablation study the effect of DyConv. We compare against padding-basee baseline with
equivalent capacity (6M parameters). Results are averaged across all evaluation datasets.
5%
10%
15%
#10
+25
#50
Metne
Imagenkew
Baseline
ImagenFew
Baseline
Imageatlew
Biseline
Imagenseww
Baseline
ImagenFew
Baseline
Imagenkew
Baseline
Disc.
0.110
0.291
BJKK
0.250
1.06
0.280
0.259
0.381
0,190
0.372
0.149
0.355
Prod
0.488
0463
0.452
0460
0.451
0.58
0.489
4.88
0.467
0.474
0.460
0.466
e-RD...
0.674
4. 190
0.578
4.88 8664
1,086
3.532
3,800
7,889
1.582
5.940
0.987
5,005
Table 5: Few-shot evaluation with and without dataset token conditioning.
5%
10%:
15%
#10
#25
#50
Metric
Cond.
Uncond.
Cond.
Uncond.
Cond.
Uncond.
Cond.
Uncond.
Cond.
Uncond.
Cond.
Uncond.
Disc.
0.11
0.10
0.08
0.08
0.06
0.06
0.25
0.25
0.19
0.18
0.14
0.15
Pred.
0.45
0.45
0.45
0.45
0.45
0.45
0.48
0.49
0.46
0.46
0.46
0.46
c-FID...
0.67
0.79
0,57
0.55
1.08
1.16
3.80
2.42
1.58
1.23
0.98
0.98
6.5
Ablation of Main Properties
We investigate the effect of our dynamic channel adaptation module (DyConv) and dataset token. We begin by evaluating whether DyConv is necessary for handling varying input sizes and improving few-shot performance. In the App. C.T we analyze how its internal configuration, specifically, the number of input and output channels, affects generative quality.
DyConv vs. padded baseline. To evaluate the necessity of DyConv, we conduct an ablation study in which the module is removed and replaced with static channel padding. Specifically, all input sequences are zero-padded to match the maximum channel dimensionality observed during pre- training. The remainder of the architecture, including the dataset token mechanism, is left unchanged, and the total number of parameters is matched to our main model. The padded baseline is pre-trained using the same procedure described in Sec. [5] To ensure fair comparison, mechanism in the loss function such that the denoising network is only penalized for predicting valid (unpadded) dimensions. This isolates the effect of DyConv from other architectural factors.
introduce a masking
The results, averaged across all evaluation datasets, are presented in Tab. 4 Our model consistently outperforms the padded baseline across all metrics, with especially large gains in Discriminative Score and contextFID. These results demonstrate that DyConv is not only effective but essential. Beyond enabling the handling of varying channel sizes, DyConv contributes a significant performance boost, making it a critical component of our architecture. In contrast, padding-bass approaches are inherently constrained by the fixed dimensionality used during pre-training, limiting their ability to generalize in real-world, multi-domain scenarios where input structures vary significantly.
Effect of dataset token. We analyze the impact of dataset token guidance during both pre-training and fine-tuning. Table compprres models fine-tuned with and without the token (Cond. vs. Uncond.) across all benchmark datasets. Interestingly, both variants achieve similar downstream performance, indicating that once the model is adapted a specific dataset, the token becomes largely redundant for fine-tuning. This suggests that the model acquires domain-agnostic temporal representations during pre-training, which transfer effectively under limited supervision. In contrast, the dataset token plays a central role during pre- training.. It enables the model to learn separate distributional modes across multiple datasets, allowing us to reliably sample from each distribution and evaluate the quality of pre-training. It also allows users to leverage the tre-trained model an invested resource, enabling targeted sampling without additional fine-tuning. To quantify this effect, we compare the generation performance on the pre- pre-training datasets when models are trained with and without the token. Without the token, the model samples from mixture of all training distributions, leading to lower-quality and less coherent generations. Averaged across all pre-training datasets, the Discriminative Score and context-FII drop from 0.393 and 4.84 (with token) to 0.252 and 11.07 (without token), as shown in Table 21 This demonstrates the key role of the token in resolving distributional ambiguity during pre- training, enabling Nomain--seciiie sampling and improving generation fidelity. While the token has little impact during fine-tuning single dataset, it remains essential for both evaluating pre-training and enabling controlled generation in mixed or unseen data settings.
6.6 Runtime and Memory Consumption
In our model, DyConv accommodates varying channel sizes while maintaining complete indepen- dence between the current sequence's channel size and the pre-train pre-train maximum channel size. This

---

design stands in contrast to the naive padding- based approach, which inherently introduces a depen- dency between the two. In this section, we evaluate the computational cost of processing a single sequence of length 24 with the average channel number encountered during pre-training under both setups, as a function of the maximum number of channels used during pre-training. Fig. illustrates a linear correlation between the number of pre-training maximum channels and the computational cost (in GFLOPs) for the na've padding solution. In contrast, DyConv exhibits a constant computational cost regardless of the channel count. In single pre-training iteration, as described in Sec. S with batch size of 2048 on an NVIDIA A6000 GPU, DyConv consumes approximately 18 GB of memory on average. Under the same setup, the naïve padding approach consumes 2.01 GB, representing substantially higher memory footprint.
7
Conclusion
Recent literature has introduced multiple approaches for time series generation based on GANs, VAËs, and Diffusion models. However, our research demonstrates that the performance of these models deteriorates significantly as the number of available training samples decreases, common constraint in real-world time series applications where generative modeling is invaluable. This paper addresses this limitation by proposing two-stage modeling framework. The first stage involves unified pre- pre-training across multiple datasets, followed by second stage where the model is finc-tuned for specific generative tasks under data-scarce conditions. In contrast to conventional approaches where models are trained on individual datasets, method exposes the model to diverse range of temporal structures and statistical properties during pre-training. This exposure facilitates the representations of time series dynamics. We validate the efficacy of our methodology through a novel benchmark specifically designed for this task and systematically analyze its key properties through comprehensive ablation studies. Our unified generative model achieves state-of-the-art performance in few-shot time series generation, outperforming all baselines by over 55% in discriminative and contextFII scores, even when fine-tuned just 5% of the data. Additionally, we demonstrate strong robustness across sequence lengths and model scales, with DyConv and dataset token guidance proving essential for efficiency and generalization in real-world, multi-domain scenarios. In conclusion, we believe that this approach has the potential to facilitate the development of large pre-trained models for time series generation, similar to the transformative advancements witnessed in image and language domains.
learning of
8 Broader Impact
Our framework advances time-series generation by pre-training a unified model across multiple datasets, promising significant gains in data synthesis, especially in data-scarce scenarios. For ex- ample, using our approach, geophysicists can generate more high-fidelity examples of earthquake waveforms in locations with rare seismic events, improving hazard assessment models. In healthcare, researchers could generate synthetic, yet realistic, electronic health records to study disease progres- sion without compromising patient privacy, or augment datasets for training diagnostic models for rare diseases where real-world data is limited.
At the same time, because the model spans high- domains like finance and biomedical data, its power to generate high-fidelity sequences also heightens significant societal risks. The potential for misuse of synthetic data is substantial; for instance, the same technology that helps medical researchers could be used to create fraudulent clinical trial data, potentially leading to the approval of ineffective treatments. Beyond direct misuse, critical danger lies in the overreliance on generated samples, as models trained on augmented datasets may become less robust to the noisy, unpredictable nature of real-world data. Furthermore, there is a risk of downstream model degradation due to hallucinations-tthe the generative model may produce data with subtle, unrealistic artifacts that, while plausible-looking cause subsequent models to fail in unforeseen ways.
Acknowledgments
This research was partially supported by the Lynn and William Frankel Center of the Computer Science Department, Ben-Gurion University of the Negev, an ISF grant 668/21, an ISF equipment grant, and by the Israeli Council for Higher Education (CHE) via the Data Science Research Center, Ben-Gurion University of the Negev, Israel.
10

---

References
[1] A. Abdulaal, Z. Liu, and T. Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21, page 1485-2494, New York, NY, USA, 2021. Association for Computing Machinery.
[2] A. Bagnall, H. A. Dau, J. Lines, M. Flynn, J. Large, A. Bostrom, P. Southam, and E. Keogh. The UEA multivariate time series classification archive, 2018. arXiv preprint / 1810005 2018.
[3] E. Ben Zaken, Y. Goldberg, and Ravfogel. BitFit: Simple caramiter-effiiient finc-tuning for transformer- based masked language-models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Short Papers), pages 1-9, Association for Computational Linguistics. 2022.
[4] N. Berman, O. Joglekar, E. Kosman, D. Di Castro, and O. Azencot. Towards general modality translation with contrastive and predictive latent diffusion bridge. In Advances in Neural Information Processing Systems (NeurIPS) 39, 2025.
[5] N. Berman, E. Kosman, D. D. Castro, and O. Azencot. Reviving life on the edge: Joint scorc-based graph generation of rich edge attributes. Trans. Mach. Learn. Res., 2024.
[6] N. Berman, I. Naiman, M. Fliasof, H. Zisling, and O. Azencot. One-step offline distillation of diffusion- based models via Koopman modeling. Advances in Neural Information Processing Systems (NeurIPS) 39, 2025.
[7] R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bersstein, J. Bohg. A. Bosselut, E. Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.00 07258, 2021.
[8] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances neural information processing systems, 33:1877-1901, 2020.
[9] L. M. Candanedo, Feldheim, and D. Deramaix. Data driven prediction models of energy use of appliances in low-energy house. Energy and Buildings, 140:81-97,201 2017.
[10] D. Cao, F. Jia, S. O. Arik. T. Plister, Y. Zheng. W. Ye, and Y. Liu. TEMPO: Prompt-based generative pre-trained transformer for time series forecasting, In The Twelfth International Conference on Learning Representattons, 2024.
[11] D. Cao, W. Ye, Y. Zhang, and Y. Liu. TimeDiT: General- purpose diffusion transformers for time series foundation model. arXiv preprint wtxx::24009022222, 2024.
[12] CDC. Illness. http://///......///////////////r 2025-05-06.
Accessed:
[13] C. Chang, W.Y Wang, W--. Peng, and T.-F. Chen. LLM4TS: Aligning pre-trained llms as data-efficient time-series forecasters. ACM Transactions on Intelligent Systems and Technology, 16(3):1-20, 2025.
[14] K. Chicaiza and M. Benalcázar. A brain-computer interface for controlling iot devices using eeg signals. In 2021 IELE ETCM, pages 1-6, 2021.
[15] C. T. Chung, S. Lee, E. King, T. Liu, A. A. Armoundas, G. Bazoukis, and G. Tse. Clinical signifi- cance, challenges and limitations in using artificial intelligence for electrocardiography based diagnosis. International journal of arrhythmia,, 23(1):24, 2022.
[16] A. Coletta, Gopalakrishnan, D. Borrajo, and Vyetrenko. On the constrained time-series generation problem. Advances in Neural Information Processing Systems, (6661048-61059 2023.
[17] A. Das, W. Kong, R. Sen, and Zhou. decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, 2024.
[18] H. A. Dau, A. Bagnall, K. Kamgar, C.-C. M. Yeh, Y. Zhu, S. Gharghabi, C. A. Ratanamahatana, and E. Kcogh. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica 1ett1129-1300 2019.
[19] A. Desai, C. Freeman. Z. Wang, and I. Beaver. Timevae: A variational auto-encoder for multivariate time series generation. arXiv preprint arXiv:2111.08095, 2021.

---

[20] P. Dhariwal and A. Q. Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer. Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems. 2021.
[21] J. O. Egede, S. Song, T. Olugbade, C. Wang. A. Williams, H. Meng, M. Aung, N. D. Lane, M. Valstar, and N.
Emopain challenge 2020: Multimodal pain evaluation from facial and bodily
expressions, 2020.
[22] G. Fadlon, I. Arbiv, N. Berman, and O. Azencot. A diffusion model for regular time series generation from irregular data with completion and masking. In Advances in Neural Information Processing Systems (NeurlPS) 39, 2025.
[23] J.-Y. Franceschi, A. Dieuleveut, and M. Jaggi. Unsupervised scalable representation learning for multivari- ate time series. Advances in neural information processing systems, 32, 2019.
[24] S. Gao, T. Koker, O. Queen, T. Hartvigsen, T. Tsiligkaridis, and M. Zimik. UniTS: A unified multi-task time series model. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[25] A. L. Goldberger, A. Amaral, L. Glass, M. Hausdorff, P. C. Ivanov, G. Mark, J. E. Mietus, G. Moody, C.-K. Peng, and H. E. Stanley. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. circulation, 101s 2000.
[26] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[27] N. Gruver, M. Finzi, S. Qiu, and G. Wilson, Large language models zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36: 19622 19635, 2023.
[28] Ho, A. Jain, and Abheel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851, 2020.
[29] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633-8646, 2022.
[30] A. Hsu, W. Khoo. N. Goyal, and M. Wainstein. Next-generation digital ecosystem for climate data mining and knowledge discovery: review of digital data collection technologies. Frontiers in big Data, 3:29, 2020.
[31] E. J. Hu, yelong shen, P. Wallis, Z. Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
[32] K. Hundman, V. Constantinou, C. Laporte. I. Colwell, and T. Soderstrom, Detecting spacecraft anomalies using Istms and nouparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD Inter- national Conference on Knowledge Discovery & Data Mining, KDD '18 page 387-395, ACM, July 2018.
[33] P. Jeha, M. Bohlke-Schneider, P. Mercado, S. Kapoor, R. S. Nirwan, V. Flunkert, J. Gasthaus, and T. Januschowski. Psa-gan: Progressive self attention gans for synthetic time series. In The Tenth International Conference Learning Representations, 2022.
[34] J. Jeon, J. Kim, H. Song. S. Cho, and N. Park. GT-GAN: General purpose time series synthesis with generative adversarial networks, Advances in Neural Information Processing Systems, 35:36999-37010, 2022.
[35] M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F Li, S. Pan, and Q. Wen. Time-LLM: Time series forecasting by reprogramming large language models. In The Twelfth International Conference on Learning Representations, 2024.
[36] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 0-225655-26777 2022.
[37] Kaufman and O. Azencot. First-order manifold data augmentation for regression learning. In Forty-first International Conference on Machine Learning, ICML, 2024.
[38] D. P. Kingma and M. Welling. Auto-encoding variational bayes. In International Conference on Learning Representations (ICLR), 2014.
12

---

[39] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitchead, A. C. Berg. W.-Y. Lo, al. Segment anything. In Proceedings the IEEE/CVE international conference computer vision. pages 4015 4026, 2023.
[40] 7. Kong, W. Ping, Huang, K. Zhao, and B. Catanzaro. Diffwave: versatile diffusion model for audio synthesis. International Conference on Learning Representations, 2021.
[41] G. Lai, W.-C. Chang, Y. Yang, and H. Liu. Modeling long- and short-term temporal patterns with deep neural networks. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2017.
[42] H. Li, S. Yu, and J. Principe. Causal recurrent variational autoencoder for medical time series generation. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 4562-8570, 2023.
[43] J. Lines, A. Bagnall, P. Caiger-Smithh and S. Anderson. Classification of household devices by electricity usage profiles. In Advances in Intelligent Data Analysis XI (IDA), volume 6936, pages 403 412, 2011.
[44] H. Liu, Z. Chen, Y. Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley. Audiol.DM: Text-to-audio generation with latent diffusion models. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 11550-21474, PMLR, 23-29 Jul 2023.
[45] H. Liu, C. Li, Q. Wu, and Y. J. Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916 2023.
[46] J. Liu, L. Zhong. J. Wickramasuriya, and V. Vasudevan. uWave: -cceleromettr--aseed personalized gesture recognition and its applications. Pervasive and Mobile Computing, 5:657-675, 2009.
[47] A. McLeod and H. Gweon. Optimal deseasonalization for monthly and daily geophysical time series. Journal of Environmental statistics, 4(11):1-11, 2013.
[48] M. Middlehurst, P. Schäfer, and A. Bagnall. Bake off redux: review and experimental evaluation of recent time series classification algorithms. Data Mining and Knowledge Discovery, 3844),1958-2031, Apr. 2024.
[49] R. Missel, K. Gyawali, J. V. Murkute, 7. Li, S. Zhou, A. AbdelWahah, J. Davis, J. Warren, J. L.. Sapp, and L.. Wang. A hybrid machine learning approach to localizing the origin of ventricular tachycardia using 12-lead electrocardiograms. Computers in biology and medicine, 126:104013, 2020.
[50] I. Naiman, N. Berman, I. Pemper, I. Arbiv, G. Fadlon, and O. Azencot. Utilizing image transforms and diffusion models for generative modeling of short and long time series. Advances in Neural Information Processing Systems, 37:121699 121730, 2024.
[51] I. Naiman, N. B. Erichson, P. Ren, M. W. Mahoney, and O. Azencot. Generative modeling of regular and irregular time series data via Koopman VAEs. In The Twelfth International Conference Learning Representations. 2024.
[52] L. Nochumsohn and O. Azencot. Data augmentation policy search for long-term term forecasting. Transactions on Machine Learning Research, 2025.
[53] R. Olszewski. Generalized feature extruction for structural pattern recognition in time-series data. Carnegie Mellon University, 2001.
[54] M. Oquab. T. Darcet, T. Moutakanni, H. V. Vo. M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabhat, V. Sharma, G. Synnaeve, H. Xu, H. Jégou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski. DINOv2: Learning robust visual features without supervision. Trans. Mach. Learn. Res., 2024.
[55] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 8748 8763. PmLR, 2021.
[56] U. Rebbapragada, P. Protopapas, C. E. Brodley, and C. Alcock. Finding anomalous periodic time series: An application to catalogs of periodic variable stars. Machine Learning, 74(3):8113133, Dec. 2008.
[57] P. Ren, R. Nakata, M. Lacour, I. Naiman, N. Nakata, J. Song, Z. Bi, O. A. Malik, D. Morozov, O. Azencot, et al. Learning physics for unveiling hidden earthquake ground motions via conditional generative modeling. arXiv preprint 2xX.::2447.15088 2024.
13

---

[58] D. Roverso. Plant diagnostics by transient classification: The aladdin approach. International Journal of Intelligent Systems, 17, 2002.
[59] I. Silva, J. Behar, R. Sameni, T. Zhu, J. Oster, G. D. Clifford, and G. B. Moody. Noninvasive fetal ecg: The physionet/computingg in cardiology challenge 2013. In Computing in Cardiology, pages 149-152, 2013.
[60] I. K. Sirot and O. Azencol. Curvature enhanced manifold sampling. In Forty-first International Conference Machine Learning, ICML, 2025.
[61] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Reprexentations, 2021.
[62] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[63] Y. Song, J. Sohl- D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative mod- eling through stochastic differential equations. In International Conference Learning Representations (ICLR). 2021.
[64] Y. Su, Y. Zhao, C. Niu, Liu, W. Sun, and D. Pei. Rohust anomaly detection for multivariate time series through stochastic recurrent neural network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 119, pages 2828-2833 ACM, 2019.
[65] G. Team, R. Anil, Borgeaud, J.-B. Alayrac, Yu, R. Soricut, J. Schalkwyk, A. M. Dai, Hauth, K. Millican,, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 1880, 2023.
[66] E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In 2012 JEEE/SS International Conference on Intelligent Robots and Syxtems, pages 5026-5033, 2012.
[67] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro,
F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 13X97, 2023.
[68] Q. Wen, L. Sun, F. Yang, X. Song. J. Gao, X. Wang, and H. Xu. Time series data augmentation for deep learning: A survey. In Proceedings the Thirtieth International Joint Conference on Artificial Intelligecee, IJCAI, pages 4653 3660. ijcai.org. 2021.
[69]
Wetterstation. Weather. https: http::/wwwwwwwcwww// bgc- jenajjnnaag...... mpg. de/wetter/
[70] G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo. Unified training of universal time series forecasting transformers. In Proceedings the 41st International Conference Machine Learning. ICML'24. JMLR.org. 2024.
[71] H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2023.
[72] X. Yi, Y. Zheng, Zhang, and T. Li. ST-MVL: Filling missing values in geo-sensory time series data. In Pruceedings of the Swenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI). IJCAI 16. pages 2704-2710, New York, NY, USA, 2016. AAAI Press.
[73] Yoon, D. Jarrett, and M. Van der Schaar. Time-series generative adversarial networks, Advances in neural information processing systems, 32, 2019.
[74] X. Yuan and Y. Qiao. Diffusion-TS: Interpretable diffusion for general time series generation. In The Twelfth International Conference on Learning Representations, 2024.
[75] H. Zhou, S. Zhang, Peng, Zhang, J. Li, H. Xiong, and W. Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11115, 2021.
[76] L. Zhou, Y. Du and J. Wu. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5826-5835, 2021.
[77] T. Zhou, P. Niu, L. Sun, R. Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advancex in neural information processing systems, 36:43322 43355, 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The related work, background, benchmarking, method and experiments sec- tions support the claims made in abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
Justification: Section 1. which presents one of our main contributions, contains a discussion about currently challenging datasets the fine-tuning corpus.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
We don't 't show any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: All models and hyper- paramettrsss extensively reported in the appendix. In addition, the code will be publicly available at the end of the double-bling blind process.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path reproducing verifying the results. 5. Open access to data and code
16

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [NA]
Justification: All datasets are public and referenced, and the code will be publicly available at the end of the double-blind process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
Polooylllllyyoyylo for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.ccpublle wwpppppgpuuiiieeeee http://ggiideeeeeecubbbbiis.. SondeeSubissiie ssiiiiolccyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All information provided inside Experimental settings section in appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Yes, provided in the appendix section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Provided in both subsection about pre- pre-training and section 6.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps///horris...ppppptt/t/tei
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Framework received a section related work and method. Additional infor- mation such as datasets, etc. were referenced.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

A Additional Dataset Details
We provide detailed descriptions of the datasets used in our framework, divided into two groups: those used for large-scale pre-training and those used for evaluating few-shot generation. The datasets span a broad range of domains, temporal structures, and channel dimensionalities, and include both real-world and synthetic sources. These datasets originate from a variety of source tasks, including classification, forecasting, and anomaly detection. In our setting, however, we do not make use the task-specific labels objectives. Instead, treat all datasets uniformly as multivariate time series, enabling a consistent generative modeling framework across diverse data types.
Name
# Samples
Task
# Variables
Domain
Stock
3661
Generation
Finance
Energy
19711
Generation
28
Energy
ETThl
8617
Forecasting
Energy
Exchange
5288
Forecasting
Finance
MSL.
58294
Anomaly Detection
55
Space
SMAP
132458
Anomaly Detection
25
Space
PSM
135160
Anomaly Detection
25
Cloud
SMD
7084
Anomaly Detection
38
Cloud
ECG5000
200
Classification
ECG
NonlvvaiveeetaBBCCCToozz
120
Classification
ECG
SelfRegulationSCP2
500
Classification
EEG
Blink
1800
Classification
EEG
ElletricDeevices
8926
Classification
Sensors
Trace
100
Classification
Sensors
FordB
3636
Classification
Sensors
UWWaeGestureLibrary
2238
Classification
Human Activity
EMOPain
968
Classification
30
Human Activity
Chinatown
20
Classification
Traffic
SharePriceInereasee
965
Classification
Finance
Table 6: Summary of pre-training datasets including number of samples, task type, number of variables, and domain. All datasets are framed as generation tasks.
A.1 Pre-training Datasets
The following datasets are used for unified pre-training of the diffusion model described in Sec. These datasets cover wide variety domains, sequence characteristics, and input sizes, allowing the model to learn generalizable representations. A summary of the pre-training datasets, including number of samples, original task type, channel size, and domain, provided Table
consists of daily historical Google stock data from 2004 2019, comprising six channels: high, low, opening, closing, adjusted closing prices, and trading volume. The data largely non-periodic and exhibits random-walk behavior typical of financial time series.
Stocks. The Stocks dataset 73
Energy. The Energy dataset is multivariate appliance energy prediction dataset [9. featuring 28 channels with correlated features. It exhibits noisy periodicity and contains continuous-valued measurements typical of real-world energy consumption data.
ETTH1. ETTh is part of the Electricity Transformer Temperature (ETT) dataset collection 75
described in App. A.2 It provides hourly readings of transformer load and oil temperature, and is characterized by strong periodic and seasonal patterns.
Exchange. The Exchange dataset contains daily exchange rates of eight foreign currencies Australia, United Kingdom, Canada, Switzerland, China, Japan, New Zealand, and Singapore, spanning the years 1990 to 2016. This dataset captures long-term financial trends and exhibits moderate temporal variability.
22

---

MSL. The MSL dataset [32 includes labeled anomalies from the Mars Science Laboratory Curios- ity rover. It features both point and contextual anomalies identified from spacecraft telemetry.
The SMAP dataset [32 contains labeled telemetry anomalies from NASA's Soil Moisture Active Passive satellite. Anomalies are categorized as point or contextual, based on their temporal characteristics.
SMAP.
PSM. The PSM dataset [ contains multivariate server telemetry data collected from multiple application nodes eBay. consists of anonymized system-level metrics and used evaluate asynchronous multivariate time series anomaly detection methods.
SMD. The SMD dataset 66 consists of multivariate monitoring data collected over five weeks from production server machines in a large Internet company. Each machine treated as separate entity with its own training and test split and labeled anomalies. The dataset is commonly used to evaluate anomaly detection under operational server conditions.
sourced from PhysioNet and contains pre-processed heartbeat segments extracted from a long ECG recording of a patient with severe congestive heart failure. Each heartbeat resampled to a fixed length and annotated into one of five classes using automated labeling.
ECG5000. The ECG5000 dataset 25
The NoniivaiiveFetallCCCTTToo dataset [59] from PhysioNet contains abdominal ECG recordings used for noninvasive fetal monitoring, Signals were collected from the thorax of pregnant subjects and are used to study fetal QRS detection in the presence of dominant maternal ECG signals.
Non
The SelfRegulationSCPP dataset 2 contains EEG recordings from an artificially respirated ALS patient performing a brain computr interface (BCT) task. The subject was instructed to regulate slow cortical potentials to move cursor up or down, guided by visual and auditory cues. Each trial consists of a 4.5-second segment sampled 256 Hz.
SelfRegulationSCCP2
Blink. The Blink dataset [14 contains EEG recordings collected during a binary classification task involving short and long blinks. Each subject was instructed to blink for 2 seconds, with data collected across multiple trials. The EEG was sampled 255 Hz over 2-second window per blink.
The ElectrieDevices dataset [43 was collected part of the UK's Powering the Nation study, aimed at analyzing household electricity usage to inform carbon reduction strategies. contains electricity consumption readings from 251 households, sampled at two-minute intervals.
ElectricDevices.
Trace. The Trace dataset 58 is synthetic 4-class subset derived from the Transient Classification Benchmark, simulating instrumentation failures in nuclear power plant. was originally designed to support research in diagnostic systems for industrial processes. All instances are z-normalized and interpolated to uniform length.
The FordB dataset 18 originates from a classification challenge the IEEE World Congress on Computational Intelligence (2008). It contains engine noise recordings used to detect the presence absence of a specific automotive subsystem symptom. Training data were collected under normal operating conditions, while test data include added noise.
FordB.
UWWavrGestureeLbbaary The LWWveGestureLibraryy dataset [46] contains accelerometer recordings of eight predefined gestures captured along the X, Y, and Z axes. The dataset was collected for personalized gesture recognition using mobile devices.
EMOPain. The EMOPain dataset 21 includes motion capture and physiological recordings from participants- both healthy individuals and those with chronic lower back pain pain-pprforming physical exercises that mimic daily activities. The dataset supports research in multimodal pain recognition and behavioral analysis.
23

---

The Chinatown dataset [8] contains time series of pedestrian counts recorded in Melbourne's Sannatoow--Sansson Street (North) throughout 2017. Each series represents a day's pedestrian volume, and the task is to classify whether the day is a weekday or weekend. The dataset was collected as part of a city-wide initiative to monitor and plan urban foot traffic.
Chinatown.
The SharePriceIncrease dataset 488 consists of 60-day time series represent- ing daily percentage changes in the closing prices NASDAQ- 100 companies prior to quarterly carnings reports. The task is to predict whether a stock's price will rise by more than 5% following the announcement. Labels are based on post- announcemmnt movement, and the dataset includes a mix of positive and negative examples derived from real-world financial data.
Sherese
A.2
Few-shot Evaluation Datasets
Name
# Samples
Task
Variables
Domain
MuJoCo
3000
Simulation
Physics
ETTml
34369
Forecasting
Electricity
ETTm2
34273
Forecasting
Electricity
ETTh2
8353
Forecasting
Electricity
Sine
10000
Synthetic Generation
Simulated
Weather
36696
Forecasting
21
Weather
ILI
581
Forecasting
Healthcare
SaugeenRiverFlow
18921
Forecasting
Weather
ECG200
200
Classification
ECG
SeliRegulationSCPP
268
Classification
EEG
StarreshtCurves
1000
Classification
Sensor
AirQuality
9357
Generation
13
Nature
Table 7: Summary of evaluation datasets used for few-shot generation, including number of samples, task type, number of variables, and domain. All datasets framed generation tasks.
We evaluate few-shot generalization on 12 datasets selected to cover diverse domains, temporal dynamics, and input dimensionalities. We treat all datasets as unconditional generative modeling tasks, even if they were originally used for tasks like classification or forecasting. A summary of all evaluation datasets, including the number of samples, task, number channels, and domain type, is provided in Table
MuJoCo.
The Multi-Joint dynamics with Contact (MuJoCo) dataset [66 consists of simulated physical trajectories in robotic environments. Each sequence captures multivariate joint positions, velocities, and contact forces. is commonly used in control and reinforcement learning settings.
ETTm1, ETTm2, ETTh2. The Electricity Transformer Temperature (ETT) datasets 75 contain
load and oil temperature readings collected from electricity transformers at regular intervals. ETTml and ETTm2 are sampled at the minute level, while ETTh2 is sampled hourly. These datasets exhibit strong periodicity and seasonal patterns.
Sine. The Sine dataset is synthetic multivariate time series dataset, where each sample consists of six channels indexed by i. Each time series is generated according to x (j) sin(nt + 0). where the frequency 1 and phase 0 are independently sampled from a uniform distribution: n,, 0 24[0, 0.1]. Both and vary across samples and channels, resulting in diverse temporal patterns.
Weather. The Weather dataset 699 includes 21 meteorological variables recorded every 10 minutes over the full year of 2020. Features include air temperature, humidity, wind speed, and other environmental indicators.
ILI. The Influenza-Like Illness (ILI) dataset 12 symptoms, reported by the CDC the United States from 2002 to 2021. Each value reflects the ratio of ILI cases to total patient visits.
contains weekly records of patient visits for II.I

---

Saugeen River Flow. The Saugeen dataset [47 is long univariate time series recording the daily average river flow (in m /s) Walkerton, Canada, from 1915 1979.
ECG200. The ECG200 dataset 53 contains univariate electrocardiogram signals representing one heartbeat per sequence, labeled as normal abnormal. It is standard benchmark for time series classification.
SellRegulationSCPL. The SeliRegulationSCPI dataset 2 includes multichannel EEG signals recorded during Brain-Computer Interface (BCI) task. The subject was instructed to self-regulate slow cortical potentials (SCPs) to move a cursor. Each trial consists of 3.5-second EEG segments sampled at 256 Hz across six electrodes.
StarL.ightCurves The Stgtrveg dataset [56 contains univariate astronomical light curves of fixed length 1024. Each curve represents the brightness of celestial object over time and is labeled into one of three variability-based categories.
AirQuality. The AirQuality dataset 722 contains hourly averaged readings from five metal oxide chemical sensors integrated into an air quality chemical multisensor device. The device was deployed at road level in a heavily polluted area of an Italian city. Data were collected continuously from March 2004 February 2005, resulting in the longest freely available on-field record chemical sensor responses related to air quality.
25

---

B Experimental Setting
This section provides detailed information about our model architecture, training procedures, hyper- parameters, and evaluation methodology. We split our discussion into pre-training and fine-tuning phases to reflect the two-stage setup used all experiments.
B.1 Model Architecture
Our generative model follows denoising diffusion framework built image-based architectures, specifically extending the Elucidated Diffusion Model (EDM) [36] and ImagenTime [50]. The model maps multivariate time series into image-like tensors via delay embedding, enabling reuse of visual backbone structures such as UNet. It incorporates two key additions: (1) a Dynamic Channel Adaptation module (DyConv) for handling varying channel counts across datasets, and (2) a dataset token for dopain-specific conditioning. These innovations make our model suitable for unified pre-training across diverse domains.
Delay embedding. Following [50] given an input time series x E R with K channels and length L, we convert each univariate channel into local trajectory matrix using delay embedding. For skip parameter m and column window size the transformation constructs a matrix:
Im+1
TL-n
22
Tm+2
IL-nt1
In
Um
FL
where 9 =
Each of the K channels is processed this way and stacked into tensor Ting E R RRxxxx which is then zero-padded to a square shape TR Xxxxn to fit the image backbone.
UNet backbone. The image Ting is passed through 2D UNet, similar to those used in diffusion models for vision. The architecture features residual blocks, downsampling and upsampling paths, and additional attention layers applied at specific resolutions. Temporal information is encoded using a noise-level embedding based on the EDM (36 design, and optional dataset-level context injected via adaptive normalization layers.
Dynamic channel adaptation (DyConv). To support variable input/output channel dimensions across datasets, we introduce DyConv-a dynamic 2D convolution module. DyConv defines canonical learnable convolutional kernel W E IR x XCx where K = 3 is the spatial kernel size and C. = C1 128 are fixed reference channel dimensions. At runtime, the kernel is resized via bicubic interpolation over the channel dimensions to produce Wi E IR. x xCxxx.x. matching the actual dataset. The same interpolation applies to the bias. DyConv is used at the first and last layers of the UNet to map between dataset-speciff inputs/outputs and shared latent channel space. This mechanism allows a single model to operate on datasets with widely varying dimensionality without re-training.
Dataset tokens. To enable domain-specifte conditioning, each dataset is assigned a unique learnable token. During training and generation, the token is embedded and injected into the denoising model via AdaGN layers [20]. modulating intermediate activations based on dataset identity. This mechanism improves sample fidelity and enables multi-domain generation from single model (see 6.c When evaluating without dataset tokens (unconditional setting), the model operates without this guidance.
Dynamic Masking Implementation. To support variable-length inputs within a fixed-size image representation, we generate a binary mask that identifies valid mon-padded) regions of each input. Specifically, after applying delay embedding and zero gero-padding to a fixed spatial size, we apply the same transformation to an all-ones tensor, producing a mask that aligns with the padded image. This mask dynamically adapts to the sequence length of each sample, while the overall image size remains fixed. During training, noise is added the entire image, but the loss computed only over the valid pixels, ensuring that the model focuses meaningful regions and ignores padded areas.
26

---

B.2 Training Procedure
We detail the procedure for each stage of our two-step framework and provide the complete pseu- docode for both the pre-trainnn pre-training and few-shot phases in Algorithm 1
Algorithm 1: Unified diffusion pre-training and few-shot adaptation
Input:
m heterogeneous datasets; T: time-to-mmaee transform; No: diffusion model; Tok: dataset token table; Σ: noise schedule; 77: learning rate
Output: Learned parameters
1: Multi-domain Pre-training:
2: foreach training step do
3: B + sample_batche ( (D(m)) aM
Jm=1
> Sample batch of series and dataset indices
4:
foreach (x, m) E B do
5:
y Tok m]
Retrieve token for dataset m D. Transform and pad to image
6:
img -- (Td__eeeee(((())
7:
- sample(E))
o(0,021):
> Sample noise level b. Apply noise D. Predict clean image Compute preconditioned loss b Parameter update
8:
2 mn
+ ing +
9:
X + Na 0, y)
L {(a)l 0
11:
(6, Tok) t AdamW(B. (,, Tok, VL,n);
12: Few-shot Adaptation;
Initialize token y" for Dnew foreach fine-g step do B < sample_ (atppleeaaachheeeeeeeee
Allocate for domain
14:
Sample few-shot batch
16:
foreach x E B do
Repeat lines 5-11 with y y";
Use domain token for adaptation
B.2.1 Pre-training Procedure
We pre-train our unified diffusion model across a diverse collection of time series datasets (see Table This stage exposes the model wide spectrum of temporal dynamics, data modalities, and channel dimensionalities, encouraging it to develop transferable representations that generalize well under data scarcity. During pre-training, each sample is first converted into an image via delay embedding (see Section B. padded to square resolution, and then diffused using Gaussian noise as defined in the Elucidated Diffusion Models (EDM) framework (36] We apply our proposed DyCony layer to dynamically adapt to varying channel dimensions across datasets and condition the denoising network on dataset token, allowing the model incorporate domain- specific signals without requiring separate models per dataset. To improve stability and generalization, we maintain an exponential moving average (EMA) of the model weights throughout training. All training runs use two NVIDIA RTX 4090 GPUs and complete in approximately 4 hours. Following EDM preconditioning, noise levels are sampled from a fixed log-normal noise schedule. The full pre- training workflow, is described in Algorithm I (Mulli-domain Pre-training). All hyperparameters specific to the pre- pre-training phase are listed in Table while the core architectural parameters, shared between both pre-training and fine-tuning, summarized in Table 9
B.2.2 Training Loss
Our approach to defining the loss function for our diffusion process follows the methodologies presented in EDM [36] and Imagen'inm [50]. More specifically, the objective of the model is to clean the noise via the following loss function:
E _ = (A(a) T 0) - 1131
(4)
where X(o) is a weighting function and l is defined as follows:
L' (2ingg σ) (a)ip + Coullr) (a) ( Cnoise (a):
(5)
27

---

Table Pre-training hyperparameters.
Parameter
Value
Optimizer
AdamW W
Learning rate
1 10-1
Batch size
2048
Epochs
1,000
EMA decay
0.9999
Weight decay
1 10
Table 9: Model architecture hyperparameters used in both pre-training and fine-tuning.
Component
Value
Delay embedding skip (m)
8
Delay embedding width (n)
8
Image resolution
x 8
UNet base channels
32
Channel multipliers
[1. 2, 2, 4]
Attention resolutions
[8, 2] 36
Diffusion steps
DyConv kernel size
3
DyConv canonical shape
[3, 3, 128, 128] Bicubic (channel dimensions)
DyConv interpolation
where No is neural network. Cakip modulates the skip connection, Gn and Cour scale the input and output magnitudes, and Cnaise maps noise level into a conditioning input for Ng. For robust training, use the same values in EDM's preconditioning (36), where the terms Casp((), Cin (in(o). Cour (ou( and Cnoise (o) are all functions o:
(skip(())
dea
Com((o)
J.0jm Vo
Ci((o)=
Va²+o
Cnoisee (o) 1
We use data = 0.3 as fixed parameter, and a(t) controls the noise level over time. These terms help stabilize the loss, which vary with Substituting Eq. ii Eq. 4ielld per-sample loss weight. To balance this, EDM sets A(o)) = =1/cou(a)) also ensuring uniform initial loss across the o range. Finally, we feed the network with an additional optional dataset token input y.
B.2.3 Few-Shot Generation Adaptation
To adapt the pre-trained diffusion model to novel domains with limited supervision, we perform fine-tuning using dedicated dataset token for each dataset. For every unseen domain, initialize a new token embedding that uniquely identifies the dataset (see Section B. This token is learned jointly with the model parameters during fine-tuning, guiding the generative process toward the target distribution while reusing the temporal representations acquired during pre-training. All architectural components, including delay embedding and DyConv, retained during fine-tuning. Fine- tine-ttning is performed for each small new dataset (see Table in accordance with our few-shot benchmark protocol, and the same configuration is used across all datasets for consistency. The model is trained for 1,000 epochs. Note that due to the small size of this dataset, training is fast, and empirically we find that 100 epochs are sufficient for most cases. An exponential moving average (EMA) of the model weights is maintained throughout the fine-tuning phase to improve stability and sample quality. Due to the data-scarce nature of our few-shot settings, the effective batch size is set to the minimum of 2048 and the number of available samples in the dataset. All datasets are fine-tuned
28

---

using the same configuration. The overall adaptation procedure follows the workflow in Algorithm (Few-shot Adaptation), with the new dataset token initialized and optimized during fine-tuning, All hyperparameters specific to the fine-tuning phase are listed in Table TO
|
Table 10: Training hyperparameters used during the fine-tuning phase.
Hyperparameter
Value
Optimizer
Adam W
Learning rate
10-4
Epochs
1,000 min(2048, # training samples)
Batch size
EMA decay
0.9999
Weight decay
x 10-6
B.3 Evaluation Protocol
We extend standardized time series generation metrics 76 broaden their applicability. allowing for consistent evaluation across both the original setup and few-shot learning setup. For each target dataset, we fine-tune the pre-trained model using either small percentage of the training data (5%, 10%, 15%) fixed number of examples (10, 25, 50). In all cases, after fine-tuning, we evaluate the model by generating samples in equal number to the original size of the test set for each dataset. This ensures fair comparison between generated and real data distributions and avoids bias from test-set size variability. We employ the following three metries to evaluate different aspects of generative performance:
Discriminative Score. To quantitatively evaluate the similarity between real and generated time series data, we adopt the framework proposed by 73 Specifically, train a post-hoc LSTM-based time series classifier to distinguish between sequences originating from the original dataset (labeled real) and those from the generated dataset (labeled synthetic). The model is trained in a standard supervised learning setup, and the classification error on held-out test serves measure of distributional similarity. We then subtract this error from 0.5, such that a score of O indicates perfect indistingussaabiiity while higher scores reflect greater divergence.
2. Predictive Score. To assess the predictive utility of the generated data, again follow an evaluation protocol proposed by [73), which tests whether synthetic data can support forecasting tasks. Specifically, next-step prediction: given a sequence of past time steps, the model forecasts the next temporal vector. The trained model is then evaluated on the original dataset using mean absolute error (MAE) as the performance metric. A low MAE indicates that the synthetic data captures the conditional temporal dynamics of the original data well enough to generalize real sequences.
train an LSTM model on the synthetic dataset to perform
3. Context-FID To measure global and contextual realism, we use the Context-Fréchet Inception Distance (Context-FID) (33. This is adaptation of the FID score used in image generation, but tailored for time series. Rather than using image- based features, Context-Fil uses embeddings extracted from contrastively trained encoder (23), separately trained for each dataset to capture temporal context. The final score reflects the Fréchet distance between the embedding distributions of real and synthetic samples. Lower values correspond to higher-quality generations.
29

---

C Additional Experiments
C.1 Impact Canonical Kernel Size in DyConv
To understand how the internal dimensionality of DyConv affects performance, conduct an ablation study varying the size of the canonical kernel. All models share the same architecture and pre-training procedure (Sec. [5), differing only in the channel dimensions of DyConv's canonical kernel. Specifically, we vary the shape K, K, Cu. C1] while keeping the kernel size fixed at K 3. Each configuration is denoted DyConv[Ca, Ci1, and evaluated the same few-shot benchmark. As shown in Table [1 The configurations DyConv[32,12) strong results, exhibiting low values in both discriminative score and contextFID. Interestingly, even the smaller DyConv[16, 128] performs reasonably well, albeit with reduced effectiveness, In contrast, the smallest configuration, DyConv[L.128) shows clear degradation across both metries, highlighting the necessity of adequate parameterization. These findings support the conclusion that increasing the capacity of DyConv may yield better performance, with diminishing performance observed in extremely limited parameter regimes.
and DyConv[ [1 128, 128] achieve similarly
Table 11: Ablation study on the effect of DyConv's internal channel dimensions.
Config
Disc.
c-FID
# Parameters
[1. 128]
0.381
16.74
1,280
[16, 128]
0,149
1,945
18,560
[32, 128]
0.138
1.612
36,992
[128, 128]
0.143
1,451
147,584
C.2 Effect of Fine-tuning Methods
We investigate two parameter-efficient fine-tuning (PEFT) strategies applied to our pre-trained diffusion model: Low-Rank Adaptation (LoRA) and Bias-Only tuning (BitFit). Both approaches aim to reduce the number of trainable parameters while enabling adaptation new datasets in the few-shot regime. In the LoRA setup BT, we inject trainable low-rank matrices into the attention layers of our UNet architecture, specifically into the projection matrices W W.. W., and W. of each attention block, using a rank of r 16. This configuration introduces only 126K trainable parameters. In the Bias-Only setup [3, all weights are frozen, and only the bias terms across all layers are updated, resulting in approximately 300K trainable parameters. As shown in Table [4] full fine-tuning consistently outperforms both LoRA and Bias-Only tuning across all subset sizes and all evaluation metrics, including contextFIID, Diseriminative Score, and Predictive Score. While LoRA and BitFit offer substantial parameter savings, their performance lags significantly behind full fine-tuning, This underscores the trade-off between efficiency and effectiveness in few-shot scenarios.
C.3 Evaluating Sequence-Nativ Baselines Under the Few-Shot Setting
To further examine the role of the underlying backbone architecture, we conducted an additional experiment comparing our magee framework to sequence-native generative models. Specif- ically, we considered DSffusionTS (74 and KoVAE [51], both of which operate directly in the temporal domain without converting sequences into images. Following the protocol of Section 6. we pre-trained both baselines on the same heterogeneous corpus as our model and fine-tuned them under identical few-shot conditions. Table 12 reports the averaged Discriminative Score (Disc.), Predictive Score (Pred.), and contextFID (c-FID) across both percentage- based and fixed-count subset regimes. Across all subset sizes and evaluation metrics, our model consistently achieves the best results, demonstrating the effectiveness of the image-based backbone in few-shot regimes compared to sequence-native alternatives.
C.4 Dataset Token Robustness Under Overlapping Domains
To evaluate the effectiveness of dataset token conditioning when domains are closely related, we pre-trained two variants of our model using only the ETT dataset family (ETTh1, ETTh2, ETTml, ETTm2): one with dataset token conditioning and one without. Both models were evaluated on the
30

---

Table 12:
Few-shot generation performance comparison between our unified model (pre-trained on sequence length 24) and sequence-native baselines. Lower is better for all metrics.
Subset size
Metric
Ours (24)
DiffusionTS
KoVAE
Disc.
0.110
0.244
0.331
5%
Pred.
0.458
0.471
0,500
c-FID
0.674
2.668
3,719
Disc.
0.083
0.223
0.342
10%
Pred.
0,452
0.464
0,491
e-FID
0.578
2.432
3,874
Disc.
0.066
0.223
0.338
15%
Pred.
0.451
0.466
0.486
e-FID
1.086
2.475
3.549
Disc.
0.259
0.347
0.375
#10
Pred.
0.489
0,516
0,517
c-FID
3,800
4.702
5.522
Disc.
0.190
0.310
0.352
#25
Pred.
0,467
0.492
0.499
c-FID
1,582
3.817
3.844
Disc.
0.149
0.287
0.354
#50
Pred.
0.460
0.480
0.491
c-FID
0.987
3.116
4.001
same ETT datasets without fine-tuning. This setup allows us to assess whether the dataset token can help disambiguate subtle inter-dataset variations during sampling. The results are summarized in Table K
Table 13: Evaluation of pre-trained models on the ETT dataset family with and without dataset token conditioning. Lower is better for all metrics.
Dataset
Metric
w/ Dataset Token
w/o Dataset Token
Disc.
0.033
0.344
ETTh1
Pred.
0.646
0.674
e-FID
0.108
5.687
Disc.
0.027
0.263
ETTh2
Pred.
0.681
0.707
c-FID
0.086
3.360
Disc.
0.007
0.232
ETTml
Pred.
0.675
0.690
c-FID
0.0210
2.920
Disc. 0.007
0.236
ETTm2
Pred.
0.694
0.718
c-FID
0.0249
1.248
Across all four datasets, using the dataset token yields substantially better scores across all metrics compared to the model trained without it. This demonstrates that dataset token conditioning remains robust even in domains with overlapping or ambiguous boundaries, enabling the model to effectively distinguish subtle distributional differences during generation.
C.5 Datasets Analysis
We investigate why our model fails to fine-tune effectively on certain datasets, such as Weather and ECG200, which remain challenging. We hypothesize that this difficulty stems from a failure to generalize to frequency distributions that were not encountered during pre-training.
31

---

Vesserstein Distance (Meen Pretrain
enerry jeretraio
ETTh (prctrane)
(peitran))
PSN
ster (unclrek))
SHD (sretrak))
Gaffeeusitorro' (pectrat))
(pretrant
Ne ECoTToruns (prcsain)
Birk lancraid)
(pectran)
EMOP (pritrent)
rutsen (pectian)
Sharenfoonc
marcc (frecue))
ETTh dineuanl
E'Imn (hnibann)
TU Crndundd
Soal Cinetuse)
Seugenttrarion (fnstunc)
Struughocurves (incuunc)
Il da mezee aooeen
Figure 4: Wasserstein distance between dataset frequency distributions.
To test this hypothesis, we analyzed the frequency content of all datasets. We applied the Fast Fourier Transform (FFT) to every signal both our pre-training and fine-tuning collections. For each dataset, we then computed a mean amplitude spectrum, a, by averaging the amplitude vectors of all signals:
N
a2
where N is the number of signals in the dataset and a; is the amplitude spectrum vector for the i-th signal.
This vector a serves as signature for the dataset's average frequency distribution. To quantify the dissimilarity in frequency content, we compute Wasserstein distance between their respective normalized mean spectra. We denote this as W(D, Î), where D and H are two different datasets from the collection of pre-trained and fine-tuned datasets. As shown in Fig. 4 our hypothesis is partially supported. The ECG200 dataset, on which our model performs poorly, exhibits a large frequency distance relative other datasets. However, Weather, another dataset where the model fails, shows a relatively small distance. This suggests that factors other than frequency distribution may be responsible for its low performance. Finally, while Non Invanv NonInvasiveFeta 1eeFeta1ECGThoaaa has the largest distance from all other datasets, this does not negatively affect performance, as it was included in the pre-training set.
C.6 Full Results of Main Table
In this section, we present the complete results of our few-shot benchmark. Tables 15 16 17 1 19
20 report Discriminative, Predictive, and contextFID scores across datasets (each table contains two datasets, together the tables represent all) and training subset sizes. Our model demonstrates marginal improvements over competing methods on several datasets, including ETTh2, ETTml, ILL, and SelfRegulationSCPPL among others. Overall, it achieves the highest performance in 141
32

---

out of 168 cases in Discriminative and contextFID metrics combined. For brevity, we use acronyms for some datasets: SRF denotes SaugeenRiverFlow. SCP1 denotes SelfRegnlationSSCP,, and SLC denotes StuveightCurves.
C.7 Full results on pre-training datasets: Cond. vs. Uncond.
Table 21 provides the complete results on all pre-training datasets, comparing models trained with and without dataset token conditioning (Cond. vs. Uncond.). These evaluations were performed directly on the pre-trate pre-training datasets without any further fine-tuning. The results demonstrate that conditioning with dataset tokens leads to consistently improved performance across most datasets and metrics, particularly in terms of contextFID and Discriminative Score. This reinforces the importance of explicit Lataset-specifee conditioning during pre-training for improving generation quality and alignment the target distribution.
Table 14: Ablation study on fine-tuning methods. We compare LoRA (126K) and Bias-Only (300K) tuning across multiple subset sizes. Lower is better. Bold indicates best.
Subset Size
Metric
Full FT (6M)
LoRA (126K)
Bias-Only (300K)
Disc.
0.110
0.317
0.340
5%
Pred. score
0.458
0.495
0.508
contextFID
0.674
5.824
8.745
Disc, score
0.083
0.312
0.332
10%
Pred. score
0.452
0.469
0.489
contextFID
0.578
5.297
8.397
Disc. score
0.066
0.302
0.337
15%
Pred. score
0.451
0.489
0.488
contextFID
1.086
6.232
7.435
Disc. score
0.259
0.332
0.363
#10
Pred. score
0.489
0.496
0.544
contextFID
3.800
7.396
9.417
Disc.
0.190
0.323
0.347
#25
Pred. score
0.467
0.484
0.514
contextFID
1.582
6.267
9.173
Disc, score
0.149
0.326
0.343
#50
Pred. score
0.460
0.494
0.517
contextFID
0.987
6.345
9.882
33

---

Table Main Table Results Part The table reports discriminative and predictive scores along
with their standard deviations, as well as the contextFID score.
dataset
Subset
Metric
Our[24]
ImagenTime
DiffTS
KoVAE
TimeGAN
AirQuality 5%
Disc.
.065t.04
.49t.0
.17#.01
.458.033
.441..0
Pred.
.011+.0
.019..01
.007+.0
.043..0
.003..0
c-FID
0.441
6.858
0.721
3,971
1.317
10%
Disc.
.085..07
.51+.01
.171.011
.48+.04
.322..12 12
Pred.
.01+.0
.009..0
.006t.0
.041..0
.005..0
c-FID
0.419
1.476
0.657
3.336
2.995
15%
Disc.
.0662.06
.64.01
.174..01
422+.02
.45+.00
Pred. .009+.0
.0092.0
.00+.00
.04..0
.002..0
c-FlI)
0.421
1.65
0.583
3.571
2.089
100%
Disc.
.065t.03
0874.04
.116..0
.333.03
.4+.07
Pred. .006+.0
.009..0
.005..0
.039..0
.004..0
c-FID
0.284
0.401
0,282
2.589
2.566
10#
Disc.
.344.003
494+.01
406+.02
488+.0
.26+.112
Pred.
.041+.0
.044..0
.044..0
.044.00
.002..0
c-FID 1.359
4.43
5.713
3.994
2.415
25#
Disc.
.222:118
.426.. 14
3744.12
.83+.01
.387+.09
Pred. .019+.0
.0172.0
.038+.01
.044..0
.023+.01
c-FID) 1.099
3.184
3.363
4.221
2.89
50#
Disc.
303+.15
406+.14
393+.04
483+.01
.65+.112 12
Pred. .019..0
.014.00
.016..0
.044..0
.004+.0
c-FID
0.841
2.706
2.599
3,433
1.603
ECG200
5%
Disc. .098:.05
.155.06
.292.007
.333.04
.663.09
Pred. 1.061
1.061+.0
1,061
1.062+.0
.455..0
c-FID
0.866
0.793
2.15
4.199
40.207
10%
Disc. 128+.08
.08t.05
.38$.007
338+.06
.385.006
Pred. 1.061+.0
1.061+.0
1.061+.0
1.062+.0
.944..0
c-FID
0.884
0.605
2.401
3.51
11,225
15%
Disc. .037+.03
0655.00
.32.066
2977.08
.38+..14 14
Pred. 1.062+.0
1.061+.0
1.06+.0
1.062+.0
.16t..0
c-FID
0.315
0.41
2,471
3,226
24.112
100%
Disc. .037+.03
.072+.04
347+.06
307+.14
.3+.13
Pred. 1.062+.0
1.063#.0
1.062#.0
1.063+.0
8692.0
c-FID
4.194
4.169
2.431
1.661
6.445
10#
Disc.
.1172.06
.117..05
.35.077
.35.066
.385.06
Pred. 1.061+.0
1.061+.0
1.06+.0
1.062+.0
.944..0
c-FID 0.496
0.805
2.859
3.983
11.225
25#
Disc. .063+.05
0022.00
345+.07
32+.07
.72+.19 12
Pred. 1.063#.0
1.062+.0
1,061+.0
1.063+.0
.72+.0
c-FID 0.148
0.232
2.03
2,338
9.289
50#
Disc. .06+.02
063+.04
.18+.1
27+.13
.33+.15
Pred. 1.062+.0
1.062+.0
1,093+.0
1.062+.0
1.048+.0
c-FID
0.113
0.172
2.493
1.662
9.121
34

---

Table Main Table Results Part The table reports discriminative and predictive scores along with their standard deviations, as well as the contextFID score.
dataset
Subset
Metric
Our[24]
ImagenTime
DiffTS
KoVAE
TimeGAN
ETTh2
5%
Disc. .034t.01
.961.11
.7+.00
.474
.49.0
Pred.
.6882.0
.707.0
.7422.01
.795.001
.921..44
c-FID
0.175
0.691
2.112
5,322
13.861
10%
Disc.
.024:.01
.252.11
.668.002
.611.01
.485.001
Pred.
.6882.0
.98+.0
.72+.01
.199.01
.834.02
c-FID
0.148
0.704
1.653
4.621
17.031
15%
Disc. .017..01
296+.1
271+.01
458+.0
.458.00
Pred. .684s.0
703+.0
.7162.01
826#.0
.725#.01
c-Fll)
0.122
0.582
1.709
4.912
7.399
100%
Disc.
.015s.01
025+.01
.74.03
.72.01
.491.00
Pred.
.6762.0
.844.0
.703+.01
8372.01
.29+.04
c-FID
0.07
0.145
1.613
4.739
15.177
10#
Disc.
2266..0
432+.08
332+.06
495+.0
.84+.02
Pred. .7172.01
.728.00
.751..0
8044.01
.8362.01
c-FID
1.718
3.173
3.662
5,996
62.157
25#
Disc. .162..02
3892.08
338+.04
495+.0
.45+.00
Pred. .7032.0
722+.0
.747.02
831+.01
1,208+.07
c-FII)
1.113
1.821
2.455
5,417
16.998
50#
Disc.
.1+.02
44+.03
.9+.05
495+.0
.83+.02
Pred. .705..0
721+.0
.752+.01
.755..01
.077.033
c-FID
0.565
1.404
2.143
5.825
35.516
ETTml
5%
Disc.
.015s.01
386t. 11
.372.02
.74.01
.55+.02
Pred. .6811.0
.688.00
.699t.01
.222..01
.24+.02
c-FID
0.042
1.036
1.9
4.823
4.671
10%
Disc.
.011+.01
071+.02
315+.02
.433.01
.6+.01
Pred.
.68+.0
686+.0
.6912.0
.332.00
8924.03
c-FIID
0.044
0.223
1.937
4.33
7.64
15%
Disc. .013..0
034+
321+.01
4662.01
.661..33
Pred.
.6782.0
.681.00
.704..0
.725..0
.526..11
c-FID
0.029
0.097
1.937
4,461
10.099
100%
Disc. .004..0
.11..0
324+.02
458+.01
.46+.05
Pred. .675..0
.675.0
.707+.01
.723.001
.775+.03
c-FID
0.011
0.025
1.867
4.876
4.543
10#
Disc.
.322.03
449+.03
441+.01
.99+.0
.46t.03
Pred. 847+.02
.778..0
912+.01
752+.01
.708+.01
c-FIl) 4.325
4.944
4.731
6.749
7.268
25#
Disc.
.216t.02
4962.0
378+.03
.764.01
4634.03
Pred. .742+.01
.78+.00
903+.02
.734..01
.701+.02
c-FID 1.623
5.115
2.929
4.695
5.186
50#
Disc.
.148t.01
4932.0
384+.02
.71+.01
.644.02
Pred. .694..0
.7032.0
.833.02
.718..0
.691+.01
c-FID) 0.651
3.77
2.317
4.943
11.111
35

---

Table Main Table Results Part The table reports discriminative and predictive scores along with their standard deviations, as well as the contextFID score.
dataset
Subset
Metric
Our[24]
ImagenTime
DiffTS
KoVAE
TimeGAN
ETTm2
5%
Disc.
.011.011
444+.01
241+.02
.452.00
.4t.88
Pred.
.7+.0
.727..0
.736*.01
.793+.01
.7062.02
c-FID
0.071
0.995
1.593
5.077
5.184
10%
Disc.
.11+.01
.055.002
.234.001
.372.01
.4822.01
Pred.
.6972.0
.71+.0
.72+.01
.888.02
1.1062.08
c-FID
0.044
0.346
1.541
4.738
6.915
15%
Disc.
.013..0
041
233+.02
407+.02
.475.00
Pred.
.
.7052.0
.725..01
.753+.0
.677..02
c-Fll)
0.06
0.203
1.529
3.528
10.711
100%
Disc.
.004..0
.011.0
228+.03
.382.04
.482.01
Pred.
.693..0
.644.0
.713..0
.7512.0
8867.01
c-FID
0.016
0.038
1.701
3.643
7.742
10#
Disc.
.33+.01
371+.01
378+.02
.874.01
.45+.0
Pred.
.771#.01
796+.02
812+.02
806+.01
.855.06
c-FID
3.679
4.72
4.015
9.069
179.951
25#
Disc.
211+.002
4872.0
346+.06
.792.01
.394..04
Pred. .7312.01
.732.0
75+.01
.785..01
.831.01
c-FIl) 1.644
4.257
2.836
5,458
7.136
50#
Disc. .135..02
472+.01
276+.02
4754.01
.55+.04
Pred. .721..0
.749..0
.768..01
.793+.02
.662+.02
c-FID
0.981
2.527
2.469
5.776
11.327
ILI
5%
Disc.
.194t.04
483+.02
.393.04
.892.01
.45+.03
Pred. 563+.0
.82+.0
.5712.0
.61.01
.5562.02
c-FID
1.135
5.51
3,274
5.828
8.697
10%
Disc.
.13+.00
459+.05
399+.02
.49+.0
.39+.02
Pred.
.555..0
575+.0
576+.01
.605..01
.54.033
c-FIID
0.489
3.29
3.286
6.098
4.965
15%
Disc. .077..04
447+.04
403+.03
.94.01
.389.08
Pred. .5542.0
.722.0
564+.01
.6062.01
.581+001
c-FID
0.416
2.56
2.805
6.039
4.594
100%
Disc. .064..03
388+.12
368+.02
489+.01
.407..33
Pred. 554+.0
5732.0
.552+.0
.605..0
.6172.02
c-FID
0.329
2.229
1.989
7.5
4.15
10#
Disc.
.292.05
491+.01
.35+.02
.934.01
.458.044
Pred. 573+.01
.688.00
.66+.02
.607..01
.45+.011
c-FIl)
2.115
7.82
5,406
6.576
6,275
25#
Disc.
.194+.04
4894.01
425+.03
495+.0
448+.03
Pred. .567.001
586+.0
.88+.01
.628..01
.536.001
c-FID 0.996
5.939
4.085
6.809
8.127
50#
Disc. .211..03
.79+.01
376+.05
.93+.01
.31+.02
Pred. .555t.0
.88.0
574+.01
.622.01
.62+.0
c-FII) 0.615
4.102
3.081
6.093
5,333
36

---

Table 18: Main Table Results - Part 4. The table reports discriminative and predictive scores along
with their standard deviations, well as the contextFID score. SRF denotes SaugeenkiverFlow and SCP1 denotes SelfReguulaiinSSPP..
dataset Subset
Metrie
Our[24]
ImagenTime
DiffTS
KoVAE
TimeGAN
SRF
5%
Disc.
.002+.0
.011+.01
1174.02
.1372.09
.08+.06
Pred.
604+.0
.605+.0
.602..0
.6052.0
.744..0
c-FID
0.016
0.102
1.134
2.44
2.416
10%
Disc.
.004..0
.008..0
.13.01
.122.008
.354.007
Pred.
.604+.0
.64+.0
.602..0
.605..0
.038..0
c-FID
0.012
0.066
1.154
2.211
78.061
15%
Disc. .006+.0
.077.00
1232.01
.109.044
.0952066
Pred. .605..0
.604+.0
.6032.0
.604.0
.7622.0
c-FID
0.038
0.033
1.087
1.494
0.961
100%
Disc.
.004..
003+.0
111+.01
0944.05
0067+.03
Pred.
.605..0
.605+.0
.603..0
.6032.0
.726t.0
c-FID
0.007
0.005
0.988
1.531
4.384
10#
Disc. .051+.02
.644.03
.388.. 14
244.01
.14.07
Pred. .605+.0
.604..0
.602+.0
.601..0
.43+.0
c-FID)
0.815
0.706
6.409
2.732
22.553
25#
Disc. 045+.02
.0a.002
304+.1 13
.022.066
.177.006
Pred.
.603.0
.604..0
.603..
.606..0
.4742.0
c-FID
0.417
0.308
7.291
3.254
3.111
50#
Disc.
013+.01
015+.01
2874.12
.172.11
.16..08
Pred. .605..0
.604..0
.602..
.604..0
.402..0
c-FID
0.179
0.233
7.597
3.519
2.612
SCPI
5%
Disc. 2339..06
3322.06
347+.04
.552.02
.433.007
Pred. 468+.01
491+.02
5644.04
A51+.01
.624+.03
c-FID
1.159
2.97
3.671
8,329
8.149
10%
Disc.
.883t.05
312+.15
275+.05
4092.03
.28+.03
Pred. 437+.0
.31+.0
534+.02
.662.001
52+.02
c-FID
0.959
2.237
2.912
8.69
5,403
15%
Disc. .067..04
.24.1
274+.03
405+.03
.188.033
Pred.
.433+.0
433+.0
472+.02
.69+.01
558+.02
c-FID
0.575
2.821
2.546
8.36
4.99
100%
Disc. .61+.004
2844.14
123+.03
382+.04
.399.04
Pred. .24+.0
.431.00
438+.01
439+.01
.415.001
c-FID
0.467
1.593
1.372
6.34
3.75
10#
Disc. 2694.04
364+.05
411+.03
434+.02
407+.07
Pred. 464.01
462+.01
568+.05
4662.01
.43+.02
c-FID
1.293
2.958
3.433
6.792
4.702
25#
Disc. .122.00
426+.07
304+.04
396+.02
448+.04
Pred. .43+.0
.33+.00
487+.02
.59+.01
.702+.02
c-FID 0.808
2.582
2.304
7.037
5.965
50#
Disc. .052+.03
404+.09
.81+.04
427+.02
36+.08
Pred. .33+.0
.43+.00
.67+.01
.461+.01
.434.01
c-FID
0.694
3.042
2.726
9.436
5.766
37

---

Table Main Table Results- Part The table reports discriminative and predictive scores along
with their standard deviations, as well as the contextFID score. SLC denotes StarLightCurves.
dataset
Subset
Metric
Our[24]
imagen'Tine
DifTS
KoVAE
TimeGAN
SLC
5%
Disc.
0042+.03
.026:.02
.096t03
.25+.00
.4+.02
Pred.
.518t.0
.501+.0
.545..0
.502..0
.474..0
c-FID
0.201
0.053
0.406
3.147
0.295
10%
Disc.
.013+.01
0177.01
.07.033
.199:.07
.0582.03
Pred.
.497..0
.98+.0
514+.0
.504..0
.407..0
c-FID
0.01
0.037
0.224
2.902
0.234
15%
Disc. .013.00
017+.01
0564..3
1122.07
.88+.03
Pred. .498t.0
5+.0
.508.00
5502..0
.592.00
c-FID
0.008
0.056
0.128
3.608
0.248
100%
Disc. .21+.02
018+.01
.364.002
244.08
.077.02
Pred. .497..0
.98+.0
.511.0
5088.0
.6232.0
c-FID
0.009
0.005
0.096
1.997
0.302
10#
Disc. .018..02
0666.00
.043..03
.11.00
133+.05
Pred. .498..0
.226t.0
.497..0
.55..0
.19+
c-FID
0.094
0.26
0.262
4.833
1.865
25#
Disc. 046+.04
026:.01
.665.00
263+.08
.062.03
Pred. .515t.0
.498..0
.498..0
.5192.0
.511.0
c-FID
0.346
0.04
0.54
3.338
0.176
50#
Disc. .188.02
.017..01
.14.06
226+.07
.44.022
Pred. 503+.0
.974.0
.576.00
.502.0
.47+.00
c-FID 0.066
0.048
0.438
2.968
0.295
Weather
5%
Disc.
.794.01
.499.00
.4162.01
.499.0
.5+.0
Pred. .061:.0
.079+.0
.07+.0
.08+00
.211.011
c-FID
3.679
9.092
7.425
12.823
39.294
10%
Disc.
433+.01
.498..0
.421..0
.482.0
.5+.0
Pred. .06+.0
.07+.0
.07+.0
104+.01
226+.0
c-FID
3.631
13.005
7.19
17.045
30.8
15%
Disc. .11.033
.495.00
.17+.01
.488.00
494+.01
Pred. .059+.0
.067..0
.0692.0
.88+.0
.1572.0
c-FID
10.86
7.214
9.683
22,476
29.475
100%
Disc.
.034+.01
204+.06
.415.00
.499.0
.49+.00
Pred. .055..0
.088t.0
.07+.0
.087+.01
.881+00
c-FID
4.97
7.2
8.871
23.638
41.502
10#
Disc.
.83+.01
.499.00
4924.01
.+.0
.494.0
Pred. .087+.0
.093+.01
.103+.01
246+.01
.022+.0
c-FID 26.481
15.305
15.016
14.954
35.694
25#
Disc.
.99+.0 0
5+.0
.82+.01
.+.0
499+.0
Pred. .068+.0
.07+.0
.082+.0
327+.03
.042..01
c-FID
8.1
13.339
12.78
20.065
22.921
50#
Disc.
.97+.0
.99+.00
.63+.01
.499.0
.49.022
Pred.
.064+.0
.074.0
.076+.0
.77+.04
.026t.0
c-FID
5.999
10.353
14.377
18.868
23.114
38

---

Table 20: Main Table Results- Part The table reports discriminative and predictive scores along with their standard deviations, as well as the contextFID score.
dataset
Subset
Metric
Our[24]
Imagen'Time
DifTS
KoVAE
TimeGAN
mujoco 5%
Disc. .111=.02
.499.00
213+.03
.47.022
.16+.02
Pred.
.04+.0
.065..0
.044..0
.062..0
.083+.0
c-FID
0.249
11.233
0.556
5.231
1.638
10%
Disc.
.074=.02
.991.0
1422.02
447+.03
4122.06
Pred.
.042.0
.062+.0
.04..0
.052+.0
.073+.0
c-FID
0.248
8.841
0.312
4.598
1.3
15%
Disc.
.059..03
.974.0
.21+.02
.34+.03
408+.05
Pred. .038t.0
.063+.0
.099..0
.046..0
.072+.01
c-FID
0.163
8.674
0.252
4.307
1.372
100%
Disc.
0088.01
25+.07
0864.01
.45+.01
.41
Pred. .033+.0
.022..0
.0362.0
.0452.0
.077+.0
c-FID
0.031
0.85
0.16
4.078
1.552
10#
Disc.
38s.02
.99+.0
.39+.1
.96+.0
.388.099
Pred. .09+.0
.666..
1422.02
.1322.01
.109+.0
c-FID
2.25
13.939
4.263
5,856
2.023
25#
Disc.
2944.00
5+.0
.32.02
.491..01
371+.04
Pred. .063+.0
,6:+.0
.052..0
.0872.0
.16+..0
c-FID
2.076
14.019
3.524
6.4
1.891
50#
Disc. 204+.02
4994.0
3+.03
.91+.0
.374.. 14
Pred. .054..0
.065..0
.048..0
.093..0
.129..0
c-FID
0.848
11.878
2.051
5,419
1.251
sine
5%
Disc.
.019+.01
2672.07
.07+.02
.162..04
.492+.0
Pred.
096+:0
.96+.0
.102+.0
.055..0
.264+.0
c-FID
0.051
2.232
0.105
1.597
6.93
10%
Disc.
.114.01
257+.1
.399.001
1192.06
.494+.0
Pred.
.094+.0
0955.0
.098..0
.055..0
267+.01
c-FID
0.048
2.257
0.056
1.827
2.12
15%
Disc. .014+.01
257+.11
.411.01
181+.05
.495+.0
Pred. .094+.0
.096..0
.098t.0
.094+.0
2692.01
c-FID
0.025
2.233
0.038
1.746
3.431
100%
Disc. 0099.00
0088.00
.018.001
.173..04
.49+.0
Pred. .094+.0
.094+.0
.0972.0
.094..0
.66+.001
c-FID
0.005
0.019
0.015
1.619
2.843
10#
Disc.
18s.02
.434.004
311+.01
.36+.04
.499+00
Pred. .12+.0
119+.0
.144.0
.098..0
2662.01
c-FID
0.974
5.662
4.035
2.457
6.733
25#
Disc. .068+.04
.34.1
3144.03
284+.05
.+.0
Pred. .098+.0
.097..0
.172+.05
.1+.0
267+.01
c-FID 0.614
4.356
3.621
2.173
3.636
50#
Disc. .087+001
312+.09
246+.01
.25.088
.499..0
Pred.
.099+.0
.097..0
.113+.0
.096+.0
.66+.01
c-FID
0.292
3,429
1.222
1.851
9.881
39

---

Table 21: Full results on pre-training datasets: conditional (with dataset token) vs. unconditional (without dataset token). Results are reported without any additional fine-tuning on the individual datasets.
Dataset
Metric
Cond.
Uncond.
Blink
contextFII
0.320
0.268
Disc score
0.028
0.050
Chinatown
contextFID 0.157
27.923
Disc score
0.188
0.175
ECG5000
contextFID 0.033
4.920
Disc score
0.016
0.176
EMOPain
contextFID 5.877
5.433
Disc score
0.491
0.492
ETThl
contextFID 0.474
1.309
Disc score
0.159
0.232
ElectrieDevices
contextFID 0.097
2.677
Disc score
0.009
0.119
Exchange
contextFID
0.160
0.353
Disc score
0.014
0.096
FordB
contextFID
0.017
2.448
Disc score
0.008
0.093
MSL.
contextFIID
28.504
29.708
Disc score
0.500
0.500
NondivaiveeetallECCToooxxx
contextFIID
0.009
5.701
Disc score
0.013
0.427
PSM
contextFID 1.468
8.195
Disc score
0.493
0.267
SMAP
contextFID
22.069
29.311
Disc score
0.499
0.423
SMD
contextFIID 17.111
22.482
Disc score
0.499
0.499
SelfRegulationSCCP2
contextFII 0.877
10.772
Disc score
0.058
0.268
SharePrice
contextFID 0.145
2.070
Disc score
0.019
0.163
Trace
contextFID
0.044
48.133
Disc score
0.083
0.215
UWaveGestureLibraay
contextFID
0.118
0.304
Disc score
0.071
0.052
energy
contextFID 14.291
8.080
Disc score
0.500
0.500
stock
contextFID 0.162
0.244
Disc score
0.034
0.051
40


Reviews:
[{"review_id": "jPRLPiSBl5", "reviewer": "Reviewer_g6T2", "summary": "This paper proposes a unified generative modeling framework for time series generation that is pre-trained across a diverse set of datasets spanning multiple domains (e.g., finance, climate, energy, biomedical) and fine-tuned for specific target domains under low-data conditions. It uses a vision-based diffusion model that transforms time series into image representations, incorporates dynamic convolution (DyConv) layers to flexibly handle varying input/output channel sizes, and uses dataset-specific tokens to guide generation. The model achieves strong few-shot performance, supports variable sequence lengths, and shows robustness across model sizes and domains.", "questions": "1. Why was delay embedding chosen over sequence-native encodings for diffusion modeling?\n\n2. What is the impact of the delay embedding parameters (e.g., window size) on generation quality?\n\n3. How robust is the dataset token mechanism when domain boundaries are ambiguous or overlapping?\n\n4. Does the image transformation degrade interpretability of the generated sequences?\n\n5. Could lightweight alternatives to DyConv (e.g., grouped or depthwise convolution) provide similar flexibility with lower cost?\n\n6. How sensitive is performance to the number and diversity of datasets used in pretraining?\n\n7. Are there cases where pretraining leads to negative transfer in fine-tuning?", "limitations": "The authors acknowledge some dataset-specific limitations in performance (e.g., on Weather and ECG200), but a deeper analysis into why the model underperforms on these datasets is missing. Understanding whether the challenges stem from noise, irregular sampling, or lack of temporal structure could provide more actionable insights. Additionally, while the model spans high-stakes domains like finance and biomedical data, the discussion on potential societal risks—such as misuse of synthetic data, overreliance on generated samples, or downstream model degradation due to hallucinations—is minimal. The paper would benefit from explicitly addressing these concerns and proposing guardrails for responsible use.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\nThe unified framework tackles a major limitation of prior generative models by enabling generalization across domains and strong performance in few-shot settings, outperforming state-of-the-art models like ImagenTime and DiffusionTS by large margins. Its use of delay-embedding image transformations leverages advances in vision diffusion models, while DyConv and dataset tokens offer architectural flexibility, efficiency, and improved fidelity across multivariate time series with varying lengths and channels. The comprehensive evaluation across datasets, sequence lengths, and model scales strengthens the validity of its claims.\n\nWeaknesses\nDespite the gains, the approach relies on image-based representations, which may introduce unnecessary inductive biases or miss native temporal dependencies compared to sequence-native architectures. The use of pretraining and fine-tuning still requires nontrivial computational resources and tuning. Additionally, while dataset tokens improve specificity, their utility diminishes without fine-tuning, and performance on some domains (e.g., Weather, ECG200) still lags in limited data settings, suggesting room for better domain adaptation or representation alignment.", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer g6T2 for their thoughtful and constructive review. We appreciate the recognition of our framework’s strengths, including its strong few-shot performance, cross-domain generalization, and architectural choices such as DyConv and dataset tokens. We also value the insightful questions and suggestions regarding interpretability, robustness, and limitations. Below, we address each point in detail and would be happy to incorporate this feedback in a final revision.\n\n**W1 + Q1:**  *the approach relies on image-based representations, which may introduce unnecessary inductive biases or miss native temporal dependencies compared to sequence-native architectures. Why was delay embedding chosen over sequence-native encodings for diffusion modeling?*\n\nOur method extends the ImagenTime framework, which converts time series data into image representations to achieve state-of-the-art results.  While representing time series as images may introduce inductive biases that could restrict the model’s ability to fully capture temporal dynamics, our experiments show that our model consistently delivers the most reliable and highest-quality generations. To further validate this, we conducted additional evaluations following the setup of Experiment 6.1, using sequence-native models like DiffusionTS and KoVAE as pretrained baselines. As illustrated in the table below, both baselines consistently underperform compared to our approach.\n\nTable 1:\n\n|percentage|metric|Our[24]|DiffTS|KoVAE|\n|-|-|-|-|-|\n|**5%**|Disc|**0.11**|_0.244_|0.331|\n||Pred|**0.458**|_0.471_|0.5|\n||cFID|**0.674**|_2.668_|3.719|\n|**10%**|Disc|**0.083**|_0.223_|0.342|\n||Pred|**0.452**|_0.464_|0.491|\n||cFID|**0.578**|_2.432_|3.874|\n|**15%**|Disc|**0.066**|_0.223_|0.338|\n||Pred|**0.451**|_0.466_|0.486|\n||cFID|**1.086**|_2.475_|3.549|\n|**10#**|Disc|**0.259**|_0.347_|0.375|\n||Pred|**0.489**|_0.516_|0.517|\n||cFID|**3.8**|_4.702_|5.522|\n|**25#**|Disc|**0.19**|_0.31_|0.352|\n||Pred|**0.467**|_0.492_|0.499|\n||cFID|**1.582**|_3.817_|3.844|\n|**50#**|Disc|**0.149**|_0.287_|0.354|\n||Pred|**0.46**|_0.48_|0.491|\n||cFID|**0.987**|_3.116_|4.001|\n\n**W2**:  *The use of pretraining and fine-tuning still requires nontrivial computational resources and tuning.*\n\nOur pretraining takes 4 training hours of 2 NVIDIA RTX-4090. This may be converted into a single RTX-4090 with $\\sim 8$ hours of training. Assuming the available pretrain model checkpoint, the finetune process itself takes on average only $\\sim 15$ minutes on a single GPU.\n\n**Q2**:  *What is the impact of the delay embedding parameters (e.g., window size) on generation quality?*\n\nThe original ImagenTime paper offers an in-depth analysis of delay embedding, outlining its advantages and limitations. Importantly, it includes an ablation study showing that delay embedding is robust across different window sizes and generally outperforms alternatives such as STFT.\n\n**Q3**:  *How robust is the dataset token mechanism when domain boundaries are ambiguous or overlapping?*\n\nTo evaluate the robustness of the dataset token in settings with closely related distributions, we pre-trained two versions of the model - one with the dataset token and one without, using only the ETT family (ETTh1/h2, ETTm1/m2).\nWe then evaluated the pre-trained models on the same ETT datasets to assess how well the token helps disambiguate subtle inter-dataset variations.\nResults in the table below show a significate improvement when sampling with dataset token opposed to without even when domain boundaries are ambiguous / overlapping.\n\nTable 2:\n\n|Dataset|Metric|w/ dataset token|w/o dataset token|\n|-|-|-|-|\n|ETTh1|Disc|0.033|0.344|\n||Pred|0.646|0.674|\n||cFID|0.108|5.687|\n|ETTh2|Disc|0.027|0.263|\n||Pred|0.681|0.707|\n||cFID|0.086|3.360|\n|ETTm1|Disc|0.007|0.232|\n||Pred|0.675|0.690|\n||cFID|0.0210|2.920|\n|ETTm2|Disc|0.007|0.236|\n||Pred|0.694|0.718|\n||cFID|0.0249|1.248|\n\n**Q4**:  *Does the image transformation degrade interpretability of the generated sequences?*\n\nNo it does not. In our work, we use an invertible transofmation to convert time series into image-shaped inputs. This means that the process of converting the time series to an image representation and then back to the original signal representation is lossless. This ensurs full interpretability of the generated sequences, inverting the image back to time-series without loosing information. \n\n**Q5**:  *Could lightweight alternatives to DyConv (e.g., grouped or depthwise convolution) provide similar flexibility with lower cost?*\n\nGrouped or depthwise convolutions indeed reduce compute, but they cannot dynamically adapt a single shared kernel across arbitrary input/output channel configurations. Depthwise convolution processes channels independently (groups = input channels), and even combined with pointwise operations it still relies on fixed Cin/Cout sizes. In contrast, DyConv uses bicubic-interpolated resizing of a canonical weight tensor to match any channel dimensions at runtime, which enables seamless generalization across diverse multivariate datasets\n\n**Q6**:  *How sensitive is performance to the number and diversity of datasets used in pretraining?*\n\nThis is an interesting question that we considered during our research, but it remains open and represents an important direction for future work. As it falls outside the scope of our current study, we do not yet have a definitive answer. We thank the reviewer for raising this point and will incorporate a brief discussion in the main paper to highlight it as a valuable open question for future investigation.\n\n**Q7**:  *Are there cases where pretraining leads to negative transfer in fine-tuning?*\n\nOur extensive empirical investigation shows that, in the majority of cases, it does not. In Figure 3A (Experiment 6.2) and Table 2 (Experiment 6.3), we show that the pre-trained model outperforms the non-pretrained model on average across datasets. To complement this, we also evaluated individual datasets. Across all cases, pretraining consistently improves both the Discriminative Score and context-FID, often by a large margin. We only observe a few isolated cases where the Predictive Score is slightly lower. Due to space limitations, we are unable to include the full per-dataset table here, but we will include it in the final revision.\n\n**Limitations 1**:  *The authors acknowledge some dataset-specific limitations in performance (e.g., on Weather and ECG200), but a deeper analysis into why the model underperforms on these datasets is missing ...*\n\nWe made extensive efforts to understand why our model underperforms on datasets like Weather and ECG200, analyzing factors such as similarity to the pretraining distribution using mutual information and FFT-based metrics using correlation analysis with linear and non-linear tools. For example, Weather had an FFT profile close to the pretraining average, while ECG200 was much farther—yet both showed poor generation quality. Overall, these experiments did not yield a clear or consistent explanation. We will include these results in the final revision and plan to explore this further in future work.\n\n**Limitations 2**:  *while the model spans high-stakes domains like finance and biomedical data, the discussion on potential societal risks—such as misuse of synthetic data, overreliance on generated samples, or downstream model degradation due to hallucinations—is minimal*\n\nThis is an important point. We will include a dedicated paragraph on this topic, and address it in the final version as follows:\n\nOur framework advances time-series generation by pre-training a unified model across multiple datasets, promising significant gains in data synthesis, especially in data-scarce scenarios. For example, using our approach, geophysicists can generate more high-fidelity examples of earthquake waveforms in locations with rare seismic events, improving hazard assessment models. In healthcare, researchers could generate synthetic, yet realistic, electronic health records to study disease progression without compromising patient privacy, or augment datasets for training diagnostic models for rare diseases where real-world data is limited. \n\nAt the same time, because the model spans high-stakes domains like finance and biomedical data, its power to generate high-fidelity sequences also heightens significant societal risks. The potential for misuse of synthetic data is substantial; for instance, the same technology that helps medical researchers could be used to create fraudulent clinical trial data, potentially leading to the approval of ineffective treatments. Beyond direct misuse, a critical danger lies in the overreliance on generated samples, as models trained on augmented datasets may become less robust to the noisy, unpredictable nature of real-world data. Furthermore, there is a risk of downstream model degradation due to hallucinations—the generative model may produce data with subtle, unrealistic artifacts that, while plausible-looking, cause subsequent models to fail in unforeseen ways."}, {"review_id": "g97YrjiXsX", "reviewer": "Reviewer_zZSF", "summary": "Proposes a new method for training unconditional time-series generation models by pretraining on heterogeneous time series data and finetuning on data from the target domain. For evaluation, it introduces a benchmark which evaluates different models under varying degrees of data scarcity.", "questions": "1. What dataset is being used to compared with in Table 1? Is the synthetic data trained on 5% of the data compared to 100% of the data, or is it compared to the 5% subset?\n2. What do you think about the originality of the synthetic data itself? Is it copies of the training data, or something more? It could be worthwhile to evaluate this empirically (e.g. using [1]), especially if the answer to the previous questions is that the synthetic data trained on 5% is compared to the 5% subset.\n3. Why is predictive score not presented for the model size study in Table 2?\n4. Could you elaborate a bit on the similarities and differences between DynConv and DyLinear?\n5. How are the datasets divided into several sequences? E.g. the stocks dataset consists of a single sequence.", "limitations": "yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n- The problem is clearly defined and thoroughly motivated\n- The model is compared to a wide range of pre-existing unconditional time-series generation models under varying degrees of data scarcity\n- The pretraining is also demonstrated to allow the model to better generalize across different sequence lengths\n- The ablation study evaluates the impact of two critical components of the approach\n- Makes valuable improvements to the ImagenTime architecture to accomodate multi-domain pretraining:\n\t- Dataset token\n\t- Dynamic convolution to handle different datasets having different number of dimensions per time step\n- Provides new insights into how to make the most out of the limited data available in unconditional time-series generation. Although pretraining techniques are common in other domains,they have not been thoroughly explored within the context of unconditional time-series generation. Ultimately, this might open up for generation of new domains in time-series which were previously inaccessible due to limited data.\n\n**Weaknesses:**\n\n- The paper focuses on relatively short sequence lengths, using sequence length 32 for the main results in Table 2, which limits real world use. This can be compared to the ImagenTime paper, which generates for up to 17k in their longest experiments.\n- Technical details that should be clarified:\n\t- What dataset is being compared to when using c-FID, discrimination score and predictive score in for the various scarcity levels? Is it the scarce variant? Or the complete dataset? Depending on the choice here, different conclusions might be drawn. If the scarce datasets are compared to the synthetic data, there is a risk that the model has learned to memorize the training data and produce copies. This would lead to very good scores, but the samples would just be copies of the original training data. There are evaluation measures that aim to measure the degree of originality of synthetic data, see e.g. [1].\n\t- The predictive score is not presented for model size ablation in Table 2\n\n[1] Alaa, A., Van Breugel, B., Saveliev, E.S., and van der Schaar, M.. (2022). How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:290-306 Available from https://proceedings.mlr.press/v162/alaa22a.html.", "was_revised": true, "final_justification": "The authors have responded to my concerns about evaluation (Q1, Q2, Q3, and Q5) and provided additional details on their methodology (Q4). This paper addresses the critical challenge of data scarcity, which is especially relevant to the field of time-series. If accepted, this can open up a new direction for future time-series generative model research with even more efficient use of the available data.", "rebuttal": "We thank Reviewer zZSF for the thoughtful and detailed review. We appreciate the recognition of our contributions, including the multi-domain pretraining setup, architectural improvements to ImagenTime, and the proposed benchmark for data-scarce generation. We also value the reviewer’s insightful questions regarding evaluation protocol, model behavior, and design decisions, which helped us clarify and strengthen the paper. Below, we address each point and would be glad to incorporate this feedback into a final revision.\n\n**W1**:  *The paper focuses on relatively short sequence lengths, using sequence length 32 for the main results in Table 2, which limits real world use. This can be compared to the ImagenTime paper, which generates for up to 17k in their longest experiments*\n\nThis is a good suggestion and we agree that extending time-series generation to long sequences is a crucial challenge for many real-world applications. Our primary goal was to investigate performance across diverse generative paradigms (GAN, VAE, Diffusion) in data-scarce settings. To ensure a fair and broad comparison, we chose a sequence length compatible with all baseline architectures, as models like TimeGAN, KoVAE, and DiffusionTS were not originally designed for long sequences and would require significant modifications. Focusing only on long sequences would have narrowed our analysis to a single method (i.e., ImagenTime), preventing the cross-paradigm study we intended. We agree that exploring our unified framework for long-sequence generation is a key direction for future work and we will add a short discussion about it in the new revision.\n\n**W2.1**: *What dataset is being compared to when using c-FID, discrimination score and predictive score in for the various scarcity levels? Is it the scarce variant?*\n\n *What dataset is being used to compared with in Table 1? Is the synthetic data trained on 5% of the data compared to 100% of the data, or is it compared to the 5% subset?*\n\n *What do you think about the originality of the synthetic data itself? is it copies of the training data, or something more*\n\nThe training dataset is a subset of the original data - depending on the setup (i.e., #10,..,5%,..), while the test dataset always comprises the full original dataset (i.e., 100\\%). We will revise the paper to make this distinction clearer. We appreciate your concern regarding potential memorization; however, given the model’s limited access to the data - especially relative to the full dataset, we believe the risk of memorization is minimal. To further support this point, we will include t-SNE and PCA visualizations of some of the fine-tuned representations in the appendix. In summary, the qualitative t-SNE/PCA results show that the generated distributions closely overlap with the input distributions while still exhibiting meaningful variability. Additionally, we plan to discuss [1] in the final manuscript to highlight the importance of assessing the originality of synthetic data.\n\n**W2.2**: *The predictive score is not presented for model size ablation in Table 2*\n\nDue to the NeurIPS format and 9-page limit, we were unable to include all metrics in Table 2 of the main paper, and reported it in the appendix. The full results, including Discriminative Score, Predictive Score, and context-FID, are shown in the complete table below. We will include the full version of this table in the final paper.\n\nTable 3:\n\n|Metric| |Base|Medium|Large|XL|XL→S Det.|\n|-|-|-|-|-|-|-|\n|Disc|FT|0.15| 0.16| 0.13| 0.13|-14.02%|\n||w/o PT| 0.35| 0.35 |0.26|0.27|-23.29%|\n| cFID| FT |4.85| 5.03|4.56|4.51|-7.37%|\n||w/o PT |15.87 |15.36|8.12|8.15|-48.62% |\n|Pred| FT | 0.4993|0.4974|0.4959|0.4965|**-0.57%**|\n||w/o PT|0.5043|0.5036|0.5007|0.5023|**-0.40%**|\n\n**Q4**: *Could you elaborate a bit on the similarities and differences between DynConv and DyLinear?*\n\nWe believe the motivation behind DyLinear was to enable a single model to support time series of multiple lengths. The main challenge arises from the fact that model instantiation occurs during pretraining, at which point we have no prior knowledge of the fine-tuning dataset, specifically, its shape. In other words, we must initialize and train model weights without knowing the final input dimensions.\n\nDyLinear addresses this cleverly by interpolating between canonical weights and the target shape required for the linear operation. Once interpolated, the layer performs a standard linear transformation using the adapted weights and the input.\n\nWe extend this idea to the convolutional setting: we define a single canonical convolution kernel and apply interpolation to reshape it according to the target dimensions. The resulting kernel is then used in the standard convolution operation.\n\n**Q5**: *How are the datasets divided into several sequences?*\n\nA major advantage of time series generation is that data from diverse tasks can often be repurposed for generative modeling. However, instead of applying a one-size-fits-all rule, we adopted dataset-specific preprocessing strategies (e.g., as done in TimeGAN and UCR/UEA benchmarks) to remain faithful to prior work and ensure reproducibility.\n\nFor all datasets, we used the original preprocessing protocols defined in their respective benchmarks. For instance, the Stocks dataset from TimeGAN was segmented using the same sliding window method as in the original implementation."}, {"review_id": "MrCSBbhGTV", "reviewer": "Reviewer_WSzz", "summary": "This paper is motivated by a key observation: even state-of-the-art generative models tend to suffer significant performance degradation in low-resource settings.\n\nTo address this challenge, the paper proposes a diffusion-based generative framework trained with a unified strategy, aiming to learn generalizable temporal representations that can bridge this performance gap. Specifically, it introduces two core components: (i) a dynamic convolution layer (DyConv) to handle channel mismatches across heterogeneous time series datasets during large-scale pretraining, and (ii) dataset label conditioning to guide the sampling process.\n\nThe paper underscores the importance of few-shot generative modeling in time series research and provides a step toward addressing this underexplored challenge.", "questions": "## Questions\nBaseline selection: The paper compares against ImagenTime, DiffTS, KoVAE, and TimeGAN. While these baselines span a broad range of generative paradigms, some—like KoVAE and TimeGAN—are relatively classical. It's unclear whether these baselines represent current state-of-the-art performance.\n\nClarity issues remain unresolved, especially those mentioned earlier regarding model explanation.\n\nDataset label conditioning ablation: The paper states that models trained without dataset label guidance experience significant performance drops if not fine-tuned. However, it's unclear where this result is reported. If a corresponding table exists, please point to it explicitly.\n\nMetric averaging ambiguity: In Section 6.1 on few-shot benchmarks, Table 1 reports the average values of Discriminator Score (Disc.), Predictor Score (Pred.), and context-based FID (c-FID). However, it is unclear whether these averages refer to the mean across different domains or the mean of multiple runs for each metric. This ambiguity could lead to misinterpretation.\n\nTerminology inconsistencies: For instance, describing the model’s ability to process variable-length time series using dynamic masks as \"variable-length time series generation\" is somewhat imprecise. Similarly, describing the experiment in Section 6.2—on pretraining with different sequence lengths—as “cross-resolution transfer” may be inaccurate.", "limitations": "yes", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "## Quality\nThe paper presents a series of designs to adapt the ImagenTime model for large-scale unified generative pretraining. In particular, DyConv is proposed to address the issue of misaligned channels across multi-domain heterogeneous time series data, and a masking mechanism is introduced to handle variable-length time series converted into images. These design choices are generally reasonable.\n\nThe ablation study for DyConv is fair. The authors remove the DyConv module and replace it with static channel padding as a baseline. To ensure fair comparison, a masking mechanism is incorporated into the loss function so that the denoising network is only penalized on valid (unpadded) dimensions. This improves the credibility of the ablation results.\n\n## Clarity\nThe explanation of the loss function (Equation 1) is somewhat unclear, particularly regarding how the weight function $\\lambda(\\sigma)$ is defined.\n\nThe model introduces a dynamic binary mask applied to the padded input sequence, which is adjusted at runtime to distinguish between valid and padded time steps. While this mask design is reasonable for handling variable-length time series converted into images, the paper fails to explain how this dynamic adjustment is implemented.\n\nThe section on dataset label conditioning states that the label is mapped to a learnable embedding and injected into the denoising network via Adaptive Group Normalization (AdaGN) [17] during training and generation. However, the implementation details are vague—this part would benefit from a formula or schematic illustration.\n\n## Significance\nThe paper clearly highlights an important issue: although generative modeling is a promising approach to mitigating data scarcity, existing models are typically designed and evaluated under the assumption of abundant training data. This makes the paper relevant to the time series generation community.\n\n## Originality\nThe proposed framework builds upon ImagenTime, a 2024 time series generative model that transforms time series into image-shaped inputs to leverage diffusion models from the vision domain. While the paper justifies using ImagenTime as the backbone and introduces DyConv and dataset label conditioning for transferability, the overall novelty remains somewhat limited.", "was_revised": true, "final_justification": "Thank you very much to the authors for the clarifications provided on several issues. However, I still find some of the key conclusions of the paper unconvincing or not particularly compelling.\n\n1. The framework builds upon ImagenTime (2024) and improves transferability through DyConv and label conditioning. Nevertheless, the overall level of innovation remains limited.\n\n2. I appreciate the authors' supplementary experiments and clarifications regarding the baselines. The inclusion of TimeVAE helps broaden the benchmark coverage to some extent. However, this addition does not fully address my previous concern: although the selected baselines span different paradigms, some are outdated and no longer representative of the current state-of-the-art. In particular, TimeVAE, introduced at ICLR 2022, is itself relatively dated. Thus, It is very hard to validate the effectiveness of your proposal. \n\n3. Furthermore, since the proposed method is based on a diffusion model framework, incorporating more recent diffusion-based time series generation models as baselines would make the evaluation more convincing. Including such models would allow for a more accurate comparison of performance and a clearer understanding of the contribution to the rapidly evolving field of time series generation.\n\n4. The evaluation process in TSGBench (VLDB 2024) has become fairly standardized, with 29 citations, which is relatively high in the time series generation community and reflects its recognition for benchmarking purposes. The current evaluation in this paper lacks several key metrics (e.g., MDD, ACD, SD, KD, ED, DTW), making it difficult to comprehensively assess the model’s generalization and performance. We encourage the authors to incorporate these metrics in future revisions to enhance the thoroughness of the evaluation.\n\n4. Regarding the previous concern about the ambiguity of average metrics, I thank the authors for their clear and detailed clarification. Although the appendix includes the complete results for each dataset and subset size, reporting the average performance across all domains as the overall result in the main text still lacks justification.\n\n5. I appreciate the authors’ thorough responses to all concerns. The clarifications in the rebuttal are likely to improve the manuscript’s readability. However, I still feel that the overall contribution to NeurIPS is somewhat limited. The paper essentially applies the ImagenTime model from NeurIPS 2024 to the task of general time series generation with pretraining, and introduces DyConv and label conditioning to enable transferability. In my view, this mainly constitutes a combination of existing methods.\n\nGiven the issues outlined above, I believe the paper still requires substantial revisions. Therefore, I will maintain my original score.", "rebuttal": "We thank Reviewer WSzz for the thoughtful and detailed review. We appreciate the recognition of our motivation, as well as the design contributions such as DyConv and dataset label conditioning. We found the questions and suggestions - particularly those regarding implementation clarity, evaluation protocol, and terminology, very helpful. These insights have allowed us to refine our explanations and improve the presentation. Below, we address each point in detail and will revise the paper to incorporate the reviewer’s feedback.\n\n**W1**: *The explanation of the loss function (Equation 1) is somewhat unclear*\n    \nWe are happy to provide more detail. Generally, our approach to defining the loss function for our diffusion process follows the methodologies presented in EDM [1] and ImagenTime [2].\n$$\n\\mathcal{L} = \\mathbb{E} _ {\\sigma,x _ \\text{img}^0,\\varepsilon} [ \\lambda(\\sigma) \\left\\| {\\Gamma_\\theta (x_\\text{img}^\\sigma;\\sigma)-x _ \\text{img}^0} \\right\\| _ 2^2 ] \\ ,\n$$\nwhere $\\lambda(\\sigma)$ is a weighting function and $\\Gamma_\\theta$ is defined as follows:\n$$\n\\Gamma _ \\theta (x _ \\text{img}^\\sigma; \\sigma) = c_{\\text{skip}}(\\sigma)x_\\text{img}^\\sigma + c_\\text{out}(\\sigma) N_{\\theta}(c_\\text{in}(\\sigma)x_\\text{img}^\\sigma;c_{\\text{noise}}(\\sigma);y) \\ .\n$$\nIn this equation, $N_{\\theta}$ represents the denoiser neural network. The terms $c_\\text{skip}(\\sigma)$, $c_\\text{in}(\\sigma)$, $c_\\text{out}(\\sigma)$, and $c_\\text{noise}(\\sigma)$ are all functions of $\\sigma$. \n\n- $c_\\text{skip}(\\sigma) = \\frac{\\sigma_\\text{data}^2}{\\sigma^2 + \\sigma_\\text{data}^2}$\n- $c_\\text{out}(\\sigma) = \\frac{\\sigma \\cdot \\sigma_\\text{data}^2}{\\sqrt{\\sigma^2 + \\sigma_\\text{data}^2}}$\n- $c_\\text{in}(\\sigma) = \\frac{1}{\\sqrt{\\sigma^2 + \\sigma_\\text{data}^2}}$\n- $c_\\text{noise}(\\sigma) = \\frac{1}{4} \\ln(\\sigma)$\n\nWe use $\\sigma_\\text{data} = 0.5$ as a fixed parameter, and $\\sigma(t)$ controls the noise level over time. These terms help stabilize the loss, which can vary with $\\sigma$.\n\nSubstituting Eq.\\~2 into Eq.\\~1 yields a per-sample loss weight. To balance this, EDM sets $\\lambda(\\sigma) = 1/c_\\text{out}(\\sigma)^2$, also ensuring uniform initial loss across the $\\sigma$ range.\n\nThough our focus wasn’t on loss design, we agree this clarification is helpful and will include it in the revised paper.\n\n**W2**: *The model introduces a dynamic binary mask applied to the padded input sequence, ... the paper fails to explain how this dynamic adjustment is implemented*\n\nWe convert time series into images using delay embedding and zero-padding. To identify valid (non-padded) regions, we apply the same transformation to an all-ones tensor, creating a binary mask.\n\nDuring training, noise is added to the full image, but loss is computed only over valid pixels.\n\n“Dynamic adjustment” refers to how the valid region (and thus the mask) changes with input length, though image size remains fixed. Each dataset has a consistent length, but lengths differ across datasets. Section 6.2 shows how our model handles length mismatches during pretraining and fine-tuning. We will clarify this in the text and release code.\n    \n**W3**: *The section on dataset label conditioning states that the label is mapped to a learnable embedding and injected into the denoising network via Adaptive Group Normalization. However, the implementation details are vague*\n    \nWe thank the reviewer for pointing this out. The Adaptive Group Normalization (AdaGN) implementation is as follows: $\\text{AdaGN}(h,y) = y_s~\\text{GroupNorm}(h) + y_b$ where $h$ is the intermediate activations of the residual block following the first convolution, and $y=[y_s,y_b]$ is obatined from a linear projection  of the timestep and the data token embedding. \n\nIn our UNet architecture, AdaGN is applied within each block, allowing the model to condition its activations on both the timestep and dataset identity. For clarity, we include a simplified UNet block code snippet below highlighting this integration. We will expand on this explanation—including the formula—in the revised version and will release the code publicly.\n\n``` python\nclass UNetBlock(torch.nn.Module):\n    ...  \n    def forward(self, x, emb):\n        orig = x\n        x = self.conv0(silu(self.norm0(x)))\n\n        # Project the embedding (timestep + data token)\n        params = self.affine(emb)\n        if self.adaptive_scale:\n            # Split parameters into scale and shift\n            scale, shift = params.chunk(chunks=2, dim=1)\n            # Apply AdaGN: scale * GroupNorm(x) + shift\n            x = silu(torch.addcmul(shift, self.norm1(x), scale + 1))\n        else:\n            # Fallback for non-adaptive scaling\n            x = silu(self.norm1(x.add_(params)))\n    ...\n```\n**W4**: *While the paper justifies using ImagenTime as the backbone and introduces DyConv and dataset label conditioning for transferability, the overall novelty remains\nsomewhat limited*\n\nWe'd like to clarify and emphasize the innovative aspects of our work. Our core contribution lies in establishing the first robust baseline for time series generation in data-scarce environments.\n\nWhile we build on ImagenTime’s idea of converting time series into image-like inputs for diffusion models, our key innovation—shown in Figure 2—is a novel design not previously explored in time series generation. By bridging research on unified training with this domain, our approach offers a strong, versatile framework that outperforms existing methods, especially in low-data settings.\n\nWe acknowledge that components like diffusion models and time series-to-image transforms aren't new, nor is the use of unified training. However, applying this paradigm to time series generation—with varying channel sizes and diverse training datasets—introduces unique challenges that our method addresses effectively. Interestingly, our unified training paradigm for time series enables analyses that are not possible in other settings, adding another novel dimension to our contribution.\n    \n**Q1**: *Baseline selection: The paper compares ... While these baselines span a broad range of generative paradigms, some-like KoVAE and TimeGAN-are relatively classical...*\n\nOur chosen baselines represent a robust and state-of-the-art set of generative models for time series, drawing primarily from recent, top-tier conferences in the field:\n\n- **ImagenTime** published at NeurIPS 2024.\n- **DiffusionTS (DiffTS)** published at ICLR 2024.\n- **KoVAE**, published at ICLR 2024.\n- **TimeGAN**, while more classical, remains a widely-used benchmark for time series generation. We included TimeGAN to span diverse generative paradigms, encompassing GAN-, VAE-, and diffusion-based models.\n\nAdditionally, we have incorporated **TimeVAE** into our evaluation for completeness. Due to limited reponse space and alignment with other reviewers, please refer to table 5 in reviewer 2Ztk response .We will include this updated table in the final version of the paper. We note our model outperform TimeVAE on all metrics and setups. We appreciate the feedback and the opportunity to clarify this point, as well as to strengthen our scarce data comparisons and benchmarking. We would be happy to address any further suggestions.\n\n**Q2**: *Dataset label conditioning ablation: The paper states that models trained without dataset label guidance experience significant performance drops if not fine-tuned. However, it’s unclear where this result is reported*\n\nThis is a subtle but important point that we will clarify in the main text.\n\nWe find that removing the dataset token during pre-training leads to a noticeable drop in evaluation metrics on the pre-training datasets. Specifically, without the token, the model achieves an average Discriminative Score of 0.252 and context-FID of 11.07. In contrast, with the token, scores improve to 0.193 and 4.84, respectively. Full results, including per-dataset comparisons, are shown in Table 18 (Appendix C.4). This demonstrates the token's key role in guiding sampling across heterogeneous datasets during pre-training.\n\nHowever, as seen in Table 4, the token is not needed during fine-tuning. Once the model is adapted to a specific dataset, the token’s original purpose fades. Still, it remains crucial when generating from mixed or unseen data distributions.\n\nWe will revise the paper to make this distinction clear and avoid confusion.\n\n**Q3**: *Metric averaging ambiguity: In Section 6.1 on few-shot benchmarks ... However, it is unclear whether these averages refer to the mean across different domains or the mean of multiple runs for each metric*\n\nWe agree that the averaging procedure should have been clarified more explicitly. The reported scores in Table 1 are averaged across all evaluation datasets for each subset size. For example, in the 100\\% row, each model's score reflects the average performance across all domains under the full-data setting. Similarly, in the 10\\% row, the reported scores are the averages across all domains under the 10\\%-data setting. For a complete picture, we've included the full, per-dataset, per-subset size results in the appendix.\n\nWe will incorporate this clarification into the paper.\n\n**Q4**: *Terminology inconsistencies: For instance, describing the model’s ability to process variable-length time series using dynamic masks as \"variable-length time series generation\" is somewhat imprecise. Similarly, describing the experiment in Section 6.2—on pretraining with different sequence lengths—as “cross-resolution transfer” may be inaccurate.*\n\nWe acknowledge that both terms may be unclear and imprecise. We will revise them to \"cross-temporal resolution transfer learning,\" which more accurately describes how our model transfers knowledge across different temporal resolutions - training on one and fine-tuning on another, as demonstrated in Section 6.2. This change will be reflected in the revised paper to improve clarity. Finally, we will use time series generation of variable-length inputs instead."}, {"review_id": "YAeBU0rWXd", "reviewer": "Reviewer_2Ztk", "summary": "This paper introduces a time series generative model based on ImagenTime. To accommodate flexible number of channels, the authors introduced dynamic convolution and learnable data token as signals for labeling data domain. The model demonstrated particular advantage in data scarcity settings. At the same time, it claims to be the first pretrained time series generative model.", "questions": "line 216: each dataset is assigned a unique token that acts as an identifier of its source domain. How is this token assigned? Is it randomly chosen? Is it the domain name converted to word embeddings? There is no information on this. \n\n\nThere is an inconsistency in the metrics reported. some experiments/analysis use three metrics: discriminative, predictive, and contextFID while some experiments/analysis only reports discriminative and contextFID. why is that?", "limitations": "The authors only mentioned one line at the end of section 6.1 saying the results on a few datasets like weather and ECG are challenging. But nothing is discussed about addressing the mentioned part. It could be due to a lack of datasets from the same domain in the pretraining set. Some analysis on similarity of pretraining and few shot dataset may help you uncover the reason. Additionally, some visualization on latent representation of learned dataset token during finetuning overlayed with dataset token specified during pretraining could be interesting.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength:\nThe modifications upon ImagenTime are intuitive. \nThe paper is easy to follow\n\nWeakness:\nThe ablation study on dataset token did not show the advantage of dataset token. The effect of dataset token is not convincing and hard to interpret. Does that mean as long as dataset token is used in pretraining, it doesn't matter if it is used during finetuning? And are the benchmark dataset all from the domains seen during the pretraining? If not, do the dataset that come from an unseen domain show a different behavior when you look at the effect of dataset token? The experiment mentioned in line 345-246 is not shown in table 4. This is probably not the right way to report results. \n\nI don't think Sine is a good dataset to include for evaluation, it might make more sense to be in the pretraining dataset. There is no distribution change for the sine dataset, so sine dataset does not pose any challenge in data-scarcity setting. The ETT datasets are also highly seasonal and probably not hard to learn in few shot settings. But stock dataset are hard but it was used in the pretraining dataset instead and there is no dataset from the financial domain in the evaluation. \n\n\n\nIt's probably not suitable to claim a systematic benchmark as a main contribution as there is only a few models compared. For example, TimeVAE which is more recent than TimeGAN is not compared. And data-scarcity setting might not be so overlooked as I remember there is such setting in the TimeVAE paper. \n\nIn figure 1A, it's only clear that your model performs better than baselines. But if your purpose it to show that the gap between #10 training samples and 100% training set for your model is smaller than baseline (drawn as x2.15), I don't think this goal is achieved. It's not obvious that the scaling behavior of your model is better than the orange and green line. You might reconsider what message should come cross.", "was_revised": true, "final_justification": "The authors provided additional exp on the challenging financial dataset evaluation where the corresponding dataset is removed from pretraining and still showed positive results. The authors also emphasized setting a standard for evaluation protocol.", "rebuttal": "We thank Reviewer 2Ztk for the detailed and constructive review. We appreciate the recognition of our motivation and design choices, as well as the thoughtful questions and critiques regarding the dataset token, evaluation setup, benchmark scope, and clarity of presentation. These comments help identify areas where further explanation and refinement are needed. Below, we address each point and will revise the paper to improve clarity and better support our contributions.\n\n**W1**: *The ablation study on dataset token did not show the advantage of dataset token. The effect of dataset token is not convincing and hard to interpret. Does that mean as long as dataset token is used in pretraining, it doesn't matter if it is used during finetuning?*\n\nThanks for highlighting this - we see that the motivation may not have been clearly conveyed.\n\nThe dataset token addresses two main purposes. **Evaluate the pretrain** – By allowing the model to learn and differentiate between the multiple data sources, we can now reliably sample from a specific distribution during pretraining and evaluate the quality of pretraining. **Use an invested resource** – The pretraining phase is already complete, and the dataset token allows users to leverage this investment by sampling from the learned distribution.\n\nThis is especially relevant in mixed distribution sampling. In fine-tuning, which involves a single distribution, the token holds little distributional value. However, we retain a task-specific token for architectural consistency. We’ll revise the manuscript to clarify this - thank you for your feedback.\n\n**W2**: *And are the benchmark dataset all from the domains seen during the pretraining?*\n\nOur evaluation includes both domains seen during pretraining and entirely new ones. For example, ETTm2 (fine-tuning) and ETTh1 (pretraining) both come from the electricity domain. In contrast, Mujoco (physics) and AirQuality (nature) are used only during fine-tuning, with no domain coverage in pretraining.\n\n**W3**: *If not, do the dataset that come from an unseen domain show a different behavior when you look at the effect of dataset token?*\n\nThe dataset token has little effect on these datasets in the context of fine-tuning, which involves tuning the networks weights towards this distribution.\n\n**W4**: *The experiment mentioned in line 345-346 is not shown in table 4*\n\nThe results in Lines 345–346 refer to the average performance across the pretraining datasets for the model without dataset token guidance, evaluated with 100% of the data and before fine-tuning. These are not shown in Table 4, which reports only fine-tuned downstream results.\n\nThe full per-dataset results are provided in Table 18 (Appendix C.4). The reported values-0.252 (Disc.) and 11.07 (context-FID) without guidance, vs. 0.193 and 4.84 with guidance, are averages across the datasets in that table. We will revise the text to clarify this and include a direct reference to Table 18.\n\n**W5**: *I don’t think Sine is a good dataset to include for evaluation ... there is no distribution change for the sine dataset, so sine dataset does not pose any challenge in data-scarcity setting.*\n\n\nWe included the synthetic sine dataset as a controlled benchmark to assess few-shot generation under low-noise, structured dynamics. While not reflective of real-world complexity, it serves as a sanity check for generalizing periodic behavior, as done in prior work (e.g., TimeGAN, ImagenTime).\n    Despite the lack of distribution shift, all models show a clear performance drop from 100% to limited data settings (Table 17), highlighting the challenge of few-shot generation even on simple signals.\n\n**W6**: *The ETT datasets are also highly seasonal and probably not hard to learn in few shot settings. But stock dataset are hard but it was used in the pretraining dataset instead and there is no dataset from the financial domain in the evaluation.*\n\nThank you for raising this valid concern. The inclusion of the Stocks (and Energy) dataset in pretraining was intentional—we aimed to expose the model to data from variety of tasks, including time series generation, with Stocks (and Energy) serving as representative examples. To directly address your concern, we conducted a new experiment in which both the Stocks and Exchange datasets were removed from the pretraining corpus—effectively eliminating all financial data from pretraining. These datasets were then included to the fine-tune group, and the benchmark was re-run accordingly.\n\nThe following tables show results for just Stocks and Exchanges dataset. Due to limited response space we comapred with second best model only. We observe that our model outperforms ImagenTime on all metrics on all setups on all datasets, even though no financial data was available in the pretrain phase.\n\nTable 4:\nStocks:\n|%|metric|Our[24]|ImagenTime|\n|-|-|-|-|\n|5%|Disc|0.018|0.204|\n||Pred|0.037|0.040|\n||cFID|0.132|0.846|\n|10%|Disc|0.017|0.306|\n||Pred|0.037|0.040|\n||cFID|0.090|0.742|\n|15%|Disc|0.018|0.316|\n||Pred|0.037|0.039|\n||cFID|0.054|0.691|\n|10#|Disc|0.138|0.275|\n||Pred|0.041|0.049|\n||cFID|0.379|3.267|\n|25#|Disc|0.119|0.206|\n||Pred|0.038|0.046|\n||cFID|0.430|2.616|\n|50#|Disc|0.054|0.247|\n||Pred|0.037|0.042|\n||cFID|0.183|1.845|\n\nExchange:\n|%|metric|Our[24]|ImagenTime|\n|-|-|-|-|\n|5%|Disc|0.046|0.470|\n||Pred|0.522|0.564|\n||cFID|0.137|3.552|\n|10%|Disc|0.036|0.47|\n||Pred|0.520|0.564|\n||cFID|0.119|3.366|\n|15%|Disc|0.044|0.473|\n||Pred|0.522|0.559|\n||cFID|0.124|3.285|\n|10#|Disc|0.352|0.496|\n||Pred|0.606|0.611|\n||cFID|2.574|8.13|\n|25#|Disc|0.279|0.476|\n||Pred|0.604|0.599|\n||cFID|2.222|5.901|\n|50#|Disc|0.178|0.491|\n||Pred|0.564|0.578|\n||cFID|0.833|4.639|\n\n**W7**: *It’s probably not suitable to claim a systematic benchmark as a main contribution as there is only a few models compared. For example, TimeVAE which is more recent than TimeGAN is not compared.*\nThank you for the comment. Our goal is to support the standardization of evaluation protocols for few-shot time series generation. Given the wide range of generative models, we had to be selective due to computational and evaluation constraints. We chose methods based on two main criteria: (1) state-of-the-art performance and (2) architectural diversity. Accordingly, we included KoVAE (ICLR 2024, VAE-based), DiffTS and ImagenTime (ICLR/NeurIPS 2024, diffusion-based), and TimeGAN, a widely used GAN baseline. We would be happy to include additional models to further strengthen the benchmark’s scope and impact.\n\n\nFollowing your suggestion, we have now incorporated TimeVAE into our evaluation. The results are shown in the table below and will be included in the final version of the paper. We note our model out perform TimeVAE on all metrics and setups\n\nTable 5:\n\n|%|metric|Our[24]|TimeVAE|\n|-|-|-|-|\n|5%|Disc|0.11|0.239| \n||Pred|0.458|0.467|\n||cFID|0.674|1.747|\n|10%|Disc|0.083|0.232|\n||Pred|0.452|0.466|\n||cFID|0.578|1.692|\n|15%|Disc|0.066|0.215|\n||Pred|0.451|0.465|\n||cFID|1.086|1.303|\n|10#|Disc|0.259|0.312|\n||Pred|0.489|0.51|\n||cFID|3.8|2.838|\n|25#|Disc|0.19|0.26|\n||Pred|0.467|0.473|\n||cFID|1.582|1.874|\n|50#|Disc|0.149|0.239|\n||Pred|0.46|0.475|\n||cFID|0.987|1.75|\n\n**W8**: *In figure 1A, it's only clear that your model performs better than baselines. But if your purpose it to show that the gap between \\#10 training samples and 100\\% training set for your model is smaller than baseline (drawn as x2.15), I don't think this goal is achieved.*\n\nThank you for the constructive feedback. The figure aimed to highlight the performance gap between the 100% and \\#10 settings to support the benchmark’s validity, rather than emphasize our model’s superiority. We acknowledge that the 100%–\\#10 gain for our model compared to other is unclear and that the \\#50–100% jump may better illustrate this point. We will revise the figure and caption accordingly.\n\n**Q1**: *each dataset is assigned a unique token that acts as an identifier of its source domain. How is this token assigned?*\n\nEach dataset is assigned a unique identifier that serves as its token. The token is mapped to a learnable embedding, trained jointly with the model, and injected into the denoising network via AdaGN to condition generation on dataset identity.\n\n**Q2**: *There is an inconsistency in the metrics reported ...*\n\nDue to the NeurIPS format and 9-page limit, we were unable to include all metrics in Table 2 of the main paper and some metrics where moved to the appendix. For the full results, including Discriminative Score, Predictive Score, and context-FID, we refer to table 3 in reviewer zZSF respons (due to limited reponse space). We will include the full version of this table in the final paper. \n\n**Limitations 1**: *the results on a few datasets like weather and ECG are challenging. But nothing is discussed about addressing the mentioned part. It could be due to a lack of datasets from the same domain in the pretraining set.*\n\n\nThank you for raising this important point—it has been a concern for us as well. We examined the link between underperformance and pretraining coverage using mutual information and FFT alignment. While Weather aligned well with the pretraining average and ECG200 did not, both showed poor results, revealing no clear pattern. Notably, ECG200 performed poorly despite ECG5000 (also ECG) being part of pretraining, suggesting domain similarity alone isn’t sufficient. Understanding these discrepancies remains an open and critical challenge, which we plan to investigate further and briefly discuss in the main paper.\n\n**Limitations 2**: *Additionally, some visualization on latent ... pretraining could be interesting*\n\nVisualizing the dataset tokens can indeed provide useful insights. We will include t-SNE and PCA plots in the paper, which show clear separation between tokens with noticeable margins. Some tokens—like those from Stocks and Exchange—reflect some domain similarity. We will include these results in the final version."}]

Author rebuttal:

Meta-review: This paper introduces a unified generative modeling framework for time series generation, leveraging a vision-based diffusion model with DyConv layers and dataset-specific tokens to support diverse domains, variable sequence lengths, and robust few-shot performance. 

The framework addresses the critical challenge of data scarcity, demonstrating strong results across finance, climate, energy, and biomedical datasets. 

While some dataset-specific limitations remain, the authors have clarified methodology and evaluation details during the rebuttal. Overall, this is a well-motivated and timely contribution that can open new directions for time-series generative modeling research. I recommend acceptance.
Decision: Accept (poster)