Title: Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Time Series

Synthetic Data Generation

LLMs
Abstract: SDForger is a flexible and efficient framework for generating high-quality multivariate time series using LLMs.
Leveraging a compact data representation, SDForger provides synthetic time series generation from a few samples and low-computation fine-tuning of any autoregressive LLM. Specifically, the framework transforms univariate and multivariate signals into tabular embeddings, which are then encoded into text and used to fine-tune the LLM.
At inference, new textual embeddings are sampled and decoded into synthetic time series that retain the original data's statistical properties and temporal dynamics. Across a diverse range of datasets, SDForger outperforms existing generative models in many scenarios, both in similarity-based evaluations and downstream forecasting tasks. By enabling textual conditioning in the generation process, SDForger paves the way for multimodal modeling and the streamlined integration of time series with textual information. The model is open-sourced at https://github.com/IBM/fms-dgt/tree/main/fms_dgt/public/databuilders/time_series.

Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation

Abstract

SDForger is a flexible and efficient framework for generating high-quality multivariate time series using LLMs. Leveraging a compact data representation, SDForger provides synthetic time series generation from a few samples and low-computation fine-tuning of any autoregressive LLM. Specifically, the framework transforms univariate and multivariate signals into tabular embeddings, which are then encoded into text and used to fine-tune the LLM. At inference, new textual embeddings are sampled and decoded into synthetic time series that retain the original data's statistical properties and temporal dynamics. Across a diverse range of datasets, SDForger outperforms existing generative models in many scenarios, both in similarity-based evaluations and downstream forecasting tasks. By enabling textual conditioning in the generation process, SDForger paves the way for multimodal modeling and the streamlined integration of time series with textual information. The model is open-sourced at https://github.com/IBM/fms-dgt/tree/main/fms_dgt/ public/databuilders/time_series .

1 Introduction

In the era of foundation models, integrating time-series analysis with language models (LLMs) has emerged as a key research priority, spurring work on specialized models for forecasting, anomaly detection and root cause analysis (Liang et al., 2024). These efforts aim to harness the representational power of LLMs to tackle complex temporal dependencies. Despite these advancements, foundation models for time series still struggle with concept drift, high variability, and face performance degradation over long horizons. In addition, the scarcity of large and diverse time-series datasets often limits model generalization, especially in domains where data collection is expensive or operationally constrained, such as climate science, finance, or plasma physics (Kit et al., 2024).

In this context, synthetic data generation has emerged as a complementary research direction to improve scalability and model performance, particularly in low-quality or data-scarce settings. In particular, synthetic data can be leveraged to fine-tune machine learning models on domain-specific distributions, reducing the need to train models from scratch and improving sample efficiency. Indeed,

several generative models for time series have been proposed, including V AE-based architectures (Desai et al., 2021; Lee et al., 2023), GANs (Pei et al., 2021; Kidger et al., 2021), and diffusion-based approaches (Zhou et al., 2023). While these methods have shown promise, they typically lack pretraining, require task-specific retraining, and often struggle with long-term dependencies, multivariate coupling, and distributional shifts (Ang et al., 2023).

Recent studies have instead explored the use of LLMs for synthetic tabular data generation (Padhi et al., 2021; Borisov et al., 2022), demonstrating that language models trained on text-encoded data can capture statistical relationships and feature dependencies effectively. However, extending these methods to time series is non-trivial: long temporal windows increase inference and training costs, while temporal and multivariate correlations require more structured modeling.

These limitations highlight the need for novel methodologies that adapt LLMs for time-series generation while addressing temporal, structural, and computational challenges. We introduce SDForger (Synthetic Data Forger), a novel framework for generating high-quality univariate and multivariate time series , even in data-scarce settings. SDForger leverages foundation models with minimal fine-tuning by operating over compact tabular embeddings derived from functional decompositions.

Key features and advantages of SDForger:

Compact basis representation SDForger uses FastICA or PCA to embed time series into low-dimensional tabular data. These capture key temporal and inter-variable structures while decoupling the embedding from sequence length, enabling efficient processing of long signals.

Text-to-sequence generation via LLMs The embedding tables are converted into structured textual prompts and used to fine-tune a language model. A guided inference approach is then used to generate structured embeddings, ensuring that the synthetic data retains the original dataset's statistical properties and feature relationships.

Flexible, lightweight architecture The framework leverages autoregressive LLMs, including lightweight models, and requires only a small number of training instances. Its modular design enables easy adaptation to different generation tasks and architectures, while its compact embedding space ensures fast inference, even for long time-series windows.

Multivariate and multimodal readiness SDForger can model complex multivariate dynamics and supports future extensions to textual conditioning, enabling generation guided by both time-series structure and external language-based context.

By combining structured embeddings with LLM-based generation, SDForger establishes a new paradigm for scalable, interpretable, and high-quality synthetic time-series generation.

Our simulations demonstrate that SDForger not only generates statistically realistic time series but also improves downstream model performance, often matching or exceeding results obtained from real data alone. This is particularly valuable in practical scenarios where access to high-quality data is limited or where distribution shifts make original training data less effective. Compared to state-of-the-art generative models, SDForger achieves competitive or superior performance across a wide range of similarity metrics and utility-based evaluations. Notably, our experiments show that even lightweight, pretrained LLMs (e.g., GPT-2) are sufficient to produce high-quality synthetic data with minimal fine-tuning, highlighting the accessibility, efficiency, and flexibility of our approach.

In the remainder of this paper, we first review related work (Section 2), then detail the SDForger framework (Section 3), describe our evaluation setup (Section 4), present extensive experimental results (Section 5), highlight the flexibility of language models (Section 6), and conclude with key takeaways and future directions (Section 7).

2 Related work

Time-series generation Recent advances in time-series generation have introduced a variety of deep generative models, including GAN-based approaches like TimeGAN (Smith and Smith, 2020), state-space models, and vector-quantized architectures such as TimeVQVAE (Lee et al., 2023). While these methods generate realistic sequences, they often struggle with multivariate dependencies, and require training from scratch for each dataset. More recent framework integrate randomly-weighted combinations of time series to improve their pretraining pipeline, e.g. Chronos (Ansari et al., 2024) adapting the Mixup (Zhou et al., 2023) methodology for time series. To address evaluation challenges, TSGBench (Ang et al., 2023) proposes a unified benchmark of similarity, fidelity, and utility metrics.

Foundation models and LLMs for time series Recent advances have introduced foundation models specifically designed for time-series tasks, offering unified frameworks for forecasting, classification, and anomaly detection (Liang et al., 2024). Examples include TFT (Lim et al., 2021), TimeGPT (Garza and Mergenthaler-Canseco, 2023), and Chronos (Ansari et al., 2024), while lightweight models like TTMs (Ekambaram et al., 2024) focus on efficient multivariate forecasting. Parallel efforts have explored adapting LLMs to time-series data by encoding numerical sequences as text (Gruver et al., 2024; Zhou et al., 2023; Jin et al., 2023), enabling zero-shot inference and transfer learning. Notably, LLMTime (Gruver et al., 2024) and GPT4TS (Zhou et al., 2023) retain most of the LLM architecture while fine-tuning only shallow layers, and Time-LLM (Jin et al., 2023) employs reprogramming to adapt to temporal tasks. Despite these innovations, most existing approaches either require task-specific pretraining or struggle to model complex structures and may benefit from synthetic data for improving generalization and robustness.

Specialized architectures for time-series generation Several architectures have been specifically designed to capture temporal and multivariate dependencies in time series. Variational autoencoders such as TimeVAE (Desai et al., 2021) and TimeVQVAE (Lee et al., 2023) use recurrent or vectorquantized structures to model sequential dynamics. GAN-based approaches, including RTSGAN (Pei et al., 2021), SDEGAN (Kidger et al., 2021), and COSCI-GAN (Seyfi et al., 2022), employ recurrent or component-wise disentangled generators to capture complex temporal patterns adversarially. Diffusion-based models, like LS4 (Zhou et al., 2023), generate sequences through learned reversetime processes. These specialized architectures complement general-purpose time-series generation methods and provide valuable baselines for evaluating synthetic data. However, these architectures are trained from scratch and cannot leverage existing pre-trained language or foundation models, limiting their scalability and adaptability across domains.

3 Methodology

In this section, we present our methodology, illustrated in Figure 1. SDForger is divided into three macro-steps: (i) Preprocessing and embedding transform the time series into tabular data (i.e., steps 1 to 3 highlighted in purple); (ii) Fine-tuning and Generation fine-tune a pre-trained LLM and generate new embedding instances (i.e., step 4 highlighted in green); (iii) Decoding reconstruct the original time-series space from the generated embeddings (i.e., steps 5 and 6 highlighted in light blue).

Original Data

6

7 = { 6 7

9

}

9+:

;

<

,

6 7

9

∈ ℝ

=×>

̃ !

!!

!

̃ !

!"

!

̃ !

!#

!

̃ !

!!

"

̃ !

!"

"

̃ !

!#

"

̃ !

"!

!

̃ !

""

!

̃ !

"#

!

̃ !

"!

"

̃ !

""

"

̃ !

"#

"

4. LLM

Inference

and

Finetuning

Time

Series

Generated Data

Tabular Data

Text

!

" =

! "

!

, … , ! "

"

∈ ℝ

# $×&

"Input:

value_1

is [blank],

value_2

is [blank] …

value_6

is [blank]. [sep]

Target:

(

()

)

[answer]

(

(*

)

[answer]

…

(

(+

*

[answer].'

(

)

*+

, ) ∈

*, +

,

6

"Input:

value_1

is [blank],

value_2

is [blank] …

value_6

is [blank]. [sep]

Target:

, (

,)

)

[answer]

, (

,*

)

[answer]

…

, (

,+

*

[answer].'

,

-

*+

, -	 ∈

*, /

+

0

!

∈ ℝ

"×.

)

" =

"

!

, … , "

"

∈ ℝ

$×&

1. PERIODICITY

AWARE

SEGMENTATION

8 = {8

1

}

1)2

3

, 	8

1

∈ ℝ

4×5

2. ICA/FPC

EMBEDDING

!

!!

!

!

!"

!

!

!#

!

!

!!

"

!

!"

"

!

!#

"

!

"!

!

!

""

!

!

"#

!

!

"!

"

!

""

"

!

"#

"

!

#!

!

!

#"

!

!

##

!

!

#!

"

!

#"

"

!

##

"

6

9

2

9

6

CHANNEL 1

CHANNEL 2

3. TEXTUAL ENCODING

5. TEXTUAL DECODING

9

2

9

6

CHANNEL 1

CHANNEL 2

6. ICA/FPC

DECODING

1 = 2

!

+	2

/

Generation

Notation Hereinafter, we introduce some basic notation. Let X = { X i } I i =1 , X i ∈ R C × L represent a collection of I instances of a multivariate time series, where each instance has length L and consists

of C channels. The task of synthetic time-series generation can now be formally defined as producing ˜ I instances of a multivariate time series { ˜ X i } ˜ I i =1 , ˜ X i ∈ R C × L conditioned on the given context X . Throughout the paper, we denote by x c i ∈ R L the i -th instance of channel c , and by X c ∈ R I × L the matrix collecting all instances associated with channel c .

Periodicity-aware segmentation In cases where only a single time series instance X 1 ∈ R C × L 1 is available, we apply a segmentation strategy to artificially create multiple instances. The segmentation procedure facilitates the estimation of the time-series distribution. Specifically, we extract I periodicity-aware windows of fixed length L < L 1 , aligning cuts with natural cycles and minimizing overlap to enhance independence and diversity. This pre-processing (described in Appendix A.1) transforms the data from a single sequence X 1 into a set X = { X i } I i =1 , where X i ∈ R C × L , preparing the data for the generation task.

3.1 From time series to tabular data

To enable tabular generation and analysis, SDForger transforms time series into structured tabular data using basis decomposition techniques . Each row represents a time series embedding obtained by projecting the signal onto a set of learned basis functions. Specifically, we adopt two decomposition methods: Functional Principal Components (FPC) (Ramsay and Silverman, 2005) and Fast Independent Component Analysis (FastICA) (Hyvarinen, 1999):

FPC identifies principal modes of variation by performing eigen-decomposition of the covariance operator. It captures directions of maximal variance, preserving correlation across components, showing effectiveness in modeling multivariate longitudinal data (Boschi et al., 2024).

FastICA extracts statistically independent components by maximizing non-Gaussianity. It decomposes a contrast function, uncovering independent latent factors that may not align with the directions of maximal variance.

Formally, for each channel c , we assume the instances ( X c 1 , . . . , X c I ) are realizations of continuous functions defined over T = [0 , L ] . We approximate each X c i as a linear combination of k c basis functions ( b c 1 , . . . , b c k c ) , where the choice of basis depends on the decomposition method (nonGaussianity-based for FastICA, covariance-based for FPC). The embedding coefficients are:

We define the embedding matrix for channel c , E c ∈ R I × k c . By concatenating the embeddings across all channels, we obtain the final embedding table: E = ( E 1 , . . . , E C ) ∈ R I × K with K = ∑ C c =1 k c .

Throughout this paper, we refer to the columns of E as embedding features. We denote the i -th row of E as E i , corresponding to the embedding vector of instance X i , and the value in its k -th column as e ik . More details on the choice of k c are given in Appendix A.2 and Appendix Table D.1.

Notably, both methods offer the advantage that their computational cost depends on the number of instances I and the number of components k c , but not on the instance length L . This decoupling allows our algorithm to handle very long time windows without a corresponding increase in computational complexity, ensuring great flexibility and scalability.

3.2 Generation of tabular data

Our data generation block consists of three key stages: encoding tabular data into text, fine-tuning an LLM, and generating synthetic embeddings.

3.2.1 From embeddings table to text

LLMs are designed to process textual information. Therefore, applying an LLM to tabular data requires converting each row into a textual format that can serve as a prompt during the fine-tuning stage. Inspired by Donahue et al. (2020), we introduce a Textual Encoder responsible for converting tabular instances E i into structured text representations using a Fill-In-The-Middle template.

Definition 1 (Textual encoder) Let P FT = {P FT i } I i =1 denote the set of fine-tuning prompts, where:

Here, the operator ⃝ denotes the concatenation and π is a random permutation of K elements.

Random Feature Order Permutation . Encoding tabular data into text can introduce unintended positional biases, as LLMs inherently process tokens in sequence. To enforce order independence (Borisov et al., 2022), we apply a random permutation π to the encoded feature-value pairs within each instance. This shuffling ensures that the model does not infer any spurious relationships based on the ordering of features within the textual representation. For K = 2 , an admissible finetuning prompt for E i is: 'Input: value_2 is [blank], value_1 is [blank] [sep] Target: e i 2 [answer] e i 1 [answer]'

3.2.2 Large language model finetuning and inference

Fine-tuning By training an LLM on structured text representations of the embedding tables, we enable it to learn meaningful patterns present in the data. Since the optimal number of fine-tuning epochs depends on the number of instances, the embedding dimension, and the LLM architecture, we implement an early stopping criterion to prevent overfitting.

Inference After fine-tuning, inference is performed by prompting the LLM with structured textual templates that mirror the training format, allowing it to autonomously generate new embedding rows.

Definition 2 (Textual inference) Given the embedding table E ∈ R I × K , we define the set of inference prompts at each generation step as P INF = {P INF g } G g =1 where:

We use a multinomial distribution sampling strategy to reduce repetition and generate more creative and diverse outputs. The model draws from its learned token probability distribution at each step, guided by the temperature parameter, which controls sampling variability. As a result, all values are internally generated by the LLM in a fully conditional and self-contained manner, highlighting the model capacity to internalize statistical and structural patterns from compact embeddings and synthesize coherent time series without external noise injection or sampling routines.

At each inference step, we generate a batch of G synthetic instances, repeating the process until the desired number of sequences is obtained or a stopping criterion is met. We denote the set of all generated text instances as: G = {G 1 , . . . , G G } . Ideally, the fine-tuned LLM should generate text instances in the following format: G g = ⃝ ( P INF g , ⃝ K k =1 (˜ a gπ ( k ) [answer] ) ) where π is the random permutation used in P INF g , and { ˜ a gπ ( k ) } K k =1 are the K numerical values inferred, which form the generated embedding table.

Retrieve embedding from text Given a generated text instance G g ∈ G , we reconstruct the corresponding tabular data by mapping the inferred embedding values { ˜ e gπ ( k ) } K k =1 to their respective features. Each textual entry is split into feature-value pairs using "[answer]" as a delimiter. Missing or unrecognized features are assigned the placeholder "NaN" . For a specific channel c , the output of an inference step s is the reconstructed embedding matrix: ˜ E c,s ∈ R G × k c , where each row corresponds to a generated instance G g and each column represents an inferred embedding feature associated with channel c . To track all generated embeddings up to step s , we define: ˜ E c, ≤ s .

In-generation filtering and stopping criterion At each inference step s , we apply an online filtering procedure that validates generated embeddings without requiring reconstruction into the time-series domain, ensuring efficient real-time evaluation. Specifically, the reconstructed embedding matrices ( ˜ E 1 ,s , . . . , ˜ E C,s ) are filtered based on three criteria: 1) Instances with missing values are discarded, as they prevent accurate reconstruction; 2) Duplicated instances are discarded to maintain diversity in the generated dataset; 3) Significantly diverging instances are discarded. This combined filtering procedure not only enforces diversity and validity among the generated instances but also provides a diagnostic signal: if a substantial fraction of samples is rejected, it may indicate that the fine-tuned LLM requires further training or more representative data. Representative examples of

discarded instances and details on the divergence detection procedure are provided in Appendix A.3, while Appendix Table D.2 reports the rejection rates observed in a representative generation scenario, illustrating the balance between filtering rigor and sample diversity.

In generation mode, SDForger employs a dynamic stopping criterion that continues generating batches of G text instances as long as sufficient diversity is preserved among the generated samples (Appendix A.4). However, for consistent comparison with baseline methods across all simulation scenarios, we fix the number of generated instances ˜ I across all algorithms. If we denote by S the final inference step, then the output of the generation process is the complete embedding table ˜ E ∈ R ˜ I × K , where, for each channel c , ˜ E c = ˜ E c, ≤ S .

3.3 Decoding: from tabular embeddings to time series

Given ˜ E , the time-series representation of generated embeddings can be efficiently recovered due to the reversible nature of the embedding technique used. For the channel c , given the generated coefficients ˜ e c ij and the corresponding basis system ( b c 1 , . . . , b c k c ) , the reconstructed time series are computed as follows Kokoszka and Reimherr (2017): ˜ x c i = ∑ k c j =1 ˜ e c ij b c j .

This formulation ensures that each generated embedding is decoded back to the original space, resulting in ˜ I synthetic instances of a multivariate time series ˜ X i ∈ R C × L .

4 Evaluation methodology

Evaluation metrics Evaluating synthetic time-series data requires balancing realism , usability , and efficiency . A strong generative model should replicate key properties of real data while supporting downstream tasks such as forecasting. We adopt a comprehensive evaluation framework comprising two categories: similarity metrics and utility metrics .

Similarity metrics , inspired by Ang et al. (2023), assess how closely the generated data matches the real data in terms of distribution, structure, and behavior. They fall into two subtypes: (i) Feature-based metrics which include Marginal Distribution Difference (MDD) , Auto-Correlation Difference (ACD) , Skewness Difference (SD) , and Kurtosis Difference (KD) , assess how well synthetic data retains key statistical properties of real data; (ii) Distance-based metrics include Euclidean Distance (ED) , Dynamic Time Warping (DTW) , and SHAP-RE (SHR), a shapelet-based reconstruction error. They quantify the similarity between synthetic and real data in raw feature space or temporal alignment. Formal definitions are provided in Appendix B.

Utility metrics assess the effectiveness of synthetic data in downstream tasks. Specifically, we fine-tune Tiny Time Mixers (TTM) (Ekambaram et al., 2024), a recent foundation model for multivariate time series, under four settings: (1) zero-shot (no fine-tuning) (2) real data only, (3) synthetic data only, and (4) real data augmented with synthetic data. This setup quantifies the impact of synthetic data on model transferability, data efficiency, and robustness.

Evaluation protocols We consider three distinct evaluation settings to assess the generative capabilities of SDForger across different structural assumptions:

Multisample generation aims to produce new instances by combining patterns from multiple existing time series. This setting reflects scenarios such as generating experimental samples, weather profiles, or patient trajectories from heterogeneous observations. It emphasizes diversity and generalization in data-rich contexts.

Univariate generation focuses on learning from a single time series to generate plausible alternative versions. This is useful for simulating counterfactual histories, seasonal variations, or stress-test scenarios in domains like finance, weather, and demand forecasting.

Multivariate generation evaluates the ability to jointly generate multiple interdependent channels. It reflects real-world settings, such as energy systems, traffic flows, or sensor networks, where channel interactions and cross-correlations are crucial for realism and downstream utility.

In the multisample case, multiple instances are available by design. In contrast, for univariate and multivariate settings, only one instance is provided; therefore, we first apply the period-aware segmentation procedure described in Section 3 to extract multiple windows from each channel.

Parameter settings We summarize here the hyperparameters for SDForger. We fix the embedding dimension to k = 3 for the multisample and univariate setting . The LLM used for generation is GPT-2 1 , fine-tuned with Adam (Diederik, 2014) optimization, a learning rate of 8 × 10 -5 , batch size 32, and a maximum of 200 epochs. Early stopping criteria is applied based on the best validation loss computed every 5 steps, patience set to 5, randomly choosing 20% of the data as a validation set.

Baselines We evaluated SDForger's performance against several baseline models for synthetic time series generation, covering different approaches. Variational autoencoders : TimeVAE (Desai et al., 2021), which models temporal dependencies with a recurrent VAE architecture, and TimeVQVAE (Lee et al., 2023), which incorporates vector quantization for better capturing discrete temporal patterns; generative adversarial networks : RTSGAN (Pei et al., 2021), which uses recurrent components for adversarial training, and SDEGAN (Kidger et al., 2021), which models time series as solutions to stochastic differential equations; and a diffusion-based model : LS4 (Zhou et al., 2023), which generates sequences via a learned reverse-time diffusion process. Hyperparameters for all baseline competitors follow those reported in their original papers, except for SdeGAN, for which we fix the number of training iterations to 1000 to balance convergence and computational cost.

Datasets We evaluated SDForger models using 12 publicly available datasets from various domains, including energy, transport, industry, weather, and finance, with sampling frequencies ranging from 2 minutes to monthly. The datasets, sourced from the Monash Time Series Forecasting Repository and other public domains, include both stationary and non-stationary time series, reflecting diverse temporal dynamics. Detailed information is provided in Appendix C.

5 Results

Following, we discuss results on Similarity-based (Section 5.1), Utility-based metrics (Section 5.2), and a condensed ablation study (Section 5.3). Complete ablations are provided in Appendix D.

5.1 Similarity-based metrics results

The similarity-based results aggregated for the multisample and univariate settings are reported in Table 1, with detailed per-dataset scores provided in Appendix Tables D.10, D.11, D.12, D.13, and D.14.

Overall performance. Different generative models exhibit complementary strengths: for instance, TimeVAE performs well on distribution-based metrics, while TimeVQVAE excels on distancebased measures such as Euclidean Distance and DTW. In contrast, SDForger achieves consistently strong and balanced performance across both metric categories , maintaining high scores without overfitting to either statistical or structural similarity (Table 1). This balanced behaviour is further confirmed by the normalized average scores per metric group and the average rank values. Such consistency indicates that SDForger not only preserves key statistical features but also captures the underlying temporal and distributional structure of the data, demonstrating strong generalization and robustness across heterogeneous temporal domains. By decoupling representation learning from generation, SDForger captures long-range dependencies while maintaining statistical realism, ultimately producing temporally coherent and domain-consistent synthetic samples.

Robustness to evaluation protocols Comparing multisample and univariate settings, we observe that model rankings and relative performances remain largely consistent, suggesting that SDForger is robust to variations in the evaluation protocol. This stability is an important advantage in practice, where test-time conditions may vary.

ICA vs. FPC The ICA embedding strategy consistently leads, particularly on distance-based metrics. The superior performance of the ICA-based variant likely comes from the nature of the components it produces. Unlike FPC, which orders components by explained variance and often concentrates most information in the first few components, ICA explicitly seeks statistically independent components. This tends to produce a more balanced and disentangled basis decomposition, where each component carries distinct information that have similar importance for data reconstruction. For our LLM-based generation pipeline, this disentanglement appears advantageous because the model can learn a joint

distribution over a set of factors that all have the same 'power'. Thus, the results suggest that the LLM is indeed better equipped to model the joint distribution when presented with independent factors rather than a hierarchy of variance-ordered components.

5.2 Utility-based metrics results

Table 2 presents the utility evaluation, where we assess the practical value of synthetic data by fine-tuning TTM on different multivariate training sources. For LS4, we modified the architecture to support multivariate generation. Furthermore, we adjust the embedding dimension k in SDForger according to the complexity of each dataset, setting k =3 for bikesharing , k =7 for etth1 , and k =5

for traffic . To determine these values, we conducted a small ablation study to identify the optimal embedding dimension for each dataset (see Table D.7).

SDForger demonstrates strong performance across datasets, as evidenced by its top average rank, with notable results on bikesharing and etth1 . In bikesharing , synthetic data from SDForger alone yields competitive scores, and combining it with real data leads to the best overall performance across metrics. On etth1 , SDForger-generated data surpasses original data in RMSE and WQL, suggesting it captures critical temporal and statistical structure. The hybrid setting (original + generated) maintains this advantage and further improves MASE. Performance on traffic is more nuanced. Here, finetuning on real data is less effective, and GAN-based methods outperform others. Nevertheless, SDForger remains competitive, especially when using synthetic data alone. This suggests that the test distribution may deviate significantly from the training set, making traditional fine-tuning less useful. Indeed, high-quality synthetic data can act as a valuable supplement or even an alternative.

In no scenario does synthetic data degrade downstream performance, underscoring the reliability and utility of SDForger-generated samples across varied forecasting contexts.

5.3 Ablation

Effect of embedding dimension k Appendix Table D.6 presents an ablation study on the number of components k used in SDForger's embedding space. A compact embedding with k =3 offers strong performance across both multisample and univariate settings, indicating that a small number of components is often sufficient to capture core temporal and structural patterns. However, the optimal value of k may vary across datasets, with more intricate dynamics potentially requiring higher-dimensional representations. In practice, users may also opt to select k based on a desired percentage of explained variance, adapting the representation to specific application needs.

Domain-level insights Appendix Tables D.3 and D.4 summarize the average normalized similarity scores per dataset. SDForger models achieve strong performance across a wide range of domains. Structured datasets such as Energy , Appliances , and Weather exhibit particularly high scores. In contrast, domains such as Tourism , Traffic , and Finance present greater challenges, likely due to their increased irregularity and noise. Nonetheless, SDForger maintains competitive results even in more complex settings, underscoring the flexibility of the proposed architecture.

Generation efficiency Appendix Table D.5 reports the average time to generate univariate sequences for three targets from the Bikesharing dataset, across two window lengths. SDForger is substantially faster than all competitors, often by one to two orders of magnitude. TimeVAE is the closest competitor but remains over 4 × slower. Notably, unlike GAN-based competitors, SDForger's generation time is independent of sequence length and scales with the number of embedding components ( k ). A minor exception occurs at k = 3 , where the reduced latent expressivity increases LLM fine-tuning time. Overall, SDForger achieves state-of-the-art efficiency without compromising quality.

LLMcomparison GPT-2 (124M) achieves performance on par with, and sometimes better than larger and more recent models such as granite-3.0 (2B) and phi-3.5 (3.8B) (Appendix Table D.9). This shows that SDForger's pipeline is effective even leveraging lightweight models. While runtime cost grows with model size, SDForger remains efficient compared to baselines (Appendix Table D.8).

Filtering Procedure. Appendix Table D.2 reports rejection statistics for a representative generation scenario. We observe that the overall discard rate remains consistently low ( < 2% ) across settings, indicating that most generated embeddings fall within a plausible norm range. The proportion of missing values increases with larger embedding dimensions, reflecting a higher likelihood of incomplete generations in longer textual outputs. Notably, the ℓ 2 norms of accepted samples closely match those of the original embeddings, while discarded ones exhibit markedly higher values, confirming that the filtering procedure effectively removes divergent or anomalous generations.

6 Shaping time series with language

SDForger is designed to naturally incorporate textual information, making it well-suited for stateof-the-art time-series generation that embraces additional multi-modal inputs. To explore this, we conduct an experiment using the bikesharing dataset. Coming from the intuition that these variables stem from a common physical process and may share latent components, we embed their three

channels (temperature, count, and humidity) into a shared ICA basis. We incorporate the channel information in the textual encoder: 'Condition: data is temp [sep] Input: value_1 is [blank]...[sep] Target: e i 1 [answer]...'

This conditioning strategy enables SDForger to generate channel-specific sequences with high fidelity. For instance, using a longitudinal k -nearest neighbor classifier (Ramos-Carreño et al., 2024) trained on real data, we achieve an accuracy of 0 . 81 in identifying the generated curves (see Figure 2). These results highlight SDForger's strong generative capacity and its ability to integrate and respond to textual cues, positioning it as a flexible and powerful baseline for multimodal time-series synthesis.

200

0

200

400

600

800

1000

1200

count

0.2

0.4

0.6

0.8

1.0

1.2

humidity

0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2

temperature

7 Conclusions

We introduced SDForger , a flexible and efficient framework for generating synthetic multivariate time series using large language models. By combining compact functional embeddings with textual conditioning, SDForger enables high-quality generation even in data-scarce settings. Extensive evaluations across multiple datasets and tasks demonstrate that SDForger consistently achieves strong similarity scores and enhances downstream forecasting performance-often matching or surpassing results obtained from real data and outperforming state-of-the-art baselines.

Ablation studies confirm the robustness of the framework across embedding strategies, dimensionality choices, and LLM architectures. SDForger is also highly efficient, with significantly lower generation times compared to its competitors. Moreover, by leveraging LLMs, SDForger enables seamless integration with textual prompts, paving the way for multimodal time-series generation, where natural language can guide not only content but also structure, semantics, or temporal context.

We believe SDForger can be further improved. Its modular design is intentionally built to support flexible experimentation, making it easy to explore enhancements or tailor components to specific needs. We see several promising directions:

Embedding Strategies While our current approach relies on linear methods like FastICA and FPC, future work could explore more expressive, nonlinear embeddings (e.g., AE) or multivariateaware methods like Multivariate FPCA or Multivariate Singular Spectrum Analysis to better capture temporal and inter-channel dependencies.

Parameter-Efficient Fine-Tuning We currently use full fine-tuning for the LLM. However, using too many components relative to the number of instances can lead to unstable fine-tuning and reduced generation quality. Incorporating PEFT techniques such as LoRA or adapters could improve scalability, efficiency, and facilitate domain adaptation.

Extension to encoder-only models Our current implementation supports only autoregressive LLMs; future work would extend the framework to encoder-only models and different generation paradigms such as masked token prediction.

Extended Utility Evaluation While we focus on forecasting, SDForger could be evaluated and optimized for broader downstream tasks such as classification or anomaly detection.

Context and Covariate Integration By design, SDForger supports integration of external covariates (e.g., categorical or textual data). Expanding this functionality could enable richer conditional generation, and multimodal transfer learning (see Section 6).

In summary, SDForger offers a flexible foundation, and we see meaningful opportunities to improve it both architecturally and in terms of task generalization.

References

Ang, Y., Q. Huang, Y. Bao, A. K. Tung, and Z. Huang (2023). Tsgbench: Time series generation benchmark. Proc. VLDB Endow. 17 (3), 305-318.

Ansari, A. F., L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. S. Rangapuram, S. P. Arango, S. Kapoor, et al. (2024). Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815 .

Borisov, V., K. Seßler, T. Leemann, M. Pawelczyk, and G. Kasneci (2022). Language models are realistic tabular data generators. arXiv preprint arXiv:2210.06280 .

Boschi, T., L. Testa, F. Chiaromonte, and M. Reimherr (2024). Fasten: an efficient adaptive method for feature selection and estimation in high-dimensional functional regressions. Journal of Computational and Graphical Statistics , 1-13.

Desai, A., C. Freeman, Z. Wang, and I. Beaver (2021). Timevae: A variational auto-encoder for multivariate time series generation. arXiv preprint arXiv:2111.08095 .

Diederik, K. (2014). Adam: A method for stochastic optimization. (No Title) .

Donahue, C., M. Lee, and P. Liang (2020). Enabling language models to fill in the blanks. arXiv preprint arXiv:2005.05339 .

Ekambaram, V., A. Jati, N. H. Nguyen, P. Dayama, C. Reddy, W. M. Gifford, and J. Kalagnanam (2024). Ttms: Fast multi-level tiny time mixers for improved zero-shot and few-shot forecasting of multivariate time series. arXiv preprint arXiv:2401.03955 .

Garza, A. and M. Mergenthaler-Canseco (2023). Timegpt-1. arXiv preprint arXiv:2310.03589 .

Godahewa, R., C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Manso (2021). Monash time series forecasting archive. arXiv preprint arXiv:2105.06643 .

Gruver, N., M. Finzi, S. Qiu, and A. G. Wilson (2024). Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems 36 .

Hyvarinen, A. (1999). Fast ica for noisy data using gaussian moments. In 1999 IEEE international symposium on circuits and systems (ISCAS) , Volume 5, pp. 57-61. IEEE.

Jablonka, K. M., C. Charalambous, E. Sanchez Fernandez, G. Wiechers, J. Monteiro, P. Moser, B. Smit, and S. Garcia (2023). Machine learning for industrial processes: Forecasting amine emissions from a carbon capture plant. Science Advances 9 (1), eadc9576.

Jin, M., S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F. Li, S. Pan, et al. (2023). Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728 .

Kidger, P., J. Foster, X. Li, and T. J. Lyons (2021). Neural sdes as infinite-dimensional gans. In International conference on machine learning , pp. 5453-5463. PMLR.

Kit, A., A. Järvinen, Y. Poels, S. Wiesen, V. Menkovski, R. Fischer, M. Dunne, A.-U. Team, et al. (2024). On learning latent dynamics of the aug plasma state. Physics of Plasmas 31 (3).

Kokoszka, P. and M. Reimherr (2017). Introduction to functional data analysis . CRC Press.

Lee, D., S. Malacarne, and E. Aune (2023). Vector quantized time series generation with a bidirectional prior model. arXiv preprint arXiv:2303.04743 .

Liang, Y., H. Wen, Y. Nie, Y. Jiang, M. Jin, D. Song, S. Pan, and Q. Wen (2024). Foundation models for time series analysis: A tutorial and survey. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining , pp. 6555-6565.

Lim, B., S. Ö. Arık, N. Loeff, and T. Pfister (2021). Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting 37 (4), 1748-1764.

Padhi, I., Y. Schiff, I. Melnyk, M. Rigotti, Y. Mroueh, P. Dognin, J. Ross, R. Nair, and E. Altman (2021). Tabular transformers for modeling multivariate time series. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 3565-3569. IEEE.

Pei, H., K. Ren, Y. Yang, C. Liu, T. Qin, and D. Li (2021). Towards generating real-world time series data. In 2021 IEEE International Conference on Data Mining (ICDM) , pp. 469-478. IEEE.

Ramos-Carreño, C., J. L. Torrecilla, M. Carbajo-Berrocal, P. Marcos, and A. Suárez (2024). scikit-fda: a python package for functional data analysis. Journal of Statistical Software 109 , 1-37.

Ramsay, J. O. and B. W. Silverman (2005). Functional data analysis (2 ed.). Springer.

Seyfi, A., J.-F. Rajotte, and R. Ng (2022). Generating multivariate time series with common source coordinated gan (cosci-gan). Advances in neural information processing systems 35 , 32777-32788.

Smith, K. E. and A. O. Smith (2020). Conditional gan for timeseries generation. arXiv preprint arXiv:2006.16477 .

Zheng, G., Y. Yang, and J. Carbonell (2016). Efficient shift-invariant dictionary learning. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pp. 2095-2104.

Zhou, L., M. Poli, W. Xu, S. Massaroli, and S. Ermon (2023). Deep latent state space models for time-series generation. In International Conference on Machine Learning , pp. 42625-42643. PMLR.

Zhou, T., P. Niu, L. Sun, R. Jin, et al. (2023). One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems 36 , 43322-43355.

Zhou, Y., L. You, W. Zhu, and P. Xu (2023). Improving time series forecasting with mixup data augmentation.

Appendix

A Implementation details

A.1 Data preprocessing: segmentation

In many scenarios, each channel consists of a single historical time series, i.e., I 0 = 1 . However, to estimate embeddings that effectively capture the temporal distribution, multiple instances per channel are necessary. Therefore, when I 0 = 1 , we segment each channel into multiple overlapping windows. Specifically, for each channel c , we construct I c windows of fixed length L c , where L c < L 0 . Without loss of generality, for simplicity in notation, we assume L c and I c are identical across all channels and denote them as L and I , respectively.

To ensure robust learning of the embedding distribution, I must be sufficiently large. Our experiments indicate that even I = 15 (i.e., 15 instances) suffices for this purpose. Once I and L are fixed, we segment the time series while minimizing the overlap between consecutive windows. The overlap step is determined by the dominant periodicity P of the channel, ensuring that window transitions align with intrinsic temporal cycles.

The set of extracted windows W is formally defined as:

where the step size s is computed as: s = max(1 , ⌊ L 0 -L I -1 ⌋ ) and then adjusted to be the nearest multiple of P to maintain consistency in periodic structure.

To determine the dominant periodicity P , we employ the Autocorrelation Function (ACF), which quantifies the similarity between the time series and its lagged versions at different time shifts. This method is robust to noise and remains effective even when periodicity is not strictly stationary.

The estimation of P follows these steps:

Compute the ACF and identify significant peaks, excluding lag 0 .

Rank the detected peaks by their autocorrelation values.

Select the highest-ranked period P such that P < L/ 2 .

By leveraging this periodicity-aware segmentation strategy, we ensure that the extracted windows align with the natural cycles of the time series. Moreover, this approach minimizes window overlap, maximizing their independence and diversity and facilitating more effective embedding computations for downstream generative tasks.

This pre-processing transforms the data from a single sequence X 1 into a set X = { X i } I i =1 , where each X i ∈ R C × L , preparing the data for the generation task.

A.2 Choice of the number of components

The choice of k c determines how well the basis representation approximates the original time-series channel c . Our frameword allows the user to either select the smallest k c that explains a predefined percentage of the total variance of the original time series or manually specify k c .

There exists an inherent trade-off in selecting k c . A higher k c captures more of the total variability but can hinder the LLM's ability to model the underlying distribution during the generation phase. Moreover, if k c is too large, the generated samples may become overly similar to the original data,

limiting diversity and the introduction of novel patterns. Conversely, choosing k c too small risks omitting essential structures and temporal characteristics, degrading the reconstruction quality.

It is important to note that the nature of the components differs between FPC and FastICA. FPC forms a parsimonious basis system, where a few components typically suffice to capture most of the variability, with components ordered by the amount of variance they explain-early components being systematically more informative. In contrast, FastICA components are unordered: each component contributes independently, without a hierarchical importance structure. As a result, FastICA generally requires more components to achieve a similar reconstruction quality compared to FPC. However, this property also makes FastICA embeddings more robust during generation, as information is distributed more evenly across components, reducing the risk that a few badly generated components disproportionately affect the synthesized curves.

To provide a quantitative intuition, Appendix Table D.1 reports the proportion of variance retained across embedding dimensions for both decomposition methods. This analysis highlights how the variance explained increases with k , and how FPC typically achieves higher cumulative variance with fewer components, while FastICA distributes information more evenly across dimensions.

In practice, we recommend keeping the total number of components K reasonably small, particularly when the number of training instances is limited. Empirically, with a training set of 30 instances, setting K > 25 often results in unstable fine-tuning and an increased rate of discarded samples due to low-quality generation. This limitation stems from the LLM's reduced ability to model high-dimensional embeddings under data-scarce conditions effectively.

A.3 In-generation filtering

Missing values and duplicated instances. To illustrate the filtering logic, we report three concrete examples of generated prompts with embedding dimension K = 4 . Following the inference template:

the model produces the following textual generations:

· Prompt 1 (valid)

Input: value_2 is [blank], value_4 is [blank], value_1 is [blank], value_3 is [blank] [sep] Target: 0.125 [answer] -0.084 [answer] 0.217 [answer] 0.041 [answer]

· Prompt 2 (duplicated)

Input: value_4 is [blank], value_1 is [blank], value_2 is [blank], value_3 is [blank] [sep] Target: -0.084 [answer] 0.217 [answer] 0.125 [answer] 0.041 [answer]

· Prompt 3 (missing value)

Input: value_1 is [blank], value_3 is [blank], value_2 is [blank], value_4 is [blank] [sep] Target: 0.182 [answer] 0.095 [answer] -0.012 [answer]

In this example, the filtering stage identifies Prompt 2 as a duplicate of Prompt 1 and discards it, while Prompt 3 is removed because it does not contain all the targets' coefficients. Only Prompt 1 is retained for reconstruction. This simple yet effective procedure ensures that the generated embedding tables remain diverse and valid before decoding into the time-series domain.

Diverging instances To ensure the quality of synthetic data, we discard generated instances whose embedding coefficients significantly deviate from the distribution of the original data. Specifically, we compute the squared ℓ 2 -norm of each embedding vector and compare it to the norms of the original embeddings. This criterion efficiently filters out extreme outliers in the latent space, without requiring reconstruction into the time-series domain.

Formally, for each channel c , let ˆ E c,s = E c ∪ ˜ E c, ≤ s -1 be the matrix containing both original embeddings and all previously accepted generated embeddings up to inference step s -1 , with ˆ E c, 0 = E c . Denote by N c, old and N c, new the sets of squared Euclidean norms of the rows of ˆ E c,s -1

and the newly generated matrix ˜ E c,s , respectively. For each newly generated row i , we compute its norm and accept it only if:

where q 1 and q 3 are the first and third quartiles of N c, old , and IQR = q 3 -q 1 . An instance is retained only if this condition is satisfied across all channels c .

This norm-based strategy is particularly well-motivated when using FPCs, due to the orthonormality of the basis. Let X c i be a time series in channel c and b c j the corresponding FPC basis. Then the L 2 -norm of X c i can be approximated by the Euclidean norm of its FPC coefficients:

where e c ij are the FPC embedding coefficients. This justifies the norm-based filtering as a direct proxy for detecting time-series samples with unusually high or low energy.

While filtering based on coefficient norms does not guarantee full statistical fidelity to the original time-series distribution, it serves as an effective mechanism to remove extreme outliers without reconstruction. Combined with additional checks for missing values and duplicates, this step helps preserve both the diversity and relevance of the generated data. A high rejection rate may indicate insufficient LLM fine-tuning or poor generalization, suggesting the need for more representative training data or additional training steps.

A.4 Stopping criterion

The stopping criterion monitors the diversity of the generated norms across all channels to determine when the generation process should stop. When using FPC, these norms correspond to the L 2 -norms of the generated curves. When using FastICA, they correspond to the norms of the embedding coefficient vectors; although not directly related to the curve norms, they still provide a useful proxy for identifying over-sampling and loss of variability.

At inference step s , for each channel c , let u c denote the number of unique values in N c, old (the set of accepted norms up to step s ), rounded to the fourth decimal place. Let ˜ I be the total number of valid instances generated so far. We define the diversity score for channel c as:

The diversity score provides a quantitative measure of how much variability remains in the generated norms. We track D c at each inference step, and the stopping condition is triggered when:

In other words, generation stops either when the maximum diversity score across channels falls below a predefined threshold λ stop, indicating reduced novelty, or when the total number of generated instances exceeds a maximum cap ˜ I max .

Monitoring the diversity score enables us to assess whether the model continues to introduce new variability in the generated data, serving as an online signal for generation quality.

If we denote by S the final inference step, then the output of the generation process is the complete embedding table ˜ E ∈ R ˜ I × K , where, for each channel c , ˜ E c = ˜ E c, ≤ S .

B Evaluation metrics

All the metrics presented below are adopted from Ang et al. (2023), except for Shapelet-based Reconstruction , which follows the definition in Zheng et al. (2016).

B.1 Feature-based evaluation

Marginal Distribution Difference MDD computes an empirical histogram for each dimension and time step in the generated series, using the bin centers and widths from the original series. It then calculates the average absolute difference between this histogram and that of the original series across bins, assessing how closely the distributions of the original and generated series align.

AutoCorrelation Difference ACD computes the autocorrelation of both the original and generated time series, then determines their difference. By contrasting the autocorrelations, we could evaluate how well dependencies are maintained in the generated time series.

Skewness Difference SD is vital for the marginal distribution of a time series, quantifying its distribution asymmetry. Given the mean (standard deviation) of the train time series T tr s as µ tr s ( σ tr s ) and the generated time series T gen s as µ gen s ( σ gen s ), we evaluate the fidelity of T gen s by computing the skewness difference between them as:

Kurtosis Difference Like skewness, KD assesses the tail behavior of a distribution, revealing extreme deviations from the mean. Using the previous notations, the kurtosis difference between T tr s and T gen s is calculated as:

B.2 Distance-based evaluation

Euclidean Distance For each original series s tr = ( x 1 , ..., x l ) and its generated s gen = ( y 1 , ..., y l ) , ED = √ ∑ l 1=1 ( x i -y i ) 2 . We take the mean of ED for all series and all samples. Given that the input time series has been preprocessed to fit within the range of [0 , 1] , ED deterministically assesses the similarity between s gen and s tr . It provides a value-wise comparison between the time series.

Dynamic Time Warping Given that ED overlooks alignment, we include DTW to capture the optimal alignment between series regardless of their pace or timing. The alignment facilitated by DTWoffers insights into the predictive quality of the generated series.

Shapelet-based Reconstructions Shapelet based RE is calculated by generated time series using shapelets extracted using shift invariant dictionary learning (SIDL) algorithm (Zheng et al., 2016). Shapelets represent local discriminative patterns present in the time-series data. We learn shift invariant patterns/shapelets on the original time-series dataset and then use the learnt dictionary to reconstruct unseen generated time series. The reconstruction error is calculated between the generated time series and their reconstruction using SIDL.

C Datasets & protocol settings

C.1 Dataset overview

Energy

Australian Electricity (Godahewa et al., 2021) contains electricity demand data from 5 states in Australia.

Appliances 2 contains house temperature and humidity conditions monitored with a wireless sensor network, and energy data logegd with m-bus energy meters averaged for 10 minutes periods.

ETTH1 3 contains oil temperatures and other covariates of electrical transformers from two stations in China, measured at 15 minutes granularity but hourly aggregated.

ECL 4 contains electricity consumption of 370 points.

Mobility and Transport

Bikesharing 5 contains the hourly and daily count of rental bikes between the years 2011 and 2012 in the Capital bike share system with the corresponding weather and seasonal information.

Traffic 6 contains observations of the number of vehicles each hour in four different junctions

Traffic Monash (Godahewa et al., 2021) contains hourly road occupancy readings from sensors in the San Francisco Bay area.

Tourism (Godahewa et al., 2021) dataset from, used for the Kaggle Tourism Forecasting competition. This dataset is non-stationary.

Nature

Carbon Capture Plant (Jablonka et al., 2023) records the emission profiles of '2-amino-2-methyl1-propanol' (AMP) and 'piperazine' (Pz) collected at every 2 minutes interval.

Weather (Godahewa et al., 2021) contains daily time series of four weather variables (rain, mintemp, maxtemp and solar radiation) measured at weather stations in Australia.

Finance

Exchange Rate (Godahewa et al., 2021) contains daily exchange rates for currencies of eight countries (Australia, British, Canada, Switzerland, China, Japan, New Zealand and Singapore) between 1990 and 2016. This dataset is non-stationary.

NN5 (Daily, Weekly) (Godahewa et al., 2021) contains cash withdrawal data from ATMs. This dataset combines stationary and non-stationary time series.

C.2 Protocols

Multisample setting For MS data preparation, we sampled I = 30 instances, each of length L = 250 , resulting in a total training sequence of 15,000 timestamps. Standard scaling per timestamp is applied. For evaluation, we generate 100 synthetic instances.

Univariate setting For UV data preparation, we used a training sequence of length L 0 = 2 , 000 , segmented into I = 30 instances of length L = 250 using our periodicity-aware segmentation strategy (cf Appendix A.1). Same standard scaling was applied. For evaluation, we generate 100 synthetic instances.

Multivariate setting For fine-tuning TTM in the MV setting, we used consecutive sequences of length L 0 = 5000 , 2500 , and 2500 for training, validation, and testing, respectively. Each set was segmented into I = 30 , 15 , and 15 instances of length L = 1120 using our period-aware segmentation strategy (Appendix A.1). Periodicity was estimated from the training set. Standard scaling per timestamp, computed on the training set, was applied consistently across all splits. For evaluation, we generated 30 instances, matching the size of the training set.

D Additional results

bikesharing

etth1

traffic

RMSE

MASE

WQL

RMSE

MASE

WQL

RMSE

MASE

WQL

0-shot

0.728

2.150

0.287

0.678

2.132

0.255

0.708

1.555

0.255

Original Data (OD)

0.495

0.822

0.178

0.658

1.820

0.232

0.702

1.995

0.283

GEN

SDF-FPC

3

0.527

0.926

0.200

0.692

1.914

0.246

0.699

2.029

0.287

SDF-FPC

5

0.530

0.918

0.198

0.693

2.003

0.252

0.662

1.837

0.262

SDF-FPC

7

0.522

0.915

0.197

0.650

1.887

0.232

0.812

2.265

0.323

SDF-ICA

3

0.514

0.899

0.194

0.647

1.829

0.233

0.730

2.068

0.294

SDF-ICA

5

0.537

0.909

0.194

0.637

1.934

0.233

0.655

1.849

0.262

SDF-ICA

7

0.517

0.898

0.193

0.626

1.820

0.224

0.790

2.189

0.312

OG+GEN

SDF-FPC

3

+ OD

0.493

0.829

0.179

0.658

1.780

0.229

0.736

2.077

0.296

SDF-FPC

5

+ OD

0.487

0.807

0.174

0.659

1.757

0.230

0.743

2.087

0.297

SDF-FPC

7

+ OD

0.492

0.821

0.177

0.666

1.754

0.231

0.706

1.993

0.283

SDF-ICA

3

+ OD

0.487

0.801

0.173

0.640

1.790

0.228

0.734

2.074

0.295

SDF-ICA

5

+ OD

0.486

0.804

0.174

0.649

1.780

0.230

0.750

2.110

0.301

SDF-ICA

7

+ OD

0.490

0.810

0.175

0.642

1.746

0.226

0.718

2.025

0.288

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction accurately summarize the key contributions and scope of the paper. Specifically, they outline SDForger's novel framework for time series generation using LMs, highlight the use of compact functional embeddings, and emphasize its flexibility, scalability, and ability to work in data-scarce regimes. These claims are substantiated throughout the paper via detailed methodology (Section 3), empirical results (Section 4), and ablation studies (Appendix D), confirming the consistency between the stated objectives and the actual contributions

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper explicitly discusses its main limitations in the conclusion. In particular, it highlights that the generation quality depends on the embedding dimensionality k and the number of training instances. When k is too large and training data is limited, the fine-tuning of the LM becomes unstable, potentially leading to low-quality or discarded generations. Additionally, the paper acknowledges that model performance may vary depending on dataset complexity and that the optimal k may be task-dependent. The need for a separate filtering step to ensure high-quality output is also discussed. These considerations reflect awareness of both computational and methodological boundaries.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include formal theoretical results, theorems, or proofs. The work is empirical and methodological in nature, focusing on the design and evaluation of a generative framework for time-series data using large language models.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides detailed information about the datasets, model components (e.g., FPC, ICA, GPT-2), hyperparameters, and evaluation metrics used in both similarity and utility experiments. The experimental setup, including instance length, number of components, fine-tuning strategy, and prompt structure, is clearly described in Section 5 and Appendix D. These details allow for the reproduction of the main results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All datasets used in our experiments are publicly available and listed in the paper. We will include the code as supplemental material at submission time and will release it as open-source upon acceptance.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The paper provides all key experimental details in Section 5 and Appendix C, including data splits, embedding dimensionality, fine-tuning parameters, optimizer, and stopping criteria. Hyperparameter choices are justified through ablation studies, and additional implementation details are included in the supplementary material.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: We do not report error bars or use repeated runs with different random seeds. Instead, we demonstrate robustness by evaluating on a wide range of datasets and metrics. We report results for each individual dataset as well as domain-level and overall aggregated scores (e.g., average rank, normalized scores), which helps mitigate datasetspecific variability and highlights consistent trends across diverse temporal settings.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We describe the compute resources in Appendix D. Experiments were run on NVIDIA V100 GPUs with 20 GB memory and 2 CPU cores. We also include generation time benchmarks (Table D.5) and specify batch sizes, training steps, and stopping criteria. The overall compute budget was moderate, as the models used are lightweight (e.g., GPT-2), and training converges in under 30 minutes for most datasets.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research fully complies with the NeurIPS Code of Ethics. We use only publicly available datasets that do not contain personal or sensitive information. No human participants or surveillance data were involved. Our method is general-purpose and does not support applications that could lead to discrimination, harm, or misuse. We also discuss potential limitations and plan to release the code under an appropriate license with documentation to support transparency and reproducibility.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [No]

Justification: The paper does not include a dedicated discussion of broader societal impacts, as it focuses on a general-purpose framework for time-series data generation using publicly available datasets. The proposed method does not directly engage with high-risk or sensitive applications, nor does it introduce privacy or fairness concerns in its current scope. We encourage responsible use aligned with the ethical deployment of generative models.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The proposed method operates on structured time-series data and leverages publicly available pretrained language models (e.g., GPT-2). It does not release any new pretrained models or scraped datasets, nor does it target high-risk applications. As such, the framework does not pose a significant risk of misuse in its current form.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All datasets and models used in this paper are publicly available and appropriately cited. Where applicable, we have acknowledged the source papers and repositories, Pretrained LMs (such as GPT-2 and Phi) are used under their respective terms of use and clearly referenced in the methodology section.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The paper introduces a new framework (SDForger) for time-series generation. The code to reproduce the methodology and results will be submitted as supplemental material and made publicly available under an open-source license upon acceptance. Documentation will be provided to support reproducibility.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: LLMs are a core component of the SDForger framework, where they are fine-tuned to generate synthetic time series from structured text prompts. The methodology, implementation details, and the impact of different LLM backbones are thoroughly discussed in Section 3 and Section 5

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes SDForger, a synthetic time series generator based on LLMs. The model proceeds as follows: During training, (1) time series are encoded to tabular data, (2) tabular data are encoded to text to tune the LLM; During inference, (1) text is sampled from LLM to decode to tabular data, (2) tabular data is decoded to time series. Extensive experiments have been conducted to validate the method's performance against existing baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- The paper seems to be clearly written and well structured.\n- The technical details seem to be rigorous.\n\nWeakness:\n- The empirical performance of the proposed method appears mixed, and the claim that SDForger achieves “consistently strong” and “balanced” performance is less convincing to me.\n\nQUESTIONS:\nFor the first bulletin in the weakness, I hope the author can explain if they believe SDForger can be improved, and if so, what are the possible approaches.\n\nFor the questions below, the authors should consider either adding an explanatory paragraph or conducting experiments to address them:\n- Can SDForger handle nonstationary time series data? The author should consider adding a corresponding experiment.\n- Does embedding time series into tabular data \"lose\" temporal dependency information? If so, about how much information is \"lost\"? Would this cause SDForger's performance to degrade when handling strongly correlated sequences?\n- The paper highlighted SDForger's \"readiness\" in the intro, but I did not find an experiment supporting this claim. The author should consider providing some actual prompts generated in the experiment process that can highlight this.\n- How do alternative embedding methods, such as AE or TSNE, compare to PCA and ICA? The author should consider adding an explanatory paragraph or conducting an ablation study.\n- What is the dimensionality of time series that SDForger can handle?\n- How does SDForger handle external covariates associated with the time series data?\n\nLIMITATIONS:\nNegative societal impact: I am concerned about the privacy issue of the model deployment in practice: does SDForger ensure that its generated data does not leak sensitive information from training data?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SDForger, a novel framework for generating synthetic multivariate time series. The core idea is to leverage the generative power of Large Language Models (LLMs) by transforming time series data into a format that an LLM can process. The methodology involves three main stages:\n1) An embedding stage where time series instances are projected onto a low-dimensional basis (using FPC or FastICA) to create a tabular representation, decoupling the representation from sequence length.\n2) A generation stage where these tabular embeddings are converted into structured text prompts, which are used to fine-tune an autoregressive LLM (e.g., GPT-2). The fine-tuned LLM then generates new textual embeddings.\n3) A decoding stage where the generated embeddings are converted back into synthetic time series.\nThe authors conduct a comprehensive empirical evaluation across 12 datasets, comparing SDForger to several VAE, GAN, and diffusion-based models. SDForger outperforms traditional generative models, such as VAEs, GANs, and diffusion models, in most settings, both in terms of quality and generation speed. The framework is also shown to be multimodal-ready, integrating textual conditioning for greater flexibility.\n\nSTRENGTHS AND WEAKNESSES:\n- Quality\n  - Strengths: The empirical evaluation is extensive and a significant strength of this paper. The authors test their method on a diverse set of 12 public datasets from various domains, which demonstrates broad applicability. The comparison against multiple modern baselines, including VAEs, GANs, and a diffusion model, is thorough. The use of a dual evaluation strategy, comprising both similarity metrics (statistical and distance-based) and a practical utility metric (downstream forecasting), provides a good assessment of the generated data.\n\n  - Weaknesses: The paper claims strong performance in \"data-scarce settings,\" but the experimental setup relies on segmenting a long time series into a minimum number of instances (e.g., I=30). This might not reflect true data scarcity, where only a few short series are available. The statistical robustness of the results could be improved. As acknowledged in the paper's checklist, the experiments do not report error bars or results from multiple runs with different random seeds. While testing on many datasets provides some evidence of robustness, it is not a substitute for assessing variance from stochastic elements like model initialization or data shuffling. The in-generation filtering process, while practical, relies on a heuristic (3 * IQR threshold) to discard outliers, which may not be optimal for all data distributions. \n\n- Clarity\n  - Strengths: The paper is well-written and logically structured. Figure 1 provides an excellent, intuitive overview of the entire SDForger pipeline, making the complex methodology much easier to understand. The step-by-step explanation of the embedding, generation, and decoding phases in Section 3 is clear and detailed.\n\n  - Weaknesses: The distinction between the three evaluation settings (multisample, univariate, multivariate) and how data is prepared for each could be introduced more explicitly in the main methodology section for better flow. Also, appendices contain important implementation details that might better serve the reader if summarized in the main text.\n\n- Significance\n  - Strengths: The paper addresses a real and growing need for synthetic time series in data-scarce environments. The framework's demonstrated efficiency is a major practical advantage, with generation times being orders of magnitude faster than many competitors (Appendix Table D.3). The most exciting aspect is the framework's extensibility to multimodal generation. The proof-of-concept in Section 6, where generation is conditioned on textual prompts, opens up a wide range of future research and applications in guided and context-aware time series synthesis. \n\n  - Weaknesses: The downstream utility improvements, while present, are sometimes modest. For the traffic dataset, GAN-based methods performed better, suggesting that SDForger may not be universally superior in all scenarios. The long-term impact will depend on how the approach scales to more complex, high-dimensional, and highly non-stationary time series data.\n\n- Originality\n  - Strengths: While prior work has used LLMs to generate tabular data, the application to time series through a functional embedding layer is a novel and creative leap. This approach cleverly reframes the sequence generation problem into a structured text generation problem, which is the native domain of LLMs. This is a fundamental departure from the dominant VAE/GAN/diffusion architectures typically used for this task.\n\n  - Weaknesses: The paper builds on established concepts (FPC/ICA, LLM fine-tuning), but their synthesis into the SDForger framework is entirely novel and applied effectively to a new problem domain.\n\nQUESTIONS:\n**Q1. Clarification on \"Data-Scarce\" Performance:**\n- The paper emphasizes the framework's effectiveness in \"data-scarce\" settings. However, the experiments use at least I=30 instances for fine-tuning. Could you elaborate on the framework's sensitivity to the number of training instances (I)? How does the generation quality and fine-tuning stability degrade when I is significantly smaller (e.g., I<10)? A clearer understanding of this boundary condition would be valuable for practitioners.\n\n**Q2. Choice of Embedding Method:**\n- The results consistently show that the ICA-based variant of SDForger outperforms the FPC-based one, particularly on distance-based metrics. Why might maximizing statistical independence (ICA) be more beneficial for this LLM-based generation pipeline than capturing maximal variance (FPC)? Does this imply that the LLM is better at learning the joint distribution from a set of disentangled factors rather than a hierarchy of variance-ordered components?\n\n**Q3. Generation Rejection Rates:**\n- The methodology includes an in-generation filtering step to discard invalid or outlier samples (e.g., those with missing values or diverging norms). Could you report the typical rejection rates observed during your experiments for different datasets or settings? A high rejection rate could signal that the fine-tuned LLM struggles to generate on-manifold embeddings, which would be an important practical limitation to understand.\n\n**Q4. Comparison with Specialized Architectures for Correlation Preservation:**\n- The paper compares against general-purpose time series generative models. However, work like COSCI-GAN (NeurIPS 2022) introduces an architecture specifically designed to preserve inter-channel correlations in multivariate time series from a common source, using a central discriminator. How do you position SDForger's implicit method of learning correlations (by encoding a full multivariate instance into a single text prompt) against such an explicit architectural approach? A discussion or comparison in the paper could further highlight the unique strengths of using an LLM for this task.\n\nLIMITATIONS:\nWhile the authors have done a good job of discussing the technical limitations of their work in the conclusion and the NeurIPS checklist, they have not adequately addressed the potential negative societal impact. In the checklist, the authors state that the work does not engage with high-risk applications. However, synthetic data generation, especially for domains like finance, can have dual-use potential.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a method for generating synthetic time series data leveraging LLMs called SDForger. SDForger converts time series data into a tabular representation through basis decomposition techniques (FastICA/FPC). These embeddings are then encoded in a text format for fine tuning of the LLM. Inference follows the inverse path and samples embeddings through text prompts from the LLM. These are then converted to the tabular format and finally back into time series by using the reverse process of the embedding technique. Additionally, the authors use a filtering heuristic on the tabular embeddings to improve quality of the synthetic time series. \n\nThe authors evaluate the synthetic time series generated by SDForger by employing similarly based metrics and compare with different generative baselines. In addition, the authors evaluate whether the synthetic data can be used to fine-tune downstream forecasting models. In their ablation study, the authors evaluate different choices of embedding dimensions, inference time, and different choices of the underlying LLM. Finally, the authors conduct a case study on one dataset to illustrate that the method could be further developed into a multi-modal synthetic data generation model.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper presents a sound and creative framework for generating synthetic time series data. This is of high significance because large scale pre-trained time series models might be bottlenecked through the availability of large datasets which stifles progress. The paper introduces an interesting method for synthetic time series dataset generation by leveraging LLMs through embedding techniques from time series to tabular to text and back. This methodology enables further research on embedding techniques and LLM fine tuning. The paper is written clearly and is easy to follow. \n\nThe weaknesses of the paper lie in the empirical evaluation of their method. I understand that synthetic data can be challenging to evaluate. The similarity based results use 12 datasets and several baseline methods. Here, it seems that TimeVAE is still a strong baseline method and that SDForger does not offer significant improvements. On the univariate setting it seems to perform well, but on the multisample setting it does not seem to improver meaningfully over baselines. Given the potential of the method for further development, I do not see this as an issue that would prevent publication, but I would kindly ask the authors to reflect that finding in their writing. Right now, the writing appears justifies SDFormers performance with “balanced performance across all metrics and domains” which I find too vague. \n\nThe utility based evaluation has the limitation that only three datasets have been considered and therefore the empirical results have low empirical power. My main concern with the utility based evaluation is that the setup is not quite clear which might impact the validity of the result. Another weakness is that the effect of the filtering heuristic is not shown. \n\nI will detail my specific questions in the Questions section below. I will consider increasing my score if my major questions are addressed.\n\nQUESTIONS:\n**Major**: \n\nHow exactly are the embedding dimensions chosen in the utility-based experiments? Do precisely does “dataset complexity” map to these specific values? How would the results change if the embedding dimension is changed for each dataset? This could be an interesting ablation. \n\nWhat part of the datasets in the utility-based evaluation are used for fine-tuning the LLM and for selecting the embedding dimension? Has the test part been excluded from training/tuning steps? If not, this could lead to data leakage and skew the results. \n\nHow do SDFormer perform without the heuristic filtering procedure? I think this could be another interesting dimension and it would be interesting to see how much the performance changes when this would be removed. Additionally, I would be curious to see how much such a filtering step changes the results for TimeVAE. I understand that the filtering is done on the tabular and might not be applicable to TimeVAE, but I think it should be possible to apply diversity/divergent filters here as well. \n\n\n**Minor**: \nWhat does “balanced performance across all metrics and domains” in the similarity-based evaluation mean specifically? Can this be quantified?\n\nLIMITATIONS:\nThe main limitation of this work lies in the empirical design in the utility-bases section that only evaluates three datasets. This should be noted in the limitation section because it is hard to judge whether the results presented here would generalize because of the small number of evaluation datasets used.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel framework for generating synthetic multivariate time series using large language models. The proposed pipeline includes transforming time series into a tabular-structured prompt, where each column contains time series components from FastICA or PCA. These prompts are then used to fine-tune an LLM, which generates new synthetic embeddings that retain the statistical and temporal properties of the original data. This work validates the proposed method through extensive experiments, demonstrating its robustness and applicability in data-scarce settings.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**: The submission is technically sound, with claims supported by experimental results across multiple domains (e.g., energy, finance, transport). The authors compare SDForger against multiple baselines (VAEs, GANs, diffusion models) using both similarity-based metrics and utility-based evaluations (forecasting). However, the results could gain more robustness by reporting the error bars and results on more datasets (e.g., long-term forecasting datasets from TSLib and probabilistic forecasting datasets from GIFT-Eval).\n\n**Clarity**: The paper is generally well-structured, but there are areas where clarity could be improved. (1) Abstract: The motivation of the work and the background it aims to solve are not clearly stated here (time series generation, forecasting, or multimodal modeling?). (2) Periodicity-aware segmentation: How is \"periodicity-aware\" reflected？ (3) LLM fine-tuning: How is the LLM  fine-tuned? Do the authors adopt some PEFT methods?\n\n**Significance**: This paper presents contributions that are likely to influence researchers working with time-series data synthetics. SDForger often can match or exceed GANs/VAEs in similarity and utility metrics. One of the biggest concerns in this work is that the improvement achieved by using this method is not obvious (Tables 1 and 2). I think the proposed method could have a large impact by extending it to other LLM architectures and demonstrating its effectiveness. \n\n\n**Originality**: I think the work is interesting because the paper provides new insights that LLMs can better understand time series via structured embeddings. While prior work (e.g., TimeGAN, TimeVAE) focused on specialized architectures (GANs, VAEs). This work bridges time-series generation and multimodal models. Technically, I think the author could conduct a comparison using non-structured transformation (e.g., w/o tabular structure or w/o PCA decomposition). This will enhance the persuasiveness of this work in terms of methodology.\n\nQUESTIONS:\n1. Line 106: How is \"natural cycles\" determined? Are there some dataset-specific configurations?\n\n2. How about the method's efficiency? Especially, the cost of converting tabular data into text representations. It seems to lead to a very long context.\n\n3. Can the author prove experiments that a structured transformation (using tabular format) effectively generates multivariate time series data?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}