CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
Authors: Shifeng Xie
Vasilii Feofanov
Jianfeng Zhang
Themis Palpanas
Ievgen Redko
Venue: ICLR 2026 Oral (2026)
Keywords: Time Series Foundation Model
Time Series Classification
Primary area: learning on time series and dynamical systems

Abstract: Time series foundation models (TSFMs) have recently gained significant attention due to their strong zero-shot capabilities and widespread real-world applications. Such models typically require a computationally costly pretraining on large-scale, carefully curated collections of real-world sequences. To allow for a sample-efficient pretraining of TSFMs, we propose CauKer, a novel algorithm designed to generate diverse, causally coherent synthetic time series with realistic trends, seasonality, and nonlinear interactions. CauKer combines Gaussian Process (GP) kernel composition with Structural Causal Models (SCM) to produce data for sample-efficient pretraining of state-of-the-art classification TSFMs having different architectures and following different pretraining approaches. Additionally, our experiments reveal that CauKer-generated datasets exhibit clear scaling laws for both dataset size (10K to 10M samples) and model capacity (1M to 783M parameters), unlike real-world datasets, which display irregular scaling behavior.

Published as a conference paper at ICLR 2026
CAUKER: CLASSIFICATION TIME SERIES FOUNDATION MODELS CAN BE PRETRAINED ON SYNTHETIC DATA
Shifeng Xie
Vasilii Feofanov Huawei Noah's Ark Lab 42.com
Jianfeng Zhang Huawei Noah's Ark Lab
Université Paris Cité Huawei Noah's Ark Lab xi kidainxieshi gea com
Themis Palpanas Université Paris Cité
Levgen Redko
Huawei Noah's Ark Lab
ABSTRACT
Time series foundation models (TSFMs) have recently gained significant atten- tion due to their strong zero-shot capabilities and widespread real-world appli- cations. Such models typically require computationally costly pre-training on large-scale, carefully curated collections of real-world sequences. To allow for sample-efficient pre-training of TSFMs, we propose CAUKER, a novel al- gorithm designed to generate diverse, causally coherent synthetic time series with realistic trends, seasonality, and nonlinear interactions. CAUKER com- bines Gaussian Process (GP) kernel composition with Structural Causal Mod- els (SCM) to produce data for sample-efficient pre-training of state-of-the-art classification TSFMs having different architectures and following different pre- training approaches. Additionally, our experiments reveal that CAU BAUKKE--gennann datasets exhibit clear scaling laws for both dataset size (10K to 10M samples) and model capacity (1M to 783M parameters), unlike real-world datasets, which display irregular scaling behavior. The source code publicly available htttps://qithn com/ coom//ht.feeeggEE/CCuuee
INTRODUCTION
Time series data are ubiquitous in applications ranging from healthcare (Gnassounou et al. 2025)
and human activity recognition (Chen et al.) 2025a) to industrial monitoring (Susto et al. 201 8). Recently, the time series community has devoted significant effort to developing large-scale pre- trained time series foundation models (TSFMs). Inspired by advances in natural language processing and computer vision, these models aim to achieve strong zero-shot performance in out-of--istribution (OOD) settings. TSFMs have been proposed for both forecasting (Ansari et al. 2024 Woo et al. 2024, BBhtthaabhhola et al. 2024) and classification tasks (Giosaami et al. [2024] Lin et al.) 2024] Feofanov et al. (025), showing promising results. TSFMs are usually trained on large-scale pre- training dataset collections gathered from different application domains. Recent works used as many
as 300 billion timepoints for model pre-training (Shi et al. 2025).
Despite the prevalence of large-scale pre-training in the development of TSFMs, several works (Hoo et al. 2024 Dooley et al.) [023 Taga al., [025 Liu et al. 2025) showed that comparable performance can be achieved by training them purely on synthetic data. The latter approach has several important advantages. First, it removes the need for time-consuming data collection and curation. This is especially important in time series classification that lacks diverse and rich pre- training corpora. Second, it allows for generating arbitrarily large datasets for model scaling. Finally, it makes the OOD evaluation more meaningful, mitigating the risk of data leakage. Inspired by the recent success of foundation models in tabular classification (Hollmann et al.) 2023). our paper proposes novel safple-efficient pre-training framework for TSFMs in classification based purely on synthetic data. Contrary to tabular and forecasting synthetic data generation pipelines, our proposal seeks to generate sequences with meaningful correlations between samples and realistic temporal

---

Published as a conference paper at ICLR 2026
dependencies within them. We provide an in-depth, large-scate scale study of its benefits compared to pre-training on commonly used time series classification corpora.
Findings Overall, our findings can be summarized as follows:
1. A carefully designed synthetic data generation pipeline can be efficiently used in training classification TSFMs. We propose such pipeline and show that it requires rethinking synthetic data generators proposed previously for tabular data and time series forecasting.
2. Pre-training on synthetic data reveals clear scaling laws both in terms of dataset size and model size. We illustrate this finding by showing that such scaling laws broken when using common classification benchmarks for pre-training, likely due to the lack of diversity in existing classification datasets.
3. Distinet from forecasting (Yao et al. 2025). where the leaderboard (with the exception of (Hollmann et al., (2023) is still dominated by models pre-trained on large-scale real-workls datasets, we show that pre-training on solely synthetic data can lead to state-of-the-ar performance in classification.
The rest of this paper is organized as follows. In Section 2 we present recent advances in TSFMs and describe commonly used pre-training datasets. Section 3 we present the problem setup considered in our work and the proposed synthetic data generation pipeline. In Section we empirically validate the effectiveness its strong generalization, scalability, and superiority over existing synthetic generation methods. Finally, we conclude our work and its limitations in Section 5
KER-generated synthetic data through extensive experiments, demonstrating
2 RELATED WORK
Time series foundation models Recent advances in TSFM have followed two primary directions:
(1) training models from scratch on large-scale, diverse time series datasets (Ansari et al.,
2024
Goswami et al. 2024 Das et al. 2024 Gao et al.)
2024 Rasul et al. 2024 Wang al. [0224
Woo
ct al.] 2024 BBhthhnabhhota et al. [2024]]] Lan et al.
2025 Gao et al.]
2024 Lin et (024 Liu et al.
2024b Cohen et al., 2024 Auer al. 2025). ) and (2) leveraging large language models (LLMs)
as backbones for time series tasks (Chang et al. 2023 Gruver et al. 2024 Zhou et al. [023] Xue
Salim 2023 Cao et al.) 2023 Jin et al.) 2023 Liu al.] 2024a). The first approach focuses on
developing architectures specifically tailored for time series, while the second approach explores encoding time series data into textual formats or extending the model's input mechanisms to natively handle sequential numeric data. Among the TSFMs mentioned above, a vast majority were proposed for time series forecasting, with only (Feofanov et al., 2025) Gao et al. 20244, Goswami et al. 2024] Chang et al. 2025 Lin et al., 2024: Zhang et al.
2025
natively supporting time series classification.
Roschmann et al. 2025) specifically target classification by contrastively pre-training encoder-only models over time series gathered from popular classification benchmarks. They achieve state-of-the-an results in this task. (oosaami et al. (2024) is an encoder-decoder model used for classification and other popular time series tasks, such as forecasting, imputation, and anomaly detection. Gao et al. (2024) relies on a custom architecture and is used in generative and prediction tasks by leveraging task-specific tokens. Finally, Chang et al. (2025) fine-tunes an LLM by adding an appropriate encoder for input data and a classification head to generate predictions.
In particular, (Feolanov et al.] | et al. 2024
Pre-training datasets The training data for TSFM generally fall into three categories: real-world. synthetic, or hybrid datasets combining the two. Models trained (or fine-tuned in case of LL--ddedd
TSFMs) exclusively on real data (Das et al. 2024 Gao et al., 2024 Rasul et al. 2024 Wang et al.
2024 Feofanov et al. 2025 Gao et al., |024|]La. Lin et al. [2024] Chang et al. 2023 Gruver et al.
2024 Zhou et al.) 2023 Xue & Salim 2023 Cao et al. 2023 Jin et al. (022) typically leverage
extensive collections (ranging from 300k to 50M distinct time series) drawn from diverse domains
such as traffic, finance and environmental monitoring. Training on these datasets, however, may be suboptimal scaling-win wise as Quan et al. (2024) obtained comparable performance using < 1% of the original 27B pre-training dataset from Woo et al.) 2024), while Yao et al. (2025) showed that famous forecasting TSFMs have very flat scaling laws in the multivariate setting. Meanwhile, forecasting
2

---

Published as a conference paper at ICLR 2026
models such Chronos (Ansari et al. 2024) and TimesFM (Das al. 2024) enhance their training corpus by incorporating synthetic time series data alongside world data. Beyond sequence-native TSFMs, there is a complementary line of work (Chen et al. 2025b) that maps time series into image-like representations and then applies vision Transformers. Finally, such methods as TimePFN (Taga et al. 2025) and ForecastPFN (Dooley al. (023) are pre-trained solely synthetic data. In all these forecasting models, synthetic data is commonly generated through structured statistical procedures, including Gaussian process (kernel-based) methods or piecewise linear and seasonal pattern constructions with additive noise (for more details, we refer the interested reader to Appendix A) To the best of our knowledge, no prior work has proposed cuasificatioo-orienttee synthetic data generation methods for training time series foundation models.
OUR CONTRIBUTIONS
We now introduce the task of zero-shot time series classification using TSFMs. We then formally present the common pre-training strategies and introduce our synthetic data generation pipeline.
3.1 PROBLEM SETUP
Zero-shot classification As done in prior work on unsupervised representation learning (Franceschi
al. 2019 Yue al. 2022), we see TSFM as encoder F RR - R2 that is kept frozen during the evaluation. For a downstream classification dataset 'D = {(T; y1) ))}} with labels y, E (1,.... C). we use TSFM obtain embeddings 2; F(x:) and train a lightweight classifier h 12 + {1. (1.....CC C} solely on {(2; 3.) At test time, unseen series x* is classified by ÿ M F=x=)) As
F is kept frozen, the resulting accuracy measures the quality of its learned representations.
To quantify OOD generalization ability, we follow Yao al. 2025 and evaluate the studied TSFMs only on samples not seen during their pre-training. In practice, if we evaluate a given TSFM on a test set from UCR (Dau et al. 2019) dataset, we ensure that the TSFM was not pre-trained on it. Our CAU CAUKER-TT KER- KER-pretrained models are trained only on BAUKKER-gnereee see UCR (or any real-world classification benchmark) during pre-training. The original Mantis and MOMENT (Feofanov et al.) 2025 Goswami et al. 2024) checkpoints, as well as other TSFMs we compare to, are pre-trained on large real-world corpora that include UCR train sets (but never UCR test data), following the standard protocol in prior work. Therefore, original Mantis and MOMENT are, to some extent, the in-distribution setup.
synthetic series and never
Self-supervised pre-training Self-supervised learning (SSL) has emerged as a powerful training paradigm for foundation models, allowing them to effectively learn discriminative representations from large-scale unlabeled datasets, significantly reducing dependency on costly data labeling (Jaiswal al. 2020). SSL methods are categorized into two principal types: contrastive learning and masked (reconstruction) learning (Liu et al. 2023). Contrastive learning focuses on distinguishing between similar (positive) and dissimilar (negative) data pairs to learn meaningful representations, Conversely, masked learning leverages reconstruction objectives by training models to predict masked parts of the input, thereby gaining robust contextual understanding (Zhang et al. 2022).
In our work, we cover both pre-training regimes. To this end, we consider Mantis (Feofanov et al. 2025), an open-source FM pre-trained contrastively, and MOMENT (Goswami et al. 2024). which is baasked-based pre-trained model. Detailed formulations of the loss functions and architecture specifics for these models are provided in the Appendix B
3.2 CAUKER: SYNTHETIC DATA GENERATION FOR TIME SERIES CLASSIFICATION
We now present our proposed synthetic data generation pipeline, termed CAUKER for Causal-Kernn generation. To develop our intuition about it, we note that the synthetic data for the time series classification task needs to combine two key ingredients. On the one hand, the generated sequences should exhibit common time series patterns such as seasonality, periodicity, and trend. On the other hand, successful classification assumes that individual time series have meaningful clustering structure that allows the trained model to successfully learn how to disentangle the underlying clusters during training. Below, we present generation pipeline that satisfies these desiderata.
3

---

Published as a conference paper at ICLR 2026
Selected kermels
Combined kernels
Generate with selected mean
SCM with selected activation functions
Generated time series
Activation
function
Figure I: An illustration of the proposed CAUKER pipeline. Kernels sampled from the kernel bank K. are randomly combined and used together with sampled mean functions form GP priors. Time series sampled from these GP priors act as rool nodes in directed acyclic graph that encodes causal dependencies between nodes, Each edge of this graph applies an activation function from predefined activation function bank and aggregates over incoming edges using a random linear transformation to propagate transformed time series through the graph. Intermediate node outputs are optionally interpolated to fixed length, forming the final synthetic dataset. This procedure yields rich, diverse, and causally consistent time series for self-supervised pre-training.
Proposed approach To proceed, we now define three banks of functions, namely: kernel, mean and activation banks denoted as IC (x,(t, ((),,))) / M (1,(()) and A (o(t):) respectively, For the kernel bank, we use the same kernel functions as Ansari et al. (2024). For mean functions, we consider a linear function az + b, exponential function ac"z, and anomaly mean function that inserts random values from (1(- 5) at random indexes. Finally, the activation functions we use for A are linear function oz + b with ~ (0..5, 2), h ~1(11,1 1)). ReLU activation, sigmoid, sine function, element- wise modulo operation mod for ~ 21[1, 5). and Leaky ReLU with a random negative slope from (00001, 0.3). For simplicity, in what follows we let J ~ S denote an i.i.d. sampling (without replacement) of n elements from a set S.
Our generative pipeline, illustrated in Figure then proceeds in five steps as follows:
Step 1. Kernel bank sampling We start by sampling candidate kernels from the kernel bank, ie,
(t)) iid. KC for some random number of candidate kernels ~ (((1,x)))
Step 2. Kernel composition We define a composite kernel based on K 1 randomly sampled binary operations and x). More formally, for random sequence [= {+, x]. we let K = x1 (1, *2 *K-1 (4,4")
Step 3.
Root nodes generation We draw M mean functions {a:(t))MM ltd. and repeat Step and Step M times to obtain composite kernels [MT) We further define M GP priors to sample from {GP(M,, nC, )
M, M ~ 2(11n
Step Activation bank sampling We sample a set of F. activation functions from the activation bank, ie, {a.)e A, E (((1nA))
Step 5. Causal graph propagation We randomly generate a directed acyclic graph (DAG) (V,E) with |8| F, | V. and M < V root nodes, i.e., nodes with in-degree zero. We then define a bijection associated with a function di, i.e., o(u;) d;- We then associate time series f, E R sampled from GP(u, kt)) to each of the M root nodes. The value ty associated with given non-root vertex "j is then calculated as follows. First, we concatenate incoming edges e, and aggregate them using randomly initialized linear layer with weights and biases W,b b ~ (0, 1), then we apply randomly sampled activation function to get by = (W [e,]+ b).
V + {o1,02..... σy such that each node U; is uniquely
A complete pseudocode of this procedure, as well as the composition and visualizations of the kernel, mean, and activation banks, are provided in Appendix C
Design choices The synthetic datasets generated using CAUKER approach effectively encode diverse, realistic patterns and causal dynamics characteristic of real-world classification problems. Un- like the kernel-only generator of Ansari et al. (2024) (Steps 1,2), which designed for forecasting
4

---

Published as a conference paper at ICLR 2026
and therefore draws zero-mean Gaussian-pres prussisnnppocess process samples that emphasize smooth trend extrapolation, our task calls for retaining the mean level itself (Step 3) as a discriminative cue - a choice that is empirically confirmed in Section[.44 Conversely, the structural causal model (SCM) generator (Steps 4,5) originally proposed for tabular classification (Hollmann et al. 2023) produces rich non-linear dependencies but lacks hallmark time series motifs such seasonality línear trends. By unifying kernel composition with an SCM processing, CAUKER inherits the periodic structure of Gaussian processes while simultaneously injecting causal semanties through directed edges. Finally, we note that different nodes of the same SCM in CAUKER can be interpreted as different channels of multivariate time series that share common causal structure. This hints at the potential of CAU for pre-training inherently multivariate models as well. We further ablate the structure of the SCM in C.3 and the impact of kernel bank composition in C.2
Positioning method While synthetic data generation has been explored for time series fore- casting and tabular classification, adapting these pipelines to time series classification TSFMs is far from trivial. prrecasting-orientee generators (e.g., kernel-based method with zero means(Ansar et al. 2024)) are optimized for smooth extrapolation and often neglect class-conditional structure and
inter-class separability. Conversely. tabular SCM generators (e.g.. TabPFN(Hollmann et al. 2023) )
discard temporal structure. Our method, CAUKER, bridges this gap by unifying compllee
Gaussian processes with structural causal models, resulting synthetic corpora that exhibit discrimi- native clusters and well-behaved scaling laws, in sharp contrast to current real- world classification collections whose heterogeneous, imbalanced composition makes them unreliable for pre-training of TSFMs.
Our experiments in Section 4 demonstrate that foundation models pre-trained on such data exhibit improved out-of-distribution generalization and meaningful scaling behavior, outperforming models trained solely on traditional synthetic benchmarks and performing with those trained on much larger real-world time series corpora. We restrict our pre-training experiments to univariate inputs and treat each node's 's trajectory as an individual series, in order to match the univariate pre-training regime of TSFMs.
4 EXPERIMENTAL RESULTS
We now empirically evaluate the effectiveness of our proposed CAUKER framework for pre-training classification TSFMs. Our experiments aim to answer the following key questions:
Q1. How does CAUKER compare to alternative synthetic data generation methods?
Q2. Do TSFMs trained CAUKER data exhibit meaningful data and model scaling laws?
Q3. Can CAUKER- KER-generated synthetic data be competitive replacement for real-world bench- marks in training TSEMs?
In all our experiments, we consider two recent TSFMs, namely Mantis and MOMENT. Mantis is an 8M encoder-only model pre-trained using contrastive learning. We use the 77M version of the MOMENT model. The latter is an encoder-decoder model pre-trained based on masked reconstruction. Considering these two models allows us compare two different pre-training paradigms previously done in Yao et al.) 2025) for forecasting. Finally, we follow Feofanov et al. (2025) and evaluate Mantis in shoo regime by learning Random Forest classifier on the embeddings of training MOMENT.,Goswami
examples of a given dataset. For MOMENT, Goowwami et al. (2024) evaluated their model using an
Support Vector Machine classifier. For both models, we report the test accuracy averaged over 128 UCR datasets, where each dataset has train and test sets following (Dau et al. 2019)
4.1 Q1: CAUKER AGAINST ALTERNATIVE SYNTHETIC GENERATORS
Experimental setup To better understand the exact contribution of the proposed CAUKER, we first start by establishing the virtues of our synthetic data generation pipeline compared to prior work. For this, we generate four different synthetic corpora, namely: 1) FPEN (Taga et al.) 2025) that uses a linear model of coregionalization
sample multivariate time series, 2) Kerne Synth (Ansari et al.) 2024) that randomly composes covariance kernels to define a Gaussian process with zero mean; 3) Mean+ KemelSynnh: our re-implementation of the KernelSynth baseline in which
5

---

Published as a conference paper at ICLR 2026
additionally add non-zero mean functions in the GP: 4) SCM (TabPEN generator), reconstruction of the structural-causal model proposed by Hollmann et al. (202)) for tabular classification We generate univariate time series with length T 512 both Mantis and MOMENT were trained on
time series this length. For a fair comparison, we fix the number of synthetic samples to 100K.
Table I:
Average zero-shot accuracy (%) on the UCR benchmark after pre-training on synthetic corpora generated by different methods.
Model
SCM
FPFN
KernelSynth
Wean-KereelSynth CAUKER (ours)
Mantis
73,49
77.52
77.70
78.20
78.31
MOMENT
59.23
70.85
69.31
72.56
74.24
Results Table [ shows a relative comparison of our proposal compared to other methods. Our first observation is that classification-tailrees tabular data generation pipeline SCM underperforms significantly compared to all other methods. This suggests that temporal dependencies are important for time series classification, differently from the forecasting setup, where TabPFN trained using SCM-generated data is among the strongest foundation models. We further note that forecasting- tailored FPFN and Kernel-Synth also provide suboptimal results, even more so for MOMENT. In the case of Mantis, the results of pre-training on these two datasets are closer to the reported performance of the Mantis model. This can be likely explained by the architecture of Mantis that incorporates strong time series classification priors into it (mean, standard deviation, and difference encoding the token generator unit). On the contrary, MOMENT a generic encoder-decoder model. We further note distinct positive effect of including non-mean functions in the GP used to generate time series in our pipeline. Finally, CAUKER improves upon this stronger baseline in both cases, highlighting the additional benefit of causal structure. The last two observations are particularly valid for MOMENT, indicating that they compensate for the lack of useful inductive biases for the task of time series classification.
Computational cost. To quantify the computational overhead introduced by the SCM layer, we compared CAUKER to the KernelSynth generator that uses the same composite kernel bank and GP implementation as our method. Both generators were asked to pro- duce N = 1,000 univariate time series length T = 512 under identical hardware and software set- tings. As reported Table 2 CAUKER is in fact slightly faster than the KernelSynth baseline. Table 2 further decomposes the CAUKER runtime: more than 99% of the time attributed to the generator itself is spent in GP kernel sampling, while constructing the SCM graph and propagating signals through it contributes
Table Overall wall-clock generation time and internal runtime breakdown for CAUKER.
Item
Time (s)
CAUKER
121.64 .64
KernelSynth
182.25
GP kernel sampling SCM structure + propagation
118.54
1.14
less than 1% of the total cost.
Sorted DTW distance matrix
This happens because CAUKER samples GPs only for the root nodes of the SCM; single set of root processes is then propa- gated through the causal graph, from which multiple nodes can be extracted as different univariate series.
140
120
100
MI distane
100
Qualitative analysis We now provide insights for why CAUKER is particularly suitable for classification. Intuitively, expect that having discriminative signal in the generated data a clustering structure defining meaningful groups of time series should enable efficient classification on previously unseen samples. To verify this, we generate 200 samples using CAUKER and calculate matrix of pairwise Dynamic Time Warping (DTW) distances (Sakoe & Chiba 1978) on them. We
150
40
20
0
50
100
150
Figure 2: Clustering structure of
CAUKER generated dataset with 200 time series.
As the original generator of (Hollmann al. 2023 is not spunesourced, we followed the algorithmic description in the paper and validated the implementation the illustrative examples provided therein.

---

Published as a conference paper at ICLR 2026
do hierarchical clustering on the obtained precomputed DTW distance matrix and sort the rows and the columns according to the obtained cluster memberships. We plot the obtained matrix in Figure 2 From it, we can observe the emergence of clusters (blocks of time series with similar intra-cluster distances) as well as the introduction of anomalies due to the anomaly mean function in the generating GP. This leads to believe that CAUKER generates data tailored specifically to classification, which may explain its superiority when pre-training TSFMs on it. More qualitative analysis with SWD (Sliced Wasserstein distance) (Bonneel al., 2015) and CKNNA (Huh al. 2024 (classwise k-nearest- neighbour alignment) can be find in the Appendix D
4.2
Q2: SCALING LAWS FOR ZERO-SHOT CLASSIFICATION WITH TSFMS
Scaling laws are fundamental to improving foundation models, underpinning their ability to generalize and demonstrate emergent capabilities with increased data and model scale. While scaling laws widely studied in language and vision, their systematic exploration in the context of zero-shot time series classification remains is currently absent. To the best of our knowledge, our work the first to thoroughly investigate scaling laws specifically in the setup of zero-shot time series classification which is of independent interest.
4.2.1 DATA SCALING LAWS
Experimental setup To investigate data scaling laws, we systematically vary the pre-training dataset sizes from two distinct sources: (1) randomly selected subsets of the real-world UEA benchmark (Bagnall et al., 2018) at increments of 0.1%, 1% ... 100%, and (2) synthetic data generated by our proposed CAUKER method, at varying scales from 10K up to 10M samples. We recall that both Mantis and MOMENT take as input univariate time series. This means that each channel of multivariate UEA datasets becomes a training sample, with total of 12M channels (train set and test set combined) from 30 different datasets. Additional details are provided in Appendix
Results As illustrated in Figures 3 experiments indicate that the classification accuracy on the UCR datasets does not monotonically increase with the size of training data when trained on subsets of the UEA dataset (left for Mantis, middle left for MOMENT). We hypothesize that this behavior may be a result domain mismatch between UEA and UCR, further exacerbated by the lack of diversity within the real-world time series of UEA. More broadly, view the irregular UEA scaling as a consequence of how current real-world classification corpora are constructed: UEA aggregates many small, heterogeneous datasets with highly unbalanced sample counts.
In contrast, LACKERR-generate datasets exhibit clear and consistent scaling laws. The accuracy steadily improves with increasing data size, demonstrating the CAUKER-generatter data's 's effective- ness in capturing diverse patterns essential for generalizing to the UCR target set. Additionally, these results also suggest an interesting contrast between model capacities: the lightweight Mantis model achieves competitive performance even with smaller training sets, likely due to the strong time series classification priors incorporated in its architecture that we have mentioned above. In contrast, the larger and more generic MOMENT model exhibits more significant accuracy gains as the training data increases, highlighting its greater capacity to leverage large-scale data for improved representation learning. This distinction underscores the importance of jointly considering model capacity and data availability when designing scalable TSFMs.
4.2.2 MODEL SCALING LAWS
Experimental setup We further assessed model scaling laws by varying the size of the MOMENT model (Small, Base, Large versions of sizes 77M, 248M, and 783M, respectively). and Mantis model (with number of parameters 0.75M, 2.59M, 8.10M) using both UEA and CAUKER- KeE--geeeetted datasets. More details on the experiments can be found in Appendix F
Results Results, as shown in Figure 3 (middle right for Mantis, right for MOMENT), indicate that models trained on real-world UEA data do not exhibit consistent perfor- mance gains with increasing model size, reinforcing the notion of limited data diversity or domain mismatch. Conversely, models trained CAUKFRRgenneaatd datasets consis-
7

---

Published as a conference paper at ICLR 2026
UEA
Caurter 1M
CauKer dataset
dalase!
UEA 10%
Cauker 100K
Caurerr
0.76
Accuracy
0.76
Accuracy
0.78
Accuarcy
0.
Acouracy
0.78
0.74
0.76
0.76
0.72
0.74
0.72
0.72
10
10
10"
10
10a
10a Data Size
10
2.59
8.30
Data Size
Model Size
Model Size
Figure 3: Scaling law of MOMENT and Mantis depending on the dataset size (left, middle left, respectively) model trained on different subsets of UEA and CauK datasets. Scaling law for the same models depending on the model size (middle right, right, respectively)
tently demonstrate increased accuracy as model size grows, clearly validating the pres- ence of model scaling laws enabled by the synthetic CAUKER- -ene-reee pre-trane pre-training data. We further notice that, apart from the single outlier of MOMENT trained on the 10M samples CAUKER corpus, every model pre-trained
CAUKER exhibits a strictly increasing UCR accuracy as its capacity grows, The small increase for MOMENT at 10M indicates that this particular encoder has reached (or is close to) saturation; a similar saturation point can be observed for Mantis once the param- eter count exceeds approximately 28M (see Appendix F for a more large-scale experiment). Conversely, the un-
Cautee
2nd Pincippl Component
stable or even degrading trend on models pre- pre-trained with larger UEA subsets is most plausibly explained by its lack of diversity. In Figure 4 (and Appendix K). we show PCA projection of CAUKER-gennen -enggreedt data, UEA and UCR collections in the embedding space of the original Mantis model. CAUKER--gneere data embeddings cover a large region in the embedding space of Mantis fully encompassing both UEA and UCR.
1st Princppal Comporent
Figure 4: Mantis embeddings of 100K time series drawn from UCR, UEA and generated by CAUKER.
Qualitative analysis A recent work by
CauKer
UEA
Bouniot et al. (2025) showed that the expressive power of pre-trained vision models can be char-
1.00
1.00
Linearitty
0.75
0.75
acterized by measuring their non-linearity. The latter depends not only on the size of the model and its architecture, but also on the pre-training dataset. To verify how TSFMs' expressive power changes depending on the pre-training dataset, we calculate the non-lincarity scores of the acti- vation functions inside Mantis as done in the orig- inal paper for vision transformers. We then plot the obtained values for the Mantis models pre- trained on CAUKER synthetic datasets of varying sizes and compare them to UEA in Figure (top row). We note that Mantis pre-trained on big- ger CAUKER synthetic datasets has a clear trend, while it barely changes when increasing the size
T
0.50
0.50
HH
0.5
0.0
10k
50k
100k
IM
10M
5%
10%
20%
30%
100%
Figure 5: (Top row) -in-liny statistics of the
Mantis models pre-trained on CAUKER synthetic datasets of varying size (left) compared UEA (right); (Bottom row) CKA similarities calculated across the hidden layers of the pre-trained models. of the UEA pre-training sample. Additionally, we validate this finding using the CKA score used to compare the similarity of internal representations of neural networks (Kornblith et al.) 2019). Lower values of CKA indicate that the hidden layers change the inputs in a more drastic, non-linear way. We see that pre-training on CAUKER exhibits structural change in the model's inner workings when the dataset size becomes larger than 100k. In case of weal-woldd UEA data, the CKA scores inside Mantis hidden layers barely change even when the pre-training sample size changes from 600K (5%) to 12M (100%). This, once again, hints at the fact that CAUKET pre- training dataset is much more diverse which aligns well with PCA projection experiment described above.
8

---

Published as a conference paper at ICLR 2026
4.3 TRAINING TIME SCALING LAWS
We now study the training time scaling law that aims at identifying the gains in terms of test accuracy that more compute given by longer optimization of the model can bring.
Experimental setup
We track the evolution of zero-shot accuracy with training epochs for Mantis and MOMENT pre-trained on two corpora, namely 10% subset of the real-world UEA benchmark and synthetic of 1M series generated by CAUKER.
Results As illustrated in Figure 6 accuracy rises steadily when the models are trained on CAUKER; ad- ditional epochs translate into consistent gains for both architectures. When pre-trained on UEA, however, accuracy curves remain flat fluctuate, especially for MOMENT, indicating that prolonged optimisa- tion yields little benefit on this dataset. These find- ings echo the data- and model-scaling observations reported earlier: causally structured, diverse CAUKER data sustains learning over long horizons.
Caudier desoet
UEA diteest
0.775
0.80
Accuracy
Accuracy
0.750
0.78
0.722
0.76
Training Epoch
Training Epoch
Figure 6: Test accuracy across epochs for MOMENT (left) and Mantis (right).
4.4 Q3: SAMPLE-EFFICIENT TREININI TSFMS USING CAUKER SYNTHETIC DATA
Experimental setup We want to study the performance and the sample efficiency of pre-training Mantis and MOMENT foundation models on different datasets. Our main goal is to show that the performance of both models pre-trained on total of 1.89M (Mantis) and 13M (MOMENT) unique time series can be almost matched by a pre- pre-training on a smaller synthetic dataset generated using CAUKER. For the latter, generate few 100k samples for Mantis and 10M for MOMENT to account for the model size difference (8M vs. 77M). As before, we include our study a 100k samples of the world UEA time
baseline given by pre-training Mantis and MOMENT series classification collection. Additionally, we also experiment with a subset of 100k time series randomly drawn from standard forecasting datasets (ETThl, ETTh2, ETTml, ETTm2, Electricity,
ExchangeRate. Illness, Traffic, Weather) (Zhou et al. 2021
Li et al. 2020 Lai et al. 2018 Matsubara
al., 2014 Li et al. 2018 Rasp et al. 2020). Although no prior work trained classification model on such data, we include it to verify whether the forecasting benchmarks can be good alternative for classification TSFM pre-training. On average over UCR datasets, classical models trained directly on the raw series achieve 68. 12% (Logistic Regression), 69.17 17%1%% (XGBoost) and 73.25% (Random Forest) accuracy. For completeness, we also report downstream fine-tuning results in Appendix M
Model
pre-train, set
Size
UCR Included?
UCR acc. (%)
CAUKER
100K
No
78.55
Training Lass
1.4
Mantis
Mantis dataset
1. 189M
Yes
78.66
a.
UEA
100K
No
76.73
02
Forecusting
100K
No
75.81
20
40
60
80
100
CAUKER
10M
No
77.49
recuracy
Time Series Pile
13M
Yes
78.85
MOMENT
CAUKER
100K
No
74.24
UEA Forecasting
100K
No
73.55
Bo.7
100K
No
73.93
Epoch Number
00
60
100
Figure Performance comparison of Mantis and MOMENT models on different pre- pre-training datasets. CAUKER -eeerreed pre-trate pre-training data allows to nearly match the performance of the original TSFMs, while being more sample-efficient. Rows with 'UCR included? Yes' correspond to in-distribution zero-shot evaluation, as the pre-training corpus contains UCR train splits (though not test data). Rows with 'UCR included? No' correspond to strictly OOD zero-shot models. Training loss and test accuracy corresponding to the first two rows illustrated in the right figure show that synthetic data is harder to train on, but leads to a smoother increase the test accuracy across epochs.

---

Published as a conference paper at ICLR 2026
Results From the results presented in Table 7 we note that the performances of Mantis and MOMENT can be almost matched by pre-training them on synthetic datasets that are ~ 20 and ~ 1.3x smaller than the original pre- pre-training datasets used by each of the papers. The accuracy drop in the case of Mantis is less than 0.1 %, while for MOMENT it barely exceeds 1 1%. This suggests that the synthetic data generated by CAUKER makes model pre-training more sample-efficiet. An additional evaluation on 17 datasets from real-world WOODS benchmark (Gagnon-Audet et al. 2023) comparing original Mantis and Mantis pre-trained on 100K CAUKER time series confirms this finding (see Appendix D Beyond UCR and WOODS, we further evaluate CAUKER pre-trained models on irregular, multivariate elinical benchmarks from (Li al.) 2023b), and find that they remain competitive with the original model in irregular settings as well, see Appendix N for details. We also note that the training loss and test accuracy of Mantis pre-trained on 100k and 1.89M time series exhibit very different behavior. For the synthetic dataset, the training loss remains higher, indicating that it is harder to Iearn, likely due to the high diversity of the generated time series. Yet, the test accuracy in this case steadily improves and surpasses the accuracy of the original model, which quickly learns the real-world pre-training dataset. This is reminiscent of the MOMENT pre-training, which only required 2 epochs (Goswami al. 2024 (even for the largest 783M) to converge.
In addition to this, the reported UCR classification accuracies of the original Mantis and MOMENT models represent in-distribution performance, since their respective training corpora include UCR train samples. In this sense, these scores may serve as a practical upper bound for zero-shot accuracy, beyond which -uitt--iistibuution generalization unlikely without direct exposure to test distributions. Finally, we note that the comparison with two other pre-training dataset candidates leads to strictly worse results.
Extension to forecasting. Although our primary contribution focuses on classification, we also experimented with applying CAUKER to forecasting TSFM. Interestingly, we observed that our pipeline transfers effectively to the forecasting setting well: Chronos models (tiny, mini, small and base) pre-trained exclusively on 0.5B timepoints CAUKER -eE-egeeeetedd series achieve zero-shot forecasting accuracy that is statistically indistinguishable from the original models pre-tra pre-trained on 84B tokens (p-value of 0.84 at the significance level of 0.05 for two-sided Wilcoxon signed rank test). We provide details and results in Appendix Importantly, this result is obtained without any task-specific modifications to the CAUKER pipeline.
5 CONCLUSION
In this work, we introduced CAUKER, a novel synthetic data generation framework tailored for time series classification. By integrating Gaussian Process kernel composition with Structural Causal Models, CAUKER generates synthetic datasets that are both temporally realistic and causally coherent. We demonstrated that TSFMs pre-trained solely on KER-generated data can match the performance of models trained larger real-world datasets. Furthermore, our study provides the first in-depth analysis of data and model scaling laws in zero-shot time series classification, establishing that such scaling effects emerge clearly when using synthetic data, but are irregular absent when training on commonly used real-world datasets.
Our findings underscore a key insight already known in vision and natural language processing: the quality and structure of pre-training data have a profound impact on the generalization performance of TSFMs. While much recent progress in time series community has focused on architectural innovations, our results suggest that equivalent gains can be achieved through principled design of synthetic training data. We hope this work encourages the community direct greater attention to the design, analysis, and benchmarking of time series training datasets, as a complementary path toward building scalable, general- purpose time series foundation models.
Limitations Similar to prior work on scaling laws in time series forecasting (Yao et al. 2025), we considered only two models that follow a different pre-training paradigm. As our study was already quite compute-intense, we believe that this choice is justified. In the same line, we didn't 't consider large-scale forecasting benchmarks such as Time-300B (Shi et al. (025) as we have observed that forecasting benchmarks are of limited utility for classification.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
Supported by EU project DataGEMS (101188416), and by YILAIOA & NextGenerationEU project HARSH (YTIОT A 0560901) that is carried out within the framework of the National Recovery
and Resilience Plan "Greece 2.0" with funding from the European Union - NextGenrrationEU. This work was granted access to the HPC resources of IDRIS under the allocation 2025-A019101264) made by GENCI.
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers, 2020. URL
https://arxiv htt////xww.orr/////255...00 00928
Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and
Doyen Sahoo. Gift-eval: A benchmark for general time series forecasting model evaluation. arxiv preprint Stxrv:2410.10392 2024.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sehastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang. Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Sohlke-SSchnidees and Yuyang Wang. Chronos: Learning the language time series, 2024. URL https: http///axvvvv.. wrw.iii.orrrooooooo abs 403.07815
Andreas Auer, Patrick Podest, Daniel Klotz, Sebastian Böck, Günter Klambauer, and Sepp Hochreiter. Tirex: Zero-shot forecasting across long and short horizons with enhanced in-context learning, 2025.
org abs 200.//255999999
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018, 2018
URL h httpss://wxvvvvoo../oo wttp://rrrrbb///...r.../. 00075
Sathya Kamesh Bhethanabhotla, Omar Swelam, Julien Siems, David Salinas, and Frank Hutter. Mamba4cast: Efficient zero-shot time series forecasting with state space models. arXiv preprint arXiv:2410.09385, 2024.
Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and radon wasser- stein barycenters of measures. J. Math. Imaging Vis., 51(1):22 45, January 2015. ISSN 0924-9907. doi: (0.0077/108511-11--5555
URL
010851-014-000-2
Quentin Bouniot, levgen Redko, Anton Mallasto, Charlotte Laclau, Oliver Struckmeier, Karol Arndt, Markus Heinonen, Ville Kyrki, and Samuel Kaski. From alexnet to transformers: Measuring the non-linyarity of deep neural networks with affine optimal transport. In CVPR, 2025. URL
aattps://arxinn owg.orgggaoo abs// 2210. 11439
Defu Cao, Furong Jia, Serean O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint 10.iv:2310.044445 2023.
Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Two-stage fine-tuning for time-series forecasting with preited 1lms. arXiv preprint arXiv:2308. 08469, 2023.
Ching Chang, Wei-Yao Wang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Aligning pre-trained Ilms as data-efficient time-series forecasters. ACM Trans. Intell. Syst. Technol. 16(3), April 2025, ISSN 2157-6904. doi: 10.1145/3719207. URL httpt:: / owt.oio.orrg101 http//11.1111111 11111111111000
Baiyu Chen, Wilson Wongso, Zechen Li, Yonchanok Khaokaew, Hao Xue, and Flora Salim. Comodo: Cross-modal video-to-imu distillation for efficient egocentric human activity recognition. arXiv preprint arXiv:2503. 07259, 2025a.
Mouxiang Chen, Lefei Shen, Zhuo Li, Xiaoyun Joy Wang, Jianling Sun, and Chenghao Liu. Visionts: Visual masked autoencoders are free-lunch zero-shot time series forecasters, 2025b. hthpp?
7/ rxiiv wwwpvv..orr//bbb//22000 1
11

---

Published as a conference paper at ICLR 2026
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyum Chen, Aakanksha Chowdhery, Sharan Narang. Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. Scaling Ceaructioo-finettndd language models, 2022. URL https: httppp:s//axxxv.ooo org/ htapg//bb///// 2210.2212022 11416
Ben Cohen, Emaad Khwaja, Kan Wang, Charles Masson, Elise Ramé, Youssef Doubli, and Othmane Abou-Amal. Toto: Time series optimized transformer for observability, 2024. URL https
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting, 2024. URL /ttp://aaxvvvvv... http://oggaababaabbbabbaa http::///aaxx...ooraa//////..... 10688
Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The ucr time series archive, 2019. URL pttttss:/azxiv.... org. wtt.:/ggg.bba.oooooooo 07758
Samuel Dooley, Gurnoor Singh Khurana, Chirag Mohapatra, Siddartha Naidu, and Colin White. Forecastpfn: Synthetically-trainee zero-shot forecasting, 2023. URL
abs/2311. 01933
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. URL
ozg aaaaab//20000 21129
Vasilii Feofanov, Songkang Wen, Marius Alonso, Romain Ilbert, Hongbo Guo, Malik Tiomoko, Lujia Pan, Jianfeng Zhang, and levgen Redko. Mantis: Lightweight calibrated foundation model for user-friendly time series classification. arXiv preprint arXiv:2502. 15637, 2025. URL https
//arxiv.orq/ htapz//abb//5502222 2502.15637
Jean- Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems, 32, 2019.
Christoohst Gagnon-Audet, Kartik Ahuja, Mohammad -avad Darvishi -aiisii Pooneh Mousavi, Guillaume Dumas, and Irina Rish. Woods: Benchmarks for out-of-distributio generalization in time series, 2023. URL
hbtp:/bs/2220 09978
Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitiik. Units: A unified multi-task time series model, 2024. URL (tLhttp:: htttps://axxi:
org abse//b///aa 2403. 00131
Théo Gnassounou, Antoine Collas, Rémi Flamary, and Alexandre Gramfort, Psdnorm: Test-time temporal normalization for deep learning on eeg signals. arXiv preprint arXiv:2503.04582. 2025.
Ary I. Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph Mietus, George B Moody, Chung- Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 101(23):e215-e22) e220, 2000.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: A family of open time-series foundation models, 2024. URL https: //tp://xxxvvoo/////
abs/2402. 03885
Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Advances Neurul Information Processing Systems, 36, 2024.
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A transformer that solves small tabular classification problems in a second, 2023. URL https: :ttttps:/aaee //arxiv org/abs 2207. 018488
12

---

Published as a conference paper at ICLR 2026
Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. The tabular foundation model tabPFN outperforms specialized time series forecasting models based on simple features. In NeurlPS 2024 Third Table Representation Learning Workshop, 2024. URL
forum? 1 d=H02X7R030C.
Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis, 2024. URL hwwp::ttpp///cc.......t http://aaxxvlggaa.e....o abs/ wttp:///0000088878
Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. A survey on contrastive self-supervised learning. Techmologies, 9(1):2, 2020.
Ming Jin, Shiyu Wang, Lintao Ma. Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yo Chen, Yux- uan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.0172, 2023.
Simon Korbllith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. ICMI,, volume 97, pp. 3519-3529. PMLR, 09-15 Jun 2019.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and serm temporal patterns with deep neural networks, 2018. /tll:/ttpe///axxx..rrr///////
07015
Tian Lan, Hao Duong Le, Jinbo Li, Wenjun He, Meng Wang, Chenghao Liu, and Chen Zhang. Towards foundation models for zero-shot time series anomaly detection: Leveraging synthetic data and relative context discrepancy, 2025. URL htt.:t/ss//aac.v..oorrbb////////
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, 2020. URL
/ttp://aaxvvvvvvm org/ abs http://aaa/1197000222
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffussio convolutional recurrent neural network: 926
Data-driven traffic forecasting, 2018. URL /tt.n/ttpss//axxx...or//////////
Zekun Li, Shiyang Li, and Xifeng Yan. Time series as images: Vision transformer for irregularly sampled time series. Advances in Neural Information Processing Systems, 36:49187-49204, 2023a.
Zekun Li, Shiyang Li, and Xifeng Yan. Time series as images: Vision transformer for irregularly sampled time series. In Advances Neural Information Prucessing Systems, pp. 49187-49204, 2023b.
Chenguo Lin, Xumeng Wen, Wei Cao, Congrui Huang, Jiang Bian, Stephen Lin, and Zhirong Wu. Nutime: Numerically melti-scaled embedding for large-scale time-series pretraining, 2024. URL
org/ abs/ 2310. 1...0.740000 07402
Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self- supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering, 35(1):55-8876 2023, doi: 10. 10111//KEDE2222130000888
Xu Liu, Juncheng Liu, Gerald Woo, Taha Aksu, Yuxuan Liang. Roger Zimmermann, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Moirai-moe; Empowering time series foundation models with sparse mixture of experts. arXiv preprint arXiv:2410. 10469, 2024a.
Xu Liu, Taha Aksu, Juncheng Liu, Qingsong Wen, Yuxuan Liang, Caiming Xiong. Silvio Savarese, Doyen Sahoo, Junnan Li, and Chenghao Liu. Empowering time series analysis with synthetic data: A survey and outlook in the era of foundation models, 2025. URL
abs/ 2bs/2566 bbs/2500..13 11411
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative prented transformers are large time series models, 2024b. URL https
// //rxiiv. www.eorg/aaaoo abs/ 2402. 02368
Yasuko Matsubara. Yasushi Sakurai. Willem G Van Panhuis. and Christos Faloutsos. Funnel: automatic mining of spatially coevolving epidemics. In Proceedings of the 20th ACM SIGKDD international conference Knowledge discovery and data mining, pp. 105-114, 2014.
13

---

Published as a conference paper at ICLR 2026
Pengrui Quan, Ozan Baris Mulayim, Liying Han, Dezhi Hong, Mario Berges, and Mani Srivastava. Reimagining time series foundation models: Metadata and state-space model perspectives. In NeurIPS Workshop Time Series in the Age of Large Models, 2024.
Stephan Rasp, Peter D. Dueben, Sebastian Scher, Jonathan A. Weyn, Soukayna Mouatadid, and Nils Thuerey. Weatherbench: A benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12(11), November 2020. ISSN 1942-2466. doi: 10.1029/ 2020ms002203. URL http:htt//cee doi http:///w..d.iir.1........ http://1..002/222000222222
Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Hena Ghonia, Rishika Bhagwatkar, Arian Khorasani, Mohammad Javad Darvishi Bayazi, George Adamopoulos, Roland Riachi, Nadhir Hassen, Marin Biloš, Sahil Garg, Anderson Schneider, Nicolas Chapados, Alexandre Drouin, Valentina Zantedeschi, Yuriy Nevmyvaka, and Irina Rish. Lag-Ilama: Towards foundation models for probabilistic time series forecasting, 2024.
Matthew A Reyna, Chris Josef, Salman Seyedi, Russell Jeter, Supreeth P Shashikumar, M Bran- don Westover, Ashish Sharma, Shamim Nemati, and Gari D Clifford. Early prediction sepsis from clinical data: the physionet/computing in cardiology challenge 2019. In 2019 Computing in Cardiology (CinC), pp. Page 1-Page 2019. doi: 10.2449/Cinn.20199.11
Simon Roschmann, Quentin Bouniot, Vasilii Feofanov, levgen Redko, and Zeynep Akata. Time series representations for classification lie hidden in pretrained vision transformers, 2025. URL
https://arxiie org/ abs 2506 08641
Hiroaki Sakoe and Seibi Chiba. Dynamic programming algorithm optimization for spoken word recognition. IEEE transactions on acoustics, speech, and signal processing, 26(1):43-44 1978.
Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time- moe: Billion-ssale time series foundation models with mixture of experts, 2025. URL, https
//a //aaxiiorr/aabs////.o.
1600
Gian Antonio Susto, Angelo Cenedese, and Matteo Terzi. Time-series classification methods: Review and applications to power systems data. Big data application in power systems, pp. 179-220, 2018.
Ege Onur Taga, M. Emrullah Ildiz, and Samet Oymak. Timepfn: Effective multivariate time series forecasting with synthetic data, 2025. URLhttps: htll:ttps:/aaxxllvoor//bbb///// 16294
Yihang Wang, Yuying Qiu, Peng Chen, Kai Zhao, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, and Chenjuan Guo. Rose: Register assisted general time series forecasting with decomposed frequency learning. arXiv preprint wxXiv:2405.17478 2024.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers, 2024. URL htt.:httts///eeee
org/abs//2 abs/ 2402. 02592
Hao Xue and Flora D Salim. Promptcast: A new prompt-based learning paradigm for time series forecasting, IEEE Transactions on Knowledge and Data Engineering, 2023.
Qingren Yao, Chao-Han Huck Yang, Renhe Jiang, Yuxuan Liang, Ming Jin, and Shirui Pan. Towards neural scaling laws for time series foundation models. In The Thirteenth International Conference on Learning Representations, 2025.
Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. Proceedings of the AAAI Conference on Artificial Intelligence, 3688) Jun. 2022. doi: 10.1609/aaai, www...3.60222
URL https: www.oassaaaaa.... org/ /t/ http:///g/...eeaaattte////////// htt.tii//ce/eeee........
Chaoning Zhang, Chenshuang Zhang, Junha Song. John Seon Keun Yi, Kang Zhang, and In So Kweon. A survey on masked autoencoder for self-supervised learning in vision and beyond. arXiv preprint NXX::2208000177 2022.
Haoran Zhang, Yong Liu, Yunzhong Qiu, Haixuan Liu, Zhongyi Pei, Jianmin Wang, and Mingsheng Long. Timesbert: bert-style foundation model for time series understanding, 2025. URL
https://arxivw www.:/owggg//oeeee
14

---

Published as a conference paper at ICLR 2026
Xiang Zhang, Marko Zeman, Theodoros Tsiligkaridis, and Marinka Zitnik. Graph-guided network for irregularly sampled multivariate time series. arXiv preprint arXiv:2110005357, 2021.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, 2021. URL
wttp/bs////aeo.oeet.////eeee btttps:/aaxxe org/ abs / 2012.001202 07436
Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Ilm. Advances in neural information processing systems, 11443322-433555 2023.
APPENDIX
The rest of this appendix is organized as follows. In Section A provide overview of pre-training datasets commonly used in TSFMs. In Section B we describe the contrastive and masked learning losses well architectural details of representative models Mantis and MOMENT. Section C introduces the CAUKER pipeline in detail, including pseudocode, kernel / mean activation banks, and hyperparameter sensitivity analysis. Section D presents additional qualitative analyses on global and local alignment between synthetic and real data. Experimental details for data scaling laws are provided Section E while model scaling experiments are discussed in Section F Further training and evaluation details are summarized in Section G Section
H reports domain-wise
UCR breakdown, and Section exxends evaluation to the WOODS benchmark. Section presenns results of pre-training Chronos on CAUKER and zero-shot forecasting, Visualization analyses of embeddings are discussed Section K Finally, we clarify the use Large Language Models for writing assistance in the last section.
A OVERVIEW OF TRE-TRAINING DATASETS FOR TIME SERIES FOUNDATION MODELS
Table 3
summarizes the pre-training datasets used by representative Time Series Foundation Models. For each model, we report whether synthetic data was used, the total number of time points and time series samples, whether the datasets are publicly available. The table is organized alphabetically by model name.
Model
Synthetic
Real
Time Points
Series Count
Open
Chronos (Ansari et al. 2024)
Yes
Yes
84B
890K
Yes
ForecastPFN (Dooley al.
2023
Yes
No
60M
300K
Yes
Mantis (Feofanov al. 2025
No
Yes
N/A
11.89M
Yes
MOMENT(Goswami et al. 2024
No
Yes
1.23B
13M
Yes
NuTime Lin al. 2024
No
Yes
60M
1.89M
Yes
TabPFN Hollmann al.
2023)
Yes
NO
N/A
9.216M
No
TimePFN
(Taga et al. 2025)
Yes
No
200M
~3M
Yes
UniTS Gao et 2024)
No
Yes
35M
6K
Yes
Table 3: Overview of pre-training datasets for Time Series Foundation Models (TSFMs).
B LOSs AND ARCHITECTURE OF MANTIS AND MOMENT
Contrastive learning loss of Mantis. Given an encoder F E R7, we consider random augmentations 6, U(T). The similarity between two augmented samples is measured after
projecting their embeddings to new dimension o' via g R2a I Specifically, the cosine similarity is defined as:
Soos (a, b)
a
V(a, e 1R?0.
|b]
15

---

Published as a conference paper at ICLR 2026
Given batch B fxx 1 1i=1 we compute pairwise similarities:
( v/) = [Scus (g a ((xi): g (xx)))))== E RR.
The Mantis encoder F and projector g optimized by minimizing the contrastive loss:
s=(0, +b)
coontrativee
Σ
Loc
T
where Ice is the cross-entropy loss and T is temperature parameter set to 0.1.
Masked learning loss of MOMENT. Given a univariate time series T TRAXT, segmented into N disjoint patches of length P. Each patch is mapped into D-dimensional embedding, replaced with learnable mask embedding [MASK] E R1xD for masked patches. The resulting embeddings fed into transformer encoder, producing transformed embeddings that then decoded by a lightweight reconstruction head hrce The masked loss for reconstruction is defined as the mean squared error (MSE):
Cmasked
Σ Tx (Fc((((M))))))
[22|
nEn
where Qi denotes the set of indices corresponding to masked patches.
Model architectures. For the masked learning approach, MOMENT leverages Transformer- architecture derived from the TS family (Chung et al., 20022 Specifically, MOMENT employs 8, 12, 24-layer Transformer encoder with hidden dimensions D) 512, 768, 1024, and 8. 12, 16 attention heads for "Small", "Base", "Large" model. The model processes input time series by segmenting them into N 64 patches of length P 8, applying positional embeddings, and then reconstructing masked patches.
al., 202T) architecture. Ini- tially, the input time series is divided into tokens, to which learnable class token is appended. Positional embeddings are added encode temporal information explicitly. The ViT unit consists of 6 transformer layers, each comprising multi-head attention with 8 heads. The final output is derived from the class token's embedding after aggregation by the transformer layers. worth noting that Mantis employs customized tokenizer. For detailed information, please refer to the original Paper"
Conversely, Mantis utilizes Vision Transformer (iT))Dosovitt
C DETAILS OF CAUKER
C.I PSEUDOCODE OF THE CAUKER
Algorithm Srrithmmmeene describes the full synthetic data generation process CAUKER. The pipeline combines the temporal structure modeled by Gaussian processes with the flexible dependency modeling of structural causal models. Specifically, the algorithm first samples number of root signals from GP priors constructed via randomly composed kernels and mean functions. It then propagates these signals through randomly generated DAG, where each edge applies a nonlinear transformation drawn from activation function bank. Finally, fixed number of node outputs are selected as observed time series variables, each interpolated to a target length. This modular and stochastic design ensures rich diversity and causal consistency in the generated synthetic data.
C.2 DETAILS OF BANKS
Figure X provides illustrative examples of the six representative kernels selected from our base kernel bank. The top row of the figure displays the covariance matrices induced by each kernel over 1024 evenly spaced time points, while the bottom row shows corresponding sample paths drawn from the Gaussian Process (GP) prior using these kernels.
Specifically, the illustrated kernels include:
'The updated number training samples (~~1.38M) is confirmed in the official repository: https:
github. com www.coo///////oo wttm://eeffaaee http://c///feeaaaaooa/aaaai////s htt////attaassssssssss////// The arXiy version initially reported ~7M.
16

---

Published as a conference paper at ICLR 2026
Algorithm I: CAUKER: Synthetic Fene-Series Generator for Classification
Input: N
// total number of samples to output
L;
// target length of each time series
// number of observed variables per sample 3 Banks K. (kernels), MM (mean fin), A (activations)
2 d:
Hyper-parameters: Kmax, Vmaxx mmax // random generator with fixed seed
// max kernels, nodes, parents
5 RNG:
Output: Dayn = {x1, (.1...., uN), n2 Rd
Function LE WWW.LLCCOOOOOOOORREEEEECC (K)
K f RNG.UniformInt Kma);
RNG.Choice(A3)
for i - 2 to K do
Kę < ENG.Choice(A)); op < RNG.Choice (( (+, x));
11
op Kg
return 13
14 Function SAMPLEMEA (M, x):
mi, mg + RNG.Choice(AN) (MM, size 2. replace True): 15
16
op + RNG.Choice((+,X) }});
17
return (p(m) m2(3));
18
D + 0;
19
while Deyal < do
V < RNG. formInt (d, Vina));
21
(V,E) RANDOMMAAG (V, P Pmax):
roots + fu E V deg (v) = 0}:
22
23
Σ RNG.Cho ce (A, size [8], replace True);
24
map each E & uniquely to an activation o. E E; foreach r E roots do
25
26
Kr + SAMPLBCCMPOSITEKSS..
(K));
27
p.()) SAMMLLEMAN (M, -);
4 ~ GP(ur, ki)) on grid [0, 1] of length L;
28
29
foreach E TopoSort(G) do
30
if v E roots then
31
continue
P. (u I (u) E E);
Concat (IuNJEP.));
34
W ~ (O0,
b~ N~, 1);
tu + (u.v) (Wz+b)
35
V' + RNG.Choice (V, size=d, Faplacc xt- stack (aack(ftoloev))
36
37
Dayn + Dayn J {x}:
38 return D Dayn
ExpSineSquared captures periodic patterns with a fixed wavelength; produces strongly oscillatory samples with global smoothness.
DotProduct induces linear trend behavior; sample paths grow or decay steadily over time.
RBF (Radial Basis Function) - generates smooth, localized fluctuations around zero with short-range correlations.
RationalQuadratic- - a scale mixture of RBF kernels, allowing for multiscale smooth variations the signal.
WhiteKernel - models uncorrelated noise; sample paths resemble pure Gaussian noise with no temporal structure.
17

---

Published as a conference paper at ICLR 2026
ConstantKernel generates flat constant signals; serves component for additive models with nonzero mean.
These six kernels represent only small subset of our full kernel bank. In practice, we construct much larger kernel bank comprising 36 distinct kernels. This is achieved by varying the hyperparameters of each kernel (e.g., length-scale, periodicity, noise level, amplitude) across range scales to capture diverse temporal dynamics. For instance, we use multiple versions of the ExpSineSquared kernel with different periodicities to simulate both high- and low-frequency periodic patterns. Similarly, we vary the length-scale of RBF and RationalQuadratto kernels to control smoothness and correlation range. An important point is that CAUKER randomly samples small number of kernels from the bank for each composite GP, enlarging or slightly modifying the bank does not increase computational cost.
Kernel
Kernel 2
Kernel 3
Kernel 4
Kernel 5
Kernel 6
Figure 8: Visualizations of covariance matrices (top) and corresponding sampled time series (bottom) from each base kernel in the kernel bank.
The images presented in Figure as illustrative examples only. During synthetic data generation, kernels are sampled from the full kernel bank, which offers significantly richer diversity than what is shown here. These base kernels are subsequently composed using random additive and multiplicative operations to define flexible Gaussian process priors for root node generation in the CAUKER pipeline.
Figure 9 presents the four representative mean functions used in synthetic data generation pipeline. Each subplot illustrates a randomly sampled instance from the corresponding function class. These functions be combined multiplicatively additively during Gaussian process sampling enrich the diversity generated signals.
Zero Mcan: bascline function returning a constant zero across the time axis, corresponding to the standard GP assumption with zero-centered priors.
Linear Mean: A simple affine transformation a-t++,, enabling trends such monotonic increases or decreases over time.
Exponential Mean: A parametric form exp(btt that introduces strong, nonlinear growth decay patterns into the signal.
Sparse Anomalies: piecewise-constant mean vector with a few randomly placed spikes, simulating rare disruptive events (e.g., faults, attacks, regime shifts).
These mean functions serve as building blocks for composing realistic non-stationary temporal structures in synthetic time series. the generation process, two functions are randomly selected and combined (either by summation elementwise multiplication), forming the final mean vector used GP sampling. The images shown in Figure D are illustrative samples; practice, stochastic variation over parameters (slopes, amplitudes, etc.) ensures that each generated series presents unique mean behavior.
Activation function bank. In addition to kernel and mean banks, CAUKER employs diverse activation function bank AA to propagate nonlinear transformations through the structural causal graph. Each edge in the DAG is randomly assigned activation from this bank, which governs how parent node values influence their children. The activation bank comprises both classical and domain-specific transformations:
Linear: Identity affine mappings arr b, preserving proportional signal propagation.
18

---

Published as a conference paper at ICLR 2026
2.50
2.25
6.02
2-
Heen
0.00
0.5
-0.02
-1
-2-
1.00
0.00
0.25
3.50
0.75
1.40
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
Time
1ime
Time
Time
Figure 9: Examples of four mean function types used in the synthetic data pipeline. Each function introduces distinct temporal structure, contributing to the diversity and realism of generated sequences.
ReLU: Rectified linear units max(0, .0), introducing sparsity and piecewise linearity.
Sigmoid: Smooth squashing function (zx) 1/(1 e"), modeling saturation effects.
Sinusoidal: Periodic modulations sin(z), inducing wave-like behaviors.
Modulo: Modular transformations mod c, yielding abrupt nonlinearities or periodic clipping.
Leaky ReLU: llope-preeerving variant of ReLU, ensuring non-zero gradients for negative inputs.
These nonlinearities enhance the diversity of functional relationships within the generated synthetic time series and allow the resulting signals to exhibit complex, structured dependencies. As illustrated in the SCM pipeline, these functions applied edge-wise to linear combinations of parent signals before assigning values to child nodes.
C.3 HYPERPARAMETER SENSITIVITY OF CAUKER
We report an ablation on two families of CAUKER hyperparameters that control the generative complexity of synthetic series: (i) co-sweep where we simultaneously increase the number of randomly sampled kernels used the composite GP prior ("Kernll k") and the maximum number of parents per node in the DAG ("Parent p"), and (ii) the graph size (number of nodes) of the DAG. F
Observations. Along the Kernel/Parents co-sweep, the data (shown in Distatistics exhibit consistent trends: Entropy, Stability, and Lumpiness increase steadily, while the Hurst decreases, indicating more heterogeneous and less persistent series as complexity grows. Despite these changes in data characteristics, the downstream UCR Accuracy remains essentially stable (fluctuations within 0.4%). When varying the graph size, the method is likewise insensitive (shown 5) the UCR accuracy varies within a narrow band, while the data statistics change only mildly. Overall, CAUKER robust to these generative hyperparameters.
Failure modes To probe failure modes, we force CAUKER to use only a single kernel family (plus SCM). DotProduct-only GPs, which produce almost linear trends, lead substantially worse performance (76. 79% UCR accuracy). In contrast, RBF-only sampling remains competitive (78.07%). underscoring the importance sufficiently rich nonlinear structure and kernel diversity.
D ADDITIONAL QUALITATIVE ANALYSIS
To complement the qualitative discussion we quantify how well synthetic corpora produced by different generators resemble the target UCR distribution, both globally and locally. Concretely, we report:
'Entropy, Hurst. Stability, and Lumpiness are computed with the same definitions in Aksu et al. 2024) higher Stability and I lampiness indicate stronger regime structurc/hettrrgeenctty while larger Hurst indicates persistent (long-memory) behavior.
19

---

Published as a conference paper at ICLR 2026
Table 4: Kernel/Parents co-sweep.
Increasing both the number of sampled kernels in the GP composition and the maximum number of parents node produces steadily higher En- trppyStablliyyILuminnss and a decreasing Hurst, while UCR accuracy stays stable.
Kernel Parents
Entropy
Hurst
Stability
Lumpiness
UCR Acc.
Kernel3 Parent2
0.4629
0.7719
0.9821
145.18
0.7848
Kernel4 Parent3
0.5034
0.7713
1.7949
1081.17
0.7854
Kernel5 Parent4
0,5352
0.7686
2.8164
6348.75
0.7850
Kerne16 Parent5
0.5522
0.7655
4.5419
1854961.61
0.7825
Kerne77 Parent6
0.6225
0.7519
11.7237
10148441.78
0.7810
Table 5: Graph size sweep. CAUKER insensitive to DAG size.
Graph Size
Entropy
Hurst
Stability
Lumpiness
UCR Acc.
10
0.5273
0.7707
2.0424
2314.14
0.7848
20
0.5716
0.7664
3.7253
1142.36
0.7811
30
0.5714
0.7665
2.1988
1252.20
0.7812
40
0.5818
0.7630
2.1085
836.93
0.7815
50
0.5889
0.7654
2.3601
50469.86
0.7785
Global proximity via the Sliced Wasserstein distance (SWD). Given empirical distributions P and Q over R2, the -order SWD is
SWD2(P,Q) Q) EB Wx ())),
where W2 is the one-dimensional Wasserstein distance between the projected marginals, and sampled uniformly the unit sphere.
Local alignment via CKNNA (classwise b-nearest-neighhour alignment) following Huh et al. (2024). Let f be frozen encoder and let S (source, synthetic pre-training) and T (target, UCR) denote labelled sets. For each x E T with label ya. we form the k-NN set Na. (2; ((S) under cosine distance in feature space and define
CKNNA&
Σ1 #T
Σ
TV= = ys].
IT K
ZEN (a:f(a)
then average over datasets.
For each UCR dataset, we compare it to five independent synthetic draws produced by (i) KER- NELSYNTH and (ii) CAUKER, using the same generator hyperparameter priors as in $4.1 Global proximity is computed by averaging SWD, over 512 random one mee-iimenioonll projections per dataset; local alignment uses features from frozen encoder and classwise k-NN agreement as above. We report mean + standard deviation across the five synthetic draws and then average across UCR datasets.
Table 6: Global and local alignment between UCR and synthetic corpora. Lower better for SWD; higher is better for CKNNA. Means + s.d. across five independent synthetic draws, then averaged over UCR datasets.
KernelSynth
CAUKER
Global SWD2
7.11 (+ 1.13)
3.1486 ( 0.2))
CKNNA (avg. over datasets)
0.014 (+ 0.03)
0.015 (+ 0.08)
20

---

Published as a conference paper at ICLR 2026
Findings. CAUKER achieves substantially smaller global discrepancy UCR than KERNELSYNTH (Table 6 SWD2), indicating a closer match at the dataset-level distribution. At the same time, its (slightly) higher CKNNA suggests at least comparable- and typically classr-classwise local neighbourhood structure transfer from pre-training to UCR. Together, these results support the view that the SCM backbone in CAUKER is crucial for shaping both global statisties and local class geometry in ways that benefit zero-shot classification.
E EXPERIMENTAL DETAILS OF SECTION 4.2.1
our scaling law experiments, we systematically evaluated the performance of two distinct models, Mantis and MOMENT, across varying dataset sizes from both real-world and synthetic sources. We adopted the official 8M parameters configuration of Mantis as released in its open-source repository, which includes 6-layer ViT encoder with 8 attention heads and hidden dimension of 256. The classification head used was a Random Forest classifier trained on frozen embeddings.
For MOMENT, used the officially supported parameters as the encoder backbone. This model structure one the pre-trained configurations endorsed in the original MOMENT framework. During training, froze the encoder and trained only the classification head, which was implemented as a Support Vector Machine (SVM). This setup mirrors the zero-shot classification evaluation protocol used prior TSFM literature.
(tggl//faantt--mmalll variant containing 77M
For both models, we varied the training data sizes as follows: for the real-world UEA dataset, subsets ranging from 0.1% % to 100% (12.7K 12.67M samples) were randomly sampled. For synthetic data, we generated samples using our CAUKER method at 10K. 50K, 100K, 500K, 1M, 5M, and 10M scales. All series were univariate with length 512. The full list of data sizes and corresponding classification accuracy values on the UCR benchmark are reported in Table
Model
Train Set
Data Size
UCR Accuracy (%)
UEA
127K
72.42
UEA
1.27M
70.49
MOMENT (77M)
UEA
633K
71.09
UEA
6.33M
72.09
UEA
12.67M
72.10
CAUKER
100K
74.24
CAUKER
500K
74.35
CAUKER
1M
75.21
CAUKER
5M
77.01
CAUKER
10M
77.49
UEA
12.7K
75.67
UEA
127K
76.21
UEA
633K
75.83
Mantis (8M)
UEA
1.27M
75.39
UEA
3.68M
76.33
UEA
12.67M
71.93
CAUKER
10K
76.91
CAUKER
50K
78.08
CAUKER
100K
78.55
CAUKER
1M
78.91
CAUKER
10M
79.09
Table 7: Exact accuracy values used in the scaling law plots (Figure
21

---

Published as a conference paper at ICLR 2026
F EXPERIMENTAL DETAILS OF SECTION 4.2.2
To investigate model scaling laws, we evaluated a range of model capacities for both MOMENT and Mantis using synthetic datasets generated by CAUKER. For MOMENT, we adopted the official series of models given by:
flan-t5-small (77M parameters),
flan-t5-base (248M parameters),
flan-t5-large (783M parameters).
For the Mantis encoder, we varied the transformer depth and width while keeping the sequence length fixed at 512 and using the same patching configuration. The model variants follows:
0.75M: hidden dim=256,
transf depth=1,
transf num heads=2,
transf mlp dim=512, transf dim head=128.
2.59M: same as above, with
8.10M: same as above, with
28.56M: same as above, with
114.14M: hidden.dim-512, transf.depthh12, Grranffmlp.dime
transf.num.heads
transf.dim.heece
All Mantis variants used the following fixed parameters: seq_len=512, num.patches-32. sca lar_aaarallllla ssscallssnnne,,, weateendii.ccaaa..e. The model output embeddings were classified using Random Forest classifier trained frozen features.
and eps www.scallr.ennn.e
This design allows to jointly assess the impact of model depth, width, and hidden dimensionality on zero-shot classification performance under a consistent synthetic data regime.
Table reports the exact accuracy values corresponding the model scaling plots shown in Figure TO For both MOMENT and Mantis, we list results under varying model sizes and dataset configurations.
Model Size
UEA 1%
UEA 10%
UEA 100%
CAUKER 100K
CAUKER 1M
CAUKER 10M
77M (MOMENT)
72.42
70.49
72.10
74.24
75.21
77.49
248M (MOMENT)
68.62
66.91
69.01
75.16
76.16
77.51
783M (MOMENT)
64.85
64.18
66.07
77.28
77.20
77.85
0.75M (Mantis)
73.25
72.81
72.77
75.10
75.67
76.44
2.59M (Mantis)
75.87
75.12
75.73
77.74
78.22
78.30
8.10M (Mantis)
76.36
75.44
72.03
78.06
78.91
79.09
28.56M (Mantis)
76.66
77.05
78.70
78.83
78.19
114.14M (Mantis) 76.60
77.29
76.97
78.42
78.86
78.81
Table 8: Exact zero-shot accuracy (%) on the UCR benchmark under different model sizes and pre-training dataset configurations.
G EXPERIMENTAL DETAILS OF SECTION 4.4
For all compared models, we adopted the best training loss epoch as the checkpoint final evaluation. Specifically, the official setting for Mantis involves training for 100 epochs, while MOMENT is typically trained for epochs. However, for our experiments, we trained Mantis for 100 epochs and MOMENT for 10 epochs to allow sufficient convergence, consistent with goal of achieving the best performance on the CAUKER and UEA datasets. For the MOMENT model, we utilized the base model Gooolle/faan-1--small with 77M parameters, trained on both the CAUKER and UEA datasets. The official MOMENT checkpoint used in our experiments (Time Series Pile), "google-t5/t5-sme -mall", has 60M parameters.
22

---

Published as a conference paper at ICLR 2026
0.79
0.73
USA
Crate
2.50
114
Model Sizc (Prameterr
Figure 10: Accuracy on UCR dataset with varying model sizes for the Mantis model trained on UEA
subsets and synthetic CAUKER data.
H DOMAIN-WISE ANALYSIS ON UCR
To assess whether the gains observed with CAUKER pre-training are uniform across application areas, we group the UCR datasets (Dau et al. 2019) by the standard Type taxonomy (e.g., ECG, Motion, Spectro, etc.) and report domain-wisg wise averages zero-shot accuracies. We compare Mantis encoder pre- pre-trained on 100K CAUKER synthetic series (CauKer100K) against (ii) the official Mantis checkpoint released by the authors For each domain we also report the absolute difference A = CauKer100f - Official.
Table 9: Domain-wise UCR accuracy (mean across datasets within each Type). A CauKer100K Official.
Type
CauKer100K
Official
A
Device
0.7209
0.7288
0.0079
ECG
0.8539
0,8271
+0.0268
EOG
0.5000
0.5304
0.0304
EPG
0.9980
1.0000
0.0020
HRM
0.8602
0.8387
+0.0215
Hemodynamics
0.7131
0.7179
-0.0048
Image
0.7798
0.7910
- -0.0122
Motion
0,7883
0.7873
+0.0010
Power
0.9667
0.9056
+0.0611
Sensor
0.7823
0.7913
- -0.091
Simulated
0.9434
0.9355
+ +0.0078
Spectro
0.7226
0.7775
- 0.0550
Spectrum
0.8126
0.7961
+0.0165
Traffic
0.9120
0.8869
+ +0.0251
Trajectory 0.5385
0.5282
+0.0103
Results and discussion Eight out of fifteen domains exhibit higher accuracy with CAU pre- training (Table 9. The only domain showing a substantial degradation (greater than 5%) Spectro (-5.50%). A plausible explanation is the prevalence of very small spectroscopy datasets (often < 50 samples), which amplifies the in-distribution advantage of the Official Mantis model that was exposed to UCR training splits during pre-training.
*The official Mantis checkpoint pre-trained corpus that includes the UCR training splits; thus, its reported constitute in-distribution performance with potential advantage on small domains.
23

---

Published as a conference paper at ICLR 2026
Table 10: WOODS summary (domain averages over constituent datasets) and overall statistics.
ERM: supervised baseline from Gagnon-Auder et al. (2023). Mantis-2M: original real-data pre- trained encoder (~~1.9MM series). CauKerl00K: the same architecture pre-trained on 100K CAUKER samples.
Domain Statistic
ERM
CauKer100K
Mantis-2M
CAP (EEG)
0.750
0.782
0.760
HAR
0.934
0.946
0.940
MI (EEG)
0.733
0.563
0.543
SEDFx (EEG)
0.7225
0.7700
0.7375
Win counts (out of 17; ties counted)
7
11
4
Average all 17 datasets
0.800
0.820
0.810
Conversely, the largest improvement observed the Power domain (+6.111%), consistent with the presence of strong periodic and quasi-seasonal motifs in power consumption profiles that are well captured by the Gaussian- process kernel composition within the CAUKER pipeline. Across the remaining thirteen domains, the performance gap remains modest (within +3%), indicating that CAUKER-based pre-training transfers robustly across diverse application areas despite using only 100K synthetic series.
cruussian-proeess
These findings support our main claim: causally structured, kernel-compont composse synthetic pre-training yields competitive (and sometimes superior) representations for zero-shot classification across het- erogeneous time-series domains, while the rare large deficit (here, Spectro) is consistent with small-sample regimes where prior exposure to the same datasets can unduly benefit in-distribution baselines.
I SUPPLEMENTARY EVALUATION ON WOODS
To further substantiate the claim in Section 4.4 that CAUKER enables sample-efficient pre-training for zero-shot classification, we evaluate on the WOODS benchmark (Gagnon-Audet al., 2023). which targets out out-of-die of-distribttion (OOD) generalization in time-series tasks. We include 17 datasets from WOODS; notably, 12 of them (CAP, SEDFx, and MI families) are EEG- EEG-based, thereby probing domain that is distributionally distant from the UCR suite used in our main evaluation. We compare three contenders:
ERM (supervised baseline). The carefully designed empirical risk minimization pipeline used in the original WOODS paper (Gagnon-Aude et al. (202)).
2. Mantis-2M (real-data pre-training). The original Mantis encoder (Feofanov et al. 2025 pre-trained on ~.899M real-world series.
3. CauKer 100K (synthetic pre-training). Mantis encoder pre-trained from scratch on only 100 CAUKER samples (Section 3); all other components and evaluation protocol are kept identical Mantis-2M.
Following zero-shot protocol, we freeze the encoder and train lightweight classifier top of embeddings using only the in- bebeenhhark training split of each WOODS dataset. Unless otherwise stated, report standard classification accuracy (averaged over the official splits) and highlight the best result per (sub)domain bold.
Results Table [000eeoummmiizze domain-level averages (aggregated over constituent datasets) and over- all statistics. Despite using roughly 20x fewer pre-training samples than Mantis-2M, CauKerl00K attains the strongest average performance and secures 12 wins (including ties) out of datasets. The advantage is pronounced on EEG- EEG-heavy families (CAP, SEDFx), while MI favors the supervised ERM baseline, suggesting that certain EEG sub-tasks may still benefit from label-rich supervised specialization. On HAR (human activity recognition), CAUKER again edges out both alternatives.
24

---

Published as a conference paper at ICLR 2026
These results corroborate central message: CAUKER pre-training yields robust and transferable representations that generalize beyond the UCR distribution, even to EEG-centric WOODS tasks. Crucially, this benefit is achieved with order of magnitude fewer pre-training samples than the real- data corpus used by Mantis-2M, reinforcing the sample-efficiency synthetic, causally structured time-series generation.
J PRE-TRAINING CHRONOS ON CAUKER AND ZERO-SHOT EVALUATION
To assess whether the proposed synthetic pipeline CAUKER also transfers to forecasting pre-training, we trained Chronos models from scratch on purely synthetic corpus of
N 1,000,000 univariate series of length
L == 512,
which amounts to approximately N x L 0.512B time points ("observations"). We abbreviate this setting as CauKer I M in the results table for consistency with our internal logs. We evaluate zero-shot manner the be disjoint from the official Chronos pre-training mixture.
benchmark, comprising 27 subsets explicitly curated to
Metric Mean Absolute Scaled Error We report Mean Absolute Scaled Error (MASE), scale-free metric where lower is better. For a seasonal period m and forecast horizon H, with ground truth [ 1T- and predictions ( MASE
T+H Σ
1
Del
MASE
(I)
H
2izm+1l0 Vi-ml
T-m
Under this normalization, the seasonal naive forecaster attains MASE P by construction, providing a meaningful baseline across heterogeneous series.
Results. Table contrasts zero-shot MASE for official Chronos checkpoints (trained on an -bs-evvsrvation mixture) against models pre-trained only on CAUKER. Despite using an order of magnitude fewer observations and no real data, CAUKER pre-training yields competitive zero-shot accuracy across Chronos model scales, and substantially outperforms Seasonal Naive baseline.
Table 11: Zero-shot forecasting the chronos-zero-stee suite (27 non-overlapping subsets). Lower MASE is better. CauKerl M denotes pre-training on 1M sequences length 512.
Model Type
Training Data
pre-training Data Size
MASE
Chronos Tiny
Official
84B Observations
0.87
MauKerIM
0.5B Observations
0.89
Chronos Mini
Official
84B Observations
0.84
CauKerIM
0.5B Observations
0.87
Chronos Small
Official
84B Observations
0.83
C
0.5B Observations
0.86
Chronos Base
Official
84B Observations
0.81
CauKerIM
0.5B Observations
0.83
Seasonal Naive
1,0000
'The label MauKerIM follows our internal shorthand and denotes the run with 1M sequences ( 0.512B observations).
25

---

Published as a conference paper at ICLR 2026
(a) Frequency Analysis
(b) Slope Analysis
Camboine Amayya
(c) Bias Analysis
(d) Combined Analysis
Figure 11: UMAP projections of embeddings produced by the CauKer pre- pre-trained encoder. Colour encodes the generating parameter for each synthetic class (green frequency, blue slope, red bias).
Discussion. Despite pre-training on as 160x fewer observations (~ 0.5B vs. 84B), the Chronos models trained on CAUKER lag the official checkpoints by only = 2-3% in zero-shot MASE. indicating that (i) CAUKER supplies sufficiently rich temporal structure for large models leverage in zero-shot regime, and (ii) purely synthetic corpus- drrpuuedesigne originally for classification can still endow forecasting FMs with strong generalization, despite the absence any real-world pre- training data. This aligns with our broader finding that principled synthetic design act effective substitute for large, curated real datasets when coupled with appropriate inductive biases and scale.
K VISUALIZATION OF EMBEDDINGS
We generated univariate time series of length L. 512 using the CAUKER pipeline. For the frequency class, 20 periodic kernels with periods evenly spaced in [50, 500) were used. For the slope class, we sampled slopes in [0.1, 10.0], and for the bias class, biases were drawn from [-5, 5]. Each parameter setting was instantiated 30 times to ensure balanced coverage across the range. We use Mantis 8M trained on 10M CauKer data to encode the time series.
The UMAP projections reveal that the encoder learned structured and disentangled representations:
the frequency, slope, and bias views (Figures) aa I1c,. we observe continuous colour gradients along one principal direction of the embedding, confirming that the encoder preserves the underlying generative factor in a smooth and ordered fashion.
In the combined view (Figure lid) embeddings from the three generation processes form distinct clusters with minimal overlap, indicating that the encoder effectively disentangles the semantic attributes of each synthetic category.
26

---

Published as a conference paper at ICLR 2026
Cautfer LEA LCR
UCK
2nd Principal Component
2nd Principal Componant
1st Principal Component
1st Principal Component
Tleax 10K.
(b) Ybmax 100K.
2nd Principal Componant
1st Principal Component
(c) 7lmax 1M.
Figure 12: PCA-visualization of Mantis embeddings for samples from UCR, UEA and CauKer- generated data. For each plot, we randomly select min Mssmpies: 7mmax) samples for each dataset, where Nsampless the dataset size and Remax {10K, 100K, 1M).
The alignment of UMAP geometry with the known generative parameters supports the conclusion that the model did not merely memorize waveform patterns, but instead internalized semantically meaningful features of the data.
These results confirm that synthetic pre-training on CAUKER enables the encoder learn robust, interpretable, and transferable representations even in the absence of real data.
Next, we evaluate the diversity of generated samples by comparing them real benchmarks such as UCR and UEA. To this end, we visualize embeddings of time series samples using PCA projection onto the first two principal components. We use the original pre-trained Mantis as the encoder and compute the PCA on the concatenation of million CauKer-generate The results are shown in Figure 12] for each plot, we matched the number UEA and CauKer samples to enable fair comparison of data distributions. As can be seen, CauKer generates more diverse samples, spanning the embedding space more uniformly. This broader coverage may facilitate pre-training by encouraging the learning of more generalizable features. This is consistent with empirical findings: CauKer data achieves comparable performance to the original Mantis pre-training dataset (which includes UCR) UCR (Table 9 and outperforms it the WOODS benchmark (Table T0
samples, UCR and UEA data.
L ATTENTION ROLLOUT ANALYSIS.
To further compare representations learned from synthetic and real data, apply the Attention Rollout (Abnar & Zuidemal 2020) to Mantis pre-trained on CAUKER KOUKKE--OOKKK and to the original
Mantis checkpoint trained 1.89M real-world corpora. As illustrated in B for randomly selected
27

---

Published as a conference paper at ICLR 2026
Libol +
Latel m
Label -
07-
(a) ECG, CAUKER- 100K
Alantco 1ou
Label
23-
27-
255
WIN
34-
0
200
400
250
400
200
400
200
455
20c
(b) ECG, real-data Mantis
Atantco
Inpu time
LAON
Label
Label
-2
Label
2.8
-80
-00
-330
-3.
(c) Fish, CAUKER-IO0K
Lobel
Libel w
Labele
Labe
23-
20
-85
-03
-35
-55
465
3.5
200
230
<le
206
400
alo
400
250
(d) Fish, real-data Mantis
Figure 13: Attention Rollout on UCR ECG and FISH samples.
representative examples from the UCR ECG and UCR Fish datasets, the CAUKER pretrained model exhibits noticeably sharper and more localized attention maps: its aggregated attention mass concentrates short subsequences containing visually salient, class-discrniinative patterns, whereas the original Mantis tends to distribute attention more diffusely along the series. This suggests that the causal structure and diversity of CAUKER encourage TSFMs to focus more strongly on discriminative temporal segments.
M ADDITIONAL EXPERIMENTS ON DOWNSTREAM FINE-TUNING
To complement the zero-shot evaluation in the main paper, we also study downstream fine-tuning of Mantis on UCR. Specifically, we follow the default fine-tuning pipeline provided in the original Mantis implementation (same optimizer, schedule, data splits. and classifier head), and compare three pre-training configurations: (i) the original Mantis model trained on the real-data corpus of 1.89M series, and (ii) two models pre-trained on CAUKER synthetic data with 100K and 1M series, respectively. Table 12 summarizes the resulting UCR test accuracies. We observe that all configurations benefit from supervised fine-tuning compared to their zero-shot counterparts (Table 7). and that increasing the size of the CAU corpus from 100K to 1M substantially narrows the performance gap to the original real-data model. This indicates that CAUKER pretrained models not only provide strong zero-shot representations, but also serve as competitive initialization for downstream supervised adaptation.
KAUKERR-retrained
28

---

Published as a conference paper at ICLR 2026
Table 12:
Downstream fine-tuning accuracy on UCR for Mantis using the default fine-tuning pipeline. All models are pre-trained with the indicated corpus and then fine-tuned under identical settings.
Pre-training corpus
pre- train series
Test aceuracy
Original Mantis real-data corpus
1.89M
0.8496
KER
100K
0.8291
CAUKER
IM
0.8457
Table 13: Zero-shot-style evaluation on irregular, multivariate clinical benchmarks. We compare the original Mantis encoder with Mantis pre-trained MAUKER-IO0K and CAUKER-IM.
Dataset
Model
AUROC
AUPRC
P12
Mantis (real-data)
0.8121
0.4340
Mantis (CAUKER-I00K)
0.7984
0,4276
Mantis (CAUKER-IM)
0.8189
0,4592
P19
Mantis (real-data)
0.8846
0.5368
Mantis (CAUKER-100K)
0.8534
0.4954
Mantis (CAUKER-IM)
0.8709
0.5005
N SUPPLEMENTARY EVALUATION ON IRREGULAR TIME SERIES
To assess whether CAUKER pre-trained encoders generalize beyond regularly sampled, fixed fixed-length benchmarks such as UCR and WOODS, we additionally evaluate Mantis on two irregular, multivariate clinical datasets, P12 (Goldberger al. 2000) and P19 (Reyna al. 2019 Both datasets consist of sparsely and irregularly sampled physiological measurements with highly imbalanced binary labels (Li et al. 2023a Zhang al. 2021).
We follow the same frozen-encoder, zero-shot-style protocol as in the main experiments: the Mantis encoder is either the original real data pre-trained checkpoint, pre-trained from scratch on CAUKER with 100K or IM synthetic series. As standard in this setting, we report AUROC and AUPRC on the held-out test split. The results in Table show that CAUKER-generatee data outperforms the original Mantis encoder on P12, whereas on P19 the CAUKER- 1M model achieves an AUROC of 0.8709 and an AUPRC of 0.5005 compared 0.8846 and 0.5368 for the real-data pretrained baseline.
O USE OF LARGE LANGUAGE MODELS
Large Language Models (LLMs) were used as a general-purposs writing assistance tool during the preparation of this manuscript. Specifically, LLMs were employed for language refinement to improve the clarity, grammar, and style of technical writing while preserving the original scientific content and authorial voice. The LLMs did not contribute to research ideation, experimental design, data analysis, or the formulation of scientific conclusions. All technical innovations, methodological contributions, experimental results, and scientific insights presented in this work are entirely the intellectual product of the human authors. The authors take full responsibility for all content, including any portions refined with LLM assistance, and have verified the accuracy and appropriateness of all information presented.
29


Reviews:
[{"review_id": "Vwi5qXQTFe", "reviewer": "Reviewer_Ztoq", "summary": "This paper proposes CAUKER, a synthetic data generation framework combining Gaussian Process kernel composition and SCM for time series foundation models for classification tasks. Unlike most prior work focusing on forecasting, CAUKER targets classification and demonstrates that synthetic pretraining can yield competitive or superior performance to real world datasets. It also reveals scaling laws for synthetic pretraining in terms of dataset and model size.", "questions": "* How does CAUKER handle multivariate dependencies beyond univariate channel concatenation?\n\n* Can CAUKER generalize to forecasting or imputation pretraining tasks?\n\n* How computationally expensive is CAUKER compared to kernel only methods?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "* Addresses a clear gap, synthetic pretraining for classification TSFMs.\n* The causal kernel composition is conceptually elegant and well motivated.\n* Benchmarks across multiple models and datasets .\n* Includes scaling law analyses for data, model, and compute.\n* Outperforms real-data pretraining in several zero-shot setups.\n* The method is explained clearly, with schematic diagrams and pseudocode.", "weaknesses": "* Both GP based and SCM based data generation already exist, the novelty lies mostly in combining them.\n* Evaluation confined to zero-shot classification. would benefit from downstream fine-tuning or transfer learning results.\n* The contribution of causal graph depth/branching remains unclear.\n* While interesting, the scaling analysis is somewhat descriptive without deeper theoretical grounding", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pN6JZqUcFn", "reviewer": "Reviewer_uHPf", "summary": "The manuscript presents CAUKER, a synthetic data generation pipeline for pretraining classification time-series foundation models. CAUKER composes Gaussian-process kernels and mean functions within a structural causal model (SCM) graph, producing causally coherent sequences for self-supervised pretraining of contrastive (Mantis) and masked-reconstruction (MOMENT) encoders. Empirically, models pretrained solely on CAUKER data achieve competitive zero-shot accuracy on UCR and exhibit monotonic scaling with both dataset size and model capacity.", "questions": "1. What are the concrete, theoretically grounded challenges when porting forecasting-oriented synthetic pipelines to classification (label generation, class balance, inter-class separability, invariance desiderata), and how does each CAUKER design choice mitigate them?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Integrating kernel composition with SCM-based propagation yields diverse dynamics and inter-series dependencies aligned with classification objectives.\n2. Evaluation across contrastive and masked-reconstruction pretraining objectives increases the generality and external validity of the findings.\n3. Experiments demonstrate data/model scaling laws and strong zero-shot transfer, offering a compelling empirical performance.", "weaknesses": "1. Pretraining on pure synthetic data and obtaining strong results is not particularly surprising, as prior work (e.g., TabPFN-TS) has already demonstrated the potential of synthetic data. This manuscript would benefit from sharper positioning of what is substantively novel in methodology part. \n2. This paper does not clearly articulate the challenges in transferring synthetic data generation methods designed for forecasting tasks to classification tasks—what the specific difficulties are and how they are addressed. The introduction reads largely as an integration of existing generators applied to classification, with empirical observations such as scaling laws, but as a research contribution this positioning feels insufficient.\n3. The evaluation scope remains narrow (largely UCR-style, often univariate and fixed-length), with limited robustness analysis on generator hyperparameters and little evidence for multivariate, irregularly sampled settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E5fG5RczLS", "reviewer": "Reviewer_Mojr", "summary": "This paper proposes CAUKER, a novel and sophisticated pipeline for generating synthetic time series data specifically tailored for the pre-training of classification-oriented Time Series Foundation Models (TSFMs). The core idea is to combine two methodologies: Gaussian Process (GP) kernel composition, which generates realistic temporal patterns (trends, seasonality), and Structural Causal Models (SCMs), which impose a causal graph structure to create complex, non-linear interactions and meaningful clusters. The authors conduct extensive experiments showing that TSFMs pre-trained on CAUKER data not only outperform models trained on other synthetic datasets but also nearly match the performance of models pre-trained on real-world corpora that are over an order of magnitude larger. A key finding is that CAUKER-generated data enables smooth and predictable scaling laws with respect to both dataset and model size, a property the authors show is absent when pre-training on standard real-world benchmarks.", "questions": "Based on these weaknesses, here my questions to the authors:\n\n*   **Question 1:** The CAUKER pipeline is composed of several stochastic modules and expertly curated function banks. How were the specific contents of these banks (e.g., the 36 kernels, the set of mean/activation functions) selected and validated? Is the performance highly sensitive to these specific choices, or is the framework robust to using a simpler, more generic set of components?\n\n*   **Question 2:** The hyperparameter sensitivity analysis in Appendix C.3 is helpful. However, to better understand the generator's failure modes, have you investigated scenarios where deliberately poor choices (e.g., using only linear activations, forcing very shallow DAGs, or using only a single kernel type) cause the method to fail or degrade to the level of the simpler baselines in Table 1?\n\n*   **Question 3:** To what extent do you believe the poor scaling on the UEA benchmark is a fundamental property of real-world time series data, versus a specific artifact of the UEA collection's composition and scale? How might CAUKER compare against a hypothetical, massive, and diverse real-world corpus curated specifically for pre-training (e.g., a \"TimeNet\")?\n\n*   **Question 4:** The study convincingly demonstrates CAUKER's benefits for Transformer-based TSFMs. How do you hypothesize the generated data would interact with models possessing fundamentally different inductive biases, such as those based on CNNs (e.g., InceptionTime) or State Space Models (e.g., Mamba), which process information more locally or linearly?\n\n*   **Question 5:** The causal graph propagation step seems central to creating discriminative structure. Does this structural property particularly favor the global receptive field of attention mechanisms? A deeper analysis of which components of CAUKER (GP vs. SCM) are most beneficial for which type of model architecture would be a valuable contribution.\n\n*   **Question 6:** The paper successfully extends CAUKER to forecasting. Does this suggest that good classification data is a superset of good forecasting data, or were any modifications to the CAUKER pipeline necessary to achieve strong forecasting performance? Specifically, are the SCM-induced non-linearities as important for forecasting as they are for classification?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "*   **Novelty and Formulation:** The primary strength of this work is its well-motivated. Rather than creating a monolithic generator, the authors identify two key requirements for classification data—realistic temporal dynamics and discriminative clustering structure—and solve them by combining the strengths of two distinct fields. Using GP kernel composition (common in forecasting) for temporal patterns and SCMs (from the causality and tabular learning literature) for creating underlying class structures is a novel and highly effective synthesis. The design choices are clearly justified (Section 3.2), and the ablation-style comparison in Table 1 convincingly demonstrates that both components are necessary for optimal performance.\n\n*   **Empirical Evidence of Scaling Laws:** The paper's most impactful result is the clear demonstration of scaling laws (Figure 3). The experiments showing that accuracy on downstream tasks increases smoothly and monotonically with more synthetic data and larger models are a significant contribution. By contrasting this with the erratic and non-scaling behavior of models trained on the real-world UEA benchmark, the authors make a powerful case for using high-quality synthetic data as a controlled \"wind tunnel\" to study and develop scalable TSFMs. This provides a valuable methodology for the community, independent of the CAUKER pipeline itself.\n\n*   **Sample Efficiency and SoTA Performance:** The paper provides strong evidence that \"quality over quantity\" is important for pre-training data. The results in Figure 7 are particularly striking, showing that pre-training a model like Mantis on just 100K synthetic samples can achieve performance nearly identical to pre-training on its original 1.89M real-world sample corpus. This has significant practical implications, as it dramatically reduces the need for expensive and difficult data collection and curation. The fact that this performance is state-of-the-art for synthetic-data pre-training validates the effectiveness of the proposed approach.\n\n*   **Experimemntal Validation:** The paper is written with outstanding clarity. The experimental validation is extensive and robust, covering comparisons to multiple baselines, scaling laws, qualitative analyses (PCA, CKA, non-linearity in Figures 4 and 5), and transferability to different benchmarks (UCR, WOODS) and even a different task (forecasting). The appendices are helpful, providing, detailed descriptions of the function banks, hyperparameter sensitivity analysis.", "weaknesses": "I have concerns about the evaluation process and specially related to the complexity of the proposed generator, the framing of its comparison to real-world data, and the scope of the architectural evaluation.\n\n1.  **High Generator Complexity and Opaque Design Choices:** The CAUKER pipeline is a complex amalgamation of multiple components: three distinct function banks (kernel, mean, activation), random kernel composition, and random DAG generation. This introduces a large number of \"meta-hyperparameters\" (e.g., the specific contents and size of the banks, the distribution of DAG parameters). While the appendix provides a sensitivity analysis for a few of these, the process for designing the function banks themselves is not fully justified. It is unclear if the chosen set of 36 kernels or the specific activation functions are uniquely effective, or if a much simpler subset could achieve comparable results. This complexity could pose a significant barrier to adoption and reproducibility for researchers who do not have the authors' expertise in this specific setup.\n\n2.  **Potential for a \"Straw Man\" Argument Against Real Data:** The paper's narrative strongly contrasts the clean scaling of CAUKER with the poor scaling of the UEA benchmark. While this is a powerful rhetorical device, it risks overgeneralizing the conclusion. The UEA archive, while a standard benchmark, is a heterogeneous collection of many small, domain-specific academic datasets; it was not designed as a large-scale, cohesive pre-training corpus in the vein of ImageNet or The Pile. The observed lack of scaling could be an indictment of the UEA dataset's specific properties (lack of diversity, domain mismatch) rather than a fundamental flaw of pre-training on real-world data in general. The paper lacks a discussion of this nuance.\n\n3.  **Limited Diversity of Tested Model Architectures:** The experiments are exclusively focused on two Transformer-based models (Mantis, which is ViT-based, and MOMENT, which is T5-based). While these represent different pre-training objectives (contrastive vs. masked reconstruction), they share a core architectural paradigm. It is an open question whether the benefits of CAUKER's data structure are universally applicable or if they are particularly well-suited to the inductive biases of attention-based models. The rich, causally-linked structures might be more effectively captured by attention than by models with different biases, such as CNNs or State Space Models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nBGyATldPJ", "reviewer": "Reviewer_zLKs", "summary": "The authors propose CAUKER, a synthetic data generation algorithm that leverages Gaussian Process kernel composition and Structural Causal Models to produce diverse time series for augmenting training data in time series classification tasks. The paper evaluates the proposed approach against other synthetic data generation techniques for several time series foundation models.", "questions": "1. Are the TSFMs pre-trained from scratch on synthetic data, or are they fine-tuned on synthetic data (using models already pre-training on real data)?\n2. How do the models perform without any synthetic data augmentation?\n\nSuggestion: It would be interesting to include the combined scaling laws for the UEA and Cauker datasets on the same plot in Figure 3 to show cross-dataset scaling laws.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a novel synthetic data generation technique leveraging structural causal models (SCMs) for time series.\n- The work is a focused study on synthetic data augmentation for time series classification, an understudied area in time series literature.\n- Two time series foundation models (TSFMs) are evaluated with supervised and contrastive learning pre-training schemes\n- Several synthetic data augmentation approaches are systematically compared in Table 1, highlighting relative effectiveness.\n- Figure 3 effectively illustrates scaling laws and the relationship between model size and performance.\n- Figure 4 provides an interesting analysis showing the diversity of principal components in synthetic data relative to non-synthetic datasets.\n- The study demonstrates that fewer synthetic samples can achieve comparable performance to real-world pre-training datasets, highlighting practical efficiency benefits.", "weaknesses": "I would be happy to increase my score if the following concerns/points are addressed.\n\n- Zero-shot evaluation methodology: The study claims to evaluate TSFMs in a zero-shot setting, but the models are allowed to be pre-trained on the training set of the same dataset used for evaluation. This means the evaluation is not strictly zero-shot, as the train and test sets are likely in-distribution (Lines 122–124): “In practice, if we evaluate a given TSFM on a test set from a UCR (Dau et al., 2019) dataset, we ensure that the TSFM was not pre-trained on it, but we allow for the train set of this same dataset to be used for pre-training.” \n\n- Missing baseline comparisons: Results without synthetic data augmentation are not reported in Table 1. Including these and quantifying the lift from augmentation would be helpful.\n\n- No text-based or experimental comparison with the synthetic data generation process used by TabPFN, which also leverages structural causal models.\n\n- No comparison with non-foundation model baselines (e.g., random forecasts, XGBoost, logistic regression).\n\n- Clarification on model pre-training: It is unclear whether the models are pre-trained from scratch on synthetic data or fine-tuned with synthetic data (using pre-trained models on real-world data). For example, the text states: “In practice, if we evaluate a given TSFM on a test set from a UCR (Dau et al., 2019) dataset, we ensure that the TSFM was not pre-trained on it, but we allow for the train set of this same dataset to be used for pre-training.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)