FeDaL: Federated Dataset Learning for General Time Series Foundation Models
Authors: Shengchao Chen
Guodong Long
Michael Blumenstein
Jing Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: Time Series Analysis
Time Series Foundation Models
Federated Learning
Primary area: learning on time series and dynamical systems
TL;DR: We propose FeDaL, a federated framework for TSFM pretraining that mitigates dataset-level biases via DBE and GBE, enabling domain-invariant representations transferable to regression and classification tasks.

Abstract: Dataset-level heterogeneity introduces significant domain biases that fundamentally degrade generalization on general Time Series Foundation Models (TSFMs), yet this challenge remains underexplored. This paper rethinks the from-scratch training of TSFMs using the paradigm of federated learning. We propose a novel Federated Dataset Learning (**FeDaL**) approach to tackle heterogeneous time series by learning dataset-agnostic temporal representations. Specifically, the distributed architecture of federated learning is a nature solution to decompose heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowledge. Moreover, based on the TSFM architecture, FeDaL explicitly mitigates both local and global biases by adding two complementary mechanisms: Domain Bias Elimination (DBE) and Global Bias Elimination (GBE). FeDaL`s cross-dataset generalization has been extensively evaluated in real-world datasets spanning eight tasks (including various regression and classification), against 54 baselines. We further analyze federated scaling behavior, showing how data volume, client count, and join rate affect model performance under decentralization. Our code is publicly available at https://github.com/shengchaochen82/FeDaL.

Published as a conference paper at ICLR 2026
FEDAL: FEDERATED DATASET LEARNING FOR GEN- ERAL TIME SERIES FOUNDATION MODELS
Shengchao Chen, Guodong Long, Michael Blumenstein, Jing Jiang Australian Al Institute, Faculty of Engineering and ITT, University of Technology Sydney shengchao- chen. uts gmai 1 {Guodong. Long, Mi chael. Bl umenstein, Jing Jing. edu. au
ABSTRACT
Dataset-level heterogeneity introduces significant domain biases that fundamen- tally degrade generalization on general Time Series Foundation Models (TSFMs), yet this challenge remains underexplored. This paper rethinks the from-scratch training of TSFMs using the paradigm of federated learning. We propose a novel Federated Dataset Learning (FeDaL) approach to tackle heterogeneous time series by learning dataset-agnoste temporal representations. Specifically, the distributed architecture of federated learning is nature solution to decompose heterogeneous TS datasets into shared generalized knowledge and preserved personalized knowl- edge. Moreover, based the TSFM architecture, FeDal explicitly mitigates both local and global biases by adding two complementary mechanisms: Domain Bias Elimination (DBE) and Global Bias Elimination (GBE). FeDaL's cross-dataset generalization has been extensively evaluated in real-world datasets spanning eight tasks (including various regression and classification), against 54 baselines. We further analyze federated scaling behavior, showing how data volume, client count, and join rate affect model performance under decentralization. Our code is publicly available at https: /tt:://gttttnn nighthh..... com/ wttp://ccoooccccceeeeeeeeee wwwp:gchhcccceeee//eeeeeee
INTRODUCTION
Time series analysis plays critical roles in decision systems such as weather forecasting (Bi et al. 2023 medical symptom classification (Wang et al. 2024c). industrial anomaly detection (Xu al,
2022 and imputing missing data in wearable sensors (Wu et al. 2020). Recent advances follow two main directions. Time Series Foundation Models (TSFMs) pretrain on large-scale, multi-domain datasets to capture transferable representations, achieving strong zero-shot or few-shot performance but remaining largely task -aek-spec,,,c, for example in forecasting (Liu et al. 2024c; Shi et al.) 2024]
Ansari et al.) 2024 Das et al. 2023) classification (Feofanov et al. 2025 In contrast, Time Se-
ries Pattern Machines (TSPMs) pursue architecture-level unification across tasks (Zhou et al. 2023
Wu al. 2022a Wang al., 2024a). However, TSPMs typically trained per dataset, restricting zero-shot generalization across unseen domains. Moreover, similar to TSFMs, they often assume centralized access diverse datasets, which is unrealistic in practice since time series data are usu- ally siloed and heterogencous (Chen et al. 2025). This gap motivates the need for a general TSFM that combines architecture-leve flexibility with data-level generalization, without centralizing data.
2023) provide promising decentralized paradigm for train- ing FMs by leveraging both public and private datasets. This strategy mitigates privacy concerns and alleviates the computational burden of centralized training, and has been extended to train general- purpose models by treating each domain or dataset as an independent client. Despite this promise, existing works (Chen et al.) (2255) face two major challenges. First, they adopt coarse- gearee-rrained treatment of heterogeneity, focusing only on broad domain-level shifts (c.g., climate vs. healthcare) while overlooking dataset-specific structural biases within each domain. Consequently, the lack of robust cross-domain invariance limits scalability, as downstream tasks still require fine- ane-ggrinedd tuning to adapt to individual datasets, preventing these models from functioning true FMs.
Federated FMs (FFMs) (Zhuang et al.
To make this concrete, we illustrate three representative dataset-level biases in Figure I (1) Tem- poral resolution bias, where sequences with different sampling rates encode inconsistent contexts

---

Published as a conference paper at ICLR 2026
under fixed window (e.g., 120 steps cover five days of hourly weather but only two hours of minute- level energy data); (2) Physical constraint bias, where unrelated physical laws (e.g.. temperature variation electric current) reduce cross-domain transferability; and (3) Pattern transition bias, where initially similar trends diverge due to exogenous events (e.g.. traffic vs. website visits), break- ing assumptions of shared temporal structures. These biases illustrate that heterogeneity arises both the client and global levels: temporal resolution and physical constraint biases primarily distort client-specific representations, while pattern transition bias amplifies during global aggregation.
Temporal Resolution Bias
Physical Constraints Blas
Pattern Transition Blas
Evers jeias, Tigi (alurmat
Homogenous Trends
Weutler floam, co
Pationn Break
Information Lost!
Traneportation
Inconsistent
Reconstruction MSE
Reconstruction
Reconstruction MSE
0.509
0.376
0.465
0.525
0.372
0.453
0,480
0.355
VedDAvg-nndependeett
FedAvg-Mixed
Stand-alone
Transformer
Figure I: Dataset-level biases. Information Lost: Low-resolution sequences capture fewer details under fixed window. Pattern Break: Abrupt structural changes across time. Settings: FeeAAgg-ndepennee (each dataset as a client), FedAvg- -Mxxed (datascts pooled then split), and Stand-alone (no aggregation). "Transformer' means that results from the local Transformer model. Setup follow Sec. 4. Lower MSE is better.
This paper introduces Federated Dataset Learning (FeDaL), a federated framework for training general TSFMs from scratch, designed capture dataset-level patterns from heterogeneous datasets. FeDaL moves beyond coarse domain-level treatment to address dataset-level heterogeneity stem- ming from differences sampling resolution, physical constraints, and transition patterns. Specif- ically, FeDal. integrates two complementary components: Domain Bias Elimination (DBE) and Global Bias Elimination (GBE), operating at the client and server levels respectively. DBE approx- imates and disentangles client-specific biases in context-agnostic manner by modeling local bias representations. GBE improves global generalization by aligning client updates via gradient-level correction and applying fine-grate fine-grained server-side tuning to mitigate residual biases during aggrega- tion. DBE and GBE enable FeDaL to produce TSEMs with comain-invarrin representations that generalize across diverse datasets while preserving privacy and scalability. Our contributions are:
1. We propose Federated Dataset Learning (FeDaL), FL framework for training general TSFMs from scratch, explicitly addressing dataset heterogeneity through context-agnostic Domain Bias Elimination (DBF) on clients and Global Bias Elimination (GBE) at server.
2. We provide the first systematic study of TSFM pretraining scaling behaviors across various tasks under federated learning, offering empirical insights into how data volume, client population, and participation rate shape model generalization in decentralized settings.
3. Extensive experiments show that FeDaL-trained models outperforms advanced baselines across forecasting, imputation, anomaly detection, and classification, while achieving highly competitive performance against centralized TSFMs with far fewer parameters.
2 PRELIMINARIES
Pre-trained Time Series Models. Pretraining has become central to time series modeling, driv- ing both task-specific models (Nie et al., 2022 Jin al. 2023) and TSFMs (Goswami et al. 2024 Woo et al.] 2024) [Shi et al. 2024). Early efforts typically rely masked reconstruction at the time-point Zeng et al., 2023) or patch level (Liu et al. 2024c| Garza & Crrgentthlle-Cannsee
2023 achieving strong zero-shot performance. More recent models such as Time-MoB (Shi et al.
2024 4 Moment (Goswami et al. 2024). Moirai (Woo et al. 2024), TimeFM (Das et al. 2024), and
Chronos (Ansari et al.) (024) replace reconstruction with sequence prediction, further boosting zero-shot forecasting. However, these models remain musk-trietted rather than universally applica- ble, unlike TSPMs that pursue architecture-level generality. In addition, they assume centralized
2

---

Published as a conference paper at ICLR 2026
access to massive datasets, unrealistic requirement since time series are often siloed and hetero- geneous (Chen et al., 2025; Zhuang et al. (002). Federated learning (FL) (McMahan et al. 2017) offers practical alternative by enabling decentralized training with privacy preservation.
Federated Foundation Models. FFMs have emerged as a promising direction for scaling FMs without centralizing data (Zhuang al. 2023). In TS. research on FFMs has begun to emerge. Time-FFM (Liu et al.] (024a) and PeFAD (Xu et al. 2022) leverages pretrained LLMs and applies lightweight tuning achieve personalized predictions. contrast to achieve multiple customized models, FFTS (Chen et al. 2025) takes different approach by pretraining TSFM on cross-domain datasets and then performing task-specific adaptation on downstream. reduces local-global dis- crepancies to mitigate domain-witg heterogeneity. However, FFTS still falls short of being true FM, since don't 't support zero-shot inference and considers heterogeneity only at a coarse level, leaving dataset-level biases largely unresolved. To bridge this gap, we propose FeDaL, an FL frame- work TSFM training framework that explicitly addresses dataset-level heterogeneity and enables training a general TSFM from scratch with multiple downstream TS analysis task support.
Problem Definition. We consider an FFM setting with a server and N clients, where each client i holds a local time series dataset D,, These datasets are heterogeneous due differences in sampling resolution, physical constraints, and temporal dynamics, which induce dataset-specific biases that may not transfer across clients. The objective is collaboratively train a general TSFM that generalizes that supports diverse downstream tasks. Formally, the global objective is F(0) = arg ming Σx= Ei (0;;;; D;), where Fi(.) denotes the local loss, and n,, are the local and total sample counts. Our key question is: Can train a general TSFM from scratch under FL con- straints that captures cataset-invariia temporal pattern while mitigating dataset-induced biases?
METHODOLOGG
Heterogeneous datasets introduce dataset-level biases, causing local models to overfit client-specific patterns that are not transferable across domains. For example, energy data may encode biases from grid cycles, while health data may reflect hospital-speciffe sampling protocols; such patterns valid locally but break down in cross-domain settings. When aggregated, this misalignment pro- duces globally biased TSFM with limited generalization, problem further amplified in large and structurally diverse datasets. To address this, we propose FeDaL (Figure 2 with two complemen- tary mechanisms: Domain Bias Elimination (DBE) on clients and Global Bias Elimination (GBE) on server, where "domain" is dataset- heterogeneity rather than broad application classes. FeDaL mitigate dataset- iinuuee biases and align temporal patterns into a unified, transferable space.
3.1 DOMAIN BIAS ELIMINATION
Domaln Blas Ellimiation
Trend Bias
DBE (as shown in Figure ad- dresses dataset-induced biases by de- composing latent representations of masked input patches into trend and seasonal components, from which trainable bias vector is derived. This bias vector is injected back during reconstruction to suppress dataset- specific artifacts (see Right, Fig- ure 2) while a regularization term aligns it with a global bias reference,
Serics
Coct-ant
Beckbvan
Head
B
Projection Head
b
Futro
by
h
Seazonal
Trend
h
Terms
Term
Cloe- det Synchesia
Sevies Representation
{b. Broadcant
Upload {b 63)
Decumppoee
Decompooee
Server
RC
Cere-wet Tuning
Bg Decoder-only Transformer
[Ganms Attantion
Global Model (Prenttooal)
Embodding
Pynatzir Conrection
Fusion
Patch Masking
Clobal Bias
encouraging the model to disentangle non-transferable local patterns from generalizable temporal structures.
Global Bias Elimination
Figure 2: FeDaL: DBE reduces dataset-induced biases on the client side, whereas GBE improves alignment on the server. Lo- cally, the plug and-play DBE block (instead of DBE mechanism) captures trend and seasonal biases from latent representations.
Following NNe et al, 2022), we adopt unsupervised patch recon- struction for local updating. For clar- ity, the client index is omitted in the following derivations. Given a masked sequence X obtained by patch-wise masking from original sequence X, its latent representation is obtained via the lo-
3

---

Published as a conference paper at ICLR 2026
cal backbone (.). We introduce a plug-and-play DBE block to explicitly approximate dataset- induced biases. Instead of directly averaging latent features, we apply decomposition (Wu et al. 202I to split them into trend and seasonal components to capture context-independent patterns as:
h h. TimeDecomp(fa- (X), T),,
(1)
where is the decomposition granularity, h, denotes the trend component, and h, the seasonal component. We then average each component to extract persistent deviations from compact repre- sentations, producing bias vectors that capture stable dataset-specific shifts. This process as:
b, Mean(h;) @ 7/., b, Mean (h,) O Ys-
())
Then we define the local bias vector b = b, + b,. Compared with plain mean of features, decom- position introduces an inductive bias: b, captures low-frequency shifts, while b, captures higher- frequency periodicities. Both are learnable, where the element- scaling factors Y: and Y enable adaptation to evolving dataset patterns. To encourage disentanglement, the model reconstructs the input by injecting b into the latent features, the process can be formulated as:
())) E Ell. (fe, (X) + b) xII
where 0 (0,, On. b). This design ensures that dataset- deviations are absorbed by b, forcing fo, to focus on transferable structure. Directly reconstructing b alone would not constrain the back- bone, whereas this additive formulation encourages disentanglement between invariant features and dataset-bound biases. To align local biases across clients, we rewrite local optimization objective Eq. 3 via introducing an explicit penalty with controlling coefficient À as:
(()) = flllfo, (fe. (X) +) x + Allb - 1
(4)
Here ba is initialized from aggregated client statistics and serves as a dynamic global prior, prevent- ing drift. Although the penalty is applied to b, its effect propagates to the backbone by constraining the separation of spurious and transferable patterns. Since bias depend on full-dataset statistics but training proceeds mini- batches adopt exponential moving average (EMA) (Zhang et al. 2015) to stabilize updates by gradually blending the old estimates with new batch-wise estimates:
by - (1 b-) + =bb"", new
b, p.)b²a + tub"...
(5)
where j is a smoothing factor. DBE can explicitly approximate dataset-specific biases, disentangle them from generalizable temporal patterns, and reduce inter-client divergence during aggregation.
3.2 GLOBAL BIAS ELIMINATION
While DBE alleviates dataset-induced biases at clients, residual misalignment persists during ag- gregation, leading divergence in global pattens. To mitigate this, we propose GBE with two components: (1) Gradient-level Dynamic Correction, which compensates for client-server drifts, and (2) Core-set Tuning, which refines the global model with sewaacypreserving client summaries. GBE reduce cross-client discrepancies and guide the global model toward domain-invarian patterns.
Gradient-level Dynamic Correction. The intuition is that naive aggregation (McMahan et al, 2017) assumes client updates unbiased estimates of the global gradient. However, when client datasets are heterogeneous, this assumption fails, and the aggregated model drifts toward client- specific directions. Inspired by (Acar et al. 2021), we maintain server-side state vector s, that records the accumulated drift between client and global models across communication rounds:
s" =-=--1_ 3 Σ O0 - 0
where Of denotes the local model from client i at round r, 0% is the global model from the previous round, and 3 is scaling factor. This state serves as a correction memory: instead of directly adopt- ing the naive FedAvg result g. we correct as 07 0r (1/B).s" Here, 0r is the weighted average of client models, and 00 is the corrected global model used for the next round. By subtracting accu- mulated drifts, this mechanism prevents client- client-specific deviations (c.g., hospital-specific rhythms in healthcare or region-specific seasonality in energy data) from dominating the shared model.
y
4

---

Published as a conference paper at ICLR 2026
Core-set Tuning Even with gradient correction, global bias may persist if clients encode different structural patterns, To refine the global model further, we perform server-side fine-tuning using compact core-sets generated by clients. Each client first samples a small batch X C X of size K" < [X'] for efficiency and privacy. From this, we construct an initial core-set Cinit- parameterized as a set of learnable vectors rather than raw samples, so that can be directly optimized via gradient descent. To approximate the client-specific pattern, the core-set is optimized by minimizing gradient discrepancies against the sampled batch, using the gradient matching (Killamsetty et al. (021) as:
Lmatchh = E VoJo VoJa(>))1
Here the loss is well-defined because gradients are aggregated at the model-parameter level; the optimization updates Cinit rather than 0 (model parameters) with learning rate 1). To protect raw data, we perturb Cinit in the Fourier domain. Specifically, we perturb only the amplitude while keeping the phase, since the phase encodes semantic temporal patterns (e.g., periodicity), whereas the amplitude carries fine-grained details as C' = F F-1(F(Cmma) + <o( where F and F-1 denote the forward and inverse Fourier transforms, and controls the noise intensity. Perturbation may distort information, so we further align C' with the sampled batch X' in latent space like Eq. 7
Lalign - lIVoso(C") - -
(8)
The final aligned core-set C is uploaded to the server. The server fine-tunes the corrected global model dor on aggregated C, from clients, producing get.r, To prevent catastrophic forgetting, we apply convex fusion as far = odai + - aa)oorr where balances stability and adaptability.
FedDaL. As shown in Algorthm T Fed- DaL proceeds in three phases. In the initialization stage, the server broadcasts
Algorithm 1 Workflow of Federated Dataset Learning
Require: Clients {c,) total rounds R. core-set size K, de-
composition period r. hyperparameters X, a,
the model and establishes a global bias reference. In local updating, each client applies DBE to decompose features, es- timate dataset-specific biases, and train the backbone with bias-regularized re- construction, while also constructing compact core-set for knowledge sum- marization. In the server aggregation stage, the server aggregates local mod- els, applies dynamic correction strategy to counter client-server drifts, and re- fines the global model using compact core-sets. After each round, the server sends the updated model 09 and the global bias be to selected clients, ensur- ing consistency across rounds. Through the interplay of DBE and GBE, our FedDaL progressively mitigates dataset- induced biases and aligns temporal pat- terns into a unified general TSFM.
Ensure: General TSFM 0 = {0,, Initialization Phase:
Server initializes gs.0
(0%, 0%)
Warm-up 1 epach, broadcasts global bias ba.0 E, bi Initialize server state 8 =
for round r = 1 to R do
Sample client subset S, and send [0--1bair-))
Local Update (each oj E S, in parallel):
Decompose features into trend/seasonal terms (Eq.
Estimate local bias by = Mean(h;,)) + Mean
Train (0r.., 0T. b by minimizing C. (Eq.
Construct core-set CJ (Eqs.7) B
Upload (05, Th.,, CT} to the server
Server Aggregation:
Aggregate models to obtain provisional global gair Update server state s" (Eq.
Apply gradient correction: for go. (1/8) Refine (eer - gat." by fine-tuning on core-sets Fuse models: 0e-r = aga. + (1 aa)at.. Update global bias 6 = [8TT Lies, by
end for
MAIN RESULTS
In this section, we comprehensively evaluate FeDaL across three dimensions: federated representa- tion learning, downstream generalization, and federated scaling behavior. Our experiments demon- strate that FeDaL (i) learns Nomain-agnoste representations in highly heterogeneous settings, (ii) enables strong generalization across forecasting, imputation, classification, and anomaly detection tasks, and (iii) exhibits favorable scaling properties under increasing data, client count and join ratio.
5

---

Published as a conference paper at ICLR 2026
4.1 FEDERATED REPRESENTATION LEARNING
Setup. We evaluate FeDaL from FL perspective, focusing its ability to handle heteroge- neous data. Experiments are conducted on two multi-domain datasets: UTSI) (Liu et al. 2024c) with domain-mixed (DM) partitioning, where each client accesses data from multiple domains, and CTSD (Chen et al. 2025) with domain-independern (DI) partitioning, where each client is tied to single domain. This allows to examine FeDaL under both mixed-domain and domain-specific heterogeneity. We compare against five FL baselines: FedAvg (McMahan et al., 2017). FedProx (Li
al.) 2020), FedPer (Arivazhagan et al. 2 FedRep (Collins et al.] 20.1), and TSFM -orietedd FFTS (Chen et al., (025). All trained with patch-wiss masked reconstruction (Nie et al.) (022) under
75%
masking ratio with length of 512. Detailed implementation are provided in Appendix B
Main Results. The main results are shown in Table FeDal consistently outperforms baselines across datasets and settings. Compared to the state-of-the-ar federated TSFM pretraining method FFTS, FeDaL reduces reconstruction MSE by average of 4.16% on USTD and 8.86% on CTSD. Notably, FeDaL adds about 2 MB per round due to transmitting core-sets, yet achieves significantly stronger generalization. These demonstrate FeDaL's 's ability balance client-specific pattern learn- ing with server-side alignment, yielding more transferable temporal patterns.
Bias Mitigation. To further assess its fia-miiigttiion capacity, we visualize the evolution of local bias features from clients in Figure 3 At R1, the biases show substantial variation across clients, reflecting strong intra- and inter-client discrepancies. As training progresses, these discrepancies shrink, then toward common space. This provides intuitive evidence that FeDal. effectively duces dataset-induced biases and promotes alignment toward nomain-invariann features.
Federated TS representation learning results aver- aged over masking ratios (20%, 35%, 50%, 75%, 90%), }. For UTSD, two heterogeneity levels (H1 and H2) are simulated as described in Appendix [B] while in CTSD each dataset treated as an independent client. Bold: the best, Underline: the second best. indicates evaluation with personalized models after server averaging, and f denotes client-side eval- uation without aggregation. Full results in Table 16
Table 1:
Round
Alignment
USTD (DM) H2
Method
HI
CTSD (DI)
Comm. Param' #
FedAvg
0.586
0.592
0.455
108.4 MB
FedProx
0.583
0.586
0.444
108.41
FedPer!
0.565
0.588
0,430
106.43 MB
FedRept
0.569
0.592
0.430
106.43 MB
FFTS
0.562
0.531
0.416
118.94 MB
Dim.
Standulone
0.571
0.567
0.447
Figure 3: Local bias change across rounds
FeDaL (Ours)
0,551
0.511
0.387
110.41 MB
(R1 R60) for select clients (Clients 3, 7, 9).
Ablation & Hyperparameter Sensitivity Results. We ana- lyze the contribution of five key components and the robust- ness of FeDaL to hyperparame- ter (Table 2 Ablations reveal that removing DBE or GBE sig- nificantly harms representation quality, confirming their role in addressing domain heterogene- ity. Disabling DBE's alignment step (Eq, 4 further degrades performance, emphasizing the im- portance of bias alignment. Removing Core-set Correction (Eq, 6) impairs global adaptation, showing their necessity for model refinement under heterogeneity. Sensitivity results (as shown in Figure 4) show: (i) overly large alignment weight À over-constrains local representations; (ii) larger core-sets bring marginal gains at the cost of privacy and communication; (ii) extreme fusion weights weaken generalization; (iv) unstable 3 values impair client-server representation blending.
Table 2: Ablation results. UTSD values are averaged over III and H2. "Vars." indicates the relative performance change compared with refees.
UTSD
CTSD
Average
Model
MSE
Vars.
MSE
Vars.
MSE
Vars.
FeDaL (Original)
0.573
0.405
0.489
w/o Alignment
0.602
5.006%
0.434
J 7.16%
0.518
16%
w/o DBE
0.637
1 11.17%
0.452
[ 17.16%
0.545
19.17%
w/o Core-set Tuning
0.590
2.97%
0.430
] 6.17%
0.510
4.57%
w/o Correction
0.600
4.71%
0.431
6.42%
0,516
5.57%
GBE
0.610
6.46%
0.444
9.63%
0.527
8.05%

---

Published as a conference paper at ICLR 2026
Sugnstyy NSE
0.60
0'10
1ith
-0
0.60
0
0.40
wegrsttructonn MBE
E413
0,89
srsnttrrccooo HSE
0.60
0.50
0
0.53
0.25
0.52
0.50
-O- USSO Dynse
-Dh CTS besel
2.20
UTEO Fette
UI8 Dokov
+O+ U150 Dater
+O= C50 base
E427
0.00
0A44
0.21
1:61
42"
MAN
150
0.40
0235
0340
6.40
10.55
240
12%%
-0
3.3%
0.3
0C 10,0
0.1 0.15
53
0:5
1024
2045
1072
45%
5120
0.32
035
0.6
07
0.5
0.9
0.01
0.1
8J
05
Aln Algement
Coro-St Size
in Model Fosion
Bin Corraciin
Figure 4: Hyperparameter sensitivity. Results on UTSD are averaged over HI and H2. 4: performance drop; 1: improvement relative to the original FeDaL. Best viewed in color and with zoom.
4.2 ADAPTION AS GENERAL TIME SERIES FOUNDATION MODELS
To evaluate the generality of F TSFMs, we test on diverse downstream tasks includ- ing long-/short-term deterministic forecasting, imputation, classification, and anomaly detection. Pretraining is conducted on LOTSA (Woo et al., (024), which consists of 23 time points parti- tioned into 174 dataset-specific clients, naturally aligning with the federated setting. For non-zero- shot task (expect long-term forecasting), we fine-tune the trained model for only one epoch. Data preprocessing details are provided in Appendix B and full results are included in Appendix C
Table 3: Full-shot Long-term forccasting results (averaged across horizons {96, 192, 336, 720} for ETT-series and Weather, and [24, 36, 48, 60) for ILI). Bold: best; Underline: second best.
Models
FeDal
EFTS
Fedire
TimeMiser
Time LLM
GFT4TS
PacNTST
TimesNet
DLincur
Fodioraer
Autodommer|
Statioaayy
LighTTS
Metrics
MAR
MAF
MAF|MSS
MSE MAR
MAF
MAF
MAF MSE MSE MAR
MSR MAF
MAF
MAR
ETTI
0.380 o 191 0.41 2.112.11120044 1111044488
444 443 0,408 0.423 0.465
0.4080423
0.455
0.450 0.422.22..333
0.440 0.40000.......... ww..0o00..9..0
0.487 4 537 0.401 0.479
CTTh2
0,3340377
0.283
0.364
0.394 0,334 3.000.3
0.330
0.379
0.427
0.416
0.450
ETTml
0.319.336
0.325
0,334
0.267.0351
0.345 0,329 0.372
0.288
0,4051.251
0.251
0.400
0,378
0,448
0.552
0.452
0.585
0.417 0.456 0,435 0.437
ETTm2
0.261
0.319
0.254
0.316
0.323 0.251 0.313
0,35
0.315
0.333 0.333.0222
0.267
0.333
333000.00
0.305
0.439
0.371 0.306
0.347 0,409 0.436
Weather
0,213
0.255
(1217
0,250 0.255
0.260 0.272 0.225
0.260
0.257
0.237
0.250
0.264
0.259
0.267
0.248
0,300
0.309
0.338
0.338
0.282
0.314
0.261111...0010 0,312
ILI
1.355
0.773
| 2899
1.9.0009144400000 1180118000 0.999
0.79
1.435
0.801
1.925
0,900
1.443
U.797
0.951
2,169
1.041
2.547
3.144(3.000 1.361
2.077 7.342
1 Count
Table 4: Average zero-shot forecasting performance across horizons (96, 192, 336, 720] for observation lengths (512, 1024, 2048, 3072]. Bold: best; Underline; second best. Full results in Table 19, Appendix C
FL Pretraining
Centralized Time Series Foundation Models
Models Metrics
PeDal. (Ours))
FedArg
Moirais
Muicai,
TimesFM
Moment
Chronos, Chronon, Time-MoEs, Time-MoF, Time-Mol...
Merrics
MSE
MAE
MSE MAE MSE MAE MSE MAE MSE MAT (MSE MAE MSE MAE MSE MAE MSE MAF
MAE MSE MAE
MSE MAE
MSE MAB
0.407
0.429
0.425
03437
0.419 0:430 0.473 0,443 O 0.683
591 0.468/0 466 400
0.394
0.419
0.412 0.426
CTTh2
0.361
0.382
0,370 0,37440.3900 0,390 0,362 0,382 0,367 0,377
0.392 0, 0,361 O. 0.809 O. 0.410
0.127
0.366
0.404
0.605
0.415
0.371
0.399
ETTml
0.360
0.300
0.364
0.420 0.3378 0,430 0,405 0,385 0:422
0.418 0.670
0.55 0.465
0.415
ETTm2
0.292
0.341
0.517 0.362 (1,322 0,365 0,511 0,337 0,329 0.543
0.328
0,346
0,316 0,365 0.310 0, 0,35
0.331
0.365
00.316
0.361
0.28 0.344
Weather
0.284
0.263.
0.277 0. 3.306 0.287 0.281 0.264 0.273
0.294 0.326
392 0.3 0 0.306
0.265 0
00.270
0 0.300
0.288
Aug.
0.335
0.365
0.348 0.361 0,352 0.357 0.361 0.372 0,365
0:407
0,400 0.465 0, 0,440 0.409 0.40990.43. 0.436 0.400
0.381
00.352 0.380
0.337 0,370
1d Count
3
god Count
4
Data Scale
2318
2318
2518
2318
2518
1008
1.13B
84B
848
500R
500R
3008
Paramse
28.42M
31.1MM
28-2M
84M
200M
385M
200M
710M
113M
453M
2.48
Long- Term Forecasting. Time series forecasting remains a critical yet challenging task in prac- tice. (1) Full-shot. We follow the setup from (Jin et al., 2023), evaluating on ETT. Weather, and Illness, excluding Traffic and ECL as they are used in pretraming, All models use a look-back window of 512, and we fine-tune the pretrained TSFM via FeDal. for five epochs. Results in Ta- show that FeDal. consistently outperforms both -fate-of-the-art deep models and LLM-based TSFMs in full-shot and few-shot scenarios, yielding significant MSE reductions. (2) Zero-shot. To further assess generalization, we perform zero-shot forecasting following the evaluation protocol of Time-MoE (Shi et al. 2024). As shown in Table 4. FeDaL achieves highly competitive zero- shot performance, surpassing FFTS by 3.8 % and Moiralaage by 6.2% in average MSE. Remarkably, FeDaL. attains these gains with only 28M parameters, far fewer than all centralized TSFMs such as Moirai (84M-91M), Chronos (200M-710M), and Time-MoF (1 13M-2.BB). This efficiency demon-
7

---

Published as a conference paper at ICLR 2026
strates that our federated pretraining strategy FeDaL can deliver strong Nomain-agnosii temporal representations without relying on the massive model scales typical of centralized approaches.
Short-Term Forecasting. To evaluate the effectiveness of FeDai-trained TSFM in short-term forecasting tasks, we conduct experiments on M4 dataset, following the protocols of GPT4TS (Zhou jet al., 2023). As shown T FeDaL significantly outperforms baselines. While it performs slightly below FFTS on SMAPE by narrow margin of 0.07%, FeDal reduces MASE by 2% to 38% and achieves 5% to 22% % improvements in SMAPE and OWA. These further confirm the ability of FeDaL to learn cross- dross-dataset representations that generalize effectively in forecasting.
Table 5: Average short-term forecasting results on M4 dataset. Bold: best; Underline: second best. None of these datasets were included in pretraining. denotes a "former" suffix. Full results are provide in Table
Model
FoDat.
FFTS
FodAug MOMENT Time-1MM (PT4TS TimesNat PatchTST N-HITS N-REATS ETS. TighTSS incar
Statinary
Avengge
11.404
12.342
14.593
11.983
12.69
12.88
12.059
12.035
12.25
14.718
13.525 13.639
13.16
12.78
12.909
MASE
L489
1.522
L.753
2.161
1.595
1808
1.836
1.623
1.625
1.698
2,408
2,111 2.095
1.775
1,756
1.771
OWA
0.818
0.831
0.926
1.103
0.859
0.94
0.955
0.869
0.869
0.896
1.172
1,011 1.511 0.949
0.93
0.939
Imputation. Imputation evaluates a model's ability to reconstruct missing values. We conduct experiments on five widely used datasets, including four ETT and Weather, where missingness is common. Following FFTS (Chen et al. 2025), we simulate varying corruption levels by randomly masking points. As shown in Table 6 FeDaL consistently outperforms all baselines. Compared to the strongest FL baselines, reduces MSE by 12.64% over FFTS and 27.62% over FedAvg. Against centralized TSFMs. FeDaL achieves 22.84% % lower error than GPT4TS and also surpasses the general-purpose TSFM Moment, despite Moment being trained on much larger centralized corpus. These confirm that FeDal. not only leverages distributed data effectively but also achieves competitive or superior generalization to centralized TSFMs under missing-data conditions.
Table 6: Average imputation performance for randomly masked time series, evaluated across four mask ratios (12.5%, 25%, 37.5%, 50%). Bold: best; Underline: second best. Full results in Table 23
Mudel
Felall
PFTS
FollAor
Momoo
TineMina
GPTTTT
PASATTT
TimesNel
Dlincar
Fallormrr
Autafommee
NS
LighTS
Mesics
MSE
MAF
MSE MAE MISE MAE MSE MSE MAE MSE MAF MAE MSE MAE MSE MAE] MSE MAE|MSE MAF MSE MAF MSE MAE
ETTbl
0.0220.00000002
0.04 n. 0.120 0.994 0.008 o 125|a002 0.005 o. a. 022 0,027 0.10lo 0.710.104 0.218 0,205 0.177 0.051 0,150
ETTh2
0013
(u817 g,ore cuuss tuess b001 a.so 0.028 0, 0002 0.09 0.102 0.208 0.377 0.022 o.e9
0.046 0,151 Du09s 0.205 0.10 0.215 0,105
ETTml
0,144
luoss 0.154 n.15 p,s2 a 167 n.ors o 192
0.224 8.1
0.20 0,329 0,088 0.187 0.284
0.573
0.573
0.201
0.306 0.24 0.163 0.214
ETTm2
0.164
(u046
0,135 2u062 1.154 2042 0.00N lupss 0.141 juus 0.103 0.436 0.099 0.146 0.119 0.250 0.250 0.142 0.259 0.16 0.279 0,055 0.156
Wasshe:
[0.04 0.048
1102 (2020
0114 pu25 anst 0.099 o.ns3 Juss 0,056 no04 (.076 1.17 0,000 0.544
(.../...ce (1.65
0.117 0.117 0.110 2.2 0.031 0.057
1 Counn
Anomaly Detection. We evaluate FeDaL-trained TSFM on five widely used multivariate datasets (SMD. MSL, SMAP, SwaT, PSM), following Moment (Goswami et al., (024) protocols for fair comparison. As shown in Table 7 FeDaL achieves the best overall performance across all datasets. It surpasses strong centralized baselines such as Modern'TCN and TSFM Moment by 2.40 % and 5.17%. %,, respectively, and further outperforms FL. methods including FedAvg and FFTS by 0.96% % and 3.69 % These results highlight our FeDaL's 's effectiveness in capturing global temporal invari- ance and its superior generalization to complex anomaly detection tasks.
Table 7: Anomaly detection results. We calculate the Fi-score (%) for each dataset and statics the average F1-score. Bold: the best, Underline: the second best. denotes a "former" suffix. Full results are in Table 24
Model
FeDal FFTS FodAvg Moment GPT4TS MTCN TimesNer FED." LightTS DLincar NS.*
Auto." Pyra." * Anomaly In. Re.
SMD
88.46
89.88
88.44
84.94
86,89
85,81
85.81
85.08
82.53
77.10
84.62
85.11
83.04
85.49
81.64
75.32
MSL
89.05
82.42
82,32
81.45
82.45
84.92
85.15
78.57
78.95
84.88
77.50
79.05
84.86
83.31
84.06 84.40
SMAP
71.70
71.38
70.78
69.43
72.88
71.26
71.52
70.76
69.21
69.26
71.09
71.12
71.09
71.18
69,92
70.40
SwaT
95.40
91.12
90.23
91.90
94.23
93.86
91.74
93.19
93.33
87.52
79.88
92.74
91.78
83.10
81.43
PSM
98.88
98.54
95.86
93.96
97.13
97.23
97.47
97.23
97.15
93.55
97.29
93.29
82.08
79.40
77.10
73.61
1 Count
3
0
Classification. Following the standard evaluation protocol from (Zhang et (025a))) We evaluate the Frrain-t TSFM on time series classification using 10 UEA (Bagnall al. 2018) and
8

---

Published as a conference paper at ICLR 2026
UEA Archive
UCR Archive
1766
10-
71.0
70ut
60
75
70
Lasa
Timasher Redernf FoW
Lighrre
Aclineer
fiom
Proeeroreer MONEEST
Festagg
Figure 5: Classification results. Full results in Table 25 & Table 26
Best viewed in color and with z0om.
91 UCR (Dau et al., 2019) subsets spanning diverse domains. We adopt Linear Probing-bai based adaption method by attaching linear classifier to the frozen TSFM, directly measuring the quality of learned representations. As shown in Figure our FeDaal-trained TSFM consistently outperforms all baselines, including task-specific models, GPT4TS (fine-tuned), and Moment. Notably, FeDaL. surpasses FL baselines such as FedAvg and FFTS by 6.4% % on UEA and /3.2% % on UCR, highlighting its ability to learn invariint features and mitigate cross-client biases.
Core-Set Analysis. To better understand the effect of core-set tuning, we visualize the representation spaces of three key stages using t-SNE in Figure 6 The initial core-sets cluster closely around the origi- nal client data, indicating that they effectively approx- imate local distributions, but may also retain sensi- tive fine-grained patterns. After applying Fourier per- turbation and semantic alignment, the refined core- sets move away from the raw data distribution while remaining semantically aligned. This suggests that perturbation strategy can partially obfuscate fine- grained details and reduce direct data leakage risks, while preserving high-level temporal semantics that are useful for global alignment. As a result, the up- loaded core-sets serve compact, privacy-aware sum- maries that enable effective server-side tuning without requiring access to raw client data. More detailed anal- ysis of the t-SNE visualizations for our proposed core-set tuning is provided in Figure 8
Original Data
Initaal Core-ort
Peeturbed Aligned Core-oet
Figure 6: T-SNE visualization of Core-set.
4.3 FEDERATED SCALING BEHAVIORS
While prior work has examined scaling laws of centralized TSFMs YYo et al, 2024), we investigate how federated pretraining scales with respect to data size, number of clients, and client participa- tion rate. Specifically, we vary: (i) data size from 40B to 231B with fixed 174 clients; (ii) client count from 30 to 174 under fixed total data; and (iii) participation rate from 10% to 100%. As shown in Figure 7 larger data consistently improves performance, more clients yield better repre- sentations (even under fixed total data), and higher participation enhances aggregation and mitigates drift. These results indicate that federated TSFM pretraining benefits from scaling in data and client diversity, emphasizing coverage and participation over model size for improved generalization.
4.4 DISCUSSION ON LARGER TIME SERIES FORECASTING MODELS
To further contextualize performance, we compare our FeDaL-trained TSFM with Time-MoE (Shi ct al. 2024), a recent large-scale TSFM designed specifically for forecasting. Time-MoB variants include models with up to 2.4B parameters trained on 300B time series data. We present results on five standard long-horiag oong-horizon forecasting benchmarks Table 20 including each model's parameter count and training data size for visual comparison. Our findings show that FeDaL consistently out- performs Time-MoEaue (113M) and Time-MoEnure (453M), and achieves comparable results to the largest variant, Time-MoFsum. (2.4B). Notably, FeDal. reaches this level of performance with only

---

Published as a conference paper at ICLR 2026
MSE
0.40
MSE
0.40
MSE
0.40
0.45
MSE
0.35
MSE
0.35
0.35
0.40
0.40
0:45
0.45
0.45
0.00
0.03
MSE
0.2
0.04
0.04
0.05
8.
Acc.
65
84
87.5
85.0
80
408
808
1208
1608
2008 2318
110
140
10%
30%
50%
70%
100%
Data Size
Number of Clients
Client Join Rate
Figure 7: Scaling behaviors across tasks. Color codes: Blue Avg. full/few-shot forecasting, Orange Zero-shot forecasting, Green Imputation, Red Classification, Purple Anomaly Detection. Y-axis for impuuatinggmmuutaiiinn is inverted (lower is better). Full details and plots in Appendix C.7
1.8% of the parameters and 70B fewer training samples, indicating substantially better efficiency. To quantify this tradeoff between performance and resource cost inspired by (Kaplan et al.) 2020 Tan & Le 2019), define the Information Gain per Cost (IGC) metric
IGC
MSE x Parameters Count Training Data Size
where a = B = 1 by default. higher IGC indicates a model training with better efficiency.
As shown in Table 8 our FeDaL achieves the highest IGC, outper- forming all Time-MoB variants in terms of cost-effectiveness: FeDaL Time-MoEnaee > Time-MoFlarge Time-MoFuttra This underscores that FeDal. not only delivers strong performance, but does so with su- perior parameter and data efficiency, making it a more scalable and practi- cal choice for real-world deployment. In addition, the TSFM trained with FeDal. demonstrates strong general- ization across diverse tasks beyond forecasting, including classification, imputation, and anomaly detection.
Table 8: Zero-shot long-term forecasting performance com- parison with Time-MoE. Bold: the best, Underline: the sec- ond best. Full results are provided in Table
Models
(Ours)
Tine-MMFF...
Tinn--Mo.argr
Time-MooF,.te
Metries
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.347
0.881
0.357
0
0,350
0.382
0.349
0,379
192
0.398
0410
0.384
0.404
0.583
0.412
0.305
0:413
FTTh
336
0.425
0.422
0.411
0.434
0.411
0.430
0.447
0:453
720
0.457
0.469
0.449
0.477
0.427
0.455
0.457
0.462
Avg.
0.407
0.429
0.600
0.424
0.394
0.419
0.412
07426
96
0.307
01355
0.305
11339
0.300
0.354
0.292
11352
ETTM2
336
192
0.387
0.349
0.372
0.395
0.391
0.351
0.418
0.386
0.417
0.564
0.425
0.365
0.347
0.570
0.419
0.401
0.419
0.557
0.447
0.361
0.366
0.405
0.557
0317
0.388
0.346
0.381
0.370
0.381
0.413
0.773
0.508
0.504
0-475
04477
Avg:
0.260
0.390
0.394
0.410
0.306
0.405
0.566
0.207
0.201
0.291
0.197
0.285
0.98
0,288
ETTm2
330
0.116
0.248
0.333
0.340
0.258
0.324
0.333
0.304
0.250
0.377
0.322
03755
0
0235
0,312
0.348
720
0.307
0
0.488
0.464
D-400
0466
0477
0.423
Avg.
0.292
0.341
0.317
0.365
0.316
0.361
0.288
0.341
5 CONCLUSION
96
0.159
0.12
0.160
0.214
0.199
0.213
0.157
0.211
Westhr
336
192
0.217
0.285
01312
0.364
0.210
0.274
1.300
1000
0.291
03215
0.266
0.322
0.255
0.308
11.20
11255
720
0.359
0.348
0.416
0,405
0:415
0.400
0.405
0.507
We introduced FeDaL, a feder- ated learning framework for train-
0.255
0.263
0.270
0,300
0.288
Averuge
0.382
0.387
0.369
ing TSFMs from scratch that explic-
Total Puram.
28.42 M
itly targets dataset-level heterogene- Training Data
3008
3008
4.081
ity. By integrating DBE at the client
side and GBE at server. FeDaL miti-
gates both local and global biases, enabling the learning of robust, domain-invariant temporal pat- terns. Extensive experiments across diverse time series analysis tasks (various regression and clas- sification) demonstrate its superior cross-domaie learning capabilities and show that FeDaL-trained models remain highly competitive against centralized TSFMs with far larger parameter scales.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We pretrain on publicly available datasets UTSD Liu et al. 2024c). CTSD Chen et al. 2025 and LOTSA (Woo al.] (024). Experiments conducted widely used benchmarks, including TSLib (Wu et al. 2022a) (ETT, Weather, and M4 for forecasting; ETT and Weather for imputa- tion; SMD, MSL, SMAP, SwaT, and PSM for anomaly detection), well as the UEA (Bagnall et al. 2018) and UCR (Dau et al.) 2019) archives. All datasets are publicly accessible. To ensure reproducibility, we detail training setups, model configurations, and hyperparameter choices in Ap- pendix B and Table
9 Furthermore, the code implementation of our proposed FeDaL is released at https: : /tt.//tttuuu..a com/ wwwngchaacheee822/eeaa
REFERENCES
Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. arXiv preprint arXiv:2111.0426. 2021.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language time series. arXiv preprint urXiv:2403.0785 2024.
Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Fed- erated learning with personalization layers. arXiv preprint arXiv: arXiv:1912.00 1ete:9912000088 2019.
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint 1.00075, 2018.
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium- range global weather forecasting with 3d neural networks. Nature, 619(7970):533-5538 2023.
Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler-Canseo and Artur Dubrawski. N-hits: Neural hierarchical interpolation for time series forecasting, 2022. URL /attps://arte /tt.:::://rrtv.orr/aa//////2 wttp://////b........
12886
Ching Chang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Two-stage fine-tuning for time-series
forecasting with pre-trained llms. arXiv preprint (xXix:2308.08400 2023.
Shengchao Chen and Ting Shu. Restyled, tuning, and alignment: Taming vims for federated non-iid medical image analysis. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 530-540. Springer, 2025.
Shengchao Chen, Guodong I Long, Jing Jiang, and Chengqi Zhang. Federated foundation models on heterogeneous time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 15839-15844 2025.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Dan Ciregan, Ueli Meier, and Jürgen Schmidhuber. Multi-column deep neural networks for image classification. In 2012 IEEE conference on computer vision and pattern recognition, pp. 3642 3649. IEEE, 2012.
Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared repre- sentations for personalized federated learning. In International conference on machine learning, pp. 2089-2099. PMLR, 2021.
Marco Cuturi and Mathieu Blondel. Soft-dtw: differentiable loss function for time-series. In International conference on machine learning, pp. 903. PMLR, 2017.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310. 1et.23300.106800 10688, 2023.

---

Published as a conference paper at ICLR 2026
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, 2024.
Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The ucr time series archive. IEEE/CAA Journal of Automatica Sinica, (16))1293-1000 2019.
Angus Dempster, François Petitjean, and Geoffrey Webb. Rocket: exceptionally fast and accu- rate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5): 145--1495, 2020.
Samuel Dooley, Gurmoor Singh Khurana, Chirag Mohapatra, Siddartha V Naidu, and Colin White. Forecastpfn: Syntheticalll-traine zero-shot forecasting. Advances in Neural Information Pro- cessing Systems, 36, 2024.
Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, Chee Keong Kwoh, Xiaoli Li, and Cuntai Guan. Time-series representation learning via temporal and contextual contrasting. arXiv preprint arXiv:2106. 14112, 2021.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with the- oretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33:3557-3568, 2020.
Lulu Feng and Shengehao Chen. Taming vision- vision-language models for federated foundation models on heterogeneous medical imaging modalities. In Proceedings of the 2025 International Confer- ence on Multimedia Retrieval, pp. 303-311, 2025.
Vasilii Feofanov, Songkang Wen, Marius Alonso, Romain Ilbert, Hongbo Guo, Malik Tiomoko, Lujia Pan, Jianfeng Zhang, and levgen Redko. Mantis: Lightweight calibrated foundation model for user-friendly time series classification. arXiv preprint arXiv:2502.1563: 2025.
Jean-Y Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems, 32, 2019.
Azul Garza and Max Bergenthaler-Canseee Timegpt-1. arXiv preprint arXiv:2310.03589, 2023.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: family of open time-series foundation models. arXiv preprint arX8v:2402.03885, 2024.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, arXiv preprint arXiv:2111.00396 2021.
Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik. Lower bounds and opti- mal algorithms for personalized federated learning. Advances in Neural Information Processing Systems, 33:2304-2315, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-17800 1997.
Ming Jin, Shiyu Wang. Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yux- uan Liang, Yuan-Fan Li, Shirui Pan, et al. Time-Ilm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728. 2023.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836/ 2020.
12

---

Published as a conference paper at ICLR 2026
Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training. In International Conference on Machine Learning, pp. 5464-5474, PMLR, 2021.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint (0X1::200100445//1 2020.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 95-104, 2018.
Arthur Le Guennec, Simon Malinowski, and Romain Tavenard. Data augmentation for time ries classification using convolutional neural networks. In ECMI/PKDD workshop on advanced analyties and learning on temporal data, 2016.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer time series forecasting. Advances in neural information processing systems, 32, 2019.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and sys- tems, 2:429-450, 2020.
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In International conference on machine learning, pp. 6357- 6368. PMLR, 2021a.
Xiaoxiao Li, Mcirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. Fedbn: Federated learning on non-onn-ee iid features via local batch normalization. arXiv preprint arXiv:2102.07623, 2021b.
Qingxiang Liu, Xu Liu, Chenghao Liu, Qingsong Wen, and Yuxuan Liang. Time-ffm: To- wards Im-empowered federated foundation model for time series forecasting. arXiv preprint arXiv:2405. 14252, 2024a.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting, In International conference learning representations, 2021.
Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. Unitime: A language-empowered unified model for cross-domain time series forecasting. In Proceedings of the ACM on Web Conference 2024, pp. 4095-4106, 2024b.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881-9899 2022.
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long- Timer: Transformers for time series analysis at scale. arXiv preprint 2rXiv:2402.02366 2024c.
Mantas Lukoševicius, Dan Popovici, Herbert Jaeger, Udo Siewert, and Residence Park. Time warp- ing invariant echo state networks. International University Bremen, Technical Report, 2006.
Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The twelfth international conference learning representations, pp. 1-43, 2024.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficinen learning deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. PMLR, 2017.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.144: 2022.
13

---

Published as a conference paper at ICLR 2026
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting, 2020. URL https://arxiv.
1905 1xc/ 10437
Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time- moe: Billion-scale time series foundation models with mixture of experts. arXiv preprint arXiv:2409.16040. 2024.
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International conference on machine learning, pp. 6105-6114 14. PMLR, 2019.
Sana Tonekaboni, Danny Eytan, and Anna Goldenberg. Unsupervised representation learning for time series with temporal neighborhood coding. arXiv preprint arXiv:2106.00050, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Shiyu Wang, Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. Timemixer+4: general time series pattern machine for universal predictive analysis. arXiv preprint arXiv:2410. 16032, 2024a.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405. 2024b.
Yihe Wang, Nan Huang, Taida Li, Yujun Yan, and Xiang Zhang. Medformer: A multi multi-granularity patching transformer for medical time-series classification. arXiv preprint arXiv:2405. 19363, 2024c.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting. arXiv preprint arXiv:2202.01388,, 2022.
Gerald Woo, Chenghao Liu, Akshat Kumar, and Doyen Sahoo. Pushing the limits of pre-training for time series forecasting in the cloudops domain. arXiv preprint arXiv:2310.05063, 2023.
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sa- hoo. Unified training of universal time series forecasting transformers, 2024. URL https:
1/arxiv.org// abs/
awtaaaa/445 22002242200002 02592
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances neural information processing systems, 34:22419-22430 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Tem- poral 2d-variation modeling for general time series analysis. arXiv preprint urXiv:2210.02116 2022a.
Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Flowformer: Linearizing transformers with conservation flows. In International Conference on Machine Learning, 2022b.
Xian Wu, Stephen Mattingly, Shayan Mirjafari, Chao Huang, and Nitesh V Chawla. Personalized
imputation on wearable-sensory time series via knowledge transfer. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 1625-1634, 2020.
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series
anomaly detection with association discrepancy, 2022. URL http::://rxiv..rr////
2110.022 02642
Peng Yan, Shengchao Chen, and Guodong Long. Federated intelligence in web: A tutorial. In Companion Proceedings of the ACM on Web Conference 2025, pp. 61-64, 2025.
Qingren Yao, Chao Huck Yang, Renhe Jiang, Yuxuan Liang, Ming Jin, and Shirui Pan. Towards neural scaling laws for time series foundation models. arXiv preprint arXiv:2410. /2360, 2024.
14

---

Published as a conference paper at ICLR 2026
Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 8980-8987 2022.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
Haoran Zhang, Yong Liu, Yunzhong Qiu, Haixuan Liu, Zhongyi Pei, Jianmin Wang, and Mingsheng Long. Timesbert: A bert-style foundation model for time series understanding. arXiv preprint arXiv:2502.21245, 2025a.
Haoran Zhang, Yong Liu, Yunzhong Qiu, Haixuan Liu, Zhongyi Pei, Jianmin Wang, and Mingsheng Long. Timesbert: A bert-style foundation model for time series understanding, 2025b. URL
sttts::/arxiv.orr/bbbaa
21245
Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, and Jose M Alvarez. Personalized feder- ated learning with first order model optimization. arXiv preprint arXiv:2012.08866 2020.
Sixin Zhang, Anna Choromanska, and Yann LeCun. Deep learning with elastic averaging sgd. Advances in neural information processing systems, 28, 2015.
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mip structures. arXiv preprint arXiv:2207.00 0/ 186, 2022.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106-11115, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-tend series forecasting. In International conference on machine learning. pp. 27268-27286. PMLR, 2022.
Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advances neural information processing systems, 36:43322-44355, 2023.
Weiming Zhuang, Chen Chen, and Lingjuan Lyu. When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306. 15546, 2023.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
This Appendix provides supplementary information and implementation details omitted from the main text, including:
More Related Work (Appendix A comprehensive review of relevant literature, covering time series foundation modcls, FL for heterogeneous data, and foundation models.
Implementation Details (Appendix B): Detailed descriptions of the techniques employed, training configurations, experimental setup, including benchmark procedures, data process- ing pipelines, and baseline methods.
* Full Results (Appendix
Complete presentation of all results discussed in the main text.
A MORE RELATED WORK
Pre-trained Time Series Models. Large Language Models (LLMs) have demonstrated strong capabilities such zero-shot adaptation (Jin et al. 2023). Yet, due the heterogeneity of time series, GPT-style TSFMs remain underexplored. Existing research can be grouped into two cate- gories. The first leverages LLMs for time series analysis. For example, FPT (Zhou al.) 2023) uses GPT-2 as a representation extractor, while LLM4TS (Chang et al. (2023) encodes senes into numerical tokens, showing scalability on forecasting tasks. Time-L.L.M (Tin et al., 2023) employs prompting techniques to enhance cross-modality reasoning, and UniTime (Liu et al. 2024b) intro-
duces domain adaptation to reduce prediction bias. However, these approaches largely depend on the LLM backbone and cross-modal design. In contrast, our model is pre-trained natively on time series, avoiding extra modality alignment. The second line pre-trains directly on large-scale time series datasets. ForecastFPN (Dooley al. 2024) trains synthetic series for vero-shot forecast- ing, while CloudOps (Woo et al. 2023) applies masked modeling for domain-specific prediction. TimeGPT-1 (Garza & Mergenthaler-Caneee
2023) represents the first commercial zero-shot API, and PreDeT (Das et al. (2023) shows strong results with a decoder-only Transformer. More recently, models such MOMENT (Goswami al., 2024), Moirai (Woo et al.] 2024). TimeFM (Das al. 2024), Chronos (Ansari et al. (024)), and Timer (Liu et al. (024c) have been trained on ultra-large datasets and publicly released, achieving competitive cross-task performance. These efforts collec- tively highlight both the potential and the resource-intenge nature of training TSFMs.
Federated Learning for Heterogeneous Data. FL. McMMaan et al. 2017 Feng & Chen 2025 Chen & Shu 2025 Yan et al. 2025) enables decentralized model training without raw data sharing. but suffers from statistical heterogeneity (non-IID data), where distributional shifts across clients degrade performance. Early approaches reduce client-server divergence via alignment or regular- ization, yet typically assume mild heterogeneity and shared representation spaces, an assumptions that collapse in highly diverse domains. Personalized Federated Learning (PFL) extends FL by tai- loring models to individual clients through regularization (Hanzely et al., 2020 Li et al.) 202 la) partial parameter sharing (Li al. 2021b Collins al. 2021). adaptive ag aggregation (Zhang al., 2020), or meta-learning (Fallah al. | 20200. While effective for vision or text tasks where low-level features transfer well (Chen et al. 2025). PFL less suitable for time series, where dataset-level dis- parities in resolution, semanties, and physical context more severe. Moreover, PFL emphasizes personalization over unified generalization, conflicting with the goal of pretraining TSFMs, which require cross-domain invariance.
B IMPLEMENTATION DETAILS
Decoder-only Transformer as Local Models We employ the Transformer as the model backbone due to its excellent scalability. Inspired by the significant advancements in Decoder-only LLMs ca-
pable of iterative generation (Garza & Mergenthaler Canseeco 2023 Zhou al. 2023), and recog- nizing the need for processing variable-length time series (Liu et al. 22244c et al. (025b), we
adopt auto-regressive approach for generative pre-training using standard Decoder-only Trans-
16

---

Published as a conference paper at ICLR 2026
former architectures. The next-token prediction can be formulated as:
N P(U) II p(u; ues), where U = {2,, U2 JN 1=1
(10)
For the tokenization of given input time series X, we employed segment-wise tokenization strat- egy, representing X as #2.......... (2.,S.....,N with a unified context length NS. In this approach, time series token is defined as consecutive segment of length S, covering the series variations: (]( .Iis} E R where i ......N. .N. Subsequently, time series segment is incorporated into the learnable position encoding, followed by the standard autoregressive Trans- former update step. Utilizing the causal attention mechanism of the Decoder-only Transformer, the model autoregressively generates the subsequent segment 8,+1 based on the previous segment s. Consequently, generative pre-trained models endowed with the flcxibility to handle vari- able context lengths during inference and excel at multi-step generation through iterative sliding and enlarging of input tokens.
Fourier-based Perturbation for Privacy Preservation To ensure privacy during core-set trans- mission, we introduce a Fourier-based perturbation mechanism that obfuscates sensitive raw tem- poral patterns while preserving high-level semantic structure. This builds on the observation that in time series, the phase of the Fourier transform captures semantic trends (e.g., shape, rhythm), while the amplitude encodes finer-graine details (e.g., scale, local fluctuations). Formally, given a core- set C. apply discrete Fourier transform (DFT) to obtain its frequency-domain representation F(C) = A + iP, where A and P denote amplitude and phase, respectively. We then perturb only the amplitude component:
č= = -A@@e
(11)
where A and P denote amplitude and phase, and C is the elementwise product. Gaussian noise only the amplitude:
We then add
A+
E E~,(0,1),
(12)
and reconstruct the perturbed core-set by
C' (A
(13)
where controls the perturbation strength and FF1 is the inverse DFT. To keep C' semantically faithful to the client data while remaining privacy-preserving, we align its latent representations to those of the sampled mini-batch X' (both of size K) via
Lalign (C')
1 K
Σ So(c) CEC
1 K
1
(14)
ZE
where ff(-) denotes the encoder that outputs latent embeddings. The semantically aligned core-set C, obtained by optimizing Lalign w.r.t. C², is then uploaded to the server for core-set tuning.
To further examine the semantic fidelity and privacy robustness of our core-set pipeline, we visualize the I-SNE embeddings of three data subsets for several clients:
T-SNE Visualization on Core-set
(i) the original local mini-batch X used for core-set construction, (ii) the gradient-matched core-set Cinit, and (iii) the final perturbed and aligned core-set C' sent to the server. As shown in Figure 8 the initial core-set (green) largely overlaps with the original batch (pink), indicating that Cinit effec- tively approximates local gradients but may still retain fine-graines information shared directly. After applying Fourier-based perturbation and semantic alignment, the refined core-set (blue) be- comes more separated from the raw data distribution while preserving overall cluster structure. This suggests that our perturbation mechanism reduces exposure of fine-grained data characteristics while maintaining high-level semantic consistency required for server-side alignment. Overall, the results indicate that the proposed pipeline achieves a practical duility-prvace trade-off without claiming formal privacy guarantees. We note that our perturbation mechanism mitigates but does not elimi- nate potential information leakage, and formal differential privacy guarantees remain future work.
17

---

Published as a conference paper at ICLR 2026
Original Dataset Clients
Initial Core-set
Perturbed Aligned Core-sct
t-SNE visualization core-set across clients. Each subfigure corresponds to a different client. The initial core-set closely aligns with the local data, while the perturbed core-set shows semantic similarity with added crivacy-preservigg shifts. Best viewed in color and zoom-in.
Figure 8:
Time Series Decomposition Time series decomposition separates sequence into interpretable components, typically trend, seasonal, and residual parts, enabling models to capture structured temporal patterns while isolating noise biases. In our framework, adopt the decomposition strategy from (Wu et al.) 2021), but apply it directly in the latent representation space instead of the raw input domain. Given the latent representation = (xx E Lxd we decompose it into trend and seasonal parts using 7-point moving average operator:
MA,(h), h, h h- h.
(15)
Here, h, captures low-frequency temporal dynamics, while h, represents higher-frequency struc- tured variations. The residual component omitted to avoid modeling unstructured noise. To estimate persistent dataset-specife deviations, average each component over the temporal di- mension:
Mean(h)), Mean(h.),
(16)
where b, and b, are compact approximations of dataset-level biases arising from heterogeneous time series distributions.
Training Configuration We evaluate FeDaL from two complementary perspectives: (1) its abil- ity to learn generic representations in heterogeneous, distributed settings (referred to as Feder- ated Representation Learning), and (2) the transferability of the learned TSFM to various down- stream time series tasks (Downstream Adaptation). The detailed configurations for both tasks
18

---

Published as a conference paper at ICLR 2026
summarized in Table For federated representation learning, the UTSD and CTSD datasets to assess FeDaL under two forms of cross domain heterogeneity: domain-mixed (DM) and (DI). For downstream adaptation, we pre-train the Decoder-only Transformer via FeDal. on LOTSA, a large-scale cross-domain dataset comprising 231 billion time points, to ensure broad coverage of temporal patterns prior to adaptation.
Table 9: Detailed training configuration across various tasks.
Configuration
Configuration
Category
Federated Representation Learning
Pretruining for TSFMs
Optimizer
SGD
SGD
Number of Clients
10 (UTSD) 18 (CTSD)
174 (LOTTS))
Batch Size
2048
2048
Client Join Ratio (0)
70%
70%
Local Epochs
10
10
Global Rounds
200
1000
Learning Rate (Local Updating)
0.01
0.01
Core-set Generation Epochs
5
5
Learning Rate (Core-set Generation)
0.005
0.01
Server-side Tuning
15
Learning Rate (Scrver-side Tuning)
0.01
0.01
Input Sequence Length
1024
3072
Learning Scheduler
StepLR
StepLR
Computation Devices
2 x Nvidia RTX A5500-24GB GPUs
16x Nvidia A100-80G GPUs
Smoother Factor (11)
0.2
0.1
0.7
0.7
0.1
0.1
(Alignment)
0.01
0.1
Core-set Size (K)
1024 (< 1%)
3072 (< 0.01%)
Decompusition Period (7)
4
8
Model Dimension
512
768
Number of Heads
4
8
Decoder Layers
4
8
Feedforward Dimension
512
768
Drupout
0.35
Activation Function
GELU
GELU
Masking Ratio
75%
50%
Patch Length
32
64
Stride
32
32
Evaluation Pipeline
Masked Patches Reconstruction
Downstream Adaption
Long-term Forecasting (Fuls-/Zero-shot)
Short-term Forecasting
Downstream Tasks
N/A
Classification Anomaly Detection
Benchmark Details We evaluate FeDaL across two major settings: Federated Representation
Learning and Downstream Adaptation. For Federated Representation Learning, use two large- scale cross-domain time series datasets: (1) UTSD containing over billion time points across domains (e.g., energy, environment, health, loT, nature); and (2) CTSD sampled from the Monash Time Series Forecasting Repository, comprising 500 million time points from 6 domains includ- ing ergonomics, transportation, health, energy, nature, and web. For Downstream Adaptation, we pretrain on the large-scale LOTSA dataset (Woo et al. 2024 3 which contains 231 billion obser- vations spanning multiple domains. We evaluate the pretraine TSFM from FeDaL. on four down- stream tasks: long- and short-term forecasting, imputation, classification, and anomaly detection. For long-term forecasting and imputation, we follow the GPT4TS (Zhou et al., 2023) protocol, using ETTh1, ETTh2, ETTml, ETTm2, Weather, and Illness datasets (details in Table 11). For short-term forecasting, we adopt the M4 dataset, again following GPT4TS, covering various tem- poral granularities (details in Table 12). For classification,
follow the TimesBERT (Zhang et al.) 2025b) setup, using 10 representative datasets from UEA and 91 from UCR to cover diverse do- mains (see Table 13). For anomaly detection, adopt the FFTS (Chen et al. 2025) benchmark,
19

---

Published as a conference paper at ICLR 2026
evaluating SMD, MSL, SMAP, SWaT, and PSM datasets (details in Table 14 overlapping with LOTSA have been excluded from the pretraining phase.
Any datasets
Table 10: Dataset statistics about the CTSD dataset. The channels indicates the number of time series. 'Min' and 'Max' denote the shortest and longest sequence lengths, respectively, while 'Fixed' signifies datasets with uniform sequence lengths.
Domain
Dataset
# Channels
Frequency
Length
Bitcoin
18
Daily
Min: 2659 Max: 4581
Economic
FRED-MD
107
Monthly
Fixed: 728
NN5
111
Daily
Fixed: 791
Pedestrain Counts
66
Hourly
Min: 576 Max: 96424
Transport
Rideshare
2304
Daily
Fixed: 541
San Francisco Traffic
862
Hourly/Weekly
Fixed: 17544
COVID Deaths
266
Daily
Min: 212 Max: 212
Health
Hospital
767
Monthly
Fixed: 84
London Smart Meters
5560
Daily
Min: 288 Max: 39648
Wind Farms
339
Minutely
Min: 6345 Max: 527040
Energy
Wind Power
Second
Fixed: 73971 147
Electricity
321
Hourly/Weekly
Fixed: 26304
KDD Cup 2018
270
Hourly
Min: 9504 Max: 10920
Nature
Oikolab Weather
3010
Daily
Min: 1332 Max: 65981
Temperature Rain
32072
Daily
Fixed: 725
Web
Web Traffic
145063
Daily
Fixed: 803
Table 11: Dataset statistics about long-term forecasting (imputation) dataset. The channels indicates the number of time series (i.e., variables), and the size is organized in (training, validation, testing). Note that Illness dataset is not included in the imputation task.
Dataset
Domain
# Channels
Frequency
Size
Forccast Length
ETThl
Power
Hourly
(8545, 2881, 2881)
(96, 136, 720)
ETTh2
Power
7
Hourly
(8545, 2881, 2881)
(96, 192, 336, 720)
ETTml
Power
Minute
(34465, 11521, 11521)
(96, 192,,36, 720)
ETTm2
Power
7
15 Minute
(34465, 11521, 11521)
{96, 192, 336, 720)
Illness
Epidemiology
Weekly
(617, 170)
{24, 36, 48, 60)
Weather
Weather
Minute
(36792, 5271, 10540)
(96, 192, 336, 720)
Table 12: Dataset statisties about short-term forecasting dataset. The channels indicates the number of time series (i.e., variables). and the size is organized in (training, validation, testing).
Dataset
Domain
# Channels
Frequency
Size
Forecast Length
M4- 14-Yearly
Demographic
Yearly
(23000, 23000)
M4- 14-Quarterly
Finance
I
Quarterly
(24000, 0. 24000)
8
M4-Monthly
Industry
Monthly
(48000, 48000)
M4- M4- 4e-eeekly
Macro
Weekly
(359, 0, 359)
13
M4-Daily
Macro
Daily
(4227, 4227)
14
M4-Hourly
Other
Hourly
(414, 0, 414)
48
20

---

Published as a conference paper at ICLR 2026
Table 13: Dataset statistics about classification dataset. The channels indicates the number of time series (i.e., variables), and the size is organized in (training, validation, testing).
Dataset
Channels
Series Length
Dataset Size
Information (Frequency)
EahanolComsentraation
1751
(261, 0, 263)
Alcohol Industry
FaceDetection
144
62
(5890, 0, 3524)
Face (250Hz)
Handwriting
152
(150, 0, 850)
Handwriting
Heartheat
405
(204, 0, 205)
Heart Beat
Japanese Vowels
12
29
(270, 0, 370)
Voice Transportation (Daily)
PEMS SF
963
144
(267, 0, 173)
SelfRegulationSCPP
6
896
(268, 0, 293)
Health (256Hz)
SelfRegulationSCCP2
1152
(200, 0, 180)
Health (256Hz)
SpakenArraicDDigiss
13
93
(6599, 0, 2199)
Voice 1025Hz)
UWaveGesturel .ibrary
3
315
(120, 0, 320)
Gesture
UCR Archive
(*, 0, *)
Table 14: Dataset statistics about anomaly detection dataset. The channels indicates the number of
time series (i.e., variables), and the size is organized (training, validation, testing).
Dataset
Channels
Series Length
Dataset Size
Information (Frequency)
SMD
38
40
(566724, 141681, 708420)
Server Machine
MSL
55
40
(44653, 11664, 73729)
Spacecraft Spacecraft
SMAP
25
40
(108146 (008146.20 27037, 427617)
SWaT
51
40
(396000, 99000, 449919)
Infrastructure
PSM
25
40
(105984, 26497, 87841)
Server Machine
Data Processing Pipeline Given the varying lengths of time series in the pretraining dataset, some sequences are too short to support the full sequence length used during pretraining (3072 points). Moreover, despite the large scale of LOTSA, we aim expose the model to as diverse a of tem- poral patterns as possible. To address these challenges, we introduce an efficient dataset processing pipeline for client-side pretraining. This pipeline (I) filters out invalid or insufficiently long time series samples, (2) augments the original time series enrich temporal diversity, and (3) flexibly handles multi-domain time series with varying numbers channels or variables a unified manner. The source code for the unified client-side data processing pipeline is provided below.
class VederatedDaaaaet__Rppeeeee Rewreeeneatiinnn (Dataset): def iiii (self, args, series_datal dict, mode='train',
Is='normal',
5
scale-True,
split_ratio=0..
sei self.initin @elf.initiallle__areeeet (args, mode, 1s, scale) self.proces__atalsereee__aa split_ratio)
10
def Baliaaizz_paramttere (self, args, mode, 1s, scale)
self.args args
self.mode mode
self.ls ls
14
sclf.scalc = scale
self.
augss.ubset___.
self self.window_stridd args. www.wwwwnwwwiiia.
18
self.seg_len args. seq_len
19
self.pred_len = prea_prea_ien
self.total len =
+ self pred__ee____
def process_data (self, series_data: dict, split_ratio: flcat):
21

---

Published as a conference paper at ICLR 2026
processed_see series skipped_series
serries
[]
24
items_ter Ltems_to_process 1ist iiveries_dataitten ()
for key, series iteme_to_prrcess::
28
required_len len self.seq_len
if 1en (series) < required_lent
skipped skipped_series series += continue
33
if scaler - standardScaler
series faaler.
n f_feeuures scriee.shape[]
38
for feature_ids idx in range (n features):
feature_series = seriesl:, feature_idxl
windows self. create widdws Teature_series reshape 1),
required_len, weff.windowsstrie
processed series. extend (windows)
47
if len (proccessed == 0:
raise ValueError NoNo valid sequences found after length filtering!")
50
self.data data = np. arrarray (processed_series)
split_ide idx int (len felef.ddtat split_ratio)
def Len (sel5) return (self.
55
def _etitemm_ index)
seg_x self.
return torch. froch.froonnungpp (seq_x)
@staticmethed
def create create_window windows (series, sindow_size, stride=32)
windows []
64
for 1 in range (0, len (scrics) window_sizo 1, stride):
windows append (scrics + window_sizel) return wi windows
Data Partitioning in Federated Time Series Representation Learning Learning domain- invariant representations from unlabeled, heterogeneous time series is crucial for advancing TSFMs in federated settings. To evaluate FeDaL. in this context, we adopt two large-scale cross-domain benchmarks: UTSD (Woo et al. (004)) (1B time points across 7 domains) with domain-mixed (DM) partitioning, and CTSD (Chen et al. 2025) (500M time points across 6 domains) with domain- independent (DI) partitioning. In DM setting, each client receives data drawn from multiple domains, while in the DI setting, each client is restricted to single domain. For better illustra- tion. examples of both partitioning strategies are shown in Figure To further assess FeDaL under varying degrees of heterogeneity, denoted HI and H2. These are generated using a customized data-loading pipeline that controls het- erogeneity through imbalance factor (parameter: i imbal 1 ance_ conce_factee factor) (set to 0.1 for H1 and 0.3 for H2) in the static method _split st..ssriees... This allows evaluation of FeDaL's robustness across environments with increasing domain skew.
construct two additional DM-based configurations
UTSD,
22

---

def [et_max_clientsscll dataset_nang str) -> int: if dataset_nams not in BEA.SSTNN____AAASSSSS CONFIGS: raise ValueError (f"lataset [dataset_name) found in Published as a conference paper at ICLR 2026
class Tetleppssenaet.oonbtseeccooo
DATASET REGISTRY: Dict [str, strl
"UTSD": "UTSD"
SINGLE_DATASTT_CORIISS [str, Dict]
"UTSD (VrsD_unbalance_ =
"dataset": "UTSD", "max_clients"; 10, "split_method"; "random", "balanced": False, " "imbalance_factor"" 0.3 by default
1
13
"UTSD unbalance_ _1":
"dataset"; "UTSD", " "max_clients"; 10, "split_method"; "random", "balanced"; False, Limbalance_Factort" 0.1 # by default
18
22
Belassmethod
+ configurations") return cls. SINGLE CONEIIS dataset name "mmax clients")
26
@classmethed
def get_total_series (cls, dataset_name: str) int:
30
if dataset_name not in cls. SINGLE_DATASET_OOOEEOOOOOO IGS: raise ValueError (f*Dataset {dataset_name) not found in conEigurations")
dataset_ DWINNNNEDATAAETTLCCCCCCCCCCCCCEE
4 SINGLE file_path = os. pat.. join 'daaasss'', 'crossdomain', ['lsss DATSSET DATASET REGISTRY dataset keyy npz')
dataset_nane ("dataset")
with np. load return le ddaaa.kyyssa... ())
as data:
@classmethed
def jet__srree_cclentt ccientt_ceeett_c data (cls, dataset name: str, client_id: int): if dataset_name not in SINNCLEEDATAAEETTTTEEEEEEEEEEEEE raise (f [dataset_name) not configured for series-based splitting")
config cls. SINGLE BANGL__DATASE_COOO IGS [dataset_name] if client_id >- ("Fxigmmmaclleeee raise ValueError (I*Ciient ID (client_io id) exceeds maximum clients (configl 'max "))
48
dataset_key
+ BEALNNGLEDATTATTLLOOOOO [ONTASST_CONIIGSsddtatteeeeeee [dataset_name) ["dataset")
file_path os.path. join ''dttasts', 1 'erossdomain',
f' DATASET REGISTRY dataset__eyalasssae___e split_methor config("split_metti
return sps._split_series (filc_path, split_method, client_id,
config "m____mmmn__cc_es_________ configl"balanced"
config "imballace__ffcttrrr
54
@stat i cmethod
23

---

Published as a conference paper at ICLR 2026
cerit_ssrieesfilll path: str, split_method: method: str, client_ia id: - int, max_clients: int, balanced: bool True, imbalance_factor: float = 0.3):
with np. ap._oaa(Eile_pathh
data:
keys list (data.keys
58
n_series lan (keys)
if split_method method == "random":
np. random. seed (42)
shuffled keys np. randem. www.raddn.eemmtatiioo.. (keys)
if balanced:
series_per clientllettttcee n_series // mlienti
extra_series n_series max_clients
start_ide client_id series_per_cliem, 4 min extra_series) end_idx start_idx series_per_elient + (1 if
client_ic extra_series else 0)
70
else:
alpha - np. np. ones (max_cl ients) * (1.0 /
imbalance_facton
proportions mp.randem. dirichlet (alpha)
series _erutssoosssssee (proportions
4 n_series) ast.ype
series_counts [-1] n_series
sum [ercscccccccsscctttll--
start_idx np. sum tep.sutterriessccutslllllll ww..um/essiiscccutteccceee..o...
76
end_idx start_idx serice_ (client_id)
cl client_series = {}
for key in shuffled_keysl
client_series [key] data [key]
else:
81
raise ValueError (E"Unknown split method: soolit_methoc))
return client_series
Baseline We compare proposed FeDaL against 54 different baselines in our experiments to demonstrate its effectiveness and superiority federated time series representation learning and downstream time series analysis tasks. In this section, introduce of these representative baselines from both federated learning and deep time series modeling. Full information of basclines can be found in T5ble The key federated learning basclines include:
FedAvg (McMahan et al. 2017): decentralized approach that enables devices to col- laboratively learn shared model by aggregating the training of deep networks on private and large datasets while reducing communication costs.
updates, allowing for
FedProx Li al. 2020):
A federated learning framework that generalizes and parameterizes FedAvg to tackle systems and statistical heterogeneity, providing conver- gence guarantees and demonstrating more robust and accurate convergence behavior.
FedPer (Arivazhagan et al. 2019): A federated learning baseline that personalizes models by keeping client-specific layers local while training shared base layers collaboratively.
FedRep (Collins al. 202T): A federated learning baseline that decouples learning into global feature representation (shared) and client-specific heads (local) for better personal- ization.
FFTS (Chen et al. 2025) FL-based decentralized framework for training time series foundation models from scratch, which incorporates Mixtureoof-Expertt mechanism
24

---

Published as a conference paper at ICLR 2026
Table 15: Baseline information pertains to federated representation learning and its applications in
forecasting, imputation, anomaly detection, and classification for downstream adaptation. Regarding Foundation Model class, are employed exclusively for the zero-shot long-term forecasting task, except for Moment, which is applied across the entire scenario.
Task
Baseline
Venue
Representation
Forecasting
Imputation
Anomaly
Classificttion
FedAvg
AISTATS
2017
Methods
FedProx
MLSys 20
X
FedPer
arXiv' 19
2019
X
X
x
FedRep
ICM1:21
2021
FFTS
25AI'25
FeDaL (Ours)
This paper
TimeMixer
2024b
ICLR'24
2024b
X
X
GPT4TS
NeurlPS 23
2025
PatchTST
ICIRR23
2122
TimesNet
ICLR'23
20224
AAAI'2
Fedformer
ICML 22
Autoformer
NeurlPS221
Stationary
Neur
LightTS
arXiv'22 3222
X
Informer
AAAI' 2021
Reformer
ICLR'20
Deep Models Traditional Machine Learning Methods
Time-LLM
ICLR
X
N-HITS
AAAI'23
202
N-BEATS
ICLR'20
ETSformer
arXiv'22
202
X
MTCN
ICLR'24
2044
Pyraformer
ICLR'22
Anomaly
ICLR'22 22
X
DTW
ICME 17
2017
TS2Vec
AAAT
T-Loss
NS
2019
X
TNC
ICLR
TS-TCC
UCAL211
TST
X
CNN
Encoder
FCN
X
X
MCDNN
CVPR'S 12
2012
ResNer
CVPR 16
2016
t-LeNet
BCML 16
2010
TWIESN
TR'06 2005
MI.P
XGBoost
KDD' 16
2016
Rocket
DMKD 20
2020
LSTM
NC'97
1997
LSTNet
SIGIR 18
2018
Transformer
NeurIPS
2017
Flowformer
ICML 22
20226
LogTransformer
NeurlP
2019
TCN
NeurPP 19
2019
X
Foundation Model
Moment
ICML'24
2024
V
Moiraismotl
ICML'24
2024
Morialaoss
ICML'24
Morialzagge
JCML'24
MIZA
TimesFM
ICML'24
3024
Chronos
TMLR
2024
X
Chronospasc
TML R 24
2124
x
Chronosiarge
TMLR
2024
25

---

Published as a conference paper at ICLR 2026
Domain-mixed (DM)
Domain-independent (DI)
Client 1
Client
Client 2
Client
Traffic
Climate
Energy
Client N
Client N
Healthcare
Web
Figure 9: Comparison between Domain-Mixed (DM) and Domain-Indepennee (DI) settings. In DM, each client contains time series from multiple domains. In contrast, in DI, each client contains time series from single domain, though the same domain may appear across different clients with non-overlapping sequences.
each client and employs a heterogeneous knowledge alignment strategy address time series heterogeneity.
Details of deep time series model baseline is as follows:
Log LogTrranffommrr (Li et al., 2019): A modified Transformer architecture that addresses the locality-agnosties and memory bottleneck issues time series forecasting by incorporat- ing convolutional self-attention and log-sparse. attention, achieving improved forecasting accuracy with reduced memory cost.
N-BEATS Oreshkin et al. 2020): A deep neural architecture that achieves state-of-the-a performance univariate time series point forecasting, using a stack of fully-connected layers with backward and forward residual links.
Reformer (Kitaev et al., 2020): This model improves Transformer by using locality- sensitive hashing for attention and reversible residual layers. It offers better memory ef- ficiency and speed lengthy sequences without sacrificing performance.
Informer (Zhou et al. 2021): An optimized Transformer-based model for long-range time series prediction. effectively, and employs a fast prediction decoder.
ProbSparse self-attention for efficiency, processes long inputs
LighTTS (Zhang et al., 2022): lightweight MLP structure. It utilizes two downsampling strategies-spacee and sequential sampling-on the MLP structure, capitalizing on the fact that downsampled time series generally maintain most of their original information.
ETSformer (Woo et al. 2022): A Transformer architecture that leverages the principle of exponential smoothing improve traditional Transformers for time-series forecasting. offering better decomposition capability, interpretability, and long-term forecasting effi- ciency.
Stationary (Liu et al. 2022: : framework that addresses the see-staaionaaizztion prob- lem in time series forecasting by combining Series Stationarization and De-stationary At- tention modules, which unify statistics for better predictability while recovering intrinsic non-stationary information.
26

---

Published as a conference paper at ICLR 2026
Autoformer (Wu et al. 2021) A decomposition architecture that leverages leveraging Auto-Core Cato-Correlation mechanism for long-term forecasting, which efficiently discovers depen- dencies and aggregates representations at the sub-scries level.
FEDformer (Zhou et al,, 2022 forecasting method that combines seasonal-tren de- composition with Trequency-enhancee Transformer, capturing both the global profile and detailed structures of time series.
Pyraformer (Liu et al. 202T): It features hierarchical pyramidal attention modules with binary trees to capture temporal dependencies across different ranges efficiently, both in time and memory complexity.
moomalyTransfomme (Xu et al., 2022): approach for unsupervised time series anomaly detection that leverages the self- cell-altention mechanism to compute association discrepancy, which captures the adjacent-concentration bias anomalies.
TimesNet (Wu et al. )022a): A framework that transforms ID time series into 2D tensors to model complex temporal variations, leveraging the multi-periodicity of time series to adaptively discover and extract intraperiod- and interperiod-variations..
PatchTST (Nie al. 2022): This method divides the time series into patches at the sub- series level for input to the Transformer. Each channel holds univariate time series, shar- ing the same embedding and Transformer weights across all series.
DLinear (Zeng et al. 2023): DLinear integrates decomposition schemes from Autoformer and FEDformer with linear layers model time series data tables. effectively summa- rizes trend and seasonal components, enhancing performance on datasets rich in trends.
N-HiTS (Challu 2022): A forecasting model that addresses the challenges of long- horizon forecasting by incorporating hierarchical interpolation and multi-rate data sampling techniques.
GPT4TS (hoo et al. 2023) : This model is designed for time series analysis across various scenarios, achieved by fine-tuning a pre-trained language model, specifically GPT2, for the time series domain.
Time-LLM (Jin et al. 2023 reprogramming framework that repurposes large language models (LL for general time series forecasting by aligning time series data with natural language modalities through text prototypes and Prompt-as-Prefh Preffxx (PaP) techniques.
TimeMixer (Wang et al. 2024b); An advanced time series forecasting baseline that em- ploys multi-scale mixing architecture with Past Decomposition Mixing (PDM) and Future Multi-predictor Mixing (FMM) modules to effectively disentangle and integrate seasonal and trend patterns.
The code is available at https://ancnymouut 4open. science/ E/FeDaL- 3DC2 This anonymous code repository intended for review purposes only.
Code Implementation
Evaluation Metrics For evaluation metrics in forecasting and imputation tasks, we utilize the mean square етrоr (MSE) and mean absolute error (MAE) for long-term forecasting. In terms of the short-term forecasting
M4 benchmark, error (SMAPE), mean absolute scaled error (MASE), and overall weighted average (OWA) as in N-BEATS. Note that OWA is specific metric utilized in the M4 competition. The calculations of
adopt the symmetric mean absolute percentage
27

---

Published as a conference paper at ICLR 2026
these metrics are follows:
1 T [YY,
MSE
(Y
(
H
MAE 1 H
Σ
Y Ynl.
H
200
H
YA Ys!
SMAPE
H
YAl + +YA
(17)
100
Y Yal.
MAPE
H
H
Y Yn
MASE
Σ
H
j--
h=1 H
SMAPE
MASE
OWA SMAPENaive MASE
where is the periodicity of the time series data. H denotes the number data points (i.e., prediction horizon in our cases). Y, and Y, are the h-th ground truth and prediction where {1, ... H...
In addition, we used Precision (P), Recall (R), and F1-Score (F1) to simply quantify the performance of our FeDal. and baselines on the anomaly detection task, these can be formulated as:
TP
P
TP + FP TP
R TP FN
(18)
P R FI = 2 X P+R
where TP (True Positives), FP (False Positives), and FN (False Negatives) represent the number of samples correctly labeled as anomalous, the number of samples incorrectly labeled as anomalous, and the number of samples that were not labeled as anomalous by the model but were actually anomalous, respectively. For classification tasks, we used Accuracy evaluate the performance.
C FULL RESULTS
This section presents the complete results across full-shot and zero-shot long-term forecasting, short- term forecasting, imputation, anomaly detection, and classification. It also includes detailed plots and analysis of federated scaling behaviors during general time series foundation model training.
C.1 TIME SERIES REPRESENTATION LEARNING
The full federated time series representation learning results are shown in Table I6 which shows proposed FeDaL outperform advanced FL methods across various datasets and setups in unsu- pervised representation learning.
Representation Learning Under Increased Heterogeneity We further investigated FeDaL's performance under increasingly challenging, high-hhttrogeneity configurations, specifically em- ploying imbalance ratios of {0.7, 0.8, 0.9}. The results shown in Table 17 illustrate FeDaL's superior generalization and robust performance over competitive baselines. This confirms its ability to effectively handle severe non-IID conditions and maintain its leading edge.
C.2 LONG-TERM FORECASTING
The full long- ong-term forecasting results are provided in Table 18 and Table 19 Table 18 demon-
strates that DeDal-trained TSFM consistently outperforms state-of-the-art deep time series models
28

---

Published as a conference paper at ICLR 2026
Table 16: Representation learning results under varying patch masking ratios. For UTSD, we sim- ulate two levels of domain heterogencity (HI and H2) as detailed in the Appendix B For CTSD, each dataset treated as Nomain-indeepende the second best. i denotes evaluation using personalized models after server averaging; $ indicates client-side evaluation without aggregation.
client. Bold indicates the best result, Underline
USTD Dataset (I0 Clients, DM)
CTSD Dataset (18 Clients, DI)
Method
20%
35%
50%
75%
90%
20%
35%
50%
75%
90%
111
112
I1]
112
111
112
111
112
111
112
FedAvg
0.550
FedProx
0.382
0.401
0.409
0.486
0.546
0.553
0.638
0.603
0.880
0.889
0.336
0.390
0.400
0.454
0.638
FodPer
0.343
0.385
0.459
0.540
0.533
0.21
0.580
0.860
0.330
0.395
0.439
0.606
FodRep!
0.390
0.413
0.451
0.560
0.529
0.537
0.621
0.592
0.853
0.860
0.352
0.375
0.386
0.440
0.598
FTTS
0.333
0.327
0.450
0.410
0.526
0.510
0.630
0.584
0.870
0.823
0.300
0.357
0.379
0.436
0.610
Stand-alone!
0.376
0.380
0.454
0.450
0.535
0.521
0.643
0.616
0.875
0.870
0.342
0.381
0.395
0.470
0.646
FeDal. (Ours)
0.348
0.300
0.436
0.422
0.521
0.489
0.596
0.549
0.852
0.795
0.310
0.319
0.343
0.405
0.560
Table 17: Federated representation learning results across different masking ratios and imbalance
ratios (H3: 0.7, H4: 0.8, H5: 0.9).
Imbalance Ratio
Method
Masking Ratio
(H)
20%
35%
50%
75%
90%
FodAvg
0.427
0.505
0.571
0.623
0.912
FedProx
0.409
0.500
0.568
0.627
0.900
FedPer
0.399
0.612
0.544
0.582
0.877
H3 (0.7)
FedRep
0.415
0.579
0.540
0.603
0.868
FFTS
0.339
0.423
0.525
0.599
0.831
FeDaL (Ours)
0,310
0.429
0,501
0.557
0,809
FedAvg
0.431
0.506
0.582
0.626
0.917
FedProx
0.412
0.509
0.577
0.630
0.926
FedPer
0.406
0.618
0.560
0.597
0.886
H4 (0.8)
FedRep
0.418
0.592
0.555
0.606
0.871
FFTS
0.350
0.435
0.537
0,601
0.844
FeDaL (Ours)
0.322
0.430
0.511
0.560
0.824
FedAvg
0.451
0.523
0.598
0.637
0.924
FedProx
0.420
0.514
0.586
0.642
0.933
FedPer
0.416
0.620
0.577
0,603
0.878
H5 (0.9)
FedRep
0.432
0.604
0.569
0.610
0.878
FFTS
0.362
0.441
0.550
0.612
0.846
FeDaL (Ours)
0.330
0.437
0.523
0.571
0.825
specifically designed for long-term forecasting. Furthermore, Table shows that under zero-shot evaluation, our approach surpasses both advanced centralized pretrained TSFMs and federated pre- training baselines, highlighting its strong generalization across domains and tasks.
Discussion Larger Time Series Models Zero-shot generalization is widely regarded as the core of strong foundation models. In Table 19 we evaluate this capability under long-term forecast- ing tasks, comparing our FeDaL-trained TSFM against range of baseline models (most of which are tailored for forecasting). The results demonstrate the superior zero-shot generalization ability of our model. To further contextualize performance, we compare our FeDaL-trained TSFM with Time- MoE (Shi et al., 2024), recent large-scale time series foundation model designed specifically for forecasting. Time-MoBE variants include models with up to 2.4B parameters trained on 300B time series data. We present results on five standard long-horizon forecasting benchmarks in Table 20 including each model's parameter count and training data size for visual comparison. Our findings show that FeDaL consistently outperforms Time-MoEhase (113M) and Time-MoBuage (453M), and achieves comparable results to the largest variant, Time-MoEults (2.4B). Notably, FeDaL reaches this level of performance with only 1.8% of the parameters and 70B fewer training samples, in- dicating substantially better efficiency. To quantify this tradeoff between performance and resource cost inspired by (Kaplan et al. 2020 Tan & Le 2019), define the Information Gain per Cost
29

---

Published as a conference paper at ICLR 2026
Table 18: Full long-term forecasting results comparing our proposed FeDal. with advanced deep
time series models. Bold: the best, Underline: the second best.
PED.
Lighe15
MAE
MSE MAE
MISE MAE
MAE
MAE
MSE
14.40
0
1440
5362
0L792
5.5 0
0,300
0.300
0.154
1440
b.56 S419 444 CAN9 11513 an
1.494
0-42444244
14432
03365
0.713
13557 0:226
192
11.311
0.400
0435
0440
0415
5412
0.48
0-10
0.433
0.421
0445
0429
9429-4420
54200
0:448 0 0,500 0,882 01534 0,500
0.538
0.504
0.415
0.462
0.792
0.53 0.206
335
11392
0.43
0454
0455
5530
0423
9.33
0.49
0.42
0.46
0491
0.59
0459
0459
04455 0.521 0 0.588 0,555
0.514
0.521
0.51 0.45
L
0.903
1 1,07 0:835
11417
11550
0.907
date
2.50
257 0.000
11.40
0.40
144500.400
(1400 1 1..0 0.857 11571
0.577
0.30
0.94
01793
(1 I1V 0,915
W1:
11367
0.325
1:296 14
10200 0.77
200
0:511
0.390 334
0:005
D.747 1 0.566 0.058 1:175
391
0.207
03537
1:765
1.535
2.606 1:317
192
11355
0.500
(1570
0.422 14434
D.139 1 1.056 C-A52 11:122 0.005
(1430
439
0.035
11.00
5.0012
15831
12 2479
E1112
135
11:59
0355
0L595
10400
0.329
1.200
04422 14452
0.00
D4677 6R2 CJ499 11:552 0.551
[1457
4:721
1.535
9.525 2:200
720
1.377
0.42
(4422 16432
0.572
6429
0:441
11:00
14450
0.403
0.515 G.5L1 11.622 1,500
0.550
0.003
11552
3,047
1.525
3.874 1,00
Avx
10334
0.377
[1554 0.304
0.334
0.983
0412
0.451
0.320
R444 11427
0:449
0.400
(-43) 11.23 0,51c
[1445 0.422
0,002
0.543
4.431
1.729
6.736
2,191
0.300
0.574
11,00
0.672
0.872
1.500 0:524
1L297
(L353 0.380
0.774 11.57
0-420
0:442
C-955
0.410
0.430
01407
2.705
0.099
11008 0.912
0.327
0.811
03322 0.44
0400 0441
0.45
0459
01121
0.507
0:42
0.435
0.455
1-212 0.97)
0.50% 0.721
720
11352
(452
0.11
0.43 0.50
0.543
0.430
0.671 0.561 0.585 0.516
0:460
0.527
0.502
1105 6.822
1.002 8541
A0g
11319
01.12
P-446
0.452
0.512
0-419
(.M61 0/714
I1157
0.20
0.257
0.20
0.257
255
0.192 0.274
0.300
0.500
1.139
0.199/1152
1.200
0.147
01:00
5.102
0:212
0.19
0.173
10220
0:11
0:345
0.00
0.231
0.002
11:42
0.304
192
IL174
0.23
12255 11250
0.199
0L234
1214
0.244
(1.220
0.354
0.244
0.219
11253
9:26
01310
0.007
0.312
22227
0.504
335
11.40
0.79
(252 0.237
5232
0.279
0.266
01216
0.319
0.345
0.350
0.203
1L306
0.399
0.300 0
0.915 01.211
0.298
0.355
0242
0.334
5.57%
0.523
11000 0.91
720
11300
0.300
(.340 0.343
0.304
0.315
0.326
0,387
0.353 0.362
0.354
0.95
0.359
3.433
0-42 0.49
0/425 01414
0.410
0.352
0.258
0.002
0.386
1,00
0344 11150 0.702
11.13
0.258
(1241 0,212
5225
0225
237
9.270
0.346
0,500
0.225
0.264
0:259
0.257
0.309
0.350
0.35% 0,582 0.358
0,314
0.273
0.334
0.201
0.312
0.004
0.545
3.500
0x056
24
1.366
0.714
11779 11.60
12255
0.223
2,005
0.651
2215
1101
1.31
0.154
2.317
1U534
1.22%
1.250 3.485 1.267 2.354 0.06.
2.521
1.000)
5.313
2.344
15.704
1.07)
(.000) 1:302
36
1.35
0.000
1,833
03.62
0.514
0.912
1.563
0.00
1.430
0.534
1,072
IU520
2,67 1,000 3.108 1.145 1.825
2.55
1.007
5.631
1.902
4.755
1.407
4.78
1.448
(8:
1.409
0.760
2.125 11076
11.03
0:061
2.100
1.553
2.254
1/575 2.09 1/855 2.000
1:200
1.082
4.703
1.400
1.02
1.004
641
1.43K
0330
2.135
11078
1.531
13:54
1.V7
2.306
1180
1.470
11750
2.127 18528
2.877
1.157
2.701
2.125 2.178 0.000
2.457
1.000
1.005
5.201
1.504
1.002 1.003
Adg
1.355
0.773
(4079
(L859)
1.415
(1900
1.925
1164
1.443
2.139
1.544
3186
1.161 2007 0.10
1.00
1.362
2,005
5.157
1.344
4.724
2.445
1" Coom
46
33
Table 19: Full results of zero-shot forecasting experiments. A lower MSE or MAE indicates better prediction. TimesFM, due to its use of Weather datasets in pretraining, is not evaluated on this dataset and is denoted by a dash (-). Bold: the best, Underline; the second best.
Federated Leurning Methods
Pretroined Time Series Foundation Models
Models
Febbal (Ours)
FFTS
Fed.Agg
Motralueest
Motralbar
Metrallagge
TImesFM
Moment
Chroaoss.can
Chronoes sMc
Chromoserer
Metries
NISE
MAE
MSE
MAE
MSE
MAE
MAE
MSE MAE
bISE
MAE
MSE
MAE
0.368
0.12
0.412
0.465
0.688
0.560
0.530
0.450
0.492
0.902
0.424
336
0.438
0.440
0.444
0.433
0.488
0.435
0.495
0.503
0.456
0.675
0.563
0.570
0.486
0.550
0.576
0.467
0.439
0.554
0.447
0.464
0.611
0.683
0.585
0615
0.582
0,583
Aug.
01.407
0.420
0.425
0.457
0.418
0.449
0.436
0.422
0.477
0.419
0.450
0.450
0.473
0.441
0.683
0.566
0.545
0.472
0.591
0.468
0.588
0.465
96
0.307
0.255
0.325
0.332
0.340
0.350
0.297
0.356
0.294
0.350
0.296
0.330
0,315
0.349
0.342
0.295
0.307
0.356
0.305
0.343
0.320
0,345
BTT22
336
192
013449
0:367
0.373
0.395
0.901
0.355
0.389
0.993
0.415
0.978
0.421
01965
0.370
1.568
(1.61
0.393
0.376
1668
113000
0.375
(
0 0.900
1351
0.121
0.300
01422
0.588
0.425
0.305
1.354
1.356
0.402
0-407
0.408
0.376
01401
0.451
0.420
0.584
0.99
0.430
0.406
0.492
0.453
0.399
730
0.401
0.406
0.409
0.122
0.427
0.441
0.41
0:426
0.436
0.433
0.423
0.418
0.443
0.454
0.395
0.454
0.604
0.553
0.501
0.477
0.603
0.511
Avg.
11336
0.081
0.510
0.374
0.990
0.401
0.361
0.364
0.302
0.383
0.367
0.32
0.392
0.406
0.361
0.408
11424
0.4.0
0.405
0.410
0.555
0.621
0.392
0.363
0.356
0.350
0.370
0.654
0.511
0.454
0.403
0.317
0.524
0.364
0.405
0.358
0.375
0.662
0.532
0.618
0,450
0.435
0:412
(1416
01436
0.072
0.683
0.18%
Avg.
0.436
0.406
0.670
0.356
0.645
0.465
56
10.207
0.283
0.202
0.278
0.219
0.269
10.214
0.258
10.205
0.273
0.211
0.274
0.202
0.270
[1250
0.335
0.200
0.201
0..99
0.274
0.197
0.271
BTTm2
336
192
0.316
0,245
0.333
0.340
0.320
0.275
0,327
0.320
0.270
0.321
0.340
0.353
0.264
0.331
0.362
0.352
0.329
0.275
1.350
13316
0.251
0.341
0.355
0.318
0,360
0.289
0.321
0.366
1.324
1.259
0.369
0.350
0.354
0.260
0
0.341
0.326
0.261
0.366
0.322
0.313
0.256
0.353
0.314
730
0:397
0.408
0.45%
03531
0.478
0.406
0.402
0.408
01497
1147%
(1060
8-4
I.394
0:409
0.553
01499
0.455
0.4%
0.416
0.415
Aug.
00.202
0.34]
0.317
0 0.362
0.902
0.965
0.307
0.347
0.311
0.337
(1
(1330
0.343
0
0.338
0.366
0.316
0.365
0.349
0.360
0.310
0.350
0.205
0.388
(0.189
0.172
0318
0.201
0.207
198
0.222
01217
(1
0211
0.243
0:355
01211
0.243
0.203
0.218
(0
0.235
Wauther
0.285
0.217
0.250
0.27%
0.340
(1247
0.283
0.265
0:300
0.271
0.256
0125)
11207
0.251
0.778
0.306
0:120
0:346
0.321
0.263
0.350
0.794
0:356
0.336
0.249
0.285
0.327
0.255
0,351
0.354
0.373
1.200
0.374
11.04
0.378
Avg.
0.217
0.305
0.275
0.286
0:257
0.261
1125
0.326
0.300
0.318
0.292
0.315
0,305
Count
17
IGC) metric as:
IGC-
(19)
MSE+ X Parameters Count" Training Data Size
where a- = 3 = 1 by default. A higher IGC indicates better efficiency. As shown in Table 20 FeDal. achieves the highest IGC, outperforming all Time-MoB variants in terms of cost-effectiveness: FeDaL > Time-MoEnme > Time-MoBBarge > Time-MoE.ttra This underscores that FeDaL not only delivers strong performance, but does so with superior parameter and data efficiency, making it a more scalable and practical choice for real-world deployment. In addition, the TSFM trained with FeDaL demonstrates strong generalization across diverse tasks beyond forecasting, including classification, imputation, and anomaly detection.
30

---

Published as a conference paper at ICLR 2026
Table 20: Zero-shot long-term forecasting performance comparison with larger-scale time series forecasting foundation models. A lower MSE or MAE indicates better prediction. A higher IGC indicates better efficiency. Bold: the best, Underline: the second best.
Comparison with Larger Forecasting Models
Models
(Ours)
Time-MoF...e
Time-MoRaerg
Time-MoEutera
Metries
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.347
0.357
0.350
0.382
0.349
0.379
192 0.398
0.410
0.384
0.404
0.388
0.412
0.395
0.413
ETThl
336
0:425
0.452
0.411
0.434
0.411
0.430
0.447
0.453
0:457
0.427
0.455
0.457
0,462
Avg.
0.407
0.429
0.400
0.424
0.394
0.419
0.412
0.426
96
0.307
0.355
0.305
0.359
0.502
0.354
0.292
0.352
ETTh2
0.349
0.351
0.347
336
0.387
0.395
0.391
0.418
0.417
0.425
0.406
0.419
720
0
0,406
0.419
0.454
0.537
0,496
0.439
0.417
Avg.
0.361
0.405
0.415
0.371
96
0.289
0.346
0.338
0.368
0.309
0.557
0.281
0.341
ETTml
192
0.317
0. 1,369
0:353
0.388
0.346
0.381
0.305
0.358
336
0.370
0,420
0.381
0,413
0.573
0,408
0.369
0.395
720
0.464
0.426
0.504
0.493
0.475
0.477
0.469
0.472
Avg.
0.360
0.390
0.394
0.415
0.376
0.405
0.356
0.391
96
0.207
0.283
0.201
0.291
0.197
0.286
0.198
0,288
ETTm2
192 0.248
0.333
0.258
0.334
0.250
0.312
0.235
0.312
336
0.316
0.340
0.324
0.373
0.337
0.375
0.293
0.348
720
0.397
0,408
0.488
0.464
0,480
0.461
0.427
0.425
Avg.
0.292
0.341
0.317
0.365
0.316
0,361
0.288
0.344
96 0.159
0.212
0.159
0.157
Weather
192 0.217
0.264
0.210
0.260
0.215
0.266
0.208
0.256
336 0.285
0.312
0.274
0.309
0.291
0.322
0.255
0.290
0.359
0.418
0.415
0.405
Arg.
0.2255
0.284
0.265
0.297
0.270
0.300
0.256
0.283
Average
0.335
0.370
0.343
0.382
0.355
0.387
0.342
0.369
1 Count
21
22
Total Param..#
28.42 M
113 M
453 M
2.4 B
Training Data
231B
300B
300B
300B
Information Gain Per Cost
4.545
10 19|
8.605
10 20 2.073
4,061 10
C.3 SHORT-TERM FORECASTING
The full short-term forecasting results are presented in Table 21 and Table 22 Specifically, Ta- ble 21 compares our FeDaL-pretraine TSFM with advanced deep time series models, while Ta- ble 22 focuses on comparisons among Fl.-based TSFM pretraining methods. Across all settings, FeDal. consistently outperforms all baselines, including task task-specific deep models, general-purpose foundation models, and alternative federated pretraining strategies, highlighting its effectiveness in learning robust and generalizable temporal representations under decentralization.
Table 21: Short-term forecasting results. Bold: the best, Underline: the second best,
Intervals Mathods FeDai. (Ours) Time-L1.M GPT4TS TimesNer PRclTTST N-HITS N-REATS ETS LighTSS Dline FED. Statioray Anto.
In.
Re.
SMAPR 13.102
13.419
15.11
15.378
13.477
13.422
13.487
18.009
14.247
16.965
14.021
13.717
13.974
14,737
16.169
Yeurly
MASE
2,812
3,005
3.565
3.554
3.019
3.056
3.036
4.487
3,109
4.283
3.016
3,073
3.134
3,418
3,800
OWA
0.748
0.789
0.911
0.918
0.792
0.795
0.795
1.115
0.827
1.058
0.81]
0.507
0.822
0.88]
0.973
SMAPE
9,508
10.110
10.597
10.165
10.38
10.185
10.561
13.376
11,364
12.165
11.1
10.958
11.338
11,360
13.315
Quanerly
MASE
1.178
1.227
1.235
1.18
1.328
1.520
1.365
1,401
0.933
0.921
1.302
1.000
1.105
12,900
13.513
12.959
13.059
13,089
14.588
14.014
13.514
13:917
13.958
20.128
Monthly
MASE
0.953
1.009
0.970
1.013
0.996
1.368
1.055
1.037
1.007
1.141
2.614
OWA
0.520
0.903
0.931
0.957
0.905
0.929
0.921
1.149
0.981
0.956
1,038
0.998
1.002
1.024
1.927
SMAPE
4,508
4.795
5.124
6.913
4.952
4.711
6.599
7.267
15,880
6.709
7.148
5,302
5.485
24,660
32.491
Others
MASE
2,590
3.17%
4.116
4.507
3.347
3.054
4.43
5.240
11.434
4.95%
4.041
4.044
3,865
20:360
33.355
OWN
0.973
1006
1.259
1.438
1.049
0.977
1.391
1.591
3.474
1.487
1.389
1.304
1.187
5.879
8.679
SMAPE
11.412
11 983
12.69
12.88
12.059
12.035
12.25
14.718
13.525
13.639
13.16
12.750
12
12.909
14,086
18.200
Average
MASE
1.489
195
1,808
1.836
6633
1.625
1,608
2.408
2.111
2.095
1.775
1.756
1.771
2.718
4.233
OWA
0.859
0.955
0.869
0.869
1.051
1
1.230
31

---

Published as a conference paper at ICLR 2026
Table 22: Short-term forecasting results based on FI.. Bold: the best, Underline: the second best.
Setting
Federated Foundation Models
Foundation Models
Intervals
Methods
FeDaL (Ours)
FFTS
FedAve
MOMENT
SMAPE 13.102
13.289
14.784
20.649
Yearly
MASE
2.812
2.909
3.257
4.757
OWA
0.748
0.781
0.866
1.230
SMAPE
9.808
10.005
10,920
10.849
Quarterly
MASE
1.112
1.190
1.367
1.305
OWA
0.847
0.877
0.957
0.968
SMAPE
12.124
11.920
13.048
14.497
Monthly
MASE
0.898
0,879
1.027
1.143
OWA
0.820
0.815
0.916
1.040
SMAPE
4.508
4,490
5,210
5.634
Others
MASE
2.890
2.907
3.657
4.102
OWA
0.973
0,994
1.154
1.240
SMAPE
11.412
11.404
12,342
14.593
Average
MASE
1.489
1.522
1.753
2.161
OWA
0.818
0.831
0.926
1.103
Table 23: Full results on imputation task. Bold: the best, Underline: the second best.
Medasi
Pebud.
FF15
GPT4TS
Tinessbet
FED.
LigheT
Mirrc
MM
MA
MAR:
MAS
MAn
just: MAD MBI MAAT sen MAl.
MSt MAn
MMS MAR Mest Man Mas Mkn
Msi Mar
MSI MAR
(ME MAR kest MAT
255
11855
10873
1015
1198
GUIZZ 0LOO0
2AGZ 5006 11 mn 1044 0.335
susss 1223
banKs 0.30 u 0.195 11e2 1e
uosz u11
busee 1:344
SaNS 0.190
k 0.146
57.5%
11005
16895
0003
CUAD 11.31 RACH 8111 Mug 1111 HOM u.303
0133 1421
2113 1223 u HG 0:219 uuee In ncas 131
cuna 0.300
11mS 0.182
11002
0255
0.166
11814
18822
01200
R0IF SHOW
DID
1001
D4T54 0.87 1822 IMA 196
D I1302
15%
0.0
2.185
57.5%
1,055
0.86
5 01055
ETNE 5087 0.023 0.91 (0,80 0.104
0.23
0.356
5.51 0.159 0 [0 6.222 0.110 251 RC27 0.1
0.305
5.155
0.93
1.175 0,303
95%
11005
14779
RuES 11.85
2 B:1
D.W R.154 131 0.317
D415 11110
11:11
12.5
1.800
16136
1L.32
CUESS 12
KISSS u
0.100 (Los3 0.200
0.125 11253
0:201 01545 U 151 0.207 ILOTU DUIIOUOO 165
DUTTA 11:92
10.14
0:254
juuce 0.194
1L138
CUAD
0.217
01102 11304
DUDAD
57.5%
0.061
R055
0.582
215
0:232
175
0.205
155 0:56
50%
01065
0131
R.C
B
0
107
2215 (4:44 245
0.791
0.000
534
0.854
0.137 0.345
0.215 0.355 i 0.503
Ave
11:47
1U035 1L
6/04
0.224
0.202
14:20
2.264
01513
C.905
11.244
0-161
0.27
12.5%
R.600
11106
1022
0.025
2.125 D40
1.50 0:257 0.153
(L117 1.319
1.101
0221
0
100 0.216
CLOBS
22212
0:135
10044
0.335
2.305
043: 10.00 0:299
25%
1.811
11194
1034
0.35
(1061
0.155
0.299
0.300
0.115
0.265
0.347
0.140
0:444 200 0:301
95%
11b1%
IL115
ja 0-11
Avg
1004
IL188
0046
2
0.065
0.163
0.307
0.426
0.119
0.25
0,299
0L165 0.27
0.252
0405
0.556
2.307
0.42
0.302
356
1.818
0885
1434
5004 005T
1OS
0.091
0065
155
5 D.11
0 B4S 1.003
1.06 163
050
DV530
0.054
0.100 10 155 0
Warfor
$7.50
10055
0050
0083
0.001
8
DLDSS
0.221 187 6.117
0.060
0.11)
5:911
11024
IU945
(U229
1L325
CJES4
(U060
0.144
(1775
1L:13
D.117
0182 0.110
ILSBN 0.20
0.009
DUESE
11251
(2344
0.104
Crant
40
C.4 TIME SERIES IMPUTATION
The full imputation results are presented in Table Our proposed FeDal.. consistently outperforms both advanced deep time series models and federated TSFM pretraining baselines, demonstrating superior generalization across heterogeneous input gaps.
C.5 TIME SERIES ANOMALY DETECTION
The full anomaly detection results are presented in Table 24 FeDaL again achieves the best perfor- mance among all evaluated methods, surpassing both advanced deep time series models and existing federated TSFM pretraining approaches.
C.6 TIME SERIES CLASSIFICATION
The full classification results are presented in Table 25 (UCR) and Table 26 (UEA). FeDaL consis- tently outperforms all baselines, including deep time series models, classical machine learning and statistical classifiers, cenerrl-purposs TSFMs (GPT4TS, MOMENT), and federated TSFM pretrain- ing methods.
32

---

Published as a conference paper at ICLR 2026
Table 24: Full results of the anomaly detection task. The P. R, and F1 represent the precision, recall, and Fi-score (%) respectively. Fl-score is the harmonic mean of precision and recall. A higher value of R. and Fl indicates a better performance. Bold: the best, Underline: the second best.
Dataset
SMD
MSL
SMAP
SWaT
PSM
Avg.
Metric
P
R
F1
R
F1
P
R
F1
P
R
F1
P
R
F1
(%)
LSTM
1997
78.52
65.47
71.41
78.04
86.22
81.33
91.06
57.49
70:48
78.06
91.72
84.34
69.24
99.53
81.67
77.97
Transformer
2017
83.58
76.13
79.56
71.57
87.37
78.68
89.37
57.12
69.70
68.84
96.53
80.37
62.75
96.56
76.07
76.88
Transans
2019
83.46
70.13
76.21
73.05
87.37
79.57
89.15
57.59
69.97
68.67
97.32
80.52
63.06
98.00
76.74
76.60
TCN
2019
84.06
79.07
81.49
75.11
82.44
78.60
86.90
59.23
70.45
76.59
95.71
85.09
54.59
99.77
70.57
77.24
Reformer
20200
82.58
69.24
75.32
85.51
83.31
84.40
90.91
57.44
70:40
72.50
96.53
82.80
59.93
95.38
73.61
77.31
Informer
2021
86.60
77.23
81.65
81.77
86.48
84.06
90.11
57.13
69.92
70.29
96.75 81.43
64.27
96.33
77.10
78.83
Anomaly"
88.91
82.23
85.49
79.61
87.37
83.31
91.85
58.11
71.18
72.51
97.32 83.10
68.35
94.72 79.40
80.50
Pyraformer
85.61
83.04
83.81
85.93
92.54
57.71
71.09
87.92
91.78
96.02
82.57
Autoformer
2021
88.06
82.35
85.11
77:27
80.92
79.05
90.40
58.62
71.12
89.85
92.74
99.08
88.15 93.29
84.26
LSSL
78.51
65.32
71.31
77:55
88.18 82.53
89.43
53.43
66.90
79.05
93.72 85.76
66.02
92.93 77.20
76.74
NSformer
2022
88.33
81.21
84,62
68.55
89.14
77.50
89.37
59.02
71.00 09
68.03
96.75 79.88
97.82
96.76 97.29
82.08
DLinear
2023
83.62
71.52
77.10
84.34
85.42
84.88
92.32
55.41
69.26
80,91
95.30 87.52
98.28
89.26
93.55
82.46
ETShomeer
2022
87.44
79.23
83.11
85.13
84.93
85.03
92.25
55.75
69.50
90,02
80.36
84.91
99.31
85.28
91.76
82.87
LighTTS
2022
87.10
78.42
82.53
82.40
75.78
78.95
92.58
55.27 69.21
91.98
94.72
93.33
98.37
95.97
97.15
84.23
PEDFormer
2022
87.95
82.39
85,08
77.14
80.07
78.57
90.47
58.10 70.76
90.17
96.42 93.19
97.31
97.16
97.23
84.97
TimesNet
3022n
88.66
83.14
85.81
83,92
86.42
85.15
92.52
58.29
71.52
86.76
97.32 91.74
98.19
96.76
97.47
86. 34
ModemnTTN
2024
87.86
83.85
85.81
83.94
85.93
84.92
93.17
57.69
71.26
91.83
95.98
93.86
98.09
96.38
97.23
86.62
MOMENT
2024
78.88
92.01
84.94
88
88.98
75.10
81.45
90.02
56.51 69:43
92.13
91.67
91.90
82
89.55
93.96
84.34
GPT4TS
2023
88.89
84,98
86.89
82,00
82.91
82.45
90.60
60.95 72.88
92.20
96.34
94.23
98.62
95.68
97.13
86.72
FedAvg
7
87.88
89.21
88.44
81.69
83.00
82.32
89.26
58.66
70.78
90.27
90.43
90.32
96.56
95.08
95. 86
85.54
FFTS
2025
89.26
90.48
89.88
89.23
87.6 64
88.42
90.64
58.88
71.38
91.22
90.99
91.12
99.00
97.85
98.54
87.86
FeDaL
(Ours)
89.15
87.78
88.46
88.87
89.
89.05
88.46
60.29
71.70
92.56
98.49
95.40
99.29
98.46
98.88
88.70
C.7 FEDERATED SCALING BEHAVIORS
While prior work explores scaling laws of centralized TSFMs in terms of model and data size, instead investigate scaling behaviors in the federated setting, focusing on three key factors: (i) pre- training data size, (ii) number of clients, and (iii) client participation rate. Specifically, we vary: (i) Data size from (40B, 80B, 120B, 160B, 200B, 231B) with fixed client count of 174; (ii) Num- ber of clients from (30, 70, 110, 140, 174} using the full 231B dataset: (iii) Client participation rate from (10%, 30%, 50%. 70%, 100%) with the original pretraining setup. All experiments follow the training protocol in Section 4.3. Results are presented in Figure We observe that: (1) Increasing data size consistently improves downstream performance, even with fixed client count; (2) More clients (with constant total data size) lead to better representations, suggesting improved learning of diverse, domain-specific patterns; (3) Higher client participation rates yield stronger results, likely due to more effective aggregation and reduced drift. These findings suggest that federated TSFM pretraining benefits from scaling in data and client dimensions. much like centralized pretraining. However, unlike the "scale-is-all" mindset focused on model size, our results highlight the impor- tance of expanding dataset coverage and client participation more efficient and federated-aligned strategies for improving generalization.
C.8 VARIANCE ESTIMATES
All experiments (except the zero-shot forecasting setting) were conducted three times, and we re- port the averaged results. Variance estimates for representative configurations are now provided in 2able27/Tabee Table 30
THE USE OF LARGE LANGUAGE MODELS
In preparing this paper, we used a Large Language Model as a general-purpose writing assistant. Its role was limited to language refinement, such improving grammar, sentence flow, and vocab- ulary usage to meet academic writing standards. All substantive aspects of the research (including conceptualization, experimental design, data analysis, and drawing conclusions) were carried out
33

---

Published as a conference paper at ICLR 2026
Table 25: Full results of the classification task on UCR Archive Dau et al. 2019). Bold: the best, Underline: the second best.
Darasots
pTW SLan
1520
TSC
T5-ICC
1ST
ChN
Eacode
FUS
SICESSS
koll::
TeachNi MITI
TWIES
MIPP
GLivar
SOMENT
CETITS Tobley
FTTS
Fibal.
71.6
77.5
76.3
00.7
09.7
25.8
41.1
47.5
713
26
76.1
53.7
29.1
10.0
52.3
47.
31.7
55.6
11.7
23.7
78.2
70.1
74.6
W
72.9
79.3
72.6
74.1
76.1
42.3
43.0
50.9
75.4
42.0
794
56.3
56.3
1031
57.1
S4.1
62.3
16.2
16.0
72.5
72.6
73.5
6:33
78.5
733
68.
689
44.7
76.1
72.6
51.1
48.0
15:0
51.
43.9
53.
73.0
11.5
68.8
07.3
08.0
AmveHied
70.3
76.6
9L2
62.8
93.8
78.9
76.6
33.3
02.5
86.9
81.1
79-2
90.4
Bosf
560
96.7
75.3
95.0
52.5
leateily
100
190
110
195.
9601
91.0
65.0
88.0
300
70.0
9011
79.0
1500
73.0
566
87.5
05.0
BisCCiden
75.0
80.0
85.0
65.0
65.0
55.0
71.
51.0
94.0
54.
00.0
93.3
80.0
50,0
02.0
74.0
90.0
58.5
95.3
25.0
85.2
87.1
95.0
CHP
1000
M55
89.5
96A
19
16.1
83.6
31.2
19.5
19.9
77.
V6.1
95.1
918
98.2
Chisaons
951
98.
98.3
93.6
91.
96.6
98.0
96.5
97.
98.3
96.5
72.6
80.5
872
K3.1
98.3
98.5
85.
968
98.
69.0
75.5
56.2
663
$1.7
66.1
58.3
5922
62.6
31.
55.4
900
56.5
72.4
06.0
65.2
Coffee
1200
1000
1000
1030
911
HOLL
100LD
95.9
1800
92.
1960
53.7
97.9
1000
1000
07.9
100.
100.
103.
CisdatY
74.9
$1.0
65.
75.
Criduat
79.
70.
71.
40.1
90.1
510
48.4
1019
53.2
66.3
36.7
742
085
106
Crep
06.5
75.6
72.
742
742
11.0
67.0
76.0
738
68.7
74.3
77.5
76.4
4.2
48.9
618
68.1
12.5
14.8
34.1
77.
77.
76.2
W6.7
96.4
954
96.1
10.0
54A
66.6
2911
87.9
30.1
ML.
86.7
96.11
989
9K-7
77.0
72.
727
755
74.1
758
76.1
718
72.9
718
78.4
79.9
43.5
70.5
647
06.9
75.5
75.1
08.9
903
80.5
$1.1
71.3
06.1
77.
754
118
272
7641
75.9
77.0
11.5
73.6
39.5
7L1
72.
66.9
76.
75.
663
77.7
200
31.2
56.2
Dedlyrt cepciome
NNN
NNN
NoN
56A
13.8
71.7
817
819
04.9
9644
NUN
KiN
NoN
73.2
97.4
18.3
90.4
97.8
952
98.
07.8
73.9
95.4
078
07.1
98.6
98.6
58.4
975
980
95.0
BOG200
77.0
92.0
940
58.
18.0
$3.0
81.6
88.4
53.8
87.
748
745
64.0
57.4
914
G4T
91.0
15.5
70.0
85.6
992
91.1
DOG
12.4
05.
M55
16.1
W6.1
92.1
16.1
1411
10.3
95.
16.0
58A
M22
MEB
KLD
W6.1
3864
911
918
93.8
76.8
1060
100d
878
678
76.3
17
86.
95.5
966
940
963
49.7
72.3
973
95.6
94.8
561
95.6
03.
97.0
Eattherakes
71.9
04.6
748
748
948
300
22.
76.8
712
100
18.2
24.5
74.
14.3
1910
744
718
710
713
FaceAll
85.0
99.
910
90.5
955
16.8
51.5
74.5
K3.0
91.5
450
946
00.5
02.4
984
963
86.3
543
81.
96.7
94.
17.5
95.4
54.
814
14.3
64.
83.1
75.9
53.1
54.2
46.
$5.9
984
90.5
Wont
090
77.1
712
05.
53.5
62A
66.8
616
61.
75.0
56-2
06.
12.5
51.
718
9611
71.4
705
704
12.6
Fiii
82.3
92.6
89
81.
81.7
730
85.5
73.4
95.1
72.0
98.1
54.0
857
12.6
87.5
848
82.5
56.3
92.0
73.1
91.
98.1
99.2
95.
M2N
MADI
26.8
16.1
MST
5300
565
14.3
91.4
91.5
9122
623
62.0
794
793
KLS
562
241
71-2
81.
7100
1914
303
511
7077
11.3
625
759
714
196
91.
73.
16.5
Thng
95.
100/
16.
66.1
177
66.3
81.9
1.4
96.
1810
1050
CosMecMalAaa
56.9
66.5
00.8
36.
36.9
208
53.
52.5
63.5
51.5
09.
11.5
723
3.6
540
535
52.
63.1
11.5
20.1
78.4
900
802
00.
16.9
516
28.
25.4
13.8
11.36
18.0
611
300
516
962
3.8
545
55.
16.3
310
641
610
664
32.3
202
285
125
173
15.
31-7
36.
32.6
254
3-.0
433
06.9
3.8
20.5
382
36.
41.5
500
163
52.3
52.0
538
M1.
365
5000
TW9
M11
86.
1200
15.5
M60
015
87-1
350
GestrcPoMii2
36.0
718
77.
542
773
943
701
29.
85.6
Centiiia A
98.6
56.1
991
96.3
49.0
96.
96.0
19.1
96.5
96.5
59.
Guadbinn Female
98.7
1000
99.7
99.7
100.0
91.7
90.5
99.2
95.2
991
100.
997
525
98.
980
91.
0914
99.1
47.5
918
1000
100.
83.8
1560
1002
1260
1030
100.0
92.2
12.3
95.
92.6
98.9
100.
1960
52.4
90.5
041
1000
92.1
90.5
52.4
91.5
100.
100.0
66.7
71.4
72.4
785
53A
73.1
06.
70.7
71.
75.8
7111
7900
63.1
76.
099
WID
78.1
75.1
776
718
795
Heriig
53.1 64.1
994
994
59,4
59.1
51.
644
572
60.0
12.5
07.
59.4
00.5
291
64.1
67.3
70.3
57.5
62.9
70.7
19.2
625
41.
26,6
29.5
65.0
3622
367
499
51A
66.8
25.
6544
16.1
39.
66.7
M14
864
$4.5
95.4
962
969
87.1
58.0
915
94.1
Meat
95.0
050
58.
960
915
855
99.
66.
00.0
11.5
95.0
66.
96.3
98.1
Mesicaliaaeo
73.7
76.9
75.0
74.7
74.7
532
63.1
66.4
71.8
62.7
72.
13.2
728
51A
64.
71.9
56.5
154
15.8
40.5
76.5
214
166
791
95.9
MAM
91.
5
91.
91
96.1
911
108
W61
66.1
711
45.1
667
50.0
03.6
03.6
63.0
65.0
53.
53
59.7
53.5
55.8
5-5
53.6
63.0
57.1
57.5
522
65.6
66.3
54.3
52.6
53.8
410
62.
851
135
35.1
24,
75.1
74.5
196
K2B
81.8
1311
530
56.3
755
51.9
815
793
518
26.4
991
61
50A
3011
195
500
26.6
56.9
61.
531
87.
881
861
768
88.5
(9.
92.4
83.9
08.
89.
Phal
90.9
78.
90.4
71.
75.9
52.5
63.8
656
759
663
76.
Hickay Desteeeeeeeinneeet
62.0
74.0
60.
60D
24.0
605
49.5
74A
41.
7044
16.0
7000
131
615
64.
$4.0
5.0
85.3
98.
166
Plase
1000
100m
99
1000
1808
$1.1
96.1
100
953
1000
06.1
14%
1060
WTT
961
07.4
988
1810
1010
PorerCCass
87.6
96.
90.0
96.
06.1
91.1
95.0
09.1
85.
92.9
87.9
100.
1000
50,0
831
00.3
09.9
09.4
36.7
98.
99.6
1000
1000
1015
81.4
844
K19
65.4
$1.5
K5.0
86.3
N7.3
19.3
87.8
N519
811
30-
57.
37.0
80.7
966
92.0
55.3
58.3
63.4
81.5
52.5
98.1
87.6
87.
55.4
SlaleGeseeWineee
00.
160
58.0
58.0
90.0
96.1
62.0
967
stapciettiz
65.0
1060
622
06.3
68.3
46.3
467
51.9
75.6
49.
18.2
57.8
500
530
545
40.1
1910
40.9
92.1
827
95.7
StapeuAli
76.8
945
72.5
77.3
93.3
67.9
$9.4
599
92.
51.0
732
1.5
643
726
68.3
14.1
81.8
23.1
$1.9
520
12.5
06.0
96.
M53
95.
63.
87.6
96
47.6
96.1
46.0
69.0
11.
519
NAN
66.0
05.
97.6
476
96.
Srep/ABBORot Suriict
72.5
982
20.
72.4
69.0
72.9
95.6
65.5
96.1
86A
73.5
42.9
72.5
692
65.6
659
89.5
58.9
92.7
933
96.2
831
87.1
839
WTT
917
34.
811
915
612
811
655
M6.1
65.
935
367
91.
Stuaeleryy
0.2
965
91.6
85.2
95.6
98.0
970
05.9
64.
91.1
00.5
93.2
62.8
Synteds
97.6
963
16.
80.8
55.
868
08.1
95.5
Crend
19.3
903
MMD
4013
59.7
M1.3
96.9
15.3
99.1
92.
15.7
82.9
923
43.1
948
912
95.1
77.
91.7
019
93.
93.0
$0.7
59.8
70.6
96.1
559
957
66.7
614
53.6
589
614
66.
96.5
561
952
919
91.0
IneSemenatiion
83.8
8011
90.0
80.2
87.3
51.5
25.2
7012
559
66.9
29.4
83.1
63.1
41.5
70.4
145
51.5
$1.5
$4.5
73.1
948
92.
9622
1000
1000
99m
1000
1000
10000
99.7
78.0
1061
983
1000
97.0
78.0
340
954
1010
71.0
9299
1000
1000
TwoLesiBCG
08.5
99.0
97.6
$7.1
83.7
596
1000
78.2
63.6
5300
96.8
655
95.5
1000
1000
99.0
46.6
99.1
1050
1000
99.4
045
25.0
80.
948
92.3
93.8
94.5
L'Woee Gon
78.
73.5
563
72.1
78.1
72.9
73.4
74.9
75.8
803
Word Lonnn
71.
71.
64.1
34.5
626
60.
6422
66.
52.
019
12.1
46.
19.3
VLLS
643
71.
2411
76.1
65.
77.0
757
0A0
65.5
630
68.
73.7
665
76.
54.8
69.6
12.1
573
556
73.6
75.7
663
78.3
78.0
764
Water
06.0
966
992
99.
904
99.1
96.1
99.7
M912
99.5
99.
99.5
89.2
91.
996
04.9
V9L3
15.9
9914
99.7
99.7
500
Wine
87.0
77%
900
45.
400
73.
10-4
618
930
764
MT
400
83.0
611
70.8
704
12.1
WindSonn
07.5
33.1
412
56R
55.1
61.
53.7
57.8
21.9
30.6
999
66.3
45.1
62.4
630
639
Avg.
26.4
852
33.4
70.
793
550
15.2
74.3
909
712
80.5
800
7608
3488
72.7
75.1
71.
78.4
55.3
56.
75.9
43.7 964
34

---

Published as a conference paper at ICLR 2026
Table 26: Full results for the classification task on UEA Archive (018)) in the Transformers indicates the name of #former. We report the classification accuracy (%) as the result. The standard deviation is within 0.1% Bold: the best, Underline: the second best.
[Classical methods
RNN
TCN
Transformers
MLP
CNN
Pre-traized FMs
Danasets/ Models
NEW
Xciiloo Redoo 1STM LaThie
TON
TorK
TSD
Fin
Ieea Light's
Thradee MITIN MIMINTFHUN BITS Fetul.
(552
NAV
MOAR
MOME
NOM
Mobr
DIME
NOST
32.3
43.7
45.2
39.9
28.9
32.7
31.9
31.6
31.2
32.6
29.7
35.7
30.0
32.1
ReeDecccion
52.9
63.3
64.7
57.7
65.7
66.7
52.8
67.
68.
67.0
65.7
68.4
68.0
66.0
66.3
67.6
66.0
67.5
68.6
70.5
63.9
65.2
69.0
70,0
Bonshriing
28.6
1588
58.8
15.2
25.8
24.6
53.
12.0
27.4
82.8
20.4
36.7
31.6
28.0
$2.5
33.8
27.0
26.1
32.1
30.6
35.1
35.0
37.2
57.3
71.7
73.
75.6
77.1
75.6
73.7
Ipunaekonedee
94.9
86.5
96.2
79.7
98.1
98.4
98.9
98.7
97.8
96.9
98.4
96.2
99.2
98.4
95.9
98.9
96.2
96.2
98.4
98.8
95.7
94.2
94.7
92.6
PEMS SF
71.1
98.3
75.1
39.9
56.7
86.1
68.
82.1
82.7
81.5
53.2
92.7
87.3
80.9
86.01
83.8
75.1
85.4
89.6
89.1
53.5
36.0
84.5
85.6
SefflapshainnsCPii
37.7
84.6
908
54.6
88.7
95.1
SefRegulaiisSC::
$3.9
48.9
53.3
46.6
52.8
52.2
55.6
53.9
56.7
53.3
53.3
50,6
51.2
54.4
55,0
56.1
50.5
51.1
57.2
60.3
55.0
56.2
55.7
59.5
SpoketArihiiighs
96.3
69.6
71.2
31.9
100,0
100,0
95.6
98.4
97.0
100.0
99.6
100.0
00.0
0000
100,0
98.8
81.4
100.0
99.0
98.7
99.4
98.0
99.5
106.0
75.0
57.8
58.4
85.9
85.3
98.5
Average Accuncy 66.00 660
72,5
48.6
71.8
70.9
70.3
71.9
71.5
72.1
70.8
71.1
72.7
70.7 71.0 73.0
67.5
70.4
73.6
74.2
72.
71.5
71.7 76.1
Averaging Between Full-shot and Few-shot for Long-term Forecasting
0.40
0.40
0.42
0.425
0.44
0.41
0.46
0.42
0.475
Data Size
Number of Clients
Client join Rate
Zero-shot Long-term Forecasting
0.35
0.35
0.35
M5E
0.40
0.40
0.40
0.45
0.45
0.45
Number Clients
Client Join
Imputation
0.030
0.03
MSE
0.04
0.040
0.05
0.045
Datt Size
Number of Clients
Client join Rabe
Classification
Accuracy
85.0
82.5
85
80
80.0
Data Size
Number of Clients
Client join Rabe
Anomaly Detection
Scoco
87.5
85
85.0
85
80
82.5
80
408
808
1208
1608
2008
2318
30
70
110
140
174
10%
30%
50%
70%
100%
Data Size
Number of Clients
Client Join Rate
Figure 10: Scaling behavior from a federated learning perspective. Note that for forecasting and imputation, the y-axis is inverted for better visualization (lower values indicate better performance).
independently by the authors The LLM was not employed for generating scientific content. We take full responsibility for the authenticity, originality, and integrity of this work.
35

---

Published as a conference paper at ICLR 2026
Table 27: Federated representation learning results with variance estimates. For USTD, only the H2 setting is reported as representative case.
Merthed
USTD
CISD
30%
Falherg
0.434 ce: (.007)
0,639 ct: 0.01)
93565 D06)
0.002 c: 0.007
0.001
0.003)
0,350 0.07)
0.388 ce: C--0333)
0.435 c2: 0.011)
Buo00 005)
0/452 0.005
Fashros
11401
04000)
UIME (1.09)
0.30M
(4:0113
(u324)
1.390C11 C4934)
(3100)
(1099)
Fallay
0.413 1:: 0.997)
0.558
(03))
0,337
0 (00)
0.992 c: 0 0.009)
0.500
0.967)
0.353
0/63)
0.313 ct 6.07)
0.385
(2: 0.00)
9.400 0.010)
0.5tg (s: 0.506
FFTS
0.337 L-: (0011
0.400
3:510
01111
0.584 cm 0.007)
0.525
cm 0.0065
0.300/1: 0092)
0.357 c2: 0/934)
0.385 (2-00000)
0:400
0.000
0.918 0.005
Fustaal
11,30 (1: (JR33)
2:099 (880
02540 (L086)
0,79 (.200)
IL339
01543 0180)
0-200 (4013)
Table 28: Long-tern term forecasting and imputation Results with Variance Estimates. We use the fore- casting horizon 720 and masking ratio 50% with MSE as the representative case.
Datasets
FeDaL (Long-term Forecasting)
FeDaL (Imputation)
ETThl
0.457 (#: 0.003)
0.030 (#: 0.001)
ETTh2
0.401 (+: 0.004)
0.022 (+: 0.002)
ETTml
0.464 (#: 0.004)
0.086 (+ 0.001)
ETTm2
0.397 (#: 0.002)
0,048 (#: 0.002)
Weather
0.359 (+ 0.005)
0.024 (+ 0.003)
Table 29: Short-term forecasting results with standard deviations (SMAPE Report).
Intervals
Yearly
Quarterly
Monthly
Others
FeDaL
13.102 (#: 0.11) 9.808 (+ 0.10)
12.124 (# 0.19)
4.508 (+ 0.07)
Table 30: Anomaly detection results with standard deviations (Fl-score Report).
Datasets
SMD
MSL
SMAP
SWaT
PSM
FeDal.
88.46 (+: 0.26)
89.05 (+ 0.12)
71.70 (+= 0.09)
95.40 (+ 0.15)
98.88 (+ 0.21)
36


Reviews:
[{"review_id": "mwDW3KOL8D", "reviewer": "Reviewer_RwPH", "summary": "This paper presents a federated pre-training framework for learning dataset-agnostic temporal representations in heterogeneous settings.\nThe method involves two mechanisms: (1) Domain Bias Elimination (DBE), which decomposes local embeddings into shared and private components, aiming to mitigate client-specific distortions, and (2) Global Bias Elimination (GBE), which corrects cross-client gradient drift via bias-aware aggregation and core-set tuning. The paper reports experiments on several datasets with numerous baselines, empirically showing improved generalization under non-IID conditions.", "questions": "What distinguishes FeDaL from existing domain-invariant FL approaches such as FedBN, MOON, or Ditto?\nCan you quantify the additional communication or computation cost introduced by DBE and GBE?\nHow sensitive is performance to the decomposition granularity (trend/seasonal split) or the hyperparameters controlling bias elimination?\nA formal or empirical measure of bias reduction (perhaps feature distribution alignment?) should be included to substantiate the central claim.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and presents results of a comprehensive experimental study. The problem that it addresses is timely and had previously not been explored enough. The proposed two-level design (DBE for local, GBE for global bias) appears modular and easy to integrate.", "weaknesses": "While the paper reports a strong engineering effort, it unfortunately lacks a clear algorithmic or theoretical innovation. The framework combines concepts from a number of prior works, including (Nie et al., 2022, Wu et al., 2021, Zhang et al., 2015, Acar et al., 2021, Killamsetty et al., 2021), to establish DBE and GBE but these are ultimately heuristic -- there is no theoretical analysis of \"bias elimination\", or of the convergence of the proposed federated scheme. Additionally, it is not clear to this reviewer whether the framework performs genuine bias correction or is the improvement achieved due to introduced regularization -- it seems plausible that a regularization could change how the model fits the data and thus improve generalization even when underlying distributional gaps across domains remains unaddressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DUWs95TlzZ", "reviewer": "Reviewer_pV6m", "summary": "This paper introduces FeDaL, a federated learning framework designed to eliminate dataset-level biases in time series foundation model (TSFM) pretraining. The method tackles the limitations of prior approaches that treat heterogeneity too coarsely, neglecting dataset-level structural biases and compromising cross-domain generalization. Specifically, FeDaL integrates two complementary components—Domain Bias Elimination (DBE) and Global Bias Elimination (GBE)—operating at the client and server levels respectively. DBE disentangles local biases by decomposing masked input representations into trend and seasonal components, injecting a trainable bias vector during reconstruction. GBE introduces a server-side state vector to track accumulated client drift and applies gradient-level correction during aggregation. The paper further analyze federated scaling behavior, demonstrating that federated TSFM pretraining benefits more from data diversity and client participation than from model size. Overall, the work offers a modular and well-motivated approach to improving generalization in federated time series modeling.", "questions": "1. The paper describes the DBE (Dataset Bias Estimation) module as “plug-and-play,” yet its integration requires latent feature decomposition, bias vector computation, and a modified reconstruction loss with an additional alignment term. Could the authors clarify in what sense the module is truly plug-and-play? Specifically, does it require architectural changes, joint optimization, or task-specific tuning? Has its generalizability across different backbone architectures or tasks been empirically validated?\n2. The paper suggests that federated learning naturally decomposes heterogeneous time series data into generalized and personalized components. However, federated learning was originally designed to enable privacy-preserving collaborative modeling, not to perform automatic representation disentanglement. It remains unclear whether this decomposition is an emergent property of the federated setup or the result of explicit modeling choices. Could the authors clarify what mechanisms, if any, enable this decomposition and whether it goes beyond the standard federated architecture?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "1. The paper addresses a well-motivated problem in federated time series modeling, focusing on dataset-level bias and cross-domain generalization.\n2. The proposed FeDaL framework is modular and clearly structured, integrating client-side and server-side bias elimination mechanisms (DBE and GBE).\n3. The gradient-level correction mechanism via a server-side state vector is a meaningful extension of FedAvg, with clear mathematical formulation.\n4. The scaling analysis in Section 4.3 provides valuable insights into federated pretraining dynamics.", "weaknesses": "1. Unclear Structure in Problem Statement: The introduction claims that existing works face “two major challenges,” but only one (coarse-grained treatment of heterogeneity) is explicitly developed. The second challenge is not clearly introduced or elaborated, which weakens the framing of the paper’s motivation.\n2. The Global Bias Elimination (GBE) module introduces a cumulative server-side state vector sr, which may be prone to instability over long training rounds if not properly scaled. While the paper provides empirical evidence supporting the choice of β, further theoretical analysis or broader sensitivity exploration would strengthen confidence in its stability.\n3. Occasional Conceptual Overreach in Terminology: Certain statements, such as “the distributed architecture of federated learning is a nature solution to decompose heterogeneous TS datasets,” are methodologically oversimplified and should be more rigorously justified. \n4. The caption of Figure 1 refers to four settings—FedAvg-Independent, FedAvg-Mixed, Stand-alone, and Transformer—yet the figure itself only visualizes the first three. The Transformer setting is neither shown in the bar plots nor explained in the figure annotations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eNp8iu1oGc", "reviewer": "Reviewer_oKpr", "summary": "The paper proposes a solution to handle heterogeneity when learning representations for time series foundational models. They focus on the federated scenario, highlighting three sources of heterogeneity: mismatch in sampling frequencies, heterogeneity in the underlying physical processes, and divergence due to exogenous events. They propose two solutions: domain bias elimination (DBE) and Global bias elimination (GBE), at the client and server levels respectively. DBE decomposes a series into trend and seasonal components per client and learns a common representation across local datasets. GBE tackles the remaining misalignments through a moving-average like process to avoid client-specific patterns from dominating the global trend. They further refine it using a core-set fine-tuning. The method is evaluated on IID and non-IID scenarios showing improvements over other federated baselines at learning representations. Comparisons are also made against other time series foundation models on several downstream tasks using the learned representations, showing improvements over the baselines.", "questions": "See the weaknesses W1-W4. Additionally I have the following minor comments.\n- The code link provided for the baselines in Appendix B page 26 is not working. \n- The imbalance factor of  H1 and H2 (Appendix page 21) seem quite small (0.1 and 0.3). I wonder how well the method would perform on UTSD if this is increased to say 0.7+. \n- The absolute number for the core-set size in table 8 is not sufficient. I think it is important to also indicate the percentage w.r.t the training data size.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- The paper shows many experimental results and ablation studies to show how it performs on multiple aspects. The breadth of baselines covered is impressive.\n-  Performance results are good.\n- The research problem is very relevant for the community", "weaknesses": "- The novelty of the method is weak. The method simply combines existing works and plugs them into the framework. For e.g. DBE applies decomposition (Wu et al. 2021) and EMA from (Zhang et al. 2015). GBE uses drift correction from (Acar et al. 2021) and core set tuning from (Killamsetty et al 2021). \n- Many of the federated baselines in Table 1 are quite old, except for FFTS. Newer baselines like Time-FFM (Liu et al. 2024) are missing.  \n- Table 1 also does not compare against strong centralized TSFM baselines, where data from all clients is pooled. For e.g. Moirai and Chronos. I understand that these comparisons have been made on downstream tasks, but it is also necessary to show these results in table 1 to see how close a distributed approach is to centralised representation learning.\n- The method’s performance improvement over the baselines seems highly dependent on all components being present (Table 2). I’m wondering if adding just one or two components to an existing baseline e.g. FFTS would make it even stronger than the proposed method.\n- The results on privacy-preservation are not clear. In the TSNE plots in the appendix and Fig. 6, the authors claim that the perturbed points (blue) are privacy-preserving. But visually, to me it looks like the green (initial core set) is further away from the original data (pink). These figures should be accompanied with a quantitative metric indicating how close or far the perturbed points are from the actual data, instead of relying on visualizations alone.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O5TTDlRHna", "reviewer": "Reviewer_CA7V", "summary": "In this work, the authors propose a novel Federated Dataset Learning (FeDaL) method to address heterogeneous time series data. FeDal mainly considers domain bias elimination (DBE) and global bias elimination (GBE), which alleviates the domain bias caused by heterogeneous time series datasets and enables learning domain-invariant time representations. Extensive experimental results on several datasets show that FeDaL outperforms existing methods in many downstream tasks.", "questions": "See in weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The presentation of the proposed method is clear, and the paper is overall well-written.\n\n2. This paper considered the challenge of time-series dataset heterogeneity, which is a significant and novel problem in FL.\n\n3. The evaluation of the proposed method is comprehensive.", "weaknesses": "1. In this work, the foundation model is trained directly on the client without considering the computational burden and parameter-efficient fine-tuning (PEFT) method.\n\n2. The paper proposes several types of bias (temporal resolution, physical constraint, pattern transition), but does not provide a reproducible quantitative metric (how to label a dataset as \"high resolution bias\"?). Therefore, it is impossible to determine which datasets are more affected by DBE.\n\n3. The experiments were compared with a large number of time series models. However, these methods are not designed for federated learning. This is not a fair and convincing comparison. At the same time, among the baselines for federated learning, only FFTS is designed for time series. More baselines for federated time series and federated time series foundation models should be considered, such as [1][2].\n\n4. It could be better to report the number of repetitions and standard error for each experiment and add a description of the variance.\n\n\n[1] Liu, Qingxiang, et al. \"Time-FFM: Towards LM-empowered federated foundation model for time series forecasting.\" In NeurIPS 2024.\n\n[2] Xu, Ronghui, et al. \"PeFAD: a parameter-efficient federated framework for time series anomaly detection.\" In KDD 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)