Title: Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
Primary area: learning on time series and dynamical systems
Keywords: Time Series Foundation Models, Time Series Forecasting, Federated Foundation Models

Abstract: Heterogeneity in time series data is more pronounced than in vision or language, as temporal dynamics vary substantially across domains and tasks. Existing efforts about time series foundation models (TSFMs) are often trained with mixed-batch strategies that merge large-scale datasets, which can cause gradient conflicts and degrade representation quality. To address this, we propose a fine-grained learning framework that distills invariant knowledge from heterogeneous series while reducing cross-domain interference. We characterize heterogeneity at two levels: inter-domain and intra-domain. To tackle this bi-level heterogeneity, we design a federated learning framework that mitigates intra-domain conflicts by enforcing domain-invariant and semantically consistent representations through local regularization, and addresses inter-domain discrepancies by enhancing cross-domain collaboration via domain-aware aggregation. Experiments across diverse benchmarks demonstrate that TSFMs trained with our method consistently outperform both centralized and federated TSFM baselines in point and probabilistic forecasting, while also achieving competitive zero-shot performance at scale, offering a flexible pathway for training TSFMs in heterogeneous environments.

TL;DR: We propose FedTRL, a federated framework for bi-level heterogeneous learning that mitigates inter- and intra-domain heterogeneity, enabling flexible TSFM pretraining with strong zero-shot generalization.