Estimating Time Series Foundation Model Transferability via In-Context
  Learning
Authors: name: Qingren Yao, name: Ming Jin, name: Chengqi Zhang, name: Chao-Han Huck Yang, name: Jun Qi, name: Shirui Pan

Time series foundation models (TSFMs) offer strong zero-shot forecasting via
large-scale pre-training, yet fine-tuning remains critical for boosting
performance in domains with limited public data. With the growing number of
TSFMs, efficiently identifying the best model for downstream fine-tuning
becomes increasingly challenging. In this work, we introduce TimeTic, a
transferability estimation framework that recasts model selection as an
in-context-learning problem: given observations on known (source) datasets, it
predicts how a TSFM will perform after fine-tuning on a downstream (target)
dataset. TimeTic flexibly organizes the observed model-data relationships as
contextual information, allowing it to adapt seamlessly to various test-time
scenarios. Leveraging the natural tabular structure formed by dataset
meta-features, model characteristics, and fine-tuned performance, we employ
tabular foundation models to serve as in-context learners. We further introduce
a novel model characterization based on entropy evolution across model layers,
capturing embedding-space distinctions and enabling TimeTic to generalize
across arbitrary model sets. We establish a comprehensive benchmark for
transferability estimation including 10 datasets, 10 foundation models, and 3
forecasting tasks. On this benchmark, TimeTic's estimation demonstrates strong
alignment with actual fine-tuned performance for previously unseen datasets,
achieving a mean rank correlation of approximately 0.6 and a 30% improvement
compared to using zero-shot performance as the transferability score.