Title: StarEmbed: Benchmarking Time Series Foundation Models on Astronomical Observations of Variable Stars
Primary area: datasets and benchmarks
Keywords: benchmark, astrophysics, light curve, photometrics, ZTF, periodic variables, time series foundation models, embedding models, classifcation

Abstract: Time series foundation models (TSFMs) are increasingly being adopted as highly-capable general-purpose time series representation learners. Although their training corpora are vast, they exclude astronomical time series data. Observations of stars produce peta-scale time series with unique challenges including irregular sampling and heteroskedasticity. We introduce $\texttt{StarEmbed}$, the first public benchmark for rigorous and standardized evaluation of state-of-the-art TSFMs on stellar time series observations (``light curves''). We benchmark on three scientifically-motivated downstream tasks: unsupervised clustering, supervised classification, and out-of-distribution source detection. $\texttt{StarEmbed}$ integrates a catalog of expert-vetted labels with multi-variate light curves from the Zwicky Transient Facility, yielding $\sim$40k hand-labeled light curves spread across seven astrophysical classes. We evaluate the zero-shot representation capabilities of three TSFMs ($\texttt{Moirai}$, $\texttt{Chronos}$, $\texttt{Chronos-Bolt}$) and a domain-specific transformer ($\texttt{Astromer}$) against handcrafted feature extraction, the long-standing baseline in the astrophysics literature. Our results demonstrate that these TSFMs, especially the $\texttt{Chronos}$ models, which are trained on data completely unlike the astronomical observations, can outperform established astrophysics-specific baselines in some tasks and effectively generalize to entirely new data. In particular, TSFMs deliver state-of-the-art performance on our out-of-distribution source detection benchmark. With the first benchmark of TSFMs on astronomical time series data, we test the limits of their generalization and motivate a paradigm shift in time-domain astronomy from using task-specific, fully supervised pipelines toward adopting generic foundation model representations for the analysis of peta-scale datasets from forthcoming observatories.

TL;DR: We propose the first benchmark for time series foundation model on stellar time series observations.