Title: Panda: A pretrained forecast model for chaotic dynamics
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: chaos, nonlinear dynamics, forecasting, physics, scientific machine learning, dynamical systems

Abstract: Chaotic systems are intrinsically sensitive to small errors, challenging efforts to construct predictive data-driven models of real-world dynamical systems such as fluid flows or neuronal activity.
Prior efforts comprise either specialized models trained separately on individual time series, or foundation models trained on vast time series databases with little underlying dynamical structure.
Motivated by dynamical systems theory, we present $\textit{Panda}$, $\textit{P}$atched $\textit{A}$ttention for $\textit{N}$onlinear $\textit{D}$yn$\textit{A}$mics.
We train $\textit{Panda}$ on a novel synthetic, extensible dataset of $2 \times 10^4$ chaotic dynamical systems that we discover using an evolutionary algorithm.
Trained purely on simulated data, $\textit{Panda}$ exhibits emergent properties: zero-shot forecasting of unseen chaotic systems preserving both short-term accuracy and long-term statistics.
Despite having been trained only on low-dimensional ordinary differential equations, $\textit{Panda}$ spontaneously develops the ability to predict partial differential equations without retraining.
We also demonstrate a neural scaling law for differential equations, underscoring the potential of pretrained models for probing abstract mathematical domains like nonlinear dynamics.

TL;DR: We introduce a novel large chaotic systems dataset and use it to pretrain a patch-based foundation model for chaos.