Title: Neural SDEs as a Unified Approach to Continuous-Domain Sequence Modeling
Primary area: generative models
Keywords: Neural Stochastic Differential Equations, Flow Matching, Diffusion Models, Continuous-Time Sequence Modeling

Abstract: Inspired by the ubiquitous use of differential equations to model continuous dynamics across diverse scientific and engineering domains, we propose a novel and intuitive approach to continuous sequence modeling. Our method interprets timeseries data as discrete samples from an underlying continuous dynamical system, and models its time evolution using Neural Stochastic Differential Equation (Neural SDE), where both the flow (drift) and diffusion terms are parameterized by neural networks. We derive a principled maximum likelihood objective and a simulationfree scheme for efficient training of our Neural SDE model. We demonstrate the versatility of our approach through experiments on sequence modeling tasks across both embodied and generative AI. Notably, to the best of our knowledge, this is the first work to show that SDEbased continuous-time modeling also excels in such complex scenarios, and we hope that our work opens up new avenues for research of SDE models in high-dimensional and temporally intricate domains.

TL;DR: We proposed a new sequence modeling approach tailored to continuous domain, as a unified framework for embodied and generative AI tasks.