Neural ODE with Differentiable Hidden State for Irregular Time Series

Authors: Yan Fang, Xu Wang, Zeyu Liang, Yudong Zhang, Pengkun Wang, Binwu Wang, Yang Wang
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: time series analysis, irregular time series, neural ordinary differential equations

Abstract:
Capturing the continuous underlying dynamics of irregular time series is essential for accurately reflecting the ongoing evolution and intricate correlations within the data. The discrete nature of current models, including RNN-based models and transformer variants, poses challenges when it comes to generalizing to the continuous-time data paradigms, which is necessary for capturing ongoing dynamics of irregular time series. 
Neural Ordinary Differential Equations (NODEs) assume a continuous latent dynamic and provide an elegant framework for irregular time series analysis. However, integrating new information while maintaining the continuity of latent dynamics remains challenging. 
To tackle this problem, we introduce Differentiable Hidden State (DHS) enhanced neural ODE, a data-dependent framework that is capable of effectively capturing temporal dependencies and ensuring the continuity of the hidden process. We leverage the theory of generalized inverses to innovatively compute attention mechanism in reverse and obtain a continuous representation. To capture more accurate temporal relationships, we introduce Hoyer metric and maximize the sparsity of it. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our model.

Decision: desk reject

Reviews and discussion:

Metareview: