Title: Variability Aware Recursive Neural Network (VARNN): A Residual-Memory Model for Capturing Temporal Deviation in Sequence Regression Modeling
Primary area: learning on time series and dynamical systems
Keywords: Time-series regression, non-stationarity, residual learning, recurrent neural networks, distribution shift, system identification.

Abstract: Real-world time series data exhibit non-stationary behavior, regime shifts, and temporally varying noise that degrade the robustness of standard regression models. We introduce the Variability-Aware Recursive (VARNN) Neural Network, a novel residual-aware architecture for supervised time-series regression that learns an explicit error memory from recent prediction residuals and uses it to recalibrate subsequent predictions. VARNN augments a feed-forward predictor with a learned error-memory state that is updated from residuals over a short context steps as a signal of variability and drift, and then conditions the final prediction at the current time. We study four configurations along two orthogonal axes: (i) residual memory as instantaneous (RM) an embedding of the current innovation only, or accumulative (ARM) that augment current innovation with past residual memory states to capture drift and volatility bursts; and (ii) the presence or absence of an activation memory (AM), which carries the previous latent activation to enrich short-term temporal dynamics and stabilize predictions under noise. Across diverse dataset domains, appliance energy, healthcare, and environmental monitoring, experimental results demonstrate that VARNN achieves superior performance and attains the lowest test MSE with minimal computational overhead over static, dynamic, and recurrent baselines. Our findings show that the VARNN model offers robust predictions under a drift and volatility environment, highlighting its potential as a promising framework for time-series learning.

TL;DR: We propose a VARNN, a residual-memory recursive neural network that improves robustness to variability in time-series regression.