Title: Neural Fourier Attention: A Framework for Learning Data-Adaptive Signal Bases
Primary area: learning on time series and dynamical systems
Keywords: Neural Fourier Attention, data-adaptive basis, frame theory, orthogonality regularization, time series forecasting, non-stationary signals, spectral representation, FFT efficiency, Harmonic-Constrained NFA, LSTM controller

Abstract: The standard approach in signal processing and deep learning is to use fixed,
predefined basis functions. This includes classic methods like the discrete Fourier
transform (DFT) and modern tools like convolutional kernels. Our work challenges
this fundamental paradigm. We introduce Neural Fourier Attention (NFA; also
referred to as a Neural Fourier Basis Generator, NFBG), a framework designed to
learn a data-adaptive, non-stationary signal transform. The core idea is to replace
projection onto a fixed basis with a neural controller that dynamically generates
basis-function parameters for each input, constructing a bespoke representation
tailored to the signal’s characteristics. Importantly, our use of the term “Attention”
does not denote a query–key–value operator; it denotes a controller that attends to
the input to generate the basis. We provide a rigorous formulation within frame
theory and propose an orthogonality regularizer that encourages near-tight frames
for stability. Extensive experiments show state-of-the-art results on challenging
benchmarks (notably on Weather) and strong average performance across the
ETT family, with ablations validating the controller design and the benefits of
controlled non-orthogonality. When to use/when not to: NFA targets signals
with complex, non-stationary periodicities (e.g., Weather/ETT); for trend-dominant
or near-random-walk series (e.g., Exchange), a hybrid pipeline that decomposes
trend/seasonality first and applies NFA to residuals is recommended.

TL;DR: Neural Fourier Attention learns data-adaptive Fourier bases with frame-theoretic regularization, improving non-stationary time-series forecasting; an FFT-based HC-NFA achieves O(N log N) efficiency with strong accuracy.