NeurIPS 2025

Generalizable, real-time neural decoding with hybrid
state-space models
Avery Hee-Woon Ryoo1,2,∗,B
Nanda H Krishna1,2,∗,B
Ximeng Mao1,2,∗,B
Mehdi Azabou3
Eva L Dyer4
Matthew G Perich1,2,†
Guillaume Lajoie1,2,5,†,B
1Mila – Quebec AI Institute
2Université de Montréal
3Columbia University
4University of Pennsylvania
5Canada CIFAR AI Chair
∗Co-first authors
†Co-senior authors
B{hee-woon.ryoo,nanda.harishankar-krishna,ximeng.mao,guillaume.lajoie}@mila.quebec
https://possm-brain.github.io
Abstract
Real-time decoding of neural activity is central to neuroscience and neurotech-
nology applications, from closed-loop experiments to brain-computer interfaces,
where models are subject to strict latency constraints. Traditional methods, includ-
ing simple recurrent neural networks, are fast and lightweight but often struggle
to generalize to unseen data. In contrast, recent Transformer-based approaches
leverage large-scale pretraining for strong generalization performance, but typically
have much larger computational requirements and are not always suitable for low-
resource or real-time settings. To address these shortcomings, we present POSSM, a
novel hybrid architecture that combines individual spike tokenization via a cross-
attention module with a recurrent state-space model (SSM) backbone to enable (1)
fast and causal online prediction on neural activity and (2) efficient generalization
to new sessions, individuals, and tasks through multi-dataset pretraining. We evalu-
ate POSSM’s decoding performance and inference speed on intracortical decoding
of monkey motor tasks, and show that it extends to clinical applications, namely
handwriting and speech decoding in human subjects. Notably, we demonstrate that
pretraining on monkey motor-cortical recordings improves decoding performance
on the human handwriting task, highlighting the exciting potential for cross-species
transfer. In all of these tasks, we find that POSSM achieves decoding accuracy
comparable to state-of-the-art...