Pretraining with Re-parametrized Self-Attention: Unlocking Generalizationin  SNN-Based Neural Decoding Across Time, Brains, and Tasks
Authors: Yuqi Yang
Tengjun Liu
Haiyan Zhang
Wang Ruixue
Xuchao Chen
Mingkang Li
Yansong Chua
Nenggan Zheng
Shaomin Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Brain-Machine Interface
Neural Spike Decoding
Spiking Neural Network
Foundation Model
Primary area: applications to neuroscience & cognitive science
TL;DR: Combining re-parametrized attention with multi-timescale dynamics, we design a lightweight pretrained SNN, achieves SOTA performance on SNN CST decoding tasks and demonstrates strong generalization to unseen conditions with low computational cost.

Abstract: The emergence of large-scale neural activity datasets provides new opportunities to enhance the generalization of neural decoding models. However, it remains a practical challenge to design neural decoders for fully implantable brain-machine interfaces (iBMIs) that achieve high accuracy, strong generalization, and low computational cost, which are essential for reliable, long-term deployment under strict power and hardware constraints.
To address this, we propose the Re-parametrized self-Attention Spiking Neural Network (RAT SNN) with a cross-condition pretraining framework to integrate neural variability and adapt to stringent computational constraints. 
Specifically, our approach introduces multi-timescale dynamic spiking neurons to capture the complex temporal variability of neural activity.
We refine spike-driven attention within a lightweight, re-parameterized architecture that enables accumulate-only operations between spiking neurons without sacrificing decoding accuracy.
Furthermore, we develop a stepwise training pipeline to systematically integrate neural variability across conditions, including neural temporal drift, subjects and tasks.
Building on these advances, we construct a pretrained model capable of rapid generalization to unseen conditions with high performance. 
We demonstrate that RAT SNN consistently outperforms leading SNN baselines and matches the accuracy of state-of-the-art artificial neural network (ANN) models with much lower computational cost under both seen and unseen conditions across various datasets. 
Collectively, pretrained-RAT SNN represents a high-performance, highly generalizable, and energy-efficient prototype of an SNN foundation model for fully iBMI. 
Code is available at [RAT SNN GitHub](https://github.com/YangYangYangYuqi/RAT-SNN).

Published as a conference paper at ICLR 2026
PRETRAINING WITH RE-PARAMETRIZED SELF- ATTENTION: UNLOCKING GENERALIZATION IN SNN- BASED NEURAL DECODING ACROSS TIME, BRAINS, AND TASKS
Yuqi Yang'; Tengjun Liu Haiyan Zhang', Ruixue Wang', Xuchao Chen', Mingkang Li', Yansong Chua"j Nenggan Zheng', Shaomin Zhang
Zhejiang University, Hangzhou, China
Intellizon Medical Ltd., Suzhou, China
3 Westlake University, Hangzhou, China
4 Scalefree Intelligence Ltd., Shanghai, China
ABSTRACT
The emergence of large-scale scale neural activity datasets provides new opportunities to enhance the generalization of neural decoding models. However, it remains a practical challenge to design neural decoders for fully implantable brain-machine interfaces (iBMIs) that achieve high accuracy, strong generalization, and low com- putational cost, which are essential for reliable, long-term deployment under strict power and hardware constraints. To address this, we propose the Re- parametrized self-Autention Spiking Neural Network (RAT SNN) with a cross-condition pre- training framework to integrate neural variability and adapt stringent computa- tional constraints. Specifically, our approach introduces multi-timescale dynamic spiking neurons capture the complex temporal variability of neural activity. We refine spike-driven attention within a lightweight, re re-parameterized architecture that enables accumulate-on operations between spiking neurons without sacri- ficing decoding accuracy. Furthermore, we develop stepwise training pipeline to systematically integrate neural variability across conditions, including neural temporal drift, subjects and tasks. Building these advances, we construct pre- trained model capable of rapid generalization to unseen conditions with high per- formance. We demonstrate that RAT SNN consistently outperforms leading SNN baselines and matches the accuracy of state-of-the-art artificial neural network (ANN) models with much lower computational cost under both and unseen conditions across various datasets. Collectively, pretrained-RAt SNN represents a high-performance, highly generalizable, and energy-efficient prototype of an SNN foundation model for fully iBMI. Code is available at RAT SNN GitHub
INTRODUCTION
Brain-machine interfaces (BMIs) have advanced significantly in enabling direct brain-to-device communication (Liu et al. 2020 Zhang et al. 2023 Rapeaux & Constandinoul 2021). Implantable BMIs (iBMIs), recording high-fidelity neural signals via cortical microelectrode arrays, offer supe- rior signal quality and control precision compared to non-invasive ones (Rapeaux & Constandinou, 2021). Prior work has demonstrated the feasibility of using iBMIs for autonomous robotic arm con- trol (Hochberg et al. 2012) Flesher et al. 202T and text generation (Pandarinath et al.) [017] Willett let al. 2001 Moses et al. 2021 Card et al. 2024). However, enabling decoding models to gener- alize across subjects, tasks, and time remains core challenge. This difficulty stems from intrinsic neural variability, including inter-subject differences, inter-task variations, and temporal drift within subjects (Driscoll al. 2022). Such variability makes it difficult to apply single model across diverse datasets: the same task might elicit different neural patterns in different subjects, while single subject may exhibit distinct activity under varying contexts (Dinstein al.) (015)..
"Equal contribution. 'Corresponding authors, shaomin @zju.edu.cn, James 44246 hotmail.com.

---

Published as a conference paper at ICLR 2026
Originally developed in natural language processing and computer vision (Wang et al. 2023). foun- dation models aim to scale learning across domains, improving generalization and minimize retrain- ing. In iBMIs, such models hold promise for improving tross-condition generalizabilityy enabling rapid deployment when subject-specific data are scarce or costly to obtain. Recent advances in large- scale neural recording have made it possible to obtain rich datasets (Perich al. [2018] Pei al.) 2021 O O'Doherty et al.) 2017: Churchland et al.) 2012). making it feasible to train foundation neural decoding models that generalize across varied conditions including sessions, subjects, and tasks.
Several Artificial Neural Network (ANN) based foundation models have recently been proposed, including POYO (Azabou al. 2023) and NDT3 et. 2025). They share common underlying paradigm: scaling up to accommodate large volumes of data. Yet. their reliance on large models and significant computational resources hinders real-time deployment
rexource-constrainee fully iBMls, which reduce infection risk by eliminating external pedestals and also promote long-term adaptation and privacy (Yik et al. 2025; Taeckens & Shah 2024 Yuste et al., 2017). But they impose strict constraints on decoder computational energy consumption, model size, and latency. Therefore, to meet the requirements of fully iBMIs, it is imperative to develop decoders that can deliver reliable and accurate performance under strict latency and power constraints.
Spiking neural networks (SNNs), mimicking biological ppske-based event-driven (spike-drice driven) na- ture Yao al. 2023b), significantly reduce computational costs by replacing multiply-an aacuuullaee (MAC) with accumulate-only (AC) operations (Yin et al. 2021). Evolving in discrete state spaces, SNNs exhibit greater robustness temporal noise (Lichtsteiner et al.. 2008), and considered advantagcous for processing discrete spatiotemporal outputs (Liang et al. 2023 Wu et al.) 2024). Especially, SNNs, whose event-driven and discrete nature aligns well with the spiking paradigm, particularly compatible with cortical spike train (CST), thereby minimizing the need for extensive data preprocessing. However, existing research on SNN- basedased CST decoding remains relatively pre- liminary, and the architectures employed are often too simplistic to either achieve high- high-performance decoding or serve as cross-condition foundation models.
To address the above challenges of building accurate, generalizable and efficient CST decoder, we propose Re-parametrized self-Attention (RAT) SNN. It features a spike-driven self-attention enhanced with re-parameterizatioon, allowing the use of batch normalization (BN) during training while maintaining AC-only operations between synapses during inference. In addition, RAT SNN introduces recurrent connections among leaky integrate-and- fire (LIF) neurons combined with dy- namic synapses to capture multi-timescale dynamics effectively. Furthermore, our method intro- duces subject-specific BN as lightweight approach accommodate distributional shifts across subjects. Through a custom cross-condition training pipeline, RAT SNN-CC achieves robust inte- gration across time, subjects, and tasks, yielding scalable, low-power SNN foundation model for generalizable neural decoding. Our main contributions can be summarized as follows:
We propose RAT SNN, a lightweight spiking neural network that integrates re- parameterized spike-driven self-attention with multi-timescale dynamics, and maintains AC-only inter-synaptic operations and architectural efficiency.
We introduce subject-specific BN along with a stepwise pretraining framework for RAT SNN, enabling the integration of neural temporal drift within an individual subject as well as other neural variability across subjects and tasks with spike-driven inference.
We evaluate method multiple datasets, encompassing diverse conditions. RAT SNN demonstrates rapid generalization, consistently outperforms state-of-the-art (SOTA) SNN baselines, and matches the performance of leading ANN models in decoding accuracy.
RAT SNN is lightweight and low computational cost model that achieves lower power consumption with fewer parameters and high decoding accuracy, and provides a promising solution for fully iBMIs.
2 RELATED WORKS
2.1 ANN-BASED CST DECODER
Numerous studies have employed ANNs for CST movement decoding (Ye & Pandarinath 2021
Ye et al. 2023 Liu al. 2022 Schneider al. 2023 Le & Shlizerman 2022 Pandarinath al.
2

---

Published as a conference paper at ICLR 2026
2018). As example for iBMIs, AEGRU (Wang et al. 2024) incorporates an autoencoder-basee training module to boost performance while maintaining training efficiency and lightweight infer- ence. Although achieving excellent performance within specific training experimental conditions, these works largely ignore the challenges posed by neural variability across subjects and tasks. To address neural drift across time, pioneering work has made significant efforts (Farshchian et al.) Degenhart et al.] 2020). NOMAD (Karpowicz) establishes bascline models on anchor sessions, then aligns subsequent sessions through dynamic time warping. While improving sessinn consistency, this alignment relies on the assumption of shared dynamics or task structure. To tackle broader neural variability, POYO (Azabou et al. (023) and NDT3 (Ye et al. 2025) employ multi- layer transformer architectures trained on large-scale neural activity data, enhancing generalization through scaling up. NDT3 utilizes simple tokenizer for neural firing rates, whereas POYO con- structs UnitEmbed UeiEEmbedrepreeentttee representations by tokenizing neural spiking sequences, enabling rapid finctuning across subjects. However, these previous methods rely large models and significant computa- tional resources, hindering deployment in resource- constrcccd fully iBMls. In this work, we intro- duce re- ce-paramerrzzaion method replace the tokenizer with subject-specifie BN layer, which can be seamlessly integrated into the linear operations between spiking neurons. This modification enables RAT SNNs to be pretrained on large datasets and then rapidly adapted to new subjects and tasks, while maintaining lightweight architecture without additional computational overhead.
2019 Karpowicz et al. 2025; Dyer et al.
2017
al. 2025) and ADAN (Farshchian al. 2019)
2.2 SNN-BASED CST DECODER
As representative lower models, SNNs have been explored for iBMIs (Yik ct al. 2025 Taeck- ens & Shah, 2024 Liao et al. 2022 Zhou et al., 2024 Liu et al. 2025 2024). However, recent studies primarily emphasize power efficiency, often yielding suboptimal decoding accuracy com- pared to ANNs. Although the SNN3d model proposed by NeuroBench demonstrates significant gains over classical SNNs, its incorporation of MAC operations between synapses substantially in- creases computational complexity leading to higher power consumption, deviating from core SNN principles (Yik al., 2025). Currently, bigRSNN stands as the best-performing SNN model re- ported for CST decoding by leveraging cross- pretraining (Liu et al.) (024).. However, these studies predominantly rely on simplistic SNN architectures, and have not effectively explored the potential of SNNs to serve as foundation model capable of addressing neural variability across time, subject, and tasks. Furthermore, Spikachu (Mentzelopoulos et al., 2025) incorporates atten- tion mechanisms but relies on ANN harmonizer to address neural variability, introducing massive MAC operations. In contrast, we propose RAT SNN, a lightweight pure SNN architecture that in- tegrates re-pat parameterized dpike- self- self-attention with timescale dynamics to achieve high decoding accuracy while maintaining AC-only operations for adhering to strict power constraints.
3
METHOD
To develop compact yet effective neural decoder, we propose RAT SNN of only layers of LIF neurons, including a CST re- parameetrized attention block and readout block (Fig. I RAT SNN in- corporates key components: 1) Dynamic synapse mechanisms with recurrent connections, which enhance the ability to capture multi-timmesale neural dynamics; 2) CST re- parameterrizaiion self- attention, which improves training efficiency and final performance; 3) Subject-specific BN and cross-condition training framework, which further enhance decoding performance and the gen- cralization ability of RAT SNN-CC. In the remainder of this section, we will introduce detailed descriptions of each of these three components.
3.1 MULTI-TEMPORAL DYNAMIC SPIKING NEURON
The CST re-parametrized attention block in RAT SNN employs LIF neurons as fundamental com- putational units (Maass) 1997), with discrete-time dynamics defined as:
[ a V Vmemm it 1] + Vsyult 1]
(1)
S[t] = Hea( H [t] Vín)
(2)
V
[ [t][ -Sit)) + Vieser
(3)
3

---

Published as a conference paper at ICLR 2026
Spike Input
CST Re parametrized Attention Block
Readout Block
Rejiadt. d)
Rep-
Cony
Rep-FC
Rep-
K Kesteylnn dif
Rep-FC
Rendout
Rep- Conv
Rep-
[n. d)
FC
FC
ypkalln d)
Hlead
(ML IP Stoee
Rep-Cony
B.N
Rep-FCC
Matris Multiplictioon
LIF Layer
Carr
Layer
Comy
BN
Recurrent LIF layer
Figure RAT SNN overall architecture. Each time step, RAT SNN takes spikes input with dimension (CST Channel Num, 1) from cortical spike train (CST), and outputs kinematic predic- attention block and readout
tions after processing through CST re- re-parametrize block. Rep-FC denotes re-parameterized fully connected layer, Rep-Conv refers to three- branch 1D convolutional layer. n and represent the number of tokens and the feature dimension of self-attention, MLP Size indicating output dimension of CST re- re-paramerrzee attention block. Head and Ro indicates number of attention heads and multi-readout block.
where H [t] denotes membrane potential before neuronal dynamics at time step Vieel [t 1], Veyn 1] are current membrane and synaptic potential, < 1 is decay coefficient. Hea(-) is the Heaviside step function, indicating that output spike S[t) emits when H [t] exceeds threshold Vin. which resets to Vreser afterwards; otherwise, [[t] continues decay without spiking (Rossbroich et al. 2022).
To simulate the temporal dynamic adaptabil- ity of biological neuron synapses and enhance the multi-timescale dynamics, we consider the recurrently connected LIF neuron with synap- tic dynamics Recurrent-LIF (Fig. [ and maintain heterogeneity among intralayer neu- rons (Perez-Nieves et al. 2021). Specifically,
fux ()
txp ()
Mas()
V Yaues
Synapee
Nambran
V Vy [t]
Σ frm. (t) + Skoe (S 1))
Figure 2: Recurrent-LIF structure. Recurrent- LIF neuron receives upstream spikes 5" and intra- (4) layer projection (S'), updates Vsyn and H, and then emits spikes 5 and updates Vmem via Hea(.).
1=1
+ + syal 1]
where (-) denotes the feedforward connec-
tions of upstream spike Sd in LIF layer with indicate different branches. And fRec(-) denotes fully-connection (FC) here. B < rep- resents the synaptic decay factor. Recurrent FC forming internal layer-wise dynamics. Together with inter-layer connections, it mimics the coexistence of long-range projection pathways and local microcircuits in biological neural systems.
the recurrent connections, we use re-parameterrization-syy
The readout block consists of five heads, each comprising two Leaky Integrator (LI) neurons to predict the two-dimensional forelimb movement velocity. LI neurons share the same membrane potential accumulation and leakage processes as LIF neurons without spiking mechanism, aiming to simulate the smooth motion characteristies of muscle groups, thereby converting the discrete spiking activity S[t] into continuous outputs:
H
v-Ll memll
[t] a UpL mem [t 1] V-Ll [t
(5)
3.2 CST RE-PARAMETRIZES SELF-ATTENTION
The main body the CST attention block is multi-head self-attention. By leveraging the binary nature of SNN, the computational complexity of the attention operation can be reduced from O()2d)
4

---

Published as a conference paper at ICLR 2026
to O as in linear attention, where >> d. In combination with Recurrent-LIF:
(6) where Qu Ks, and V. respectively represent query(Q), key(K) and value(V) of i-th attention head with spike representation, and RLIF(.) denotes the temporal dynamies of Recurrent-LIF.
Attentions; RLIF V..)) RLIF(Qs, (K,, V..)
Previous studies have shown that convolution layer (Conv) can effectively enhance the performance of spike-driven attentions (Yao et al., 2023b). However, considering the discontinuity of electrode channel indices and inter-subject differences, we integrate parallel Conv and FC for Qs/Ks/Vs, s, where FC captures global features and Conv focuses on local features across CST channels:
2z/Ka/V, (LIF(RRp-Conv(Xe (Xcc) Rep-FC(XcST))
(7)
where Xest denotes the input CST spikes at current time step and Rep-Conv() and Rep-FC(() denote the re-parameterized Conv and FC.
Moreover, previous experiments have shown that overly deep SNNs can impair decoding accuracy and real-time efficiency in CST decoding for BCI systems and other applications (Rossbroich al, 2022). Therefore, we merge the MLP block and the output layer in the attention block from prior spike-driven transformer designs (Yao et al., 2023a b Zhou et al. 2023 2022), reducing the depth from the classic five layers to three: this shallower structure not only helps improve accuracy but also reduces model size and computational operations.
To further reduce model depth, we compute Qs/Ks/Vs directly from the input CST spikes and, since they lack membrane potentials, replace the membrane-shorteut (MS) in spike-driven transformers with spike-element-wisg shortcut (Fang et al. 2021) followed by a linear layer (a "surrogate short- cut"):
U. (AttenAttnntion, + Rep-FC(Xcst))
where Ug denotes the output spikes of CST re-parametrized attention block at current time step. Surrogate shortcut enables changing the attention output dimension (MLP Size > X d, Fig. D. providing flexible output dimensionality and improving performance without increasing depth.
3.2.1 RE-PARAMETRIZED SELF-ATTENTION
Furthermore, structural re- re-parameterization was originally proposed to address issues such as gra- dient vanishing and suboptimal performance in VGG (Ding et al. 2021). However, has also been found to be highly beneficial for SNNs Yao et al. 2023a), enabling the fusion of complex inter-layer structures in SNNs into a single linear connection, thereby promoting training stability and keeping the pure AC operations (without MAC operations) between LIF neurons during the inference phase.
Building on this, we extend the CST attention block to the CST re-parametrized attention block that leverages BN during training to improve performance and accelerate convergence. Specifically, introduce two re-parameterized structures: Rep-Conv (ID Conv) and Rep-FC.
Inspired by RepConv2d in RepVGG (Ding al., 2021), Rep-Conv employs multiple parallel branches, including 1 x 1 kernel Conv ()) kernel Conva(-), optionally larger kernels Conv.(.) 5, .,... and identity mapping id(.) which can be viewed Conv operation with an identity kernel (-). Given an input X
Rep-Conv(X.) = CConv,(X3 E.N, BN, X.+ bb)
iEK
TEN
where * denotes the Conv operation, W,, b, and BN(.) denotes kernel, bias and BN of the i-th branch, K - {0, 1,3, The output of Rep-Conv weighted sum of outputs from all branches, which can be re-parameterized into single Conv during inference:
Rep-Conv(Xe)
[ PAD(W.RT)) x X.
Σ
b
Rep
(10)
Tex:
where W.Rop W.
Vo 7/h Rop bi-mi 7 3, denote re-parameterized Conv kernel and bias Vapte??
of branch i, while (4s, de 7t 3r and are parameters of corresponding BN. And PAD(.) means padding the Conv kernel of all branches to the largest one. Similarly, with weight matrix W, given an input X,, Rep-FC can be expressed as:
Rep-FC(X,) BN(W X,) W.RePp X,
(11)
5

---

Published as a conference paper at ICLR 2026
3.3 STEPWISE TRAINING FRAMEWORK
In this work, we encapsulated composite dataset composed of multiple subjects and tasks from different sources. Each subject contributes to one or more sessions, where each session represents data collected during a distinct time period. The data were split into training, validation, and testing sets at the session level. Depending on the scale of the training set, this dataset was organized into three hierarchical levels: cross-condition (encompassing multiple subjects and tasks), cross-sessiie session (same as in-subject), and single-sens session.
Due to the variability of neural activity, there exist various distributional differences across subjects, tasks and sessions. As a result, even a well-trained cross-condition model may struggle to generalize effectively when transferred directly different scales and enable the model to gradually adapt from the broader cross-condition distribu- tion the more specific session-level distribution, we divide the training pipeline of RAT SNN-CC into four distinct stages based on dataset granularity: 1) cross-condition pretraining, 2) cross-session retraining. 3) single-session finetuning, and 4) optional lightweight finetuning (Fig.
specific session. To better leverage the potential of data
Notice that, benefiting from the re- parameterization, assign an independent BN for each condition during the cross-condition pretraining phase (Fig 3] top). During each epoch, BN are switched dynamically according to the subject identity of the current batch, enabling RAT SNN-CC to better accommo- date the distributional drifts across different conditions. For datasets with only session or generalization experiments, we skip the cross-session retraining step and directly perform finetuning on the target session. In the lightweight finetuning stage, first finetune the model with an activity upper bound (AUB) of each individual LIF neuron after structural re-parameterization to reduce the overall firing rate. Then, we provide an optional iterative prune step (masked the smallest of weights) and following retrain to further reduce model connection size and computational complexity.
Rep-Connectinn
BN
BN
Cape
BN
Cosy
BN
3-T Capy
BN
Cany
BN
FC
PC
BN
RIT
CST
RTT
CST
Cross-session Retraining
MAZE
CST
RIT
CST
Cenner-out
CST
Single sessie- Finetuning
Lightweight Finetuning
AUB
Happ- Cannecliiii
Cross-condition
Prunc (opc3
Figure 3: Training pipeline. (a) Cross-condition: training across different experimental conditions, with a specific BN assigned to each subject. (b) Cross-session:
4 EXPERIMENTS
training on multi-session data from the target condition, with BN fixed to the corresponding sets. (c) Finetune the target session. (d) Further optimize via structural re- parameterization and pruning.
4.1 DATASETS AND EXPERIMENT SETUP
key advantage of Pretained-RAT SNN lies in ability to leverage the discrete spatiotemporal processing capabilities of SNNs with pretrain- ing strategies. This enables effective integration of neural variability across conditions, including across time, subjects, tasks, experimental environments. To this end, we integrate data from mul- tiple datasets (Table D). All these data are derived from electrophysiological recordings of the M1, PMd, and S1 regions in monkeys, spanning 103 sessions across six subjects. Neural activity in these regions has been extensively studied, deep learning tools and benchmarks have already been established as well (Pei al. 202T).
These neural recordings correspond to movements of subjects while performing three different tasks: random target task (RTT), maze target reach (MAZE), and center-out(COO, Specifically, RTT re- quires subjects to continuously and self-pacedly move cursor the screen via finger motion (or joystick in RTT-joystick) to reach randomly appearing targets. MAZE, the subject has to move the cursor around obstacles to reach designated target within each trial. And CO requires the sub- ject to use a joystick to control the cursor from screen center to peripheral targets per trial. Data from MAZE and CO were trial-segmented due to their discrete trial structure.

---

Published as a conference paper at ICLR 2026
Table I:
Datasets used in this work. means being used in cross-condition dataset, while ***
means being used in cross-session dataset. RTT: Random Target Task, CO: Center-Out.
Study
Subject
Brain Area
Channels
Task
Sessions
NHP O'Doherrty el al.
2017
Monkey I, L
MI,
96/192
RTT
37
NLB MAZE (Pet et al. 2021
Monkey Je
MI. PMd
192
MAZE
NLB_RTT (Pei cl al. 2021
Monkey I
96
RTT
Perich datasct [Dver al
Monkey M. J. C.
MI. PMd
192
RTT-joystick. CO
Private dataset L.u al 2025
Monkey C05
CO
Table 2: Performance (R² x 100) on NHP dataset O'Doherrty et al. 2017 *-SS": single-session from scratch; "-CS": cross-session pretrained on same subject: "- "-CC", cross-condition pretrained across sessions, subjects, and tasks. Results without standard deviation (std) are from published works. Best and second-best models are in bold and underlined, respectively. "-P" indicates pruned.
Model
Monkey
Monkey I.
Average
Classical
KalmanFilter Dangi
2011)
21.13
14.20
17.67
ANN2d Yik el 2025
59.28
55.82
57.55
ANN3d Yik el al.
2025
64.27
58.90
61.60
GRU (Wang et al. 2024
71.00
62.33
67.00
ANN
AEGRU Wang
2024)
72.00
67.00
69.50
POYO-SS Azabou
2023
0.83
60.53++2.32
61.43+ 1.25
POYO-CS Azahou et al.
20233
70.99+00.2
69.63-42.32
70.31+0.18
SNN2d Yik et al.
2025
59.30
55.83
$7.57
SNN3d Yık ct 2025)
64.20
62.40
63.30
SNN
bigRSNN-SS [Luu
2024
70.46=0.224
65.40+00.88
67.94+00.37
bigRSNN-CS (Luu ct al.
2024
70.89-#0455
68.70++0.36
69.79-+0.22
RAT SNN-SS
72.22=0.388
66.30
09026+0055
RAT SNN-CS
7..26=0.14
68.63+00.44
71.45+0.15
Ours
RAT SNN-CC
74.06
70.40+0.68
72.23+0.38
RAT SNN-CC-P
73.18.0.117
70.75.1.24
71.96:1..1
We constructed the cross-condition dataset with Monkey I, Monkey L in NHP (O'Doherty et al.] 2017). Monkey Je in NLB (Pei et al. 202T). and Monkey CO5 in private dataset (Luu et al. 2025) for Cross-condition RAT SNN (-CC). Cross- RAT SNN (-CS) was pre- trained across all available sessions from Monkey I, Monkey L, and Monkey CO5 spectively, followed by finetuning on individual sessions. In contrast, single-session RAT SNN (-SS) was trained directly the target session. The test segments were selected from three representative sessions of Monkey I and Monkey L as recommended by Neu- roBench (Yik al.] 2025), and from randomly selected sessions in the remaining datasets.
Table 3: Performance (R² x 100) on Monkey CO5. 44n means generalize result on unseen session.
4.2 TESTING ON SINGLF-SESSION
Model
C05 2022
C05 2025
We first evaluated the performance of RAT
Ours SNNAAN
AEGRU
76.05-10.43
52.39-40.81
SNN at single-session level on Monkey I and Monkey L (Tabl22). Results show that RAT
POYO-SS
77.05+00.80
64.93+22.12
bigRSNN-SS SS
81.72-10.35
62.15:0..33
SNN-SS achieves decoding accuracy compa- rable to SOTA SNN (cross-session pretrained bigRSNN) and ANN (POYO) models, while also maintaining competitive performance un- der other conditions (Table3] 4). Notably, RAT SNN delivers these results with 600K parame- ters, significantly fewer than bigRSNN (1.2M)
bigRSNN-CS
83.56-10.54
66.45-+-0.8
RAT SN
81,58:0.42
66.71
RAT SNN-CS
84.34-1.19
66.77-t1.11
RAT SNN-CC
85.07
6
RAT SNN-CC-P
83.88=0..11
67.17=:1.18
and POYO-SS (1.9M).

---

Published as a conference paper at ICLR 2026
Table Performance (R² 100) on NLB dataset unseen session. "-"; not applicable.
Yik al. 2025
means generalize result on
NLB MAZE Monkey Je
NLB_RTT
Model
Monkey I
GRU Azaaou et al
2023
88,87
59.51
FADDLFADS (Pandarinath
2018
90.62
59.31
ANN
NDT Ye & Pandarinath
2021
89.29
58.95
NDT-Sup (L.u ct al. 2072
87.08
46.21
EIT ([iuu et al. (0022)
87.91
46.91
POYO-SS (Azabou ct al.
202.3
94.70
68.50
POYO-1 (Azabou et al.
2023
94.82
73.78
bighSNN-SS (Lau ct al. 2024
90.38
71.66-F0.54
SNN
bigRSNN-CS [Lu al.
2024
+-0.52
RAT SNN-SS
90.76+00.0
76.34-10.54
RAT SNN-CS
78.55++0.19
Ours
RAT SNN-CC
91.16:10.19
78.70110.66
RAT SNN-CC-P
91.02:+0.15
18.08
(8)
80
(5)
(8)
75
Chas-condilen BAT saglermii loAl SNN
Average Training Epochs
200
70
R x2xx7 100
65
130
60
85
100
50
30
Hels 400 A77
Arp
Sab 301 477
$4..8 sa
ABGRU
NiRSSNN-s
RAY SNN-59
RAT SWN-CS
RAT SNN-CC
POYO-SS
POY04
(-S]]--CC--SS))
Valadriie Cavee (Bak Mtt XTD
Vasdite R Came Set CO
Valiatia R 100
Valduee R 300
Yatcab
Epoch
Uppcch
Epoch
Figure 4: Result of neural temporal drift generalization on NLB RTT. (a) R² com- pared to baselines. (b) Training epochs parison. (c) Learning curves, shaded area de- notes std, aligned to fastest converging seed.
Figure 5: Generalization to unseen conditions. (a) Average training epochs RAT SNN-SS and RAT SNN-CC unseen subject sessions and un- seen subject with unseen task (KTT-joystick) ses- sions. (b-c) Learning curves example sessions.
4.3 GENERALIZING ACROSS NEURAL TEMPORAL DRIFT
Furthermore, we evaluated the performance RAT SNN-CS on seen sessions and its generalization capability to unseen sessions. On seen sessions, cross-session pretraining effectively improved the decoding accuracy RAT SNN, enabling to surpass single-session version and baselines in most sessions (Table B}. Although this is not standard generalization experiment, exploring ways to enhance decoding performance on seen sessions can provide valuable insights for neuroscience
research based Al-AI for neuroscience (Zador et al. 2023 Chen al. 2022).
8

---

Published as a conference paper at ICLR 2026
To evaluate temporal generalization of RAT SNN-CS, the NLB RTT dataset PPe al. 2021
(same animal and task as NHP, Monkey D). Remarkably, although RAT SNN-SS already achieved excellent performance on NLB RTT, even substantially surpassing POYO-1 (pretrained on cross- condition, 11 11.8M), RAT SNN-CS further improved performance (Table 4 The learning curves further demonstrate that RAT SNN-CS converges faster than its single-session counterpart (Fig. + proving that RAT SNN-CS can rapidly generalize to unseen sessions from different time periods, demonstrating excellent capability of addressing neural temporal drift.
4.4 GENERALIZING ACROSS NEURAL VARIABILITY BETWEEN SUBJECTS AND TASKS
After demonstrating that RAT SNN possesses strong capabilities in integrating cross-session data and generalizing to neural temporal drift, we further validate its ability to integrate across conditions. After cross-condition pretraining, RAT SNN-CC achieved superior performance unseen sessions of seen subject and seen conditions, highlighting the effectiveness of cross-condition pretraining and its capacity to integrate neural variability (Table 3 4
both seen and
Notably, compared to POYO, RAT SNN-CC consistently outperformed its single-session version across multiple datasets, suggesting more substantial performance gains and superior integration of heterogeneous data.
Furthermore, we evaluated the generalization capability of RAT SNN-CC to unseen conditions (without cross-session re-training). Result show that when generalizing an entirely unseen subject performing an unseen task (RTT-joystick), the RAT SNN-CC achieves faster convergence and superior performance compared the version trained from scratch, highlighting its strong cross- condition generalization (Fig. 5
4.5 STRUCTURAL ABLATION STUDY
Using the small-scale dataset (Monkey C05) as an example, we conduct ablation experiments on the RAT SNN-SS to evaluate how CST re-parameterized attention block affects decoding accuracy.
Comparing our CST attention block (MLP size= 512) with the classical spike-driven transformer block (SDT SNN) and RAT SNN-192 (MLP size= 192, the same output dimension as the SDT block and equal to the input size), our method yields higher decoding accuracy, and enlarging its output dimension via the "surrogate shortcut" further improves performance (Table 5).
Valdake 82 Cave (CB5 on
Training Eppoche
(00
Validaiie R² x 1.000
F2-
500
Aveenge
210
20
SNN
Epoch
Table 5: Ablation study results RAT SNN-SS on Monkey CO5.
Viltedea Crouge
(d)
100
Model SDT SNN
CO5 2022
C05 C05_2025
50
80.09-10.23
65.21110.57
Valtbanteen R' 1:00
Validriee R² x 100
RAT SNN-192
80.49:00.8
66.6110.79
RepPC SNN-HI
81.09--0.40
66.26:10.5
RepFC SNN-H3
79.47+0.96
65.74++100
RAT SNN-SS
81.5810.42
66.7111048
1
RAI ENN
10
RAT SNN
wid geggenstteiiiiiie
Table 6:
Ablation study results on re-
Epoch
400
100
Epoch
400
parameterization Monkey C05.
Model
R' x 100
Epoch 545::84
Figure 6: Performance comparison between RAT SNN and w/o re-parameterization ver- sion. (a) Average training epochs for both mod- els on Monkey C05. (b-d) 122 learning curves.
w/o re re-param
67.58=2.50
RAT SNN-SS
81.58 0.42
179-167
Also, compared with fully connected recurrent SNN baselines (RepFC SNN), method out- performs both hengle-hidden-lyerr (RepFC SNN-HII and 3-hidden-layer (-H3) variants (Table (5) showing the effectiveness of CST attention block in capturing spatial dependencies in CST.

---

Published as a conference paper at ICLR 2026
Furthermore, investigate the impact of the ce-parameterizationn method on RAT SNN (Table 6 This strategy both accelerates learning and improves decoding accuracy: relative to the standard RAT SNN, the w/o parameterization variant shows 17.16% drop in R and requires roughly three times more training epochs to converge (Fig. 6).
4.6 REDUCING COMPUTATIONAL CONSCMPTION WITH HIGH PERFORMANCE
To evaluate computational cost of RAT SNN,
we used NeuroBench (ik et al. 2025) to count
Table 7: Comparison of synaptic operations across different Models.
ACS and MACs on NHP dataset (O D'Dheeyyy
et al. 2017) as energy metrics.
With the pe-aammettrrzztiinn method, RAT SNN involves only synaptic AC operations dur- ing inference, aligning with the spike-driven nature of energy-efficient SNNs (Table (7). Furthermore, with simple pruning strategy, RAT SNN-CC-P substantially reduces compu- tational cost while maintaining comparable per- formance (Wilcoxon test, p-0.3828, Table 2). Compared to basic SNN model (SNN2d), al- though incurring higher ACs, RAT SNN-CC-P still remains energy efficient given the substan- tial performance gap. Compared to bigRSNN, RAT SNN-CC-P has a substantially lower com- putational cost and smaller parameter size (150K 1.2M), making it more suitable for deploy- ment in fully implantable iBMIs.
Model
Synaptic Operations
Effective MACs Effective ACs
ANN2d
4970
0
ANN3d
11507
0
AEGRU
25095-1-9
0
POYO
1730507#3
810339
SNN2d
0
414
SNN3d
32256
5831
bigRSNN
0
42003-+6281
RAT SNN-CC
0
65307=841
RAT SNN-CC-P
0
21020#:5515
Furthermore, RAT SNN significantly more efficient than other operation-efficient ANN-based decoders (AE- GRU and basic ANN models), especially considering that MAC operation consumes approximately 31 times the energy an AC operation (Yin et al., (02T). Therefore, RAT SNN demonstrates outstanding performance-to- energy ratio, making a highly promising candidate for future fully iBMI applications.
RAT-SNN-DC.
70-
ADGRU
68
BEJSSNN
POYO
6 6AR830
5 CONCLUSION
58
+ANK2S
In this work, propose RAT SNN, CST decoder that integrates re-parameterized self-attention mecha- nisms. RAT SNN achieves superior decoding perfor- mance through cross-condition pretraining and possesses strong generalization capabilities, enabling rapid finetun- ing on unseen data to overcome neural temporal drift in individual subjects as well as neural variability across other cross-condition scenarios. Meanwhile, RAT SNN features a lightweight architecture and strictly adheres to the spike-drive driven nature of low-power SNNs, realizing a prototype SNN foundation model with high performance, strong generalization, and low energy consumption (Fig
Ing(Compurations Cost)
Figure Comparison summary between RAT SNN and baselines. Computational Cost = Effective ACs + Effective MACs 31 (Yin al. 1021) Bubble size represents the model size.
ACKNOWLEDGMENTS
This work is supported by: the STI 20jo-Major Projects (No. (N0.2022ZD00088644) the Key R&D Pro- gram for Zhejiang (No. 2021C03003, 2022R52033, 2023C03081, 2023C03026, 2024C03002), and the Lingang Laboratory (Grant No-7709-188--08..
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Mehdi Azabou, Vinam Arora, Venkataramana Ganesh, Ximeng Mao, Santosh B. Nachimuthu, Michael Jacob Mendelson, Blake Aaron Richards, Matthew G. Perich, Guillaume Lajoie, and Eva L. Dyer. A unified, scalable framework for neural population decoding. In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
Malyaban Bal and Abhronil Sengupta. Spikingbert:
Distilling bert train spiking language models using implicit differentiation. Proceedings of the AAAI Conference on Artificial Intelligence, 38 (10):10998-1 1006, March 2024. ISSN 2374-3468.
Nicholas S. Card, Maitreyee Wairagkar, Carrina lacobacci, Xianda Hou, Tyler Singer-Clark, Fran- cis R. Willett, Erin M. Kunz, Chaofei Fan, Maryam Vahdati Nia, Darrel R. Deo, Aparn Srini- vasan, Eun Young Choi, Matthew F. Glasser, Leigh R. Hochberg, Jaimie M. Henderson, Kiarash Shahlaie, Sergey D. Stavisky, and David M. Brandman. An accurate and rapidly calibrating speech neuroprosthesis. New England Journal of Medicine, August 2024.
Guozhang Chen, Franz Scherr, and Wolfgang Maass. A data-based large-scale model for primary visual cortex enables brain-like robust and versatile visual processing. Science Advances, 8(44): eabq7592, November 2022.
Mark M. Churchland, John P. Cunningham, Matthew T. Kaufman, Justin D. Foster, Paul Nuyu- jukian, Stephen I. Ryu, and Krishna V. Shenoy. Neural population dynamics during reaching. Nature, 487(7405):51-56, July 2012. ISSN 1476-4687.
Siddharth Dangi, Suraj Gowda, Rodolphe Héliot, and Jose M. Carmena. Adaptive kalman filtering for closed-loop brain-machine interface systems. In 2011 5th International IEEF/EMBS Confer- ence on Neural Engineering, pp. 609-612, April 2011.
Alan D. Degenhart, William E. Bishop, Emily R. Oby, Elizaheth C. Tyler-Kabara, Steven M. Chase, Aaron P. Batista, and Byron M. Yu. Stabilization of brain-computer interface via the alignment of lowddmeeniional spaces of neural activity. Nature Biomedical Engineering, 4(7):672-685, July 2020. ISSN 2157-846X.
Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg: Making vgg-style convnets great again. In 2021 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 13728-13737, June 2021.
Ilan Dinstein, David J. Heeger, and Marlene Behrmann. Neural variability: Friend or foe? Trends in Cognitive Sciences, 19(6):222-333 222222222,,22 June 2015. ISSN 1364-6613.
Laura N. Driscoll, Lea Duncker, and Christopher D. Harvey. Representational drift: Emerging theo- ries for continual learning and experimental future directions. Current Opinion in Neurobiology, 76: 76:10,,,,, October 2022. ISSN 0959-4388.
Eva L. Dyer, Mohammad Gheshlaghi Azar, Matthew G. Perich, Hugo L. Fernandes, Stephanie Naufel, Lee E. Miller, and Konrad P. Körding. A cryptography-base approach for movement decoding. Nature Biomedical Engineering, 1(12):967-97) December 2017. ISSN 2157-846X.
Wei Fang. Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. In Proceedings of the 35th International Conference Neural Information Processing Systems, NIPS '21, pp. 21056-21069, Red Hook, NY, USA, December 2021. Curran Associates Inc. ISBN 198---713--4539
Ali Farshchian, Juan A. Gallego, Joseph P. Cohen, Yoshua Bengio, Lee Miller, and Sara Solla. Adversarial domain adaptation for stable brain-machine interfaces, January 2019.
Sharlene N. Flesher, John E. Downey, Jeffrey M. Weiss, Christopher L. Hughes, Angelica J. Herrera, Elizabeth C. Tyler-Kabare Michael L. Boninger, Jennifer L. Collinger, and Robert A. Gaunt. A brain- computr interface that evokes tactile sensations improves robotic arm control. Science, 372(6544):311-886 May 2021.
11

---

Published as a conference paper at ICLR 2026
Leig R. Hochberg, Daniel Bacher, Beata Jarosiewicz, Nicolas Y. Masse, John D. Simeral, Joern Vogel, Sami Haddadin, Jie Liu, Sydney S. Cash, Patrick van der Smagt, and John P. Donoghue. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 485 (7398):372-375, May 2012. ISSN 1476-4687.
Brianna M. Karpowicz, Yahia H. Ali, Lahiru N. Wimalasena, Andrew R. Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma. Daniel B. Rubin, Ziv M. Williams, Sydney S. Cash, Leigh R. Hochberg, Lee E. Miller, and Chethan Pandarinath. Stabilizing brain-computer terfaces through alignment of latent dynamics. Nature Communcaations, 16(1))6662, May 2025. ISSN 2041-1723.
Trung Le and Eli Shlizerman. Stndt: Modeling neural population activity with spatiotemporal trans- formers. Advances in Neural Information Processing Systems, 35:17926 17939, December 2022.
Ling Liang, Xing Hu, Lei Deng, Yujie Wu, Guoqi Li, Yufei Ding, Peng Li, and Yuan Xie. Exploring adversarial attack in spiking neural networks with spike-compatible gradient. IEEE Transactions on Neural Networks and Learning Systems, 34(5):2569-258,, May 2023. ISSN 2162-2388.
Jiawei Liao, Lars Widmer, Xiaying Wang, Alfio Di Mauro, Samuel R. Dason-Tomaszewski Cyn- thia A. Chestek, Luca Benini, and Tackwang Jang. An energy-efficien spiking neural network for finger velocity decoding for implantable brain-machine interface. In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 134-137, June 2022.
Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. A 128 times 128 120 db 15 jss latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid-State Circuits, 43(2):566- 576, February 2008. ISSN 1558-173X.
Ran Liu, Mehdi Azabou, Max Dabagia, Jingyun Xiao, and Eva Dyer. Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformers. Advances in Neural Information Processing Systems, 35:2377-2391, December 2022.
Tengjun Liu, Julia Gygax, Julian Rossbroich, Yansong Chua, Shaomin Zhang, and Friedemann Zenke. Decoding finger velocity from cortical spike trains with recurrent spiking neural networks, In 2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1-5, October 2024.
Tengjun Liu, Yansong Chua, Yuxiao Ning. Pengfu Liu, Yiwei Zhang, Tuoru Li, Guihua Wan, Zi- jun Wan, Weidong Chen, and Shaomin Zhang. motorsma: spiking recurrent neural network inspired by brain topology for the effective and efficient decoding of cortical spike trains. Biomed- ical Signal Processing and Control, 99: 10644,, January 2025. ISSN 1746-8094.
Zhengwu Liu, Jianshi Tang, Bin Gao, Peng Yao, Xinyi Li, Dingkun Liu, Ying Zhou, He Qian, Bo Hong, and Huaqiang Wu. Neural signal analysis with memristor arrays towards high- efficiency brain-m interfaces. Nature Communications, 11(1):4234, August 2020. ISSN 2041-1723.
Wolfgang Maass. Networks of spiking neurons: The third generation of neural network models. Neural Networks, 1099):155-1161, December 1997. ISSN 0893-6080.
Georgios Mentzelopoulos, loannis Asmanis, Konrad Kording, Eva L. Dyer, Kostas Danilidiis, and Flavia Vitale. A scalable, causal, and energy efficient framework for neural decoding with spik- ing neural networks. In The Thirty-Ninth Annual Conference on Neural Information Processing Systems, October 2025.
David A. Moses, Sean L. Metzger, Jessie R. Liu, Gopala K. Anumanchipalli, Joseph G. Makin, Pengfei Sun, Josh Chartier, Maximilian E. Dougherty, Patricia M. Liu, Gary M. Abrams, Ade- lyn Tu-Chan, Karunesh Ganguly, and Edward F. Chang. Neuroprosthesis for decoding speech in a paralyzed person with anarthria. New England Journal of Medicine, 385(3):217-227, July 2021. ISSN 0028-4793.
Joseph E. O'Doherty, Mariana M. B. Cardoso, Joseph G. Makin, and Philip N. Sabes. Nonhuman primate reaching with multichannel sensorimotor cortex electrophysiology, May 2017.
12

---

Published as a conference paper at ICLR 2026
Chethan Pandarinath, Paul Nuyujukian, Christine H. Blabe, Brittany L Sorice, Jad Saab, Francis R Willett, Leigh Hochberg, Krishna V Shenoy, and Jaimie Henderson. High performance communication by people with paralysis using intracortical brain-computer interface. eLife, 6: e18554, February 2017. ISSN 2050-084X.
Chethan Pandarinath, Daniel J. O'Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D. Stavisky, Jonathan C. Kao, Eric M. Trautmann, Matthew T. Kaufman, Stephen I. Ryu, Leigh R. Hochberg. Jaimie M. Henderson, Krishna V. Shenoy, L. F. Abbott, and David Sussillo. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature Methods, 15(10):805-815, October 2018. ISSN 1548-7105.
Felix Joel Ye, David Zoltowski, Angi Wu, Raeed Chowdhury, Hansem Sohn, Joseph O' O'Doherty, Krishna V. Shenoy, Matthew Kaufman, Mark Churchland, Mehrdad Jazayeri, Lee Miller, Jonathan Pillow, Il Memming Park, Eva Dyer, and Chethan Pandarinath. Neural latents bench- mark 21: Evaluating latent variable models of neural population activity. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 1, December 2021.
Nicolas Perez-Nieves, Vincent C. H. Leung, Pier Luigi Dragotti, and Dan F. M. Goodman. Neural heterogeneity promotes robust learning. Nature Communications, 12(1):5791, October 2021. ISSN 2041-1723.
Matthew G. Perich, Juan A. Gallego, and Lee E. Miller. A neural population mechanism for rapid learning. Neuron, (114)))444-77.20 November 2018. ISSN 0896-6273.
Adrien B Rapeaux and Timothy G Constandinou. Implantable brain machine interfaces: First-in- human studies, technology challenges and trends. Current Opinion Biotechnology. 72:102-114 December 2021. ISSN 0958-1669.
Julian Rossbroich, Julia Gygax, and Friedemann Zenke. Fluctuation-drivee initialization for spiking neural network training. Neuromorphic Computing and Engineering, 2(4):044016, December 2022. ISSN 2634-4386.
Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, 61 617(7960):3003368 May 2023. ISSN 1476-4687.
Elijah A. Taeckens and Sahil Shah. A spiking neural network with continuous local learning for robust online brain machine interface. Journal of Neural Engineering. 20(6):066042, January 2024. ISSN 1741-2552.
Hanchen Wang, Tianfan Fu, Yuanqi Du Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chan- dak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, Anima Anandkumar, Karianne Bergen, Carla P. Gomes, Shirley Ho, Pushmeet Kohli, Joan Lasenby, Jure Leskovec, Tie- Yan Liu, Arjun Manrai, Debora Marks, Bharath Ramsundar, Le Song, Jimeng Sun, Jian Tang, Petar Veličković, Max Welling, Linfeng Zhang, Connor W. Coley, Yoshua Bengio, and Marinka Zitnik. Scientific discovery the age of artificial intelligence. Nature, 620(7972):47-60, August 2023. ISSN 1476-4687.
Yuanxi Wang, Zuowen Wang, and Shih-Chii Liu. Leveraging recurrent neural networks for predict- ing motor movements from primate motor cortex neural recordings. In 2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1-5, October 2024.
Francis R. Willett, Donald T. Avansino, Leigh Hochberg, Jaimie M. Henderson, and Krishna V. Shenoy. high-peeffrmance brain-to-text communication via handwriting. Nature, 593(7858); 249-254 May 2021. ISSN 1476-4687.
Yujie Wu, Bizhao Shi, Zhong Zheng, Hanle Zheng, Fangwen Yu, Xue Liu, Guojie Luo, and Lei Deng. Adaptive spatiotemporal neural networks through complementary hybridization. Nature Communications. 15(1):3355, August 2024. ISSN 2041-1723.
Man Yao, JiaKui Hu, Tianxiang Hu, Yifan Xu, Zhaokun Zhou, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer v2: Meta spiking neural network architecture inspiring the design of next next-generation neuromorphic chips. In The Twelfth International Conference on Learning Representations, October 2023a.
13

---

Published as a conference paper at ICLR 2026
Man Yao, JiaKui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer. In Thirty-Seer -erre--eeenth Conference on Neural Information Processing Systems, November 2023b.
Joel Ye and Chethan Pandarinath. Representation learning for neural population activity with neural data transformers. Neurons, Behavior, Data analysis, and Theory, 5(3):1118, August 2021.
Joel Ye, Jennifer Collinger, Leila Wehbe, and Robert Gaunt. Neural data transformer 2: Multi- context pretraining for neural spiking activity. Advances in Neural Information Processing Sys- tems, 36:80352-80074 December 2023.
Joel Ye, Fabio Rizzoglio, Adam Smoulder, Hongwei Mao, Xuan Ma. Patrick Marino, Raeed Chowd- hury, Dalton Moore, Gary Blumenthal, William Hockeimer, Nicolas G. Kunigk, J. Patrick Mayo, Aaron Batista, Steven Chase, Adam Rouse, Michael L. Boninger, Charles Greenspon, Andrew B. Schwartz, Nicholas G. Hatsopoulos, Lee E. Miller, Kristofer E. Bouchard, Jennifer L. Collinger, Leila Wehbe, and Robert Gaunt. A generalist intracortical motor decoder, February 2025.
Jason Yik, Korneel Van den Berghe. Douwe den Blanken, Younes Bouhadjar, Maxime Fabre. Paul Hueber, Weijie Ke, Mina Khoei, Denis Kleyko, Noah Pacik-Nelson, Alessandro Pierro, Philipp Stratmann, Pao-Sheng Vincent Sun, Guangzhi Tang. Shenqi Wang. Biyan Zhou, Soikat Hasan Ahmed, George Vathakkattil Joseph, Benedetto Leto, Aurora Micheli, Anurag Kumar Mishra, Gregor Lenz, Tao Sun, Zergham Ahmed, Mahmoud Akl, Brian Anderson, Andreas G. Andreou, Chiara Bartolozzi, Arindam Basu, Petrut Bogdan, Sander Bohte, Sonia Buckley, Gert Cauwen- berghs, Elisabetta Chicca, Federico Corradi, Guido de Croon, Andreea Danielescu, Anurag Daram, Mike Davies, Yigit Demirag, Jason Eshraghian, Tobias Fischer, Jeremy Forest, Vitto- rio Fra, Steve Furber, Michael Furlong, William Gilpin, Aditya Gilra, Hector A. Gonzalez, Giacomo Indiveri, Siddharth Joshi, Vedant Karia, Lyes Khacef, James C. Knight, Laura Kriener, Rajkumar Kubendran, Dhireesha Kudithipudi. Shih-Chii Liu, Yao-Hong Liu, Haoyuan Ma. Rajit Manohar, Josep Maria Margarit-Tauuke Christian Mayr, Konstantinos Michmizos, Dylan R. Muir, Emre Neftei, Thomas Nowotny, Fabrizio Ottati, Ayea Ozcelikkale, Priyadarshini Panda, Jongkil Park, Melika Payvand, Christian Pehle, Mihai A. Petrovici, Christoph Posch, Alpha Renner, Yu- lia Sandamirskaya, Clemens S. Schaefer, André van Schaik, Johannes Schemmel, Samuel Schmidgall, Catherine Schuman, Jae-sun Seo, Sadique Sheik, Sumit Bam Shrestha, Manolis Sifalakis, Amos Sironi, Kenneth Stewart, Matthew Stewart, Terrence C. Stewart, Jonathan Tim- check, Nergis Tömen, Gianvito Urgese, Marian Verhelst, Craig M. Vineyard, Bernhard Vogginger, Amirreza Yousefzadch, Fatima Tuz Zohora, Charlotte Frenkel, and Vijay Janapa Reddi. The neu- robench framework for benchmarking neuromorphic computing algorithms and systems. Nature Communications, 16(1) (6(1):1545, February 2025. ISSN 2041-1723.
Bojian Yin, Federico Corradi, and Sander M. Bohté. Accurate and efficient time-domain classi- fication with adaptive spiking recurrent neural networks. Nature Machine Intelligence, 3(10): 905-913, October 2021. ISSN 2522-5839.
Rafael Yuste, Sara Goering, Blaise Agüera y Arcas, Guoqiang Bi, Jose M. Carmena, Adrian Carter, Joseph J. Fins, Phoebe Friesen, Jack Gallant, Jane E. Huggins, Judy Illes, Philipp Kellmeyer, Eran Klein, Adam Marblestone, Christine Mitchell, Erik Parens, Michelle Pham, Alan Rubel, Norihiro Sadato, Laura Specker Sullivan, Mina Teicher, David Wasserman, Anna Wexler, Meredith Whit- taker, and Jonathan Wolpaw. Four ethical priorities for neurotechnologies and ai. Nature, 551 17679):159-163, November 2017. ISSN 1476-4687.
Anthony Zador, Sean Escola, Blake Richards, Bence Ölveczky, Yoshua Bengio, Kwabena Boa- hen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, James DiCarlo, Surya Ganguli, Jeff Hawkins, Konrad Körding, Alexei Koulakov, Yann LeCun, Timothy Lillicrap, Adam Marblestone, Bruno Olshausen, Alexandre Pouget, Cristina Savin, Terrence Sejnowski, Eero Simoncelli, Sara Solla, David Sussillo, Andreas S. Tolias, and Doris Tsao. Catalyzing next- generation artificial intelligence through neuroai. Nature Communicationss 14(1):1597, March 2023. ISSN 2041-1723.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural Computation, 1514)1511-1144, June 2018. ISSN 0899-76677.
14

---

Published as a conference paper at ICLR 2026
Qiaosheng Zhang, Sile Hu, Robert Talay, Zhengdong Xiao, David Rosenberg, Yaling Liu, Guanghao Sun, Anna Li, Bassir Caravan, Amrita Singh, Jonathan D. Gould, Zhe S. Chen, and Jing Wang. A prototype closed-loop machie interface for the study and treatment of pain. Nature Biomedical Engineering, 7(4):533-545, April 2023. ISSN 2157-846X.
Biyan Zhou, Pao-Sheng Vincent Sun, Jason Yik, Charlotte Frenkel, Vijay Janapa Reddi, and Arindam Basu. Grand challenge neural decoding for motor control of non-human primates. In 2024 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1-5, October 2024.
Chenlin Zhou, Liutao Yu, Zhaokun Zhou, Zhengyu Ma, Han Zhang, Huihui Zhou, and Yonghong Tian. Spikingformer: Spike-driven residual learning for transformer-basse spiking neural net- work. arXiv preprint arXiv:2300. 1954, 2023.
Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. arXiv preprint arXiv:2209. 15425 2022.
Rui-Jie Zhu, Ziqing Wang, Leilani Gilpin, and Jason K. Eshraghian. Autonomous driving with spik- ing neural networks. Advances in Neural Information Processing Systems, 37:136782-136801 December 2024.
APPENDIX
A DATASETS
We conducted experiments five datasets, involving various subjects and tasks, as summarized in Table I Based on these datasets, we established cross-condition dataset incorporating data from multiple conditions, including different subjects, tasks, and sessions. Notably, the subject Monkey in NHP dataset (O'Doherty et al., 2017) was included in NLB dataset (Pei et al. (021). with different sessions. Private dataset Liu et al. (2025) includes 10 sessions of Monkey COS recorded in 2022 (C05 2022), and 2 sessions recorded in 2025 (C05_2025). The RTT task in Perich dataset (Perich et al. (018)) is different from other dataset, with subjects could not touch the screen to move cursor but with joystick.
We evaluated RAT SNN with the standardized brain-inspired benchmarking framework, Neu- roBench Yik al. 2025). All samples from the datasets were uniformly downsampled at 4 intervals, with z-score normalization applied per-session basis. The model was trained and vali- dated using overlapping 2-second samples, while testing employed continuous long sequences to simulate real-time processing. For each session, 10% of the data was reserved for validation and 25% for testing following NeuroBench recommendations (Fig. { (Yik al., 2025). except for the Perich dataset. For Perich dataset, we reserved 10% of the data for validation and 10% for testing as recommended by POYO (Azabou et al. 2023)
A.1 DATASETS FOR MULTIPLE TRAINING STAGES
We constructed three hierarchical datasets corresponding to four training stages of RAT SNN
(Fig. 99. 1) The cross-condition dataset is used in the cross-condition pretraining stage of cross-
S1
$2
S3
Session
Trnal
Trail 2
Tral 3
10% for
Trail
Figure 8: Dataset segmentation schematic. Each session consists of multiple trials. We divide the data into training, validation, and test sets based on trials. Using NHP dataset (O'Doherty et al. 2017) as segmentation example.
15

---

Published as a conference paper at ICLR 2026
CST
Soodeyy L
Morkay I
RIT
Maricey
Necka
Nonicy 99143
505-00, XIn STT
Birjeyeck
CST
RIT
CST
Morky SM
Moekey
Monicyy
565400 53325
Modian C: 52140
Mantay 500
Mociay
Canteazi
Monkey Cus
MAZE
Makka NLB
Cros-cradities Dataset
Cros-sessio Dutasets
Siagle-session Dutasets
Figure 9: Datasets for Multiple Training Stages. We constructed three datasets corresponding to the cross-condition, cross-session, and single-session levels. Left: The cross-condition dataset includes data from different subjects and tasks. Middle: The cross-session dataset includes multiple sessions from the same subject. Right: The single-session dataset includes a single session from single subject, specific session indices are annotated on right. Different subject monkeys are represented by different colors, and overlapping boxes with arrows indicate multiple sessions from the same subject.
condition RAT SNN (RAT SNN-CC), and includes data from different subjects and tasks. 2) The cross-session dataset is used in both the cross-session pretraining stage of cross-session RAT SNN (RAT SNN-CS) and the cross-session retraining stage of RAT SNN-CC. It consists of multiple sessions from the same subject. 3) The single-session dataset is used in the training stage of the single-session RAT SNN (RAT SNN-SS), as well as in the single-session finetuning and lightweight finetuning stages of both the RAT SNN-CC and RAT SNN-CS. It contains data from single session of a single subject.
The cross-condition dataset comprises data from different subjects and tasks, including Monkey I and Monkey I. from NHP dataset OO'Doherty et al. 2017), Monkey Je from NLB MAZE (Pei et al.) 2021], and Monkey C05_ 20_2222 from our private dataset (Liu al. (2025). Cross-condition pretrain- ing was performed using this combined dataset to train the RAT SNN-CC, which was subsequently finetuned on individual downstream subjects and tasks.
The cross-session datasets includes multiple sessions from the same subject. In this study, we used all available sessions from Monkey I, Monkey L (NHP dataset), and Monkey C05_2022 (private dataset) to train and evaluate the RAT SNN-CS.
The single-session dataset contains only one session from a single subject. For NHP dataset, we selected three representative sessions from Monkey I and Monkey L, as recommended by NeuroBench 222 01, indy 20166660.01, indy 20170131 02, loco 20170210.09 _02, loco_201110 20370301 .05 Yik et al. (2025)).
For NLB MAZE and NLB RTT (Pei al. 202 }, we used the only available sessions from Monkey Je ("MAZE 000128") and Monkey ("RTT 000129"", respectively.
For Perich dataset (Perich et al. 2018) and the private dataset
(Liu al. 2025
we randomly selected early sessions from Monkey CO5 (05_2022012009411
c05 005_20200110 14093926, c05 2022011413225: and c05 2025022114411,,,
c05 20250218095 128),
Monkey
M (m.201401 www.rrrnnnmmaaaee reaching.
m_20140115 1al.40011..aanomm..mm caadom_target_rrachhe Monkey
m.20144 40203 m02001 wetrseeeeeeeruteeeeeee centchn.uu.rracchingg.
cotterr.ut.reaching! t_20130820 1a000200rannomtttee handommtargettrrachnnge
m wtt.2014211ccmmee
(t.201308) center.out_reaching
and Monkey C
(c 20161011 I center out reaching,
с 20161013 center out reaching), prioritizing temporally
consecutive recordings to ensure result stability.
16

---

Published as a conference paper at ICLR 2026
Cross-session Petraining Dataset
Cross-session Petraining Rep-Connection
RTT
CST
BN
Monkey 801-37
P-t Cony
Single-session Finetuning Data
Finctuncing
Single-session Finetuning Rep-Connction
Monkey
BN
S01
3"1 Cony
BV
RTT
CST
Monkey
FC
BN
S02
Monkey
Lightweight Finctuning
S03
Rup AUB
Rep- Connectionn
Reped-Conv
Unseen Session Generalizarion
RTT
CST
Frene (opl.)
Reped-FCC
Monkey
NLB
Figure 10: Cross-session training pipeline. Using Monkey as an example: RAT SNN was first pretrained on the cross-session dataset (from NHP dataset), then separately finetuned three sions from NHP dataset to evaluate decoding performance on seen sessions, and the extra session from RTB to assess generalization capability on unseen sessions.
B TRAINING
B.
PRETRAINING FOR NEURAL TEMPORAL DRIFT GENERALIZATION
Since a subject's dataset typically contains multiple sessions collected on different dates, with some sessions separated by intervals even years, neural activity continuously changes over time-Neural Temporal Drift (Driscoll et al. 2022). Consequently, cross-session neural variability represents one of the primary challenges in iBMI decoding.
In this work, We enhance RAT SNN's neural temporal drift generalization performance through cross-session pretraining. And we further divide the neural temporal drift generalization perfor- mance evaluation into components: generalization capabilities sessions. The generalization capability on seen sessions indicates that the model can cover neu- ral temporal drift in observed data during training and extract underlying features during dynamic changes. The generalization capability on unseen sessions indicates that the model can rapidly adapt feature changes in unseen sessions based on underlying features extracted from seen sessions, thereby simulating long-term deployment scenarios in real iBMI applications.
sessions and unseen
Specifically, we pretrained RAT SNN on the cross-session datasets, which includes all available ses- sions from Monkey I, Monkey L, and Monkey CO5 respectively. Pretraining and finetuning pipeline shown in Fig. T The pretraining stage was conducted on the cross- cross-session datasets respectively, followed by finetuning on individual sessions of the same subject to evaluate decoding performance on seen sessions. Additionally, we finetuned the model on the NLB RTT and CO5.2025 to assess its generalization capability across temporal intervals unseen sessions. After cross-session pre- training, the finetuning process comprises two distinct phases: 1) Single-session finetuning: the model architecture remains unchanged teristics through conventional finetuning. 2) Lightweight finetuning with pruning (Fig. [0] Right, Section B.3
directly adapt RAT SNN to target session charac-
17

---

Published as a conference paper at ICLR 2026
Cross-conditions Dataset
RTT
CST
Monkay Monkey L
Cross-conditions Pretraining
Cop-Conncction
MAZEE
Monkey Je
Come
3-1 Cons
FC
BN
Center out
Monkey CV5
BN Fixed
Cross-session Retraining Dataset
Cross-session Retraining
Mookey CO5
Rep-Cennection
BN
I Cuay
BN
Menkeey
Monkey
Figure 11: Cross-condition training pipeline on subject with multiple session. RAT SNN was first pretrained on the cross-condition dataset, which includes data from different subjects and tasks from Monkey I, Monkey L, Je, and C05, For Monkey I, L and CO5, which have multiple sessions, RAT SNN-CC was then retrained on all training sessions of the same subject, and then finetuned on individual sessions with pipeline shown in the Fig TO
Crus--onddiions Dataset
Crosi-veditiens Pretraining
RIT
Morley
Corr
BS%
Cotte
MAZE
CST
Markes Je
BN Fixed
Single sesdion Finetuning
Rap Tep-CCannccike
Coneraui
CST
Mackey
Sieglescsion Data-Seca Condition
Masoay la
Lightweight Finetuning
Bap AUB
Repod-Caav
Prane
Singl-sesion Data-UUnsen Goneralication
Unseen subject Secn Task
Morley
Morkez
Mesley
Unseen subject Unseen Task
KTTjeverck
CST
Figure 12: Cross-condition training pipeline on subject with single session. RAT SNN was first pretrained on the cross-condition dataset. For seen Monkey Je and unseen Monkey M, T. C, RAT SNN-CC directly finetuned on the single-session data.
18

---

Published as a conference paper at ICLR 2026
B.2 CROSS-CONDITION PRETRAINING FOR NEURAL VARIABILITY GENERALIZATION ACROSS SUBJECTS AND TASKS
Foundation models for iBMI application need to possess two important characteristics: the ability to leverage cross-condition data to improve model decoding accuracy, and the ability to rapidly adapt to different subjects and tasks. Therefore, decoder models must effectively handle neural variability arising from different conditions including different subjects and tasks. When generalize to specific individuals, due to differences in experimental data richness across subjects, some subjects contain abundant data, allowing models to perform cross-session retraining on sufficient multiple session data to improve decoding accuracy. Other subjects only contain single-session data, so models must also possess the capability to directly finetune based on a limited amount of data to adapt specific individual characteristics.
Based on above, we designed two different training strategies for RAT SNN-CC: 1) For subjects containing multi-session data, RAT SNN-CC undergoes pretraining on the cross-condition dataset, followed by retraining on all training sessions, and then finetuning on the target session (Fig II], representing the highest decoding accuracy after fully utilizing all available data; 2) For subjects containing only single-session data or unseen condition generalization task, RAT SNN-CC is di- rectly finetuned on the target session (Fig 12). encompasses both seen condition and unseen con- dition scenarios (including unseen subject with seen task and unseen subject with unseen task). The former represents ability to leverage large amounts of heterogeneous neural activity to improve decoding accuracy with limited samples, while the latter represents ability rapidly adapt and im- prove decoding accuracy when facing unseen conditions, which is closer to real iBMI generalization scenarios. worth noting that for unseen conditions, we first average the parameters of all seen condition BN layers to initialize the BN parameters for the unseen condition, ensuring that the model can rapidly adapt to the features of unseen conditions.
B.3 LIGHTWEIGHT FINETUNING AND PRUNING
To reduce the energy consumption of RAT SNN and further stabilize the parameter space after re-parameterization, we merge the synaptic operation layers using the re-parameterization method after single-session finetuning, and then perform additional finetuning with activity upper bound (AUB) regularization term. This enables the model to further adjust within the local parameter space, removing unnecessary active connections introduced by pretraining, and thus better adapting to the characteristics of the target session (Fig. 10 Right).
Additionally, we provide an optional pruning step further reduce the computational cost of RAT SNN. Specifically, we setting the lowest prune percentage of weights to zero, and then perform masked retraining after pruning to recover performance. The detailed pruning procedure is shown in Algorithm|] [ In this way. we can effectively reduce the computational cost of the model without significantly affecting its performance.
B.4 PROPAGATE STYLE
Our method is built upon Stork (Zenke & Ganguli 2018), which adopts a layer-connection prop- agation sequence inspired by the simulation of biological neural systems. Specifically, during the forward pass, Stork first updates the dynamics of all neural layers, followed by the update of all connections. This design implies that input at time step only begins to influence the first neural layer at time t 1, and the second layer at + 2 (Fig. 13 Left).
While such a strategy may be advantageous in other domains with the biological plausibility, it in- troduces increasing latency between input and output in time series predict tasks which accumulates with model depth. To mitigate this issue, we employ sequential spike propagation scheme (Fig. [3 Right), where the dynamics of each neural layer and connections each time step, enabling the input at time step to influence the output within the same time step and allowing RAT SNN to decrease the latency between input and output.
updated sequentially within
19

---

Published as a conference paper at ICLR 2026
Algorithm Iterative pruning
Input: RAT SNN parameter 8, R² of validation before pruned Ri ,start prune percentage p. per- formance decrease tolerance max prune percentage Pmazs prune precision list ps(pi, P2 .....]
(where p1 > P2 > ...)
Output: Pruned RAT SNN parameter Opruned
1: precision idex i + 0, R² last + 0, prumed + False
2: while p S Pmaz do
3: R² val, 8' Prune&Retrain
percentage +- p, model parameter <- 6)
4: if R2 val RD -le Rg. then
{if performance drop exceeds tolerance) iddp <een(ps) then
5:
6: idæ, f- idx, + 1, 4 p- ps(idxpl, continue {try smaller prune percentage)
7:
else if pprnned = F False then
8:
p ps (idxp), continue continue {if not get pruned model yet, continue to try]
9: else
break continue {if already get pruned model with minimum percentage, stop)
11:
end if
else
13:
if R val > R² last then
Ra last R² val, Opraned 6' (if performance improve, save]
15:
end if
16: p f p + pslidkpp] {update prune percentage}
17: end if
18: end while
19: if Bgraned == False then
20:
Byraned t {if never get a pruned model, return original model]
21: end if
22: return Ppruned
Input
Connection ................................
Connection
Connection
Storkesttyle Propagate
Sequence Propagate
Figure 13: Comparison between two propagation styles. Left: Stork-style propagation, Right: quential propagation. The internal propagation order within each time step is indicated by ascending numbers.
B.5 BATCH NORMALIZATION IN RAT SNN
We apply two BN strategies in the CST re-parameterized attention block. First, for the Qs/Ks/Vs computation
head-wise BN (i.e., separate BN for each attention head) to preserve each head's 's independence before the attention calculation. Second, after the attention operation we apply element-wise BN to the output of other Rep-FC to ensure training stability.
20

---

Published as a conference paper at ICLR 2026
B.6 TRAINING DETAILS
Before training, RAT SNN is initialized using fluctuation-driven initialization (Rossbroich et al., 2022), and optimized with the SMORMS3 optimizer combined with cosine learning rate scheduling to accelerate convergence (Rossbroich et al. 2022 Liu et al. 2024). We employed surrogate gra- dient descent along with a homeostatic activity regularization term prevent neuron silencing for training (Rossbroich et al. 2022). Surrogate gradient descent minimized the root mean square error (RMSE) between the predicted y; and target finger velocities y/:
NT U
RMSE
0:)²
(12)
where Nr is the total time steps. To prevent neuron silencing during training, we add a homeostatic regularization term (activity lower bound, ALB) the loss function:
CALB Feg
AALB
max (0, ΦALB fr:)]²
(13)
where N is the total number of spiking neurons, fr, is the firing rate of neuron i, ΦLLB is the lower bound of the firing rate, and \ALB is the corresponding regularization strength coefficient. The overall loss function is defined as:
L= LRMSE + LALB
(14)
For lightweight finetuning stage, we further impose an AUB constraint on the average firing rate of spiking neurons:
N
CAUB
AAUB
max (0, dAUB]]
(15)
Lightt =
CRMSE CALB
+ CAUB
(16)
rag
where dAUB is the upper bound of the firing rate and AAUB is the corresponding regularization
strength coefficient.
The number of epochs for cross-condition pretraining is 120, while all other training stages adopt early stopping to ensure full convergence. Notably, we equalized input sizes across subjects by replicating shorter recordings. We trained single-session model with random seeds (1-5) to ensure result stability. For pretrained model, we used 2 different random seeds for pretraining, then applied random seeds respectively during the megle-session finetuning process. resulting 4 seed models. All training completed on a single Nvidia 4090D) GPU, with cross-condition pretraining taking under two days, and the remaining stages finishing within one day.
B.7 HYPER-PARAN PARAMETERS SETTINGS
For self-attention, set the number of attention heads to 4, with qkv dimensions (n, d) (48, 1). We configure RepConv with a three-branch structure using kernel sizes of 3. 1, and (identity map- ping), employing padding ensure the output dimensions equal the input dimensions. The specific hyperparameter settings are shown in Table g And following Liu et al. (2024), the parameters for LIF and LI neurons shown in Table9
B.8 BASELINE SET
In this work, we compare our method with various baseline models to comprehensively evaluate the decoding performance and energy efficiency of RAT SNN. For the performance of these baseline models on the corresponding datasets, we first use the results reported in the original publications as the baseline performance. We select bigRSNN (Liu et al., 2024) and its pretrained version as the major SNN-based baseline models, and AEGRU (Wang al. (0244) and POYO (Azabou al. 2023) as the major ANN-based baseline models, representing the current state-of- tet-hh--ttii in the SNN and ANN fields, respectively. For the results of major baseline models that reported in the
21

---

Published as a conference paper at ICLR 2026
Table 9: Time Constants and Training Parame- ters of LIF and LI Neurons
Table 8: Hyperparameter Settings for RAT SNN
Parameter
Value
Parameter
Value
Input size
192
Synaptic time constant (s)
1.00E-02
Attention head num (Fihend)
LIF membrane time constant (s)
2.00E-02
Dimension of qkv (n, d)
(192, 1)
membrane time constant (s)
5.00F-02
Rep-conv kernel size
Time constants heterogeneous
True
Stride
Time constants trainable
True
Dilation
L2 strength of ALB AALB
100
Bias
False
L,2 threshold of ALB ØLLB
1.00E-03
MLP size
(512,1)
L2 strength of AUB AAUB
0.01
Readout head num
L2 threshold of AUB ΦAUB
10
learning rate
1.00E-02
Dropout rate
0.3
original papers for certain datasets, we evaluate these baseline models using the testing framework provided by NeuroBench YYik et al. 2025).
For bigRSNN and AEGRU, we train and test them the corresponding datasets using their recom- mended hyperparameters. For POYO, since its testing framework differs from that of NeuroBench, modified its pretraining procedure according the NeuroBench testing standard on the NHP dataset, and trained and tested it with the provided hyperparameters.
Since different baseline models adopt different training strategies and preprocessing pipelines (e.g., the random offset sampler in POYO and Gaussian noise augmentation and jitter in AEGRU), it is difficult to define fully unified preprocessing procedure that aligns all baselines. Therefore, for the baseline models in this paper, we use the hyperparameters and preprocessing strategies recom- mended by the original authors ensure that our evaluation faithfully reflects the performance of each method.
B.9 COMPUTATIONAL COST ESTIMATION
To evaluate the computational cost of RAT SNN and baseline models, we used the computational cost estimation method provided by NeuroBench (Yik et al. 2025). Specifically, we calculated the total effective MACs and ACs the same test data metrics for computational cost. Furthermore, following Yin et al. (202T), we set the energy consumption of a single MAC operation to be 31 times that of single AC operation estimate the energy efficiency of each model.
The estimation method provided by NeuroBench only considers the computational cost of basic linear and RNN modules in the model, but does not provide calculation method for attention modules. Therefore, for multi-head attention modules, we first determine whether the data is binary before each matrix multiplication to distinguish between AC and MAC operations. We then binarize the input data, and based on the output of the matrix multiplication, count the number of AC and MAC operations according the binary determination results.
B.10 AC-ONLY OPERATIONS SELF-ATTENTION
In RAT SNN, the self-attention operates entirely with AC operations. This achieved by binarizing the Q, K, and V matrices to Qs, Ks, and Vs by LIF layers before performing matrix multiplications. Specifically, according to Eq. (6), we first compute (K''V.,) Because K., and Vs, contain only 0/1 values, the matrix product can be implemented using AC-only operations (the identity element under multiplication leaves any number unchanged, and this property extends to matrix multiplication). When multiplying Q. with V.....). s; V..). Qs, is also binary, this step can likewise be realized by ACs. Thus the entire attention computation avoids MAC operations and uses only ACs, consistent with the spike-driven transformer (Yao al. 2023a
22

---

Published as a conference paper at ICLR 2026
Table 10: Performance comparison on NHP dataset, averaged across session 01-03 from Monkey I and L. Values without standard deviations are cited from published results. Results are reported as R 100. Best performing model in bold and second best model underlined.
Monkey
Monkey I.
Method
Session
Session 02
Session 05
Avv
Session 01
Session
Session 03
I. Avg
KF
35,00
0.60
27.80
21.13
9.20
19.60
13.80
14.20
ANN2d
62.17
57.27
ANN3d
67.50
59.40
65.90
64.27
$9.70
52.80
64,20
58.90
74.00
61.00
AEGRU
76.00
65.00
76.00
72.00
68.00
62.00
70.00
67.00
POYO-SS
73.02:40.70
55.60+1.99
38.38-43.44
62.334-0.85
64,27-2.66
57.63-43.50
59 79.70-1.78
60.534-2.32
POYO CS
77.53.10.73
63.70.0.82
71.73
70.99
72.42 1.12
65.98 1.49
70.48
69.63.12.32
SNN24
63.30
52,40
02.20
59.30
57.30
47.50
62.70
SNN3d
69.70
57,70
65.20
64,20
62.30
56.80
68.10
62.40
biRRSNN-SS
77.43-0.37
1740776776
74.78-00.73
70.46+-0.24
67.24-0.31
59.32:41.75
82-82-0.81
65.40-4088
bigRSNN-CS
76.96=00.25
58.53#1.42
77.17=0.69
70.89+0.45
70.02=0.66
62, 02.6990.83
73.38=0064
68, 70:00
RAT SNN-SS
08.06:10.77
63.29100.9
1.033=1.01
72.22:10.38
66.88=243
60.87:22.51
71.15=0.87
66.30:10.95
RAT SNN CS
79.68.10.29
64.07.0.61
79.03
1..14
1.21
62.11.10.80
74.68.0.55
68.6310.44
RAT SNN-CC
80.13:10.32
63.04221004
79.01=0.446
74.061:0.42
68.99=1.80
66.16:10.32
76.05=0.80
70.40:0068
RAT SNN CC-P
78.36.10.57
62.6110.63
78.57
73.18.1017
70.05 0.42
66.55.11.91
75.64.11.56
1..75
C EVALUATION DETAILS
Due to space limitations in the main text, we provide detailed session-specific decoding results here. For NHP dataset, selected three representative sessions from Monkey I and Monkey L to eval- uate multiple versions RAT SNN, labeled as Session 01, Session 02, Session 03 (Table[O). For private dataset, selected three random sessions recorded in 2022 from Monkey C05 to evaluate multiple versions of RAT SNN, labeled as CO5_1. 1, CO5_2, CO5_3 (Table[T). The specific results for NLB MLB_MAZE and NLB.RTT have already been presented in the main text.
The results demonstrate that all versions of RAT SNN perform excellently on NHP dataset, and as the scope of pretraining data increases, RAT SNN's decoding accuracy gradually improves. This proves that RAT SNN can effectively overcome neural temporal drift and neural variability across different subjects and tasks through cross-session and cross-condition training, thereby enhancing decoding accuracy.
For private dataset, RAT SNN also performs excellently across three sessions of Monkey CO5, and when generalizing to the unseen sessions CO5 2025, RAT SNN's decoding accuracy remains at the highest level with faster convergence, demonstrating the generalization capability across long temporal scales(Fig 14..
Table 11: Performance comparison (R² x 100, mean + std) on CO5_ C05_2022 and CO5_ CO5_2025 2025. **" means generalize result on unseen session.
Monkey CDS 2022
Monkey 2025
Method
Session
Session 02
Session 03
2022 Avg
Session 01
Session 02
2025 Avg
AEGRU
77.5810.39
18.15:1047
72.43-0.89
76.05:10.43
55.07:10.22
49,71=1.20
52.39=0.81
POYO-SS
78.22-+0.93
74.37:10.76
78.57=1.52
77.05:+0.80
64.6841..7
65.19:2..91
64.93-2.12
bigRSNN-SS
12.10:1:.71
85.40:10.28
77.64=0.76
81.72+1..35
65.35:10.89
58.9621.08
62.15:00633
bigRSNN-CS
84.62:10.84
86. 19:1922222
79.88-0.94
83.56:1.554
68.4110.37
64.49:10.29
66.45-2..2
RAT SNN-SS
84.24:10.28
85.29-10.68
75.20=0.88
81.58:0:42
69.17:10.75
64.25+11001
66.71:0:48
RAT SNN-CS
85.37-1:0366
87.52-10.88
03..6
84.3411..19
68.91111.03
64.64::1211
66.77+1.11
RAT SNN- CC
85.94::0.21
87.73-10.17
81.53 0.61
85.07::0.21
68 68.21110.10
64.87:t11599
66.54:0.34
RAT SNN-CC-P
84.98:10.32
87.2944..59
7.83
83.88-10111
60.09=:1.08
65.25::1.29
67.17=1.18
We use Table 12 to compare the decoding accuracy of pretrained RAT SNN models of different scales with RAT SNN-SS on seen sessions across datasets, and Table 13 to compare both decoding accuracy and convergence speed unseen sessions. Together, these results highlight RAT SNN's ability to improve decoding performance through cross-session and cross-condition pretraining.
We evaluate the generalization capability of RAT SNN-CC to unseen conditions by directly finetun- ing it on unseen subjects (including both seen tasks and unseen tasks), and compare its performance with the mained-from-scrate version (RAT SNN-SS). The comparison results have been presented
23

---

Published as a conference paper at ICLR 2026
Table 12: Performance comparison on seen sessions.
Dataset
Session
KAT-NNNSS
RAT-SNNCS
RAT SNN-CC
session
78.06:10777
79.68-1..29
0.31
Monkey
session 2
63.29:10.99
64.07=0.61
63.04::1.04
session 3
75.33+1.01
79.03=0.54
79.01=0.46
session
66.88 +2443
69.12+1.22
68.99-11.88
Monkey L
session 2
6
62.1
66.16-10.30
session 3
71.15=0.87
74.68
76.05-10.80
session 1
04,24
85.37
85.94:10.21
Monkey C05_2022
session 2
05.2
87.5240.38
87.73:10.17
session 3
75.20=0.88
00.13
81.53+10.61
Monkey Je (NLB_MAZI)
90.76.00.30
91.16:10.19
Table 13: Performance comparison unseen sessions.
Monkey CO5 CO5_202 2025
Monkey
Model
Metric
Session
Session
NLB_RTT
R x 100
69.17-10.75
64.25++
76.54+00.54
RAT SNN-SS
Epoch
10.19
68.40-16.54
171.60++44.2
Ri2 x 100
68.9111.03
6..6411.22
78.55:10.19
RAT SNN-CS
Epoch
28.50155007
20.2512.006
38.25119.37
Rd x 100
68.21
64.87
78.70-10.66
RAT SNN-CC
Epoch
28.75
2..55+1..00
126.75+40.17
in the main text. Here supplement the individual learning curve comparison results for Monkey
M, T, and C (Fig 15 TableT4.
Table 14: Performance comparison on unseen conditions on sessions used in Fig 5 and Fig
Metric
Monkey M
Monkey T
Monkey C
Rt x 100
86.99-10.51
74.84-10.32
93.21
RAT SNN-SS
Epoch
231.56+:36.40
304.13:144.9
364.75-168.26
Rd x 100
87.1110.11
76.2911.09
93.3010.11
RAT SNN-CC
Epoch
167.38+48.11
106.63+166665
221.13::3.228
C.I COMPARISON WITH SPIKACHU
Spikachu Mentzelopoouos et al. 2025) is a recently proposed SNN-based neural decoding model that also incorporates attention mechanisms. While its evaluation datasets and strategies differ from ours, on the common NLB MAZE and NLB RIT datasets, Spikachu achieved R²² scores of 0.78 and 0.56, respectively, which are lower than our method (RAT SNN-CC-p: 0.9102 and 0.7808).
Notably, Spikachu employs an ANN harmonizer before the SNN module to address neural activity variability across conditions, introducing substantial number of MAC operations. In contrast, our approach is a pure SNN model that maintains AC-only operations between synapses during inference. Furthermore, RAT SNN features a smaller architecture with fewer layers, resulting in lower inference latency and reduced parameter count. According to the reported results, the cross- condition version Spikachu (Spikachu-mp) consumes 4.75 JJ and 4.96 ,J on NLB MAZE and NLB RTT. respectively. Based the same energy evaluation metric (Bal & Sengupta 2024 Zhu et al. 2024). RAT SNÑ-CC-p consumes significantly less energy. approximately 4351.83 3 157.23 47 + 1207.11 pJ these datasets.
pJ and 8386.47
Although Spikachu offers an effective solution for handling cross-condition variability in SNNs and provides valuable insights for model improvement, the trade-offs in energy consumption and
24

---

Published as a conference paper at ICLR 2026
(a)
(b)
70.0
(a)
YEAARA (Da MXL RCT.
(b)
Average Training Epochs
50
1103000-11 103
Test R2 x 1000
65.0
40
62.5
30
60.0
20
57.5
(c)
Epeck
10
55.0
AEGRU POYO-S9
bigRsiNN-Ss
RAT SNN-S9
RAT SNM-CS
RAT SNN-CC
-CC-CC
Validetion x 100
(c)
Yaldertonn R3 Cheme TOL
16
-0
RAT SNN 85 SNN-CC
RAT SNN CS
10
15
25
50
35
(6
Epoch
Figure 14: Result of neural temporal drift generalization monkey C05 2025. (a) R² compared to baseline models. (b) Training
epochs required for convergence. (c) Learning curves, with the shaded area denotes std and aligned the fastest converging seed. Using session 01 as example. [S-S] denotes single- session models, [C-S] cross-session models, and [C-C] cross-condition models.
Figure 15: Supplementary learning curves for Monkey M, T, and C. The shaded area repre- sents std, and the curve length aligned to the fastest converging seed. RTT here means ran- dom target task with joystick.
decoding accuracy may stem from its deep architecture. A similar phenomenon regarding model depth was also observed in our ablation studies (Section F).
D CROSS-CONDITION PRETRAINING EFFECTIVENESS
To rule out the possibility that the effect of cross-condition pretraining is solely due to the increased amount of pretraining data, we designed a controlled experiment comparing RAT SNN-CC with RAT SNN-CS trained on the same amount of pretraining data.
Specifically, we constructed a small-scale cross-condition dataset containing six ses- sions: "indy_20 0102220661150111 "c05 202200179944011" and "c05.2022012001 2022012009415"",, (Monkey I, L. and C05). totaling 870,822 timestamps (3,483.29 seconds).
2o1dyy20110927..00 which are from three different subjects
Ploco_20170301
We also constructed a small-scale cross-session dataset containing six sessions: "indy 2002009015000 ".."". "indy-201608 "00", "indy-201 211.20160411.00 and "indy 201642001,,,,,001, all from Monkey I, totaling 871,949 times- tamps (3,487.80 seconds).
.0"", "indy 20160411.000
We trained RAT SNN-CC-mini and RAT SNN-CS ANN-CS-mini on these two datasets, respectively, and evaluated them the unseen Session 01-03 of Monkey (as recommended by NeuroBench). The
25

---

Published as a conference paper at ICLR 2026
results show that RAT SNN-CC-ming outperforms RAT ANN-CS-ming the unseen sessions of Monkey (3) demonstrating that the effectiveness of cross- cross-condition pretraining is not solely due to the increased amount of pretraining data. Compared to training single condition, in- troducing a greater diversity of neural activity patterns can more effectively improve the decoding accuracy of RAT SNN.
Table 15: Performance comparison (R² 100, mean + std) of RAT CC-mini on Monkey I.
and RAT SNN-
Method
Session 01
Session 02
Session 03
Average
RAT SNN-CS-mini
0..77
62.82:41.23 67.92:1..88
76.08-00.77
72.36:H0.30
RAT SNN-CC-mini
78.11
77.52+10.43
74.52:2.44
E SUPPLEMENT FOR ABLATION STUDY
The above comparative experiments have demonstrated the importance of cross-session pretraining and cons-condition pretraining for improving the decoding accuracy of RAT SNN. To further vali- date the impact of our design on decoding accuracy, we investigated the effects of each component of RAT SNN on decoding accuracy across both single-sessinn and cros-ccondition dimensions.
E. ABLATION STUDY ON SINGLE-SESSION RAT SNN
Using the small-scale dataset (Monkey C05) an example, we conduct ablation experiments on the RAT SNN-SS to evaluate how design affects decoding accuracy. The ablation study assesses from three perspectives: (1) the overall model architecture, (2) the configuration of LIF layers, and (3) the re-parameterization method.
E.1.1 ABLATION STUDY ON THE OVERALL MODEL ARCHITECTURE
We first compare the decoding performance of CST attention block with classical spike-driven trans- former block (represented by (Yao et al. (223a)) on the CST task. Specifically, we design RAT SNN variant based on the spike-driven transformer block, denoted as SDT SNN, while keeping the remaining architecture identical to RAT SNN. We then evaluate SDT SNN, RAT SNN-192 (where the output dimension of the CST attention block is set to 192, matching that of SDT SNN), and the final version RAT SNN-512 (shown as RAT ENN-SS above), in order assess how the two attention modules affect decoding accuracy. The ablation results confirm the advantage of RAT SNN's CST attention block in terms of decoding accuracy (80.08 for SDT SNN vs. 80.49 for RAT SNN-192), as well as the performance gains obtained by increasing the output dimension of the attention module (80.49 for RAT SNN-192 vs. 81.58 for RAT SNN-512, T6).
We additionally construct an SNN model that contains only RepFC-LIF layers, denoted as RepFC SNN, evaluate the contribution of the attention module decoding accuracy. Specifically, RepFC SNN-HI replaces the CST attention block in RAT SNN-512 with a single RepFC-LIF layers of size 512, while RepFC SNN-H3 replaces it with three RepFC-LIF layers of sizes [192, 192. 512], matching the depth and width of RAT SNN. The ablation results show that introducing the attention module substantially improves the decoding accuracy of RAT SNN, indicating that the attention mechanism plays an important role in capturing spatial dependencies in neural activity (81.09 for RepPC SNN-HI vs. 79.47 for RepFC SNN-H3 vs. 81,58 for RAT SNN-512, T6ble
We further investigate how increasing the depth of the CST attention block affects decoding per- formance. The results show that deeper attention block actually degrades the decoding accuracy of RAT SNN (79.53 for RAT SNN-192-A2 vs. 81.58 for RAT SNN-512). Combined with the comparison between RepFC SNN-H3 and RepFC SNN-HI (81.09 for RepFC SNN-HI vs. 79.47 for RepFC SNN-H3), this is consistent with previous findings and indicates that, for the CST task, increasing model depth is detrimental rather than beneficial to improving decoding accuracy (T.
In addition, we compare a w/o attentionnRRegeee variant of RAT SNN, in which the RepFC branch is removed from the CST attention block, with the standard RAT SNN to evaluate the impact of
26

---

Published as a conference paper at ICLR 2026
Table 16: Ablation study on RAT SNN-SS on Monkey C05 (R² x 100, mean std).
Model
CO5 .022
CO5.2022
Architecture
SDT SNN
80.091:.233
65.21=0.557
RAT SNN-192
80,49-+0.38
66.61
RAT SNN-192-A2
79.53++0.22
66,43+0044
RepFC SNN-HI
81.09+0.440
66.26+00355
RepFC SNN-H3
79.47=0.96
65.74:1100
w/o syn dynamic
79.13+0.50
58.0440.52
w/o recurrent
79.50+00.45
60,48+1167
RAT SNN-512
0..58:0.42
66.71
86
wio ottentoonRepFCC
84
RAT SNN
Test R2 x 100
82
80
78
76
74
C06_01
C05_00 005_02
C05_00 03
Average
Figure 16: Performance comparison between the standard RAT SNN and the w/o attention-
RepFC version. The dashed line in the boxplot represents average value..
the RepFC branch on decoding accuracy. The ablation results for attention-RepFe demonstrate that RepFC branch can effectively improve decoding accuracy of RAT SNN. Although the improvement is modest (81.35 vs. 81,58), across all three sessions of Monkey CO5, the addition of the RepFCC branch leads varying degrees of improvement decoding accuracy of RAT SNN, indicating that the performance gains from this strategy are consistent (Fig. 16). Table
E.1.2 ABLATION STUDY ON THE CONFIGURATION OF LIF LAYERS
In this part, we mainly examine the impact of recurrent connections and synaptic dynamics on decoding accuracy (TableT6). The ablation results show that both structures provide substantial im- provements in decoding performance, with the removal of synaptic dynamics leading to decrease in decoding accuracy from 81.58 79.13, and the removal of recurrent connections resulting in drop to 79.50.
1.3 ABLATION STUDY ON THE RE-PARAMETERRIZATION METHOD
This section compares the impact of the ee-aarameterization method on RAT SNN-SS. Before train- ing, we re-parameterize RAT SNN obtain w/o re-parameterization version RAT SNN that does not include multi-branch Conv and BN structures.
The ablation results for the re-parameterization method demonstrate that this strategy not only effec- tively improves the learning speed of RAT SNN but also enhances decoding accuracy. Specifically, compared to the standard RAT SNN, the w/o re- parameterizztiion version 17.16% decrease in decoding accuracy (R² drops from 81.58+0.42 to 67.58+2.50),, while the train- ing epochs required for full convergence increase by approximately 3 times (average training epochs increase from 179+-67 to 545+ 84, Table
RAT SNN shows a
27

---

Published as a conference paper at ICLR 2026
Table 17:
Ablation study results on single-session RAT SNN components. The top performing models in each category are indicated in boldface.
Method
Avg/ 100
Avg./Epoch
w/o re-param
67.58+2.50
545+84
w/o Aeten
81.35:+00.3
213+165
RAT SNN-SS
81.5810.42
179167
Table 18: Performance comparison (Rt x 100) between subject-specific BN and subject-shared BN on cross-condition RAT SNN. the top performing models in each category are indicated boldface.
Session
Shared BN
Specific BN
CO5_1
8.82
84.79:10.64
C05 2
85.66=1.277
86.84:00.52
CO5 3
7..01=1885
79.05=11.53
C05 Avg.
82.31=0.98
83.56++0.41
LI
78.29=0.57
78.30:10.71
L2
60.52++1811
62.16:10.47
L3
75.15=1.21
76.35:10.97
LAvg.
71.32.10.93
72.27-00.66
LNLB_RTT.
0.70
77.89
L
1..90
68.17+2.00
I,2
6.28
57.45++..9
L.3
71.84=0.17
72.10
LvAvg.
6..2
65.91
E.2 ABLATION STUDY ON CROSS-CONDITION RAT SNN
For RAT SNN-CC, we primarily compared the decoding accuracy between subject-specifie BN and subject-shared BN.
Due to the high computational resource and time requirements for cross-condition training, we used subset of the cross-condition dataset for comparative experiments. Specifically, for the data from Monkey I, L, and COS in the cross-condition dataset, we selected only the first sessions from each for comparison, reducing the total session count from 58 19. The subsequent finetuning approaches for both subject-specific BN RAT SNN and subject-shared BN RAT SNN remained consistent with the aforementioned methods.
Across the 4 sessions of Monkey CO5, subject-specific BN RAT SNN consistently outperforms the subject-shared BN version. other dataset, subject-specific BN RAT SNN similarly outperforms the bobjectt-hhrrdd BN version, with only session 2 from Monkey L showing inferior performance (8). This indicating that during the cross-condition pretraining stage, subject-specific BN can better extract the underlying distributions of neural activity under different conditions, thereby improving the decoding accuracy RAT SNN.
E.3 ABLATION STUDY ON LIGHTWEIGHT FINETUNING STRATEGY
In this work, we advocate combining direct finetuning with lightweight finetuning and AUB regu- larization during the lightweight finetuning stage target session after pretraining, To validate the effectiveness of this finetuning strategy, we compare the results of direct finetuning with lightweight finetuning.
Specifically, we pretrained RAT SNN on the cross cross-session dataset, then directly finetuned on the target session to obtain the Pre-LF version of RAT SNN. Subsequently, performed lightweight finetuning on Pre-LF version to obtain the Post-LF version. We compared these two versions to eval- uate the impact of lightweight finetuning decoding accuracy (9). Similarly, conducted
28

---

Published as a conference paper at ICLR 2026
Table Performance comparison (R² 100) before and after lightweight finetuning across cross- session RAT SNN. Bold indicates the best result in each row.
Session
Pre-LF
Post-LF
CO5_1
84.93=0.60
85.37:10.66
CO5_2
86.41
87.52-10.38
C05
80.34+-2.01
80.13++0.56
CO5 Avg.
83.89+0.71
84.34
L1
78.81=0.56
79.69
L2
62.22+t1.10
64.07=0.611
L3
78.54=0031
79.03+10.54
LAvg.
73.19:10.49
74.26:10.14
LNLB_RTT
78.41
78.15
L1
66.59-F1.50
69.12+1.20
L22
59.51+2.448
02.0
L3
74.58
74.68
L.
66.89110.42
68.6310.44
Table 20: Performance comparison (R²
100) before and after lightweight finetuning and after
pruning across cross-condition RAT SNN
Session
Pre-LF
Post-LF
After prune
CO5_1
86.05 0.15
85.94
84.98=0.32
CO5 2
86.91=0.43
87.73
87.29-10.59
CO5_3
82.19=1.23
81.53:10.61
0.83
C05_Avg.
85.05=0.31
85.07
83.88=0.11
T_
79.21=0.39
0.332
7.537
L2
61.96=1.38
63.0411.04
62.61-0.63
77.70=0.30
79.01
78.57
LÁvg.
72.96=0.41
7.40
73.18=0.17
INLB RTT
76.73=0.91
78.70
78.08=0.75
L.
67.07=2.59
68.9941.80
7..05=0.42
L.2
64.65=1.59
66.16
66.55
L3
76.33=1.48
76.0540.80
1.564
L Avg.
69.35 1.01
7..40+0.68
70.75 + 1.24
NLB MAZE
0,395=0.30
91.1611.119
91.022-0.15
analogous comparative experiments on the cross-condition scene with additional iterative pruning (T)
The results demonstrate that lightweight finetuning can effectively improve decoding accuracy across both cross-session and cross-condition scenarios. This demonstrates that RAT SNN perform more refined search in the local parameter space through lightweight finetuning, thereby improving decoding accuracy. And iterative pruning no harm to decoding accuracy with effective effect to reduce the computational consumption of RAT SNN.
F DISCUSSION ON THE DEPTH OF SNN IN CST DECODING TASK
In image-related applications, deep SNN models often achieve better performance than shallow ones. However, many neuromorphic domains, shallow SNNs actually outperform deeper models, observed in tasks such as Spiking Heidelberg Digits and Therefore, in the CST decoding task with pretraining (to minimize the adverse effects of limited compared SNNs with 1-3 hidden layers and found that, when
(Rossbroich et al.) 2022).
dataset size deep models). using the simplest fully connected RSNN architecture, increasing the number of hidden layers not only fails improve decoding performance but in fact leads to degradation (Fig. I7p.
The potential reasons for this phenomenon may include: (1) Inaccurate gradient estimation: surro- gate gradients used to approximate the non-differentiable spike function, and the approximation
29

---

Published as a conference paper at ICLR 2026
error accumulates more severely as the network depth increases. (2) Downstream neuron death in deep SNN models and increased sensitivity of deeper SNNs to initialization (Rossbroich et al, 2022). (3) Differences in data structure: in natural image tasks, the spatial structure is highly com- plex, requiring multiple layers of convolutions and nonlinearities to hierarchically extract features from local edges semantic concepts. In contrast, most neuromorphic benchmarks (including the CST task) typically involve "event streams + relatively simple spatial structure," where the key to decoding often lies in temporal patterns and spiking rhythms rather than rich high-level spatial mantics.
Therefore, in the CST decoding task, using a shallow SNN is in fact better choice. We compress the depth of the spike transformer block and adopt a single-layer attention structure, making the model more advantageous in terms of both model size and computational operations.
1.0
0.3778
6.5477
6.4923
3.0
5W0-17i0den
SVW-2h
Shi-Shidsen
Figure 17: Performance comparison between 1-3 hidden layers SNN. These SNN model are tested on Monkey and Monkey L with random seed 1-5, the hidden LIF layer size is 64, The large variance is caused by differences in CST signals across subjects.
G THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, all research, ideation, and writing were conducted by the authors. Copilot was used solely as an assistive tool for polishing the manuscript and code annotation. All content generated with Copilot's s assistance was thoroughly reviewed and revised by the authors ensure the original- ity and quality of the work.
30


Reviews:
[{"review_id": "6vmtnGLY4k", "reviewer": "Reviewer_XSu9", "summary": "The authors build on existing Spiking neural networks architecture using attention heads by introducing structural reparametrization in the self-attention blocks. The reparametrization comes with free batch normalisation, and the authors assign to each condition (ie. behaviour task) an independent batch normalisation during pre-training. The resulting model is then put to the bench using a corpus of extracellular neurophysiology recordings in non-human primate performing reaching tasks. \nThe RAT-SNN model overperform previous SNNs and matches ANNs, at a lower computational expense (pruned version of 150k parameters compared to ~2M for ANNs). Furthermore, the pre-trained model is shown to systematically converge in fewer iterations than zero-shot attempts.", "questions": "- What experimental benchmark would make SNNs fail compared to ANNs ? Is there a more drastic protocol where pre-trained models are shown a completely unseen Maze task in an unseen subject where the ANNs take a strong lead ? Answering this question would make for a stronger paper.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Motivation is clear, SNN could enable generalizable models and be a lower power alternative than conventional ANNs for BMI applications.\n- Performance on neural decoding is comparable to ANNs for this experiment, with an ablation study showing the reparametrization having a 0.15 increase on R2.\n- The experiment follows a proposed framework for neuromorphic modeling benchmarking for model for baseline comparisons where applicable and for computational (Neurobench)", "weaknesses": "The experimental protocol is narrow in the sense that tasks are all very similar, using the same recording modalities in the same cortical areas of primates. One hypothesis could be that ANNs will generalize better than SNNs when the breadth of datasets will expand. Here maybe the experimental protocol is a bit too easy, without enough \"unseen\" subjects/tasks benchmarking. Even if ANNs were to take a strong scoring lead in such a case compared to SNNs, this wouldn't reduce the appeal of the presented method for real-time applications.\n- I struggled to follow the experimental protocol and understand the terms single-session, cross-session, cross-condition: in all the performances of table 2 and 3 the pre-training always seems to include all datasets, while truly \"unseen\" benchmarks are only introduced in figure 5. Maybe a visual or a table recap of what is used in each training phase and in test would be more clear.\n- Private dataset is... private, making any reproducibility or competition attempt impossible at the moment.\n\n\n## Minor comments\n- typo line 961: we employs\n- the github repository is unaccessible", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qjn1pTTKdO", "reviewer": "Reviewer_a8mk", "summary": "This paper proposes a novel spiking neural network model for neural decoding across heterogeneous sessions, called RAT SNN. Incorporating a re-parameterized architecture, RAT SNN achieves consistent improvement over leading SNN baselines and comparable with ANN baseline POYO. Overall, with SNN being a light-weight model, I believe this is a valuable addition to the community, hence I am leaning towards acceptance.", "questions": "1. In appendix (line 740 page 14), the training and validation setups are mentioned to be overlapping 2-sec samples and continuous long sequence respectively. Were the same setup used for POYO as well? Because POYO by default seems to be working with 1-sec segment, with no length generalization in validation. I believe it is worth mentioning that when discussing POYO’s scores.\n2. Could the authors comment on the potential reason cross-condition results are sometimes worse than cross-session or single session? For example, for monkey I and L, there is a ~2% improvement from SS to CS, while CC in monkey I is worse than CS. Considering that from SS to CS, the dataset increase is not that big (1 session vs 3 sessions), whereas CC has much more data. Would it be a data issue (maybe dataset is too diverse) or a model issue?\n3. In addition, have the authors tried POYO in CC? It seems to be a bigger improvement from SS to CS for POYO, so it would be interesting to compare dataset scaling capability between ANN and SNN.\n4. Being at the core of the architecture, it would be better to formally define re-parameterization-style layer.\n5. Pruning is not properly defined in section 3.3, but there are multiple mentions of that in the results table. This could cause confusion.\n6. In Table 1, perich dataset is included but I didn’t find the results on that dataset. Could the author point to where in the paper that dataset is evaluated?\n7. In line 374 page 7, it is mentioned that RTT was used to test transferability to unseen sessions. How about Maze dataset whose results are also in Table 4? Could the authors comment on the experimental setup on that dataset, based on my understanding it couldn’t be in-distribution.\n8. Please note that the provided code link is not valid, as all the files are shown not found\n9. In equation (4), please mention specifically which index to sum over with respect to the feed-forward function\n10. Please fix the numbering error in appendix", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Propose a novel SNN architecture including re-parameterized self attention and condition-specific batch normalization\n- Proposed model show improved performance (in or out of distribution) over previous SNN models and comparable results as non-SSN transformer, which is known to have better decoding performance than SNN\n- Scaling analysis shows that the proposed model can benefit from training over more and diverse datasets, which is a promising trait in the era of large-scale pretraining.", "weaknesses": "- The method section is in general difficult to follow, especially for someone not very familiar with SNN. It would make it easy to include some high-level description on what particular layers do, instead of laying out detailed equations and terminology\n- Effect on dataset scaling is still not clear.. Despite having more data, cross-condtion training is sometimes worse than cross-subject training and even single sessionm in terms of decoding performance, which warrants further investigation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sjldqRpX0v", "reviewer": "Reviewer_tbCB", "summary": "This paper proposes a re-parameterized self-attention spiking neural network (SNN) architecture for neural signal decoding.\nThe network employs spiking neurons to process cortical spike train signals, achieving lightweight structure and low-power consumption.\nFurthermore, based on existing datasets, they construct a hybrid dataset that integrates cross-condition, cross-session, and cross-subject data, and conduct pretraining on this dataset to alleviate the generalization problem commonly faced by neural decoding models.", "questions": "1. The authors state that the proposed model uses only four layers of LIF neurons, yet the paper does not provide a detailed description of the overall network architecture. Moreover, a four-layer configuration is considered very shallow for a typical SNN. The fact that the authors claim to achieve SOTA performance in various aspects—and even outperform ANN-based models—with such a shallow network raises doubts about the fairness of the comparison.\n\n2. The authors present the iterative equations for LIF neurons in Equations (1)–(3), but the formulas appear to be incorrect. In Equation (1), it should be $V_{syn}[t]$.\n\n3. In Section 3.2, the authors state that the re-parameterized channel attention consists of re-parameterized convolutional and FC layers. However, I disagree with this claim and believe it represents over-packaging of the concept. In my view, an attention mechanism should dynamically adjust its focus based on the input, helping the model emphasize important parts while filtering out irrelevant information. Moreover, channel attention is a well-established concept introduced many years ago, typically involving the use of convolution to generate attention scores across channels, which then guide the model to reduce redundancy and enhance salient features. Therefore, I do not consider a module composed merely of convolutional and fully connected layers to qualify as a genuine channel attention mechanism.\n\n4. At the end of Section 3.2, the authors mention that global BN is applied to the output of the multi-head attention mechanism. However, I do not clearly understand what the authors mean by global BN. If their intention is to distinguish global BN from head-wise BN, then I wonder—haven’t previous works that employ self-attention already used what could be considered global BN by default? Moreover, has head-wise BN ever actually been used or reported in prior studies?\n\n5. Throughout the paper, the authors repeatedly emphasize the concept of cross-condition when constructing datasets and conducting generalization tests. However, they do not clearly define what exactly a condition refers to. Is it a concept parallel to cross-subject and cross-session, or does it represent a broader notion that encompasses cross-subject, cross-session, and other variations?\n\n6. What are the model parameters and the time step length used in the SNN in this paper? Additionally, how is the time step defined or segmented for the CST signals?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper provides a well-grounded analysis of the motivation, clearly identifying that the major challenge in current brain-signal decoding models lies in their poor generalization ability. It also highlights the low-power and lightweight model requirements in edge-side deployment scenarios. Building upon this, the paper logically introduces the use of spiking neural networks (SNNs), which are not only inherently well-suited for processing cortical spike train (CST) signals, but also naturally meet these computational constraints, forming a coherent and well-structured line of reasoning.\n\n2. The experiments in this paper are thorough. A hybrid dataset was specifically designed to meet the requirements of pretraining, and extensive evaluations were conducted across multiple datasets.", "weaknesses": "1. The authors’ innovations in training methodology and dataset integration are indeed appealing; however, the network architecture design lacks sufficient novelty. For instance, the re-parameterization operation that fuses Batch Normalization (BN) with Convolution/FC layers has already been widely adopted in CNN-based SNN architectures. Similarly, the multi-branch convolutional structure inspired by RepVGG and its inference-time merging into a single convolution are also common practices in the ANN domain. In this paper, the authors merely apply these existing techniques, yet devote substantial space to describing the re-parameterization process, even emphasizing the “re-parameterized self-attention” in the title—while no genuinely innovative architectural design is evident in this aspect.\n\n2. The variable notation is inconsistent. In the annotation of Figure 1, it is stated that $n_\\text{input}$ represents the input dimension, which suggests that n should denote a dimensional quantity. However, later in the text, the authors write that “$n$ and $d$ represent the size of the LIF layers,” a statement whose meaning is unclear—particularly regarding which symbol refers to the number of layers. Furthermore, in Section 3.2, when analyzing the SNN self-attention mechanism, the paper claims that the computational complexity is reduced from O($n^2d$) to O($nd^2$). In this context, $n$ is evidently used to denote the number of tokens, while $d corresponds to the feature dimension, which contradicts the earlier definition. Overall, the inconsistent and ambiguous use of symbols significantly affects the clarity and readability of the paper.\n\n3. The paper provides insufficient analysis of the neuron design. In the design of LIF neurons, the authors propose R-LIF, which introduces a recurrent structure across time to enhance the extraction of temporal information. Specifically, a temporal skip connection is added to integrate past spike firings and synaptic inputs. However, the paper does not provide a detailed analysis of the motivation behind this design, nor does it discuss any possible biological justification for it. Moreover, such an operation is not novel in the context of SNN neuron design.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7LpctAmlmS", "reviewer": "Reviewer_c3gQ", "summary": "This paper proposes RAT SNN, a Spiking Neural Network integrating re-parameterized self-attention with a cross-condition pre-training framework. It aims to provide a neural decoding solution for iBMI that offers high accuracy, strong generalization capabilities, and high energy efficiency.", "questions": "It is strongly recommended that the authors move the reparameterization effectiveness experiments from the appendix to the main body of the paper.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper's strength lies in its ingenious integration of various advanced technologies to address a critical and highly practical problem (efficiency and generalization bottlenecks in iBMI). It further demonstrates an excellent balance between decoding accuracy and computational efficiency through extremely detailed and convincing experiments.", "weaknesses": "I am not familiar with the iBMI field, so I can only raise the following weaknesses from the SNN perspective.\n\n1.The method primarily represents a sophisticated combination and engineering optimization of existing techniques (e.g., RepVGG-style re-parameterization, spiking Transformers, and LIF layer with recurrent connections) within a specific application scenario, rather than proposing an entirely new algorithm or theoretical framework.Weakness\n\n2.During the inference process of the attention module, how does matrix multiplication avoid MAC operations? Please provide a more detailed explanation.\n\n3.Is the use of the attention mechanism effective on a network with only 4 layers? The paper lacks corresponding ablation experiments, such as comparing with a regular SNN that uses reparameterization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)