SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decoding
Authors: Trung Le
Hao Fang
Jingyuan Li
Tung Nguyen
Lu Mi
Amy L Orsborn
Uygar Sümbül
Eli Shlizerman
Venue: NeurIPS 2025 poster ()
Keywords: intracortical brain-computer interfaces
Primary area: neuroscience_and_cognitive_science

Abstract: Intracortical Brain-Computer Interfaces (iBCI) decode behavior from neural population activity to restore motor functions and communication abilities in individuals with motor impairments. A central challenge for long-term iBCI deployment is the nonstationarity of neural recordings, where the composition and tuning profiles of the recorded populations are unstable across recording sessions. Existing approaches attempt to address this issue by explicit alignment techniques; however, they rely on fixed neural identities and require test-time labels or parameter updates, limiting their generalization across sessions and imposing additional computational burden during deployment. In this work, we address the problem of cross-session nonstationarity in long-term iBCI systems and introduce SPINT - a Spatial Permutation-Invariant Neural Transformer framework for behavioral decoding that operates directly on unordered sets of neural units. Central to our approach is a novel context-dependent positional embedding scheme that dynamically infers unit-specific identities, enabling flexible generalization across recording sessions. SPINT supports inference on variable-size populations and allows few-shot, gradient-free adaptation using a small amount of unlabeled data from the test session. We evaluate SPINT on three multi-session datasets from the FALCON Benchmark, covering continuous motor decoding tasks in human and non-human primates. SPINT demonstrates robust cross-session generalization, outperforming existing zero-shot and few-shot unsupervised baselines while eliminating the need for test-time alignment and fine-tuning. Our work contributes an initial step toward a robust and scalable neural decoding framework for long-term iBCI applications.

SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decoding
Trung Le Hao Fang', Jingyuan Li', Tung Nguyen?, Lu MI Amy Orsborn', Uygar Sümbül', Eli Shlizerman"
University of Washington
2 University of California, Los Angeles
Tsinghua University
Allen Institute
Abstract
Intracortical Brain- Crain-Computer Interfaces (iBCI) decode behavior from neural popu- lation activity restore motor functions and communication abilities in individuals with motor impairments. A central challenge for long-term iBCI deployment is the nonstationarity of neural recordings, where the composition and tuning pro- files of the recorded populations are unstable across recording sessions. Existing approaches attempt address this issue by explicit alignment techniques; how- ever, they rely on fixed neural identities and require test-time labels or parameter updates, limiting their generalization across sessions and imposing additional com- putational burden during deployment. In this work, we address the problem of cross-session nonstationarity in long-tend mer-term iBCI systems and introduce SPINT Spatial Permutation Invariant Neural Transformer framework for behavioral decoding that operates directly on unordered sets of neural units. Central to our ap- proach is a novel context-depenuent positional embedding scheme that dynamically infers unit-specific identities, enabling flexible generalization across recording sessions. SPINT supports inference variable-size populations and allows few- shot, gradient-free adaptation using a small amount of unlabeled data from the test session. We evaluate SPINT on three multi-session datasets from the FALCON Benchmark, covering continuous motor decoding tasks in human and non-human primates. SPINT demonstrates robust cross-session generalization, outperforming existing zero and few-shot unsupervised baselines while eliminating the need for test-time alignment and fine-tuning, Our work contributes initial step toward a robust and scalable neural decoding framework for long-term iBCI applications.
Introduction
Motor behavior arises from the complex interplay between interconnected neurons, each possessing distinct functional properties [1] Deciphering the highly nonlinear mapping from the activity of these neural populations to behavior has been major focus of intracortical Brain- Brain-Computee Interfaces (iBCI), whose applications have enabled individuals with motor impairments to control external devices 2 restore communication abilities through typing 55. handwriting 66] 71, and speech 8 10 [2].
Despite the remarkable capabilities, iBCI systems suffer from performance degradation over extended periods of time, largely attributed the nonstationarities of the recorded populations 13] Sources of
"ooreeponniig author: shlizee @uw.edu. Code is available at http://s///ggtttubbooeee/eeeeeee
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

nonstationarities include shifts in electrode position, tissue impedance changes, and neural plasticity [4 [3] [6]. These nonstationarities lead to changes in the number and identity of neural units picked up by recording electrodes over time. Such changes in population composition alter the learned mapping from neural activity to behavior, preventing decoders trained on previous sessions from maintaining robust performance in new sessions. To ensure robust behavior decoding over future recording sessions, one approach has focused on training deep networks using many sessions, attempting to achieve decoders that are robust to cross-session variability [7 [8] This zero-shot approach requires months of labeled training data, necessitating extensive data collection from the user. While recent methods targeting cross-subject generalization may alleviate some of this burden [19 20 21 22]. degradation over long-term use still remains, advocating for the adoption of adaptive methods [1 23 24 25 26 These adaptive methods leverage the now-dimensional
manifold underlying population activity, which has been shown to preserve a consistent relationship with behavior over long periods of time [14 27). Depending on the use of labels at test time, they
can be categorized into supervised [21 28 semi-supervised 77. or unsupervised [24, 25 29].
with varying levels of success and practical utility in weal-world iBCI [26].
Despite the variety of technical approaches, these works share common design philosophy: they adopt fixed view of the neural population, assigning fixed identities and order for neural units during training. While this treatment achieves high decoding performance on held-in sessions (within- session generalization), the decoders suffer from out- of-distribution performance degradation when evaluated on held-out sessions with different sizes and unit membership (cross-session generalization). To enable transfer of the pretrained model to novel sessions, explicit alignment procedures with gradient updates to adapt model parameters are necessary, imposing disruptive and costly computation overhead for iBCI users. With these limitations of existing approaches, we advocate for the view that an ideal, universal iBCI decoder should be invariant to the permutation of the neural population by design, and should be able to seamlessly handle inference of variable-sized, unordered set of neural units with minimal data collected from the new setting.
In this work, we address the problem of cross-session nonstationarity in long-term iBCI systems and introduce SPINT a pprmuuaaiooninvaariant framework that can decode motor behavior from the activity of unordered sets of neural units. We contribute toward an iBCI design that can predict behavior covariates from continuous streams of neural observations and adapt gradient- free to novel sessions with few-shot unlabeled calibration data. At the core of our methods is a permutation-invaric transformer with a novel context-dependent positional embedding that allows flexible identification of neural unit identities on-the-fly. We evaluate our approach on three movement decoding datasets from the FALCON Benchmark (26). demonstrating robust cross-session generalization motor tasks in human and non-human primates. Our model outperforms zero-shot and few-shot unsupervised baselines, while not requiring any retraining or fine-tuning overhead.
In summary, the contributions of this work include:
We present a transformer-based permutation cermutation-imvarian invariant framework with a novel context-dependent positional embedding for few-shot unsupervised behavioral decoding. Our flexible, lightweight model enables ingestion of unordered sets of neural units during training and facilitates out-of-the- box inference on unseen neural populations.
We evaluate our model on three motor behavioral decoding datasets in the FALCON Benchmark, showing robust gradient-free generalization to unseen sessions in the presence of cross-session nonstationarities.
2 Related Work
Decoding motor behavior from intracortical population activity: A variety of computational models have been proposed model population spiking dynamics and decode motor behavior, starting with linear models such as population vectors [I]. linear regression [30]. Kalman filter [31 32
33
Nonlinear methods have also been developed: a non-exhaustive list includes generalized linear models
34 35
36
37], latent variable models [38] 39 40 41), leveraging deep neural networks including
recurrent neural networks (RNN) 27
42 43 and transformers [44 28
46 22
48
49].
Most of these works assume fixed view of the population across time and have limited direct transferability to unseen populations. We adopt the view of neural population as an unordered set of neural units, offering more flexibility in transferring across population compositions.

---

(A)
160
Channel
Channel N
Nonstationary neural activity over sessions
Channel
1.80
80
Session
88898855
Chanmele
8889
1.0
0.6
0.0
Session 1
Session 2
Bn
Seceinn-
Human IBCI
-0.6
Shifted channel
Shifted
channel N
Covatatee cleriee
88
Time
Session K
Charmete
0.
3
0.0
Monkey iBCI
Timeboi
Figure 1: Nonstationarities in long-term iBCL (A) Examples of iBCI systems in human and non-human primates. Spiking activity is recorded from multichannel electrode arrays together with behavior covariates, c.g.. 7 degree-of-freedoon upper limb. Neural activity exhibits nonstationarities over recording sessions. (B) Systematic changes in neuron positions, including the introduction or loss of neurons in the vicinity of electrodes and the shifts of the entire electrode array can contribute to instability of neural recordings over time. Illustrations were created with BioRender.com.
robotic arm control or electromyography from the
Cross-session neural alignment methods: Neural nonstationarities pose challenges for maintaining decoder performance over long periods of time. To tackle this issue, alignment techniques were proposed to align the testing sessions to match the distribution of a training session, usually performed in the latent space. These techniques vary from linear methods using Canonical Correlation Analysis
(CCA) 14 [0)] linear stabilizer 233 linear distribution alignment (24, to nonlinear using generative
adversarial networks (GAN) [50 229] RNN
(25] 7 [5]], and diffusion models [52]. All these approaches, however, still require explicit alignment procedures with model parameter updates and test-time labels in some cases to adapt the pretrained decoder to unseen populations. On the other hand, our proposed context-dependent positional embedding scheme allows flexible, gradient- adaptation in sessions without any labels.
Recent advances in foundation models for neural decoding demonstrate promising transfcrability across sessions, subjects and tasks through pretraining on large and diverse corpora of neural recordings (2 28 22
Foundation models for neuroscience:
47 48 53 49). While these approaches aim to build generic pretrained models that are then adapted to diverse downstream settings via parameter finetuning, SPINT instead focuses reducing calibration burden in weal-world iBCI systems by achieving gradient-free cross -eoss-session generalization under the limited data regime. SPINT, however, also ingests data from multi-session recordings during training to achieve this goal. Architecturally, SPINT employs the cross-attention mechanism to integrate information across variable number of tokens similar POYO-style models 222
47 48 while also operates binned spike counts similar to NDT-style frameworks [2 28 53 and POYO+ in spirit 47 Unlike these models, SPINT adopts much shallower design-une using only single cross-attention layer-and tokenizes data by treating each neural unit's 's temporal context window as token, analogous to the spatial tokenization in [46]] (4), and distinct from the temporal tokenization in 44 55), patch-based tokenization (1]. or spike-based based tokenization in [22 48)
Permutation-invarian neural networks for set-structured inputs: While conventional neural networks are designed for fixed dimensional data instances, in many set-structured applications such as point cloud object recognition or image tagging, the inputs have no intrinsic ordering, advocating for a class of models that are permutation-invarti DeepSets, introduced set average pooling approach serving as a universal approximator for any set function (56). Follow-up works [57 58 extended this pooling method to include max-pooling and attention mechanisms (59) We take inspiration from these works to design our universal neural behavior decoding framework, leveraging the permutation- invariant property of the cross-attention mechanism [60].
by design (7]. One such work,
unit identifier and permutation-invarit
3

---

M-shot neural calibration trials
Neural ID encoder
Trial-wise pooling
Learnalle behavior queries
000
Window
Last step prediction
Cross- attention
8mo
Behavior covariates
Unordered neural activity
Context-dependent neural ID
ID-informed neural activity
Figure 2: SPINT architecture. The model performs continuous behavioral decoding by predicting behavior covariates at the last timestep given past window of activity from an unordered set of neural units. The universal Neural ID Encoder infers identities of the units using few-shot unlabeled calibration trials, while the cross-attention mechanism selectively aggregates information from the units to decode behavior.
3 Approach
3.1 A permutationn-nnariinn framework for few-shot continuous behavioral decoding
We study the problem of real-time, cross-session iBCI decoding, where behavior needs to be decoded in causal manner from continuous stream of neural observations. Concretely, within a single session s, let Xi, denote the binned spiking activity of neural unit i at time I, X.:: denote all the activity of unit i, and X., denote the activities of all units at time Given a past observation window of population activity X X E NR where N₂ is the number of recorded neural units in session and W is the length of the observation window, we aim to estimate the corresponding last time step of behavior output Y; E BR where the dimensionality of behavior covariates. Model parameters are fitted with gradient descent using labeled data from & training (held-in) sessions and evaluated on testing (held-out) sessions without gradient updates or labels. At our disposal on each held-out session a short calibration period X1,1c] E TRT'' consisting of M-shot variable-length trials lasting for T' timesteps, be used for cross-session adaptation.
Traditional approaches consider the population activity at each timestep as a "token" and decode behavior by modeling temporal dynamics of population activity [39
By treating temporal snapshots of population activity X E IR as input vectors, these approaches assume fixed number and order neural units, requiring explicit spatial re-alignment when applied to another session with different size and order 27
27 44
25 29 20 24]. Recent methods incorporating factorized spatial- face similar challenges, while approaches with explicit spatiotemporal still require fine-tuning unit identity in novel sessions. These design choices hinder out- oft-of-the-bor generalizability of neural decoders across sessions, as a universal decoder should ideally be invariant to the permutation and size of the input population.
temporal modeling
45
tokens [21] 28
47
48
To realize this goal, we treat windows of individual neural units X,,-w+1: E R as an unordered set of tokens and aggregate information from these units to decode behavior using the cross-attention mechanism (59). compensate for the loss consistent order that the decoder can leverage for behavior decoding. we embed notion of neural identity each unit based on its spiking signature during few-shot, unlabeled calibration period X. in the same session. X. is either provided in limited amount in held-out sessions at test time, or is artificially sampled from held-in sessions

---

during training. This context-dependeet neural identity is inferred by universal neural identity encoder that shared across units and sessions, enabling gradient-free adaptation novel population compositions at test time.
3.2 Encoding identity of neural units
Let X E RMX be the trialized version of Xx RTT. X, is the collection of M calibration trials of neural unit interpolated to fixed length T'. We infer neural identity E; E RW of unit i by neural network [DEncoder:
E, IDEncoder(X;) ([ppol(((X())) (xxx)))
(I)
where to and ó are multi-layer fully connected networks and pool is the mean pooling operation across M trials. Due the permutation invariant nature of the mean operation and the fact that e and á are applied trial-wise, IDEncoder is invariant to the order of M calibration trials by design [56].
3.3 Decoding behavior via selective aggregation of information from neural population units
After inferring the identity for each unit from its calibration period, add E, to all X; windows to form identity-aware representations Z. Zi contains the time-varying activity of each unit while also being informed of the unit's stable identity within session. In matrix form:
Z - X + E
where Zs's, X₁'s, E,'s constitute rows of the Z, X, matrices.
We leverage the cross-attention mechanism to selectively aggregate information from population units and decode behavior outputs:
QK
Y CrossAttm (Q. softmax
V
(3)
VTx
ZWv E R. are projections of the identity-ifformed neural activity Z, and Q E R RBxW is a Iearnable matrix to query the behavior from Z. Q is initialized at random and learned during training. In future work, SPINT can be extended to accommodate multi-task setup by having multiple Q matrices, one for each behavior task, as in [47) We use the standard cross-attention module with pre- pre-normalization and feedforward layers (see details in Appendix).
where K ZWK,
Proposition 1. Cross-attention with idemtity-informet neurul activity (Equation permutation of neural units, i.e.,
is invariant to the
CrossAttn Q, Z, Z) = (q,
(4)
where PR is the row permutation matrix. (See proof in Appendix).
E in Equations 2 and 3 can be understood as a special kind of positional embedding for attention mechanism, where E is equivariant to the order of tokens (neural units), i.e., permuting the rows in X also permutes the rows E accordingly. Hence, unlike traditional positional embeddings in the transformer literature where positional embeddings are fixed entities, our proposed E is context dependent. This context-dependent positional embedding enables cross-session generalization by design, as F is stable for all samples within the same context (session), and can readily adapt in gradient-free manner new populations with arbitrary size and order.
After cross-attention, we project down by a fully connected layer to a one-dimensional vector representing the predicted behavior covariates at the last timestep, based on which we compute the mean squared error (MSE) between the predicted and the ground truth behavior covariates. The IDEncoder and cross-attention module are trained in end-to-end manner using this MSE objective.
5

---

3.4 Encouraging model robustness to inconsistent population composition
Neural population distributes its computation among many neural units, allowing us effectively decode behavior even though we can only record neural activity with a limited number of electrodes. Leveraging this insight and in order to encourage model robustness to different compositions of neural membership across recording sessions, we employ dynamic channel dropout to avoid overfitting to the population composition seen during training. Unlike classical population dropout methods where or a variable fraction of neurons/timesteps sampled from a conservative range [64] 22 is removed during training, we uniformly sample a dropout rate between O and 1 each training iteration and remove population units with the sampled dropout rate. With dynamic channel dropout, we not only encourage the model to be robust the unit membership but also encourage it to be robust to the size of the population, leading to improved cross- sooss-session generalization without additional tuning of dropout rates (see Section 4.8 and Figure
only a fixed fraction of neurons/timesteps 66
63
44
3.5 Gradient-free, few-shot cross-session adaptation in unseen neural populations with variable size and order
The overall framework is depicted in Figure| 2 We use labeled data from all training sessions to train all model parameters following the above pipeline. The model naturally digests populations of arbitrary size and order in all sessions without any need of session-specific alignment layer or fixed positional embeddings for neural units, hence having the potential to scale up to large amount of data. When testing on a held-out session, we reuse the trained IDEncoder and only need a few unlabeled calibration trials to infer identities of neural units in the test session, without the need of gradient descent updates to fine-tune session-specific alignment layers or unit/session embeddings. With these benefits, our proposed model removes the time and computation overhead usually required for re-calibrating neural decoders before each session, and facilitates its applicability in real-world iBCI settings where test-time labels are inherently unavailable.
4 Experiments
4.1 Datasets and evaluation metrics
We evaluate our approach on three continuous motor decoding tasks from the Few-shot Algorithms for Consistent Neural Decoding (FALCON) Benchmark (26). Specifically, we evaluate SPINT on the MI, M2, and HI datasets. In MI, monkey reached to, grasped, and manipulated an object in a variety of locations (4 possible objects, 8 locations), while neural activity was recorded from precentral gyrus and intramuscular electromyography (EMG) was recorded from 16 muscles [65] 66 67 68). In M2, a monkey made finger movements to control a virtual hand and acquired cued target positions while neural activity from the precentral gyrus and 2-D) actuator velocities were captured [69] In H1,
human subject attempted to reach and grasp with their right hand according to a cued motion for a -f flegree-of-rreedom freedomm robotic arm control 3 70 71). Each dataset comprises multiple labeled held-in sessions used to train the decoder (spanning 4, 4, and 6 days for MI, M2, and HI, respectively), and multiple held-out sessions for model evaluation (spanning 3, 4, and 7 days for MI, M2, and HI, respectively). Each held-out session provides a few public calibration trials (with optional labels) used for decoder calibration, after which the decoder is evaluated on a private test split. Cross-session performance is quantified by the mean and standard deviation of R² between the predicted and ground truth behavior covariates across all held-out sessions. All evaluation results were obtained on the held-out private split by submitting models to the EvalAI platform (72).
4.2 Baselines
We compare SPINT with zero-shot (ZS) and few-shot unsupervised (FSU) baselines, since SPINT the intersection of these two approaches. Similar FSU approaches, SPINT makes use of a few unlabeled calibration samples in the held-out sessions; however, unlike conventional FSU approaches. SPINT does not require gradient updates for model parameters at test time, therefore bearing resemblance to ZS methods in terms of practical utility. We call this new class of model gradient-free few-shot unsupervised (GF-FSU).
ZS Wiener Filter and 7. RNN: Wiener Filter is a linear model that predicts the current behavior as weighted sum of previous timesteps 73). In addition to the Wiener Filter, we also compare with

---

Class
M1
M2
H1
Wiener Filter (WF)
OR
0.53 + 0.04
0.26 + 0.03
0.21 0.04
RNN
OR
0.75 + 0.05
0.56
0.44
NDT2 Multi 21
OR
0.78 0.04
0.58 #.08
0.63
NDT2 Multi (1
FSS
0.59 0.07
0.43
0.52
WF
7S
0.34 0.06
0.06
0.16 0.03
RNN
ZS
-0.40
0.09 0.20
CycleGAN + WF [29]
FSU
0.43 0.04
0.06
0.02
NoMAD + WF [5]
FSU
0.49 0.03
0.20
0.13
SPINT (Ours)
GF-FSU
0.66 + 0.07
0.26 + 0.13
0.29 - 0.15
Table Performance comparison against oracles (OR), few-shot supervised (FSS), few-shot unsuper- vised (FSU), and zero-shot (ZS) methods. Our SPINT approach belongs to a special class which we termed gradient-free few-shot unsupervised (GF-FSU), where models perform adaptation based on few-shot unlabeled data but without any parameter updates at test time. Results are reported as mean + standard deviation R2 across held-out sessions.
simple RNN baseline (implemented as an LSTM 74).. The WF and RNN models were fitted using a single held-in session and evaluated zero-shot on the held-out sessions.
An FSU method where Generative Adversarial Network (GAN) is trained using calibration data from held-out session (day K) to transform day K's population activity form resembling activity from a held-in session (day 0), allowing decoders pretrained on day 0 to be reused on day
CycleGAN 129
NoMAD 25] Another FSU method where a dynamical model and a decoder are trained on day to predict behavior from the inferred dynamics. Then on day K, alignment network is trained to match the distribution of neural latent states to that of day allowing the fixed model and decoder to transfer to day K.
Wiener Filter, RNN and Transformer Oracles (OR): We include the Wiener Filter, RNN, and NDT2 a transformer for neural data [21), trained on private held-out labeled data to serve as upper bounds for model performance.
NDT2 Multi (FSS) [21]: Similar to NDT2 Multi OR, but only trained on held-in and held-out few-shot calibration data with supervision.
4.3 SPINT outperforms zero-shot and few-shot unsupervised baselines continuous motor decoding tasks
We show in Table the performance of SPINT in comparison with ZS and FSU approaches. SPINT outperforms all ZS and FSU baselines across all three datasets, while requiring no retraining or fine-tuning of model parameters. Improvement is most prominent in MI. where the amount of training data the largest (~5x more data than HI and ~6x more data than M2 in terms of recording time). Notably, SPINT surpasses Wiener Filter oracles in all datasets, which private labeled data. SPINT even outperforms the FSS method NDT2 Multi on M1 dataset while unlike NDT2, it does not require access to test-time labels model parameter updates. As we focus on cross-session transferability, all our experimental results show the cross-session performance. We include comparison on within-session performance in the Appendix.
trained with access the
4.4 SPINT requires only minimal amount of unlabeled data for adaptation
To gauge the data efficiency of our model at test time, we trained and tested the model with varying number of calibration trials used to infer the neural unit IDs. We show in Figure ([)) that SPINT could achieve reasonable cross-sessie sessisn generalization with a small number few-shot trials. In M1 dataset, the model could even achieve similar performance as the best model (which uses all available calibration trials) with only one single trial. This study demonstrates the practical utility of SPINT in online iBCI, relieving the burden of data collection and label collection users test time.
7

---

M1
M2
H1
(A)
0.81
0.6
2 0.4
12
0.2
R4 0.2
0.2
0.1
0.0
6
8
10
13
25
33
Number of callbration trials
Number of calibration trial
Number calibration trias
(B)
0.4
0.4
R2 0.6
-0.2
0.0
training days
Number training days
Number training
(C) 0.8
0.4
0.4
R
0.2
0.2
60
40
population szz (%)
Neural population size
population
Figure 3: Scaling analyses. Cross-session performance of SPINT against number of calibration trials (A), training days (B), and population sizes (C) across M1, M2, and H1 datasets. Bars represent mean R2 across held-out sessions, whiskers represent standard error of the mean of R² across held-out sessions.
4.5 SPINT performance scales well with the amount of training data
Thanks the flexible coamuutation-nvaaiian transformer network and the context-dependent positional embeddings, SPINT can ingest populations with arbitrary sizes and orders. These design choices give SPINT the ability to scale naturally with large amounts of training data. We demonstrate this scaling ability in Figure (B), where we observe clear trend cross-session performance when we data from more held-in days to train the model, with the best performance achieved when using all available training data on each dataset. This ability suggests potentials of SPINT as a large-scale pretrained model for iBCI when trained on larger datasets beyond the FALCON benchmark.
4.6 SPINT is robust to variable population composition
Our proposed dynamic channel dropout encourages robustness of SPINT variable input population size and membership. We test this robustness by training SPINT on the full held-in populations with dynamic channel dropout and evaluating variable-sized held-out populations (Figure C At each evaluation batch, we randomly sample a subset of the original population and measure the R2 obtained when the model makes predictions based on this limited subset. We observe robust performance in MI with reasonable performance drop when the population gets increasingly smaller, with the model still achieving mean held-out R. of 0.52 when only 20% of the original population remains, outperforming other ZS and FSU baselines with the full population.
4.7 SPINT maintains low-latency inference for iBCI systems
A critical consideration in iBCI system deployment is the ability of the system to perform behavior decoding in real time. We designed SPINT with this consideration in mind, using only one layer of cross-attention and two three-layer fully connected networks for IDEncoder. In Table 2 we report the latency achieved by SPINT as compared other methods. Latency is defined as the amount of time method requires to process the evaluation data divided by the duration of the evaluation data (72). The ratio less than signifies the approximation to real-time iBCI inference. SPINT achieves 13 latency on MI and M2, and 0.14 latency on HI, matching or outperforming transformer baselines, while being significantly below In practice, SPINT could be potentially faster in terms of deployment time, as it climinates the need for explicit alignment step required by conventional iBCI systems.

---

Class
M1
M2
H1
Wiener Filter (WF)
OR
0.06
0.08
0.14
RNN
OR
0.04
0.04
0.08
NDT2 Multi
OR
0.15
0.10
2.29
NDT2 Multi
FSS
0.13
0.10
0.30
WF
7S
0.06
0.08
0.15
RNN
ZS
0.03
0.01
0.02
CycleGAN + WF
FSU
0.07
0.09
0.16
NoMAD + WF
FSU
0.99
0.91
1.03
SPINT (Ours)
GF-FSU
0.13
0.13
0.14
Table 2: Inference latency of SPINT against oracles (OR), few-shot supervised (FSS), few-shot
unsupervised (FSU) and zero-shot (ZS) methods on held-out sessions (lower is better).
M1
M2
H1
(A)
0.8
Contast-dependent DD
0.21
Context dependent ID
Context-dapendent
Absolute PE
Absolute PE
Absolute PE
Without PE
Without
Without PE
-0.3
0.0
-0.8
-0.1
(B)
Dynamic dropout
Dynamic dropoul
Dynamic dropout
No dropout
No dropout
No
0.3
0.2
8:0.4 0.4
0.2
0.1
0.0
0.0
0.0
Figure 4: Ablation Study. Analyses showing the critical roles our proposed context-dependent ID against fixed positional embeddings (PE) and no positional embeddings (A), our dynamic channel dropout against no dropout (B). Results are shown across M1, M2, and H1 datasets. Bars represent mean 1² across held-out sessions, whiskers represent standard error of the mean of R2 across held-out sessions.
4.8 Ablation Study
We perform ablation studies to demonstrate the benefits of our context-dependent positional embed- dings and dynamic channel dropout techniques. In [ we compare our context-dependent positional embeddings with fixed (absolute) positional embeddings used in the vanilla transformer [59], and with no positional embeddings. The conventional fixed positional embeddings break property of the cross-attention mechanism, thus not able to generalize to populations with different compositions in held-out sessions. With no positional embeddings, the model is permutation-invarte
by design; however, the loss of information about neural unit functional identities hinders the model's ability to decode the behavior these units encode. We achieve the best of both worlds by our proposed context-dependent positional embeddings, being both permutation invariant while retaining neural identities for behavior decoding.
To demonstrate the effectiveness of our proposed dynamic channel dropout technique. we compare the cross-session performance of SPINT with dynamic channel dropout and without dropout. We show in Figure 4B that dynamic channel dropout serves as an effective regularization technique by preventing the model from overfitting to the population composition in training sessions.
Discussion
In this work, we introduce SPINT, a permutatioon-nnvarian transformer designed for cross-session intracortical motor decoding. SPINT features context-dependent neural ID embeddings that dynamically infers unit identities, allowing the model to handle unordered, variable-sized populations

---

across sessions. These components enable SPINT to adapt to new sessions using only small amount of unlabeled calibration data, with no need for test-time labels or gradient updates. We demonstrate SPINT's cross-session generalization ability on three continuous motor decoding tasks from the FALCON Benchmark, where consistently outperforms existing zero-shot and few-shot unsupervised baselines, even surpassing few-shot supervised and oracle models in some instances. SPINT's light- weight design also enables low-latency inference, making it well-suited for world iBCI applications.
Our study represents an initial attempt at a flexible, gradient-free framework for consistent behavioral decoding and opens up several promising avenues for future research. Within the framework of the FALCON Benchmark, we demonstrated that our approach scales effectively with increasing amount of training data, albeit from single subject and behavior task. Exploring the applicability of our approach under more diverse settings (cross-task, cross-region, cross-subject) and on non-motor behaviors is an important direction for future research 44 end-to-end training for the unit identifier together with the rest of the network ties the identification of the neural identities to task-specific behavioral decoding and thus requires access to labels to train the unit identifier, which might be limited in real-world iBCI settings. Disentangling the training of the unit identifier with behavioral decoding by means of self-supervised approaches such as contrastive or predictive learning can potentially alleviate the reliance on behavior labels for this stage, albeit the cost of additional training overhead and potential loss of behavior-relevant neural features essential for downstream decoding.
48 49]. On the technical side, our
Lastly, although our study shows robust and low-latency behavioral decoding in silico, its performance remains to be evaluated in vivo, where sensory feedback from closed-loop control may induce modulation in the neural activity inputs and hardware-specifie constraints could affect the online inference latency [75]. To this end, our preliminary assessment represents an initial step toward a light- plug-and-play decoding framework for long-term iBCI systems.
6 Acknowledgment
The authors acknowledge the partial support of HDR Institute: Accelerated AI Algorithms for Data-Driven Discovery (A3D3) National Science Foundation grant PHY-2117997 (TL, HF, JL, AO, ES) and EFRI-BRAID-211 2223492 (ES, TL). The authors also acknowledge the partial support by the Departments of Electrical Computer Engineering, Applied Mathematics and Bioengineering. The authors are thankful to the Center of Computational Neuroscience, the eScience Center at the University of Washington and the Allen Institute for Brain Science. TL and US wish to thank the founder of the Allen Institute, P.G. Allen, for his vision, encouragement, and support. The authors also thank the FALCON organizers Joel Ye, Brianna Karpowicz, Chethan Pandarinath, Thiago Scodeler, and the EvalAI team for their technical support with the datasets and submissions.
References
[1] Apostolos P Georgopoulos, Andrew B Schwartz, and Ronald E Kettner. Neuronal population coding of movement direction. Science, 233(4771):1416-1419, 1986.
[2] Leigh R Hochberg, Daniel Bacher, Beata Jarosiewiez, Nicolas Masse, John D Simeral, Joern Vogel, Sami Haddadin, Jie Liu, Sydney S Cash, Patrick Van Der Smagt, et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 485(7398):372-37, 2012.
[3] Jennifer L Collinger, Brian Wodlinger, John E Downey, Wei Wang, Elizabeth C Kabara, Douglas J Weber, Angus JC McMorland, Meel Velliste, Michael L Boninger, and Andrew B Schwartz. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet, 381 (9666:555776, 2013.
[4] Chethan Pandarinath, Paul Nuyujukian, Christine Blabe, Brittany L. Sorice, Jad Saab, Fran- cis R Willett, Leigh R Hochberg, Krishna V Shenoy, and Jaimie M Henderson. High perfor- mance communication by people with paralysis using an intracortical brain-computer interface. elife, 6:554 2017.
10

---

[5] Nishal P Shah. Matthew S Willsey, Nick Hahn, Foram Kamdar, Donald T Avansino, Chaofei Fan, Leigh Hochberg. Francis Willett, and Jaimie M Henderson. A flexible intracortical brain-computer interface for typing using finger movements. bioRxiv, 2024.
[6] Francis R Willett, Donald Avansino, Leigh R Hochberg, Jaimic M Henderson, and Kr- ishna V Shenoy. high-performance brain-to-text communication via handwriting, Nature, 593(7858):249-2555 254,, 2021.
[7] Chaofei Fan, Nick Hahn, Foram Kamdar, Donald Avansino, Guy Wilson, Leigh Hochberg, Kr- ishna V Shenoy, Jaimie Henderson, and Francis Willett. Plug-and-pha stability for intracortical brain-computer interfaces: a one-year demonstration of seamless brain-to-text communication. Advances in neural information processing systems, 36:42258-422770 2023.
[8] Francis R Willett, Erin M Kunz, Chaofei Fan, Donald T Avansino, Guy H Wilson, Eun Young Choi, Foram Kamdar, Matthew F Glasser, Leigh R Hochberg, Shaul Druckmann, et al. A high-performance speech neuroprosthesis. Nature, 620(7976):033110033 2023.
[9] Jingyuan Li, Trung Le, Chaofei Fan, Mingfei Chen, and Eli Shlizerman. Brain-to-text decoding with context-aware neural representations and large language models. Journal of Neural Engineering, 22(5):056026, 2025.
[10] Nicholas S Card, Maitreyee Wairagkar, Carrina lacobacci, Xianda Hou, Tyler Singer-Clark, Francis R Willett, Erin M Kunz, Chaofei Fan, Maryam Vahdati Nia, Darrel R Deo, et al. An accurate and rapidly calibrating speech neuroprosthesis. New England Journal of Medicine, 391(7):609-618, 2024.
Clark,
[11] Maitreyee Wairagkar, Nicholas S Card, Tyler Singer-Claie Clark, Xianda Hou, Carrina Iacobacci, Lee M Miller, Leigh R Hochberg, David M Brandman, and Sergey Stavisky, An instantaneous voice-synthesis neuroprosthesis. Nature, pages 1-8, 2025.
[12] Erin M Kunz, Benyamin Abramovich Krasa, Foram Kamdar, Donald T Avansino, Nick Hahn, Nicholas S Card, Justin J
Seonghyun Yoon, Akansha Singh, Samuel R Jude, et al. Inner speech in motor cortex and implications for speech neuroprostheses. Cell, 188(17):465844673 2025.
[13] Cynthia A Chestek, Vikash Gilja, Paul Nuyujukian, Justin D Foster, Joline M Fan, Matthew T Kaufman, Mark M Churchland, Zuley Rivera-Alvidrez, John P Cunningham, Stephen Ryu, et al. Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. Journal of neural engineering, 8(4):045005, 2011.
[14] Juan A Gallego, Matthew G Perich, Raeed H Chowdhury, Sara A Solla, and Lee E Miller. Long-term stability of cortical population dynamics underlying consistent behavior. Nature neuroscience, 23(2):260-2) 270-270, 2020.
[15] János A Perge. Mark L Homer, Wasim Q Malik, Sydney Cash, Emad Eskandar, Gerhard Frichs, John P Donoghue, and Leigh Hochberg. Intra-day signal instabilities affect decoding perfor- mance in an intracortical neural interface system. Journal of neural engineering, 10(3):036004, 2013.
[16] John E Downey, Nathaniel Schwed, Steven M Chase, Andrew B Schwartz, and Jennifer L Collinger. Intracortical recording stability in human brain-computer interface users. Journal of neural engineering, 15(4):046016, 2018.
[17] David Sussillo, Sergey D Stavisky, Jonathan C Kao, Stephen Ryu, and Krishna V Shenoy. Making brain-machine interfaces robust to future neural variability. Nature communications, 7114:1374, 2016.
[18] Thomas Hosman, Tsam Kiu Pun, Anastasia Kapitonava, John Simeral, and Leigh R Hochberg. Months-long pigf-perf fixed Istm decoder for cursor control in human intracortical brain-computer interfaces. In 2023 / 11th International IEFF/EMBS Conference on Neural Engineering (NER), pages 1-5. IEEE, 2023.

---

[19] Fabio Rizzoglio, Ege Altan, Xuan Ma, Kevin L Bodkin, Brian M Dekleva, Sara A Solla, Ann Kennedy, and Lee E Miller. From monkeys humans: observation-based emg brain comp-ce interface decoders for humans with paralysis. Journal of Neural Engineering, 20(5):060040, 2023.
[20] Mostafa Safaie, Joanna C Chang, Junchol Park, Lee E Miller, Joshua T Dudman, Matthew G Perich, and Juan A Gallego. Preserved neural dynamics across animals performing similar hehaviour. Nature, 623(7988):765-777 2023.
[21] Joel Ye, Jennifer Collinger, Leila Wehbe, and Robert Gaunt. Neural data transformer 2: multi- context pretraining for neural spiking activity. Advances in Neural Information Processing Systems, 36:80352-8033 80374, 2023.
[22] Mehdi Azabou, Vinam Arora, Venkataramana Ganesh, Ximeng Mao, Santosh Nachimuthu, Michael Mendelson, Blake Richards, Matthew Perich, Guillaume Lajoie, and Eva Dyer. A unified, scalable framework for neural population decoding. Advances in Neural Information Processing Systems, 36:44937-44956 2023.
[23] Alan D Degenhart, William E Bishop, Emily R Oby, Elizabeth C Tyler- Kabara, Steven M Chase, Aaron P Batista, and Byron M Yu. Stabilization of brain computer interface via the alignment of low-dimensional spaces of neural activity. Nature biomedical engineering, 4(7):672-6855 2020.
[24] Eva L Dyer, Mohammad Gheshlaghi Azar, Matthew G Perich, Hugo L Fernandes, Stephanie Naufel, Lee E Miller, and Konrad P Körding. A cryptography-baat approach for movement decoding. Nature biomedical engineering, 1(12):97-976. 2017.
[25] Brianna M Karpowicz, Yahia H Ali, Lahiru N Wimalasena, Andrew R Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma, Lee E Miller, and Chethan Pandarinath. Stabilizing brain-computer interfaces through alignment of latent dynamics. BioRxiv, pages 2022-04, 2022.
[26] Brianna Karpowicz, Joel Ye, Chaofei Fan, Pablo Dostado-Marcos Fabio Rizzoglio, Clayton Washington, Thiago Scodeler, Diogo de Lucena, Samuel aoson-Tomaszewski Matthew Mender, et al. Few-shot algorithms for consistent neural decoding (falcon) benchmark. Advances in Neural Information Processing Systems, 37:76578-76615, 2024.
[27] Chethan Pandarinath, Daniel J O' Ohea, Jasmine Collins, Rafal Jozefowicz, Sergey D Stavisky, Jonathan C Kao, Eric M Trautmann, Matthew T Kaufman, Stephen Ryu, Leigh R Hochberg, et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature methods, 15(10))805-815 2018
[28] Joel Ye, Fabio Rizzoglio, Adam Smoulder, Hongwei Mao, Xuan Ma, Patrick Marino, Raeed Chowdhury, Dalton Moore, Gary Blumenthal, William Hockeimer, et al. A generalist intracorti- cal motor decoder. bioRxiv, pages 2025-02, 2025.
[29] Xuan Ma, Fabio Rizzoglio, Kevin L Bodkin, Eric Perreault, Lee E Miller, and Ann Kennedy. Using adversarial networks to extend brain computer interface decoding accuracy time. elife, 12:e84296, 2023.
[30] Liam Paninski. Maximum likelihood estimation of cascade point-process neural encoding models. Network: Computation Neural Systems, 15(4):243, 2004.
[31] Wei Wu, Yun Gao, Elie Bienenstock, John Donoghue, and Michael Black. Bayesian population decoding of motor cortical activity using kalman filter. Neural computation, 18(1):80-118, 2006.
[32] Tobias Pistohl, Tonio Ball, Andreas Schulze-Bonhager Ad Aertsen, and Carsten Mehring. Pre- diction of arm movement trajectories from recording in humans. Journal of neuroscience methods, 167(1)105-1114 2008
[33] Wei Wu, Jayant E Kulkarni, Nicholas G Hatsopoulos, and Liam Paninski. Neural decoding of hand motion using a linear state-space model with hidden states. IEEE Transactions on neural systems and rehabilitation engineering. 17(4):370-378, 2009.
12

---

[34] Wilson Truccolo, Uri T Eden, Matthew R Fellows, John P Donoghue, and Emery N Brown. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. Journal of neurophysiology. 93(2):1074-1089) 2005.
[35] Byron M Yu, Afsheen Afshar, Gopal Santhanam, Stephen Ryu, Krishna V Shenoy, and Maneesh Sahani. Extracting dynamical structure embedded in neural activity. Advances in neural information processing systems, 18, 2005.
[36] Jonathan W Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher, Alan M Litke, EJ Chichilnisky, and Eero P Simoncelli. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature, 454(7207):995 999, 2008
[37] Vernon Lawhern, Wei Wu, Nicholas Hatsopoulos, and Liam Paninski. Population decoding of motor cortical activity using a generalized linear model with hidden states. Journal of neuroscience methods, 189(2):677-280, 2010.
[38] Anqi Wu, Nicholas Roy, Stephen Keeley, and Jonathan W Pillow. Gaussian process based non- linear latent structure discovery in multivariate spike train data. Advances in neural information processing systems, 30, 2017.
[39] Byron M Yu. John P Cunningham, Gopal Santhanam, Stephen Ryu, Krishna V Shenoy, and Maneesh Sahani. Gaussian-procest factor analysis for low-dimensional single-trial analysis of neural population activity. Advances neural information processing systems, 21, 2008.
[40] Timothy D Kim, Thomas Z Luo, Jonathan Pillow, and Carlos D Brody. Inferring latent dynamics underlying neural population activity via neural differential equations. In International Conference on Machine Learning, pages 5551-5561. PMLR, 2021.
[41] Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617(7960))360- 368, 2023.
[42] Wei-Hsien Lee, Brianna M Karpowicz, Chethan Pandarinath, and Adam G Rouse. Identifying distinct neural features between the initial and corrective phases of precise reaching using autolfads. Journal of Neuroscience, 44(20), 2024.
[43] Shreya Saxena, Abigail A Russo, John Cunningham, and Mark M Churchland. Motor cortex activity across movement speeds is predicted by network-level strategies for generating muscle activity. Elife, 11:667620, 2022.
[44] Joel Ye and Chethan Pandarinath. Representation learning for neural population activity with neural data transformers. Neurons, Behavior, Data analysis, and Theory, 2021.
[45] Ran Liu, Mehdi Azabou, Max Jingyia Xiao, and Eva Dyer. Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformers. Advances in neural information processing systems, 35:2377-2391, 2022.
[46] Trung Le and Eli Shlizerman. Stndt: Modeling neural population activity with spatiotemporal transformers. Advances in Neural Information Processing Systems, 35: 17926 119226--7933 17939, 2022.
[47] Mehdi Azabou, Krystal Xuejing Pan, Vinam Arora, Ian Jarratt Knight, Eva L Dyer, and Blake Aaron Richards. Multi-session, multi-task neural decoding from distinct cell-types and brain regions. In The Thirteenth International Conference on Learning Representations.
[48] Avery Hee-Woon Ryoo, Nanda H Krishna, Ximeng Mao, Mehdi Azabou, Eva L Dyer, Matthew G Perich, and Guillaume Lajoie. Generalizable, real-time neural decoding with hybrid state-space models. arXiv preprint arXiv:2506.05320. 2025.
[49] Yizi Zhang, Yanchen Wang. Mehdi Azabou, Alexandre Andre, Zixuan Wang, Hanrui Lyu, The International Brain Laboratory, Eva Dyer, Liam Paninski, and Cole Hurwitz. Neural encoding and decoding at scale. arXiv preprint arXiv:2504.08201, 2025.
[50] Ali Farshchian, Juan A Gallego, Joseph P Cohen, Yoshua Bengio, Lee E Miller, and Sara A Solla. Adversarial domain adaptation for stable brain-machine interfaces. arXiv preprint arXiv:1810.00045, 2018.
13

---

[51] Ayesha Vermani, II Memming Park, and Josue Nassar. Leveraging generative models for unsupervised alignment of neural time series data. In The Twelfth International Conference on Learning Representations, 2023.
[52] Yule Wang, Zijing Wu, Chengrui Li, and Anqi Wu, Extraction and recovery of spatio-temporal structure in latent dynamics aligmment with diffusion models. Advances in Neural Information Processing Systems, 36:38988-3900 -90005, 2023.
[53] Yizi Zhang, Yanchen Wang. Donato Jiménez-Benetó, Zixuan Wang, Mehdi Azabou, Blake Richards, Renee Tung, Olivier Winter, Eva Dyer, Liam Paninski, et al. Towards a" universal translator" for neural dynamies at single-cell, snnll--pikke resolution. Advances in Neural Information Processing Systems, 0080000-852225 2024.
[54] Ziyu Lu, Wuwei Zhang, Trung Le, Hao Wang. Uygar Sümbül, Eric Todd SheaBrown, and Lu Mi. Netformer: An interpretable model for recovering dynamical connectivity in neuronal population dynamics. In The Thirteenth International Conference on Learning Representations, 2025.
[55] Lu Mi, Trung Le, Tianxing He, Eli Shlizerman, and Uygar Sümbül. Learning time-invariant rep- resentations for individual neurons from population dynamics. Advances in Neural Information Processing Systems, 36:46007-46026, 2023.
[56) Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
[57] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
[58] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutation-invarian neural networks. In International conference machine learning, pages 3744-3753. PMLR, 2019.
[59] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[60] Hengyuan Xu, Liyao Xiang, Hangyu Ye, Dixi Yao, Pengzhi Chu, and Baochun Li. Permutation equivariance of transformers and its applications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5987-5996, 2024.
[61] Jingyuan Li, Leo Scholl, Trung Le, Pavithra Rajeswaran, Amy Orsborn, and Eli Shlizerman. Amag: Additive, multiplicative and adaptive graph neural network for forecasting neuron activity. Advances in Neural Information Processing Systems, 36:8988-900 2023.
[62] Mohammad Reza Keshtkaran and Chethan Pandarinath. Enabling hyperparameter optimization in sequential autoencoders for spiking neural data, Advances neural information processing systems, 32, 2019.
[63] Mohammad Reza Keshtkaran, Andrew R Sedler, Raeed Chowdhury, Raghav Tandon, Diya Basrai, Sarah L Nguyen, Hansem Sohn, Mehrdad Jazayeri, Lee E Miller, and Chethan Pandari- nath. A large-scale neural network training framework for generalized estimation of single-trial population dynamics. Nature Methods, 19(12))1522-1577 2022.
[64] Mehdi Azabou, Mohammad Gheshlaghi Azar, Ran Liu, Chi-Heng Lin, Erik C Johnson, Kiran Bhaskaran-Nain WashU-St Louis, Max Dabagia, Bernardo Avila-Pires Lindsey Kitchell, et al. Mine your own view: A self-supervised approach for learning representations of neural activity.
[65] Adam G Rouse and Mare Schieber. Spatiotemporal distribution of location and object effects in reach-to -oaaggapp kinematics. Journal of neurophysiology, 1 (1466):368832288 2015.
14

---

[66] Adam G Rouse and Mare H Schieber. Spatiotemporal distribution of location and object effects in the electromyograppic activity of upper extremity muscles during reach-to- to-grop........ Journal of neurophysiology, 115(6)
2016.
[67] Adam G Rouse and Mare H Schieber. Spatiotemporal distribution of location and object effects in primary motor cortex neurons during reach-to to to--too---...... Journal of Neuroscience, 36(41):10640- 10653, 2016.
[68] Adam G Rouse and Marc H Schieber. Condition-dependent neural dimensions progressively shift during reach to grasp. Cell reports, 25(1 1):31583-1166, 2018.
[69] Samuel R Nason, Matthew J Mender, Alex K Vaskov, Matthew S Willsey, Nishant Ganesh Kumar, Theodore A Kung, Parag G Patil, and Cynthia A Chestek. Real-time linear prediction of simultaneous and independent movements of two finger groups using intracortical brain- machine interface. Neuron, 109(19):3164 3177, 2021.
[70] Wodlinger, JE Downey, EC Tyler-Kabara, AB Schwartz, ML. Boninger, and Collinger. Ten-dimensional anthropomorphic arm control in a human brain- machine interface: difficulties, solutions, and limitations. Journal of neural engineering, 12(1):016011, 2014.
[71] Sharlene N Flesher, John Downey, Jeffrey M Weiss, Christopher L Hughes, Angelica Herrera, Elizabeth TTyler-Kabare Michael L Boninger, Jennifer L. Collinger, and Robert A Gaunt. A brain-computer interface that evokes tactile sensations improves robotic arm control. Science, 372(6544):831-833 2021.
[72] EvalAI.
Falcon benchmark challenge.
https://eval. http//eebbhhaallengee
challenger wwtppaeee/e//alaaaeeeeeae 2025. Accessed: 2025-05-12.
[73] Norbert Wiener. Extrapolation, interpolation, and smoothing of stationary time series. The MIT press, 1964.
[74] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, ((8):1735-178)) 1997.
[75] Amy L Orsborn, Helene G Moorman, Simon A Overduin, Maryam M Shanechi, Dragan F Dimitrov, and Jose M Carmena. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron, 1286):330-11333 2014.
[76] Diederik Kingma. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We describe the limitations of the work in the Discussion section.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide the proof the main paper's proposition in the Appendix section.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: Implementation details for reproducing experimental results are provided in the Appendix.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The data used is already publicly available. We will make the code publicly available upon acceptance of the paper.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Implementation details for reproducing experimental results provided in the Appendix.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide error bars in all of our barplots.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Implementation details for reproducing experimental results are provided in the Appendix.
16

---

9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://///aaarr....co///a//////a
Answer: [Yes]
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
14. Crowdsourcing and research with human subjects
Question: crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
17

---

A Appendix
A.1 Within-session performance comparison
We include the within-session performance comparison between SPINT and baselines in Table A This table is similar Table 1 in the main paper, but with metrics obtained EvalAI's private splits within the held-in in sessions. As observed from the table, SPINT also consistently outperforms ZS and FSU baselines on the held-in splits.
Class
M1
M2
H1
Wiener Filter (WF)
OR
0.54
0.27 0.02
0.24 0.02
RNN
OR
0.75
0.59
0.59
NDT2 Multi 21
OR
0.00
0.62
0.65
NDT2 Multi [2]
FSS
0.00
0.63
0.62
WF
ZS
0.46
0.15
0.20
RNN
ZS
0.52
0.20
0.31
CycleGAN + WF 29
FSU
0.61 0.02
0.32
0.04
NoMAD WF 25
FSU
0.64
0.05
0.21 0.06
SPINT (Ours)
GF-FSU
0.77 + 0.02
0.59 0.01
0.47 + 0.06
Table Al: Within-session performance comparison against oracles (OR), few-shot supervised (FSS), few-shot unsupervised (FSU), and zero-shot (ZS) methods. Our SPINT approach belongs to a special class which termed Gradient- Free Few-Shot Unsupervised (GF-FSU), where models perform adaptation based on few-shot unlabeled data but without any parameter updates at test time. Results reported mean + standard deviation R² across held-in sessions, achieved on EvalAI private held-in splits.
A.2 Proof of SPINT's permutation-innaaiine
Let PR. Pe be the row and column permutation matrices of the same permutation π (Pc = PR P and POPR I). Also let X' PRX and (xCy- = P be the row-permuted neural windows and row-permuted calibration trials.
Since the ID embedding of each neural unit is computed individually from the set of calibration trials for that unit:
E, IDEncode = e (o(x
(A1)
permuting the neural units in the original population (neural windows X or calibration trials X will permute the embedding matrix E in the exact same order, i.e., = Pr..
It follows that:
Z'=X'+E' P PR(X + E) PRZ
(A2)
In other words, Z is equivariant to the permutation of neural units.
Cross-attention performed on Z' then becomes:
CrossAttn (Q, ZzZz" CrossAttn(Q (Q, PRZ, PRZ)
QW P.
softmax
PrZWv
Vdx
QW Pc
softmax
PREWV
Vdk
(A3)
QWIZT
softmax
PePRZWv
Vdz
QWTZ
softmax
ZWv
Vdz
CrossAttn(Q,22 2,
18

---

QW 22
softmax @Q x
Pe because an element always normalized with the same group of elements in the same row regardless of whether column permutation is performed before or after softmax.
where softmax Vidy
Vdd
Equation A3 concludes Proposition 1 in the main paper.
We note that multi-layer perceptron (MLP), layer normalization, and residual connection are ap- plied row-wise and hence do not affect the overall permutation-invariance property of our SPINT framework.
A.3 Correlation of attention scores and firing statistics
We ask whether the attention scores SPINT assigns for each neural unit are correlated with its firing statistics. To answer this question, in each held-out calibration window, we measure the average attention scores over behavior covariates, and its firing statistics (mean/standard deviation) over the held-out calibration trials, then calculate the Pearson's 's correlation between these two quantities using all held-out calibration windows. We show the results Table
We observe that the attention scores correlate moderately with the mean and the standard deviation of the neural unit's firing rates, with higher correlation for the standard deviation than the mean, suggesting that SPINT might be extracting neural units that are active (having high mean firing rates) and behaviorally relevant (having high variance throughout the calibration periods where behavior varied) to pay attention to in behavioral decoding.
M1
M2
HI 0.51 + 0.04 0.57
p attentioo scores, mean firing rates) p (attention scores, standard deviation of firing rates)
0.33 + 0.16
0.76 = 0.03
0.45 0.16
0.87 + 0.02
+ 0.03
Table A2: Pearson's correlation between attention scores for each neural unit and that unit mean/standard deviation of firing rates during the held-out calibration periods. Results are re- ported as the mean correlation + standard deviation across held-out sessions. All p-values are less than 0.05.
A.4
Implementation details
A.4.1 Data preprocessing
For neural activity, we use the binned spike count obtained by unit threshold crossing with the standard bin size of 20ms set forth by the FALCON Benchmark. We follow FALCON's continuous decoding setup for all three M1, M2, and H1 datasets, where rather than decoding trialized behavior from the trialized neural activity (often performed in non-causal manner), the last step of a neural activity window, mimicking the online, causal iBCI decoding. To construct the length-W neural window the beginning of each session, we pre-pad the session neural time series with (W - 1) zeros. We discard the windows whose last time step belongs to a non-evaluated period as defined by FALCON, e.g., inter-trial periods where there is registered kinematics.
decode behavior
Our IDEncoder infers neural unit identity from trialized calibration trials. As calibration trials vary in length, we interpolate all calibration trials the same length T, where T 100 for M2 and T = 1024 for MI and H1. We use the Python library iiaayyinnerpoolett interppi with a cubic spline for interpolation. Note that only perform interpolation for neural calibration trials to synchronize their trial lengths. We still use the raw spike counts for the neural windows, conforming with the continuous decoding setup.
A.4.2 Behavior output scaling
For M2 and HI, since values behavior covariates are relatively small, during training we scale the network behavior predictions by factor of 0.2 and 0.05 for M2 and H1, respectively, effectively asking the model to predict 5x and 20x the original behavior values. The MSE loss and R² metrics computed between the scaled predicted outputs and the original ground truth values.
19

---

Dropout
0
0.2
0.4
0.6
0.8
DD [0,1]
R
0.51 0.13 0.62 + 0.10 0.63 + 0.10 0.63 = 0.10 0.60 = 0.09
0.64 = 0.10
Table A3: SPINT's cross- session performance against dynamic dropout and different choices of fixed
dropout rates. Results are reported as mean + standard deviation across held-out calibration sessions. DD [0,1] stands for dynamic dropout with variable dropout rates between and 1.
DD range
10, 0.1]
[0, 0.2]
[0, 0.3]
[0, 0.4]
[0, 0.5
[0.1]
R
0.59 + 0.07 0.59 + 0.07 0.61 + 0.10
0.62 + 0.07
0.63 + 0.07 0.64 + 0.10
Table A4: SPINT's cross-session performance across different ranges of dynamic dropout. Results reported as mean + standard deviation across held-out calibration sessions.
# heads
16
32
64
R 0.62 0.08 0.63 0.09
0.64 + 0.10
0.65 0.11
0.64 + 0.10
Table A5: SPINT's cross-session performance for different rross-attention head counts. Results are reported mean = standard deviation across held-out calibration sessions.
# self-attention layers R
0
1
2
3
4
0.64 + 0.10 0.63
0.57 = 0.13
0.61 0.10
0.60 = 0.15
Table A6: SPINT's cross-session performance for different number of self-attention layers. Results are reported as mean + standard deviation across held-out calibration sessions.
# cross-attention layers R
2
3
4
5 0.62 + 0.13
0.64 + 0.10 0.65 + 0.10 0.65 + 0.10 0.64 + 0.11
Table A7: SPINT's cross-session performance for different number of cross- aooss-attnttion layers. Results are reported as mean + standard deviation across held-out calibration sessions.
Window size
50
100
200
400
600
R
0.65 + 0.10
0.64 + 0.10
0.64 + 0.10
0.60 + 0.10
0.61 + 0.09
Table A8: SPINT's cross-session performance for different context window sizes. Results are reported as mean + standard deviation across held-out calibration sessions.
A.4.3 Inferring neural unit identity
We follow the permutation-invaree invariant framework in (56) for inferring identity E, of neural unit i:
E (XX)) MLP2
M Σ (MLP ( (X))))
(A4)
where M is the number of calibration trials, X is the neural activity of the 7th calibration trial of neural unit i, MLP, and MLP2 are two 3-layer fully connected networks. MLP; projects the length- T' trials hidden dimension H, and MLP2 projects the length-// // hidden features length-W neural unit identity output.
A.4.4 Behavioral decoding by cross-attention
After neural identity for all units E is inferred, we add it to the neural window input X to form the identity-awer aware neural activity Z, i.e., Z X E. We then use the cross-attention mechanism in the latent space to decode last step behavior covariates. Specifically:
Zin MLP (Z)
(A5)
3. = Q + Cross Attn(Q Layer Z7n)) Layer Zin))
(A6)
Zont = Ž MLP atth < (TayerNor((2))
(A7)
Y = MLP out (Zout)
(A8)
20

---

A.4.5 Hyperparameters
We include the notable hyperparameters used to optimize SPINT in Table A9 We train and evaluate models for each MI, M2, and HI dataset separately. We train the models using all available held-in sessions and evaluate on all available held-out sessions. We use Adam optimizer 76 for all training.
M1
M2
H1
Batch size
32
32
32
Window size
100
50
700
Max trial length
1024
100
1024
Number of IDEncoder layers
3.3
3,3
3,3
Number of cross attention layers
Hidden dimension
1024
512
1024
Behavior scaling factor
0.2
0.05
Learning rate
le-5
5e-5
le-5
Table A9: Hyperparameters used train SPINT the M1, M2, and HI datasets.
We include representative hyperparameter sweeps demonstrating SPINT's robustness to hyperparam- eter choices Tables A3
A8 This robustness allows SPINT effectively capture long-range context while maintaining a minimalist architecture without compromising generalizabil- ity. All sweep results were obtained on 20% of calibration trials held out from each session of the M1 dataset rather than on the EvalAl test split.
A4
A5
A6
A7
A.4.6 Computational resources
SPINT was trained using a single A40 GPU, consuming less than 2GB of GPU memory with batch
size of 32 and taking around 12 hours, hours, and 8 hours finish 50 training epochs for MI, M2, and HI, respectively. We select checkpoints for evaluation at epoch 50 in all M1, M2, and H1 datasets.
21


Reviews:
[{"review_id": "ksdv8J7SXU", "reviewer": "Reviewer_fpRR", "summary": "The authors propose a framework to address the variability and ordering in the placement of electrodes in the context of real-time, cross-session iBCI decoding and develop a spatial-permutation invariant neural transformer for decoding motor behavior from intracortical spikes. Spike windows are turned into tokens and aggregated via cross-attention.To compensate for the loss of positional information a neural ID encoder derives context-dependent embeddings from a handful of unlabeled \"calibration\" trials collected at the start of each new session. Next these embeddings are regularized by channel-dropout that removes an arbitrary fraction of units during training. The method has state of the art performance in three multi-day datasets from the FALCON benchmark. It also maintains real time inference without test-time gradients.", "questions": "1. Could contrastive or predictive objectives remove the need for labelled data for the neural ID encoder? \n2. Would a lightweight temporal transformer on top of the cross-unit attention benefit decoding behaviors with longer latencies?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Permutation invariant and unit agnostic pipeline: Removes the rigid channel ordering assumption enabling training and testing on different population of different size and composition.\n2. Context-dependent neural id embeddings. \n3. light weight , low-latency architecture: This is particularly significant for closed loop BCI. \n4. Consistently outperforms previous approaches on FALCON without test-time gradients. \n\nWeaknesses:\n1. Shallow architecture limits long range modeling. Moreover, Neural ID Encoder completely forgets the within-trial structure. \n2. Restricted evaluation to 1 human and 2 monkey and behavio limited to arm/hand tasks \n3. Gradient -free adaptation still needs session specific calibration data.", "was_revised": true, "final_justification": "After reading the authors' rebuttal to my questions and the other reviewers' questions I conclude that there are two important weaknesses in the paper. First, the setting is restrictive especially the fact that it requires calibration data that are session-specific hinders real world applications. Also, the architecture itself needs a lot of tuning to find windows that are able model long-range dependencies since there are no increasing receptive fields. Such validation parameter would require even more data! This last weakness is not apparent in the restricted experimentation of the paper since the windows have already been tuned. I will keep my score as is.", "rebuttal": "We thank the reviewer for a thoughtful review and valuable feedbacks. We provide point by point clarifications and answer questions below.\n\n**W1. re:** shallow architecture limiting long range modeling, neural ID Encoder forgetting the within-trial structure.\n\nWe understand the reviewer’s concern regarding the long range modeling with shallow architecture. We would like to clarify a point that might have been a source of confusion: the IDEncoder uses the whole trial as input, as opposed to the sliding context window that the cross-attention module uses. Therefore the IDEncoder would be able to learn the within-trial structures. \n\nSecondly, the shallow architecture of SPINT does not necessarily limit long range performance. The opposite might be true for CNNs where each layer has limited receptive field; however, in SPINT, fully connected layers and attention layers are used to model both temporal and spatial population responses, and each temporal step (or spatial unit) is accessible to all other timesteps (or spatial units) regardless of how far apart they are in the context. As long as the sliding window is long enough to include sufficient context for behavior decoding, the model should perform well as we showed in our analyses.\n\n**W2. re:** restricted evaluation to 1 human and 2 monkeys, behavior limited to arm/hand tasks.\n\nWe appreciate the reviewer's feedback regarding the number of subjects and diversity of the tasks.\nWe provided additional results demonstrating the benefits of using SPINT in cross-subject scenarios (please see table 1 & 2 in response to reviewer MJ6k). The focus of the  hand/arm tasks are mostly based on the availability of the human/monkey experiments. Due to the limitation of recording devices, most neural recordings are collected while monkeys are sitting, leading to the popularity of arm/hand tasks in the literature.\n\n**W3. re:** gradient-free adaptation still needs session specific calibration data.\n\nIndeed, our proposed SPINT framework still requires few session-specific calibration trials although it does not need labels for those trials (thus appreciably improving its practicality). Compared to current state-of-the-art (SOTA) few-shot decoding approaches (NoMAD and CycleGAN) which rely on fine-tuning to achieve desirable motor decoding performance, SPINT is a gradient-free approach that removes the need for test-time fine-tuning. \n\nMore broadly, an ideal BCI decoder needs to work under the zero-shot operation regime, meaning that no calibration data and no model fine-tuning are needed. Our approach represents a practicable and effective step towards this goal via a minimal use of calibration trials (in M1 dataset we could achieve 0.65 held-out $R^2$ even with 1 calibration trial, outperforming other SOTA methods).\n\n**Q1. re:** whether contrastive or predictive objectives could remove the need for labelled data for the neural ID encoder\n\nWe thank the reviewer for bringing up this insightful idea. We did briefly consider using unsupervised learning objectives such as contrastive or predictive loss to learn the representation for each neural unit. We found that these self-supervised objectives have difficulty in capturing the behavior-relevant aspects of neuronal identity, resulting in performance loss. Additionally, contrastive and predictive objectives require pretraining which necessitates additional training steps on downstream behavior decoding tasks. Thus we opted for the current SPINT design instead of using the contrastive or predictive objectives. We will add this discussion to the revised version of the paper.\n\n**Q2. re:** whether a lightweight temporal transformer on top of the cross-unit attention would benefit decoding behaviors with longer latencies\n\nWe thank the reviewer for this suggestion. The ability to model long-range information of SPINT depends on the length of the sliding context window. We performed sweeping experiments to find the best window length that can yield the best performance and found that longer context does not always translate to better performance. \n\n| Window length | 50           | 100     | 200           | 400           | 600           |\n|-------------|---------------|----------------|---------------|---------------|---------------|\n| $R^2$    | 0.65$\\pm$0.10 | 0.64$\\pm$0.10  | 0.64$\\pm$0.10 | 0.60$\\pm$0.10 | 0.61$\\pm$0.09 |\n\nTable 1: SPINT held-out performance across different window lengths.  Results are reported as mean $\\pm$ standard deviation across held-out M1 sessions.\n\nWe will incorporate these results and the discussion in the revised version of the paper."}, {"review_id": "78VWDozpSy", "reviewer": "Reviewer_MJ6k", "summary": "This paper introduces SPINT, a spatial permutation-invariant transformer architecture for behavioural decoding from neural population activity, in order to tackle the problem of dynamically inferring neural unit identities on new sessions or days and adapting to non-stationarities over time in intracortical brain-computer interface applications. SPINT uses a neural unit identity encoder that is equivariant under permutation of neural unit/channel orderings in the dataset/recording source, and invariant to the ordering of the calibration trials. While this encoder is trained on the pretraining dataset, it is not finetuned when generalising to subsequent sessions or days and is thus used in a zero-shot parameter update-free manner (only the unit identity embeddings are updated using the calibration trials), which the authors refer to as their few-shot unsupervised and gradient-free unit identification scheme. The authors also use a dynamic dropout scheme to improve robustness to variable neural population sizes. Post identity encoding of a time chunk of neural activity, behaviour is decoded using a cross-attention readout. The authors demonstrate results on 3 motor decoding datasets of the FALCON Benchmark for spiking activity-based neural decoders (2 nonhuman primate and 1 human dataset), and demonstrate better performance compared to other unsupervised baseline methods, while also showing some improvements with increasing training data and calibration trials, and with comparable latency to some Transformer-based methods.", "questions": "* Could the authors clarify details on the dropout ablation?\n* What if you were to train or finetune the `IDEncoder` MLPs with the calibration data since labels are available for those trials? Does it max out performance/lead to considerable gains?\n* The model has no self-attention or other intermediate layers, the embedded units seem to be directly used to decode behavior by a fairly small cross-attention layer. Were additional parameters not helpful or necessary to see better performance? Did you try scaling up the model size? I anticipate that this may not help much in the single dataset/subject setting, but in the large-scale multi-dataset/subject pretraining case, I anticipate that smaller models will underperform larger overparameterised models.\n* Details on the learnable behaviour queries seem sparse. Could the authors specify additional details on the behaviour queries? Given that only one behaviour at the final timestamp is decoded per context window of neural activity, is there only one behaviour query (initialised at random and learnt during pretraining)? Could there be multiple queries for multiple task readouts?\n* The approach currently uses binned spike counts but especially given the cross-attention's ability to handle variable number of input tokens, binning isn't necessary and the authors could use individual spike tokenisation à la POYO. Have the authors tried this scheme? It could potentially yield slightly better performance since there is no loss of information from the binning component, and spike timings could be important [15].\n* I see from Table S3 that quite a bit of context is needed for some tasks (M1: $W$ = 100 = 2s, M2: $W$ = 50 = 1s – which is fine, i.e., on par with NDT-2 and POYO, H1: $W$ = 700 = 14s – a lot!), I understand this can work in a sliding window but I'm just curious. Have the authors tried varying these values/doing a search for the best values? Why is so much context needed for some of these tasks?\n\n15. Xie, Weizhen, et al. \"Neuronal sequences in population bursts encode information in human cortex.\" Nature (2024): 1-8.", "limitations": "Yes, the authors have discussed the limitations of their work. There is no discussion on negative societal impact. While powerful iBCI decoders may generally have a positive impact on society, it may be worth recognising that a lot of these advances have been born out of animal research and may be misused as well; in addition there are concerns about the privacy of what is being decoded and when e.g. in decoding speech in humans.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n* The writing and figures are clear and easy to understand.\n* The authors have performed experiments on a recently introduced standard benchmark for neural decoding and have included important ablations and few-shot + data scaling results.\n* The method works with variable-sized neural populations thanks to the cross-attention mechanism.\n* The method is simple and can be used for lightweight, update-free adaptation – a key asset – across days in single subject and single task setups.\n\n**Weaknesses**\n\n* All experimental results are on single subject evaluation setups, with no experiments on cross-subject transfer for heldout subjects. While this is a consequence of the FALCON benchmark not having multi-subject data (all of M1, M2, and H1 are single subject datasets), it would be important to study how well SPINT performs in the multi-subject setting using other datasets as done in prior work [1-3] without which it is hard to evaluate the claims of it being a \"plug and play\" decoding method. The lack of this evaluation also diminishes the significance of the few-shot finetuning paradigm: the model cannot take advantage of multi-dataset pretraining and would have to be retrained for every new subject from scratch, limiting the finetuning scheme to the single subject setting across sessions/days. Baselines discussed in the paper, such as NDT-2 [1], and other key methods such as POYO [2] and NDT-3 [3] (latter is concurrent work) have studied this multi-subject setting, and some of these works demonstrate strong zero-shot or few-shot performance on new days too [1]. Furthermore, in my opinion, the \"plug and play\" argument for SPINT vanishes because quite a good amount of pretraining data is needed for good performance in the single subject case (ref. Fig. 3B, especially middle and right panels where multiple days of data are needed to get decent decoding performance), and there is no amortization of these costs through pre-training.\n* On a related note, while there are three datasets considered here, all are from a single benchmark while several of the mentioned baselines including the NDT models [1,3] have been tested on other public datasets [4-6]. Evaluating SPINT on these datasets would allow for better comparison to a wider range of existing approaches and improve the evaluations in this paper.\n* Another key point to me is the lack of comparison to POYO/POYO+ [2,7] (although the former is cited), given several architectural similarities. POYO can also accept unordered neural units as inputs, where it would learn a channel/unit embedding for each neuron, although through supervised learning for each new dataset. It is thus equivariant to the ordering of units. While SPINT takes in spike counts, where there is one token per unit, POYO is generally provided individual spike tokens, although it may also be used with spike counts [2,7,8] ([8] is concurrent work). While SPINT doesn't use self-attention layers, POYO uses a cross-attention encoder and decoder, and the setup in SPINT is very similar to POYO's behaviour queries and the overall Perceiver IO [9] framework. Note also that POYO uses a unit dropout scheme to improve robustness to population variability, so that is yet another similarity. Given these similarities, and the public availability of POYO code through `torch_brain` (along with a checkpoint) [10], I think it would be important for authors to consider this as a baseline on other public datasets (if not the FALCON benchmark, which POYO has not been submitted to). I do acknowledge that each approach has its own assets and pitfalls: POYO _requires_ supervised finetuning but excels at cross-task and cross-subject transfer, SPINT doesn't require parameter finetuning at all but has not been evaluated in a multi-subject setting.\n* The proposed approach which considers fixed $W$-length windows of neural activity may not work for attempted speech decoding in the FALCON H2 dataset, since decoding is from a fixed context window and there is no history as you would have in a recurrent architecture (unlike concurrent work [8]). This dataset only contains trial-level sentence annotations and not dense labels, so chunking up trials (which can be 10s+ long) and decoding phonemes from chunks would probably not yield good performance (as such, H2 only has recurrent baselines on the FALCON leaderboard).\n* The unit embedding scheme used here is tied to a single, specific behavioural task on which the model is trained. This limitation is acknowledged in the discussion, but it is a key obstacle to building cross-task models (currently, models would have to be retrained for each task). The authors could consider a readout scheme like POYO+ [7] to allow for multi-task decoding, but the advantages are limited without large-scale multi-dataset pretrainability. I do agree with the authors' suggestion that SSL approaches are the way forward for this problem.\n* This is relatively minor compared to my other concerns, but the channel dropout scheme introduced here with variable dropout rates is not entirely novel, and I'm curious about the exact setup used in the ablation study shown in Fig. 4B. To begin with, standard dropout [12] with a fixed dropout rate $p$ would itself dropout a variable number of units in each forward pass, just with a mean fraction of $p$ units dropped out. Secondly, models like POYO [2] and MYOW [11] have used unit dropout schemes with a minimum threshold for the number of units already. Thirdly, adaptive and variable dropout schemes are already prevalent in the deep learning literature [13,14], but have not been as widely adopted as standard dropout, potentially due to limited gains at large scale. Finally, about Fig. 4B, when the authors mention \"without\" dynamic dropout, do they mean there is no dropout at all, or just standard, fixed $p$ dropout? Of course I would expect consistent gains with (any type of) dropout as opposed to not having dropout at all, but given that the authors list the \"dynamic\" dropout as a contribution, I would expect an ablation to show performance comparisons with standard unit dropout [2,11] ideally with different values of $p$ in the ablation study. Could the authors clarify this?\n\n**References**\n\n1. Ye, Joel, et al. \"Neural data transformer 2: multi-context pretraining for neural spiking activity.\" Advances in Neural Information Processing Systems 36 (2023): 80352-80374.\n2. Azabou, Mehdi, et al. \"A unified, scalable framework for neural population decoding.\" Advances in Neural Information Processing Systems 36 (2023): 44937-44956.\n3. Ye, Joel, et al. \"A Generalist Intracortical Motor Decoder.\" bioRxiv (2025): 2025-02.\n4. Perich, Matthew G., Juan A. Gallego, and Lee E. Miller. \"A neural population mechanism for rapid learning.\" Neuron 100.4 (2018): 964-976. (https://dandiarchive.org/dandiset/000688?search=000688&pos=1)\n5. O’Doherty, Joseph E., et al. \"Nonhuman primate reaching with multichannel sensorimotor cortex electrophysiology.\" Zenodo (2017). (https://zenodo.org/records/3854034)\n6. Churchland, Mark M., et al. \"Neural population dynamics during reaching.\" Nature 487.7405 (2012): 51-56. (https://dandiarchive.org/dandiset/000070?search=churchland&pos=2)\n7. Azabou, Mehdi, et al. \"Multi-session, multi-task neural decoding from distinct cell-types and brain regions.\" The Thirteenth International Conference on Learning Representations.\n8. Ryoo, Avery Hee-Woon, et al. \"Generalizable, real-time neural decoding with hybrid state-space models.\" arXiv preprint arXiv:2506.05320 (2025).\n9. Jaegle, Andrew, et al. \"Perceiver io: A general architecture for structured inputs & outputs.\" arXiv preprint arXiv:2107.14795 (2021).\n10. Code: https://github.com/neuro-galaxy/torch_brain/tree/main/examples/poyo; checkpoint: https://github.com/neuro-galaxy/poyo\n11. Azabou, Mehdi, et al. \"Mine your own view: A self-supervised approach for learning representations of neural activity.\"\n12. Srivastava, Nitish, et al. \"Dropout: a simple way to prevent neural networks from overfitting.\" The journal of machine learning research 15.1 (2014): 1929-1958.\n13. Ba, Jimmy, and Brendan Frey. \"Adaptive dropout for training deep neural networks.\" Advances in neural information processing systems 26 (2013).\n14. Li, Zhe, Boqing Gong, and Tianbao Yang. \"Improved dropout for shallow and deep learning.\" Advances in neural information processing systems 29 (2016).", "was_revised": true, "final_justification": "The authors addressed my questions, clarified their motivation and goals, and performed several experiments during the rebuttal phase. I think the method is neat and shows some good results on FALCON, a standard benchmark, so I'm happy to recommend acceptance.", "rebuttal": "We thank the reviewer for an insightful review, thoughtful questions, and valuable feedbacks. We address the reviewer's concerns and questions below:\n\n**W1:**\nWe thank the reviewer for raising the question about SPINT’s ability to transfer across subjects. Our focus in this paper was to isolate the challenge of cross-session nonstationarity within a subject, which remains a central issue in iBCI systems. When we said SPINT contributed an initial step toward a plug-and-play decoding framework for long-term iBCI, we referred specifically to the out-of-the-box cross-session adaptation within a subject, where we demonstrated gradient-free generalization without test-time labels.\nWhile we designed SPINT specifically for this practical usage, we agree with the reviewer that cross-subject evaluation would bring further values to the method. We therefore conducted additional experiments on a dataset of an extra monkey performing the M1 task (M1-B) (https://dandiarchive.org/dandiset/001209). While this extra dataset is not part of the official FALCON Benchmark and does not have a private held-out test set on EvalAI, the subject in this dataset (Monkey X) performs the same reach-to-grasp task as Monkey L in the official M1-A dataset, and the FALCON organizers curated its data into held-in and held-out sessions following the same standardized format of the benchmark. We did not observe straight-out transferability when evaluating one subject’s checkpoint on another subject’s held-out sessions. However, with full model weight supervised fine-tuning on the new subject’s held-in sessions, the pretrained SPINT quickly adapts to the new subject, maintaining a gap over the model trained from scratch on the new subject in terms of cross-session generalization (Tables 1 & 2). The EvalAI platform was down during the rebuttal period, therefore we show the performance on the public calibration portion of the held-out sessions. \n\n| | Epoch 1 | Epoch 2 | Epoch 3 | Epoch 4 | Epoch 5 | Epoch 50 |\n|-|-|-|-|-|-|-|\n| Monkey X from scratch | 0.59 $\\pm$ 0.06 | 0.60$\\pm$ 0.07 | 0.60$\\pm$0.07 | 0.59$\\pm$0.07 | 0.59$\\pm$0.07 | 0.64 $\\pm$ 0.08 |\n| Monkey L -> Monkey X | 0.62$\\pm$0.06   | 0.64$\\pm$0.06  | 0.62$\\pm$0.07 | 0.63$\\pm$0.07 | 0.65$\\pm$0.07 | 0.67 $\\pm$ 0.08 |\n\nTable 1: Cross-session performance of SPINT on subject X when finetuned from Monkey L vs. trained from scratch. Results are reported as mean $\\pm$ standard deviation across held-out calibration sessions.\n\n| | Epoch 1 | Epoch 2 | Epoch 3 | Epoch 4 | Epoch 5 | Epoch 50 |\n|-|-|-|-|-|-|-|\n| Monkey L from scratch | 0.48$\\pm$0.10 | 0.52$\\pm$0.08 | 0.53$\\pm$0.07 | 0.54$\\pm$0.08 | 0.56$\\pm$0.08 | 0.64 $\\pm$ 0.10|\n| Monkey X -> Monkey L | 0.57$\\pm$0.07 | 0.56$\\pm$0.10 | 0.60$\\pm$0.07 | 0.60$\\pm$0.07 | 0.60$\\pm$0.07 | 0.63 $\\pm$ 0.09 |\n\nTable 2: Cross-session performance of SPINT on subject L when finetuned from Monkey X vs. trained from scratch.\n\nThese results highlight the subject transferability of SPINT, where cross-session generalization ability on a new subject is maintained and efficiently achieved within a few epochs of fine-tuning.\n\n**W2 & W3:** We chose the FALCON Benchmark for our analyses because it contains standardized splits with clear constraints on the number of test-time samples, label availability, and latency requirements in real-world iBCI systems that we aspire for SPINT. We also appreciate the reviewer highlighting the similarities and differences between SPINT and POYO. We hope the appreciation we have for POYO and other baselines was clear in our paper. To provide further clarity, we will include a discussion on the unique aspects of POYO and SPINT, and will cite POYO+ and POSSM papers in the final version of our paper. \n\nDirect comparison of SPINT and POYO on the FALCON Benchmark is not trivial, as the benchmark’s continuous decoding evaluation paradigm strictly requires participating models to operate on binned spikes, which POYO does not natively support with its unit tokenization scheme. Following the reviewer’s recommendation, we conducted extra experiments to evaluate SPINT on the suggested Perich et al., 2018 dataset [1] that POYO was also trained on. Due to resource constraints and the lack of standardized few-shot evaluation splits on this dataset as we have in FALCON, we selected 10 and 2 sessions of Monkey C performing the Center-Out (CO) reaching task as held-in and held-out sessions, respectively. We reserved 32 trials on each held-out session as the few-shot calibration set. We binned the spike trains with a 20-ms bin width and followed FALCON’s continuous decoding paradigm on this dataset. SPINT demonstrated good cross-session generalizability on this dataset, achieving $R^2$ of 0.83 $\\pm$ 0.01 on the heldout sessions, without fine-tuning or using labels in the held-out sessions. For comparison, POYO-mp trained on 100 sessions (that is, POYO-mp was trained on more data) achieved $R^2$ of 0.9675 $\\pm$ 0.0079 by fine-tuning unit ID and $R^2$ of 0.9708 $\\pm$ 0.0116 by fine-tuning the entire model on the held-out sessions. When fine-tuned on the held-out sessions, SPINT could further achieve held-out $R^2$ of 0.88 $\\pm$ 0.01. These results show the promise of SPINT on datasets beyond the FALCON Benchmark.\n\n[1] Perich, Matthew G., Juan A. Gallego, and Lee E. Miller. \"A neural population mechanism for rapid learning.\" Neuron 100.4 (2018): 964-976. (https://dandiarchive.org/dandiset/000688?search=000688&pos=1)\n\n**W4:** Due to the character limits, we kindly ask the reviewer to check our Q3 response to Reviewer CKq1.\n\n**W5:** We appreciate the reviewer’s comment on the cross-task trainability. As the reviewer suggested, a modification to the SPINT architecture to employ a readout scheme as in POYO+ will facilitate SPINT’s training across behavior tasks. SPINT could be extended to have this multi-dataset pretrainability in future works. We will add this perspective to the Limitations section in the revised manuscript.\n\n**W6:** We thank the reviewer for bringing up similar dropout schemes employed in previous works that we missed. We will cite the references the reviewer suggested, and revise our wording so as to not overclaim dynamic dropout as our main contribution. One relatively minor difference between SPINT’s dropout scheme and that of earlier implementations is that dynamic dropout uses the full valid interval of probabilities in SPINT (i.e., [0,1]), instead of a limited low-probability range (i.e., [0, 0.2]). In Figure 4B, by “without dynamic dropout” we meant with no dropout at all. Due to the character limits, we kindly refer the reviewer to the dynamic dropout table in W2 response to Reviewer CKq1 for further dropout analyses.\n\n**Q1:** Please see the above comments in W6.\n\n**Q2:** Although we designed SPINT for few-shot unsupervised adaptation, SPINT could also be used for few-shot supervised learning, as the reviewer correctly suggested. We performed experiments where we either fine-tune the IDEncoder or the whole model using calibration data on the few labeled samples of held-out sessions, and observed that this led to improved cross-session performance. Fine-tuning the IDEncoder obtained held-out $R^2$ of 0.69 $\\pm$ 0.10, and fine-tuning all model weights obtained held-out $R^2$ of 0.70 $\\pm$ 0.10, as compared to the gradient-free unsupervised SPINT with held-out $R^2$ of 0.64 $\\pm$ 0.10. We will report the new results in the revised manuscript.\n\n**Q3:** Intended to be lightweight, our SPINT model does not have self-attention layers and only has 1 cross-attention layer for simple information aggregation. To analyze the scaling effect of model size, we provide the results for two sets of experiments: 1) similar to POYO, we added self-attention layers before the cross-attention layers, with the number of self-attention layers ranging from 1 to 4. 2) We scaled up the cross-attention layers from 1 to 5. Our results are listed below.\n\n| Number of self-attention layers | 0 (Ours)  | 1 | 2 | 3 | 4 |\n|:--:|-|-|-|-|-|\n|      $R^2$     | 0.64$\\pm$0.10 | 0.63$\\pm$0.13 | 0.57$\\pm$0.13 | 0.61$\\pm$0.10 | 0.60$\\pm$0.15 |\n\n| Number of cross-attention layers | 1 (Ours)  | 2| 3 | 4 | 5 |\n|:--:|-|-|-|-|-|\n|      $R^2$     | 0.64$\\pm$0.10 | 0.65$\\pm$0.10 | 0.65$\\pm$0.10 | 0.64$\\pm$0.11 | 0.62$\\pm$0.13 |\n\nOur results indicate that scaling up model size by adding self-attention and cross-attention layers does not significantly improve the performance. This might be due to the small scale of the FALCON datasets, as the reviewer noted. We will incorporate these results and the associated interpretation into the revised manuscript.\n\n**Q4:** Our single behavior query was initialized at random and learned during training. The behavior query Q has dimension BxW, where B is the number of behavior covariates and W is the neural window length. For multi-task extension, there can be multiple queries for decoding behavior in the corresponding task.\n\n**Q5:** We agree with the reviewer that the individual spike tokenization in POYO could potentially work better since spike timings contain helpful information that can be lost in the binning process. However, using POYO spike tokenization in the context of the FALCON continuous decoding paradigm is difficult, as the benchmark strictly requires participating models to adopt the binned spike format for evaluation. Please also see our additional comments in W2 and W3.\n\n**Q6:** We performed a hyperparameter search to find the reported optimal sliding window size. Heterogeneity in the task structure leads to different optimal sizes for each task, as a trial in H1 typically lasts for ~20s, much longer than trials in M1 and M2. Due to the character limits, we refer to the window size table in Q2 from Reviewer fpRR."}, {"review_id": "ugL1F6jDQD", "reviewer": "Reviewer_CKq1", "summary": "The paper tackles cross‐session drift in intracortical BCIs caused by shifting neural signals. It introduces SPINT, a transformer that treats neurons as unordered tokens and infers their identities from a small set of unlabeled calibration data. SPINT uses context‐dependent position embeddings and dynamic channel dropout to adapt without gradient updates. It outperforms zero‐ and few‐shot baselines on multi‐session motor decoding benchmarks.", "questions": "1. How well does SPINT handle very large shifts in neural signals between sessions?  \n2. Can SPINT adapt with only one or two calibration trials?  \n3. Could this method work for non-motor tasks (e.g., speech decoding)?  \n4. Are there any privacy or ethical concerns with long-term neural recording?  \n\nI will take into account other reviewers’ opinions when evaluating the paper further.", "limitations": "No.\n\nThe authors do outline methodological limitations in the Discussion section, but they omit any consideration of potential negative or dual-use societal impacts.\n\nSuggestions for improvement:\n1. Add a “Broader Impacts” subsection that discusses risks of misuse (e.g., privacy breaches, uneven access to iBCI technologies).\n2. Reflect on ethical concerns around long-term neural recording and consent, and outline safeguards (e.g., data anonymization, IRB oversight).\n3. Consider potential downstream harms (e.g., neuro-surveillance) and propose mitigation strategies.", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper is well written and organized, with diagrams and pseudo-code that make the approach easy to follow.  \n2. The experiments are comprehensive and rigorously compare SPINT to zero-shot, few-shot, and oracle baselines on three primate and human datasets, showing clear performance gains.  \n3. SPINT adapts with only a small amount of unlabeled calibration data and requires no gradient updates, making it practical for real-world BCIs.  \n4. Combining context-dependent position embeddings with dynamic channel dropout is a novel solution for session drift without explicit alignment.\n\n**Weaknesses**\n1. No failure cases are shown, so it’s unclear how SPINT handles extreme signal drift or very limited calibration data.  \n2. Hyperparameter choices—such as dropout rates and attention head counts—are not well justified, making sensitivity hard to assess.  \n3. Evaluation is limited to motor decoding, leaving its generality to other BCI tasks untested.", "was_revised": true, "final_justification": "After reading other reviewers' reviews and authors' rebuttal. I agree that this work is acceptable.", "rebuttal": "We thank the reviewer for providing valuable suggestions and feedbacks on our manuscript. We address the reviewer's concerns and questions in our point-by-point responses below: \n\n**W1. re: failure cases**\n\nWe thank the reviewer for the comments. We show in the paper that SPINT can handle well the signal drift across held-in and held-out sessions, which can be three weeks to one month apart in the extreme cases of all three datasets. We also showed in Figure 3 our analyses on how SPINT performed with very limited calibration data. Except for the M1 dataset where SPINT can perform well even with one calibration trial, we observed that with one calibration trial in M2 and H1, SPINT performance degraded with held-out $R^2$ around zero in these extreme cases. \n\n**W2. re: sensitivity to hyperparameter choices**\n\nWe appreciate the reviewer’s request for more details on the hyperparameter choices of fixed dropout rates and attention head counts. While we employed dynamic dropout instead of the conventional fixed dropout, we provide below the sweeping analysis on different choices for fixed dropout rate together with different range of dynamic dropout rates for reference. We also include sweeping experiments on attention head counts as the reviewer suggested. SPINT’s performance on held-out sessions remains stable across different choices of dropout rates and attention head counts. The results demonstrate SPINT’s robustness to the choices of hyperparameters. \n\n| Fixed dropout rate | 0             | 0.2           | 0.4           | 0.6           | 0.8           | dynamic dropout [0, 1] |\n|---------------|---------------|---------------|---------------|---------------|---------------|------------------------|\n| $R^2$         | 0.51$\\pm$0.13 | 0.62$\\pm$0.10 | 0.63$\\pm$0.10 | 0.63$\\pm$0.10 | 0.60$\\pm$0.09 | 0.64$\\pm$0.10          |\n\nTable 1: SPINT's cross-session performance against dynamic dropout and different choices of fixed dropout rates. Results are reported as mean $\\pm$ standard deviation across held-out calibration sessions.\n\n| Dynamic dropout ranges | [0, 0.1]             | [0, 0.2]           | [0, 0.3]          | [0, 0.4]          | [0, 0.5]           | [0, 1] (Ours)|\n|---------------|---------------|---------------|---------------|---------------|---------------|------------------------|\n| $R^2$         | 0.59$\\pm$0.07 | 0.59$\\pm$0.07 | 0.61$\\pm$0.10 | 0.62$\\pm$0.07 | 0.63$\\pm$0.07 | 0.64$\\pm$0.10          |\n\nTable 2: SPINT's cross-session performance against different choices of range for dynamic dropout. Results are reported as mean $\\pm$ standard deviation across held-out calibration sessions.\n\n| # Attention heads | 4             | 8             | 16            | 32            | 64            |\n|-------------------|---------------|---------------|---------------|---------------|---------------|\n| $R^2$             | 0.62$\\pm$0.08 | 0.63$\\pm$0.09 | 0.64$\\pm$0.10 | 0.65$\\pm$0.11 | 0.64$\\pm$0.10 |\n\nTable 3: SPINT's cross-session performance against different choices of attention head counts. Results are reported as mean $\\pm$ standard deviation across held-out calibration sessions.\n\n**W3. re: evaluation limited to motor decoding**\n\nWe appreciate the reviewer’s concern. While this is driven mainly by the availability of standardized datasets and evaluation platforms, we will mention this perspective in the Limitations section of the revised manuscript.\n\n**Q1. re: SPINT handling of large shift between sessions**\n\nPlease see our response in W1\n\n**Q2. re: whether SPINT can adapt with one or two trials**\n\nYes SPINT can adapt with as few as one or two calibration trials. We showed in Figure 3 of our main paper SPINT’s performance on M1 and H1 datasets when the number of calibration trials are restricted to these values.\n\n**Q3. re: whether SPINT can work on non-motor tasks**\n\nIn the absence of actual experiments, we speculatively think that SPINT’s lightweight and neuronal identity based approach will be useful in non-motor tasks as well. \nOur algorithm can be extended to speech and text coding with the following adjustments. In general, for speech/text coding, the data is of variable length and has sparse labels. To address the above difficulties, we can 1) apply zero-padding or learnable padding for the variable-length data so they can have the same maximum length. This padding batch trick is widely used in LLM that can be easily incorporated in our data processing pipelines. 2) The issue of no dense label in text/speech decoding can be solved via the connectionist temporal classification (CTC) loss during model training. In the model inference phase, one can use either greedy or beam search approaches for the final character/phoneme decoding. This will serve as an exciting future extension of our work.\n\n**Q4. re: privacy and ethical concerns of long-term neural decoding**\n\nWe work on public datasets within the framework of the FALCON Benchmark and focused more on algorithm development to realize the goal of consistent motor decoding. Thus, our in-silico experiments did not involve any human or animal experiments. In terms of data privacy and ethics protocol in the collection of neural data recording, the FALCON Benchmark has properly addressed these points in their publication.\n\n**L1. re: broader impacts**\n\nWe thank the reviewer for suggesting this point. We added them in the discussion section by considering 1) the ethical concerns in iBCI data collection; 2) privacy of iBCI data usage and distribution; 3) the safety of iBCI technology.\n\nBroader Impacts. iBCI technologies raise significant ethical and societal considerations that require proactive mitigation strategies. First, we advocate for consistent consent protocols and enhanced IRB oversight to address challenges of long-term neural data recording and collection, particularly for vulnerable populations with potential neurological and neuropsychiatric disorders. Second, neural data's inherently identifying nature necessitates privacy-preserving techniques beyond traditional anonymization, including different privacy regularizations to prevent risky inference of subject information. Third, we acknowledge dual-use risks including potential neuro-surveillance applications and cognitive manipulation, alongside equity concerns where high costs may exacerbate healthcare disparities. To mitigate these risks, we recommend developing open-source frameworks, establishing international governance standards similar to those proposed for AI-related algorithms and systems, implementing technical safeguards against unauthorized access, and conducting regular ethical impact assessments. To this end, we hope the above considerations can underscore the need for interdisciplinary collaboration between technologists, ethicists, and policymakers to ensure responsible development and deployment of future iBCI technologies."}, {"review_id": "4wsNk6gjc6", "reviewer": "Reviewer_Fnbe", "summary": "Authors introduce SPINT, a neural decoder that is invariant to unit activity, thanks to a context dependent positional encoding. They show improved performance compared to other zero or few-shot methods on the FALCON dataset.", "questions": "First, i find section 3.5 a bit unclear. From the rest of the paper and the architecture design, I would have expected that IDEncoder is fine-tuned, while the cross attention layer remains fixed in the few shot experiment. Lines 190-192 seems to indicate the contrary? I am not sure I understand this part correctly and believe this could be clarified further.  \n\nAlso, authors claim that this is an attempt at a gradient-free framework. I believe that their method could be applicable in more complex datasets as a few-shot decoder (i.e. not gradient free), but don't believe that it will work as a zero-shot method. I would appreciate if authors can explain why they claim their method to be a step towards a gradient-free method, or remove this claim (I don't believe it's a drawback of the paper since few-shot is already sufficient in my opinion, but I would appreciate a further explanation if the method is truly zero-shot). To clarify this further, in Table 1, it would help to show which methods are OR, FSS, FSU, and ZS (you could have four \"rows\" for each of them and add it in the table). \n\nIn Figure 3A, it is very surprising that the method works on M1 with 1 calibration trial only. Could the authors give some intuition behind this result? \n\nThe authors claim that the neural population can be of arbitrary size. Isn't it bounded by a maximum population size? How do you deal with populations of increasing size? The problem of having \"disappearing neurons\" is adressed, but neurons can appear in the recordings as well (it can also be an artefact of the spike sorting / preprocessing pipeline). \n\nFinally, in in-vivo recordings, often neurons get \"mixed up\" together, rather than \"permuted\", in the sense of spikes from one neuron get assigned to the other and vice versa, as a result of the spike sorting pipeline. Do you think the method will be robust to such noise, and could you augment your permutations (maybe add such a perturbation on top of the channel dropout) to make it robust to these artefacts?", "limitations": "Yes, the authors have discussed the limitations. \nAs mentioned, two important challenges are to align across task, regions and subjects, and to extend the method to in vivo recordings.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: It is a very good paper, building on a great idea (being invariant to unit ID), with sound design choices and good evaluation. \n\nWeaknesses: I don't see no major weakness to this paper, but as the authors mentioned in the Discussion section, it will be important for future work to evaluate SPINT in vivo, where the noise and perturbations to unit assignment and spike detection can be much more complex.", "was_revised": true, "final_justification": "I wish to keep my positive score.", "rebuttal": "We thank the reviewer for the insightful review, valuable suggestions and constructive feedbacks. We provide clarification to the reviewer questions in our point-by-point responses below:\n\n**Q1. re: clarification on whether IDEncoder is fine-tuned in the new sessions.**\n\nWe thank the reviewer for the questions. The IDEncoder is trained with behavior labels during held-in sessions and its weights are frozen during inference on held-out sessions. IDEncoder’s role is to identify units from their temporal activations in calibration trials. We assume this identity is largely preserved for each individual unit across sessions, only their order is changed due to shifts in electrodes between sessions. Therefore, with sufficient training during held-in sessions, IDEncoder can infer this preserved identity out-of-the-box on the held-out sessions without any fine-tuning. \n\n**Q2. re: clarification on gradient-free, zero-shot, and few-shot terminologies**\n\nWe thank the reviewer for the comments. By SPINT being a few-shot unsupervised method we mean SPINT would need a few unlabeled calibration trials from the held-out sessions for adaptation. This adaptation does not require any parameter updates in either the IDEncoder or the rest of the model as we elaborate in the response above, therefore SPINT is a *gradient-free* framework. The reviewer is correct that SPINT is not a zero-shot method, as it would still need a few calibration trials at test time. However, unlike existing few-shot baselines where model weights are adapted with gradients at test time (NoMAD and CycleGAN), SPINT can be adapted in the forward-only manner. Therefore we created a new class of method called *gradient-free few-shot unsupervised* to properly distinguish SPINT with the existing baselines.\n\nOne model can belong to different model categories (ZS, FSU, FSS, OR), depending on whether they have access to the held-out calibration neural activity and behavior labels. In Table 1 of our paper we followed the reporting format in the FALCON paper [1] to mention each model’s class in the column next to their names.\n\n**Q3: re: intuition how SPINT can work with only one calibration trial on M1**\n\nWe agree with the reviewer that this is a surprising and interesting result. The ability to generalize even with one calibration trial can be attributed to the consistency of the animal behavior and the relatively large size of the M1 dataset (5 and 6 times larger than the H1 and M2 datasets, respectively). After being sufficiently trained on a sufficient amount of structured data, when presented with even a single trial at test time, the IDEncoder can extract distinct features from the temporal responses of the units, and the cross-attention mechanism can selectively aggregate these information for behavior decoding.\n\n**Q4: re: robustness of SPINT against test populations with larger size than the training population size**\n\nWe appreciate the reviewer’s thoughtful question. We conducted additional experiments on to gauge the robustness of SPINT when the population is larger than the size seen during training. Specifically, we artificially duplicated a random subset of units in the population and evaluated SPINT’s cross-session performance on this larger population. We found that SPINT is robust to the increased population size, even up to twice the size of original population (Table 1). Since the EvalAI platform was down during the rebuttal period, we show here the performance on the public calibration portion of the held-out sessions on M1 dataset.\n\n| Population size | 100%           | 120%            | 140%           | 160%           | 180%           | 200%           |\n|-----------------|----------------|-----------------|----------------|----------------|----------------|----------------|\n| Held-out $R^2$  | 0.64$\\pm$ 0.10 | 0.63 $\\pm$ 0.09 | 0.63$\\pm$ 0.09 | 0.63$\\pm$ 0.09 | 0.63$\\pm$ 0.08 | 0.63$\\pm$ 0.09 |\n\nTable 1: SPINT robustness to increased size of population. Results are reported as mean $\\pm$ standard deviation across held-out calibration sessions.\n\n**Q5. re: robustness of SPINT against noise in spike assignment**\n\nWe thank the reviewer for raising an interesting question. We agree with the reviewer that noise introduced by the spike sorting pipeline could negatively impact SPINT performance. We conducted additional experiments on the M1 dataset where we artificially shuffle spikes among a subset of random units and evaluated SPINT cross-session performance:\n\n| Augmentation rate | 0.0           | 0.05          | 0.1            | 0.2           | 0.4           | \n|-------------------|---------------|---------------|----------------|---------------|---------------|\n| Held-out $R^2$    | 0.64$\\pm$0.10 | 0.61$\\pm$0.08 | 0.57$\\pm$ 0.09 | 0.53$\\pm$0.08 | 0.38$\\pm$0.05|\n\nTable 2: SPINT robustness to noise in spike assignment. Results are reported as mean $\\pm$ standard deviation across held-out calibration sessions.\n\nThis is somewhat expected, as SPINT relies on consistent spiking features in a unit’s responses to accurately identify the unit. With spikes being assigned to incorrect units, this ability to extract unit identity would be impacted, leading to a degraded decoding performance as a result. Our initial attempts to introduce this noise corruption during training did not result in improved performance; however, we agree with the reviewer that this is an interesting avenue to explore in future extensions of the method.\n\n\n[1] Brianna Karpowicz, Joel Ye, Chaofei Fan, Pablo Tostado-Marcos, Fabio Rizzoglio, Clayton Washington, Thiago Scodeler, Diogo de Lucena, Samuel Nason-Tomaszewski, Matthew Mender, et al. Few-shot algorithms for consistent neural decoding (falcon) benchmark. Advances in Neural Information Processing Systems, 37:76578–76615, 2024."}]

Author rebuttal:

Meta-review: This paper introduces SPINT, a transformer-based framework designed to address the challenge of changing neural populations in long-term intracortical Brain-Computer Interfaces (iBCIs). The model's primary contribution is its ability to operate on unordered sets of neurons, using a novel positional embedding to dynamically infer neural identities, which allows for robust generalization across different recording sessions. The authors demonstrate that SPINT outperforms existing methods and uniquely enables few-shot, gradient-free adaptation, eliminating the need for computationally expensive alignment or fine-tuning during deployment.

The reviewers were in strong agreement about the paper's positive contributions, consistently praising its core idea as novel, clever, and highly relevant to the BCI field. They found the proposed permutation-invariant transformer, which treats neurons as an unordered set to handle cross-session drift, to be a sound and well-designed solution. The reviewers also positively noted the model's strong performance, highlighting that it outperformed baselines on a standard benchmark. They especially appreciated the practical aspects of the method, such as its lightweight architecture and its efficient, gradient-free adaptation scheme, which they saw as a significant advantage for real-world applications. The clarity of the writing and the overall quality of the presentation were also common points of positive assessment.

Despite the positive reception of the core concept, the reviewers also converged on several critiques, primarily concerning the limited scope of the evaluation. They consistently pointed out that the experiments were restricted to only three single-subject datasets and were focused exclusively on motor decoding tasks. This narrow evaluation raised questions about the model's generalizability to new subjects, different tasks, and more complex in-vivo conditions. Another key point of agreement was that while the adaptation method was innovative, the necessity of collecting new, session-specific calibration data for each use was a practical limitation that tempered the model's "plug-and-play" claims. Some reviewers also felt the paper could have been strengthened by a more thorough comparison to other similar architectures and a deeper exploration of the model's failure cases.

Nonetheless, with the overall positive reviews, and scores post-rebuttal, a decision of accept (poster) was reached.
Decision: Accept (poster)