Exploring the trade-off between deep-learning and explainable models for brain-machine interfaces
Authors: Luis Hernan Cubillos
Guy Revach
Matthew Mender
Joseph T Costello
Hisham Temmar
Aren Hite
Diksha Anoop Kumar Zutshi
Dylan Michael Wallace
Xiaoyong Ni
Madison M. Kelberman
Matt Willsey
Ruud Van Sloun
Nir Shlezinger
Parag Ganapati Patil
Anne Draelos
Cynthia Chestek
Venue: NeurIPS 2024 poster (2024)
Keywords: brain-machine interfaces
neural decoders
safety
kalman filter
real-time processing
Primary area: neuroscience_and_cognitive_science
TL;DR: We show an explainable brain decoder that combines the Kalman filter and RNNs to predict finger movements with high accuracy

Abstract: People with brain or spinal cord-related paralysis often need to rely on others for basic tasks, limiting their independence. A potential solution is brain-machine interfaces (BMIs), which could allow them to voluntarily control external devices (e.g., robotic arm) by decoding brain activity to movement commands. In the past decade, deep-learning decoders have achieved state-of-the-art results in most BMI applications, ranging from speech production to finger control. However, the 'black-box' nature of deep-learning decoders could lead to unexpected behaviors, resulting in major safety concerns in real-world physical control scenarios. In these applications, explainable but lower-performing decoders, such as the Kalman filter (KF), remain the norm. In this study, we designed a BMI decoder based on KalmanNet, an extension of the KF that augments its operation with recurrent neural networks to compute the Kalman gain. This results in a varying “trust” that shifts between inputs and dynamics. We used this algorithm to predict finger movements from the brain activity of two monkeys. We compared KalmanNet results offline (pre-recorded data, $n=13$ days) and online (real-time predictions, $n=5$ days) with a simple KF and two recent deep-learning algorithms: tcFNN (non-ReFIT version) and LSTM. KalmanNet achieved comparable or better results than other deep learning models in offline and online modes, relying on the dynamical model for stopping while depending more on neural inputs for initiating movements. We further validated this mechanism by implementing a heteroscedastic KF that used the same strategy, and it also approached state-of-the-art performance while remaining in the explainable domain of standard KFs. However, we also see two downsides to KalmanNet. KalmanNet shares the limited generalization ability of existing deep-learning decoders, and its usage of the KF as an inductive bias limits its performance in the presence of unseen noise distributions. Despite this trade-off, our analysis successfully integrates traditional controls and modern deep-learning approaches to motivate high-performing yet still explainable BMI designs.

Exploring the trade-off between deep-learning and explainable models for brain-machine interfaces
Luis H. Cubillos', Guy Revach", Matthew J. Mender', Joseph Costello', Hisham Temmar', Aren Hite', Diksha Zutshi', Dylan M. Wallace', Xiaoyong Ni², Madison M. Kelberman', Matthew S. Willsey Ruud J.G. van Sloun', Nir Shlezinger". Parag Patil', Anne Draelos', Cynthia A. Chestek
DDpartments of Electrical & Computer Engineering, Biomedical Engineering. Robotics, Computational Medicine & Bioinformatics, and Neurosurgery, University of Michigan, USA 2 Biointerfaces Institute and Nearoscience Institute, University of Michigan, USA Department of Information Technology and Electrical Engineering, ETH Zürich, Switzerland Department of Electrical Engineering, Eindhoven University of Technology, Netherlands, School of Electrical and Computer Engineering, Ben-Garion University, Israel
Abstract
People with brain spinal cord-related paralysis often need to rely others for basic tasks, limiting their independence. A potential solution is brain-machine interfaces (BMIs), which could allow them voluntarily control external devices (e.g., robotic arm) by decoding brain activity to movement commands. In the past decade, deep-learning decoders have achieved state-of-the-ar results in most BMI applications, ranging from speech production to finger control. However, the 'black-box' nature of deep-learning decoders could lead unexpected behaviors, resulting in major safety concerns in real-world physical control scenarios. In these applications, explainable but lower-performing decoders, such as the Kalman filter (KF), remain the norm. In this study, we designed a BMI decoder based on Kalman- Net, an extension of the KF that augments its operation with recurrent neural networks to compute the Kalman gain. This results in varying "trust" that shifts between inputs and dynamics. We used this algorithm predict finger movements from the brain activity of two monkeys. We compared KalmanNet results offline (pre-recorded data, πi = 13 days) and online (real-time predictions, n = days) with simple KF and two recent deep-learning algorithms: tcFNN (non-ReFlT version) and LSTM. KalmanNet achieved comparable or better results than other deep learning models in offline and online modes, relying on the dynamical model for stopping while depending more on neural inputs for initiating movements. We further validated this mechanism by implementing heteroscedastic KF that used the same strategy, and it also approached state-of-the- performance while remaining in the explainable domain of standard KFs. However, also see downsides to KalmanNet. KalmanNet shares the limited generalization ability existing deep-learning decoders, and its usage of the KF an inductive bias limits its performance in the presence of unseen noise distributions. Despite this trade-off, our analysis successfully integrates traditional controls and modern deep-learning approaches to motivate high-performing yet still explainable BMI designs.
Introduction
Millions of people worldwide live with a ncurological or spinal cord condition that limits their ability to interact with the world [T]. These patients have rely others for basic tasks such as cating,
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

moving, or even communicating with their loved ones, which severely limits their independence. Existing assistive devices, such as sip-and-put controllers or eye trackers, can take advantage of small residual movements-iff available-but are limited in the degrees of freedom (DoF) they can control. Brain-machine interfaces (BMIs) have started gaining traction in the past decades as a viable alternative that can control multiple DoFs while taking advantage of the existing hardware for computation: the brain.
Brain-machine interfaces work by reading electrical activity from the brain, decoding the activity with decoding algorithm, and then using the algorithm's 's output to interact with an external device. Similar to existing applications such as language and image generation (2]-(6) and understanding
(7 deep learning has become the norm in decoding algorithms for BMIs. New BMI applications such as handwriting detection from a person unable to move (TT) and real-time speech production
from a person unable to speak [12 [4] have been enabled recently through the use of deep learning
algorithms. Additionally, many deep learning models have demonstrated sate-of-the-arr results in BMI applications that previously only used linear decoders, such as real-time prediction of finger
movement
15
and cursor control
[8 [9
These promising results using deep learning algorithms are enabled, in part, by an increase in the complexity of the models [20). This can make interpreting their structure and results very difficult [1O]. 21 22]. is also difficult to predict the algorithm's 's behavior with out-of diittri-iiiiittiion inputs. This complexity has caused deep learning algorithms to be seen as "black boxes," limiting their practical application in more sensitive domains such as vehicular control or lower limb prosthetics. Interaction with the physical world inevitably involves safety concerns. There, conventional control approaches that use explainable linear models are the norm, usually involving Kalman filters (KFs) because they are not as likely to exhibit unexpected behaviors. In BMIs, KFs and explainable linear models are still regularly employed in applications where the safety of the user is in question, such as when moving robotic arm Thus, researchers are confronted with the need to sacrifice potentially better decoder performance in the name of safety and explainability. Here we ask: can we have the best of both worlds?
23 or controlling paralyzed arm through electrical stimulation (24).
In this paper, we examine the trade-offss between linear, explainable models and high- gigh-peffrrming deep-lcarning approaches. To interpolate between these two regimes (model-base and data-drin -riaenriven). we modify and retrain the recently proposed KalmanNet 25 which combines the long-standing KF with the abstractness of deep models by augmenting its operation with set of flexible recurrent neural networks (RNNs). We demonstrate the effectiveness of KalmanNet in real-time BMI experiments where monkeys with brain implants perform dexterous finger task and show that we can match or outperform two previously published algorithms: tcFNN (non-ReFIT version) (16) and LSTM [17). By leveraging the explainable portion of KalmanNet and its velocity dependence, we explain how this approach works through abrupt changes in the relative trust of the brain or dynamical models. Under the injection of additional noise, however, noise, the LSTM actually outperforms all other models for parge-mannitude Finally, tested the ability of the models to generalize across task contexts and found better generalization for the simple KF than for all deep-learning-base based models.
find that while KalmanNet can reject maggituee -iitt--diittiittiinn noise.
2 Related Work
Our work builds on previous modifications of the KF for BMI decoding, as well as advancements in deep learning and model-based techniques. Modifications to the standard KF,, such as adapting it to non-linear dynamics observation models, common for BMI decoding. For example, Li et al. (66 used an unscented KF with a quadratic neural tuning model, achieving superior performance over the traditional KF, while others have shown that online adaptation of the KF parameters can have significant impacts on performance [29. Additionally, modifying the trust variable of the KF-the Kalman gain-has been used to improve the computational efficiency by pre-fixing the trust value (30]. Our work further extends this concept by allowing the Kalman gain to vary over time, enabling more flexible trust with a deep RNN. Pure recurrent and nerd-forwrrd neural networks have shown promise in BMIs. Feed-forwand
networks with a temporal convolution layer have outperformed the KF an online finger control task when using the ReFIT recalibration method 166. while RNN approaches, such as (31 and (17) have achieved even better results in predicting hand and fingers position and velocities in primates. KalmanNet incorporates the benefits of these deep learning approaches with the knowledge of real-world dynamics to improve the explainability

---

of the model while achieving similar performance. Thus, our work also aligns with broader efforts in model-based deep learning performance and explainability. For example, physies-informed neural networks (33) incorporate physical knowledge into the loss function as regularizer. KalmanNet can be framed similarly, with
(32]. where domain knowledge is combined with deep learning to enhance
real-world dynamics serving as regularizer for the underlying deep neural networks.
3 Methods
3.1 Neural Decoders
Throughout this study, we compared the performance of four neural decoders, three of which have been studied before in a BMI context: the Kalman filter (KF, (34), the temporal convolutional feed-forward network (tcFNN (16), and a decoder based on the long short-term memory recurrent neural network (LSTM (17). Here, we additionally adapt recent model, KalmanNet (KNet, (25)). to the BMI setting. Each decoder took in processed neural features and predicted movement kinematics (positions and velocities of the fingers; see Technical Appendix for more details).
Kalman filter. The KF algorithm tracks a state from noisy observations using a state-space (SS) description of the dynamics and the relationship between the observations and the tracked state. These are modeled as linear functions assuming Gaussian noise [35] Ch. 4]. our study, the KF tracks state composed of the position and velocity of both finger groups (total length of four). Like all algorithms in this study, it uses the 50ms binned spiking band power (SBP (36) of the brain activity as observations of the state (Figure D B). The KF computes the level of trust in the dynamics versus the observations with a variable called the Kalman gain (Figure C), and then uses it to merge both sources of information. When the linear Gaussian SS representation holds, the KF is known to achieve the optimal mean-squared error (MSE) [35] Ch. 4]. Equation shows how the Kalman gain (KG) is used to output kinematics prediction by interpolating between the prediction given by the dynamics and the one given by the brain activity: a high value of the Kalman gain up-weights the brain activity as the main source of information, while low value ensures the dynamics are the dominant term. The trainable parameters for the KF correspond to the linear observation model (C), the linear dynamies model, and the noise covariances of the state and the observations.
Predicted Dynamics Kinematics
Brain Activity
x
(I KG x C)+
KG
(1)
Prediction
tcFNN. The temporal convolutional ford-forward neural network (teFNN, [6]. (37) is "pure"
(data-driven) deep learning approach that uses a temporal convolutional layer on the input, four
fully-connected linear layers, and two-dimensional output to predict finger velocities. It has been shown to predict finger velocities better than linear approaches in offline and online BMI experiments
[16]. (37). The specific tcFNN architecture used throughout this paper was presented in [37 (see Technical Appendix for more details). Here, however, the ReFIT process [27]. which showed the best results for online experiments in [6 was not included, to compare only the baseline performance across algorithms.
LSTM. The long short-term memory (LSTM 17 network is type of RNN that uses memory cells and gates to control what information to remember and what forget. The LSTM has achieved state-of-the-art results in finger movement predictions, outperforming the tcFNN, the KF, and other linear and non-linear approaches [17]. The specific LSTT architecture used throughout this paper
was presented in [7).
KalmanNet. KalmanNet (25) is a tracking algorithm that uses (possibly mismatched) SS repre- sentation combined with deep learning [38]; It uses the KF as an inductive bias, by incorporating deep learning architecture based on RNNs that augments the Kalman gain computation (Figure [ C). This inclusion of the RNN allows for more flexible modulation of trust between dynamics and observations while maintaining the linear model used to model dynamics (evolution model, A) and the relationship between states and observations (observation model, C). It differs from the KF in that does not need an explicit model of the noise covariance of dynamics or observations, nor does it need to track the state covariance through time. The parameters of KalmanNet are the SS model parameters (necessary for the KF) and the trainable parameters of the included RNNs. Here, modified KalmanNet to use the same dynamics and observation models that have shown success
3

---

in the KF in the past [5]. (3), modified the loss function to account for the difference in scales between positions and velocities, and optimized the hyperparameters of the network for predicting finger positions and velocities.
3.2 Task and Data Acquisition
Two non-human primates, Monkeys N and W. were implanted with microelectrode arrays the hand area (precentral gyrus) of the right motor cortex of the brain. Monkey N was implanted with two 64-channel arrays and Monkey W with a single 96-channel array; both cases, recordings were limited to the best 96 channels. We recorded the SBP (power in the 100-1000Hz band) each channel, a useful and low-power feature for BMI applications (36]. The SBP was averaged in 50ms nonoverlapping bins for each channel. For training and testing, we only used the "active" channels (i.e., those with at least one threshold crossing per second on average, approximately 20-30 total channels each day) for KalmanNet. For all other decoders, used all channels that had threshold crossings morphologically consistent with action potentials [16]. (approximately 60-70 total channels, which included the "active" channels).
Both monkeys were trained to do 2-degree-offfreedom (2-DoF) dexterous finger task in which they had to move the index and muddle-ring-smale (MRS) fingers independently to acquire targets shown on a screen (Figure [ A). To successfully acquire targets, monkeys had to move their fingers to the required position and hold for minimum hold time (750 ms). A new set of targets appeared on each trial, and the monkey had five seconds to acquire them; if the monkey could not acquire the targets during that time, the trial was labeled as unsuccessful. On hand control trials, finger kinematics were measured using resistive bend sensors and shown to the monkey in real-time using custom visualization tool (MSMS (40).. On brain control trials, the 50-ms binned SBP was fed in real-time to a decoder that predicted finger kinematics and controlled the output visualization. The two modes-hand control and bruin control-were almost identical, with the exception being the reduced hold time (500 ms) during brain control trials to lower their difficulty. Analyses of performance during hand and brain control trials are referred to offline and online, respectively.
Neural
Spiking
Brain Activity
Decoder
Kalman fiber
Prodetion
Finger
(Kalmanitee
Brain Control
Real-Time
updates
6FFWN
releditis
Hand Control
Kalman filter
KalmanNet
Obsenvation Model
Observation: State
Finger kinematics
Flexion
State Conortanc
Osservation heite
Estrection
Footurc
CNCP
Extension
Kalman
Kalman
Gain
Catn
Figure 1: Task and Neural Decoders. (A) We trained monkey to do a 2-DoF finger task (shown on screen) while brain activity and finger kinematics (index and middle-ring-smaa (MRS) traces shown below) are recorded. The monkey do the task in hand control, using his hand, in brain control. (B) In brain control, the SBP of each brain channel is extracted and binned every 50 milliseconds. Each neural decoder takes and KalmanNet differ in how they compute the Kalman gain: the KF uses the equation shown, while KalmanNet uses a set RNNs.
bin and predicts position and velocity, or only velocity. (C) The KF
3.3 Decoder Training and Testing
For offline decoder testing, decoders were trained a supervised manner using a pre-recorded set of 400-500 hand-control trials from each day with synchronized brain activity and finger kinematics. The decoders were then tested on different set of 100-500 hand-control trials from the same day to evaluate offline performance. We trained decoders daily, as micro-movements in the microclectrode

---

arra can cause large changes in the recorded neural activity across days [4T]. We trained and tested 13 different instances of each decoder overall, 8 using Monkey N's data, and using Monkey W's data, and they were evaluated by their ability to predict finger positions and velocities (for KF, KalmanNet, and L.TTM) or just finger velocities (for the tcFNN).
When running online (real-time trials), decoders were trained on similar initial set of 400-500 hand-control trials and then tested in pairs each day in an ABA manner: decoder for set of trials, decoder 2 for another set, and then back to decoder 1 for another set, to account for possible shifts in signals or monkey behavior. All decoders were tested using a similar number of trials in each set. KalmanNet was compared online with the KF on four days, with the tcFNN on two days (during which the KF was also run), and with the LSTM on one day. All days on which the monkey attempted to do online trials with KalmanNet and at least one more decoder were included in our analysis. Only Monkey N was used for online experiments.
3.4 Performance Metrics
We used different metrics to determine decoder performance when tested offline versus online. When testing a decoder offline, we have access to the ground-truth finger kinematics and thus can directly measure prediction accuracy. We used the Pearson correlation coefficient and the MSE between the ground truth and the predictions and then averaged across fingers to get a single value for each position and velocity. To compare offline performance across models, we used combination of frequentist and Bayesian statistical methods. First, paired-sample I-tests on the MSEs as well as in the Fisher Z-transformed correlation coefficients [42] determined whether the difference was significant (a = 0.05) under the null hypothesis assumption. Second, we computed the Bayes factor (Bo1) to determine the ratios of likelihoods between the null and alternative hypotheses.
When testing a decoder online, without access to ground-truth finger kinematics, we can simply measure how well the monkey completed the trials. We used four metrics for online performance: success rate, trial times, path efficiency, and throughput. The success rate is the ratio of the number of trials successfully completed to the number of trials attempted. Trial time is the time taken to acquire both finger targets minus the hold time: higher trial times mean a slower or more erratic movement. The path efficiency measures the smoothness of the movements by computing the ratio between the distance between the start position and the target and the distance traveled by the monkey when completing the trial: higher path efficiencies mean smoother paths. Finally, the information throughput (measured in bits/s) accounts for each trial's s difficulty (distance between start and target) and completion time (16). with higher throughput representing faster target acquisitions. Note that all metrics except the success rate were computed only with successful trials. Since there is some variability in performance across days due to mivro-movements of the electrode array (4T), we normalized all the metrics- except success rate-frve froe-fom each online day by the average performance of KalmanNet during that day, to be able to aggregate each decoder's performance across testing days.
3.5 KalmanNet as a Non-linear Trust System
To understand the mechanism used by KalmanNet to accurately predict finger kinematics, we inspected the network's 's output, which corresponds to the Kalman gain matrix in the KF. For easier interpretation, we computed the Frobenius norm of KalmanNet's Kalman gain and compared it to the norm of the predicted velocities. As a metric of the relationship, we used the Pearson correlation coefficient between the Frobenius norm of the Kalman gain and the vector norm of the velocities over time across all 13 offline days tested for both monkeys (n = for Monkey N, n = 5 for Monkey W).
Additionally, to replicate KalmanNet's observed correlation between Kalman gain and velocity in the regular KF, we modified the KF to have heteroscedastic process noise. We refer this model as the heteroscedastic Kalman filter (HKF). The process noise determines how noisy we believe the dynamics to be: with noisier dynamics, the KF will trust observations more (higher Kalman gain), and with less noisy dynamics, the KF will trust the dynamics more (lower Kalman gain). Thus, to force the Kalman gain to covary with the velocity, we made the process noise, specifically the parts related to the velocities, increase linearly with the norm of the ground truth velocity (see Technical Appendix for more details). Note that this model needs the ground truth velocity to properly vary the process noise, which makes it unsuitable for online experiments.
5

---

3.6 Robustness to Real- Weal-World Operation
To test the models' offline robustness to real-world operation, we first simulated noise spikes on the data. This may happen in real-world scenario with, for example, movement artifacts loose electrical connections, and we investigated whether this would have unsafe consequences when controlling a physical device, such as big spikes in the velocity predictions. The noise spikes were simulated by adding fixed magnitude to all neural channels of the testing dataset for a given number of 50-millisccond bins. We varied the magnitude from 0.1 to 100 times the standard deviation of the training set SBP and varied the duration from one to five 50-millisecond bins. While this may seem like a large amount of noise, for BMI applications, it is possible to have unexpected mV level noise added to μV level neural signals. The noise was injected randomly for 5% of the total time steps on each testing day, and we tested it offline across 13 different days for both monkeys. We measured the impact of the noise in the models with the MSE of the velocity normalized by the baseline velocity MSE (no noise added to the testing dataset).
Additionally, we tested the decoders' ability to generalize to unseen task contexts, a key property for decoders to work in real-life scenarios rather than just in highly constrained research settings 337. [43]. On six different days, we modified the task by having the wrist flexed by 30 degrees and adding spring to resist flexion and facilitate extension [3] Then, we trained off-context and on-context decoders. Off-contex: decoders were trained on a normal set of trials (no wrist or spring modifications) and then tested on the last 20% of the spring+ wrist context trials. On-context decoders were trained on the first 80% of the wprin context trials and then tested on the last 20% (see Figure 6 A). We evaluated the generalization performance of each algorithm by measuring the percentage increase in velocity MSE from the on-context to the off-context decoders.
4 Results
4.1
Offline and Online Performance
To determine how well KalmanNet could fit the data compared to state-of-the art deep learning techniques, we first assessed its offline performance in predicting finger kinematies from brain data. We evaluated KalmanNet over n = 13 days across the two monkeys, comparing it with the KF and the deep learning based tcFNN and LSTM models.
The LSTM was the highest- performing approach in a previous study [77]. In terms of correlation with velocity, which controls the visualization when running online, KalmanNet did not significantly differ from the LSTM (p- = 0.64, Ba = 3.3) and had a significantly higher correlation than the KF (p 1e-7, Bo = 0.048) and the tcFNN (p < 0.001, Bo 0.062: Figure B). All approaches had similar correlations with position, other than the tcFNN, which is a velocity-only approach. Regarding velocity MSE, there was also no significant difference between KalmanNet and the LSTM (p 0.72, Bo1 = 3.4) or between KalmanNet and the tcFNN (p = 0.29, Bo1 = 2.1). However, KalmanNet significantly outperformed the KF (p < 0.01, Boi 0.14). In terms of position, there were no significant differences in correlation (KNet vs. LSTM, p = 0.32, Bb1 = 2.4: KNet vs KF, p 0.93, By = 3.5: LSTM vs. KF, p 0.08, Bo1 = 1.1) or MSE (KNet vs. LSTM, p = 0.26, BuI = 2.0: KNet vs KF, p = 0.77, Bu1 = 3.5: LSTM vs. KF, = 0.1, Bu1 = 1.1), although KalmanNet had higher variance in MSE. Overall, is important to note that the absence of p-values below the significance level for position and velocity correlations and MSE between the LSTM and KalmanNet should not be interpreted as definitive evidence that the two models are equivalent. Instead, it indicates that the data did not provide strong enough evidence to conclude that the models were different. However, the Bayes factor (Bo1) values between 2 and 3.4 suggest that the models may have comparable performance, with the null hypothesis (models are equivalent) being least twice as likely as the alternative hypothesis (models are different). This suggests a higher likelihood that any observed differences are not substantial, though it remains essential to consider the variability and sample size when interpreting these results.
However, because this is motor control application, it is important to evaluate performance online, with a user the loop. Therefore, we tested all algorithms online across five sessions (number of trials: KF = 601, tcFNN = 576, KNet = 2801, LSTM 393) and compared their performance in terms of success rate, throughput, trial times, and path efficiencies. Figure 3 (right) shows the performance of all approaches normalized to the performance of KalmanNet (except for the success

---

Position
Index
Velocity
0.8
Comclation
Velocity
MSE
lo-7
08
Conclation
Position
M5E
1.0
0.7
Index
Flexion
Correlation Caefficient
4.5
0.6
Ero pavenbs Meenn
Eeternsin
03
-06
KNees
04
KIket
0.3
0.
004
0.
00
00
LSTM
011
011
0.
KRET
LSTR
RP
KNET
LSTR
00
KNET
LSTMi
KRF7
LSTM
00
Figure 2: Offline Performance. (A) Traces of ground truth index position and velocity (in blue) versus the predictions from each neural decoder. Note that tcFNN only predicts finger velocity. (B) Velocity (left) and position (right) performance in terms of correlation and MSE for each neural decoder. Square markers and error bars denote the mean and the standard error of the mean, respectively. Tested across 13 days from both monkeys.
rate). We found that KalmanNet outperformed the KF and tcFNN across all metrics (Figure 3 Supplemental Videos 1 and 2, respectively), allowing the monkey to perform faster and more accurate trials. Compared to the LSTM, KalmanNet achieved lower throughputs (p < 1E 16) and higher trial times (p < 1E 5), meaning slower trials overall. However, it had higher overall success rates (97% vs. 90%) and more efficient and smoother paths (p < 1 7: Figure 3 Supplemental Video 2). which is consistent with the inclusion KalmanNet's dynamics model. The online example traces tell similar story, with the KF and tcFNN showing more oscillations in the path and more unsuccessful trials (Figure 3 A).
100%
Success Rates
MRS Pos.
90%
Extension - Flexion
Index
1s
80%
Throughput
Talled
Normalized Metric
1.25
1.00
KNet
Trial Times
1.25
1.00
Path Efficiency
Time
tcFNN
KNet
LSTM
Figure 3: Online Performance. (A) Traces the positions of the index (blue) and middle-ring-smal (yellow) fingers during online control, across neural decoders. Blue-filled boxes represent index tar- gets the flexion-extension range, while yellow-filled boxes represent MRS targets. Green-outlined boxes represent successful trials, while red-outlined boxes show failed trials. (B) Online metrics of performance. Throughput, trial times, and path efficiencies are normalized to the corresponding KalmanNet value for each day. Tested on monkey N across T 601 (KF), 576 (tcFNN), 2801 (KNet), 393 (LSTM) trials in total of five days.
4.2 Modulating Trust between Dynamies and Observations
Notably, KalmanNet can match or nearly match the highly nonlinear black-box LSTM performance. In KalmanNet, the structure of the linear KF intact, such that the performance improvement comes from the RNNs used to compute the Kalman gain. This is the KF's metric of trust that determines how to interpolate between dynamies and observations (see Methods). To analyze this, we first inspected the regular KF's Kalman gain and observed that it converges to fixed value in the first few seconds of session (inset Figure 4 A, orange line), line with known steady-state behavior [35] Ch. 4.3]. We then evaluated the KalmanNet gain computation during inference. Interestingly, we found
7

---

that it correlated strongly with KalmanNet's output velocity, both offline (mean + standard error of the mean 0.79 = 0.02) and online (0.60 = 0.02; Figure center). This correlation suggests that KalmanNet behaved as non-linear trust system, switching rapidly towards trusting the neural activity when the velocity was high and quickly switching to trusting the dynamics when needed to stop.
A
Offline
g.gain (gain-Velocity
Online
D Velocity Prediction
1e-5
08-
MRE
134e
Normalized Kalman Gain Norm
25
Fivdered vilodty
0.06
Velocity Mean Squared Enor
Vildty
Srabiltation
W.en W.eN ApoRA
Comelation Coefooent
Kalman Cain Norm
0.0
Norn Melociy
Comrelation feefitennt
0.5
6.6
4.26
8.4
1s
5/2
02-
0.02
0.0
0.0
Offline
Online
Figure 4: KalmanNet as a Non-linear Trust System. (A) Offline trace showing the norm of the Kalman gain for KalmanNet (in green) and for the KF (in orange), together with the predicted velocity for KalmanNet (black). The inset shows the first five seconds of trials, during which KF's Kalman gain converges
fixed value [35 Ch. 4]. (B) Average correlations between Kalman gain and velocity for KalmanNet, HKF, and KF, for offline (n 13 days, across both monkeys) and online (n days, only Monkey N) trials. (C) Online trace of the norm of the Kalman gain for KalmanNet (green), together with the predicted velocity (black). (D) Offline velocity prediction performance comparison between KF, HKF, and KalmanNet, terms of correlation coefficient (left) and MSE (right), Arrows represent the addition of heteroscedastic process noise transform the KF into HKF. Tested across n 13 days and both monkeys.
To further explore whether this observed modulation of trust is the key to KalmanNet's improved performance, we used our knowledge of the true velocity to manipulate the noise covariance of the regular KF to achieve the same effect. Specifically, we created process noise that increased with high velocities and decreased with low velocities, forcing the Kalman gain to covary with the output velocity (HKF model; Kalman gain correlation with velocity output across days and monkeys = 0.21 + 0.03; Figure B). We found that this modification made the new model perform well KalmanNet when predicting velocity, both in terms of correlation (HKF 0.64 vs. KNet = 0.65. p 0.57) and MSE (HKF = 3.4E vs. KNet = 3.3E 7, p 13), which showcases the benefits of the trust modulation mechanism observed in KalmanNet.
KF model with heteroscedastic
4.3 Robustness to Noise and Change in Task Context
The primary motivation for using a principled KF rather than a black box neural network its safe operation, under based SS models to improve kinematic predictions from noisy sensors. Therefore, we evaluated these approaches in the face of artificialll-injecte channels (see Methods). This allowed us to study the effect on the output velocity under various real-world scenarios. Given the small magnitude of brain signals, noise artifacts can be much larger than the signal features of interest. We modeled those with extreme noise additions 100 times the standard deviation in the training data. Surprisingly, KalmanNet had the largest increase in MSE associated with injected noise (by a factor of ~ 4300 for noise on three consecutive time bins, when the noise magnitude was 100x the std. dev.). It exhibited high-velocity spikes the noise injection points (Figure A and Supplemental Figure 99. Perhaps surprisingly, the best performer was the LSTM, which changed very little from the no-noise case. showed just a 2x increase in velocity MSE when injected with noise level 100 times the training data's 's standard deviation. It is important to note that the noise injected was not present in the training data, so even the regular KF performed poorly. However, unmodeled noise is a common scenario in BMIs stemming from, for example, problems with the physical connection to the recording electrodes or electrode movement with respect to neurons 41
changing circumstances. The KF conventionally leverages physics-
sensor noise of varying magnitudes and duration across

---

Magnitude =
Duration =
B
Duration =
C
Magnitude
Indox
Normal Ized wewllleeeelllle Velocity MSE
Velocity MSE
2.0-
Finger velocity (a.u.)
HgC
103
KAET
0.001
LSTM
2.00
LSTM
0.000
102
1.5
1:50
-0.001.
101
1.5
100ms
-0.003
100
1.00
10-1 100
16l
102
Noice Magnitude (multiples of std. std.
Nosse Duration
Figure 5: Robustness to Injected Noise. (A) Offinn traces across neural decoders as compared to the ground truth index velocity (black). Grey columns represent noise injections of 5 times the standard deviation of the training dataset distribution, for duration of one time bin. (B) Change in normalized velocity MSE as the noise magnitude changes, for a fixed noise duration of time bins. (C) Change in normalized velocity MSE the noise duration changes, for fixed noise magnitude of 1x the training dataset distribution's 's standard deviation. Tested on 13 days for Monkeys N and W.
Another potential benefit of KF with simple linear model is the ability to generalize to new contexts. Therefore, we evaluated all decoders by slightly varying the monkey task. We trained all models on the normal task context and then tested them in a new context [37] [43] (off-context performance), which contained spring to resist flexion, as well as 30-degree angle change in wrist position (Figure 6 A). We then compared their performance to training and testing the new context (on-context performance). The normal KF showed the lowest drop in absolute performance (7.98% MSE), exhibiting good generalization to the new model but still showing the lowest absolute performance. On the other hand, all neural network models, including KalmanNet, showed substantially higher MSE increases due to the context shift (34.20% for teFNN, 44.45% for KNet, 50.34% for LSTM).
Generalization
le
Velocity MSE
Contest
Normal Context
Add spring
Spring i Wrist Context
Mean Squared Erro
Context
Change wristank ongl
3.0
All Trials
100%
80% Toin
20%
50.5%
Trals
Test
T
Context
20-
Off-Context
LSTM
Figure 6: Generalization Across Task Contexts. (A) Illustration of how off-context and on-context decoders are trained. Monkeys performed a variation of the task that included a change in wrist angle and a spring that resisted flexion. Off-contextt decoders were trained on a normal set of trials from the same day and then tested on the last 20% of the trials of the spring+wrist context. On-context decoders were trained and tested on the same spring+wrist context. (B) Change in velocity MSE between on and off-context decoders for each decoder. Tested across n 6 days of Monkey N.
5 Discussion
In this paper, we explored the trade-offs between explainable lincar models and deep-learning approaches to BMI decoding. We modified and retrained KalmanNet (25), which fuses deep learning techniques with the classic KF, We showed that KalmanNet can match or outperform existing state-of- the-art decoders in offline and online modalities. We also showed that the explainability of KalmanNet can be leveraged to improve a linear model up to the -fate-of-the-ar -ttth-art art standard but that it comes at the cost of greater error when faced with -fi------itrttttionn noise.
KalmanNet approached the state-of-thed ortt---rttt-tt performance offline, achieving comparable velocity and position predictions to the LSTM [17). This performance does not necessarily transfer to online trials,
as has been studied previously [9]. [44.. and as we observed here with the tcFNN model. Offline, the tcENN had significantly better velocity predictions than the KF, but online, it had overall worse

---

success rates, throughput, trial times, and path efficiencies. This lower online performance may follow from the fact that the monkey is in the loop, reacting real-time to the output of the decoder. which
may change the dynamics of the problem versus testing decoders offline (19). [44] Note, however, that we did not implement ReFIT [16]. [27] for teFNN, a recalibration technique that can improve
tcFNN online performance [T6]; similar method could potentially be developed for all decoders (albeit with potential differences in performance improvements), and thus we decided to compare all models with their baseline non-ReFIT performances (see Technical Appendix). KalmanNet, on the other hand, was able to work well online and did similarly well to the LSTM: it acquired targets more slowly but had higher success rates well smoother paths, which may be more desirable for applications controlling physical devices. Moreover, while we currently use existing SS modeling that is learned separately from the filtering task, the fact that KalmanNet converts the KF into a machine-led machine-learning model indicates that one can potentially learn the observation model (C) and its computation of the Kalman gain. As demonstrated in (45), this could lead to both improved MSE performance and provide information on the usefulness of existing approaches for forming SS representations in BMI.
KalmanNet functioned as a non-linear trust system, quickly re-weighting the contributions to its predictions from either the model inputs or internal dynamics depending the desired output. For stopping. since the dynamical model provides an explicit term for exponentially decreasing the output velocity (see Technical Appendix), KalmanNet learned to trust the dynamical model. For fast movements, only the neural activity contains this information. Therefore, it chose to put more confidence in the neural inputs. We constructed new linear model that enforced this observed behavior (HKF), and it matched offline state-of-the-ar results, albeit assuming that the velocity level is known. This showcases some of the potential benefits of using more explainable and, therefore, trustworthy model compared to a black-box decoder. Additionally, the smart switching between dynamics and inputs is an interesting behavior that could be enforced even in black-box deep-learning decoders to potentially improve decoding. Similar approaches were recently explored for speech neuroprostheses, using a combination of language model (which can be thought of as language "dynamics") and an RNN predicting phonemes from neural activity [14]. One could modify that approach by, for example, having model that learns to trust the neural activity for starting words or when faced with uncommon transitions between phonemes, and to trust the language model on more common phoneme transitions or for being silent. This approach would be similar to the behavior we observed in KalmanNet.
A disadvantage of combining neural network with Kalman filter is that the overall method carries over some limitations from both approaches. We found that when faced with out-of-distribution noise, KalmanNet had the highest increases in MSE, up to 1000x, when faced with noise 100x the original distribution's 's noise. This effect likely follows from the similar behavior exhibited by the KF, which increased its MSE 100x with the same noise magnitude. Nonetheless, the fact that KalmanNet preserves the operation of the KF indicates that it can be extended to outlier-robust variations of the KF, e.g.. [46). or alternatively, combined with hypernetworks for handling multiple noise levels [47). Interestingly, the pure RNN approach (the LSTM) had the lowest increase in error at high noise magnitudes. This result may be unexpected for control-focused researchers, who often choose linear approaches due to the perceived higher safety of the more explainable models.
In conclusion, the results shown here with KalmanNet suggest that we may not need to sacrifice explainability to achieve results in line with the state-of-the-art deep-learning models and that using a mechanism based on gating trust between dynamics and observations may provide valuable insight for developing future models.
6 Acknowledgements
We thank Eric Kennedy for animal and experimental support and Professor Hans-Andrea Loeliger for helpful discussions. This work was supported by NSF GRFP 1841052, NSF EFRI BRAID 2223822, CASI award (1021865.01 from the Burroughs Wellcome Fund, NSF NCS 1926576, NIH RO1-NS- 10513-01--11 NIH U41 NS129436 and the Agencia Nacional de Investigacion Desarrollo (ANID) of Chile.
10

---

References
[1] B. S. Armour, E. A. Courtney-Long, M. H. Fox, H. Fredine, and A. Cahill, "Prevalence and Causes of Paralysis- Unnied States, 2013," American Journal of Public Health, Oct. 2016.
[2] D. W. Otter, J. R. Medina, and J. K. Kalita, "A survey of the usages of deep learning for natural language processing," IEEE transactions on neural networks and learning systems, 2020. [3] L. Jíao and J. Zhao, "A survey on the new generation of deep learning in image processing," Ieee Access, 2019.
[4] K. Zhou, Y. Yang, 'T. Hospedales, and T. Xiang, "Deep domain-adversarial image generation for domain generalization," in Proceedings of the AAAI conference on artificial intelligence, 2020.
[5] M. Kang and J. Park, "Contragan: Contrastive learning for conditional image generation," Advances in Neural Information Processing Systems, 2020.
[6] M. Ding, Z. Yang, W. Hong, W. Zheng, C. Zhou, D. Yin, J. Lin, X. Zou, Z. Shao, H. Yang, et al., "Cogview: Mastering text-to-image generation via transformers," Advances in Neural Information Processing Systems, 2021.
[7] L. Deng, J. Li, J.-T. Huang, K. Yao, D. Yu, F. Seide, M. Seltzer, G. Zweig, X. He. J. Williams, et al., "Recent advances in deep learning for speech research microsoft," in 2013 IEEE international conference on acoustics, speech and signal processing, IEEE, 2013.
[8] Y. Guo. Y. Liu, A. Oerlemans, S. Lao, S. Wu, and M. S. Lew, "Deep learning for visual understanding: A review," Neurocomputing, 2016.
[9] R. Ranjan, S. Sankaranarayanan, A. Bansal, N. Bodla, J.-C. Chen, V. M. Patel, C. D. Castillo, and R. Chellappa, "Deep learning for understanding faces: Machines may be just good, or better, than humans," IEEE Signal Processing Magazine, 2018.
[10] W. Samek, T. Wiegand, and K.-R. Müller, "Explainable artificial intelligence: Understanding. visualizing and interpreting deep learning models," arXiv preprint arXiv: / 7708. 08296, 2017. [11] F. R. Willett, D. T. Avansino, L. R. Hochberg, J. M. Henderson, and K. V. Shenoy, "High- performance brain-to- communication via handwriting," Nature, May 2021.
[12] D. A. Moses, L. Metzger, J. R. Liu, K. Anumanchipalli. G. Makin, Sun, J. Chartier, M. E. Dougherty, P. M. Liu, G. M. Abrams, et al., "Neuroprosthesis for decoding speech in a paralyzed person with anarthria," " New England Journal of Medicine, 2021.
[13] S. L.. Metzger, K. T. Littlejohn, A. B. Silva, D. Moses, M. Seaton, R. Wang, M. Dougherty, J. R. Liu, P. Wu, M. A. Berger, al., "A high- high-performance neuroprosthesis for speech decoding and avatar control," Nature, 2023.
[14] F. R. Willett, E. M. Kunz, C. Fan, D. T. Avansino, G. H. Wilson, E. Y. Choi, F. Kamdar, M. F. Glasser, L. R. Hochberg, S. Druckmann, K. V. Shenoy, and J. M. Henderson, "A high-performance speech neuroprosthesis, Nature, Aug. 2023.
[15] S. R. Nason, M. J. Mender, A. K. Vaskov, M. S. Willsey, N. Ganesh Kumar, T. A. Kung, P. G. Patil, and C. A. Chestek, "Real-time linear prediction of simultaneous and independent movements of two finger groups using intracortical brain-machine interface," Neuron, Oct. 2021.
[16] M. S. Willsey, S. R. Aason-Tomaszewskke S. R. Ensel, H. Temmar, M. J. Mender, J. T. Costello, P. G. Patil, and C. A. Chestek, "Real-time brain-machine interface in non-human primates achieves high-velocity prosthetic finger movements using shallow feedforward neural network decoder," Nature Communications, Nov. 2022.
[17) J. Costello, Temmar, L. Cubillos, M. Mender, D. Wallace, M. Willsey, P. Patil, and C. Chestek, "Balancing memorization and generalization in RNNs for high performance brain- machine Interfaces." Advances in Neural Information Processing Systems, Dec. 2023.
[18] W. Wu, M. Black, Y. Gao, M. Serruya, A. Shaikhouni, J. Donoghue, and E. Bienenstock, "Neural decoding of cursor motion using kalman filter," Advances in neural information processing systems, 2002.
[19] D. R. Deo, F. R. Willett, D. T. Avansino, L. R. Hochberg, J. M. Henderson, and K. V. Shenoy, "Brain control of bimanual movement enabled by recurrent neural networks." Scientific Reports, Jan. 2024.
[20] C. C. Aggarwal et al., Neural networks and deep learning. Springer, 2018.
11

---

[21] P. Linardatos, V. Papastefanopoulos, and S. Kotsiantis, "Explainable ai: A review of machine learning interpretability methods," Entropy, 2020.
[22] S. Oh, B. Schiele, and M. Fritz, "Towards reverse-enginerring black-box neural networks," Explainable AI: interpreting, explaining and visualizing deep learning, 2019.
[23] B. Wodlinger, J. Downey, F. Tyler-Kabara, A. Schwartz, M. Boninger, and J. Collinger, "Ten- dimensional anthropomorphic arm control in a human brain-machine interface: Difficulties, solutions, and limitations."" Journal of neural engineering, 2014.
[24] A. B. Ajiboye, F, R. Willett, D. R. Young, W. D. Memberg, B. A. Murphy, J. Miller, B. L. Walter, J. A. Sweet, H. A. Hoyen, M. W. Keith, P. H. Peckham, J. D. Simeral, J. P. Donoghue, L. R. Hochberg, and R. F. Kirsch, "Restoration of reaching and grasping movements through brain-controlle muscle stimulation in person with tetraplegia: A proof-of-concept demonstration." T The Lancet, May 2017.
[25] G. Revach, N. Shlezinger, X. Ni, A. L. Escoriza, R. J. G. van Sloun, and Y. C. Eldar, "Kalman- Net: Neural Network Aided Kalman Filtering for Partially Known Dynamics," IEEE Transac- tions on Signal Processing, 2022.
[26] Z. Li, J. E. O'Doherty, T. L.. Hanson, M. A. Lebedev, C. S. Henriquez, and M. A. L. Nicolelis, "Unscented Kalman Filter for Brain-Machine Interfaces," " PLOS ONE, Jul. 2009.
[27] V. Gilja, P. Nuyujukian, C. A. Chestek, J. P. Cunningham, B. M. Yu, J. M. Fan, M. M. Churchland, M. T. Kaufman, J. C. Kao, S. I. Ryu, and K. V. Shenoy, "A high-performance neural prosthesis enabled by control algorithm design," Nature Neuroscience, Dec. 2012.
[28] C. S. L. Tsui, J. Q. Gan, and S. J. Roberts, "A self- paced brain-comppi computer interface for controlling a robot simulator: An online event labelling paradigm and an extended Kalman filter based algorithm for online training." Medical & Biological Engineering & Computing, Mar. 2009.
[29] S. Dangi, S. Gowda, R. Héliot, and J. M. Carmena,' "Adaptive Kalman filtering for closed-loop Brain-Machin Interface systems," in 2011 5th International IEEE/EMBS Conference on Neural Engineering, Apr. 2011.
[30] W. Q. Malik, W. Truccolo, E. N. Brown, and L. R. Hochberg, "Efficient Decoding With Steady- State Kalman Filter in Neural Interface Systems," IEEE Transactions on Neural Systems and Rehabilitation Engineering, Feb. 2011.
[31] D. Sussillo. P. Nuyujukian, J. M. Fan, J. C. Kao. S. D. Stavisky, S. Ryu, and K. Shenoy, "A recurrent neural network for closed-loop intracortical brain- machine interface decoders," Journal of Neural Engineering, Mar. 2012.
[32] N. Shlezinger, J. Whang, Y. C. Eldar, and A. G. Dimakis, "Model-Based Deep Learning." Proceedings of the IEEE, May 2023.
[33] M. Raissi, P. Perdikaris, and G. E. Karniadakis, "Physics- -nnforee neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations," Journal of Computational Physics, Feb. 2019.
[34] R. F. Kalman, "A New Approach Linear Filtering and Prediction Problems," Journal of Basic Engineering, Mar. 1960.
[35] J. Durbin and S. J. Koopman, Time series analysis by state space methods. OUP Oxford, 2012.
[36] S. R. Nason, A. K. Vaskov, M. S. Willsey, E. J. Welle, H. An, P. P. Vu, A. J. Bullard, C. S. Nu,
J. C. Kao, K. V. Shenoy, T. Jang. H.-S. Kim, D. Blaauw, P. G. Patil, and C. A. Chestck, "A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain- wrainne interfaces," Nature Biomedical Engineering, Oct. 2020.
[37] H. Temmar, M. S. Willsey, J. T. Costello, M. J. Mender, L. H. Cubillos, J. L. Lam, D. M. Wallace, M. M. Kelberman, P. G. Patil, and C. A. Chestek, Artificial neural network for brain-m interface consistently produces more naturalistic finger movements than linear methods, Mar. 2024.
[38] N. Shlezinger, J. Whang, Y. C. Eldar, and A. G. Dimakis, "Model-based deep learning." " Proceedings of the IEEE, 2023.
[39] A. K. Vaskov, Z. T. Irwin, S. R. Nason, P. P. Vu, C. S. Nu, A. J. Bullard, M. Hill, N. North, P. G. Patil, and C. A. Chestek, "Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter," Frontiers in Neuroscience, 2018.
[40] R. Davoodi and G. F. Loeb, "MSMS software for VR simulations of neural prostheses and patient training and rehabilitation." in MMVR, 2011.
12

---

[41] J. A. Perge, M. L. Homer, W. Q. Malik, S. Cash, E. Eskandar, G. Friehs, J. P. Donoghue, and L. R. Hochberg, "Intra-day signal instabilities affect decoding performance in an intracortical neural interface system," Journal of Neural Engineering, Apr. 2013.
[42] X.-L. Meng and D. B. Rubin, "Performing likelihood ratio tests with mullippy-immutte data sets," Biometrika, 1992.
[43] M. J. Mender, S. R. Nason- Tomaszewski, H. Temmar, J. T. Costello, D. M. Wallace, M. S. Willsey, N. Ganesh Kumar, T. A. Kung, P. Patil, and C. A. Chestek, "The impact of task context on predicting finger movements in brain-machine interface," eLife, J. A. Pruszynski,
T. R. Makin, and C. Ethier, Eds., Jun. 2023.
[44] J. P. Cunningham, P. Nuyujukian, V. Gilja, C. A. Chestek, S. I. Ryu, and K. V. Shenoy, "A closed-loop human simulator for investigating the role of feedback control brain-machine interfaces," Journal of Neurophysiology, Apr. 2011.
[45] I. Buchnik, D. Steger, G. Revach, R. J. van Sloun, T. Routtenberg, and N. Shlezinger, "Latent- KalmanNet: Learned Kalman filtering for tracking from high-dimensional signals," IEEE Transactions Signal Processsing, 2024.
[46] S. Truzman, G. Revach, N. Shlezinger, and Klein, "Outlier-isennittiv Kalman filtering using NUV priors," in JEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023.
[47] X. Ni, G. Revach, and N. Shlezinger, "Adaptive KalmanNet: Data-driven Kalman filter with fast adaptation." in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024.
13

---

A Technical Appendix
A.1
Kalman filter
A.1.1 Structure
The regular Kalman filter was first introduced by Kalman (3) and provides a reliable way of tracking state variables over time, given periodic sensor measurements that contain some information about the state. In our case, the state to track is composed of the kinematics of both finger groups:
[Pix Pmurs Vidx Umrs
(2)
Where p; represents the flexion angle in arbitrary units (from O to 1) for finger group while uy represents the flexion velocity in arbitrary units/s for finger group i. The sensor measurements, in our case, correspond to the spiking band power (SBP) of the neural channels. Thus, observation at time & would be:
y1
[y1 3/2 Unl
Where corresponds to the number of neural channels used during the experiment. The KF predicts a new state by first computing the Kalman gain, which uses information about the noise covariance of the dynamics (W) and observations (Q) to determine the optimal interpolation between dynamics and observation predictions (equation
-tj-11 = Art-1/t-1
(4)
Pi|t-1 AP +W
(5)
Ki (CP- + +Q)
(6)
Fejt (I K,C)=.p-1 + Kiye
In equations 4 through 7 A represents the linear dynamical model that encodes the natural evolution of the state through time, C. the linear observation model that relates the observations to the state, and Peje the covariance of the state prediction. Equation 7 uses those models, together with the Kalman gain, compute prediction of the state at time Taje- When running the Kalman filter online, since the monkey can see in real-time the position of each finger group of the virtual hand, we set the covariance of the position estimate zero, which has shown improve results previously [15] 39 Additionally, to control the virtual hand on the screen in online trials, we calculated its position using the following equation: pos, = 0.02 pos, + 0.98(pos, + 50 vel,), which allowed us to interpolate between the position predicted directly (pôs,). and the position predicted through the velocity (vel.). The velocity term allows for smooth movements, while the position term helps to stabilize the overall position prediction, and the specifie values were found empirically.
A.1.2 Model Training
The Kalman filter parameters (dynamics model A, observation model C, and noise covariances Q and W)) were computed using a combination of domain knowledge and a data-driven approach. The linear dynamics model A, which represents the evolution of the state through time, had the structure shown in equation X [39]. The model integrates velocity (with dt 50ms being the bin size) and assumes no relationship between positions at time
and those at time t, well as no relationship between velocities at time t - and velocities at time t. The velocity transition sub-matrix, represented in & with Avel, was learned from an initial calibration run (see Methods) using a least-squares approximation. Note that across days, the diagonal elements of And were always strictly greater than and lower than 1, representing an exponential attenuation of the velocity through time: if input is applied to the movement, the dynamical model allows for fast stopping.
equation
14

---

0
0
1
0
dt
A
(8)
0
0
Avel
0
0
The observation model C was computed in similar way. The only difference is that assumptions were made about the structure, and a bias term was added to account for the difference hase levels. Then, given and C, we computed the noise covariance for the dynamics (W) and the observations (Q) using the maximum likelihood estimation. For W, we additionally set the noise covariance of the
position terms to zero, assuming that the error only propagated through the velocity
[39]
In the training Kalman filter model, an ASUS TUF A15 laptop equipped with an NVIDIA 1660 Ti GPU was utilized. The model took approximately 10 seconds to train for each day.
A.2
KalmanNet
A.2.1
Structure
The main difference between KalmanNet and the Kalman Filter is that KalmanNet uses a set of recurrent neural networks (RNNs) to compute the Kalman Gain, In our implementation, KalmanNet uses the same linear dynamics (A) and observation (C) models, but now does not require an explicit definition of the noise covariances. The full architecture, explained in more detail in (25). is shown in Figure Briefly, it attempts to compute the Kalman gain by using gated recurrent units (GRUs, a type of recurrent neural network) to implicitly model the noise covariance of the dynamics (GRU 2 in Figure 7. the noise covariance of the observations (GRU in Figure 7 and the prediction covariance of the observations (GRU 3 in Figure 7 This implicit modeling of the noise covariances allows KalmanNet to dynamically shift the Kalman gain depending on the current state and inputs to choose better interpolation between states and observations. Overall, the set of equations for KalmanNet is reduced to Aqqutiios/aandl with K, corresponding to the output of the network at time f.
The inputs to the KalmanNet network are variations on the Kalman filter variables: the observation difference (Fi, eq. 9). the innovation difference (F2, eq. 10), the forward evolution difference (F3,
eq. I1). and the forward update difference (F4, eq. 12 25
Fi = y/ 9t-1
(9)
F2 y/ Crret-1
(10)
F3 1t-1|t-1 I2-2[t-2
(11)
F4 = 1t-1[t-1 It-1[t-2
(12)
When running online, the position of the virtual hand was calculated as for the Kalman filter, integrating the velocity prediction and using 2% the position prediction for stabilization.
A.2.2 Model Training
To train the KalmanNet network, the loss function used was scaled mean-squared error (MSE) that accounted for the differences in scale between position and velocity, computed between the predicted and ground truth output. The network does not produce the predicted state directly; instead, it generates the Kalman gain, for which we do not have a ground truth. Consequently, for each time step, standard update and prediction procedure network output as a component of the computation (using equations and 77. Sequences of sixty time bins (equivalent to three seconds of data) were passed through the network, and then the loss was computed as the average scaled MSE between the output at each time step and the ground truth state from the training data.
executed, incorporating the KalmanNet
The KalmanNet network for each day was trained for 300 iterations, each taking batch of random sequences of length 60 in the training data (Figure Since to produce output, KalmanNet needs do a forward pass on the network, as well as a Kalman filter step, most of the py optimizations for network training could not be used, which slowed down training. Overall, the network training
15

---

F2
Unnar
Linear
F3
Limeer
RellU
Linear
Linear
ReLU
Kalman
Gain
Figure 7: KalmanNet architecture Diagram of the components of the KalmanNet network. consists of three GRUs plus seven linear + ReLU layers that try to model the normal way of computing the Kalman gain 25 Fi through F4 correspond to the input features from equations through 12
took around 10-15 minutes per day on an Asus TUF A15 laptop equipped with an NVIDIA 1660 Ti GPU.
Velocity
Correlation
MSE
0.7
0.6
0.5
3.0
0.3
0.2
2.5
0.1
10
1
100
150
length (number 50ms bins)
Figure 8: Sensitivity analysis of sequence length during training. Offline velocity correlation (left) and MSE (right) for KalmanNet, under different sequence lengths employed during training. The horizontal axis represents the number of 50ms bins; the one used throughout corresponds to 60, or equivalently, three seconds. Computed across all n = 13 days for both monkeys.
A.3 Heteroscedastic Kalman filter
The heteroscedastic Kalman filter (HKF) equivalent to the KF all but the process noise covariance (W). ) To enforce behavior similar to what observed with KalmanNet's output covarying with the
16

---

velocity, we forced the dynamics noise covariance in the HKF to also covary with velocity. At time t, and given base dynamics noise model W (shown in equation 13). we modified the velocity-related terms using the ground truth velocity of index and MRS at time I, Ut to produce new time-synced noise model W (equation TT). In the new noise model, a corresponds to a constant scaler and Umax to the maximum absolute velocity observed across all experiments.
0
0
0
0
0
0
W
22
(13)
0
0
Tighte yi
0
0
0
W₁ = a
0 0
pitt
2
lof
Cutt giurs
(14)
gittt Of
Note that since we are changing the noise model, rather than the Kalman gain directly, the output Kalman gain of the HKF does not perfectly track the velocity, the Kalman gain takes few steps to account for changes in the noise model.
A.4
tcFNN
The tcFNN stands for temporally convolved Feedforward Neural Network, and was introduced by [16] and then further studied by [37]. It consists of an initial time convolutional layer that takes in three time bins (150ms) of neural activity followed by four linear layers using ReLU activation functions, batch normalization, and 50% dropout. Note that tcFNN only predicts velocity outputs,
rather than position and velocity, the other decoders tested in this study do. See 116 and 37 for more details on architecture and implementation.
The tcFNN was trained for 15 epochs using the hyperparameters used in (37. and the training took around I minute per day on an Asus TUF A15 laptop equipped with an NVIDIA 1660 Ti GPU.
When running online, the position of the virtual hand was calculated by integrating the velocity prediction since no position prediction was available. Additionally, note that previous work with tcFNN has shown that implementing a recalibration feedback intention-traintee initial training can substantially improve online performance by assuming that errors during online trials are mostly due to errors in decoder output. In this work, we decided to compare to the base (non-ReFTT) tcFNN, as ReFIT has also been shown to greatly improve performance in the Kalman filter and similar concept could also be implemented for recurrent approaches, such as the LSTM and the KalmanNet network.
(ReFIT) step after the
A.5 LSTM
The long short-term memory (LSTM) network is a type of recurrent neural network that has more control over what to remember and what to forget about long-term relationships between inputs. In this study, we refer to LSTM as the architecture used in [17), consisting of single-layer LSTM with linear layer mapping its hidden output to position and velocity predictions. See [17] for more details on architecture and hyperparameters.
The LSTM used the same hyperparameters as in 17 and was trained with scheduler that adapted the learning rate according to the validation loss, and stopped after the validation loss stopped improving. The training took around 2 minutes per day on an Asus TUF A15 laptop equipped with an NVIDIA 1660 Ti GPU.
When running online, the position of the virtual hand calculated as for KalmanNet and the Kalman filter, integrating the velocity prediction and using 2% of the position prediction for stabilization.
17

---

B Supplemental Figures
KF
HKF
Resistance to in-distribution noise
x
Nose Magnitude (multiples of std. dev.)
Normalized velocity MSE
30
100
tcFNN
Velocity MSE
1.4
0.1
LSTM
10
0.1 Noise Magnitude (multiples sta. dev.
1.0
Noise Duration (50 ms bins)
Figure 9: Resistance to noise injection. (A) Offline velocity MSE for KalmanNet (green) and LSTT (brown) across 1=13 days for both monkeys, with noise values closer to those present in the training data. A noise of zero magnitude is equivalent (B) Full product of normalized velocity MSE across models for all values of noise magnitude and duration. The logarithmic color bar on the right represents the MSE value for each combination of noise magnitude and duration, normalized to each model's 's baseline performance (without noise).
adding noise (i.e., baseline shown in Figure 2
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of our work directly in part of the results, as well as in part of the discussion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We describe the full experimental setup well the architectures of the deep-learning models used.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
20

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Code will be released at publication time.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ praips.cc/publicc/uuieeee CoodeSumeeieeeeee ssi iessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details on training and architecture can be found in the Technical Appendix, well as in the code be released at publication time.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All results are shown with error bars representing the standard error of the mean.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
21

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details on compute resources are shown in the Technical Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: All animal experiments detailed in this paper were approved by the Institutional Animal Care and Use Committee of [Anonymized University].
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss in the introduction how brain-machine interfaces can change people's 's lives, and good BMI decoders can play big role in that.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
22

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The base models used in this study are widely available.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
23

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.


Reviews:
[{"review_id": "ao7M9W7WtX", "reviewer": "Reviewer_586k", "summary": "- This approach studies a few approaches that can be used for neural decoding. The baseline approaches are blackbox DNNs and a vanilla Kalman filter. The proposed approach, the KalmanNet, is a hybrid model, in which a DNN is used to control the gain on a Kalman filter.\n- Approaches such as Kalman filtering have the benefit of being more interpretable, at the potential expense of performance. Despite this they find comparable results with pure deep learning approaches.\n- Other aspects of the trade-off between the \"traditional\" KF approach and the DNN approaches are explored.\n- By examining the KalmanNet's predicted gain, the authors can measure at which time points the KalmanNet relied more on observation and when it relied more on the prior.", "questions": "- Line 93 mentions dynamics model $A$, but this does not appear in equation 1. Is it supposed to?\n- What do online and offline refer to? Simply the presence of ground truth finger measurements?\n- A basic question: what is the difference between modifying the Kalman gain using a network and building a black box network that takes the observations directly? Is it a difference in expressive power? Is it a difference in safety guarantees?\n- A related question: what are the benefits of explainability in this context? What are some things that we get for knowing that we are relying on the observations? \n- line 223: What is $p$? Pearson's $r$ correlation? The significance of the difference between approaches?\n- Did any modifications to KalmanNet need to be made to accomodate this domain?\n- Figure 5: Show the whole product of duration and magnitude?", "limitations": "Limitations are discussed adequately. Negative societal impacts are not discussed, but I think this is less relevant here.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper is well written; it is of general interest to the BMI community to see the results of exploring this tradeoff\n- The benchmarked models are relevant to the ones currently popular in the BMI field\n- There are specific takeaways for future BMI decoding models given in the conclusion\n- Data for evaluation seems to have been newly collected for this study (is this indeed the case?) which represents significant novelty", "weaknesses": "- Overall, I think the technical novelty is a little limited. That is, the application of KalmanNet is a good engineering contribution, but the presented modifications to the KalmanNet seem mainly to adapt an existing model to this particular decoding task.\n- The main contribution of this paper seems to be an empirical comparison of existing methods on a specific neural decoding problem. This might have limited significance to the broader machine learning community.\n- It's mentioned that the KF has a safer operation (line 268). Can this still be said of KalmanNet, since the contribution of KF can be zeroed out at anytime by the network?\n- If I read section 3.3 correctly, it seems that the KalmanNet is not as robust to injected noise as the LSTM, which is unfortunate. I don't think the authors should be penalized for disclosing this result. They should be commended. But it does hurt the significance of the proposed modified KalmanNet approach, which we might have expected to be more robust to noise.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed comments and suggestions, as well as for recognizing the main strengths of the paper. We have addressed the paper weaknesses brought up by the reviewer in the general rebuttal. Here, we will address each of the reviewer’s questions.\n### Equation 1:\nWe thank the reviewer for noticing this oversight. We propose modifying the referred sentence to the following:\n\n>The trainable parameters for the KF correspond to the linear observation model (C), the linear dynamics model, and the noise covariances of the state and the observations.\n### Online and offline experiments:\nWe thank the reviewer for this question, which allows us to expand on one of the paper's key contributions. We refer to offline experiments as those in which we analyzed offline the brain activity and finger kinematics from the monkey doing the task in hand control. For these, we have the ground truth readings of the finger measurement, as the reviewer stated, which means we can determine the error in predictions by each model. Offline analysis of neural data is a domain in which these neural network tools are widely used and have very high performance.\n\nIn online experiments, there is a user in a control loop reacting moment by moment to how well the output movements match their desired movements. This is the domain for which Kalman filters are much more commonly used than neural networks due to their physically stable, easily controllable dynamics. In our application, we have the monkey control animated fingers with his brain signals in real-time using one of the tested decoders. We feed those signals every 50ms to the model we are testing and use that model to predict the desired finger velocity and then we move the virtual hand based on those predictions. The monkey sees the movement of the virtual hand and reacts to try to acquire targets as fast as possible. Offline performance is not necessarily predictive of online performance (Refs #19, 35 from main). The user’s reactions to movement at each timestep can generate brain activity that looks very different from the training data, consequently affecting the predictions. We saw this effect in this paper with the tcFNN model. Thus, our work included the very essential online experiments, beyond just offline modeling, to validate that the tested models would work in a real-life scenario.\n\nTo clarify the difference between these two experiment modalities, and relate them to the descriptions of hand and brain control, we propose adding a sentence at the end of the paragraph starting on line 129:\n\n>Analyses of performance during hand and brain control trials are referred to as offline and online, respectively.\n### Kalman gain with a network vs a black box:\nWe thank the reviewer for this question. The biggest difference between modifying the Kalman gain using a network and building a black box network that directly takes the observations lies in the framework in which KalmanNet operates. In KalmanNet, the network does not predict the velocities directly but rather just determines at every time point how much to trust the linear observation model versus how much to trust the linear dynamical model. This allows KalmanNet to incorporate domain knowledge in the state-space model, a classic advantage of using the Kalman filter in any application. In our case, this domain knowledge is reflected by our dynamics model, which models the physics of finger positions and velocities, and the observation model, which determines the relationship between brain measurements and finger kinematics, both of which have been informed by previous work (Ref #31, 36 in main). Thus, and also for safety, KalmanNet essentially never directly zeroes out the contribution of the Kalman filter, but rather adds additional flexibility to the model to choose whether to trust dynamics or sensor measurements at any time point. \n### Benefits of explainability:\nPlease see the general rebuttal for a thorough response to this question. \n### p, Pearson's r correlation:\nIn line 223, the p represents the p-value of the difference between correlations. The value of 0.64 means we did not find enough evidence to conclude that the two models had different correlations. On the suggestion of another reviewer, we have also added a computation of the Bayes factors, to determine the ratio of likelihoods for the null versus alternative hypothesis in each comparison (please see response to reviewer xWsH for full explanation).\n### Domain modifications to KalmanNet:\nWe thank the reviewer for this question. Our novel modifications to make KalmanNet work in this domain can be separated into two parts: the state-space model and the training. The biggest difference between the original KalmanNet and the one used in this work is that we created a new state-space model (dynamics and observation models) based on the specific domain of application (finger movements and brain sensor measurements). The state-space model used the structure described in section A.1 of the Technical Appendix, but briefly: we used a kinematic model of the physical position dynamics via velocity integration, and learned from the training data the parameters for the velocity dynamics as well as the relationship between observations and kinematics. For model training, we modified the base loss function to account for the different scales for our two predictors, position and velocity, and increased the input sequence length to improve performance and encourage smoothness in the output. To show this, we will include a new figure that shows the change in velocity MSE with different sequence lengths during training to the Technical Appendix, as seen in the attached PDF (Supplemental Figure 2).\n### Figure 5: \nWe thank the reviewer for the suggestion. We propose adding the values from the product of duration and magnitude as a figure to the Technical Appendix (Supplemental Figure 3), shown in the attached PDF.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "roFkc2boF7", "reviewer": "Reviewer_JYj7", "summary": "This paper addresses the trade-off between performance and explainability in brain-machine interface (BMI) decoders. The authors introduce KalmanNet, a novel decoding algorithm that combines the traditional Kalman filter (KF) with deep learning techniques, specifically recurrent neural networks (RNNs).\n\nKey contributions:\n\n1. Development of KalmanNet: A hybrid model that maintains the interpretable structure of the KF while leveraging the flexibility of RNNs to compute the Kalman gain dynamically.\n2. Comprehensive evaluation: The authors conduct both offline and online experiments using neural data from two non-human primates performing a 2-DoF dexterous finger task. They compare KalmanNet against standard KF, tcFNN, and LSTM decoders.\n3. Performance and explainability balance: KalmanNet achieves comparable or better performance than state-of-the-art deep learning models while maintaining a degree of explainability.\n4. Behavioral analysis: The paper provides insights into KalmanNet's decision-making process, showing how it adjusts trust between the dynamical model and neural observations.\n\nThe paper demonstrates a promising direction for developing BMI decoders that balance high performance with interpretability, potentially enabling safer and more effective neural prosthetics. It also provides valuable insights into integrating control theory and deep learning in the context of neural decoding.", "questions": "1. KalmanNet shows higher sensitivity to injected noise compared to LSTM. Could you provide more insight into why this occurs?\n2. Could you provide a more detailed analysis of KalmanNet's sensitivity to hyperparameter choices?", "limitations": "The authors have adequately addressed the limitations in the checklist.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Originality:\n    - The paper presents a novel approach (KalmanNet) that creatively combines traditional control theory (Kalman filter) with modern deep learning techniques (RNNs).\n    - Using RNNs to compute Kalman gain dynamically is innovative and addresses a long-standing challenge in BMI decoder design.\n    - The introduction of the Heteroscedastic Kalman Filter (HKF) as an analytical tool to understand KalmanNet's behavior is an original contribution.\n2. Quality:\n    - The experimental design is relatively comprehensive, including both offline and online evaluations, which is crucial for BMI applications. Also, The comparison with state-of-the-art methods (KF, tcFNN, LSTM) provides a robust benchmark for the proposed method.\n3. Clarity:\n    - The paper is well-structured and follows a logical flow from problem statement to methodology, experiments, and conclusions.\n4. Significance:\n    - The performance of KalmanNet, matching or exceeding deep learning models while maintaining some interpretability, represents a significant advancement in the field.", "weaknesses": "1. Limited generalization: The model's poor performance in new task contexts is a significant weakness. The authors could explore transfer learning techniques to improve the robustness and generalization capabilities of KalmanNet.\n2. Insufficient comparison with state-of-the-art methods: The paper lacks comparison with more recent and advanced approaches, particularly transformer-based models which have shown superior performance in various domains. These models also offer various interpretability techniques that could be relevant to this work.\n3. Lack of ablation studies: The paper does not provide a detailed analysis of the contribution of each proposed module. Comprehensive ablation studies would help understand the specific gains from different components of KalmanNet.\n4. Inadequate analysis of interpretability: While improved interpretability is claimed as a key advantage of KalmanNet, the paper lacks a detailed analysis and concrete examples demonstrating this interpretability in practical scenarios.\n5. Questionable paper structure: The absence of a dedicated related work section hinders reviewers' ability to quickly grasp the context of relevant prior work. Additionally, the discussion section is disproportionately long, and some of this space could be better utilized for more detailed experimental analysis and insights.\n\nThese weaknesses, if addressed, could significantly strengthen the paper and provide a more comprehensive evaluation of the proposed method in the context of current BMI research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their very thorough review of the paper, for recognizing its main strengths and contributions, and for suggesting specific avenues for improving our work. We have addressed the weaknesses and questions raised by the reviewer below.\n### Weaknesses:\nFirst in terms of a comparison with better state of the art models, we acknowledge that transformer based models have surpassed other models, such as the LSTM, in many applications. However, in previous work [1], with a user in the loop, a transformer that works particularly well for this type of data offline did not perform as well as an LSTM online. This may be most likely explained by the transformer overfitting to the offline brain data and having too complex dynamics for the user to control in a closed loop setting. A recent theoretical study noted that transformers can accurately approximate a Kalman Filter offline [2], but in practice it may not properly discover the domain dynamics from training data alone. Another recent study has shown that transformers may be able to replace the GRUs in KalmanNET [3], which is of strong interest to our group for future work. \n\nRegarding the generalization issue raised by the reviewer, note that while KalmanNet has a big percentage increase in error when testing on a different context, the overall velocity error still falls below that of the Kalman filter and is comparable to the other deep-learning models (see Figure 6). None of these models were optimized for generalization and multiple previous studies have found that generalizing to other tasks in the domain of brain-machine interfaces is a challenging problem [4-5]. In our work, we felt it was important to show that KalmanNet carried over the generalization disadvantages common in non-optimized deep learning models but, as the reviewer suggests, techniques such as transfer learning or data augmentation can have a great impact on the models’ generalization ability and we plan to explore them in future work.\n\nWe thank the reviewer for suggesting ablation studies. The original KalmanNet paper (Ref #25 in main) did some ablation work by modifying the architecture and the state-space models, arriving at the architecture we used in our paper. We also did a high-level ablation study of our own by comparing the Kalman filter, KalmanNet, and the heteroscedastic Kalman filter (HKF). The only difference between the KF and KalmanNet is how the Kalman gain is computed, while the only difference between the HKF and the KF is that the noise model changes over time. With our work, we showed that the flexible modulation of the Kalman gain was the key to the better performance in KalmanNet, and we further proved that by emulating that behavior in a linear model. We have also included a new figure (Supplemental figure 2) in the Technical Appendix showing the sensitivity of KalmanNet to the sequence length used during training, also attached to the rebuttal.\n\nWe hope that the comments in the general rebuttal on the benefits of explainability help assuage some of the reviewer’s concerns. Additionally, we want to thank the reviewer for suggesting a section specifically about related work: we will condense the discussion and some of the introduction and introduce more details in relevant prior work. Specifically, some of the references we will include, grouped by topics are: Variations on KFs (e.g., adaptive, extended) for BMI applications [6-9]; deep learning models for BMI [5, 10-11] and Refs #16-19 from the main text; and model-based deep learning [12]. \n### Higher sensitivity to injected noise:\nWe thank the reviewer for their question. We have addressed this point in the general rebuttal, but briefly: we exposed the models to extreme, out-of-distribution, and non-Gaussian noise, which falls into the worst case for the Kalman filter-based models. The LSTM showing high robustness to this extreme noise was interesting and unexpected and we plan to explore it further in future work.\n### Analysis of KalmanNet's sensitivity to hyperparameter choices:\nWe thank the reviewer for the suggestion. We propose including a more detailed sensitivity analysis on some key parameters during KalmanNet training: length of each sequence, learning rate, and training time. We have included a new figure in the supplement (Supplemental Figure 2) showing the variation in offline MSE across days for different sequence lengths during training, and we will include the learning rate and training time analyses upon publication. Note, however, that the main objective in BMI experiments is to perform well online with a user in the loop, which does not necessarily follow from offline results (Refs #19 and 35 from main text). Online experiments are difficult to perform, necessitating some use of offline data in parameter optimization. We found that the chosen hyperparameters striked a good balance between overfitting to the training data and performing well online.\n\n### References:\n1. Costello, J., ... & Chestek, C. (2024). NeurIPS, 36.\n2. Goel, G., & Bartlett, P. (2024). 6th Annual Learning for Dynamics & Control Conference, 1502-1512.\n3. Wang, J., Geng, X., & Xu, J. (2024). arXiv preprint arXiv:2404.03915.\n4. Mender, M. J., ... & Chestek, C. A. (2023). Elife, 12, e82598.\n5. Temmar, H., ... & Chestek, C. A. (2024). bioRxiv, 2024-03.\n6. Li, Z., … & Nicolelis, M. A. (2009). PloS one, 4(7), e6243.\n7. Dangi, S., …, & Carmena, J. M. (2011). 5th International IEEE/EMBS Conference on Neural Engineering, 609-612.\n8. Tsui, C. S. L., Gan, J. Q., & Roberts, S. J. (2009). Medical & biological engineering & computing, 47, 257-265.\n9. Malik, W. Q., …, & Hochberg, L. R. (2010). IEEE TNSRE, 19(1), 25-34.\n10. Pandarinath, C., ... & Sussillo, D. (2018). Nature methods, 15(10), 805-815.\n11. Sussillo, D., …, & Shenoy, K. (2012). Journal of neural engineering, 9(2), 026027.\n12. Shlezinger, N., …, & Dimakis, A. G. (2023). Proceedings of the IEEE, 111(5), 465-499.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hzr63z4R9M", "reviewer": "Reviewer_xWsH", "summary": "The study provides a rigorous comparison of four neural decoders typically used in BCIs: Kalman Filter, KalmanNet, tcFNN, and LSTM in both offline and online conditions on a NHP performing a 2 degree of freedom dexterous finger task. Authors explore the trade-off in decoding capabilities of these decoders with their explainability. Authors find that the KalmanNet which incorporates deep-learning techniques by estimating the Kalman Gain through GRUs can attain comparable performance to fully \"Black-Box\" methods using LSTMs, showing there exists methodology which can still retain the performance of deep-learning techniques without fully sacrificing the explainability of linear models. Furthermore, authors also analyze the behavior of the GRUs of the KalmanNet by analyzing the corresponding Kalman Gain, and show that the behavior of KalmanNet can be replicated by a explainable heteroskedastic Kalman Filter. Authors also discuss some limitations of these deep learning approaches (including KalmanNet) when dealing with out-of-distribution inputs.", "questions": "- Why is the variance of MSE in the position so high in Figure 2?\n- Just out of my curiosity, the KalmanNet seems similar in spirit to Extended Kalman Filter, where the Kalman gain is also updated using a known non-linearity in the dynamics of the system. I would like to hear authors' opinion on how extended Kalman filters where the non-linearity can be introduced in the dynamics compare against the methodologies tested in this work, particularly KalmanNet which seems to do something similar where the Kalman gain is updated non-linearly through a GRU. It might provide another avenue for understanding KalmanNet by analyzing if the non-linear trust introduced by the KalmanNet can be modeled as a non-linearity introduced in the dynamics of the system.", "limitations": "Yes authors adequately discuss the limitations of the work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to follow.\n- The paper tackles an important problem of understanding the trade-off between the performance of \"black-box\" decoders with superior decoding abilities and simpler linear decoders  with more explainability in the context of BCI.\n- The authors provide rigorous comparison between 4 different decoders with varying levels of \"black-box-ness\" starting from the conventional linear Kalman Filter, to KalmanNet (which introduces non-linearity by estimating the Kalman Gain using a GRU architecture), and two conventional deep-learning approaches: LSTM & tcFNN on both online and offline data collected from NHPs.\n- The authors are able to show that it is possible to adapt traditional linear approaches by incorporating deep learning approaches (i.e., KalmanNet) while still retaining some amount of explainability without sacrificing the superior performance.\n- Authors are also able to explain the KalmanNet using the explainable Kalman Filter with heteroscedastic noise models.\n- Authors also demonstrate that deep learning approaches (including KalmanNet) are typically unable to handle out-of-distribution inputs (a known problem in deep learning method).\n- Authors also show a surprising result that LSTM are more robust to input noise compared to other methodologies tested, which is at the very least quite an intriguing result.", "weaknesses": "- Please do not use $p>0.05$ as evidence to conclude that there is no significant difference (lines 221-230). The only scientific conclusion that can be made when $p>0.05$ is that using this test we *cannot conclude* that a significant difference exists not that there is not a significant difference (emphasis on conclude). Please run a proper statistical equivalence test using which it can be concluded that the two quantities being compared are equivalent.  \n- Please specify the statistical tests used whose p-values have been reported.\n- The argument that there is no difference between the MSE in position between KF, KalmanNet, and LSTM based on p-values > 0.05  is misleading. From figure 2,  it is clear that KalmanNet struggles to estimate the position.\n- The error bars for online metrics are missing in Fig 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for highlighting the strengths and main contributions of the paper as well as for suggesting specific avenues for improvement. We have addressed the statistical weakness brought up by the reviewer as well as responding to their questions below.\n### Weaknesses:\nWe thank the reviewer for the various comments and suggestions regarding the statistical analyses of the paper, which we will add in greater detail in a revised main text. The p-values shown in the paper were the result of conducting paired-sample t-tests on the mean squared errors and on the Fisher Z-transform of the correlation values. From the reviewer's suggestion, we also added Bayes paired-sample t-tests for each of the comparisons, to give readers more information about the likelihood of the null hypothesis (models have similar performance) versus the alternative hypothesis (models differ in performance). Specifically, we propose adding the following after the last sentence in line 162, both to specify the statistical test we ran, as well as introduce the new Bayes factor analysis:\n\n>To compare offline performance across models, we used a combination of frequentist and Bayesian statistical methods. First, paired-sample t-tests on the MSEs as well as in the Fisher Z-transformed correlation coefficients [1] determined whether the difference was significant ($\\alpha =0.05$) under the null hypothesis assumption. Second, we computed the Bayes factor ($B_{01}$) to determine the ratios of likelihoods between the null and alternative hypotheses.\n\nAdditionally, we propose changing lines 221 through 230 to the following:\n\n>In terms of correlation with velocity, which controls the visualization when running online, KalmanNet did not significantly differ from the LSTM ($p = 0.64, B_{01}=3.3$) and had significantly higher correlation than the KF ($p < 1E−7, B_{01}=0.048$) and the tcFNN ($p < 0.001, B_{01}=0.062$; Figure 2, B). All approaches had similar correlations with position, other than the tcFNN, which is a velocity-only approach. Regarding velocity MSE, there was also no significant difference between KalmanNet and the LSTM ($p = 0.72, B_{01}=3.4$) or between KalmanNet and the tcFNN ($p = 0.29, B_{01}=2.1$). However, KalmanNet significantly outperformed the KF ($p < 0.01, B_{01}=0.14$). In terms of position, there were no significant differences in correlation (KNet vs. LSTM, $p = 0.32, B_{01}=2.4$; KNet vs KF, $p=0.93, B_{01}=3.5$; LSTM vs. KF, $p=0.08, B_{01}=1.1$) or MSE (KNet vs. LSTM, $p=0.26, B_{01}=2.0$; KNet vs KF, $p=0.77, B_{01}=3.5$; LSTM vs. KF, $p = 0.1, B_{01}=1.1$), although KalmanNet had higher variance in MSE.\n\nFinally, regarding the error bars representing the standard error of the mean of online metrics in Figure 3: they are present, but are very small in part due to the large number of trials we conducted. The trial numbers can be found in line 234, but for easier access to the reader we propose adding them to the end of the caption, as follows:\n\n>Tested on monkey N across T=601 (KF), 576 (tcFNN), 2801 (KNet), 393 (LSTM) trials in a total of five days.\n### High MSE variance in Figure 2:\nWe thank the reviewer for their question. The higher variance stems from KalmanNet having a bias in the position prediction on some days, which increases the MSE without decreasing the correlation. This effect most likely stems from bias shifts in the neural data, a known problem in brain-machine interfaces [2], which can greatly affect the bias in the linear observation model used in KalmanNet. However, when using KalmanNet online, the virtual hand is driven mostly by predicted velocity, with the predicted position serving only as a stabilization parameter (see equation in line 479). \n### Similarity to EKF:\nWe thank the reviewer for this insightful question. The extended Kalman filter, as well as other varieties such as the unscented Kalman filter, indeed are similar to KalmanNet in the sense that they allow for non-linearities to be introduced either in the dynamics or in the relationship between sensor information and the tracked state. In a linear Kalman filter, the Kalman gain is computed by propagating the noise variance matrices through the linear system. In an extended Kalman filter, on the other hand, since the system is not linear anymore, the Kalman gain is computed by linearizing the system at the predicted state at every time point, and then proceeding as in the linear Kalman filter. This linearization then depends on the state estimate, and the filter can diverge if the system is not linearized around the correct point. In contrast, KalmanNet computes the Kalman gain by implicitly computing the noise of each source of information by using the differences between what it observes and what it predicts (see section A.2.1 of the Technical Appendix). Thus, the difference between the extended Kalman filter and KalmanNet lies in how the noise is propagated through the system, which determines how the Kalman gain is computed. This allows KalmanNet for a much faster switching in which information source to trust, which proved beneficial in this application.\n\nThe reviewer also raises an interesting question about whether the non-linear trust introduced by KalmanNet could be modeled as non-linearities introduced in the dynamics. The heteroscedastic Kalman filter (HKF) model presented in the paper presents some evidence towards this: by making the noise on the observations covary with the velocity, the HKF could match the performance with KalmanNet but the HKF’s Kalman gain did not perfectly track the velocity (see Figure 4B). It is not clear whether introducing a non-linearity in the dynamics could make the Kalman gain perfectly track the velocity, but it is definitely an interesting avenue for further understanding KalmanNet and potentially using that to inform model development.\n### References:\n1. Meng, X. L., & Rubin, D. B. (1992). Biometrika.\n2. Degenhart, A. D., …, & Yu, B. M. (2020). Nature BME.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their helpful questions, suggestions, and generally supportive comments. Our work demonstrates the tradeoffs of using KalmanNet, an explainable algorithm that combines deep learning with the Kalman filter (KF), to predict finger movements from brain data. We have addressed the weaknesses and questions raised by the reviewers here and in each review. We believe that with the addition of the proposed modifications this work is greatly improved.
### Novelty and significance:
We appreciate the reviewers recognizing our novel approach and comprehensive evaluation of our decoders in online BMI tasks with newly collected data. We wish to clarify that, yes, all data collected during online tasks are new data specifically for this project and used for comparing the real-time performance of the 4 decoders, 2 of which are new (KalmanNet, HKF). Additionally, 3 of the 13 days used for offline evaluation were new data collected specifically for this project. Upon publication, we will release all data for public usage.

We will revise the explanation of our novel approach in the text to clarify that the primary novelty is in combining a state space model framework and a deep learning architecture for BMI applications. However, we also suggest that these results are significant for any controls application with a user in the loop, interacting with the physical world. KFs are widely used due to safety concerns in these applications, which may limit performance compared to state of the art machine learning techniques. Our investigation of the behavior of KalmanNet during the finger movement task led us to create a new, explainable, and small linear model: a novel and significant advance for models with BMI applications. Thus, we show the benefit of potentially understanding the algorithms “mechanisms”, which is not usually available for black-box models. Additionally, we demonstrated that explainable models do not necessarily perform worse than their pure ‘black-box’ counterparts with novel online experiments.  Finally, we show an intriguing result in which an LSTM substantially outperformed both a normal KF and KalmanNet in the case of extreme noise injection, which was counter to our original intuition.
### Explainability:
Some reviewers raised concerns about the benefits of KalmanNet’s explainability. First, throughout robotics and controls, engineers generally choose “white-box” (explainable) approaches at the cost of performance, due to safety concerns stemming from “black box” approaches. This is important not only for brain machine interfaces which control a prosthesis or muscle stimulation, but for any robotics application interacting with the physical world. Autonomous vehicles make greater use of KFs than deep learning models, despite possible higher performance. Second, explainability is helpful for refining the design of algorithms. After observing KalmanNet’s behavior, we generated a simpler linear decoder (HKF), with only a fraction of parameters, that matched the performance of the deep learning model. Third, this result may even shed light on the mechanisms by which the brain drives motor output. KalmanNet learned to trust the brain for producing high velocities and trust dynamics for stopping. This is consistent with prior work in systems neuroscience recording from the motor cortex [1,2]. 
### Noise injection:
Reviewer 586k pointed out that we should not be penalized for disclosing the fact that KalmanNet has a  weakness to extreme noise. Reviewer xWsH shared our view that this was a very intriguing result. Indeed, we performed this analysis because intuitively, one would expect the KF to do well in the face of noisy inputs, where it can rely on its dynamical model and avoid unsafe movements. The fact that the LSTM outperformed both KalmanNet and a regular KF in this regime is a very interesting result in our opinion.

This occurs because the KF architecture is subject to an underlying assumption of zero mean Gaussian noise and is known to suffer when this assumption is violated [3-4]. With KalmanNet, although we do not explicitly model the noise, the model is subject to the same assumptions. We exposed these models to extreme levels of noise, up to 100x the standard deviation of the neural data. We want to show this data for two reasons: First, neural signals are very small, and 100x errors are possible. Second, we want to highlight the surprising success of the LSTM for this difficult problem. 

We have now included another figure (Supplemental Figure 1) that compares the MSE for velocity predictions when the noise is less extreme (up to 1x standard deviation). There, the difference between models is small, suggesting that when operating within the expected noise magnitudes, KalmanNet can match the LSTM performance. We will rephrase (line 273):

>Given the small magnitude of brain signals, noise artifacts can be much larger than the signal features of interest. We modeled those with extreme additions of noise ~100 times the variance in the training data.

Pragmatically, this low robustness to out-of-distribution noise is also a known weakness of the KF and many techniques have been developed to address it. For example, the outlier insensitive KF (OIKF) [5] explicitly models outliers as random variables with unknown variance. The outlier robust KF (ORKF) [4], modifies the noise model to allow for non-Gaussian and heavy-tailed noise. Since KalmanNet works under the same framework, we could apply these techniques to improve KalmanNet’s robustness to noise.

### References:
1. Saleh, M., …, Hatsopoulos, N. G. (2010). Journal of Neuroscience, 30(50), 17079-17090.
2. Reina, G. A., ..., Schwartz, A. B. (2001). Journal of neurophysiology, 85(6), 2576-2589.
3. Huber, P. J. (1992). Breakthroughs in statistics: Methodology and distribution, 492-518.
4. Agamennoni, ..., Nebot, E. M. (2011). IEEE ICRA, 1551-1558.
5. Truzman, S., ..., Klein, I. (2023). ICASSP, 1-5.

Meta-review: Range of rating was only 1 point difference suggesting very good agreement between reviewers. Rounded average score was 6 (accept)
Something in which all reviewers agree and I do as well is that this paper is very well written. Moreover, all reviewers agree on an accepting score. I could see myself having been a bit more tight here as I believe the paper is exploiting a bit of opportunism, but I do not want this comment of mine to detract from the fact that it is indeed a good paper. There was some criticisms wrt the interpretation of stats, and again I agree with the reviewers on this regard, but as proven by the favourable scores this was not perceived as critical.
Rebuttal was nicely done by the authors but discussion was hopeless. I intervened asking participation from one of the reviewers who seemed disengaged, but even that did not seem to have triggered a more spicy argumentation. Anyhow, I reckon everyone was aware that this paper was a clear acceptance and hence not worth spending more time on defending (or yielding) positions.
No issues with ethics were highlighted.
Decision: Accept (poster)