Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication
Authors: Chaofei Fan
Nick Hahn
Foram Kamdar
Donald Avansino
Guy H Wilson
Leigh Hochberg
Krishna V. Shenoy
Jaimie M. Henderson
Francis R Willett
Venue: NeurIPS 2023 spotlight (2023)
Keywords: brain-computer interface
self-training
continual online learning

Abstract: Intracortical brain-computer interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). 
However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. 
This requires iBCI users to stop using the iBCI and engage in supervised data collection, making the iBCI system hard to use. 
In this paper, we propose a method that enables self-recalibration of communication iBCIs without interrupting the user. 
Our method leverages large language models (LMs) to automatically correct errors in iBCI outputs. 
The self-recalibration process uses these corrected outputs ("pseudo-labels") to continually update the iBCI decoder online. 
Over a period of more than one year (403 days), we evaluated our Continual Online Recalibration with Pseudo-labels (CORP) framework with one clinical trial participant. 
CORP achieved  a stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods. 
Notably, this is the longest-running iBCI stability demonstration involving a human participant. 
Our results provide the first  evidence for long-term stabilization of a plug-and-play, high-performance communication iBCI, addressing a major barrier for the clinical translation of iBCIs.

Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Texs
Communication
Chaofei Fan Nick Hahn', Foram Kamdar². Donald Avansino Guy H. Wilson', Leigh
Mochhergl00...1 Krishna V. Shenoyl 455.89 Jaimie M. Henderson 22. and Francis R. Willett
Bio-X Program, Stanford University
Department Neuroscience, Stanford University Department of Neurosurgery, Stanford University
Department Neurobiology, Stanford University
Department of Bioengineering, Stanford University
DDeparmeent Computer Science, Stanford University Wu Tsai Neurosciences Institute. Stanford University
8 Howard Hughes Medical Institute at Stanford University Department of Electrical Engineering, Stanford University
School of Engineering and Carney Institute for Brain Science, Brown University
VA RR&D Center for Neurorestoration and Neurotechnology. VA Providence Healthcare System
12 Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School "oorreppnneenc:: istfan stanford.edu Equal contribution
Abstract
Intracortical computr interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. This requires iBCI users to stop using the iBCI and engage in supervised data collection, making the iBCI system hard to In this paper, propose a method that enables self-recalibration of communication iBCIs without interrupting the user. Our method leverages large language models (LMs) automatically correct errors iBCI outputs. The self-recalibration process uses these corrected outputs ""pseudo-abeele online. Over a period of more than one year (403 days), we evaluated our Continual Online Recalibration with Pseudo-labels (CORP) framework with one clinical trial participant. CORP achieved stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods. Notably. this is the longest-running iBCI stability demonstration involving human participant. Our results provide the first evidence for long-teng term stabilization of a plug-and and--nd--pl -lay, high reghhperformanc communication iBCI, addressing major barrier for the clinical translation of iBCIs.
continually update the iBCI decoder
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Introduction
The ability to communicate with others is ubiquitous, everyday function. Neurological disorders such as amyotrophic lateral sclerosis (ALS) can often lead to loss of speech. Restoring lost com- munication can "improve relationships, increase participation in family and community life, offer a greater sense of independence and help a patient make important medical decisions" [4]. Nonin- vasive assistive technologies such eye-tracking and electroencephalograahy (EEG) have helped reestablish communication channels with patients but suffer from low information transfer rates [29 More recently, progress in intracortical brain-computer interfaces (iBCIs) that record neural activity at single-neuron resolution offers a promising path to restore rapid communication [3 47 For example, [47 demonstrated -eh-pperforance iBCI that can accurately decode attempted handwriting movements into text from neural activity recorded with microelectrode arrays placed in the motor cortex.
Despite these advances, numerous challenges must be addressed before iBCIs can be effectively utilized by those who require them. One key challenge is to maintain accuracy in the face of nonstationary neural recordings. iBCIs often require daily recalibration to maintain accuracy. During the recalibration process, users must cease using the iBCIs for some length of time, diminishing the advantages of high-performance iBCls. Nonstationarities are caused by various factors, including microelectrode array movement 44), device degradation (50), single neuron physiological changes
33 9 3), and variability at the behavioral level 7 46 These changes occur across different
timescales, with single and multi-unit baseline firing rates changing within minutes [22 33 and
response selectivity often varying over days (7). As these changes accumulate, an iBCI decoder fit to specific time period gradually becomes outdated, resulting in need for frequent recalibration. While recent attempts to create inherently robust decoders (20, have demonstrated potential in mitigating this issue, they only postpone the inevitable need for recalibration.
In this work, we present an alternative approach called CORP: Continual Online Recalibration with Pseudo-labels. CORP leverages the structure in language enable self-recalibration of communi- cation iBCls without interrupting the user (i.e., plug-and-play Specifically, CORP uses language models (LMs) to automatically correct communication iBCI text outputs, and continually retrains the decoder using these corrected outputs ( ("pseudo-labels") LMs calculate probabilities for word sequences based on their likelihood of occurrence in the language. placing strong prior on what the decoded text should look like that can help correct errors that would otherwise result in improbable sentences. Using LM thus results in decoder outputs that contain fewer errors than the raw iBCI decoder outputs alone. CORP uses online stochastic gradient descent (SGD) to retrain the iBCI decoder to produce the labuls-labels using the Connectionist Temporal Classification (CTC) [16] loss. We use various techniques such as a replay buffer and data augmentation to overcome the challenge of online SGD with very limited data. Note that CORP agnostic to the communication task and can recalibrate any iBCI decoder that maps input neural signals to text outputs. CORP is particularly beneficial scenarios where users rely on the iBCI system for continuous communication, and supervised data collection is either impractical disruptive.
We built prototype self-recalibrating handwriting iBCI system and assessed its long-term per- formance with one participant in a pilot clinical trial. Over a period of more than one year (403 days). our participant used the iBCI system monthly and wrote on average 58.3 sentences per usage session. With CORP, the average accuracy was stable at 93.84% + 2.28 (mean + standard deviation), sigmificantly outperforming other recalibration methods. To our knowledge, our assessment represents the longest-running iBCI recalibration study to date, lasting 9 months longer than previous work 200] CORP has the potential to remove a significant barrier to plug- aud--and-lay communication iBCIs. This study represents an initial step in applying machine learning (ML) methods that address nonstationary data distributions to neural network based iBCI decoders. We believe that collaboration between the ML and iBCI community can lead to improved methods that specifically target the type of data distribution shifts seen for iBCIs. To that end, we released the data and code here for future iBCI stability research.
httpe://ggthub..c//cffaaa//m
2

---

Handwriting iBCI Decoder
Online Recalibrator
TX Features
Update
RNN
session's data
Previous
paper
pezd
Language Model
Send data
data:
data: thannoee
thersapprr
Combine and augment
The attempts write sentence. Output from the language model plus features are sent to recalibrator,
After finishes, recalibration the background. The user will start with a recallbrated model next
Figure I: Overview of the self-recalie recalibcaigg handwriting iBCI system. On the left is the handwrit- ing iBCI decoder that decodes threshold-crossing (TX) neural features into text real-time using an RNN and a language model (LM).. On the right is the online recalibrator that continually updates the RNN decoder with pseudo-labels from the LM. The recalibrator mixes new data with previously collected data and uses stochastic gradient descent (SGD) to update the RNN.
2 Related work
Many unsupervised recalibration algorithms have been developed for iBCI decoders that attempt to transform neural data from new days to look like neural data collected on earlier days by aligning their distributions [10 11 27 (24). These algorithms rely on the assumption that the neural signals can be mapped to stable low-dimensional subspace which encodes a majority of task-related modulation
10 3 [44 45). Unsupervised recalibration algorithms have achieved some success in cursor iBCIs,
where a user controls a cursor on a computer screen via iBCI, but have not yet been demonstrated for iBCIs that decode more complex behaviors like handwriting or speech. Attempted handwriting or speech may require larger number of neural dimensions decode successfully compared to cursor iBCIs, and hence may be more difficult to align. We found that the Factor Analysis Stabilizer cettribution-alignnen
method, did not appear provide a benefit on our handwriting iBCI task. Further comparisons between CORP and these unsupervised distribution- alignment methods should be done the future. Additionally, is worth noting that CORP has the potential to be integrated with these distribution-alignnee approaches, whereby the neural data is initially de-noised using the harness the strengths of both methods.
[10. a wwweeetyy-prrposeeeee
-alibuttiio-llggmenttt strategies. This combination could potentially
Self-training is technique that involves using a pre-existing classifier to generate predictions ("pseudo-labels") on a large unlabeled dataset, followed by training new model using these pscudo- labels. This approach has been widely employed in various machine learning applications 51 18
Self- self-training as means to recalibrate iBCIs has been investigated for cursor tasks 22 49 26 Our work extends iBCI self-training to handwriting decoding, richer task setting with higher dimensional outputs. We also leveraged LMs to provide robust priors, resulting in stable performance over more extended time period. Self-training in speech recognition also utilizes LMs (33; however, our approach is distinct due to its unique ability to handle continually shifting data distributions, while also operating in the data-limited regime of online iBCI control. L-Mguided recalibration has also been explored for non-invasive BCI (6). However, the inputs and decoding methods of non-invasive BCI differ significantly from those of iBCI.
Continual learning is an approach that enables models to learn from a large number of tasks sequen-
tially, without forgetting knowledge obtained from the preceding tasks [17 31. This is particularly important in the context of our work, where the iBCI decoder must continually adapt to changes in the neural signals over time. To address the issue of catastrophic forgetting in continual learning, variety of techniques have been developed technique (37. In future research, we plan to investigate additional approaches to further enhance the performance of our iBCI recalibration method.
[3 36 37 In this work, we employ the replay buffer
3 Methods
In this section, we give an overview of the handwriting iBCI system and the Continual Online Recalibration with Pscudo-labels (CORP) framework (Figure
3

---

3.1 Handwriting iBCI
Our handwriting iBCI system was developed based on the framework outlined in 47 To train and evaluate the system, the user was instructed to copy sentences displayed on a computer screen by attempting to handwrite them letter by letter. Neural activity was recorded with two 96-channel silicon microelectrode arrays (Blackrock Neurotech). These microelectrodes were placed in the "hand knob" region of the dorsal motor cortex of our clinical trial participant (full details in Section
Binned threshold-crossing counts were used as features for neural decoding, and were computed by counting the number of times the voltage time series on given electrode crossed an amplitude threshold set at -4.5 times the standard deviation of the voltage signal. The iBCI decoder comprises shared Gated Recurrent Unit (GRU) RNN [8] backbone and day-specific affine transform layers for each recording session. The neural features first undergo an affine transformation via the day-specifis layer before being processed by the shared GRU backbone. Given the small amount of collected data, we used 2-layer GRU with 512 hidden units to decode the neural features into sequence of character probabilities, comprising 26 English letters and punctuation marks (periods, commas, apostrophes, question marks and spaces).
3-gram language model (LM), along with beam search algorithm [34,, processed the RNN character probability output and translated it into words in real-time, which were displayed on the screen. The 3-gram LM was trained on the Open WebTex22 [T5] corpus using Kaldi [34]. It uses a 130,000 word vocabulary taken from the CMU Pronouncing Dictionary [. After the user finished writing each sentence, we employed the large language model (LLM) GPT2-XL 35] to rescore the 3-gram LM's n-best outputs, and the top-scored result was displayed on the screen as the final decoded sentence.
The iBCI decoder was trained using a combination of data from the same participant in [47] and newly collected data. The training was done with an NVIDIA A100 GPU and took about 2 hours. We used the Connectionist Temporal Classification (CTC) loss [16] as opposed to the cross-entropy (CE) loss when training the GRU. The advantage of using the CTC loss that it eliminates the need for labeling the exact timing of each character while maintaining performance on par with a CE-trained model.
3.2 Continual online recalibration with pseudo-labels
Formally, the iBCI recalibration problem can be defined as follows. Let f x ++ y be an iBCI decoder that maps neural activity into text outputs E IR xxC where T is the number of time bins and C is the number of features. Suppose we are given pretrained iBCI decoder fe, with model parameters dg- fo, has been trained on a dataset D = (Zi,e) V/., it < k). Ti,t denotes the neural activity for trial on day and yi., denotes the corresponding ground- truth text label. D is collected before day k. On a new day k. we want to decode a new neural signal Zy.A into text. Due to nonstationarity, z does not come from the same distribution as Zi, it k. and therefore decoder fo, 's parameters require updating:
0 = argmin
U..k)
0
L is a distance function that measures the difference between decoded text Új.u and ground-truth text Uj.k- We use Levenshtein distance for L.
There are two major challenges to solve the above problem. The first arises from the fact that we do not know kj as we want the user continuously use the iBCI system without stopping for supervised data collection. The second challenge is that at the beginning of day k, we only have a very small sample of z. (e.g, from few seconds a minute). Fine-tuning neural network models on small datasets can lead to overfitting and catastrophic forgetting [3] 36). In the following sections, we describe how we address these two challenges.
3.2.1 Pseudo-labels
Although we do not know the ground-truth label for Ty,t the decoded output Új.e is mostly accurate. In [7], the author reported 1.5% character error rate when evaluating a pretrained decoder on a new day that is less than a week away. However, when evaluating on a new day that was more than two weeks away, the accuracy dropped significantly. This observation suggests that using

---

LM-corrected outputs as pseudo-labels for continual recalibration could potentially maintain the decoder's 's accuracy over time, as long as recalibration performed frequently enough. Formally, we replace the ground-truth label Uj.A with the pseudo-label Ui.k in Eq.
3.2.2 Continual online recalibration
On each new day k; the recalibration process is initiated immediately after the user completes writing the first sentence. This process runs continuously in the background, updating the decoder after every sentence. Such a continual online recalibration poses an optimization challenge, distinct from traditional deep neural network optimization, which typically assumes access to large, correctly labeled datasets. In our case, we only have a limited amount of data with pseudo-labels, and due to within-day nonstationarity, the distribution of j may be also subject to change. We surveyed the literature on continual learning to identify methods that would enable us to optimize our GRU decoder using small amounts of pseudo-labeled data.
Replay buffer A replay buffer in continual learning is fixed-size buffer, typically denoted by M, which stores data pairs (I Vi.c)) [37]. Its purpose is to balance the learning process by combining new data with previously collected data, mitigating issues such as data sparsity, overfitting, and catastrophic forgetting. We use a replay buffer to combine data collected prior to day k and data from day k. p percent of the replay buffer is reserved for day k's data and the remaining space is allocated for data from previous days. The incorporation of new data alongside existing data addresses the data sparsity issue, preventing the model from overfitting limited new dataset. Furthermore, this approach safeguards the model against forgetting previously learned information.
Data augmentation Data augmentation is a technique commonly employed in machine learning to artificially expand the training dataset by generating new samples through the application of various transformations. This approach enhances the model's ability to generalize by exposing it to a wider range of data variations, leading to improved performance and robustness. To enrich the recalibration data, we also employ data augmentation techniques. Following [47,, we introduce two types of artificial noise to the neural features. Firstly, we add white noise directly the input feature vectors at each time step, providing the model with a more diverse set of inputs. Subsequently, we introduce artificial random offsets to the means of the neural features, to make the decoder model more robust to baseline firing rates drift [22 43 1O). This combination of augmentation techniques serves to enhance the model's 's overall performance and stability.
Stopping criteria Online recalibration requires balancing time, accuracy, and nonstationarity. The recalibration process should be swift, enabling users to access the updated model promptly, while accounting for the time needed for neural network training to improve model accuracy. To strike balance, we need to determine an optimal point for stopping the recalibration process in order to minimize waiting time and maximize accuracy. Note that unlike typical machine learning settings with fixed validation sets, we lack a validation set for two reasons: the first is that we have a very limited amount of data; the second is that due to nonstationarity, performing well on data collected in the past does not guarantee good performance in future.
Our approach involves setting loss threshold during recalibration and selecting an appropriate learning rate. During recalibration, a training loss is computed for every gradient update step taken on the training data. Since lower training loss values correlate with higher accuracy (assuming no substantial overfitting has occurred), we stop the recalibration once the loss falls below a certain threshold. However, loss thresholds that are too low may prolong recalibration and risk overfitting to imperfect pseudo-labels. Thus, we must find an optimal loss threshold considering time constraints and pseudo-label quality. In addition to the loss threshold, the learning rate also affects the recalibra- tion accuracy and time. A trade-off exists between small and large learning rates, with large rates requiring fewer gradient update steps but potentially causing instability, and smaller rates offering stability at the cost of increased time. The optimal learning rates and loss thresholds depend on the problem. In the next section, we conduct an offline sweep to analyze the trade-offs. We find a loss threshold and learning rate combination that effectively balances time, accuracy, and nonstationarity.
4 Results
We assessed the effectiveness of CORP with our clinical trial participant (referred to as 'T5'), who part of the BrainGate2 pilot clinical trial. T5 is right- man who was 69 years old at the time
5

---

No Reca ibbratio
Recalibration with CORP b
No Recalibration
Recalibration with CORP
100
02 00200
2000
100
9,999
02200
Bay
Ery.
D033403
Character Enor Rate (%)
240
222
80
Word Error Rate (%)
80
60
60
20
20
100
200
300
400
500
600
100
200
300
400
500
600
# of Sentences
# of Sentences
Figure 2: An evaluation of CORP demonstrates high-performance over 403 days. The character error rate (CER) and word error rate (WER) of each sentence across 15 evaluation sessions. Day-O was train and evaluate the seed model. Each subsequent day compared the no-recalibration blocks (blue) and the recalibration blocks (orange) where CORP was used. Details about each day can be found in the Supplement. (a) The CER of raw RNN-decoded outputs (pre-LM). (b) The WER of LM-decoded outputs.
of the study. He was diagnosed with C4 AIS-C spinal cord injury eleven years prior to this study. This study was conducted under an FDA investigational device exemption, with institutional review board approval (see details in Supplement..
Over a 403-day time period, T5 used our handwriting iBCI system in real-time each month to copy sentences displayed on the screen (15 evaluation sessions in total). Each evaluation session consisted of side-by-side comparison of online handwriting performance with and without CORP recalibration. Every session was divided into four "blocks", with each block containing 10-20 sentences. T5 instructed read sentences displayed on a computer screen and attempt to write them. After finishing a sentence, T5 would verbally report this, and we would manually stop the decoder and start recalibration. T5's writing speed was 73.2 + 11.0 characters per minute on average. All sentences were randomly sampled from the Brown corpus with no repetition [2].
The first block served as a warm-up for T5 to get reacquainted with the handwriting task after hiatus. fixed set of 10 sentences was used for the warm-up block. The second block employed frozen seed model, trained on combination of data from [7] and data collected prior to this evaluation (21 sessions total). The third and fourth block featured recalibration using the same seed model but unfrozen, which was continually recalibrated using CORP. In a few sessions (details in the Supplement), the third and fourth block were run before the second block to mitigate the order effect.
Plug-and-play long-term stability CORP successfully maintained the seed model's high accuracy over a 403-day evaluation period without requiring any supervised data collection.
Figure 2 compares the character error rate (CER) of the RNN-decoded outputs (without an LM) between the no-recalibration blocks (original seed model) and the recalibration blocks (CORP- recalibrated model) for each evaluation day. On day-0, seed model trained and its CER was 3.22%. similar to what was reported in [47]. Starting on day-28, in the no-recalibration blocks the CER gradually increased and then fluctuated due to nonstationarity in the neural data. In contrast, the CORP recalibration blocks exhibited stable and low average CER of 6.35% + 1.75. During each evaluation day, the recalibration block's CER started higher than the day before but progressively decreased over the course of the day, demonstrating successful continual recalibration as the participant wrote more sentences. some recalibration blocks (e.g. day-61), the CER initially decreased, increased, and then decreased again, revealing that nonstationarity can occur within single day, and CORP can effectively address Notably, there were extended breaks between some sessions (e.g. 105 and 180) and CORP was still able to successfully recalibrate the decoder, showcasing its resilience and adaptability even in the face of long gaps in usage. Remarkably, the recalibration accuracy remains high and stable even after 405 days (6.18% CER). Figure Z shows a similar comparison for the LM-decoded outputs, with performance measured word error rate (WER) after the LM is applied. T5 could clearly perceive the performance difference between the no-recalibration blocks and the recalibration blocks, and expressed a preference for the system recalibrated with CORP.

---

b
35
Word Error Rate (%)
30
Word Error Rate (%)
25
20
4.0
15-
10
2.0
5
10 20
30 40
0
10 15
20
of Recallbration Sentences
of Training Days
Figure 3: Data needs for recalibration and seed model training. Shaded region represents 95% confidence interval taken across 10 random seeds, computed via bootstrap resampling. (a) Word error rates continue to decrease as more sentences are used for recalibration, but only a handful of sentences are needed to regain the majority of the compromised accuracy. (b) Using more days of data to train seed model enhances the recalibration accuracy, but performance does not appear to improve beyond 10 days.
CORP can achieve reasonable recalibration accuracy with minimal data. Figure 3a shows the recalibration WER as a function of the number of sentences used for recalibration (results were obtained from re-analyzing the data offline while using different amounts of training data). After approximately 10 sentences, CORP recovers most of the lost accuracy. Increasing the number of sentences can further improve recalibration accuracy (albeit at a reduced rate).
high-quality seed model is important for maintaining consistent recalibration accuracy. Training seed model on an extensive dataset allows the model to capture a wide range of variations in the underlying patterns, which in turn enhances its generalization capabilities. Figure Biguee demonstrates the benefits of using more data for training. However, it also reveals diminishing return after about 10 days' worth of data. This plateau likely occurs because the model has already learned substantial amount from the available patterns and information within those 10 days, resulting in diminishing improvements as more data added.
Comparison of recalibration methods Table [ compares the performance of CORP to other recalibration methods in online and offline settings. In the online setting, the average WER is measured using the recorded decoder outputs from all the online evaluation sessions. In the offline setting, we used the recorded neural activity from recalibration blocks from day-28 to day-228 to compare different recalibration methods. Offline recalibration was run with 10 random seeds to compute the confidence interval using bootstrapping.
Table 1: Word error rate (WER) comparison of different recalibration methods. The online results were average WER from the no-recalibration and recalibration blocks. The offline results were computed using the recalibration blocks from day-28 to day-228. CORP's offline performance was improved due to hyperparameter search. The 95% confidence interval was computed via bootstrap resampling across 10 random seeds.
Method
Offline WER % (15%
Online WER %
No Recalibration
23. 15 1-11
26.51
FA Stabilizer
23.03 (2.64.23277
CORP
3.68 (3.21,466)
6.16
CORP (Ground Truth)
2.05 (2.00.112)
Offline analysis allowed us to compare CORP to an alternative approach: unsupervised stabilization of the distribution neural data. We compared CORP to Factor Analysis (FA) Stabilizer, an unsupervised recalibration method originally developed for cursor iBCI tasks. The FA Stabilizer requires a reference day to estimate the FA subspace, and aligns all other days' data to that reference day. We used day-O the reference day and aligned all other days' data to day-O. An FA seed
7

---

20.0
Learning Rate
0.001
Character Error Rate (%)
16.0
0.004
0.01
Loss Threshold
12.0
0,4
0.6
8.0
1.0
2.0
4.0
4.0
8.0
16,0
32.0
0.0
10a
10a
Mean Train Steps
Figure 4: Exploring the balance between recalibration accuracy and time requirements. The relationship between recalibration accuracy, measured by character error rate (CER) on RNN-decoded outputs (as opposed to word error rate, which is noisier), and time required for recalibration, measured terms of the number of gradient updating steps. Each color-coded line represents the varying loss thresholds at constant learning rate. The size of the point represents the loss threshold. Shaded region represents 95% confidence interval taken across random seeds, computed via bootstrap resampling.
model was trained on the aligned training data, frozen, and then evaluated on the aligned evaluation data. used 96 latent dimensions for FA seed model training and recalibration as this was the best performing configuration. More details on the hyperparameters for the FA Stabilizer can be found in the Supplement, The results show that the FA Stabilizer was unsuccessful the handwriting recalibration task, which stands in contrast to previous studies where it demonstrated success in cursor iBCI task. This could potentially be because the FA Stabilizer assumes that a low dimensional latent structure is preserved across days and can be realigned using Procrustes alignment. Handwriting, however, has a higher intrinsic dimensionality than iBCI cursor control (see Supplement) and thus may be more challenging realign. Nevertheless, many other promising Iistribution-alignmee methods have been proposed 11 27 24 and future work should assess these methods and further probe their potential mechanisms of failure and improvement. In theory, distribution -obbbttionnlllgmmmnt methods should be able to be combined with CORP for enhanced performance.
Note that the "No Recalibration WER" is different between the online and offline settings because, in the online setting, the no-recalibration blocks used different sentences from the recalibration blocks. When evaluating offline, tested both methods the same set of sentences. Additionally, the offline CORP has lower WER because we further optimized its aeper-parameterss offline. We also compared CORP to continuous recalibration using the actual ground truth-labels, which sets lower bound on the WER. With ground truth labels, we achieved a word error rate of 2%, showing that CORP is close to the lowest achievable error rate (1.6% difference), but with some room for improvement.
Balancing recalibration accuracy and time CORP recalibration introduces minimal latency the end-user's interaction with the system. On average, the recalibration procedure took approximately seconds to complete (compared to an average of 36 seconds to write each sentence). During evaluation, the participant was requested to pause before writing the subsequent sentence until the recalibration was finished. However, real-world application, this recalibration could operate seamlessly in the background, unbeknownst to the user. The short run-time of CORP ensures that the recalibrated model is promptly available for use.
To achieve balance between recalibration accuracy and time, we explored a range of loss thresholds and learning rates. Both loss threshold and learning rate together determine the recalibration time and accuracy, as shown in Figure 4 For any fixed learning rate, initially, as the loss threshold is reduced, the recalibration CER also decreases, but then increases as the model begins overfitting to imperfect labels. Very small learning rates (e.g., learning rate = 0.001) not only take more time to recalibrate but also lead sub-optimal recalibration CER. Conversely, large learning rates might achieve the optimum faster, but cause instability when the loss threshold is small (e.g.. the

---

train steps and CER increase dramatically for learning rate = 0.01 when the loss threshold is less than 1). Overall, CORP can work well with a wide range of learning rates and loss thresholds. This is advantageous for its practical use, as it eliminates the need for extensive hyperparameter tuning for each new user.
Ablation on CORP Table presents the effects of various ablations on the accuracy of CORP's recalibration, highlighting the importance of certain key components.
Table 2: Ablation of key components in CORP, measured by character error rate (CER) on RNN- decoded outputs to make all ablations comparable to the "No Language Model", The 95% confidence interval computed via bootstrap resampling across 10 random seeds.
Ablation
CER % (95% CI)
CORP
4.80 [468, 4.98)
No Data Augmentation
5.87 [5.77,6011]
No Replay Buffer
6.42 16.33, 6,521
No GRU Backbone Training
7.00 [6.82. 7.19)
No Language Model
19.22 (17.29, 22.18)
The Language model (LM) makes the most significant contribution to recalibration accuracy. When the LM is not incorporated, shown in the "No Language Model" ablation, the CER increases significantly when relying solely on RNN-dccoded outputs as pseudo-labels. This underscores the LM's proficiency in correcting decoding errors induced by nonstationarities, and emphasizes its significance in the CORP framework.
The next ablation, "No GRU Backbone Training", freezes the shared GRU backbone of our decoder while only training the day-specific affine input layer. Since CER only increases by 2% the backbone is frozen, this ablation results suggest that nonstationarities largely cause the neural representation to gradually rotate from the original space into a new one, aligning with previous findings [47]. Nonetheless, recalibrating both the backbone GRU and the day-specific layers yields improved recalibration accuracy, indicating that factors beyond mere rotation of the neural representation are in play.
Lastly, the ablations "No Replay Buffer" and "No Data Augmentation", which remove the use of the replay buffer and noise augmentation respectively, both result in CER increase. Despite the increase being modest, these components may still important during the initial stages of recalibration when data availability is limited.
Limitations One limitation of CORP lies in its depen-
dence on the accuracy of the pseudo-labels. In Figure 5 we examine the impact of pscudo-label accuracy on recalibra-
IMN-Dwcoded UN-Oscoded
tion accuracy. To simulate a varying pseudo-label accuracy, we randomly altered a specific percentage of letters in the ground truth sentences (replacing letters, inserting new let- ters, and deleting letters). As the CER of the lseudo-labels increased, the recalibrated RNN's CER also increased, fol- lowing an almost linear relationship. The sublinear relation- ship can likely be attributed to the redundancy of letters in sentences. Interestingly, after auto-correction by an LM,
Character Error Rate (%)
40
20
the recalibration CER increased much more slowly than the pseudo-label CER. This again shows the importance of leveraging an LM for pseudo-label-basee recalibration.
10
30
Pseudo-label Character Rate (%)
Considering these findings, we do not anticipate pseudo- label quality to be major concern in practice. This is because future clinically viable iBCIs are expected to have
Figure 5: Recalibration error rate increases sub-linearly with respect to the pseudo-label error rate.
a high decoding accuracy (which should be achievable with
higher channel count devices, e.g. (28 39 32) Users are
also likely to utilize the iBCI frequently, resulting in small nonstationarities most of the time. Under

---

these conditions, we believe that the pseudo-labels will have high accuracy, allowing CORP to sustain the iBCT's accuracy indefinitely.
5 Discussion
iBCIs offer the potential to restore communication abilities lost due to neurological disease or injury. In this study, we presented CORP, continual online recalibration method to address a nonstationarity of neural signals, which is a key challenge in iBCIs. CORP leverages language models to perform relf-recalibration of any communication iBCI that maps neural activity text. Through an extensive evaluation of handwriting iBCI system, we demonstrated the efficacy of CORP in maintaining high decoding accuracy over an extended period of more than one year (403 days), achieving plug-and-pin stability. This is the longest-running iBCI stability demonstration involving human participants. Our findings have important implications for the clinical translation of iBCIs. By maintaining decoding accuracy in the presence of nonstationarity, CORP can help to ensure reliable and stable iBCI performance, which is crucial for user acceptance and practical utilityy Additionally, the reasonable time and computational resources associated with CORP facilitates the implementation of iBCIs in various clinical and home settings.
However, there are some limitations to the current study. The reliance pseudo-label accuracy in CORP may pose challenges in cases where the iBCI system exhibits substantial nonstationarity or low decoding accuracy. Nonetheless, we expect that future iBCIs will continue to improve in performance as recording devices improve (e.g., [28] 39 (2 mitigating this concern. Furthermore, the present study focused on a handwriting iBCI system with one participant, and the generalizability of our findings to other iBCI tasks and more participants remains to be established.
In future work, several avenues could be explored to further enhance the recalibration process. For instance, combining setrriuttionallignmen approaches [1 [4 27] with CORP might harness the strengths of both methods. Various methods developed for addressing distribution shift in machine
It is also important to examine the performance of CORP in other iBCI applications, such as speech decoding 48). to gain valuable insights into its broader applicability and potential for further advancements the iBCI field.
learning could also investigated [41
2 5 19
25
21
38 42
In conclusion, CORP represents a promising approach to address the challenges of nonstationarity in iBCI systems. By enabling effective online recalibration without the need for additional supervised data collection, CORP has the potential to tackle one of the remaining technical roadblocks for clinically translating gegh-performnce communication iBCIs.
6 Acknowledgements
We thank participant T5 and his caregivers for their generously volunteered time and effort as part of the BrainGate2 pilot clinical trial: B. Davis, K. Tsou and S. Kosasih for administrative support. Support was provided by the Office of Research and Development, Rehabilitation R&D Service, Department of Veterans Affairs (nos. N2864C and A2295R). Wu Tsai Neurosciences Institute, Howard Hughes Medical Institute, Larry and Pamela Garlick, Simons Foundation Collaboration on the Global Brain and NIH-NIDCD UOIDC017844, NIH-NIDCD R01DC014034, NIH-NIBIB 201EB028171.
References
[1] The CMU pronouncing dictionary. 2023-05-16.
cs. cmu. www..eu/u...accccmm wwwpee/iuuincmmeeeeeeeeeeeeeeeee cmmuictt
Accessed:
[2] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 1907.0283, 2019.
[3] Vivek R Athalye, Karunesh Ganguly, Rui M Costa, and Jose M Carmena. Emergence of coordinated neural dynamics underlies neuroprosthetic learning and skillful control. Neuron, 9344)955-970, 2017.
[4] Alisa Brownlee and Merisa Paloveak. The role of augmentative communication devices in the medical management of als. NeuroRchabiliiation 22(6):445 450, 2007.
10

---

[5] Fabio M Carlucci, Antonio D' Innocentee Silvia Bucci, Barbara Capulo. and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2229-2222222222 2238,223,22222 2019.
[6] Xinlin Chen, Leslie Collins, and Boyla O Mainsah. Language model-guided classifier adaptation for brain-computer interfaces for communication. In 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 1642-1647. IEEE, 2022.
[7] Cynthia A Chestek, Aaron P Batista, Gopal Santhanam, M. Yu Byron, Afsheen Afshar, John P Cunningham, Vikash Gilja, Stephen Ryu, Mark M Churchland, and Krishna V Shenoy. Single-neuron stability during repeated reaching in macaque premotor cortex. Journal of Neuroxcience, 27(40)):0742210755) 2007.
[8] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encnder-decoder for statistical machine translation. arXiv preprint arXiv: 1078, 2014.
[9] Mark M Churchland and Krishna V Shenoy. Temporal complexity and hetcrogencity of single-ncuron activity premotor and motor cortex. Journal of neurophysiology, 97(6):4235 4255--575,, 2007.
[10] Alan D Degenhart, William E Bishop. Emily R Oby, Elizabeth Tyler-Kabara, Batista, and Byron M Yu. Stabilization of a brain computer interface via the alignment of low-dimensional spaces of neural activity. Nature biomedical engineering, 4(7):672-685, 2020.
Kabara, Steven Chase, Aaron
[11] Ali Farshchian, Juan A Gallego, Joseph P Cohen, Yoshua Bengio. Lee E Miller, and Sara A Solla. Adversarial domain adaptation for stable brain-machine interfaces. arXiv preprint arXiv: 1810.00055 2018.
[12] W Nelson Francis and Henry Kucera. Brown corpus manual.
[13] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135, 1999.
[14] Juan A Gallego, Matthew G Perich, Lee Miller, and Sara A Solla. Neural manifolds for the control of movement. Newron, 94(5):978 9944994,,,, 2017.
[15] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang. Horace He, Anish Thite, Noa Nabeshima. Shawn Presset, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint
2020.
[16] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 669-376, 2006.
[17] Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 24(12): 102-))))0-814400 1040, 2020.
[18] Junxian He, Jiatao Gu, Jiajun Shen, and Marc' Aurelio Ranzato. Revisiting self-training for neural sequence generation. arXiv preprint arXiv: (r.Xiv::90..1//. 1999913998.. 2019.
[19] Christina Heinze-Deml and Nicolai Meinshausen. Conditional variance penalties and domain shift robustness. arXiv preprint /71vv:/710./14000 2017.
[20] Tommy Hosman, Tsam Kiu Pun, Kapitonava, John D Simeral, and Leigh R Hochberg. Months-long interfaces.
rerfooerformance fixed LSTM decoder for cursor control in human intracortical In 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, 2023.
[21] Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning, pages 2029 2037. PMLR, 2018.
[22] Beata Jarosiewicz, Anish A Sarma. Daniel Bacher. Nicolas Y Masse, John D Simeral, Brittany Sorice. Erin M Oakley, Christine Blabe, Chethan Pandarinath, Vikash Gilja, et al. Virtual typing by people with tetraplegia using self-calibrating intracortical comptce interface. Science translational medicine, 7(113)313ra177 313rll 179, 19,, 2015.
[23] Jacob Kahn, Ann Lee, and Awni Hannun. Self traintrainng for end-to-end speech recognition. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7084-7088. IEEE, 2020.
11

---

[24] Brianna M Karpowicz, Yahia H Ali, Lahiru N Wimalasena, Andrew R Sedler, Mohammad Reza Keshtkaran. Kevin Bodkin, Xuan Ma, Lee E Miller, and Chethan Pandarinath. Stabilizing brain-compuler interfaces through alignment of latent dynamics. bioRxiv. pages 2022-04, 2022.
[25] Da Li. Yongxin Yang. Yi-Zhe Song, and Timothy Hospedales. Learning generalize: Meta- Meta-learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[26] Zheng Li, Joseph E O'Doherty. Mikhail A Lebedev, and Miguel AL Nicolelis. Adaptive decoding for brain-machine interfaces through bayesian parameter updates. Neural computation, 23(12):3162-322 2011.
[27] Xuan Ma, Fabio Rizzoglio, Eric J Perreault, Lee E Miller, and Ann Kennedy. Using adversarial networks to extend brain computer interface decoding accuracy over time, bioRxiv, pages 2022-08, 2022.
[28] Elon Musk al. An integrated brain-machine interface platform with thousands of channels. Journal of medical Internet research. 21(10)ee1114, 2019.
[29] Chethan Pandarinath and Sliman J Bensmaia. The science and engincering behind sensitized brain- controlled bionic hands. Physiological Reviews, 102(2):551 6044, 2022.
[30] Chethan Pandarinath, Paul Nuyujukian, Christine H Blabe, Brittany L Sorice, Jad Saab, Francis R Willett, Leigh Hochberg, Krishna V Shenoy, and Jaimie M Henderson. High performance communication by people with paralysis using an intracortical brain-computer interface. Elife, 6:e18554, 2017.
[31] German Parisi, Ronald Kemker, Jose I. Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural networks, 113:54-71, 2019.
[32] Angelique C Paulk, Yoav Kfir, Arjun R Khanna, Martina L. Mustroph, Eric M Trautmann, Dan J Soper, Sergey D Stavisky, Marleen Welkenhuysen. Barundeb Dutta, Krishna V Shenoy, Large-scale neural recordings with single neuron resolution using Neuropixels probes in human cortex. Nature Neuroscience, 25(2):252-263 2022.
[33] János A Perge, Mark L Homer, Wasim Q Malik, Sydney Cash, Emad Eskandar, Gerhard Frichs, John P Donoghue, and Leigh R Hochberg. Intra-day signal instabilitics affect decoding performance in an intracortical neural interface system. Journal of neural engineering, 10(3):036004, 2013.
[34] Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Modicek, Yanmin Qian, Petr Schwarz, et al. The Kaldi speech recognition toolkit. In IEEE 201/ workshop on automatic speech recognition and understanding, number CONE. IEEE Signal Processing Society, 2011.
[35] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9. 2019.
[36] Anthony Robins. Catastrophic forgetting, rehearsal and pseudorchearsal. Connection Science, 722):123 146, 1995.
[37] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning, Advances in Neural Information Processing Systems, 32, 2019.
[38] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint MeXiv:1911.08731 2019.
[39] Kunal Sahasrabuddhe, Aamir A Khan, Aditya P Singh, Tyler M Stem, Yeena Ng, Aleksandar Tadić, Peter Orel, Chris LaReau, Daniel Pouzzner, Kurtis Nishimura, et al. The Argo: a high channel count recording system for neural recording in vivo. Journal of neural engineering. 18(1)015002, 2021.
[40] Gopal Santhanam, Michael Linderman, Vikash Gilja, Afsheen Afshar, Stephen Ryu, Teresa Meng, and Krishna V Shenoy. HermesB: a continuous neural recording system for freely behaving primates. IEEE Transactions on Biomedical Engineering, 54(1 1:2077222072000 2007.
[41] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCU 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Pari III 14, pages 443-450. Springer, 2016.
[42] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self superviperv for generalization under distribution shifts, In International conference on machine learning, pages 9229-99222299 9248. PMLR, 2020.
12

---

[43] David Sussillo, Paul Nuyujukian. Joline M Fan. Jonathan C Kao, Sergey D Stavisky, Stephen Ryu, and Krishna Shenoy. recurrent neural network for closed-loop intracortical wrainnmachine interface decoders. Journal of neural engineering, 9())026027, 2012.
[44] David Sussillo, Sergey D Stavisky. Jonathan C' Kao, Stephen Ryu, and Krishna V Shenoy. Making brain-machine interfaces robust future neural variability. Nature communications, 7(1):13749, 2016.
[45] Eric M Trautmann, Sergey D Stavisky, Subhaneil Lahiri, Katherine Ames, Matthew Kaufman, Daniel O'Shea, Saurabh Vyas, Xulu Sun, Stephen Ryu, Surya Ganguli, et Accurate estimation of neural population dynamics without spike sorting. Neuron, 103(())292- 200-29222-,,, 2019.
[46] Wilson Truccolo, Gerhard M Friehs. John P Donoghue, Leigh Hochberg. Primary motor cortex tuning to intended movement kinematics in humans with tetraplegia. Journal of Neuroxcience, (865):1163-1173
2008.
[47] Francis R Willett, Donald T Avansino, Leigh R Hochberg, Jaimie M Henderson, and Krishna V Shenoy. High-performanee brain-to-text communication via handwriting. Nature, 593(7858):249- 254, 2021.
[48] Francis R Willett, Erin M Kunz, Chaofei Fan, Donald T Avansino, Guy Wilson, Eun Young Choi, Foram Kamdar, Leigh R Hochberg, Shaul Druckmann, Krishna V Shenoy, et al. A high-performance speech neuroprosthesis. BioRxiv, pages 2023 01, 2023.
[49] Guy И Wilson, Francis R Willett, Elias A Stein, Foram Kamdar, Donald T Avansino, Leigh R Hochberg. Krishna V Shenoy, Shaul Druckmann, and Jaimie M Henderson. Long-term unsupervised recalibration of BCIs. bioRxiv, pages 2023 02, 2023.
Kabara. [50] Kevin Woeppel. Christopher Hughes, Angelica J Herrera, James R Eles, Elizabeth C Tyler-Kabara, Robert Gaunt, Jennifer Collinger. and Xinyan Tracy Cui. Explant analysis of utah electrode arrays implanted in human cortex for brain-computer-nnterfacce Frontiers in Bioengineering and Biotechnology, 9:1137, 2021.
[51] Qizhe Xie, Minh-Thang Thang Luong, Eduard Hovy, and Quoe V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10687-10698, 2020.
13


Reviews:
[{"review_id": "RLbKg3fIPY", "reviewer": "Reviewer_miLw", "summary": "Edit: I have read the rebuttal and am satisfied with the authors' comments. I stand by my score and would like to see this paper accepted.\n\n########################################################################\n\nIn this work, the authors tackle the issue of non-stationarity in decoding technologies specifically for handwriting recognition. They propose several methodological steps to address like the use of a replay buffer, data augmentation and most importantly, an n-gram language model. They use data from a longitudinal study spanning over 200 days where they collect data from a single participant. Their model comprises an affine transform for each day followed by a 2-layer RNN that outputs character probabilities. These probabilities are then fed into a pertained n-gram language model that then generates plausible words (with beam search). The top-n words are then fed into GPT2-XL to rerank and the best word is displayed back to the participant. The output of the LM is also fed into a calibrator that uses this data along with some percentage of past data to recalibrate the RNN.\n\nThe authors show that this model achieves very good character error rate and reasonable word error rate over a model that doesn't recalibrate for stationarity and an alignment method. Through several experiments, they also show the usefulness of different model components, hyperparameter choices and the reliability of the pseudo-labels.", "questions": "How long does the assumption of some stationarity hold? The authors say hat prior work found only a 1.5% error rate but is this something we can evaluate with respect to the drop in calibration error to find the most optimal window?\n\nRe hyperparameter tuning in figure 4: It was not clear what the shaded region represented here- standard error across all decoded characters? Also, how stable is this across different days, ie., does the non-stationarity affect the plots substantially?\n\nThere were limited details provided on the affine transform for each day. How is the affine transform for each day trained and what role does it play in the online decoding system?\n\nClarification: Since the LM outputs word n-grams, I assume that this is segmented into characters to provide the pseudo `y's`?\n\nMinor: What `p`was used in the final model? (esp for Table 4) Similarly, what is the `n` used for the GPT2-XL ranking step?\n\nI would also encourage the authors to allude to the effect of replay buffer size and other modeling parameters on CER in the main text.", "limitations": "Yes, potential limitations were discussed.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Strengths:\n1. Very interesting applications of ML techniques to handwriting decoding.\n2. Well written and easy to follow.\n3. Achieves very good performance and authors clearly show through many experiments how their method alleviates the problem of non-stationarity.\n4. Good analyses on model choice with ablations and many different parameter sets.", "weaknesses": "Weaknesses: No major weaknesses apart from some questions below and a few missing details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Question 1**\n\nThank you for this insightful question regarding the assumption of stationarity and the optimal window for recalibration. The determination of the optimal window is a complex issue, primarily because the nature of nonstationarity in iBCI systems is unknown and can be highly unpredictable. Factors such as variability between individual participants, the recording modality used, and the specific tasks being performed can all influence the degree and pattern of nonstationarity. Given these complexities, a comprehensive analysis of this problem requires collaboration with more iBCI research labs and the collection of additional data. In our future work, we plan to pursue these collaborations to gain a deeper understanding of the underlying factors affecting nonstationarity and to develop more effective methods for managing it.\n\n> **Question 2**\n\nThe shaded region in Figure 4 represents a confidence interval taken across 10 random seeds (we’ll update the figure caption to indicate this), computed via bootstrap resampling. Each data point in the figure is the average character error rate of all the recalibration sentences. This figure is to show that the recalibration process is stable around the operating points.\n\nUnfortunately we didn't compare these hyperparameters across different recalibration sessions to assess the effect of nonstationarity on these hyperparameters. However, given the stable recalibration accuracy in Figure 2 and the tight confidence interval around the operating point in Figure 4, we believe that nonstationarity has little influence on these hyperparameters. Thank you for bringing this to our attention, and we hope this explanation clarifies your query.\n\n> **Question 3**\n\nThe affine transform is defined as:\n\n$y = Ax + b$, $x \\in \\mathbb{R}^{c \\times 1}, A \\in \\mathbb{R}^{c \\times c}, b \\in \\mathbb{R}^{c \\times 1}$\nwhere $x$ is the input, $y$ is the transformed input, and $c$ is the input dimension.\n\nEach session day has its own affine transform layer. The affine transform layers are trained together with the RNN. For a new session, a new affine transform is created and its weights are initialized with the previous session’s. During online decoding, the input neural features are transformed by the affine layer first before being processed by the RNN.\n\nMore details about it can be found in [46]. We’ll include the above in the revised manuscript.\n\n> **Question 4**\n\nYes. LM outputs words, which are then converted into character-level pseudo-labels.\n\n> **Question 5**\n\nDuring online evaluation, we set p = 0.6, n = 100. We’ll add these to Table 4 in the Supplementary.\n\n> **Question 6**\n\nSince we only have a few hundred sentences of data in total, we loaded all the data into the replay buffer. During recalibration, the replay buffer samples BATCH_SIZE * p of sentences from the new data, and BATCH_SIZE * (1 - p) from the past data. We'll make this point clear in the revised manuscript.\n"}, {"review_id": "86bBQmxaZz", "reviewer": "Reviewer_3txU", "summary": "This work focuses on ", "questions": "-\t“The second block employed a frozen seed model, trained on a combination of data from [46] and data collected prior to this evaluation (21 sessions in total).” Are the “data collected prior to this evaluation” from the current participant? Also referred to as “newly collected data” earlier in the paper. Are these 21 sessions prior to day 0?\n\n- “updating the decoder after every sentence.” How is the end of a sentence detected? Automatically?\n\n- Equation 1: \\theta_k, where k denotes a day implies that the recalibration uses all the data from that day. Is Equation 1 supposed to be \\theta_{x, k}, where x refers to sentence?\n\n-\tWhat are the character error rates of the LM-decoded outputs (Figure 2b)? This is to assess whether the use of LM-based correction at word level introduces errors at the character level (vs. Figure 2a with RNN-decoder outputs). (Can be inferred based on simulations in figure 5).\n\n-\tFigure 2: It would be useful to include results from recalibration with the ground truth labels. Why are the amounts of data collection different on day 0 and day 105 different? If understood correctly, day 0 does not have four blocks. This needs to be specified/clarified in the text/caption.\n\n-\tInconsistency:  Over an approximately 8-month period, our participant used the iBCI system monthly and wrote on average 57.7 sentences per usage session.” vs. “X’s writing speed was 69.5 ± 8.6 characters per minute on average.” \n\n-\tWhat is “per-frame labeling”?\n\n-\tx_{i, t}, y_{i, t}: define subscript i. \n\n-\tDefine all acronyms and variables in the captions and provide more context such that the captions standalone to understand the content of the presented information without necessarily referencing the text. Figure and table captions should be more informative to minimise confusion/misinterpreting the CER% or WER% results across figures/tables. For example, the mismatch between the average online WER % with CORP in table 1 vs. figure 2 is explained in the text and not the figure caption. Same with Figure 3. Captions should state if results are from offline vs. online analysis, specific blocks used during recalibration, etc., for clarity.\n\n-\tCheck that the contrast between line styles is preserved when figures are in grayscale. \n", "limitations": "- More discussion is needed on the societal impact of the BCI technology. In particular, how effectively the BCI communicates a user’s intent when there is no alternative. There is the potential concern that the LM may be more dominant than the user’s intent, particularly in cases with low BCI prediction accuracy. \n\n- “we do not anticipate pseudolabel quality to be a major concern in practice. This is because future clinically viable iBCIs are expected to have a high decoding accuracy...  Users are also likely to utilize the iBCI frequently, resulting in small nonstationarities most of the time. ... we believe that the pseudo-labels will have high accuracy, allowing CORP to sustain the iBCIs accuracy indefinitely.” Given the low sample sizes and no current data from “future clinically viable iBCIs”, these claims are questionable. There are issues related to recording quality with long term use of intracortical electrodes. \n", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "Originality: Related work in Chen et al., IEEE SMC 2022 uses a language model for pseudo label corrections during BCI self-recalibration, though the study is focused on simulations with EEG data from a longitudinal study with participants with ALS using the P300 speller. This work is a creative combination of existing ideas to develop a new method to recalibrate BCIs for communication by using language models to improve pseudolabel quality and enhancing continuous learning during recalibration via the use of a replay buffer and data augmentation. \n\nQuality: This paper presents results from a longitudinal online BCI study to demonstrate utility of the proposed approach, which is the gold standard in evaluating BCI algorithms. The inclusion of results from offline analysis also enhances the paper.  \n\nClarity: The paper is very well-written and organised. Areas needing clarity and suggestions to improve readability are noted below.\n\nSignificance: This work is highly relevant to developing automated approaches to periodically recalibrate BCIs for communication for long-term BCI use with minimal user disruptions.  The approach is applicable to general BCIs for communication. Results from a longitudinal online study with a BCI user from a target end user population increase the impact of the paper.\n", "weaknesses": "-\tLow sample size. The paper presents results from one participant with generally high performance level, so difficult to assess the utility across a broad range of user performance levels.  The low sample size is understandable given the challenge with conducting studies in target BCI end-user populations; in particular, this is an iBCI study, in contrast to a non-invasive BCI study. The authors recognise the limitation of the lack of generalisability of results given the low sample size. The authors include results from simulations using data from the current participant to investigate the impact of a broad range of character error rates on the recalibration performance (Figure 5).\n\n-\tPotential order effects due to lack of randomisation of the no recalibration block (block 2) and the recalibration blocks (blocks 3 and 4). If understood correctly, the RNN decoder is updated with the data from the current calibration block and does not rely on data from the seed model block, so the testing order could be randomised daily to mitigate order effects. \n\n-\tThere is the confound of the recalibration blocks (blocks 3 and 4) displaying the LM-decoded outputs (“the top-scored result was displayed on the screen as the final decoded sentence.”) vs. the no recalibration blocks (block 2) displaying the RNN-decoded outputs. This difference in feedback may potentially impact the BCI user experience (mental state, motivation, etc.) and further compound order effects as the user is aware given the fixed block order.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Related work in Chen et al., IEEE SMC 2022 ...**\n\nThank you for pointing out this related work. We’ll add it to the revised manuscript.\n\n> **Weakness 2**\n\nWe appreciate this suggestion. We collected new data recently (two additional days - 278 and 300, shown in the attached pdf). On day 300, we ran recalibration blocks first followed by  the no-recalibration block. The results are still consistent with all other sessions. We’ll keep the session order randomized in all of our upcoming sessions and include this new data in the final paper.\n\n> **Weakness 3**\n\nIt appears there was a lack of clarity in our methods description. Please note that both the recalibration blocks and the no-recalibration blocks display the LM-decoded outputs. We apologize for any confusion this may have caused, and we will make sure to clarify this point in the revised manuscript.\n\n> **Question 1**\n\nYes, all the prior data is from the same participant.\nThe 21st session is day 0. In this session, we collected 60 sentences, trained a decoder with 50 of those sentences combined with the previous 20 sessions’ data, and evaluated the decoder’s performance on the remaining 10 sentences. We’ll make this clear in the revised manuscript.\n\n> **Question 2**\n\nIn this study, the participant would indicate to us when they had finished, and we would manually stop the decoder (since this particular participant can still speak, although he is paralyzed from the neck down). However, we believe the end of the sentence could automatically be detected without much difficulty, and this would be important for a locked-in user (the main target population of such a device)  One potential method is to use the output of the RNN itself. The RNN we used in this study is capable of outputting a special \"blank\" state when the participant is not writing. If the RNN outputs this blank state for more than a certain threshold of time (t seconds), we can infer that the participant has finished writing.\n\n> **Question 3**\n\nYes, thank you for catching this error. It should be \\theta_{j, k}, where j refers to a sentence sample of day k. We will update Eq 1 in the revised manuscript.\n\n> **Question 4**\n\nThe average character error rate on LM-decoded outputs is 1.9% ± 0.6 (5.9% ± 1.4 for RNN-decoded outputs). The fact that LM can significantly reduce word error rate (6.3% ± 2.3 for LM-decoded outputs and 25.1% ± 5.6 for RNN-decoded outputs) means that using LM can reduce both word error rate and character error rate. It’s not shown in Figure 2 due to space limitations.\n\n> **Question 5**\n\nThank you for this suggestion. The data used to plot Figure 2 was collected during online evaluation. Unfortunately, as we didn’t run any ground truth recalibration blocks online, we cannot include this as an additional online baseline. In the future, we will consider including such blocks for comparison as we agree they would be valuable.\n\nDay-0 was intended only to establish a baseline performance. As mentioned above, we collected 60 open-loop sentences in that session, and used the first 50 together with past data to train a RNN decoder and evaluated the decoder on the last 10 sentences. The performance shown in Figure 2 is on those 10 sentences.\n\nOn day-105, we had some technical issues so only one recalibration block was collected.\n\nWe’ll revise the manuscript to make this clear.\n\n> **Question 6**\n\nWe would like to clarify that these  two statements refer to different metrics. The first statement refers to the total number of sentences written per session. The second statement refers to the rate of writing in characters per minute. These two metrics are distinct and provide different insights into the participant's usage and performance with the iBCI system. We hope this clarification is helpful, but please let us know if this does not resolve the issue. \n\n> **Question 7**\n\nPer-frame labeling means that each decoding frame (20ms time windows) needs to be assigned a ground-truth label. However, since our participant is tetraplegic, it’s impossible to get such labels. In [46], the authors used a hidden markov model to force align the neural data with the text to generate per-frame labels.\n\n> **Question 8**\n\ni indexes the trials in day t. We’ll revise the manuscript to include this.\n\n> **Question 9**\n\nThank you for these helpful suggestions. We’ll revise the manuscript to include these clarifications.\n\n> **Question 10**\n\nThanks for this suggestion. We’ll revise the manuscript to make sure that lines are distinguishable when in grayscale\n\n> **Limitation 1**\n\nWe appreciate your feedback on the societal impact. We recognize the importance of accurately communicating a user's intent, and while handwriting iBCIs have shown high accuracy (>95% CER), a more comprehensive metric may be needed. We also acknowledge concerns about the LM dominating user intent, especially with low BCI accuracy, but note that this can be controlled via a weight parameter (Equation 3 in the Supplement). We will incorporate these points into the revised manuscript.\n\n> **Limitation 2**\n\nWe acknowledge that the quality of pseudo-labels is indeed a crucial factor for the success of our proposed method. While we anticipate that future clinically viable iBCIs will have high decoding accuracy, we understand that this is a hypothesis that needs to be tested with more extensive data and over longer periods. Similarly, we recognize that there are potential issues related to the recording quality with long-term use of intracortical electrodes. This is an area that requires further investigation and we are committed to exploring this in our future work.\n\nWe will revise our manuscript to more clearly acknowledge these concerns and the need for further research. We will also temper our claims to more accurately reflect the current state of knowledge and the limitations of our study.\n"}, {"review_id": "onHxoff9WE", "reviewer": "Reviewer_8J1g", "summary": "There is a vastly growing literature reconstructing continuous language from non-invasive brain recordings using popular deep-learning models. These papers typically use invasive recordings from surgically implanted electrodes, while decoders that use non-invasive recordings can only identify stimuli from among a small set of letters, words, or phrases. \n\nThis paper contributes to that literature by introducing a new approach, CORP, a continual online recalibration method for intracranial Brain-Computer Interface (iBCI) that maps neural activity to text. Specifically, the authors use a language model (RNN) to automatically correct errors in iBCI outputs by continually updating the iBCI decoder with an online learning method. The experimental results revealed that the proposed framework achieved a stable decoding accuracy of 93.71% in an online handwriting iBCI task, significantly outperforming other baseline methods.", "questions": "* Did the authors interpret the states of RNN in a continual online learning setup? Is there any trend across days? Which information is overwriting more in the language model?", "limitations": "The authors have discussed several limitations and made future directions for the research community.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper contains the following key contributions:\n\n* The novelty of this work: Different from previous continuous language reconstruction methods from non-invasive brain recordings, the authors build a framework to leverage the structure in language to enable self-recalibration of communication iBCIs without interrupting the user.\n* The proposed framework is agnostic to the communication task and can recalibrate any iBCI decoder that maps input neural signals to text output.\n\nOriginality:\n\n* The prototype of a self-recalibrating handwriting iBCI system and assessed its performance over a period of time is interesting.", "weaknesses": "Weaknesses:\n\n* Although the paper's main idea is quite interesting, however, the experiment was conducted on one participant in a pilot clinical trial. Hence, the proposed method needs to be tested on more subjects.\n\n* Authors have not discussed recent brain decoding works:\n\n    [1] Semantic reconstruction of continuous language from non-invasive brain recordings, Jerry Tang, Amanda LeBel, Shailee Jain, Alexander G. Huth\n\n    [2] Decoding speech from non-invasive brain recordings, Alexandre DÃ©fossez, Charlotte Caucheteux, JÃ©rÃ©my Rapin, Ori Kabeli, and Jean-RÃ©mi King\n\n* In the above works, the authors reconstruct the continuous language from non-invasive brain recordings (fMRI and MEG) with better perplexity. \n\n* Did the authors compare their methods on any existing datasets?\n\n* Since the RNN language model is limited in handling long-term memory information and vanishing gradient problems, did the authors try with recent pretrained Transformer language models? We may expect better accuracy since the Transformer model was pretrained on larger corpora. Moreover, we may expect that a replay buffer is not required in the case of Transformer models.\n\nQuality: The paper supports its claims with few details. Specifically, the methodology and experimental details are limited.\n\nClarity: The paper needs a lot of improvement with clear methodology details and motivation behind using the RNN model. The information provided in the submission needs to be more comprehensive to reproduce the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our manuscript. We would like to clarify that our work differs significantly from the recent non-invasive language decoding works. We'll make this difference clear below, and would like you to consider re-evaluate our work.\n\n> **Weakness 1**\n\nWe acknowledge the limitation of the low sample size and understand the need for more extensive testing across additional users. Conducting studies with intracortical brain-computer interfaces (iBCIs) presents unique challenges, particularly in recruiting participants, compared to non-invasive BCIs. The nature of iBCIs, which involve invasive procedures, inherently limits the number of participants we can involve in our studies. However, we are committed to addressing it in the future work by collaborating with other iBCI research labs and testing our method on more participants. \n\n> **Weakness 2&3**\n\nWe appreciate your suggestion to compare our method with recent brain decoding works. However, it's important to note that the context and objectives of our work differ significantly from those of the studies you mentioned.\n\nOur primary aim is to develop clinically viable BCI devices that can restore communication for individuals with paralysis. Two crucial requirements guide our work: the user must be able to freely express themselves by typing whatever message they desire, and the system must maintain high accuracy. In our referenced work [46] where a handwriting iBCI was first demonstrated, the authors used an RNN to decode brain signals into text with an accuracy exceeding 95% (measured as character error rate). Their method is general enough to express any English sentence. This paper extends [46] to address the issue of non-stationary neural recordings. We used n-gram language models and large language models to automatically correct the RNN decoder’s output, which is then used for online recalibration of the RNN. Our results indicated that with CORP, the RNN decoder’s performance can be maintained around 95% for a long period of time without burdening the user to collect new calibration data. \n\nThe two papers you mentioned aim to solve a different problem. They attempt to decode text from brain signals (recorded using fMRI or MEG) while the participant is listening to a speech. In a clinical setting, it's unclear how this approach could help restore communication, which would require decoding text when a participant is speaking. Moreover, their decoding method can only achieve better than chance accuracy. For instance, in [1] Table 1, the word error rate (WER) is > 90%, meaning that 9 out of 10 words decoded are incorrect. While performance might be better when assessed with semantic metrics, our goal is to enable the user to express the exact text of their intended message.\n\nIn summary, while we appreciate the suggestion, a direct comparison between our work and the two mentioned papers is not entirely appropriate due to the significant differences in context, objectives, and methodologies. We hope this clarifies our position and thank you for your insightful feedback.\n\n> **Weakness 4**\n\nIn the field of intracortical BCIs (iBCIs), the only datasets available are those that study the long-term stability of cursor decoding. Unfortunately, there are  currently no existing datasets specifically for handwriting iBCIs. Data collection for iBCIs is a challenging task, due to the very limited number of clinical trial participants. One of the significant contributions of our work is the 8-month long dataset we collected, which will be published alongside this paper. We believe that this dataset will be a valuable resource for the iBCI community, enabling researchers to explore more methods for addressing the issue of non-stationarity.\n\n> **Weakness 5**\n\nWe appreciate your question regarding the use of Transformer models in place of RNNs. We would like to clarify that in this work no RNN language model was used. We used an RNN only to decode brain signals into character probabilities. A language model (3-gram LM + a transformer-based LLM) was then used to transform these probabilities into a string of words. The replay buffer was used to store recent data for the purpose of online recalibration of the RNN decoder. This is necessary regardless of the type of model used.\n\nWe hope this clarifies the methodology of our work.\n\n> **Question 1**\n\nWe appreciate your suggestion, but our RNN is not a language model. Please refer to the answer above. If you have more questions regarding the RNN decoder, we’d be happy to answer.\n"}, {"review_id": "7TkvPX97hL", "reviewer": "Reviewer_hfNe", "summary": "The paper proposes a self-recalibrating brain-computer interface system where inputs come from implanted electrodes in patient's motor cortex and the outputs characters that are patient images to hand-write. The output is then corrected by a language model to match to the most probable desired sentence.\n\nThe recalibration in needed because in the real biological system there are many factors that change over time and the input data distribution drifts because of that. While it is possible to recalibrate the system by running a dedicated session, this is not optimal is it requires downtime, participation of a technician and ~daily mental effort from the patient.\n\nThis work proposes a smart trick to used LLM-corrected word predictions as new labels for continuous retraining: the input signal distribution drift is gradual on the next day, while the drift has already occurred, the language model can still correctly recover what the intended word were. And these corrected words can be used as new labels to retrain the system a bit and this cancel the drift. If performed often the system will never drift too far and will always be able to self-correct.\n\nThe experimental results from one test subject with implanted microelectrode array confirm that the error rate of the system stays stable thanks to continuous recalibration and it is shows that it would degrade without the proposed mechanism.", "questions": "Fig 1: What does the arrow from \"Language model\" to \"Current session's data\" mean? Is it somehow part of LM's training to predict session data? Or does it provide labels and the data in \"Tx Fetures\" is the same as in \"Current session's data\"?\n\n126: How does the system know that the subject has finished thinking a sentence? Is there a special \"stop\"-though and if so - how reliably it can be detected?\n\nQ1: What was X's subjective feedback on the performance of the system after a few months? While the numbers clearly show that the system was more accurate, what it also noticeable from the user's perspective?\n\nQ2: How are spaces handles? I would image there is no clear motor imagery to think about to imagine making a space, how does the patient do that?\n\n361: Perhaps this should go to Limitations?", "limitations": "I would be curious to know about the natural limits of speed of writing with this approach. After all having a direct brain connection would seem like an opportunity to forgo \"clunky\" written language and make a more direct transmission possible. Just curious what are the authors' thought on feasibility of this and what could be the way to do it.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is very clearly written and well-structured, to the extent that I find myself in trouble performing my role as a reviewer's and ask questions, because I had to delete most most of them as I progressed through the pages :)\n\nThe application is, of course, amazing and the fact that this work was tested on a human subject with implanted electrodes makes it a unique contribution to the field of ML applications to BCIs.", "weaknesses": "I could not come up with any. I think paper is a very clear account of what was done and presents sufficient support for the claims.\n\nPerhaps more test subjects would be beneficial, but even with one subject the results stand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Question 1**\n\nThe arrow from \"Language model\" to \"Current session's data\" represents the generation of pseudo-labels by the language model. We will clarify this in the revised manuscript.\n\n\n> **Question 2**\n\nIn this study, the participant would indicate to us when they had finished, and we would manually stop the decoder (since this particular participant can still speak, although he is paralyzed from the neck down). However, we believe the end of the sentence could automatically be detected without much difficulty, and this would be important for a locked-in user (the main target population of such a device)  One potential method is to use the output of the RNN itself. The RNN we used in this study is capable of outputting a special \"blank\" state when the participant is not writing. If the RNN outputs this blank state for more than a certain threshold of time (t seconds), we can infer that the participant has finished writing.\n\n\n> **Question 3**\n\nYes, our participant X provided positive feedback about the self-recalibration system. He reported that he could clearly perceive the performance difference between the no-recalibration blocks and the recalibration blocks, and he expressed a preference for the recalibration system. We will include X's subjective feedback in the revised manuscript.\n\n> **Question 4**\n\nA user would write‘>’ to indicate space, similar to [46].\n\n> **Question 5**\n\nWe will mention the one subject limitation in the Limitations in the revised manuscript.\n\n> **Limitations**\n\nIn our study, participant X achieved a writing speed of 69.5 ± 8.6 characters per minute. In a previous study [46], the authors reported that their participant was able to write as fast as 90 characters per minute. For a comprehensive review of different BCIs' communication speeds, refer to this blog post (https://www.paradromics.com/blog-post/enabling-connection-ii-bci-for-assistive-communication). As for the possibility of a more direct transmission between the brain and a computer, bypassing the need for written language, this is indeed a fascinating topic. However, achieving this goal would require significant advancements in our understanding of the brain and how it encodes information. Right now, intracortical BCIs focus on motor areas of the brain that represent the intention to move. Therefore, speeds are tied to motor production, for example how long it takes to write a letter or speak a sound. In the future, it may be possible to record from different areas of the brain that may bypass the need to formulate a motor intent, if the neural code can be understood. \n"}, {"review_id": "ebOWmYzFTs", "reviewer": "Reviewer_QkLb", "summary": "This paper introduces a method called Continual Online Recalibration with Pseudo-labels (CORP) that allows for self-recalibration of intracortical brain-computer interfaces (iBCIs) without the need for user interruption. iBCIs are used to restore communication abilities in individuals with neurological disorders like ALS but require frequent recalibration due to changes in neural recordings over time. The proposed method utilizes large language models (LMs) to automatically correct errors in iBCI outputs and uses these corrected outputs as \"pseudo-labels\" to update the iBCI decoder in real time. The CORP framework was evaluated over an 8-month period with a single participant in a clinical trial. The results showed a stable decoding accuracy of 93.71% in an online handwriting task, outperforming other baseline methods. This study demonstrates the longest-running iBCI stability with a human participant and presents a plug-and-play, high-performance communication iBCI that addresses a significant challenge in the clinical application of iBCIs.", "questions": "If a participant does not use the iBCI for an extended period without recalibrating the RNN model, the initial Word Error Rate (WER) and Character Error Rate (CER) can be very high. In such cases, using pseudo-labels generated with the help of LLMs may potentially mislead the training of the RNN model. Has this scenario been tested or are there any alternative approaches?", "limitations": "As mentioned by the authors in the paper, the main limitation of CORP lies in the pseudo-labels generated by the language model. If the error rate of the pseudo-labels is high, it directly affects the Character Error Rate (CER) of the recalibrated RNN model. Therefore, it is crucial to select more powerful language models to minimize errors in the pseudo-labels.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. originality: This work presents an online recalibration method for iBCI which uses LLM to perform re-calibration for communication iBCI.\n2. quality and clarity: This paper provides detailed experimental validation of the effectiveness of the proposed method. The figures and text are clear, easy to understand, and flow smoothly, ensuring a clear expression of ideas.\n3. significance: The CORP method proposed in this paper achieves higher accuracy and stability in brain-to-text communication through intracortical brain-computer interfaces (iBCIs) than other recalibration methods. It has been validated on the longest-running iBCI involving human participants, proving its effectiveness and potential.", "weaknesses": "1. The method presented in this paper has only been validated on a single subject, which may raise concerns about the limited sample size and lack of generalizability.\n2. The stopping criteria mentioned in the article are based on whether the loss falls below a certain threshold to determine the termination of training. This criterion may not be sufficiently objective. It could be considered to explore alternative quantitative metrics for evaluating the training performance of the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Weakness 1**\n\nWe acknowledge the limitation of low sample size. In the future, we plan to address this limitation by collaborating with other BCI labs to test our method on a broader range of subjects. We believe that such collaborations will provide valuable insights and help us refine our approach. Additionally, we plan to publish the code associated with our method. This will allow other researchers to test our method on their data, further contributing to the validation and generalizability of our findings.\n\n> **Weakness 2**\n\nWe agree that the stopping criteria based on whether the loss falls below a certain threshold may not be sufficiently objective. We will consider alternative quantitative metrics for evaluating the training performance of the model in future work.\n\n> **Question 1**\n\nIn our paper, we have attempted to address this issue in the limitation section by simulating varying pseudo-label accuracy. We acknowledge that our method could fail if the initial error rate is too high, typically indicating that the neural data has changed significantly. In such cases, supervised recalibration could be used to rescue the system. However, this is an area where more research is needed. Understanding the nature of the nonstationarity in neural data is a complex challenge, and further investigation is required to develop methods that can automatically address this problem more effectively.\n\n> **Limitations**\n\nThank you for the suggestion of using more powerful language models. Recent advancements in large language models (LLMs) offer promising avenues for improving the accuracy of pseudo-labels which we plan to explore in future work. "}]

Author rebuttal:
We would like to express our sincere gratitude for your time and effort in reviewing our manuscript. Your constructive feedback and insightful questions have been invaluable in helping us improve the quality of our work.

We acknowledge the limitation of our study regarding the low sample size. Conducting studies with intracortical brain-computer interfaces (iBCIs) presents unique challenges, particularly in recruiting participants, compared to non-invasive BCIs. The nature of iBCIs, which involve invasive procedures, inherently limits the number of participants we can involve in our studies. We understand the implications of this limitation on the generalizability of our results and appreciate your understanding in this regard. In the future, we plan to address this limitation by collaborating with other iBCI labs to test our method on a broader range of subjects. Additionally, we plan to publish the code associated with our method. This will allow other researchers to test our method on their data, further contributing to the validation and generalizability of our findings.

The field is at an early stage in its  efforts to address nonstationarity in iBCIs. Our work represents an initial step towards developing a solution that can maintain the stability of iBCIs over extended periods. We hope that the encouraging results we have obtained so far and the publication of our dataset will stimulate interest in this problem among the broader machine learning community. We believe that the involvement of more researchers in this area will accelerate progress towards a robust solution to the nonstationarity problem in iBCIs.

Once again, we thank you for your thoughtful reviews and look forward to your continued feedback as we strive to improve our work.

An updated Figure 2 is attached to show the performance of CORP over a 300-day period, extending the original study from 228 days.


Meta-review: The idea of using language-model input to help recalibrate brain-computer interfaces (BCIs) is interesting and well-motivated, and the methods in this submission are well-executed and promising.  While concerns were raised about the limitation to a single subject for the multi-month experimental evaluation, given the invasive nature of intracortical BCIs, most reviewers find this reasonable and still worthy of publication.  Please do carefully revise to clarify the various points discussed in the reviews and rebuttals.
Decision: Accept (spotlight)