DOI: 10.5061/dryad.1jwstqk6g
title: Data from: Long-term unsupervised recalibration of cursor BCIs

description: Data accompanying the manuscript "Long-term unsupervised
 recalibration of cursor BCIs", consisting of closed-loop cursor
 control datasets collected with participant T5. The dataset includes
 historical sessions as well as new online tests of recalibration methods
 collected specifically for this manuscript. It also contains results of
 parameter optimizations and cursor control simulations. Personal use data
 collected with participant T11 (Figure 6) is not included, as it may
 contain PHI. The README.md file describes each of the four data formats
 included. This data is meant to be used with the accompanying github
 repository: guyhwilson/nonstationarities: Unsupervised recalibration
 project. Original abstract from the manuscript: Intracortical
 brain-computer interfaces (iBCIs) require frequent recalibration to
 maintain robust performance due to changes in neural activity that
 accumulate over time. Compensating for this nonstationarity would enable
 consistently high performance without the need for supervised
 recalibration periods, where users cannot engage in personal use of their
 device. Here we introduce a hidden Markov model (HMM) to infer what
 targets users are moving toward during iBCI use. We then retrain the
 system using these inferred targets, enabling unsupervised adaptation to
 changing neural activity. Our approach outperforms distribution alignment
 methods in large-scale, closed-loop simulations over two months, and in
 closed-loop with a human iBCI user over one month. Leveraging an offline
 dataset spanning five years of iBCI recordings, we further show how
 recently proposed data distribution-matching approaches to recalibration
 fail over long time scales. Only target-inference methods appear capable
 of enabling long-term unsupervised recalibration, while
 distribution-matching methods appear to accumulate compounding error over
 time. Finally, we show offline that our approach also performs well on
 freeform datasets of a person using a home computer with an iBCI. Our
 results demonstrate how task structure can be used to bootstrap a noisy
 decoder into a highly-performant one, thereby overcoming one of the major
 barriers to clinically translating BCIs.
description: ## Overview This data is meant to be used with the corresponding github
 repository:
 [https://github.com/guyhwilson/nonstationarities](https://github.com/guyhwilson/nonstationarities) "Installation" and "Main figures" sections below detail how to install the repository and which notebooks to reference for each figure. The "Data formatting" section details the four file types used to store the data. ## Installation First install all python packages (tested on Python 3.9): pip install -r requirements.txt If cupy does not install, it can be removed from requirements.txt if you just want to run the main figure notebooks (see below). Then install this repository as a python package: pip install -e . ## Main figures Notebooks that reproduce the main figures are as follows in the nonstationarities repository ([https://github.com/guyhwilson/nonstationarities](https://github.com/guyhwilson/nonstationarities)): Figure 1 - Nonstationarity characterization (notebooks/Figure1_NeuralDriftCharacteristics.ipynb) Figure 2 - Stabilizer & HMM visualization (notebooks/Fig2_ExampleApproaches.ipynb) Figure 3 - Offline methods comparison on session pairs (notebooks/Fig_3_and_Supp_Figs_3_4.ipynb) Figure 4 - Comparison of recalibration strategies in simulation (simulator_notebooks/Fig_4a_4b.ipynb, simulator_notebooks/Fig_4c.ipynb, simulator_notebooks/Fig_4d.ipynb, simulator_notebooks/Fig_4e.ipynb) Figure 5 - Closed-loop performance comparison in T5 (notebooks/Fig_5.ipynb) Figure 6 - Offline evaluation of PRI-T and RTI on freeform, personal use data (notebooks/Fig_6.ipynb; but note that T11 personal use data was not able to be released becuase it may contain PHI, so this cannot be run) Before running these notebooks on your computer, change DATA_DIR and FIG_DIR in utils/preprocessing/preprocess.py to point to the unzipped, downloaded data directory and an empty directory where figures should be saved. Also note that the Figure 3 notebook depends on decoding analysis results created by the Figure 1 notebook, so run that one first. ## Data formatting ### Dataset & directory structure overview There are a few different data sources included here: historical and newer T5 datasets used for offline analysis of neural activity and recalibration methods, T5 datasets collected to compare recalibration methods online over a 1 month timespan, hyperparameter sweep results for various recalibration methods, and BCI simulation results. See the directory structure below for an overview, with descriptions provided in parentheses (). The sections further below detail the format of each type of data. ├── simulator (BCI simulation results) │   ├── HP_sweeps (Hyperparameter sweeps for simulation) │   │   └── regular │   └── performance (Simulator performance numbers) │   ├── efficiency │   ├── instability_analysis │   ├── regular │   └── SNR └── T5 ├── methods (Results of different recalibration techniques applied to T5 data) │   ├── ADAN │   ├── HMM │   ├── HMM-Stabilizer │   └── Stabilizer └── raw_data (participant T5 data) ├── historical (historical datasets used for offline analysis/comparison of recalibration methods) ├── new (newer datasets used for offline analysis/comparison of recalibration methods) └── one_month_recal (datasets where recalibration methods were compared online over 1 month) ### T5 closed-loop cursor control datasets (retrospectively analyzed offline) These are saved as .mat files in the T5/raw_data/historical folder (sessions 2016.09.26 - 2020.02.26) and T5/raw_data/new folder (2021.04.26 - 2021.07.26), and were originally intended for loading using MATLAB. To obtain minimally processed results using scipy, access the data via: dat = scipy.io.loadmat("/path/to/mat_file.mat")["dataset"][0][0] The variable "dat" is now a list with entries: dat[0][0] (list[str]) - integer labels for each block dat[1] (list[float]) - 2D target size at each timestep dat[2] (list[float]) - 2D cursor size at each timestep dat[3] (list[int]) - corresponding block number for each timestep dat[4] (list[float]) - system clock dat[5] (list[float]) - clock time on NSPs dat[6] (list[float]) - 2D cursor position at each timestep dat[7] (list[float]) - 2D target position at each timestep dat[8] (list[int]) - indicator at each timestep for if cursor is on target dat[10] (list[float]) - 2D decoded velocity at each timestep dat[12] (list[int]) - 192D array of binned firing rates at each timestep dat[13] (list[float]) - RMS threshold for spike detection dat[15] (2D float) - trial start and stop indices in MATLAB convention (1-indexing, inclusive end range) dat[17] (list[int]) - active trial times dat[18] (list[str]) - list of block names dat[19] (list[int]) - trial success indicator (1 = success, 0 = failed) To obtain more nicely formatted data in Python, you can use the DataStruct object in the accompanying repository. For example: from utils.preprocessing.preprocess import DataStruct dat = DataStruct("/path/to/mat_file.mat") ### T5 online recalibration experiment datasets These data are stored as individual blocks for each closed-loop experiment in: T5/raw_data/one_month_recal/t5.YYYY.MM.DD/Data/TaskData/FittsTask/YYYY-MM-DD_block_Z.csv The files can be loaded using pandas dataframes: df = pd.read_csv("YYYY-MM-DD_block_Z.csv"). The data are also stored in a .pkl format for use with the accompanying code repository (example notebooks load the .pkl files; for example, see: notebooks/Fig_5.ipynb). Each row in the dataframe corresponds to a timepoint in the recording. Columns in the dataframe are: target_pos (2D float) - position of the target on the screen at each timestep cursor_pos (2D float) - position of the cursor on the screen cursor_vel (2D float) - instantaneous cursor velocity cursor_color (RGB int tuple) - color of the cursor target_color (RGB int tuple) - color of the target state (int) - an integer-valued game state indicator. 0 -- regular game play, 1 -- hovering over target, 2 -- success, 3 -- failure. trial_counter (float) - UNIX timestamp (UTC) of elapsed trial time; unused hold_counter (float) - UNIX timestamp (UTC) of elapsed hold time; unused bias_killer (2D float) - bias correction state alpha (float) - exponential smoothing value beta (float) - gain value max_trial_time_sec (float) - system's max allowed trial time hold_time_sec (float) - hold time needed for dwell-based selection of target target_radius_options (float, list[float]) - target radius size options; chosen randomly on each new trial target_radius (float) - radius of the current trial's target xPC_clock (float) clock times as measured on xPC system neural (192-D float) - binned spike counts at each timestep from channels raw_vel (2D float) - the raw velocity signal as decoded, prior to exponential smoothing and gain adjustment task_clock (byte str) - clock time on system used for task handling in seconds Decoders used during the session can be found in: T5/raw_data/one_month_recal/t5.YYYY.MM.DD/Data/Decoders/Fitts/YYYY-MM-DD_block_Z_decoder_A.mat Decoders can be loaded using scipy.io: dec = scipy.io.loadmat("T5/raw_data/one_month_recal/t5.YYYY.MM.DD/Data/Decoders/Fitts/YYYY-MM-DD_block_Z_decoder_A.mat") The variable "dec" is a dictionary which contains two variables: "coef" (2 x 192 numpy matrix of coefficients) and "bias" (1 x 2 numpy matrix of intercept terms). ### Recalibration technique sweep results Results are stored for each technique in: T5/XXX/test/scores_ID_YY.npy where XXX corresponds to a specific method (e.g. "ADAN"). The .npy files are generated from a parallel compute job and should be combined to return the full sweep results for a given method. Each such file is a list of dictionaries, with individual entries having key-value pairs: file (str) - corresponds to which pair of files was used for measuring recalibration performance (reference and new sessions) R2_score (float) - R2 performance on test split from the new session r2_score (float) - pearson correlation of predictions with intention vector from new session\ days_apart (int) - number of days apart between reference and new session norecal_R2_score (float) - R2 score on new day without recalibration norecal_pearson_r (float) - pearson correlation on new day without recalibration meanrecal_R2_score (float) - R2 score on new day with mean adaptation meanrecal_pearson_r (float) - pearson correlation on new day with mean adaptation suprecal_R2_score (float) - R2 score on new day with supervised retraining suprecal_pearson_r (float) - pearson correlation on new day with supervised retraining Additionally, the hyperparameters used for the recalibration on these specific data are listed in additional entries, "batch_size" (int), or "n_components" (int). In all cases these are singletons of type float, int, or string. ### Simulator sweeps Results for various sweeps are stored as .npy files at: simulator/performance/regular/ (main results from Fig. 4A and B) simulator/performance/efficiency/ (efficiency sweeps from Fig. 4C) simulator/performance/SNR/ (SNR sweeps from Fig. 4D) simulator/performance/instability_analysis/ (Fig. 4E) For the main results, each .npy file is a Time x Methods array of floats containing the average trial time for each recalibration method on each day. There are 61 days and 8 methods tested (ordering: 'Gain', 'Supervised', 'PRI-T', 'PRI-T (static)', 'Stabilizer', 'Stabilizer (static)', 'RTI', 'RTI (static)'). For efficiency and SNR sweeps, each .npy contains a dictionary of the results from a single simulator run, with key-value pairs: method (str) - recalibration method used ttt (float array) - individual trial times in seconds neuralTuning (3D float) - channels x 3 of mean FR and XY encoding weights For the efficiency sweeps, there is also a field "nSimSteps' (int) corresponding to the length of each simulation in steps. Both file groups also contain hyperparameters for each recalibration method used. For the instability analysis in Fig. 4E, the .npy files are dictionaries containing data from simulation runs after a critical error threshold has been passed, and are of the form: ttt (list[float]) - list of trial times on threshold passing day stabilizers (object) - singleton list of Stabilizer objects ss_decoder_dicts (dict) - singleton of corresponding dictionary containing stabilizer config n_days_to_threshold (list[int]) - singleton; number of days until threshold passed corrvals (list[float]) - angular error between linear kinematics decoder and encoding subspace new_cfgs (list[dict]) - singleton containing simulator state on threshold passing day ## Human subjects data Data is anonymized and referenced by a subject code alone (participant T5). Informed consent for data sharing was obtained.