Tracking Functional Changes in Nonstationary Signals with Evolutionary Ensemble Bayesian Model for Robust Neural Decoding
Authors: Xinyun Zhu
Yu Qi
Gang Pan
Yueming Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Brain-machine-interface
Bayesian filter
state-space model
evolutionary computation

Abstract: Neural signals are typical nonstationary data where the functional mapping between neural activities and the intentions (such as the velocity of movements) can occasionally change. Existing studies mostly use a fixed neural decoder, thus suffering from an unstable performance given neural functional changes. We propose a novel evolutionary ensemble framework (EvoEnsemble) to dynamically cope with changes in neural signals by evolving the decoder model accordingly. EvoEnsemble integrates evolutionary computation algorithms in a Bayesian framework where the fitness of models can be sequentially computed with their likelihoods according to the incoming data at each time slot, which enables online tracking of time-varying functions. Two strategies of evolve-at-changes and history-model-archive are designed to further improve efficiency and stability. Experiments with simulations and neural signals demonstrate that EvoEnsemble can track the changes in functions effectively thus improving the accuracy and robustness of neural decoding. The improvement is most significant in neural signals with functional changes.

Tracking Functional Changes in Nonstationary Signals with Evolutionary Ensemble Bayesian Model for Robust Neural Decoding
Xinyun Zhu 1.2.4, Yu Qi Gang Pan Yueming Wang
{ (zhuxinyun, qiyu, gpan, ymingwang @zju. edu cn Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou, China
2 College of Computer Science and Technology, Zhejiang University, Hangzhou, China 3 Affiliated Mental Health Center & Hangzhou Seventh People's 's Hospital and the
MOE Frontier Science Center for Brain Science and Brain-machine Integration, Zhejiang University School of Medicine, Hangzhou, China.
'The State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China
Abstract
Neural signals are typical nonstationary data where the functional mapping between neural activities and the intentions (such as the velocity of movements) can occa- sionally change. Existing studies mostly from an unstable performance given neural functional changes. We propose a novel evolutionary ensemble framework (EvoEnnemblee) to dynamically cope with changes in neural signals by evolving the decoder model accordingly. EvoEnsemble integrates evolutionary computation algorithms Bayesian framework where the fitness of models can be sequentially computed with their likelihoods according to the incoming data at each time slot, which enables online tracking of varying functions. Two strategies of vvolve-at-change and castory-model-archee signed further improve efficiency and stability. Experiments with simulations and neural signals demonstrate that EyoEnsemble can track the changes in func- tions effectively thus improving the accuracy and robustness of neural decoding. The improvement is most significant in neural signals with functional changes.
fixed neural decoder, thus suffering
are de-
Introduction
Nonstationary signals widely exist in the real world, where the properties and functions can change continuously over time. Neural signals are typical nonstationary signals, where the inherent dynamics in the neural systems, the plasticity of synapses driven by learning and adaptation contribute to the variability of neural encoding of the brain [1]]. The nonstationary property of neural signals forms a challenging problem in the brain-computer interfaces (BCIs) field 3 D. because the decoding accuracy will degrade over time given changing neural functions and fixed neural decoder.
Considering the nonstationary neural signals, the neural encoding model, can be presented by:
V.- hy (xx)) q.-
(1)
where y, denotes the neural signal we observed time t, the function hx(-) is the neural encoding model that changes over time, and ø, denotes the state to be encoded such as the velocity of computer cursor, and q, biological external noises. Typical neural decoders such OLE (TO) and Kalman filters (KF) III mostly assume a stationary hy (-), namely he(-) = ho(-), (t = 1, 1,2,3....
"Corresponiing authors: Yu Qi and Yueming Wang
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

which can be oversimplified especially for online BCI systems. Recent studies showed that, in online BCI control, neural encoding functions change significantly with the influence of real-time feedback
such as the speed of the cursor and errors in control [22 13 The functional changes in neural encoding frequently occur over time, even in a single target reaching trial [4
How to cope with the functional changes in the neural system? Idcally, a neural decoder should be capable of adjusting itself along with changes in neural signals. To this end, there are two main types of solutions. The first one is re-calibration of neural decoders periodically or manually when the performance degrades [5] 16 to maintain the control accuracy. Brandman et al. proposed a framework for rapid calibration, which removed the traditional open-loop calibration phase and reduced the original 2-5 min calibration interval to 2-5 with Bayesian parameter updating. However, such a training process can still disturb the BCI users and usually can not cope with changes in the short-term such as single trials. Another way is to design an adaptive or dynamic model Wang et al. proposed a framework that used dual state-space models to estimate both kinematics and neural tuning functions. However, this method does not directly model the neural variability. Qi et al. proposed novel dynamic ensemble Bayesian filter (Dy (yEnsemble) which obtained the state-of-the-art performance with nonstationary neural signals (20]. DyEnsemble constructs a pool of encoding models, and adaptively assembles decoder from these models online according to neural signals with a recursive Bayesian filter, which allows the neural decoder adjust its functions to cope with the changes in neural signals. However, DyEnsemble employs static model pool, only addressing neural changes in a certain range. The ability can be limited given continuous functional changes in long-term neural recordings.
Regarding the encoding changes in neural signals as a functional tracking problem, we propose an evolutionary ensemble Bayesian filter (EvoEnsemble) to track the neural encoding functions over time by incorporating evolutionary computation in a recursive Bayesian framework. Specifically, EvoEnsemble maintains population (i.e., a set of candidate functions) to estimate the neural functions, where the changing fitness of the population is dynamically computed by the likelihood given neural signals (observation) over time. In this way, the population evolution can be driven by the changes in neural signals. Meanwhile, the vime-varying functions can be tracked by the posterior distribution of the candidate functions with Bayesian model averaging rules.
The main contributions of this study can be summarized as follows: (1) We propose a new framework that extends evolutionary computation in Bayesian process to achieve robust state estimation with neural functional changes; (2) We propose a particle-based solution to simultaneously track the functional changes and estimate the state (e.g., the velocity of a cursor) sequentially online; (3) Two strategies of evolve-at-changes and mistory-model-acciee are proposed to improve the computational efficiency and estimation stability.
Experiments with simulations and neural signals demonstrate the superiority of EvoEnsemble against traditional decoders such as Kalman filters, and the improvement is most significant with functional changes in neural signals.
2 State-space formulation with time-varying neural functions
Considering the temporal changes in neural functions, the state-space model can be defined by
xt = g(zx=-) 14-1: 3/- = h4 (x,) + q:
(3)
where denotes the time slot. D, E RRds is the state we want to estimate; Yr E RRd is the observation; g(-) is the state transition function, with Gaussian transition noise nu; and h-(-) the time-varying observation function which is dependent on f, with Gaussian observation noise qr-
In typical motor decoding problem, given sequence of observed neural signals, a neural decoder aims to estimate the corresponding kinematic states in a sequential manner. Here we define the state D; with d, 2, which contains the velocities of movements in x-axis and y-axis, and y, is the firing rates of d, neurons.
The EvoEnsemble approach consists of two phases, one is the calibration phase and the other is the test phase. During the calibration phase, training sample pairs are collected and used to learn the transition function, well as set of initial encoding models to approximate the timevnryying

---

Prior Kinematics
Observed Neural Signals
Model Pool
Model Likethood
Differential Evolution
Model Pool
Encoding
Meeel
Companing
Beslve
Medd
Mecal
Model Evolution Process
Encoded Neural Signals
beke towads Indildual
t+1
Dynamic Model Assembling Process
Beyesian Model
Model Lkkllhood
Averaging
Model Uikllhood
Assemble
Model Lielihoodd
Assarbble cawdd
C-1
I+1
Figure 1: Illustration of the model evolution process (above) and the dynamic model assembling process (below). At cach time slot, EvoEnsemble encodes the prior kinematics with candidate models and then calculates the model likelihoods through comparing the encoded neural signals with the observed neural signals. Afterwards, the model likelihoods are used to evolve the model pool and assemble the models simultancously.
observation function. Once the calibration phase is over, the transition function is fixed. Mcanwhile, set of initial particles are randomly drown from a Gaussian distribution, to approximate the posterior of the kinematic states. During the test phase, the particle likelihoods and model likelihoods calculated through comparing the neural signals encoded by the models with the observed ones. The observation function changes along with the incoming neural signals through adaptively assembling the encoding models in the model pool, where the assembling weights (i.e., the posterior probabilities of the models) updated from the model likelihood. When all models' likelihoods in the model pool are small, the model pool will be updated by EvoEnsemble through a differential evolution algorithm, where model fitness is equal to the model likelihood.
Tracking functional changes with EvoEnsemble
EvoEnsemble tracks the changes of function (-(-) in Eq. with a population of candidate functions (model pool) which evolves with changes in data over time. Then it estimates fe(-) at each time slot t by assembling the candidate functions in the model pool with a Bayesian model averaging algorithm. To this end, there several key problems: 1) how to evolve the model pool over time; 2) how to dynamically assemble the model pool to an optimized estimation of h2(:); and 3) how to sequentially estimate the state 2 given h,(-).
Focusing on the problems above, the EvoEnsemble can be divided into three main components: 1) population evolution algorithm to evolve the model pool in time (See Fig. above); 2) dynamic Bayesian model ensemble approach estimate h-) from the model pool (See Fig. below); and 3) particle filter to estimate given ha(-). Further, two strategies of evolve-at-changes and
history-model-areheee -rrchvvveeeeevveeeeeeeeeeee are proposed to improve the efficiency and stability of the sequential process.
3.1 Population evolution in the Bayesian framework
In this section, we first introduce how to initialize the population, and then show how to dynamically compute the fitness of the population with the model likelihoods in the Bayesian framework. After that, we present the evolutionary process with the differential evolution (DE) algorithm.
3.1.1 Population initialization
The candidate model pool, M (m (-){k = 1,2,
first initialized by set of encoding models that transform kinematics into neural signals. To enrich the diversity of the model pool, each model is trained with a different subset of the training data, which contains randomly divided segment. Here use linear encoding models and the models are fitted with the least square algorithm.
..., N.... N),

---

We denote the linear mapping matrix as M and its parameters as vector p Thus, for linear case, the candidate model pool can also be denoted as M {p* xp 1,2, .... N). where p* is the ktt individual in the evolution population. The specific operation of getting subset indices is given in Appendix
3.1.2 Fitness definition by the model likelihood
The purpose of evolution is to search for the best individual in the population and evolve other individuals towards it. Therefore, a fitness function that evaluates the quality of the model plays core role in evolutionary computation.
In the Bayesian framework, the fitness of model can be defined by its likelihood given observation data. A model with a high likelihood demonstrates good fitness. Since the candidate models in the model pool are neural encoding functions, which predict neural signals given kinematic parameters, we firstly infer the prior distribution of kinematics, namely æt, by Eq. (2) then for each candidate
function m* (.) we can infer the neural signal o by Eq. 3 and compute the likelihood of the observed neural signal yt given Dř The likelihood 1 indicates the fitness of m² m²)).. The fitness computation process is specified as follows.
Step1: Prior kinematics prediction. First, given the kinematics 2;-1 at time f - 1, we can get the prior kinematics p( (z)--) at time through the state transition function g(-) in Eq. (
Step2: Neural encoding with different models. Second, we can generate different neural signals
from the prior kinematics with different encoding models in the model pool. The neural signals encoded by the tt model at time / is " (xx) M
Step3: Likelihood computing for each model. Third, we compute each model's likelihood by comparing its encoded neural signals of with the incoming neural signals yr- One model's likelihood indicates how likely the observed neural signals are encoded by this model. The more likely it is, the greater the value is. The marginal likelihood of the 1 model, given the observed neural signals at time can be represented as follows:
where the p(æ, 1 the probability of a certain kinematic given all the neural signals before, and p" (z,) ) is the model likelihood given the certain kinematic. A particle- based solution towards will be given Section 3.3, where the model likelihoods are obtained by summing the likelihoods of set of particles.
Step4: Fitness definition by the model likelihood. Finally, define the fitness value of the individual p* at time f using the likelihoods of recent time slots:
S<
p" (y,,
(5)
pre
j=!
where fpre is the length of the recent time slots. Note that, the neural decoding is sequential problem, and we do not update the model pool at each time point. That is. we usually accumulate data for several previous time slots for one update process.
3.1.3 Population evolution with the DE
With the initial model pool and the fitness function, computation algorithms. We use the DE algorithm with real-value encoding rather than those evolutionary computation algorithms with binary encoding. DE is global optimization algorithm that follows an iterative population initialization, mutation, crossover and selection procedure. The canonical DE is introduced Appendix B, and here we present the DE in model evolution.
evolve the population with evolutionary
Current-to- torpest mutation. Mutation obtains evolution directions from individual differences. mutated population of the Gh generation [v ..,,,..N N.. comes from the parent population I ls = 1, 2, .....,, NN according certain mutation strategy.

---

To avoid the premature convergence problem and diverse the population 21 we adopt DD/cuurrent-oo-pees// with archive strategy following 22
= * Fk (PG phest
+
P
where D phest is randomly chosen from the top 100 + 5% of the population, and usually E [0.05, 0.2]. Th and ri are randomly selected indices in the range of N.. being distinct from each other and also the index k. The F is list of the positive mutation factor and P randomly selected from collection storing the inferior individuals. See more details in Appendix C.
Crossover. Crossover randomly selects component from either parent or mutated vectors in each dimension. The individual after crossover is a D-dimension vector fuğ = u2 bb,)
fuk G. if ra, S C. j-
P
P.
otherwise
where Jrand is randomly chosen from [1, DJ. and the CR is a list of the positive crossover factor.
Selection. In the selection stage, the fitness function f(-) determines whether individual stays in the new population or not:
( if ff f(p
P
[
otherwise
Usually, one model pool evolution process iterates for hundreds of generations (mutation, crossover, selection) until satisfies the pre-set conditions.
3.2 Estimating (-) with dynamic Bayesian model ensemble
A Bayesian model ensemble approach (20] is employed to estimate he(-) at each time slot given the model pool. Specifically, h-).), can be represented as a weighted combination models in the poo! by:
he())
Σ
w m m-),
(9)
where wi is the assembling weight, changing with neural signals over time. It can be dynamically obtained with the dynamic Bayesian ensemble algorithm.
Specifically, the assembling weight w of each model m²()) at time t computed by the posterior probability of each model given the observation neural signals y.
p(ha(-) = m² (-) (p (Vrlyo.-1) ΣN pphc)())))) m (-) (m0
p(he(-) (- (-)
(10)
( (yelyoo-1))
where p(h+(-) = m²(()l1o.e11) is the prior probability of m² (-) at time t, which is given by the posterior probability of (.) at time / - 1; and pk (3) (/:c-1) is the likelihood as in Eq. 44.
3.3 Estimating x. with particle-based solution
With h+()) at hand, we can estimate the posterior distribution of state x, by
p(ar)yoor)) Σ E(ar h+(+) = m²()). yoo) = m²))))oo.)
(11)
where p( p(x)) = m+), Yo:)) is the posterior probability of the state with association to the 4 model, which can be estimated recursively with the Bayesian filter (i.e., the particle filter). And p(hx(-) = m²x() (-)(-))oo)) is the model weight coming from Eq. (10 with Bayesian model averaging rules.
Specifically, we propose particle-based solution to compute the model likelihoods and state posterior simultaneously (see Appendix D for details). Suppose there is set of particles {a (2) ...,, (Nv))) ). and the normalised importance weights of particle s under the hypothe- .,.,.Napar. Npars the particle-bast based marginal likelihood can
k(s) 5 1, ..., sis he(-) m²) are given by w.
5

---

be calculated as follows:
Newr
P Σ
(V.lyoaa-))
k(e) w w pep"((n
(y, (
(12)
where Dom w((()) 1. Further, the particle- based state posterior can be calculated by:
Now
p( (he(.) = m mm=(()) voa)
w
k()) 5(x, 2!
(13)
s=1
where 5 is the Dirac delta function. When the Npur -> co, the approximation approaches the true posterior density.
3.4 Strategies for model pool update
propose two strategies, namely evolve-at-changes and history-mode--arciive, to decide when to update and how to update the model pool, respectively.
The evolve-at-changes strategy. The default setting of the EvoEnsemble is update the model pool at regular interval tup. However, too large an interval may lead to untimely updates, while small an interval may lead unnecessary updates. Evolve-at-changes is a strategy to choose the update time adaptively according to the models' likelihoods. Specifically, we maintain list of log- likelihood Lvmxx-eerr where keeps the maximum liglliielshoods of all the candidate models each time slot. Every time new neural signal comes in, we average the latest three values in Lmmx-ever as current log-lite bogllikelhhodd leur, and average the three values before the latest three values in Lmax-eevrr as previous log-likelihood /pr. Meanwhile, we add ratio Tup for flexibility. Thus, the model pool will update once leur < Tup ppre. Besides, we limit the interval between two updates
or more.
The distory-model--rehhee pool using all individuals evolved by DE. However, DE tends evolve the models all towards the best model and leaves the model pool lacking diversity, which is insufficient to characterize the variability neural signals. Thus, we establish history model archive Mhis; to store the best models in each time slot, whose size is in line with the main model pool. When the iterations of DE finish, we randomly choose models in the history model archive, replacing the models in the evolved model pool. Again, we define ratio 'e to determine how many history models to be preserved in the main model pool, which can written as Npre Tpm N. So the final updated model pool "Ne
strategy. The default setting of the EvoEnsemble is to update the model
{p*1k 1,2,... N} {pi P
Phis , N1+
p p M2 .... p. J..
4 Experiments and results
4.1 Performance with simulations
To evaluate the performance of EvoEnsemble with different conditions of functional changes, we simulate several conditions of changing functions, including functions with slow or rapid monotonic changes, functions with slow rapid non-monotonic changes, and a piece-wisely changing function
(as shown by the red lines in Fig. 2 (a), see details Appendix D). Specifically, the function's
first parameter for conditions 4 the same: h 7E-3t + 1, 1 5 t 300, while the second parameters are:
(1) A 7E-4t - 1.9;
(2) h 1.12E-2f + 2.6;
(3) h/ 9.8E-6t 2.8E-3t 3.4;
(4) !/ 3.43F-5/2 +
4 4.2E-31 1.7;
For condition 5, the parameters wisce-wisely change time:

---

(a)
Condition1
Condition2
Condition3
Condition4
Condition
(b)
Condition4
12'
(c)
2000
2000
2000
2000
2000
1000
1000
1000
1000
1000
(d)
0.8
75
150
22h
900
150
22b
300
75
180
22t
300
75
250
228
300
300
620
Ad
Ground
EvoEnsemble(w/o DE)
EvoEnsembic
(e)
0.6
0.5
-15
0.
DE
-2.5
groundd
RMSE
0.6 0.1
srowe tie medial
-3.
Exairoembte
KF
EvoEnsembleew/o
EvoEnsemble
Figure 2: (a) Function parameter by tracking performance of EvoEnsemble (yellow) and Evo w/o DE) (blue) in five conditions, comparing with the ground truth (pink). (b) Visualization of model ensemble and model evolution process with EvoEnsemble(w/o DE) (above) and EvoEnsemble (below). (c) Comparisons of candidate model likelihoods change. (d) Comparisons decoding performance change. (e) Overall performance of R², CC, RMSE in five conditions comparing between four different decoders.
4,
0 < < S 168,
5,
< <t < 168,
1E-2t + 2.32,
168 < t S 345,
-2E-2t 8.36,
168 < t S 345,
(5)
5.77,
345 < S 517,
1.46,
345 < S 517,
2E-22 + 1.611E1,
517 < 650;
1E-2/ 3.71,
517 < S 650.
Performance of functional tracking. We first evaluate the functional tracking ability of EvoEnsem- ble with different conditions. Fig. 2 (a) illustrates the ground truth (red lines) and the EvoEnsemble estimated (yellow lines) function's ' parameters h2 compared with EvoEnsemble without DE- model evolution (blue lines). Results demonstrate that EvoEnsemble tracks the changes in functions closely over time for all five conditions. To further analyze how EvoEnsemble tracks the functional changes with the evolutionary model pool, present the internal tracking process within EvoEnsem- ble Fig. (b). It shows that the candidate functions in the model pool can effectively follow changes of the true function he with the model evolution process, such that the dynamic estimation of by is obtained. To quantitively evaluate the functional tracking process, we compute the likelihood of the functions estimated over time in Fig. 2 (c). EvoEnsemble keeps high likelihood to the neural data with different changes, suggesting that EvoEnsemble can effectively capture the changing functions in time.
Performance of state estimation. Here we evaluate the state estimation performance with different criteria of correlation coefficient (CC), root mean squared error (RMSE), and determination of coefficients (R²) between the estimated and ground-truth -truh state sequences. Fig. 2 (d) illustrates the R² of the estimated states. Results show that, without the model evolution process, the performance decreases gradually as the function changes, and the R² decrease to 0.552, 0, 0.187, O, and 0.896 with the five conditions respectively. EvoEnsemble is more robust functional changes that the R²s 0.975, 0.759, 0.970, 0.764, and 0.986 for the five conditions, respectively. Fig. 2 (e) further compare the state estimation performance with different neural decoders. Overall, EvoEnsemble achieves the superior performance with all the conditions, with stable high R², CC and low RMSE. Specifically, EvoEnsemble obtains CCs of 0.894, 0.905, 0.952, 0.895, and 0.997 with the five conditions, which are 12.6%, 105.2 10..2%, 1.7%, 155.0%, and 4.7% higher than Kalman filters (see details in Appendix E).
These results suggest the necessity and importance of functional tracking for robust state estimation with time-van varying functions, and demonstrate that EvoEnsemble can track the changing functions effectively to improve the accuracy and robustness of state estimation.
Effectiveness of the evolve-at-changes strategy. The evolve-at-changes strategy aims to control 'when to update' the model pool to balance the trade-off between efficiency and accuracy: frequently
7

---

(a)
Condition4
(b)
Condition4
(d)
Condition1
No updato
R² lnyli
0.9
Pout clnalgy
Final Pi 0
Desiguamiig ogg
R² logiit
0.7
No Jp0000
Foog updit
0.8
time
0.5
Update regularty
x² loylit
(c)
Condition3
0.4
conditions
Erofisannlee
Final
loglik
Foe Hocey
1. 2
Final RP 0:387
24 0.5
Updato stratagy
dogiik
Evolution works when the function parameter deviates
-i-conddonn
(ndepteedddd. Final
02
0.4
0.8
0.6
Tyme
Figure 3: (a) The changes of log- likelihood and decoding R in five different model update settings, where the dashed line represents the update time point. (b) The changes of R over time under different update ratio Tip settings, where the asterisks denote the update time point. (c) The maximum log-likelihood of the models coming from history and from evolution after each update (above) in condition (below). (d) Comparison of the decoding R² between EvoEnsemble without mesttry-model-archive strategy and with history-model-archive strategies (above). The decoding R² under different historical model preserving ratios Tpe (below).
updating the model pool helps track the changes functions more precisely while requiring more computational costs. Here compare the state estimation performance using different model updating rules with simulation 4. Fig. 3 (a) compares the log-likelihood of models and the state estimation R² with five different settings of 1) no updating: 2) updating with regular time interval; and 3) updating with the evolve-at-changes strategy. Results demonstrate that with the proposed strategy, EvoEnsemble achieves high Ra of 0.741 comparable to 0.764, while reducing 72.2% of model update costs, which well balances the efficiency and accuracy. Fig. 3 (b) analyzes the influence of updating ratio Tup using R² together with the update timing (with the asterisks). Results demonstrate that the decoding performance is relatively stable when the update ratio is 0,4 to 0.9.
Effectiveness of the history-model-archis strategy. aiscevv is proposed to improve the stability of state estimation with the information of historical models. Fig. 3 (c) shows the maximum log- ligelllelssooss of the models from the historical model pool (blue bars) and the current model pool (red bars) after every update in condition 3, where a non non-monotonic functional change is involved (as shown in the lower panel of Fig. (c)) From time slots 0 to 150 where the function gradually deviates from the initial, models in the current model pool gradually take the dominant part track the changing functions with model evolution. While from time slots 180 to 300 where the function comes back to the initial, the likelihood of historical models gradually increases. We further evaluate the decoding performance of R with different history preserve ratio Tpre- Fig. 3 (d) (above) shows that the history-model-archin five conditions. Fig. (d) (below) tests the influence of history preserve ratio (rpe)) parameter in the history-model modee--oolllahhhe strategy, which shows that the optimal rpe relies on the conditions of data. Thus Tpe can be selected hyper parameter for each dataset.
strategy improves the decoding performance in all
4.2 Performance with neural signals
We evaluate our EvoEnsemble approach on four datasets with neural signals recorded from a macaque monkey (Data-M) and human with tetraplegia (Data-P1/2/3).
Monkey neural datasets. The monkey data is from the open Zenodo dataset 23 where the monkeys were trained to perform a self-paced reaching task in an 8 * 8 grid (as shown in Fig. 4 (a)). Neural spike signals are recorded from the MI area. We use the data segment of indy-20170 2401, and we select 20 channels according to the CC between neural signals and kinematics. The neural data are smoothed with window of 3 time slots. The hand velocities are obtained from the position by a discrete derivative.
8

---

(a)
Data
(c)
Data M
(d)
Data
Ground
Extimatinn
Data
02
Vacaque morkey 'indy"
atl gri reaching
(b)
Data
Data P3
Partide
liter
Zime
2ime
Nurran with tetrollggi
3 center
ExcEreeriiee
EvoEreemble (wio
EvoErsemble
Figure 4: (a) Illustration of the grid reaching paradigm with a macaque monkey, (b) Illustration of center-out paradigm with a human participant. (c) The neural decoding performance R², CC, RMSE of the four neural datasets. (d) The tracking performance for four exemplary parameters of Data-M with EvoEnsemble(w/o ww// DE) and EvoEnsemble.
Clinical neural datasets. The clinical dataset was collected with a human subject, with two 96- channel Utah arrays implanted in the left primary motor cortex. All the clinical and experimental procedures were approved by the Medical Ethics Committee. The BCI experiment paradigm is center-out 2D cursor control with eight directions (as shown in Fig. (b)). In the online closed- BCI control, there calibration phase with two observation blocks and three ortho-impedance assistant training blocks similar to [24], where the ortho- sitho-impedance assistance ratios of the three blocks are 0.7,0.5, and 0.3 respectively. After the training blocks, full brain control block where no impedance assistance is applied. We use the training blocks for decoder training, and the sixth block for the test. Neural signals are sorted manually before each day's s experiment. Similar to Data-M, 20 neurons with top CCs between neural signals and kinematics are selected. Data-P1/2/3 were collected in 2020 2000//0//// respectively.
Neural decoding performance. The neural decoding performance with Data-M and Data-P1/21 presented in Table ! and Fig. (c). Overall, EvoEnsemble outperforms the other decoders in all the datasets. Compared with the static decoders such as the Kalman filter, EvoEnsemble obtains CCs of 0.764, 0.547, 0.561 and 0.487 with Data-M and Data-P1/2/3 respectively, which are 7. 7.2%, 39.2%, 26.1% % and 43.2 33.2% higher than the Kalman filter. The improvement is most significant in the Data-P datasets with closed-loop BCI control. We think it is partly because that the feedback the closed-loop BCI control process may lead to frequent functional changes in neural signals (as shown Fig. S1). The functional changes lead to unstable performance with static approaches of particle filters (PF) and Kalman filters. such that low CCs of 0.337, 0.437, 0.343 (PF) and 0.393, 0.445, 0.340 (KF) are obtained. With the DyEnsemble approach (00 where the neural decoder dynamically adjusts along with changes in neural signals, the performance increases 0.513, 0,485, and 0.472 for the three datasets respectively. However, without the model evolution process, the performance is limited given substantial changes in functions (in Data- D1/2, see Fig. S1). While EvoEnsemble further improves the performance by 6.6% and 15.7% on Data P1/2 with the functional tracking ability.
The three adaptive neural decoders of Kalman filter (recalib), DyEnsemble and EvoEnsemble obtain higher performance compared with static ones. Kalman filter (recalib) recalibrates the neural decoder every 20 time slots to cope with changes in neural functions [16]: and DyEnsebble increases the model pool with two strategies of connection dropout and weight perturbation (20]. Compared with the Kalman filter (recalib), EvoEnsemble improves the CCs by 8.4%, 14.7%, 2.4% and 10.9% with the four datasets respectively; and the improvement are 4.5%, 6.6%, 15.7% and 3.2% respec- tively compared with the EnEneeble approach. The results demonstrate the superior ability of EvoEnsemble in accurate and robust neural decoding with neural functional changes.
Functional tracking performance. In Fig. (d), we further illustrate the functional tracking performance of EvoEnsemble with and without the model evolution process. In each subfigure, the red line indicates the function parameter computed in every temporal time slot given neural signals and the true movement trajectories, which we regard as the ground truth of the parameter

---

Table 1: Decoding performance (CC) of different decoders.
Decoder
Data-M
Data-PI
Data-P2
Data-P3
Particle filter
0.699 1 0.004
0.337 1 0.006
0.437 1: 0.017
0.343 1 0.005
Kalman filter
0.713 + 0.000
0.393 = 0.000
0.445 + 0.000
0.340 - 0.000
Kalman filter (recalib)
[6
0.705 + 0.000
0,477 = 0.000
0.548 0.000
0.439 : 0.000
DyEnsemble 20
0.731 = 0.001
0.513 = 0.001
0.485 + 0.001
0.472 = 0.001
EvoEnsemble (ours)
0.764 + 0.007
0.547 = 0.011
0.561 + 0.017
0.487 = 0.011
over time. The yellow lines indicate the estimation the parameters with both EvoEnsemble (right) and EvoEnsemble (w/o DE) (left). Results demonstrate that, although EvoEnsemble (w/o DE) demonstrates its model adaptation ability to certain extent, the range of functional adjustment limited with fixed model pool. Thus EvoEnsemble (w/o DE) can be effective with slight functional changes in a certain range, and face difficulties with substantial changes in functions. On the other hand, the EvoEnsemble model can closely follow the changes in the parameters over time, such that it improves the functional tracking ability as well as the neural decoding accuracy (see Fig. S2 and Fig. S3 for more examples).
5 Conclusions
We propose an evolutionary ensemble Bayesian model (EvoEnsemble) for accurate and robust neural decoding under functional changes in neural signals. It incorporates evolutionary computation in a Bayesian filter framework which enables the temporal evolution of neural decoder driven by the model likelihood given incoming neural signals. Experiments with both simulations and neural signals strongly demonstrate the necessity and importance of the functional tracking for robust state estimation with time-varying functions. EvoEnsemble can track the changing functions effectively with both simulation and neural signals to improve the accuracy and robustness in state estimation. One limitation of our approach is the computational efficiency of the evolution process, since requires hundreds of generations for one model pool update process. Our future work may focus on improving the efficiency of evolutionary computation. The framework of EvoEnsemble is beneficial to wide range of problems in the nonstationary signal processing area.
6 Acknowledgment
This work was partly supported by the grants from National Key R&D Program of China (2088YFA0701000), Key R&D Program of Zhejiang (2022C0301 1), Natural Science Foundation of China (61906166, U1909202, 61925603), the Fundamental Research Funds for the Central Universi- ties, the Starry Night Science Fund of Zhejiang University Shanghai Institute for Advanced Study (SN-ZJU-SIAS-002), and the Lingang Laboratory (LGG-SS-002202-04).
References
[1] Yael Mandelblat-Cerf, Rony Paz, and Eilon Vaadia. Trial-to-trial variability of single cells in motor cortices is dynamically modified during visuomotor adaptation. Journal of Neuroscience, 29(48):15053-1500 2009.
[2] Stefano Fusi, Earl K Miller, and Mattia Rigotti. Why neurons mix: high dimensionality for higher cognition. Current opinion in neurobiology, 37:66-74, 2016.
[3] Meel Velliste, Sagi Perel, M Chance Spalding, Andrew S Whitford, and Andrew B Schwartz. Cortical control of a prosthetic arm for self-feeding. Nature, 453871999: 2008.
[4] Ujwal Chaudhary, Niels Birbaumer, and Ander Ramos-Murguialda, Brain- computer interfaces for communication and rehabilitation. Nature Reviews Neurology, 12(9):513, 2016.
[5] Chad Bouton, Ammar Shaikhouni, Nicholas V Annetta, Marcia A Bockbrader, David Friedenberg, Dylan M Nielson, Gaurav Sharma, Per B Sederberg. Bradley C Glenn, W Jerry Mysiw, et Restoring cortical control of functional movement human with quadriplegia. Nature, 533(77022:247-2550 2016.
10

---

[6] Yoh Inoue, Hongwei Mao, Steven B Suway, Josue Orellana, and Andrew B Schwartz. Decoding arm speed during reaching. Nature communications, 9(1):1-14, 2018.
[7] Yangang Li. Yu Qi, Yiwen Wang. Yueming Wang, Kedi Xu, and Gang Pan. Robust neural decod- ing by kernel regression with siamese representation learning. Journal of Neural Engineering. 18(5):056062, 2021.
[8] Yu Qi, Ling Ding, Yueming Wang, and Gang Pan. Learning robust features from nonstation- ary brain signals by multi-scale domain adaptation networks for seizure prediction. IEEE Transactions on Cognitive and Developmental Systems, 2021.
[9] Yueming Wang, Kang Lin, Yu Qi, Qi Lian, Shaozhe Feng, Zhaohui Wu, and Gang Pan. Estimating brain connectivity with varying-length time lags using recurrent neural network. IEEE Transactions on Biomedical Engineering, 65(9): 1953-1963,,, 2018.
[10] Steven M Chase, Andrew B Schwartz, and Robert E Kass. Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms. Neural networks, 22(9): 1203-121,, 2009.
[1I] Vikash Gilja, Chethan Pandarinath, Christine Blabe, Paul Nuyujukian, John D Simeral, Anish A Sarma, Brittany I. Sorice, János A Perge, Beata Jarosiewiez, Leigh Hochherg, et al. Clinical translation of a high-performance neural prosthesis. Nature medicine, 21110):1122 2015.
[12] Mark M Churchland and Krishna V Shenoy. Temporal complexity and heterogeneity of single- neuron activity in premotor and motor cortex. Journal of neurophysiology. 97(6):4235-422 4257, 2007.
[13] David Sussillo, Sergey D Stavisky, Jonathan C Kao, Stephen Ryu, and Krishna V Shenoy. Making brain-machine interfaces robust to future neural variability. Nature communications, 13749, 2016.
[14] SB Suway, J Orellana, AJC McMorland, GW Fraser, Z Liu, M Velliste, SM Chase, RE Kass, and AB Schwartz. Temporally segmented directionality in the motor cortex. Cerebral Cortex, 28(7):2326-235 23399, 2018.
[15] Leigh R Hochberg, Daniel Bacher, Beata Jarosiewicz, Nicolas Y Masse, John D Simeral, Joern Vogel, Sami Haddadin, Jie Liu, Sydney S Cash, Patrick Van Der Smagt, et al. Reach and grasp by people with tetraplegia using neurally controlled robotic arm. Nature, 485(7398):372-3755 2012.
[16] David M Brandman, Tommy Hosman, Jad Saab, Michael C Burkhart, Benjamin E Shanahan, John Ciancibello, Anish A Sarma, Daniel J Milstein, Carlos Vargas-Irwin Brian Franco, al. Rapid calibration of an intracortical brain computer interface for people with tetraplegia. Journal of neural engineering, 15(2):026007, 2018
[17] Uri T Eden, Loren M Frank, Riccardo Barbieri, Victor Solo, and Emery N Brown. Dynamic analysis of neural encoding by point process adaptive filtering. Neural computation, 16(5): 971-998. 2004.
[18] Yiwen Wang and Jose C Principe. Tracking the non-stationary neuron tuning by dual kalman filter for brain machine interfaces decoding. In 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 1720-1723, IEEE, 2008
[19] Yiwen Wang, Xiwei She, Yuxi Liao, Hongbao Li, Qiaosheng Zhang, Shaomin Zhang, Xiaoxiang Zheng, and Jose Principe. Tracking neural modulation depth by dual sequential monte carlo estimation on point processes for mraine interfaces. IEEE Transactions on Biomedical Engineering, 63(8):1728-174) 2015.
[20] Yu Qi, Bin Liu, Yueming Wang, and Gang Pan. Dynamic ensemble modeling approach to nonstationary neural decoding brain-computer interfaces. arXiv preprint arXiv: 1911,00714 2019.
11

---

[21] A Kai Qin, Vicky Ling Huang, and Ponnuthurai N Suganthan. Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE transactions on Evolutionary Computation, 13(2):398-417, 2008.
[22] Jingqiao Zhang and Arthur C Sanderson. Jade: adaptive differential evolution with optional external archive. IEEE Transactions on evolutionary computation, 13(5):945-958, 2009.
[23] Joseph E O'Doherty, MMB Cardoso, JG Makin, and PN Sabes. Nonhuman primate reaching with multichannel sensorimotor cortex electrophysiology. Zenodo http://doi org/10. 0//1000221///oooo 583331, 2017.
[24] N. A. Fitzsimmons, M. Lebedev, D. Peikon, and Miguel A.L. Nicolelis. Extracting kinematic parameters for monkey bipedal walking from cortical neuronal ensemble activity. Frontiers in Integrative Neuroscience, 3:1-19, 2009.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A] You are strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license the code and datasets? [Yes] See Section ??.
Did you include the license the code and datasets? [No] The code and the data proprictary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answerss below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the pa- per's contributions and scope? [Yes] We clearly state our work in the abstract and introduction.
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [N/A] We believe our work is relevant to basic decoding algorithm design and we do not forsee any negative impact rather positive one. Our work can help people with tetraplegia control the external devices more stably.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material
URL)? [Yes] We provide the simulation data and code with instructions in the Appendix.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We specify the parameter settings in the Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Fig. 2 Fig. and Table [
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] We only run basic computer.
12

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasnng new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4.3.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes] See Section 4.3.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] See Section 4.3.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "vTpdmUkSkAo", "reviewer": "Reviewer_g44o", "summary": "The authors propose an evolutionary ensemble method for decoding neural signals. The key challenge that this work addresses is the fact that the underlying function between the variable of interest and the signal changes over time (\"functional changes\"). The authors suggest that their method can discover and adapt to functional changes and tested its performance on synthetic and neural datasets.", "questions": "see above.", "limitations": "see above.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: the study is well motivated, and the algorithm proposed is well defined and well analyzed.\n\nWeakness: I would like to see performance comparison against more modern ML methods. I am no expert in the area but I would surprised if OLE and Kalman filters are meaningful targets for comparison. The following review (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7470933/pdf/ENEURO.0506-19.2020.pdf) appears to suggest that there are many NN based algorithms that take the functional change problem into account.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "s60qjuQwi8", "reviewer": "Reviewer_YBUd", "summary": "The work addresses the task of neural decoding (estimating intended user action from observed neural brain signals) for a changing observation model ($h_t$ in eq. 1) over time. The method involves initially training an ensemble model via minimizing a squared error loss on training data (section 3.1.1) and then dynamically evolving models in the pool towards members with a high observation likelihood score (fitness measure) computed from the current observed neural signal (sections 3.1.2-3.1.3). This adaptive procedure is iterated over time. The overall model prediction is then made from a weighted combination of ensemble members by Bayesian model averaging (section 3.2).\n\nUsing this learned and dynamically evolving observation model, the $x_t$ state (ex: intended cursor velocity) is estimated using a particle filter (section 3.3).\n\nFinally, two additional strategies are employed to control the model pool update frequency and encourage diversity (section 3.4). The __evolve-at-changes__ strategy only updates the model pool once a running average of the likelihood drops below a certain threshold. The __history-model-archive__ strategy enforces that a certain proportion of historic models be kept in the model to encourage diversity.\n\nAn ablation study of the different model components is done in experimental section 4.1 involving an artificially generated observation model.\n\nEvaluation on real neural signal datasets and against other adaptive neural decoder schemes (Kalman filter with fast recalibration [13] and DyEnsemble [17]) is done in section 4.2. The proposed EvoEnsemble method yields improved performance compared to baselines.\n", "questions": "__This question is copied from the Weaknesses section:__  \nI feel it would be useful to have more intuition as to why this approach beats the Kalman filter with recalibration method [13]. In the introductory section, it is stated that this method “usually can not cope with changes in the short-term such as single trials” although this is more an assert of an end-effect (poor performance) instead of a reasoning for it. Discussion in the experimental section likewise does not seem to go into depth as to why EvoEnsemble outperforms this baseline (bottom of page 8). It is stated that the Kalman filter with recalibration is set to update every 20 slots. Was this value optimally chosen and is it similar to the rate that EvoEnsemble determines using its dynamic update strategy? Put another way, is EvoEnsemble beating out this method due to difference in update rate or does is update mechanism itself responsible for the delta in performance? Is there further intuition as to why EvoEnsemble outperforms this baseline. \n", "limitations": "The authors briefly state in the Conclusion section that “One limitation of our approach is the computational efficiency of the evolution process, since it requires hundreds of generations for one model pool update process. Our future work may focus on improving the efficiency of evolutionary computation.”\n\nLikewise - although not a major limitation – I would question how much manual hyperparameter tuning may be required to achieve optimal performance per dataset. Ideally these could be limited.\n", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "__General Strengths:__  \n- The dynamic ensemble scheme successfully updates to changes in the neural observation model. Furthermore – from my understanding – updates are done based solely on the observed neural signal whereas other methods such as the Kalman filter with fast recalibration uses a $z_t$ surrogate (indicating the direction to target) as a surrogate for the hidden $x_t$ state to do recalibration.\n- I found the experimental ablation study (section 4.1) particularly insightful in illustrating the effects of the different method additions.\n- Beats other adaptive neural decoder baselines.\n\n__General Weaknesses:__  \n- There seems to be a relatively larger number of steps and hyperparameters which are empirically motivated and I would potentially be concerned how much manual hyperparameter tuning may be required to achieve optimal performance per dataset.  This seems especially true for the strategies for model pool updating (section 3.4). \n- I feel it would be useful to have more intuition as to why this approach beats the Kalman filter with recalibration method [13]. In the introductory section, it is stated that this method “usually can not cope with changes in the short-term such as single trials” although this is more an assert of an end-effect (poor performance) instead of a reasoning for it. Discussion in the experimental section likewise does not seem to go into depth as to why EvoEnsemble outperforms this baseline (bottom of page 8). It is stated that the Kalman filter with recalibration is set to update every 20 slots. Was this value optimally chosen and is it similar to the rate that EvoEnsemble determines using its dynamic update strategy? Put another way, is EvoEnsemble beating out this method due to difference in update rate or does is update mechanism itself responsible for the delta in performance? Is there further intuition as to why EvoEnsemble outperforms this baseline.\n\n__Originality and Significance:__  \nTo the best of my knowledge, the work is an original novel contribution to using an evolutionary ensemble for neural decoder and beats similar baselines.\n\n__Quality:__  \nThe quality is good. The ablation study on simulated data (section 4.1) highlights the application of the different method components. This is followed by practical evaluation against related baseline methods on several real neural datasets (section 4.2).\n\n__Clarity:__  \nIn general, the submission is clear. There are a few sections with somewhat awkward wording though this is minor.\n\n__Minor Additional Feedback:__  \nAt the top of page 3 (lines 83-84), the text seems to indicate that Fig. 1 is simultaneously above and below.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fRHqPz0e6eD", "reviewer": "Reviewer_Lhh8", "summary": "This paper proposed a neural decoding method to process non-stationary data for brain-computer interface, called  “EvoEnsemble”. \nThe main advantage of the method is that the model can be adaptive to the variation of neural signals through evolution in the Bayesian framework. ", "questions": "1) The DE algorithm tends to evolve all models towards the same one,  is this evolution approach really better than a well-defined model pool, considering we may can utilize the statistical features of neural signals to well define a good model pool?\n2) How about we restrict models in the pool to be disentangled  (such as orthogonal base functions), will this works better?\n3) How to understand Fig.2 condition 3, the h^2_t tracking result is quite different from the ground truth, although the likelihood and R^2 are high?\n4) In neural decoding tasks, the trend of state x is essential for the application. Still, the decoding results, which always seem to fluctuate up and down with respect to the ground truth, influence the decoding effect.\n5) Fig. 2(a) the vertical coordinates should be “h^2_t” rather than “h^1_t”\n6) In Eq(9) “\\sum_{i=1}^{N}…” should be “\\sum_{k=1}^{N}…”\n7) In Eq(11) “\\sum_{i=1}^{N}…” should be “\\sum_{k=1}^{N}…”\n", "limitations": "The authors mentioned the limitation of computational efficiency, but did not evaluate it. ", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1) The simulation experiments support that the method can capture the non-stationary characteristic of neural signals.\n2) The paper is clearly written and well-organized, and the technical contents are well explained.\n\nWeaknesses\n1)  The method is built upon the previous work, DyEnsemble (Qi et al. 2019) by introducing evolution of the model pool,  which is OK, but the novelty is not strong under the normal criterion of NeurIPS.\n2) The differential evolution (DE) algorithm tends to evolve models towards the best one, even though each candidate model evolves independently. This may make the model pool eventually becomes useless..\n3) The computational efficiency of evolutionary computation is not evaluated in the paper, which is key for an online decoding method.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "44NDgWriY1Z", "reviewer": "Reviewer_3dMs", "summary": "The paper proposes a method for adaptively updating neural decoders in Brain Machine Interfaces (BMI). Evidently the method consists in maintaining a population of decoders, and evolving them according to a genetic algorithm, with fitness determined by the likelihood of the decoders given the data. Various experiments purport to demonstrate the superior performance of the method and the effects of ablating various components.\n", "questions": "1- Line 71, we are told that we want to use the neural data y(t) to predict certain kinematic values x(t), e.g. a cursor position. Yet the whole method apparently consists in training decoders for the opposite task, i.e. predicting neural values y(t) from the kinematics x(t). Please explain this choice, and why we couldn't simply train y -> x predictors. It may be obvious to you, but not to a general reader.\n\n2- In Step 1 (line 111) we are told that we get the \"prior kinematics\" x(t) from x(t-1) through the \"transition function\" g. What is this transition function? If it already gives us x(t), what is the need of predicting it in the first place?\n\n3-  In Step 2, we supposedly use the decoders to predict neural signals y_hat / x(t), where x(t) comes from the \"prior kinematics\" from Step 1. But when I look at Equation 4, and in fact at all other equations in the paper, the estimated distribution over x(t) is instead computed from the previous y (presumably with the \"particle system\" of section 3.3)? So which one is actually used to estimate x(t)? Where are the \"prior kinematics\" used exactly?\n\n4- In Eq 4, it seems that the likelihood of each decoder (p^k) is computed by marginalizing over the possible kinematics x(t), weighted by their *estimated* probability given previous y. We are told \"a particle solution towards it\" in Section 3.3. Towards what?\n\n5- In section 3.3, apparently we should learn how p(x(t) / y(t'<t)) is estimated for any x. Unfortunately this section is incomprehensible. We are told of \"particles\" x1...xn (chosen how?), with \"important weights\" (would that be \"importance weights\"?)  called omega^k(s)_t... Where do these omegas come from? What do they represent? How are they computed? \n\n\n6- More generally: x(t) is estimated from the decoders (Equation 11). Decoders are evaluated based on their output for estimated x(t). This seems circular. At which point does *real* data about x(t) enter the system? AFAICT, in the paper, every single value of x(t) used in the equations is estimated,  with nothing to anchor it to real data.  So the predictions could be absolutely random, as long as the decoders can use them to predict neural data y(t) accurately? Surely this is not the case! I suspect some part of the description is missing.\n\n7- How are the model predictions  y-hat(t) compared with real neural data y(t)? That is, how is p^k(y_t / t_t) (in Equation 4, right-hand side, first factor in the integral) actually computed?\n\n8- The experiments are similarly lacking in description:\n\n- In section 4.1 (simulations) what is h_t? How are the x(t) and the y(t) generated? Also, what is \"E\" ? \n- In section 4.2 (monkey reaching) what is being tracked? \n- The description of the human clinical dataset is particularly lacking. What is the x(t) being tracked? What is an \"ortho-impedance\"? \n\nI note that in all of these, the \"transition function\" g(t) is never mentioned, leaving me confused about  what exactly it represent.\n\n\n9- More pressingly for the actual impact of the method, it is briefly mentioned that the evolutionary algorithm is run for \"hundreds of generations\" (line 149). How long does that take? Does it mean that the method cannot be applied online to real-time data? That would be a serious limitation.\n\n", "limitations": "The authors briefly mention that their method requires running an algorithm for hundreds of generations, for every update of the decoder pool (?). However, they do not quantify the time taken. Most importantly, they do not say whether this prevents the method from being applied online, in real-time, in \"live\" BMIs (as opposed to the offline datasets used in the experiments). \n\nPlease explicitly state whether the method is currently applicable to online, real-time neural data from live BMIs or not.\n\nSocial impact does not seem relevant in the current state of the method.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nThe method seems potentially interesting. Unfortunately the extremely confused description makes it impossible to understand, much less reproduce the method. This applies both to the description of the method itself, as well as the experiments, as explained below. \n\nUPDATE: After the author's response and clarifications, the paper is much improved and I have updated my score towards acceptance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept