Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose an evolutionary ensemble method for decoding neural signals. The key challenge that this work addresses is the fact that the underlying function between the variable of interest and the signal changes over time ("functional changes"). The authors suggest that their method can discover and adapt to functional changes and tested its performance on synthetic and neural datasets.
strengths_and_weaknesses: Strength: the study is well motivated, and the algorithm proposed is well defined and well analyzed.

Weakness: I would like to see performance comparison against more modern ML methods. I am no expert in the area but I would surprised if OLE and Kalman filters are meaningful targets for comparison. The following review (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7470933/pdf/ENEURO.0506-19.2020.pdf) appears to suggest that there are many NN based algorithms that take the functional change problem into account.
questions: see above.
limitations: see above.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The work addresses the task of neural decoding (estimating intended user action from observed neural brain signals) for a changing observation model ($h_t$ in eq. 1) over time. The method involves initially training an ensemble model via minimizing a squared error loss on training data (section 3.1.1) and then dynamically evolving models in the pool towards members with a high observation likelihood score (fitness measure) computed from the current observed neural signal (sections 3.1.2-3.1.3). This adaptive procedure is iterated over time. The overall model prediction is then made from a weighted combination of ensemble members by Bayesian model averaging (section 3.2).

Using this learned and dynamically evolving observation model, the $x_t$ state (ex: intended cursor velocity) is estimated using a particle filter (section 3.3).

Finally, two additional strategies are employed to control the model pool update frequency and encourage diversity (section 3.4). The __evolve-at-changes__ strategy only updates the model pool once a running average of the likelihood drops below a certain threshold. The __history-model-archive__ strategy enforces that a certain proportion of historic models be kept in the model to encourage diversity.

An ablation study of the different model components is done in experimental section 4.1 involving an artificially generated observation model.

Evaluation on real neural signal datasets and against other adaptive neural decoder schemes (Kalman filter with fast recalibration [13] and DyEnsemble [17]) is done in section 4.2. The proposed EvoEnsemble method yields improved performance compared to baselines.

strengths_and_weaknesses: __General Strengths:__  
- The dynamic ensemble scheme successfully updates to changes in the neural observation model. Furthermore – from my understanding – updates are done based solely on the observed neural signal whereas other methods such as the Kalman filter with fast recalibration uses a $z_t$ surrogate (indicating the direction to target) as a surrogate for the hidden $x_t$ state to do recalibration.
- I found the experimental ablation study (section 4.1) particularly insightful in illustrating the effects of the different method additions.
- Beats other adaptive neural decoder baselines.

__General Weaknesses:__  
- There seems to be a relatively larger number of steps and hyperparameters which are empirically motivated and I would potentially be concerned how much manual hyperparameter tuning may be required to achieve optimal performance per dataset.  This seems especially true for the strategies for model pool updating (section 3.4). 
- I feel it would be useful to have more intuition as to why this approach beats the Kalman filter with recalibration method [13]. In the introductory section, it is stated that this method “usually can not cope with changes in the short-term such as single trials” although this is more an assert of an end-effect (poor performance) instead of a reasoning for it. Discussion in the experimental section likewise does not seem to go into depth as to why EvoEnsemble outperforms this baseline (bottom of page 8). It is stated that the Kalman filter with recalibration is set to update every 20 slots. Was this value optimally chosen and is it similar to the rate that EvoEnsemble determines using its dynamic update strategy? Put another way, is EvoEnsemble beating out this method due to difference in update rate or does is update mechanism itself responsible for the delta in performance? Is there further intuition as to why EvoEnsemble outperforms this baseline.

__Originality and Significance:__  
To the best of my knowledge, the work is an original novel contribution to using an evolutionary ensemble for neural decoder and beats similar baselines.

__Quality:__  
The quality is good. The ablation study on simulated data (section 4.1) highlights the application of the different method components. This is followed by practical evaluation against related baseline methods on several real neural datasets (section 4.2).

__Clarity:__  
In general, the submission is clear. There are a few sections with somewhat awkward wording though this is minor.

__Minor Additional Feedback:__  
At the top of page 3 (lines 83-84), the text seems to indicate that Fig. 1 is simultaneously above and below.

questions: __This question is copied from the Weaknesses section:__  
I feel it would be useful to have more intuition as to why this approach beats the Kalman filter with recalibration method [13]. In the introductory section, it is stated that this method “usually can not cope with changes in the short-term such as single trials” although this is more an assert of an end-effect (poor performance) instead of a reasoning for it. Discussion in the experimental section likewise does not seem to go into depth as to why EvoEnsemble outperforms this baseline (bottom of page 8). It is stated that the Kalman filter with recalibration is set to update every 20 slots. Was this value optimally chosen and is it similar to the rate that EvoEnsemble determines using its dynamic update strategy? Put another way, is EvoEnsemble beating out this method due to difference in update rate or does is update mechanism itself responsible for the delta in performance? Is there further intuition as to why EvoEnsemble outperforms this baseline. 

limitations: The authors briefly state in the Conclusion section that “One limitation of our approach is the computational efficiency of the evolution process, since it requires hundreds of generations for one model pool update process. Our future work may focus on improving the efficiency of evolutionary computation.”

Likewise - although not a major limitation – I would question how much manual hyperparameter tuning may be required to achieve optimal performance per dataset. Ideally these could be limited.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposed a neural decoding method to process non-stationary data for brain-computer interface, called  “EvoEnsemble”. 
The main advantage of the method is that the model can be adaptive to the variation of neural signals through evolution in the Bayesian framework. 
strengths_and_weaknesses: Strengths: 
1) The simulation experiments support that the method can capture the non-stationary characteristic of neural signals.
2) The paper is clearly written and well-organized, and the technical contents are well explained.

Weaknesses
1)  The method is built upon the previous work, DyEnsemble (Qi et al. 2019) by introducing evolution of the model pool,  which is OK, but the novelty is not strong under the normal criterion of NeurIPS.
2) The differential evolution (DE) algorithm tends to evolve models towards the best one, even though each candidate model evolves independently. This may make the model pool eventually becomes useless..
3) The computational efficiency of evolutionary computation is not evaluated in the paper, which is key for an online decoding method.


questions: 1) The DE algorithm tends to evolve all models towards the same one,  is this evolution approach really better than a well-defined model pool, considering we may can utilize the statistical features of neural signals to well define a good model pool?
2) How about we restrict models in the pool to be disentangled  (such as orthogonal base functions), will this works better?
3) How to understand Fig.2 condition 3, the h^2_t tracking result is quite different from the ground truth, although the likelihood and R^2 are high?
4) In neural decoding tasks, the trend of state x is essential for the application. Still, the decoding results, which always seem to fluctuate up and down with respect to the ground truth, influence the decoding effect.
5) Fig. 2(a) the vertical coordinates should be “h^2_t” rather than “h^1_t”
6) In Eq(9) “\sum_{i=1}^{N}…” should be “\sum_{k=1}^{N}…”
7) In Eq(11) “\sum_{i=1}^{N}…” should be “\sum_{k=1}^{N}…”

limitations: The authors mentioned the limitation of computational efficiency, but did not evaluate it.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a method for adaptively updating neural decoders in Brain Machine Interfaces (BMI). Evidently the method consists in maintaining a population of decoders, and evolving them according to a genetic algorithm, with fitness determined by the likelihood of the decoders given the data. Various experiments purport to demonstrate the superior performance of the method and the effects of ablating various components.

strengths_and_weaknesses: 
The method seems potentially interesting. Unfortunately the extremely confused description makes it impossible to understand, much less reproduce the method. This applies both to the description of the method itself, as well as the experiments, as explained below. 

UPDATE: After the author's response and clarifications, the paper is much improved and I have updated my score towards acceptance.
questions: 1- Line 71, we are told that we want to use the neural data y(t) to predict certain kinematic values x(t), e.g. a cursor position. Yet the whole method apparently consists in training decoders for the opposite task, i.e. predicting neural values y(t) from the kinematics x(t). Please explain this choice, and why we couldn't simply train y -> x predictors. It may be obvious to you, but not to a general reader.

2- In Step 1 (line 111) we are told that we get the "prior kinematics" x(t) from x(t-1) through the "transition function" g. What is this transition function? If it already gives us x(t), what is the need of predicting it in the first place?

3-  In Step 2, we supposedly use the decoders to predict neural signals y_hat / x(t), where x(t) comes from the "prior kinematics" from Step 1. But when I look at Equation 4, and in fact at all other equations in the paper, the estimated distribution over x(t) is instead computed from the previous y (presumably with the "particle system" of section 3.3)? So which one is actually used to estimate x(t)? Where are the "prior kinematics" used exactly?

4- In Eq 4, it seems that the likelihood of each decoder (p^k) is computed by marginalizing over the possible kinematics x(t), weighted by their *estimated* probability given previous y. We are told "a particle solution towards it" in Section 3.3. Towards what?

5- In section 3.3, apparently we should learn how p(x(t) / y(t'<t)) is estimated for any x. Unfortunately this section is incomprehensible. We are told of "particles" x1...xn (chosen how?), with "important weights" (would that be "importance weights"?)  called omega^k(s)_t... Where do these omegas come from? What do they represent? How are they computed? 


6- More generally: x(t) is estimated from the decoders (Equation 11). Decoders are evaluated based on their output for estimated x(t). This seems circular. At which point does *real* data about x(t) enter the system? AFAICT, in the paper, every single value of x(t) used in the equations is estimated,  with nothing to anchor it to real data.  So the predictions could be absolutely random, as long as the decoders can use them to predict neural data y(t) accurately? Surely this is not the case! I suspect some part of the description is missing.

7- How are the model predictions  y-hat(t) compared with real neural data y(t)? That is, how is p^k(y_t / t_t) (in Equation 4, right-hand side, first factor in the integral) actually computed?

8- The experiments are similarly lacking in description:

- In section 4.1 (simulations) what is h_t? How are the x(t) and the y(t) generated? Also, what is "E" ? 
- In section 4.2 (monkey reaching) what is being tracked? 
- The description of the human clinical dataset is particularly lacking. What is the x(t) being tracked? What is an "ortho-impedance"? 

I note that in all of these, the "transition function" g(t) is never mentioned, leaving me confused about  what exactly it represent.


9- More pressingly for the actual impact of the method, it is briefly mentioned that the evolutionary algorithm is run for "hundreds of generations" (line 149). How long does that take? Does it mean that the method cannot be applied online to real-time data? That would be a serious limitation.


limitations: The authors briefly mention that their method requires running an algorithm for hundreds of generations, for every update of the decoder pool (?). However, they do not quantify the time taken. Most importantly, they do not say whether this prevents the method from being applied online, in real-time, in "live" BMIs (as opposed to the offline datasets used in the experiments). 

Please explicitly state whether the method is currently applicable to online, real-time neural data from live BMIs or not.

Social impact does not seem relevant in the current state of the method.
review_title: null

Meta-review: metareview: The review ratings/confidences were 5/2, 6/2, 4/4, and 5/2. Although the average rating of 5 was just above the acceptance threshold, I think that it should somehow be discounted by the lower confidence levels. Although I myself does not have expertise in the field of BCI, as for the reviewers' evaluation, I think that they basically agreed on the following points:
- The problem is well motivated.
- The proposed method was built on DyEnsemble with some empirically-motivated extensions to have decoder models dynamically evolving. One could then argue that the proposal is not groundbreaking but somehow incremental.
- The authors showed experimentally that the proposed method works well compared with a number of other existing methods.

I also noticed that the authors made revision (adding a paragraph at the end of Section 2: It can be observed in the August 10 revision), which would have improved readability of this paper. I would thus recommend acceptance of this paper, provided that there is room for it.
confidence: Less certain
Decision: Accept