Paper:
Information-based Adaptive Stimulus Selection to Optimize Communication Efficiency in Brain-Computer Interfaces
Stimulus-driven brain-computer interfaces (BCIs), such as the P300 speller, rely on using a sequence of sensory stimuli to elicit specific neural responses as control signals, while a user attends to relevant target stimuli that occur within the sequence. In current BCIs, the stimulus presentation schedule is typically generated in a pseudo-random fashion. Given the non-stationarity of brain electrical signals, a better strategy could be to adapt the stimulus presentation schedule in real-time by selecting the optimal stimuli that will maximize the signal-to-noise ratios of the elicited neural responses and provide the most information about the user's intent based on the uncertainties of the data being measured. However, the high-dimensional stimulus space limits the development of algorithms with tractable solutions for optimized stimulus selection to allow for real-time decision-making within the stringent time requirements of BCI processing. We derive a simple analytical solution of an information-based objective function for BCI stimulus selection by transforming the high-dimensional stimulus space into a one-dimensional space that parameterizes the objective function -the prior probability mass of the stimulus under consideration, irrespective of its contents. We demonstrate the utility of our adaptive stimulus selection algorithm in improving BCI performance with results from simulation and real-time human experiments.

Review:
The authors consider the problem of adaptively selecting stimulus presentations online for a BCI based on evoked potentials.   They focus on the problem of presenting the best “flash groups” in a P300 spelling application.  They derive a method to select stimuli that maximize mutual information between a target character and the information likely to be extracted from EEG signals (in the form of classifier scores).  They derive their methods with an eye towards computational efficiency to allow their method to work online during BCI use and demonstrate their method with simulated and real, online experiments.   The authors do a very nice job of motivating the need for online stimulus selection for evoked BCI work and it is clear the authors are addressing an important problem.  However, I feel the paper, in it’s current form, may not be appropriate for NIPS.     My first concern is the work presented here may be very application specific.  It’s not clear how the methods presented here would generalize to problems outside of the evoked BCI paradigm.  Additionally, it seems that many of the technical contributions of the paper (in particular the form of optimization objective) are motivated by the need to run their algorithm in real time and leverage special structure (binary representation) specific to the P300 spelling paradigm.    My other second concern with the paper is that while the authors have simplified the objective function, it’s not clear to me the importance of this. The search space is still (if I understand correctly) over a set of 2^M possible stimulus configurations, which can potentially be quite large.  It seems the authors are currently searching this space with a Greedy algorithm.  It then seems the authors have turned a hard search problem into a slightly less difficult one, but from a technical point of view, it seems this may be an incremental improvement.    Finally, the authors are to be commended for honestly reporting their results on real data, but as they point out, the current results leave room for improvement, as classification accuracy of the target stimuli with their adaptive strategy decreases with time.   The authors indicate they have good reason to believe they may understand the cause of this, but it would be helpful to see this demonstrated in practice.    Finally, a minor concern with the paper is one of technical clarity.  I found the derivations throughout sections 2.2 – 2.3 a bit hard to follow.   For example, it is not clear to me that 3a is the correct equation for the conditional mutual information between Y_{t+1}^h and C*.  It may very well be, but it seems to be there is a term missing in the denominator of the log term.      Response to authors rebuttal: I thank the authors for their detailed and clear response.  After reviewing their response, I now see how their formulation of the objective fundamentally reduces the complexity of the objective which must be calculated during run-time.  Additionally, while I still find the performance during real experiments somewhat concerning, I also acknowledge the substantial decreases in expected stopping times achieved with their method in these real experiments.  For this reason, I am revising my overall score upward.