Paper:
A fundamental challenge in designing brain-computer interfaces (BCIs) is decoding behavior from time-varying neural oscillations. In typical applications, decoders are constructed for individual subjects and with limited data leading to restrictions on the types of models that can be utilized. Currently, the best performing decoders are typically linear models capable of utilizing rigid timing constraints with limited training data. Here we demonstrate the use of Long Short-Term Memory (LSTM) networks to take advantage of the temporal information present in sequential neural data collected from subjects implanted with electrocorticographic (ECoG) electrode arrays performing a finger flexion task. Our constructed models are capable of achieving accuracies that are comparable to existing techniques while also being robust to variation in sample data size. Moreover, we utilize the LSTM networks and an affine transformation layer to construct a novel architecture for transfer learning. We demonstrate that in scenarios where only the affine transform is learned for a new subject, it is possible to achieve results comparable to existing state-of-the-art techniques. The notable advantage is the increased stability of the model during training on novel subjects. Relaxing the constraint of only training the affine transformation, we establish our model as capable of exceeding performance of current models across all training data sizes. Overall, this work demonstrates that LSTMs are a versatile model that can accurately capture temporal patterns in neural data and can provide a foundation for transfer learning in neural decoding.

Review:
The paper describes an approach to use LSTM’s for finger classification based on ECOG. and a transfer learning extension of which two variations exists. From the presented results, the LSTM model is not an improvement over a basic linear model. The transfer learning models performs better than subject specific models on a subset of the subjects. Overall, I think the problem Is interesting but the technical description and the evaluation can be improved. I am not confident in the analysis of the model. Additionally, the citations are not always correct and some related work is not referenced at all. For the reasons above, I am not willing to recommend the paper for acceptance at his point.
The paper tackles a problem that is challenging and interesting. Unfortunately, the dataset size is limited. 
This is common for brain data and makes evaluation much more difficult.
 The paper states that all hyper-parameters were optimized on 75% of subject B data.
The actual model training was done using cross-validation. 
So far this approach seems more or less correct but in this case I would argue that subject B should not be considered for evaluation since its data is heavily used for hyper-parameter optimization and the results obtained on this subject are at risk of being biased.
Omitting subject B from the analysis, each non-transfer learning method  performs best on one of the remaining subjects.
Therefore it is not clear that an LSTM model is an improvement. 
For transfer learning (ignoring B again) only C and D are improved but it is unclear what the variance is.
In the BCI community there are many approaches that use transfer learning with linear models. I think that it would be interesting how linear model transfer learning would fare in this task. 
A second issue that might inflate the results is the fact that the data is shuffled randomly. While this is common practice for most machine learning tasks, it is dangerous when working with brain data due to changes in the signal over time. As a result, selecting random samples might inflate the accuracy compared to having a proper train and test set that are separated in time. Ideally the cross-validation should be done using contiguous folds. 
I am not quite sure whether it should be possible to have an accuracy above chance level half a second before movement onset? How long does motor preparation take? I am not familiar with this specific subject, but a quick search gave me a reaction time for sprinters of .15 seconds. Is it possible that cue processing activity was used to obtain the classification result? Please discuss this effect because I am do not understand why it should be possible to get above chance level accuracy half a second before movement onset. 
There are also several technical aspects that are not clear to me. I am confident that I am unable to re-implement the proposed method and their baseline given the information provided.
LDA baseline:
—————————
For the LDA baseline, how is the varying sequence length treated? 
Ledoit wolf analytic  regularization is used, but it isn not referenced. If you use that method, cite the paper. 
The claim that LDA works for structured experimental tasks but not in naturalistic scenarios and will not generalize when electrode count and trial duration increases is a statement that might be true. However, it is never empirically verified.  Therefore this statement should not be in the paper. 
HMM baseline
—————————
How are the 1 and the 2 state HMM used w.r.t. the 5 classes? It is unclear to me how they are used exactly. Is there a single HMM per class? Please be specific. 
LSTM Model
—————
What is the random and language model initialization scheme? I can only find the sequence auto-encoder in the Dai and Le paper. 
Model analysis
——————————-
It is widely accepted in the neuroimaging community that linear weight vectors should not be interpreted directly. It is actually impossible to do this.  Therefore this section should be completely re-done. Please read the following paper on this subject.
Haufe, Stefan, et al. "On the interpretation of weight vectors of linear models in multivariate neuroimaging." Neuroimage 87 (2014): 96-110.
References
———— 
Ledoit wolf regularization is used but not cited. Fix this.
There is no citation for the random/language model initialization of the LSTM model. I have no clue how to do this without proper citation.
Le at al (2011) are referenced for auto-encoders. This is definitely not the right citation. 
Rumelhart, Hinton, & Williams, 1986a; Bourlard & Kamp, 1988; Hinton & Zemel, 1994 and Bengio, Lamblin, Popovici, & Larochelle, 2007; Ranzato, Poultney, Chopra, & LeCun, 2007 are probably all more relevant.
Please cite the relevant work on affine transformations for transfer learning especially the work by morioka et al who also learn an input transferm.
Morioka, Hiroshi, et al. "Learning a common dictionary for subject-transfer decoding with resting calibration." NeuroImage 111 (2015): 167-178.