# Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory
## 6 Conclusions  
This paper aimed to develop a subject's shared system for recognizing seven phonemes and four words collected during imaginary speech from the Kara One Database. To achieve the proposed goal, the database was preprocessed and features were computed, in order to decode the hidden information regarding imaginary speech. The features were based on computing the cross-covariance in the frequency domain and were introduced to a CNNLSTM neural network for final classification in the desired classes.  
During our research, we observed that, by computing the cross-covariance in frequency domain, the feature space turned into a space where the utterances could be separated easily. This conclusion was drawn after analyzing the feature space in two dimensions by computing the LDA algorithm for feature reduction.  
Another feature comparison was made between the extracted features and the features obtained after the first CNNLSTM layer. Tables 4 and 5 represent the Euclidean distances between the mean of each class, and it can be easily seen that these distances increased after the CNNLSTM layer. This suggests that the feature space of the input vectors changed, so that the classes became more separable. This is one of the advantages of deep neural networks, especially CNN networks, which use the first layers for feature extraction before the actual classification of the data.  
This paper also showed an improvement for the system performance when using a CNNLSTM neural network, with respect to CNN. The accuracy increased from 37% to 43% when using CNNLSTM and the same processing chain of the database. The advantage of CNNLSTM is the consideration of both the spatial and temporal connections. CNN uses the convolution for the spatial connection between the channels, while LSTM brings the long-term memory that is vital for the non-stationary time variant signals, such as EEG.  
The proposed system considers the portability of a possible real-time portable device. Therefore, we also studied the system's behavior when reducing the number of channels for classification. We separated the electrodes into main areas, i.e., frontal, central, and occipital, studied the areas involved in imaginary speech production, and selected the channels accordingly for the anatomical speech brain areas study. We concluded that 93% of the information is concentrated in the anatomical speech brain areas, obtaining an accuracy of 40% for 29 channels used, in comparison to the 62 used in the beginning. Even if the accuracy dropped by 3% using only 29 electrodes, the system brings more advantages in a matter of time for execution, portability, comfort, and cost.  
Conceptualization, A.-L.R. and O.G.; Methodology, A.-L.R. and O.G.; Software, A.-L.R.; Data curation, A.-L.R.; Writing - original draft, A.-L.R.; Writing - review & editing, O.G.; Supervision, O.G. All authors have read and agreed to the published version of the manuscript. This research received no external funding.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **No. Channels** & **Time [s]** & **Accuracy [\%]** \\ \hline L/R Frontal + Central + Occipital & 34 & 0.0258 & 35.18/35.66 \\ \hline L/R ASBA & 16 & 0.0068 & 40.27 \\ \hline L + R Frontal & 23 & 0.0167 & 35.82 \\ \hline L + R Central & 27 & 0.0156 & 36.78 \\ \hline L + R Occipital & 16 & 0.0089 & 26.06 \\ \hline L + R ASBA & 29 & 0.0209 & 40.27 \\ \hline \hline \end{tabular}
\end{table}
Table 11: _Cont._