# Ensemble Voting-Based Multichannel EEG Classification in a Subject-Independent P300 Speller
## 3 Proposed Methodology
### Ensemble Voting  
Ensemble learning is a technique of combining several models to achieve more stable results. Recently, ensemble learning has become a popular choice for brain signals' features classification. For instance, a number of classifiers, such as SVM, can be applied on the same data. The models have different hyperparameters for classifying the P300 speller data [27]. Apart from the SVM, a popular choice for EEG classification is to ensemble several CNN classifiers [30] with different architectures. Ensemble learning exhibits stable results for P300 component classification; however, it always uses more computational resources to be trained because it contains not one but several models. That is why the proposed methodology is designed for GT. Here, the model is trained on a merged dataset from different subjects and does not require retraining for the new user. Using the above-described classifiers soft-voting ensemble models were designed, as shown in Figure 3. Essentially, four different ensemble architectures were designed: the LDA-\(k\)NN model (Figure 3a), LDA-SVM-\(k\)NN (Figure 3b), LDA-SVM-\(k\)NN-CNN (Figure 3c), and the W-LDA-SVM-\(k\)NN model(Figure 3d). Three of these models used simple averaging voting, while the last was a weighted-averaging voter. As mentionedin Section 3, CNN uses 2D EEG data directly, while LDA, SVM and \(k\)NN models require channel-averaging before features extraction. We decided to combine the LDA and \(k\)NN models in order to achieve the most efficiency in terms of a time complexity ensemble voting model. The fusion of LDA, \(k\)NN, and SVM and its weighted version were assumed to be more accurate than LDA-\(k\)NN, as SVM is one of the best models for P300 classification. Combining CNN with the LDA and/or \(k\)NN in one ensemble voter seemed to not be effective in terms of computational complexity, as CNN requires much more time to process the data than LDA and \(k\)NN. SVM however requires more computational resources, due to the kernel trick and optimal hyperplane construction. Thus, CNN was added to the fusion of LDA, SVM, and \(k\)NN in order to see whether 2D data classification could improve the existing ensemble model. Despite the fact that the LDA-SVM-\(k\)NN-CNN ensemble model may require much more  
Figure 3: Ensemble averaging models: (**a**) LDA-\(k\)NN model; (**b**) LDA-SVM-\(k\)NN model; (**c**) LDA-SVM-\(k\)NN-CNN model; (**d**) W-LDA-SVM-\(k\)NN model. time to process the data, it is assumed that it can outperform other ensemble voting models in terms of accuracy. The classification results of the simple ensemble-averaged voting models were computed as follows  
\[P_{avg}(X|y=1)=\frac{\sum_{i=1}^{N}P_{i}(X|y=1)}{N}, \tag{5}\]  
where \(P_{i}(X|y=1)\) is the \(i\)th classifier's prediction of EEG vector \(X\) containing the target P300 component. \(N\) is the number of classifiers in the ensemble voting model. It can be assumed that weighted voting would be more efficient for the P300 speller rather than just ensemble averaging. For instance, the weighted voting based on CNN, SVM, and stepwise LDA was introduced for the P300 speller with the aim of improving EEG classification [35]. In this work, we assume that the weighted ensemble voting based on LDA, SVM, and \(k\)NN classifiers can improve the performance of the system. Figure 3d represents the weighted voting model's structure (W-LDA-SVM-\(k\)NN), where each result of the inner classifiers is multiplied by the weight \(w_{i}\) as  
\[P_{w}(X|y=1)=\frac{\sum_{i=1}^{N}w_{i}P_{i}(X|y=1)}{\sum_{i=1}^{N}w_{i}}. \tag{6}\]  
In order to find the optimal weights, GS or random search (RS) can be applied. In GS it is necessary to run through the grid of the possible triplets of weights, while in RS hyperparameters are sampled randomly. As there are only three weights needing to be found, random combinations of three weights are generated quite fast. The most optimal weights combination is further selected among the generated parameters without any aliasing.