# Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming
## 3 Results
### Classifier Accuracy for Limited Training Data  
It is of interest to keep the calibration time before BCI operation as short as possible. We mimic this problem by training the classifier with as few training epochs as possible. We evaluate the performance of all classifiers for different levels of available training data and apply the cross-validation procedure nine times (the number of blocks in the training fold) for all subjects, keeping the corresponding number of blocks in the training folds and dropping the rest. Figures 3 and A1 show each classifier's accuracy relative to the data availability. We statistically compare the two newly proposed classifiers, stbf-struct and stbf-shrunk, for different levels of training data availability using a one-sided paired [PERSON] rank-sum test with Holm correction for the multiple pairwise comparisons between classifiers. We performed this analysis three times: by only using the first trial of a block, by averaging epochs across the first two trials of a block, and across the first five trials of a block. Results validated on one trial are reported in Table 1, two-trial results in Table 2, and five-trial results in Table 3.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**1 Trial**} & \multicolumn{8}{c}{**Nb. of Training Blocks**} \\ \cline{2-10}  & **1** & **2** & **3** & **4** & **5** & **6** & **7** & **8** & **9** \\ \hline stbf-struct \(>\) stbf-struct \(>\) stbf-shrunk & **0.005** & **0.030** & **0.015** & 0.543 & 0.284 & 0.159 & – & – & 0.952 \\ stbf-struct \(>\) stbf-semp & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** \\ stbf-struct \(>\) xdawn+rg & 0.086 & **0.002** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** \\ \hline stbf-shrunk \(>\) stbf-semp & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** \\ stbf-shrunk \(>\) xdawn+rg & – & 0.499 & 0.071 & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **<0.001** & **0.001** \\ \hline \hline \end{tabular}
\end{table}
Table 1: \(p\)-values calculated via the one-sided paired Wilcoxon rank-sum test with Holm correction using one testing trial for different classifiers and levels of data availability. \(p\)-values \(<\) 0.05 are considered significant and marked bold.  
Figure 2: Average cross-validated stbf-struct accuracy using one trial per block for validation over all 21 subjects relative to the number of iterations used to estimate the Kronecker–Toeplitz-structured shrunk covariance. Error bars represent the first and third quartiles. The accuracy does not improve when using more than one iteration. (**A**) Results for 1, 2, and 5 trials using only the first block in each training fold for training. (**B**) Results for 1, 2, and 5 trials using all nine training blocks in the training folds.  
The tables show that stbf-struct has a significant advantage over stbf-shrunk when the number of training blocks is low. This effect is present for 1-, 2-, and 5-trial evaluations. This advantage decreases (the \(p\)-value increases) when adding more training blocks. Both stbf-struct and stbf-shrunk perform significantly better than stbf-emp for all evaluated settings. Compared to xdawn+rg, stbf-struct also has significantly higher accuracy in almost all evaluated settings, except when using only one training block. stbf-shrunk does not outperform xdawn+rg when training data are scarce but gains a significant advantage when using more training data.