[33] Florian Yger, Maxime Berar, and Fabien Lotte. "Riemannian approaches in brain-computer interfaces: a review". In: *IEEE Transactions on Neural Systems and Rehabilitation Engineering* 25.10 (2016), pp. 1753-1762.

[34] Marie-Constance Corsi et al. "Functional connectivity ensemble method to enhance BCI performance (FUCONE)". In: *IEEE Transactions on Biomedical Engineering* (2022).

[35] Vernon J Lawhern et al. "EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces". In: *Journal of neural engineering* 15.5 (2018), p. 056013.

[36] Robin Tibor Schirrmeister et al. "Deep learning with convolutional neural networks for EEG decoding and visualization". In: *Human brain mapping* 38.11 (2017), pp. 5391-5420.

[37] Martín Abadi et al. *TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems*. Software available from tensorflow.org. 2015. URL: https://www.tensorflow.org/.

[38] Student. "The probable error of a mean". In: *Biometrika* (1908), pp. 1-25.

[39] Frank Wilcoxon. "Individual comparisons by ranking methods". In: *Breakthroughs in statistics*. Springer, 1992, pp. 196-202.

[40] Samuel A Stouffer et al. *The american soldier: Adjustment during army life.* (studies in social psychology in world war ii), vol. 1. Princeton Univ. Press, 1949.

[41] Kisung You and Hae-Jeong Park. "Geometric learning of functional brain network on the correlation manifold". In: *Scientific reports* 12.1 (2022), pp. 1-13.

[42] Olivier Ledoit and Michael Wolf. "A well-conditioned estimator for large-dimensional covariance matrices". In: *Journal of multivariate analysis* 88.2 (2004), pp. 365-411.

[43] Daniel Bartz and Klaus-Robert Müller. "Covariance shrinkage for autocorrelated data". In: *Advances in neural information processing systems* 27 (2014).

[44] David Sabbagh et al. "Manifold-regression to predict from MEG/EEG brain signals without source modeling". In: *Advances in Neural Information Processing Systems* 32 (2019).

[45] M. Moakher. "A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices". In: *SIAM J. Matrix Anal. Appl.* 26.3 (Apr. 2005), pp. 735-747.

[46] Marco Congedo, Alexandre Barachant, and Rajendra Bhatia. "Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review". In: *Brain-Computer Interfaces* 4.3 (2017), pp. 155-174.

[47] X. Pennec, P. Fillard, and N. Ayache. *A Riemannian Framework for Tensor Computing*. Tech. rep. 5255. INRIA, Sophia Antipolis, July 2004.
<figure><img src="image_6.png" /><figcaption>Figure 6: Results for Right vs Left hand classification, using WS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure 6: Results for Right vs Left hand classification, using WS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Figure 7: Plot (a) shows the best hyper-parameters for the ACM+MDM algorithm for each session and subject in the left hand right hand task, using the BNCI2014001 dataset. The red box defines the grid search domain. The heat-map in plot (b) shows the area that contains the hyper-parameter that provide a classification score of 99% of the maximum score obtained. These results are computed for Subject 1 in BNCI2014001 Session E in a right hand vs left hand task. The heat-map of plot (c) shows the maximal occurrence of best area of hyper-parameters across subjects and sessions of the BNCI2014001 dataset. It is obtained by counting the number of non empty boxes of plots similar to plot (b) over all sessions and subjects.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 8: Total computational time (training+testing) vs ROC AUC criterion for the MOABB within-session evaluation of BNCI2014001 with the task right vs left hand classification. Similar behaviors occur in the other evaluation procedures, tasks and datasets.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 7: Plot (a) shows the best hyper-parameters for the ACM+MDM algorithm for each session and subject in the left hand right hand task, using the BNCI2014001 dataset. The red box defines the grid search domain. The heat-map in plot (b) shows the area that contains the hyper-parameter that provide a classification score of 99% of the maximum score obtained. These results are computed for Subject 1 in BNCI2014001 Session E in a right hand vs left hand task. The heat-map of plot (c) shows the maximal occurrence of best area of hyper-parameters across subjects and sessions of the BNCI2014001 dataset. It is obtained by counting the number of non empty boxes of plots similar to plot (b) over all sessions and subjects.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 8: Total computational time (training+testing) vs ROC AUC criterion for the MOABB within-session evaluation of BNCI2014001 with the task right vs left hand classification. Similar behaviors occur in the other evaluation procedures, tasks and datasets.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 9: Total computational time (training+testing) for the MOABB within-session evaluation for subject 1 of BNCI2014001 with the task right hand vs feet using ACM+MDM without the grid search hyper-parameter selection but setting specifically the order. Computational times at order 1 correspond to standard covariance. Similar behaviors occur in the other evaluation procedures, subjects, tasks and datasets.</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>Figure 9: Total computational time (training+testing) for the MOABB within-session evaluation for subject 1 of BNCI2014001 with the task right hand vs feet using ACM+MDM without the grid search hyper-parameter selection but setting specifically the order. Computational times at order 1 correspond to standard covariance. Similar behaviors occur in the other evaluation procedures, subjects, tasks and datasets.</figcaption></figure>
# A Supplementary Material

## A.1 Riemann geometry

Over the past 10 years, Riemannian approaches have shown interesting performances for SPD matrix classification [8]. The key idea behind this method is to map the covariance matrix into an appropriate geometrical space. The space of SPD matrix is a differentiable manifold with a natural Riemann structure [9].

The spaces of $n \times n$ real square and real symmetric matrices are denoted by $M(n)$ and $S(n)$, respectively: $S(n) = \{\mathbf{S} \in M(n), \mathbf{S}^T = \mathbf{S}\}$. The space of SPD matrices is defined as $P(n) = \{\mathbf{P} \in S(n), \mathbf{P} > 0\}$. In this way, we can represent a matrices $\mathbf{P} \in S(n)$ as a point on a Riemann manifold of dimension $n(n+1)/2$. Since the space of SPD matrix is a manifold of non zero curvature, we cannot expect that concepts of Euclidean geometry preserve the manifold structure. It is possible to define a distance between two SPD matrix $\mathbf{P}_1$ and $\mathbf{P}_2$ in $P(n)$ as the length of the geodesic that connects $\mathbf{P}_1$ and $\mathbf{P}_2$ on the Riemann manifold. In this paper, we use the affine-invariant metric [45]

$$ \delta_R(\mathbf{P}_1, \mathbf{P}_2) = \left\| \log \left( \mathbf{P}_1^{-\frac{1}{2}} \mathbf{P}_2 \mathbf{P}_1^{-\frac{1}{2}} \right) \right\|_F = \left[ \sum_{i=1}^{n} \log^2 \lambda_i \right]^{1/2} \quad (7) $$

where $\| \cdot \|_F$ is the Frobenius norm, $\text{Log}()$ is the logarithm of a matrix and $\lambda_i$ are the eigenvalues of $\mathbf{P}_1^{-\frac{1}{2}} \mathbf{P}_2 \mathbf{P}_1^{-\frac{1}{2}}$. The choice of the metric is not unique but we choose this metric because it possesses relevant properties for BCI applications: invariance under reordering, invariance under congruent transformation and invariance under inversion [46]. To define the mean of $m$ SPD matrices $(\mathbf{P}_1, ..., \mathbf{P}_m)$, we use the concept of Fréchet mean [45]

$$ \bar{\mathbf{P}} = \underset{\mathbf{P} \in P(n)}{\operatorname{argmin}} \sum_{i=1}^{m} \delta_{R}^{2}(\mathbf{P}, \mathbf{P}_{i}) \quad (8) $$

$\bar{\mathbf{P}}$ is called the geometric mean in the Riemannian sense or the center of mass of $(\mathbf{P}_1, ..., \mathbf{P}_m)$. Note that, for $m > 2$, no closed-form expression is known, so we need to employ an iterative algorithm [47]. Due to the properties of Eq. (7), the geometric mean inherits the properties of congruence invariance [46].

For every point $\mathbf{P}_i \in P(n)$, in a complete Riemann space, we can define a point $\mathbf{S}_i \in S(n)$ in the tangent space $\mathbf{T}_P$ to the Riemann surface at point $\mathbf{P} \in P(n)$. The tangent space has zero curvature by definition and allows us to use standard ML algorithms and other classical tools. Mappings from the Riemann surface to the tangent space and from the tangent space to the Riemann surface, are provided respectively by the Exp and Log maps [45] (see Fig. A1):

$$ \operatorname{Exp}_{\mathbf{P}}(\mathbf{S}_i) = \mathbf{P}^{1/2} \operatorname{Exp}(\mathbf{P}^{-1/2} \mathbf{S}_i \mathbf{P}^{-1/2}) \mathbf{P}^{1/2} \operatorname{Log}_{\mathbf{P}}(\mathbf{P}_i) = \mathbf{P}^{1/2} \operatorname{Log}(\mathbf{P}^{-1/2} \mathbf{P}_i \mathbf{P}^{-1/2}) \mathbf{P}^{1/2} \quad (9) $$

In the context of a Riemann geometry based classification algorithm, we estimate a spatial covariance matrix for each epoch with the standard approach of the spatial sample covariance estimator

$$ \operatorname{Cov}(\mathbf{X}) = \frac{1}{Time - 1} \sum_{i=1}^{Time} \mathbf{X}_i \mathbf{X}_i^T, \quad (10) $$

where $T$ is the transpose operator. This is not the only option for estimating the covariance. We could have used other estimators, but this one has the advantage of being unbiased when the number of electrodes is much smaller than the temporal instants contained in an epoch.

As the covariance matrix belongs to the SPD space, it is possible to classify the different mental states using the Riemannian framework. Note that the classification can be performed either
# A Supplementary Material

## A.1 Riemann geometry

Over the past 10 years, Riemannian approaches have shown interesting performances for SPD matrix classification [8]. The key idea behind this method is to map the covariance matrix into an appropriate geometrical space. The space of SPD matrix is a differentiable manifold with a natural Riemann structure [9].

The spaces of $n \times n$ real square and real symmetric matrices are denoted by $M(n)$ and $S(n)$, respectively: $S(n) = \{\mathbf{S} \in M(n), \mathbf{S}^T = \mathbf{S}\}$. The space of SPD matrices is defined as $P(n) = \{\mathbf{P} \in S(n), \mathbf{P} > 0\}$. In this way, we can represent a matrices $\mathbf{P} \in S(n)$ as a point on a Riemann manifold of dimension $n(n+1)/2$. Since the space of SPD matrix is a manifold of non zero curvature, we cannot expect that concepts of Euclidean geometry preserve the manifold structure. It is possible to define a distance between two SPD matrix $\mathbf{P}_1$ and $\mathbf{P}_2$ in $P(n)$ as the length of the geodesic that connects $\mathbf{P}_1$ and $\mathbf{P}_2$ on the Riemann manifold. In this paper, we use the affine-invariant metric [45]

$$ \delta_R(\mathbf{P}_1, \mathbf{P}_2) = \left\| \log \left( \mathbf{P}_1^{-\frac{1}{2}} \mathbf{P}_2 \mathbf{P}_1^{-\frac{1}{2}} \right) \right\|_F = \left[ \sum_{i=1}^{n} \log^2 \lambda_i \right]^{1/2} \quad (7) $$

where $\|\cdot\|_F$ is the Frobenius norm, $\text{Log}(\cdot)$ is the logarithm of a matrix and $\lambda_i$ are the eigenvalues of $\mathbf{P}_1^{-\frac{1}{2}} \mathbf{P}_2 \mathbf{P}_1^{-\frac{1}{2}}$. The choice of the metric is not unique but we choose this metric because it possesses relevant properties for BCI applications: invariance under reordering, invariance under congruent transformation and invariance under inversion [46]. To define the mean of $m$ SPD matrices $(\mathbf{P}_1, ..., \mathbf{P}_m)$, we use the concept of Fréchet mean [45]

$$ \bar{\mathbf{P}} = \underset{\mathbf{P} \in P(n)}{\operatorname{argmin}} \sum_{i=1}^{m} \delta_R^2(\mathbf{P}, \mathbf{P}_i) \quad (8) $$

$\bar{\mathbf{P}}$ is called the geometric mean in the Riemannian sense or the center of mass of $(\mathbf{P}_1, ..., \mathbf{P}_m)$. Note that, for $m > 2$, no closed-form expression is known, so we need to employ an iterative algorithm [47]. Due to the properties of Eq. (7), the geometric mean inherits the properties of congruence invariance [46].

For every point $\mathbf{P}_i \in P(n)$, in a complete Riemann space, we can define a point $\mathbf{S}_i \in S(n)$ in the tangent space $\mathbf{T}_P$ to the Riemann surface at point $\mathbf{P} \in P(n)$. The tangent space has zero curvature by definition and allows us to use standard ML algorithms and other classical tools. Mappings from the Riemann surface to the tangent space and from the tangent space to the Riemann surface, are provided respectively by the Exp and Log maps [45] (see Fig. A1):

$$ \operatorname{Exp}_{\mathbf{P}}(\mathbf{S}_i) = \mathbf{P}^{1/2} \operatorname{Exp}(\mathbf{P}^{-1/2} \mathbf{S}_i \mathbf{P}^{-1/2}) \mathbf{P}^{1/2} \operatorname{Log}_{\mathbf{P}}(\mathbf{P}_i) = \mathbf{P}^{1/2} \operatorname{Log}(\mathbf{P}^{-1/2} \mathbf{P}_i \mathbf{P}^{-1/2}) \mathbf{P}^{1/2} \quad (9) $$

In the context of a Riemann geometry based classification algorithm, we estimate a spatial covariance matrix for each epoch with the standard approach of the spatial sample covariance estimator

$$ \operatorname{Cov}(\mathbf{X}) = \frac{1}{Time - 1} \sum_{i=1}^{Time} \mathbf{X}_i \mathbf{X}_i^T, \quad (10) $$

where $T$ is the transpose operator. This is not the only option for estimating the covariance. We could have used other estimators, but this one has the advantage of being unbiased when the number of electrodes is much smaller than the temporal instants contained in an epoch.

As the covariance matrix belongs to the SPD space, it is possible to classify the different mental states using the Riemannian framework. Note that the classification can be performed either
<figure><img src="image_10.png" /><figcaption>Figure A1: A Riemann manifold and the maps from/to the tangent space at point P.</figcaption></figure>

on the Riemann surface, using algorithms specifically defined on the Riemann manifold as Minimum Distance to the Mean (MDM), or by moving points to tangent space and using standard classification algorithms such as Support Vector Machine (SVM) or Logistic Regression [17].

<table><caption>Table A1: Parameters for ML pipelines</caption><thead><tr><th>Pipeline</th><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td>"COV+EN"</td><td>l1 ratio</td><td>[0.15, 0.30, 0.45, 0.60, 0.75]</td></tr><tr><td rowspan="2">"TANG+SVM"</td><td>C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>Kernel</td><td>"linear", "rbf"</td></tr><tr><td>"CSP+LDA"</td><td>nfilter</td><td>[1, 2, 3, 4, 5, 6, 7, 8]</td></tr><tr><td rowspan="4">"ACM+TANG+SVM"</td><td>C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>Kernel</td><td>"linear", "rbf"</td></tr><tr><td>Order</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr><tr><td>Lag</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr><tr><td rowspan="2">"ACM+MDM"</td><td>Order</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr><tr><td>Lag</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr></tbody></table>
<figure><img src="image_12.png" /><figcaption>Figure A1: A Riemann manifold and the maps from/to the tangent space at point P.</figcaption></figure>

on the Riemann surface, using algorithms specifically defined on the Riemann manifold as Minimum Distance to the Mean (MDM), or by moving points to tangent space and using standard classification algorithms such as Support Vector Machine (SVM) or Logistic Regression [17].

<table><caption>Table A1: Parameters for ML pipelines</caption><thead><tr><th>Pipeline</th><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td>"COV+EN"</td><td>l1 ratio</td><td>[0.15, 0.30, 0.45, 0.60, 0.75]</td></tr><tr><td rowspan="2">"TANG+SVM"</td><td>C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>Kernel</td><td>"linear", "rbf"</td></tr><tr><td>"CSP+LDA"</td><td>nfilter</td><td>[1, 2, 3, 4, 5, 6, 7, 8]</td></tr><tr><td rowspan="4">"ACM+TANG+SVM"</td><td>C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>Kernel</td><td>"linear", "rbf"</td></tr><tr><td>Order</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr><tr><td>Lag</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr><tr><td rowspan="2">"ACM+MDM"</td><td>Order</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr><tr><td>Lag</td><td>[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]</td></tr></tbody></table>
<table><caption>Table A2: Common parameters for DL pipelines.</caption><thead><tr><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td>Epoch</td><td>300</td></tr><tr><td>Batch Size</td><td>64</td></tr><tr><td>Validation Split</td><td>0.2</td></tr><tr><td>Loss</td><td>Sparse Categorical Crossentropy</td></tr><tr><td>Optimizer</td><td>Adam<br>Learning Rate = 0.001</td></tr><tr><td>Callbacks ES</td><td>Early Stopping<br>Patience = 75<br>Monitor = Validation Loss</td></tr><tr><td>Callbacks LR</td><td>ReduceLROnPlateau<br>Patience = 75<br>Monitor = Validation Loss<br>Factor = 0.5</td></tr></tbody></table>
<table><caption>Table A2: Common parameters for DL pipelines.</caption><thead><tr><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td>Epoch</td><td>300</td></tr><tr><td>Batch Size</td><td>64</td></tr><tr><td>Validation Split</td><td>0.2</td></tr><tr><td>Loss</td><td>Sparse Categorical Crossentropy</td></tr><tr><td>Optimizer</td><td>Adam<br>Learning Rate = 0.001</td></tr><tr><td>Callbacks ES</td><td>Early Stopping<br>Patience = 75<br>Monitor = Validation Loss</td></tr><tr><td>Callbacks LR</td><td>ReduceLROnPlateau<br>Patience = 75<br>Monitor = Validation Loss<br>Factor = 0.5</td></tr></tbody></table>
<figure><img src="image_11.png" /><figcaption>Figure A2: Results for Right vs Left hand classification, using CS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>Figure A2: Results for Right vs Left hand classification, using CS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>Figure A3: Results for Right hand vs Feet classification, using WS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Figure A3: Results for Right hand vs Feet classification, using WS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>Figure A4: Results for Right hand vs Feet classification, using CS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>Figure A4: Results for Right hand vs Feet classification, using CS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Figure A5: Results for Right hand vs Left hand vs Feet classification, using WS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_16.png" /><figcaption>Figure A5: Results for Right hand vs Left hand vs Feet classification, using WS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>Figure A6: Results for Right hand vs Left hand vs Feet classification, using CS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>
<figure><img src="image_17.png" /><figcaption>Figure A6: Results for Right hand vs Left hand vs Feet classification, using CS evaluation. Plot (a) provides a combined meta analysis (over all datasets) of the different pipelines. It shows the significance that the algorithm on the y-axis is better than the one on the x-axis. The color represents the significance level of the difference of accuracy, in terms of t-values. We only show significant interactions (*p* < 0.05). Plot (b) is a box plot of the score with the error of the different pipelines for every dataset considered. Plots (c), (d), (e), (f), (g) and (h) show the meta analysis of ACM+TGSP+SVM (Grid) against respectively Cov+EN, FgMDM, CSP+LDA, DeepConvNet, EEGNet and ShallowConvNet. We show the standardized mean differences of p-values computed as one-tailed Wilcoxon signed-rank test for the hypothesis given as title of the plot. The gray bar denotes the 95% interval. \* stands for *p* < 0.05, \*\* for *p* < 0.01, and \*\*\* for *p* < 0.001.</figcaption></figure>