[70] R. Leeb, F. Lee, C. Keinrath, R. Scherer, H. Bischof, and G. Pfurtscheller. Brain-computer communication: motivation, aim, and impact of exploring a virtual apartment. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 15(4):473-482, 2007.

[71] A.-L. Ligozat, J. Lefevre, A. Bugeau, and J. Combaz. Unraveling the hidden environmental impacts of AI solutions for environment life cycle assessment of AI solutions. *Sustainability*, 14(9):5172, 2022.

[72] Z. Lin, C. Zhang, W. Wu, and X. Gao. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. *IEEE transactions on biomedical engineering*, 53(12):2610-2614, 2006.

[73] F. Lotte and C. Guan. Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. *IEEE Transactions on biomedical Engineering*, 58(2):355-362, 2010.

[74] F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi. A review of classification algorithms for EEG-based brain-computer interfaces. *Journal of neural engineering*, 4(2):R1, 2007.

[75] F. Lotte, L. Bougrain, A. Cichocki, M. Clerc, M. Congedo, A. Rakotomamonjy, and F. Yger. A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update. *Journal of neural engineering*, 15(3): 031005, 2018.

[76] A. S. Luccioni, S. Viguier, and A.-L. Ligozat. Estimating the carbon footprint of bloom, a 176b parameter language

model. *Journal of Machine Learning Research*, 24(253):1-15, 2023.

[77] S. J. Luck. *An introduction to the event-related potential technique*. The MIT Press, 2014.

[78] V. Martínez-Cagigal, J. Thielen, E. Santamaria-Vazquez, S. Pérez-Velasco, P. Desain, and R. Hornero. Brain-computer interfaces based on code-modulated visual evoked potentials (c-VEP): A literature review. *Journal of Neural Engineering*, 18(6):061002, 2021.

[79] M. Moakher. A differential geometric approach to the geometric mean of symmetric positive-definite matrices. *SIAM journal on matrix analysis and applications*, 26(3):735-747, 2005.

[80] T. Moreau, M. Massias, A. Gramfort, P. Ablin, P.-A. Bannier, B. Charlier, M. Dagréou, T. D. la Tour, G. Durif, C. F. Dantas, Q. Klopfenstein, J. Larson, E. Lai, T. Lefort, B. Malézieux, B. Moufad, B. T. Nguyen, A. Rakotomamonjy, Z. Ramzi, J. Salmon, and S. Vaiter. Benchopt: Reproducible, efficient and collaborative optimization benchmarks. In *Advances in Neural Information Processing Systems (NeurIPS)*, volume 36, New-Orleans, LA, USA, Nov. 2022. Curran Associates, Inc.

[81] J. Müller-Gerking, G. Pfurtscheller, and H. Flyvbjerg. Designing optimal spatial filters for single-trial EEG classification in a movement task. *Clinical neurophysiology*, 110(5):787-798, 1999.

[82] S. Nagel and M. Spüler. World's fastest brain-computer interface: combining EEG2Code with deep learning. *PloS one*, 14(9):e0221909, 2019.
[83] M. Nakanishi, Y. Wang, Y.-T. Wang, Y. Mitsukura, and T.-P. Jung. Enhancing unsupervised canonical correlation analysis-based frequency detection of SSVEPs by incorporating background EEG. In *2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society*, pages 3053–3056. IEEE, 2014.

[84] M. Nakanishi, Y. Wang, Y.-T. Wang, and T.-P. Jung. A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials. *PloS one*, 10(10): e0140703, 2015.

[85] M. Nakanishi, Y. Wang, X. Chen, Y.-T. Wang, X. Gao, and T.-P. Jung. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. *IEEE Transactions on Biomedical Engineering*, 65(1):104–112, 2017.

[86] C. S. Nam, A. Nijholt, and F. Lotte. *Brain–computer interfaces handbook: technological and theoretical advances*. CRC Press, 2018.

[87] V. P. Oikonomou, G. Liaros, K. Georgiadis, E. Chatzilari, K. Adam, S. Nikolopoulos, and I. Kompatsiaris. Comparative evaluation of state-of-the-art algorithms for SSVEP-based BCIs. *arXiv preprint arXiv:1602.00904*, 2016.

[88] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. PyTorch: An imperative style, high-performance deep learning library. In *Advances in*

*Neural Information Processing Systems* 32, pages 8024–8035, 2019.

[89] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. *Journal of Machine Learning Research*, 12:2825–2830, 2011.

[90] J. Peirce, R. Hirst, and M. MacAskill. *Building experiments in PsychoPy*. Sage, 2022.

[91] C. R. Pernet, S. Appelhoff, K. J. Gorgolewski, G. Flandin, C. Phillips, A. Delorme, and R. Oostenveld. EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. *Scientific data*, 6(1):103, 2019.

[92] H. Ramoser, J. Muller-Gerking, and G. Pfurtscheller. Optimal spatial filtering of single trial EEG during imagined hand movement. *IEEE transactions on rehabilitation engineering*, 8(4):441–446, 2000.

[93] Y. Renard, F. Lotte, G. Gibert, M. Congedo, E. Maby, V. Delannoy, O. Bertrand, and A. Lécuyer. Openvibe: An open-source software platform to design, test, and use brain–computer interfaces in real and virtual environments. *Presence*, 19(1):35–53, 2010.

[94] A. Riccio, L. Simione, F. Schettini, A. Pizzimenti, M. Inghilleri, M. O. Belardinelli, D. Mattia, and F. Cincotti. Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis. *Frontiers in human neuroscience*, 7:732, 2013.

[95] B. Rivet, A. Souloumiac, V. Attina, and G. Gibert. xDAWN algorithm to en-
[83] M. Nakanishi, Y. Wang, Y.-T. Wang, Y. Mitsukura, and T.-P. Jung. Enhancing unsupervised canonical correlation analysis-based frequency detection of SSVEPs by incorporating background EEG. In *2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society*, pages 3053–3056. IEEE, 2014.

[84] M. Nakanishi, Y. Wang, Y.-T. Wang, and T.-P. Jung. A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials. *PloS one*, 10(10): e0140703, 2015.

[85] M. Nakanishi, Y. Wang, X. Chen, Y.-T. Wang, X. Gao, and T.-P. Jung. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. *IEEE Transactions on Biomedical Engineering*, 65(1):104–112, 2017.

[86] C. S. Nam, A. Nijholt, and F. Lotte. *Brain–computer interfaces handbook: technological and theoretical advances*. CRC Press, 2018.

[87] V. P. Oikonomou, G. Liaros, K. Georgiadis, E. Chatzilari, K. Adam, S. Nikolopoulos, and I. Kompatsiaris. Comparative evaluation of state-of-the-art algorithms for SSVEP-based BCIs. *arXiv preprint arXiv:1602.00904*, 2016.

[88] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. PyTorch: An imperative style, high-performance deep learning library. In *Advances in*

*Neural Information Processing Systems* 32, pages 8024–8035, 2019.

[89] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. *Journal of Machine Learning Research*, 12:2825–2830, 2011.

[90] J. Peirce, R. Hirst, and M. MacAskill. *Building experiments in PsychoPy*. Sage, 2022.

[91] C. R. Pernet, S. Appelhoff, K. J. Gorgolewski, G. Flandin, C. Phillips, A. Delorme, and R. Oostenveld. EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. *Scientific data*, 6(1):103, 2019.

[92] H. Ramoser, J. Muller-Gerking, and G. Pfurtscheller. Optimal spatial filtering of single trial EEG during imagined hand movement. *IEEE transactions on rehabilitation engineering*, 8(4):441–446, 2000.

[93] Y. Renard, F. Lotte, G. Gibert, M. Congedo, E. Maby, V. Delannoy, O. Bertrand, and A. Lécuyer. Openvibe: An open-source software platform to design, test, and use brain–computer interfaces in real and virtual environments. *Presence*, 19(1):35–53, 2010.

[94] A. Riccio, L. Simione, F. Schettini, A. Pizzimenti, M. Inghilleri, M. O. Belardinelli, D. Mattia, and F. Cincotti. Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis. *Frontiers in human neuroscience*, 7:732, 2013.

[95] B. Rivet, A. Souloumiac, V. Attina, and G. Gibert. xDAWN algorithm to en-
hance evoked potentials: application to
brain–computer interface. *IEEE Transactions on Biomedical Engineering*, 56
(8):2035–2043, 2009.

[96] S. RMJ. The american soldier, vol. 1: Adjustment during army life, 1949.

[97] C. Rommel, J. Paillard, T. Moreau, and A. Gramfort. Data augmentation for learning predictive models on EEG: a systematic comparison. *Journal of Neural Engineering*, 19(6):066020, 2022.

[98] R. N. Roy. *Neuroergonomics and physiological computing contributions to human-machine interaction*. PhD thesis, Université Paul Sabatier, 2022.

[99] Y. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and J. Faubert. Deep learning-based electroencephalography analysis: a systematic review. *Journal of neural engineering*, 16(5): 051001, 2019.

[100] A. Salami, J. Andreu-Perez, and H. Gillmeister. EEG-ITNet: An explainable inception temporal convolutional network for motor imagery classification. *IEEE Access*, 10:36672–36685, 2022.

[101] G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R. Wolpaw. BCI2000: a general-purpose brain-computer interface (BCI) system. *IEEE Transactions on biomedical engineering*, 51(6):1034–1043, 2004.

[102] R. Scherer, J. Faller, E. V. Friedrich, E. Opisso, U. Costa, A. Kübler, and G. R. Müller-Putz. Individually adapted imagery improves brain-computer interface performance in end-users with disability. *PloS one*, 10(5):e0123727, 2015.

[103] R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstetter, K. Eggensperger, M. Tangermann,

F. Hutter, W. Burgard, and T. Ball. Deep learning with convolutional neural networks for EEG decoding and visualization. *Human brain mapping*, 38(11): 5391–5420, 2017.

[104] S. Schneider, A. Baevski, R. Collobert, and M. Auli. wav2vec: Unsupervised pre-training for speech recognition. *arXiv preprint arXiv:1904.05862*, 2019.

[105] J. Shin, A. von Lühmann, B. Blankertz, D.-W. Kim, J. Jeong, H.-J. Hwang, and K.-R. Müller. Open access dataset for EEG+ NIRS single-trial classification. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 25(10): 1735–1745, 2016.

[106] J. Sosulski and M. Tangermann. Spatial filters for auditory evoked potentials transfer between different experimental conditions. In *GBCIC*, 2019.

[107] J. Sosulski and M. Tangermann. UMM: Unsupervised mean-difference maximization. *arXiv preprint arXiv:2306.11830*, 2023.

[108] J. Sosulski, D. Hübner, A. Klein, and M. Tangermann. Online optimization of stimulation speed in an auditory brain-computer interface under time constraints. *arXiv preprint arXiv:2109.06011*, 2021.

[109] D. Steyrl, R. Scherer, J. Faller, and G. R. Müller-Putz. Random forests in non-invasive sensorimotor rhythm brain-computer interfaces: a practical and convenient non-linear classifier. *Biomedical Engineering/Biomedizinische Technik*, 61(1):77–86, 2016.

[110] Q. Sun, L. Zheng, W. Pei, X. Gao, and Y. Wang. A 120-target brain-computer interface based on code-modulated vi-
hance evoked potentials: application to
brain–computer interface. *IEEE Transactions on Biomedical Engineering*, 56
(8):2035–2043, 2009.

[96] S. RMJ. The american soldier, vol. 1: Adjustment during army life, 1949.

[97] C. Rommel, J. Paillard, T. Moreau, and A. Gramfort. Data augmentation for learning predictive models on EEG: a systematic comparison. *Journal of Neural Engineering*, 19(6):066020, 2022.

[98] R. N. Roy. *Neuroergonomics and physiological computing contributions to human-machine interaction*. PhD thesis, Université Paul Sabatier, 2022.

[99] Y. Roy, H. Banville, I. Albuquerque, A. Gramfort, T. H. Falk, and J. Faubert. Deep learning-based electroencephalography analysis: a systematic review. *Journal of neural engineering*, 16(5): 051001, 2019.

[100] A. Salami, J. Andreu-Perez, and H. Gillmeister. EEG-ITNet: An explainable inception temporal convolutional network for motor imagery classification. *IEEE Access*, 10:36672–36685, 2022.

[101] G. Schalk, D. J. McFarland, T. Hinterberger, N. Birbaumer, and J. R. Wolpaw. BCI2000: a general-purpose brain-computer interface (BCI) system. *IEEE Transactions on biomedical engineering*, 51(6):1034–1043, 2004.

[102] R. Scherer, J. Faller, E. V. Friedrich, E. Opisso, U. Costa, A. Kübler, and G. R. Müller-Putz. Individually adapted imagery improves brain-computer interface performance in end-users with disability. *PloS one*, 10(5):e0123727, 2015.

[103] R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstetter, K. Eggensperger, M. Tangermann,

F. Hutter, W. Burgard, and T. Ball. Deep learning with convolutional neural networks for EEG decoding and visualization. *Human brain mapping*, 38(11): 5391–5420, 2017.

[104] S. Schneider, A. Baevski, R. Collobert, and M. Auli. wav2vec: Unsupervised pre-training for speech recognition. *arXiv preprint arXiv:1904.05862*, 2019.

[105] J. Shin, A. von Lühmann, B. Blankertz, D.-W. Kim, J. Jeong, H.-J. Hwang, and K.-R. Müller. Open access dataset for EEG+ NIRS single-trial classification. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 25(10): 1735–1745, 2016.

[106] J. Sosulski and M. Tangermann. Spatial filters for auditory evoked potentials transfer between different experimental conditions. In *GBCIC*, 2019.

[107] J. Sosulski and M. Tangermann. UMM: Unsupervised mean-difference maximization. *arXiv preprint arXiv:2306.11830*, 2023.

[108] J. Sosulski, D. Hübner, A. Klein, and M. Tangermann. Online optimization of stimulation speed in an auditory brain-computer interface under time constraints. *arXiv preprint arXiv:2109.06011*, 2021.

[109] D. Steyrl, R. Scherer, J. Faller, and G. R. Müller-Putz. Random forests in non-invasive sensorimotor rhythm brain-computer interfaces: a practical and convenient non-linear classifier. *Biomedical Engineering/Biomedizinische Technik*, 61(1):77–86, 2016.

[110] Q. Sun, L. Zheng, W. Pei, X. Gao, and Y. Wang. A 120-target brain-computer interface based on code-modulated vi-
sual evoked potentials. *Journal of Neuroscience Methods*, 375:109597, 2022.

[111] M. Tangermann, K.-R. Müller, A. Aertsen, N. Birbaumer, C. Braun, C. Brunner, R. Leeb, C. Mehring, K. J. Miller, G. Mueller-Putz, et al. Review of the BCI competition IV. *Frontiers in neuroscience*, page 55, 2012.

[112] J. Thielen, P. van den Broek, J. Farquhar, and P. Desain. Broad-band visually evoked potentials: re (con) volution in brain-computer interfacing. *PloS one*, 10(7):e0133797, 2015.

[113] J. Thielen, P. Marsman, J. Farquhar, and P. Desain. From full calibration to zero training for a code-modulated visual evoked potentials for brain–computer interface. *Journal of Neural Engineering*, 18(5):056007, 2021.

[114] M. Tietz, T. J. Fan, D. Nouri, B. Bossan, and skorch Developers. skorch: A scikit-learn compatible neural network library that wraps PyTorch, July 2017. URL https://skorch.readthedocs.io/en/stable/.

[115] E. Vaineau, A. Barachant, A. Andreev, P. C. Rodrigues, G. Cattan, and M. Congedo. Brain invaders adaptive versus non-adaptive P300 brain-computer interface dataset. *arXiv preprint arXiv:1904.09111*, 2019.

[116] G. Van Veen, A. Barachant, A. Andreev, G. Cattan, P. C. Rodrigues, and M. Congedo. Building brain invaders: EEG data of an experimental validation. *arXiv preprint arXiv:1905.05182*, 2019.

[117] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. *Advances in neural information processing systems*, 30, 2017.

[118] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental algorithms for scientific computing in python. *Nature Methods*, 17:261–272, 2020.

[119] Z. Wan, R. Yang, M. Huang, N. Zeng, and X. Liu. A review on transfer learning in EEG signal analysis. *Neurocomputing*, 421:1–14, 01 2021.

[120] Y. Wang, X. Chen, X. Gao, and S. Gao. A benchmark dataset for SSVEP-based brain–computer interfaces. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 25(10):1746–1752, 2016.

[121] X. Wei, A. A. Faisal, M. Grosse-Wentrup, A. Gramfort, S. Chevallier, V. Jayaram, C. Jeunet, S. Bakas, S. Ludwig, K. Barmpas, et al. 2021 BEETL competition: Advancing transfer learning for subject independence & heterogeneous EEG data sets. In *NeurIPS 2021 Competitions and Demonstrations Track*, pages 205–219. PMLR, 2022.

[122] F. Wilcoxon. Individual comparisons by ranking methods. In *Breakthroughs in Statistics: Methodology and Distribution*, pages 196–202. Springer, 1992.

[123] Z. Wu and D. Yao. The influence of cognitive tasks on different frequencies
sual evoked potentials. *Journal of Neuroscience Methods*, 375:109597, 2022.

[111] M. Tangermann, K.-R. Müller, A. Aertsen, N. Birbaumer, C. Braun, C. Brunner, R. Leeb, C. Mehring, K. J. Miller, G. Mueller-Putz, et al. Review of the BCI competition IV. *Frontiers in neuroscience*, page 55, 2012.

[112] J. Thielen, P. van den Broek, J. Farquhar, and P. Desain. Broad-band visually evoked potentials: re (con) volution in brain-computer interfacing. *PloS one*, 10(7):e0133797, 2015.

[113] J. Thielen, P. Marsman, J. Farquhar, and P. Desain. From full calibration to zero training for a code-modulated visual evoked potentials for brain–computer interface. *Journal of Neural Engineering*, 18(5):056007, 2021.

[114] M. Tietz, T. J. Fan, D. Nouri, B. Bossan, and skorch Developers. skorch: A scikit-learn compatible neural network library that wraps PyTorch, July 2017. URL https://skorch.readthedocs.io/en/stable/.

[115] E. Vaineau, A. Barachant, A. Andreev, P. C. Rodrigues, G. Cattan, and M. Congedo. Brain invaders adaptive versus non-adaptive P300 brain-computer interface dataset. *arXiv preprint arXiv:1904.09111*, 2019.

[116] G. Van Veen, A. Barachant, A. Andreev, G. Cattan, P. C. Rodrigues, and M. Congedo. Building brain invaders: EEG data of an experimental validation. *arXiv preprint arXiv:1905.05182*, 2019.

[117] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. *Advances in neural information processing systems*, 30, 2017.

[118] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental algorithms for scientific computing in python. *Nature Methods*, 17:261–272, 2020.

[119] Z. Wan, R. Yang, M. Huang, N. Zeng, and X. Liu. A review on transfer learning in EEG signal analysis. *Neurocomputing*, 421:1–14, 01 2021.

[120] Y. Wang, X. Chen, X. Gao, and S. Gao. A benchmark dataset for SSVEP-based brain–computer interfaces. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 25(10):1746–1752, 2016.

[121] X. Wei, A. A. Faisal, M. Grosse-Wentrup, A. Gramfort, S. Chevallier, V. Jayaram, C. Jeunet, S. Bakas, S. Ludwig, K. Barmpas, et al. 2021 BEETL competition: Advancing transfer learning for subject independence & heterogeneous EEG data sets. In *NeurIPS 2021 Competitions and Demonstrations Track*, pages 205–219. PMLR, 2022.

[122] F. Wilcoxon. Individual comparisons by ranking methods. In *Breakthroughs in Statistics: Methodology and Distribution*, pages 196–202. Springer, 1992.

[123] Z. Wu and D. Yao. The influence of cognitive tasks on different frequencies
steady-state visual evoked potentials. *Brain topography*, 20:97–104, 2007.

[124] F. Yger, M. Berar, and F. Lotte. Riemannian approaches in brain-computer interfaces: a review. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 25(10):1753–1762, 2016.

[125] W. Yi, S. Qiu, K. Wang, H. Qi, L. Zhang, P. Zhou, F. He, and D. Ming. Evaluation of EEG oscillatory patterns and cognitive process during simple and compound limb motor imagery. *PloS one*, 9(12):e114853, 2014.

[126] Y. Zhang, G. Zhou, J. Jin, X. Wang, and A. Cichocki. Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis. *International journal of neural systems*, 24(04): 1450013, 2014.

[127] Z. Zhang, S. hua Zhong, and Y. Liu. TorchEEG, 2024. URL https://torcheeg.readthedocs.io.

[128] B. Zhou, X. Wu, Z. Lv, L. Zhang, and X. Guo. A fully automated trial selection method for optimization of motor imagery based brain-computer interface. *PloS one*, 11(9):e0162657, 2016.

## Appendix A. Detailed pipelines evaluation

This section provides details regarding the automatic parametrization and the evaluation conducted in the benchmark.

### Appendix A.1. Specialized grid search

The parametrization of evaluated pipelines should be generic to ensure a fair evaluation and automatic to avoid information leak-

age. A dictionary structure containing the parameter to search for each element of the machine learning pipeline is implemented to ensure this process when an evaluation is launched. As shown below, the `param_grid` structure, with names matching the `pipeline` structure, could be passed to the function `evaluation.process()`.

```python
pipelines = {}
pipelines["GridSearchEN"] = Pipeline(
    steps=[
        ("Covariances", Covariances("cov")),
        ("Tangent_Space",
         TangentSpace(metric="riemann")),
        (
            "LogistReg",
            LogisticRegression(
                penalty="elasticnet",
                l1_ratio=0.70,
                intercept scaling=1000.0,
                solver="saga",
                max_iter=1000,
            ),
        ),
    ]
)

param_grid = {}
pipelines["GridSearchEN"] = {
    "LogistReg_11_ratio":
        [0.15, 0.30, 0.45, 0.60, 0.75],
}

evaluation = WithinSessionEvaluation(
    paradigm=paradigm,
    datasets=dataset,
    overwrite=True,
    random_state=42,
    HDF5_path=path,
    n_jobs=-1,
)

result = evaluation.process(pipelines,
    param_grid)
```

### Appendix A.2. Other evaluation types

The proposed benchmark focus on within-session evaluation, but it is possible to conduct
steady-state visual evoked potentials. *Brain topography*, 20:97–104, 2007.

[124] F. Yger, M. Berar, and F. Lotte. Riemannian approaches in brain-computer interfaces: a review. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 25(10):1753–1762, 2016.

[125] W. Yi, S. Qiu, K. Wang, H. Qi, L. Zhang, P. Zhou, F. He, and D. Ming. Evaluation of EEG oscillatory patterns and cognitive process during simple and compound limb motor imagery. *PloS one*, 9(12):e114853, 2014.

[126] Y. Zhang, G. Zhou, J. Jin, X. Wang, and A. Cichocki. Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis. *International journal of neural systems*, 24(04): 1450013, 2014.

[127] Z. Zhang, S. hua Zhong, and Y. Liu. TorchEEG, 2024. URL https://torcheeg.readthedocs.io.

[128] B. Zhou, X. Wu, Z. Lv, L. Zhang, and X. Guo. A fully automated trial selection method for optimization of motor imagery based brain-computer interface. *PloS one*, 11(9):e0162657, 2016.

## Appendix A. Detailed pipelines evaluation

This section provides details regarding the automatic parametrization and the evaluation conducted in the benchmark.

### Appendix A.1. Specialized grid search

The parametrization of evaluated pipelines should be generic to ensure a fair evaluation and automatic to avoid information leak-

age. A dictionary structure containing the parameter to search for each element of the machine learning pipeline is implemented to ensure this process when an evaluation is launched. As shown below, the `param_grid` structure, with names matching the `pipeline` structure, could be passed to the function `evaluation.process()`.

```python
pipelines = {}
pipelines["GridSearchEN"] = Pipeline(
    steps=[
        ("Covariances", Covariances("cov")),
        ("Tangent_Space",
         TangentSpace(metric="riemann")),
        (
            "LogistReg",
            LogisticRegression(
                penalty="elasticnet",
                l1_ratio=0.70,
                intercept scaling=1000.0,
                solver="saga",
                max_iter=1000,
            ),
        ),
    ]
)

param_grid = {}
pipelines["GridSearchEN"] = {
    "LogistReg_11_ratio":
        [0.15, 0.30, 0.45, 0.60, 0.75],
}

evaluation = WithinSessionEvaluation(
    paradigm=paradigm,
    datasets=dataset,
    overwrite=True,
    random_state=42,
    HDF5_path=path,
    n_jobs=-1,
)

result = evaluation.process(pipelines,
    param_grid)
```

### Appendix A.2. Other evaluation types

The proposed benchmark focus on within-session evaluation, but it is possible to conduct
<table><caption>Table A1: Parameter used in Grid Search. For the ACM+TS+SVM pipeline, we reduce the hyperparameter search due to computational constraint (both order and lag to [1–5] for datasets with more than 60 electrodes - Cho2017, Lee2019-MI, PhysionetMI and Weibo2014. While we select parameters to [1 – 3] for datasets with more than 100 electrodes - Schirrmeister2017 and GrosseWentrup2009)</caption><thead><tr><th>Pipeline</th><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td rowspan="3">CSP + SVMGrid</td><td>csp_nfilter</td><td>[2 - 8]</td></tr><tr><td>svc_C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>svc_kernel</td><td>["rbf", "linear"]</td></tr><tr><td>EnGrid</td><td>logisticregression_l1 ratio</td><td>[0.20, 0.30, 0.45, 0.65, 0.75]</td></tr><tr><td rowspan="2">LogVarGrid</td><td>svc_C</td><td>[0.01, 0.05, 0.1, 0.5, 1, 5, 10, 50, 100]</td></tr><tr><td>svc_C</td><td>[0.5, 1, 1.5]</td></tr><tr><td rowspan="2">Tangent Space SVM (SVM) Grid</td><td>svc_C</td><td>["rbf", "linear"]</td></tr><tr><td>svc_kernel</td><td></td></tr><tr><td rowspan="4">ACM + TANG + Support Vector Machine (SVM) Grid</td><td>augmenteddataset_order</td><td>[1 - 10]</td></tr><tr><td>augmenteddataset_lag</td><td>[1 - 10]</td></tr><tr><td>svc_C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>svc_kernel</td><td>["rbf", "linear"]</td></tr></tbody></table>

<figure><img src="image_11.png" /><figcaption>Figure A1: Cross-session evaluation</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Figure A2: Cross-subject evaluation</figcaption></figure>

other evaluation types like cross-session or cross-subject. The structure of the cross-session evaluation is shown in Figure A1 and Figure A2 shows the structure of the cross-subject one.

## Appendix B. Machine learning algorithms

This section describes the machine learning pipelines and the choices of implementation.

### Appendix B.1. CSP-baseline pipelines

Mathematically, the CSP algorithm seeks to find spatial filters by solving a generalized eigenvalue problem. Let $X_1$ and $X_2$ be the data matrices of the band-pass filtered EEG signals for two conditions, each with dimensions of (time × channel). The CSP algorithm constructs discriminative and common activity matrices, $S_d$ and $S_c$, respectively. These matrices are defined as follows:

$$S_d = \Sigma_1 - \Sigma_2 \quad \text{and} \quad S_c = \Sigma_1 + \Sigma_2, \quad (B.1)$$
<table><caption>Table A1: Parameter used in Grid Search. For the ACM+TS+SVM pipeline, we reduce the hyperparameter search due to computational constraint (both order and lag to [1–5] for datasets with more than 60 electrodes - Cho2017, Lee2019-MI, PhysionetMI and Weibo2014. While we select parameters to [1–3] for datasets with more than 100 electrodes - Schirrmeister2017 and GrosseWentrup2009)</caption><thead><tr><th>Pipeline</th><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td rowspan="3">CSP + SVMGrid</td><td>csp_nfilter</td><td>[2 - 8]</td></tr><tr><td>svc_C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>svc_kernel</td><td>["rbf", "linear"]</td></tr><tr><td>EnGrid</td><td>logisticregression_l1 ratio</td><td>[0.20, 0.30, 0.45, 0.65, 0.75]</td></tr><tr><td rowspan="2">LogVarGrid</td><td>svc_C</td><td>[0.01, 0.05, 0.1, 0.5, 1, 5, 10, 50, 100]</td></tr><tr><td>svc_C</td><td>[0.5, 1, 1.5]</td></tr><tr><td rowspan="2">Tangent Space SVM (SVM) Grid</td><td>svc_C</td><td>["rbf", "linear"]</td></tr><tr><td>svc_kernel</td><td></td></tr><tr><td rowspan="4">ACM + TANG + Support Vector Machine (SVM) Grid</td><td>augmenteddataset_order</td><td>[1 - 10]</td></tr><tr><td>augmenteddataset_lag</td><td>[1 - 10]</td></tr><tr><td>svc_C</td><td>[0.5, 1, 1.5]</td></tr><tr><td>svc_kernel</td><td>["rbf", "linear"]</td></tr></tbody></table>

<figure><img src="image_10.png" /><figcaption>Figure A1: Cross-session evaluation</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure A2: Cross-subject evaluation</figcaption></figure>

other evaluation types like cross-session or cross-subject. The structure of the cross-session evaluation is shown in Figure A1 and Figure A2 shows the structure of the cross-subject one.

## Appendix B. Machine learning algorithms

This section describes the machine learning pipelines and the choices of implementation.

### Appendix B.1. CSP-baseline pipelines

Mathematically, the CSP algorithm seeks to find spatial filters by solving a generalized eigenvalue problem. Let $X_1$ and $X_2$ be the data matrices of the band-pass filtered EEG signals for two conditions, each with dimensions of (time × channel). The CSP algorithm constructs discriminative and common activity matrices, $S_d$ and $S_c$, respectively. These matrices are defined as follows:

$$S_d = \Sigma_1 - \Sigma_2 \quad \text{and} \quad S_c = \Sigma_1 + \Sigma_2, \quad (B.1)$$
where $\Sigma_1 = X_1^T X_1$ and $\Sigma_2 = X_2^T X_2$ are the estimates of the condition covariance matrices. Then, the objective of CSP is to find spatial filters $v_j \in \mathbb{R}^C$ that maximize the EEG signal bandpower variance between examples from different conditions while simultaneously minimizing its variance between examples from the same condition. This translates into:

$$ \arg\max_V \frac{V^T S_d V}{V^T S_c V} \qquad (B.2) $$

which is optimized by solving the following generalised eigenvalue problem [92]:

$$ S_d v = \lambda S_c v, \qquad (B.3) $$

and selecting the filters $v$ that yield the largest eigenvalues $\lambda$.

For classification purposes, CSP utilizes log-variance features extracted from the filtered signals projected onto the CSP filters. Typically, a small number of patterns (2 to 6) are selected based on the corresponding eigenvalues. The patterns, denoted as $a_j$, provide insights into the specific information captured by the corresponding filters $v_j$. Each filter $v_j$ extracts the activity spanned by pattern, $a_j$ while canceling out other activities spanned by different patterns. This allows for discrimination between different mental states based on the log-variance features. Linear classifiers, such as linear discriminant analysis, are commonly used due to the approximately Gaussian distribution of the log-variance features. In MOABB, we implemented two different decision head, the SVM and Linear Discriminant Analysis (LDA), with a GRIDSEARCH in some parameters [11].

## Appendix B.2. CCA-based pipelines

Based on the work of Hotelling, the CCA aims at finding a canonical space where

the correlation of two sets of variables is maximized. Considering the total covariance matrix $C$ of two sets of variables $x$ and $y$, the within-set covariances matrices are respectively denoted $C_{xx}$ and $C_{yy}$ and the between-sets covariance matrices are $C_{xy} = C_{yx}^\top$. The canonical correlation is defined as:

$$ \rho = \max_{w_x, w_y} \frac{w_x^\top C_{xy} w_y}{\sqrt{w_x^\top C_{xx} w_x w_y^\top C_{yy} w_y}} \qquad (B.4) $$

with $w_x$ and $w_y$ the projection vectors that maximizes canonical correlation. The solution is provided by a generalized eigenproblem formulated as $C_{xy}C_{y}^{-1}yC_{yx}w_x = \lambda^2 C_{xx}w_x$. As $C_{xx}$ and $C_{yy}$ are symmetric positive definite matrices, it is possible to rewrite the problem as symmetric standard eigenproblem $Ax = \lambda x$ that could be solved with any linear algebra library.

In SSVEP, the CCA is applied on a set of $x$ EEG trials and a set $y$ of reference sinusoid signals as described in [72]. The frequency of the sine and cosine reference signals should match the stimulus frequency and its harmonics. The obtained $w_x$ could be seen as filter acting on EEG signal to enhance the signal components synchronized with the visual stimulus.

Indeed, the choice reference signals is crucial to obtain robust results and could be complex to parameterize. The multiset canonical correlation analysis (MsetCCA) generalizes the CCA for multiple references [126], with the objective to learn the optimal reference signals without constraint on the sine/cosine shape of the reference set.

The goal of filters learned from CCA is to enhance the EEG activity generated by cortical stimulus response. The cortical signal component unrelated to the task could be filtered out as well, as introduced in task-related component analysis (TRCA) [85] for
where $\Sigma_1 = X_1^T X_1$ and $\Sigma_2 = X_2^T X_2$ are the estimates of the condition covariance matrices. Then, the objective of CSP is to find spatial filters $v_j \in \mathbb{R}^C$ that maximize the EEG signal bandpower variance between examples from different conditions while simultaneously minimizing its variance between examples from the same condition. This translates into:

$$ \arg\max_v \frac{V^T S_d V}{V^T S_c V} \qquad (B.2) $$

which is optimized by solving the following generalised eigenvalue problem [92]:

$$ S_d v = \lambda S_c v, \qquad (B.3) $$

and selecting the filters $v$ that yield the largest eigenvalues $\lambda$.

For classification purposes, CSP utilizes log-variance features extracted from the filtered signals projected onto the CSP filters. Typically, a small number of patterns (2 to 6) are selected based on the corresponding eigenvalues. The patterns, denoted as $a_j$, provide insights into the specific information captured by the corresponding filters $v_j$. Each filter $v_j$ extracts the activity spanned by pattern, $a_j$ while canceling out other activities spanned by different patterns. This allows for discrimination between different mental states based on the log-variance features. Linear classifiers, such as linear discriminant analysis, are commonly used due to the approximately Gaussian distribution of the log-variance features. In MOABB, we implemented two different decision head, the SVM and Linear Discriminant Analysis (LDA), with a GRIDSEARCH in some parameters [11].

## Appendix B.2. CCA-based pipelines

Based on the work of Hotelling, the CCA aims at finding a canonical space where

the correlation of two sets of variables is maximized. Considering the total covariance matrix $C$ of two sets of variables $x$ and $y$, the within-set covariances matrices are respectively denoted $C_{xx}$ and $C_{yy}$ and the between-sets covariance matrices are $C_{xy} = C_{yx}^\top$. The canonical correlation is defined as:

$$ \rho = \max_{w_x, w_y} \frac{w_x^\top C_{xy} w_y}{\sqrt{w_x^\top C_{xx} w_x w_y^\top C_{yy} w_y}} \qquad (B.4) $$

with $w_x$ and $w_y$ the projection vectors that maximizes canonical correlation. The solution is provided by a generalized eigenproblem formulated as $C_{xy}C_{y}^{-1}yC_{yx}w_x = \lambda^2 C_{xx}w_x$. As $C_{xx}$ and $C_{yy}$ are symmetric positive definite matrices, it is possible to rewrite the problem as symmetric standard eigenproblem $Ax = \lambda x$ that could be solved with any linear algebra library.

In SSVEP, the CCA is applied on a set of $x$ EEG trials and a set $y$ of reference sinusoid signals as described in [72]. The frequency of the sine and cosine reference signals should match the stimulus frequency and its harmonics. The obtained $w_x$ could be seen as filter acting on EEG signal to enhance the signal components synchronized with the visual stimulus.

Indeed, the choice reference signals is crucial to obtain robust results and could be complex to parameterize. The multiset canonical correlation analysis (MsetCCA) generalizes the CCA for multiple references [126], with the objective to learn the optimal reference signals without constraint on the sine/cosine shape of the reference set.

The goal of filters learned from CCA is to enhance the EEG activity generated by cortical stimulus response. The cortical signal component unrelated to the task could be filtered out as well, as introduced in task-related component analysis (TRCA) [85] for
SSVEP. In this case, an optimization process is designed to find the $w_x$ that maximized the inter-trial covariance of EEG signals.

### Appendix B.3. Riemannian pipelines

As described in subsection 4.2, the Riemannian pipelines could be directly defined on the manifold. This is the case of the Minimum Distance to Mean (MDM) classifier, that computes the average of each class with Eq. (2) and use the distance defined in Eq. (1) between an unseen sample and each class center to make a prediction.

To mitigate the effect of increased dimensions, i.e., for EEG recorded with a high number of electrodes, a geodesic filtering could be applied before MDM classification, namely the FgMDM. This filtering step implements a linear discriminant analysis in the tangent space to project all trials on a single hyperplane and the projection back to the manifold.

The second option is to project the samples in the tangent space, vectorize samples that are symmetric matrices and train a classifier on those vectorized data. Algorithms such as logistic regression (LR), logistic regression under $\ell_1$ and $\ell_2$ norm penalties (ElasticNet, EL) or support vector machine (SVM) have been described in the literature.

When considering MI, Riemannian classifiers operate directly on covariance matrices estimated from the EEG signals. The transient information of ERP require to first estimate a average ERP or, better, an average ERP filtered XDAWN. For SSVEP, the relevant information is spectral and the covariances need to be estimated on the bandpass filtered signal for each stimulation frequency. All those preprocessing are described in great detail in [124, 29].

## Appendix C. Experimental results

This section provides a global overview of the pipeline scores for the different paradigms with raincloud plots. The pipelines are grouped by categories: Deep learning, Riemannian, and Raw. The small points correspond to the scores obtained on the individual sessions, with an exception for the last row, i.e. *Average*, where they correspond to the average score over one dataset. The curves above indicate a density estimation of these individual scores. The diamond shapes connected by vertical lines indicate the within-dataset average. Finally, the boxes and horizontal black bars indicate the quartiles.

## Appendix D. Detailed results for each pipeline

This section provides the tables indicating the ROC-AUC for binary classification problems and the accuracy for multi-class tasks. The main objective is to provide a reference benchmark that could be easily reproduced to verify the results or used as-is to compare with new pipelines. It is thus possible to save energy and resources, avoiding reproducing already existing validated results with a simple copy-paste. To this end, the tables provide here the average result of pipelines across all subjects for a given dataset using within-session evaluation. We did not provide subject-by-subject results for space reasons and we know that this benchmark is meant to evolve with new pipelines, datasets, and evaluation methods. We mirror those tables on a website that will allow further additions and available up-to-date results.
SSVEP. In this case, an optimization process is designed to find the $w_x$ that maximized the inter-trial covariance of EEG signals.

### Appendix B.3. Riemannian pipelines

As described in subsection 4.2, the Riemannian pipelines could be directly defined on the manifold. This is the case of the Minimum Distance to Mean (MDM) classifier, that computes the average of each class with Eq. (2) and use the distance defined in Eq. (1) between an unseen sample and each class center to make a prediction.

To mitigate the effect of increased dimensions, i.e., for EEG recorded with a high number of electrodes, a geodesic filtering could be applied before MDM classification, namely the FgMDM. This filtering step implements a linear discriminant analysis in the tangent space to project all trials on a single hyperplane and the projection back to the manifold.

The second option is to project the samples in the tangent space, vectorize samples that are symmetric matrices and train a classifier on those vectorized data. Algorithms such as logistic regression (LR), logistic regression under $\ell_1$ and $\ell_2$ norm penalties (ElasticNet, EL) or support vector machine (SVM) have been described in the literature.

When considering MI, Riemannian classifiers operate directly on covariance matrices estimated from the EEG signals. The transient information of ERP require to first estimate a average ERP or, better, an average ERP filtered XDAWN. For SSVEP, the relevant information is spectral and the covariances need to be estimated on the bandpass filtered signal for each stimulation frequency. All those preprocessing are described in great detail in [124, 29].

## Appendix C. Experimental results

This section provides a global overview of the pipeline scores for the different paradigms with raincloud plots. The pipelines are grouped by categories: Deep learning, Riemannian, and Raw. The small points correspond to the scores obtained on the individual sessions, with an exception for the last row, i.e. *Average*, where they correspond to the average score over one dataset. The curves above indicate a density estimation of these individual scores. The diamond shapes connected by vertical lines indicate the within-dataset average. Finally, the boxes and horizontal black bars indicate the quartiles.

## Appendix D. Detailed results for each pipeline

This section provides the tables indicating the ROC-AUC for binary classification problems and the accuracy for multi-class tasks. The main objective is to provide a reference benchmark that could be easily reproduced to verify the results or used as-is to compare with new pipelines. It is thus possible to save energy and resources, avoiding reproducing already existing validated results with a simple copy-paste. To this end, the tables provide here the average result of pipelines across all subjects for a given dataset using within-session evaluation. We did not provide subject-by-subject results for space reasons and we know that this benchmark is meant to evolve with new pipelines, datasets, and evaluation methods. We mirror those tables on a website that will allow further additions and available up-to-date results.
<table><caption>Table D1: Summary of performances via average on all the motor imagery datasets, for classification using all the labels. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>AlexandreMotorImagery</th><th>BNCI2014-001</th><th>PhysionetMotorImagery</th><th>Schirrmeister2017</th><th>Weibo2014</th><th>Zhou2016</th><th>Average</th></tr></thead><tbody><tr><td>ACM+TS+SVM</td><td>69.37±15.07</td><td><b>77.82±12.23</b></td><td>55.44±14.87</td><td>82.50±10.20</td><td><b>63.89±11.01</b></td><td><b>85.25±4.06</b></td><td>72.38</td></tr><tr><td>CSP+LDA</td><td>61.04±17.22</td><td>65.99±15.47</td><td>47.73±14.35</td><td>72.97±10.42</td><td>39.45±11.87</td><td>82.96±5.20</td><td>61.69</td></tr><tr><td>CSP+SVM</td><td>62.92±16.89</td><td>66.88±15.22</td><td>48.52±14.62</td><td>75.89±10.55</td><td>44.08±11.95</td><td>83.08±5.33</td><td>63.56</td></tr><tr><td>DLCSPauto+shLDA</td><td>60.63±17.91</td><td>66.31±15.36</td><td>46.85±14.65</td><td>72.82±10.44</td><td>38.84±11.97</td><td>82.06±5.57</td><td>61.25</td></tr><tr><td>DeepConvNet</td><td>37.71±4.56</td><td>35.29±8.26</td><td>27.68±3.91</td><td>56.78±18.11</td><td>24.17±9.80</td><td>55.69±5.61</td><td>39.55</td></tr><tr><td>EEGITNet</td><td>36.04±3.43</td><td>35.55±6.35</td><td>26.15±4.95</td><td>70.44±14.68</td><td>25.78±8.00</td><td>50.68±16.27</td><td>40.77</td></tr><tr><td>EEGNeX</td><td>37.71±9.64</td><td>45.62±15.29</td><td>26.69±5.64</td><td>67.56±14.15</td><td>30.22±11.02</td><td>56.42±11.29</td><td>44.03</td></tr><tr><td>EEGNet-8.2</td><td>43.96±8.62</td><td>60.46±20.20</td><td>29.04±7.03</td><td>76.99±13.05</td><td>35.35±14.05</td><td>83.34±3.58</td><td>54.86</td></tr><tr><td>EEGTCNet</td><td>34.17±1.86</td><td>41.65±13.73</td><td>25.79±3.85</td><td>71.11±11.96</td><td>17.95±3.88</td><td>37.19±2.57</td><td>37.98</td></tr><tr><td>FBCSP+SVM</td><td>65.00±17.56</td><td>66.53±12.05</td><td>45.49±12.54</td><td>75.94±8.59</td><td>45.21±10.05</td><td>81.99±4.65</td><td>63.36</td></tr><tr><td>FgMDM</td><td>65.63±15.63</td><td>70.14±15.13</td><td>55.04±14.17</td><td>82.97±10.08</td><td>56.94±9.26</td><td>83.07±4.96</td><td>68.97</td></tr><tr><td>MDM</td><td>60.62±13.69</td><td>61.60±14.20</td><td>42.96±12.98</td><td>52.03±10.11</td><td>33.41±8.67</td><td>76.05±7.10</td><td>54.45</td></tr><tr><td>ShallowConvNet</td><td>50.00±12.94</td><td>72.47±16.50</td><td>41.87±12.50</td><td>85.13±9.57</td><td>48.94±10.36</td><td>85.02±3.78</td><td>63.91</td></tr><tr><td>TS+EL</td><td><b>69.79±13.75</b></td><td>72.38±14.85</td><td><b>59.93±14.07</b></td><td><b>85.53±9.40</b></td><td>63.84±8.77</td><td>84.54±4.93</td><td><b>72.67</b></td></tr><tr><td>TS+LR</td><td>69.17±14.79</td><td>71.97±15.46</td><td>58.55±14.06</td><td>84.60±9.28</td><td>62.76±8.39</td><td>84.88±4.63</td><td>71.99</td></tr><tr><td>TS+SVM</td><td>67.92±12.74</td><td>70.76±15.08</td><td>58.46±15.15</td><td>84.41±9.56</td><td>61.47±9.62</td><td>83.66±4.55</td><td>71.11</td></tr><tr><td>Average</td><td>55.73</td><td>61.34</td><td>43.51</td><td>74.85</td><td>43.27</td><td>74.74</td><td>58.91</td></tr></tbody></table>

<table><caption>Table D2: Summary of performances via average on all the P300 datasets, for classification using left vs. right motor imagery task. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>BNCI2014-001</th><th>BNCI2014-004</th><th>Cho2017</th><th>GrosseWentrup2009</th><th>Lee2019-MI</th><th>PhysionetMotorImagery</th><th>Schirrmeister2017</th><th>Shin2017A</th><th>Weibo2014</th><th>Zhou2016</th><th>Average</th></tr></thead><tbody><tr><td>ACM+TS+SVM</td><td><b>91.71±10.30</b></td><td><b>82.67±15.33</b></td><td>73.56±14.54</td><td>86.60±15.12</td><td>83.05±13.97</td><td>63.55±21.24</td><td>85.82±13.98</td><td>68.97±23.45</td><td>84.78±13.33</td><td>95.03±4.76</td><td>81.57</td></tr><tr><td>CSP+LDA</td><td>82.34±17.26</td><td>80.10±14.93</td><td>71.38±14.54</td><td>76.44±20.95</td><td>76.88±17.41</td><td>65.75±17.37</td><td>77.23±18.43</td><td><b>72.30±21.79</b></td><td>80.72±15.29</td><td>93.15±6.88</td><td>77.63</td></tr><tr><td>CSP+SVM</td><td>83.07±16.53</td><td>79.27±15.68</td><td>71.92±14.25</td><td>77.81±21.27</td><td>77.27±16.73</td><td>65.71±17.90</td><td>79.24±20.07</td><td>70.11±22.19</td><td>79.84±15.86</td><td>92.96±7.86</td><td>77.72</td></tr><tr><td>DLCSPauto+shLDA</td><td>82.75±16.69</td><td>79.87±15.11</td><td>71.16±14.53</td><td>76.40±20.83</td><td>76.69±17.23</td><td>65.07±17.68</td><td>77.02±18.48</td><td>70.34±23.30</td><td>80.16±15.23</td><td>92.56±7.21</td><td>77.2</td></tr><tr><td>DeepConvNet</td><td>82.07±15.52</td><td>72.36±18.53</td><td>71.67±12.91</td><td>82.38±15.39</td><td>70.65±15.76</td><td>59.57±16.77</td><td>81.23±17.39</td><td>56.03±19.18</td><td>73.64±15.78</td><td>94.42±6.21</td><td>74.4</td></tr><tr><td>EEGITNet</td><td>75.27±16.37</td><td>65.10±15.32</td><td>57.20±12.21</td><td>72.19±14.71</td><td>59.17±11.72</td><td>52.71±11.11</td><td>74.66±20.52</td><td>52.18±16.78</td><td>59.35±14.06</td><td>69.41±14.66</td><td>63.72</td></tr><tr><td>EEGNeX</td><td>66.28±13.22</td><td>66.53±17.10</td><td>53.28±10.60</td><td>57.00±7.52</td><td>55.12±10.05</td><td>51.20±10.63</td><td>68.58±19.37</td><td>49.02±17.58</td><td>57.97±15.65</td><td>61.56±14.60</td><td>58.65</td></tr><tr><td>EEGNet-8.2</td><td>77.15±19.33</td><td>69.50±19.50</td><td>66.79±16.34</td><td>83.02±18.08</td><td>65.67±16.43</td><td>59.55±15.95</td><td>80.20±18.13</td><td>57.99±17.28</td><td>66.46±21.78</td><td>94.84±2.83</td><td>72.12</td></tr><tr><td>EEGTCNet</td><td>67.46±20.81</td><td>69.70±19.55</td><td>58.34±12.63</td><td>68.45±16.27</td><td>55.68±12.75</td><td>55.90±12.74</td><td>75.62±22.33</td><td>51.26±16.77</td><td>63.16±18.32</td><td>82.24±9.40</td><td>64.78</td></tr><tr><td>FBCSP+SVM</td><td>84.44±16.00</td><td>80.39±16.05</td><td>67.91±15.63</td><td>79.65±18.63</td><td>75.07±16.97</td><td>58.45±13.93</td><td>81.44±17.89</td><td>65.63±21.64</td><td>76.81±18.88</td><td>92.64±5.01</td><td>76.24</td></tr><tr><td>FgMDM</td><td>86.53±12.14</td><td>79.28±15.25</td><td>72.90±12.70</td><td>87.02±13.20</td><td>81.34±13.93</td><td><b>68.46±19.06</b></td><td>86.71±13.79</td><td>70.86±23.36</td><td>78.41±14.85</td><td>92.54±6.67</td><td>80.41</td></tr><tr><td>LogVar+LDA</td><td>77.96±15.09</td><td>78.51±15.25</td><td>64.49±10.08</td><td>78.71±11.69</td><td>66.21±12.06</td><td>61.94±14.41</td><td>78.44±13.76</td><td>61.78±22.77</td><td>74.13±10.40</td><td>88.39±8.57</td><td>73.06</td></tr><tr><td>LogVar+SVM</td><td>75.86±16.45</td><td>78.30±15.18</td><td>65.46±11.71</td><td>81.73±12.40</td><td>73.83±13.85</td><td>62.35±16.87</td><td>79.42±13.66</td><td>61.38±22.68</td><td>74.85±11.33</td><td>88.47±8.50</td><td>74.17</td></tr><tr><td>MDM</td><td>81.69±14.94</td><td>77.66±15.78</td><td>63.39±13.69</td><td>64.29±8.04</td><td>70.23±13.87</td><td>54.76±16.79</td><td>61.53±16.41</td><td>62.99±21.25</td><td>58.80±16.13</td><td>90.70±7.11</td><td>68.6</td></tr><tr><td>ShallowConvNet</td><td>86.17±13.74</td><td>72.36±18.05</td><td>73.84±14.95</td><td>86.53±13.00</td><td>75.83±15.04</td><td>65.19±15.80</td><td>84.82±15.29</td><td>60.80±19.27</td><td>79.10±12.63</td><td><b>95.65±5.55</b></td><td>78.03</td></tr><tr><td>TRCSP+LDA</td><td>79.84±16.28</td><td>79.78±15.22</td><td>71.85±13.84</td><td>78.29±16.66</td><td>76.26±15.41</td><td>67.24±17.23</td><td>79.14±15.91</td><td>67.30±23.19</td><td>79.33±14.43</td><td>93.53±6.38</td><td>77.25</td></tr><tr><td>TS+EL</td><td>86.44±13.20</td><td>79.75±15.44</td><td><b>76.23±14.21</b></td><td><b>89.25±12.00</b></td><td><b>84.74±13.19</b></td><td>67.91±20.03</td><td><b>88.65±12.98</b></td><td>68.68±23.64</td><td><b>85.29±12.10</b></td><td>94.35±6.04</td><td><b>82.13</b></td></tr><tr><td>TS+LR</td><td>87.41±12.58</td><td>80.09±15.01</td><td>75.01±13.71</td><td>87.60±13.20</td><td>83.09±13.46</td><td>67.28±19.19</td><td>87.22±13.83</td><td>69.31±23.06</td><td>83.62±13.88</td><td>94.16±6.33</td><td>81.48</td></tr><tr><td>TS+SVM</td><td>86.48±13.58</td><td>79.41±15.26</td><td>74.62±14.19</td><td>88.08±13.58</td><td>83.57±14.08</td><td>68.18±19.92</td><td>87.64±13.48</td><td>68.45±24.25</td><td>83.72±14.28</td><td>93.37±6.30</td><td>81.35</td></tr><tr><td>Average</td><td>81.1</td><td>76.35</td><td>68.47</td><td>79.02</td><td>73.18</td><td>62.15</td><td>79.72</td><td>63.44</td><td>74.74</td><td>89.47</td><td>74.76</td></tr></tbody></table>
<table><caption>Table D1: Summary of performances via average on all the motor imagery datasets, for classification using all the labels. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>AlexandreMotorImagery</th><th>BNCI2014-001</th><th>PhysionetMotorImagery</th><th>Schirrmeister2017</th><th>Weibo2014</th><th>Zhou2016</th><th>Average</th></tr></thead><tbody><tr><td>ACM+TS+SVM</td><td>69.37±15.07</td><td><b>77.82±12.23</b></td><td>55.44±14.87</td><td>82.50±10.20</td><td><b>63.89±11.01</b></td><td><b>85.25±4.06</b></td><td>72.38</td></tr><tr><td>CSP+LDA</td><td>61.04±17.22</td><td>65.99±15.47</td><td>47.73±14.35</td><td>72.97±10.42</td><td>39.45±11.87</td><td>82.96±5.20</td><td>61.69</td></tr><tr><td>CSP+SVM</td><td>62.92±16.89</td><td>66.88±15.22</td><td>48.52±14.62</td><td>75.89±10.55</td><td>44.08±11.95</td><td>83.08±5.33</td><td>63.56</td></tr><tr><td>DLCSPauto+shLDA</td><td>60.63±17.91</td><td>66.31±15.36</td><td>46.85±14.65</td><td>72.82±10.44</td><td>38.84±11.97</td><td>82.06±5.57</td><td>61.25</td></tr><tr><td>DeepConvNet</td><td>37.71±4.56</td><td>35.29±8.26</td><td>27.68±3.91</td><td>56.78±18.11</td><td>24.17±9.80</td><td>55.69±5.61</td><td>39.55</td></tr><tr><td>EEGITNet</td><td>36.04±3.43</td><td>35.55±6.35</td><td>26.15±4.95</td><td>70.44±14.68</td><td>25.78±8.00</td><td>50.68±16.27</td><td>40.77</td></tr><tr><td>EEGNeX</td><td>37.71±9.64</td><td>45.62±15.29</td><td>26.69±5.64</td><td>67.56±14.15</td><td>30.22±11.02</td><td>56.42±11.29</td><td>44.03</td></tr><tr><td>EEGNet-8,2</td><td>43.96±8.62</td><td>60.46±20.20</td><td>29.04±7.03</td><td>76.99±13.05</td><td>35.35±14.05</td><td>83.34±3.58</td><td>54.86</td></tr><tr><td>EEGTCNet</td><td>34.17±1.86</td><td>41.65±13.73</td><td>25.79±3.85</td><td>71.11±11.96</td><td>17.95±3.88</td><td>37.19±2.57</td><td>37.98</td></tr><tr><td>FBCSP+SVM</td><td>65.00±17.56</td><td>66.53±12.05</td><td>45.49±12.54</td><td>75.94±8.59</td><td>45.21±10.05</td><td>81.99±4.65</td><td>63.36</td></tr><tr><td>FgMDM</td><td>65.63±15.63</td><td>70.14±15.13</td><td>55.04±14.17</td><td>82.97±10.08</td><td>56.94±9.26</td><td>83.07±4.96</td><td>68.97</td></tr><tr><td>MDM</td><td>60.62±13.69</td><td>61.60±14.20</td><td>42.96±12.98</td><td>52.03±10.11</td><td>33.41±8.67</td><td>76.05±7.10</td><td>54.45</td></tr><tr><td>ShallowConvNet</td><td>50.00±12.94</td><td>72.47±16.50</td><td>41.87±12.50</td><td>85.13±9.57</td><td>48.94±10.36</td><td>85.02±3.78</td><td>63.91</td></tr><tr><td>TS+EL</td><td><b>69.79±13.75</b></td><td>72.38±14.85</td><td><b>59.93±14.07</b></td><td><b>85.53±9.40</b></td><td>63.84±8.77</td><td>84.54±4.93</td><td><b>72.67</b></td></tr><tr><td>TS+LR</td><td>69.17±14.79</td><td>71.97±15.46</td><td>58.55±14.06</td><td>84.60±9.28</td><td>62.76±8.39</td><td>84.88±4.63</td><td>71.99</td></tr><tr><td>TS+SVM</td><td>67.92±12.74</td><td>70.76±15.08</td><td>58.46±15.15</td><td>84.41±9.56</td><td>61.47±9.62</td><td>83.66±4.55</td><td>71.11</td></tr><tr><td>Average</td><td>55.73</td><td>61.34</td><td>43.51</td><td>74.85</td><td>43.27</td><td>74.74</td><td>58.91</td></tr></tbody></table>

<table><caption>Table D2: Summary of performances via average on all the P300 datasets, for classification using left vs. right motor imagery task. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>BNCI2014-001</th><th>BNCI2014-004</th><th>Cho2017</th><th>GrosseWentrup2009</th><th>Lee2019-MI</th><th>PhysionetMotorImagery</th><th>Schirrmeister2017</th><th>Shin2017A</th><th>Weibo2014</th><th>Zhou2016</th><th>Average</th></tr></thead><tbody><tr><td>ACM+TS+SVM</td><td><b>91.71±10.30</b></td><td><b>82.67±15.33</b></td><td>73.56±14.54</td><td>86.60±15.12</td><td>83.05±13.97</td><td>63.55±21.24</td><td>85.82±13.98</td><td>68.97±23.45</td><td>84.78±13.33</td><td>95.03±4.76</td><td>81.57</td></tr><tr><td>CSP+LDA</td><td>82.34±17.26</td><td>80.10±14.93</td><td>71.38±14.54</td><td>76.44±20.95</td><td>76.88±17.41</td><td>65.75±17.37</td><td>77.23±18.43</td><td><b>72.30±21.79</b></td><td>80.72±15.29</td><td>93.15±6.88</td><td>77.63</td></tr><tr><td>CSP+SVM</td><td>83.07±16.53</td><td>79.27±15.68</td><td>71.92±14.25</td><td>77.81±21.27</td><td>77.27±16.73</td><td>65.71±17.90</td><td>79.24±20.07</td><td>70.11±22.19</td><td>79.84±15.86</td><td>92.96±7.86</td><td>77.72</td></tr><tr><td>DLCSPauto+shLDA</td><td>82.75±16.69</td><td>79.87±15.11</td><td>71.16±14.53</td><td>76.40±20.83</td><td>76.69±17.23</td><td>65.07±17.68</td><td>77.02±18.48</td><td>70.34±23.30</td><td>80.16±15.23</td><td>92.56±7.21</td><td>77.2</td></tr><tr><td>DeepConvNet</td><td>82.07±15.52</td><td>72.36±18.53</td><td>71.67±12.91</td><td>82.38±15.39</td><td>70.65±15.76</td><td>59.57±16.77</td><td>81.23±17.39</td><td>56.03±19.18</td><td>73.64±15.78</td><td>94.42±6.21</td><td>74.4</td></tr><tr><td>EEGITNet</td><td>75.27±16.37</td><td>65.10±15.32</td><td>57.20±12.21</td><td>72.19±14.71</td><td>59.17±11.72</td><td>52.71±11.11</td><td>74.66±20.52</td><td>52.18±16.78</td><td>59.35±14.06</td><td>69.41±14.66</td><td>63.72</td></tr><tr><td>EEGNeX</td><td>66.28±13.22</td><td>66.53±17.10</td><td>53.28±10.60</td><td>57.00±7.52</td><td>55.12±10.05</td><td>51.20±10.63</td><td>68.58±19.37</td><td>49.02±17.58</td><td>57.97±15.65</td><td>61.56±14.60</td><td>58.65</td></tr><tr><td>EEGNet-8,2</td><td>77.15±19.33</td><td>69.50±19.50</td><td>66.79±16.34</td><td>83.02±18.08</td><td>65.67±16.43</td><td>59.55±15.95</td><td>80.20±18.13</td><td>57.99±17.28</td><td>66.46±21.78</td><td>94.84±2.83</td><td>72.12</td></tr><tr><td>EEGTCNet</td><td>67.46±20.81</td><td>69.70±19.55</td><td>58.34±12.63</td><td>68.45±16.27</td><td>55.68±12.75</td><td>55.90±12.74</td><td>75.62±22.33</td><td>51.26±16.77</td><td>63.16±18.32</td><td>82.24±9.40</td><td>64.78</td></tr><tr><td>FBCSP+SVM</td><td>84.44±16.00</td><td>80.39±16.05</td><td>67.91±15.63</td><td>79.65±18.63</td><td>75.07±16.97</td><td>58.45±13.93</td><td>81.44±17.89</td><td>65.63±21.64</td><td>76.81±18.88</td><td>92.64±5.01</td><td>76.24</td></tr><tr><td>FgMDM</td><td>86.53±12.14</td><td>79.28±15.25</td><td>72.90±12.70</td><td>87.02±13.20</td><td>81.34±13.93</td><td><b>68.46±19.06</b></td><td>86.71±13.79</td><td>70.86±23.36</td><td>78.41±14.85</td><td>92.54±6.67</td><td>80.41</td></tr><tr><td>LogVar+LDA</td><td>77.96±15.09</td><td>78.51±15.25</td><td>64.49±10.08</td><td>78.71±11.69</td><td>66.21±12.06</td><td>61.94±14.41</td><td>78.44±13.76</td><td>61.78±22.77</td><td>74.13±10.40</td><td>88.39±8.57</td><td>73.06</td></tr><tr><td>LogVar+SVM</td><td>75.86±16.45</td><td>78.30±15.18</td><td>65.46±11.71</td><td>81.73±12.40</td><td>73.83±13.85</td><td>62.35±16.87</td><td>79.42±13.66</td><td>61.38±22.68</td><td>74.85±11.33</td><td>88.47±8.50</td><td>74.17</td></tr><tr><td>MDM</td><td>81.69±14.94</td><td>77.66±15.78</td><td>63.39±13.69</td><td>64.29±8.04</td><td>70.23±13.87</td><td>54.76±16.79</td><td>61.53±16.41</td><td>62.99±21.25</td><td>58.80±16.13</td><td>90.70±7.11</td><td>68.6</td></tr><tr><td>ShallowConvNet</td><td>86.17±13.74</td><td>72.36±18.05</td><td>73.84±14.95</td><td>86.53±13.00</td><td>75.83±15.04</td><td>65.19±15.80</td><td>84.82±15.29</td><td>60.80±19.27</td><td>79.10±12.63</td><td><b>95.65±5.55</b></td><td>78.03</td></tr><tr><td>TRCSP+LDA</td><td>79.84±16.28</td><td>79.78±15.22</td><td>71.85±13.84</td><td>78.29±16.66</td><td>76.26±15.41</td><td>67.24±17.23</td><td>79.14±15.91</td><td>67.30±23.19</td><td>79.33±14.43</td><td>93.53±6.38</td><td>77.25</td></tr><tr><td>TS+EL</td><td>86.44±13.20</td><td>79.75±15.44</td><td><b>76.23±14.21</b></td><td><b>89.25±12.00</b></td><td><b>84.74±13.19</b></td><td>67.91±20.03</td><td><b>88.65±12.98</b></td><td>68.68±23.64</td><td><b>85.29±12.10</b></td><td>94.35±6.04</td><td><b>82.13</b></td></tr><tr><td>TS+LR</td><td>87.41±12.58</td><td>80.09±15.01</td><td>75.01±13.71</td><td>87.60±13.20</td><td>83.09±13.46</td><td>67.28±19.19</td><td>87.22±13.83</td><td>69.31±23.06</td><td>83.62±13.88</td><td>94.16±6.33</td><td>81.48</td></tr><tr><td>TS+SVM</td><td>86.48±13.58</td><td>79.41±15.26</td><td>74.62±14.19</td><td>88.08±13.58</td><td>83.57±14.08</td><td>68.18±19.92</td><td>87.64±13.48</td><td>68.45±24.25</td><td>83.72±14.28</td><td>93.37±6.30</td><td>81.35</td></tr><tr><td>Average</td><td>81.1</td><td>76.35</td><td>68.47</td><td>79.02</td><td>73.18</td><td>62.15</td><td>79.72</td><td>63.44</td><td>74.74</td><td>89.47</td><td>74.76</td></tr></tbody></table>
<table><caption>Table D3: Summary of performances via average on all the P300 datasets, for classification using right hand vs. feet tasks motor imagery task. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>AlexandreMotorImagery</th><th>BNCI2014-001</th><th>BNCI2014-002</th><th>BNCI2015-001</th><th>BNCI2015-004</th><th>PhysionetMotorImagery</th><th>Schirrmeister2017</th><th>Weibo2014</th><th>Zhou2016</th><th>Average</th></tr></thead><tbody><tr><td>ACM+TS+SVM</td><td><b>86.56±12.26</b></td><td><b>97.32±3.35</b></td><td><b>88.60±10.71</b></td><td><b>93.01±8.09</b></td><td><b>62.60±14.62</b></td><td>93.33±8.46</td><td>98.67±3.06</td><td><b>93.25±4.12</b></td><td><b>97.18±3.00</b></td><td><b>90.06</b></td></tr><tr><td>CSP+LDA</td><td>77.19±17.58</td><td>91.52±10.39</td><td>80.98±14.79</td><td>88.52±10.75</td><td>54.02±11.33</td><td>86.41±13.96</td><td>97.02±5.17</td><td>88.59±6.36</td><td>95.20±3.17</td><td>84.38</td></tr><tr><td>CSP+SVM</td><td>78.59±20.14</td><td>91.04±10.35</td><td>81.21±15.30</td><td>89.19±10.08</td><td>52.08±11.05</td><td>88.04±12.57</td><td>97.50±4.90</td><td>88.64±5.90</td><td>94.95±3.53</td><td>84.58</td></tr><tr><td>DLCSPauto+shLDA</td><td>77.03±18.93</td><td>91.54±10.37</td><td>80.45±15.52</td><td>88.87±10.42</td><td>53.02±10.75</td><td>86.81±13.34</td><td>96.95±5.22</td><td>88.48±6.53</td><td>94.43±3.41</td><td>84.18</td></tr><tr><td>DeepConvNet</td><td>61.88±19.05</td><td>88.27±12.19</td><td>87.56±11.25</td><td>88.12±13.19</td><td>57.08±12.29</td><td>71.49±15.88</td><td>95.90±7.14</td><td>79.29±12.63</td><td>95.92±3.66</td><td>80.61</td></tr><tr><td>EEGITNet</td><td>47.50±9.46</td><td>75.98±13.09</td><td>70.90±17.50</td><td>71.95±16.76</td><td>51.41±6.40</td><td>54.69±11.97</td><td>96.04±8.62</td><td>62.54±12.32</td><td>80.40±17.12</td><td>67.93</td></tr><tr><td>EEGNeX</td><td>52.34±14.81</td><td>64.36±13.49</td><td>69.95±20.12</td><td>72.34±19.83</td><td>53.02±9.69</td><td>51.77±12.06</td><td>89.49±16.91</td><td>60.18±11.70</td><td>64.80±16.64</td><td>64.25</td></tr><tr><td>EEGNet-8,2</td><td>64.22±16.01</td><td>88.55±14.92</td><td>83.93±16.31</td><td>90.43±11.75</td><td>54.20±8.20</td><td>73.78±15.59</td><td>96.50±8.07</td><td>78.15±14.46</td><td>94.58±3.21</td><td>80.48</td></tr><tr><td>EEGTCNet</td><td>61.09±22.06</td><td>75.21±18.53</td><td>73.92±19.02</td><td>77.21±18.55</td><td>51.22±5.84</td><td>57.03±13.25</td><td>97.15±7.70</td><td>62.37±12.42</td><td>85.46±16.42</td><td>71.19</td></tr><tr><td>FBCSP+SVM</td><td>80.78±18.86</td><td>93.55±6.29</td><td>80.39±16.83</td><td>91.57±7.66</td><td>52.51±9.82</td><td>83.97±12.43</td><td>97.40±4.18</td><td>88.27±7.91</td><td>94.63±3.94</td><td>84.78</td></tr><tr><td>FgMDM</td><td>79.84±17.80</td><td>93.52±8.18</td><td>84.77±11.26</td><td>90.18±9.77</td><td>58.31±12.63</td><td>89.67±10.65</td><td>98.48±3.45</td><td>88.56±4.63</td><td>96.04±2.67</td><td>86.6</td></tr><tr><td>MDM</td><td>74.22±21.19</td><td>89.13±10.38</td><td>77.48±14.11</td><td>86.20±12.99</td><td>48.45±9.62</td><td>81.78±11.64</td><td>84.67±13.13</td><td>65.18±9.75</td><td>92.21±4.31</td><td>77.7</td></tr><tr><td>ShallowConvNet</td><td>64.22±18.33</td><td>93.00±8.05</td><td>87.60±12.05</td><td>91.41±10.88</td><td>57.23±12.36</td><td>74.75±14.98</td><td>98.06±4.35</td><td>88.70±5.60</td><td>97.06±1.86</td><td>83.56</td></tr><tr><td>TS+EL</td><td>81.41±21.36</td><td>94.45±6.74</td><td>85.98±11.38</td><td>91.19±8.49</td><td>58.70±13.37</td><td>94.09±7.17</td><td>98.56±3.01</td><td>92.32±3.98</td><td>96.59±2.82</td><td>88.14</td></tr><tr><td>TS+LR</td><td>83.75±17.47</td><td>94.45±7.06</td><td>85.86±11.01</td><td>91.09±8.71</td><td>61.01±14.22</td><td>93.15±7.40</td><td>98.60±3.08</td><td>91.53±4.53</td><td>96.76±2.58</td><td>88.47</td></tr><tr><td>TS+SVM</td><td>82.66±18.16</td><td>94.01±7.60</td><td>86.19±11.50</td><td>90.81±8.95</td><td>62.55±15.30</td><td><b>94.27±7.19</b></td><td><b>98.72±2.92</b></td><td>91.84±4.25</td><td>96.11±2.99</td><td>88.57</td></tr><tr><td>Average</td><td>72.08</td><td>88.49</td><td>81.61</td><td>87.01</td><td>55.46</td><td>79.69</td><td>96.23</td><td>81.74</td><td>92.02</td><td>81.59</td></tr></tbody></table>

<table><caption>Table D4: Summary of performances via average on all the P300 datasets, for classification using all the labels. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>BNCI2014-008</th><th>BNCI2014-009</th><th>BNCI2015-003</th><th>BrainInvaders2012</th><th>BrainInvaders2011a</th><th>BrainInvaders2014a</th><th>BrainInvaders2014b</th><th>BrainInvaders2015a</th><th>BrainInvaders2015b</th><th>Cattan2019-VR</th><th>EPFLP-300</th><th>Huebner2017</th><th>Huebner2018</th><th>Lee2019-ERP</th><th>Soulsbl2019</th><th>Average</th></tr></thead><tbody><tr><td>ERPCov+MDM</td><td>74.30±9.77</td><td>81.16±10.13</td><td>76.79±10.95</td><td>78.77±10.32</td><td>80.59±9.36</td><td>71.62±11.17</td><td>78.57±12.36</td><td>80.02±10.07</td><td>75.04±15.85</td><td>80.76±10.07</td><td>71.97±10.88</td><td>94.47±8.26</td><td>95.15±3.72</td><td>74.43±13.26</td><td>68.17±13.59</td><td>78.79</td></tr><tr><td>ERPCov(svdn4)+MDM</td><td>75.42±9.91</td><td>84.52±8.83</td><td>76.93±11.26</td><td>79.02±10.53</td><td>82.07±8.46</td><td>72.11±11.64</td><td>76.48±12.83</td><td>77.92±10.33</td><td>77.09±15.81</td><td>80.67±9.47</td><td>71.44±10.20</td><td>96.21±6.50</td><td>96.61±1.89</td><td>82.47±12.56</td><td>70.63±13.79</td><td>79.97</td></tr><tr><td>XDAWN+LDA</td><td>82.24±5.26</td><td>64.03±3.91</td><td>78.62±7.19</td><td>64.41±4.14</td><td>76.74±7.16</td><td>66.60±7.54</td><td>83.73±10.62</td><td>76.02±10.46</td><td>77.22±13.73</td><td>67.16±6.11</td><td>62.98±5.38</td><td>97.74±2.84</td><td>97.54±1.58</td><td>96.45±3.93</td><td>67.49±7.44</td><td>77.27</td></tr><tr><td>XDAWNCoV+MDM</td><td>77.62±9.81</td><td>92.04±5.97</td><td><b>83.08±7.55</b></td><td><b>88.22±5.90</b></td><td>90.97±5.52</td><td>80.88±11.01</td><td>91.58±10.02</td><td>92.57±5.03</td><td>83.48±12.05</td><td>88.53±7.34</td><td>83.20±9.05</td><td>98.67±2.09</td><td>97.78±1.04</td><td>97.70±2.68</td><td>86.07±7.15</td><td>88.79</td></tr><tr><td>XDAWNCoV+TS+SVM</td><td><b>85.61±4.43</b></td><td><b>93.43±5.11</b></td><td><b>82.95±8.57</b></td><td><b>90.99±4.79</b></td><td><b>92.71±4.92</b></td><td><b>85.77±9.75</b></td><td><b>91.88±9.94</b></td><td><b>93.05±4.98</b></td><td><b>84.56±12.09</b></td><td><b>90.68±6.29</b></td><td><b>84.29±8.53</b></td><td><b>98.69±1.78</b></td><td><b>98.47±0.97</b></td><td><b>98.41±2.03</b></td><td><b>87.28±6.92</b></td><td><b>90.58</b></td></tr><tr><td>Average</td><td>79.04</td><td>83.01</td><td>79.67</td><td>80.28</td><td>84.61</td><td>75.4</td><td>84.45</td><td>83.91</td><td>79.48</td><td>81.56</td><td>74.78</td><td>97.04</td><td>97.11</td><td>89.89</td><td>75.93</td><td>83.08</td></tr></tbody></table>

<table><caption>Table D5: Summary of performances via average on all the P300 datasets, for classification using left vs. right motor imagery task. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>Kalunga2016</th><th>Lee2019-SSVEP</th><th>MAMEM1</th><th>MAMEM2</th><th>MAMEM3</th><th>Nakanishi2015</th><th>Wang2016</th><th>Average</th></tr></thead><tbody><tr><td>CCA</td><td>25.40±2.51</td><td>23.86±3.72</td><td>19.17±5.01</td><td>23.60±4.10</td><td>13.80±7.47</td><td>8.15±0.74</td><td>2.48±1.01</td><td>16.64</td></tr><tr><td>MsetCCA</td><td>22.67±4.23</td><td>25.10±3.81</td><td>20.50±2.37</td><td>22.08±1.76</td><td>27.60±3.01</td><td>7.10±1.50</td><td>4.00±1.10</td><td>18.43</td></tr><tr><td>MDM</td><td><b>70.89±13.44</b></td><td>75.38±18.38</td><td>27.31±11.64</td><td>23.12±6.29</td><td>34.40±9.96</td><td>78.77±19.06</td><td>54.77±21.95</td><td>52.09</td></tr><tr><td>TS+LR</td><td>70.86±11.64</td><td><b>89.44±13.84</b></td><td><b>53.71±24.25</b></td><td><b>39.36±12.06</b></td><td><b>42.10±14.33</b></td><td><b>87.22±15.96</b></td><td><b>67.52±20.04</b></td><td><b>64.32</b></td></tr><tr><td>TS+SVM</td><td>68.95±13.73</td><td>88.58±14.47</td><td>50.58±23.34</td><td>34.80±11.76</td><td>40.20±14.41</td><td>86.30±15.88</td><td>59.58±20.57</td><td>61.28</td></tr><tr><td>TRCA</td><td>24.84±7.24</td><td>64.01±15.27</td><td>24.24±6.65</td><td>24.24±2.93</td><td>23.70±3.49</td><td>83.21±10.80</td><td>2.79±1.03</td><td>35.29</td></tr><tr><td>Average</td><td>47.27</td><td>61.06</td><td>32.58</td><td>27.87</td><td>30.3</td><td>58.46</td><td>31.86</td><td>41.34</td></tr></tbody></table>
<table><caption>Table D3: Summary of performances via average on all the P300 datasets, for classification using right hand vs. feet tasks motor imagery task. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>AlexandreMotorImagery</th><th>BNCI2014-001</th><th>BNCI2014-002</th><th>BNCI2015-001</th><th>BNCI2015-004</th><th>PhysionetMotorImagery</th><th>Schirrmeister2017</th><th>Weibo2014</th><th>Zhou2016</th><th>Average</th></tr></thead><tbody><tr><td>ACM+TS+SVM</td><td><strong>86.56±12.26</strong></td><td><strong>97.32±3.35</strong></td><td><strong>88.60±10.71</strong></td><td><strong>93.01±8.09</strong></td><td><strong>62.60±14.62</strong></td><td>93.33±8.46</td><td>98.67±3.06</td><td><strong>93.25±4.12</strong></td><td><strong>97.18±3.00</strong></td><td><strong>90.06</strong></td></tr><tr><td>CSP+LDA</td><td>77.19±17.58</td><td>91.52±10.39</td><td>80.98±14.79</td><td>88.52±10.75</td><td>54.02±11.33</td><td>86.41±13.96</td><td>97.02±5.17</td><td>88.59±6.36</td><td>95.20±3.17</td><td>84.38</td></tr><tr><td>CSP+SVM</td><td>78.59±20.14</td><td>91.04±10.35</td><td>81.21±15.30</td><td>89.19±10.08</td><td>52.08±11.05</td><td>88.04±12.57</td><td>97.50±4.90</td><td>88.64±5.90</td><td>94.95±3.53</td><td>84.58</td></tr><tr><td>DLCSPauto+shLDA</td><td>77.03±18.93</td><td>91.54±10.37</td><td>80.45±15.52</td><td>88.87±10.42</td><td>53.02±10.75</td><td>86.81±13.34</td><td>96.95±5.22</td><td>88.48±6.53</td><td>94.43±3.41</td><td>84.18</td></tr><tr><td>DeepConvNet</td><td>61.88±19.05</td><td>88.27±12.19</td><td>87.56±11.25</td><td>88.12±13.19</td><td>57.08±12.29</td><td>71.49±15.88</td><td>95.90±7.14</td><td>79.29±12.63</td><td>95.92±3.66</td><td>80.61</td></tr><tr><td>EEGITNet</td><td>47.50±9.46</td><td>75.98±13.09</td><td>70.90±17.50</td><td>71.95±16.76</td><td>51.41±6.40</td><td>54.69±11.97</td><td>96.04±8.62</td><td>62.54±12.32</td><td>80.40±17.12</td><td>67.93</td></tr><tr><td>EEGNeX</td><td>52.34±14.81</td><td>64.36±13.49</td><td>69.95±20.12</td><td>72.34±19.83</td><td>53.02±9.69</td><td>51.77±12.06</td><td>89.49±16.91</td><td>60.18±11.70</td><td>64.80±16.64</td><td>64.25</td></tr><tr><td>EEGNet-8,2</td><td>64.22±16.01</td><td>88.55±14.92</td><td>83.93±16.31</td><td>90.43±11.75</td><td>54.20±8.20</td><td>73.78±15.59</td><td>96.50±8.07</td><td>78.15±14.46</td><td>94.58±3.21</td><td>80.48</td></tr><tr><td>EEGTCNet</td><td>61.09±22.06</td><td>75.21±18.53</td><td>73.92±19.02</td><td>77.21±18.55</td><td>51.22±5.84</td><td>57.03±13.25</td><td>97.15±7.70</td><td>62.37±12.42</td><td>85.46±16.42</td><td>71.19</td></tr><tr><td>FBCSP+SVM</td><td>80.78±18.86</td><td>93.55±6.29</td><td>80.39±16.83</td><td>91.57±7.66</td><td>52.51±9.82</td><td>83.97±12.43</td><td>97.40±4.18</td><td>88.27±7.91</td><td>94.63±3.94</td><td>84.78</td></tr><tr><td>FgMDM</td><td>79.84±17.80</td><td>93.52±8.18</td><td>84.77±11.26</td><td>90.18±9.77</td><td>58.31±12.63</td><td>89.67±10.65</td><td>98.48±3.45</td><td>88.56±4.63</td><td>96.04±2.67</td><td>86.6</td></tr><tr><td>MDM</td><td>74.22±21.19</td><td>89.13±10.38</td><td>77.48±14.11</td><td>86.20±12.99</td><td>48.45±9.62</td><td>81.78±11.64</td><td>84.67±13.13</td><td>65.18±9.75</td><td>92.21±4.31</td><td>77.7</td></tr><tr><td>ShallowConvNet</td><td>64.22±18.33</td><td>93.00±8.05</td><td>87.60±12.05</td><td>91.41±10.88</td><td>57.23±12.36</td><td>74.75±14.98</td><td>98.06±4.35</td><td>88.70±5.60</td><td>97.06±1.86</td><td>83.56</td></tr><tr><td>TS+EL</td><td>81.41±21.36</td><td>94.45±6.74</td><td>85.98±11.38</td><td>91.19±8.49</td><td>58.70±13.37</td><td>94.09±7.17</td><td>98.56±3.01</td><td>92.32±3.98</td><td>96.59±2.82</td><td>88.14</td></tr><tr><td>TS+LR</td><td>83.75±17.47</td><td>94.45±7.06</td><td>85.86±11.01</td><td>91.09±8.71</td><td>61.01±14.22</td><td>93.15±7.40</td><td>98.60±3.08</td><td>91.53±4.53</td><td>96.76±2.58</td><td>88.47</td></tr><tr><td>TS+SVM</td><td>82.66±18.16</td><td>94.01±7.60</td><td>86.19±11.50</td><td>90.81±8.95</td><td>62.55±15.30</td><td><strong>94.27±7.19</strong></td><td><strong>98.72±2.92</strong></td><td>91.84±4.25</td><td>96.11±2.99</td><td>88.57</td></tr><tr><td>Average</td><td>72.08</td><td>88.49</td><td>81.61</td><td>87.01</td><td>55.46</td><td>79.69</td><td>96.23</td><td>81.74</td><td>92.02</td><td>81.59</td></tr></tbody></table>

<table><caption>Table D4: Summary of performances via average on all the P300 datasets, for classification using all the labels. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>BNCI2014-008</th><th>BNCI2014-009</th><th>BNCI2015-003</th><th>BrainInvaders2012</th><th>BrainInvaders2011a</th><th>BrainInvaders2014a</th><th>BrainInvaders2014b</th><th>BrainInvaders2015a</th><th>BrainInvaders2015b</th><th>Cattan2019-VR</th><th>EPFLP-300</th><th>Huebner2017</th><th>Huebner2018</th><th>Lee2019-ERP</th><th>Soulsbl2019</th><th>Average</th></tr></thead><tbody><tr><td>ERPCov+MDM</td><td>74.30±9.77</td><td>81.16±10.13</td><td>76.79±10.95</td><td>78.77±10.32</td><td>80.59±9.36</td><td>71.62±11.17</td><td>78.57±12.36</td><td>80.02±10.07</td><td>75.04±15.85</td><td>80.76±10.07</td><td>71.97±10.88</td><td>94.47±8.26</td><td>95.15±3.72</td><td>74.43±13.26</td><td>68.17±13.59</td><td>78.79</td></tr><tr><td>ERPCov(svdn4)+MDM</td><td>75.42±9.91</td><td>84.52±8.83</td><td>76.93±11.26</td><td>79.02±10.53</td><td>82.07±8.46</td><td>72.11±11.64</td><td>76.48±12.83</td><td>77.92±10.33</td><td>77.09±15.81</td><td>80.67±9.47</td><td>71.44±10.20</td><td>96.21±6.50</td><td>96.61±1.89</td><td>82.47±12.56</td><td>70.63±13.79</td><td>79.97</td></tr><tr><td>XDAWN+LDA</td><td>82.24±5.26</td><td>64.03±3.91</td><td>78.62±7.19</td><td>64.41±4.14</td><td>76.74±7.16</td><td>66.60±7.54</td><td>83.73±10.62</td><td>76.02±10.46</td><td>77.22±13.73</td><td>67.16±6.11</td><td>62.98±5.38</td><td>97.74±2.84</td><td>97.54±1.58</td><td>96.45±3.93</td><td>67.49±7.44</td><td>77.27</td></tr><tr><td>XDAWNCoV+MDM</td><td>77.62±9.81</td><td>92.04±5.97</td><td><strong>83.08±7.55</strong></td><td><strong>88.22±5.90</strong></td><td>90.97±5.52</td><td>80.88±11.01</td><td>91.58±10.02</td><td>92.57±5.03</td><td>83.48±12.05</td><td>88.53±7.34</td><td>83.20±9.05</td><td>98.67±2.09</td><td>97.78±1.04</td><td>97.70±2.68</td><td>86.07±7.15</td><td>88.79</td></tr><tr><td>XDAWNCoV+TS+SVM</td><td><strong>85.61±4.43</strong></td><td><strong>93.43±5.11</strong></td><td><strong>82.95±8.57</strong></td><td><strong>90.99±4.79</strong></td><td><strong>92.71±4.92</strong></td><td><strong>85.77±9.75</strong></td><td><strong>91.88±9.94</strong></td><td><strong>93.05±4.98</strong></td><td><strong>84.56±12.09</strong></td><td><strong>90.68±6.29</strong></td><td><strong>84.29±8.53</strong></td><td><strong>98.69±1.78</strong></td><td><strong>98.47±0.97</strong></td><td><strong>98.41±2.03</strong></td><td><strong>87.28±6.92</strong></td><td><strong>90.58</strong></td></tr><tr><td>Average</td><td>79.04</td><td>83.01</td><td>79.67</td><td>80.28</td><td>84.61</td><td>75.4</td><td>84.45</td><td>83.91</td><td>79.48</td><td>81.56</td><td>74.78</td><td>97.04</td><td>97.11</td><td>89.89</td><td>75.93</td><td>83.08</td></tr></tbody></table>

<table><caption>Table D5: Summary of performances via average on all the P300 datasets, for classification using left vs. right motor imagery task. Intra-session validation. Bold numbers represent the best score in each dataset.</caption><thead><tr><th>pipeline</th><th>Kalunga2016</th><th>Lee2019-SSVEP</th><th>MAMEM1</th><th>MAMEM2</th><th>MAMEM3</th><th>Nakanishi2015</th><th>Wang2016</th><th>Average</th></tr></thead><tbody><tr><td>CCA</td><td>25.40±2.51</td><td>23.86±3.72</td><td>19.17±5.01</td><td>23.60±4.10</td><td>13.80±7.47</td><td>8.15±0.74</td><td>2.48±1.01</td><td>16.64</td></tr><tr><td>MsetCCA</td><td>22.67±4.23</td><td>25.10±3.81</td><td>20.50±2.37</td><td>22.08±1.76</td><td>27.60±3.01</td><td>7.10±1.50</td><td>4.00±1.10</td><td>18.43</td></tr><tr><td>MDM</td><td><strong>70.89±13.44</strong></td><td>75.38±18.38</td><td>27.31±11.64</td><td>23.12±6.29</td><td>34.40±9.96</td><td>78.77±19.06</td><td>54.77±21.95</td><td>52.09</td></tr><tr><td>TS+LR</td><td>70.86±11.64</td><td><strong>89.44±13.84</strong></td><td><strong>53.71±24.25</strong></td><td><strong>39.36±12.06</strong></td><td><strong>42.10±14.33</strong></td><td><strong>87.22±15.96</strong></td><td><strong>67.52±20.04</strong></td><td><strong>64.32</strong></td></tr><tr><td>TS+SVM</td><td>68.95±13.73</td><td>88.58±14.47</td><td>50.58±23.34</td><td>34.80±11.76</td><td>40.20±14.41</td><td>86.30±15.88</td><td>59.58±20.57</td><td>61.28</td></tr><tr><td>TRCA</td><td>24.84±7.24</td><td>64.01±15.27</td><td>24.24±6.65</td><td>24.24±2.93</td><td>23.70±3.49</td><td>83.21±10.80</td><td>2.79±1.03</td><td>35.29</td></tr><tr><td>Average</td><td>47.27</td><td>61.06</td><td>32.58</td><td>27.87</td><td>30.3</td><td>58.46</td><td>31.86</td><td>41.34</td></tr></tbody></table>
<figure><img src="image_13.png" /><figcaption>Figure C1: Distributions of ROC-AUC scores on the right hand vs left hand MI task of the pipelines grouped by category.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure C2: Distributions of ROC-AUC scores on the right hand vs left hand MI task.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>Figure C1: Distributions of ROC-AUC scores on the right hand vs left hand MI task of the pipelines grouped by category.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure C2: Distributions of ROC-AUC scores on the right hand vs left hand MI task.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>Figure C3: Accuracy scores averaged over all subjects and session of each SSVEP dataset, per pipeline category.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Figure C4: Box-plot representing classification accuracy averaged over all the sessions of all the subjects of all the datasets of the SSVEP paradigm and over all pipelines of the corresponding category (*Riemannian, Raw*). Box-plots are overlaid with strip-plots, where each point represents the classification accuracy of one within-session evaluation.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Figure C5: ROC-AUC scores on the ERP classification task of the different pipelines.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>Figure C3: Accuracy scores averaged over all subjects and session of each SSVEP dataset, per pipeline category.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Figure C4: Box-plot representing classification accuracy averaged over all the sessions of all the subjects of all the datasets of the SSVEP paradigm and over all pipelines of the corresponding category (*Riemannian, Raw*). Box-plots are overlaid with strip-plots, where each point represents the classification accuracy of one within-session evaluation.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Figure C5: ROC-AUC scores on the ERP classification task of the different pipelines.</figcaption></figure>