# Riemannian geometry for combining functional connectivity metrics and covariance in BCI

Sylvain Chevallier, Marie Constance Corsi, Florian Yger, Fabrizio de Vico Fallani

► To cite this version:

Sylvain Chevallier, Marie Constance Corsi, Florian Yger, Fabrizio de Vico Fallani. Riemannian geometry for combining functional connectivity metrics and covariance in BCI. Software Impacts, 2022, 12, pp.100254. 10.1016/j.simpa.2022.100254. hal-03772666

HAL Id: hal-03772666

https://hal.science/hal-03772666v1

Submitted on 21 Sep 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Original software publication

# Riemannian geometry for combining functional connectivity metrics and covariance in BCI

<figure><img src="image_2.png" /></figure>

Check for
updates

Sylvain Chevallier<sup>a</sup>, Marie-Constance Corsi<sup>b,*</sup>, Florian Yger<sup>c</sup>, Fabrizio De Vico Fallani<sup>b</sup>

## ARTICLE INFO

### Keywords:

Functional connectivity

Riemannian geometry

Brain-computer interface

Ensemble learning

## ABSTRACT

Brain-computer interfaces allow interactions based on brain activities detected in electroencephalography. Despite important improvements in the last decade, some subjects still achieve poor performances without any identified cause. On the one hand, State-of-the-art methods for online decoding are based on covariance matrices seen as elements of a Riemann manifold. On the other hand, functional connectivity is a powerful method to characterize the brain activity. The proposed software combines functional connectivity and covariance within a Riemannian framework to increase the robustness of brain-computer interfaces.

### Code metadata

Current code version

Permanent link to code/repository used for this code version

Permanent link to Reproducible Capsule

Legal Code License

Code versioning system used

Software code languages, tools, and services used

Compilation requirements, operating environments & dependencies

If available Link to developer documentation/manual

Support email for questions

v1.0 (preprint)

https://github.com/SoftwareImpacts/SIMPAC-2021-189

N.A.

BSD 3-clauses

git

Python

see https://github.com/mccorsi/FUCONE/blob/main/requirements.txt

https://github.com/mccorsi/FUCONE/blob/main/README.md

sylvain.chevallier@uvsq.fr

## 1. Introduction

Brain-computer interfaces allow to communicate without requiring any muscular capabilities and is therefore widely investigated for assisting people with motor deficiencies. The electroencephalography (EEG), as a low-cost and light-weight technology, is a very common choice for recording the electric potential generated by brain activities [1]. Despite the large range of clinical applications, BCI fails in being used outside the laboratory. One of the main challenges to overcome is the high inter-subject variability, referred in the literature as the “BCI inefficiency” phenomenon, associated with a non-negligible percentage of the users who cannot control the BCI device even after several training sessions. Among the promising approaches to tackle this issue is the improvement of the neural decoders [2]. For this purpose, studies have elicited new features that relied on covariance matrices, like $Cov = \frac{1}{T-1}XX^T$ for an EEG signal $X$ of $T$ signal samples, and on adjacency matrices. Those adjacency matrices are

based on specific functional connectivity estimators, in particular the imaginary coherence and the instantaneous coherence. They result from the coherency defined for a given frequency bin $f$ as, $Coh_{ij}(f) = \frac{S_{ij}(f)}{\sqrt{S_{ii}(f)S_{jj}(f)}}$, where $S_{ij}(f)$ is the cross-spectral density and $S_{ii}(f)$ the auto-spectral density. The imaginary coherence (ImCoh) corresponds to the imaginary part of the coherency [3]. The instantaneous coherence (Instantaneous) corresponds to the real part of the coherency [4]. The software described hereafter proposes an original solution that takes advantage of complementary features to improve the decoder's performance.

## 2. A Riemannian take on functional connectivity for brain-computer interfaces

Riemannian geometry-based methods are now the gold standard in motor imagery-based BCI [5]. They have been applied to covariance

<footnote><sup>a</sup> LISV - UVSQ - Université Paris-Saclay, France</footnote>

<footnote><sup>b</sup> Inria-Paris, Aramis Project-team, Paris Brain Institute, France</footnote>

<footnote><sup>c</sup> LAMSADE, CNRS, Université Paris-Dauphine, PSL Research University, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: marie.constance.corsi@gmail.com (M.-C. Corsi).</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 1. (A) Overview of the proposed approach. Cov stands for covariance, Instantaneous stands for instantaneous coherence and ImCoh stands for imaginary coherence, and EN stands for Elastic-net. (B) Replicability assessments and comparison with state-of-the-art pipelines. Analysis performed with 2-class datasets. lhrh stands for left- vs right-hand.</figcaption></figure>

matrices that are symmetric and positive-definite matrices (SPD) and obtained from EEG signals. Here, in order to take into account the user's specificity and more particularly the interconnected nature of brain functioning, we considered, in addition to covariance matrices, connectivity information extracted from functional connectivity estimators [6]. Our solution tackle the inter-subject variability by combining multiple feature space to extract robust representation (see Fig. 1-A).

We follow the FAIR principles to ensure the reproducibility of your work, grounding the open science inside our software production. We develop a Python code, hosted and versioned on GitHub, that follow the scikit-learn API [7] for designing machine learning pipelines. Our approach is built upon existing open source libraries, such as PyRiemann<fnref n="1" /> for Riemannian geometry tools, MNE [8] for handling EEG files and MOABB<fnref n="2" /> to conduct the benchmark.

In this software, we introduce three novel developments: (i) new functional connectivity estimators, (ii) new projection method on Riemannian manifold and (iii) new evaluation for benchmarks. For (i), MNE already propose connectivity estimators, but they are suited to offline analysis and not online decoding. We adapt these existing estimators to handle trial-based estimation. The connectivity matrices yield by these estimators are positive semi-definite, meaning that they are not necessarily full rank, and thus could fail to be SPD. We compute the projection on the manifold of SPD matrices of the instantaneous coherence (Instantaneous) and the imaginary coherence (ImCoh) to ensure that consistency of our processing steps. With (ii), we develop a projection method on the manifold of SPD matrices to ensure that we could apply Riemannian approaches on these matrices. At last, we develop in (iii) a new benchmark method based on MOABB classes for facilitating the selection of model parameters (see Fig. 1-B).

## 3. Impact overview

The developed software has already been used in several studies. In [9], the diversity of the classifiers based on connectivity and

covariance is highlighted, indicating that it is a good candidate for ensemble learning. This approach won a competition on clinical BCI data [10] and its application to another dataset is shown in [11]. A thorough analysis of connectivity estimators, preprocessing step and ensemble learning method developed in [11] along with a complete evaluation on multiple datasets and different experimental conditions.

There are several research questions that can be pursued as a result of our software. First, we introduce new features in BCI, by demonstrating that it is possible to apply Riemannian classifiers on functional connectivity matrices. There are several estimators, like non-linear or robust ones, that could be evaluated. Next, we rely on an ensemble method to combine multiple classifiers for improving the robustness of our decoding. It is possible to improve this ensemble method by incorporating domain-specific knowledge. Last, dimensionality reduction methods could be applied to project connectivity and covariance in lower dimensional manifold.

It should be noted that our software development is a work in progress. We want to improve the replicability of our approach by contributing to the libraries that help to build our software. We will contribute to pyRiemann<fnref n="3" /> and MOABB<fnref n="4" /> to integrate the development made in our work. Brain-computer interfaces is a multidisciplinary topic that requires advanced signal processing techniques, state-of-the-art machine learning classifiers and good knowledge in neuroscience. This requires joint efforts to ensure a high software quality and open contributions to allows for replicable studies.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

<footnote n="1">¹ https://pyriemann.readthedocs.io</footnote>

<footnote n="2">² https://neurotechx.github.io/moabb/</footnote>

<footnote n="3">³ https://pyriemann.readthedocs.io/</footnote>

<footnote n="4">⁴ https://neurotechx.github.io/moabb/</footnote>
## Acknowledgments

FDVF and MCC acknowledge support from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement No. 864729). FY acknowledges the support of the ANR, France as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute).

## References

[1] Wolpaw, et al., Brain-computer interfaces for communication and control, Clin. Neurophysiol. 113 (6) (2002) 767-791, http://dx.doi.org/10.1016/S1388-2457(02)00057-3.

[2] Lotte, et al., A review of classification algorithms for EEG-based brain-computer interfaces: A 10-year update, J. Neural Eng. (2018) http://dx.doi.org/10.1088/1741-2552/aab2f2.

[3] Nolte, et al., Identifying true brain interaction from EEG data using the imaginary part of coherency, Clin. Neurophysiol. 115 (10) (2004) 2292-2307, http://dx.doi.org/10.1016/j.clinph.2004.04.029.

[4] Pascual-Marqui, Instantaneous and lagged measurements of linear and non-linear dependence between groups of multivariate time series: frequency decomposition, 2007, http://arxiv.org/abs/0711.1455.

[5] Yger, et al., Riemannian approaches in brain-computer interfaces: A review, IEEE Trans. Neural Syst. Rehabil. Eng. 25 (10) (2017) 1753-1762, http://dx.doi.org/10.1109/TNSRE.2016.2627016.

[6] De Vico Fallani, et al., Graph analysis of functional brain networks: practical issues in translational neuroscience, Philos. Trans. R. Soc. B 369 (2014) 1653, http://dx.doi.org/10.1098/rstb.2013.0521.

[7] Buitinck, et al., API design for machine learning software: experiences from the scikit-learn project, in: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, 2013, pp. 108-122.

[8] Gramfort, et al., MEG and EEG data analysis with MNE-Python, Front. Neurosci. 7 (267) (2013) 1-13, http://dx.doi.org/10.3389/fnins.2013.00267.

[9] Chevallier, et al., Extending Riemannian brain-computer interface to functional connectivity estimators, in: IROS Workshop on Bringing Geometric Methods to Robot Learning, Optimization and Control, 2020, https://hal.archives-ouvertes.fr/hal-03015390.

[10] Corsi, et al., Riemannian geometry on connectivity for clinical BCI, in: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, 2021, pp. 980-984, http://dx.doi.org/10.1109/ICASSP39728.2021.9414790.

[11] Corsi, et al., Functional connectivity ensemble method to enhance BCI performance (FUCONE), 2021, arXiv:2111.03122 [Cs, q-Bio] http://arxiv.org/abs/2111.03122.