<figure><img src="image_1.png" /></figure>

HAL
open science

# Holographic EEG: multi-view deep learning for BCI

Igor Carrara, Bruno Aristimunha, Sylvain Chevallier, Marie-Constance Corsi,
Théodore Papadopoulos, Marie-Constance Corsi

<figure><img src="image_2.png" /></figure>

## To cite this version:

Igor Carrara, Bruno Aristimunha, Sylvain Chevallier, Marie-Constance Corsi, Théodore Papadopoulos, et al.. Holographic EEG: multi-view deep learning for BCI. Journée CORTICO 2023 - Collectif pour la Recherche Transdisciplinaire sur les Interfaces Cerveau-Ordinateur, May 2023, Paris, France. . hal-04095900

HAL Id: hal-04095900

https://hal.science/hal-04095900

Submitted on 12 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# HOLOGRAPHIC EEG: multi-view deep learning for BCI

Inria

UNIVERSITÉ CÔTE D'AZUR

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

NEURO

MOD

<figure><img src="image_7.png" /></figure>

ARAMIS LAB
BRAIN DATA SCIENCE

université PARIS-SACLAY

UFABC

I. Carrara¹,²*, B. Aristimunha³, ⁴, S. Chevallier⁴, MC. Corsi⁵, T. Papadopoulos¹, ²

¹ Université Côte d'Azur (UCA), Nice, France,
² Université de Lille

³ Université de Lille
⁴ ARAMIS, Inria, Paris Brain Institute, Paris, France

² Centre Inria d'Université Côte d'Azur, Sophia Antipolis, France,

⁴ Université Paris-Saclay, Paris, France

<figure><img src="image_8.png" /></figure>

Brain Computer Interfaces or **BCI** can be defined as a technology that measures brain activity and translates that activity into instructions for a digital system. We will focus on **non-invasive (EEG)-based BCI (BCI-EEG)**.

Let us consider a typical EEG signal after preprocessing. In the figure on the left, we show an EEG signal measured with 4 different electrodes. Our work will be focused on **Motor Imagery (MI) BCI**, so we want to understand if the subject, for example, is thinking about moving the left or the right hand.

**Can we exploit multiple features to learn inherent SPD representations of the EEG data in a self-supervised manner, to later improve cross-subject classification ?**

<figure><img src="image_9.png" /></figure>

## METHODS: Holographic-EEG

### Standard Riemann approach [1]:

Extract sample spatial covariances which are Symmetric
Positive Definite (SPD) matrices and classify them using
Minimum Distance to Mean (MDM) or classical machine
learning procedure on the tangent space (SVM, LDA, ...)

From the EEG signal, we can extract different SPD matrices:

- Spatial covariance (as in the standard case).

- Augmented Covariance [2]

- Coherence [3]

Inspired by recent advances in Computer Vision [4], we want to use these different matrices in a **semi self-supervised contrastive framework**.

The idea is to **adjust the weights of the different SPDNet [5] Deep Learning networks in the architecture** (on the right) so that the different SPD representations for a same data sample are pushed closer (maximizing the Mutual Information) while those of different samples are pushed apart.

### Use of NT-Xent Contrastive Loss [4]

$$ \mathcal{L}^{\text{self}} = - \sum_{i \in I} \log \frac{\exp(z_i \cdot z_{j(i)} / \tau)}{\sum_{a \in A(i)} \exp(z_i \cdot z_a / \tau)} $$

We use the SPDNet architecture [5] instead of classical Convolutional Networks because we work with SPD matrices. Similarly, we are using the RiemannAdam optimizer [6] (instead of the

standard Adam algorithm) because we need to constrain the weights in the BiMap layers to be on the SPD Riemannian manifold.

Figure extracted from [5]

<figure><img src="image_10.png" /></figure>

The self-supervised learnt SPDNets are then used in a classification network which classification layer is trained for a specific task using a small labelled training set.

<figure><img src="image_11.png" /></figure>

Figure extracted from [1]

<figure><img src="image_12.png" /></figure>

## RESULTS

In order to test our approach and to increase the reliability of our results, we use the **MOABB framework** [7]. Results are preliminary and are tested only on **BNCI2014001 dataset** on task **right hand vs left hand** in a **cross subject evaluation. We compare the results against the state of the art algorithms [8]**. In this experiment, we use only spatial covariance and coherence

<figure><img src="image_13.png" /></figure>

The state of the art algorithm use all labelled data to train the algorithm. Our self supervised **HolograpichEEG** use only 20% of the training labelled data.

No statistical differences between **HolograpichEEG** and other non deep learning ML algorithms can be seen. In some cases, our method is superior (CSP+LDA, FgMDM). DL architectures in general outperform **HolograpichEEG** (but with a 5 times bigger training set).

## TAKE HOME MESSAGE

**Contrastive learning** can be used to **learn meaningful representations of EEG signals from different subjects**. Further research is required to fully explore the potential of self-supervised contrastive learning and really improve over the state of the art in EEG-BCI.

<figure><img src="image_14.png" /></figure>

**Acknowledgements:** This work has been partially financed by a EUR DS4H/Neuromod fellowship.

**References:**

[1] Alexandre Barachant et al. "Riemannian geometry applied to BCI classification". In: International conference on latent variable analysis and signal separation. Springer. 2010, pp. 629-636.

[2] Igor Carrara et al. "Classification of BCI-EEG based on augmented covariance matrix" arXiv preprint arXiv:2302.04508 (2023).

[3] Marie-Constance Corsi et al. "Functional connectivity ensemble method to enhance BCI performance (FUCONE)." IEEE TBE (2022)

[4] Chen, Ting, et al. "A simple framework for contrastive learning of visual representations." International conference on machine learning. PMLR, 2020.

[5] Huang, Zhiwu et al. "A riemannian network for spd matrix learning." Proceedings of the AAAI conference on artificial intelligence. Vol. 31. No. 1. 2017.

[6] Max Kochurov et al. "Geoopt: Riemannian optimization in pytorch." arXiv preprint arXiv:2005.02819 (2020).

[7] Alexandre Barachant et al. "MOABB: trustworthy algorithm benchmarking for BCIs". In: Journal of neural engineering 15.6 (2018), p. 066011.

[8] Fabien Lotte et al. "A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update". In: Journal of neural engineering (2018)