<figure><img src="image_1.png" /></figure>

HAL
open science

# What are BCIs, and how do they work? How to ease the reproducibility in BCI?

Marie-Constance Corsi, Arthur Desbois

► To cite this version:

Marie-Constance Corsi, Arthur Desbois. What are BCIs, and how do they work? How to ease the reproducibility in BCI?. 5th International Neuroergonomics Conference, Jul 2024, Bordeaux, France. hal-04701062

HAL Id: hal-04701062

https://inria.hal.science/hal-04701062v1

Submitted on 5 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
WHAT ARE BCIS, AND HOW DO THEY WORK?
HOW TO EASE THE REPRODUCIBILITY IN BCI?

<figure><img src="image_3.png" /></figure>

Arthur Desbois & Marie-Constance Corsi

Inria NERV team, Paris Brain Institute

<figure><img src="image_4.png" /></figure>

ICM Institut du Cerveau

<figure><img src="image_5.png" /></figure>

Inserm
Institut national
de la santé et de la recherche médicale

<figure><img src="image_6.png" /></figure>

SORBONNE
UNIVERSITÉ
<figure><img src="image_7.png" /></figure>
<figure><img src="image_8.png" /></figure>
<figure><img src="image_9.png" /></figure>

Non-invasive tools

EEG

<figure><img src="image_10.png" /></figure>

NIRS

<figure><img src="image_11.png" /></figure>

Near Infrared Spectroscopy
(Fazli et al, 2012)

MEG

<figure><img src="image_12.png" /></figure>

fMRI

<figure><img src="image_13.png" /></figure>

Functional MRI
(Sitaram et al, 2009)
<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

Adapted from (Corsi, 2023)
# Brain recording w/ M/EEG – Evoked responses

Nomenclature: the latency, the amplitude, the shape and the polarity

- Nxxx: one negative wave @ xxx ms (EEG)

- Pxxx: one positive wave @ xxx ms (EEG)

- Mxxx: one wave @ xxx ms (MEG)

<figure><img src="image_16.png" /></figure>

Adapted from [Campagne, 2014]

## Components

- Early components (exogenous): related to stimulus characteristics

- Late components (endogenous): related to the task, to the subject's stat

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

Adapted from [Corsi, 2023]
# Brain recording w/ M/EEG – Oscillatory activity

## Spontaneous activity, characteristics:

- Frequency

- Amplitude

- Shape

- Localization

- Psychopsychological context

- Duration

- Vanishing

<figure><img src="image_19.png" /></figure>

Adapted from [Corsi, 2023]
<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

Adapted from
[Gonzalez-Astudillo, ..., Corsi, & De Vico Fallani, JNE, 2020]
<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

Adapted from
[Gonzalez-Astudillo, ..., Corsi, & De Vico Fallani, JNE, 2020]
<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>
Control

- Prosthesis (Fifer et al, 2014)

- Wheelchair (Carlson & Millan, 2013)

- Quadcopter (LaFleur et al, 2013)

<figure><img src="image_26.png" /></figure>

Communication

- Verbal & nonverbal communication (Jin et al, 2012; Hwang et al, 2012; Kashihara, 2014)

- Silent talk (Naci et al, 2013)

<figure><img src="image_27.png" /><figcaption>BCI & communication</figcaption></figure>

Neurological disorders treatment

- Stroke (Prasad et al, 2010)

- Spinal cord injury (King et al, 2013)

- Consciousness (Chatelle et al, 2012)

- Psychiatric disorders (Arns et al, 2017)
* Inria software development w/ OpenViBE for:

* Robotic device control

* Stroke rehabilitation

* Better monitoring general anesthesia

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /><figcaption>LORIA projects
Courtesy of S. Rimbert</figcaption></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /><figcaption>ARAMIS projects
Courtesy of T. Venot</figcaption></figure>
## Examples of French BCI laboratories

- LORIA team (Nancy, France)

- Hybrid team (Rennes, France)

- Potioc team (Bordeaux, France)

- NERV team (Paris, France)

## Most salient disciplines:

- EEG Signal Processing & Machine Learning

- Clinical Neuroscience

- Human-Computer Interaction & BCI

- Computational Neuroscience

- Invasive BCI research

- Ethics

<figure><img src="image_33.png" /><figcaption>BCI labs localization in France</figcaption></figure>
<figure><img src="image_34.png" /><figcaption>P300 Speller</figcaption></figure>

<figure><img src="image_35.png" /><figcaption>Illustrations from BCI2000 website</figcaption></figure>

<figure><img src="image_36.png" /><figcaption>Adapted from [Lotte et al, 2015]</figcaption></figure>
<figure><img src="image_37.png" /></figure>
Motor execution vs Rest

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

Power decrease

<figure><img src="image_40.png" /></figure>

Motor imagery vs Rest

<figure><img src="image_41.png" /></figure>

<figure><img src="image_42.png" /></figure>

Desynchronization effect
(Pfurtscheller et al, 1999)
Underlying idea

Taking advantage of a neurophysiological phenomenon to establish a
communication between the brain and the computer
# AN EXAMPLE OF BCI EXPERIMENT

<figure><img src="image_43.png" /></figure>
## Behavioral properties

- Movement / preparation for movement : Event-related desynchronization (ERD) (Pfurtscheller, G, Lopes da Silva, FH, 1999)

- With relaxation/post-movement period : ERS

## Why using it in BCI ?

- Mu/Beta activity modulation by motor-imagery, a way to communicate

- Use of power spectra

- To establish this communication :

- Spatial selection

- Frequency selection

<figure><img src="image_44.png" /></figure>

Illustrations from BCI2000 website
<figure><img src="image_45.png" /></figure>
<figure><img src="image_46.png" /></figure>

$r^2$

<figure><img src="image_47.png" /></figure>

$r^2$
<figure><img src="image_48.png" /></figure>

Acquisition

Features extraction

Classification

$$ \text{cursor velocity} = \left( \sum_{i=1}^{N} \sum_{j=1}^{M=1} \text{Features}_{i,j} \text{Classifier}_{i,j,k} - \text{Offset} \right) \times \text{Gain} $$

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>

Time series

<figure><img src="image_52.png" /></figure>

Feedback

<figure><img src="image_53.png" /></figure>

Connected device

Control interface

<figure><img src="image_54.png" /></figure>

Classifier parameters
from training session

Power spectra

<figure><img src="image_55.png" /></figure>

Data from kth window/buffer

<figure><img src="image_56.png" /></figure>

<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>
<figure><img src="image_59.png" /></figure>
Great potential

<figure><img src="image_60.png" /></figure>

Poor usability

<figure><img src="image_61.png" /></figure>

(Ahn & Jun, 2015)

**Problem:** Current BCIs fail to detect the mental intentions in ~30% of users –
**BCI inefficiency** (Thompson, 2018)
# Controlling a BCI – A tale of two learners

## Machine-centered approaches

- Signal processing (Vidaurre et al, 2011)

- Classification algorithms (Lotte et al, 2018)

⇒ Rely on EEG signals

## User-centered approaches

- Neurophysiological patterns (Blankertz et al, 2010; Ahn et al, 2015)

- Human factors & Cognitive profile (Hammer et al, 2012; Jeunet et al, 2015)

⇒ Lack of reliable markers
<table><thead><tr><td>Type</td><td>Examples of methods used in BCI</td></tr></thead><tbody><tr><td>Linear classifiers</td><td><ul><li>Linear Discriminant Analysis (LDA) and its regularized version</li><li>Support Vector Machine (SVM)</li></ul></td></tr><tr><td>Neural networks (NN)</td><td><ul><li>Multi-Layer Perceptron (MLP)</li></ul></td></tr><tr><td>Non-linear Bayesian classifiers</td><td><ul><li>Bayes quadratic classifiers</li><li>Hidden Markov Models (HMMs)</li></ul></td></tr><tr><td>Nearest neighbour classifiers</td><td><ul><li>k-Nearest Neighbour (kNN)</li><li>Mahalanobis distance classifiers</li></ul></td></tr><tr><td>Classifier combinations</td><td><ul><li>Boosting</li><li>Voting</li><li>Stacking</li></ul></td></tr></tbody></table>

Current challenges:

- Low signal-to-noise ratio of EEG signals

- Non-stationarity over time of EEG signals

- Inter/intra-subject variabilities

- Limited amount of training data for calibration

- Low reliability and performance of current BCIs
- Adaptive classifiers

- Classification of matrices

- Classification of tensors

- Transfer learning

- Deep learning
# Open challenge – small datasets

## Motivations

- New subject/protocol – necessity to try out different experimental conditions

- Working with patients means often short sessions to avoid tiredness and lack of motivation

- Data change over time (non-stationary feature distributions) – cf rehabilitation training

## Methods

- LDA variants with domain-specific regularizations [Sosulski et al, 2021]

- Transfer learning from earlier sessions/other subjects/different tasks [Kobler et al, 2022]

- Deep Learning and data augmentation [Rommel et al, 2022]

## Resources

- Workshop dedicated to small datasets (BCI meeting 2023) by M. Tangermann – materials (slides and code)
CONCLUDING REMARKS
* BCI

* Promising tool for clinical applications

* Multidisciplinary domain

* Growing interest in the last few years with the AI

* BCI learning & inter-subject variability

* Improving the classifier / signal processing

* Improving instructions

* Finding (new) subject-related predictors

* Groups & events

* International: BCI society, international society

* Cybathlons: competitions to promote BCI and to test the finest algorithms with end users !

* In France: CORTICO, French association to promote BCI
To go further...

* Tools – with many tutorials

* Performing online experiments : [OpenViBE](https://openvibe.org), an Inria software

* Open datasets to test algorithms & check their replicability: [MOABB](https://moabb.org/)

* M/EEG data analysis : [MNE-Python](https://mne-py.org/)

* Classification tools : [Scikit-learn](https://scikit-learn.org)

* Extracting and selecting features in BCI: [HappyFeat](https://github.com/happyfeat/HappyFeat) an Inria software
# To go further...

* Tools – with many tutorials

* Performing online experiments : [OpenViBE](https://openvibe.org), an Inria software

* Open datasets to test algorithms & check their replicability: [MOABB](https://moabb.org/)

* M/EEG data analysis : [MNE-Python](https://mne-py.org/)

* Classification tools : [Scikit-learn](https://scikit-learn.org)

* Extracting and selecting features in BCI: [HappyFeat](https://www.happyfeat.org) an Inria software

* M/EEG

* [Chapter book](https://www.example.com)

* Origins of the signals,

* M/EEG experiments,

* Data analysis,

* Features extraction & selection,

* Brain disorders
## To go further...

* Tools – with many tutorials

* Performing online experiments : [OpenViBE](https://openvibe.org), an Inria software

* Open datasets to test algorithms & check their replicability: [MOABB](https://moabb.org/)

* M/EEG data analysis : [MNE-Python](https://mne-py.org/)

* Classification tools : [Scikit-learn](https://scikit-learn.org/)

* Extracting and selecting features in BCI: [HappyFeat](https://github.com/happyfeat/HappyFeat) an Inria software

* M/EEG

* [Chapter book](https://www.example.com)

* [Github repo](https://github.com)

* E/MEG data visualization,

* Data extraction (ERD/S),

* Classification
Thank you for your attention!

<figure><img src="image_62.png" /></figure>

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

marie-constance.corsi@inria.fr

arthur.desbois@inria.fr

<figure><img src="image_65.png" /></figure>

MConstanceCorsi
ADesboisDev

<figure><img src="image_66.png" /></figure>

mccorsi
AsteroidShrub