# Dynamic brain networks in motor imagery-based BCI

Tristan Venot, Arthur Desbois, Fabrizio de Vico Fallani

► To cite this version:

Tristan Venot, Arthur Desbois, Fabrizio de Vico Fallani. Dynamic brain networks in motor imagery-based BCI. Graz BCI Conference, Sep 2024, Graz, Austria. 10.3217/978-3-99161-014-4-031. hal-04854131

HAL Id: hal-04854131

https://inria.hal.science/hal-04854131v1

Submitted on 23 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# DYNAMIC BRAIN NETWORKS IN MOTOR IMAGERY-BASED BCI

T. Venot¹, A. Desbois¹, F. De Vico Fallani¹

¹Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière, F-75013, Paris, France

E-mail: tristan.venot@icm-institute.org

**ABSTRACT:** Using the interactions between brain regions has great potential as new features to discriminate between mental tasks for brain computer interface (BCI). Network approaches applied to electroencephalographic (EEG)-derived functional connectivity has been recently used to identify discriminating brain organizational features in offline classification scenarios. However how those network properties temporally vary during the task, is still poorly understood. *A contrario*, the dynamics of event related desynchronization/synchronization resulting from local power spectra is widely known and used for online motor imagery-based BCIs. Here, we explored the offline time-frequency properties of dynamic brain networks in two subjects performing three sessions of MI-BCI for the control of a robotic arm. Results were compared to standard time-frequency power spectra and discussed in light of future implementation for online scenarios.

## INTRODUCTION

When dealing with motor imagery (MI) BCI, one of the central challenges is finding features both discriminant and interpretable from a neurophysiological perspective[1, 2]. More over, they often depend on the correct execution of the mental task. Performing motor imagery is tricky, mental strategies may vary from one subject to the next and features need to be robust to such variabilities[3]. When performing motor imagery, the main information that can be transferred into a command is the event related desynchronization (ERD) responsible for power spectral density variations in the frequency bands associated to motor task, $\alpha$ (8-12 Hz) and $\beta$ (13-30 Hz)[4]. However, this information is local, and the brain is a dynamic system whose regions constantly interact together. To capture those interactions, network approaches applied to functional connectivity [5] shows to be relevant as it studies the spectral correlation between electrodes (in the sensor space); the use of connectivity allows to apply metrics coming from network science such as node strength (NS)[6], which captures the amount of connections a node (in our case an electrode) has with the others. The interactive information coming from connectivity has been explored to differentiate MI from rest[7] however its use using network metrics as feature for classification is still not employed despite having the potential

to complete the local information of the PSD. Here, we want to investigate how the node strength evolves over time during a motor imagery task with respect to resting state and how the parameters used to estimate the functional connectivity will have an impact on the performances of a classifier, Figure 1 gives a representation of the offline approach. To this end, we used EEG data recorded from two subjects during a MI-BCI control of a robotic arm in three different sessions[8]. Our preliminary results points towards different conclusions. First, as it is the case for ERD/ERS, there is a need to average over time points to obtain subsequent performances to temper effects of variability coming from the spectral/coherence estimation. Second, adding the information of node strength as a complementary feature for the classification tend to improve the performances. Third, the temporal dynamics of node strength shows to fluctuate more than ERD/ERS on short windows which makes them more difficult to interpret.

## MATERIALS AND METHODS

### *Experimentation:*

The two subjects (2 F), aged 24.5 ± 1.5 years, right-handed, provided informed consent and participated voluntarily in the protocol. The protocol was approved by Inria's national ethical committee as part of the BCIPRO protocol (authorization number 2021-35 - ref SICOERLE n°179). Experiments took place in the controlled environment of the EEG/MEG center within the neuroimaging core facility of the Paris Brain Institute.

A robotic arm facing the subjects reaches for objects on an augmented table used to show visual stimuli and neurofeedback (this table consists of a screen lying under a plexiglas, that displays visual cues directly underneath the objects to grasp). Subjects gaze towards a target to make the robot reach it and perform motor imagery for the robot to grasp the target. Each subject performs the control over the robot during three sessions where the robot moves before, during or after the subjects perform MI or rest. More details on the protocol can be found in the Braccio protocol [8].

In this study, we focus on motor imagery of the right hand closing and resting state trials lasting for three seconds. The acquisition uses BrainAmp 64 EEG cap, 500 Hz sampling frequency with TP9 and TP10 as reference
<figure><img src="image_2.png" /><figcaption>Figure 1: Representative view of the brain network dynamics through functional connectivity captured by node strength over time $s(t)$. $W_{ij}(t)$ corresponds to the imaginary coherence (IC) between two electrodes calculated for each instant $t$ by $IC_{ij}[f](t) = \frac{|\int (P_j[f])|}{\langle P[f] \cdot P_j[f] \rangle^{1/2}}$, with $P$ the power and $f$ the frequency, i and j are the couples of electrodes.</figcaption></figure>

and ground. During the experimentation, we train an Linear discriminant Analysis (LDA) classification algorithm on PSD features selected from the $R^2$ between MI and Rest trials for each subjects. These features are spectral amplitudes averages over trial for specific electrodes and frequency bins. To determine which features to select, we evaluated the highest $R^2$ values in the sensorimotor cortex (electrodes of lines C and CP) in the $\alpha$ and $\beta$. We selected 3 features for those two subjects: C3, CP3, C1 for subject 1 and C3, CP3, CP1 for subject 2. Each session is composed of two phases: first, 3 runs of control over the robot (Phase 1), then based on a training over the features of the 3 runs, a second phase of 2 runs (Phase 2), each run consists of 10 MI/10 Rest trials. Motor imagery and resting state trials lasted for 4 seconds, however only the last 3 seconds were kept to take into account the reaction time of subjects.

**Network metrics estimation:** To estimate the functional connectivity, we use imaginary coherence as it is more robust to volume conduction compared to spectral coherence[9]. Spectral properties were computed using Burg autoregressive (AR) method with a model order of 19, a frequency resolution of 0.5 Hz and make the parameters of windowing set to 0.33 s and overlap 53% to have a number of time points arbitrarily set to 18 points for 3 seconds of trial. The motivation of the choice of the filter order is based on two preliminary studies i) where we could identify a certain stability of the subject's patterns when we made the AR filter vary from 19 to 30 ii) using a particle swarm algorithm to optimize difference between

MI and rest for each subject, we identify 19 as the average filter. This operations were done using HappyFeat software (Inria) [10]. Based on the connectivity matrix, we use a local network metric called node strength (NS)[6], i.e. the average of all connection over each electrode in each condition. We then compute the average of the temporal node strength over the trials in Fig 2.

**Power spectrum estimation:** To estimate power spectral density, we also use Burg autoregressive method, set to a model of 19, a frequency resolution of 0.5 Hz and the window 0.25 s and 38% of overlap. For power spectrum only, a common average reference (CAR) was applied.

**Classification:** The classification algorithm used is a 2 class LDA, Phase 1 is used for the training and Phase 2 as a validation test. Brain features are electrodes at certain frequency bins used for the different sessions, they are selected after computing the $R^2$ statistical test between trials of MI and resting state and with neurophysiological relevance - in the motor cortex in the $\alpha$ or $\beta$ band). We compare performance obtained with two different training approaches. A first method consists in using each estimated spectral window for each trial (both in PSD and NS) as a feature. It means that for 30 trials lasting for 3 seconds with 18 points, we trains the algorithm on $30 \times 18 \times N \times M$ features per class (N being the number of electrodes and M of frequency bins). The second approach consists in averaging over time windows the features, meaning that the algorithm will be trained on $30 \times N \times M$ features per class. In a first step, we use the same features for PSD and NS, then we select specific features for NS corresponding to its specific $R^2$ map, and finally we combine the information coming from the two sources of information (NS and PSD).

## RESULTS

**Average Temporal dynamics over trials:** In a first step, we want to compare the trial-averaged evolution of NS compared to the PSD. As expected the separation in $\alpha$ and $\beta$ bands for PSD is clear and the amplitude corresponding to each condition is stable in time. Results are however more peculiar for NS, indeed, even though the evolution is averaged across trials, we still notice some strong oscillatory patterns that makes the separation between tasks more complex. From this, two different hypotheses can be made: first, the AR method used for the coherence estimation is more sensitive to noise in the context of short windows which forbids from using it to study the resulting node strength dynamics. The other hypothesis is that NS possesses properties different from the PSD on its temporal dynamic, ERD/ERS producing a stable pattern during MI/Rest task whereas node strength is intrinsically more oscillating.

We observe that Imaginary Coherence Node strength (NS -ImCoh) seems to follow same patterns as ERD/ERS as shown in Fig 2. Indeed we observe a decrease of Node strength from resting state to motor imagery mainly centered on 10-12Hz which is the expected behaviour. The
<figure><img src="image_3.png" /><figcaption>Figure 2: *Time-frequency maps of power and network-related dynamics:* Top: Relative difference of average spectral estimation across trials between motor imagery and resting state ($\frac{MI-Rest}{Rest}$) for subject 2, session 3, electrode C3 selected. Bottom: Relative difference of average imaginary coherence estimation across trials between motor imagery and resting state ($\frac{MI-Rest}{Rest}$) for subject 2, session 3, electrode C3 selected.</figcaption></figure>

most interesting detail is that the most intense moment of desynchronization (in PSD) corresponds to the peak of the NS difference. Also, the dynamic seems to separate more the tasks in the $\alpha$ band than in the $\beta$ band. Finally, it is necessary to mention that the difference is far superior with PSD than with NS-ImCoh, however, the subtle changes of dynamics might be more easily captured even though the noise in the computation limits our interpretation. It is to note that the PSD and the NS do not use the same parameters of windowing and overlap hence intrinsic differences, when the coherence is computed using the same parameters as PSD, patterns are even less visible. This is due to the fact that coherence is more sensitive to noise and requires more information hence wider windowing for it to reveal relevant information.

**Investigating inter trials and temporal variability through classification:** If we evaluate each time point as a single feature (of the pre-selected electrodes at a specific frequency bin) and train the algorithm on all the time points and compare it to the average feature over all the time points and train the algorithm on the average features, it appears clear that the performances favour the average features. In both cases (PSD and NS-imCOH), averaging tends to increase class differentiation. Quite surprisingly the difference of intensity between NS and PSD does not seem to have an effect on performances: indeed, PSD, while slightly superior in average, is not necessarily better than node strength when training on each

separate time point. Even though, the $R^2$ statistical test between MI and resting state trials shows higher scores for PSD than for NS. Altogether, linear machine learning algorithm (such as LDA) are sensitive to the noisy time points in PSD and NS, which stresses the importance for averaging along the trial to obtain good levels of discrimination.

**Combining information:** Two separate elements are to mention regarding the features and their resulting accuracies as shown in Fig 3. First, the node strength and PSD do not carry the same information: indeed if the same choices of features are made for both modalities (NS and PSD) and we base ourself on PSD, NS accuracies are lower. However, if other features are selected based on the specific NS $R^2$, NS based algorithm is showing accuracy improvement. Second, The interesting result we reveal is that if features of network and power are combined, performance always increase (in both training over each time windows and on average over trial). This tends to indicate the complementary nature of the two approaches, indeed while the local PSD information provides the majority of the information, the distributed information given by the network via the node strength has a role to play.

<figure><img src="image_4.png" /><figcaption>Figure 3: *Accuracy Comparison* LDA trained on motor imagery vs resting state trials of phase 1 and tested on phase 2 based on relevant neurophysiological features selected using the $R^2$ in NS and PSD either with all the time points as separate features as if it was instantaneous (blue) or the average features over the time points (red). Top: Subject 1 on session 3. Bottom: Subject 2 on session 2. NS:Node Strength, PSD:Power Spectral Density</figcaption></figure>
*What does network dynamics reveal?*: Studying networks behaviour to discriminate between mental tasks is relatively new and yet to be used in online BCI paradigm[7, 11]. Even though connectivity using imaginary coherence is known to decrease during a motor imagery task with respect to a resting state in certain frequency bands [5], the temporal dynamics associated are still poorly studied. Here we propose to explore with preliminary results what are those dynamics in order to know how they could be integrated to the BCI context. We find that node strength follows a similar trend as ERD/ERS even though the data is more sensitive to noise which limits the amount of interpretations regarding the neurophysiological process.

*Auto regressive method needs fine tuning to overcome its limitations:* Spectral estimation is always subjected to the rule of its estimator and the use of any estimator over short time windows is a challenge. AR method (Burg) has been studied so far in the domain of power spectral density [12, 13] but its use for computing coherence has remained marginal compared to welch or multitaper which are not suited for short time window estimation[14]. Our first results show that auto regressive method with short windows and overlap are far more sensitive to noise with coherence compared to PSD which limits the ways we can use such method. This is especially revealed by the important decrease of performance when taking each time point as a feature. To contrast this effect, two approaches could be used, if it is for offline analysis, averaging over trials could filter those noises even though the estimation could still be erroneous. Using larger time windows could be the definitive solution to limit this noisy computational phenomenon.

*Can we use continuous MI BCI within this framework?*:
One of the many problematic regarding BCI is the use of discrete or continuous feedback, which produce different effects on subjects[15, 16]. Continuous feedback can be used if the features they rely on are estimated on short time windows, the intrinsic noise of EEG makes. The logical follow up of our endeavour on the temporal dynamics of node strength is to interrogate the use of this feature in a feedback context. Here, our few results tend to demonstrate that continuous BCI which could use node strength as features will be highly impacted by the spectral estimation noise. Even though connectivity measures seems to improve performance when added to PSD, it is necessary to stress the use of an averaging over time series which means doing pseudo-continuous or discrete feedback.

## CONCLUSION

In this contribution, we investigate new forms of features that could be used for Motor imagery BCI relying on a network approach to EEG. We use data coming from subjects who perform during three sessions MI BCI to control the seizing of object with a robotic arm. We evaluate using imaginary coherence their network dynamics dur-

ing the MI task. We find out patterns similar to ERD/ERS but with some more subtle phenomenon which might be hidden due to the spectral estimator used. These results have to be tempered by the low amount of subjects and will require more of them to strengthen the conclusions. Nevertheless, based on those preliminary result, we advocate for the use of features averaged over time to maximize the differences that could be spotted and to use PSD and NS combined as they seem to be complimentary in the information they provide.

## ACKNOWLEDGEMENTS

FDVF acknowledges support from the European Research Council (ERC), Grant Agreement No. 864729 and from the Agence Innovation Defense DGA. The authors declare that they have no competing interests.

## REFERENCES

[1] Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002;113(6):767-791.

[2] Lebedev MA, Nicolelis MAL. Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. Physiological Reviews. 2017;97(2):767-837.

[3] Guillot A, Collet C. Contribution from neurophysiological and psychological methods to the study of motor imagery. Brain Research Reviews. 2005;50(2):387-397.

[4] Pfurtscheller G, Silva FH Lopes da. Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clin Neurophysiol. 1999;110(11):1842-1857.

[5] Cattai T, Colonnese S, Corsi MC, Bassett DS, Scarano G, De Vico Fallani F. Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2021;29:1168-1177.

[6] Gonzalez-Astudillo J, Cattai T, Bassignana G, Corsi MC, Fallani FDV. Network-based brain-computer interfaces: Principles and applications. J. Neural Eng. 2021;18(1):011001.

[7] Leeuwis N, Yoon S, Alimardani M. Functional Connectivity Analysis in Motor-Imagery Brain Computer Interfaces. Frontiers in Human Neuroscience. 2021;15.

[8] Venot T, Desbois A, Corsi MC, Hugueville L, Saint-Bauzel L, Fallani FDV. Intentional binding enhances hybrid BCI control. arXiv:2309.12195 [q-bio]. Oct. 2023. [Online]. Available: http://arxiv.org/abs/2309.12195 (visited on 02/26/2024).

[9] Nolte G, Bai O, Wheaton L, Mari Z, Vorbach S, Hallett M. Identifying true brain interaction from EEG data using the imaginary part of coherency. Clinical Neurophysiology. 2004;115(10):2292-2307.
[10] Desbois A, Venot T, De Vico Fallani F, Corsi MC. *HappyFeat*—An interactive and efficient BCI framework for clinical applications. Software Impacts. 2024;19:100610.

[11] Corsi MC, Chevallier S, Fallani FDV, Yger F. Functional Connectivity Ensemble Method to Enhance BCI Performance (FUCONE). IEEE Trans Biomed Eng. 2022;69(9):2826-2838.

[12] Bufalari S, Mattia D, Babiloni F, Mattiocco M, Marciani MG, Cincotti F. Autoregressive spectral analysis in Brain Computer Interface context. Conf Proc IEEE Eng Med Biol Soc. 2006;2006:3736-3739.

[13] Krusienski DJ, McFarland DJ, Wolpaw JR. An evaluation of autoregressive spectral estimation model order for brain-computer interface applications. Conf Proc IEEE Eng Med Biol Soc. 2006;2006:1323-1326.

[14] Diez P, Laciar E, Mut V, Avila E, Torres A. A Comparative Study of the Performance of Different Spectral Estimation Methods for Classification of Mental Tasks. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference. 2008;2008:1155-8.

[15] Carabalona R. Attitude of the subject towards feedback and its implications for BCI use in neurorehabilitation. In: 2010 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL 2010). Nov. 2010, 1-4.

[16] Carino-Escobar RI, Rodríguez-García ME, Carrillo-Mora P, Valdés-Cristerna R, Cantillo-Negrete J. Continuous versus discrete robotic feedback for brain-computer interfaces aimed for neurorehabilitation. Front Neurorobot. 2023;17:1015464.