# Genetic algorithm applied to hyperparameter selection for fMRI neurofeedback scores prediction from EEG signals

Caroline Pinte, Claire Cury, Pierre Maurel

► To cite this version:

Caroline Pinte, Claire Cury, Pierre Maurel. Genetic algorithm applied to hyperparameter selection for fMRI neurofeedback scores prediction from EEG signals. IABM 2024 - Colloque Français d'Intelligence Artificielle en Imagerie Biomédicale, Mar 2024, Grenoble, France. pp.1-1, 2024. [hal-04529036](https://hal.inria.fr/hal-04529036)

HAL Id: hal-04529036

https://inria.hal.science/hal-04529036

Submitted on 2 Apr 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

# Can neural networks help improve portability of combined EEG – fMRI neurofeedback?

## Genetic algorithm applied to hyperparameter selection for fMRI neurofeedback scores prediction from EEG signals

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

Caroline Pinte

<figure><img src="image_8.png" /></figure>

@Caroline_Pinte

<figure><img src="image_9.png" /></figure>

caroline.pinte@irisa.fr

## Introduction

* Neurofeedback (NF) training is more effective with simultaneous EEG-fMRI acquisitions [1].

* However, the use of MRI is expensive and draining for the subject.

* So, we need to reduce its use.

→ We propose a method based on a one-dimensional convolutional neural network (1D CNN).

## Objective

Design a **neural network** model from simultaneous EEG-fMRI data to predict **fMRI NF** scores from **EEG** signals alone.

## Methods

1. **Data:** We acquired simultaneous EEG-fMRI data on 15 healthy subjects in a motor imagery context, described in [2]. This includes fMRI and EEG data for three NF runs of five minutes per subject, with their respective NF scores.

2. **Preprocessing:** We compute a supervised learning dataset, where a certain number of timesteps from a selection of EEG channels become one sample that has one fMRI NF score as target.

3. **Genetic search:** We use a genetic algorithm to search for the 1D CNN architecture hyperparameters.

<figure><img src="image_10.png" /></figure>

4. **Training:** We train the 1D CNN model. Since we have 15 subjects with 3 runs each, the training dataset consists of the first run of every subject, the validation dataset of the second runs and the test dataset of the third runs.

5. **Analysis:** After the model is trained, fMRI NF scores are predicted, and compared to the ground truths with the MSE and Pearson's correlation metrics. Then, we add EEG NF scores to our predictions to compare it with bimodal EEG-fMRI NF scores.

## Discussion

We are currently working on improving the correlations, which are negatively impacted by:

* Noisy nature of the EEG modality

* Lack of diversity in data

## Conclusion

* The interest of this method compared to the previous modeling [3] is to train a **single deep learning model** for **all subjects**, which would eventually allow the use of the model to predict these fMRI NF scores directly online.

* An important aspect of the issue is to **interpret this model**, in order to better understand links between EEG and fMRI.

<figure><img src="image_11.png" /><figcaption>**Fig. 1:** Architecture of the 1D CNN used to predict fMRI NF scores from raw EEG signals. The input sample consists of 2400 timesteps from 25 EEG channels selected above the motor area.</figcaption></figure>

## Results

<table><caption>### Predicting fMRI NF scores:</caption><thead><tr><td></td><td>fMRI NF scores</td><td>EEG-fMRI NF scores</td></tr></thead><tbody><tr><td>Mean MSE</td><td>0.13</td><td>0.41</td></tr><tr><td>Mean correlation</td><td>0.19</td><td>0.60</td></tr></tbody></table>

<figure><img src="image_12.png" /><figcaption>**Fig. 2:** Comparison between fMRI NF scores and fMRI NF scores prediction for two example test runs.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Fig. 3:** Comparison between EEG-fMRI NF scores and EEG NF scores ground truths to which fMRI NF scores predictions are added for the same two example test runs.</figcaption></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

1. Perronnet et al. "Unimodal versus bimodal EEG-fMRI neurofeedback of a motor imagery task." (2017)

2. Lioi et al. "Simultaneous EEG-fMRI during a neurofeedback task, a brain imaging dataset for multimodal data integration." (2020)

3. Cury et al. "A sparse EEG-informed fMRI model for hybrid EEG-fMRI neurofeedback prediction." (2020)

<figure><img src="image_17.png" /></figure>

Caroline Pinte¹, Claire Cury¹, Pierre Maurel¹

¹University of Rennes, CNRS, Inria, Inserm, IRISA UMR 6074, EMPENN — ERL U 1228, Rennes, France.