# Electrophysiological brain imaging based on simulation-driven deep learning in the context of epilepsy

Zuyi Yu, Amar Kachenoura, Régine Le Bouquin Jeannès, Huazhong Shu, Paul Berraute, Anca Nica, Isabelle Merlet, Laurent Albera, Ahmad Karfoul

► To cite this version:

Zuyi Yu, Amar Kachenoura, Régine Le Bouquin Jeannès, Huazhong Shu, Paul Berraute, et al., Elec-trophysiological brain imaging based on simulation-driven deep learning in the context of epilepsy. NeuroImage, 2023, Neuroimage, 285, pp.120490. 10.1016/j.neuroimage.2023.120490. hal-04381963

HAL Id: hal-04381963

https://hal.science/hal-04381963

Submitted on 30 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Electrophysiological brain imaging based on simulation-driven deep learning in the context of epilepsy

Zuyi Yu, Amar Kachenoura, Régine Le Bouquin Jeannès, Huazhong Shu, Paul Berraute, Anca Nica, Isabelle Merlet, Laurent Albera, Ahmad Karfoul

► To cite this version:

Zuyi Yu, Amar Kachenoura, Régine Le Bouquin Jeannès, Huazhong Shu, Paul Berraute, et al., Elec-trophysiological brain imaging based on simulation-driven deep learning in the context of epilepsy. NeuroImage, 2023, Neuroimage, 285, pp.120490. 10.1016/j.neuroimage.2023.120490. hal-04381963

HAL Id: hal-04381963

https://hal.science/hal-04381963v1

Submitted on 30 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

# Electrophysiological brain imaging based on simulation-driven deep learning in the context of epilepsy

Zuyi Yu<sup>a,b,c,d</sup>, Amar Kachenoura<sup>c,d</sup>, Régine Le Bouquin Jeannès<sup>c,d</sup>, Huazhong Shu<sup>a,b,*</sup>, Paul Berraute<sup>f</sup>, Anca Nica<sup>c,e</sup>, Isabelle Merlet<sup>c,d</sup>, Laurent Albera<sup>c,d,**</sup>, Ahmad Karfoul<sup>c,d</sup>

<figure><img src="image_3.png" /></figure>

## ARTICLE INFO

### Keywords:

Electrophysiological source imaging

Inverse problem

Simulation-driven deep learning

Multi-scale strategy

## ABSTRACT

Identifying the location, the spatial extent and the electrical activity of distributed brain sources in the context of epilepsy through ElectroEncephaloGraphy (EEG) recordings is a challenging task because of the highly ill-posed nature of the underlying Electrophysiological Source Imaging (ESI) problem. To guarantee a unique solution, most existing ESI methods pay more attention to solve this inverse problem by imposing physiological constraints. This paper proposes an efficient ESI approach based on simulation-driven deep learning. Epileptic High-resolution 256-channels scalp EEG (Hr-EEG) signals are simulated in a realistic manner to train the proposed patient-specific model. More particularly, a computational neural mass model developed in our team is used to generate the temporal dynamics of the activity of each dipole while the forward problem is solved using a patient-specific three-shell realistic head model and the boundary element method. A Temporal Convolutional Network (TCN) is considered in the proposed model to capture local spatial patterns. To enable the model to observe the EEG signals from different scale levels, the multi-scale strategy is leveraged to capture the overall features and fine-grain features by adjusting the convolutional kernel size. Then, the Long Short-Term Memory (LSTM) is used to extract temporal dependencies among the computed spatial features. The performance of the proposed method is evaluated through three different scenarios of realistic synthetic interictal Hr-EEG data as well as on real interictal Hr-EEG data acquired in three patients with drug-resistant partial epilepsy, during their presurgical evaluation. A performance comparison study is also conducted with two other deep learning-based methods and four classical ESI techniques. The proposed model achieved a Dipole Localization Error (DLE) of 1.39 and Normalized Hamming Distance (NHD) of 0.28 in the case of one patch with SNR of 10 dB. In the case of two uncorrelated patches with an SNR of 10 dB, obtained DLE and NHD were respectively 1.50 and 0.28. Even in the more challenging scenario of two correlated patches with an SNR of 10 dB, the proposed approach still achieved a DLE of 3.74 and an NHD of 0.43. The results obtained on simulated data demonstrate that the proposed method outperforms the existing methods for different signal-to-noise and source configurations. The good behavior of the proposed method is also confirmed on real interictal EEG data. The robustness with respect to noise makes it a promising and alternative tool to localize epileptic brain areas and to reconstruct their electrical activities from EEG signals.

## 1. Introduction

ElectroEncephaloGraphy (EEG) is a non-invasive and safe tool to

measure synchronized brain activities, which uses an array of electrodes to record voltage fluctuations (Busch and VanRullen, 2010; Sun et al., 2022; Yu et al., 2022). The distribution of the scalp potential differences

<footnote><sup>a</sup> Laboratory of Image Science and Technology, Southeast University, Nanjing 210096, PR China</footnote>

<footnote><sup>b</sup> Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Nanjing 210096, PR China</footnote>

<footnote><sup>c</sup> University Rennes, INSERM, LTSI-UMR 1099, Rennes F-35042, France</footnote>

<footnote><sup>d</sup> Centre de Recherche en Information Biomédicale Sino-français (CRIBs), Rennes F-35042, France</footnote>

<footnote><sup>e</sup> Centre Hospitalier Universitaire (CHU) de Rennes, service de neurologie, pôle des neurosciences de Rennes, Rennes F-35042, France</footnote>

<footnote><sup>f</sup> Thales, Cholet F-49300, France</footnote>

<footnote>* Corresponding author at: Laboratory of Image Science and Technology, Southeast University, Nanjing 210096, PR China.</footnote>

<footnote>** Corresponding author at: University Rennes, INSERM, LTSI-UMR 1099, Rennes F-35042, France.
E-mail addresses: shu.list@seu.edu.cn (H. Shu), laurent.albera@univ-rennes1.fr (L. Albera).</footnote>
<figure><img src="image_2.png" /></figure>

# Electrophysiological brain imaging based on simulation-driven deep learning in the context of epilepsy

Zuyi Yu<sup>a,b,c,d</sup>, Amar Kachenoura<sup>c,d</sup>, Régine Le Bouquin Jeannès<sup>c,d</sup>, Huazhong Shu<sup>a,b,*</sup>, Paul Berraute<sup>f</sup>, Anca Nica<sup>c,e</sup>, Isabelle Merlet<sup>c,d</sup>, Laurent Albera<sup>c,d,**</sup>, Ahmad Karfoul<sup>c,d</sup>

<figure><img src="image_3.png" /></figure>

Check for
updates

## ARTICLE INFO

### Keywords:

Electrophysiological source imaging

Inverse problem

Simulation-driven deep learning

Multi-scale strategy

## ABSTRACT

Identifying the location, the spatial extent and the electrical activity of distributed brain sources in the context of epilepsy through ElectroEncephaloGraphy (EEG) recordings is a challenging task because of the highly ill-posed nature of the underlying Electrophysiological Source Imaging (ESI) problem. To guarantee a unique solution, most existing ESI methods pay more attention to solve this inverse problem by imposing physiological constraints. This paper proposes an efficient ESI approach based on simulation-driven deep learning. Epileptic High-resolution 256-channels scalp EEG (Hr-EEG) signals are simulated in a realistic manner to train the proposed patient-specific model. More particularly, a computational neural mass model developed in our team is used to generate the temporal dynamics of the activity of each dipole while the forward problem is solved using a patient-specific three-shell realistic head model and the boundary element method. A Temporal Convolutional Network (TCN) is considered in the proposed model to capture local spatial patterns. To enable the model to observe the EEG signals from different scale levels, the multi-scale strategy is leveraged to capture the overall features and fine-grain features by adjusting the convolutional kernel size. Then, the Long Short-Term Memory (LSTM) is used to extract temporal dependencies among the computed spatial features. The performance of the proposed method is evaluated through three different scenarios of realistic synthetic interictal Hr-EEG data as well as on real interictal Hr-EEG data acquired in three patients with drug-resistant partial epilepsy, during their presurgical evaluation. A performance comparison study is also conducted with two other deep learning-based methods and four classical ESI techniques. The proposed model achieved a Dipole Localization Error (DLE) of 1.39 and Normalized Hamming Distance (NHD) of 0.28 in the case of one patch with SNR of 10 dB. In the case of two uncorrelated patches with an SNR of 10 dB, obtained DLE and NHD were respectively 1.50 and 0.28. Even in the more challenging scenario of two correlated patches with an SNR of 10 dB, the proposed approach still achieved a DLE of 3.74 and an NHD of 0.43. The results obtained on simulated data demonstrate that the proposed method outperforms the existing methods for different signal-to-noise and source configurations. The good behavior of the proposed method is also confirmed on real interictal EEG data. The robustness with respect to noise makes it a promising and alternative tool to localize epileptic brain areas and to reconstruct their electrical activities from EEG signals.

## 1. Introduction

ElectroEncephaloGraphy (EEG) is a non-invasive and safe tool to

measure synchronized brain activities, which uses an array of electrodes to record voltage fluctuations (Busch and VanRullen, 2010; Sun et al., 2022; Yu et al., 2022). The distribution of the scalp potential differences

<footnote><sup>a</sup> Laboratory of Image Science and Technology, Southeast University, Nanjing 210096, PR China</footnote>

<footnote><sup>b</sup> Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Nanjing 210096, PR China</footnote>

<footnote><sup>c</sup> University Rennes, INSERM, LTSI-UMR 1099, Rennes F-35042, France</footnote>

<footnote><sup>d</sup> Centre de Recherche en Information Biomédicale Sino-français (CRIBs), Rennes F-35042, France</footnote>

<footnote><sup>e</sup> Centre Hospitalier Universitaire (CHU) de Rennes, service de neurologie, pôle des neurosciences de Rennes, Rennes F-35042, France</footnote>

<footnote><sup>f</sup> Thales, Cholet F-49300, France</footnote>

<footnote>* Corresponding author at: Laboratory of Image Science and Technology, Southeast University, Nanjing 210096, PR China.</footnote>

<footnote>** Corresponding author at: University Rennes, INSERM, LTSI-UMR 1099, Rennes F-35042, France.
E-mail addresses: shu.list@seu.edu.cn (H. Shu), laurent.albera@univ-rennes1.fr (L. Albera).</footnote>
can be visualized by establishing topographical maps. The high temporal resolution of EEG makes it attractive and leads to broad applications in the areas of diagnosis and brain-computer interface devices (Numata et al., 2019). However, the low spatial resolution of EEG is a significant technical limitation caused by the differences of conductivity between brain tissues (He et al., 2018). To address this limitation, Electrophysiological Source Imaging (ESI), which is also known as the EEG inverse problem, can help to improve the spatial resolution of EEG significantly and bridge the gap between scalp recordings and neural generators by inferring the active brain source distributions from the external measurements on the scalp (Gross et al., 2001; Bore et al., 2021; He et al., 2011; Baillet et al., 2001; Martínez-Vargas et al., 2019). Moreover, ESI can help neuroscientists to better understand the brain mechanisms and help practitioners to plan surgery of epileptogenic areas in drug-resistant patients (Liu et al., 2018). However, it is challenging to solve the EEG inverse problem without adding further constraints as it is highly ill-posed and different source distributions can produce same topographies of EEG activity on the scalp (He et al., 1987; Scherg and Von Cramon, 1986).

Previous ESI methods proposed in the EEG community have focused on regularization techniques or a priori assumptions to face limitations induced by the relatively small number of electrodes and the volume conduction (Pascual-Marqui, 2002; Dale and Sereno, 1993; Van Veen et al., 1997). Specifically, the equivalent current dipole model requires making an assumption on the number of sources, which is suitable to estimate the locations of focal cortical activities (Kiebel et al., 2008). But this model is limited in generating detectable scalp potentials and providing information on source extents due to its reliance on single or few dipoles. In contrast, the distributed dipole model, which enforces regularization constraints to restrict the solution space, is a widely-used and more realistic approach (Liu et al., 2015; Cai et al., 2018). This model divides the cortical surface into thousands of triangles, which are represented by dipoles. This dipolar grid is dedicated to reconstruct the 3D distribution of the neural activity based on the assumption that EEG measures the summation of the synchronous activity of millions of neurons, which is modeled by one or several clusters (patches) of brain dipoles (Wu et al., 2015; Abeyratne et al., 2001).

To obtain a unique solution, various methods employing regularization techniques in both deterministic and probabilistic frameworks have been proposed. The foremost and most commonly used distributed inverse solution is the Minimum Norm Estimates (MNE). This approach, MNE (Hämaläinen and Ilmoiemi, 1994), involves the utilization of the L2-norm as a regularizer to meet the hypothesis of minimizing the sources with minimum energy. However, MNE tends to produce sources situated on the cortical surface while overlook deeper sources. A variation of MNE, known as Weighted MNE (wMNE) (Dale and Sereno, 1993), addressed this issue by weighting the sources differently according to their respective distance from the scalp electrodes. Another variation of the MNE solution is the Low Resolution Electromagnetic Tomography (LORETA) (Pascual-Marqui et al., 1994) method, where a Tikhonov regularization-based on the Laplacian operator of the sources was employed. This resulted in a smoother source distribution. In addition, Linearly Constrained Minimum Variance (LCMV) beamforming (Van Veen et al., 1997), which is the most widely employed beamforming technique, has also been utilized to tackle the ESI problem. This method functions as a spatial filter, enabling the passage of signals originating from specific brain locations, typically represented by dipoles, while simultaneously attenuating signals from other locations. However, the LCMV approach cannot accurately locate correlated sources. Regarding the Bayesian framework, the Maximum Entropy on the Mean (MEM) technique (Grova et al., 2006) can be accounted for. In fact, the MEM technique aims to identify the most spread-out distribution by maximizing entropy. However, MEM does not take into account the valuable temporal information in EEG signals.

With the rapid development of computing power, the deep learning concept has been widely used in various fields including image

classification (Krizhevsky et al., 2017), natural language processing (Nadkarni et al., 2011) and biomedical signals analysis (Li et al., 2020). But such an approach seems to have attracted little attention from researchers in the field of ESI. The deep learning concept, as a data-driven approach, allows us to capture the sophisticated mapping between source and scalp spaces through large-scale training data (Awan et al., 2019). Although the training process is time consuming to obtain suitable weights for the ESI task, this technique can predict the position of active sources efficiently and quickly (Zorzos et al., 2021). Recently, a model named Deep Brain Neural Network (DeepBraNNet), based on the Long Short-Term Memory (LSTM) architecture, has been used to solve the EEG inverse problem, which avoids the explicit inversion of the lead field matrix and outperforms the classical methods (Bore et al., 2021). But a single type of LSTM network may lack sufficient robustness for complex noisy EEG signals, whereas such a robustness is an essential property in clinical applications. Sun et al. (2022) proposed a Deep learning-based Source Imaging Framework (DeepSIF) consisting of multiple fully connected layers and LSTM, which shows an effective performance for source imaging.

This paper proposes a novel simulation-driven deep learning-based method, named MS-ESI (Multi-Scale network for Electrophysiological Source Imaging), which can precisely delineate the epileptogenic zone. Epileptic High-resolution 256-channels scalp EEG (Hr-EEG) signals are simulated in a realistic manner to train the proposed patient-specific model. More particularly, a computational neural mass model developed in our team is used to generate the temporal dynamics of the activity of each dipole while the forward problem is solved using a patient-specific three-shell realistic head model and the boundary element method. A Temporal Convolutional Network (TCN) is considered in the proposed model to capture local spatial patterns. To enable the model to observe the EEG signals from different scale levels, the multi-scale strategy is leveraged to capture the overall features and fine-grain features by adjusting the convolutional kernel size. Then, the LSTM is used to extract temporal dependencies among the computed spatial features. The performance of the proposed method is evaluated through three different scenarios of realistic synthetic interictal Hr-EEG data as well as on real interictal Hr-EEG data acquired in three patients with drug-resistant partial epilepsy, during their presurgical evaluation. A performance comparison study is also conducted with two other deep learning-based methods, namely DeepSIF (Sun et al., 2022) and DeepBraINNet (Bore et al., 2021), three classical regularized minimum norm estimates, namely SISSY (Source Imaging based on Structured SparsitY) (Becker et al., 2017), wMNE (weighted Minimum Norm Estimate) (Dale and Sereno, 1993) and sLORETA (standardized Low Resolution Electromagnetic Tomography) (Pascual-Marqui, 2002), and the probabilistic source imaging technique cMEM (coherent Maximum Entropy on the Mean) (Chowdhury et al., 2013). The results obtained on simulated data demonstrate that the proposed method outperforms the existing methods for different signal-to-noise and source configurations. The good behavior of the proposed method is also confirmed on real interictal EEG data. The robustness with respect to noise makes it a promising and alternative tool to localize epileptic brain areas and to reconstruct their electrical activities from EEG signals.

The remainder of this paper is organized as follows: The proposed method is described in Section II. Section III is devoted to the experimental results. The efficiency of the proposed method is discussed in Section IV and Section V gives a conclusion of this work.

## 2. Material and methods

### 2.1. Problem formulation

EEG is a noninvasive method to record the electrical activity of the brain using an array of N sensors placed on the scalp. The electric potential generated by pyramidal cortical neurons propagates to the scalp surface through the head which is a conducting medium, and the current
can be visualized by establishing topographical maps. The high temporal resolution of EEG makes it attractive and leads to broad applications in the areas of diagnosis and brain-computer interface devices (Numata et al., 2019). However, the low spatial resolution of EEG is a significant technical limitation caused by the differences of conductivity between brain tissues (He et al., 2018). To address this limitation, Electrophysiological Source Imaging (ESI), which is also known as the EEG inverse problem, can help to improve the spatial resolution of EEG significantly and bridge the gap between scalp recordings and neural generators by inferring the active brain source distributions from the external measurements on the scalp (Gross et al., 2001; Bore et al., 2021; He et al., 2011; Baillet et al., 2001; Martínez-Vargas et al., 2019). Moreover, ESI can help neuroscientists to better understand the brain mechanisms and help practitioners to plan surgery of epileptogenic areas in drug-resistant patients (Liu et al., 2018). However, it is challenging to solve the EEG inverse problem without adding further constraints as it is highly ill-posed and different source distributions can produce same topographies of EEG activity on the scalp (He et al., 1987; Scherg and Von Cramon, 1986).

Previous ESI methods proposed in the EEG community have focused on regularization techniques or a priori assumptions to face limitations induced by the relatively small number of electrodes and the volume conduction (Pascual-Marqui, 2002; Dale and Sereno, 1993; Van Veen et al., 1997). Specifically, the equivalent current dipole model requires making an assumption on the number of sources, which is suitable to estimate the locations of focal cortical activities (Kiebel et al., 2008). But this model is limited in generating detectable scalp potentials and providing information on source extents due to its reliance on single or few dipoles. In contrast, the distributed dipole model, which enforces regularization constraints to restrict the solution space, is a widely-used and more realistic approach (Liu et al., 2015; Cai et al., 2018). This model divides the cortical surface into thousands of triangles, which are represented by dipoles. This dipolar grid is dedicated to reconstruct the 3D distribution of the neural activity based on the assumption that EEG measures the summation of the synchronous activity of millions of neurons, which is modeled by one or several clusters (patches) of brain dipoles (Wu et al., 2015; Abeyratne et al., 2001).

To obtain a unique solution, various methods employing regularization techniques in both deterministic and probabilistic frameworks have been proposed. The foremost and most commonly used distributed inverse solution is the Minimum Norm Estimates (MNE). This approach, MNE (Hämaläinen and Ilmoiemi, 1994), involves the utilization of the L2-norm as a regularizer to meet the hypothesis of minimizing the sources with minimum energy. However, MNE tends to produce sources situated on the cortical surface while overlook deeper sources. A variation of MNE, known as Weighted MNE (wMNE) (Dale and Sereno, 1993), addressed this issue by weighting the sources differently according to their respective distance from the scalp electrodes. Another variation of the MNE solution is the Low Resolution Electromagnetic Tomography (LORETA) (Pascual-Marqui et al., 1994) method, where a Tikhonov regularization-based on the Laplacian operator of the sources was employed. This resulted in a smoother source distribution. In addition, Linearly Constrained Minimum Variance (LCMV) beamforming (Van Veen et al., 1997), which is the most widely employed beamforming technique, has also been utilized to tackle the ESI problem. This method functions as a spatial filter, enabling the passage of signals originating from specific brain locations, typically represented by dipoles, while simultaneously attenuating signals from other locations. However, the LCMV approach cannot accurately locate correlated sources. Regarding the Bayesian framework, the Maximum Entropy on the Mean (MEM) technique (Grova et al., 2006) can be accounted for. In fact, the MEM technique aims to identify the most spread-out distribution by maximizing entropy. However, MEM does not take into account the valuable temporal information in EEG signals.

With the rapid development of computing power, the deep learning concept has been widely used in various fields including image

classification (Krizhevsky et al., 2017), natural language processing (Nadkarni et al., 2011) and biomedical signals analysis (Li et al., 2020). But such an approach seems to have attracted little attention from researchers in the field of ESI. The deep learning concept, as a data-driven approach, allows us to capture the sophisticated mapping between source and scalp spaces through large-scale training data (Awan et al., 2019). Although the training process is time consuming to obtain suitable weights for the ESI task, this technique can predict the position of active sources efficiently and quickly (Zorzos et al., 2021). Recently, a model named Deep Brain Neural Network (DeepBraNNet), based on the Long Short-Term Memory (LSTM) architecture, has been used to solve the EEG inverse problem, which avoids the explicit inversion of the lead field matrix and outperforms the classical methods (Bore et al., 2021). But a single type of LSTM network may lack sufficient robustness for complex noisy EEG signals, whereas such a robustness is an essential property in clinical applications. Sun et al. (2022) proposed a Deep learning-based Source Imaging Framework (DeepSIF) consisting of multiple fully connected layers and LSTM, which shows an effective performance for source imaging.

This paper proposes a novel simulation-driven deep learning-based method, named MS-ESI (Multi-Scale network for Electrophysiological Source Imaging), which can precisely delineate the epileptogenic zone. Epileptic High-resolution 256-channels scalp EEG (Hr-EEG) signals are simulated in a realistic manner to train the proposed patient-specific model. More particularly, a computational neural mass model developed in our team is used to generate the temporal dynamics of the activity of each dipole while the forward problem is solved using a patient-specific three-shell realistic head model and the boundary element method. A Temporal Convolutional Network (TCN) is considered in the proposed model to capture local spatial patterns. To enable the model to observe the EEG signals from different scale levels, the multi-scale strategy is leveraged to capture the overall features and fine-grain features by adjusting the convolutional kernel size. Then, the LSTM is used to extract temporal dependencies among the computed spatial features. The performance of the proposed method is evaluated through three different scenarios of realistic synthetic interictal Hr-EEG data as well as on real interictal Hr-EEG data acquired in three patients with drug-resistant partial epilepsy, during their presurgical evaluation. A performance comparison study is also conducted with two other deep learning-based methods, namely DeepSIF (Sun et al., 2022) and DeepBraINNet (Bore et al., 2021), three classical regularized minimum norm estimates, namely SISSY (Source Imaging based on Structured SparsitY) (Becker et al., 2017), wMNE (weighted Minimum Norm Estimate) (Dale and Sereno, 1993) and sLORETA (standardized Low Resolution Electromagnetic Tomography) (Pascual-Marqui, 2002), and the probabilistic source imaging technique cMEM (coherent Maximum Entropy on the Mean) (Chowdhury et al., 2013). The results obtained on simulated data demonstrate that the proposed method outperforms the existing methods for different signal-to-noise and source configurations. The good behavior of the proposed method is also confirmed on real interictal EEG data. The robustness with respect to noise makes it a promising and alternative tool to localize epileptic brain areas and to reconstruct their electrical activities from EEG signals.

The remainder of this paper is organized as follows: The proposed method is described in Section II. Section III is devoted to the experimental results. The efficiency of the proposed method is discussed in Section IV and Section V gives a conclusion of this work.

## 2. Material and methods

### 2.1. Problem formulation

EEG is a noninvasive method to record the electrical activity of the brain using an array of N sensors placed on the scalp. The electric potential generated by pyramidal cortical neurons propagates to the scalp surface through the head which is a conducting medium, and the current
lead to electric potential differences, which are collected by surface electrodes. A specific number of simultaneously active neuronal populations, which can be modeled by a grid of contiguous dipoles located on the cortex surface, is necessary to achieve a measurable signal with sufficient amplitude on the scalp surface. The EEG forward problem is commonly defined as the process of calculating the scalp potentials generated by pyramidal neurons, which can be formulated as:

$$X = GS \qquad (1)$$

where $X = [x_1, ..., x_T, ..., x_T] \in \mathbb{R}^{N \times T}$ is the observed EEG signals recorded by N electrodes at T time points, $S = [s_1, ..., s_T, ..., s_T] \in \mathbb{R}^{M \times T}$ denotes the source signals on the cortical surface, M denotes the number of dipoles, $G \in \mathbb{R}^{N \times M}$ gives the lead field matrix which describes the complex relationship between the source signal at a certain location and the observed scalp signals.

An extended source is defined as the union of clusters of grid dipoles, called patches, with fully correlated signals. We will consider several extended sources if they are not fully correlated. To differentiate the extended sources of interest and the noisy background activity, the data model can be rewritten as:

$$x(t) = \sum_{k=\Omega_c} s_k(t)g_k + \sum_{c=\Omega_b} s_c(t)g_c = X_c + X_b \qquad (2)$$

where $x(t) \in \mathbb{R}^T$ is the observed EEG signal recorded at t time point, $\Omega_c$ denotes the index set of dipoles forming the eth extended source, $g_k$ and $s_k$ represent the kth column of G and the kth row of S, respectively. Similarly, $\Omega_b$ denotes the index set of dipoles forming the background area, $g_c$ and $s_c$ represent the cth column of G and the cth row of S, respectively. The matrix $X_c$ denotes the EEG data generated from the extended sources of interest and $X_b$ represents the background activity. The adjective “extended” will be omitted in the sequel for the sake of simplicity.

## 2.2. Multi-Scale network for Electrophysiological Source Imaging (MS-ESI)

Our ultimate goal is to establish a patient-specific deep learning model for brain source imaging in the context of epilepsy using HR-EEG recordings. While real HR-EEG, i.e. 256 channels, data can be intuitively considered for training the model in a supervised learning manner, there are several challenges to overcome: (i) the lack of simultaneous

acquisition of HR-EEG and Stereotactic EEG (SEEG) in clinical routine and (ii) the lack of a precise location of the epileptic sources in clinical practice even supposing that such a joint acquisition is supposed to be accessible. To overcome these limitations, one potential alternative solution worth considering is the use of simulated HR-EEG recordings that are generated in a realistic manner. This approach can provide a larger and more diverse dataset for training the deep learning model, compensating for the scarcity of real epileptic EEG data. By simulating EEG recordings that closely resemble real-world scenarios, we can improve the training process and enhance the ability of the model to adjust well to different source positions and cortical areas. To sum up, fully simulated HR-EEG recordings representing precise locations of the underlying epileptic sources in the cortical surface were used here together with simulated intracerebral electrical activity to train the proposed patient-specific model as illustrated in Fig. 1 (Training stage). More precisely, realistic interictal epileptiform spikes, as relevant biomarkers of epileptic zones, generated using a neural mass model, developed in our team for several years (Wendling et al., 2002; Wendling et al., 2005), were considered as source neural activities. These simulated sources, along with a patient-specific head model, were used to solve the forward problem in EEG. Hence, the obtained simulated patient-specific HR-EEG data and the simulated intracerebral electrical activity were considered for the training step. Finally, the clinical epileptic HR-EEG data were employed to evaluate the performance of the model as also shown in Fig. 1 (Testing stage). However, it is crucial to bear in mind that when dealing with a new patient, the only necessary adjustment is to replace the current head model with the patient's own, as the simulated intracerebral electrical activity remains consistent, regardless of the specific patient under study. Although the training process of deep learning is relatively time-consuming, it is an offline process, and once the model is trained; it can be saved for processing any new data. More comprehensive information regarding the generation of training data are provided in Section 2.3.

The proposed MS-ESI model, detailed in Fig. 2, for brain source imaging from HR-EEG recordings comprises three essential stages: spatial feature extraction using TCN, temporal dependency revealing through LSTM and linear regression using dense neural network. It relies mainly on the combination of TCN and LSTM to assess reliable characterization of each time sample and to emphasize their temporal correlation in the feature space. In fact, the combination of TCN and LSTM in the proposed model results in a hierarchical representation of EEG signals: TCN captures local spatial patterns, while LSTM extracts temporal

<figure><img src="image_4.png" /><figcaption>Fig. 1. The global patient-specific flow of the proposed ESI procedure.</figcaption></figure>
lead to electric potential differences, which are collected by surface electrodes. A specific number of simultaneously active neuronal populations, which can be modeled by a grid of contiguous dipoles located on the cortex surface, is necessary to achieve a measurable signal with sufficient amplitude on the scalp surface. The EEG forward problem is commonly defined as the process of calculating the scalp potentials generated by pyramidal neurons, which can be formulated as:

$$X = GS \qquad (1)$$

where $X = [x_1, ..., x_T, ..., x_T] \in \mathbb{R}^{N \times T}$ is the observed EEG signals recorded by N electrodes at T time points, $S = [s_1, ..., s_T, ..., s_T] \in \mathbb{R}^{M \times T}$ denotes the source signals on the cortical surface, M denotes the number of dipoles, $G \in \mathbb{R}^{N \times M}$ gives the lead field matrix which describes the complex relationship between the source signal at a certain location and the observed scalp signals.

An extended source is defined as the union of clusters of grid dipoles, called patches, with fully correlated signals. We will consider several extended sources if they are not fully correlated. To differentiate the extended sources of interest and the noisy background activity, the data model can be rewritten as:

$$x(t) = \sum_{k=\Omega_c} s_k(t)g_k + \sum_{c=\Omega_b} s_c(t)g_c = X_c + X_b \qquad (2)$$

where $x(t) \in \mathbb{R}^T$ is the observed EEG signal recorded at t time point, $\Omega_c$ denotes the index set of dipoles forming the eth extended source, $g_k$ and $s_k$ represent the kth column of G and the kth row of S, respectively. Similarly, $\Omega_b$ denotes the index set of dipoles forming the background area, $g_c$ and $s_c$ represent the cth column of G and the cth row of S, respectively. The matrix $X_c$ denotes the EEG data generated from the extended sources of interest and $X_b$ represents the background activity. The adjective “extended” will be omitted in the sequel for the sake of simplicity.

## 2.2. Multi-Scale network for Electrophysiological Source Imaging (MS-ESI)

Our ultimate goal is to establish a patient-specific deep learning model for brain source imaging in the context of epilepsy using HR-EEG recordings. While real HR-EEG, i.e. 256 channels, data can be intuitively considered for training the model in a supervised learning manner, there are several challenges to overcome: (i) the lack of simultaneous

acquisition of HR-EEG and Stereotactic EEG (SEEG) in clinical routine and (ii) the lack of a precise location of the epileptic sources in clinical practice even supposing that such a joint acquisition is supposed to be accessible. To overcome these limitations, one potential alternative solution worth considering is the use of simulated HR-EEG recordings that are generated in a realistic manner. This approach can provide a larger and more diverse dataset for training the deep learning model, compensating for the scarcity of real epileptic EEG data. By simulating EEG recordings that closely resemble real-world scenarios, we can improve the training process and enhance the ability of the model to adjust well to different source positions and cortical areas. To sum up, fully simulated HR-EEG recordings representing precise locations of the underlying epileptic sources in the cortical surface were used here together with simulated intracerebral electrical activity to train the proposed patient-specific model as illustrated in Fig. 1 (Training stage). More precisely, realistic interictal epileptiform spikes, as relevant biomarkers of epileptic zones, generated using a neural mass model, developed in our team for several years (Wendling et al., 2002; Wendling et al., 2005), were considered as source neural activities. These simulated sources, along with a patient-specific head model, were used to solve the forward problem in EEG. Hence, the obtained simulated patient-specific HR-EEG data and the simulated intracerebral electrical activity were considered for the training step. Finally, the clinical epileptic HR-EEG data were employed to evaluate the performance of the model as also shown in Fig. 1 (Testing stage). However, it is crucial to bear in mind that when dealing with a new patient, the only necessary adjustment is to replace the current head model with the patient's own, as the simulated intracerebral electrical activity remains consistent, regardless of the specific patient under study. Although the training process of deep learning is relatively time-consuming, it is an offline process, and once the model is trained; it can be saved for processing any new data. More comprehensive information regarding the generation of training data are provided in Section 2.3.

The proposed MS-ESI model, detailed in Fig. 2, for brain source imaging from HR-EEG recordings comprises three essential stages: spatial feature extraction using TCN, temporal dependency revealing through LSTM and linear regression using dense neural network. It relies mainly on the combination of TCN and LSTM to assess reliable characterization of each time sample and to emphasize their temporal correlation in the feature space. In fact, the combination of TCN and LSTM in the proposed model results in a hierarchical representation of EEG signals: TCN captures local spatial patterns, while LSTM extracts temporal

<figure><img src="image_4.png" /><figcaption>Fig. 1. The global patient-specific flow of the proposed ESI procedure.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Fig. 2.** The schematic diagram of the proposed model. TCN (#K, KS, D) stands for a TCN layer with #K, KS and D denoting the number of used kernels, the kernel size and the set of used dilatation factors (here $D = \{1, 2, 4, 8, 16\}$), respectively.</figcaption></figure>

dependencies among these spatial features. This integration enables the model to enhance its ability to adjust and to improve the overall performance of brain source imaging.

TCN is a type of deep learning architecture that is specifically designed for sequential data processing such as time series or natural language sequences. TCN captures long-term trends using a hierarchy of temporal convolutional filters. Two types of TCN can be found: (i) Encoder-Decoder TCN (ED-TCN) which uses temporal convolutions, pooling, and up-sampling to capture long-range temporal patterns and (ii) dilated TCN where dilated convolutions and skip connections across layers are used instead of pooling and up-sampling. Dilated convolutions apply filters across larger regions by skipping input values with a specific step defined using a dilatation factor. This approach which is used in our proposed model efficiently handles large amounts of data sequence (Lea et al., 2017). Regarding the LSTM model, it is a type of recurrent neural network that predicts sequences using feedback connections. It consists of a chain of repeated cells with three interconnected layers: forget gate, input gate and output gate. These gates are composed of a sigmoid neural network layer, followed by point-wise multiplication operations (Hochreiter and Schmidhuber, 1997). This design allows the LSTM model to selectively remove or add information to the flow of data through each cell. The LSTM cell's first layer is the "forget gate layer", which determines whether certain information should be disregarded. The subsequent "input gate layer," coupled with a "tanh" layer, filters and updates new input data, contributing to the cell state's update. The "tanh" layer combines information from previous layers and modifies each state value accordingly. Lastly, the "output gate layer" produces the final output based on the cell state (Hochreiter and Schmidhuber, 1997).

More precisely, the proposed neural network structure consists, for each time sample in the Hr-EEG recordings of length $T$, of a set of three TCN layers, which are employed to perform a multi-scale spatial feature extraction using a set of 64 kernels with varying size: 3, 5 and 7 as illustrated in Fig. 2. These multiple filters allow us to capture both local and global spatial features, through dilated spatial convolutions with zero-padding and multiple dilatation factors taken in the set $D = \{1, 2, 4, 8, 16\}$. Next, the set of generated feature maps for each time sample are concatenated to form a sample feature map of size $256 \times 64$. Finally, to encompass the entire EEG input signal, the $T$ sample feature maps are stacked in such a way a global feature map of size $256 \times 192 \times T$ is obtained. As far as the temporal-dependency extraction stage is

concerned, a LSTM model with $T$ LSTM cells (each cell comprises 128 units) are used to capture a long-range temporal dependency among the $T$ computed sample feature maps. The LSTM output is then introduced to a dense layer with a number of neurons equals to the number $D$ of grid dipoles on the cortical surface (source space) as shown in Fig. 2. The Minimum Squared Error (MSE) criterion is exploited as a cost function during the backpropagation phase using the Adam algorithm, with learning rate of 0.001, as optimizer. Table 1 recapitulates the key parameters of the proposed model.

To avoid the model overfitting a 5% dropout layer was incorporated into our designed model. Besides, 10% of the training samples were used as a validation set to monitor the model's performance during the training process. Together with the dropout method, an early stop strategy was also used to prevent the model overfitting.

## 2.3. Simulated data generation

Realistic synthetic data are simulated in order to train the MS-ESI model and to quantitatively evaluate the performance of our approach in particular context of extended interictal epileptic sources. We generated a High-resolution 256-channels scalp EEG (Hr-EEG) using a three-shell realistic head model including the brain, the skull and the scalp, whose conductivity are $0.33 \Omega^{-1}m^{-1}$, $0.00825 \Omega^{-1}m^{-1}$, and $0.33 \Omega^{-1}m^{-1}$, respectively. The high-resolution 15,002 vertices cortical mesh was built from the segmentation of the gray-white matter interface from a patient 3D T1-weighted MRI using Brain Visa software (Rivière et al., 2009). Each vertex of the mesh has been associated with an elementary

<table><caption>**Table 1**
The key parameters of the proposed model.</caption><thead><tr><td>Configuration</td><td>Value</td></tr></thead><tbody><tr><td>Dilated conv1D</td><td>Kernel number=64; Kernel_size=1 × 3; 1 × 5; 1 × 7</td></tr><tr><td>Dilation factors</td><td>1, 2, 4, 8, and 16</td></tr><tr><td>Dropout layer</td><td>0.05</td></tr><tr><td>LSTM</td><td>Unit=128</td></tr><tr><td>Fully connected layer</td><td>Neurons_number = 128; 15,002</td></tr><tr><td>Learning rate</td><td>0.001</td></tr><tr><td>Batch_size</td><td>64</td></tr><tr><td>Epochs</td><td>100</td></tr><tr><td>Optimizer</td><td>Adam</td></tr><tr><td>Loss</td><td>MSE</td></tr><tr><td>Monitor-metric</td><td>val_loss</td></tr></tbody></table>
<figure><img src="image_5.png" /><figcaption>**Fig. 2.** The schematic diagram of the proposed model. TCN (#K, KS, D) stands for a TCN layer with #K, KS and D denoting the number of used kernels, the kernel size and the set of used dilatation factors (here $D = \{1, 2, 4, 8, 16\}$), respectively.</figcaption></figure>

dependencies among these spatial features. This integration enables the model to enhance its ability to adjust and to improve the overall performance of brain source imaging.

TCN is a type of deep learning architecture that is specifically designed for sequential data processing such as time series or natural language sequences. TCN captures long-term trends using a hierarchy of temporal convolutional filters. Two types of TCN can be found: (i) Encoder-Decoder TCN (ED-TCN) which uses temporal convolutions, pooling, and up-sampling to capture long-range temporal patterns and (ii) dilated TCN where dilated convolutions and skip connections across layers are used instead of pooling and up-sampling. Dilated convolutions apply filters across larger regions by skipping input values with a specific step defined using a dilatation factor. This approach which is used in our proposed model efficiently handles large amounts of data sequence (Lea et al., 2017). Regarding the LSTM model, it is a type of recurrent neural network that predicts sequences using feedback connections. It consists of a chain of repeated cells with three interconnected layers: forget gate, input gate and output gate. These gates are composed of a sigmoid neural network layer, followed by point-wise multiplication operations (Hochreiter and Schmidhuber, 1997). This design allows the LSTM model to selectively remove or add information to the flow of data through each cell. The LSTM cell's first layer is the "forget gate layer", which determines whether certain information should be disregarded. The subsequent "input gate layer," coupled with a "tanh" layer, filters and updates new input data, contributing to the cell state's update. The "tanh" layer combines information from previous layers and modifies each state value accordingly. Lastly, the "output gate layer" produces the final output based on the cell state (Hochreiter and Schmidhuber, 1997).

More precisely, the proposed neural network structure consists, for each time sample in the Hr-EEG recordings of length $T$, of a set of three TCN layers, which are employed to perform a multi-scale spatial feature extraction using a set of 64 kernels with varying size: 3, 5 and 7 as illustrated in Fig. 2. These multiple filters allow us to capture both local and global spatial features, through dilated spatial convolutions with zero-padding and multiple dilatation factors taken in the set $D = \{1, 2, 4, 8, 16\}$. Next, the set of generated feature maps for each time sample are concatenated to form a sample feature map of size $256 \times 64$. Finally, to encompass the entire EEG input signal, the $T$ sample feature maps are stacked in such a way a global feature map of size $256 \times 192 \times T$ is obtained. As far as the temporal-dependency extraction stage is

concerned, a LSTM model with $T$ LSTM cells (each cell comprises 128 units) are used to capture a long-range temporal dependency among the $T$ computed sample feature maps. The LSTM output is then introduced to a dense layer with a number of neurons equals to the number $D$ of grid dipoles on the cortical surface (source space) as shown in Fig. 2. The Minimum Squared Error (MSE) criterion is exploited as a cost function during the backpropagation phase using the Adam algorithm, with learning rate of 0.001, as optimizer. Table 1 recapitulates the key parameters of the proposed model.

To avoid the model overfitting a 5% dropout layer was incorporated into our designed model. Besides, 10% of the training samples were used as a validation set to monitor the model's performance during the training process. Together with the dropout method, an early stop strategy was also used to prevent the model overfitting.

## 2.3. Simulated data generation

Realistic synthetic data are simulated in order to train the MS-ESI model and to quantitatively evaluate the performance of our approach in particular context of extended interictal epileptic sources. We generated a High-resolution 256-channels scalp EEG (Hr-EEG) using a three-shell realistic head model including the brain, the skull and the scalp, whose conductivity are $0.33 \Omega^{-1}m^{-1}$, $0.00825 \Omega^{-1}m^{-1}$, and $0.33 \Omega^{-1}m^{-1}$, respectively. The high-resolution 15,002 vertices cortical mesh was built from the segmentation of the gray-white matter interface from a patient 3D T1-weighted MRI using Brain Visa software (Rivière et al., 2009). Each vertex of the mesh has been associated with an elementary

<table><caption>**Table 1**
The key parameters of the proposed model.</caption><thead><tr><td>Configuration</td><td>Value</td></tr></thead><tbody><tr><td>Dilated conv1D</td><td>Kernel number=64; Kernel_size=1 × 3; 1 × 5; 1 × 7</td></tr><tr><td>Dilation factors</td><td>1, 2, 4, 8, and 16</td></tr><tr><td>Dropout layer</td><td>0.05</td></tr><tr><td>LSTM</td><td>Unit=128</td></tr><tr><td>Fully connected layer</td><td>Neurons_number = 128; 15,002</td></tr><tr><td>Learning rate</td><td>0.001</td></tr><tr><td>Batch_size</td><td>64</td></tr><tr><td>Epochs</td><td>100</td></tr><tr><td>Optimizer</td><td>Adam</td></tr><tr><td>Loss</td><td>MSE</td></tr><tr><td>Monitor-metric</td><td>val_loss</td></tr></tbody></table>
current dipole. The mesh comes from the segmentation of the cortex (gray matter). This cortical surface includes the insula, the basal frontal and part of the interhemispheric structures (commissures). It mainly models the basal part of the hippocampal structure, entorhinal and parahippocampal cortex. But the hippocampus itself is not part of the mesh nor the internal nuclei. The 256-channel Biosemi system was used as the electrode template. Besides, the forward problem was solved using the Boundary Element Method (Gramfort et al., 2010) (BEM) to calculate the lead field matrix G (256 × 15,002), which quantifies the contribution of each dipole of the mesh at the level of 256 scalp electrode positions. The temporal dynamics of the activity of each dipole, were simulated using a computational neural mass model developed in our team for several years (Wendling et al., 2005). The parameters of this model can be adjusted to generate either background-like activity or interictal-spikes. To model an extended source (or patch), the Brainstorm software was used to randomly select a triangular grid on the cortical surface as a centered seed. Then, the patch size was iteratively increased until it reached the desired extent as per the requirements. These steps were then replicated to generate a sufficient number of epileptic source locations for both the training and testing stages. Dipoles associated to triangles within the patch were assigned with highly correlated interictal spike activities using an appropriate setting of coupling parameters between populations. Dipoles that do not belong to a patch were attributed background activity with an amplitude that is adjusted to the amplitude of the epileptic spikes, where the SNR was computed as $SNR = 10\log_{10}\frac{\|X_e\|^2}{\|X_b\|^2}$.

## 2.4. Clinical data

A 256-channel Hr-EEG was recorded in three patients for one hour, at 1000 Hz following the procedure approved by the National Ethics Committee for the Protection of Persons (CPP, agreement number 2012-A01227-36). Patients gave their written informed consent to participate in this study. For each patient, a specific head model is derived by segmenting the surfaces of the brain, the skull, and the scalp from the patient MRI. As for simulated data, the source space was composed of 15,000 dipoles corresponding to the vertices of the cortical surface mesh. The lead field matrix of each patient is computed using the BEM method implemented in OpenMEEG (Gramfort et al., 2010). For each patient, spikes were characterized according to their voltage distribution and averaged to improve SNR.

* **Patient 1:** The scalp EEG recordings showed interictal spike activity with maximal amplitude at the dorsal frontal left electrode AF5h. Twenty-three spikes centered in a 1 s epoch were considered: they were classified by shape leading to three groups of spikes and to three averaged spikes. Source imaging was performed on each of the three spikes. A weighted average of the three solutions was considered as result at the output of each of the seven methods.

* **Patient 2:** The scalp EEG recordings indicated that interictal spike activity maximal amplitude occurred at the frontal basal electrode AFP9h, located above the subject's left eyebrow. Eight spikes away from the artefacts were visually selected from the interictal epileptic activity. Each of them was extracted using a one-second window centered on the maximum point and these eight windows were averaged.

* **Patient 3:** The maximal spike activity occurred at the left parietal electrode CP1. Eighteen spikes centered in a 1 s epoch were averaged.

## 2.5. Model training and testing

According to the studied clinical context (partial epilepsy) three scenarios corresponding to three different source configurations were considered to train the MS-ESI model: (i) single patch sources, (ii) two

distant uncorrelated patches where Euclidean and Geodesic mean distance between all vertices of the two uncorrelated patches were 9.41 cm and 10.41 cm, respectively, and (iii) two close correlated patches configuration where the mean Euclidean and Geodesic distance between the two patches were 4.60 cm and 5.58 cm, respectively.

To ensure the robustness of the three-deep learning-based methods, namely MS-ESI, DeepSIF, and DeepBraiNNet, in different scenarios, the training dataset was carefully constructed by blending realizations from scenarios (i), (ii), and (iii) with SNR equal to 10 dB. The SNR level of 10 dB, in the training set, is chosen to mimic the realistic SNR of denoised interictal EEG data. More precisely, the training dataset is composed of 440 realizations of each scenario, which gives a total of 1320 examples. In addition, 10 % of these realizations were randomly kept for the validation stage to avoid the model overfitting. As far as the evaluation on simulated data is concerned, 100 realizations, for each scenario and each SNR level (from -15 to +10 dB by a step of 5 dB), were used as testing set to assess the performance of the proposed method and the six other state-of-art algorithms considered in this study. In order to prevent information leakage, the data employed in the training set is not recycled in the testing set. This strict demarcation guarantees that the performance of the learning-based methods is evaluated using completely novel and separate data, preserving the integrity of the assessment process. It is important to note that all parameters of DeepSIF, DeepBraiNNet and SISSY were rigorously fixed in accordance to the original papers (Sun et al., 2022; Bore et al., 2021; Becker et al., 2017), respectively. Regarding the wMNE, sLORETA and cMEM approaches, they were used as suggested by one of the most commonly used open-source Matlab toolbox: Brainstorm (Tadel et al., 2011). Regarding the performance evaluation on real data, a patient-specific approach is adopted, as pointed out before, to learn the MS-ESI, DeepSIF, and DeepBraiNNet models. This entails that for each tested patient, a new model was learned using the same training set of simulated intracerebral electrical activity with a patient-specific head model. Finally, all conducted experiments were carried out on a PC with an Intel Xero W-2295 CPU and an NVIDIA RTX A5000 GPU.

## 2.6. Performance metrics

The proposed framework is quantitatively evaluated in terms of source localization and the source reconstruction. To calculate the performance criteria, we first determine the active patch by thresholding the intensity of the estimated sources at the time sample with the highest power, which corresponds to the maximum of the epileptic spike in our context. To do so, a suitable threshold $\beta$ (between 0 and 1) is applied for each evaluated ESI approach for which all sources of intensity superior or equal to $\beta$ times the absolute maximal intensity is retained; i.e. if the amplitude of a reconstructed source exceeds the fixed threshold $\beta$, it is then considered as an epileptic source, otherwise, it is regarded as background.

The localization performance is assessed using two criteria: the Dipole Localization Error (DLE) (Becker et al., 2017) and the Normalized Hamming Distance (NHD). Other usual classification scores, namely, sensitivity (Sen) specificity (Spe), False Detection Rate (FDR) and F1-score are also provided in supplementary material. The DLE measures the similarity between the original and estimated sources configuration, and is defined as:

$$DLE = \frac{1}{2N} \sum_{m \in \Omega} \min_{n \in \Omega} \|\mathbf{p}_m - \mathbf{p}_n\| + \frac{1}{2\bar{N}} \sum_{n \in \Omega} \min_{m \in \Omega} \|\mathbf{p}_m - \mathbf{p}_n\| \quad (3)$$

where $\Omega$ and $\bar{\Omega}$ denote, respectively, the original and estimated sets of indices of all dipoles of an active patch. $\bar{N}$ are the numbers of original and estimated active dipoles while $p_m$ and $p_n$ stand respectively for the position of the m-th and the n-th source dipoles. Regarding the NHD-based metric, after a suitable thresholding, we derive for each method a binary map, by setting to one all the retained dipoles,
current dipole. The mesh comes from the segmentation of the cortex (gray matter). This cortical surface includes the insula, the basal frontal and part of the interhemispheric structures (commissures). It mainly models the basal part of the hippocampal structure, entorhinal and parahippocampal cortex. But the hippocampus itself is not part of the mesh nor the internal nuclei. The 256-channel Biosemi system was used as the electrode template. Besides, the forward problem was solved using the Boundary Element Method (Gramfort et al., 2010) (BEM) to calculate the lead field matrix G (256 × 15,002), which quantifies the contribution of each dipole of the mesh at the level of 256 scalp electrode positions. The temporal dynamics of the activity of each dipole, were simulated using a computational neural mass model developed in our team for several years (Wendling et al., 2005). The parameters of this model can be adjusted to generate either background-like activity or interictal-spikes. To model an extended source (or patch), the Brainstorm software was used to randomly select a triangular grid on the cortical surface as a centered seed. Then, the patch size was iteratively increased until it reached the desired extent as per the requirements. These steps were then replicated to generate a sufficient number of epileptic source locations for both the training and testing stages. Dipoles associated to triangles within the patch were assigned with highly correlated interictal spike activities using an appropriate setting of coupling parameters between populations. Dipoles that do not belong to a patch were attributed background activity with an amplitude that is adjusted to the amplitude of the epileptic spikes, where the SNR was computed as $SNR = 10\log_{10}\frac{\|X_e\|^2}{\|X_b\|^2}$.

## 2.4. Clinical data

A 256-channel Hr-EEG was recorded in three patients for one hour, at 1000 Hz following the procedure approved by the National Ethics Committee for the Protection of Persons (CPP, agreement number 2012-A01227-36). Patients gave their written informed consent to participate in this study. For each patient, a specific head model is derived by segmenting the surfaces of the brain, the skull, and the scalp from the patient MRI. As for simulated data, the source space was composed of 15,000 dipoles corresponding to the vertices of the cortical surface mesh. The lead field matrix of each patient is computed using the BEM method implemented in OpenMEEG (Gramfort et al., 2010). For each patient, spikes were characterized according to their voltage distribution and averaged to improve SNR.

* **Patient 1:** The scalp EEG recordings showed interictal spike activity with maximal amplitude at the dorsal frontal left electrode AF5h. Twenty-three spikes centered in a 1 s epoch were considered: they were classified by shape leading to three groups of spikes and to three averaged spikes. Source imaging was performed on each of the three spikes. A weighted average of the three solutions was considered as result at the output of each of the seven methods.

* **Patient 2:** The scalp EEG recordings indicated that interictal spike activity maximal amplitude occurred at the frontal basal electrode AFP9h, located above the subject's left eyebrow. Eight spikes away from the artefacts were visually selected from the interictal epileptic activity. Each of them was extracted using a one-second window centered on the maximum point and these eight windows were averaged.

* **Patient 3:** The maximal spike activity occurred at the left parietal electrode CP1. Eighteen spikes centered in a 1 s epoch were averaged.

## 2.5. Model training and testing

According to the studied clinical context (partial epilepsy) three scenarios corresponding to three different source configurations were considered to train the MS-ESI model: (i) single patch sources, (ii) two

distant uncorrelated patches where Euclidean and Geodesic mean distance between all vertices of the two uncorrelated patches were 9.41 cm and 10.41 cm, respectively, and (iii) two close correlated patches configuration where the mean Euclidean and Geodesic distance between the two patches were 4.60 cm and 5.58 cm, respectively.

To ensure the robustness of the three-deep learning-based methods, namely MS-ESI, DeepSIF, and DeepBraiNNet, in different scenarios, the training dataset was carefully constructed by blending realizations from scenarios (i), (ii), and (iii) with SNR equal to 10 dB. The SNR level of 10 dB, in the training set, is chosen to mimic the realistic SNR of denoised interictal EEG data. More precisely, the training dataset is composed of 440 realizations of each scenario, which gives a total of 1320 examples. In addition, 10 % of these realizations were randomly kept for the validation stage to avoid the model overfitting. As far as the evaluation on simulated data is concerned, 100 realizations, for each scenario and each SNR level (from -15 to +10 dB by a step of 5 dB), were used as testing set to assess the performance of the proposed method and the six other state-of-art algorithms considered in this study. In order to prevent information leakage, the data employed in the training set is not recycled in the testing set. This strict demarcation guarantees that the performance of the learning-based methods is evaluated using completely novel and separate data, preserving the integrity of the assessment process. It is important to note that all parameters of DeepSIF, DeepBraiNNet and SISSY were rigorously fixed in accordance to the original papers (Sun et al., 2022; Bore et al., 2021; Becker et al., 2017), respectively. Regarding the wMNE, sLORETA and cMEM approaches, they were used as suggested by one of the most commonly used open-source Matlab toolbox: Brainstorm (Tadel et al., 2011). Regarding the performance evaluation on real data, a patient-specific approach is adopted, as pointed out before, to learn the MS-ESI, DeepSIF, and DeepBraiNNet models. This entails that for each tested patient, a new model was learned using the same training set of simulated intracerebral electrical activity with a patient-specific head model. Finally, all conducted experiments were carried out on a PC with an Intel Xero W-2295 CPU and an NVIDIA RTX A5000 GPU.

## 2.6. Performance metrics

The proposed framework is quantitatively evaluated in terms of source localization and the source reconstruction. To calculate the performance criteria, we first determine the active patch by thresholding the intensity of the estimated sources at the time sample with the highest power, which corresponds to the maximum of the epileptic spike in our context. To do so, a suitable threshold $\beta$ (between 0 and 1) is applied for each evaluated ESI approach for which all sources of intensity superior or equal to $\beta$ times the absolute maximal intensity is retained; i.e. if the amplitude of a reconstructed source exceeds the fixed threshold $\beta$, it is then considered as an epileptic source, otherwise, it is regarded as background.

The localization performance is assessed using two criteria: the Dipole Localization Error (DLE) (Becker et al., 2017) and the Normalized Hamming Distance (NHD). Other usual classification scores, namely, sensitivity (Sen) specificity (Spe), False Detection Rate (FDR) and F1-score are also provided in supplementary material. The DLE measures the similarity between the original and estimated sources configuration, and is defined as:

$$DLE = \frac{1}{2N} \sum_{m \in \Omega} \min_{n \in \Omega} \|\mathbf{p}_m - \mathbf{p}_n\| + \frac{1}{2\bar{N}} \sum_{n \in \Omega} \min_{m \in \Omega} \|\mathbf{p}_m - \mathbf{p}_n\| \quad (3)$$

where $\Omega$ and $\bar{\Omega}$ denote, respectively, the original and estimated sets of indices of all dipoles of an active patch. $\bar{N}$ are the numbers of original and estimated active dipoles while $p_m$ and $p_n$ stand respectively for the position of the m-th and the n-th source dipoles. Regarding the NHD-based metric, after a suitable thresholding, we derive for each method a binary map, by setting to one all the retained dipoles,
otherwise they are set to zero. Mathematically, the NHD is defined as follows:

$$NHD = \frac{\sum_{i=1}^{Q} |\alpha(i) - \hat{\alpha}(i)|}{\sum_{i=1}^{Q} |\alpha(i) + \hat{\alpha}(i)|} \quad (4)$$

where $\alpha$ and $\hat{\alpha}$ stand, respectively, for a vector representation of the true binary map (ground truth) and the estimated one and Q denotes the total number of dipoles constituting the cortical surface.

Regarding the construction of the sources times courses, the Pearson Correlation Coefficients (PCCs) between the estimated time courses and the true ones are considered as an evaluation criterion. More precisely, the Averaged PCC is calculated by averaging the PCCs computed on the times courses associated to the overlapped dipoles between the estimated patches and the those for the original patches. In the experiments where we have more than one patch, the mean of the PCCs for all patches is computed. Finally, all the results are averaged over 100 realizations of the testing set with different patch signals configurations and SNRs.

To evaluate the statistical significance of the obtained results in terms of source localization and source reconstruction, a one-tailed t-test was conducted. Statistical significance was determined at $p < 0.05$. This analysis compared the results of the proposed MS-ESI in terms of DLE, NHD, and APCCS with the corresponding values of the six other methods (obtained from the 100 realizations of the testing set).

## 3. Results

A precise knowledge about the epileptogenic zones is generally unavailable for real data, performance evaluations are mostly based on realistic simulated data for which the ground truth is known and controlled. Thus, in this section, we firstly study the effectiveness of the proposed MS-ESI method in the context of interictal epileptic extended sources localization, by using realistic synthetic data (100 realizations). Besides, the well-known inverse crime issue and the impact of the limited number of surface electrodes were also discussed. Additionally, to corroborate the effectiveness of the proposed pipeline, real EEG measurements are also included. Finally, the MS-ESI algorithm was compared with several existing ESI algorithms, including DeepSIF, (Sun et al., 2022) DeepBraiNNet (Bore et al., 2021), SISSY (Becker et al., 2017), wMNE (Dale and Sereno, 1993), sLORETA (Pascual-Marqui, 2002) and cMEM (Chowdhury et al., 2013), for all experiments.

### 3.1. Application on realistic synthetic interictal EEG data

#### 3.1.1. Thresholds selection

As pointed out before, for the localization of the epileptogenic zone, all the PCCs of grid dipoles that exceed a certain threshold are associated to the extended source. Thus, the selection of a suitable threshold for each method is crucial in determining the performance of source imaging. A subjective threshold will lead to an unfair performance comparison between different methods, particularly as their performance can vary significantly with the chosen threshold. While determining the best threshold may be challenging, it is possible to identify a suitable relative threshold for all the methods under study. Such a suitable relative threshold will be the one that provides the best compromise between the source localization and source reconstruction. To set this suitable threshold, a set of threshold candidates is evaluated in the context of a single patch scenario for an SNR of 10 dB. Fig. 3 illustrates the line charts for the two metrics, DLE and NHD, as a function of the threshold $\beta$. Except for sLORETA, the DLE and the NHD curves show an almost equivalent trend. However, in the case of sLORETA, the DLE is inversely proportional to the value of $\beta$, whereas the NHD tends to deteriorate when the threshold is set to 0.8 and 0.9. Fig. 3, shows also that the optimal thresholds $\beta$ for MS-ESI, DeepSIF, DeepBraiNNet, SISSY, wMNE, sLORETA, and cMEM are 0.4, 0.4, 0.4, 0.4, 0.4, 0.7, and 0.2, respectively. Thus, for all the results that will be presented hereafter, the threshold $\beta$ is fixed accordingly. Note also that a study for the choice of the optimal threshold for the SNRs of 0 dB and -10 dB was also carried out (see supplementary material). The curves of the DLEs and NHDs present similar trends as those observed in the scenario with an SNR of 10 dB. Additionally, we also used the well-known Otsu's method (Otsu, 1979) to determine the optimal threshold for each algorithm. The results obtained, as elaborated in the supplementary materials, distinctly demonstrate that when utilizing the optimal Otsu's threshold, the DLEs and NHDs for each method were lower than when employing our thresholding approach. Consequently, for all the results presented below, the threshold $\beta$ was determined in accordance with the proposed methodology.

#### 3.1.2. Experiment I: single patch scenario

Once the adequate threshold is fixed for each method, the proposed model is firstly evaluated in the case of one patch as a function of SNR. Fig. 4(a)-(c) present the boxplot of the DLE, the NHD and the APCC metrics, respectively, for an SNR varying in a range from -15 dB to 10 dB. As expected, the performance of all methods is affected by increasing the noise, but the MS-ESI method exhibits the best behavior, in terms of source localization, compared to the other ones. Indeed, it can be

<figure><img src="image_6.png" /><figcaption>Fig. 3. The line charts for two metrics with different thresholds under one patch scenario with SNR of 10 dB. (a) DLE and (b) NHD.</figcaption></figure>
otherwise they are set to zero. Mathematically, the NHD is defined as follows:

$$NHD = \frac{\sum_{i=1}^{Q} |\alpha(i) - \hat{\alpha}(i)|}{\sum_{i=1}^{Q} |\alpha(i) + \hat{\alpha}(i)|} \quad (4)$$

where $\alpha$ and $\hat{\alpha}$ stand, respectively, for a vector representation of the true binary map (ground truth) and the estimated one and Q denotes the total number of dipoles constituting the cortical surface.

Regarding the construction of the sources times courses, the Pearson Correlation Coefficients (PCCs) between the estimated time courses and the true ones are considered as an evaluation criterion. More precisely, the Averaged PCC is calculated by averaging the PCCs computed on the times courses associated to the overlapped dipoles between the estimated patches and the those for the original patches. In the experiments where we have more than one patch, the mean of the PCCs for all patches is computed. Finally, all the results are averaged over 100 realizations of the testing set with different patch signals configurations and SNRs.

To evaluate the statistical significance of the obtained results in terms of source localization and source reconstruction, a one-tailed t-test was conducted. Statistical significance was determined at $p < 0.05$. This analysis compared the results of the proposed MS-ESI in terms of DLE, NHD, and APCCS with the corresponding values of the six other methods (obtained from the 100 realizations of the testing set).

## 3. Results

A precise knowledge about the epileptogenic zones is generally unavailable for real data, performance evaluations are mostly based on realistic simulated data for which the ground truth is known and controlled. Thus, in this section, we firstly study the effectiveness of the proposed MS-ESI method in the context of interictal epileptic extended sources localization, by using realistic synthetic data (100 realizations). Besides, the well-known inverse crime issue and the impact of the limited number of surface electrodes were also discussed. Additionally, to corroborate the effectiveness of the proposed pipeline, real EEG measurements are also included. Finally, the MS-ESI algorithm was compared with several existing ESI algorithms, including DeepSIF, (Sun et al., 2022) DeepBraiNNet (Bore et al., 2021), SISSY (Becker et al., 2017), wMNE (Dale and Sereno, 1993), sLORETA (Pascual-Marqui, 2002) and cMEM (Chowdhury et al., 2013), for all experiments.

### 3.1. Application on realistic synthetic interictal EEG data

#### 3.1.1. Thresholds selection

As pointed out before, for the localization of the epileptogenic zone, all the PCCs of grid dipoles that exceed a certain threshold are associated to the extended source. Thus, the selection of a suitable threshold for each method is crucial in determining the performance of source imaging. A subjective threshold will lead to an unfair performance comparison between different methods, particularly as their performance can vary significantly with the chosen threshold. While determining the best threshold may be challenging, it is possible to identify a suitable relative threshold for all the methods under study. Such a suitable relative threshold will be the one that provides the best compromise between the source localization and source reconstruction. To set this suitable threshold, a set of threshold candidates is evaluated in the context of a single patch scenario for an SNR of 10 dB. Fig. 3 illustrates the line charts for the two metrics, DLE and NHD, as a function of the threshold $\beta$. Except for sLORETA, the DLE and the NHD curves show an almost equivalent trend. However, in the case of sLORETA, the DLE is inversely proportional to the value of $\beta$, whereas the NHD tends to deteriorate when the threshold is set to 0.8 and 0.9. Fig. 3, shows also that the optimal thresholds $\beta$ for MS-ESI, DeepSIF, DeepBraiNNet, SISSY, wMNE, sLORETA, and cMEM are 0.4, 0.4, 0.4, 0.4, 0.4, 0.7, and 0.2, respectively. Thus, for all the results that will be presented hereafter, the threshold $\beta$ is fixed accordingly. Note also that a study for the choice of the optimal threshold for the SNRs of 0 dB and -10 dB was also carried out (see supplementary material). The curves of the DLEs and NHDs present similar trends as those observed in the scenario with an SNR of 10 dB. Additionally, we also used the well-known Otsu's method (Otsu, 1979) to determine the optimal threshold for each algorithm. The results obtained, as elaborated in the supplementary materials, distinctly demonstrate that when utilizing the optimal Otsu's threshold, the DLEs and NHDs for each method were lower than when employing our thresholding approach. Consequently, for all the results presented below, the threshold $\beta$ was determined in accordance with the proposed methodology.

#### 3.1.2. Experiment I: single patch scenario

Once the adequate threshold is fixed for each method, the proposed model is firstly evaluated in the case of one patch as a function of SNR. Fig. 4(a)-(c) present the boxplot of the DLE, the NHD and the APCC metrics, respectively, for an SNR varying in a range from -15 dB to 10 dB. As expected, the performance of all methods is affected by increasing the noise, but the MS-ESI method exhibits the best behavior, in terms of source localization, compared to the other ones. Indeed, it can be

<figure><img src="image_6.png" /><figcaption>Fig. 3. The line charts for two metrics with different thresholds under one patch scenario with SNR of 10 dB. (a) DLE and (b) NHD.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Fig. 4. Boxplot of DLE (a), NHD (b) and APCC (c) metrics for the proposed MS-ESI and other six existing ESI ones for different SNR values. Case of a single epileptic patch.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Fig. 4. Boxplot of DLE (a), NHD (b) and APCC (c) metrics for the proposed MS-ESI and other six existing ESI ones for different SNR values. Case of a single epileptic patch.</figcaption></figure>
noticed from Fig. 4 that the performance of the MS-ESI method is relatively stable from -10 dB to 10 dB. For a very low SNR (e.g. -15 dB), despite the fact that the performance of the MS-ESI exhibits in this case a minor decrease, it continues to outperform the other techniques, especially for source localization metrics (i.e. DLE and NHD). Even if sLORETA seems robust against noise, its performance remains low in terms of NHD. The aforementioned algorithms show also a consistent behavior in terms of other statistical metrics (e.g. F1-score) as shown in the provided supplementary material (see Fig. S1). The results, supported by statistical analysis using t-test, indicate good statistical significance with a confidence level of 95 %, whatever the compared method and the studied SNRs (see Tables S1 and S2 in the supplementary material for more details). Concerning the APCC, all methods demonstrate a good performance for an SNR greater or equal to -5 dB. For low SNR values (e.g. -10 and -15 dB), MS-ESI, DeepSIF, DeepBraiNNet, wMNE and sLORETA are still robust, whereas SISSY and cMEM seem to be less effective. Also, it can be generally noticed that the performance of MS-ESI is globally quasi equivalent compared with the commonly used methods, namely sLORETA and wMNE, for any SNR level. This demonstrates that the MS-ESI method is also very competitive in reconstructing the source signal. In relation to the APCC metric, the study showed significant superiority (with a p-value < 0.05) of the MS-ESI over the other methods considered, except for DeepBraiNNet, wMNE and sLORETA, as indicated in Table S3 of the supplementary materials.

To complete our performance analysis study, a visual inspection on the performance of each ESI method is provided in Fig. 5. This figure shows two examples of source imaging in the case of a single epileptic source (for SNR= 5 dB) with two different locations on the cortex: (i) a superficial patch and (ii) a deep patch. In accordance with the overall results presented in Fig. 4(a) and 4(b), deep learning-based methods, especially DeepSIF and the proposed method MS-ESI are very effective, regardless the patch location. The SISSY algorithm, which employs two sparsity constraints on the target sources, presents a very good behavior for the superficial patch but its performance seems less relevant in the case of deep sources. Regarding wMNE, sLORETA and cMEM, even if these methods target well the epileptic zone, they lead to blurred source localization results for both cases. Furthermore, the signals reconstructed by the seven methods together with their corresponding ground truth signal being randomly selected from testing samples under -5 dB SNR are shown in Fig. 6. According to the latter figure, it can be noticed that the performance of all methods remains acceptable even for a relatively bad SNR of -5 dB. It's worth noting that, as depicted in Table 2, with the exception of the cMEM algorithm, all the methods employed in this study exhibited short execution times for localizing epileptic sources and reconstructing their time courses. For instance, for Hr-EEG signals comprising 256 channels and 200 time samples, MS-ESI, DeepSIF, DeepBraiNNet, and sLORETA required less than 0.6 s for execution, while SISSY and cMEM took a longer time to complete the

process.

### 3.1.3. Experiment II: two uncorrelated patches with different source vertices and large distance

To deal with a multi-focal epilepsy context, the proposed model is evaluated on Scenario 2, which consists of two uncorrelated patches with different source vertices and a large distance. Fig. 7 shows the boxplot of three metrics for the proposed method and the other six state-of-art ones. The MS-ESI method significantly outperforms the other methods whatever the metrics as shown in Fig. 7 (all the p-values of Tables S4 and S5 of supplementary materials are lower than 0.05). This fact confirms the stability and robustness of MS-ESI even with multiple uncorrelated sources. In fact, even in such a scenario, which is clearly more complex than the one including one patch, the MS-ESI method still maintains a stable performance across an SNR range from -5 dB to 10 dB. In contrast to MS-ESI, the methods wMNE, sLORETA and cMEM do not maintain the same level of stability as observed in the single patch scenario. In fact, the performance of these methods shows relatively a large fluctuation under different SNRs. In terms of APCC, it is worth noting that the MS-ESI method consistently achieves the excellent APCC across all SNR values. This confirms again its ability to accurately reconstruct the source signal even in difficult scenarios such as for a low SNR value together with two highly distant coactivated sources. Regarding the significant improvement of the time course reconstruction, i.e. APCC, of the MS-ESI model, the p-values given in Table S6 are in line with the boxplot of Fig. 7(c), i.e., MS-ESI exhibited significant differences compared to DeepSIF, SISSY, and cMEM. However, this significant is not always demonstrated for DeepBraiNNet, wMNE, and sLORETA.

Fig. 8 illustrates examples of source imaging for two uncorrelated source patches which are randomly selected from the testing samples for an SNR of 5 dB. All methods globally target the location of the source patches. Compared with the MS-ESI method, DeepSIF and DeepBraiNNet lead to a more extensive reconstruction. Under the condition of two uncorrelated source patches, SISSY is unable to maintain high performance. For instance, in the case where we have at least one deep patch (Case 2 in the Fig. 8), SISSY overestimates the spatial extent of the deep source. As in the single patch case, the remaining methods including wMNE, sLORETA, and cMEM cannot accurately delimitate the spatial location and tend to give blurred source localization results (as for Scenario 1). Additionally, we conducted random inspections of multiple trials in Scenario 2 and computed, for each method, the success rate of detecting two uncorrelated patches (as presented in Table 3). Successful detection was determined based on two criteria: i) the reconstructed sources needed to have an overlap with the ground truth sources exceeding 70 %, and ii) both patches had to be detected regardless of their position in the source space. Table 3 illustrates that MS-ESI achieved a 95 % success rate, fulfilling both of these conditions,

<figure><img src="image_8.png" /><figcaption>Fig. 5. Source imaging for single source patch under 5 dB SNR. Case 1: superficial source patch. Case 2: deep source patch.</figcaption></figure>
noticed from Fig. 4 that the performance of the MS-ESI method is relatively stable from -10 dB to 10 dB. For a very low SNR (e.g. -15 dB), despite the fact that the performance of the MS-ESI exhibits in this case a minor decrease, it continues to outperform the other techniques, especially for source localization metrics (i.e. DLE and NHD). Even if sLORETA seems robust against noise, its performance remains low in terms of NHD. The aforementioned algorithms show also a consistent behavior in terms of other statistical metrics (e.g. F1-score) as shown in the provided supplementary material (see Fig. S1). The results, supported by statistical analysis using t-test, indicate good statistical significance with a confidence level of 95 %, whatever the compared method and the studied SNRs (see Tables S1 and S2 in the supplementary material for more details). Concerning the APCC, all methods demonstrate a good performance for an SNR greater or equal to -5 dB. For low SNR values (e.g. -10 and -15 dB), MS-ESI, DeepSIF, DeepBraiNNet, wMNE and sLORETA are still robust, whereas SISSY and cMEM seem to be less effective. Also, it can be generally noticed that the performance of MS-ESI is globally quasi equivalent compared with the commonly used methods, namely sLORETA and wMNE, for any SNR level. This demonstrates that the MS-ESI method is also very competitive in reconstructing the source signal. In relation to the APCC metric, the study showed significant superiority (with a p-value < 0.05) of the MS-ESI over the other methods considered, except for DeepBraiNNet, wMNE and sLORETA, as indicated in Table S3 of the supplementary materials.

To complete our performance analysis study, a visual inspection on the performance of each ESI method is provided in Fig. 5. This figure shows two examples of source imaging in the case of a single epileptic source (for SNR= 5 dB) with two different locations on the cortex: (i) a superficial patch and (ii) a deep patch. In accordance with the overall results presented in Fig. 4(a) and 4(b), deep learning-based methods, especially DeepSIF and the proposed method MS-ESI are very effective, regardless the patch location. The SISSY algorithm, which employs two sparsity constraints on the target sources, presents a very good behavior for the superficial patch but its performance seems less relevant in the case of deep sources. Regarding wMNE, sLORETA and cMEM, even if these methods target well the epileptic zone, they lead to blurred source localization results for both cases. Furthermore, the signals reconstructed by the seven methods together with their corresponding ground truth signal being randomly selected from testing samples under -5 dB SNR are shown in Fig. 6. According to the latter figure, it can be noticed that the performance of all methods remains acceptable even for a relatively bad SNR of -5 dB. It's worth noting that, as depicted in Table 2, with the exception of the cMEM algorithm, all the methods employed in this study exhibited short execution times for localizing epileptic sources and reconstructing their time courses. For instance, for Hr-EEG signals comprising 256 channels and 200 time samples, MS-ESI, DeepSIF, DeepBraiNNet, and sLORETA required less than 0.6 s for execution, while SISSY and cMEM took a longer time to complete the

process.

### 3.1.3. *Experiment II: two uncorrelated patches with different source vertices and large distance*

To deal with a multi-focal epilepsy context, the proposed model is evaluated on Scenario 2, which consists of two uncorrelated patches with different source vertices and a large distance. Fig. 7 shows the boxplot of three metrics for the proposed method and the other six state-of-art ones. The MS-ESI method significantly outperforms the other methods whatever the metrics as shown in Fig. 7 (all the p-values of Tables S4 and S5 of supplementary materials are lower than 0.05). This fact confirms the stability and robustness of MS-ESI even with multiple uncorrelated sources. In fact, even in such a scenario, which is clearly more complex than the one including one patch, the MS-ESI method still maintains a stable performance across an SNR range from -5 dB to 10 dB. In contrast to MS-ESI, the methods wMNE, sLORETA and cMEM do not maintain the same level of stability as observed in the single patch scenario. In fact, the performance of these methods shows relatively a large fluctuation under different SNRs. In terms of APCC, it is worth noting that the MS-ESI method consistently achieves the excellent APCC across all SNR values. This confirms again its ability to accurately reconstruct the source signal even in difficult scenarios such as for a low SNR value together with two highly distant coactivated sources. Regarding the significant improvement of the time course reconstruction, i.e. APCC, of the MS-ESI model, the p-values given in Table S6 are in line with the boxplot of Fig. 7(c), i.e., MS-ESI exhibited significant differences compared to DeepSIF, SISSY, and cMEM. However, this significant is not always demonstrated for DeepBraiNNet, wMNE, and sLORETA.

Fig. 8 illustrates examples of source imaging for two uncorrelated source patches which are randomly selected from the testing samples for an SNR of 5 dB. All methods globally target the location of the source patches. Compared with the MS-ESI method, DeepSIF and DeepBraiNNet lead to a more extensive reconstruction. Under the condition of two uncorrelated source patches, SISSY is unable to maintain high performance. For instance, in the case where we have at least one deep patch (Case 2 in the Fig. 8), SISSY overestimates the spatial extent of the deep source. As in the single patch case, the remaining methods including wMNE, sLORETA, and cMEM cannot accurately delimitate the spatial location and tend to give blurred source localization results (as for Scenario 1). Additionally, we conducted random inspections of multiple trials in Scenario 2 and computed, for each method, the success rate of detecting two uncorrelated patches (as presented in Table 3). Successful detection was determined based on two criteria: i) the reconstructed sources needed to have an overlap with the ground truth sources exceeding 70 %, and ii) both patches had to be detected regardless of their position in the source space. Table 3 illustrates that MS-ESI achieved a 95 % success rate, fulfilling both of these conditions,

<figure><img src="image_8.png" /><figcaption>Fig. 5. Source imaging for single source patch under 5 dB SNR. Case 1: superficial source patch. Case 2: deep source patch.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Fig. 6. Reconstructed signal of all methods and their corresponding ground truth signal which are randomly selected from the testing samples for an SNR of -5 dB. (a) MS-ESI. (b) DeepSIF. (c) DeepBraiNNet. (d) SISSY. (e) wMNE. (f) sLORETA. (g) cMEM.</figcaption></figure>

<table><caption>**Table 2**
Executive time for all ESI methods.</caption><thead><tr><th></th><th>MS-ESI</th><th>DeepSIF</th><th>DeepBraiNNet</th><th>SISSY</th><th>wMNE</th><th>sLORETA</th><th>cMEM</th></tr></thead><tbody><tr><th>Executive time (s)</th><td>0.2063</td><td>0.5981</td><td>0.1101</td><td>13.3016</td><td>0.1674</td><td>0.1929</td><td>2824.3667</td></tr></tbody></table>

followed by DeepSIF (90 %) and DeepBraiNNet (85 %). Among the traditional methods, only SISSY demonstrated the ability to correctly detect both uncorrelated patches with a 60 % success rate. It is important to note that, due to their inherently blurry solutions, methods like wMNE, sLORETA, and cMEM never met the criteria for a successful detection.

### 3.1.4. Experiment III: two correlated patches with different source vertices and short distance

A more challenging scenario is considered here to test the ability of the considered ESI methods to separate two correlated relatively close patches with different source vertices. The performance of the proposed method and other baseline methods is evaluated using the previous three metrics, DLE, NHD and APCC, and the boxplot of these three metrics presented in Fig. 9. Even in this challenging scenario, the MS-ESI method still exhibits the best performance among all the methods as shown in Fig. 9. It is worth noting that SISSY yields a relatively better DLE than DeepSIF and DeepBraiNNet across an SNR range from 0 dB to -10 dB. This is probably due to the fact that DeepSIF and DeepBraiNNet focus only on single-scale features and cannot effectively characterize EEG signals from different scales. Similar to Scenario 2, although the APCC of wMNE and sLORETA is slightly higher than that of the proposed MS-ESI method in scenarios with an SNR of 0 dB, 5 dB and 10 dB, the performance of these conventional methods declines drastically with an increasing noise level. This results in a much lower APCC compared to the MS-ESI method across an SNR range from -5 dB to -15 dB. These results demonstrate the robust performance of the MS-ESI method regardless the distance or the correlation among simultaneously active sources. Note that a similar behavior of the methods under study is also observed using other statistical metrics as shown in the provided supplementary materials (Fig. S3). Tables S7, S8 and S9, in the supplementary material, showed that the proposed method significantly outperforms other methods in terms of DLE and NHD. Concerning the APCC metric, except for wMNE and sLORETA methods (for some SNR

levels), a significant improvement of the results obtained with MS-ESI was also observed.

Similarly to Scenario 1 and Scenario 2, two examples of source imaging for two correlated source patches are also given in Fig. 10. These examples are randomly selected from the testing dataset. Even if we observe a slight degradation in the reconstruction quality of MS-ESI under this more challenging scenario, this approach still outperforms the other ESI methods and is able to separate the two closest patches in both cases. Interestingly, contrary to the other methods, the DeepSIF one shows inability to localize one of the two deep patches as shown in Fig. 10 (Case 2). Additionally, depending on the specific case being studied, DeepBraiNNet and SISSY are not able to distinguish between the two patches, as they treat them as a single large patch. As far as wMNE and sLORETA are considered, they fail to recover all patches. Finally, in both studied cases, the solutions of cMEM include high amplitude at a position that does not correspond to the initial source configuration. In the context of Scenario 3, we also examined the success rate of detecting two correlated patches. In this case, a successful detection required the considered methods to meet not only the previously mentioned conditions (i) and (ii) from Experiment II but also a third condition: (iii) the capability to distinguish between closely correlated sources. Even in this challenging scenario, as shown in Table 4, the MS-ESI approach displayed promising results with a success rate of 60 %, outperforming all the other methods considered in the study (the highest success rate of 45 % was achieved by DeepBraiNNet).

### 3.1.5. Experiment IV: model errors

In the preceding experiment, we addressed the EEG inverse problem, where the same model parameters were utilized for both forward and inverse modeling, a situation commonly referred to as the "inverse crime". To mitigate this concern, we employed a lead field matrix to simulate scalp EEG signals (for the forward problem) that differed from the one used to solve the EEG inverse problem. This was achieved by making slight adjustments to the positions of the scalp electrodes. It's
<figure><img src="image_9.png" /><figcaption>Fig. 6. Reconstructed signal of all methods and their corresponding ground truth signal which are randomly selected from the testing samples for an SNR of -5 dB. (a) MS-ESI. (b) DeepSIF. (c) DeepBraiNNet. (d) SISSY. (e) wMNE. (f) sLORETA. (g) cMEM.</figcaption></figure>

<table><caption>**Table 2**
Executive time for all ESI methods.</caption><thead><tr><th></th><th>MS-ESI</th><th>DeepSIF</th><th>DeepBraiNNet</th><th>SISSY</th><th>wMNE</th><th>sLORETA</th><th>cMEM</th></tr></thead><tbody><tr><td>Executive time (s)</td><td>0.2063</td><td>0.5981</td><td>0.1101</td><td>13.3016</td><td>0.1674</td><td>0.1929</td><td>2824.3667</td></tr></tbody></table>

followed by DeepSIF (90 %) and DeepBraiNNet (85 %). Among the traditional methods, only SISSY demonstrated the ability to correctly detect both uncorrelated patches with a 60 % success rate. It is important to note that, due to their inherently blurry solutions, methods like wMNE, sLORETA, and cMEM never met the criteria for a successful detection.

### 3.1.4. Experiment III: two correlated patches with different source vertices and short distance

A more challenging scenario is considered here to test the ability of the considered ESI methods to separate two correlated relatively close patches with different source vertices. The performance of the proposed method and other baseline methods is evaluated using the previous three metrics, DLE, NHD and APCC, and the boxplot of these three metrics presented in Fig. 9. Even in this challenging scenario, the MS-ESI method still exhibits the best performance among all the methods as shown in Fig. 9. It is worth noting that SISSY yields a relatively better DLE than DeepSIF and DeepBraiNNet across an SNR range from 0 dB to -10 dB. This is probably due to the fact that DeepSIF and DeepBraiNNet focus only on single-scale features and cannot effectively characterize EEG signals from different scales. Similar to Scenario 2, although the APCC of wMNE and sLORETA is slightly higher than that of the proposed MS-ESI method in scenarios with an SNR of 0 dB, 5 dB and 10 dB, the performance of these conventional methods declines drastically with an increasing noise level. This results in a much lower APCC compared to the MS-ESI method across an SNR range from -5 dB to -15 dB. These results demonstrate the robust performance of the MS-ESI method regardless the distance or the correlation among simultaneously active sources. Note that a similar behavior of the methods under study is also observed using other statistical metrics as shown in the provided supplementary materials (Fig. S3). Tables S7, S8 and S9, in the supplementary material, showed that the proposed method significantly outperforms other methods in terms of DLE and NHD. Concerning the APCC metric, except for wMNE and sLORETA methods (for some SNR

levels), a significant improvement of the results obtained with MS-ESI was also observed.

Similarly to Scenario 1 and Scenario 2, two examples of source im-
aging for two correlated source patches are also given in Fig. 10. These
examples are randomly selected from the testing dataset. Even if we
observe a slight degradation in the reconstruction quality of MS-ESI
under this more challenging scenario, this approach still outperforms
the other ESI methods and is able to separate the two closest patches in
both cases. Interestingly, contrary to the other methods, the DeepSIF one
shows inability to localize one of the two deep patches as shown in
Fig. 10 (Case 2). Additionally, depending on the specific case being
studied, DeepBrainNNet and SISSY are not be able to distinguish be-
tween the two patches, as they treat them as a single large patch. As far
as wMNE and sLORETA are considered, they fail to recover all patches.
Finally, in both studied cases, the solutions of cMEM include high
amplitude at a position that does not correspond to the initial source
configuration. In the context of Scenario 3, we also examined the success
rate of detecting two correlated patches. In this case, a successful
detection required the considered methods to meet not only the previ-
ously mentioned conditions (i) and (ii) from Experiment II but also a
third condition: (iii) the capability to distinguish between closely
correlated sources. Even in this challenging scenario, as shown in
Table 4, the MS-ESI approach displayed promising results with a success
rate of 60 %, outperforming all the other methods considered in the
study (the highest success rate of 45 % was achieved by DeepBraiNNet).

### 3.1.5. Experiment IV: model errors

In the preceding experiment, we addressed the EEG inverse problem, where the same model parameters were utilized for both forward and inverse modeling, a situation commonly referred to as the "inverse crime". To mitigate this concern, we employed a lead field matrix to simulate scalp EEG signals (for the forward problem) that differed from the one used to solve the EEG inverse problem. This was achieved by making slight adjustments to the positions of the scalp electrodes. It's
<figure><img src="image_10.png" /><figcaption>Fig. 7. Boxplot of DLE (a), NHD (b) and APCC (c) metrics for the proposed MS-ESI and other six existing ESI ones for different SNR values. Case of two uncorrelated patches.</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Fig. 7. Boxplot of DLE (a), NHD (b) and APCC (c) metrics for the proposed MS-ESI and other six existing ESI ones for different SNR values. Case of two uncorrelated patches.</figcaption></figure>
<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Source imaging for two uncorrelated source patches under 5 dB SNR. Case 1: superficial source patch. Case 2 includes one superficial and deep source patch, which are shown in the top view and bottom view.</figcaption></figure>

<table><caption>**Table 3**

Successful detection rate, DLE and NHD at the output of all ESI methods for the two uncorrelated sources scenario.</caption><thead><tr><th></th><th>MS-ESI</th><th>DeepSIF</th><th>DeepBraiNNet</th><th>SISSY</th><th>wMNE</th><th>sLORETA</th><th>cMEM</th></tr></thead><tbody><tr><td>Successful detection rate</td><td>95 %</td><td>90 %</td><td>85 %</td><td>60 %</td><td>0 %</td><td>0 %</td><td>0 %</td></tr><tr><td>DLE</td><td>1.2735±0.8276</td><td>1.9670±1.9509</td><td>2.1698±1.2265</td><td>2.3722±2.0206</td><td>-</td><td>-</td><td>-</td></tr><tr><td>NHD</td><td>0.2612±0.1115</td><td>0.3317±0.1931</td><td>0.3600±0.1394</td><td>0.3682±0.1780</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

important to note that there are alternative methods to circumvent this inverse crime, such as employing distinct grids for the forward model and the inverse one, or modifying the skull conductivity between the forward model and the source localization solution. To be specific, for each of the 100 realizations in the testing set, a new lead field matrix was generated by randomly shifting the electrode positions by approximately 1 centimeter. The resulting DLEs and NHDs, with an SNR value of 10 dB, for all methods across the three studied scenarios were provided in Figs. 11 and 12, respectively. It's evident that, as in the three prior experiments, the DL-based methods outperformed the classical ones in all scenarios. Specifically, the obtained results were nearly indistinguishable with and without the inverse crime for the easier scenarios 1 and 2. As for the challenging scenario 3 (involving 2 correlated sources), all methods exhibited slightly reduced efficiency. Notably, the proposed MS-ESI method maintained its superior performance in this context compared to the other methods.

### 3.1.6. *Experiment V: influence of the number of scalp EEG electrodes*

Even if the HR-EEG is increasingly used in clinical settings, the standard EEG systems in medical applications generally have 19 to 32 channels. Thus, to evaluate the effectiveness of the proposed MS-ESI approach on low-resolution EEG data, the experiments I to III are reproduced for a limited number of electrodes equal 32 and a realistic SNR of 10 dB. Figs. 13 and 14 showed the obtained results for DLE and NHD metrics. Similarly to the experiment IV, the obtained result for scenarios 1 and 2 were globally equivalent than those obtained from HR-EEG (256 electrodes). As far as the scenario 3 was concerned, all methods exhibit reduced performances when only 32 electrodes were exploited. However, the MS-ESI model was still the more efficient method with a reasonable performance.

## 3.2. Application on clinical data

The above experimental results on simulated signals showed that the

MS-ESI method accurately localized the source position and reconstructed the spatial extent, whatever the studied scenario. To further verify whether the MS-ESI method can maintain its good performance on real data, we propose to qualitatively measure the efficiency of the commonly used pipelines on real HR-EEG data acquired in patients with drug-resistant partial epilepsy, during their presurgical evaluation.

For the three patients, Fig. 15 illustrates source imaging of real EEG interictal spikes, the reconstructed interictal spike activity at the electrode for which it has maximal amplitude as well as the PCC between the latter and the corresponding recorded interictal spike. The magenta dots indicate SEEG contacts with interictal activity and the black circle indicate the maximum of amplitude of the reconstructed brain sources. For patients 2 and 1, the implantation was in Frontal and temporal regions, for patient 3 it was in frontal and parietal regions. For all patients, both orthogonal and oblique trajectories were considered for the electrode implantation (Rollo et al., 2020). In addition, all these three patients have SEEG recordings. The top view of source imaging for patients 1 and 3 using cMEM is provided in Fig. S4 of the supplementary materials.

**Patient 1:** The most circumscribed solutions were observed with MS-ESI, DeepSIF, DeepBraiNNet, and SISSY. More extended solutions were obtained with wMNE, sLORETA and cMEM which also showed activations in the contralateral hemisphere. These activations were in the left frontal pole, i.e. the closest to SEEG contacts that showed the maximal intracerebral spike activity with DeepSIF and SISSY. The solution obtained with MS-ESI, DeepBraiNNet, and MS-ESI localizes a well circumscribed region, posterior to the contacts showing the maximal intracerebral spike activity. There was no implanted electrode in this region. MS-ESI, DeepBraiNNet, DeepSIF and SISSY exhibited a good performance. On the contrary, sLORETA and cMEM showed activated areas away from the maximal intracerebral activity, and wMNE generated large activation including the contralateral frontal region. All the PCCs are high, showing a good EEG signal reconstruction for all the methods, with a slightly higher PCC for SISSY and wMNE. **Patient 2:**
<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Source imaging for two uncorrelated source patches under 5 dB SNR. Case 1: superficial source patch. Case 2 includes one superficial and deep source patch, which are shown in the top view and bottom view.</figcaption></figure>

<table><caption>**Table 3**

Successful detection rate, DLE and NHD at the output of all ESI methods for the two uncorrelated sources scenario.</caption><thead><tr><th></th><th>MS-ESI</th><th>DeepSIF</th><th>DeepBraiNNet</th><th>SISSY</th><th>wMNE</th><th>sLORETA</th><th>cMEM</th></tr></thead><tbody><tr><td>Successful detection rate</td><td>95 %</td><td>90 %</td><td>85 %</td><td>60 %</td><td>0 %</td><td>0 %</td><td>0 %</td></tr><tr><td>DLE</td><td>1.2735±0.8276</td><td>1.9670±1.9509</td><td>2.1698±1.2265</td><td>2.3722±2.0206</td><td>-</td><td>-</td><td>-</td></tr><tr><td>NHD</td><td>0.2612±0.1115</td><td>0.3317±0.1931</td><td>0.3600±0.1394</td><td>0.3682±0.1780</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

important to note that there are alternative methods to circumvent this inverse crime, such as employing distinct grids for the forward model and the inverse one, or modifying the skull conductivity between the forward model and the source localization solution. To be specific, for each of the 100 realizations in the testing set, a new lead field matrix was generated by randomly shifting the electrode positions by approximately 1 centimeter. The resulting DLEs and NHDs, with an SNR value of 10 dB, for all methods across the three studied scenarios were provided in Figs. 11 and 12, respectively. It's evident that, as in the three prior experiments, the DL-based methods outperformed the classical ones in all scenarios. Specifically, the obtained results were nearly indistinguishable with and without the inverse crime for the easier scenarios 1 and 2. As for the challenging scenario 3 (involving 2 correlated sources), all methods exhibited slightly reduced efficiency. Notably, the proposed MS-ESI method maintained its superior performance in this context compared to the other methods.

### 3.1.6. *Experiment V: influence of the number of scalp EEG electrodes*

Even if the HR-EEG is increasingly used in clinical settings, the standard EEG systems in medical applications generally have 19 to 32 channels. Thus, to evaluate the effectiveness of the proposed MS-ESI approach on low-resolution EEG data, the experiments I to III are reproduced for a limited number of electrodes equal 32 and a realistic SNR of 10 dB. Figs. 13 and 14 showed the obtained results for DLE and NHD metrics. Similarly to the experiment IV, the obtained result for scenarios 1 and 2 were globally equivalent than those obtained from HR-EEG (256 electrodes). As far as the scenario 3 was concerned, all methods exhibit reduced performances when only 32 electrodes were exploited. However, the MS-ESI model was still the more efficient method with a reasonable performance.

## 3.2. Application on clinical data

The above experimental results on simulated signals showed that the

MS-ESI method accurately localized the source position and reconstructed the spatial extent, whatever the studied scenario. To further verify whether the MS-ESI method can maintain its good performance on real data, we propose to qualitatively measure the efficiency of the commonly used pipelines on real HR-EEG data acquired in patients with drug-resistant partial epilepsy, during their presurgical evaluation.

For the three patients, Fig. 15 illustrates source imaging of real EEG interictal spikes, the reconstructed interictal spike activity at the electrode for which it has maximal amplitude as well as the PCC between the latter and the corresponding recorded interictal spike. The magenta dots indicate SEEG contacts with interictal activity and the black circle indicate the maximum of amplitude of the reconstructed brain sources. For patients 2 and 1, the implantation was in Frontal and temporal regions, for patient 3 it was in frontal and parietal regions. For all patients, both orthogonal and oblique trajectories were considered for the electrode implantation (Rollo et al., 2020). In addition, all these three patients have SEEG recordings. The top view of source imaging for patients 1 and 3 using cMEM is provided in Fig. S4 of the supplementary materials.

**Patient 1:** The most circumscribed solutions were observed with MS-ESI, DeepSIF, DeepBraiNNet, and SISSY. More extended solutions were obtained with wMNE, sLORETA and cMEM which also showed activations in the contralateral hemisphere. These activations were in the left frontal pole, i.e. the closest to SEEG contacts that showed the maximal intracerebral spike activity with DeepSIF and SISSY. The solution obtained with MS-ESI, DeepBraiNNet, and MS-ESI localizes a well circumscribed region, posterior to the contacts showing the maximal intracerebral spike activity. There was no implanted electrode in this region. MS-ESI, DeepBraiNNet, DeepSIF and SISSY exhibited a good performance. On the contrary, sLORETA and cMEM showed activated areas away from the maximal intracerebral activity, and wMNE generated large activation including the contralateral frontal region. All the PCCs are high, showing a good EEG signal reconstruction for all the methods, with a slightly higher PCC for SISSY and wMNE. **Patient 2:**
<figure><img src="image_12.png" /><figcaption>Fig. 9. Boxplot of DLE (a), NHD (b) and APCC (c) metrics for the proposed MS-ESI and other six existing ESI ones for different SNR values. Case of two correlated patches.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>**Fig. 9.** Boxplot of DLE (a), NHD (b) and APCC (c) metrics for the proposed MS-ESI and other six existing ESI ones for different SNR values. Case of two correlated patches.</figcaption></figure>
<figure><img src="image_13.png" /><figcaption>Fig. 10. Source imaging for two correlated source patches under 5 dB SNR. Case 1: superficial source patch. Case 2: deep source patch.</figcaption></figure>

<caption>**Table 4**</caption>

Successful detection rate, DLE and NHD of all ESI methods under two correlated sources scenario.

<table><thead><tr><th></th><th>MS-ESI</th><th>DeepSIF</th><th>DeepBraiNNet</th><th>SISSY</th><th>wMNE</th><th>sLORETA</th><th>cMEM</th></tr></thead><tbody><tr><th>Successful detection rate</th><td>60 %</td><td>25 %</td><td>45 %</td><td>40 %</td><td>0 %</td><td>0 %</td><td>0 %</td></tr><tr><th>DLE</th><td>2.7687±2.7195</td><td>5.3079±2.8948</td><td>4.3067±3.1396</td><td>2.8346±2.6474</td><td>-</td><td>-</td><td>-</td></tr><tr><th>NHD</th><td>0.3489±0.1258</td><td>0.5214±0.1683</td><td>0.4431±0.1260</td><td>0.4245±0.0925</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

<figure><img src="image_14.png" /><figcaption>Fig. 11. Boxplot of DLE for each method based on new testing samples with 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 12. Boxplot of NHD for each method based on new testing samples with 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

Spatially limited solutions were obtained with MS-ESI, DeepSIF, DeepBraiNNet while larger activations appeared with SISSY, wMNE, sLORETA and cMEM. The maximal amplitude of solutions from MS-ESI, DeepSIF and DeepBraiNNet were the closest to the maximal intracerebral spike activity located in the left mesial orbitofrontal region.

Relatively close source activity was also obtained using cMEM, SISSY and sLORETA regarding the maximal amplitude of their solution. The solution of wMNE was localized in the lateral orbitofrontal region, remotely from the main interictal SEEG activity. All the PCCs are very high, showing a very good EEG signal reconstruction for all the methods,
<figure><img src="image_13.png" /><figcaption>Fig. 10. Source imaging for two correlated source patches under 5 dB SNR. Case 1: superficial source patch. Case 2: deep source patch.</figcaption></figure>

<caption>Table 4</caption>

Successful detection rate, DLE and NHD of all ESI methods under two correlated sources scenario.

<table><thead><tr><th></th><th>MS-ESI</th><th>DeepSIF</th><th>DeepBraiNNet</th><th>SISSY</th><th>wMNE</th><th>sLORETA</th><th>cMEM</th></tr></thead><tbody><tr><th>Successful detection rate</th><td>60 %</td><td>25 %</td><td>45 %</td><td>40 %</td><td>0 %</td><td>0 %</td><td>0 %</td></tr><tr><th>DLE</th><td>2.7687±2.7195</td><td>5.3079±2.8948</td><td>4.3067±3.1396</td><td>2.8346±2.6474</td><td>-</td><td>-</td><td>-</td></tr><tr><th>NHD</th><td>0.3489±0.1258</td><td>0.5214±0.1683</td><td>0.4431±0.1260</td><td>0.4245±0.0925</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

<figure><img src="image_14.png" /><figcaption>Fig. 11. Boxplot of DLE for each method based on new testing samples with 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 12. Boxplot of NHD for each method based on new testing samples with 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

Spatially limited solutions were obtained with MS-ESI, DeepSIF, DeepBraiNNet while larger activations appeared with SISSY, wMNE, sLORETA and cMEM. The maximal amplitude of solutions from MS-ESI, DeepSIF and DeepBraiNNet were the closest to the maximal intracerebral spike activity located in the left mesial orbitofrontal region.

Relatively close source activity was also obtained using cMEM, SISSY and sLORETA regarding the maximal amplitude of their solution. The solution of wMNE was localized in the lateral orbitofrontal region, remotely from the main interictal SEEG activity. All the PCCs are very high, showing a very good EEG signal reconstruction for all the methods,
<figure><img src="image_16.png" /><figcaption>Fig. 13. Boxplot of DLE for each method under three scenarios with 32 electrodes and 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 14. Boxplot of NHD for each method under three scenarios with 32 electrodes and 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

with a slightly higher PCC for MS-ESI, SISSY and sLORETA. **Patient 3:** The most restricted solutions were obtained with MS-ESI and cMEM. Unfortunately, SISSY, DeepSIF and DeepBraiNNet showed bilateral activations and wMNE and sLORETA gave extended solutions. The most accurate localization with respect to the maximal intracerebral spike activity was obtained with MS-ESI in the postcentral gyrus. Similarly, the maximal activation obtained with DeepBraiNNet also collocated with SEEG interictal activity but the solution was more extended than with MS-ESI. The results obtained with the other approaches were less congruent with SEEG, with a maximal activation in the left inferior parietal, left precentral, left prefrontal or right basal temporal regions for DeepSIF, wMNE, sLORETA and cMEM respectively. A good EEG signal reconstruction was obtained by MS-ESI, DeepBrainNet, WMNE and sLORETA with high PCC values.

## 4. Discussion

To reduce the number of feasible solutions and achieve a relatively optimal one for the inverse problem, the conventional numerical ESI methods explicitly incorporate prior information into the objective equation in the form of regularization terms. The sparsity constraints introduced by SISSY (Becker et al., 2017) effectively enhance the capability to obtain focused and sparse solutions, but its performance is often limited to locating superficial epileptic sources and is less efficient for deep sources. wMNE (Dale and Sereno, 1993) addresses the problem of MNE favoring cortical surface sources by weighting different sources based on their distances from scalp electrodes, providing a solid mathematical foundation and clear formulation. However, its estimated source localizations are still blurred. sLORETA (Pascual-Marqui, 2002) is renowned for its robustness to noise in EEG signals and has a solid

theoretical foundation, but it suffers from spatial blurring and generates overly diffuse estimates. cMEM (Chowdhury et al., 2013) imposes spatial constraints derived from anatomical or physiological considerations of the brain to guide the solution toward realistic source configurations. However, it may be sensitive to the selection of some parameters, and its computational complexity is relatively high. The effectiveness of these conventional ESI methods depends on the accuracy of their assumptions and their ability to define additional mathematical terms accurately based on prior information. Recent advances in DL framework have confirmed the power of this technique in tackling various real-life applications among which biomedical engineering. However, it is commonly known that the performance of such technique is heavily subject to the size of the training dataset. In many applications, such a constraint is however not always fulfilled. For instance, in the context of epilepsy, as stated before, the availability of a joint registration of surface EEG and SEEG signals, which is mandatory for DL-based ESI, is infeasible in clinical routine. In this paper, this limitation is addressed by means of realistic simulated EEG and intracerebral electrical activity as training dataset generated using a neural mass model developed in our group. (Wendling et al., 2002; Wendling et al., 2005) It worth noting here that the only clinical parameter used in our method is the one of the patient lead field matrix which is systematically computed from the available patient's MRIs. Recently, a DL-based ESI approach has been introduced in (Bore et al., 2021) where a LSTM model was mainly employed. Despite its promising results, such models capture only time dependence among time samples where possible spatial dependencies are omitted. To simultaneously extract spatial and temporal information, the DeepSIF (Sun et al., 2022) model that incorporated both fully connected networks and LSTM was proposed to implicitly learn rich characteristics of underlying brain sources.
<figure><img src="image_16.png" /><figcaption>Fig. 13. Boxplot of DLE for each method under three scenarios with 32 electrodes and 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 14. Boxplot of NHD for each method under three scenarios with 32 electrodes and 10 dB SNR. (a) one patch. (b) two uncorrelated patches. (c) two correlated patches.</figcaption></figure>

with a slightly higher PCC for MS-ESI, SISSY and sLORETA. **Patient 3:** The most restricted solutions were obtained with MS-ESI and cMEM. Unfortunately, SISSY, DeepSIF and DeepBraiNNet showed bilateral activations and wMNE and sLORETA gave extended solutions. The most accurate localization with respect to the maximal intracerebral spike activity was obtained with MS-ESI in the postcentral gyrus. Similarly, the maximal activation obtained with DeepBraiNNet also collocated with SEEG interictal activity but the solution was more extended than with MS-ESI. The results obtained with the other approaches were less congruent with SEEG, with a maximal activation in the left inferior parietal, left precentral, left prefrontal or right basal temporal regions for DeepSIF, wMNE, sLORETA and cMEM respectively. A good EEG signal reconstruction was obtained by MS-ESI, DeepBrainNet, WMNE and sLORETA with high PCC values.

## 4. Discussion

To reduce the number of feasible solutions and achieve a relatively optimal one for the inverse problem, the conventional numerical ESI methods explicitly incorporate prior information into the objective equation in the form of regularization terms. The sparsity constraints introduced by SISSY (Becker et al., 2017) effectively enhance the capability to obtain focused and sparse solutions, but its performance is often limited to locating superficial epileptic sources and is less efficient for deep sources. wMNE (Dale and Sereno, 1993) addresses the problem of MNE favoring cortical surface sources by weighting different sources based on their distances from scalp electrodes, providing a solid mathematical foundation and clear formulation. However, its estimated source localizations are still blurred. sLORETA (Pascual-Marqui, 2002) is renowned for its robustness to noise in EEG signals and has a solid

theoretical foundation, but it suffers from spatial blurring and generates overly diffuse estimates. cMEM (Chowdhury et al., 2013) imposes spatial constraints derived from anatomical or physiological considerations of the brain to guide the solution toward realistic source configurations. However, it may be sensitive to the selection of some parameters, and its computational complexity is relatively high. The effectiveness of these conventional ESI methods depends on the accuracy of their assumptions and their ability to define additional mathematical terms accurately based on prior information. Recent advances in DL framework have confirmed the power of this technique in tackling various real-life applications among which biomedical engineering. However, it is commonly known that the performance of such technique is heavily subject to the size of the training dataset. In many applications, such a constraint is however not always fulfilled. For instance, in the context of epilepsy, as stated before, the availability of a joint registration of surface EEG and SEEG signals, which is mandatory for DL-based ESI, is infeasible in clinical routine. In this paper, this limitation is addressed by means of realistic simulated EEG and intracerebral electrical activity as training dataset generated using a neural mass model developed in our group. (Wendling et al., 2002; Wendling et al., 2005) It worth noting here that the only clinical parameter used in our method is the one of the patient lead field matrix which is systematically computed from the available patient's MRIs. Recently, a DL-based ESI approach has been introduced in (Bore et al., 2021) where a LSTM model was mainly employed. Despite its promising results, such models capture only time dependence among time samples where possible spatial dependencies are omitted. To simultaneously extract spatial and temporal information, the DeepSIF (Sun et al., 2022) model that incorporated both fully connected networks and LSTM was proposed to implicitly learn rich characteristics of underlying brain sources.
<figure><img src="image_18.png" /><figcaption>**Fig. 15.** Source imaging of real EEG recordings, the reconstructed interictal spike activity at the electrode for which it has maximal amplitude as well as the PCC between the latter and the corresponding recorded interictal spike. (a) Patient 1. (b) Patient 2. (c) Patient 3.</figcaption></figure>

However, the drawbacks of the fully connected network in acquiring
spatial information and its high number of training parameters to some
extent limit the performance of this model in extracting spatiotemporal
information. The spatial dependency is relevant in EEG inverse problem
as they permit to establish the appropriate spatial filter that targets
epileptic sources. To this end, our approach takes advantage of both

spatial and temporal dependencies by combining TCN with LSTM: the
goal of TCN is to capture local spatial patterns whereas the role of LSTM
is to reveal temporal dependencies.

As shown in Section 3.1 giving results on realistic synthetic interictal
EEG data, while the time course reconstruction of epileptic sources
seems to be slightly better for the proposed MS-ESI method compared to
<figure><img src="image_18.png" /><figcaption>**Fig. 15.** Source imaging of real EEG recordings, the reconstructed interictal spike activity at the electrode for which it has maximal amplitude as well as the PCC between the latter and the corresponding recorded interictal spike. (a) Patient 1. (b) Patient 2. (c) Patient 3.</figcaption></figure>

However, the drawbacks of the fully connected network in acquiring spatial information and its high number of training parameters to some extent limit the performance of this model in extracting spatiotemporal information. The spatial dependency is relevant in EEG inverse problem as they permit to establish the appropriate spatial filter that targets epileptic sources. To this end, our approach takes advantage of both

spatial and temporal dependencies by combining TCN with LSTM: the goal of TCN is to capture local spatial patterns whereas the role of LSTM is to reveal temporal dependencies.

As shown in Section 3.1 giving results on realistic synthetic interictal EEG data, while the time course reconstruction of epileptic sources seems to be slightly better for the proposed MS-ESI method compared to
the other considered DL-based methods, MS-ESI significantly improved the performance in terms of source localization especially for challenging scenarios (Scenarios 2 and 3).

Three challenges in ESI framework are commonly known and worth to be discussed here: localizing i) focal epileptic sources, ii) deep epileptic sources and iii) close correlated epileptic sources.

**Localization of focal epileptic sources:** as can be stated from the obtained results, all DL-based methods together with SISSY generally succeed in localizing focal superficial epileptic sources. This is contrary to the other considered methods, namely wMNE, sLORETA and cMEM which to a large extent provide blurred solutions, for all conducted experiments.

**Localization of deep epileptic sources:** As shown in Table 3, the proposed MS-ESI method shows surprisingly its superiority in targeting deep epileptic patches compared to all the methods under study. In fact, rather than the proposed MS-ESI model, several inadequate outcomes can be observed for the other six tested methods. Some of them over estimate the spatial extent of the epileptic patch location while others entirely miss this deep patch.

**Recognizing close correlated sources:** As stated before, this case stands for the most challenging situation being faced in the framework of EEG inverse problem. Traditional numerical methods often face challenges when it comes to distinguishing and separating two closely correlated epileptic sources. As mentioned earlier, the proposed MS-ESI model stands out as the only approach achieving a detection rate exceeding 50 % among all the methods tested. The MS-ESI model's ability to gain a more profound insight into complex EEG characteristics and relationships from various perspectives leads to improved accuracy in source localization. This advancement holds significant importance for both neuroscientific research and clinical diagnostic applications.

Accurate source localization and source signal reconstruction are two desirable goals for the ESI techniques. It is generally believed that precise source signal reconstruction for dipoles within the source patch is a prerequisite for source localization. However, if the ESI methods, such as wMNE, sLORETA and cMEM, can only reconstruct the source signal for a very small number of dipoles within the source patch, the spatial extent of the source will be underestimated no matter how accurate its reconstructed signal is. On the contrary, although the signal reconstruction of the proposed method is slightly lower than that of the classical methods, it can reconstruct the signal of equivalent quality for the majority of dipoles within the source patch. This demonstrates that the proposed method can make a trade-off between the accurate source localization and satisfactory source signal reconstruction. In addition, our statistical analysis showed that the proposed method significantly outperforms the baseline methods in terms of DLE and NHD, whatever the SNR level.

Moreover, note that the choice of an optimal threshold that guarantees the best localization results is crucial in the ESI framework. Optimally, the best method is the one for which no thresholding is applied or at least a low threshold is to be used. Regarding this constraint, the proposed MS-ESI method is very competitive especially for real data where no ground truth is available. Clearly, MS-ESI belongs to the set of methods that give more focal solutions whatever the analyzed patient. As pointed out previously, a good localization of epileptic patches is mandatory for clinical perspective, but being able to have a good reconstruction of the neural activities of the current dipoles constituting these patches is also of great importance for a deep investigation of brain behavior, for instance through the inference of brain connectivity. Thus, the most effective ESI method stands for the one that provides the best source localization-time course reconstruction trade-off. In view of this criterion, and according to our obtained results, the proposed MS-ESI approach is the one for which the latter trade-off is maximized.

As shown in Section 3.2, the MS-ESI method maintains its good performance on real Hr-EEG data acquired in three patients with drug-resistant partial epilepsy, during their presurgical evaluation. By

considering simultaneously the results obtained for the three patients, the three methods based on deep learning provide a better source localization-time course reconstruction trade-off than SISSY, wMNE, sLORETA and cMEM. The APCCs computed over the three patients are equal to 0.9122, 0.8782 and 0.9306 for MS-ESI, DeepSIF and Deep-BrainNNet, respectively, showing a good EEG signal reconstruction. But MS-ESI is the only one which localizes a well circumscribed region whatever the considered patient, hence the best source localization-time course reconstruction trade-off offered by MS-ESI. Finally, regarding the execution time shown in Table 2, it should be noted that even significant learning time is required to learn the model, in an offline mode, this issue is not a matter when the model is applied on new data. Indeed, once a deep learning model is trained offline, it can rapidly process new scalp data in real-time or near-real-time to determine the location of the epileptic source. This makes it a potential candidate for real-time source imaging in clinical practice.

The limitations of the proposed work primarily revolve around two key aspects: the availability of experimental data and the deep learning model used. In terms of experimental data, validating the proposed method with real data presented a challenge. It is indeed a complex task to assess the accuracy of source localization solutions when simultaneous EEG/SEEG recordings are not available. Determining the precise locations of these remote regions remains subjective, making the use of realistic simulated data, where ground-truth information is accessible, a reasonable approach. In this study, the effectiveness of the proposed method was confirmed on a limited scale, involving only three patients for whom clinical experts provided some indications of ground truth. These promising results must be validated on a larger dataset before considering clinical application. Moreover, the current work utilized HR-EEG data with 256 channels, as high-density electrodes enhance spatial resolution, the availability of such HR-EEG system may not be consistent across all clinical settings. Future work will necessitate systematic validation of the model's performance under different electrode configurations to assess its applicability in various clinical conditions. Regarding the deep learning model, different deep learning architectures will influence the performance of source EEG signal reconstruction. In future work, comprehensive comparative experiments will be constructed to obtain a relatively optimal model. This will include systematic testing of encoding-decoding, autoencoder, and attention mechanisms. An additional effort will concern the question of our model explainability as an emerging topic to enhance transparency of DL models.

## 5. Conclusion

This paper introduces a novel neural network framework, MS-ESI, designed for the identification and reconstruction of extended epileptic sources. MS-ESI leverages TCN for extracting relevant spatial features and LSTM for capturing temporal characteristics from high-resolution scalp EEG signals. A pivotal aspect of our study is the use of realistic simulated data for model training, addressing the challenge of lacking co-registration between EEG and SEEG signals. In comparison to traditional numerical methods, our deep learning-based approach autonomously learns prior information and potential features from a large training dataset. Experimental results on simulated data showcased that, when compared to state-of-the-art methods, MS-ESI excelled in efficiently localizing the source position, particularly for deep and correlated epileptic sources. This superior performance was further validated on real high-resolution EEG data obtained from three patients with drug-resistant partial epilepsy during their presurgical evaluation. Given its impressive capabilities, MS-ESI shows promise for clinical EEG applications. Future work will focus on evaluating the proposed approach on a broader real-world dataset and exploring its explainability.
the other considered DL-based methods, MS-ESI significantly improved the performance in terms of source localization especially for challenging scenarios (Scenarios 2 and 3).

Three challenges in ESI framework are commonly known and worth to be discussed here: localizing i) focal epileptic sources, ii) deep epileptic sources and iii) close correlated epileptic sources.

**Localization of focal epileptic sources:** as can be stated from the obtained results, all DL-based methods together with SISSY generally succeed in localizing focal superficial epileptic sources. This is contrary to the other considered methods, namely wMNE, sLORETA and cMEM which to a large extent provide blurred solutions, for all conducted experiments.

**Localization of deep epileptic sources:** As shown in Table 3, the proposed MS-ESI method shows surprisingly its superiority in targeting deep epileptic patches compared to all the methods under study. In fact, rather than the proposed MS-ESI model, several inadequate outcomes can be observed for the other six tested methods. Some of them over estimate the spatial extent of the epileptic patch location while others entirely miss this deep patch.

**Recognizing close correlated sources:** As stated before, this case stands for the most challenging situation being faced in the framework of EEG inverse problem. Traditional numerical methods often face challenges when it comes to distinguishing and separating two closely correlated epileptic sources. As mentioned earlier, the proposed MS-ESI model stands out as the only approach achieving a detection rate exceeding 50 % among all the methods tested. The MS-ESI model's ability to gain a more profound insight into complex EEG characteristics and relationships from various perspectives leads to improved accuracy in source localization. This advancement holds significant importance for both neuroscientific research and clinical diagnostic applications.

Accurate source localization and source signal reconstruction are two desirable goals for the ESI techniques. It is generally believed that precise source signal reconstruction for dipoles within the source patch is a prerequisite for source localization. However, if the ESI methods, such as wMNE, sLORETA and cMEM, can only reconstruct the source signal for a very small number of dipoles within the source patch, the spatial extent of the source will be underestimated no matter how accurate its reconstructed signal is. On the contrary, although the signal reconstruction of the proposed method is slightly lower than that of the classical methods, it can reconstruct the signal of equivalent quality for the majority of dipoles within the source patch. This demonstrates that the proposed method can make a trade-off between the accurate source localization and satisfactory source signal reconstruction. In addition, our statistical analysis showed that the proposed method significantly outperforms the baseline methods in terms of DLE and NHD, whatever the SNR level.

Moreover, note that the choice of an optimal threshold that guarantees the best localization results is crucial in the ESI framework. Optimally, the best method is the one for which no thresholding is applied or at least a low threshold is to be used. Regarding this constraint, the proposed MS-ESI method is very competitive especially for real data where no ground truth is available. Clearly, MS-ESI belongs to the set of methods that give more focal solutions whatever the analyzed patient. As pointed out previously, a good localization of epileptic patches is mandatory for clinical perspective, but being able to have a good reconstruction of the neural activities of the current dipoles constituting these patches is also of great importance for a deep investigation of brain behavior, for instance through the inference of brain connectivity. Thus, the most effective ESI method stands for the one that provides the best source localization-time course reconstruction trade-off. In view of this criterion, and according to our obtained results, the proposed MS-ESI approach is the one for which the latter trade-off is maximized.

As shown in Section 3.2, the MS-ESI method maintains its good performance on real Hr-EEG data acquired in three patients with drug-resistant partial epilepsy, during their presurgical evaluation. By

considering simultaneously the results obtained for the three patients, the three methods based on deep learning provide a better source localization-time course reconstruction trade-off than SISSY, wMNE, sLORETA and cMEM. The APCCs computed over the three patients are equal to 0.9122, 0.8782 and 0.9306 for MS-ESI, DeepSIF and Deep-BrainNNet, respectively, showing a good EEG signal reconstruction. But MS-ESI is the only one which localizes a well circumscribed region whatever the considered patient, hence the best source localization-time course reconstruction trade-off offered by MS-ESI. Finally, regarding the execution time shown in Table 2, it should be noted that even significant learning time is required to learn the model, in an offline mode, this issue is not a matter when the model is applied on new data. Indeed, once a deep learning model is trained offline, it can rapidly process new scalp data in real-time or near-real-time to determine the location of the epileptic source. This makes it a potential candidate for real-time source imaging in clinical practice.

The limitations of the proposed work primarily revolve around two key aspects: the availability of experimental data and the deep learning model used. In terms of experimental data, validating the proposed method with real data presented a challenge. It is indeed a complex task to assess the accuracy of source localization solutions when simultaneous EEG/SEEG recordings are not available. Determining the precise locations of these remote regions remains subjective, making the use of realistic simulated data, where ground-truth information is accessible, a reasonable approach. In this study, the effectiveness of the proposed method was confirmed on a limited scale, involving only three patients for whom clinical experts provided some indications of ground truth. These promising results must be validated on a larger dataset before considering clinical application. Moreover, the current work utilized HR-EEG data with 256 channels, as high-density electrodes enhance spatial resolution, the availability of such HR-EEG system may not be consistent across all clinical settings. Future work will necessitate systematic validation of the model's performance under different electrode configurations to assess its applicability in various clinical conditions. Regarding the deep learning model, different deep learning architectures will influence the performance of source EEG signal reconstruction. In future work, comprehensive comparative experiments will be constructed to obtain a relatively optimal model. This will include systematic testing of encoding-decoding, autoencoder, and attention mechanisms. An additional effort will concern the question of our model explainability as an emerging topic to enhance transparency of DL models.

## 5. Conclusion

This paper introduces a novel neural network framework, MS-ESI, designed for the identification and reconstruction of extended epileptic sources. MS-ESI leverages TCN for extracting relevant spatial features and LSTM for capturing temporal characteristics from high-resolution scalp EEG signals. A pivotal aspect of our study is the use of realistic simulated data for model training, addressing the challenge of lacking co-registration between EEG and SEEG signals. In comparison to traditional numerical methods, our deep learning-based approach autonomously learns prior information and potential features from a large training dataset. Experimental results on simulated data showcased that, when compared to state-of-the-art methods, MS-ESI excelled in efficiently localizing the source position, particularly for deep and correlated epileptic sources. This superior performance was further validated on real high-resolution EEG data obtained from three patients with drug-resistant partial epilepsy during their presurgical evaluation. Given its impressive capabilities, MS-ESI shows promise for clinical EEG applications. Future work will focus on evaluating the proposed approach on a broader real-world dataset and exploring its explainability.
## CRediT authorship contribution statement

**Zuyi Yu:** Conceptualization, Methodology, Software, Investigation, Visualization, Formal analysis, Writing – original draft. **Amar Kachenoura:** Conceptualization, Writing – original draft. **Régine Le Bouquin Jeannès:** Writing – review & editing, Supervision. **Huazhong Shu:** Resources, Funding acquisition, Writing – review & editing, Supervision. **Paul Berraute:** Software, Validation. **Anca Nica:** Writing – review & editing. **Isabelle Merlet:** Data curation. **Laurent Albera:** Resources, Writing – review & editing, Supervision. **Ahmad Karfoul:** Conceptualization, Writing – original draft.

## Declaration of Competing Interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled.

## Data availability

The authors do not have permission to share data.

## Acknowledgements

This research was supported by the National Key Research and Development Program of China (2022YFE0116700), the National Natural Science Foundation of China (62171125, 61876037).

## Data availability

The data are not publicly available due to privacy or ethical restrictions.

## Code availability

The code that supports the findings of this study are available on reasonable request from the corresponding author.

## Data and code availability statement

The code that supports the findings of this study are available on reasonable request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.neuroimage.2023.120490.

## References

* Busch, N.A., VanRullen, R., 2010. Spontaneous EEG oscillations reveal periodic sampling of visual attention. Proc. Natl. Acad. Sci. 107 (37), 16048-16053.

* Sun, R., Sohrabpour, A., Worrell, G.A., He, B., 2022. Deep neural networks constrained by neural mass models improve electrophysiological source imaging of spatiotemporal brain dynamics. Proc. Natl. Acad. Sci. 119 (31), e2201128119.

* Yu, Z., Albera, L., Le Bouquin Jeannes, R., Kachenoura, A., Karfoul, A., Yang, C., Shu, H., 2022. Epileptic seizure prediction using deep neural networks via transfer learning and multi-feature fusion. Int. J. Neural Syst. 32 (07), 2250032.

* Numata, T., Kiguchi, M., Sato, H., 2019. Multiple-Time-Scale analysis of attention as revealed by EEG, NIRS, and pupil diameter signals during a free recall task: a multimodal measurement approach. Front. Neurosci. 13, 1307.

* He, B., Sohrabpour, A., Brown, E., Liu, Z., 2018. Electrophysiological source imaging: a noninvasive window to brain dynamics. Annu. Rev. Biomed. Eng. 20, 171-196.

* Gross, J., Kujala, J., Hämäläinen, M., Timmermann, L., Schnitzler, A., Salmelin, R., 2001. Dynamic imaging of coherent sources: studying neural interactions in the human brain. Proc. Natl. Acad. Sci. 98 (2), 694-699.

* Bore, J.C., Li, P., Jiang, L., Ayedh, W.M., Chen, C., Harmah, D.J., Yao, D., Cao, Z., Xu, P., 2021. A long short-term memory network for sparse spatiotemporal EEG source imaging. IEEE Trans. Med. Imaging 40 (12), 3787-3800.

* He, B., Yang, L., Wilke, C., Yuan, H., 2011. Electrophysiological imaging of brain activity and connectivity—challenges and opportunities. IEEE Trans. Biomed. Eng. 58 (7), 1918-1931.

* Bailleul, S., Mosher, J.C., Leahy, R.M., 2001. Electromagnetic brain mapping. IEEE Signal Proc. Mag. 18 (6), 14-30.

* Martínez-Vargas, J.D., Duque-Muñoz, L., Vargas-Bonilla, F., López, J.D., Castellanos-Dominguez, G., 2019. Enhanced data covariance estimation using weighted combination of multiple Gaussian kernels for improved M/EEG source localization. Int. J. Neural Syst. 29 (06), 1950001.

* Liu, K., Yu, Z.L., Wu, W., Gu, Z., Li, Y., Nagarajan, S., 2018. Variation sparse source imaging based on conditional mean for electromagnetic extended sources. Neurocomputing 313, 96-110.

* He, B., Musha, T., Okamoto, Y., Homma, S., Nakajima, Y., Sato, T., 1987. Electric dipole tracing in the brain by means of the boundary element method and its accuracy. IEEE Trans. Biomed. Eng. 6, 406-414.

* Scherg, M., Von Cramon, D., 1986. Evoked dipole source potentials of the human auditory cortex. Electroenceph. Clin. Neurophysiol. 65 (5), 344-360.

* Pascual-Marqui, R.D., 2002. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find. Exp. Clin. Pharmacol. 24, 5-12. Suppl D.

* Dale, A.M., Sereno, M.I., 1993. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: a linear approach. J. Cogn. Neurosci. 5 (2), 162-176.

* Van Veen, B.D., Van Drongelen, W., Yuchtman, M., Suzuki, A., 1997. Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans. Biomed. Eng. 44 (9), 867-880.

* Kiebel, S.J., Daunizeau, J., Phillips, C., Friston, K.J., 2008. Variational Bayesian inversion of the equivalent current dipole model in EEG/MEG. Neuroimage 39 (2), 728-741.

* Liu, K., Yu, Z.L., Wu, W., Gu, Z., Li, Y., 2015. Straps: a fully data-driven spatio-temporally regularized algorithm for M/EEG patch source imaging. Int. J. Neural Syst. 25 (04), 1550016.

* Cai, C., Sekihara, K., Nagarajan, S.S., 2018. Hierarchical multiscale Bayesian algorithm for robust MEG/EEG source reconstruction. Neuroimage 183, 698-715.

* Wu, W., Nagarajan, S., Chen, Z., 2015. Bayesian Machine Learning: EEG\MEG signal processing measurements. IEEE Signal Proc. Mag. 33 (1), 14-36.

* Abeyratne, U.R., Zhang, G., Saratchandran, P., 2001. EEG source localization: a comparative study of classical and neural network methods. Int. J. Neural Syst. 11 (04), 349-359.

* Hämäläinen, M.S., Ilmoniemi, R.J., 1994. Interpreting magnetic fields of the brain: minimum norm estimates. Med. Biol. Eng. Comput. 32, 35-42.

* Pascual-Marqui, R.D., Michel, C.M., Lehmann, D., 1994. Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. Int. J. Psychophysiol. 18 (1), 49-65.

* Grova, C., Daunizeau, J., Lina, J.M., Bénar, C.G., Benali, H., Gotman, J., 2006. Evaluation of EEG localization methods using realistic simulations of interictal spikes. Neuroimage 29 (3), 734-753.

* Krizhevsky, A., Sutskever, I., Hinton, G.E., 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60 (6), 84-90.

* Nadkarni, P.M., Ohno-Machado, L., Chapman, W.W., 2011. Natural language processing: an introduction. J. Am. Med. Inform. Assoc. 18 (5), 544-551.

* Li, Y., Yu, Z., Chen, Y., Yang, C., Li, Y., Li, X.A., Li, B., 2020. Automatic seizure detection using fully convolutional nested LSTM. Int. J. Neural Syst. 30 (04), 2050019.

* Awan, F.G., Saleem, O., Kiran, A., 2019. Recent trends and advances in solving the inverse problem for EEG source localization. Inverse Probl. Sci. En. 27 (11), 1521-1536.

* Zorzos, I., Kakkos, I., Ventouras, E.M., Matsopoulos, G.K., 2021. Advances in electrical source imaging: a review of the current approaches, applications and challenges. Signals 2 (3), 378-391.

* Becker, H., Albera, L., Comon, P., Nunes, J.C., Gribonval, R., Fleureau, J., Guillotel, P., Merlet, I., 2017. SISSY: an efficient and automatic algorithm for the analysis of EEG sources based on structured sparsity. Neuroimage 157, 157-172.

* Chowdhury, R.A., Lina, J.M., Kobayashi, E., Grova, C., 2013. MEG source localization of spatially extended generators of epileptic activity: comparing entropic and hierarchical bayesian approaches. PLoS ONE 8 (2), e55969.

* Wendling, F., Bartolomei, F., Bellanger, J.J., Chauvel, P., 2002. Epileptic fast activity can be explained by a model of impaired GABAergic dendritic inhibition. Eur. J. Neurosci. 15 (9), 1499-1508.

* Wendling, F., Hernandez, A., Bellanger, J.J., Chauvel, P., Bartolomei, F., 2005. Interictal to ictal transition in human temporal lobe epilepsy: insights from a computational model of intracerebral EEG. J. Clin. Neurophysiol. 22 (5), 343.

* Lea, C., Flynn, M.D., Vidal, R., Reiter, A., Hager, G.D., 2017. Temporal convolutional networks for action segmentation and detection. Proc. IEEE Conf. Comput. Vis. Pattern Recogn 156-165.

* Hochreiter, S., Schmidhuber, J., 1997. Long short-term memory. Neural Comput. 9 (8), 1735-1780.

* Rivière, D., Geffroy, D., Denghien, I., Souedet, N., Cointepas, Y., 2009. BrainVISA: an extensible software environment for sharing multimodal neuroimaging data and processing tools. Neuroimage (47), S163.

* Gramfort, A., Papadopoulo, T., Olivi, E., Clerc, M., 2010. OpenMEEG: opensource software for quasistatic bioelectromagnetics. Biomed. Eng. 9, 1-20. Online.
## CRediT authorship contribution statement

**Zuyi Yu:** Conceptualization, Methodology, Software, Investigation, Visualization, Formal analysis, Writing – original draft. **Amar Kachenoura:** Conceptualization, Writing – original draft. **Régine Le Bouquin Jeannes:** Writing – review & editing, Supervision. **Huazhong Shu:** Resources, Funding acquisition, Writing – review & editing, Supervision. **Paul Berraute:** Software, Validation. **Anca Nica:** Writing – review & editing. **Isabelle Merlet:** Data curation. **Laurent Albera:** Resources, Writing – review & editing, Supervision. **Ahmad Karfoul:** Conceptualization, Writing – original draft.

## Declaration of Competing Interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled.

## Data availability

The authors do not have permission to share data.

## Acknowledgements

This research was supported by the National Key Research and Development Program of China (2022YFE0116700), the National Natural Science Foundation of China (62171125, 61876037).

## Data availability

The data are not publicly available due to privacy or ethical restrictions.

## Code availability

The code that supports the findings of this study are available on reasonable request from the corresponding author.

## Data and code availability statement

The code that supports the findings of this study are available on reasonable request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.neuroimage.2023.120490.

## References

* Busch, N.A., VanRullen, R., 2010. Spontaneous EEG oscillations reveal periodic sampling of visual attention. Proc. Natl. Acad. Sci. 107 (37), 16048-16053.

* Sun, R., Sohrabpour, A., Worrell, G.A., He, B., 2022. Deep neural networks constrained by neural mass models improve electrophysiological source imaging of spatiotemporal brain dynamics. Proc. Natl. Acad. Sci. 119 (31), e2201128119.

* Yu, Z., Albera, L., Le Bouquin Jeannes, R., Kachenoura, A., Karfoul, A., Yang, C., Shu, H., 2022. Epileptic seizure prediction using deep neural networks via transfer learning and multi-feature fusion. Int. J. Neural Syst. 32 (07), 2250032.

* Numata, T., Kiguchi, M., Sato, H., 2019. Multiple-Time-Scale analysis of attention as revealed by EEG, NIRS, and pupil diameter signals during a free recall task: a multimodal measurement approach. Front. Neurosci. 13, 1307.

* He, B., Sohrabpour, A., Brown, E., Liu, Z., 2018. Electrophysiological source imaging: a noninvasive window to brain dynamics. Annu. Rev. Biomed. Eng. 20, 171-196.

* Gross, J., Kujala, J., Hämäläinen, M., Timmermann, L., Schnitzler, A., Salmelin, R., 2001. Dynamic imaging of coherent sources: studying neural interactions in the human brain. Proc. Natl. Acad. Sci. 98 (2), 694-699.

* Bore, J.C., Li, P., Jiang, L., Ayedh, W.M., Chen, C., Harmah, D.J., Yao, D., Cao, Z., Xu, P., 2021. A long short-term memory network for sparse spatiotemporal EEG source imaging. IEEE Trans. Med. Imaging 40 (12), 3787-3800.

* He, B., Yang, L., Wilke, C., Yuan, H., 2011. Electrophysiological imaging of brain activity and connectivity—challenges and opportunities. IEEE Trans. Biomed. Eng. 58 (7), 1918-1931.

* Bailleul, S., Mosher, J.C., Leahy, R.M., 2001. Electromagnetic brain mapping. IEEE Signal Proc. Mag. 18 (6), 14-30.

* Martínez-Vargas, J.D., Duque-Muñoz, L., Vargas-Bonilla, F., López, J.D., Castellanos-Dominguez, G., 2019. Enhanced data covariance estimation using weighted combination of multiple Gaussian kernels for improved M/EEG source localization. Int. J. Neural Syst. 29 (06), 1950001.

* Liu, K., Yu, Z.L., Wu, W., Gu, Z., Li, Y., Nagarajan, S., 2018. Variation sparse source imaging based on conditional mean for electromagnetic extended sources. Neurocomputing 313, 96-110.

* He, B., Musha, T., Okamoto, Y., Homma, S., Nakajima, Y., Sato, T., 1987. Electric dipole tracing in the brain by means of the boundary element method and its accuracy. IEEE Trans. Biomed. Eng. 6, 406-414.

* Scherg, M., Von Cramon, D., 1986. Evoked dipole source potentials of the human auditory cortex. Electroenceph. Clin. Neurophysiol. 65 (5), 344-360.

* Pascual-Marqui, R.D., 2002. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find. Exp. Clin. Pharmacol. 24, 5-12. Suppl D.

* Dale, A.M., Sereno, M.I., 1993. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: a linear approach. J. Cogn. Neurosci. 5 (2), 162-176.

* Van Veen, B.D., Van Drongelen, W., Yuchtman, M., Suzuki, A., 1997. Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans. Biomed. Eng. 44 (9), 867-880.

* Kiebel, S.J., Daunizeau, J., Phillips, C., Friston, K.J., 2008. Variational Bayesian inversion of the equivalent current dipole model in EEG/MEG. Neuroimage 39 (2), 728-741.

* Liu, K., Yu, Z.L., Wu, W., Gu, Z., Li, Y., 2015. Straps: a fully data-driven spatio-temporally regularized algorithm for M/EEG patch source imaging. Int. J. Neural Syst. 25 (04), 1550016.

* Cai, C., Sekihara, K., Nagarajan, S.S., 2018. Hierarchical multiscale Bayesian algorithm for robust MEG/EEG source reconstruction. Neuroimage 183, 698-715.

* Wu, W., Nagarajan, S., Chen, Z., 2015. Bayesian Machine Learning: EEG\MEG signal processing measurements. IEEE Signal Proc. Mag. 33 (1), 14-36.

* Abeyratne, U.R., Zhang, G., Saratchandran, P., 2001. EEG source localization: a comparative study of classical and neural network methods. Int. J. Neural Syst. 11 (04), 349-359.

* Hämäläinen, M.S., Ilmoniemi, R.J., 1994. Interpreting magnetic fields of the brain: minimum norm estimates. Med. Biol. Eng. Comput. 32, 35-42.

* Pascual-Marqui, R.D., Michel, C.M., Lehmann, D., 1994. Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. Int. J. Psychophysiol. 18 (1), 49-65.

* Grova, C., Daunizeau, J., Lina, J.M., Bénar, C.G., Benali, H., Gotman, J., 2006. Evaluation of EEG localization methods using realistic simulations of interictal spikes. Neuroimage 29 (3), 734-753.

* Krizhevsky, A., Sutskever, I., Hinton, G.E., 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60 (6), 84-90.

* Nadkarni, P.M., Ohno-Machado, L., Chapman, W.W., 2011. Natural language processing: an introduction. J. Am. Med. Inform. Assoc. 18 (5), 544-551.

* Li, Y., Yu, Z., Chen, Y., Yang, C., Li, Y., Li, X.A., Li, B., 2020. Automatic seizure detection using fully convolutional nested LSTM. Int. J. Neural Syst. 30 (04), 2050019.

* Awan, F.G., Saleem, O., Kiran, A., 2019. Recent trends and advances in solving the inverse problem for EEG source localization. Inverse Probl. Sci. En. 27 (11), 1521-1536.

* Zorzos, I., Kakkos, I., Ventouras, E.M., Matsopoulos, G.K., 2021. Advances in electrical source imaging: a review of the current approaches, applications and challenges. Signals 2 (3), 378-391.

* Becker, H., Albera, L., Comon, P., Nunes, J.C., Gribonval, R., Fleureau, J., Guillotel, P., Merlet, I., 2017. SISSY: an efficient and automatic algorithm for the analysis of EEG sources based on structured sparsity. Neuroimage 157, 157-172.

* Chowdhury, R.A., Lina, J.M., Kobayashi, E., Grova, C., 2013. MEG source localization of spatially extended generators of epileptic activity: comparing entropic and hierarchical bayesian approaches. PLoS ONE 8 (2), e55969.

* Wendling, F., Bartolomei, F., Bellanger, J.J., Chauvel, P., 2002. Epileptic fast activity can be explained by a model of impaired GABAergic dendritic inhibition. Eur. J. Neurosci. 15 (9), 1499-1508.

* Wendling, F., Hernandez, A., Bellanger, J.J., Chauvel, P., Bartolomei, F., 2005. Interictal to ictal transition in human temporal lobe epilepsy: insights from a computational model of intracerebral EEG. J. Clin. Neurophysiol. 22 (5), 343.

* Lea, C., Flynn, M.D., Vidal, R., Reiter, A., Hager, G.D., 2017. Temporal convolutional networks for action segmentation and detection. Proc. IEEE Conf. Comput. Vis. Pattern Recogn 156-165.

* Hochreiter, S., Schmidhuber, J., 1997. Long short-term memory. Neural Comput. 9 (8), 1735-1780.

* Rivière, D., Geffroy, D., Denghien, I., Souedet, N., Cointepas, Y., 2009. BrainVISA: an extensible software environment for sharing multimodal neuroimaging data and processing tools. Neuroimage (47), S163.

* Gramfort, A., Papadopoulo, T., Olivi, E., Clerc, M., 2010. OpenMEEG: opensource software for quasistatic bioelectromagnetics. Biomed. Eng. 9, 1-20. Online.
Tadel, F., Baillet, S., Mosher, J.C., Pantazis, D., Leahy, R.M., 2011. Brainstorm: a user-
friendly application for MEG/EEG analysis. Comput. Intell. Neurosci. 1–13, 2011.

Otsu, N., 1979. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man. Cy. 9 (1), 62–66.

Rollo, P.S., Rollo, M.J., Zhu, P., Woolnough, O., Tandon, N., 2020. Oblique trajectory angles in robotic stereo-electroencephalography. J. Neurosurg. 135 (1), 245–254.
Tadel, F., Baillet, S., Mosher, J.C., Pantazis, D., Leahy, R.M., 2011. Brainstorm: a user-
friendly application for MEG/EEG analysis. Comput. Intell. Neurosci. 1–13, 2011.

Otsu, N., 1979. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man. Cy. 9 (1), 62–66.

Rollo, P.S., Rollo, M.J., Zhu, P., Woolnough, O., Tandon, N., 2020. Oblique trajectory angles in robotic stereo-electroencephalography. J. Neurosurg. 135 (1), 245–254.