# Improved estimation of EEG evoked potentials by jitter compensation and enhancing spatial filters

Antoine Souloumiac, Bertrand Rivet

► To cite this version:

Antoine Souloumiac, Bertrand Rivet. Improved estimation of EEG evoked potentials by jitter compensation and enhancing spatial filters. ICASSP 2013 - 38th IEEE International Conference on Acoustics, Speech and Signal Processing, May 2013, Vancouver, Canada. pp.1222-1226, 10.1109/ICASSP.2013.6637845 . cea-01830773

HAL Id: cea-01830773

https://cea.hal.science/cea-01830773v1

Submitted on 24 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
IMPROVED ESTIMATION OF EEG EVOKED POTENTIALS BY JITTER COMPENSATION
AND ENHANCING SPATIAL FILTERS

Antoine Souloumiac*

CEA, LIST,
Laboratoire Outils d'Analyse des données,
91191 Gif-sur-Yvette CEDEX, France

Bertrand Rivet

GIPSA-lab, CNRS-UMR5216
Grenoble University,
Grenoble, France

ABSTRACT

We propose in this paper a new technique to investigate the Event-Related Potentials, or Evoked-Response Potentials, in the electroencephalographic signal. The multidimensional electroencephalographic signal is first spatially filtered to enhance the Evoked-Response Potentials using the xDAWN algorithm and, second, the single trial latencies (whatever their origins: physiological or electronical) are estimated by maximizing a cross correlation without any a priori model. The performance of this approach is illustrated on two classical P300-Speller electroencephalographic databases (BCI Competition II and III). The single-trial distribution of P300 Evoked-Response Potential is deblurred using the proposed resynchronization algorithm for applications in particular to Brain Computer Interfaces.

***Index Terms***— Brain Computer Interface, spatial filter, jitter compensation, single trial, Event-Related Potential

# 1. INTRODUCTION

Brain Computer Interfaces (BCI) allow a subject to control a device without any muscular activity [1]. They are generally based on the acquisition and the analysis of scalp recorded electroencephalography signals (EEG). In several BCI systems, the subject is submitted to different classes of stimulations generating corresponding cerebral responses referred as event-related potentials (ERP) or evoked potentials [2]. The subject is asked to choose one class of stimuli called the target stimuli. Then the BCI system provides the subject with a mixture of target and non-target stimulations, records every cerebral responses and detects which is the target class that had been intentionally chosen by the subject. This detection is based on the fact that the ERP related to a target stimulation, the well-known P300 ERP, is different from the response related to non-target stimulations. This is the principle of the famous P300-Speller BCI [3, 4]. More specifically in the P300-Speller case, the subject is watching a screen with

a 6-by-6 grid of 36 letters and figures and the stimulation consists in highlighting randomly one row or one column of this grid. The subject chooses one symbol and is asked to focus on its brightness. Each time the chosen symbol column or row is highlighted one P300 ERP is generated by the brain. P300 or P3 means that the ERP is positive and emitted roughly 300 milliseconds after the target stimulation.

Obviously, the ERP detection algorithms take advantage of the synchronization between the stimulus and the evoked potential. Nevertheless, the 300 milliseconds latency is not constant and can be affected by different kinds of variability. Firstly, the “natural” latency between the stimulus perception and the ERP depends for instance on the current cerebral work load. Secondly, the delay is also affected by any jitter between the stimulation device clock and the EEG recording device clock. This “electronic” delay can be minimized by using specific synchronization hardware but this is complex and not commonly done in practice. Both “natural” and “electronic” jitter amplitude are often similar to the P300 temporal width (i.e. several tenths of milliseconds). Consequently, it is clear that such a jitter amplitude significantly reduces the relevance of the estimated ERP shapes for neuroscience/cognitive interpretations or the global performance of many classification algorithms and the corresponding ERP-based synchronous BCIs. It also prevents from merging different EEG recording sessions in one homogeneous data set and can create a discrepancy between the training and test data sets.

A solution to this problem is to resynchronize the stimulations and EEG timescales. We propose to do it by achieving a single-trial detection and resynchronization of each ERPs. This resynchronization has several applications: assess the jitters affecting an existing database and reduce them, merge several EEG databases affected by inhomogeneous jitter distributions, evaluate the jitter amplitude of a given BCI device and minimize it by tuning electronic parameters, improve the target/non-target classification performance, improve the characterization of the ERP distribution, detect unknown ERPs that so far had remained hidden due to desynchronization. Of course, this method can be generalized to any detectable ERP, and any other areas where a system is char-

<footnote>*This work was supported by ANR (France) grant ANR-2009-CORD-017.</footnote>
acterized by investigating the responses to short stimulations.

The paper is organized as follow: the resynchronization method using xDAWN is described in the second section. Section 3 shows the performance of the proposed method on actual BCI data. Finally, Section 4 concludes this paper by a discussion and perspectives.

## 2. ERP ENHANCEMENT AND LATENCY ESTIMATION

The proposed method is based on two steps: the ERPs are first enhanced by one-dimensional spatial filtering (Section 2.1) and then the latency of each single trial ERP is estimated by intercorrelation with a mean ERP profile (Section 2.2).

### 2.1. The xDAWN algorithm

With P300-speller BCI paradigm, the multidimensional EEG signal is modeled as the sum of

* the target-ERP component,

* other ERP components also related to the stimulations (non-target ERP or the union of target and non-target ERP for instance), and

* the remaining EEG independent of any stimulations.

Each ERP component is assumed to be also the superposition of one multi-dimensional pattern (i.e. a matrix) at the rhythm of the corresponding stimulations. The xDAWN algorithm [5, 6] provides an estimation of each ERP pattern and the spatial filters which maximize the relative power of the ERP component of interest versus the other ERPs and the remaining stimulation independent EEG.

More specifically, let's denote $N_E$ the number of electrodes, $N_S$ the number of EEG time samples and $X$ the $N_S \times N_E$ EEG signal matrix (one column by electrode). If the ERP pattern is a $N_P \times N_E$ matrix denoted $A$ then we assume the one-ERP EEG model

$$X = DA + N \quad (1)$$

where $D$ is Toeplitz $N_S \times N_P$ whose entries are 0 or 1; the first column entries are 1 if the time index is the starting time (i.e. onset) of one occurrence of the corresponding ERP, and 0 otherwise. In the case of several superimposed ERPs of respective patterns $A_1, A_2$, etc. we have

$$X = D_1A_1 + D_2A_2 + \dots + N = DA + N \quad (2)$$

with the same definition of the Toeplitz matrices $D_1, D_2$, etc. and, with $D$ the horizontal concatenation of the $D_1, D_2$, etc. and $A$ the vertical concatenation of $A_1, A_2$, etc. A least squares estimation of the concatenated ERP patterns gives

$$\hat{A} = (D^T D)^{-1} D^T X \quad (3)$$

Each ERP pattern $\hat{A}_1, \hat{A}_2$, etc. is obtained as the corresponding horizontal slice of $\hat{A}$.

As a matter of fact, we noted that the target ERPs estimations $\hat{A}_1$ are often corrupted by the periodic rythm of flashes (every 175 milliseconds in the data sets that are used below). Therefore, we postulate that a second ERP $A_2$ is generated synchronized to any row or column highlighting. We observed in practice that this two patterns modelization and the corresponding enhancement with the xDAWN spatial filters yield an estimation of $A_1$ that is acceptably immune to the flashes steady-state visual evoked potential.

The spatial filter $v_1$ for enhancing the target-ERP of pattern $A_1$ and Toeplitz matrix $D_1$ are computed as the first generalized eigenvectors which maximize the Rayleigh quotient (a kind of target-ERP to EEG ratio)

$$v_1 = \arg\max_{v} \frac{v^T \hat{A}_1^T D_1^T D_1 \hat{A}_1 v}{v^T R_X v} \quad (4)$$

$R_X$ is the covariance matrix of the whole EEG signal $X$.

### 2.2. Single-trial ERP delays estimation

The spatial filter $v_1$ which yields the highest ERP enhancement is generally efficient enough to allow a single-trial detection of most of the P300 ERPs. The latency of each ERP is estimated by maximizing the intercorrelation between the ERP-enhanced EEG (i.e. $Xv_1$) and the ERP template profile after the same spatial filtering of the ERP pattern (i.e. $\hat{A}_1v_1$).

Note that the estimation of optimal spatial filters for ERP enhancement (Section 2.1) is based on the a priori knowledge of the ERP latency via the $D$ Toeplitz matrix. Therefore, the proposed estimation of the ERPs jitter allows a correction of the $D$ matrix by adjusting the stimuli onsets and the computation of corrected spatial filters. As a consequence, the full procedure iterates the two described steps

* spatial filter enhancing the ERPs for a given set of stimuli onsets

* correction of the stimuli onsets by estimating the latency of each ERP.

Based on our tests (Section 3), only few iterations are necessary to reach convergence (typically two).

## 3. RESULTS ON ACTUAL P300-SPELLER EEG DATA

The efficiency of the proposed ERP resynchronization technique is validated on two P300-Speller databases available online: the training data set IIb of the BCI Competition II [7] (which is generally considered as an easy set), and the more challenging training data set II of the BCI Competition III [8].
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Intercorrelation as a function of time for all the target ERP trials of BCI II in Fig. 1(a) and BCI III in Fig. 1(b); the different recording sessions are separated by black lines.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Histograms of P300 ERP jitter latencies after the first (left plots) and second (right plots) iterations.</figcaption></figure>

**3.1. Databases**

Both EEG data sets have been recorded at 240Hz with $N_E = 64$ electrodes. The P300-Speller paradigm has been implemented with 15 repetitions of 2 target stimulations (the right row or the right column of the grid is highlighted) and 10 non-target stimulations (wrong rows or columns). The training data set IIb of the BCI Competition II contains two sessions (10 and 11) and several runs; all the runs of both sessions have been concatenated to create a unique EEG signal matrix $X$. The training data set II of the BCI Competition III contains two subjects (A and B) and 85 blocks each corresponding to 85 symbols. We discarded the EEG blocks that are visually extremely corrupted by artifacts (8 for subject A and 10 for subject B) and concatenated the remaining 152 blocks.

**3.2. Estimation of the ERP jitters**

Figure 1 shows the color coded intercorrelation amplitude as a function of the time on the x-axis and the target stimulation on the y-axis. The different sessions, runs and/or subjects have been separated by black lines. The target ERP jitters are very clearly correlated to the run and the session in Figure 1(a). Figure 1(b) shows a more discrete but systematic difference of timing between subject A and subject B. Both figures give an idea of the dispersion of the delays between the stimulation and the actual generation of the target ERP.

**3.3. Convergence of the proposed method**

In this section, we show that the proposed iterative two steps estimation procedure converges for the two data sets in very few iterations. Figure 2 shows the histograms of the estimated target ERP times of occurrence after one and two iterations. Both first iteration histograms (left plots) show the amplitude of the latency jitter (the width of the distribution) and the side

bins amplitude allow to evaluate the probability of finding a maximum to the intercorrelation function in the necessary limited interval of research (here from -30 to +30 and -50 to +50 sampling periods of roughly 4 milliseconds). The second iteration histograms (right plots) show the convergence of the proposed two steps procedure since the jitter latencies after the second iteration are mainly concentrated around 0.

**3.4. ERP distributions characterization**

The ERPs are classically identified by averaging separately target and non-target one-dimensional EEG epochs (one electrode of interest) and detecting by comparison a difference between these temporal profiles [2]. Averaging epochs yields the ERPs enhancement by a kind of temporal filtering. This temporal enhancement is very efficient but the individual ERPs are lost; single-trial characterization is no longer possible. The main interest of ERPs enhancement by spatial filtering is that it allows the characterization of the ERPs distribution (the detection of individual ERPs are even possible in favorable situations). This section investigates these ERP distributions and the blurring effect of the proposed resynchronization method. The distribution of general P300 ERPs and the specific case of two successive P300 ERPs (close target stimulations) are shown.

In all this article, the figures presenting the distributions of ERPs are built in the same way. All the epochs of the ERP of interest are selected in the enhanced channel (i.e. $Xv_1$), then these epochs are superimposed and histograms are computed on each column. In few words, the distribution figures show the color-coded density of ERP trajectories in each pixel.

Figure 3(a) shows the general P300 ERP distribution in the data set IIb of the BCI Competition II without resynchronization (bottom plot) and with resynchronization (top plot). The P300 ERP variability and its ability to be detected by thresholding can be evaluated on these figures. Figure
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** P300 ERP distribution without (bottom plots) and with (top plots) resynchronization. The data set IIb BCI II is in Fig. 3(a) and the data set II BCI III in Fig. 3(b).</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Successive P300 ERP distributions without resynchronization, data set IIb BCI II. From top to bottom, the target stimulations are separated by 175, 350, 525, 700 and 875 ms.</figcaption></figure>

3(b) shows the same distributions but using the data set II of the BCI Competition III without and with resynchronization, which illustrates the variability of the P300 ERP from one data set to another and from one subject to another.

When two successive target stimulations are very close in time (here 175 milliseconds) the second ERP is slightly delayed and of lower amplitude. Figures 4 and 5 illustrate this physiologic phenomenon by showing the corresponding ERP distributions. In this case too, the resynchronization blurs the distribution by compensating latency jitters.

# 4. RELATION TO PRIOR WORK & DISCUSSION

The ERP latency jitter estimation is an old issue in neuroscience and its estimation dates back to the late sixties. Woody [9] estimates the single trial P300 latencies by maximizing the cross-correlation between the one dimensional EEG signal (one electrode) and a P300 template (generally a sine wave). Jaśkowski [10] proposes to jointly estimate

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Successive P300 ERP distribution without resynchronization, data set II BCI III. From top to bottom plots, the target stimulations are separating of 175, 350, 525, 700, 875ms.</figcaption></figure>

the single trial latency and amplitude of P300 via maximum likekihood approach used by Pham [11] for other ERPs. More recently, Li et al. [12] uses spatial filtering to improve the signal (P300) to EEG ratio; latencies and amplitudes are also estimated by maximizing cross correlation between the trial and a model of P300 involving gamma functions. On the contrary, our approach is only based on the EEG data and makes no assumption on the P300 profile (i.e. no parametric model as gamma functions), which is directly estimated from the data using the xDAWN algorithm. It allows in particular to provide an estimation of the ERPs distribution instead of a priori model parameters. The proposed method, based on the xDAWN algorithm, also takes into account the possible inter-classes overlapping when estimating the ERP profile used for the resynchronization. Finally, the resynchronization step provides more accurate spatial filters since the output signal to noise ratio (4) is higher after few iterations than without latencies jitter estimations.

A new method for compensating the latency jitter between a stimulation event and the corresponding EEG event-related potential (ERP) is proposed. This iterative technique shows its good convergence behavior since only a few number of iterations (typically one or two) are necessary. Coupled with the ERP enhancing spatial filtering algorithm xDAWN, it allows to deblur efficiently the ERP distributions. Several ERPs are investigated: the P300 and two successive P300 with several delays between them. The performance of the method are shown on two classical P300-Speller EEG databases of different difficulty.

The proposed technique allows to estimate accurately the ERP times of occurrence when these ERPs are approximately localized in time (the target stimulation times are known in the P300-Speller training data sets used here). It would be of main interest to extend this method to contexts where this a priori knowledge is unknown like in BCI test data sets.
## 5. REFERENCES

[1] B. Z. Allison, E. W. Wolpaw, and J. R. Wolpaw, "Brain-computer interface systems: progress and prospects," *Expert Review of Medical Devices*, vol. 4, no. 4, pp. 463-474, 2007.

[2] S. J. Luck, *An Introduction to the Event-Related Potential Technique*, The MIT Press, 2005.

[3] L. A. Farwell and E. Donchin, "Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials," *Electroencephalography and Clinical Neurophysiology*, vol. 70, no. 6, pp. 510-523, December 1988.

[4] E. Donchin, K.M. Spencer, and R. Wijesinghe, "The mental prosthesis: assessing the speed of a P300-based brain-computer interface," *IEEE Transactions on Rehabilitation Engineering*, vol. 8, no. 2, pp. 174-179, June 2000.

[5] B. Rivet, A. Souloumiac, V. Attina, and G. Gibert, "xDAWN algorithm to enhance evoked potentials: Application to brain-computer interface," *IEEE Transactions on Biomedical Engineering*, vol. 56, no. 8, pp. 2035-2043, August 2009.

[6] B. Rivet and A. Souloumiac, "Optimal linear spatial filters for event-related potentials based on a spatio-temporal model: Asymptotical performance analysis," *Signal Processing*, vol. 93, pp. 387-398, 2013.

[7] B. Blankertz, K.R. Müller, G. Curio, T.M. Vaughan, G. Schalk, J.R. Wolpaw, A. Schlögl, C. Neuper, G. Pfurtscheller, T. Hinterberger, M. Schröder, and N. Birbaumer, "The BCI Competition 2003: progress and perspectives in detection and discrimination of EEG single trials," *IEEE Transaction Biomedical Engineering*, vol. 51, pp. 1044-1051, June 2004.

[8] B. Blankertz, K.R. Müller, D.J. Krusienski, G. Schalk, J.R. Wolpaw, A. Schlögl, G. Pfurtscheller, J. del R. Millán, M. Schröder, and N. Birbaumer, "The BCI competition. III: Validating alternative approaches to actual BCI problems," *IEEE Transaction Neural System Rehabilitation Engineering*, vol. 14, pp. 153-159, June 2006.

[9] C. D. Woody, "Characterization of an adaptive filter for the analysis of variable latency neuroelectric signals," *Medical and biological engineering*, vol. 5, pp. 539-553, 1967.

[10] P. Jaśkowski and R. Verleger, "Amplitudes and latencies of single-trial ERP's estimated by a maximum-likelihood method," *IEEE Transactions on Biomedical Engineering*, vol. 46, no. 9, 1999.

[11] D. T. Pham, J. Mocks, W. Kohler, and T. Gasser, "Variable latencies of noisy signals: Estimation and testing in brain potential data," *Biometrika*, vol. 74, pp. 525-533, 1987.

[12] R. Li, J. C. Principe, M. Bradley, and V. Ferrari, "A spatiotemporal filtering methodology for single-trial erp component estimation," *IEEE Transactions on Biomedical Engineering*, vol. 56, no. 1, pp. 83-92, 2009.