# Detecting P300-ERPs Building a Post-validation Neural Ensemble with Informative Neurons from a Recurrent Neural Network

Christian Oliva, Vinicio Changoluisa, Francisco B. Rodríguez, Luis F.

Lago-Fernández

► To cite this version:

Christian Oliva, Vinicio Changoluisa, Francisco B. Rodríguez, Luis F. Lago-Fernández. Detecting P300-ERPs Building a Post-validation Neural Ensemble with Informative Neurons from a Recurrent Neural Network. 19th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2023, León, Spain. pp.90-101, 10.1007/978-3-031-34111-3_9. hal-05449789

HAL Id: hal-05449789

https://inria.hal.science/hal-05449789v1

Submitted on 9 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Detecting P300-ERPs Building a Post-validation Neural Ensemble with Informative Neurons from a Recurrent Neural Network

Christian Oliva<fnref n="1" />(✉), Vinicio Changoluisa<fnref n="2" />(✉), Francisco B. Rodríguez<fnref n="1" />(✉), and Luis F. Lago-Fernández<fnref n="1" />(✉)

**Abstract.** We introduce a novel approach for detecting the sample-level temporal structure of P300 event-related potentials. It consists of extracting the most informative neurons from a Recurrent Neural Network and building a post-validation neural ensemble (PVNE). The weights connecting the recurrent and the output layers are used to rank the recurrent neurons according to their relevance when generating the network's output. A set of neurons is selected according to their positions in this ranking, and their individual predictions are then combined to obtain the final model's output. This procedure discards neurons whose role could be more related to maintaining the network's hidden state than to detecting the P300 events, with an overall performance increase. The use of L1 regularization notably emphasizes this effect. We compare the performance of this approach with both Elman and LSTM RNNs and show that the PVNE is able to detect the sample-level temporal structure of P300 event-related potentials, outperforming the standard models. Sample-level prediction also allows for real-time monitoring of the EEG signal generation related to ERPs.

**Keywords:** LSTM · Elman RNN · Bayesian Linear Discriminant Analysis · Output Neurons Ensemble · Neurons relevance · Neurons interpretability · Brain-machine interface · Inter- and intra-subject variability

## 1 Introduction

Event-related potentials (ERPs) are commonly used for exploring the human mind and brain [19]. Electroencephalography (EEG) is one of the most widely

<footnote n="1">¹ Grupo de Neurocomputación Biológica, Departamento de Ingeniería Informática, Escuela Politécnica Superior, Universidad Autónoma de Madrid, Madrid, Spain
{christian.oliva,f.rodriguez,luis.lago}@uam.es</footnote>

<footnote n="2">² Grupo de Investigación en Electrónica y Telemática, Universidad Politécnica Salesiana, Quito, Ecuador
fchangoluisa@ups.edu.ec</footnote>
used ways to monitor brain activity related to ERPs, due to its good temporal
resolution and relatively low cost. ERPs appear in EEG as positive and negative
voltage deflections, maintaining a specific temporal structure according to the
brain activity under study. When a set of frequent and infrequent stimuli is
presented, a positive deflection is generated at 300 ms. This voltage deflection is
known as P300, and it is one of the most widely used ERPs. From the seminal
article presented by Sutton et al. [28] to date, many research works have been
searching to understand its structure and functionality. Currently, the P300 ERP
is known to be related to several cognitive processes, such as memory, learning,
or attention [19].

One of the most widespread uses of P300 is its application in Brain-Computer Interfaces (BCIs) as a control signal [2]. In this context, a P300 is evocated when the subject detects one infrequent stimulus (called target stimulus) within a set of other (non-target) stimuli, and can be used to control an external device without any usual peripheral pathways. Despite the wide acceptance of P300-based BCIs, there are still some problems to be solved. One is the inter- and intra-subject variability, present due to different factors, such as age or stress [2, 10]. Several methodologies have recently been proposed to deal with these variability problems [5-7]. Another problem is the low information transfer rate (ITR), due to the standard approaches that require processing entire one-second windows, increasing the target detection time [27].

There exist different techniques to detect P300s trial by trial. The most com-
mon is the use of machine learning models, such as discriminant analysis, support
vector machines, or artificial neural networks (see [26] for a complete review of
the different methods to detect visual P300-ERPs, and [18] for classification algo-
rithms in general EEG-based BCIs). Currently, there are few pieces of evidence
for the use of recurrent neural network (RNN) models in the P300-based BCI
prediction [9,29,30], and some works that exist on LSTM perform a combina-
tion with convolutional neural networks (CNNs) to retrieve spatial information
[1,3,12,16]. P300 ERP detection has much redundant information since the many
EEG recording electrodes process different but related events [14,18]. It is also
known that, due to the inter-variability between users, the P300 event does not
always appear in the same waveform (latency and amplitude) [5,7,17,24]. Our
work presents a continuous training mode from the input signals with P300-
ERPs using recurrent neural networks (RNNs), based on previous research [21].
We extend this idea by selecting the most informative neurons according to their
weights and combining their individual predictions in order to obtain the final
output. The introduction of L1 regularization in the output layer appears to
be decisive in this process. It allows for the creation of a post-validation neural
ensemble (PVNE) which uses the output layer's weights as a metric for defining
the neuron's voting, increasing the models' performance in ERP-based BCIs.

The remaining of the article is organized as follows. First, in Sect. 2, we
describe the dataset used in our experiments, the preprocessing we apply to the
data, and the models that we use. In Sect. 3 we present and analyze our results.
Finally, in Sect. 4 we present the conclusions and further lines of research.
# 2 Materials and Methods

## 2.1 Data Description and Preprocessing

We use the dataset created by Hoffman et al. [14], whose paradigm is based on a six-choice P300. The dataset contains eight users, which have reacted to random sequences of six images (classes) with an Inter-Stimulus Interval (ISI) of 400 ms. Each image is flashed for 100 ms and, during the remaining 300 ms, nothing is displayed. The EEG signals were recorded at a 2048 Hz sampling rate from the standard 32 electrodes of the 10-20 international system [15]. The dataset is filtered with a sixth-order forward-backward Butterworth bandpass filter, cut-off frequencies are set to 1.0 Hz and 12.0 Hz, and its signals are downsampled from 2048 Hz to 32 Hz.

**Standard Preprocessing.** The standard preprocessing consists of extracting single windows of duration 1000 ms starting at the stimulus onset. In these conditions, the classification problem consists of using the 1000 ms windows to predict the corresponding class: *Target stimulus*, when the target image appears, versus *Non-Target stimulus*, when it does not [14]. This approach is followed by most research works, and we apply it to prepare the data for the Bayesian Linear Discriminant Analysis (BLDA) model introduced in Sect. 2.2, which we use as benchmark. In the context of a P300-based BCI, however, this procedure can result in a low ITR, since full one second signals must be processed prior to making a decision.

**Sample-Level Preprocessing.** Sample-level preprocessing, first implemented in [21], offers a methodology for training a Recurrent Neural Network to provide a continuous output, instead of classifying 1000 ms windows. Given that the exact window of the P300 event is not a priori known, two hyperparameters are used to appropriately define the network's expected output ($Y_{True}$) for each time step. The two hyperparameters are an *offset*, which represents the starting time of the P300 event, and a *window size*, which represents its duration. The *offset* and the *window size* define an *interest interval* where the P300 event is assumed to occur. The networks are thus trained to output 1 in this interval and 0 elsewhere. Figure 1 shows an example 1000 ms window, with the average signal recorded by the *Fp1* electrode when target stimuli are presented to user 7 (top), and the expected output, $Y_{True}$, considering an *offset* of 437.5 ms and a *window size* of 250 ms.

## 2.2 Models

**Bayesian Linear Discriminant Analysis (BLDA).** BLDA is the method originally used by Hoffman et al. [14] for the P300-ERP recognition problem. It is an extension of Fisher's Linear Discriminant Analysis (FLDA) [4] that runs the regression in a Bayesian framework. It automatically estimates the
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Example of a 1000 ms window when a target stimulus is presented at time *t* = 0 ms. The top figure shows the average *Fp1* electrode signal from user 7, sampled 32 Hz. The bottom figure is an example of the expected output, *Y*<sub>True</sub>, using an *offset* of 437.5 ms and a *window size* of 250 ms.</figcaption></figure>

degree of regularization, which prevents overfitting in noisy and high-dimensional environments. In our experiments we combine BLDA with the standard data preprocessing. We use the scikit-learn implementation [25].

**Elman Recurrent Neural Network (Elman RNN).** The Elman RNN [11] is one of the simplest neural models that introduces recurrent connections. Here we use the standard Keras SimpleRNN implementation [8], adding one output fully connected layer with a single sigmoid neuron. This output represents the probability of *Y*<sub>True</sub> being 1, according to the sample-level preprocessing described in Sect. 2.1.

**Long-Short Term Memory Networks (LSTM).** LSTM networks [13] are one of the most commonly used RNN models, and they are a usual benchmark in many sequence modeling applications. The LSTM architecture introduces memory cells and gate units to mitigate the effects of the vanishing and the exploding gradients problems. We use the Keras implementation [8] and, as before, the final model includes a fully connected output layer with sigmoid activation, which is used to predict the value of *Y*<sub>True</sub> using the sample-level preprocessing.
## 2.3 From Sample-Level to Window Classification

The recurrent neural networks, trained with sample-level preprocessed data, provide a continuous output signal with a frequency 32 Hz. As we already explained, this sample-level output represents, for each time step, the probability of being in one of the P300 interest intervals hyperparameterized by the offset and the window size. In order to transform this sample-level signal into a classification label (target/ERP-P300 vs non-target/non-ERP-P300) applicable to complete 1000 ms windows, we follow the approach in [21] (see Fig. 2). For each 1000 ms window, we obtain the maximum network output in the interest interval and use it as the predicted P300 probability. We apply a threshold of 0.5 to obtain the final predictions. This transformation allows for the comparison with other methods, such as BLDA, fitted using the standard preprocessing.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Diagram of the sample-level rebuilt to window classification. A 1000 ms window with 32 electrodes is the input to the recurrent model, either an Elman or an LSTM network in our experiments. It returns one output for each time step, corresponding to the 32 samples of the 1000 ms input signal. Then, we get the interest interval with the offset (375 ms) and the window size (125 ms), colored in blue, and finally, we get the maximum value in this bounded window as the prediction probability.</figcaption></figure>

## 2.4 Post-validation Neural Ensemble

As an alternative to the previous approach, we propose a novel strategy to transform the continuous network output so that it can be used to classify the 1000 ms windows. It consists of building an ensemble with the top-K most informative neurons from the recurrent layer, which act as single voters. The network output, for either the Elman or the LSTM models, is given by:

$$y_t = \sigma(\mathbf{w}^T \mathbf{h}_t + b), \quad (1)$$

where $\mathbf{h}_t$ is the activation vector of the recurrent layer at time t. The weight vector $\mathbf{w}$ determines how the neurons in the recurrent layer contribute to the network's output. These weights adjust the output of individual neurons from $\mathbf{h}_t$ in Eq. 1, so we can interpret the output layer as a weighted voting of individual neurons (plus a bias $b$). Hence, we can use the absolute value of each weight, $|w_i|$, as a measure of the relative importance of its corresponding neuron $i$. Then, the recurrent neurons are ranked according to this relevance, and the activations of
the K most relevant neurons are combined to build the final decision. We have named this procedure Post-Validation Neural Ensemble (PVNE). Note that this process discards the neurons that are more weakly connected to the output layer. The activity of these discarded neurons could be more relevant to the computation of the next hidden state than to the construction of the network's output, and hence the construction of the PVNE could have a positive effect on the final predictions. Our results seem to support this hypothesis.

The following section describes how the activation of each of the K-selected neurons is used to determine the neuron's vote, and how the individual votes are combined to compute the final ensemble's decision.

**Voting of Individual Neurons.** Figure 3 shows the activation of a highly ranked neuron from the recurrent layer of an Elman RNN trained with user 7, for all the target (red lines) and non-target (blue lines) stimuli. Thick lines represent averages over stimuli. It is interesting to note the big difference between target and non-target averages in the *interest interval* defined by the *offset* and the *window size* hyperparameters. We have observed a correlation between the position of a neuron in the ranking and the difference between its average response to the target and the non-target stimuli in the selected interval.

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Activation of a neuron from the hidden layer of an Elman RNN trained with user 7. The blue curve represents the neuron's activation along the time of the corresponding neuron when tested on any of the non-target stimuli, and the red one does the same when tested on any of the target stimuli. Vertical blue lines delimit the *interest interval*. The green horizontal line is a threshold $\theta_i$ at the middle point between the target, $\bar{h}_i^T$, and the non-target, $\bar{h}_i^{NT}$, averages for this example neuron. (Color figure online)</figcaption></figure>

Motivated by this observation, we decided to define an activation threshold, $\theta_i$, as the middle point between the target, $\bar{h}_i^T$, and the non-target, $\bar{h}_i^{NT}$, averages for neuron i:

$$ \theta_i = \frac{1}{2}(\bar{h}_i^T + \bar{h}_i^{NT}) \qquad (2) $$
The average values $\bar{h}_i^T$ and $\bar{h}_i^{NT}$ are evaluated at the middle of the interest interval. The neuron is considered to vote in favor of the presence of a P300 event if its activation $h_i$ in the middle point of the interest interval lies on the target side of this threshold. That is, neuron's $i$ vote is given by:

$$v_i = \begin{cases} 1, & \text{if } (h_i - \bar{h}_i^{NT})/(\theta_i - \bar{h}_i^{NT}) > 1, \\ 0, & \text{otherwise,} \end{cases} \qquad (3)$$

where $v_i = 1$ means preference for a P300 event. Finally, the PVNE prediction is the hard voting of the top $K$ neurons chosen from the defined ranking score:

$$p = \frac{1}{K} \sum_{i=1}^{K} v_i. \qquad (4)$$

The model predicts as target (P300 event) if $p$ is greater than 0.5.

# 3 Analysis and Results

In this work, we present a novel approach by modifying the output layer's activation in the evaluation phase to test the models using Elman and LSTM recurrent networks. In Sect. 3.1, we describe the experiments. In Sect. 3.2, we analyze the output layer weights and discuss the importance of applying L1 regularization. Then, in Sect. 3.3, we show the results and give some final observations.

## 3.1 Experiments

The complete dataset has 8 users and four different sessions for each one. For each user, we test the models by averaging a simple validation (3 training sessions, 1 test session) of the four available sessions per user. For validation, we perform a 3-fold cross-validation using the three available training sessions (2 sessions for training, 1 for validation) to look for the best hyperparameters (offset and window size) and thus find the interest interval. Once the optimal hyperparameters have been found, a new model is trained using the whole training data and evaluated using the test session. In the particular case of PVNE, the additional hyperparameter $K$ (number of voters in the ensemble) is chosen in two different manners: i) fixed for all users (for instance, PVNE-K5 refers to using the top 5 neurons), or ii) the best $K$ for each user (PVNE-KBest refers this second option).

As a metric for evaluation, we use the balanced accuracy, $BAC = (recall + specificity)/2$, because this problem has an intrinsic imbalance: there are five negative stimuli against only one positive stimulus. The standard classification consists of getting the model's prediction probability for a 1000 ms window to be or not a P300 wave. As our models provide a continuous output (one output for each time step) we use the procedures described in Sects. 2.3 and 2.4 to transform this sample-level output into an event classification.
## 3.2 Output Layer Analysis

Choosing the appropriate *offset* and *window size* is crucial when training recurrent neural networks to predict P300 events in a sample-level mode [21] because we know that, due to the inter-variability between users, the P300 event does not always appear in the same temporal window. For the experiments, we introduce the L1 regularization penalty with parameter $\gamma = 0.05$ into the model. We use L1 because of its aggressive impact on the neurons’ weights, since we want the model to discard those neurons which are not relevant in the output layer.

In this case, the analysis of the output layer takes a significant role since it is crucial to understand the post-validation ensemble model. To analyze the output weights, we show in Fig. 4 two example bar plots for each neuron in the hidden layer with the absolute value of its output weights. The top figure represents the non-regularized settings, and the bottom figure represents the regularized settings. Both plots were obtained with the best *interest interval* (*offset* = 375 ms and *window size* = 125 ms) when the network is trained with User 7 and tested with Session 4. Similar results have been observed with all other users and sessions.

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** For each neuron in the hidden layer, the bar plots represent the absolute value of its output weights. The top figure represents the non-regularized settings, and the bottom figure represents the regularized settings with the best *interest interval* (*offset* = 375 ms and *window size* = 125 ms) when the network is trained with User 7 and tested with Session 4. Similar results have been observed with all other users and sessions.</figcaption></figure>

There exists a substantial difference between the two figures. When we train the network without L1 regularization (top figure), almost all neurons influence the model’s prediction in its output. However, when applying regularization to the output weights (bottom figure), there are a few neurons that determine the model’s prediction. Hence, some neurons are determinant to give a proper prediction, while the rest of the neurons are used to process the previous time steps and keep the relevant information from the temporal sequence. This result lines up with previous research ideas [22,23]. Therefore, if some neurons differ from themselves when identifying P300 or not P300, we confirm that these neurons
can somehow classify the data individually, so their voting in the PVNE will be more meaningful.

## 3.3 Results

In the remaining of this section, we show the test BAC averaged over the eight available users. Please note that the *interest interval* (*offset* and *window size*) is different for each user, as well as the value of *K* if we refer to PVNE-KBest (see Sect. 3.1). We show in Table 1 the test BAC for different settings and models: the first row is trained with the standard preprocessing since we are replicating the state of the art Hoffman's BLDA model [14], and the rest of them are trained following the sample-level preprocessing.

<table><caption>**Table 1.** Test Balanced Accuracy obtained with all proposed models when training with and without L1 regularization.</caption><thead><tr><th>#</th><th>Models</th><th>L1reg = 0 L1reg = 0.05</th></tr></thead><tbody><tr><td>1</td><td>BLDA [14]</td><td>0.71 ± 0.05</td></tr><tr><td>2</td><td>Elman</td><td>0.66 ± 0.06 0.71 ± 0.06</td></tr><tr><td>3</td><td>Elman-PVNE-K5</td><td>0.62 ± 0.05 0.73 ± 0.06</td></tr><tr><td>4</td><td>Elman-PVNE-K10</td><td>0.64 ± 0.05 0.73 ± 0.06</td></tr><tr><td>5</td><td>Elman-PVNE-K15</td><td>0.64 ± 0.06 0.73 ± 0.06</td></tr><tr><td>6</td><td>Elman-PVNE-K20</td><td>0.65 ± 0.06 0.73 ± 0.06</td></tr><tr><td>7</td><td>Elman-PVNE-KBest</td><td>0.66 ± 0.06 <b>0.75 ± 0.06</b></td></tr><tr><td>8</td><td>LSTM [21]</td><td>0.69 ± 0.05 0.69 ± 0.06</td></tr><tr><td>9</td><td>LSTM-PVNE-K5</td><td>0.67 ± 0.05 0.70 ± 0.06</td></tr><tr><td>10</td><td>LSTM-PVNE-K10</td><td>0.68 ± 0.05 0.71 ± 0.06</td></tr><tr><td>11</td><td>LSTM-PVNE-K15</td><td>0.68 ± 0.06 0.71 ± 0.07</td></tr><tr><td>12</td><td>LSTM-PVNE-K20</td><td>0.68 ± 0.06 0.71 ± 0.07</td></tr><tr><td>13</td><td>LSTM-PVNE-KBest</td><td>0.69 ± 0.05 0.72 ± 0.07</td></tr></tbody></table>

From the table, we can extract two main ideas. First, to apply or not L1 regularization notably affects the final performance of the models (compare columns L1reg = 0 and L1reg = 0.05). In the case of the LSTMs, the variation in the results is more modest. However, the performance of Elman networks is strongly affected. Second, the regularization scheme seems to be more relevant than the network architecture. In fact, the Elman network slightly outperforms the LSTM in the best configuration, so we could claim that complex models, such as LSTMs, are not needed to solve this problem. Finally, the Elman-PVNE approach improves the state of the art in more than 4% of test BAC when the best *K* is chosen for each user (compare rows 1 and 7). Intuitively, we should only perform this experiment if the output layer has been regularized, but it can
also be applied if there is no regularization penalty. As expected, the best performance is obtained when L1 regularization turns off the unnecessary neurons. When it is not the case, the use of PVNE has no benefits in the model performance (note that rows 2 and 7 have the same BAC when no L1 regularization is applied).

# 4 Conclusions and Discussion

In this article, we have introduced a novel approach that consists of a post-validation neural ensemble (PVNE) methodology for detecting the sample-level temporal structure of P300 event-related potentials. This neural ensemble is made by selecting neurons according to the importance of the weights that connect the recurrent layer with the network's output. Subsequently, the individual neural predictions are combined to obtain the final PVNE decision.

We have compared the performance of Elman RNNs against the standard LSTM, and we have shown that, for this particular problem, models as complex as LSTMs are not indispensable since better results are obtained with an Elman network (compare rows 2 and 8 from Table 1 when L1 has been applied). In addition, PVNE with its best settings outperforms the state of the art (BLDA [14]) by 4% (compare rows 1 and 7 from Table 1). Furthermore, applying PVNE on a recurrent network, either Elman or LSTM, improves the standard network (compare rows 2 and 7, and 8 and 13 from Table 1). This last result lines up with other research ideas [22,23] in which a few neurons determine the model's prediction, and the rest are used to process and keep the relevant information from the temporal sequence. L1 regularization reduces the number of active neurons in the output layer, differentiating the decision-maker neurons from the rest.

This sample-level methodology opens the way to a study of the EEG signal in ERP-based BCIs that will improve the information transfer rate, which is a pending challenge [27]. Sample-level prediction allows real-time monitoring of the EEG signal generation related to ERPs since this methodology can help locate the temporal information from a target stimulus. This way, some approaches, such as those presented in this work, help to manage inter- and intrasubject variability, a latent problem in BCIs. Likewise, the proposal of this work helps to gain knowledge into the interpretability of the EEG signal. In a clinical problem (as in any machine learning problem) it is essential to verify that the high-accuracy measurement is the result of an adequate representation of the problem and not of possible artifacts in the data [20].

Our work contributes to techniques that seek to understand how the model has learned. Our analysis of the output layer departs from the importance of applying regularization, described in Sect. 3.2, and could give us more insight into the internal interpretability of the models. Further research could focus on the analysis of the internal behavior of the recurrent layer to ensure confidence in the signal features and get the most relevant input electrodes in the temporal decision-making. Another interesting research could be the addition of
some mechanisms for the *offset* and *window size* hyperparameters adjustment as trainable parameters of the network. This way, the potential model could learn specific inter- and intrasubject differences from each user.

**Acknowledgements.** This work has been partially funded by Spanish project PID2020-114867RB-I00, (MCIN/AEI and ERDF- "A way of making Europe"), Universidad Politécnica Salesiana 034-02-2022-03-31 and by Predoctoral Research Grants 2015-AR2Q9086 of the Government of Ecuador through SENESCYT.

## References

1. Abibullaev, B., Zollanvari, A.: A systematic deep learning model selection for P300-based brain-computer interfaces. IEEE Trans. Syst. Man Cybern. Syst. **52**(5), 2744-2756 (2021)

2. Allison, B.Z., Kübler, A., Jin, J.: 30+ years of P300 brain-computer interfaces. Psychophysiology **57**(7), e13569 (2020)

3. Bashivan, P., Rish, I., Yeasin, M., Codella, N.: Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks (2015)

4. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)

5. Changoluisa, V., Varona, P., Rodríguez, F.B.: An electrode selection approach in P300-based BCIs to address Inter-and Intra-subject variability. In: 2018 6th International Conference on Brain-Computer Interface (BCI), pp. 1–4. IEEE (2018)

6. Changoluisa, V., Varona, P., Rodriguez, F.B.: A fine dry-electrode selection to characterize event-related potentials in the context of BCI. In: Rojas, I., Joya, G., Català, A. (eds.) IWANN 2021. LNCS, vol. 12861, pp. 230–241. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85030-2_19

7. Changoluisa, V., Varona, P., Rodríguez, F.B.: A low-cost computational method for characterizing event-related potentials for BCI applications and beyond. IEEE Access **8**, 111089–111101 (2020)

8. Chollet, F., et al.: Keras (2015). https://keras.io

9. Craik, A., He, Y., Contreras-Vidal, J.L.: Deep learning for electroencephalogram (EEG) classification tasks: a review. J. Neural Eng. **16**(3), 031001 (2019)

10. van Dinteren, R., Arns, M., Jongsma, M.L., Kessels, R.P.: P300 development across the lifespan: a systematic review and meta-analysis. PloS One **9**(2), e87347 (2014)

11. Elman, J.L.: Finding structure in time. Cogn. Sci. **14**(2), 179–211 (1990)

12. Havaei, P., Zekri, M., Mahmoudzadeh, E., Rabbani, H.: An efficient deep learning framework for P300 evoked related potential detection in EEG signal. Comput. Methods Programs Biomed. **229**, 107324 (2023)

13. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. **9**(8), 1735–1780 (1997)

14. Hoffmann, U., Vesin, J.M., Ebrahimi, T., Diserens, K.: An efficient P300-based brain-computer interface for disabled subjects. J. Neurosci. Methods **167**, 115–25 (2008)

15. Hu, L., Zhang, Z. (eds.): EEG Signal Processing and Feature Extraction. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-9113-2

16. Joshi, R., Goel, P., Sur, M., Murthy, H.A.: Single trial P300 classification using convolutional LSTM and deep learning ensembles method. In: Tiwary, U.S. (ed.) IHCI 2018. LNCS, vol. 11278, pp. 3–15. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04021-5_1
17. Li, F., et al.: Inter-subject P300 variability relates to the efficiency of brain networks reconfigured from resting- to task-state: evidence from a simultaneous event-related EEG-fMRI study. NeuroImage **205**, 116285 (2020)

18. Lotte, F., et al.: A review of classification algorithms for EEG-based Brain-Computer Interfaces: a 10 year update. J. Neural Eng. **15**(3), 031005 (2018)

19. Luck, S.J.: An introduction to the event-related potential technique. MIT press, second edn. (2014)

20. Montavon, G., Samek, W., Müller, K.R.: Methods for interpreting and understanding deep neural networks. Digital Sig. Process. **73**, 1–15 (2018)

21. Oliva, C., Changoluisa, V., Rodríguez, F.B., Lago-Fernández, L.F.: Precise temporal P300 detection in brain computer interface EEG signals using a long-short term memory. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds.) Artificial Neural Networks and Machine Learning - ICANN 2021, pp. 457–468. Springer International Publishing, Cham (2021)

22. Oliva, C., Lago-Fernández, L.F.: On the interpretation of recurrent neural networks as finite state machines. In: Tetko, I.V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11727, pp. 312–323. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30487-4_25

23. Oliva, C., Lago-Fernández, L.F.: Separation of memory and processing in dual recurrent neural networks. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds.) ICANN 2021. LNCS, vol. 12894, pp. 360–371. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86380-7_29

24. Ouyang, G., Hildebrandt, A., Sommer, W., Zhou, C.: Exploiting the intra-subject latency variability from single-trial event-related potentials in the P3 time range: a review and comparative evaluation of methods Neurosci. Biobehav. Rev. **75**, 1–21 (2017)

25. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. **12**, 2825–2830 (2011)

26. Philip, J.T., George, S.T.: Visual P300 mind-speller brain-computer interfaces: a walk through the recent developments with special focus on classification algorithms. Clin. EEG Neurosci. **51**(1), 19–33 (2020)

27. Ramadan, R.A., Vasilakos, A.V.: Brain computer interface: control signals review. Neurocomputing **223**, 26–44 (2017)

28. Sutton, S., Braren, M., Zubin, J., John, E.R.: Evoked-potential correlates of stimulus uncertainty. Science **150**(3700), 1187–1188 (1965)

29. Tal, O., Friedman, D.: Recurrent Neural Networks for P300-based BCI. CoRR abs/1901.10798 (2019), http://arxiv.org/abs/1901.10798

30. Zhang, X., Yao, L., Wang, X., Monaghan, J., McAlpine, D., Zhang, Y.: A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers. J. Neural Eng. **18**(3), 031002 (2021)