# Tagging Latency Estimator: A Standalone Software for Estimating Latency of Event-Related Potentials in P300-based Brain-Computer Interfaces

Grégoire Cattan, Cesar Mendoza

► To cite this version:

Grégoire Cattan, Cesar Mendoza. Tagging Latency Estimator: A Standalone Software for Estimating Latency of Event-Related Potentials in P300-based Brain-Computer Interfaces. Journal of Open Source Software, 2023, 8 (81), pp.4114. 10.21105/joss.04114. hal-03932533

HAL Id: hal-03932533

https://hal.science/hal-03932533

Submitted on 6 Feb 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Tagging Latency Estimator: A Standalone Software for Estimating Latency of Event-Related Potentials in P300-based Brain-Computer Interfaces

Grégoire H. Cattan¹ and Cesar Mendoza²

1 IBM, Cloud and Cognitive Software, Poland 2 IHMTEK, France

DOI: 10.21105/joss.04114

## Software

- Review ⚙️

- Repository ⚙️

- Archive ⚙️

Editor: Olivia Guest ⚙️

## Reviewers:

- @raybecker

- @ekinney-lang

Submitted: 08 September 2021

Published: 10 January 2023

## License

Authors of papers retain copyright and release the work under a Creative Commons Attribution 4.0 International License (CC BY 4.0).

## Summary

Event-related potentials (ERPs) are small potentials elicited by the brain in response to an external stimulation. They are measured using an electroencephalogram (EEG). Differences in the onset time and amplitude of ERPs reflect different sensory and high-level brain processing functions, such as the recognition of symbols, the correctness of presented information, or changes in a subject's attention (Luck, 2012). For these reasons, ERPs are a useful tool for describing the processing of information inside the brain, with practical applications in the domain of brain-computer interfaces (Wolpaw & Wolpaw, 2012).

To detect and evaluate an ERP in an ongoing electroencephalogram (EEG), it is necessary to tag the EEG with the exact onset time of the stimulus. A precise hardware method is then used to assess the latency between the tag and the exact onset of the stimulus on screen (Andreev et al., 2019). This method relies on a photodiode, placed in front of a stimulus which records the exact moment where the stimulus actually lightens on the screen. The latency is then computed by subtracting the time when the EEG is tagged to the actual apparition of the stimulus on screen.

A fixed latency engenders a constant offset which can be easily removed. However, the failure to control the tagging pipeline causes problems when interpreting ERPs thus leading to contradictory conclusions (Amin et al., 2015; Käthner et al., 2015; Pegna et al., 2018) - such as confusing two ERPs. This is particularly true when comparing ERPs elicited by stimuli presented on different platforms as these platforms usually introduce latencies that differ due to specific hardware and software configurations (Cattan et al., 2021). Another common problem, is the display of stimuli that don't match the position of the photodiode. In fact, different stimuli have different latencies as different parts of the screen don't refresh at the same time.

Analysis of the tagging pipeline (Cattan et al., 2018) have led to the development of a theoretical framework to interpret and eventually correct the measured latency, based on high-level configuration, such as the position of the photodiode (if known), the distribution of the stimuli on screen, the screen orientation or the number of cameras within the screen - like in virtual reality where the screen is split in two.

## Statement of need

TaggingLatencyEstimator is a standalone software developed in Unity which provides a C# implementation for Cattan et al. (2018).

As briefly summarized in the Summary subsection, the complexity of the tagging pipeline is a problem under-estimated in the scientific literature, which could lead to the misinterpretation of the ongoing brain processing functions. It is indeed difficult to separate the latency due to
software, hardware, or physiological component. This software intends to remove such biases
in the analysis, by estimating the latency and variability solely due to hardware and software
components.

Some of the use cases include (non-exhaustively):

* *Same experiment, different platforms.* The same visual stimulation is displayed on a PC vs TV monitor for example. There is a significant delay between the onset of the ERP under the two conditions. The software provides a confidence interval inside which the delay can solely be explained by the software and hardware setups.

* *Different experiments, same platform.* Two different visual stimuli, such as a word and an image are displayed on the screen. There is a significant delay between the onset of the ERPs. The software provides a confidence interval inside which the delay can solely be explained by software and hardware setup.

* *Same experiment, same platform, different runs.* A subject participates in two runs of the same experiment, consisting of the flashing of stimuli on the screen. There is a significant delay between the onset of the ERP under the two runs. The software provides a confidence interval inside which the delay can solely be explained by software and hardware setup.

* *Same experiment, same platform, same session.* A subject participates in an experiment session, consisting of the flashing of stimulation on the screen. A positive potential picks about 350ms after the onset of a stimulus, and after subtracting the latency measured by the photodiode. This potential could correspond to either a P300a or a P300b. The software estimates that an additional 50ms can be removed from the analysis due to the distribution of the stimuli on the screen. This suggests that the ERP is rather a P300a.

To our knowledge, there is no software or tools which may facilitate the correct estimation and interpretation of such latency. In addition, although this software originated from experiments with visually-induced ERPs, the principles it implements could benefit research with other types of ERP - in particular when the stimuli may appear at different locations in the same experiment, such as with spatialized songs.

Based on the model described in Cattan et al. (2018), an early version of this software was used in Korczowski et al. (2019a); Korczowski et al. (2019b); Korczowski et al. (2019c); Korczowski et al. (2019d); Vaineau et al. (2019); Van Veen et al. (2019); Cattan (2019); Cattan et al. (2021) thereby outlining the need for such an implementation.

## References

Amin, H. U., Malik, A. S., Mumtaz, W., Badruddin, N., & Kamel, N. (2015). Evaluation of passive polarized stereoscopic 3D display for visual amp;amp; mental fatigues. *2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)*, 7590–7593. https://doi.org/10.1109/EMBC.2015.7320149

Andreev, A., Cattan, G., & Congedo, M. (2019). *Engineering study on the use of Head-Mounted display for Brain-Computer Interface (Technical {Report} No. 1)*. GIPSA-lab. https://hal.archives-ouvertes.fr/hal-02166844

Cattan, G. (2019). *De la réalisation d'une interface cerveau-ordinateur pour une réalité virtuelle accessible au grand public* [Thesis, Université Grenoble Alpes]. https://tel.archives-ouvertes.fr/tel-02357203

Cattan, G., Andreev, A., Maureille, B., & Congedo, M. (2018). *Analysis of tagging latency when comparing event-related potentials (Technical {Report} No. 1)*. Gipsa-Lab ; IHMTEK. https://hal.archives-ouvertes.fr/hal-01947551
Cattan, G., Andreev, A., Mendoza, C., & Congedo, M. (2021). A Comparison of Mobile VR Display Running on an Ordinary Smartphone With Standard PC Display for P300-BCI Stimulus Presentation. *IEEE Transactions on Games*, 13(1), 68–77. https://doi.org/10.1109/TG.2019.2957963

Käthner, I., Kübler, A., & Halder, S. (2015). Rapid P300 brain-computer interface communication with a head-mounted display. *Frontiers in Neuroscience*, 9, 207. https://doi.org/10.3389/fnins.2015.00207

Korczowski, L., Cederhout, M., Andreev, A., Cattan, G., Rodrigues, P. L. C., Gautheret, V., & Congedo, M. (2019a). *Brain Invaders calibration-less P300-based BCI with modulation of flash duration Dataset (bi2015a)* [Research {Report}]. GIPSA-lab. https://doi.org/10.5281/zenodo.3266930

Korczowski, L., Cederhout, M., Andreev, A., Cattan, G., Rodrigues, P. L. C., Gautheret, V., & Congedo, M. (2019b). *Brain Invaders Cooperative versus Competitive: Multi-User P300-based Brain-Computer Interface Dataset (bi2015b)* [Research {Report}]. GIPSA-lab. https://doi.org/10.5281/zenodo.3267307

Korczowski, L., Ostaschenko, E., Andreev, A., Cattan, G., Rodrigues, P. L. C., Gautheret, V., & Congedo, M. (2019c). *Brain Invaders calibration-less P300-based BCI using dry EEG electrodes Dataset (bi2014a)* [Research {Report}]. GIPSA-lab. https://doi.org/10.5281/zenodo.3266223

Korczowski, L., Ostaschenko, E., Andreev, A., Cattan, G., Rodrigues, P. L. C., Gautheret, V., & Congedo, M. (2019d). *Brain Invaders Solo versus Collaboration: Multi-User P300-based Brain-Computer Interface Dataset (bi2014b)* [Research {Report}]. GIPSA-lab. https://doi.org/10.5281/zenodo.3267301

Luck, S. J. (2012). Event-related potentials. In *APA handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and psychometrics* (pp. 523–546). American Psychological Association. https://doi.org/10.1037/13619-028

Pegna, A. J., Darque, A., Roberts, M. V., & Leek, E. C. (2018). Effects of stereoscopic disparity on early ERP components during classification of three-dimensional objects: *Quarterly Journal of Experimental Psychology*. https://doi.org/10.1080/17470218.2017.1333129

Vaineau, E., Barachant, A., Andreev, A., Rodrigues, P. L. C., Cattan, G., & Congedo, M. (2019). *Brain Invaders Adaptive versus Non-Adaptive P300 Brain-Computer Interface dataset* (Research {Report} No. 1). GIPSA-LAB. https://hal.archives-ouvertes.fr/hal-02103098

Van Veen, G. F. P., Barachant, A., Andreev, A., Cattan, G., Rodrigues, P. L., & Congedo, M. (2019). *Building Brain Invaders: EEG data of an experimental validation* (Research {Report} No. 1). GIPSA-lab. https://hal.archives-ouvertes.fr/hal-02126068

Wolpaw, J., & Wolpaw, E. W. (2012). *Brain-Computer Interfaces: Principles and Practice*. Oxford University Press, USA. ISBN: 978-0-19-538885-5