# EEG Markers of Acceleration Perception in Virtual Reality

Gaël van der Lee, Francois Cabestaing, Anatole Lécuyer, Reinhold Scherer, Hakim Si-Mohammed

► To cite this version:

Gaël van der Lee, Francois Cabestaing, Anatole Lécuyer, Reinhold Scherer, Hakim Si-Mohammed. EEG Markers of Acceleration Perception in Virtual Reality. BCI 2024 - 9th Graz Brain-Computer Interface Conference, Graz University of Technology, Sep 2024, Graz, Austria. pp.249-253, 10.3217/978-3-99161-014-4-044. hal-04796294

HAL Id: hal-04796294

https://hal.science/hal-04796294v1

Submitted on 21 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# EEG MARKERS OF ACCELERATION PERCEPTION IN VIRTUAL REALITY

Gaël Van der Lee¹, François Cabestaing¹, Anatole Lécuyer², Reinhold Scherer³,
Hakim Si-Mohammed¹

¹Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRISTAL, F-59000 Lille, France

²Inria Rennes, IRISA, France

³University of Essex, Colchester, United Kingdom

E-mail: gael.vanderlee@univ-lille.fr francois.cabestaing@univ-lille.fr
hakim.simohammed@univ-lille.fr

**ABSTRACT:** This study investigates neural patterns of acceleration in virtual reality (VR) using electroencephalography (EEG). Participants experienced accelerating white spheres in VR while EEG signals were recorded. Significant EEG differences were found at the fronto-central region between acceleration and slow speed, regardless of direction, and at the central region depending on the acceleration direction. Topographic responses also show differences in spacial patterns between the conditions. These findings give insights into the perception of acceleration in the brain and show potential for passive BCI applications.

## INTRODUCTION

Although there are numerous definitions of VR [1], it can be succinctly defined as “A real or simulated environment in which a perceiver experiences telepresence” [2]. The user’s psychological response to immersion in VR, termed “presence” [3], provides valuable insights into user behavior and experience. Understanding user responses to specific stimuli in VR is crucial for enhancing overall user experience and mitigating issues such as cybersickness [4].

Recently, there has been growing interest in leveraging BCIs to enhance VR experiences, aiming to create more immersive and interactive environments [5]. This paper investigates brain responses to acceleration in VR to identify potential markers of acceleration perception. Our experimental setup involved the simultaneous use of a 16-electrode EEG headset and a VR headset. Participants were presented with stimuli consisting of moving white spheres within the VR environment. These spheres initially moved at a slow constant speed before undergoing a sudden acceleration, either forward or backward, followed by a return to the initial speed. This experimental paradigm allowed us to examine the neural responses associated with the perception of acceleration in different directions.

The study of acceleration perception in VR is particularly relevant given the association between the subjective sensation of movement called vection and the user's experi-

ence in VR. Specifically, cybersickness has been associated with vection [6] and could be better understood through that prism.

Thus, studying the perception of acceleration serves as a foundational step in investigating vection and its associated neural correlates.

We identify two potential neuromarkers:

1. A frontal marker of visual acceleration characterized by a positive potential between 300ms and 700ms after the start of the acceleration.

2. A signal differentiating the direction of the acceleration, whether it was forward or backward in the central region.

To the best of our knowledge, these findings have not been previously reported in the literature. They entail two potential meanings for the fields of neuroscience and BCIs. Firstly, they highlight fundamental cortical responses associated with acceleration perception. Secondly, they pave the way for passive BCIs that utilize this neuromarker to tailor the user experience accordingly. By identifying markers of acceleration perception, we can develop algorithms capable of discerning user attention towards acceleration events. Aligning detected acceleration events with corresponding neuromarkers offers insight into user engagement with such stimuli.

In summary, this study contributes to the growing body of literature on BCIs and VR by hinting at the neural basis of acceleration perception and its potential applications in human-computer interaction and immersive technology.

## MATERIALS AND METHODS

**Objective:** This protocol aims to generate acceleration perception responses using Virtual Reality (VR). To achieve this goal, a user study was designed to trigger a potential through two types of trials: (1) sudden forward acceleration *FA*₁ and (2) sudden backward acceleration *BA*₁ (see Figure 1). Following this event, the participant slowed back down to their initial speed by either a forward (*FA*₂) or a backward (*BA*₂) acceleration.
<figure><img src="image_2.png" /></figure>

Figure 1: Illustration of a trial: Evolution of speed over time is depicted. Dashed durations represent variable delays, with one of the three delays chosen randomly. The second delay is selected to ensure that the cumulative sum of delays amounts to 6 seconds.

**Participants and Ethics:** Twenty healthy participants with normal or corrected-to-normal vision (12 men, 8 women; age $\mu$ = 28.1, $\sigma$ = 7.99, min = 20, max = 56) took part in the experiment. Ethical approval for this study was obtained from the Ethics Committee of the University of Lille with approval number 2021-526-S97. The study adhered to the principles outlined in the Declaration of Helsinki. Written informed consent was obtained from all participants, who were explicitly informed of their right to withdraw from the experiment at any time without repercussion. Special attention was given to inform the participants that they can withdraw should they experience cybersickness. Data was anonymized and stored in compliance with the General Data Protection Regulation (GDPR). Participants were given their anonymized ID and could withdraw their consent at any time after the experiment, removing their recordings from the dataset.

**Experimental Setup:** The VE was displayed using a Valve Index Head-Mounted Display (HMD) connected to a DELL PRECISION 3640 personal computer with an NVIDIA RTX 3080 video card. EEG data were recorded using OpenVibe 3.1.0 software and a g.GAMMAcap2 EEG cap from g.tec medical engineering GmbH® (Austria) with 14 electrodes positioned at FPz, Fz, F1, F2, FCz, FC1, FC2, Cz, C1, C2, CPz, CP1, CP2, Pz and a reference electrode placed on the right earlobe. The VE was created using the Unity game engine software (version 2020.3.11f1).

**Trial Design:** Each trial lasted 17 seconds and consisted of four phases:

1. The *Static* phase: The environment fades in over 2 seconds and remains still.

2. The *Slow speed* phase: The environment accelerates to a speed of 3m/s over 2 seconds and maintains this speed for a variable duration of 1 to 5 seconds. This phase is used to have an EEG baseline of visual stimulation without strong speed or acceleration.

<figure><img src="image_3.png" /></figure>

Figure 2: Depiction of the visual experience presented to participants, featuring a virtual scene composed of point clouds and a central crosshair.

3. The *Acceleration* phase: Participants experience a sudden forward acceleration $FA_1$ or backward acceleration $BA_1$ of 12m/s² for 1 second. The resulting speed is maintained for two seconds before returning to the initial slow speed with either a forward acceleration $FA_2$ or a backward acceleration $BA_2$.

4. The *End* phase: The environment maintains a speed of 3m/s for a duration matching the slow speed phase before fading out over 2 seconds.

**Environment:** The VE consisted of a minimalistic environment with stationary white spheres arranged cylindrically around the participant, following what has been done in the literature [7]. Participants were instructed to focus on a red crosshair at the center of the visual field to minimize ocular movements. Spheres gradually became visible from 150 meters away and were updated in real-time to reflect the current speed. The participant's view can be seen in Figure 2. Participants experienced 78 trials organized into four blocks. Each condition was evenly distributed amongst the trials, with 39 of each forward and backward trials, as well as 26 of each duration before acceleration (1, 3 or 5 seconds).

**Data Processing:** EEG data were processed using MNE-python for filtering and epoching. Noisy channels were identified and excluded, and data were re-referenced using common average referencing (CAR). Data were re-sampled to 128Hz and filtered from 0.3 to 10Hz using a 4th order Butterworth filter. Epochs ranged from 0.5s before stimulus onset to 1s after stimulus offset. Epochs containing voltage exceeding 125µV were rejected. Data were stored in EEG Brain Imaging Data Structure (BIDS) standard for easy sharing. Visualization was performed using the seaborn library.

## RESULTS

In our subjects, we split the EEG data between different conditions:

* **Baseline:** This is taken during the slow speed phase, as shown in Figure 1. Where the subject is going at a constant speed of 3m/s.

* $FA_1, BA_1, FA_2, BA_2$ As described in the *Trial Design* subsection.
<figure><img src="image_4.png" /><figcaption>Figure 3: Topographic map comparison of the average response for all subjects to baseline (top) and the first acceleration, *FA*₁ and *BA*₁, averaged (bottom)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Mean values of the FCz electrode for *FA*₁ (blue), *BA*₁ (orange), and baseline (green). The 95% confidence interval is shown for each event. The shaded gray area corresponds to the time period where there was a statistically significant difference between the signals. *FA*₁ and *BA*₁ present similar patterns distinguishable from the baseline.</figcaption></figure>

In this study, we evaluate the significance of observed differences using a non-parametric bootstrapping approach. First we perform 10,000 resamples on our data with replacement. Then we compute the 95% confidence intervals which correspond to the range between the 2.5th and 97.5th percentiles of the resampled data distribution. These confidence intervals are represented as shaded areas in the figures. Additionally, we utilize topographic maps to underscore spatial differences between conditions, with cubic interpolation applied to obtain values between electrodes.

**Marker of acceleration:** The spacial response shows
differences between the baseline and the acceleration
condition as shown in Figure 3 which presents a much
higher positive peak along the central regions and es-
pecially the fronto-central region peaking at 600ms. A
better temporal representation can be found in Figure 4,
which shows a significant difference at electrode FCz be-
tween periods of strong acceleration (*FA*₁ or *BA*₁) com-
pared to slow speed (*3m/s*), regardless of acceleration di-
rection. We see the characteristic strong positive potential
between 300 and 700ms after acceleration onset. Both
*FA*₁ and *BA*₁ follow a similar pattern in this region. This
particular pattern in the region could represent a marker

<figure><img src="image_6.png" /><figcaption>Figure 5: A comparison of the average topographic map across all subjects for FA₁ (top) and BA₁ (bottom)</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6: Mean values of the Cz electrode for *FA*₁ (blue) and *BA*₁ (orange). The 95% confidence interval is shown for each event. The shaded gray area corresponds to the time period where there was a statistically significant difference between the signals. The Cz presents a significant difference between *FA*₁ and *BA*₁.</figcaption></figure>

of acceleration, regardless of direction.

**Marker of direction:** The spacial response to the di-
rection of acceleration also shows differences. Compar-
ing the topographic maps in Figure 5, we find a stronger
negativity in the parietal region for the *FA*₁ condition
around 400ms, that becomes a frontal and positive over
the course of the next 200ms. The *BA*₁ condition shows a
much stronger negativity, especially around the central
electrodes, that increases until 600ms. Looking at the
Cz electrode, a significant difference was found between
forward and backward acceleration, as seen in Figure 6.
The *BA*₁ condition differentiates from the *FA*₁ condition
by showing a higher peak around 400ms and keeping a
stronger positive potential until 700ms after acceleration
onset.

**Return to slow speed:** Responses when returning to the slower speed were not as pronounced as the responses we found in the initial acceleration condition. We did find a distinct marker for the return to slow speed section during the Acceleration phase, as seen in Figure 7. First we observe a small difference in conditions around 350–500ms, which is followed by a more pronounced late marker. This late marker showed a significant difference between the *FA*₂ and *BA*₂ conditions between 750–1000ms after acceleration onset.
<figure><img src="image_8.png" /><figcaption>Figure 7: Mean values of the Pz electrode for *FA*₂ (brown) and *BA*₂ (purple). The 95% confidence interval is shown for each event. The shaded gray area corresponds to the time period where there was a statistically significant difference between the signals. The Pz presents a significant difference between *FA*₂ and *BA*₂.</figcaption></figure>

DISCUSSION

*Interpretation:* The identification of distinct neural ac-
tivation patterns in response to acceleration stimuli opens
the door for a deeper understanding of acceleration per-
ception circuits in the brain. Further understanding of
the neural mechanisms underlying acceleration percep-
tion may reveal insights into motor control processes.
Some of these responses, notably the one exhibited in
Figure 6 bears resemblance to P300 responses from the
literature in its timing and location [8]. As the P300 is
associated with surprise and decision-making, the per-
ception of a sudden acceleration could trigger a decision-
making process in the brain.

Importantly, these findings lay the foundation for future
research on passive BCIs that use acceleration percep-
tion as a neural signal. Acceleration perception could be
used in a similar manner to established passive BCI sig-
nals such as mental workload [9, 10] or changes in error-
related potentials [11]. Such systems could adapt a user's
environment and inputs, knowing if the user perceived an
acceleration and in which direction he perceived it. For
example, a passive BCI could detect if a driver in a vehi-
cle is paying attention to the road by using the markers of
acceleration perception along with an accelerometer.

*Limitations and future work:* In this paper, we find pat-
terns when going from a slow speed to a high speed (*FA*₁
and *BA*₁), but very different patterns when starting from
a high speed and going back to normal (*FA*₂ and *BA*₂).
All four patterns are unique, we do not explain this dif-
ference, and it warrants further study to be better under-
stood. Moreover, while this study shows a signal specific
to acceleration perception and direction, these findings
are limited to specific conditions: the subject is sitting, in
VR, with a stimulation consisting of white spheres.

Continued research in this direction could uncover how
the response evolves as we add sensory indicators of ac-
celeration such as sounds. One could also compare this
signal to those exhibited ecological experiences of accel-
eration in VR as well as in real-world scenarios. Beyond

the perception of motion, this opens the door to under-
standing how the brain perceives self-motion induced by
visual stimuli, or vection, which bears importance for the
field of VR and cybersickness. Thus, another direction
could be correlating this signal with the subjective per-
ception of the participant.

CONCLUSION

In this study, we find neural responses associated with
perceptual changes induced by sudden accelerations in
VR. We find different spatial responses characteristic of
both acceleration and acceleration direction. We also un-
cover differences in EEG signals at electrodes FCz and
Cz during acceleration perception, suggesting the exis-
tence of distinct neural markers for acceleration direction.
These findings have implications for the development of
passive BCIs and the enhancement of virtual reality ex-
periences. By leveraging these neural markers, future re-
search can design adaptive BCIs and create more immer-
sive and interactive VR environments. Overall, this study
contributes to advancing our understanding of the neu-
ral mechanisms underlying acceleration perception and
paves the way for innovative applications in brain com-
puter interfaces.

ACKNOWLEDGMENTS

This research was funded by the European Union ERA-
NET CHIST-ERA 2020 research and innovation pro-
gram under grant number CHIST-ERA-20-BCI-003 -
“GENESIS” and the French Research Agency (ANR) un-
der grant number ANR-21-CHRA-0001-01.

REFERENCES

[1] Xiong J, Hsiang EL, He Z, Zhan T, Wu ST. Augmented reality and virtual reality displays: Emerging technologies and future perspectives. Light: Science & Applications. 2021;10(1):216.

[2] Biocca F, Levy MR. Communication in the Age of Virtual Reality. Routledge (2013).

[3] Bowman DA, McMahan RP. Virtual Reality: How Much Immersion Is Enough? Computer. 2007;40(7):36–43.

[4] Nesbitt K, Davis S, Blackmore K, Nalivaiko E. Cor-
relating reaction time and nausea measures with tradi-
tional measures of cybersickness. Displays. 2017;48:1–
8.

[5] Tan X, Li Y, Gao Y. Combining brain-computer interface with virtual reality: Review and prospect. In: 2017 3rd IEEE International Conference on Computer and Communications (ICCC). Dec. 2017, 514–518.

[6] Keshavarz B, Riecke BE, Hettinger LJ, Campos JL. Vection and visually induced motion sickness: How are they related? Frontiers in Psychology. 2015;6.
[7] Keshavarz B, Philipp-Muller AE, Hemmerich W, Riecke BE, Campos JL. The effect of visual motion stimulus characteristics on vection and visually induced motion sickness. Displays. 2019;58:71-81.

[8] Polich J. Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology. 2007;118(10):2128-2148.

[9] Luong T, Martin N, Raison A, Argelaguet F, Diverrez JM, Lécuyer A. Towards Real-Time Recognition of Users' Mental Workload Using Integrated Physiological Sensors Into a VR HMD. In: ISMAR 2020 - IEEE International Symposium on Mixed and Augmented Reality. Virtual, Brazil, Nov. 2020, 1-13.

[10] Roy RN, Bonnet S, Charbonnier S, Campagne A. Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Jul. 2013, 6607-6610.

[11] Dehais F et al. Monitoring Pilot's Mental Workload Using ERPs and Spectral Power with a Six-Dry-Electrode EEG System in Real Flight Conditions. Sensors. 2019;19(6):1324.