HemodynamicVR - Adapting the User’s Field Of View during
Virtual Reality Locomotion Tasks to Reduce Cybersickness using
Wearable Functional Near-Infrared Spectroscopy
Hiroo Yamamura

Keio University
Yokohama, Japan
usurai-ligno@kmd.keio.ac.jp

Holger Baldauf

Keio University
Germany
holger.baldauf@gmail.com

Kai Kunze

Keio University
Yokohama, Japan
kai@kmd.keio.ac.jp

Figure 1: HemodynamicVR concept: Using hemoglobin changes measured by fNIRS to determine cybersickness symptoms
and limiting the field of view during walking segments

ABSTRACT
We present HemodynamicVR, a virtual reality headset combinedwith functional near-infrared spectroscopy (fNIRS). We believe thatsensing brain activity will enable novel interactions in virtual reality.
In this paper, we assess a user’s cybersickness based on the changeof their total hemoglobin concentration measured via an fNIRS
device, and we try to mitigate that by changing the field of view(FOV) in real-time. In this experiment, participants experienced
VR locomotion with an added variable FOV controlled by velocitychanges and by fNIRS. The results suggest that fNIRS can detectcybersickness as registered by the qualitative SSQ test.

CCS CONCEPTS• Computer systems organization → Embedded systems; Redundancy; Robotics; • Networks → Network reliability.

KEYWORDS
fNIRS, VR locomotion, Cybersickness, FOV
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
AHs ’21, February 22–24, 2021, Rovaniemi, Finland© 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-8428-5/21/02. . . $15.00
https://doi.org/10.1145/3458709.3458994

ACM Reference Format:
Hiroo Yamamura, Holger Baldauf, and Kai Kunze. 2021. HemodynamicVR- Adapting the User’s Field Of View during Virtual Reality Locomotion
Tasks to Reduce Cybersickness using Wearable Functional Near-Infrared
Spectroscopy. In Augmented Humans International Conference 2021 (AHs’21), February 22–24, 2021, Rovaniemi, Finland. ACM, New York, NY, USA,
5 pages. https://doi.org/10.1145/3458709.3458994

1

INTRODUCTION

As the first virtual reality devices (e.g., Oculus Quest 2) are entering
the mainstream consumer market1 , there are still a lot of unsolved
problems in terms of interaction paradigms, especially related topersonal adaptability. A simple example illustrating this is thatinterpupillary distances cannot be adjusted correctly for some users(e.g., 3 predefined IPD settings on the Oculus Quest 2), leading toan increase in cybersickness [6, 17]. Our research focuses on using
physiological signals to dynamically personalize the user’s virtualenvironment according to the user’s current state. In this paper,
we explore the use of a wearable fNIRS device to adjust users’ VRexperiences.
Problems with virtual reality systems often arise due to a mismatch between a user’s perception of motion and the system’s
corresponding rendering of motion. Sensory conflict theory provides a framework to understand these problems [11]. We explore aparticular common problem in VR locomotion using a gaming controller without physically moving. The gaming controller method
is widely used as a movement method in a VR space which is much
1 https://www.theverge.com/2020/10/30/21541535/oculus-quest-2-preorders-sales-developers-zuckerberg

223

AHs ’21, February 22–24, 2021, Rovaniemi, Finland

Anon al.

bigger than the available physical space. Although it is easy to implement and widely used in VR games, it increases cognitive load,
sensory conflict and can cause cybersickness[3]. We propose todetect the higher cognitive load related to sensory conflict duringlocomotion sequences in order to adjust the field of view (FOV)of the user automatically. The contributions of this paper can besummarized as follows:• We present a prototype implementation that dynamicallycontrols the user’s FOV in virtual reality using fNIRS.
• We show in a study with 11 participants that we can detect
total hemoglobin change (Hbt) with a high enough level ofsensitivity to identify cybersickness during VR locomotion
events.
• We compare the dynamic FOV adjustment using fNIRS with astandard method using movement velocity in an experimentwith 10 participants.

2

symptoms of VR sickness [4]. It was suggested that gastric tachycardia, blinking, heart rate and EEG delta waves were positively
correlated with SSQ scores [10]. When cybersickness symptoms
were mild, respiratory variability and ventilation were reduced, andwhen severe sickness occurred, markedly larger respiration andincreased respiratory variability were observed[15].
There is a great deal of individual variation in the degree ofcybersickness[16]. We follow a study from Bruck et al. postulatingthat subjects experiencing cybersickness have increased cerebral
blood flow (total hemoglobin concentration increase) during movement sequences (caused by symptoms like an increased breathingrate) [4]. This principle is the basis of our system.
Other researchers are exploring fNIRS setups in virtual reality.
Yet, to our knowledge, we are the first to present a fully interactive
system with an evaluation in 2 user studies.

RELATED WORK

Our work can be seen in the context of implicit interactions, using
physiological signals to recognize the user’s state and seamlessly
provide feedback. There is a lot of work in implicit interactionsin VR/AR environments using physiological signals [1, 5, 12, 14].
Common sensing technologies use various targets: tracking facial
expressions to body movement, eye tracking and brain activity
sensing (mostly EEG). The applications also vary widely, for example, redirected walking and re-targeting of hand movements using
visual/haptic illusions[12, 13]. As the field is so broad, we will focus
on this paper’s use case: locomotion related research and works
related to cybersickness.
The symptoms evoked by cybersickness are similar to simulator
sickness. Furthermore, teleportation is a well-known techniquefor reducing cybersickness[2]. Since the screen changes instantly,
teleportation reduces the sensation of moving and can suppress
cybersickness. A study comparing severity of sickness for steering(the user continuously perceives the scene along the path to thedestination) and jumping reported significantly lower scores forjumping. In the same experiment, steering tended to be preferred
for the task of allowing users to freely explore the VR space[18].
Besides, changing the screen instantly reduces immersion, so it is
not suitable for VR locomotion. Controlling the FOV can also reducecybersickness[7]. Both methods can reduce cybersickness, but at
the expense of immersion and presence. In other words, there is atrade-off between avoiding cybersickness and reducing immersion.
Some studies have minimized decreasing presence by dynamicallycontrolling the FOV. They obtain the user’s movement velocity
and angular velocity and control the FOV accordingly to ensure asense of presence while reducing cybersickness[7]. There are somequantitative metrics for evaluating cybersickness. The simulator
sickness questionnaire (SSQ) is a widely used tool to assess them[9].
Some studies have attempted to assess cybersickness using physiological signals. Currently, no physiological indicators that canaccurately assess cybersickness have been established, but someseem to be possible. An increase in arousal leads to changes in
respiratory rate and causes a decrease in the concentration of carbon dioxide in the cerebral bloodstream. These changes may cause

224

Figure 2: Experimental setup: Participant wearing the VIVE
VR headset and an fNIRS (model HOT-1000 from Hitachiconnected to the computer over Bluetooth).

3

APPROACH

As mentioned in the related work, serveral researchers are focusing
on intuitive interactions in virtual/ augmented reality. The mostobvious related to cognition and higher level reasoning are EEG
and eye tracking.
We decided to use fNIRS for a dynamic detection of cybersickness
during walking events. The signal is also affected by movement, yet
the resulting noise is less than for EEG. One problem with fNIRS
measurements is the delay (from 30 sec. to 3 min). As cybersickness
during locomotion usually builds up slowly, it is applicable for ouruse cases. Additional advantages of using fNIRS is that we do not
require user-dependent calibration (eye tracking) or careful sensor
placement (EEG). Our system is looking for relative changes intotal hemoglobin (Hbt) concentration measured in the prefrontal
cortex. It detects symptoms of cybersickness in subjects by measuring changes in cerebral blood flow (hemoglobin concentration
changes) caused by hyperventilation using fNIRS based on theexperiments of Bruck et al.[4]. In the following, we present our
prototype implementation, as well as an initial test detecting cybersickness using our prototype, and a study to evaluate a shrinking
FOV if cybersickness is recognized.

HemodynamicVR

4

AHs ’21, February 22–24, 2021, Rovaniemi, Finland

HEMODYNAMICVR PROTOTYPE

We use an HTC VIVE VR headset with an fNIRS device (model
HOT-1000 provided by Hitachi, Figure 2) connected over Bluetooth
to a computer running Windows 10. The application was developedwith Unity version 2019.4.12f1. The VR space was created from free
Unity assets (Nature Starter Kit2 and Farm Animals Set).

5

PROTOTYPE EVALUATIONIn the following, we present several use cases of the HemodynamicVR prototype to show its functionality and provide evidence thatfNIRS can be used for implicit VR interactions. All experiments
were conducted under the guidelines and approval of the ethics
committee of X (blinded for review).

5.1

Use case: Detecting Cybersickness

In a use case evaluation, we assess if we can detect cybersickness
episodes in users using the HemodynamicVR prototype. Our hypothesis, based on Buck et al.[4]: if the user is moved significantly inthe VR environment, there is an increased cognitive load indicatedby an increase in total hemoglobin change.
To induce cybersickness in real-time, the study participants experience VR in an uncontrolled state. The user is passively moved
in random directions on a VR forest landscape. We ask the user topress a button if they feel sickness. We use the HemodynamicVR
prototype to record the Hbt values during the experience. Users canstop at anytime; the experiment lasted for a maximum of 5 minutes.
We performed the SSQ test before and after the experience. We had
11 participants (5 female, average age 24.2, std=1.33).
From the results, it seems only for 7 participants the Hbt values
increased when experiencing cybersickness. 4 participants pressed
the button sometimes (indicating they felt sick), yet their Hbt values
did not increase significantly. These 4 participants’ score difference(before experiment - after experiment) was very low on the SSQ
score. Comparing them with the 7 participants in terms of SSQ, we
can detect a significant difference in the 2 groups (Mann–Whitney’s
U test, p=0.05). For the 4 participants, the SSQ scores are significantly lower than for the 7 others, meaning although they had
indicated they had felt sick, it seems like it had been relatively mildand does not register as cybersickness according to the SSQ score.
The experimental results suggest that fNIRS can detect cybersickness when its degree is above the level registered by the qualitative
SSQ test. As we hypothesized, participants with a strong increasein cybersickness (SSQ score) had an average increase of 0.56 in Hbt
value compared to the Hbt value when they did not feel sick. We set
this mean value as a threshold for the occurrence of cybersickness.

5.2

decide the minimum FOV, the subjects’ FOV was narrowed slowly
while they were moving. They were not informed in advance that
the FOV would change. The subjects were instructed to verballytell the experimenter if they felt a loss of immersion. They werealso interviewed after the experiment. We also varied the rate at
which the FOV narrowed while they were moving to determine
the appropriate rate of FOV change. We then instructed them toverbally tell us when they felt uncomfortable regarding their vision.
As a result of the short FOV test, most subjects did not reportany discomfort/loss of immersion when being limited to 10 percentof their FOV plus blur (see Figure3). In minimum FOV, 10 percentof the subject’s FOV is completely blocked and the blur comes inat the boundary where transparency is 0 (Figure 3).

Use case: Field of View (FOV) Adjustment

We conducted a short FOV test to experimentally determine the
FOV that could retain immersion during VR locomotion without
inducing sickness. Specifically, we determined the appropriate parameters for the minimum FOV to limit cybersickness during locomotion events and to retain as much immersion as possible.
11 participants (6 female, average age 24.9 years, std = 2.21)
conducted the setup. In the FOV use test, subjects wore the VIVE
headset and moved in virtual space. The velocity was set to one
meter per second and they were free to move with the joystick. To

225

Figure 3: Field of View adjustment (rendering for the left
eye).

6

DYNAMIC FOV ADJUSTMENT

To show the usefulness of HemodynamicVR, we implemented a
dynamic FOV adjustment based on the cybersickness and FOV
use case evaluations. An increase in Hbt value during locomotion
events (moving with a gaming controller) leads to a decrease in
FOV. We compare it to a standard implementation of a restricted
FOV using velocity: the faster the user moves, the smaller the FOV
gets. We use the empirically determined parameters described in
the previous section for the FOV.

6.1

Experimental Setup

We use the HemodynamicVR prototype for the experimental setup.
We created an animal search game in a forest with the Unity gameengine (randomly spawning animals, see Figure 4). Participants
have to move using a gaming controller and search and count the
animals in a given area. This task was chosen as it requires exploring
the area as well as moving the head around. The position of theanimals was randomised. Participants wore the HemodynamicVR.
The participants’ movement speed is 1 m/sec at maximum(around average walking speed). The experiment is divided into
two parts, each lasting 5 minutes. The parts are counter-balanced.
In the velocity session, we reduce cybersickness by changing the
FOV based on the subject’s velocity (Figure1). In the fNIRS session,
we change the FOV based on the subject’s Hbt value during theexperience. We obtain the average of Hbt values during the firstminute before starting the experience and use this as the baseline.

AHs ’21, February 22–24, 2021, Rovaniemi, Finland

Anon al.

Figure 6: SSQ score comparing the FOV adjustment using velocity and fNIRS

Figure 4: Subjects look for animals in this VR environment
The FOV shrinks when Hbt is greater than or equal to the set value
based on the cybersickness evaluation. We record the Hbt values
during the sessions. After and before the session, the user fills out
the SSQ for cybersickness [9]. After each session, the user also
answers the immersion questionnaire (IQ) [8].
This experiment was conducted with 10 participants (6 female,average age 24.5 years, age standard deviation 1.27).

6.2

Results

Figure 5 is a violin plot of the mean Hbt values in the velocity and
fNIRS sessions. Compared to the velocity session, the mean Hbt
values were significantly reduced in the fNIRS session (Wilcoxon
signed-rank test, p=0.05).

Figure 7: IQ score comparing the FOV adjustment using velocity and fNIRS

7

Figure 5: Violin plots: Hbt values, velocity vs fNIRS session
A comparison of the total SSQ scores showed a decrease in 6 participants (Figure 6). But both of total SSQ scores are compared andno significant differences were identified (Wilcoxon signed-rank
test, p=0.05). As a result of comparing the total IQ scores, an increasein scores of 6 participants was confirmed (Figure 7). However, nosignificant differences were identified (Wilcoxon signed-rank test,
p=0.05). We learned from the interviews that one of the reasons
why the SSQ of the fNIRS session increased more than that of thevelocity session was that two participants’ eyes were tired fromconcentrating too much on the game in fNIRS session. The subjects
reported an increase in scores of the eye-related items ("eye strain","difficulty focusing") and also an increase in IQ scores on "focus"and "losing track of time".

226

DISCUSSION AND CONCLUSIONSIn this study, we were able to detect cybersickness taking the current
cognitive load of the user into account. As tolerance for sickness
could vary given the current state of mind and condition of the user(for example: fatigued versus well-rested, hungry versus satiated,etc.) and the VR experiences they play (users might be more prone
to sickness for specific scenarios, e.g. underwater, in the sky), FOV
control by velocity only cannot adapt to these factors. Therefore, weneed a dynamic solution adapting the VR experience to the current
sickness condition of the user. The questionnaire didn’t confirm asignificant difference between the two methods, but this may be
due to the fact that some of the questions apply to both focus on
the game and sickness. A disadvantage is that the weight of the
HMD is increased by our prototype due to the additional fNIRS
device, leading to more fatigue for the user. There are limitations
to our system’s initial implementation as higher hemoglobin levels
can also be induced by other cognitive states (e.g., a user trying tosolve a very hard math problem). This is a current limitation we
addressed by only measuring hemoglobin changes when the useris actually moving (in walking segments). In future work, we alsoneed to take cognitive tasks while walking into account, predict orrecognize them and adjust the FOV accordingly.
In this paper, we present a prototype implementation that dynamically controls the user’s FOV in VR using fNIRS. We detected
Hbt changes with a high enough level of sensitivity to identify
cybersickness. We also controlled the FOV using two methods(velocity and fNIRS) and compared the effects of each method on
cybersickness reduction.

HemodynamicVR

AHs ’21, February 22–24, 2021, Rovaniemi, Finland

REFERENCES[1] Sergi Bermúdez i Badia, Aleksander Valjamae, Fabio Manzi, Ulysses Bernardet,
Anna Mura, Jônatas Manzolli, and Paul FM J Verschure. 2009. The effects of
explicit and implicit interaction on user experiences in a mixed reality installation:
The synthetic oracle. Presence: Teleoperators and Virtual Environments 18, 4 (2009),
277–285.
[2] Laurenz Berger and Katrin Wolf. 2018. WIM: fast locomotion in virtual reality
with spatial orientation gain & without motion sickness. In Proceedings of the
17th International Conference on Mobile and Ubiquitous Multimedia. 19–24.
[3] Costas Boletsis. 2017. The new era of virtual reality locomotion: A systematic
literature review of techniques and a proposed typology. Multimodal Technologies
and Interaction 1, 4 (2017), 24.
[4] Susan Bruck and Paul A. Watters. 2011. The factor structure of cybersickness.
Displays 32, 4 (oct 2011), 153–158. https://doi.org/10.1016/j.displa.2011.07.002[5] Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D
Wilson. 2017. Sparse haptic proxy: Touch feedback in virtual environments
using a general passive prop. In Proceedings of the 2017 CHI Conference on Human
Factors in Computing Systems. 3718–3728.
[6] James E Cutting. 1997. How the eye measures reality and virtual reality. Behavior
Research Methods, Instruments, & Computers 29, 1 (1997), 27–36.
[7] A. S. Fernandes and S. K. Feiner. 2016. Combating VR sickness through subtle
dynamic field-of-view modification. In 2016 IEEE Symposium on 3D User Interfaces(3DUI). 201–210.
[8] Charlene Jennett, Anna L Cox, Paul Cairns, Samira Dhoparee, Andrew Epps,
Tim Tijs, and Alison Walton. 2008. Measuring and defining the experience of
immersion in games. International journal of human-computer studies 66, 9 (2008),
641–661.
[9] Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal.
1993. Simulator sickness questionnaire: An enhanced method for quantifying
simulator sickness. The international journal of aviation psychology 3, 3 (1993),

227

203–220.
[10] Young Kim, Hyun Kim, Eun Kim, Heedong Ko, and Hyun-Taek Kim. 2005. Characteristic changes in the physiological components of cybersickness. Psychophysiology 42 (10 2005), 616–25. https://doi.org/10.1111/j.1469-8986.2005.00349.x[11] Eugenia M Kolasinski. 1995. Simulator sickness in virtual environments. Vol. 1027.
US Army Research Institute for the Behavioral and Social Sciences.
[12] Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick
Baudisch. 2017. Providing haptics to walls & heavy objects in virtual reality bymeans of electrical muscle stimulation. In Proceedings of the 2017 CHI Conferenceon Human Factors in Computing Systems. 1471–1482.
[13] Fabien Lotte, Josef Faller, Christoph Guger, Yann Renard, Gert Pfurtscheller,
Anatole Lécuyer, and Robert Leeb. 2012. Combining BCI with virtual reality:
towards new applications and improved BCI. In Towards Practical Brain-Computer
Interfaces. Springer, 197–220.
[14] Sebastian Marwecki, Andrew D Wilson, Eyal Ofek, Mar Gonzalez Franco, andChristian Holz. 2019. Mise-unseen: Using eye tracking to hide virtual reality
scene changes in plain sight. In Proceedings of the 32nd Annual ACM Symposium
on User Interface Software and Technology. 777–789.
[15] Chizuru Nakagawa. 2008. "Studies on motion sickness using physiological responses". (2008).
[16] Lisa Rebenitsch and Charles Owen. 2014. Individual variation in susceptibility tocybersickness. In Proceedings of the 27th annual ACM symposium on User interface
software and technology. 309–317.
[17] Kay Stanney, Cali Fidopiastis, and Linda Foster. 2020. Virtual Reality Is Sexist:
But It Does Not Have to Be. Frontiers in Robotics and AI 7 (2020), 4. https://doi.org/10.3389/frobt.2020.00004[18] T. Weißker, A. Kunert, B. Fröhlich, and A. Kulik. 2018. Spatial Updating and
Simulator Sickness During Steering and Jumping in Immersive Virtual Environments. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).
97–104. https://doi.org/10.1109/VR.2018.8446620