Poster Session

UIST '20 Adjunct, October 20–23, 2020, Virtual Event, USA

Pleasant Locomotion – Towards Reducing Cybersickness
using fNIRS during Walking Events in VR
Hiroo Yamamura, Holger Baldauf, Kai Kunze
Graduate School of Media Design, Keio University, Yokohama, Japan
usurai-ligno@kmd.keio.ac.jp,holger.baldauf@gmail.com,kai@kmd.keio.ac.jp

ABSTRACTMoving in virtual reality without causing cybersickness is still
an unsolved and difﬁcult problem, especially if the virtual
space is much larger than the real space and the virtual reality environment asks for quick movements. Many methods
to reduce cybersickness are proposed but most of them also
reduce immersion. In this paper, we explore the use of fNIRS
as an additional modality to detect the level of cybersickness
for movement events in VR. We try to mitigate the sickness anindividual feels by narrowing the ﬁeld of vision based on thesickness level detected via measuring increased deoxygenatedhemoglobin (Hb) with fNIRS. Our overall goal is to reduce cybersickness in virtual reality applications using physiologicalsignals and appropriate adjustments with little to no impact on
immersion.
Author Keywords

fNIRS; virtual reality; VR locomotion; simulator sickness;cybersickness

Figure 1. Experimental Setup: Participant wearing the VIVE VR headset and an fNIRS (the HOT-1000 from Hitachi connected to the computer over Bluetooth).

CCS Concepts•Human-centered computing → Interaction design; Empirical studies in interaction design ; Systems and tools forinteraction design;

teleportation: A user can move instantly by changing the position of the virtual camera [1]. Narrowing the ﬁeld of view(FOV) has the potential to reduce cybersickness [7]. Most of
the methods that reduce cybersickness are trade-offs between
cybersickness and reduced immersion. Therefore, the FOV
should only be narrowed when a user’s cybersickness is detected but no other suitable methods of reducing sickness forthis user have been found. We developed a system that detects
a user’s cybersickness in real-time by using functional nearinfrared spectroscopy (fNIRS) and it can narrow their FOV toreduce cybersickness at the appropriate time for each user. We
had two sessions during which we measured the degree of cybersickness, one with our system and one with no narrowing of
the FOV. To objectively evaluate cybersickness, we measured
cerebral blood ﬂow and heart rate. In addition, the simulator
sickness questionnaire (SSQ) was conducted to subjectively
evaluate that. Based on these results, we propose an optimal
VR locomotion system for each user. Our contributions are
as follows: (1) We introduce fNIRS as a novel approach foranalysing cybersickness. (2) We propose an adaptive system
for handeling varying degrees of tolerance to cybersickness
on an individual user level based on fNIRS data. (3) In an
initial test, we observed an increase in hemoglobin related toa stress event of locomotion for a participant who experienced
cybersickness.

INTRODUCTION

VR locomotion has to provide a sensation of moving in VR
space that has a different structure than the physical space[18, 17, 15, 16, 21]. Comfortable VR locomotion requiresa solution to "cybersickness" [2]. Cybersickness consists ofsymptoms similar to motion sickness that may happen whena user moves in a virtual reality environment (VRE). Thesesymptoms are also described as simulator sickness. The difference of deﬁnition between simulator sickness and cybersickness is that simulator sickness is a subset of motion sickness
experienced from travel through VRE. By contrast, cybersickness is the more general term [13]. There are many methods
to reduce cybersickness. One of the most used methods isPermission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor proﬁt or commercial advantage and that copies bear this notice and the full citationon the ﬁrst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
UIST ’20 Adjunct, October 20–23, 2020, Virtual Event, USA© 2020 Copyright is held by the author/owner(s).
ACM ISBN 978-1-4503-7515-3/20/10.
https://doi.org/10.1145/3379350.3416184

56

Poster Session

UIST '20 Adjunct, October 20–23, 2020, Virtual Event, USA

device (model Hot 1000 provided by Neu, Figure 1). First,
we measured cerebral blood ﬂow change for 5 minutes while
the subject was stationary and 5 minutes during a locomotion
event (using a controller to move). Based on this we can see anincrease in deoxygenated hemoglobin (Hb) for the movement
part (blue line in Figure 3) for the movement (locomotion
event). The SSQ administered after the recording showed ascore of 48 points indicating cybersickness.

RELATED WORK AND APPROACH

In a second test, we set an experimental threshold basedon the initial recording of double increase in deoxygenatedhemoglobin (Hb) to introduce a narrowed FOV to mitigatecybersickness. The Hb change is shown in the orange line
(Figure 3). Apart from that, the setup is the same. When introducing the narrowed FOV based on the hemoglobin change
the SSQ score after the locomotion event was 30, moving fromthe perceivable to the unnoticeable part of the SSQ score forcybersickness.

Figure 2. HMD with integrated fNIRS device with proposed interaction.

Cybersickness seems to have high individual variation [19].
There is a lot of work to quantify, detect and mitigate cybersickness, often also called virtual reality sickness[4, 6, 11, 20].
Our approach follows research using physiological signals toestimate the degree of cybersickness in real-time while theuser experiences it. We try to apply mitigating methods tolimit the sickness for the individual user. In this work, we add
fNIRS as a potential detection technology for cybersickness.
To the best of our knowledge, we are unaware of any work
using a commercial fNIRS for adaptive cybersickness detection and mitigation. So far only medical equipment has beenused [8]. There is also work that shows it is easy to integrate
fNIRS with headsets [10].

As shown in previous works, heart rate (recorded also over thefNIRS sensors) is elevated for the case of cybersickness [9],as shown in Figure 4.

The most common way to assess cybersickness is the use of
the Simulator Sickness Questionnaire (SSQ). The SSQ assesses the symptoms caused by simulator sickness, such as
nausea, headache and vertigo. The SSQ comes with major advantages/disadvantages of any questionnaire metric. The usermight forget the actual scale and it might impact immersion if
administered during a VR experience.

Figure 3. Initial data recordings: deoxygenated hemoglobin (Hb) for a
movement event where cybersickness occurred (blue) and where it wasmitigated by a narrowed FOV (orange).

There is a lot of research that assesses cybersickness using
physiological signals. So far, there is no robust detection mechanism for it [5]. Bruck et al.’s study suggests that increased
arousal leads to changes in respiratory rate and low carbon
dioxide levels [3]. Kim et al.’s study suggests that the total
severity of cybersickness had a signiﬁcant positive correlation
with eye blink rate, heart period, and electroencephalogram(EEG) delta wave etc. [9]. Nakagawa et al.’s study reported
that when participants feel low sickness, their respiratory variability and tidal volume are decreased [14]. In this research,
we used fNIRS to measure changes in cerebral blood ﬂow
and heart rate. Increased cerebral blood ﬂow and heart rate
have been related to cybersickness. The major advantages offNIRS are that it is non invasive and less susceptible to data
corruption by movement artifacts compared to EEG[12].

Figure 4. Heart rate for stationary (not sick), FOV not narrowed (sickness occurred) and FOV narrowed (no sickness while moving).

CONCLUSION

We are working on using fNIRS as a novel technology toassess and mitigate cybersickness. It is affordable and can beeasily integrated in any VR headset (2 LEDs plus sensors),
it provides several physiological signals that are related tocybersickness and our initial data recording seems promising.
Based on these insights we plan an experimental setup toevaluate the use of fNIRS in relation to cybersickness eventsand how to prevent them.

The use of fNIRS to diagnose and mitigate cybersickness
could make it possible to provide an individualized solution
without interfering with the subject’s sense of immersion.
EXPERIMENTAL INSIGHTS

For an initial data exploration we tested our setup with one participant. The subject uses an HMD with an integrated fNIRS
57

Poster Session

UIST '20 Adjunct, October 20–23, 2020, Virtual Event, USA

REFERENCES

Hung. 2017. Estimating the simulator sickness inimmersive virtual reality with optical ﬂow analysis. In
SIGGRAPH Asia 2017 Posters. 1–2.

[1] Laurenz Berger and Katrin Wolf. 2018. WIM: fast
locomotion in virtual reality with spatial orientation gain
& without motion sickness. In Proceedings of the 17th
International Conference on Mobile and Ubiquitous
Multimedia. 19–24.

[12] Sarah Lloyd-Fox, Anna Blasi, and C.E. Elwell. 2009.
Illuminating the developing brain: The past, present andfuture of functional near infrared spectroscopy.
Neuroscience and biobehavioral reviews 34 (08 2009),
269–84. DOI:

[2] Costas Boletsis. 2017. The new era of virtual reality
locomotion: A systematic literature review of techniques
and a proposed typology. Multimodal Technologies andInteraction 1, 4 (2017), 24.

http://dx.doi.org/10.1016/j.neubiorev.2009.07.008[13] Michael McCauley and Thomas Sharkey. 1992.
Cybersickness: Perception of Self-Motion in Virtual
Environment. Presence 1 (01 1992), 311–318. DOI:

[3] Susan Bruck and Paul A. Watters. 2011. The factor
structure of cybersickness. Displays 32, 4 (oct 2011),
153–158. DOI:

http://dx.doi.org/10.1162/pres.1992.1.3.311

http://dx.doi.org/10.1016/j.displa.2011.07.002[14] Chizuru Nakagawa. 2008. "Studies on motion sickness
using physiological responses". (2008).

[4] Henry Been-Lirn Duh, Donald E. Parker, and Thomas A.
Furness. 2001. An “independent visual background”
reduced balance disturbance envoked by visual scene
motion: implication for alleviating simulator sickness.
In Proceedings of the SIGCHI conference on humanfactors in computing systems. 85–89.

[15] Benjamin I Outram, Yun Suen Pai, Tanner Person, Kouta
Minamizawa, and Kai Kunze. 2018. AnyOrbit: Orbital
navigation in virtual environments with eye-tracking. In
Proceedings of the 2018 ACM Symposium on Eye
Tracking Research & Applications. 1–5.

˙ nska,´[5] Natalia Duzma
Paweł Strojny, and Agnieszka
Strojny. 2018. Can Simulator Sickness Be Avoided? A
Review on Temporal Aspects of Simulator Sickness.
Frontiers in Psychology 9 (2018).

[16] Yun Suen Pai, Zikun Chen, Liwei Chan, Megumi Isogai,
Hideaki Kimata, and Kai Kunze. 2018. Pinchmove:
improved accuracy of user mobility for near-ﬁeld
navigation in virtual environments. In Proceedings ofthe 20th International Conference on Human-Computer
Interaction with Mobile Devices and Services. 1–11.

[6] Tobias Feigl, Daniel Roth, Stefan Gradl, Markus Wirth,
Marc Erich Latoschik, Bjoern M Eskoﬁer, Michael
Philippsen, and Christopher Mutschler. 2019. Sick
Moves! Motion Parameters as Indicators of Simulator
Sickness. IEEE transactions on visualization andcomputer graphics 25, 11 (2019), 3146–3157.

[17] Yun Suen Pai and Kai Kunze. 2017. Armswing: Usingarm swings for accessible and immersive navigation inar/vr spaces. In Proceedings of the 16th International
Conference on Mobile and Ubiquitous Multimedia.
189–198.

[7] A. S. Fernandes and S. K. Feiner. 2016. Combating VR
sickness through subtle dynamic ﬁeld-of-view
modiﬁcation. In 2016 IEEE Symposium on 3D UserInterfaces (3DUI). 201–210.

[18] Kirill Ragozin, Kai Kunze, Karola Marky, and Yun SuenPai. 2020. MazeRunVR: An Open Benchmark for VR
Locomotion Performance, Preference and Sickness inthe Wild. In Extended Abstracts of the 2020 CHI
Conference on Human Factors in Computing Systems(CHI EA ’20). Association for Computing Machinery,
New York, NY, USA, 1–8. DOI:

[8] Alireza Mazloumi Gavgani, Rachel H.X. Wong,Peter R.C. Howe, Deborah M. Hodgson, Frederick R.
Walker, and Eugene Nalivaiko. 2018.
Cybersickness-related changes in brain hemodynamics:
a pilot study comparing transcranial Doppler andnear-infrared spectroscopy assessments during a virtualride on a roller coaster. Physiology & Behavior 191(2018), 56–64.

http://dx.doi.org/10.1145/3334480.3383035[19] Lisa Rebenitsch and Charles Owen. 2014. Individual
variation in susceptibility to cybersickness. InProceedings of the 27th annual ACM symposium on
User interface software and technology. 309–317.

[9] Young Kim, Hyun Kim, Eun Kim, Heedong Ko, andHyun-Taek Kim. 2005. Characteristic changes in thephysiological components of cybersickness.
Psychophysiology 42 (10 2005), 616–25. DOI:

[20] Misha Sra, Abhinandan Jain, and Pattie Maes. 2019.
Adding Proprioceptive Feedback to Virtual Reality
Experiences Using Galvanic Vestibular Stimulation. In
Proceedings of the 2019 CHI Conference on Human
Factors in Computing Systems. 1–14.

http://dx.doi.org/10.1111/j.1469-8986.2005.00349.x[10] Aleksandra Landowska, Sam Royle, Peter Eachus, andDavid Roberts. 2018. Testing the Potential of
Combining Functional Near-Infrared Spectroscopy withDifferent Virtual Reality Displays—Oculus Rift andoCtAVE. In Augmented Reality and Virtual Reality.
Springer, 309–321.

[21] Sam Tregillus and Eelke Folmer. 2016. Vr-step:
Walking-in-place using inertial sensing for hands freenavigation in mobile vr environments. In Proceedings ofthe 2016 CHI Conference on Human Factors in
Computing Systems. 1250–1255.

[11] Jiun-Yu Lee, Ping-Hsuan Han, Ling Tsai, Rih-Ding
Peng, Yang-Sheng Chen, Kuan-Wen Chen, and Yi-Ping
58