Virtual Reality
https://doi.org/10.1007/s10055-021-00571-w

ORIGINAL ARTICLE

NapWell: An EOG‑based Sleep Assistant Exploring the Effects of Virtual
Reality on Sleep Onset
Yun Suen Pai1 · Marsel L. Bait1 · Juyoung Lee2 · Jingjing Xu1 · Roshan L Peiris3 · Woontack Woo2 ·Mark Billinghurst4 · Kai Kunze1
Received: 11 June 2020 / Accepted: 10 August 2021
© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2021

Abstract
We present NapWell, a Sleep Assistant using virtual reality (VR) to decrease sleep onset latency by providing a realistic
imagery distraction prior to sleep onset. Our proposed prototype was built using commercial hardware and with relatively low
cost, making it replicable for future works as well as paving the way for more low cost EOG-VR devices for sleep assistance.
We conducted a user study ( n = 20) by comparing different sleep conditions; no devices, sleeping mask, VR environmentof the study room and preferred VR environment by the participant. During this period, we recorded the electrooculography(EOG) signal and sleep onset time using a finger tapping task (FTT). We found that VR was able to significantly decrease
sleep onset latency. We also developed a machine learning model based on EOG signals that can predict sleep onset with across-validated accuracy of 70.03%. The presented study demonstrates the feasibility of VR to be used as a tool to decrease
sleep onset latency, as well as the use of embedded EOG sensors with VR for automatic sleep detection.
Keywords Virtual reality · Sleep onset · Electrooculography

1 Introduction
Sleep, like eating and drinking, is biologically imperative
to all living beings, while also carrying other benefits like
improving one’s memory and cognitive abilities (Milner andCote 2009). Power naps specifically show us that sleeping
can improve alertness, productivity, and mood which is especially useful in scenarios like night shift work or prolonged
driving sessions (Mednick et al. 2002). However, many modern technologies prevent us from taking a nap by providing
mental stimulation, sudden notifications, as well as modifying light-based cues (e.g. increased blue-light after sunset
which can promote alertness). Many efforts have been madeto use technology to fall asleep faster, from pharmacological* Yun Suen Pai
pai@kmd.keio.ac.jp
1

Keio University Graduate School of Media Design,
Yokohama, Japan

2

Korea Advanced Institute of Science and Technology,
Daejeon, Korea

3

Rochester Institute of Technology, New York, USA

4

The University of Auckland, Auckland, New Zealandagents to mechanical and physiological efforts such as cortical electrical stimulation and acoustic stimulation (Daviset al. 1939; Lee et al. 2019; Akert et al. 1951). However,while these technologies attempt to directly influence thenap time for a user, they do not control the environment theuser sleeps in nor do they improve a users’ sleep hygiene(how routine and effective sleep is for a person). One critical element to sleep hygiene in power naps is a proper nap
environment, free of too much mental stimulation, little (ifany) blue light, and be comfortable and/or familiar (Muzet
2007). This kind of environment may be hard to achievein the physical world, especially in circumstances where aperson is traveling, wishes to nap in a crowded space (e.g.
an airport) or in an office or workplace. Most of us resort toimagery distraction to place ourselves mentally elsewhere
to cope with such a situation (Harvey and Payne 2002). A
common solution could be to simply wear noise-cancelingheadphones, or listen to some soft audio that suits the imagined environment.
In this work, we present a study to understand if VR
environments can assist in imagery distraction by providing a virtual environment prior to the nap to decrease oursleep onset latency (time taken to transit from wakefulness to sleep). During the nap, we log the participants’

13Vol.:(0123456789)

Virtual Reality

eye movements using electrooculography (EOG) while
employing a behavioral finger tapping task (FTT) task todetect sleep onset time ground truth. We also used thegathered EOG signal to train a machine learning model
for automatic sleep onset detection. This allows us toconceptualize an all-in-one prototype VR head-mounted
display (HMD) that can automatically detect sleep onset
using embedded sensors in the HMD itself. Below are thecontributions of this work:
1. We conducted and present results of a pilot survey( n = 157) to determine the sleep habits and preferred
nap environments.
2. We present to our knowledge the first study ( n = 20 )exploring the effects of VR on sleep using both physiological and behavioral measures.
3. Our study shows that VR can significantly reduce sleep
onset latency compared to no devices at all, and on
average also provided longer sleeping time and lesser
drowsiness onset latency.
4. We conceptualize a VR prototype with integrated EOG
sensors and a machine learning model that can automatically detect sleep onset with a cross-validated accuracyof 70.03%.

2 Related WorkIn this section, we look into the past decades of work that
focused on understanding the parameters that influence oursleep quality and sleep pattern. HCI research has touchedon improving sleep before, such as work by Liang and Ploderer (2016) and Ehleringer and Kim (2013) who investigated if commercial wearable devices improve sleep andthe possibility of a wearable device to improve sleep quality for dementia patients, respectively. There are several
good overviews regarding sleep assessment, the benefitsof sleep, and pervasive technologies related to sleeping(Schmidt et al. 2012; Ibáñez et al. 2018). There are also
some contactless sleep tracking works using smart homeinfrastructure (Adib et al. 2015). SleepThermo (Katsumata
et al. 2019) was a wearable device that monitors body temperature with respect to sleep quality. Shirazi et al. (2013)use an alarm clock smartphone application to track andshare sleep behavior. There are other works that use smartphones or fitness trackers for sleep quality tracking (Minet al. 2014; Choe et al. 2015).
Closest to our work, Kitson et al. explore lucid dreaming as a tool for introspection in Virtual Reality (Kitson
et al. 2018). Semertzidis et al. (2019) also explore thepre-sleep state, yet in a much more artistic and qualitativeevaluation. We see our work as being complementary to13this research, as we are evaluating sleep onset and utilize
EOG (compared to EEG in Semertzidis et al.’s study). In
this work, we mainly focus on the various sleep detection methods used and VR’s effect and immersion on thehuman mind.

2.1 Sleep detection
When someone falls asleep, several markers are present
such as a decrease in attention and change in physiological signals (Ogilvie 2001). In most related work, methods to detect sleep onset largely fall into two categories;
physiological and behavioral-type methods (Scott andLack 2017). Physiological detection refers to the use ofphysiological signals such as electrooculography (EOG),
polysomnographic (PSG) (gold-standard for sleep studies),
electroencephalogram (EEG), and so on to detect sleep
onset. Detecting sleep onset can be a complicated andcostly process, usually conducted only at traditional sleep
labs with expensive PSG equipment and a trained specialist. PSG detects sleep onset specifically based on a reduction of alpha waves and the dominance of low voltage
waves (Berry et al. 2012). However, there has been somerelated work on using simple EEG electrodes which arecheap and easily accessible to detect sleep onset (Zhang
et al. 2014). EEG has also proven to be reliable enoughto even detect microsleep (Rothkrantz 2016). However,signals from EEG require intense filtering due to noise.
Otherwise, other more invasive methods are required,such as implants through the skull for a better signal-tonoise ratio. There have also been low-cost approaches withrelatively high accuracy, such as the use of commerciallyavailable heart rate sensors (Okamura et al. 2016). However, one of the earliest approaches used to detecting sleep
is to recognize the rapid eye movements (REM) via electrooculography signals (Boukadoum and Ktonas 1986).
Both EEG and EOG are non-invasive measures; however,EEG signals try to measure the brains potentials thougha thick skull and are thus affected by the wearer’s fullphysical and physiological state. In contrast, EOG senses
the electrical potential difference between the cornea andthe retina only, leading to overall less noisy signals. Whencoupled with a head-mounted wearable, this will cause
more issues for EEG as well, since a minor shift in electrode placement due to the straps on the head will lead tonoisy signals once again (Gupta et al. 2020). EOG need
only be in contact with the user’s face. Furthermore, AR/
VR head-mounted displays are often in contact with theface, making EOG a more logical approach. EOG has also
proven to be able to outperform EEG-based methods insleep stage classification (Rahman et al. 2018).
Behavioral detection, on the other hand, can be further
divided into two categories; active and passive. Active

Virtual Reality

behavioral detection refers to assigning a task that requires
minimal effort for the participant to perform while trying
to fall asleep. Once he/she stops performing it, that timeis registered as the sleep onset time. It was first proposedby Blake et al. (1939) who asked his participants to holda spoon when falling asleep. Once the spoon dropped,
the time was recorded as sleep onset, which also correlates with a drop in alpha waves of the brain. Reaction
time (RT) tasks are also often employed for this detection
method (Liberson and Liberson 1966; Ogilvie and Wilkinson 1984). RT generally relies on audio feedback, where
the participant needs to react to audio stimuli. However,
one key issue with active behavioral detection is that there
is a possibility for the stimuli to instead disrupt the processof falling asleep (Ogilvie 2001). This is evident, especiallywhen employing RT because it promotes “readiness” fromthe participants, which increases arousal (Ogilvie andWilkinson 1988). This resulted in the introduction of passive behavioral detection which refers to gathered sensor
data from user behaviour, usually actigraphy which is theuse of inertial sensors. Another approach used a pressure
mattress to analyze the heat map of the user’s sleeping
posture and behaviour (Metsis et al. 2014). There havealso been studies for detecting long-term sleep patterns
using a combination of inertial, ambient light, and time
data (Borazio and Van Laerhoven 2012). However, thisdoes not mean that the passive approach is superior to
the active, because the recent introduction of devices andapplications that uses very discreet stimulus and requiresvery minimal response significantly reduces any disruption
to sleep onset (Scott and Lack 2017).
Between physiological and behavioral sleep detection,
both methods have a reasonable degree of accuracy. With
PSG being the gold standard yet not easily accessible,
behavioral methods have only shown a small difference ofabout 2 to 3 min (Connelly 2004; Lack and Mair 1995; Scottet al. 2018). This is due to a slight discrepancy that exists
between sleep onset for physiological signals and behavioral activities. People can still possibly give a behavioral
response to a stimuli even after PSG has determined sleep
onset because most people only stop responding during the
beginning of N2 sleep (Ogilvie and Wilkinson 1988; Ogilvie
et al. 1989). One area where active behavioral sleep detection is potentially more beneficial than both physiologicaland passive behavioral methods is for short power naps,
because the signals gathered using the other two methods
require signal processing steps, whereas active methods
are relatively instantaneous simply by observing the user’s
response to stimuli. However, it is still possible for actigraphy to detect naps with relatively high accuracy if the nap
duration is at least 30 min (Kanady et al. 2011). Sleep onset
time can be complex because it is a dynamic process marked
by the change on both physiological and behavioral statesat different points in time. In this study, we actually use an
active behavioral task to first detect sleep onset when napping, then use that data as a label for a physiological-based
detection, allowing our final prototype to the best compromise between them.
Regarding the factors that influence sleep, the primaryreason that led us to want to understand the effects of VR
towards sleep onset is due to imagery distraction in sleep
studies. Imagery distraction was a proven method to help
insomniac patients fall asleep (Harvey and Payne 2002).
This is because imagery distraction occupies the cognitive
space that prevents the participant from re-engaging withworrisome thoughts that causes insomnia. This brings us to
the use of VR, which is very similar to imagery distraction inthat, it is capable of placing us virtually into another space.
However, it has, to our knowledge, never been used as asleep-assist tool, though the closest literature we can find isfor stress relief and meditation.

2.2 Sleep assist tools
There has been a myriad of methods employed by people
to aid them in their sleep, either with the use of devices or
through some form of sleep ritual. Typically, it is advisable to alter one’s habit, such as not sleeping too late, doing
more exercise, eat healthily and so on Brown et al. (2002),
yet there will always be times where an external device or
stimulus is preferable.
An eye or sleep mask is one of the most common tool
to assist sleep. It does not serve as a stimulus, but rather tofurther block out light that can suppress melatonin. Studies have shown that a sleep mask can result in more REM
time, shorter REM latency, less arousal, and elevated melatonin levels (Hu et al. 2010). It was also found to be able toreduce long awakenings for patients in intensive care units(Demoule et al. 2017; Yazdannik et al. 2014). Additionally,
patients with acute coronary syndrome also found sleep
masks to be a cheap and viable method over drug therapy toincrease sleep quality (Daneshmandi et al. 2012). This has
led to works like the Smart Eye Mask (Matsui et al. 2017)that integrated photo-reflectors into sleep masks to classify
sleep stages. This method, however, adds bulk and weight
onto the eye mask which was meant to be light and soft.
Another potential negative impact of eye masks is that theypress against the user’s eyes, leading to possible discomfort
on the eyes.
One of the most common tool is to use audio stimulus via
binaural sounds to induce sleep. When an acoustic beat of
two tones is played in each ear simultaneously, the generated
binaural beat induces brain signals that can assist in sleep(Lee et al. 2019; MORALES-COBAS et al. 1995). Several
researches look into this specific frequency, such as by Jirakittayakorn and Wongsawat (2018) who found that binaural

13

Virtual Reality

beat at 3Hz and found sleep latency to be relatively shorter.
A lot of these works, like the sleep mask, also specificallytarget those with health conditions that prevents them fromsleeping. Pedemonte et al. (2014) used auditory feedback
during sleep to treat participants with idiopathic tinnitus andfound significant changes in the brain activity. Audio feedback remains a popular choice for sleep assistance because
the responsiveness of the auditory system during sleep isstill partially preserved (Peña et al. 1999).

2.3 VR for stress relief, therapy, and meditation
The perceived environment and atmosphere can greatly
influence a person’s cognitive performance, mood, and physiology (Zhao et al. 2017). Even though studies regarding theuse of VR on sleep is relatively rare to our knowledge, themost relatable use cases would be for stress relief, therapy,
or meditation. For example, post-traumatic stress disorder(PTSD) can occur as a result of a disastrous event or experience and has been studied most in war veterans. PTSD
symptoms include sleep disruptions (next to a multitude ofother indicators). Studies with Vietnam War veterans in VR
to desensitize the life-changing event showed that this treatment can significantly decrease their symptoms including
sleep disruptions (Rothbaum et al. 1999).
There is also a lot of work using virtual reality and mixed
reality for relaxation, stress relief, and mindfulness training(Kosunen et al. 2016; Amores et al. 2016; Roo et al. 2017;
Bernardino et al. 2016). We see our work complementary,
as we focus directly on naps and the impact of VR on napping properties.
Meditation can be relatable to naps because it requires theuser to be in a calm state of mind. For example, RelaWorld(Kosunen et al. 2016) combines neurofeedback with VR as ameditation system to induce deeper relaxation, feeling of thepresence, and a deeper level of meditation. For therapy, VR
has consistently been used especially for exposure therapy
for phobia treatment (Powers and Emmelkamp 2008), tomental and eating disorders (Ferrer-Garcia et al. 2013).
Both meditative experiences and therapy can also be
combined, such as the development of the Meditation Chamber (Shaw et al. 2011). This device uses immersive virtualenvironments with biofeedback technologies (galvanic skin
response and heart rate) to reduce stress and anxiety. The
main negative feedback, though, was that the weight of thedevice hampers the experience. A recent work employed
the use of VR for meditative walks as a way to treat chronic
pain (Gromala et al. 2015). The developed virtual meditative
walk (VMW) provides a peaceful environment with changing weather that depends on the user’s physiological state.
The effects of VR on the human mind are primarily due
to the sense of the presence it simulates, as this aids in perception and consciousness (Sanchez-Vives and Slater 2005).

13

Cognitively, we know that we do not exist at a virtual space,but sub-consciously, we respond to it. Therefore, we wish toemulate this effect in our study on understanding the relationship between VR and naps. How can VR influence sleep
onset time, and how to conceptualize a prototype HMD forthis purpose?

3 Behavioural sleep and drowsiness onset
detection
The first step in this work would be to detect sleep onset
accurately. As seen from related work, it can be challenging
to detect sleep onset without access to any specialized hardware. More lightweight detection as proposed by Okamura
et al. (2016) use a smartphone application with the specificintention of detecting sleep onset. Their work uses finger
tapping during a beep or vibration where, if the user does
not do so after 4 indications, the system perceives the user
to have fallen asleep and the application records the time.
We used a similar approach, the finger tapping task (FTT)
proposed by Casagrande et al. (1997). It is proven to be better than an auditory-based reaction task since it interferesless with the sleep onset process and thus reduces arousal.
We developed an FTT smartphone application, as shownin Fig. 2 that requires the user to tap the screen continuously.
The smartphone was strapped to the participant’s preferredhand using a velcro strap to make it effortless to hold andeasily accessible (Ogilvie and Wilkinson 1988). The numberof taps, tap rate, and time for each tap is then logged into thedevice. If the user does not tap for more than 300 seconds,
then he/she is deemed to have fallen asleep (Casagrande
et al. 1997). The user’s tapping rate also allows for drowsy
onset detection. Since each user taps their finger at theirown pace, we first find their average tapping speed during
their first 50 taps. The drowsy onset time is defined as when
the average tapping frequency decreases tenfold. We findthis estimation of drowsy onset time to be a suitable value
because it falls into the range of time for Stage 1 latency(Casagrande et al. 1997). Sleep duration can be found bylooking at the highest number of ms between a tap without
any interruption.
Additionally, we log certain physiological signals
from the user during their napping duration. We chose touse EOG sensing extracted from the JINS MEME smart
glasses1 as shown in Fig. 3, which is designed for everyday
use Uema and Inoue (2017). Logging of the JINS MEME
data is achieved via a Macbook running a Python scriptthat received the data through Bluetooth with a sampling
rate of 100Hz. To calculate EOG values, JINS combines
1

https://​jins-​meme.​com/​en/.

Virtual Reality

the capacitance difference from the two nose pads for thehorizontal EOG component, and the difference between
nose pads and nose bridge for the vertical EOG component(Ishimaru et al. 2014). We log the EOG signal to develop a
prototype that is able to detect sleep onset automatically. We
present the experimental setup in the following.

5 User study

4 Pilot survey

For the hardware, we choose the Google DayDream View
VR2 headset with the Asus Zenfone AR3 as our primary VR
tool due to several reasons. A lightweight, wireless headset is essential to maintain the comfort of the user and to
avoid any cable entanglement common with commercial
VR devices tethered to a desktop computer. Secondly, theweight of the headset is also a primary concern. Since the
DayDream View uses mobile VR, it is overall lighter thandesktop connected HMDs. Finally, the DayDream View uses
fabric and soft-like texture to ensure comfort for the user.
The HMD is integrated with EOG sensors taken directlyfrom a pair of JINS MEME smart glasses4 (Fig. 3). The sensors’ vertical position is adjustable so that it just rests on theparticipant’s face and that it would be comfortable for different face shapes. The batteries are attached to the left side.
The power and pairing button is attached to the right side of
the HMD. A Macbook5 is placed on a nearby table to readand log the EOG signals in real-time. The chair used in the
study is the GTRacing Gaming Chair6 that can recline backward until 90◦. The phone used for the FTT is a Samsung
Galaxy S87, though any Android phone running Android
5.0 and above should be compatible. For the VR conditions,the participants also used a pair of Sony Noise Cancelling
headphones8 that plays the audio of the virtual environment.
Initially, we had the participants also use the Apple Watch9
and Xiaomi Mi Band10 for passive behavioral detection andcompare the onset time with the FTT. However, pilot testing
showed that they were unable to detect short periodic naps
accurately. We, therefore, choose to exclude them.

Before initiating the experiment, we first ran a survey togather relevant data regarding peoples’ general sleep behavior. The survey was deployed to social media platforms(Facebook, Twitter, WeChat, etc.) for a week. We successfully gathered data from a total of 157 participants (52males) from 33 different countries (27.7% are from Indonesia, and 35.5% are from the Philippines). We found that
46.5% of the participants sleep for only 6 h every night, and56.7% of them believe that their sleep quality can be further
improved. 46.5% of the respondence find it difficult to fallasleep. 59.9% of them are relying on some form of activity or habit to fall asleep faster at night. One of the maincauses is work or school-related stress, followed by a noisy
environment.
Regarding naps, only 22.3% of them take naps very orquite often, with the majority of them being not that often.
The reason for taking a nap or lack thereof is related to work.
Too much work prevents nap time, while also causing someof them to be too tired to work continuously. When asked toimagine a place where they could fall asleep more quickly,
the top five choices are their own room (73.1%), a hotel
room (35.9%), by the clouds (17.3%), the beach (15.4%),
and finally a lodge by the woods (14.1%) as shown in Fig. 4.
We believe that their own room, or a hotel room scored thehighest because most people find these places familiar. The
other preferences are mostly natural surroundings, supported
by literature (Felsten 2009). We use these results for thebasis of our selected VR environments to assist naps. The
survey also included the Epworth Sleepiness Scale (ESS)
Questionnaire (Johns 1991), which showed an ESS score of
9.6 in average across our 157 participants. They fall into thenormal range of ESS scores (majority not having any formof sleep disorder). We provided the link for the full resultsof our questionnaire (Bait 2019).

To understand how VR can effect sleep onset, we design
the experimental setup akin to taking a nap at work in the
middle of the day.

5.1 Apparatus

5.2 Study design
The independent variable is the device used, with four
distinct conditions: (1) napping without any device as a2

https://​vr.​google.​com/​daydr​eam/.
https://​www.​asus.​com/​Phone/​ZenFo​ne-​AR-​ZS571​KL/.
4
https://​jins-​meme.​com/​en/.
5
https://​www.​apple.​com/​nz/​macbo​ok-​air/.
6
https://​www.​gtrac​ingch​air.​com/.
7
https://​www.​samsu​ng.​com/​global/​galaxy/​galaxy-​s8/.
8
https:// ​ w ww. ​ s ony. ​ c o. ​ n z/ ​ e lect ​ ronics/ ​ h eadb ​ a nd- ​ h eadp ​ h ones/​ w h-​
1000x​m3.
9https://​www.​apple.​com/​apple-​watch-​series-5.
10
https://​www.​mi.​com/​global/​mi-​smart-​band-4.
3

13

Virtual Reality

Fig. 1  (left) An illustration of the prototype that combines EOG sensors from JINS Meme with Google Daydream, (right) and a participant of
the study using the prototype on a reclining office chair with a smartphone for a finger tapping task to detect sleep onset manually

Fig. 2  Screen capture of the FTT application logging the time stamp,
tap number, and tap rate

Fig. 3  Prototype used for the
study, with embedded, adjustable EOG sensors shown on thefront view, and the battery andpower switch on the top

13

baseline, (2) using a sleeping mask, (3) with a VR HMD
showing the view of the study room itself (baseline for VR),and with (4) a VR HMD displaying the participant’s preferred virtual environment (includes headphones playing
audio of the selected environment). These conditions werechosen by taking into consideration how each items anddevices are used conventionally, as opposed to the sensory
feedback provided. For example, the sleeping mask condition does not include any form of audio feedback or earplugs
as it is being used as a standalone item (Liang et al. 2015). A
standalone VR experience, on the other hand, often includes
both visual and audio feedback (Jerald 2015). For all conditions, the room light is left on to simulate an actual officeworking hour, as shown in Fig. 6.
For monitoring purposes, we avoided being in the sameroom as the participant when they were napping. Therefore,
the FTT application was designed to send packets of data toa mobile device so that the tapping activity can be monitored
remotely. When the tapping rate decreases, it indicates thatthe participant is feeling drowsy, and when it stops, it is anindication that the participant has fallen asleep.

Virtual Reality
Fig. 4  Survey results regarding
preferred locations to fall asleep
easily with a sample size of 157

Fig. 5  Screenshot of the VR application, showing the (1) home
screen with instructions to load the “own room” video into the application, (2) the main menu where the user can select using the DayDream controller, (3) the environment for the “study room” condi-

tion, (4) the “own room” condition which differs according to theparticipants’ room, (5) the “hotel room”, (6) the “by the clouds”, (7)the “beach”, and (8) the “lodge by the woods”

5.3 Virtual environments

length so that the looping is not obvious. We initially tested
with virtual environments built with a game engine, but thegraphical quality needs to be on par with desktop-VR toachieve high realism. However, we got low framerates since
it is just running off from a mobile phone. We, therefore,chose to use 360◦ video instead, which maintains realism

The prepared virtual environments for the fourth condition
are the top 5 results from the previously conducted pilot
survey (see Section PILOT SURVEY). Figure 5 depicts
the different environments. Each clip is at least 15 min in13

Virtual Reality
Fig. 6  The participants experiencing (left) the baseline condition with no devices, (middle)
with the use of a sleeping mask,and (right) with VR (the “studyroom” and “preferred VR”
condition)

while requiring low computing power. Participants whochose the “own room” environment were required to inform
us at least a day before the experiment for the VR condition, where we lend the participant a 360◦ camera (the Ricoh
Theta S) to capture a video of their own bed room. The
video includes the ambient sound of the room as well. Oncethey did, we directly imported the environment into the VR
application. Additionally, a blue-light filter was added into
the environment to negate the effects of blue light on sleep
onset (Green et al. 2017) (all conditions requiring the VR
display to be on uses the filter). The dependant variable isthe sleep onset time.

5.4 Participants
We recruited a total of 20 participants (7 female), aged
between 22 to 29 (mean = 25.05, STD = 1.79) to participate in this within-subject study. The experiment itself isconducted in a regular office room to simulate naps in aworking office environment. Each of the participants may
freely wear any attire that they are comfortable with, and any
restrictions were excluded. The room temperature is maintained between 25 to 28 degrees Celsius. The noise level ofthe study room itself is around 32dB, which is normal fora typical quiet room. Each participant was required to haveat least 7 to 8 h of sleep the night before the experiment isconducted. Furthermore, we ensured that each of the experiments is conducted at least 3 h after the participant had ameal, to ensure that he/she would not be affected by caffeine,sugar, or alcohol intake (Okamura et al. 2016). The study
was performed according to ethical rules and regulations of
Keio University.

setup. The recorded video for the participant’s room will
be deleted at the end of his/her session. Each of them alsocame on four different days to participate for each of theconditions.
After we briefed them regarding the procedure, the participants lie down on the reclined office chair (see Fig. 1).
We used an angle of 40◦ for the user study because it is an
optimal angle for naps on a reclining chair as proven by
Nicholson and Stone (1987). Depending on the assigned
condition, we then put on the HMD, sleeping mask, or no
device at all, onto the participant. The FTT phone was also
securely strapped to the desired hand with velcro where they
are instructed to continuously tap on the screen. A nearby
Macbook was set up to record the logged EOG data, whereas
the FTT data are stored directly in the phone. Each participant was given a maximum period of 1 hour for taking a nap.
For participants who have fallen asleep, the FTT data will
indicate this based on the duration between taps. However,if the participant did not fall asleep, we take the maximum
allocated time of 1 hour for the duration.

6 Results and discussion
In this section, we divide the results into two main sections; the tapping data analysis to extract sleep onset
latency, drowsy onset latency and sleeping duration, and thesleep onset classification accuracy. Of the 20 participants
recruited, 8 of them chose their own room as the preferred
VR condition. 5 participants chose the beach side, followedby another 5 who chose the lodge in the woods. Only 2 participants chose the hotel room environment.

5.5 Procedure

6.1 Tapping data analysis

Prior to the experiment, we ask the participants about theirpreferred VR environment among the available options (theirown room, the hotel room, being among the clouds, by thebeach, and a lodge by the woods). We also show them a shortpreview. If they chose the “own room” scenario, we lend a360◦ camera to them for a recording for 15 min so that we
can import it into the application during the experiment day.
On the experiment day, participants filled out a consent
form detailing personal data saved and the experimental

Figure 7 shows a sample result from a participant for the“preferred VR” condition. The red area represents the participant still being awake, the period between each taps
being about 500 ms. This illustrates how a tap data generally look for a participant falling asleep. When they start
becoming drowsy, the yellow region shows the taps slowing
down, with the period between each tap being about 40,000
to 50,000 ms. This is aligned with our estimation that thetapping speed decreases by ten times the average tapping

13

Virtual Reality
Fig. 7  A sample FTT result
from a participant for duration
between each tap. This is an
extracted time window of about
33 minFig. 8  Extracted results from the FTT, which shows the sleep onset latency (left), maximum nap duration (middle), and drowsiness onset latency(right) for all 20 participants

speed as it enters Stage 1 of sleep latency (Casagrande et al.
1997). In this stage, they have not fallen asleep yet, but are in
the transitional phase between fully awake and fully asleep.
Finally, the green region shows that the participant had
fallen asleep, with the tap period being above 300,000 ms.
Following the work by Casagrande et al. (1997), this study
defines Asleep as when the inter-tap interval (ITI) is greaterthan 300,000 ms, Drowsiness as when the ITI is between
50,000 ms and 300,000 ms, and Awake as when the ITI is
shorter than 50,000 ms. Figure 8 shows the ITI time course
and the corresponding brain states (Awake, Drowsy, andAsleep) defined behaviorally.
The left chart of Fig. 8 shows the sleep onset latency data,
the middle shows the maximum nap duration, and the rightshows the drowsiness onset latency data for all 20 participants. The amount of participants that did actually fall asleep
for each condition differs. For the “no device” condition, a

total of only 8 participants managed to fall asleep. For the“sleeping mask” condition, a total of 13 participants successfully fell asleep. Finally, for both the “VR study room”
and “preferred VR” conditions, a total of 17 participants
managed to fall asleep. Looking at the results for sleep onset
latency, the average time for falling asleep in the VR study
room and preferred VR conditions are 1,089,950 ms (18.16
min) and 1,222,700 ms (20.38 min), respectively. These
results are both faster than the time it took on average forparticipants to fall asleep for the conditions with no devices
and wearing a sleeping mask, which took 2,553,550 ms(42.56 min) and 1,995,100 ms (33.25 min), respectively. We
performed a Shapiro–Wilk test to determine the normalityof our results. For the sleep onset time, we found that theresults were significant ( p < 0.05), meaning that we cannot assume normality in the data. Next, we ran the Friedman test and found that there was a statistically significant

13

Virtual Reality

difference in sleep onset latency depending on the sleep
condition ( x2 (2) = 18.648, p < 0.001). Post-hoc analysis
with Wilcoxon signed-rank test was conducted with Bonferroni correction applied, resulting in a significance level ofp < 0.0125. The median (IQR) perceived effort levels for theconditions with no device, sleeping mask, VR view of study
room and preferred VR environment were 60 min (13.14
min to 60 min), 29.21 min (10.69 min to 60 min), 8.84 min(6.04 min to 20.51 min), and 10.98 min (6.16 min to 30.96
min), respectively. There were no significant differences
between the sleep conditions of no device with sleep mask(z = −1.642, p > 0.0125), the sleep mask with the VR study
room ( z = −2.025, p > 0.0125), or the VR study room withthe preferred VR environment (z = −1.16, p > 0.0125), eventhough there is an overall reduction in sleep onset latency
between no device with the sleep mask, and both VR conditions with the sleep mask. However, there was a statistical significant reduction in sleep onset latency between
no devices with VR study room ( z = −2.765, p < 0.0125)
and no device with preferred VR environment ( z = −2.504,
p < 0.0125).
From these results, we can observe that being in VR doessignificantly decrease sleep onset latency, allowing users tofall asleep faster as opposed to without any devices. A participant mentioned that since the HMD feels similar to asleeping mask, he/she feels inclined to keep his/her eyes
closed most of the time while having them on. Since therendered environment was exactly that of the physical space,
there was nothing to further distract them or to examine
further, such as a different environment which could potentially feel more exciting to view, at least for the first time.
However, there was a lack of significance when comparing
with the sleep mask, or with the VR baseline condition. A
trend is visible where on average, there is a drop of sleep
onset latency from no device, to the sleeping mask, andfinally the VR conditions. When worn, both the sleeping
mask and the use of VR effectively blocks any form of visual stimulus from the environment. However, VR includes
a display that instead shows a different environment which
may increase distraction, whereas the sleeping mask is simple but lightweight and comfortable, stated by a participant
who preferred the sleeping mask. These factors contributeto the lack of significance between sleeping mask and VR.
On the positive side, unlike the use of a sleeping mask thatmerely covers the eyes (Liang et al. 2015), VR is instead a
combination of audio and visual stimulus, with content specifically designed to induce a sense of calmness, not unlike
meditation or yoga (Gromala et al. 2015). Three participants
did mention that the environment helped them imagine thatthey were there, which leads to better imagery distraction
and sleep onset latency reduction. One participant claimed
that he/she has been having a better sleep after participating in the experiment in preferred VR condition. Maybe the13

inclusion of a display for distraction and instilling calmness is useful for sleeping. Yet, further studies are needed to
explore the effects of the displayed environment, such as byexperimenting with lighter HMD, more content options, andsofter padding. Our initial assumption was that participants
who chose the “own room” option for the VR environment
would be able to fall asleep faster compared to those whochose other environments, according to Fig. 4. However, ourresults do not indicate any significance between the “ownroom” and preferred VR” condition. This could be due toseveral factors, such as the VR environments being captured
only with a 360◦ camera, thus lacking depth and realism
to be perceived as the actual environment completely. Secondly, the videos used for all the environment were relatively
pixelated and low resolution, further effecting the perceived
realism negatively.
Next, we look at the nap duration time, where a longer
duration indicates a longer rest time. On average, the participants slept for a maximum of 989,450 ms (16.5 min) and
779,238 ms (13 min) for the VR in a study room and preferred VR environment, respectively. Both these durations
are higher than the conditions with no devices and sleeping
mask, with a nap duration of 583,364 ms (9.72 min) and
778,734 ms (12.98 min), respectively. However, there was no
statistically significant difference in nap duration dependingon the sleep condition ( x2 (2) = 4.296, p > 0.05).
Finally, we repeated the procedure for the drowsinessonset latency. On average, the participants show signs offeeling drowsy for the preferred VR environment after
296,800 ms (4.95 min). This is relatively faster than theother three conditions. Participants began to show signs ofdrowsiness after 490,000 ms (8.17 min), 731,000 ms (12.18
min), and 695,000 ms (11.58 min) for the no device, sleep
mask, and VR of study room, respectively. However, there
was no statistically significant difference in drowsiness onset
latency depending on the sleep condition ( x2 (2) = 6.075,
p > 0.05).
P2 and P15 mentioned that they were able to sleep longer
largely due to the presence of the background noise fromthe VR environments, since the visuals itself could not haveany effect once they have fallen asleep. However, the VRcontent did help in making them feel drowsy faster, eventually leading to faster sleep onset. P1, who has no prior
VR experience, did find the HMD to be heavy and made it
harder to move around during sleep. However, he/she didmention that HMDs feel heavy in general, and not specificto our use case. When putting on the HMD, P5, P7, and P9
mentioned that the placement of the EOG sensors causes aslight discomfort to the face and chose to loosen the strap a
bit more. Additionally, P5 mentioned that he/she could only
sleep properly while in pajamas. Only P14 mentioned thathe/she dislikes any form of noise as well as wearing any kindof device while sleeping.

Virtual Reality

Fig. 9  Envisioned prototype with (1) adjustable EOG sensors andface cushion, (2) an outward display showing the sleep mode andremaining period, (3) integrated headphones, (4) the home screen

showing several napping durations, and (5) the selection screen presenting 8 choices of VR environments

6.2 Sleep onset classification

Table 1  Confusion Matrix

In this section, we discuss further on the use of commercial EOG sensors to automatically detect sleep onset as our
second main contribution. This solution allows an HMD
integrated with it to be able to know when the user falls
asleep depending on the eye movement, thus disabling thedisplayed content at the right time. This is an important issue
to consider for a wearable sleep-assist tool. Even though ourparticipants did not complain about this, it was previously
found that audio or visual feedback still received after the
onset of sleep increases the probability of arousal (Zhang
et al. 2014). Yet, we look at this as the secondary contribution of the paper and an initial step towards conceptualizing NapWell as a working prototype. Even though the FTTprocedure can be used to detect sleep onset, it is not ideal
when suggesting NapWell as a standalone device. A prediction model would be able to achieve this automatically forthe user.
We use the previously collected EOG data, as well as
the data from the FTT application to label it for supervised
learning. The raw data from the JINS MEME glasses are
3-axis accelerometer, 3-axis gyroscope, and EOG sensors
at the left and right nose pad, for a total of 8 features. We
then generated a sliding window with a period of 10 seconds which are non-overlapping for each of these features.
Next, we calculated the mean, standard deviation, and variance for each of these features within that time frame, fora total of 24 features. To reduce the dimension of the data,
we ran the principal component analysis (PCA) procedure
until only 6 features remain. PCA essentially creates new

True Awake
True Asleep

Predicted Awake

Predicted Asleep

5967521461960110698

Table 2  Results of the main classification metrics

Awake
AsleepAverage/Total

Precision

Recall

f1-score

support

0.80
1.00
0.86

1.00
0.43
0.83

0.89
0.60
0.80

596752
256894853646transformed dimensions based on the original data and thedesired amount of output dimensions11. This includes themost dominant trend from the data samples (Masai et al.
2018). After that, we filtered the features using the moving
average filter. Finally, we fed the features with a train-test
split of 80–20 into a logistic regression classification algorithm. The splitting was performed prior to PCA to avoid
any biases from the test data, where we split depending on
participant (data from 4 random participants out of 20 wereselected as test data).
From the test results of our model, we obtained an
accuracy of 82.88% with the full confusion matrix shownin Table 1 and details shown in Table 2. The errors in the11

https://​scikit-​learn.​org/​stable/​modul​es/​gener​ated/​sklea​r n.​decom​
posit​ion.​PCA.​html#​sklea​rn.​decom​posit​ion.​PCA.​fit_​trans​form.

13

Virtual Reality

classification are mainly due to false positives of the detection. There are no false negatives. To validate the accuracy,
we ran the model through K-fold cross-validation with K = 6
and obtained an average accuracy of 70.03%. Some of thecontributing factors are due to the tightness of the worn
HMD. To obtain better data with less noise, the HMD should
ideally be as tight as possible. The misclassifications could
possibly be due to if the HMD was loose on the head of theuser for comfort as mentioned by some of the participants(the EOG not having full contact). To improve this in future
works, we should make a signal check before starting thenap, to make sure the signal to noise ratio is better. Nevertheless, we believe this model can be further improved infuture iterations.

6.3 Conceptual prototype
Based on the obtained results, this section conceptualizes
it into a prototype. The key takeaway from the results are:1. VR environments do improve sleep onset time. The“own room” VR option was the most popular choice.
2. embedded EOG sensors can detect sleep onset withrelatively high accuracy, though one needs to adjust the
EOG sensors
From these finding, we illustrated a HMD prototype shown
on Fig. 9 that is meant to maximize comfort with soft faceand ear paddings. The EOG sensor is sponge or textile-basedso that it can be pressed firmly on the face without discomfort (Krishnan et al. 2018) and its position is highly adjustable to accommodate different faces. We also conceptualize an outward display showing remaining nap time, which
could be useful at office environments. The user will first bepresented with an interface that shows a room environment,with 3 separate options for different nap duration. After
selecting an option, the next page shows up to 8 choicesof virtual environments to choose from. The “own room”
option will allow users to upload their own 360◦ video of
their room if they have one. The system then automaticallydetects sleep onset based on a generalized pre-trained model
suggested in section 6.2, without the need to perform theFTT procedure.

7 Limitations and future works
To further improve this study, we suggest experimenting
with different kinds of HMD, as well as a higher resolution
for the video. Even though Google Daydream itself is light(around 450 grams), the use of a mobile phone for VR display makes the content more pixelated. The EOG electrodes
should also be softer, such as a sponge or textile-based

13

material to be more comfortable (Krishnan et al. 2018). The
use of a VR HMD for sleep undeniably adds bulk and weightto the user’s face, though unlike a sleep mask, it does notpress against the eyes directly. A custom HMD that mounts
on the head and can be flipped down with minimal face contact, such as the Lynx12 headset design, could help reduceweight and increase comfort. In the next phase, we plan toperform the user study for proper night sleep in a sleep lab.
We also plan to use PSG equipment to obtain more accurate
physiological signals that can ascertain the sleep quality ofthe participants.
When recruiting the participants, we ask that each ofthem have a gap of 3 h since their last meal, as proposedby Okamura et al. (2016). However, there are also studies
that showed that sleep might be best induced when we are
well fed and with low cognitive load (Wells et al. 1997). We
also did not monitor what the participants did within that 3
h gaps; for example, some participants could have went fora jog, thus being more tired and susceptible to being sleepy.
Future studies may include methods to measure a user’s
blood sugar and cognitive load level prior to the experiment.
As mentioned previously, the independent variable in the
proposed experiment is not the different sensory feedback
modality, but rather regarding the use of different device
and its affect towards sleep onset. For our future works, wewould like to extend this by investigating how each sensory
feedback as well as the combination of feedbacks can affectsleep onset.
On the VR experience side, our study was only limited
to five environments which we established from the pilot
survey. Yet, each individual has their own specific preference regarding the VR environment they choose as their
sleep assist tool. Therefore, future iterations of NapWell will
look into (1) a VR environment creator that allows users toeasily build and deploy their preferred environment, and (2)a method to easily share user-created content so that usersof NapWell who share the same preferred environment can
upload their creation to our dedicated servers to share withother users. Furthermore, we consider this work as a preliminary study towards using VR imagery to promote sleep.
Future works can look into how each environment can befurther manipulated based on the EOG data, such as adding
dynamic relaxing imagery or other sleep-inducing patterns
depending on the eye’s saccades and blink rate.

8 Conclusion
We propose NapWell, an EOG-based sleep assistant thatis integrated into VR HMDs to reduce sleep onset latency.
We first conducted and presented a pilot survey from 157
12

https://​lynx-r.​com/.

Virtual Reality

participants to establish the sleep habits and preferred nap
environments of the general public. We then presented auser study that explores the effects of VR on sleep using
both physiological and beahavioral measures. We found that
VR can significantly reduce sleep onset latency compared tousing no devices while also providing overall longer sleep
time and lesser drowsiness onset latency. Finally, we conceptualized a low-cost VR prototype with integrated consumer-ready EOG sensors. Our machine learning model
could detect sleep onset win a cross-validated accuracy of
70.03%. The conducted experiment and conceptualized prototype serve as a first step towards the broader concept of
introducing VR as a sleep-assist tool. We showed that VR
was able to influence sleep by reducing sleep onset latency
during naps by augmenting the perceived environment of
the user.
Acknowledgements This work was partly supported by Institute ofInformation & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (No. , WISE ARUI/UX Platform Development for Smartglasses), as well as the Japan
Society for the Promotion of Science (JSPS) Kakenhi Grant Number
18H03278.

References
Adib F, Mao H, Kabelac Z, Katabi D, Miller RC (2015) Smart homesthat monitor breathing and heart rate. pp 837–846
Akert K, Koella W, Hess R Jr (1951) Sleep produced by electrical stimulation of the thalamus. Am J Physiol Legacy Content
168(1):260–267
Amores J, Benavides X, Maes P (2016) Psychicvr: increasing mindfulness by using virtual reality and brain computer interfaces. In:
Proceedings of the 2016 CHI Conference Extended Abstracts on
Human Factors in Computing Systems, ACM, pp 2
Bait M (2019) Sleep survey. [online] google docs 28, https://​docs.​
google.​com/​forms/d/​1Nso7​25KJg​YKm1K​jODso​snBh2​FBuzA​
IejCg​a4m4v​Nodk/​viewa​nalyt​ics
Bernardino C, Ferreira HA, Chambel T (2016) Towards media for
wellbeing. In: Proceedings of the ACM international conference
on interactive experiences for TV and Online Video, ACM, pp
171–177
Berry RB, Brooks R, Gamaldo CE, Harding SM, Marcus CL, Vaughn
BV et al (2012) The aasm manual for the scoring of sleep andassociated events. Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine,p 176
Blake H, Gerard RW, Kleitman N (1939) Factors influencing brain
potentials during sleep
Borazio M, Van Laerhoven K (2012) Combining wearable and environmental sensing into an unobtrusive tool for long-term sleep
studies. In: Proceedings of the 2nd ACM SIGHIT international
health informatics symposium, ACM, pp 71–80
Boukadoum A, Ktonas P (1986) Eog-based recording and automateddetection of sleep rapid eye movements: a critical review, andsome recommendations. Psychophysiology 23(5):598–611
Brown FC, Buboltz WC Jr, Soper B (2002) Relationship of sleep
hygiene awareness, sleep hygiene practices, and sleep quality inuniversity students. Behav Med 28(1):33–38

Casagrande M, De Gennaro L, Violani C, Braibanti P, Bertini M (1997)
A finger-tapping task and a reaction time task as behavioral measures of the transition from wakefulness to sleep: Which task interferes less with the sleep onset process? Sleep 20(4):301–312
Choe EK, Lee B, Kay M, Pratt W, Kientz JA (2015) Sleeptight: lowburden, self-monitoring technology for capturing and reflecting on
sleep behaviors. In: Proceedings of the 2015 ACM international
joint conference on pervasive and ubiquitous computing, ACM,
pp 121–132
Connelly L (2004) Behavioural measurements of sleep onset: A comparison of two devices. PhD thesis
Daneshmandi M, Neiseh F, SadeghiShermeh M, Ebadi A (2012) Effectof eye mask on sleep quality in patients with acute coronary syndrome. J Caring Sci 1(3):135
Davis H, Davis PA, Loomis AL, Harvey EN, Hobart G (1939) Electrical reactions of the human brain to auditory stimulation during
sleep. J Neurophysiol 2(6):500–514
Demoule A, Carreira S, Lavault S, Pallanca O, Morawiec E, Mayaux
J, Arnulf I, Similowski T (2017) Impact of earplugs and eye mask
on sleep in critically ill patients: a prospective randomized study.
Critical Care 21(1):1–9
Ehleringer EH, Kim SJ (2013) The wearable lullaby: Improving sleep
quality of caregivers of dementia patients. In: CHI ’13 Extended
Abstracts on Human Factors in Computing Systems, ACM, New
York, NY, USA, CHI EA ’13, pp 409–414, https://​doi.​org/​10.​1145/​24683​56.​24684​29, http://​doi.​acm.​org/​10.​1145/​24683​56.​
24684​29
Felsten G (2009) Where to take a study break on the college campus:
an attention restoration theory perspective. J Environ Psychol29(1):160–167
Ferrer-Garcia M, Gutiérrez-Maldonado J, Riva G (2013) Virtual reality based treatments in eating disorders and obesity: a review. J
Contemp Psychother 43(4):207–221
Green A, Cohen-Zion M, Haim A, Dagan Y (2017) Evening light
exposure to computer screens disrupts human sleep, biologicalrhythms, and attention abilities. Chronobiol Int 34(7):855–865
Gromala D, Tong X, Choo A, Karamnejad M, Shaw CD (2015) The
virtual meditative walk: Virtual reality therapy for chronic pain
management. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM, New York,NY, USA, CHI ’15, pp 521–524, https://d​ oi.o​ rg/1​ 0.1​ 145/2​ 70212​ 3.​27023​44, http://​doi.​acm.​org/​10.​1145/​27021​23.​27023​44
Gupta K, Hajika R, Pai YS, Duenser A, Lochner M, Billinghurst M(2020) Measuring human trust in a virtual assistant using physiological sensing in virtual reality. In: 2020 IEEE Conference on
Virtual Reality and 3D User Interfaces (VR), IEEE, pp 756–765
Harvey AG, Payne S (2002) The management of unwanted pre-sleep
thoughts in insomnia: distraction with imagery versus general
distraction. Behav Res Ther 40(3):267–277
Rf Hu, Jiang Xy, Zeng Ym, Chen Xy, Zhang Yh (2010) Effects of
earplugs and eye masks on nocturnal sleep, melatonin and cortisol in a simulated intensive care unit environment. Critical Care
14(2):1–9
Ibáñez V, Silva J, Cauli O (2018) A survey on sleep assessment methods. PeerJ 6:e4849
Ishimaru S, Kunze K, Uema Y, Kise K, Inami M, Tanaka K (2014)
Smarter eyewear: using commercial eog glasses for activity recognition. In: Proceedings of the 2014 ACM International Joint
Conference on Pervasive and Ubiquitous Computing: AdjunctPublication, pp 239–242
Jerald J (2015) The VR book: Human-centered design for virtual reality. Morgan & Claypool
Jirakittayakorn N, Wongsawat Y (2018) A novel insight of effects ofa 3-hz binaural beat on sleep stages during sleep. Front Human
Neurosci 12:387

13

Virtual Reality
Johns MW (1991) A new method for measuring daytime sleepiness:the epworth sleepiness scale. Sleep 14(6):540–545
Kanady JC, Drummond SP, Mednick SC (2011) Actigraphic assessment of a polysomnographic-recorded nap: a validation study. J
Sleep Res 20(1pt2):214–222
Katsumata K, Noda Y, Isokawa N, Katayama S, Okoshi T, Nakazawa J (2019) Sleepthermo: The affect of in-cloth monitored
body temperature change during sleep on human well-being.
In: Adjunct Proceedings of the 2019 ACM International Joint
Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, ACM, New York, NY, USA, UbiComp/ISWC’19 Adjunct, pp 1174–1177, https://​doi.​org/​10.​1145/​33411​62.​
33470​79, http://​doi.​acm.​org/​10.​1145/​33411​62.​33470​79
Kitson A, Schiphorst T, Riecke BE (2018) Are you dreaming?: A
phenomenological study on understanding lucid dreams as atool for introspection in virtual reality. In: Proceedings of the2018 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI ’18, pp 343:1–343:12,
https://​doi.​org/​10.​1145/​31735​74.​31739​17, http://​doi.​acm.​org/​
10.​1145/​31735​74.​31739​17
Kosunen I, Salminen M, Järvelä S, Ruonala A, Ravaja N, Jacucci G(2016) Relaworld: neuroadaptive and immersive virtual reality meditation system. In: Proceedings of the 21st international
conference on intelligent user interfaces, ACM, pp 208–217
Krishnan A, Kumar R, Venkatesh P, Kelly S, Grover P (2018) Lowcost carbon fiber-based conductive silicone sponge eeg electrodes. In: 2018 40th annual international conference of theIEEE engineering in medicine and biology society (EMBC),
IEEE, pp 1287–1290
Lack L, Mair A (1995) The relationship between eeg and a behavioral measure of sleep onset. Sleep Res 24:218
Lee M, Song CB, Shin GH, Lee SW (2019) Possible effect of binaural beat combined with autonomous sensory meridian response
for inducing sleep. Front Human Neurosci 13:425
Liang SF, Kuo CE, Lee YC, Lin WC, Liu YC, Chen PY, Cherng
FY, Shaw FZ (2015) Development of an EOG-based automatic sleep-monitoring eye mask. IEEE Trans Instrum Meas
64(11):2977–2985
Liang Z, Ploderer B (2016) Sleep tracking in the real world: A qualitative study into barriers for improving sleep. In: Proceedingsof the 28th Australian Conference on Computer-Human Interaction, ACM, New York, NY, USA, OzCHI ’16, pp 537–541,
https://​doi.​org/​10.​1145/​30109​15.​30109​88, http://​doi.​acm.​org/​
10.​1145/​30109​15.​30109​88
Liberson W, Liberson CW (1966) Eeg records, reaction times, eyemovements, respiration, and mental content during drowsiness. In: Recent advances in biological psychiatry, Springer,
pp 295–302
Masai K, Sugiura Y, Sugimoto M (2018) Facerubbing: Input technique
by rubbing face using optical sensors on smart eyewear for facial
expression recognition. In: Proceedings of the 9th Augmented
Human International Conference, pp 1–5
Matsui S, Terada T, Tsukamoto M (2017) Smart eye mask: eye-mask
shaped sleep monitoring device. In: Proceedings of the 2017 ACM
International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, pp 265–268
Mednick SC, Nakayama K, Cantero JL, Atienza M, Levin AA, Pathak
N, Stickgold R (2002) The restorative effect of naps on perceptual
deterioration. Nature Neurosci 5(7):677
Metsis V, Kosmopoulos D, Athitsos V, Makedon F (2014) Non-invasive analysis of sleep patterns via multimodal sensor input. Personal Ubiquitous Comput 18(1):19–26

13

Milner CE, Cote KA (2009) Benefits of napping in healthy adults:
impact of nap length, time of day, age, and experience with napping. J Sleep Res 18(2):272–281
Min JK, Doryab A, Wiese J, Amini S, Zimmerman J, Hong JI (2014)
Toss’n’turn: smartphone as sleep and sleep quality detector. In:
Proceedings of the SIGCHI conference on human factors in computing systems, ACM, pp 477–486
Morales-Cobas G, Ferreira MI, Velluti RA (1995) Firing of inferior
colliculus neurons in response to low-frequency sound stimulation
during sleep and waking. J Sleep Res 4(4):242–251
Muzet A (2007) Environmental noise, sleep and health. Sleep Med
Rev 11(2):135–142
Nicholson A, Stone BM (1987) Influence of back angle on the qualityof sleep in seats. Ergonomics 30(7):1033–1041
Ogilvie RD (2001) The process of falling asleep. Sleep Med Rev
5(3):247–270
Ogilvie RD, Wilkinson RT (1984) The detection of sleep onset:
behavioral and physiological convergence. Psychophysiology
21(5):510–520
Ogilvie RD, Wilkinson RT (1988) Behavioral versus eeg-based monitoring of all-night sleep/wake patterns. Sleep 11(2):139–155
Ogilvie RD, Wilkinson RT, Allison S (1989) The detection of sleep
onset: behavioral, physiological, and subjective convergence.
Sleep 12(5):458–474
Okamura T, Isoyama N, Lopez G (2016) A method to detect accurately
falling asleep and awakening time. In: Adjunct Proceedings of the
13th international conference on mobile and ubiquitous systems:
computing networking and services, ACM, pp 47–52
Pedemonte M, Testa M, Díaz M, Suarez-Bagnasco D (2014) The
impact of sound on electroencephalographic waves during sleepin patients suffering from tinnitus. Sleep Sci 7(3):143–151
Peña JL, Pérez-Perera L, Bouvier M, Velluti RA (1999) Sleep andwakefulness modulation of the neuronal firing in the auditory
cortex of the guinea pig. Brain Res 816(2):463–470
Powers MB, Emmelkamp PM (2008) Virtual reality exposure therapy for anxiety disorders: A meta-analysis. J Anxiety Disord
22(3):561–569
Rahman MM, Bhuiyan MIH, Hassan AR (2018) Sleep stage classification using single-channel EOG. Comput Biol Med 102:211–220
Roo JS, Gervais R, Frey J, Hachet M (2017) Inner garden: Connecting
inner states to a mixed reality sandbox for mindfulness. In: Proceedings of the 2017 CHI conference on human factors in computing systems, ACM, pp 1459–1470
Rothbaum BO, Hodges L, Alarcon R, Ready D, Shahar F, Graap K,
Pair J, Hebert P, Gotz D, Wills B et al (1999) Virtual reality
exposure therapy for ptsd vietnam veterans: a case study. J TraumStress Official Pub Int Soc Traum Stress Stud 12(2):263–271
Rothkrantz L (2016) Automatic detection system of micro sleeps ofcar drivers based on eeg analysis. In: Proceedings of the 17th
international conference on computer systems and technologies
2016, ACM, pp 214–221
Sanchez-Vives MV, Slater M (2005) From presence to consciousness
through virtual reality. Nature Rev Neurosci 6(4):332
Schmidt A, Shirazi AS, Van Laerhoven K (2012) Are you in bed withtechnology? IEEE Pervasive Comput 11(4):4–7
Scott H, Lack L (2017) The revival of active behavioural devices formeasuring sleep latency. SM J Sleep Disord 3(3):1015
Scott H, Lack L, Lovato N (2018) A pilot study of a novel smartphone application for the estimation of sleep onset. J Sleep Res
27(1):90–97
Semertzidis NA, Sargeant B, Dwyer J, Mueller FF, Zambetta F (2019)
Towards understanding the design of positive pre-sleep througha neurofeedback artistic experience. In: Proceedings of the 2019
CHI Conference on Human Factors in Computing Systems, ACM,
New York, NY, USA, CHI ’19, pp 574:1–574:14, https://​doi.​org/​

Virtual Reality
10.​1145/​32906​05.​33008​04, http://d​ oi.​acm.​org/​10.​1145/​32906​05.​
33008​04
Shaw C, Gromala D, Song M (2011) The meditation chamber: towards
self-modulation. In: Metaplasticity in virtual worlds: Aesthetics
and semantic concepts, IGI Global, pp 121–133
Shirazi AS, Clawson J, Hassanpour Y, Tourian MJ, Schmidt A, Chi EH,
Borazio M, Van Laerhoven K (2013) Already up? Using mobile
phones to track & share sleep behavior. Int J Human Comput Stud71(9):878–888
Uema Y, Inoue K (2017) Jins meme algorithm for estimation and tracking of concentration of users. In: Proceedings of the 2017 ACM
International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, ACM, New York, NY, USA, UbiComp ’17, pp 297–300, https://d​ oi.​org/​10.​1145/​31230​24.​31231​
89, http://​doi.​acm.​org/​10.​1145/​31230​24.​31231​89
Wells AS, Read N, Uvnas-Moberg K, Alster P (1997) Influences of
fat and carbohydrate on postprandial sleepiness, mood, and hormones. Physiol Behav 61(5):679–686

Yazdannik AR, Zareie A, Hasanpour M, Kashefi P (2014) The effectof earplugs and eye mask on patients perceived sleep quality inintensive care unit. Iranian J Nursing Midwifery Res 19(6):673
Zhang Z, Guan C, Chan TE, Yu J, Ng AK, Zhang H, Kwoh CK (2014)
Automatic sleep onset detection using single eeg sensor. In: 2014
36th Annual International Conference of the IEEE Engineeringin Medicine and Biology Society, pp 2265–2268, https://​doi.​org/​
10.​1109/​EMBC.​2014.​69440​71
Zhao N, Azaria A, Paradiso JA (2017) Mediated atmospheres: A multimodal mediated work environment. Proc ACM Interact Mob
Wearable Ubiquitous Technol 1(2):31:1–31:23, https://d​ oi.o​ rg/1​ 0.​1145/​30900​96, http://​doi.​acm.​org/​10.​1145/​30900​96
Publisher's Note Springer Nature remains neutral with regard tojurisdictional claims in published maps and institutional affiliations.

13