ThermoQuest - A Wearable Head Mounted Display to Augment
Realities with Thermal Feedback
Kirill Ragozin

Xiaru Meng

Roshan Lalintha Peiris

Keio University
Japankirill@kmd.keio.ac.jp

Keio UniversityJapanmengxiaru@kmd.keio.ac.jp

Rochester Institute of Technology
United Statesroshan.peiris@rit.edu

Katrin Wolf

George Chernyshov

Kai Kunze

Beuth University of Applied Sciences
Germanykatrin.wolf@acm.org

Keio University
Japan
chernyshov@kmd.keio.ac.jp

Keio UniversityJapan
kai@kmd.keio.ac.jp

Figure 1: Prototype of the ThermoQuest system worn by a user experiencing cold and hot immersive virtual environments.

ABSTRACTKEYWORDS

We present ThermoQuest, a self-contained wearable head-mounted
display system for enhancing Virtual Reality experiences with temperature feedback. It’s constructed with commodity hardware elements, featuring 6 Peltier elements on the rim of the headset touching the users face. We explain the design and implementation of anaffordable wearable thermal VR prototype build with commodity
hardware. In a user study with 15 participants, we show evidence
for a significant difference in reported presence in VR between
thermal VR and control conditions (p > 0.017) in a counterbalanced
experimental setup. We end with a discussion and use cases of thepresented VR prototype and similar systems.

Multimodal Interfaces, Thermal Feedback, Virtual Reality
ACM Reference Format:
Kirill Ragozin, Xiaru Meng, Roshan Lalintha Peiris, Katrin Wolf, George
Chernyshov, and Kai Kunze. 2021. ThermoQuest - A Wearable Head Mounted
Display to Augment Realities with Thermal Feedback. In 20th International Conference on Mobile and Ubiquitous Multimedia (MUM 2021), December 5-8, 2021, Leuven, Belgium. ACM, New York, NY, USA, 5 pages.
https://doi.org/10.1145/3490632.3490649

1

INTRODUCTION

As governments and healthcare experts around the world call forchanging lifestyle in response to the Covid-19 pandemic, the development and usage of remote communication and touchless technologies are rapidly becoming an essential part of the “new normal”.
At the same time, the absence of touch and physical contact highlights their critical importance in human life from school to hospitalto care facilities. The struggle of deafblind individuals who rely ontouch alone for communication especially sheds light on how critical this sense is in human communication, since they are unable totake advantage of the conventional auditory and visual telepresencetechnologies even as their usage is promoted under stay-at-homemeasures.
In our research, we are working on technologies to use feedback
to the human somatosensory subsystem (i.e. pressure, temperature,
texture, pain and body position) to encode, model and deliver information from computing systems in a minimally obtrusive way. In
this paper we focus on thermal perception, equipping a commodity,

CCS CONCEPTS• Human-centered computing → Virtual reality; Interaction
devices.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
MUM 2021, December 5-8, 2021, Leuven, Belgium© 2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8643-2/21/05. . . $15.00
https://doi.org/10.1145/3490632.3490649

62

MUM 2021, December 5-8, 2021, Leuven, Belgium

Kirill Ragozin, Xiaru Meng, Roshan Lalintha Peiris, Katrin Wolf, George Chernyshov, and Kai Kunze

Figure 2: Prototype of the virtual reality headset with 6 embedded Peltier elements (left) showing the thermal preview of hot
actuation (middle) and cold actuation (right).
wearable VR headset with affordable Peltier elements to provide
thermal feedback.
Our contributions are as follows:

Close to our work, Peiris et al. present ThermoVR, a HMD connected to a computer, using some five thermal feedback prototype
modules to provide hot/cold feedback on the user’s face [10]. Chenet al. also present some thermal design patterns for the same typeof thermal feedback prototypes [1]. Yet, these systems use thermal
feedback prototypes from a specific producer. These elements are
not readily available for other researchers and DIY makers. Compared to this our prototype uses off-the-shelf products and is easy toassemble for researchers and enthusiasts. We use the works as basis
for our feedback design, combining the 6 actuators in 3 channels(left, front, right).
To the best of our knowledge, ThermoQuest is the first fully
wearable HMD with thermal feedback using affordable commodity
elements. We are also unaware of any research evaluating thermal
feedback in standalone VR headsets.

• We present the design and implementation of an affordable,
wearable thermal VR HMD prototype built with commodity
hardware.
• We show evidence for a significant difference in reported
presence in VR between thermal VR and controll conditions(p > 0.017) in a counterbalanced experimental setup with 15
participants.
• We end with a discussion and use cases of the presented VR
prototype and similar systems.

2

RELATED WORK

Somatosensory feedback over pressure, tactile and thermal actuators is often used as an additional output channel in human computer interaction, from simulating different materials, over thermal
notifications, navigation scenarios to increase presence in virtual
reality environments [2, 5, 8, 14–18, 20]. For brevity, we will focus
on thermal feedback related to human computer interaction andthe enhancement of mixed/virtual reality applications.
Nakashige et al. presented an early work augmenting a computer mouse to enhance digital media with temperature information [8]. There are several continuations of this work, enhancing
game controllers and mobile phones with thermal interactions,adding situational awareness or simple hot/cold notifications toapplications [7, 19]. Also form factors evolved from interaction
devices and peripherals to wearables. Peiris et al. present a braceletwith thermal feedback, evaluating perception thresholds and application cases [9]. Moving away from notifications, immersion
and situational awareness, Tewell et al. explore the use of thermal
feedback also for navigation tasks [16].
Presence and immersion evaluations in virtual environments are
subjective and usually recorded over questionnaires. Although there
is an ongoing discussion about validity and effectiveness, thermal(and other somatosensory) feedback has been shown to increase
immersion [3, 4, 6, 11, 12, 14, 15, 17, 18]. Ranashinghe et al. use
additional hardware setups, focusing on thermal actuation on theneck, throat and behind the ear to increase immersion simulating
wind and temperature [13]. These works often apply extensive
hardware setups in larger lab environments.

3

IMPLEMENTATION

The device is designed to fit the standard Oculus Quest HMD. Peltierelements were mounted into the removable foam face interface ofthe HMD. In total we used 6 elements, 15x15mm in size, rated at
2A 3.7V max. 6 elements are split into 3 channels: front (2 on theforehead), left and right (1 on temple and 1 on cheek). Each of 3groups is controlled with an Allegro A3909 H-bridge chip, allowing
us to pass current through the element in either direction, providing
both cold and hot sensations on the same element. Each channel ispowered through a PTC fuse for safety reasons.
The feedback module is controlled by an esp32 module, acting asa Wifi Access Point (AP) with Oculus connecting to this AP directly
and communicating with the thermal device over WiFi. The device
is powered from a 3.7v lithium-polymer battery with another PTC
fuse on the power line.
The thermal sensations were provided in repetitive pulses. For
hot sensations the element was powered on for 4 seconds followed
by a 6 second cool down period. For cold sensations the elements
were powered on for 5 seconds and powered off for another 5seconds. Since we did not use any cooling mechanism on the peltierelements we could not power them continuously in order to avoid
overheating.

4

METHODOLOGY

A mobile VR headset prototype capable of providing thermal feedback in short repeating pulses was created for the experiment.

63

ThermoQuest - A Wearable Head Mounted Display to Augment Realities with Thermal Feedback

MUM 2021, December 5-8, 2021, Leuven, Belgium

Figure 3: Pictures from inside the VR environment showing thermally charged objects that generate cold and hot sensations.
Proximity to the waterfall triggers cold feedback, while the fire produces heat.
Additionally, an immersive virtual environment was created with
Unreal Engine to facilitate the thermal feedback interactions. The
experiment featured three controlled conditions. To account forthe possible ordering effects, the trials between participants followed a counterbalanced order. With this study we have captured
quantitative data for the self-estimated presence in virtual environment using Witmer & Singer Presence Questionnaire (version 3.0).
Qualitative feedback regarding the user experience was gathered aswell. The experiment was conducted in a closed room with ambient
temperature set at a constant level for all the participants. The
procedure of the study was as follows:from 1 to 7. Additionally, they were provided a secondary questionnaire where they were asked to fill in a free form the answers to
the following questions:• What supported your feeling of immersion in the VR scene?
• What reduced your feeling of immersion in the VR scene?
• What did you like about the presented feedback?
• What did you NOT like about the presented feedback?

1) Informed Consent. The participant was welcomed to the study
location and and was informed about the details of the procedureof the study. Then, we explained the consent form and the dataprotection policy which the participants were asked to read andsign.
2) Personal Data Gathering. The participant then was asked to indicate their age and sex on the digital spreadsheet.
3) Experiencing Virtual Environment. The person conducting theexperiment would then select an appropriate trial category for theparticipant and hand over the virtual reality HMD to the participant. The experience was set up to start after a 30 second delay,
providing just enough time for the user to put the headset on andget comfortable. Virtual environment featured six scenes spread
across three virtual worlds. Each scene has lasted 30 seconds after
which the users virtual avatar would be teleported to the new view
location. Thermal feedback was provided passively as an ambientenvironmental cue and the user was not able to affect it in any way.
Two virtual scenes were neutral and featured no thermal feedback,two scenes contained objects as temperature sources (waterfall andfireplace), and two were thematically charged with temperature(winter environment and desert environment). The user was only
equipped with the VR headset and could interact with the environment though rotation and small motion based on the physicalposition of the headset. No other interactions were provided.

Figure 4: User wearing the prototype of mobile headmounted display with thermal feedback.

5) Trials with Different Conditions. The experiment was then repeated for the remaining two conditions. For example, if the firsttrial was without any feedback (A), the consecutive trial would
contain normal thermal feedback (B) and then reversed thermal
feedback (C), repeating steps (3) and (4) for each one.
5) Semi-structured Interview. After all three trials were complete,
the participant was asked to elaborate on their answers to thequestions in section (4) and to share their general opinion and comments regarding the overall experience, if they had anything to add.

4) Filling the Questionnaire. After going through the virtual experience, the participant had filled in the 32 questions of Witmer &Singer Presence Questionnaire (version 3.0), rated on a Likert scale64

MUM 2021, December 5-8, 2021, Leuven, Belgium

5

Kirill Ragozin, Xiaru Meng, Roshan Lalintha Peiris, Katrin Wolf, George Chernyshov, and Kai Kunze

RESULTS

from a visually small object (fireplace). One participant had said
that when feeling normal thermal feedback on the face, he felt moreengaged with the experience and wished that it would be possible
to interact with the environment to feel the feedback on different
parts of the body, specifically hands. When asked about the unexpected feedback, the majority of participants have described it as“weird” and “uncomfortable”. One had claimed that “mismatching
feedback doesn’t make sense and doesn’t add anything positive
to the experience”. Several participants have exhibited an elevatedsense of alertness and the desire to “figure out what is wrong” thatlasted for several seconds, before they could realize that it’s just an
opposite temperature setting and nothing else in the environment
had changed. One explained that such disconnect “feels like it’s acue for me to look at things”. This suggests a possible intentionaluse of the unexpected thermal feedback technique as a way of temporarily inducing the sense of unease in the experience. Several
users felt that pulsating feedback, chosen due to technical limitations, was limiting and wished to feed more “even” sensation; twohave expressed the desire to either have more touch-points or havethe feedback provided by a consistent mechanism, where individual thermal stimuli could not be distinguished. Many users havereported that after going through trials with both normal feedback
and without feedback they felt “relaxed and very peaceful”, confirming the assumption of potentially using such type of experience
for light recreation and alleviating stress. One user had expressed
the feeling of stronger feedback intensity when switching between
the polar opposite scenes from warm to cold, saying “when thesituation changed from the desert to snow, the difference of thetemperature is too big, so I feel even colder”. Two have described
the feedback as “too strong”, to the point where they felt eventhe thermal stimuli aligned with the virtual scene have distracted
them the experience. During the interview two participants havementioned that they actually have felt more immersed with theenvironment featuring unexpected feedback, either because it wasconsistent with their life experience (such as feeling the comforting
warmth in the winter), or because they felt compelled to create asensible explanation (such as comparing warm feedback from waterfall to the heat of the hot spring). Overwhelmingly, the versionof the experience featuring regular thermal feedback was referred
to as “the best”, with participant saying that “feedback matching
the scene has a great positive impact on the immersion”, “the soundand the heat changes supported my feeling” and “thermal feedback
is cool”.

A total of 15 participants (6 male, 9 female, age 21 to 30) have taken
part in the experiment.

Figure 5: Median presence scores with standard deviationsfor 3 experimental conditions: A - no feedback, B - expected
feedback, C - unexpected feedback.
Initial analysis of the data conducted with a Friedman test had
shown that there was a statistically significant difference in thereported sense of presence depending on the type of feedback presented to the users while in VR, x 2 = 19.600, p = 0.000055. Post
hoc analysis with Wilcoxon signed-rank tests was conducted witha Bonferroni correction applied, resulting in a significance level set
at p < 0.017. Median presence evaluation levels for the no feedback,normal expected feedback and reversed unexpected feedback were131 (104 to 174, std = 17.578), 149 (129 to 163, std = 18.019) and
119 (106 to 128, std = 20.786), respectively. There were significant
differences between all three trial category pairs: no feedback andnormal feedback trials (Z = −3.238, p = 0.001204), no feedback
and reversed thermal feedback (Z = −2.473, p = 0.013394), normal
feedback and reversed feedback (Z = −3.409, p = 0.000652).

6

DISCUSSION

Statistical analysis of the W&S questionnaire suggests that regular temperature feedback was effective in increasing the senseof presence in virtual environment, while reversed thermal had
feedback decreased that sense. Many users have reported that audio played a significant part in evaluating the experience. As such,one participant had said that “I can feel objects surrounding meby listening to the background sound”. Some have also mentioned
the aesthetic qualities of the environment and reported that thepolygonal nature of the chosen style had a minor negative impact
on their sense of immersion, due to being “not realistic”. Others,
however, enjoyed the look of the environment, claiming that “I likethe sound and visual design”. Some participants felt limited by thelack of interactions with the environment. As for the temperature,
the following reactions were observed. Two users have reported asense of inconsistency when perceiving “intense” warm feedback

7

CONCLUSIONS AND FUTURE WORKThe results of the experiment seem to support the original hypothesis that addition of the thermal feedback to the VR experiencewould elevate the sense of presence and immersion, while the useof an “unexpected” feedback would help achieve the opposite result.
Additionally, it suggests potential use of the “unexpected” feedback
as an intentional design technique aimed at making the user either
confused, uncomfortable and more alert for the upcoming eventin the scene. Lastly, it provides several directions for the future
hardware design, suggesting to change the stimuli distribution to amore continuous range and to integrate a cooling module, so that65

ThermoQuest - A Wearable Head Mounted Display to Augment Realities with Thermal Feedback

the feedback could be used with a faster onset and be consistent
without pulsations.

MUM 2021, December 5-8, 2021, Leuven, Belgium[19] Graham Wilson, Martin Halvey, Stephen A Brewster, and Stephen A Hughes.
2011. Some like it hot: thermal feedback for mobile devices. In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems. ., ., 2555–2564.
[20] Dennis Wolf, Michael Rietzler, Leo Hnatek, and Enrico Rukzio. 2019. Face/On:
Multi-Modal Haptic Feedback for Head-Mounted Displays in Virtual Reality.
IEEE transactions on visualization and computer graphics 25, 11 (2019), 3169–3177.

ACKNOWLEDGMENTS
This work is supported by the JSPS Kakenhi Kiban B Grant No.
18H03278.

REFERENCES[1] Zikun Chen, Roshan Lalintha Peiris, and Kouta Minamizawa. 2017. A thermal
pattern design for providing dynamic thermal feedback on the face with head
mounted displays. In Proceedings of the Eleventh International Conference on
Tangible, Embedded, and Embodied Interaction. ., ., 381–388.
[2] George Chernyshov, Kirill Ragozin, Cedric Caremel, and Kai Kunze. 2018. Hand
motion prediction for just-in-time thermo-haptic feedback. In Proceedings of the24th ACM Symposium on Virtual Reality Software and Technology. ., ., 1–2.
[3] George Chernyshov, Kirill Ragozin, Jiajun Chen, and Kai Kunze. 2018. Dubhap:
A sensory substitution based superhuman sport. In Proceedings of the First Superhuman Sports Design Challenge: First International Symposium on Amplifying
Capabilities and Competing in Mixed Realities. ., ., 1–6.
[4] Felix Hülsmann, Julia Fröhlich, Nikita Mattar, and Ipke Wachsmuth. 2014. Windand warmth in virtual reality: implementation and evaluation. In Proceedings ofthe 2014 Virtual Reality International Conference. ., ., 1–8.
[5] Shuichi Ino, Shunji Shimizu, Tetsuro Odagawa, Mitsuru Sato, M Takahashi, T
Izumi, and T Ifukube. 1993. A tactile display for presenting quality of materials
by changing the temperature of skin surface. In Proceedings of 1993 2nd IEEE
International Workshop on Robot and Human Communication. IEEE, ., ., 220–224.
[6] Dhruv Jain, Misha Sra, Jingru Guo, Rodrigo Marques, Raymond Wu, Justin Chiu,
and Chris Schmandt. 2016. Immersive terrestrial scuba diving using virtual reality.
In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in
Computing Systems. ., ., 1563–1569.
[7] Vasil Kotsev, Atanas Nikolev, Kasper Pawlak, and Markus Löchtefeld. 2017. Investigating the usage of thermal feedback as an active game element. In Proceedingsof the 16th International Conference on Mobile and Ubiquitous Multimedia. ., .,91–95.
[8] Mutsuhiro Nakashige, Minoru Kobayashi, Yuriko Suzuki, Hidekazu Tamaki, andSuguru Higashino. 2009. " Hiya-Atsu" media: augmenting digital media withtemperature. In CHI’09 Extended Abstracts on Human Factors in Computing
Systems. ., ., 3181–3186.
[9] Roshan Lalitha Peiris, Yuan-Ling Feng, Liwei Chan, and Kouta Minamizawa.
2019. Thermalbracelet: Exploring thermal haptic feedback around the wrist. In
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. .,., 1–11.
[10] Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Minamizawa. 2017. Thermovr: Exploring integrated thermal haptic feedback withhead mounted displays. In Proceedings of the 2017 CHI Conference on Human
Factors in Computing Systems. ., ., 5452–5456.
[11] Kirill Ragozin, George Chernyshov, and Kai Kunze. 2019. HeatSense–Thermal
Sensory Supplementation for Superhuman Sports. In 2019 IEEE Conference on
Virtual Reality and 3D User Interfaces (VR). IEEE, ., ., 1831–1833.
[12] Kirill Ragozin, Dingding Zheng, George Chernyshov, and Danny Hynds. 2020.
Sophroneo: Fear not. A VR Horror Game with Thermal Feedback and Physiological Signal Loop. In Extended Abstracts of the 2020 CHI Conference on Human
Factors in Computing Systems. ., ., 1–6.
[13] Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, David Tolley, and Ellen YiLuen Do. 2017. Ambiotherm: enhancing sense of presence in virtual reality bysimulating real-world environmental conditions. In Proceedings of the 2017 CHI
Conference on Human Factors in Computing Systems. ., ., 1731–1742.
[14] Nimesha Ranasinghe, Pravar Jain, Nguyen Thi Ngoc Tram, Koon Chuan Raymond
Koh, David Tolley, Shienny Karwita, Lin Lien-Ya, Yan Liangkun, Kala Shamaiah,
Chow Eason Wai Tung, et al. 2018. Season traveller: Multisensory narration forenhancing the virtual reality experience. In Proceedings of the 2018 CHI Conferenceon Human Factors in Computing Systems. ., ., 1–13.
[15] Thomas B Sheridan. 1992. Musings on telepresence and virtual presence. Presence:
Teleoperators & Virtual Environments 1, 1 (1992), 120–126.
[16] Jordan Tewell, Jon Bird, and George R Buchanan. 2017. Heat-nav: Using temperature changes as navigation cues. In Proceedings of the 2017 CHI Conference on
Human Factors in Computing Systems. ., ., 1131–1135.
[17] Dangxiao Wang, Kouhei Ohnishi, and Weiliang Xu. 2019. Multimodal haptic
display for virtual reality: A survey. IEEE Transactions on Industrial Electronics
67, 1 (2019), 610–623.
[18] Eric Whitmire, Hrvoje Benko, Christian Holz, Eyal Ofek, and Mike Sinclair. 2018.
Haptic revolver: Touch, shear, texture, and shape rendering on a reconfigurable
virtual reality controller. In Proceedings of the 2018 CHI Conference on Human
Factors in Computing Systems. ., ., 1–12.

66