Frisson Waves: Exploring Automatic Detection, Triggering andSharing of Aesthetic Chills in Music Performances
YAN HE, Keio University Graduate School of Media Design, JapanGEORGE CHERNYSHOV, Keio University Graduate School of Media Design, JapanJIAWEN HAN, Keio University Graduate School of Media Design, JapanDINGDING ZHENG, Keio University Graduate School of Media Design, Japan
RAGNAR THOMSEN, Keio University Graduate School of Media Design, JapanDANNY HYNDS, Keio University Graduate School of Media Design, JapanMUYU LIU, Cleveland Institute of Music, United States
YUEHUI YANG, Keio University Graduate School of Media Design, JapanYULAN JU, Keio University Graduate School of Media Design, JapanYUN SUEN PAI, Keio University Graduate School of Media Design, Japan
KOUTA MINAMIZAWA, Keio University Graduate School of Media Design, JapanKAI KUNZE, Keio University Graduate School of Media Design, Japan
JAMIE A WARD, Goldsmiths, University of London, United Kingdom
Frisson is the feeling and experience of physical reactions such as shivers, tingling skin, and goosebumps. Using entrainment
through facilitating interpersonal transmissions of embodied sensations, we present "Frisson Waves" with the aim to enhance
live music performance experiences. "Frisson Waves" is an exploratory real-time system to detect, trigger and share frisson
in a wave-like pattern over audience members during music performances. The system consists of a physiological sensing
wristband for detecting frisson and a thermo-haptic neckband for inducing frisson. In a controlled environment, we evaluate
detection (n=19) and triggering of frisson (n=15). Based on our findings, we conducted an in-the-wild music concert with
48 audience members using our system to share frisson. This paper summarizes a framework for accessing, triggering andsharing frisson. We report our research insights, lessons learned, and limitations of "Frisson Waves".
CCS Concepts: • Human-centered computing → Interaction techniques; Interaction devices; Interactive systems
and tools; Participatory design; Collaborative and social computing.
Additional Key Words and Phrases: wearable computing, frisson, haptics, live performance, physiological sensing
Authors’ addresses: Yan He, yann@kmd.keio.ac.jp, Keio University Graduate School of Media Design, Japan; George Chernyshov, Keio
University Graduate School of Media Design, Japan, chernyshov@kmd.keio.ac.jp; Jiawen Han, Keio University Graduate School of Media
Design, Japan, hanjiawen@kmd.keio.ac.jp; Dingding Zheng, Keio University Graduate School of Media Design, Japan, zheng208@kmd.keio.ac.
jp; Ragnar Thomsen, Keio University Graduate School of Media Design, Japan, ragnar@kmd.keio.ac.jp; Danny Hynds, Keio University Graduate
School of Media Design, Japan, dannyhynds@kmd.keio.ac.jp; Muyu Liu, Cleveland Institute of Music, United States, mliu@idyllwildarts.org;
Yuehui Yang, Keio University Graduate School of Media Design, Japan, yuehuiyang@kmd.keio.ac.jp; Yulan Ju, Keio University Graduate Schoolof Media Design, Japan, yulan-ju@kmd.keio.ac.jp; Yun Suen Pai, Keio University Graduate School of Media Design, Japan, pai@kmd.keio.ac.jp;
Kouta Minamizawa, Keio University Graduate School of Media Design, Japan, kouta@kmd.keio.ac.jp; Kai Kunze, Keio University Graduate
School of Media Design, Japan, kai@kmd.keio.ac.jp; Jamie A Ward, Goldsmiths, University of London, United Kingdom, jamie@jamieward.net.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that
copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first
page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy
otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions frompermissions@acm.org.
© 2022 Association for Computing Machinery.
2474-9567/2022/9-ART118 $15.00
https://doi.org/10.1145/3550324
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118118:2•He et al.

ACM Reference Format:
Yan He, George Chernyshov, Jiawen Han, Dingding Zheng, Ragnar Thomsen, Danny Hynds, Muyu Liu, Yuehui Yang, Yulan
Ju, Yun Suen Pai, Kouta Minamizawa, Kai Kunze, and Jamie A Ward. 2022. Frisson Waves: Exploring Automatic Detection,
Triggering and Sharing of Aesthetic Chills in Music Performances. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 3,
Article 118 (September 2022), 23 pages. https://doi.org/10.1145/3550324

1

INTRODUCTION

Fig. 1. Frisson Waves: Music performance with EDA/heart-rate base frisson detection and frisson sharing using a thermal
neckband.

When music deeply resonates with us, many experience a sudden feeling of excitement and chills, otherwise
known as frisson. Frisson is a psycho-physiological phenomenon commonly described as having goosebumps
or feeling shivers down one’s spine. This sensation can be evoked by intense emotions or it can be triggered
via external stimuli [2, 46]. Previous research reported most people sometimes experience "the chills", which isoftentimes described as the feeling of goosebumps and shivers on the neck, scalp, and spine, when listening tomusic [35]. While some people may never experience this sensation. Studies link the experiencing of aesthetic
chills to moments of peak pleasure and emotional responses [2, 3, 47]. Individuals who experience frisson tend tofeel higher degrees of pleasure than those who do not experience frisson [16]. Based on these findings, we assume
enhancing frisson sensations may increase the aesthetic experience, especially for those who never experiencefrisson naturally.
In our research, we present the first in-depth exploration of frisson from detection, over triggering to sharing
this complex feeling. Frisson is an intimate personal experience, there are a lot of divergent descriptions of frisson
as a feeling (e.g. chills down the spine, goosebumps on arms or neck) [2]. Only around 30% of people can actually
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:3

experience it. However, it seems at least the timing of frisson in musical pieces can be more clearly defined. The
feeling is usually very short: usually around 4–10 seconds. It can be described as pleasurable [28, 56]. Typicalstimuli can be found in loud parts of music and parts that violate some expectation (introducing a pattern and thenbreaking with it, therefore violating some kind of muscial expectation of the listener) [8]. In a live performancesuch as music concerts, speeches, or movies, verbal communication is usually not encouraged. Therefore, theexciting moments of frisson are often difficult to communicate and share with each other. Previous works related
to goosebumps, shiver sharing and transmission exist, such as by Neidlinger et al. [33] who explored externalizing
the feeling of goosebumps in their work ”AWElectric”. Cooper et al. [14] found that people’s temperature goes
down if they see others shiver. However there is a gap in the literature about sharing frisson. To this end, we
present "Frisson Waves".
We investigate if sharing frisson can lead to a deeper understanding of other people’s aesthetic experienceand further elicit an expressive appreciation of the musical piece. Thus far frisson has been relatively underexplored [16]. To start to address this, we explore the feasibility of sharing frisson through a combination oflab studies and in-the-wild concerts, and we investigate the potential influence of this on the audience’s live
experience through multi-modal evaluation methods. We aim to augment the aesthetic experience throughenhancing the capability of feeling frisson which is usually affected by music appreciation and training [16],personality [50], openness [13], physiological and neural features [42]. While it may not be possible to changethese individual characteristics directly, we aim to mediate frisson between people using a thermohaptic approachthat allows neighboring people to perceive frisson beyond their conscious control [24]. For example, if a person
who is sensitive to music and appreciates music feels frisson at a specific moment in a piece, a person who
receives our actuation might also feel this frisson or at the very least have their attention drawn to how special
that moment is. The concept is akin to the intuition that sometimes we gain appreciation for a piece of music only
after reading favourable reviews. In essence, our system aims to provide feedback via a real-time physiological"review" during the concert.
Going to a concert is a collective experience. The physical presence is a predictor of connectedness between bothartist and the audience [36]. Three primary perspectives can be identified for people who go to live performancesrather than tuning in to records: concerts are defined as live events, they include communal, social components
and they focus on a specific kind of music. Experiencing live events usually makes audience members more
susceptible to spontaneous synchronization, and entrainment [53]. Previous research findings suggest that ahigher synchrony of physiological signals among audience members in classical music concerts may have apositive influence on their aesthetic experience and social connectedness [36, 54]. Performers and audience
members gathering in the same space and at the same time draw their attention on the music piece together.
As a quintessential example of rhythmic join action, music can elicit synchronized reactions more than otherscenarios [25], which provides researchers more opportunities to investigate psychological and physiologicalentrainment. By further sharing intimate feelings of frisson, the emotional contagion, connectedness, and evenphysiological synchrony between people might be further enhanced. We therefore hypothesize that increasing thesynchrony of psycho-physiological phenomena with external stimuli between audience members may positively
influence their aesthetic experience and social connectedness.
The wider aim of the Frisson Waves project is to positively influence the concert experience by increasing
psycho-physiological synchrony through mediating frisson between audience members. In essence the work
aims to turn aesthetic perception inward: to explore intimate reactions, enabling a sense of attention and an act oflistening that is individually transformative as well as it is collective. It is envisioned as a communal assemblageof frisson experiences to evoke synchronization and connectedness among spectators and to provide a social
component of implicit communication. Towards this we aim to mediate audience sensations and experiencesby creating a new design space for bi-directional performer-audience interactions, which we evaluate during alive classical performance. In this paper, we summarize our exploration of sharing frisson sensation through a
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:4•He et al.

self-built system with the hope that this article will stimulate the interest of others in this hitherto unexplored
field.
Frisson Waves is an exploratory work at the intersection of performance and science that consists of twoparts: the development of an unobtrusive wearable system that can detect, trigger, and share frisson; and a seriesof experimental concerts that trial the deployment of the system. The main contributions of this paper are as
follows:• 1) We present and discuss the concept of detecting, triggering and potentially sharing frisson amongaudience members during real-world classical music concerts.
• 2) We describe an open integrated wearable system that consists of a wrist band, back-end processing, andoutput. Our proposed system enables the capturing and translating of physiological signals into thermal
information to induce frisson. We evaluate the system for frisson recognition (n=19), and frisson triggering(n=15). We detail information about the design process, as well as how these sensing modalities can beused to detect frisson, with a special focus on sensing heart rate and electrodermal activity (EDA). The data
and analysis is open-sourced and can be accessed publicly under https://osf.io/rzpn3/.
• 3) We conducted a proof-of-concept concert performance using our frisson sharing system. The concert
was performed by five musicians in front of an audience of 48 people. Frisson was mediated between asubset of audience members and a live feed of psycho-physiological data from the audience was used tocontrol various synthesizers. We discuss the outcomes and implications of sharing psycho-physiological
data among audience members and demonstrate the opportunities of this new design space.

2

BACKGROUND

Recently, the HCI, UbiComp and other related communities have shown increased interest in the embodied interaction and emotion-elicitation fields. For example, experience sharing from exploring autonomous sensory meridian
response (ASMR) to animating awe for embodiment and externalizing cues for communication [1, 27, 32, 33].
Niu et al. discussed design implications for social interactions based on embodied communication principles [34].
Our work is inspired by these research directions, focusing on aesthetics-induced frisson and exploring potential
interpersonal interaction approaches using physiological sensing and interoceptive interactions. In the following,
we will highlight some research on defining and measuring, as well as exploratory works on triggering andsharing frisson.

2.1

Embodied Aesthetic Chills: Frisson

Frisson is a feeling that describes the experience of body states like goosebumps, shivers, and skin tingling. Frisson
is described as a sudden strong feeling of excitement often experienced together with so-called goosebumps [30].
Due to the numerous descriptive terms that have been associated with this experience (chills, thrills, goosebumps,
shivers, and goosetingles), the succinct French term frisson makes it beneficial to provide continuity to this bodyof research [13, 51].
Frisson is known as aesthetic chills or musical chills which refer to a set of bodily sensations, such as shivers, or
piloerection (goosebumps), transient paresthesia (skin tingling), sometimes along with mydriasis (pupil dilation).
The embodied feeling of frisson induced body sensations is a physical feeling of shivers down your spine andtingling on the nape of your neck and the back of your arms [30, 46]. Having goosebumps is a physiologicalphenomenon where small bumps appear on the skin surface as the hairs stand up. Frisson can be caused by cold,fear or a strong emotional stimulus. Frisson or aesthetic chills is personal, with highly individual differences rather
than a shareable experience among people [7]. However, while some people have a strong frisson response [35],not all people perceive these moments of frisson naturally.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:5

From a psychology perspective of frisson definition, frisson is a set of body reactions such as goosebumps
or shivers that can be induced by intense and discrete emotions [38]. Mori et al. [31] indicated that the peakemotional response, aesthetic chills could be characterised by mixed emotion rather than basic emotions or
two emotional dimensions. Frisson has distinct varieties and is not equivalent to a single dimension in valencearousal-power space. Instead, frisson is constructed, characterized by groupings in, and between, both bodily
activity and emotional experience data based on the multiple correspondence analysis: warm chills (warmth,
smiling, happiness, stimulated and relaxed), cold chills (coldness, frowning, sadness and anger), and moving
chills (lump in the throat, tears, affection, tenderness, being moved and intensity) [2]. Frisson, as well as aesthetic
chills, appear in the middle of multidimensional scaling space which shows a spectrum of responses ranging
from visceral responses to cognitive responses, which reflects it has both visceral and abstract components [42].
Humans uniquely appreciate aesthetics, experiencing pleasurable responses to complex stimuli that confer
no clear intrinsic value for survival [42]. Embodied aesthetics refers to embodied responses and resonance ofaesthetic emotions and experiences. Aesthetic chills are related to aesthetic emotions [47], Perlovsky et al. [40]suggest that aesthetic emotions are emotions that we feel when we are in aesthetic activities such as dancing,
painting and gymnastics, which has elements related to pure beauty or when we appreciate things. There are a
lot of works focusing on the qualitative aspects of frisson, introducing different scales and surveys for recording
and measurement [39, 43–45, 52].

2.2

Measuring of Frisson

There are many ways to detect and measure chills or goosebumps, such as physiological sensing, optical measurement, or self assessment. Through recording the spatial frequency changes in the visible domain, it is possible toaccurately detect the onset of goosebumps. Research from Benedek et al. [6] combined the optical continuous
measurement of piloerection and self-report method to record goosebumps which provided a reliable mappingof emotion and occurrences of visible piloerection. However, the manifestations of frisson do not necessarily
cause piloerection and depending on the type of skin and device placement, optical measurement may not besuitable for our case. A lighter and simpler wearable device is preferable in live music performances. Kim etal. [26] previously explored measuring mechanical changes in the skin to detect goosebumps through a capacitive
skin piloerection sensor. This is a consistent and convenient measurement though it’s not applicable for large
live performances due to the limited skin area and costs.
The work of Grewe et al. [20] revealed the relationships of physiological responses, emotions and chills.
Reportedly, heart rate (HR) and electrodermal activity (EDA) features are related to chills and frisson. Heart ratevariability (HRV), as the flucturation in the time intervals between adjacent heart beats, could reflect dynamic
affective processes [48]. EDA is an authoritative indicator of emotional arousal associated with human behavior.
It reflects the sympathetic nervous system responses related to emotion. Benedek et al. [5] also found the increaseof phasic electrodermal activity and increased respiration depth are associated with piloerection. The data set
from Zhang et al. [57] includes 457 people’s simultaneous EDA responses to music emotion recognition. Greweet al. [19] looked into EDA and chills stimulated through four different sensory domains (tactile, aural, visual,gustatory).
In summary, frisson can be assessed via a combined self-report and physiological (e.g. galvanic skin response)
measurement. Detecting physiological signals such as EDA, respiratory rate and HRV combined with self-report
are the most efficient and common ways that do not require bulky and complicated setups and can be deployedin large scale settings.

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:6•He et al.

2.3

Triggering of Frisson

In live music performances, there are various factors that typically constitute frisson: cognitive factors such as
personal attachment to the performing content, individual differences such as individual characters or openness
toward things and experiences, social and environmental context, acoustic properties, as well as aesthetic emotions.
Based on the natural factors that cause frisson, there are a lot of related works exploring triggering frisson fromexternal stimulus. Interoception interaction provides the possibility to trigger frisson feelings through driving
embodied reactions such as goosebumps and shivers. Grewe et al. [19] proved that among different sensory
domains, tactile feedback is the most effective trigger. In "The Thermal Feedback Influencer" [23] project, sudden
cold thermal feedback when listening to music also proved to be the obvious trigger of frisson and could enhance
the music listening experience. The project "Frisson" [24] created a spine-attached thermal band as an aesthetic
prosthesis to induce emotions from the body up through interoceptive technologies. Work from Fukushima etal. [17] and "MAGHair" [9] controlled and stimulated body hair to augment the feeling of surprise and awe. Colver
et al. [13] found that aural stimuli such as aesthetic works of music or sound are also suitable for triggering
frisson.

2.4

Sharing of Frisson

Since not everyone could feel frisson [35], and due to the lack of communication in typical music concerts, we
believe sharing frisson and turning individual musical experiences into collective ones can create a new form oflistening. Previous works have showed frisson sharing exists. Neidlinger et al. [33] have explored externalizing
and sharing the feeling of goosebumps with color visualizations and inflatable textiles in the work ”AWElectric”.
Cooper et al. [14] found that people’s temperature goes down if they see others shiver. However, there was noevaluation as of yet on the user experience. So far, no attempts have been made to share frisson veritably andcollectively during live performances to create cues for interpersonal implicit communication. To bridge this gap,
we introduce an interactive system based on physiological sensing and frisson haptics that detects, induces, andshares frisson.

3

DESIGN PROCESSThe system evolved in an iterative co-design process with professional classical music performers, humanfactors/haptics researchers, and user experience designers. The system was developed iteratively over thecourse of four studies: an in-the-wild study piloting the design requirements for frisson feedback during a live
performance, a lab study validating a method for automatic frisson detection, a lab study evaluating the frisson
inducing device, and a final in-the-wild study evaluating the complete system during a live performance:• Exploring Frisson In-Situ Concert I. This preliminary in-the-wild study, discussed in Section 3.1, aimed
to understand the audience’s expectations of feeling and sharing frisson. The study confirmed the positive
feedback of feeling frisson and the spectator’s tendency to share frisson, as well as piloted the systemdesign requirements.
• Physiological sensing wristband and frisson detection ML model is introduced in 4.1 and 4.2. Thislab study aims to detect frisson accurately by utilizing a physiological sensing wristband and developing
an ML model. The physiological sensing wristband records and extracts EDA and HRV that can be used todetect when someone has frisson. The ML model processes the wirelessly streamed physiological data forreal time classification of the data as frisson or non-frisson. The model with the best performance presented
an average accuracy score of 85.78% with an average precision score of 81.75%• Frisson induction neckband is introduced in 4.3. This lab study aims to trigger frisson efficiently bydeveloping a neckband with Peltier thermoelectric cooler modules (TEC). This study confirms the neckband
is able to trigger 93% of 15 participants to feel frisson, 73% of them felt intense frisson.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:7• Frisson-sharing server and Frisson-Sharing Concert II. is introduced in 4.4 and 5.1. The frisson sharing
server controls the operation of all the devices and manages data processing and recording. The aim of theConcert II. study was to determine if frisson sharing can positively impact an audience’s musical experience.
This study found several spectators’ decrease in enjoyment caused by the neckband, and the uncontrolledinterpersonal relationships of the live study both meant it was difficult to draw a quantitative conclusion
on enhanced frisson occurrences and enjoyment. However, higher physiological synchrony and relatively
higher self-reported social connectedness are found in the audience who shared frisson compared withthose who did not.

3.1

Field Study: Live Concert I. Exploring Frisson In-Situ

In order to better understand the impact of frisson and how performers and spectators think toward sharing frisson
in a wild live performance, we held an experimental concert with 20 audience members (female=13; male=7)
between 20 and 60 years of age (mean age=29.5, standard deviation=11.91). COVID-19 infection prevention
measures were implemented in this concert by disinfecting all devices and keeping an appropriate distance.
The experimental concert included a 40 minutes classical piano program, featuring “Chopin Prelude, Op.28
No.15”, “Chopin Ballade, No.1”, “Grieg: Piano Concerto in A minor, Op.16 Cadenza & Coda”, and “Rachmaninoff:
Études-Tableaux, Op.39, No.1”. These works offer a wide range of textures, dynamics, tempos, and implied
emotions which are regarded as aesthetic emotion-elicitation pieces and allowed us to observe the aesthetic
responses of the audience in many different ways.
Each pieces’ frisson occurrences, related embodied responses and qualitative comments of frisson were accessedwith questionnaires. Based on subjective responses from 20 audience members, 69.72% have experienced frissonand regard it as a positive component of an enjoyable live music performance. Wishing to experience morefrisson was reported by 83.33%. 85% of the spectators, who reported they usually communicate with their friends
in concert contexts such as eye contact, holding hands, talking or whispering and so on. This suggests that people
have the tendency to communicate with others in such scenarios. Through the interview of spectators who felt
frisson, one of them reported "there is the deep touch of sublime that only happens when I am in this concert";
other user comments included "feeling deep breathing", "increasing temperature" ,"starting to sweat", "feelinggoosebumps", "heartbeat goes faster and faster", "a lump in the throat", and "eyes getting wet, tears falling". Mostof them reported they would like to share their visceral reactions and exciting moments such as frisson responses
with other spectators without interfering with the performances, which shows the possibility of a silent implicit
interaction approach that could facilitate meaningful interpersonal communication.
This pilot study was done to understand and confirm the audience’s expectations of feeling, sharing frissonand social needs during a live performance. Based on our observations, we decided to design an unobtrusive
frisson-sharing system which can be deployed at scale.

3.2

Design Concept

We designed two hardware devices for triggering and detecting frisson (see as Figure. 2). The first is a wristband
measuring EDA and blood volume pulse (BVP) (see as Figure. 2.(2)). The second device is a neckband withhaptic actuators and Peltier thermoelectric modules to induce frisson (see as Figure. 2.(1)). With this setup we
can detect with a certain degree of precision the occurrences of frisson or onset of intense emotions using thephysiological data. In the case of detection, frisson is spread or shared to the audience members sitting next tothe one experiencing frisson. In the case of successful induction of frisson, we assume that it is detected and willcontinue spreading through the audience akin to ripples or waves in the water [21].
The flow of sharing frisson is as shown in Figure. 3. Firstly, physiological data recorded from the wristbands isstreamed to the server in real time. Secondly, the server processes the received data, extracts certain features
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:8•He et al.

Fig. 2. Thermo-haptic Neckband and Wristband

Fig. 3. System Development Frameworkand detects frisson using the trained model every 10 seconds and operates within a 2 minute adjustable sliding
window. Thirdly, the system activates the thermo-haptic neckbands around the detected audience members likea drop of water falling down to create the rippling wave pattern. Thereby, the system enables the transmission offrisson in a pattern like a rippling wave.

4

SYSTEM DEVELOPMENT AND EVALUATION

We developed a system with two devices, one for detecting and one for triggering frisson, and conducted a series
of user studies to evaluate them. The final version of the complete system was used during a live concert (seeSection 5.1) held on June, 25th, 2021.

4.1

Wristband: Frisson Self-report and Physiological Data

Automatic detection of frisson is performed using a smart physiological sensing wristband [12] (see as Figure. 2.(2))
that measures BVP and EDA. The devices utilize ESP32 modules and use WiFi to stream the recorded data. BVP
is measured via a reflection-based optic plethysmograph with the sampling rate of 50Hz. EDA as the changeof skin conductance is measured with a Wheatstone bridge, an instrumentation amplifier and ADC with 16-bit
resolution in +-2V range, with basic low-pass RC filters on both. EDA sampling rate is 4.545 Hz, 12-bit resolution.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:9

Readings are streamed to the server software where the heart rate variability (HRV) and EDA-related features
are extracted to be fed into the frisson detection model since studies showed EDA and HRV physiological signals
have the most obvious pattern changes regard to aesthetic chills [4, 19, 20]. The frisson detection model runs
every 10 seconds for each participant and operates with an adjustable sliding window size around one minute. To
ensure an adequate amount of data, the server sent data in a window larger than one minute (approximately 2
minutes) window each time. The full system description is shown in Figure. 3

4.2

A Machine Learning Model: Frisson Recognition

In order to train the model, we processed the recorded EDA and blood volume pulse (BVP) data as well as when
the participants pressed the frisson-report button for data labeling.
Experiment Procedures The same Thermo-Haptic neckband and wristband were used with a counterbalanced
order of 3 parts, including part A, part B, and part C. Part A is a five minute excerpt from Gustav Holst’s "The
Planets: Jupiter, the Bringer of Jollity", approximately 4:00-9:00. According to previous studies, this particular
5 minutes is the part most successful in provoking chills [4]. Part B is a three minute cold thermal feedback
stimulus session through the neckband with no music stimulation. The cold feedback onset period is 8 seconds:
3 seconds cold feedback "on" and 5 seconds cold feedback "off". We will introduce more details on the thermal
feedback in section 4.3. Part C is a 5 minute piano recording from Frédéric Chopin’s "Prelude, Op. 28, No. 15". It
was recorded from the first live concert we held and rated most likely to have frisson from the audience. In thisstudy, we only turn on the Thermo-Haptic Neckband in one music part throughout three sessions: A and B or B
and C.
We recruited 33 volunteers (female= 17; male = 16) from 7 countries aged 22-37 years old (mean = 26.09) for
the 30 minutes study. Once the study began, participants filled out the demographic questionnaire followed by
the explanation of frisson while the investigators helped them put on the wristband with a frisson-report button
and the neckband with two thermal modules placed on the back of their neck. Then investigators then explained
the definition of frisson and when to press the button.
Model Training We use a support vector machine classification (SVM) algorithm in our frisson detectionreferring to some previous works using physiological data to detect physical or mental phenomenon [41, 49].
Our model was trained using the features extracted from a sliding window of 1 minute moving every 1 second.
(Note that during the real-time detection stage, to ensure sufficient data in the event of data loss, 2 minutes ofdata is sent, from which only 1 minute is ultimately classified at a rate of every 10 seconds.) The window waslabeled as a frisson event if the button was pressed within the window. We removed the participant’s data if thedata was either too noisy to process or too few frisson events were reported, which left us 19 participants’ data intotal. Each participant’s raw EDA data was passed through a 2nd order Butterworth low-pass filter (0.5 Hz). And
Each participant’s raw BVP data was passed through a 4nd order Butterworth low-pass filter (4 Hz). We extracted
four EDA features and three HRV features using Neurokit2 [29] from the filtered signals.
According to the feature importance by a Random Forest Classifier, we selected the following features andnormalized the values to remove individual differences:• EDA-Tonic: EDA Tonic component value• EDA-Phasic: EDA Phasic component value• Tonic-diff-60: the change EDA Tonic component value in 60 seconds• Tonic-diff-30: the change EDA Tonic component value in 30 seconds• HRV-MeanNN: the average of normal sinus beats’ interbeat intervals (NN)• HRV-pNN50: the percentage of adjacent NN intervals that differ from each other by more than 50 ms• HRV-pNN20: the percentage of adjacent NN intervals that differ from each other by more than 20 ms
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:10 •He et al.

Fig. 4. One subject’s EDA signal plots of natural and triggered frisson events lasting for around eight seconds. A similar
trend of filtered EDA signal for natural and triggered frisson events could be visually inspected.

Fig. 5. Confusion matrix for the prediction model with LOPO-CV. A row represents an instance of the actual classes whereas
a column represents an instance of the predicted classes.Each number represent the sample numbers falling into eachquadrant.The overall sensitivity is 92.81% and the overall specificity is 76.06%.

Biosignals of both the original frisson and synthetically generated frisson events were analyzed and plotted.
Firstly, we visually inspected the trend and change of biosignals and found the two types of events were similar
enough to be both considered as frisson experiences. Figure. 4 shows one subject’s example of frisson events(natural and triggered) and the trend of filtered EDA signal recorded. Moreover, we trained a trial model using
the data from three subjects combined who reported frisson in both natural sessions and triggering sessions. The
model was trained on the data in the natural session and tested on the triggering session with an accuracy of
80.42% as an initial result. Both of the results supported our concept.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:11

We further developed our model using all the data from 19 participants and applied leave one participant out
cross validation (LOPO-CV) to divide data into training and testing sets. The classifier with the best performance
presented an average accuracy score of 85.78% (sd = 11.23%) with an average precision score of 81.75% (sd =12.48%). Figure. 5 illustrates the overall performance of the model.

4.3

Thermo-haptic Neckband: Triggering Frisson

Our system uses a thermo-haptic neckband(see as Figure. 2.(1)) that includes two Peltier TECs fixed on a neckband.
Each TEC has an exposed heatsink attached to the side opposite to the user’s skin. The device is controlled by an
ESP32 module and communicates with the system server using WiFi. It is powered from a 1000 mAh 3.7V battery
which ensures several hours of constant usage. There was 3W of heat transfer between the sides of the peltier
module, the exact temperatures depend on the thermal conductivity of the neck - Peltier element thermal joint.
Since the skin heat conductance, temperature, and neck geometry are highly individual, we test and chosen that
3W of heat transfer is a generalizable estimate. In the event of activation each thermal module is turned on for 5
seconds with a consecutive 5 seconds of cool-down time, allowing the heatsink to dispose of the accumulated
heat, which ensures constant performance of the thermal actuation. In total, each TEC consumes up to 1.2A ofcurrent at fully charged battery voltage of 4.2V. As the battery discharges, the current drops to 0.9-1A, which
provides sufficiently stable cold sensations. For safety reasons the battery is protected with a 2.5A PPTC fuse.
Thermo-haptic Neckband Study In order to test whether the Thermo-Haptic Neckband [22] can trigger
frisson at all we conducted a simple user test with 15 volunteers (female=7; male=8) aged 22-37 (mean =26.4).
Participants were explained the concept of frisson while the investigators helped them put on the wristband witha frisson-report button. They were asked to press the button as long as they experienced frisson or any form ofshivers. The two Peltier TEC modules of the neckband were placed on the back of their neck for one minute andprovided pulses of cold thermal feedback to trigger frisson. Afterwards, the participants were asked to report if
they experienced frisson-like feelings or not and were asked to rate the intensity of their frisson experience on a
7-point Likert scale (1= not at all; 7= very intense frisson feeling). The two TECs were pumping about 5-5.5W ofheat and reached a surface temperature of 24°C.
Results We analyzed three sets self-report data: if the participants pressed the button during the test, if theyreported had frisson and if they had higher than a level 4 intensity of frisson according to the interview. If yes,
we marked the feedback data as 1, otherwise 0. Three one-tailed T-tests were performed. A total of 52 buttonpresses were recorded from 11 out of the 15 participants (M=.73; p=.034). 14 out of 15 participants reported theyexperienced frisson-like feelings during the test (M=.93; p<.001). 11 participants consciously experienced higher
than level 4 frisson during the test (M= .73; p=.034). This result suggested that the neckband could successfully
trigger frisson.

4.4

Frisson Sharing Server

The server software controls each device via a TCP/IP network connection. The server records all the data fromthe wristbands and manages the data processing and recording. The server starts a python script for each device
that extracts the EDA and HRV features necessary for the frisson classification model and runs the classifier
every 10 seconds for each device. If an occurrence of frisson is detected, python script reports it back to theserver. Then the server commands all the neckbands adjacent to the participant who had just experienced frisson
to activate and apply cold feedback to their wearers.
The server side is developed in Qt C++ and is capable of handling hundreds of simultaneously active devices.
It is capable of re-streaming the data of the selected participants to other computers for generation of real-timevisuals or audio. During the final performance mentioned in 5.1, we streamed live heart rate data to an Ableton
Max for Live patch to generate real-time soundscapes. Although the data has to be buffered on the wristband
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:12 •He et al.

side, the server is gradually de-buffering it for the stream and produces a smooth stream at about 50 Hz. The
delay caused by several transmission buffers averages at 400 ms.

5LIVE CONCERT II. SHARING FRISSON

The purpose of the concert was both to test the system’s viability at detecting and triggering frisson experiencesin the wild, and to investigate and compare any differing effects on frisson-sharing and non-sharing groups.
Specifically questions related to their frisson experience, their physiological synchrony and their sense ofconnectedness. We tested our complete system in an in-the-wild study where we organized a Frisson Waves
Concert on June 25th, 2021 at Kawasaki Symphony Hall Assembly Room.

5.1

Concert Information

Five professional musicians, each with roughly 20 years of experience, curated and composed a musical program
based on their artistic interpretations to evoke the audience members aesthetic responses. In the following, we
describe the concert program.
The concert consisted of three sessions. Session 1 was a 20 minute interactive ambient music performance
titled "Reflections on Chopin Prelude Op.28 No.15" was based around two electronic artists generating soundswith laptops and synthesizers from the audience’s real-time heartbeats and frisson physiological signals using
custom-built patchers in Max/MSP. These patchers function by receiving data from the server via OSC (opensound control) protocol, which is then distributed by the artists to several filters, triggers, and sound modifiers.
One violinist and one pianist performed with the electronic artists together with audiences’ physiological feedback
loop, which transformed the performance from a more structured classical work into a semi-improvisational
performance piece. We composed this piece to express the lost feeling in the gloomy periods and bring theaudience implicitly together with us to complete this piece.
Session 2 was a 21 minute classical piano program of "Beethoven: Sonata No.30 Op.109". In the opinion of the
pianist, this sonata, composed by Beethoven in 1820, is very compact both spiritually and technically. It has awide variety of emotions, including melancholy, joy, and a feeling of grace. It sometimes portrays resentment andconflict as well. In particular, the third movement, which begins with a naive and romantic theme and consists ofsix variations and a coda, expresses various human emotions skillfully such as deep love and conflict hidden in
Beethoven’s innermost feelings.
Session 3 was an 18 minutes classical piano program of two pieces by Frederic Chopin. The first was "Nocturne
Op.27 No.1". This nocturne, composed in 1835, was written in a deeply sorrowful tonality in C sharp minor.
A theme consisting of a wide-range chord in the left hand and a simple melody that echoes above it in theright hand. Chopin’s delicacy has changed from the mysterious grace to the dramatic appearance of Mazurka
in the middle part and the sudden appearance of Mazurka comes as if to express the national feelings towards
Chopin’s home country Poland, with the piece then returning to the main theme quietly. It revealed his loveof his homeland and the resentment and anxiety that dwelled somewhere in his heart. The second piece was"Preludes Op.28 No.18 24". These preludes, which were completed on Mallorca in 1838, were influenced by Bach’s
equal temperament and were composed with one song each, covering all 24 tonalities. The length and difficultyof each piece are different. Although there is no unity between the songs, the characters of each tonality are
expressed in a delicate, graceful, and bold way. That is typical of Chopin’s works, and the harmony of the songs
before and after is well maintained. Each piece is a straightforward projection of Chopin’s music and feels for life.
It is a collection of preludes that skillfully expresses human emotions. Chopin composed these pieces to cover thetravel expenses for his escape to Mallorca with George Sand.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •5.2

118:13

Audience

48 audience members in total attended the concert (female=28; male=19, 1 other or preferred not to say) between
19 and 83 years (mean =38.53, sd=15.09). The audience registered voluntarily through a concert poster posted onsocial media with knowing the time, location, music programs of the concert. When they came to the concert, we
prepared flyers to illustrate the concert and system implementation and delivered to each participant together
with the consent forms about data usages and photography before they entered the hall. COVID-19 infection
prevention measures were implemented in this concert such as regulating the distance between all participants
as well as sanitizing all the wearable devices before and after usage. The participants could stop wearing thedevice or leave the hall anytime they want.
48 audience members were able to choose to sit in one of two groups with knowing the experience difference:sharing-frisson and non-sharing groups. The frisson-sharing group consisted of 24 audience members wearing
our neckbands and wristbands (see as Figure. 1). The non-sharing group of 24 audience members wore only thewristbands.
We designed the questionnaire which included the audience’s frequency of feeling musical frisson or aesthetic
chills both in general and in this experimental live concert. These two questions were accessed with a 5-point
Likert scale (1= Never, 3=Sometimes, 5=Very often). Intensity of frisson feeling in this concert were accessedwith a 7-point Likert scale of the question "I have felt very intense frisson during this concert." (1= Disagree
Strongly, 4= Neutral/Mixed, 7= Agree Strongly). Enjoyment and engagement of this concert are accessed witha 7-point Likert scale (1= disagree strongly, 4= Neutral/Mixed, 7= Agree Strongly). For audiences who are in
the frisson-sharing group, "feeling frisson induced by the neckband more than other similar concerts" and"decline of enjoyment because of wearing neckband" was assessed using a 7-point Likert scale (1= Disagree
Strongly, 4= Neutral/Mixed, 7= Agree Strongly). Besides measuring frisson occurrences, the study also measuresconnectedness, the experience of relatedness between people, which is a central concept in understanding andevaluating communication media, in particular awareness systems [55]. Referred to study [54], which explores
physiological synchrony with connectedness in live concerts, we accessed connectedness score with a 7-point
Likert scale (1= Disagree Strongly, 4= Neutral/Mixed, 7= Agree Strongly) through this question “I felt connectedwith people sitting around me”. To eliminate the effect of interpersonal relationship, it’s accessed with rating thenumber of acquaintances who are sitting adjacent to the user. Questionnaires are also accessible together withthe dataset in our supplementary material (Appendix. A.1).

5.3

System Implementation

We implemented the system mentioned in section4 with the same setup, data streams from the physiologicalsensing wristbands in4.1 are managed by the server software in4.4, and the server records data and classifies
frisson with the machine learning model in 4.2 every 10 seconds. If a frisson signal is detected of one participant,the server commands and activates all the neckbands mentioned in 4.3 adjacent to the participant.

6

CONCERT II. FEEDBACK AND DISCUSSION

This section presents and discusses the outcomes of implementing the Frisson Waves system in-the- wild. Overall,
the audience evaluated the concert with the system positively with relatively high enjoyment levels (mean= 6.2,
sd= 1.01) from both groups. Compared with their previous experiences in similar events, 62.5% of the audience
reported that they enjoyed more in this concert. On the contrary, 14% of the audience reported they used tohave a better experience without. We further investigated the possible reasons and found 7 of them found theneckbands uncomfortable, which may suggest the necessity of improving the neckband’s user experience.
With the audience’s immersion throughout the concert in mind, we did not include real-time self-reporting forfrisson considering the risk of distraction and detriment of enjoyment. This resulted in a limited evaluation on
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:14 •He et al.

Fig. 6. Example of frisson transmission effect matched with seat number in Live Concert II.

the frisson prediction model’s performance in-the-wild and distinguishing between natural frisson and synthetic
frisson in a statistical way. However, researchers were monitoring the system during the whole concert to observe
the feasibility of the system. Here, we mainly discuss the following aspects of the feedback from audience’s
subjective reported frisson feeling, connectedness and physiological states.

6.1

Enhanced Frisson Occurrences

We observed the artefact Frisson Waves effect as we designed for example in Figure. 6. The frisson signal was
detected from subject 33, adjacent neckbands were activated and subject 34 successfully triggered frisson thenshared to subject 38.
For all the 48 audience members, according to the questionnaire answers, 60% of them were familiar with theconcept of frisson, 20% were familiar to a certain extent, and the remaining 20% audience members were notfamiliar with frisson. On a daily basis, about half of the audience reported feeling frisson between "sometimes" to"often", while the other half reported feeling frisson "rarely" to "never". However, the numbers of feeling frissonin relation to music are slightly different. 60% of the audience reported to feel frisson in relation to music, while
40% felt frisson in relation to music "rarely" to "never". In terms of frisson experiences in this concert, 40% ofaudience members reported that they felt more frisson during this concert than usual, while 8.3% felt less frisson.
6.25% reported feeling no frisson at all and the rest found it hard to answer. We found a strong correlation exists
between the reported frequency of experiencing frisson normally and frisson during this concert (R= .62, p< .001).
For the frisson-sharing and non-sharing groups, there is a noticeable difference in the reported number offrisson occurrences during the concert. Although the non-sharing group had reported feeling frisson in relation
to music more often than the other group, the non-sharing group did not experience more frisson during theconcert than usual. 41.67% of the non-sharing group reported that they had more frisson than usual, 54% offrisson-sharing group reported that they had more frisson than usual, the frisson-sharing group had an 8.8%
increase of frisson than usual, which suggests that the frisson-sharing mechanism does increase the number offrisson occurrences. This leads us to think that increasing frisson in a group which is originally less familiar
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:15

with frisson is proof of the adequacy of the system. This result suggests our system worked as we expected for
detecting and actuating neckbands to trigger frisson in a live concert scenario.

6.2

Enhanced Connectedness

In order to evaluate our hypothesis that the sense of connectedness in the frisson-sharing group would behigher than in the non-sharing group, we compared the subjective connectedness score between the two groups.
However, these scores were found to correlate with the number of acquaintances sat adjacent to one another(R=.31, p= .03). Among the frisson-sharing group, the number of adjacent acquaintances (mean= 1.08, sd= 1.06) is 40%
less (p = .017) than the non-sharing group (mean=1.8 ,sd=1.32). The connectedness score in the frisson-sharing
group (mean= 3.54, sd=1.53) is 16% less (p = .11) than in the non-sharing group(mean=4.17, sd=1.85). We cannot
state a significant difference (p = .11) of the sense of connectedness within the two comparison groups, though weassume that the score should have a much stronger decrease in the frisson-sharing group based on the significantly
fewer (p = .017) acquaintances. Since it was an experiment in-the-wild study, the audience were allowed to picktheir preferred seats, the groups were not balanced by the personal relationships, and the non-sharing group
happened to know each other much better than the frisson-sharing group. This highlights the importance oftaking personal relationships between participants into consideration when evaluating connectedness. To be able
to evaluate how well the system enhances connectedness among audience members, further experiments will beneeded under tighter conditions.

6.3

Enhanced Physiological Synchrony of EDA

Besides subjective feedback from audience members, we explored the physiological data collected and calculated
the physiological synchrony (PS) of each group. PS occurs when the “physiological activity between two or more
people” becomes associated or interdependent” [37]. Therefore, we consider PS as a feasible metric to quantify
the experience of physiological connectedness. Following the prior work of Gashi et al. [18], we conducted thedecomposition into EDA tonic and EDA phasic, normalized the two components to remove individual differences,and processed dynamic time warping (DTW) to calculate the accumulated distance between every two persons
in each group [18]. After removing noisy and incomplete datasets, we have the EDA data from 9 audience(non-sharing: 4, sharing: 5) in the first session, 16 audience (non-sharing: 7, sharing: 9) in the second session,and 16 audience (non-sharing: 7, sharing: 9) in the third session. Figure. 9 shows one audience’s EDA signal andFigure. 8 shows one audience’s BVP signal as examples. Both of the data were collected during the first session.
Figure. 9 and Figure. 8 can be found in the Appendix. A.3. Figure. 7. presents the distribution of accumulated
distances normalized with the signal length between each pair within the group. According to the t-test results,
we found the normalized accumulated distances of EDA tonic in the frisson-sharing group (mean = .19, sd = .11)
was significantly larger than that in the non-sharing group (mean = .10, sd = .06),t(56) = 3.51, p<.001. However,the normalized accumulated distances of EDA phasic in the frisson-sharing group (the first session: mean = .003,
sd = .002; the second session: mean = .003, sd = .002) was significantly smaller than that in the non-sharing group(the first session: mean = .008, sd = .002; the second session: mean = .005, sd = .002) for both the first session, t(15)= -4.50, p < .001, and the second session, t(56) = -2.81, p < .05.
The EDA tonic component indicates the slow change of skin conductance levels while the EDA phasic
component reflects the quick and prompt change of skin conductance response [10, 11, 15]. Therefore, our findings
may suggest wearing the neckband and sharing frisson could provide audience members more physiologicallysimilar experiences in terms of short-term and sudden feelings of arousal compared with the long-term trend incertain concert sessions.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:16 •He et al.

Fig. 7. Comparison of paired similarity of EDA tonic (a) and EDA phasic (b) between every two audience members in two
groups. Each point represents the signal distance between one intragroup pair of audience members. The smaller the distance,
the similar the pair of data. In the third session, the distances of EDA tonic in the frisson-sharing group was significantly
larger than that in the non-sharing group. While, in the first two sessions, the distances of EDA phasic in the frisson-sharing
group was significantly smaller than that in the non-sharing group. (*p<.05,**p<.005,***p<.001)

7

DISCUSSION

We discuss the social impact of creating and sharing frisson, introspection from the exploratory process, as well
as visionary possibilities which could augment human perception in musical experiences.

7.1

Qualitative Difference between Artificial Frisson and Natural Frisson

In order to better understand the qualitative experience of the artificial frisson induced by our technology, weinterviewed three audience members in the frisson-sharing group after "Frisson Concert II." Subject 30 claimed torarely experience natural frisson when listening to music, stating it would only happen once every few years.
However, whenever they do experience natural frisson while listening to a piece of music, the listening experiencewould leave a profound and lasting impression on them. Subject 30 then goes on to explain that the synthetic
frisson similarly increased their attention to the piece of music. However, the listening experience did not leave
as deep of an impression on them after the concert was over. Subject 33 stated that the technology successfully
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:17

induced synthetic frisson for them. However, they argue that naturally occurring frisson would typically be
accompanied by a feeling of resonance with the music. This would be lacking for the synthetic frisson. Subject 27
reported that natural frisson would come from an emotional engagement with some particular parts of the music
while cold synthetic frisson would just come from the cold. At the same time, synthetic frisson also made theconcert more stimulating as a whole. Based on these interviews, we presume that there is a qualitative difference
between the artificial frisson caused by our technology and the frisson that people naturally feel when listening
to music. We believe that this qualitative difference between artificial and natural frisson opens up new creative
opportunities, as it allows us to explore the complex interplay between music and cognition.

7.2

Sharing Frisson Constitutes a Collective Experience

Both subject 30 and 33 reported a sense of surprise when they noticed someone close to them experiencing
frisson. This aroused their curiosity as to what kind of experience the other person was having in that moment
and why they might resonate so strongly with that moment of music. Subject 27 claimed they started to noticeother audience’s presence through the clues of feeling others’ frisson. Subject 30 also stated that they enjoyedbeing able to share how they resonated with the music with others. This suggests that sharing frisson between
audience members can constitute a unique kind of collective empathetic experience.

7.3

Future Applications for Frisson Sharing

Beyond using the system in a music concert context, we believe that the system can be applied to wider scenarios
related to live experience as well. For example, our system could potentially be used to mediate frisson between
audience members at a movie screening in theaters, watching dance performances, visiting museums, listening
to speeches, and other performing scenarios. Especially during a pandemic, when talking or direct contact are
discouraged, implicit communication approaches are crucial for fulfilling human social needs. Beyond that, oursystem could possibly also be used to share bodily sensations across remote locations. One potential application
for frisson sharing between remote locations might be online concerts, where the audience is watching from home.
Being able to share bodily sensations across remote locations might help to facilitate the collective experiencethat concert goers typically long for. There’s also a potential to develop user dependency of a similar system forsharing human emotions and feelings to create empathetic collective experiences. We could also imagine thatthrough this design process and system platform, other emotions and feelings could also be accessed, shared,
and communicated to others. Elevating the affordances of physiological data sensing to create a feedback loop
between and beyond audience and performers, Potentially, Frisson Waves opens up new horizons for facilitatinga new form of "cybernetic togetherness".

7.4

Neckband in a Ubiquitous Context

We demonstrated our system in a live classical music concert. Beyond this, we believe that the system can beapplied to a variety of other ubiquitous contexts as well. Our current implementation comes in the form of a
neckband. This form factor allows the device to be taken on and off quickly. For a more personalized experience,
it would be possible to integrate our system into clothes, for example a shirt or a turtleneck sweatshirt that isworn directly on the body. Locations other than the neck may be suitable for triggering frisson as well, and will
be explored in future studies.

7.5

Frisson Sharing Design Guideline

Besides the concept, framework, and system which we demonstrated, we would also like to discuss the lessons
we learned which other designers might consider when developing similar systems. Designing a data sensing
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:18 •He et al.

tracking protocol would help recording and streaming valid data in less controllable in-the-wild studies. As with
reporting problematic data source from the server, researchers could help participants wear the devices witha stabler connection in time. Testing the frisson model in a controlled environment before in-the-wild studies
would provide a convincing baseline to evaluate the model’s performance and distinguish artificial frisson andnatural frisson. A user-friendly re-design of the neckbands would help to reduce the constrained impact on
the user experience. A larger user group or a more socially controlled method would be more accessible for
evaluating the impact on connectedness and togetherness. We hope Frisson Waves can give insights for thosewhom would like to design similar systems through providing a formidable starting point for developing frisson
sharing systems.

8

LIMITATIONSThere are still some limitations with our in the wild field study. Firstly, to avoid the risk of distraction we did
not include real-time self-reporting for frisson in our field study. This may limit our evaluation of the frissonprediction model’s performance in the wild, making it difficult to distinguish between natural and synthetic
frisson. Moreover, not all recorded data from the in the wild study was successfully saved. Therefore, we were
not able to explore every audience member’s data and compare between groups. Thus, we may have missed someinsightful differences between frisson-sharing and non-sharing groups. Furthermore, a re-design of the neckband
is warranted as to reduce its impact on the user experience. To give a more direct and tangible understanding of
how much temperature has to change to elicit a feeling of frisson, we will add a thermal sensor to the surface ofeach Peltier of the neckband to measure the temperature presented to the participants for future experiments.
To have a more accurate understanding of how long the thermal feedback activation timing is more efficient to
the trigger frisson, We will conduct a more detailed control group experiment in the future. For evaluating theimpact on connectnedness and togetherness, a more controlled method and or a larger sample size is required.
Frisson Waves explores a new form of "cybernetic togetherness" by allowing one person to perceive anotherperson’s response to and appreciation of live music. We hypothesize that sharing embodied sensations such asfrisson during a live concert can lead to a new way of appreciating the music. However, our approach might
potentially interrupt the natural music appreciation process. For example, person A may feel frisson at similar
moment as person B, albeit a few seconds later. In this scenario, person A’s natural frisson response would overlap
with the synthetic frisson that has been triggered by person B’s sensation. What happens at this overlap of naturaland synthetically induced frisson has not been explored as part of this study. Person A’s natural experience may
be augmented as a result of this overlap, however it might also be interrupted. Possibly, an entirely new sensation
will emerge from this overlap in stimulation. This possible occurrence has not been mentioned by the participants
or investigated by the study. For a conclusive answer, further investigation of this occurrence is warranted.
This exploratory work raises questions: As we are headed towards a future of interconnectedness, the potentialof artificial interruption leads us to reconsider affective augmentation and manipulation. In an increasinglyinterconnected world where technology mediates how and when we feel, we need to have more responsibility
to design such technology. Frisson Waves opens up a much-needed dialog about these questions. While theexperience of sharing your intimate feelings can be empowering, it can also be dangerous. Where to draw theline? Do we need to draw a line?

9

CONCLUSION

This work introduces Frisson Waves, a system that detects and triggers frisson and can be used to create a sharedfrisson effect among audiences during musical performances. The system works by translating physiologicalsignals into thermal information, enabling us to automatically detect frisson through a smart wristband with aProc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:19trained real-time frisson classifier, trigger frisson through a smart neckband with two Peltier thermal modules,
and share frisson through our open integrated server.
We devised and piloted the concept during an initial live performance, we conducted several studies to evaluate
the system, and we tested the final system during a 90 minute concert for 48 spectators. We demonstrated that asimple wearable device with embedded Peltier TECs can induce clearly reported frisson in 93% of participants(N=15) . We demonstrated a classifier trained on 19 participants with 85% accuracy and were able to process datafrom the other 48 audience members in real time and in the wild. The feedback from the concert suggests thatfrisson is shareable using our system. The system is shown to enhance the frisson feeling for people who are notfamiliar with frisson, and to enhance the feeling of connectedness between audiences.
Besides deepening our understanding of frisson and how to share this phenomenon, this research also looks
into the implication of a frisson-sharing system. We hope that the questions raised in this work around artificial
and natural experience, individual and collective experience, along with the technical contributions of Frisson
Waves, will provide some help and inspiration to those seeking to design similar systems or experiences.

ACKNOWLEDGMENTS
This work is partially supported by the Cybernetic Being project under JST Moonshot R&D Program (Grant
number JPMJMS2013). Special thanks to J.Y.PLANNING, Studio Apollon, Hua, and Shaowei for the kind supportand help. Thanks to the many people who offered help and feedback on this project and concert production, andexperiment organization.
We also want to thank Renate Kunze for her loving support, encouragement, and kindness.

REFERENCES[1] Josh Andres, MC Schraefel, Rakesh Patibanda, and Florian’Floyd’ Mueller. 2020. Future InBodied: A Framework for Inbodied Interaction
Design. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction. 885–888.
[2] Scott Bannister. 2019. Distinct varieties of aesthetic chills in response to multimedia. PloS one 14, 11 (2019), e0224974.
[3] Scott Bannister. 2020. A vigilance explanation of musical chills? Effects of loudness and brightness manipulations. Music & Science 3(2020), 2059204320915654.
[4] Scott Bannister and Tuomas Eerola. 2018. Suppressing the chills: Effects of musical manipulation on the chills response. Frontiers in
psychology 9 (2018), 2046.
[5] Mathias Benedek and Christian Kaernbach. 2011. Physiological correlates and emotional specificity of human piloerection. Biologicalpsychology 86, 3 (2011), 320–329.
[6] Mathias Benedek, Barbara Wilfling, Reingard Lukas-Wolfbauer, Björn H Katzur, and Christian Kaernbach. 2010. Objective and continuous
measurement of piloerection. Psychophysiology 47, 5 (2010), 989–993.
[7] Giacomo Bignardi, Rebecca Chamberlain, Sofieke T Kevenaar, Zenab Tamimy, and Dorret I Boomsma. 2022. On the etiology of aesthetic
chills: a behavioral genetic study. Scientific Reports 12, 1 (2022), 1–11.
[8] Anne J Blood and Robert J Zatorre. 2001. Intensely pleasurable responses to music correlate with activity in brain regions implicated inreward and emotion. Proceedings of the national academy of sciences 98, 20 (2001), 11818–11823.
[9] Roger Boldu, Mevan Wijewardena, Haimo Zhang, and Suranga Nanayakkara. 2020. MAGHair: A Wearable System to Create Unique
Tactile Feedback by Stimulating Only the Body Hair. In 22nd International Conference on Human-Computer Interaction with Mobile
Devices and Services. 1–10.
[10] Wolfram Boucsein. 2012. Electrodermal activity. Springer Science & Business Media.
[11] John T Cacioppo, Louis G Tassinary, and Gary Berntson. 2007. Handbook of psychophysiology. Cambridge university press.
[12] Gheorghe Cernisov. 2020. An unobtrusive wearable sensing platform for psychological and social dynamics. Ph.D. Dissertation. Keio
University.
[13] Mitchell C Colver and Amani El-Alayli. 2016. Getting aesthetic chills from music: The connection between openness to experience andfrisson. Psychology of Music 44, 3 (2016), 413–427.
[14] Ella A Cooper, John Garlick, Eric Featherstone, Valerie Voon, Tania Singer, Hugo D Critchley, and Neil A Harrison. 2014. You turn me
cold: evidence for temperature contagion. PloS one 9, 12 (2014), e116126.
[15] Michael E Dawson, Anne M Schell, and Diane L Filion. 2017. The electrodermal system. (2017).
[16] Rémi de Fleurian and Marcus Pearce. 2020. Chills in music: An integrative review. PsyArXiv 1 (2020), 1–50.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:20 •He et al.

[17] Shogo Fukushima and Hiroyuki Kajimoto. 2012. Facilitating a surprised feeling by artificial control of piloerection on the forearm. In
Proceedings of the 3rd Augmented Human International Conference. 1–4.
[18] Shkurta Gashi, Elena Di Lascio, and Silvia Santini. 2019. Using unobtrusive wearable sensors to measure the physiological synchrony
between presenters and audience members. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1
(2019), 1–19.
[19] Oliver Grewe, Björn Katzur, Reinhard Kopiez, and Eckart Altenmüller. 2011. Chills in different sensory domains: Frisson elicited byacoustical, visual, tactile and gustatory stimuli. Psychology of Music 39, 2 (2011), 220–239.
[20] Oliver Grewe, Reinhard Kopiez, and Eckart Altenmüüller. 2009. The chill parameter: Goose bumps and shivers as promising measures inemotion research. Music Perception 27, 1 (2009), 61–74.
[21] Yan He, George Chernyshov, Dingding Zheng, Jiawen Han, Ragnar Thomsen, Danny Hynds, Yuehui Yang, Yun Suen Pai, Kai Kunze, andKouta Minamizawa. 2021. Frisson Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music Performances. In
SIGGRAPH Asia 2021 Emerging Technologies. 1–2.
[22] Y He, D Zheng, G Chernyshov, R Thomsen, J Han, D Hynds, Y S Pai, K Kunze, and Kouta Minamizawa. 2021. Frisson Waves: Sharing
Frisson to Create Collective Empathetic Experiences for Music Performances. In 2021 IEEE World Haptics Conference (WHC). IEEE,
591–591.
[23] Yuri Ishikawa, Anzu Kawazoe, George Chernyshov, Shinya Fujii, and Masashi Nakatani. 2018. The Thermal Feedback Influencer:
Wearable Thermal Display for Enhancing the Experience of Music Listening. In International AsiaHaptics conference. Springer, 162–168.
[24] Abhinandan Jain, Adam Haar Horowitz, Felix Schoeller, Sang-won Leigh, Pattie Maes, and Misha Sra. 2020. Designing Interactions
Beyond Conscious Control: A New Model for Wearable Interfaces. Proceedings of the ACM on Interactive, Mobile, Wearable and UbiquitousTechnologies 4, 3 (2020), 1–23.
[25] Peter E Keller, Giacomo Novembre, and Michael J Hove. 2014. Rhythm in joint action: psychological and neurophysiological mechanisms
for real-time interpersonal coordination. Philosophical Transactions of the Royal Society B: Biological Sciences 369, 1658 (2014), 20130394.
[26] Jaemin Kim, Dae Geon Seo, and Young-Ho Cho. 2015. Capacitive Skin Piloerection Sensors for Human Emotional State Cognition.
Transactions of the Korean Society of Mechanical Engineers B 39, 2 (2015), 147–152.
[27] Josephine Klefeker, Libi Striegl, and Laura Devendorf. 2020. What HCI Can Learn from ASMR: Becoming Enchanted with the Mundane.
In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[28] Stefan Koelsch. 2014. Brain correlates of music-evoked emotions. Nature Reviews Neuroscience 15, 3 (2014), 170–180.
[29] Dominique Makowski, Tam Pham, Zen J. Lau, Jan C. Brammer, François Lespinasse, Hung Pham, Christopher Schölzel, and S. H. Annabel
Chen. 2021. NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior Research Methods (02 Feb 2021). https://doi.org/10.3758/s13428-020-01516-y[30] Jonathon McPhetres. 2021. The physiological study of emotional piloerection: A review and guide for future research. (2021).
[31] Kazuma Mori and Makoto Iwanaga. 2017. Two types of peak emotional responses to music: The psychophysiology of chills and tears.
Scientific reports 7, 1 (2017), 1–13.
[32] Kristin Neidlinger, Lianne Toussaint, Edwin Dertien, Khiet P Truong, Hermie Hermens, and Vanessa Evers. 2019. Emotional prosthesis
for animating awe through performative biofeedback. In Proceedings of the 23rd International Symposium on Wearable Computers.
312–317.
[33] Kristin Neidlinger, Khiet P Truong, Caty Telfair, Loe Feijs, Edwin Dertien, and Vanessa Evers. 2017. AWElectric: that gave me goosebumps,
did you feel it too?. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 315–324.
[34] Shuo Niu, Hugh S Manon, Ava Bartolome, Nguyen Binh Ha, and Keegan Veazey. 2022. Close-up and Whispering: An Understanding ofMultimodal and Parasocial Interactions in YouTube ASMR videos. In CHI Conference on Human Factors in Computing Systems. 1–18.
[35] Emily C Nusbaum and Paul J Silvia. 2011. Shivers and timbres: Personality and the experience of chills from music. Social Psychologicaland Personality Science 2, 2 (2011), 199–204.
[36] Kelsey E Onderdijk, Dana Swarbrick, Bavo Van Kerrebroeck, Maximillian Mantei, Jonna K Vuoskoski, Pieter-Jan Maes, and Marc Leman.
2021. Livestream experiments: the role of COVID-19, agency, presence, and social context in facilitating social connectedness. Frontiersin psychology 12 (2021), 1741.
[37] Richard V Palumbo, Marisa E Marraccini, Lisa L Weyandt, Oliver Wilder-Smith, Heather A McGee, Siwei Liu, and Matthew S Goodwin.
2017. Interpersonal autonomic physiology: A systematic review of the literature. Personality and Social Psychology Review 21, 2 (2017),
99–141.
[38] Jaak Panksepp. 1995. The emotional sources of" chills" induced by music. Music perception 13, 2 (1995), 171–207.
[39] Matthew Pelowski, Helmut Leder, Vanessa Mitschke, Eva Specker, Gernot Gerger, Pablo PL Tinio, Elena Vaporova, Till Bieg, and Agnes
Husslein-Arco. 2018. Capturing aesthetic experiences with installation art: An empirical assessment of emotion, evaluations, and mobile
eye tracking in Olafur Eliasson’s “Baroque, Baroque!”. Frontiers in Psychology 9 (2018), 1255.
[40] Leonid Perlovsky. 2014. Aesthetic emotions, what are their cognitive functions? Frontiers in Psychology 5 (2014), 98.
[41] Ming-Zher Poh. 2011. Continuous assessment of epileptic seizures with wrist-worn biosensors. Ph.D. Dissertation. Massachusetts Instituteof Technology.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:21[42] Matthew E Sachs, Robert J Ellis, Gottfried Schlaug, and Psyche Loui. 2016. Brain connectivity reflects human aesthetic responses tomusic. Social cognitive and affective neuroscience 11, 6 (2016), 884–891.
[43] Alfonsina Scarinzi. 2015. Aesthetics and the embodied mind: beyond art theory and the cartesian mind-body dichotomy. Springer.
[44] Klaus R Scherer and Eduardo Coutinho. 2013. How music creates emotion: a multifactorial process approach. The emotional power ofmusic (2013), 121–145.
[45] Ines Schindler, Georg Hosoya, Winfried Menninghaus, Ursula Beermann, Valentin Wagner, Michael Eid, and Klaus R Scherer. 2017.
Measuring aesthetic emotions: A review of the literature and a new assessment tool. PloS one 12, 6 (2017), e0178899.
[46] Félix Schoeller, AJH Haar, Abhinandan Jain, and Pattie Maes. 2019. Enhancing human emotions with interoceptive technologies. Physics
of life reviews 31 (2019), 310–319.
[47] Felix Schoeller and Leonid Perlovsky. 2016. Aesthetic chills: Knowledge-acquisition, meaning-making, and aesthetic emotions. Frontiersin psychology 7 (2016), 1093.
[48] Fred Shaffer and Jay P Ginsberg. 2017. An overview of heart rate variability metrics and norms. Frontiers in public health (2017), 258.
[49] Lin Shu, Jinyan Xie, Mingyue Yang, Ziyi Li, Zhenqi Li, Dan Liao, Xiangmin Xu, and Xinyi Yang. 2018. A review of emotion recognition
using physiological signals. Sensors 18, 7 (2018), 2074.
[50] Paul J Silvia and Emily C Nusbaum. 2011. On personality and piloerection: Individual differences in aesthetic chills and other unusual
aesthetic experiences. Psychology of Aesthetics, Creativity, and the Arts 5, 3 (2011), 208.
[51] John A Sloboda. 1991. Music structure and emotional response: Some empirical findings. Psychology of music 19, 2 (1991), 110–120.
[52] Robert Stecker. 2006. Aesthetic experience and aesthetic value. Philosophy Compass 1, 1 (2006), 1–10.
[53] Dana Swarbrick, Dan Bosnyak, Steven R Livingstone, Jotthi Bansal, Susan Marsh-Rollo, Matthew H Woolhouse, and Laurel J Trainor.
2019. How live music moves us: head movement differences in audiences to live versus recorded music. Frontiers in psychology 9 (2019),
2682.
[54] Wolfgang Tschacher, Steven Greenwood, Hauke Egermann, Melanie Wald-Fuhrmann, Anna Czepiel, Martin Tröndle, and Deborah
Meier. 2021. Physiological synchrony in audiences of live concerts. Psychology of aesthetics, creativity, and the arts (2021).
[55] Daniel T Van Bel, Karin CHJ Smolders, Wijnand A IJsselsteijn, and YAW De Kort. 2009. Social connectedness: concept and measurement.
In Intelligent Environments 2009. IOS Press, 67–74.
[56] Robert J Zatorre and Valorie N Salimpoor. 2013. From perception to pleasure: music and its neural substrates. Proceedings of the National
Academy of Sciences 110, Supplement 2 (2013), 10430–10437.
[57] Kejun Zhang, Hui Zhang, Simeng Li, Changyuan Yang, and Lingyun Sun. 2018. The pmemo dataset for music emotion recognition. In
Proceedings of the 2018 acm on international conference on multimedia retrieval. 135–142.

A SUPPLEMENTARY MATERIALS
A.1 Dataset
The 33 sets of physiological data recording in the frisson detection ML model study consisted of audience EDA,BVP signals, and self-report frisson labels. We have 33 recordings in total (male = 16; female = 17). By eliminating
incomplete or noisy data records, the sets of data from the recruited participants were 19 recordings.
The 48 sets of physiological data recording in Concert II. three sessions consisted of audience EDA and BVP
signals over three sessions in Concert II. We have 48 recordings in total (male = 19; female = 28, prefer not tosay=1). By eliminating incomplete or noisy data records, the breakdown sets of data from the recruited participants
for each session was: 1st, 9 (male= 3; female=6), 2nd, 16 (male=6; female=10), and the 3rd, 16 (male= 5; female=9).
We uploaded the dataset together with the questionnaires used in the two concerts (Exploring Frisson in
In-Situ Concert I. and Sharing Frisson Concert II.) to the following link: https://osf.io/rzpn3/.

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

118:22 •A.2

He et al.

Descriptive Statistic

Table 1. Comparisons of Mean and Standard Deviation Values of Concert II. Subjective Feedback and EDA Features between
Frisson-sharing Group and Non-sharing Group.***p<.001)

Dependent Variables
Enjoyment (1-7 Likert Scale)Engagement (1-7 Likert Scale)
Frisson Frequency (1-5 Likert Scale)
Frisson Intensity (1-7 Likert Scale)
Frisson Frequency Induced by Neckband (1-7 Likert Scale)Connectedness Score (1-7 Likert Scale)
Number of Acquaintance Sitting Around
Session 1
Normalized Accumulated Distances of EDA Tonic Session 2
Session 3
Session 1
Normalized Accumulated Distances of EDA Phasic Session 2
Session 3

A.3

Frisson-sharing
Mean(SD)
6.13(0.99)
4.86(1.73)
2.83(1.05)
4.08(1.74)
4.33(1.74)
3.54(1.53)
1.08(1.06)
0.184(0.067)
0.181(0.142)
0.192(0.107)
0.003(0.002)
0.003(0.001)
0.002(0.002)

Non-sharing
Mean(SD)
6.29(1.04)
5.21(1.28)
2.67(1.00)
4.33(1.81)/
4.17(1.85)
1.99(1.30)
0.223(0.092)
0.119(0.041)
0.103(0.058)
0.008(0.002)
0.005(0.002)
0.004(0.002)

Statistics
t
-0.59
-0.76
1.03
-0.49
/
-0.76
-2.32*-0.9891.936
3.515***-4.505***-2.809***-1.197

Plot Examples

Fig. 8. BVP signal example collected from one audience member during the first interactive session of the concert. The
left-top plot presents a period of BVP raw signal. The green line chart shows one of the HRV feature used in the modeltraining.

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.

Frisson Waves: Exploring Automatic Detection, Triggering and Sharing of Aesthetic Chills in Music Performances •118:23Fig. 9. EDA signal example collected from one audience member during the first interactive session of the concert. The upper
plot presents the EDA signal after a low-pass filter. The orange markers highlight peaks in skin conductance response (SCR
Peaks) which is related to sudden change of EDA. The middle and the bottom plots present Tonic and Phasic components
extracted from EDA signal [15] which are key features in the prediction model.

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 6, No. 3, Article 118. Publication date: September 2022.