Haptic Empathy: Conveying Emotional Meaning throughVibrotactile Feedback
Yulan Ju

Dingding Zheng

Danny Hynds

yulan-ju@kmd.keio.ac.jp
Keio University Graduate School ofMedia Design
Yokohama, Japanzheng208@kmd.keio.ac.jp
Keio University Graduate School ofMedia Design
Yokohama, Japandannyhynds@kmd.keio.ac.jp
Keio University Graduate School ofMedia Design
Yokohama, Japan

George Chernyshov

Kai Kunze

Kouta Minamizawa

chernyshov@kmd.keio.ac.jp
Keio University Graduate School ofMedia Design
Yokohama, Japan

kai@kmd.keio.ac.jp
Keio University Graduate School ofMedia Design
Yokohama, Japankouta@kmd.keio.ac.jp
Keio University Graduate School ofMedia Design
Yokohama, Japan

Figure 1: a) Experiment material - TECHTILE tookit. b) Experiment part one - recording vibration samples. c) Experiment parttwo - recognising emotions from recorded vibration samples.

ABSTRACTCCS CONCEPTS

Touch plays an essential role in communicating emotions and intensifying interpersonal communication[4]. A lot of research focuseson how to create or improve haptic interfaces looking into challenges and possibilities that the haptic technology can ofer[30].
The objective of this research is to investigate whether people canshare subjective feelings through simple vibrotactile feedback. In an
initial experiment, we used the TECHTILE toolkit[19] to record 28
vibration sample sets for 4 diferent emotions (joy, anger, sadness,relaxation). We then replayed the vibrations to test how well they
could be recognized. The results support the hypothesis that people
can use vibration feedback as a medium for expressing specifc
subjective feelings. It also indicates some universalities in afective
vibrotactile stimuli that even strangers with little to no knowledge
about the senders could recognize the emotional meanings.

• Human-centered computing → Human computer interaction (HCI); Haptic devices; User studies.

KEYWORDS
afective computing, emotion encoding, emotion recognition, emotion expression, haptic interfaces, vibration
ACM Reference Format:
Yulan Ju, Dingding Zheng, Danny Hynds, George Chernyshov, Kai Kunze,
and Kouta Minamizawa. 2021. Haptic Empathy: Conveying Emotional Meaning through Vibrotactile Feedback. In CHI Conference on Human Factors
in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts),
May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 7 pages.
https://doi.org/10.1145/3411763.3451640

1
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor proft or commercial advantage and that copies bear this notice and the full citationon the frst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specifc permission and/or afee. Request permissions from permissions@acm.org.
CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japan© 2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8095-9/21/05. . . $15.00
https://doi.org/10.1145/3411763.3451640

INTRODUCTION

Nonverbal information plays a crucial role in human-human social
interaction, often communicating emotional states between individuals. Touch, as one of the primary methods of human interaction,
has been widely studied and applied in human-computer interaction. Although research on functions of adding tactile qualities tocomputer applications extended from the cognitive function, mostof the current research has concentrated on the cognitive side. Our
work focused on haptic technology in afective communication.

CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japan

In general, haptic technology has contributed to afective communication in two main forms. First, some of the work focusedon the relationships between emotion elicitation and tactile touch
has shown that haptic stimulation can evoke emotional responses[25, 26]. Second, several research studies investigated how haptic
stimuli can intensify emotional information received through othermodalities [13]. As one of the most commonly used haptic feedback
methods, vibration has been widely used in this feld to enhanceother stimuli (e.g.adding vibrotactile activation to a visual representation) [5, 8, 12, 15, 25, 31]. To be more specifc, most of theresearch in haptic-based afective communication has focused on
the use of tactile interfaces, which are manifested as tangible objects or robots, wearable items, or contactless devices (i.e., mid-air
stimulation devices) [4].
Although vibration is mostly investigated as an auxiliary modality for communicating emotions, the study of using the single vibration modality to express feelings/emotions is not well explored[9, 21]. Some projects designed haptic expression icons to convey basic information like facial expressions [15]. Some researchers
have also worked on the taxonomy of vibrotactile stimulation. However, those studies’ vibrotactile stimuli were well designed in two
measurable quantities (intensity and frequency).
Based on the premise of individual diferences in haptic perception and preferences, we investigated whether unprocessed vibration samples can be used as a new communication method to convey
emotional information. To this end, we present an initial study design and some qualitative results to investigate 1) whether people
can use simple vibration feedback to express subjective feelingsand 2) whether people can identify emotions conveyed by simple
vibration feedback. To be more specifc, we have built a 28-set emotional vibration database for four emotions: joy, anger, sadness, andrelaxation. We have tested 196 emotional vibration trials in total
and analyzed each sample’s characteristics through their rhythmic
graphs. The results indicated the universality in afective vibrotactile stimuli in which strangers with little to no knowledge about
the sender could recognize the emotional meanings. Furthermore,
we propose a new direction in accessing universal haptic empathy,
which ofers the possibility for haptic-based afect detection.

2

RELATED WORK

Many psychologists believe emotions embrace both universalismand diferentialism[27]. On the one hand, emotion is a primary
human functioning mechanism that is relatively invariant acrossraces and cultures, just like perception, cognition, or learning. On
the other hand, as identifed by diferent language labels, emotions
are diferentiated from physiological symptoms, expressive behavior, motivation, and subjective feelings. Based on cross-culturalresearch by Kövecses, Z[14], speakers of a given language appear
to feel that some of the emotion words are more basic than others.
To be more specifc, happiness, sadness, anger, fear, and love are
known as fve general and possible universal categories of emotion
in 11 languages. In other words, these fve emotions can lead to aconsistent understanding of diferent language contexts.
In the feld of the universality of emotion, facial expressions
have a long and vital history in psychology. Many tests have beendeveloped to assess emotion recognition ability, and reliability and

Yulan and Dingding, et al.

validity have been well explored [6, 11, 16, 22, 23]. Among thoseresearch and tests, the ability to decode nonverbal cues from theface, body, and voice have been widely used. Despite the fact thattouch provides a compelling method of eliciting and modulating
emotion, applying haptic sensors for afect detection has been anunderstudied topic compared to facial and vocal displays. Matthew
and his colleagues noted that previous research on this subject did
not produce promising results[4]. However, these works opened apossibility of developing a multimodal system of afect detectionthat can be applied to several domains, such as people with autism
spectrum disorders who do not share emotions through typicalsocial cues.
Many studies discussed the efectiveness of using the haptic
channel to communicate afective information through direct andmediated methods [4]. Bailenson et al. investigated the possibility of
encoding emotional information through a haptic device [1]. Theirwork indicated that humans recognize emotions through haptic
modality at 33% above chance (1/4). Many research designed haptic
tools to explore the relationship between haptics and emotions. For
instance, Haptic Face Display (HFD) were designed to study vibrotactile pattern and elicited emotional responses [17]. Similarly, somestudies developed tactile icons to enrich user experience [30, 32].
The results of these tests indicated the possibility of employing a
sense of touch as an alternative way of communication. However,the design of the vibrotactile patterns was based on the authors’
own intuition, the ecological validity of the emotional qualitiesof the stimuli may be insufcient. POKE, a tactile-sharing device,
was designed for people with intimate relations to communicate
afections through tactile feedback and built their own tactile vocabularies [20]. Seif and her coworker created a 120-item databaseand compiled research on tactile language into fve taxonomies[28].
Although there is not yet agreement on an afective tactile design
language [10], those studies show that touch can augment or eventake the place of other emotional information cues.
Existing afective communication techniques have been widelyused in many areas such as virtual reality, smart surveillance, natural interfaces, enhanced human-robot interaction, gaming andentertainment, online social interaction, etc.[29]. The majority oftechniques use text, speech, gesture, and facial expression information to communicate afection. These techniques mainly augment two of the human senses in computer-mediated emotional
communication: visual and auditory[18]. Although touch plays apivotal role in human communication, research on communicating
afection by touch is still limited. Therefore, we have attempted
to explore more possibilities of applying the sense of touch tocomputer-mediated emotional communication.

3

METHOD

Our goal is to understand how an emotion can be transmitted between people using haptics. Firstly, we need to investigate whether
vibrations can be used as a medium for emotions. Secondly, toinvestigate what emotions can be conveyed.

3.1

Participants

Seven graduate school students (3 males and 4 females) between
ages of 24 to 30 years participated in our study. Three of them were

Haptic Empathy: Conveying Emotional Meaning through Vibrotactile Feedback

CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japan

3.3

Design and Procedure

There are two major parts to this study: using vibration patterns
to express certain emotions (see Fig.1.b) and recognizing emotions
from recorded vibrations (see Fig.1.c). Each participant was frstly
introduced to the “TECHTILE toolkit”. In each trial they were asked
to create a 10 second vibration pattern by any means (e.g. tapping,rubbing, pressing, etc.). The same 10 second recording was thenplayed back to them. The practice trials lasted for 3 minutes.

Figure 2: Russell’s circumplex model of afect[24]very familiar with the TECHTILE toolkit, one of them was mildly
familiar with TECHTILE, and three of them never tried TECHTILE
before. To remove emotional perception variations caused by cultural diferences, all participants were Chinese. All the participants
were students at Keio University’s Graduate School of Media Design.
No compensation was ofered.

3.2

Materials

3.2.1 Demographic Qestionnaire. A background information questionnaire was designed to collect participants’ biological information (sex, age and race) and general background of familiarity ofhaptic devices.
3.2.2 TECHTILE Toolkit (see Fig. 1.a) is a device capable of recording and playback of vibrotactile patterns [19]. This device consistsof a haptic recorder, a tactile display, and a signal transceiver. We
used it to collect tactile sensations created by the participants andplay the recordings back to them. Participants are able to share
their experiences through this “create-record-play” process.
3.2.3 Vibrotactile Emotions Data Base. All of the vibration trials
used in this study are recorded from the participants when they
were asked to express their emotions. More details are described in
3.3.1.
Russell’s dimensional circumplex model of afect is used to map
emotions in the quantitative two-dimensional space of valence andarousal (see Figure 2). This work focus on four emotions: joy, anger,sadness, and relaxation (one for each quadrant). The emotions fromthe same quadrant may be difcult to distinguish but the diference
increases for emotions from diferent quadrants. For instance, depressed and gloomy are close to sad, but distinct from angry. We
chose the four most basic emotions to avoid complications related
to the language barriers.
3.2.4 Dificulty Rating Survey. A self-report difculty rating survey
was designed to record how the participants felt about expressing
or recognising the four selected emotions by vibration. It used a 5point Likert scale from 1 (extremely easy) to 5 (extremely difcult).

3.3.1 Expression: Emotions to Vibrations . After the practice trials,
the participants were asked to try expressing four emotions (joy,anger, sadness, relaxation) within a 10 second recording. For eachtrial, the researcher would verbally tell the participant the designated emotion and then allowed the participants to record no morethan 10 seconds. After each recording, participants were allowed tocheck the recording and redo it if they deem it necessary. To reducethe impact of the emotion fuctuations generated by the previous
emotion expression on the latter sample, the interval between eachsample recording was 60 seconds. At the end of this session, we
asked participants how they felt about the task and whether theyused any strategies to express the emotions via the haptic device.
3.3.2 Recognition: Vibrations to Emotions. In the second session,
we asked the same participants to distinguish the 4 selected emotions conveyed by each vibration sample. To avoid hearing thesound of the vibration, participants were asked to wear noisecanceling headphones and listen to white noise during the wholeexperiment. There were 7 vibrotactile samples for each emotion
collected from the frst session, for 28 viboratactile sets in total(including the samples participants created by themselves). All thevibrotactile samples were presented in random order using the“TECHTILE toolkit”. After each trial, participants were asked torate from 1 to 5 how difcult it was to determine each emotion conveyed by vibration. The interval between presenting each sample
was 15 seconds. After all the trials, participants were asked to provide a further description of the whole experiment. We asked thesame question about how they felt about the task and whether theyused any strategies to distinguish the emotions. The researchers
explained the purpose of this study at the end.
For each trial, if the emotion the participant chose matchedwith the intended input emotion when the vibration sample wasrecorded, we considered the task to have an accurate result andmarked it as 1; if the participant failed to identify the vibration
sample representing the emotion type as the same as the intended
input, we considered the task to be a false result and mark it as 0.

4

RESULTSThe current study linked four diferent emotions expressed andrecognized through vibrations. One tail z-test was performed to test
whether the accuracy rate was signifcantly higher than random
results (1/4).
Among 49 test trials for each emotion (joy, anger, sadness, relaxation) conveyed through vibration, there were 35 matched result
for joy, 32 for anger, 25 for relaxation, and 23 for sadness (See Table2). Based on that, the accuracy rate of joy was the highest 71.43%,
followed by anger 65.30%, relaxation 51.02%, and sadness 46.94%. It
is interesting to note several anomalies in the data. E.g. anger was

CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japan

Yulan and Dingding, et al.

Table 1: Recognition of 4-Types-Emotion Vibrations Samples of 7 Participants

P1
P2
P3
P4
P5
P6
P7
Average

All Trials (28) i
Difculty Correct Accuracy
3.43
14
50.0%
2.79
15
53.6%
2.89
20
71.4%
2.18
18
64.3%
2.57
13
46.4%
2.46
20
71.4%
2.46
15
53.6%
2.68
16.22
57.91%Own (4) ii
Correct Accuracy
1
25.0%
3
75.0%
4
100.0%
2
50.0%
2
50.0%
4
100.0%
250.0%
2.57
64.30%

Others (24) iii
Correct Accuracy
13
54.2%
12
50.0%
16
66.7%
16
66.7%
11
45.8%
16
66.7%
13
54.2%
13.86
57.74%

Recognized by Others (24) ix
Correct
Accuracy
14
58.3%
13
54.2%
16
66.7%
15
62.5%
11
45.8%
17
70.8%
11
45.8%
13.86
57.74%i All Trials: All 28 emotional vibration fles created by these 7 participants ;ii Own : The emotional vibrations created by this participant;iii Others: The emotional vibrations created by others only– left out self input fles;
ix Recognized by others: the emotional vibrations created by this participant and recognized by others.

Table 2: Mean Accuracy and Difculty levels of the Recognition of 4-Types-Emotion Vibrotactile Emotion Samples

Emotion
Joy
Anger
Sad
Relax

Difculty
2.31
2.67
2.93
2.91

Total

2.68

All Vibration (n= 49)
Correct AR of All (M)
35
71.4
32
65.3
23
46.9
25
51.0
28.75

58.67

p-value
Correct< .00001 ***31< .00001 ***31
0.0002***17< .00001 ***22< .00001***

24.25

Others’ Vibration (n= 42)
AR of Reg. Oths (M) p-value
73.8< .00001***
73.8< .00001***40.5
0.0102*52.4< .00001***
57.74< .00001 ****p <.05, ***p <.001, Population SD =0.062
AR= Accuracy Rate, M= Mean, recognition accuracy values are in percent.
All Vibration: all 49 tested trials.
Others’ Vibration: 42 tested trials that excluded the self input ones.

sometimes confused with joy, however, joy was almost never confused with anger (see confusion matrix on Fig.3). The same holds
true for anger-sadness and sadness-relaxation pairs. There was one
participant who failed to choose any emotion for one of the anger
vibration trials. That result was removed from confusion matrix
since it was not mistaken for any other emotions, yet remained
as an incorrect result for the calculation. We can only speculate
about the underlying reasons at this point. Further investigation ofthis phenomenon is required. For 28 tested trials, the accuracy rate
for each participant ranged from 46.43% to 71.43% (Mean= 0.579;
SD=0.095). See Table 1.
Analysis showed the vibration pattern recognition was significantly higher than by chance. After performing z-test, all of theresults for four emotions are signifcant at p < .05 See Table 2.
These results are based on counting all 28 feedback sets from eachparticipant including the 4 emotional vibrations trials created bythemselves. Nevertheless, in the case of omitting self input emotional vibration trials, the accuracy rate did not difer much as
showed in Table 2 when n= 42. Performing z-test without including

the recognition of participants’ own input, we still found the recognition of all four emotional vibrations were signifcantly higher
than random results. All of the results are signifcant at p < .05.
Overall, because the p-value of all performed z-tests for four
emotion recognition samples were still less than .05, we reject thenull hypothesis. Therefore, we have sufcient evidence to say theemotions of anger, joy, sadness and relaxation can be expressed andrecognized by vibrations.
The average difculty levels perceived by the participants foreach emotion conveyed by vibration ranged from 2.31 to 2.93 (seeas Table 2). The sadness and relaxation vibration trials were rated as
more difcult to tell compared to anger and joy. This result matchwith the order of their accuracy rate for these four emotions. The
perceived difculty of recognition somewhat correlates with thecorrect recognition rate (R = 0.44, R square = 0.2, STDE = 0.601,
signifcance = 0.0169). This would imply that the vibration patterns
which were deemed confusing by the participants had a lower
correct recognition rate. This may seem obvious, but such a fnding
supports the validity of the gathered data.

Haptic Empathy: Conveying Emotional Meaning through Vibrotactile Feedback

Figure 3: Confusion matrix for vibration patterns recognition.

5

DISCUSSION

The early results of this experiment supported the hypothesis thatvibration can be utilized as a medium for expression and recognition of four selected emotions– joy, anger, sadness, and relaxation.
Some may think the general accuracy (46.9% - 71.4%) from current result is not high enough; yet, according to a work testingemotion recognition in face, voice, and body, the accuracy of these
traditional emotion cues also averagely ranged from 56% to 70%[2]. If the current accuracy rate remains with a large sample, itwould be interesting to explore how the haptic emotion cue can becomparable with those traditional emotion cues.
We also found joy and anger, compared with sadness and relaxation, were easier to both express and recognize through vibration.
This tendency is also consistent with other studies on traditionalemotion expression cues [2, 7]. The results also show that the vibrotactile emotion samples collected from people who are betterat recognizing emotions from vibrations also had a higher recognition rate by other participants. In other words, people who are
better at recognizing emotions are also better at expressing emotion through vibrations. Regression shows that the rate of correct
recognition by other participants correlates with the rate of therecognition rate of other participants’ patterns (R = 0.87, R squared= 0.76, STDE = 0.047, signifcance = 0.0108 ). This leads us to hypothesize the existence of some form of haptic empathy, or somedimension of emotional intelligence leading to this result. Again,
we have to recognize the low number of participants (n=7) and theensuing consequences. There are also some cases that individuals
who are poor at expressing one emotion through vibrations are relatively capable of expressing other emotions. There is one emotional
vibration sample created by one participant which received 0% accuracy in the study. The vibration fle 25-joy was not recognizedby any participants including the creator P7. However, created by
the same participant, the vibration fle 26-anger was recognizedby all 7 participants. The vibration fle 11-sad was not recognized

CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japanby anyone besides the creator P3 but all the other three emotional
vibrations had relatively high recognition feedback (between 6 to
7). It would be interesting to see how well these individuals areat expressing those particular emotions in general and compare
their performances with vibrotactile emotion feedback. A further
research study with a bigger sample size may be needed to conclude
more fndings.
We exported the recordings of the emotional vibrations to spectrograms and found that certain rhythmic characteristics wereassociated with specifc emotions. Due to limited space, we only
picked the highest recognition rate fle for each emotion in Fig. 4
as an example, yet the similar patterns are shared across the highly
recognized vibration fles:
1. Joy - rhythmic patterns are mostly consistent and prominent.
The rhythms tend to be more quantized than others.
2. Anger - rhythmic patterns are very prominent and constant.
These patterns are more driving and abstract than joy.
3. Sadness - rhythmic content is sparse and tends to come inshort patterns/gestures, with large sections of inactivity.
4. Relaxation - rhythmic content/density is quite sparse. The
amount of rhythmic content is nearly non-existent.
Analyzing waveforms and spectrograms, we deduced that joy
and anger share similarities in terms of rhythmic structure. These
patterns tended to consist of dense clusters of rhythms interspersedwith moments of inactivity. In joy, the rhythms were typically moreconsistent, whereas in anger the rhythms were noticeably more
abstract and disjointed. Much like the relation of joy and anger,sadness and relaxation share similarities in rhythmic content. With
these two emotions, the rhythmic content was quite sparse, withmuch of the waveforms containing large amounts of inactivity. In
sadness were small clusters of rhythmic content. In relaxation, there
were nearly no discernible rhythmic peaks. Apart from these twosimilar groups, we also found a connection between joy and relaxation in terms of rhythmic consistency. For example, the waveforms
shown for joy and relaxation show that the activity is occurring ina mostly repetitive or consistent manner. On the other hand, anger
and sadness were quite the opposite. These waveforms show rather
inconsistent and varied rhythmic content. Though the other highly
recognized sample fles shared the patterns, since the sample size
was small, more research will be necessary.
The relation of haptic interpretations of emotions was most identifable between joy and anger, as well as sadness and relaxation,
in regards to rhythmic density. In addition, the rhythmic variationof these waveforms showed connections between joy and relaxation, as well as anger and sadness. Further study will be needed
to determine what these connections fully imply and how theseconnections can be utilized for further studies.
In conclusion, the results suggest 1) people can share subjective
emotions through simple vibrotactile feedback; 2) people who are
better at recognizing emotions from vibrations are better at expressing emotions through a haptic device; 3) people have certain
common understandings about haptic interpretations of emotions.

6

FUTURE WORK

Though the early results of this work have shown a potential connection between emotions and vibrotactile feedback, there may be

CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japan

Yulan and Dingding, et al.

Figure 4: The Rhythmic Graphs of Four Emotional Vibrations from the database visualised with SonicVisualiser[3]. The top
images are the waveform and the bottom images are the spectrogram. (a) 9-Joy (AR=100%), (b) 26- Anger (AR=100%), (c) 3Sadness (AR=57.1%), (d) 24- Relaxation (AR=100%). AR= Accuracy Rate of the sample fle.
some possible limitations that could be addressed in future research.
For example, the recruited participants are all non-native English
speakers from the same country, potentially making the results only
applicable to this specifc group of people. The possible impacts ofgender, age, language contexts, and cultural backgrounds on this
topic are also worth exploring. In addition, some data we did notrecord this time could also be benefcial to include in future studies.
For instance, the response time of recognizing vibration samples
can also help for better understanding the relationship between
haptic empathy and emotion recognition ability. Furthermore, thelink between haptic empathy and emotional intelligence is worthyof further investigation. With smartphones all-pervading today,
vibrotactile haptic devices are accessible and afordable. Thus exploring haptic emotion research through vibrotactile patterns canlead to great potential of further applications.
Although the number of participants in this study is insufcientthus far, existing results have inspired us to acquire a larger sampleof participants in our future work. In March 2021, we are going tocollaborate with local schools and apply our experiment’s resultsin workshops with autistic children. We believe that this kind of
vibrotactile communication may help autistic children to share theiremotions more easily, and the “vibrolanguages" could even be a
fun way to enhance the understanding of emotions. It would be anice opportunity to explore whether or not using vibration stimulation can help autistic children scafold diferent representations
for emotion and encourage them to be creative and express theiremotions.

REFERENCES[1] Jeremy N Bailenson, Nick Yee, Scott Brave, Dan Merget, and David Koslow. 2007.
Virtual interpersonal touch: expressing and recognizing emotions through haptic
devices. Human–Computer Interaction 22, 3 (2007), 325–353.
[2] Tanja Bänziger, Didier Grandjean, and Klaus R Scherer. 2009. Emotion recognition
from expressions in face, voice, and body: the Multimodal Emotion Recognition
Test (MERT). Emotion 9, 5 (2009), 691.
[3] C. Cannam, C. Landone, and M. Sandler. 2010. Sonic Visualiser: An Open Source
Application for Viewing, Analysing, and Annotating Music Audio Files. In Proceedings of the ACM Multimedia 2010 International Conference. Firenze, Italy,
1467–1468.

[4] M. A. Eid and H. Al Osman. 2016. Afective Haptics: Current Research and Future
Directions. IEEE Access 4 (2016), 26–40. https://doi.org/10.1109/ACCESS.2015.
2497316[5] Haruna Fushimi, Daiya Kato, Youichi Kamiyama, Kazuya Yanagihara, Kouta
Minamizawa, and Kai Kunze. 2017. atmoSphere: designing cross-modal musicexperiences using spatial audio with haptic feedback. In ACM SIGGRAPH 2017
Emerging Technologies. 1–2.
[6] Matthew J Hertenstein, Rachel Holmes, Margaret McCullough, and Dacher Keltner. 2009. The communication of emotion via touch. Emotion 9, 4 (2009), 566.
[7] Holger Hofmann, Henrik Kessler, Tobias Eppel, Stefanie Rukavina, and Harald C
Traue. 2010. Expression intensity, gender and facial emotion recognition: Women
recognize only subtle facial emotions better than men. Acta psychologica 135, 3(2010), 278–283.
[8] Gijs Huisman and Aduén Darriba Frederiks. 2013. Towards tactile expressionsof emotion through mediated touch. In CHI’13 Extended Abstracts on Human
Factors in Computing Systems. 1575–1580.
[9] Ali Israr and Freddy Abnousi. 2018. Towards pleasant touch: vibrotactile grids
for social touch interactions. In Extended Abstracts of the 2018 CHI Conference on
Human Factors in Computing Systems. 1–6.
[10] Cathrine V. Jansson-Boyd. 2011. Touch matters: exploring the relationship between consumption and tactile interaction. Social Semiotics 21 (2011), 531 –
546.
[11] Laura Janusik. 2017. Profle of Nonverbal Sensitivity (PONS). 522–529. https://doi.org/10.1002/9781119102991.ch58[12] Pafan Julsaksrisakul, George Chernyshov, Masashi Nakatani, Benjamin Tag, andKai Kunze. 2017. Nene: An interactive pet device. In Proceedings of the 2017
ACM International Joint Conference on Pervasive and Ubiquitous Computing andProceedings of the 2017 ACM International Symposium on Wearable Computers.
89–92.
[13] Mark L Knapp, Judith A Hall, and Terrence G Horgan. 2013. Nonverbal communication in human interaction. Cengage Learning.
[14] Zoltán Kövecses. 2003. Metaphor and emotion: Language, culture, and body inhuman feeling. Cambridge University Press.
[15] Sreekar Krishna, Shantanu Bala, Troy McDaniel, Stephen McGuire, and Sethuraman Panchanathan. 2010. VibroGlove: an assistive technology aid for conveying
facial expressions. In CHI’10 Extended Abstracts on Human Factors in Computing
Systems. 3637–3642.
[16] David Matsumoto, Jef LeRoux, Carinda Wilson-Cohn, Jake Raroque, Kristie
Kooken, Paul Ekman, Nathan Yrizarry, Sherry Loewinger, Hideko Uchida, Albert
Yee, et al. 2000. A new test to measure emotion recognition ability: Matsumotoand Ekman’s Japanese and Caucasian Brief Afect Recognition Test (JACBART).
Journal of Nonverbal behavior 24, 3 (2000), 179–209.
[17] Troy McDaniel, Shantanu Bala, Jacob Rosenthal, Ramin Tadayon, Arash Tadayon,
and Sethuraman Panchanathan. 2014. Afective Haptics for Enhancing Access toSocial Interactions for Individuals Who are Blind. 419–429. https://doi.org/10.
1007/978-3-319-07437-5_40[18] Hongying Meng and Nadia Bianchi-Berthouze. 2013. Afective state level recognition in naturalistic facial and vocal expressions. IEEE Transactions on Cybernetics
44, 3 (2013), 315–328.
[19] Kouta Minamizawa, Yasuaki Kakehi, Masashi Nakatani, Soichiro Mihara, andSusumu Tachi. 2012. TECHTILE toolkit: a prototyping tool for design andHaptic Empathy: Conveying Emotional Meaning through Vibrotactile Feedback

education of haptic media. In Proceedings of the 2012 Virtual Reality International
Conference. 1–2.
[20] Young-Woo Park, Kyoung-Min Baek, and Tek-Jin Nam. 2013. The roles of touch
during phone conversations: long-distance couples’ use of POKE in their homes.
In Proceedings of the SIGCHI conference on human factors in computing systems.
1679–1688.
[21] Young-Woo Park, Sungjae Hwang, and Tek-Jin Nam. 2011. Poke: emotional touch
delivery through an infatable surface over interpersonal mobile communications.
In Proceedings of the 24th annual ACM symposium adjunct on User interface
software and technology. 61–62.
[22] Hallee Pitterman and Stephen Nowicki Jr. 2004. A test of the ability to identify
emotion in human standing and sitting postures: The Diagnostic Analysis of
Nonverbal Accuracy-2 Posture Test (DANVA2-POS). Genetic, Social, and General
Psychology Monographs 130, 2 (2004), 146–162.
[23] Ronald E Riggio. 2005. The Social Skills Inventory (SSI): measuring nonverbal andsocial skills. The sourcebook of nonverbal measures: Going beyond words (2005),
25–33.
[24] James A Russell. 1980. A circumplex model of afect. Journal of personality andsocial psychology 39, 6 (1980), 1161.
[25] Katri Salminen, Veikko Surakka, Jani Lylykangas, Jukka Raisamo, Rami Saarinen,
Roope Raisamo, Jussi Rantala, and Grigori Evreinov. 2008. Emotional and behavioral responses to haptic stimulation. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems. 1555–1562.

CHI ’21 Extended Abstracts, May 8–13, 2021, Yokohama, Japan[26] Suana Sanchez, Heng Gu, Kai Kunze, and Masahiko Inami. 2015. Multimodal
literacy: storytelling across senses. In Adjunct Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing and Proceedingsof the 2015 ACM international symposium on wearable computers. 1257–1260.
[27] Klaus R Scherer and Harald G Wallbott. 1994. Evidence for universality and cultural variation of diferential emotion response patterning. Journal of personality
and social psychology 66, 2 (1994), 310.
[28] H. Seif, K. Zhang, and K. E. MacLean. 2015. VibViz: Organizing, visualizing
and navigating vibration libraries. In 2015 IEEE World Haptics Conference (WHC).
254–259. https://doi.org/10.1109/WHC.2015.7177722[29] Jianhua Tao and Tieniu Tan. 2005. Afective computing: A review. In International
Conference on Afective computing and intelligent interaction. Springer, 981–995.
[30] Tampereen Yliopisto, Grigori Evreinov, Leena Vesterinen, Antti Nyman, Jalo
Kääminen, Jarno Jokinen, and Deepa Vasara. 2005. vSmileys: Imaging Emotions
through Vibration Patterns. (01 2005).
[31] Steve Yohanan, J Hall, K MacLean, E Croft, MV der Loos, Matthew Baumann,
Jonathan Chang, Dana Nielsen, and Susana Zoghbi. 2009. Afect-driven emotional
expression with the haptic creature. Proceedings of UIST, User Interface Software
and Technology (2009), 2.
[32] Yongjae Yoo, Taekbeom Yoo, Jihyun Kong, and Seungmoon Choi. 2015. Emotional
responses of tactile icons: Efects of amplitude, frequency, duration, and envelope.
In 2015 IEEE World Haptics Conference (WHC). IEEE, 235–240.