International Journal
Transactionsof JapanofSociety
Affectiveof Engineering
Kansei Engineering
J-STAGE Advance Published Date: 2023.03.08
doi: 10.5057/ijae.IJAE-D-22-00024

ORIGINAL ARTICLE

Timing of Vibratory Stimuli to the Upper Body for Enhancing Fear andExcitement of Audio-visual Content
Ibuki TARA*, Shogo OKAMOTO*, **, Yasuhiro AKIYAMA*** and Hidetaka OZEKI** Nagoya University, Furo-cho, Chikusa-ku, Nagoya-shi, Aichi 464-8603, Japan** Tokyo Metropolitan University, 6-6 Asahigaoka, Hino-shi, Tokyo 191-0065, Japan*** Shinshu University, 3-15-1, Tokida, Ueda-shi, Nagano 386-0018, Japan

Abstract: Vibratory stimuli to viewers’ upper bodies enhance emotional experiences during audio-visual content. Studies haveinvestigated the effects of stimulation conditions, such as vibration waveform and spatiotemporal patterns. We investigated how thetiming of vibratory stimuli affect viewers’ skin conductance responses and subjective reports of audio-visual experiences. Vibratory
stimuli were administered to the viewers’ torso at various timings around a monster’s sudden appearance in horror videos, and jump
performances in figure skating videos. For horror videos, vibratory stimuli elicited fear. The fear of horror videos was maximized whenthe vibratory stimuli started simultaneously with the monster’s first appearance. For the jump performance scenes in figure skating
videos, although the most exciting timing varied by participant, the vibration should be intensified as performers launch off the ice.
These findings will help develop entertaining audio-visual content.
Keywords: Vibratory stimuli, Emotion, Audio-visual contents

1. INTRODUCTION
Vibratory stimuli are used to enhance the experiences ofaudio-visual content [1-12]. For example, Lemmens et al.
[1] and Karafotias et al. [2] presented vibratory stimuli toviewers’ upper bodies during several videos using jackets
with vibrators installed and enhanced viewers’ subjective
ratings for the quality of experiences such as immersion
and realism. In addition, Branje et al. [8] reported anincrease in average skin conductance levels whilst watching horror videos when converting their soundtracks tovibratory signals and delivering them to viewers’ wholebodies.
Most earlier studies reported that the vibratory stimuli
increased the subjective rating of audio-visual content. In
contrast, the physiological responses did not necessarily
increase by vibratory stimuli and depended on content
genre [1, 6, 9]. For example, Lemmens et al. investigated
how vibratory stimuli changed subjective ratings andphysiological responses for various genres of emotionallyevocative videos [1]. In their study, irrespective of theemotions evoked by the videos, subjective scores
increased due to vibratory stimuli. However, the effectsof the vibration on the physiological responses were only
apparent in videos that evoked negative emotions such asanger and anxiety.
Vibratory stimuli are composed of various characteristics, such as vibration waveform, body parts to stimulate,and spatiotemporal patterns of vibrators. Earlier studies
Received: 2022.08.15 / Accepted: 2023.02.02

investigated how these characteristics affect users’ emotions
or feelings [2, 13-22]. For example, Okamoto et al. [22]reported that a certain vibratory frequency on one’s backinduced a pleasant feeling. Further, vibration timing can
be one of the design parameters because the temporalconsistency between multimodal stimuli is a determinantof sensory integration [23, 24]. For example, several tensof millisecond of temporal gap can be recognizable for
haptic cues [25]. However, this parameter has not been
investigated in earlier studies, in which they matched thecommencement of the vibration to emotional scenes partly
because most studies utilized the soundtracks or salient
visual motions in the videos to determine the timing of
the vibratory cues [2-4, 6-8, 10, 12]. Hence, earlier studies
do not inform us of the effects of the delay or lead invibration timing. When the vibration is appropriately
timed with the audio-visual content, the vibratory stimuli
are associated with events in the videos. However, if thevibration timing dramatically deviates from the audiovisual content, the experience may be tainted because theviewers may not associate the vibratory stimuli with thescene. Therefore, vibration timing is an essential element
for enhancing the experiences of the audio-visual content.
Thus, to enhance emotional experiences, it will be helpful
to understand good timing and an acceptable range ofdeviations in vibration timing.
This study investigated the effects of the vibration
timing on viewers’ physiological response and subjective
rating of audio-visual content and determined the best
Copyright © 2022 Japan Society of Kansei Engineering.
All Rights Reserved.

Advance Publication

timing. In the experiment, the vibratory stimuli areadministered to viewers’ upper bodies in context withemotionally evocative scenes in horror and figure skating
videos.
This study extends our previous study [26] that investigated how the vibration timing affected the viewers’
excitement to figure skating videos. This study also
targets fear in addition to excitement of sports scenes
which are both arousing emotions. The former has anegative valence, and the latter has a positive or neutral
one. Multiple emotions such as fear and excitement
should be investigated to confirm the generality of thestudy. Furthermore, our previous study [26] failed to
discuss individual video differences. The effects of theirpotential differences should be considered to conclude
the effects of vibration on emotional experiences.
This study was approved by the Institutional Review
Board, Hino Campus, Tokyo Metropolitan University(#21-46).
2. METHODS
2.1 Apparatus
The audio-visual content was played using a signage
player (BS/HD220, BrightSign, Inc., United States) that
controls the video in response to external digital signals.
The signage player had a 21.5-inch monitor with an aspectratio of 16:9 and headphones for video and audio output.
The vibratory stimuli were presented by voice coilmotors (Vp408, Acouve Laboratory, Inc., Japan). As
shown in Figure 1, the vibrators were attached firmly tothe left and right sides of the shoulders, chest, abdomen,
and waist of participants using a strap vest. All the voicecoil motors were actuated synchronously.
Trigger signals to the signage player and voice coil
actuators were synchronized using a microcomputer(mbed NXP LPC1768, ARM Ltd., England). The microcomputer output the vibration waveform signals to the

Figure 1: Vibration vest (adapted from [26])Vibrators were attached firmly to the left and right sidesof shoulders, chest, abdomen, and waist of the vest.

vibrators at the predetermined time for each video.
The vibratory signals were then amplified by an audio
amplifier (FX-2020A+, NFJ North Flat Japan Ltd., Japan)
and input to voice coil actuators. The control frequency ofthe vibration waveform was 1000 Hz.
We measured the skin conductance responses [27] ofviewers’ hands when watching horror or figure skating
videos. Skin conductance has been used to indicate
emotional arousal, including feelings of fear and excitement [27-31]. Furthermore, the skin conductancemeasurement is suitable for evaluating the viewers’
responses to instant stimuli such as monster’s appearance
scenes and jump performance scenes because it starts
rising after the stimulation within 1–2 seconds. We useda measurement unit (AP-U030m II, Nihon Santeku Ltd.,
Japan) and an amplifier (MaP1720CA, Nihon Santeku
Ltd., Japan) to record the skin conductance response. For
this measurement, electrodes were attached to the second
and third fingers of the viewers’ left hand. The signalof skin conductance response was recorded using anoscilloscope (TDS2004C, Tektronics, Inc., United States)
at 1000 Hz.
2.2 Audio-visual content
Participants watched five horror videos and six figureskating videos. Each horror video was about two to threeminutes long and included a scene where a monster
suddenly appeared. The titles of the horror videos wereBlackout, Chain letter, Fear filter, Splat, and Cam closer.
Each figure skating video was approximately one minutelong and was of different male performers. We selected
two performances from the PyeongChang Olympics 2018,
three from the Audi Cup of China 2016, and one from theWorld Figure Skating Championships 2015. Each video
started with the beginning of the performance and ended
after the first jump performance was completed. The
monster’s appearance and jump performance scenes, ofwhich moments of beginning could be clearly defined,
were used as audio-visual stimuli to evoke viewers’ fearand excitement.
2.3 Vibratory stimuli
1) Preliminary experiment to determine the vibratory
stimuli
We designed different vibratory stimuli for horror videos
and figure skating videos. These patterns were determinedby a preliminary experiment with five participants, excluding the authors, where effective vibratory frequency,
amplitude profiles, and strength were set through participant agreement. Earlier studies demonstrated that theseAdvance Publication
Timing of Vibratory Stimuli to the Upper Body for Enhancing Fear and Excitement of Audio-visual Content

properties affect feelings [13, 32-34]. The parameters andmodulation methods were varied, that is, amplitude or
frequency modulation. A set of vibratory stimuli wasprepared and based on the results of participant voting foreach horror and figure skating video, stimuli for the main
experiment were determined.
We modulated the amplitude of the sinusoidal waves
at a constant frequency of either 20 Hz, 50 Hz, 70 Hz,
and 110 Hz. All the participants agreed that the sounds
associated with the vibration at the frequencies greaterthan 110Hz deteriorated the video contents. Hence,
frequencies higher than 110 Hz were not used in the
preliminary experiment. The waveforms of the amplitude
were either square, triangular, sawtooth, or reverse
sawtooth waves. Furthermore, the vibration intensity wascontrolled at two levels, that is, weak or strong vibration.
To reduce the number of stimuli to be tested, we used across table with frequency, waveform, and intensity as
parameters, and then designed and compared sixteen typesof vibratory stimuli. Similarly, we applied frequencymodulated vibration. Frequencies linearly changed from
20 Hz to 110 Hz and from 110 Hz to 20 Hz. Two intensity levels were applied to these frequency-modulated
stimuli. In total, suitable stimuli were selected for the main
experiment from 20 vibratory stimuli (16 amplitudemodulated + 4 frequency-modulated) through voting by
five participants. For each of horror and figure-skating
videos, each participant voted for three vibratory stimuli
that would enhance the emotional experiences whilelooking at videos. For each type of videos, one vibratory
stimulus gained four votes, of which features aredescribed in Section 2.3 2). Note that through theseprocesses, we did not seek optimal vibrations regarding
emotional effects; however, we intended to select reasonably effective vibrations.
2) Vibratory stimuli adopted for the main experiment
For the horror videos, we adopted a frequency-modulated
vibration, of which frequency was decreased linearly
from 110 Hz to 20 Hz over a second. The voltage amplitude to the voice coils was constant. The maximum
acceleration of the vibrators’ surface was approximately
150 m/s2 when the vibration frequency was 110 Hz and7 m/s2 for 20 Hz.
We adopted an amplitude-modulated vibration of 70 Hz
for the figure skating videos. The amplitude was modulated
by a triangular wave, increasing linearly from 0 over
640 ms after onset and decreasing to 0 for another
640 ms. The maximum acceleration of the vibrators’
surface was 60 m/s2 when the vibration amplitude was at
its peak.

3) Preliminary experiment to determine presentation
timings of vibratory stimuli
The levels of the vibration timing were determined byanother preliminary experiment involving the same five
participants. At this stage, the vibration was presented incontext with several horror or figure skating videos atseveral timings while shifting forward or backward theonset of the vibration by 100 ms from the reference
timing. For the reference timing of the horror videos, thevibration started simultaneously with the monster’s
appearance. We determined the reference timing of the
figure skating videos, where the peak of the vibration
coincided with the performer’s take-off from the ground.
We determined the upper and lower limits, that is, delay
and lead, of the vibration timings that the participants
could barely associate the vibration with the monster’s
appearance or jump performance scenes. The intervals
between the upper and lower limits were set such that fouror all five participants could marginally recognize whichof the two neighboring timings was earlier.
4) Presentation timings adopted for the main experiment
For the horror videos, five conditions were adopted. The
vibratory stimulus was not administered in one condition.
In the other four conditions, the vibrations were differently timed. As shown in Figure 2 (a), for the reference
timing, the vibration started simultaneously with the
instance of the monster’s appearance. In two conditions,vibration onset was shifted to 400 ms earlier or later than
the reference. In the last condition, the onset was 700 ms
later than the reference.
(a) Vibration timings for horror videos(b) Vibration timings for figure skating videos

Figure 2: (a) Four vibration timings for horror videos.
At the reference time point, a monster suddenly appears.
(b) Five vibration timings for figure skating videos (adapted from[26]). At the reference timing, the peak of the vibration amplitude
coincides with the performer’s take-off from the ground.

Advance Publication

For the figure skating videos, six conditions were adopted.
In one condition, the vibration was not presented, whereas
in another, the vibration was presented so that its peak
matched the performer’s take-off. As shown in Figure 2 (b),the other four vibration timings included 200 ms or 400 ms
earlier and 200 ms or 400 ms later than the reference
timing.
2.4 Participants
Eighteen (eight women) and twenty-four (twelve
women) university students (mean and standard deviationof ages: 22.0 ± 1.5) participated in the experiment of thehorror and figure skating videos, respectively, after
providing written informed consent. Nine participants
watched both the horror and figure skating videos, andthe other participants watched either of them. They had
not seen any of the videos used in the experiment andwere not aware of the objective.
2.5 Procedures
The horror and figure skating video trials wereconducted separately at intervals of at least 30 min.
For those who participated in both types of trials, thevideos’ order, that is, horror or figure skating videos, wasrandomized.
The participant wore vibration-presenting vests, skinconductance sensors, and headphones, and was seated ina chair during the experiment. The skin conductanceelectrodes were attached after his/her fingers were wipedwith an alcohol sheet and dried. The participant wasadministered the vibratory stimulus thrice before theexperiment began to avoid his/her reactions toward an
unfamiliar upper body vibration.
In the experimental trials, the participant watchedvideos under several conditions of the vibratory stimulus.
In the horror video trials, the vibratory stimulus waspresented for four of the five videos with different
timings. Similarly, in the figure skating video trials,
the vibration was presented in five of the six videos.
The order and combination of video and vibration
conditions were randomized for each participant.
Before watching each video, the participant relaxed
for at least 3 min. The video was started again only
after the skin conductance responses were visuallyjudged as stable.
After watching each video, the participant rated theirfear levels in response to the monster’s appearance or
excitement levels in reaction to the skaters’ jump performances. Participants used a 9-point graded scale to ratelevels of fear and excitement, respectively, experienced

Figure 3: Change in the skin conductance response to a stimulus
The increase in skin conductance response from the commencementof the rising to its peak was used for the analysis.

at the monster’s appearance during the horror video andthe jump performance during the figure skating video.
The scale’s two extremities were labeled as only a little
excited/scared to very excited/scared. The participants
watched either five horror or six figure skating videos ina row. Then, they were encouraged to make the firstvideo of each genre a reference and comparatively rate
the succeeding videos.
2.6 Analysis
The skin conductance value starts rising 1–2 seconds
after the stimulation and peaks within 1–2 seconds, andtapers after the peak, as shown in Figure 3. We analyzed
the increase in the skin conductance response from thecommencement of the increase to the peak, which is
generally called an amplitude.
We investigated the effects of vibratory stimuli and theirtiming on skin conductance amplitudes and questionnairescores. These values were standardized (z-scored) for
each participant. Six participants in the horror video trials
and five participants in the figure skating trials were
excluded from the skin conductance amplitude analysis
because they exhibited no response to vibratory andaudio-visual stimuli potentially because of their skin andphysiological conditions.
The main interest of the present study is the effects ofthe timing of vibratory stimuli. However, different videos
evoked varying levels of fear or excitement. Hence, thevideos’ effects were explicitly considered. We used
two-way ANOVA with the vibration timing and video
being two factors to investigate their effects on the skinconductance and questionnaire scores. The statisticalanalyses were conducted in Matlab (2021b, MathWorks,
United States).

Advance Publication
Timing of Vibratory Stimuli to the Upper Body for Enhancing Fear and Excitement of Audio-visual Content

3. RESULTS
3.1 Results for horror videos
For the horror videos, Table 1 summarizes the results of
two-way ANOVA. As listed in Table 1 (a) and (b), theTable 1: Summary of two-way ANOVA for horror videos(a) Amplitude of skin conductance response
Vibration timing

d.o.f.

F

p

4

5.15

2.3 × 10−3

Video

4

1.20

0.33

Vibration timing × video

16

0.51

0.93

Error

35

(b) Subjective feare
Vibration timing

d.o.f.

F

p

4

6.90

1.0 × 10−4

Video

4

2.34

0.065

Vibration timing × video

16

0.61

0.86

Error

65

(a) Normalized skin conductance amplitudes(b) Normalized questionnaire scores for fear

Figure 4: Effects of vibration timings when horror videos
were watched
Error bars are standard errors.
*, **, and *** indicate significance levels of p < 0.05, 0.01, and 0.001,
respectively, with Bonferroni correction of factor 10.

vibration timing exhibited a significant effect on the skinconductance amplitudes and subjective fear, respectively,
whereas the effects of videos and interaction were insignificant. The levels of fear were not very different between
the five videos used in the experiment.
Figure 4 shows the means and standard errors of theskin conductance amplitudes and subjective fears by thevibration timing. As post-hoc tests, the amplitudes andsubjective fears were compared between all the pairs oftiming conditions including no-vibration condition byt-tests with Bonferroni correction of factor 10 (5C2).
As shown in Figure 4 (a), for all the vibratory conditions, the skin conductance amplitudes were greater thanthat of no-vibration condition indicating that thevibratory stimuli led to the increases of physiologicalactivities irrespective of the temporal gap with themonster-appearance scene.
As shown in Figure 4 (b), the subjective fear was thehighest when the vibration ignited with the monster’s
appearance. The fear levels for no-delay condition (0-mscondition) and 400-ms condition were greater than thatfor no-vibration condition.
3.2 Results for figure-skating videos
Table 2 shows the results of two-way ANOVA for figureskating videos. For the amplitude of skin conductance
response and questionnaire score, the vibration timing
was the unique significant factor. The effects of vides
were not significant, and the interaction between thevibration timing and video was also insignificant.
Figure 5 (a) and (b) show the means and standard
errors of the normalized amplitudes of skin conductanceand subjective excitement scores, respectively, by timing
conditions. For each pair of the different timing conditions and no-vibration condition, t-test was performed
with Bonferroni correction of factor 15 (6C2).
Significant differences in the skin conductance wereobserved between the no-vibration condition and allconditions with the vibration, which indicates thatthe vibratory stimuli to the upper body evoked physiological activities. In contrast, no differences were seen
between any vibration conditions with different vibration timings.
Regarding the questionnaire scores of excitements,
no-vibration condition and −400-ms condition exhibited
smaller values than the other conditions with vibratory
stimuli. No significant differences were seen between
0-ms, 200-ms, and 400-ms conditions. For these vibratory
conditions with zero or greater delays, the questionnairescores were equally large.

Advance Publication

Table 2: Summary of two-way ANOVA for figure-skating videos

4. DISCUSSION(a) Amplitude of skin conductance response
d.o.f.

Fp

Vibration timing

5

4.82

7.0 × 10−4

Video

5

2.15

0.068

Vibration timing × video

25

0.97

0.52

Error

78

d.o.f.

Fp

Vibration timing

5

11.4

7.6 × 10−9

Video

5

0.28

0.80

Vibration timing × video

25

0.34

0.94

Error

108

(b) Subjective excitement(a) Normalized skin conductance amplitudes(b) Normalized questionnaire scores for excitement

Figure 5: Effects of vibration timings when figure-skating
videos were watched
Error bars are standard errors.
*, **, and *** indicate significance levels of p < 0.05,0.01, and 0.001,
respectively, with Bonferroni correction of factor 15.

Regarding the questionnaire scores, majority of thevibration conditions led to higher scores than theno-vibration condition for both the horror and figureskating videos. Upper body vibrations increased fear
reactions to the monster’s appearance and excitement
responses when the skaters performed technically difficult
jumps. Previous studies have demonstrated that vibratory
stimuli enhance emotional experiences and improve theimmersive quality of audio-visual content [1-12, 18, 32-34].
However, earlier research has not examined basic
emotions such as fear and excitement. These studies
primarily investigated arousal and valence and found thatvibratory stimuli increase arousal. Fear and excitement
are classified as arousing emotions [35], and our results
agree with these early studies. Hence, it can be concluded
that upper body vibrations impact fear and excitement.
Participants reported the strongest fear reaction for
the reference vibration timing, as shown in Figure 4 (b).
The fear was the strongest in horror videos when thevibratory stimulus was administered simultaneously withthe monster’s appearance. The vibration effects became
weaker if the timing was shifted by 400 ms. According
to introspective reports, when the vibratory stimulus
was administered earlier than the reference timing,
participants were surprised at the sudden, unexpected
stimulus timing; however, the experience of fear was not
influenced by the vibration because of their temporal
gap. They also reported that the fear was not heightenedwhen the vibration occurred later than the reference
timing.
As shown in Figure 5 (b), most participants reported
that vibration timing at the reference point or later largely
matched the jump performance scenes in figure skating
videos. Perhaps the most exciting moment in the figureskating jump performance depended mainly on theviewers’ perspectives. Some participants reported that the
instance when the performer was in the air was the most
exciting. Others stated that the performer’s landing,occurring later than all other experimental timing conditions, was most exciting. In contrast, few participants
rated timings earlier than the reference point as most
exciting. According to participant introspective reports,
the vibration started too early for the jump performancescenes when the timing was shifted by 400 ms earlier than
the reference point.
One area remaining to be clarified concerns the lackof agreement between various vibration timing effects
on skin conductance amplitudes and subjective ratings,

Advance Publication
Timing of Vibratory Stimuli to the Upper Body for Enhancing Fear and Excitement of Audio-visual Content

especially for the horror videos. For these videos, thecorrelation coefficient between the standardized skinconductance amplitudes and subjective fear levels was
0.43 (t = 3.63, p = 3.00 × 10−4) indicating that the relationships were moderately or weakly consistent. The −400-ms
timing was significantly higher than the no-vibration
condition for the skin conductance responses. In contrast,
as shown in Figure 4 (b), the no-vibration and −400-ms
timing conditions were not significantly different for
subjective fear. This inconsistency between the skinconductance and subjective fear can be explained by the
introspective reports of the participants as aforementioned. They commented that the −400-ms timing startled
them because the vibration was unexpectedly earlier thanthe horrific scene. However, they recognized that such
early vibration did not temporarily match the monster’s
appearance in the video. For the figure skating videos, as
shown in Figure 5, all vibration timing effects on skinconductance were significant; however, the −400-ms
timing did not significantly differ from the no-vibration
condition regarding the subjective excitement. Hence, forthe −400-ms timing, the questionnaire and physiologicalresponses mismatched. Nonetheless, in general, thestandardized skin conductance amplitudes and subjective
excitement levels were moderately consistent with thecorrelation coefficient of 0.65 (t = 9.05, p < 2.22 × 10−15).
We demonstrated the emotional effects of the upperbody vibratory stimulation synchronized with scenes ofhorror or excitement. However, some limitations remain
unaddressed. Regarding the figure skating videos, wefound that the moments judged most exciting differedamong participants, based on their introspective reports.
Figure skaters’ jump performance scenes have someunique qualities that may account for this variation.
Hence, this study’s results related to the figure skating
videos may not be directly applicable to other sports
videos. We adopted a within-subjects design for theexperiment where participants experienced all videos andvibration conditions. Participants could familiarize themselves with the horror and figure skating videos andvibratory stimuli. Between-subjects designs are ideal foravoiding habituation effects. However, it is difficult tocompare skin conductance amplitudes among different
participants because of substantial individual differences.
Furthermore, skin conductance measurement largely
depends on electrode contact conditions. Although thisstudy addressed fear and excitement, in general, thetypes of emotion that can be influenced by upper-bodyvibration have not been firmly established. In earlier andcurrent studies, some successes have been reported.

However, only a few studies producing either negative
results or those where emotions could be only minimallyaltered have been reported. Furthermore, the optimal
vibratory stimuli to influence emotions still remain
unclear. Nonetheless, we established the effects of thevibration timing, that is, one of the many conditions, as
an important condition for successfully influencing
emotions.
In earlier studies, the mechanism for the upper body
vibration to influence the emotional experiences withaudio-visual content has not been thoroughly discussed.
We speculate that the vibratory stimulation of the visceralsensation affects the changes in emotional experiences.
Visceral or interoceptive stimuli affect emotions [36-39].
Changes in interoceptive sensations caused by mechanical vibration to visceral organs may be attributed to theemotional content of videos. A similar hypothesis was
also considered in [39, 40].
5. CONCLUSION
We investigated vibratory stimuli effect when administered to the upper bodies on skin conductance amplitude
and subjective emotional experience scores in reaction
to horror and figure skating videos. The rated fear andexcitement scores in response to the horror and skating
videos, and the skin conductance responses became larger
when the vibratory stimuli were administered in the
expected context at the peak point of fear or excitement.
The fear response to horror videos was maximized whenthe vibration started simultaneously with the beginning ofstartling scenes. If the vibration timing deviated by
400 ms or more from this timing, the fear experience was
not enhanced. Although the most exciting vibration timing
for the figure skating jump performance video scenes
depended on viewers’ perspectives, based on our results,
the vibration should be intensified after the skaterlaunches off the ice. This study’s findings will be
useful for enriching immersive emotional experiencesin response to audio-visual content. Once the designers
find the best vibration timing, a temporal shift of a few
hundred milliseconds must be avoided. Such deviations
spoil the viewers’ emotional experiences. However,deviations of a few tens of milliseconds might beacceptable. One future research challenge will be to
examine the use of videos with prolonged emotional
scenes, unlike those used in this study. Long-lasting
romantic scenes and rally scenes in tennis games areexamples that might be addressed in the future.

Advance Publication

REFERENCES
1. Lemmens, P., Crompvoets, F., Brokken, D., van den
Eerenbeemd, J., and de Vries, G.-J.; A body-conforming
tactile jacket to enrich movie viewing, IEEE World
Haptics 2009-Third Joint EuroHaptics Conference
and Symposium on Haptic Interfaces for Virtual
Environment and Teleoperator Systems, pp.7-12,
2009.
2. Karafotias, G., Teranishi, A., Korres, G., Eyssel, F.,
Copti, S., and Eid, M.; Intensifying emotional reactions via tactile gestures in immersive films, ACM
Transactions on Multimedia Computing, Communications, and Applications, 13(3), pp.1-17, 2017.
3. Kim, M., Lee, S., and Choi, S.; Saliency-driven realtime video-to-tactile translation, IEEE Transactions
on Haptics, 7(3), pp.394-404, 2013.
4. García-Valle, G., Ferre, M., Breñosa, J., and Vargas, D.;
Evaluation of presence in virtual environments:
Haptic vest and user’s haptic skills, IEEE Access, 6,
pp.7224-7233, 2017.
5. Ueoka, R., and Ishigaki, K.; Development of thehorror emotion amplification system by means of
biofeedback method, International Conference on
Human Interface and the Management of Information,
pp.657-665, 2015.
6. Krogmeier, C., Mousas, C., and Whittinghill, D.;
Human-virtual character interaction: Toward understanding the influence of haptic feedback, Computer
Animation and Virtual Worlds, 30(3-4), e1883, 2019.
7. Nanayakkara, S., Taylor, E., Wyse, L., and Ong, S. H.;
An enhanced musical experience for the deaf: Designand evaluation of a music display and a haptic chair,
SIGCHI Conference on Human Factors in Computing
Systems, pp.337-346, 2009.
8. Branje, C., Nespoil, G., Russo, F., and Fels, D. I.; The
effect of vibrotactile stimulation on the emotional
response to horror films, Computers in Entertainment,
11(1), 4, 2014.
9. Tajadura-Jiménez, A., Väljamäe, A., and Västfjäll, D.;
Self representation in mediated environments: Theexperience of emotions modulated by auditoryvibrotactile heartbeat, CyberPsychology & Behavior,
11(1), pp.33-38, 2008.
10. Fröhlich, J., and Wachsmuth, I.; The visual, the auditory
and the haptic – A user study on combining modalities
in virtual worlds, International Conference on Virtual,
Augmented and Mixed Reality, pp.159-168, 2013.
11. Sakurai, S., Katsumura, T., Narumi, T., Tanikawa, T.,
and Hirose, M.; Evoking emotions in a story usingtactile sensations as pseudo-body responses with

contextual cues, International Conference on Human
Interface and the Management of Information, pp.241250, 2014.
12. Tara, I., Okamoto, S., Akiyama, Y., and Yamada, Y.;
Heightening fear response to horror video scenes byapplying vibratory stimuli to upper bodies of viewers,
IEEE Global Conference on Life Sciences and Technologies, pp.254-256, 2022.
13. Hasegawa, H., Okamoto, S., Ito, K., and Yamada, Y.;
Affective vibrotactile stimuli: Relation between
vibrotactile parameters and affective responses,
International Journal of Affective Engineering, 18(4),pp.171-180, 2019.
14. Arafsha, F., Alam, K. M., and El Saddik, A.;
Emojacket: Consumer centric wearable affective jacket
to enhance emotional immersion, IEEE International
Conference on Innovations in Information Technology,
pp.350-355, 2012.
15. Ur Réhman, S., Khan, M. S. L., Li, L., and Li, H.;
Vibrotactile TV for immersive experience, Signal andInformation Processing Association Annual Summit
and Conference, pp.1-4, 2014.
16. Kim, Y., Cha, J., Ryu, J., and Oakley, I.; A tactile
glove design and authoring system for immersive
multimedia, IEEE Multimedia, 17(3), pp.34-45, 2010.
17. Bassel, C., and Schiff, B. B.; Unilateral vibrotactile
stimulation induces emotional biases in cognition andperformance, Neuropsychologia, 39(3), pp.282-287,
2001.
18. Chandra, Y., Tag, B., Peiris, R. L., and Minamizawa, K.;
Preliminary investigation of across-body vibrotactile
pattern for the design of affective furniture, IEEE
Haptics Symposium, pp.671-676, 2020.
19. Mazzoni, A., and Bryan-Kinns, N.; Mood glove: A
haptic wearable prototype system to enhance moodmusic in film, Entertainment Computing, 17, pp.9-17,
2016.
20. Raisamo, J., Raisamo, R., and Surakka, V.; Evaluating the effect of temporal parameters for vibrotactile
saltatory patterns, International Conference on Multimodal Interfaces, pp.319-326, 2009.
21. Kelling, C., Pitaro, D., and Rantala, J.; Good vibes:
The impact of haptic patterns on stress levels, International Academic Mindtrek Conference, pp.130-136,
2016.
22. Okamoto, S., Tamada, A., Sumiya, T., and Yokoi, S.;
Design of pleasant vibrotactile stimulation through aseat, IEEE Global Conference on Consumer Electronics, pp.54-55, 2017.
23. Shimada, S., Fukuda, K., and Hiraki, K.; Rubber handAdvance Publication
Timing of Vibratory Stimuli to the Upper Body for Enhancing Fear and Excitement of Audio-visual Content

illusion under delayed visual feedback, Plos One,
4(7), e6185, 2009.
24. Fujisaki, W., Shimojo, S., Kashino, M., and Nishida, S.;
Recalibration of audiovisual simultaneity, Nature
Neuroscience, 7(7), pp.773-778, 2004.
25. Okamoto, S., Konyo, M., Saga, S., and Tadokoro, S.;
Identification of cutaneous detection thresholds
against time-delay stimuli for tactile displays, IEEE
International Conference on Robotics and Automation, pp.220-226, 2008.
26. Tara, I., Okamoto, S., Akiyama, Y., and Yamada, Y.;
How important is the timing of vibrotactile stimuli
for the excitation of an emotional response foraudio-visual content?, IEEE Global Conference on
Consumer Electronics, pp.157-159, 2021.
27. Boucsein, W.; Electrodermal activity, Springer
Science & Business Media, 2012.
28. Egger, M., Ley, M., and Hanke, S.; Emotion recognition from physiological signal analysis: A review,
Electronic Notes in Theoretical Computer Science,
343, pp.35-55, 2019.
29. Baig, M. Z., and Kavakli, M.; A survey on psychophysiological analysis & measurement methods in
multimodal systems, Multimodal Technologies andInteraction, 3(2), 37, 2019.
30. Nikula, R.; Psychological correlates of nonspecificskin conductance responses, Psychophysiology, 28(1),pp.86-90, 1991.
31. Öhman, A., and Mineka, S.; Fears, phobias, andpreparedness: Toward an evolved module of fear andfear learning, Psychological Review, 108(3), pp.483522, 2001.
32. Macdonald, S. A., Brewster, S., and Pollick, F.;
Eliciting emotion with vibrotactile stimuli evocativeof real-world sensations, International Conference
on Multimodal Interaction, pp.125-133, 2020.
33. Seim, C. E., Ritter, B., Flavin, K. E., Lansberg, M. G.,
and Okamura, A. M.; Affective ratings of vibrotactile
signals in older adults with and without history ofstroke, IEEE World Haptics Conference, pp.457-462,
2021.
34. Yoo, Y., Yoo, T., Kong, J., and Choi, S.; Emotional
responses of tactile icons: Effects of amplitude,frequency, duration, and envelope, IEEE World
Haptics Conference, pp.235-240, 2015.
35. Russell, J. A.; A circumplex model of affect, Journalof Personality and Social Psychology, 39(6), pp.11611178, 1980.
36. Phillips, M. L., Gregory, L. J., Cullen, S., Cohen, S.,
Ng, V., Andrew, C., Giampietro, V., Bullmore, E.,

Zelaya, F., Amaro, E., Thompson, D. G., Hobson, A. R.,
Williams, S. C. R., Brammer, M., and Aziz, Q.; The
effect of negative emotional context on neural andbehavioural responses to oesophageal stimulation,
Brain, 126(3), pp. 669-684, 2003.
37. Craig, A. D.; How do you feel? Interoception: The
sense of the physiological condition of the body,
Nature Reviews Neuroscience, 3(8), pp.655-666,
2002.
38. Fukudo, S.; Stress and visceral pain: Focusing on
irritable bowel syndrome, Pain, 154, pp.63-70, 2013.
39. Fukushima, S.; A study on enhancement of emotional
experience for audio-visual content, PhD thesis,
University of Electro-Communications, 2013.
40. Sakurai, S., Narumi, T., Tanikawa, T., and Hirose, M.;
Making emotion and decision via affecting selfperception, Emotional Engineering, 4, pp.165-187,
2016.

Ibuki TARA (Non-member)
Ibuki Tara received the Master degree and B.E. inengineering from Nagoya University, in 2020 and
2022, respectively. He is a mechanical engineer
who studies autonomous robots.

Shogo OKAMOTO (Member)
Shogo Okamoto received the Ph.D. degree in information sciences from Tohoku University, in 2010.
From 2010, he has been with Nagoya University.
He is also currently an Associate Professor with
the Department of Computer Science, Tokyo
Metropolitan University. His research interests
include haptics, assistive robotics, human-centered informatics, andaffective engineering.

Yasuhiro AKIYAMA (Non-member)
Yasuhiro Akiyama received the B.E. degree inengineering from the Tokyo Institute of Technology,
Tokyo, Japan, in 2006, and the M.S. and Ph.D.
degrees in engineering from The University ofTokyo, Tokyo, in 2008 and 2011, respectively.
From 2016 to 2022, he was an Assistant Professor
with Nagoya University, Japan. He is currently an Associate Professor
with Shinshu University. His main research interests include mechanicalsafety, human–robot interaction, and biomechanics.

Hidetaka OZEKI (Non-member)
Hidetaka Ozeki received the B.E. degree in engineering from Nagoya
University in 2022. He is a mechanical engineer.