Tactile Stimulus is Essential to Increase Motivation for Touch
Interaction in Virtual Environment
Kana Higashinoa,b, *, Mitsuhiko Kimotoa,c, Takamasa Iioa,b,, Katsunori
Shimoharab, and Masahiro Shiomiaa

ATR-IRC, Kyoto, Japan;b

Doshisha Univ., Kyoto, Japan;cKeio Univ., Kanagawa, Japan;*corresponding author: Kana Higashino, higashino2019@sil.doshisha.ac.jp
Kana Higashino received 2020 a B.S. degree from Doshisha University, Kyoto, Japan, where she
is currently in the master’s degree program. Her research interests include social touch, social
robots, and human-robot interaction.
Mitsuhiko Kimoto received M. Eng. and Ph.D. degrees from Doshisha University, Kyoto, Japan,
in 2016 and 2019. He is currently a JSPS Research Fellow (PD) at Keio University, Kanagawa,Japan. His research interests include human-robot interaction, human-agent interaction, andinteractive AI.
Takamasa Iio received the Ph.D degree from Doshisha University, Kyoto, Japan, in 2012. He is
an associate professor at Doshisha University, Kyoto, Japan. His research interests include social
robotics, group conversation between humans and multiple robots and social behaviors of robots.
Katsunori Shimohara respectively received B.E. and M.E. degrees in Computer Science andCommunication Engineering and a Doctor of Engineering degree from Kyushu University,
Fukuoka, Japan in 1976, 1978, and 2000. From 2001 to 2006, he was the director of the
Network Informatics Laboratories and the Human Information Science Laboratories, Advanced
Telecommunications Research Institute (ATR) International, Kyoto, Japan. He is currently aprofessor in the Department of Information Systems Design, Faculty of Science andEngineering in the Graduate School of Science and Engineering, Doshisha University, Kyoto,
Japan. His research interests include community systems design, relationality design, andrelationality-oriented systems design.
Masahiro Shiomi received M. Eng. and Ph.D. degrees in engineering from Osaka University in2004 and 2007. During that same time, he was an intern researcher at the Intelligent Robotics

and Communication Laboratories (IRC). He is currently a group leader in the Agent Interaction
Design department at Interaction Science Laboratories (ISL) and the Advanced
Telecommunications Research Institute International (ATR). His research interests include
human-robot interaction, social touch, robotics for childcare, networked robots, and field trials.

Tactile Stimulus is Essential to Increase Motivation for Touch
Interaction in Virtual SpaceThis paper reports the effectiveness of a tactile stimulus in a virtual environment
to increase people's motivations during a monotonous task by comparing a touchwith only visual stimuli and another with both visual and tactile stimuli. Although
touch interaction with robots showed various positive effects such as improved
motivation, visual and haptic stimuli were not separated due to the form of thetouch in physical environments. Virtual environments enable us to investigate such
effects by separating the modality of touch: a visual-only-touch and a visual-tactile
touch. We experimented in a virtual environment where participants did a
monotonous task after experiencing either a visual-only-touch or a visual-tactiletouch by an agent to compare them in the context of motivation improvements andthe perceived impressions. The experimental results showed that a visual-tactile
touch significantly increases the motivations of the participants for monotonous
tasks. On the other hand, their likability of the agents is not significantly changedby the touch modalities.
Keywords: Human-Robot Interaction, Social Touch, Virtual Reality

1. Introduction
Physical touches influence peoples’ behaviors and perceptions in the interaction
between humans and robots [1, 2]. Past studies reported that a robot’s touch encourages
motivation [3], persuasion [4], pro-social behaviors [5], self-disclosures [6], and increases
pain or stress-buffering effects [7, 8]. Touch interaction enables robots to provide positive
impressions [9] and convey different emotions by changing the touch characteristics [10,11].
These studies identified the importance of the physical existence of robots in thecontext of social interaction. However, it remains unknown whether a physical touch isessential to reproduce such effects because recent advances in virtual reality applications
allow interactions with others by such pseudo-haptic stimuli as visually touchingbehaviors. A recent study in virtual reality (VR) applications identified the effectivenessof touch interaction in VR environments, even though the touch interaction only includes
visual stimuli [12]. From another perspective, studies about body transfer illusion effects
which are well known as rubber hand illusion, investigated the effects of visual and tactile
stimulus toward people’s perceptions [13-18], but these studies less focused on positiveeffects in touch interaction with others.
In other words, we are interested in whether pseudo-touch interaction, i.e., only
visual-touch interaction, also provides similar effects that change people’s behaviors andperceptions. Therefore, we experimentally compare the motivation improvement effects
between only visual-touch interaction and both visual-tactile-touch interactions by
integrating a virtual reality application and a physical robot.
Although this paper is an extended version of previous work by Higashino et al.
[19], it contains additional descriptions of related works, additional experiment resultswith a modified experiment design, and more detailed discussions.

Fig. 1 Touch interaction with a virtual agent

2. Related work
2.1 Effects of visual and tactile sensation toward body transfer illusion effects
Manipulating combinations between visual and tactile sensations is a major
approach to investigating the effects of body transfer illusion in human science literature,
known as “rubber hand illusion” [13]. A series of these studies reported how body transfer
illusions occurred due to various combinations between visual and tactile sensations, andthey also noted the importance of synchronizations between these stimuli to cause suchillusion effects [14-16]. Recent studies used a virtual environment to investigate bodytransfer illusion effects and reported the usefulness of VR applications as a tool for
manipulating visual and tactile sensations [17, 18].
These studies provided essential knowledge about body ownership of people anddesigns guidelines for manipulating combinations between visual and tactile sensations;however, these studies did not focus on the positive effects of touch in the context ofmotivation improvements. In other words, these studies mainly focused on therelationships between tactile stimuli in touch interaction, i.e., less focused on motivation
improvements, which is the main aim of our research.

2.2 Touch interaction in physical environments
Robotics researchers have broadly investigated the effectiveness of touch
interaction with physical agents, i.e., robots. They focused on the influences of perceived
impressions of robots [20], including their conveyed emotions [11] as well as behavior
changes [6] and physiological effects [7]. From another perspective, some studies haveinvestigated the effects of observing touch interaction between robots and people in thecontext of trust for the observed robots [21] because the essential role of physical contact
is building relationships with others in human-human interaction [22].

Several studies focused on shake-hands interaction in telecommunication settingsand compared the effects of the only-visual-touch and visual-tactile-touch stimuli. For
example, Nakanishi et al. have developed a robotics hand, integrated it into atelecommunication application, and reported that shaking hands via the robotic hand
improved perceived impressions [23]. Bevan et al. have investigated the effects ofshaking hand interaction by using a physical robot in telecommunication settings, andphysical interaction is effective for negotiation contexts [4].
A series of such studies provided rich evidence about the usefulness of robot
touches in interaction with people. However, most studies only addressed the mixed
effects of visual and haptic stimuli in touch interaction due to robots' physical presence
and were less focused on motivation improvement contexts. A few studies focused on
observational touch effects and investigated the effects of only-visual-touch stimuli [21][24], but the robots in these studies did not directly interact with participants. Therefore,
even though they described the effectiveness of robot’s touch interaction, it remains
unknown whether a touch stimulus is essential for these positive effects.

2.3 Touch interaction in virtual environments
Touch interaction effects have also been broadly investigated in the field of virtual
reality studies. Similar to human-robot touch interaction in physical environments,
researchers focused on the influences of the perceived impression by touch with visual
stimuli [12, 25-27] and audio stimuli [28]. Moreover, due to advanced haptic devices thatcan work in such environments, recent studies investigated the effects of visual-tactiletouch stimuli in virtual environments. For example, simple haptic gloves or arm straps
are widely used in these studies to investigate touch interaction effects in virtualenvironments [29-33]. Some studies employed physical robots to reproduce more realistic
tactile stimuli for users in virtual environments [34, 35]. One merit of experiments with

virtual environments is how simply the interacting agents' appearance can be changed; acouple of studies investigated the appearance effects in touch interaction by integrating
virtual and physical agents [8, 36].
Similar to robotics research works, these studies provided rich knowledge about
touch interactions in virtual environments. However, they still failed to clearly compare
the effects of interaction between only-visual-touch and visual-tactile-touch stimuli
toward the motivation improvements.

2.3 Position of this study
Past studies provided interesting knowledge about visual-only-touch and visualhaptic touch interaction, as described above. However, they did not clearly compare theeffects of these factors in the context of motivation improvements. Therefore, our study
will provide complemental knowledge from an applicational perspective to utilize
manipulating combinations between visual and tactile sensations.

3. System
Figure 2 shows an overview of our system, which is composed of four hardware
components: a laptop computer, a head-mounted display, a touch controller, and a robot.
The laptop computer ran an application for a virtual reality environment to control boththe agent and the robot based on an experiment scenario. We used a head-mounted display,
Oculus Rift S (Fig. 3, left) for the experiment. The Oculus Touch controller (Fig. 3, right)
detected the user's left hand's position.
Our developed system has eight software components as follows. First, the scenario
manager in the computer loads an experiment scenario that consists of sequences for theagent’s actions like utterances and motions. The scenario manager tells the information

about the agent’s actions to the behavior interpreter, which explains them as commands
that it sends to the agent controller. If the command is a touch, it is also sent to the robot
controller to move the robot's arm. The agent controller is connected with the VR world
manager to update the VR world’s state. The visual/audio renderer reflects the updated
state. The head-position-sensing and the hand-position-sensing modules send the users’
head and hand positions to the behavior interpreter to control the agent’s behaviors. These
positions are used by the agent to realize eye-contact behaviors and to adjust its touch
motion.
PC
Scenario manager

Behavior
interpreter

Agent controller

VR world manager

Robot controller

Hand-position
sensingHead-position
sensing

Visual/Audio
renderer

Robot

Touch controller

Hardware component

HMD
Software component

Fig. 2 An architecture of the developed system.

Fig. 3 A head mount display and a controller

Fig. 4 Scenes of the virtual agent’s motions (hand clapping and waving)

3.1 Virtual agent
We used a 3D virtual agent that resembles a teddy bear. The agent performed three types

of motions: autonomous regular, scenario-based, and touch motions. The autonomous
regular motions denote an idle motion and eye contact. The agent slightly swung its headin the idle motion to make eye contact with users in the VR world. The scenario-based
motions were executed based on a script. Figure 4 shows the hand-clapping and waving
motions. The touch motion was defined in a scenario as well as the scenario-based
motions, but the motions were adjusted to touch the user's left-hand by using the sensing
modules. We used speech synthesis software to prepare the agent's utterances in advance.

3.2 Robot as a tactile stimuli device
Fig. 5 showed Sota as a physical robot in this experiment. It is tabletop-sized and has
eight degrees of freedoms (DOFs): three for its head, two for each arm, and one for its
lower body. It is 28 cm high. To provide tactile stimuli, of course, we did not need to use
a humanoid robot, but we use Sota as a tactile stimuli device due to its ease of use. In thisexperiment, we only used the robot to touch the user's hand. We covered its right handwith fluffy fabric. The robot's touch motion was synchronized with the touch motion by
the virtual agent (Fig. 6).

Fig. 5 The robot with fluffy fabricFig. 6. Syncronaization of touch motions between the agent and the robot

3.3 Scenario
The scenario decides the behaviors of both the virtual agent and the robot. The scenario

contents are shown in Table 1. We describe the details of three sub-sessions in the
scenario.

3.3.1 PracticeThis experiment's instructions were shown on a screen in the headset, and the agent next
to the instructions window read aloud them. Fig. 7 shows an image from the perspectiveof the users’s viewpoint. In the virtual environment, the users are seeing the task window,
the virtual agent, and their virtual left hand. They did several practices of the task. After
that, the agent announced to move the next session, i.e., the fixed-time session.
Table 1 Scenario contents
SessionPractice

Utterance
Hello. My name is Teddy. Nice to meet you. Please read the instructions on the screen.

MotionWavingClick the start button when you are ready to begin.
Fixed-time

The practice is over. Next is the five-minute, second session. Good luck.

Clapping

After 30 sec

You are off to a good start. You are getting faster than in your practice.

Clapping or Touch

After 60 sec

One minute has passed. Keep working.

Clapping or Touch

After 90 sec

You are working on your tasks at a good pace. Keep it up.

Clapping or Touch

After 120 sec

You seem to be getting used to it. You did more tasks than most other participants.

Clapping or Touch

Agent

Task window

Virtual left handFig. 7 Image seen in practice sessions

3.3.2 Fixed-time
The duration of the fixed time session is five minutes. At the beginning of the session,the agent requested the users to continue the task during this session after explaining thesession's duration. The agent praised the participant every 30 seconds in the fixed-time
session, in which three types of motions were performed. After five minutes, the agent
again announced to move the next session, i.e., the free-time session.

3.3.3 Free-time
The fixed time session duration is 15 minutes, but the agent did not explain the maximum
time information. At the beginning of the session, the agent again requested the users tocontinue the task during this session, but it did not explain the time period. On the other
hand, the agent explained that they could quit the task whenever. The agent kept a distanceand neither praised nor touched the users in this session. When the users pressed the ESK
key (this operation was described on the screen in the headset), the free-trial session was
finished, or the maximum time had expired.

3.5

Task

As an experimental task, we used a monotonous drag-and-drop task, which has been used
to investigate the effects of touch effects in a human-robot experiment (Fig. 7) [3]. In thistask, a circle and a square are shown on the screen in the headset. The user is asked todrag the circle and drop it into the square. When the circle is dropped into the square, thecircle disappears, and a new one appears in the first position. The users repeatedly did
this operation. The speed of the mouse cursor was intentionally set low.

4. Experiment
4.1 Hypotheses and predictions
Past studies reported the effectiveness of touch interaction with robots in thecontext of motivation improvements [3]. However, these studies less focused on theeffects of visual and tactile stimuli in touch interaction. Therefore it remains unknown
whether tactile stimuli are essential for motivation improvement effects. This lack ofinvestigation raises one simple question. Are tactile stimuli essential for improving
motivation via touch interaction? Even though past studies investigating visual-onlytouch effects did not delve into motivation improvements, these studies reported thatvisual-only-touch changed peoples’ perceptions [12, 25-27]. Another past study
concluded that an imagined touch, e.g., without tactile stimuli, positively affected stressbuffering [37].
To answer our simple question, we experimentally compared the effects of visualonly and visual-tactile touches in a virtual environment. We expected visual-tactile touch
to have an advantage compared to visual-only-touch because past studies reported that aphysically stimulating C-tactile fiber at an appropriate speed (5~10 cm/second) provided
comfortable feelings [38], which increased positive impressions and changed people’s
behaviors [3]. Therefore, we made the following predictions:
Prediction 1: People who are both visually and physically touched by the agent
will do more requested tasks than those only visually touched by it.
Prediction 2: People who are both visually and physically touched by the agent
will have more positive impressions of it than people who are only visually touched by it.

4.2 Conditions
We designed the experiment with a between-participant design and prepared two
conditions to compare the effects of the visual-only and visual-tactile-touches. The
following are the details:-

Visual-only-touch condition: The agent virtually touches the participants’ virtual
left hands without tactile stimuli.

-

Visual-tactile-touch condition: The agent virtually touches the participants’virtual left hands. At the same time, the robot physically touched their left hand toprovide tactile stimuli.
Due to unnatural and a mismatch between visual and tactile stimuli, we did notprepare a touch-only condition, i.e., the agent does not virtually touch the participants’hands, but the robot physically touches their hands.
We note that we did not include the gender factor in this study because [19] did
not show any significant differences through detailed analysis. In addition, the experimentresults of [19] showed that the no-touch condition and the visual-only-touch condition
showed similar trends compared to the visual-tactile-touch condition. Different from thegender factor, if we remove both conditions, we could not investigate the effects of visualtactile-touch in this context. Our main aim of this series of studies is to investigate themeaning of tactile stimuli in touch interaction. If we keep the no-touch condition, thecomparison could not purely evaluate tactile stimuli. Therefore we kept the visual-onlytouch condition as an alternative condition.

4.3 Environment
Figure 8 shows experimental scenes where participants are doing tasks in bothconditions (left: visual-only-touch condition, right: visual-tactile-touch condition). In thevisual-tactile-touch condition, the robot’s arm is touching the participant’s left hand.

Fig. 8 Experimental environment: left, without Sota, and on right, with it.

4.4 Participants
In this experiment,24 people have participated (every 22 females and males. Theirages ranged from 21 to 54, and their average was 35.7 (S. D.=11.0)). We assigned 11
females and 11 males to each condition. We could not correctly gather the data of four
participants during the experiments due to the misunderstanding of the instructions andinterruptions (i.e., we excluded four participants). Therefore we gathered valid resultsfrom nine males and 11 females in the visual-only-touch condition and 11 males and nine
females in the visual-tactile condition.

4.5 Procedure
Before the experiment, the participants were given a brief description of its
purpose and procedure. Our institution’s ethics committee approved this research forstudies with human participants. Written, informed consent was obtained from all of them.
The experimenter first explained the details of the systems without the robot. Thenthe participants checked the positions of the mouse and the controller and put on theheadset. After participants started to wear it, the experimenter placed the robot in thevisual-tactile condition (Fig. 8) and left the room. After the system preparation, theinstructions appeared on the screen in the virtual environment. The participants practiced
the task several times in the practice session, performed it for five minutes in the fixedtime session, and continued the task again during the free-time session until they decided
to stop or the 15-minute time limit passed.
Finally, the experimenter entered the room, hid the robot from the participants,
asked them to remove their headsets, and gave them questionnaires. At the end of theexperiment, the experimenter explained how the tactile stimuli are provided to theparticipants in the visual-tactile condition.

4.6 Measurements
We investigated the task motivation by measuring a subjective item: the free-time
session's working time. In addition, we measured an objective item to investigate theirimpressions of the agent with a likeability scale [39].

5 Results
5.1 Verification of prediction 1
Figure 9 shows the working times in the free-time sessions in each group. We
conducted a t-test whose results showed significant differences between the conditions(t(38) =2.027, p=0.050). Thus, the participants in the visual-tactile-touch condition
significantly did more tasks than the participants in the visual-only-touch condition;
prediction 1 was supported.

5.2 Verification of prediction 2
Figure 10 shows the perceived likeability of the participants in each group. We
conducted a t-test whose results did not show significant differences between theconditions (t(38) =0.555, p=0.582). Thus, the visual-tactile touch did not increase theparticipants' positive impressions compared to the visual-only-touch; prediction 2 was notsupported.

Seconds400** : p < .05

7

350

6

300

5

250
200

4

150

3

100

2

50

1

0
Visual-only-touch

Visual-tactile-touch

Fig. 9 Working time in free-time sessions

Visual-only-touch

Visual-tactile-touch

Fig. 10 Perceived likeability

5.3 Additional analysis
To investigate the visual-tactile-touch effects more, we conducted additional
analysis about the number of tasks and the time in each drag task. About the former one(mean: 113.56 times and 182.10 times for the visual-only-touch condition and thevisual-tactile-touch condition), we conducted a t-test whose results showed a significant
trend between the conditions (t(38) =1.928, p=0.061). About the latter one (mean: 0.57
second and 0.55 second for the visual-only-touch condition and the visual-tactile-touchcondition), we conducted a t-test whose results did not show a significant difference
between the conditions (t(38) =0.472, p=0.640).
These results suggested that the effect of the increase in input features did not
significantly increase the needed time for each task, and the proposed condition may
have a weak positive effect to increase the number of tasks, although the significant
effect was shown.

6 Discussion
6.1 Implication
This study showed the importance of tactile stimuli in the context of motivation
improvements by touch interaction. As shown in our results, the visual-tactile-touchcondition participants did the task longer than the participants in the visual-only-touchcondition. These results suggest a promising possibility for using robotic devices toprovide tactile stimuli in virtual reality applications that support education, rehabilitation,and so on. One advantage of integrating such applications and physical devices isavoiding the appearance effects of these devices. As described in the related work section,
appearances influence touch effects. Virtual environments enable developers to overwrite
the appearances of devices by HMDs.

We note that the aim of this paper is to investigate whether touch stimulus iseffective for motivation improvement. It is different to investigate whether touch stimulus
is the only factor for motivation improvement. In the context of motivation improvement
applications for social robots and/or virtual agents, they usually use various interaction
modalities for the purpose. Use of touch stimulus (not only visual touch) becomes acandidate modality for such interaction based on our experiment results, i.e., our study
provides evidence of the usefulness for motivation improvement to use tactile sensation.

6.2 Why did the visual-tactile-touch increased the task time?
The results of our study showed the motivation improvement effects of the visualtactile touch interaction compared to the visual-only touch interaction. We did not thinkthat simply increasing the number of input features caused the effect. Since past studies
that investigated touch effects increased the number of input one by one (e.g., no-touch,passive-touch, and active touch conditions), but the studies only changed their behavior
and impressions in the active touch conditions [3, 6]. These results indicate that the
number of input features not linearly influences people’s behaviors and perceptions. In
this section, we clarify why these differences occurred.
Interestingly, our experiment results did not show significant differences in the
likeability impressions between conditions. One possible implication is due to the ceiling
effect because the average likeability values in both conditions are about six, even thoughseven is this scale’s maximum value in this study. Therefore, these results showed that a
psychological index is not directly related to motivation improvement, which contradicts
results compared to past studies that suggest some psychological indexes are related tosuch efforts [40] [41].
Therefore, we thought that using other measurements would be useful to discuss
the reasons why the task time was long in the visual-tactile touch condition, such asbiometric or brain-related activities. For example, past studies measured biometric orbrain-related activity data to investigate how people react to stimuli [42-45]. We note thatthese studies are mainly focused on the reaction of people during visual/tactile sensations,
i.e., less focused on motivation improvement context, but still useful for discussion. In
addition, measuring brain activities using fMRI [46] or hormone changes [47] would
provide more depth insight. However, the participants needed to move their arms
frequently due to the task set. Such actions would make a lot of artifact noise while using
fMRI and other types of devices to measure brain-related activities such as EEG/NIRS.
Investigations of hormone changes also have difficulties because such changes need arelatively long period, and appropriate hormone types related to motivation improvement
are still unveiled. Although solving these problems is out of the scope of this study, it
would be important to investigate the visual-tactile touch interaction effects from various
evaluation perspectives.

6.3 Different modalities to improve motivations
This study investigated the effects of visual-tactile touch interaction for
motivation improvements, but of course, different kinds of modalities would be usefulfor motivation improvement. This section discusses possible modalities for motivation
improvements, which can be used simultaneously with visual/tactile stimulus.
For example, past studies investigated visual effects such as lighting patterns ofrobot’s LEDs, moving patterns, and speaking behaviors toward perceived impressions
and effective personality expressions [48, 49]. Although these studies are less focused on
motivation improvements, investigating the effects of these designs would be interesting
future work. Moreover, the appearances of the agents would influence people’s behaviors[50] [8]; therefore, using different avatars or changing avatars during interaction would
contribute to the motivation improvement of users.

Therefore, one possible future work of this study is to compare the effects of each
modality and explorer more effective combinations between these modalities. For thispurpose, sensing people’s reactions would be important to change modalities that arecommon strategies in the education context [51] and build friendly relationships between
users and agents via long-term interaction [52, 53].

6.4 Gender effects
This study did not include the gender factor in the experiment settings based on
our past study [19]. However, related studies investigating behavior changes via humanrobot touch interaction discussed gender effects because the results showed consistent
trends, although some of their results did not show significant differences [3][8].
Therefore, we also discuss the gender effects in the context of touch interaction with our
results.
Table 2 shows the values of the working time separated by the participants’genders. We note that these values are not significantly different between the genders due
to high standard deviations, but showed a similar trend in [19], i.e., the working time ofmale participants seemed to be more encouraged than female participants by tactile touch
stimuli. On the other hand, the working time of female participants in the visual-onlytouch condition is relatively longer than male participants.
One possible reason for fewer gender effects in this study is due to agents’appearance and voice. In this study, we used an animal-like appearance and a child-likevoice.

Still,

relatedstudiesinvestigatingthegendereffectsusedtypicalfeminine/masculine appearances and voices for interacting agents [8, 12]. The use of avirtual application enables us to change agent’s appearance easily. Therefore,
investigating the gender effects with different avatars is one of the interesting future
works described above.

Table 2 Average and S.D of the working time in each gender
Male

FemaleVisual-only-touch

169.8 (132.4)

232.9 (151.0)

Visual-tactile-touch

339.7 (241.0)

293.5 (170.0)

6.5 Limitations
This study has several limitations. We only investigated the effects of touch
interaction using a specific avatar, i.e., a teddy bear. Even though the appearance effectof avatars is not the main target of this study, investigating such effects is important touse different avatars. In addition, we only visualized the participants’ hands in the virtualenvironments. Showing the entire body and/or different appearances would produce
different consequences because several past studies reported that avatar appearanceschange perceived impressions and user behaviors [54, 55].

7 Conclusion
Even though touch interaction provides several positive effects in both humanrobot interaction in physical environments and human-agent interaction in virtualenvironments, the effects of touch stimuli have not been thoroughly investigated.
Therefore, we experimentally compared the effectiveness of visual-only-touch andvisual-tactile-touch by a virtual agent in a virtual environment in the context of motivation
improvement effects. We developed a system that provides both stimuli by integrating avirtual reality application and a physical robot.
Our experiment results showed that the participants in the visual-tactile touchcondition did more requested monotonous tasks than participants in the visual-only-touchcondition. In other words, the results highlighted the importance of tactile stimuli in touchinteraction for motivation improvements. On the other hand, perceived impressions about
the robot’s likeability were not significantly different between conditions.

Acknowledgements
This research work was supported in part by JST CREST Grant Number
JPMJCR18A1, Japan JSPS KAKENHI Grant Numbers JP18H03311 and JP 19J01290,
and JST, PRESTO Grant Number JPMJPR1851, Japan.

References:[1] J. B. F. van Erp, and A. Toet, “Social Touch in Human–Computer Interaction,”
Frontiers in Digital Humanities, vol. 2, no. 2, pp. 2, 2015.
[2] M. Shiomi, H. Sumioka, and H. Ishiguro, “Survey of Social Touch Interaction
Between Humans and Robots,” Journal of Robotics and Mechatronics, vol. 32,
no. 1, pp. 128-135, 2020.
[3] M. Shiomi, K. Nakagawa, K. Shinozawa, R. Matsumura, H. Ishiguro, and N.
Hagita, “Does A Robot’s Touch Encourage Human Effort?,” International
Journal of Social Robotics, vol. 9, pp. 5-15, 2016.
[4] C. Bevan, and D. Stanton Fraser, “Shaking hands and cooperation in tele-present
human-robot negotiation,” in Proceedings of the Tenth Annual ACM/IEEEInternational Conference on Human-Robot Interaction, pp. 247-254, 2015.
[5] M. Shiomi, A. Nakata, M. Kanbara, and N. Hagita, “A Hug from a Robot
Encourages Prosocial Behavior,” in 2017 26th IEEE International Symposium
on Robot and Human Interactive Communication (RO-MAN), pp. 418-423,
2017.
[6] M. Shiomi, A. Nakata, M. Kanbara, and N. Hagita, “Robot Reciprocation of Hugs
Increases Both Interacting Times and Self-disclosures,” International Journal ofSocial Robotics, pp. 1-9, 2020.
[7] N. Geva, F. Uzefovsky, and S. Levy-Tzedek, “Touching the social robot PARO
reduces pain perception and salivary oxytocin levels,” Scientific Reports, vol.
10, no. 1, pp. 9814, 2020.
[8] M. Shiomi, and N. Hagita, “Audio-Visual Stimuli Change not Only Robot’s Hug
Impressions but Also Its Stress-Buffering Effects,” International Journal ofSocial Robotics, pp. 1-8, 2019.
[9] C. J. Willemse, and J. B. van Erp, “Social Touch in Human–Robot Interaction:
Robot-Initiated Touches can Induce Positive Responses without Extensive Prior
Bonding,” International Journal of Social Robotics, vol. 11, no. 2, pp. 285-304,
2019.
[10] M. Teyssier, G. Bailly, C. Pelachaud, and E. Lecolinet, “Conveying Emotions
Through Device-Initiated Touch,” IEEE Transactions on Affective Computing,2020.
[11] X. Zheng, M. Shiomi, T. Minato, and H. Ishiguro, “What Kinds of Robot's Touch
Will Match Expressed Emotions?,” IEEE Robotics and Automation Letters, pp.
127-134, 2019.

[12] J. Swidrak, and G. Pochwatko, “Being Touched by a Virtual Human.:
Relationships Between Heart Rate, Gender, Social Status, and Compliance,” inProceedings of the 19th ACM International Conference on Intelligent Virtual
Agents, pp. 49-55, 2019.
[13] M. Botvinick, and J. Cohen, “Rubber hands ‘feel’touch that eyes see,” Nature, vol.
391, no. 6669, pp. 756-756, 1998.
[14] M. Tsakiris, and P. Haggard, “The rubber hand illusion revisited: visuotactile
integration and self-attribution,” Journal of Experimental Psychology: Human
Perception and Performance, vol. 31, no. 1, pp. 80, 2005.
[15] M. Costantini, and P. Haggard, “The rubber hand illusion: sensitivity andreference frame for body ownership,” Consciousness and cognition, vol. 16, no.
2, pp. 229-240, 2007.
[16] S. Shimada, K. Fukuda, and K. Hiraki, “Rubber hand illusion under delayed visualfeedback,” PloS one, vol. 4, no. 7, pp. e6185, 2009.
[17] W. A. IJsselsteijn, Y. A. W. de Kort, and A. Haans, “Is this my hand I see before
me? The rubber hand illusion in reality, virtual reality, and mixed reality,”
Presence: Teleoperators and Virtual Environments, vol. 15, no. 4, pp. 455-464,
2006.
[18] M. Slater, B. Spanlang, M. V. Sanchez-Vives, and O. Blanke, “First person
experience of body transfer in virtual reality,” PloS one, vol. 5, no. 5, pp.
e10564, 2010.
[19] K. Higashino, M. Kimoto, T. Iio, K. Shimohara, and M. Shiomi, “Effects of Social
Touch from an Agent in Virtual Space: Comparing Visual Stimuli and VirtualTactile Stimuli,” in 2020 29th IEEE International Conference on Robot andHuman Interactive Communication (RO-MAN), pp. 768-774.
[20] N. T. Fitter, and K. J. Kuchenbecker, “How Does It Feel to Clap Hands with a
Robot?,” International Journal of Social Robotics, vol. 12, pp. 113-127, 2020.
[21] T. Arnold, and M. Scheutz, “Observing robot touch in context: How does touch
and attitude affect perceptions of a robot's social qualities?,” Proceedings of the2018 ACM/IEEE International Conference on Human-Robot Interaction, pp.
352-360, 2018.
[22] T. Field, “Touch for socioemotional and physical well-being: A review,”
Developmental Review, vol. 30, no. 4, pp. 367-383, 2010.
[23] H. Nakanishi, K. Tanaka, and Y. Wada, “Remote handshaking: touch enhances
video-mediated social telepresence,” in Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems, pp. 2143-2152, 2014.
[24] Y. Okada, R. Taniguchi, A. Tatsumi, M. Okubo, M. Kimoto, T. Iio, K. Shimohara,
and M. Shiomi, “Effects of Touch Behaviors and Whispering Voices in RobotRobot Interaction for Information Providing Tasks,” in 2020 29th IEEE
International Conference on Robot and Human Interactive Communication (ROMAN), pp. 7-13.
[25] F. Biocca, J. Kim, and Y. Choi, “Visual touch in virtual environments: An
exploratory study of presence, multimodal interfaces, and cross-modal sensory
illusions,” Presence: Teleoperators and Virtual Environments, vol. 10, no. 3, pp.
247-265, 2001.
[26] M. Fusaro, M. Lisi, G. Tieri, and S. M. Aglioti, “Touched by vision: howheterosexual, gay, and lesbian people react to the view of their avatar being
caressed on taboo body parts,” 2020.

[27] K. Nagamachi, Y. Kato, M. Sugimoto, M. Inami, and M. Kitazaki, “Pseudo
Physical Contact and Communication in VRChat: A Study with Survey Method
in Japanese Users,” 2020.
[28] F. Biocca, Y. Inoue, A. Lee, H. Polinsky, and A. Tang, “Visual cues and virtual
touch: Role of visual stimuli and intersensory integration in cross-modal haptic
illusions and the sense of presence,” Proceedings of presence, pp. 410-428,
2002.
[29] G. Huisman, “Social Touch Technology: A Survey of Haptic Technology forSocial Touch,” IEEE Transactions on Haptics, vol. 10, no. 3, pp. 391-408, 2017.
[30] F. Boucaud, Q. Tafiani, C. Pelachaud, and I. Thouvenin, “Social Touch in Humanagent Interactions in an Immersive Virtual Environment,” in, pp. 129-136, 2019.
[31] P. Sykownik, and M. Masuch, “The Experience of Social Touch in Multi-User
Virtual Reality,” in 26th ACM Symposium on Virtual Reality Software andTechnology, pp. 1-11, 2020.
[32] A. Haans, and W. A. IJsselsteijn, “The virtual Midas touch: Helping behavior after
a mediated social touch,” IEEE Transactions on Haptics, vol. 2, no. 3, pp. 136140, 2009.
[33] G. Huisman, M. Bruijnes, J. Kolkmeier, M. Jung, A. D. Frederiks, and Y.
Rybarczyk, “Touching virtual agents: embodiment and mind,” in International
Summer Workshop on Multimodal Interfaces, pp. 114-138, 2013.
[34] Y. Yixian, K. Takashima, A. Tang, T. Tanno, K. Fujita, and Y. Kitamura,“ZoomWalls: Dynamic Walls that Simulate Haptic Infrastructure for Roomscale VR World,” in Proceedings of the 33rd Annual ACM Symposium on User
Interface Software and Technology, pp. 223-235, 2020.
[35] R. Suzuki, H. Hedayati, C. Zheng, J. L. Bohn, D. Szafir, E. Y.-L. Do, M. D. Gross,and D. Leithinger, “RoomShift: Room-scale Dynamic Haptics for VR with
Furniture-moving Swarm Robots,” in Proceedings of the 2020 CHI Conferenceon Human Factors in Computing Systems, pp. 1-11, 2020.
[36] A. Kotranza, B. Lok, C. M. Pugh, and D. S. Lind, “Virtual humans that touch
back: enhancing nonverbal communication with virtual humans throughbidirectional touch,” in 2009 IEEE Virtual Reality Conference, pp. 175-178,
2009.
[37] B. K. Jakubiak, and B. C. Feeney, “Keep in touch: The effects of imagined touch
support on stress and exploration,” Journal of Experimental Social Psychology,
vol. 65, pp. 59-67, 2016.
[38] G. K. Essick, A. James, and F. P. McGlone, “Psychophysical assessment of theaffective components of non‐painful touch,” Neuroreport, vol. 10, no. 10, pp.
2083-2087, 1999.
[39] C. Bartneck, D. Kulić, E. Croft, and S. Zoghbi, “Measurement instruments for theanthropomorphism, animacy, likeability, perceived intelligence, and perceived
safety of robots,” International Journal of Social Robotics, vol. 1, no. 1, pp. 7181, 2009.
[40] K. Takemura, “The effect of interpersonal sentiments on behavioral intention ofhelping behavior among Japanese students,” The Journal of Social Psychology,
vol. 133, no. 5, pp. 675-681, 1993.
[41] B. J. Fogg, “Persuasive technology: using computers to change what we think anddo,” Ubiquity, vol. 2002, no. December, pp. 5, 2002.
[42] K. C. Armel, and V. S. Ramachandran, “Projecting sensations to external objects:
evidence from skin conductance response,” Proceedings of the Royal Society of
London. Series B: Biological Sciences, vol. 270, no. 1523, pp. 1499-1506, 2003.

[43] T. Tsuji, H. Yamakawa, A. Yamashita, K. Takakusaki, T. Maeda, M. Kato, H.
Oka, and H. Asama, “Analysis of electromyography and skin conductance
response during rubber hand illusion,” in 2013 IEEE Workshop on Advanced
Robotics and its Social Impacts, pp. 88-93, 2013.
[44] N. Arizono, Y. Ohmura, S. Yano, and T. Kondo, “Functional connectivity analysis
of NIRS data under rubber hand illusion to find a biomarker of sense ofownership,” Neural plasticity, vol. 2016, 2016.
[45] N. Kanayama, M. Hara, and K. Kimura, “Virtual reality alters cortical oscillations
related to visuo-tactile integration during rubber hand illusion,” Scientificreports, vol. 11, no. 1, pp. 1-13, 2021.
[46] H. Ramakonar, E. A. Franz, and C. R. Lind, “The rubber hand illusion and its
application to clinical neuroscience,” Journal of Clinical Neuroscience, vol. 18,
no. 12, pp. 1596-1601, 2011.
[47] M. Ide, and M. Wada, “Salivary oxytocin concentration associates with thesubjective feeling of body ownership during the rubber hand illusion,” Frontiersin human neuroscience, vol. 11, pp. 166, 2017.
[48] S. Song, and S. Yamada, “Expressing Emotions through Color, Sound, andVibration with an Appearance-Constrained Social Robot,” in Proceedings of the2017 ACM/IEEE International Conference on Human-Robot Interaction,
Vienna, Austria, pp. 2-11, 2017.
[49] S. Whittaker, Y. Rogers, E. Petrovskaya, and H. Zhuang, “Designing personas forexpressive robots: Personality in the new breed of moving, speaking, andcolorful social home robots,” ACM Transactions on Human-Robot Interaction(THRI), vol. 10, no. 1, pp. 1-25, 2021.
[50] K. Terada, L. Jing, and S. Yamada, “Effects of agent appearance on customer
buying motivations on online shopping sites,” in Proceedings of the 33rd annual
acm conference extended abstracts on human factors in computing systems, pp.
929-934, 2015.
[51] T. Belpaeme, J. Kennedy, A. Ramachandran, B. Scassellati, and F. Tanaka,“Social robots for education: A review,” Science robotics, vol. 3, no. 21, pp.
eaat5954, 2018.
[52] T. Kanda, M. Shiomi, Z. Miyashita, H. Ishiguro, and N. Hagita, “Acommunication robot in a shopping mall,” Robotics, IEEE Transactions on, vol.
26, no. 5, pp. 897-913, 2010.
[53] T. Kanda, R. Sato, N. Saiwaki, and H. Ishiguro, “A two-month field trial in anelementary school for long-term human–robot interaction,” IEEE Transactions
on Robotics, vol. 23, no. 5, pp. 962-971, 2007.
[54] N. Yee, and J. Bailenson, “The Proteus effect: The effect of transformed selfrepresentation on behavior,” Human communication research, vol. 33, no. 3, pp.
271-290, 2007.
[55] N. Yee, J. N. Bailenson, and N. Ducheneaut, “The Proteus effect: Implications oftransformed digital self-representation on online and offline behavior,”
Communication Research, vol. 36, no. 2, pp. 285-312, 2009.