Observation of Mirror Reflection and Voluntary Self-Touch EnhanceSelf-Recognition for a Telexistence Robot
Yasuyuki Inoue∗

Fumihiro Kato†MHD Yamen Saraiji‡

Charith Lasantha Fernando§

The University of Tokyo, JapanThe University of Tokyo, Japan

Keio University, Japan

Keio University, Japan

Susumu Tachi¶
The University of Tokyo, JapanA BSTRACT

temporal delay between motor input from and sensory feedback tothe operator’s body is inherent in the control system. If the systemdelay is short enough that the operator’s motion is replicated immediately, the correspondence between motion execution and visualfeedback is obvious to the operator. Such sensory-motor couplingleads to a sense of agency (SoA), which is the operator’s subjective
awareness that the robot can be controlled by his/her will [2]. Conversely, when the delay is too long, normal expectations of sensorymotor coupling collapse and the operator’s SoA of the robot declines. Therefore, motor input and sensory feedback response speed
plays crucial role in constructing telexistence systems.
Furthermore, in telexistence systems, the appearance of the robot
body differs from that of the operator’s real body, even if the robot
has a similar kinematic configuration to a human body. This aesthetic difference makes it difficult for the operator to identify withthe robot. In particular, when watching the robot’s body in a mirror,
the operator feels uncanny because his/her face has changed, which
may lead to a decrease in the sense of body ownership. However,considering that real-time mirror reflection of a virtual avatar affects
body ownership [5], it is probable that mirror self-recognition arises
for the robot’s face instead of the operator’s real face. The mark test
paradigm, which is a standard method for verifying the presence ofmirror recognition [6], can be applied to test this assumption.

In this paper, we analyze the subjective feelings about the bodyof the operator of a telexistence system. We investigate whether amirror reflection and self-touch affect body ownership and agency
for a surrogate robot avatar in a virtual reality experiment. Results
showed that the presence of tactile sensations synchronized with theview of self-touch events enhanced mirror self-recognition.
Keywords: telexistence, body ownership, agency, self-touch, mirror recognition
Index Terms: H.5.1 [Human-centered computing]: Interaction
paradigms—Virtual reality
1I NTRODUCTION

Telexsitence [1] can be realized using a remote-controlled humanoid robot that replicates the body movement of the operator;
therefore, it is an important that the operator be able to control therobot as naturally and skillfully his/her own body. To achieve thisresult, a remote surrogate robot for telexsitence systems must possess similar dynamic characteristics to a human body. However, itis unclear how much similarity is necessary to provide operatorswith the feeling that the robot as can be used as his/her own body.
To clarify this issue, the author focuses on three subjective feelings
about the body: sense of body ownership (SoO), sense of agency,
and mirror self-recognition.
Sense of body ownership is a subjective awareness that bodyparts seen within one’s view belong to oneself [2]. Body ownershipis normally applied to one’s real body, but it is sometimes illusorily misapplied to artificial objects, as in the rubber-hand illusion[2] [3]. Recent studies using virtual reality (VR) techniques haveshown that similar illusions can occur with many other body parts,and even with the whole body [4]. These studies have revealed
that congruency of visual, tactile, and somatosensory information
is essential to establish body ownership. Therefore, telexistencesystems should allow the operator to experience multisensory congruency, as experienced with his/her real body.
In addition, the mobility of the robot is another crucial issue inthe development of telexistence systems because it has significantinfluence on the sense of naturalness of voluntary movement. Humanoid robots can be controlled using the operator’s actions (i.e.,master-slave manipulation), meaning that the operator can understand how to move the robot without special training. However,
because of limitations in mechanics and signal communication, a2M ETHOD

Aims: We investigate whether the presence of self-related information in telexistence systems affects the operator’s subjective
feelings of his/her body. More specifically, we evaluate how SoO,
SoA, and mirror self-recognition ratings change based on robot mobility and sensory-motor congruency when performing a motor task
in a behavioral experiment.
Participants: Eight healthy adults with normal vision were recruited for participation in the experiment. After receiving instructions and explanation, they gave written informed consent. This
research was approved by the ethics committee of the University ofTokyo.
Apparatus: An optical motion-capture system (Optitrack
Prime13W × 6) and a wide field-of-view dead-mounted display(Oculus DK2) were used to construct a VR environment. Seven
tracking markers were attached on the participant’s body to record
his/her upper-body movement. Data gloves for both hands (5DT,Data Glove 14 Ultra) were worn to measure their finger movements. The physiological response of participants was recorded using a skin conductance measurement unit (NIHONSANTEKU APU030m). Two wet gel electrodes were attached to the palm of theparticipant’s left hand. The VR environment was created in Unity
5.3.1 working on a Windows PC (Intel Core i7-6700, NVIDIA
GeForce GTX 980). The avatar used in the experiment was anelaborate 3D model of the existing telexistence robot, “Telesar V,”
which has the same hierarchical structures of mechanical linkages,
joint mobility, and body shape [7].
Procedure: In VR, participants held a virtual mirror in theirleft hand; using the mirror reflection, they were able to watch the∗ e-mail:y-inoue@tachilab.org† e-mail:fumihiro.kato@tachilab.org‡ e-mail:yamen@kmd.keio.ac.jp§ e-mail:charith@kmd.keio.ac.jp¶ e-mail:tachi@tachilab.org

2017 IEEE Virtual Reality (VR)March 18-22, 2017, Los Angeles, CA, USA
978-1-5090-6646-9/17/$31.00 c 2017 IEEE345

Figure 1: Experimental setup (left) and VR image (right)

robot’s body instead of their own. The task for participants was tofind a small cube around the robot body in the VR environment andto whisk them off using their right hand. The cube randomly appeared at one of four positions near a body part (left cheek, centreof chest, left shoulder, and left forearm) every 4 seconds. Participants continued the motor task for 60 seconds and then responded
to the six statements below: Q1: “I felt as if my body became arobot”, Q2: “I felt as if the robot moved according to my will”,
Q3: “I felt as if I watched my body in the mirror”, Q4: “I felt as
if there were two bodies there”, Q5: “I felt as if the robot movedindependently”, and Q6: “I felt as if there was another robot inthe mirror”. Q1 and Q4 are statements that address body ownership, Q2 and Q5 concern agency, and Q3 and Q6 are related to mirror self-recognition. Participants made a subjective rating for eachstatement based on a 7-point Likert scale ranging from -3 (stronglydisagree) to +3 (strongly agree). One trial consisted of the motor
task and evaluation described above, which was repeated four times
with changing experimental conditions in a 2 × 2 within-subjects
design. One factor was motor delay (the right hand either moved
immediately or was delayed by 500 ms); the other was self-touchsensation (the cube appeared either on or distant from the surfaceof the robot. Note that the virtual cube was presented only visually, therefore tactile sensation was absence when touching thosedistant from body. Each condition was presented to participants ina random order. The procedure above was one experimental block.
Participants performed four experimental blocks with five minutes
intermission between each.
After finishing all trials, participants performed an extra trial regarding mark test. In this trial, the mirror reflection temporarily
disappeared, though the frame was still in their hand. While theparticipant waited for the recovery of the mirror reflection, an experimenter covertly rubbed the participant’s cheek using a urethane
stick; simultaneously, a gash appeared on the robot’s face. Threeseconds after the rub stimulus, the mirror reflection appeared again
and participants noticed the gash on the robot’s face. After watching the face, they were instructed to answer two additional questions: Q7: “I felt as if my face got hurt” and Q8: “I felt as if
there was a wounded robot there”. During this trial, the participant’s skin conductance response (SCR) was recorded. These data
were analyzed offline and the change in SCR before and after themark test were extracted for each participant to evaluate the correlation between physiological sweating caused by face changes andresponses to Q7 and Q8 regarding mirror recognition.
3432
10-1

432
10-1

ANo deley
Deley
without
self-touchwithself-touch

B
MirrorAgencyOwnershipownership score (Figure 2A).

without
self-touchwithself-touch

43
2
10-1

C

without
self-touchwithself-touch

Figure 2: Summarized questionnaire scores for body ownership (A),agency (B), and mirror recognition (C)Correlation between SCR and questionnaire: Correlation
analysis was performed for the change rate of SCR against the summarized scores of Q7 and Q8; moderate correlation was found between them (r = 0.630).
4C ONCLUSION
In this study, we found that observation of a mirror reflection affected body recognition for a telexistence robot, especiallywhen performing voluntary self-touch and observing this event withconsistent sensory-motor feedback. With motion delay, the presence of tactile sensation through self-touch degrades mirror selfrecognition, probably because motion delay breaks the consistency
between visual, tactile, and motor information. This finding is analogous to previous studies emphasizing the effect of mirror observation [5] and self-touch [8], though direct evidence of improvementof body ownership was not observed in this experiment. A mark test
paradigm was introduced to investigate whether a threat reaction
occurs through mirror reflection and correlation between SCR andsubjective evaluation was found, meaning that operator’s subjective
acceptance/repudiation about the robot body can be evaluated using
this paradigm.
ACKNOWLEDGEMENTS
This work was supported by JSPS KAKENHI Grant Number
JP26240030.
R EFERENCES[1] S. Tachi. Telexistence (2nd Edition). Singapore: World Scientific, 2015.
[2] M. Tsakiris, S. Schtz-Bosbach, and S. Gallagher. On agency and bodyownership: Phenomenological and neurocognitive reflections. Consciousness and Cognition, 16:645–660, 2007.
[3] M. Botvinick and J. Cohen. Rubber hands feel touch that eyes see.
Nature, 391:756, 1998.
[4] O. Blanke and T. Metzinger. Full-body illusion and minimal phenomenal selfhood. Trends in Cognitive Science, 13(1):7–13, 2009.
[5] M. Gonzalez-Franco, D. Perez-Marcos, B. Spanlang, and M. Slater.
The contribution of real-time mirror reflections of motor action on virtual body ownership in an immersive virtual environment. pages 111–114. IEEE Virtual Reality, March 2010.
[6] G. G. Gallup. Chimpanzees: self-recognition. Science, 167:86–87,1970.
[7] M. Y. Saraiji, C. L. Fernando, M. Furukawa, K. Minamizawa, andS. Tachi. Virtual telesar - designing and implementation of a modular based immersive virtual telexistence platform. pages 595–598.
IEEE/SICE, December 2012.
[8] M. Hara, P. Pozeg, G. Rognini, T. Higuchi, K. Fukuhara, A. Yamamoto,T. Higuchi, O. Blanke, and R. Salomon. Voluntary self-touch increases
body ownership. Frontiers in Psychology, 6:1509, 2015.

R ESULT

Questionnaires: The scores collected for each questionnaire
were summarized within body ownership (Q1 and Q4), agency (Q2and Q5) and mirror recognition (Q3 and Q6). Repeated-measure
two-way ANOVAs were performed for these three categories andshowed that the main effect of motion delay was significant for
the agency score (F(1,7) = 8.04, p < 0.05, Figure 2B). Additionally, there was significant interaction between delay and touch on
the mirror recognition score (F(1,7) = 6.09, p < 0.05, Figure 2C).
There were no significant differences or interaction for the body346