International Conference on Artificial Reality and TelexistenceEurographics Symposium on Virtual Environments (2015)
M. Imura, P. Figueroa, and B. Mohler (Editors)

Development of Mutual Telexistence System using Virtual
Projection of Operator’s Egocentric Body Images
MHD Yamen Saraiji†1 , Charith Lasantha Fernando1 , Kouta Minamizawa1 and Susumu Tachi2
1 Graduate School of Media Design, Keio University, Japan
2 Institute of Gerontology, The University of Tokyo, Japan

A

B

C

Figure 1: Our proposed mutual Telexistence system using projected body visuals. (a) user’s first person view with his hands
being projected on remote place, and (b) a remote participant interacting with user’s projected hands. And in (c) the user
explaining to a remote participant about several items using the projected hands
Abstract
In this paper, a mobile telexistence system that provides mutual embodiment of user’s body in a remote place isdiscussed here. In this system, a fully mobile slave robot was designed and developed to deliver visual and motion
mapping with user’s head and body. The user can access the robot remotely using a Head Mounted Display (HMD)
and set of head trackers. This system addresses three main points that are as follows: User’s body representation ina remote physical environment, preserving body ownership toward the user during teleoperation, and presenting
user’s body interactions and visuals into the remote side. These previous three points were addressed using virtual
projection of user’s body into the egocentric local view, and projecting body visuals remotely. This system isintended to be used for teleconferencing and remote social activities when no physical manipulation is required.
Categories and Subject Descriptors (according to ACM CCS): H.5.1 [Information Interfaces and Presentation]:
Multimedia Information Systems—Artificial, augmented, and virtual realities

1. Introduction
Teleoperated robots have been widely used in several applications related to communication and operation. For communication purposes, we rely not only on what we see, hearand say, but also we utilize our bodies as a way to communicate and embody our internal mental states to the others [BP75]. Telepresence type of systems generally provides† yamen@kmd.keio.ac.jp
c The Eurographics Association 2015.

mean to navigate and have a mediated video/audio communication over the internet, such as Telepresence robots[TMEHB11, LS∗ 11]. Furthermore, these systems provides
minimum representation of user’s body state using a display
showing user’s face. However, the interface of these systems
disconnects user’s perception of presence in the target remote place. And the user fails to observe his body being immersed in the teleoperated robot side, as well as the remoteparticipants does not have clear awareness of user’s bodystate and actions.

MHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System

In contrast with Telepresence systems, Telexistence systems provide the human operator a real-time sensation of
being presented at a place different from where he physically located at, and to be able to interact with the remoteenvironment [TMFF12]. These type of systems usually uses
a Head Mounted Display (HMD) to deliver the user an immersive, first point of view (FPV) vision of the remote place.
In addition, Telexistence in concept enables the observers in
the remote environment to see an avatar representation of theoperator.
In this paper, we focus on the topic of Telexistence into
remote environment using real body captured images. We
addressed the following points:
1. User’s body representation in a physical environment.
2. Preserving body ownership during teleoperation.
3. Presenting user’s body visuals to remote observers.
This paper describes how to accomplish a low-cost mobile
Telexistence platform that enables the user to have visualawareness of his real arms and hands in the remote place.
Also, the remote observers can understand the intended interactions clearly although the operator does not have physical representation of his arms and hands. Figure 1 shows thedifferent scenarios of using virtual projection of user’s body
into local and remote surfaces. Figure 1 (A) shows what theuser sees in the remote place while using his hands. And in
Figure 1 (B), user’s hands are projected to remote surfaces
making it easy to understand what the user is pointing at.
Also it shows in Figure 1 (C) how the remote participants
can understand precisely what the user is pointing at.
2. Related Work
Several works addressed the mutual presentation of user’s
body into remote places. Systems which deploys a tangible
representation for user body were proposed [LFOI14,IK92],
in this type of systems the remote participants can understand user’s hands motion and position using a visualand tangible surface. For teleconferencing and collaborative
tasks, [KNK10] uses life-sized video to show user’s bodybeing overlaid into a shared display in the remote place. Remote participants can see user’s gaze and gestures in theirlocal space. However, these type of systems do not aim
for immersive teleoperation. Other type of systems whichaims for full scale immersive telecommunication using virtual representation for both local and remote sides has beenpresented. [STB∗ 12, OSS∗ 13] demonstrated the concept of“Beaming” in which the user can teleoperate into a remotephysical space using full 3D representation of that space.
The project demonstrates a general framework for such purpose.
For Telexistence type of systems, the topic of mutual
telecommunication was addressed in TelesarPHONE system [TKN∗ 08]. TelesarPHONE projects user’s body visuals that were captured from external camera into slave robot

body. The system uses retro-reflective projection technology(RPT) [IKT03] to provide single point of view image projection to the observer. However, this system requires to view
the operator from specific points of view based on the positions of the acquired images in respect to operator’s body. In
addition, the user does not observe the remote environment
from an egocentric position, but observes using set of displays. The work presented by [FFK∗ 12] addressed the physical representation of user’s body as an avatar representation,in which the human operator upper body is fully replicatedand mapped as a humanoid robot. In this type of systems,
the user has an immersive experience of presence in the remote place, with the capability to manipulate objects using
the robotics arms. However, in this type of full scale robotic
systems, the user fails to see his real body visuals in the robot
place, but instead he would see mechanical representation.
In addition, the scale of these systems is inconvenient to be
used for mobile and telecommunication situations due to thecost and complexity.
To present user’s body visuals into a virtual environment,
several works investigated the appropriate methods toachieve that. In the work of [YHK96] developed a “Whatyou can see is what you can feel” system, which is used todirectly manipulate and touch virtual objects using hands
and a video-see-through display. This system requires video
keying technique using a distinguishable background color
from hands colors, user hands are segmented from thebackground and superimposed into virtual environment.
In this work [BSRH09], egocentric images of user’s bodyare captured using a video-see-through HMD, and superimposed into virtual environment. In this method, the user
has to train the system for his skin color in order to be
captured effectively. In [TAH12], demonstrated the usage of
depth array sensor to capture user’s hands interaction andsuperimpose it remotely for visual guidance applications.
Though the hands are captured from a different point ofview from user’s eyes, the 3D geometric data are reconstructed and matched with his view. A different approach
was proposed by [SFF∗ 13] which uses model-based image
segmentation. In this method, egocentric body visuals arealso captured using a video-see-through HMD, but theimages are masked from the background using humanoid
virtual body representing the tracked state of user’s body.
The user is equipped with set of optical trackers and data
gloves to track his body, arms and hands posture, andmapped to the virtual mask.
Recent work started to address an important cue fortelecommunication, facial expressions and gaze representation. SphereAvatar [OSS12] uses a 360◦ spherical display
to present face visuals of the local user into a remote place.
Though the visual representation is the face is mapped intoa virtual 3D head, however it helps to identify the user fromany viewpoint around the display. Also in [PS14] a cylindrical display is used to project multi-viewpoint capturedc The Eurographics Association 2015.

MHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System

To address these limitations, we propose a mutual virtualembodiment method for lightweight Telexistence robots thatlacks physical arms. This method uses virtual hands that arecaptured from user’s side, and present those hands in user’sview as superimposed pictures on the remote images. Also,
the hands are projected remotely using a small projector
mounted on the robot head and aligned with head movement.
The virtual hands can be projected onto a physical table, remote user, or to any remote surfaces in order to provide theclue of user’s hands interaction and intended actions. These
virtual hands also provides the awareness for the user about
his body, which are necessary for the sense of body presence.

3. Design Considerations
As described in [BS10, Bio97], three types of presence contribute in user’s awareness of being presented in a specific environment: spatial presence, self presence, and social presence. Spatial presence is to be capable to interact
with the environment. Self presence can be described as tobe able to observe our bodies being presented in the environment, and aware of its posture at any given moment. The
social presence is how we observe social interaction withother people within one environment. Figure 2 shows thethree points with some examples of the target experience foreach.
To address the first key point “spatial presence” the user
should have direct access to the environment as if he is located there, with the freedom to move and navigate via an alternative representation of his body. The user should be able
to move freely and independently his head and body, andaccording to that, the slave robot should follow and updateuser’s visuals of the remote place. We avoid using any physical or tangible controller (such as a joystick or keyboard)
to control motion and rotation speed of the slave robot. Thisis important because if the user is aware of the presence ofa physical controller, then the coherence between the local
and remote place will break. So an intuitive and natural interface is required to maintain spatial coherence. To moveand navigate in virtual environments, locomotion interfacesc The Eurographics Association 2015.

Social Presencesual
Vis essdy
Bo aren
Aw

Co
m Vis
m ua
un l
ica
tio

n
Collaborative
Tasks

A previously proposed mobile, low-cost telexistence system [SFM∗ 14] provides operators own hands visuals by using Computer Generated (CG) hands and overlaid into operator’s HMD along with remote avatar vision. However
there was no representation of the hands in the avatar robot
place. As a result, the remote participants were not capable
to understand user’s gestures and posture with respect to his
avatar body. Therefore, sometimes the remote participants
get confused due to lack of visual clues of the operator interactions.

Self Presence

PhysicalMovement

images of user’s head into a remote place, achieving higher
realistic visuals of the face and gaze compared with the 3D
representation.

User

Sensory
Immersive

NaturalNavigation

Spatial Presence
Figure 2: Three main factors for sense of presence.

such as [IF96] usually uses treadmill floor to allow the users
to walk infinitely by constraining his body position. Other
type of navigation in virtual environments was suggested
by [LJFKZ01] in which the user decides the moving vector
by leaning his body. The latter method in comparison withthe former one has less fatigue effect on the user when thesystem is used for long period since the user can navigate
while on seat.
The second point “self presence” is the fact the user
should have physical awareness of his body’s presence. The
user validates his existence in a specific place by observing
his body’s visuals as he expects, maintaining the ownership
relation with his body. Several works addressed the representation of our bodies in virtual/physical environments as listed
in the previous section, mainly two types were discussed:
physical robotic representation, and image-based representation. In this work, we found that observing body visuals
is an effective factor to maintain the seamless sense of presence for the user, so image-based method is developed which
captures egocentric images of user’s body visuals, and superimpose it into the remote place.
The final point we addressed in this paper is “social presence”. In order for the user to communicate effectively withother people in a different location, mutual communication
between both sides should be maintained. As in “spatialpresence” the user is aware of the surroundings and people around, those people should be capable to understandwhat the user wants in return. It is commonly to user an LCD
panel only to show user’s body, however this method is notcapable to provide spatial interaction in the 3D space. Asan alternative, we propose to project user’s body visuals in
robot side, so the user can visually affect in the remote place,allowing remote observers to visualize his body.

MHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System

4. System Implementation

Table 1: Robot/IR Cameras, HMD, and Projector’s FoV andResolution

The developed system is divided into a Master-Slave Telexistence systems as described in Figure 3 . The master sideis the operating side where the user is located, and it contains a set of tracking tools that are used to capture user’s
head movement integrated with a wide angle HMD (Model
No: Oculus Rift DK2). The HMD was customized to contain a front Infrared (IR) Camera to capture user’s egocentric images, more specifically hands images. The IR Camera used is part of a commercially available product (Model
No: Leapmotion). User’s cockpit communicates with Robotavatar over wireless network 5.8 Ghz band, which handles
real-time stereo-images video streaming from robot side, as
well as control commands from user’s side.
Pico Projector

HMD &Leap motion

Horizontal FoV
100◦
75◦
135◦
45◦

HMD
Robot Camera
IR Camera
Pico Projector

perimposed on the visual stream from the robot side, so theuser can have visual awareness of his hands presence. The
position and size of the captured hands are preserved in the
FPV with his real hands, so the pointing remains natural. In
the robot side, Those hands are sent and projected using thepico projector. The projected hands serves as a shadow ofuser’s hands which follows its motion and gestures.
Hands Superimpose
Segmentation

HMD &IR Camera

Master Side (Local)

Slave Side (Remote)Motion Vectors

UDP Control Bus

Video Processing

Motion Capture

Projected Hands
Head Position +Orientation

Image Distortion
Correction

x264 Decodeing

UDP Depay
Port 7000

Figure 3: Proposed system overview.
In the robot side (slave), a 3D printed 3 Degrees of Freedom (DOF) head was designed to physically map user’s headrotational motion at the remote place. HD stereo cameras
and binaural microphones are used to enable bidirectional
visual and auditory communication to the user from robot
side. The robot provides to the remote participants user’svideo and voice using a LCD display and a speaker mounted
on the front side of the robot. The robot also contains a picoprojector to display user’s hands projection.The robot designed with fully wireless and mobile platform that allows
free motion in remote places.
Table 1 summarizes the current setup for the cameras andprojector in the robot side, as well as user’s HMD and IR
camera. Because its not always possible to maintain the same
FoV for all the components, image size correction needs totake place. Further details about correction method will follow in the next sections.
The captured Hands movement and visuals are used toprovide visual feedback to user’s side, as well as to be projected in the robot’s side. The user observe his own hands
motion over robot’s vision. To present user’s hands in the
remote place, the captured egocentric images are first segmented to isolate the hands from the background, then su-

Operator Side - Local

Hands Correction

Port 7001

Video Streaming

Display

Hands Projection
Projection

IR Capture

Network Connection - Wireless 5.8Ghz

Correction

Robot BaseResolution
1920x1080
1280x720 (per eye)
640x240 (per eye)
1280x720

Stereo Cameras &Pico Projector

YUV420 Stereo
Capture (1280x720)
x264 Encoding
bitrate: 3500 kbps

UDP Payload

Robot Control

4.1. System Overview

Head & Base
Motion Control

UDP Read
Port 6000

Avatar Side - Remote

Figure 4: System data flow diagram. Main components of
Mutual Telexistence System.

The overall data flow and main components of the system
are described in Figure 4. Stereo images are captured fromthe robot side, stitched to one image and encoded together
on a single stream. This is important to ensure the synchronization between both eyes even if some frames dropped depending on network’s reliability. The stream is encoded using H264 video encoder with a bit-rate of 3500 kilobits per
second (kbps). The video stream is sent over a UDP channel
to the user side. On the user’s side, video stream is decodedand visually corrected to match HMD’s Field of view (Fov),then displayed inside the HMD. For media encoding andencapsulation, we used an open source library "GStreamer
1.0".
For hands displaying and projection, the hands are captured from user’s FPV, and processed locally to be displayedover the remote images. Also those images are sent to theremote side using H264 encoding (similar to the previous).
The robot side handles those images, and correct the sizeand distortion of the images, and project them using the picoprojector. Further details will follow in the next subsections.
c The Eurographics Association 2015.

MHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System

4.2. User’s Side Overview
4.2.1. Hands Capturing and Segmentation

Rotate Left

In the user side, the hands are captured using an IR camera mounted on the front of the HMD. The camera provides
110◦ field of view which covers HMD FoV, and thus it is possible to capture user hands with no cropped areas. Though
the resolution of the cameras are relatively low (640x240),
up sampling step is necessary to smooth out the edges. The
advantages of using IR camera compared with RGB camerais the possibility to capture objects close to the camera using
the returned intensity, in our case we capture hand visuals
effectively. However there is a resulting noise from the background. We apply a nonlinear filtering function on the captured images, this function removes the pixels which color
intensity are below a certain threshold:Side
Movement

Rotate Right

ForwardMovement

Figure 6: Body as a joystick for motion control.

Stereo Cameras( 1
1
P Gamma , if P Gamma ≥ threshold
Filter(P) =0,otherwise

Binocular Audio(1)

Pico Projector

3 Axis HeadP ∈ [0, 1]

Figure 7: Top/Side views of camera/projector FoV and projection size.

4.3. Avatar Robot Overview
Figure 5: Captured hands IR images before (a) and after(b) applying the filter.

The procedure of applying the filter is implemented in the
GPU using shader language. The results of applying the filter
can be seen in Figure 5.
4.2.2. Avatar Robot Motion Control
To enable the motion in the remote place, the control mechanism should avoid any explicit controllers. A hand-free control was implemented to fulfill this condition using body asa joystick. The user controls robot motion by leaning or rotating his body to move forward or rotate to left and right.
Figure 6 outlines the motion vectors relative to user’s head.
User’s motion is captured using Oculus DK2 tracker,
which outputs 3D position H pos and head Euler angles Hang .
When the user connects to the robot, H pos and head panning values are calibrated to zero. While connected, Hang
directly controls robot’s head angles (Tilt, Pan, Roll), so theuser head rotation is mapped 1:1 with robot’s head.
c The Eurographics Association 2015.

4.3.1. Head Design
Robot’s head provide the user spatial mapping using a 3
DOF part to control (Roll,Pan,Tilt) rotation based on user’s
head movement. High torque servo motors used in this design (Model No: HerkuleX DRS-0201). Visual mapping
is done using stereo HD cameras (Model No: See3CAM
CU130) with a fixed interpupillary distance (65mm). The
camera outputs images at resolution 1280x720@50 Frames
per seconds (FPS) using YUV420 format. To provide egocentric images of user’s body into robot side, a pico projector (Model No: Lumex Beampod MX-65) is used to projectuser’s hands remotely. The projector is aligned with the eyes
movement so the relative distance between eyes-hands remains the same as in the user. Head components alignment
can be seen in Figure 7.
4.3.2. Projector Calibration
Due to the displacement between the projector position andthe camera position in the robot side, and the difference between both fields of view, the projection of the hands directly
will result mismatch scale and position when observed by
the operator from the FPV. Thus it is necessary to measureMHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System

this displacement and scale by calibrating the projector withrespect to one of the cameras. The goal of the calibration
process is to determine the amount of displacement (dx,dy)
between the projected image and camera’s captured region.
Also to extract the relative scale between the projected image
and the field of view of the camera (Rw,Rh). This process isdone at a projection distance (D). Figure 8 shows an illustration of the top and side views of calibration setup. The parameters (Wp,Hp), (Wc,Hc) represents the size of projection
and capture for the projector and the camera respectively.
An automated process is done to extract those parameters
by projecting a chessboard image into a specific distance(D) which is set to 100 cm as representation of hands reach.
The relative scale of the projected images (Rw,Rh) is calculated as the ratio between (Wp,Hp) and (Wc,Hc). This ratio
is used as a cropping factor for the projected hands, and thedisplacement (dx,dy) is used to shift the cropping region ofhands images.
Top View

Wp

Figure 9: User’s hands being super imposed locally.

This would results the user to fail to determine the visual distance of his hands. To correct this distortion, simple
trigonometry is used to calculate the scaling factor:

Side View

dx

CameraSuper Imposed
Hands

dy

Wc

CameraHp

Projector

Hc

Projector

D

D

Figure 8: Top/Side views of camera/projector FoV and projection size.
This calibration gives matching results for images projected at the calibrated distance (D). However it is affected
when the images are projected at different plane, resulting
mismatching size and shift of hands position. This behavior
is intended as a shadow of the hands, so even though the user
can see the projected hands, he still understands this acts asa precise pointing interface in the remote place.
4.4. Hands Presentation
4.4.1. Presenting to User
The processed hands are used locally in the user side by superimposing them over the remote visuals of the robot side.
By doing this, the user remains aware of his body though
there is no physical representation in the remote side. Also,
since the hands are image-based captured from his FPV, theuser knows that the presented hands are his own, thus preserving his body ownership sensation. Figure 9 shows what
the user sees from his FPV when he uses his hands.
Since of the captured images FoV can be different from
HMD’s FoV (depending on IR camera’s FoV), hands size
will be different from the size of our observed real hands.

IRtan FoV2HandScale =tan FoV2HMD(2)

Because of the hands are being captured, processed, andsuperimposed locally, the speed of interaction of the hands
is not affected by any network delays or packet drops. For
example, when latency occurs from robot side (mechanicalor visual stream), the user would remain able to see his hands
follows his body regardless of the robot side. This helped toreduce the sense of time delay and visual sickness when the
network become unstable for some reason.
4.4.2. Presenting to remote participants
User hand images are streamed remotely to robot side, andare projected from robot’s point of view using a pico projector mounted on its head. Those hands are aligned with user
hands position and motion, and allows the remote participants to see the gesture of his hands. Figure 10 shows thehands being projected on a trivial surface, where user hand
gesture can be seen remotely. Depending on projector’s lumens, the hands might be difficult to see in a well lit room.
In the current implementation, we are using a 65 lumen projector to render the hands.
4.5. Technical Evaluation
Evaluating the speed of transmission in any telexistence system is necessary to be known before conducting user evaluation for it. The two main concerns regarding the latency
are: body perception latency, and remote visual feedback latency. Those two factors are necessary to be minimized toan acceptable level. âĂIJAcceptable levelâĂİ has two cases,
one for each of the previous:c The Eurographics Association 2015.

MHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System

Projected Hands
Imagestating, no tangible controllers were used. The robot communicates with the user over an IP network, and a low-latency
video stream from robot side is sent to the user over this network. Using this method, it is possible for the participants tounderstand where the user is pointing at or what the user isintended to do with his hands in the remote environment.
6. Acknowledgement
This work was supported by JSPS KAKENHI Grant#26240030

Figure 10: Illustration of the projected hands into a physical table at robot’s side.

References• The first factor related to body perception affects user’s
kinaesthesia, which is the awareness of his body position
in respect to the motion. When user experience latency towards the presented body, the level of presence is reduced
accordingly.
• Visual feedback latency mainly affects the operation efficiency of the robot (navigating for example), and also the
motion sickness the user would experience if the visualfeedback did not match his motion head motion.

[BP75] B ENTHALL J., P OLHEMUS T.: The body as a medium of
expression. Allen Lane, 1975. 1

For body perception related latency, the processing ofbody and hands visuals are entirely done locally, no network
is involved in this process. The measured speed of capturing the hands images, filtering them and rendering them iswithin the range 15-20ms (60-50 FPS).
Regarding visual feedback latency, the video stream fromthe robot to user is passed over IP network, so process of encoding, payloading over network, and decoding the images
will add significant overhead. In an ideal system, latency
does not exceed one frame (15ms) however due to encoder’srequirements, extra frames are needed to do encoding. H264
video encoder is used in this system which handles image
size 1280x720@60FPS at bitrate 3500 kbps. The measured
Capture-to-Display (CTD) latency was 100±20ms.
5. Conclusion
In this paper, we proposed a mutual Telexistence mobile system which uses virtual projection of egocentric body visuals
into local and remote sides. The user maintains the sense ofownership of his body while operating in a different location
by superimposing his FPV body visuals on top of remoteenvironment’s visuals. Body visuals are also presented remotely by projecting the captured egocentric images into remote space using a pico projector mounted on robot’s head.
To provide spatial mobility in the remote place, we designed
and developed a lightweight Telexistence platform with a 3
DOF head and mobile base. The user controls robot’s navigation and speed using his body motion by leaning or roc The Eurographics Association 2015.

[Bio97] B IOCCA F.: The cyborg’s dilemma: Progressive embodiment in virtual environments. Journal of Computer-MediatedCommunication 3, 2 (1997), 0–0. 3[BS10] B RACKEN C. C., S KALSKI P.: Immersed in media: Telepresence in everyday life. Routledge, 2010. 3[BSRH09] B RUDER G., S TEINICKE F., ROTHAUS K., H IN RICHS K.: Enhancing presence in head-mounted display environments by visual body feedback using head-mounted cameras. In
CyberWorlds, 2009. CW’09. International Conference on (2009),
IEEE, pp. 43–50. 2[FFK∗ 12]F ERNANDO C. L., F URUKAWA M., K UROGI T., K A MURO S., S ATO K., M INAMIZAWA K., TACHI S.: Design oftelesar v for transferring bodily consciousness in telexistence. InIntelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on (2012), IEEE, pp. 5112–5118. 2[IF96] I WATA H., F UJI T.: Virtual perambulator: a novel interface device for locomotion in virtual environment. In Virtual
Reality Annual International Symposium, 1996., Proceedings ofthe IEEE 1996 (1996), IEEE, pp. 60–65. 3[IK92] I SHII H., KOBAYASHI M.: Clearboard: A seamlessmedium for shared drawing and conversation with eye contact.
In Proceedings of the SIGCHI conference on Human factors incomputing systems (1992), ACM, pp. 525–532. 2[IKT03] I NAMI M., K AWAKAMI N., TACHI S.: Optical camouflage using retro-reflective projection technology. In Proceedingsof the 2nd IEEE/ACM International Symposium on Mixed andAugmented Reality (2003), IEEE Computer Society, p. 348. 2[KNK10] K UNZ A., N ESCHER T., K UCHLER M.: Collaboard: anovel interactive electronic whiteboard for remote collaboration
with people on content. In Cyberworlds (CW), 2010 International Conference on (2010), IEEE, pp. 430–437. 2[LFOI14] L EITHINGER D., F OLLMER S., O LWAL A., I SHII H.:
Physical telepresence: shape capture and display for embodied,
computer-mediated remote collaboration. In Proceedings of the27th annual ACM symposium on User interface software andtechnology (2014), ACM, pp. 461–470. 2[LJFKZ01] L AV IOLA J R J. J., F ELIZ D. A., K EEFE D. F.,Z ELEZNIK R. C.: Hands-free multi-scale navigation in virtualenvironments. In Proceedings of the 2001 symposium on Interactive 3D graphics (2001), ACM, pp. 9–15. 3[LS∗ 11] L AZEWATSKY D., S MART W. D., ET AL .: An inexpensive robot platform for teleoperation and experimentation. In
Robotics and Automation (ICRA), 2011 IEEE International Conference on (2011), IEEE, pp. 1211–1216. 1

MHD Yamen Saraiji, Charith Lasantha Fernando, Kouta Minamizawa, & Susumu Tachi / Development of Mutual Telexsitence System[OSS12] OYEKOYA O., S TEPTOE W., S TEED A.: Sphereavatar:a situated display to represent a remote collaborator. In Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (2012), ACM, pp. 2551–2560. 2[OSS∗ 13] OYEKOYA O., S TONE R., S TEPTOE W., A LKURDI
L., K LARE S., P EER A., W EYRICH T., C OHEN B., T ECCHIA
F., S TEED A.: Supporting interoperability and presence awareness in collaborative mixed reality environments. In Proceedings of the 19th ACM Symposium on Virtual Reality Software andTechnology (2013), ACM, pp. 165–174. 2[PS14] PAN Y., S TEED A.: A gaze-preserving situated multiview telepresence system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2014), ACM,
pp. 2173–2176. 2[SFF∗ 13] S ARAIJI M. Y., F ERNANDO C. L., F URUKAWA M.,
M INARNIZAWA K., TACHI S.: Real-time egocentric superimposition of operator’s own body on telexistence avatar in virtualenvironment. In ICAT (2013), pp. 35–39. 2[SFM∗ 14] S ARAIJI M. Y., F ERNANDO C. L., M IZUSHINA Y.,
K AMIYAMA Y., M INAMIZAWA K., TACHI S.: Enforced telexistence: teleoperating using photorealistic virtual body and haptic feedback. In SIGGRAPH Asia 2014 Emerging Technologies(2014), ACM, p. 7. 3
[STB∗ 12] S TEED A., T ECCHIA F., B ERGAMASCO M., S LATER
M., S TEPTOE W., OYEKOYA W., P ECE F., W EYRICH T.,
K AUTZ J., F RIEDMAN D., ET AL .: Beaming: an asymmetrictelepresence system. IEEE computer graphics and applications,
6 (2012), 10–17. 2[TAH12] T ECCHIA F., A LEM L., H UANG W.: 3d helping hands:a gesture based mr system for remote collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference
on Virtual-Reality Continuum and its Applications in Industry(2012), ACM, pp. 323–328. 2[TKN∗ 08] TACHI S., K AWAKAMI N., N II H., WATANABE K.,
M INAMIZAWA K.: Telesarphone: Mutual telexistence masterslave communication system based on retroreflective projection
technology. SICE Journal of Control, Measurement, and System
Integration 1, 5 (2008), 335–344. 2[TMEHB11] TAKAYAMA L., M ARDER -E PPSTEIN E., H ARRIS
H., B EER J. M.: Assisted driving of a mobile remote presence system: System design and controlled user evaluation. In
Robotics and Automation (ICRA), 2011 IEEE International Conference on (2011), IEEE, pp. 1883–1889. 1[TMFF12]TACHI S., M INAMIZAWA K., F URUKAWA M., F ER NANDO C. L.: TelexistenceâĂŤfrom 1980 to 2012. In Intelligent

Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on (2012), IEEE, pp. 5440–5441. 2[YHK96] YOKOKOHJI Y., H OLLIS R. L., K ANADE T.: What
you can see is what you can feel-development of a visual/haptic
interface to virtual environment. In Virtual Reality Annual International Symposium, 1996., Proceedings of the IEEE 1996
(1996), IEEE, pp. 46–53. 2

c The Eurographics Association 2015.