「エンタテインメントコンピューティングシンポジウム（EC2014）」2014 年 9 月

テレイグジスタンスの研究（第 78 報）- AR によるレイヤー情報を使用したテレイグジスタンスへの応用Study on Telexistence LXXVIII
Layered Information Overlay on Telexistence
using Augmented Reality Techniques
CHAIWAT MEETANATHARVORNMHD YAMEN SARAIJI
CHARITH LASANTHA FERNANDO KOUTA MINAMIZAWA
SUSUMU TACHI
Graduate School of Media Design, Keio University, Japan. {m_code, yamen, charith, kouta, tachi}@tachilab.org
In this paper, we present a method for providing augmented reality information based on a selective layered based hierarchy inubiquitous telexistence robots. User can receive selective information dynamically depend on the interest and the distance
between remote robot and desired object. Layered information is rendered on the remote environment via image based marker
boards that also acts as information cards or posters for local users in the remote environment. In this paper, the implementationof the image based AR posters, frame templates in ubiquitous telexistence robots has been discussed and the effectiveness oflayered information is evaluated through a set of experiments.
Keyword: Augmented Reality, Telexistence, Robot

1. Introduction

a technique to selectively display the required content whennecessary.

With the advancement of technology, telecommunication
infrastructure has become popular in day-to-day interactions
among humans. Video conferencing, Skype and similar
technologies allow expressing ones visual and auditory
expressions. With busy lifestyle people use these technologies toconnect with people remotely, yet still many of them prefer tomeet their friends/family physically specially if they are aroundbecause lack of ability to present oneself in a remoteenvironment. Telepresence and Telexistence robots are twotechnologies available for presenting yourself in a remote

Figure 1: Layered information overlay with 3 levels ofenvironment with ones own movements. Thus the users feels

information

more engaged with the remote area as if they were physically

The information is selected in a way that it provides thepresent there. With telexistence, we have achieved human-likenecessary information to the operator depending on his

neck movements to visually interact with a remote object in 3

preference and what he is looking at the moment. Moreover, by

dimensional spaces through previous versions of TELESARV

providing augmented reality layered information in telexistence,[1] [2]. Even though telexistence allow users to feel that they

it can help navigating to where the user is intended to visit,

existed in the remote environment, due to the narrow FoV andreduce time consumption in finding something that is interested,

the quality of the video, it is sometimes hard to find thing thatand reduce the need of asking for help from remote participants.

have not being seen before. Moreover, navigation inside a

Proposed layered Information is further categorized such asbuilding is very difficult as the reference frame is not known

object properties, personal information and virtual controllers

and it sometimes confuses the user.

for house hold appliances. With this system, user can feel

To address this issue, we provide layered AR information in

comfortable to look at the object he/she is interesting andthe telexistence environment when the operator needs thereceive the appropriate information at the appropriate distance.

information. However, too much information will confuse theUser can get closer to the object to learn more. This kind ofoperator and thus may disturb his/her vision that can make itinteraction is very intuitive for human behavior [3]. As a result,

hard to perform remote manipulations. Therefore, we introduce

user have no need to learn how to select information, just oneⓒ2014 Information Processing Society of Japan1

sentence of instruction, user will know how to do the interaction

Personal space

0-75

centimeters

Social space75-120centimeters

Public space120-760

centimeterswhich increase the ease of use and satisfaction on user interface.
Above layered information based guidance can be applied for
not only robot but also for everyday head mounted display
techniques. In recent technology, it can be say that the future

Table 2: Meaningful distance in humanhead mounted display can become a widely use and augmented
reality itself will become one of the important technology. At

2.3 Remote Virtual Controllerthat time, layered based information also can be applied for any

Similar to analog data in name card information, there may bekind of information provider and become standard of interaction

various home appliances in the remote environment that the user

for such method.

wants to interact. However, the analog interface between the
remote side and the users vision does not allow interacting.

2. Design Considerations

Therefore we propose grabbing a virtual copy of existing

The design is to provide 3 types of overlay information as

controllers in to your hands and then control the device via it.

below. To detect where to display AR content, image based AR

For example, if the user wants to turn ON the TV on the remotetags are being used. These tags are similar to project informationside, he could grab a virtual copy of the real remote from thecards or posters in a exhibition venue. This local participants

place that real remote is placed. This virtual copy is fully

can get enough information by reading the poster information

functional and it can be a gateway between the remote appliancewhile the same poster can act as an AR marker for online

and the telexistence operator. This way operator not only feels

participants so that more dynamic content can be rendered on tothat his body is situated in the remote side, but also he canthe telexistence vision system. Here is the table shows relation

naturally interact with the devices n the other side.

between distance and interest in human behavior.
High interest

10 - 120

centimeters

Medium interest

120 - 230

centimeters

Low interest

230 - 500

centimeters

3. Implementation
3.1 Hardware System Overview

Table 1: Relative between distance and interest in
human behavior
These overlay information will provides as Low, Medium andHigh depend on distance thus, when user feel more interest in
project, he/she can move closer to get more information.
2.1 Object properties
In this category, it is mostly focus about different object
properties in the remote environment. Depending on the users
background such as engineer, designer, or sales person, theseoverlay information can be customized. Furthermore, when the

Figure 2: Hardware System Overview

user goes closer to the object, the insides of the object can be

There are 2 parts of the hardware system, Master side and slave

seen in various layers. Here again the inside information can beside. On the master side, the telexistence user sits in a cockpit

customized based on the user preferences.

where his body motion is tracked and sent to the remote side.
On the slave side, Wi-Fi (Wireless 802.11ac) is used to repeat

2.2 Personal Information

the wireless data that is received over Internet. A custom madeName card is traditional way to provide information for local

robot, TELUBee v.2 is used as the robot base platform. The

audience. However, small name cards cannot be seen whenrobot can locomotion in indoors when the user manipulates via aattending in a remote environment, furthermore these analog

Wii Pad. Robots head motion is synced with the users headdata cannot be used to interact with that person. Our design is tomotion. Robot has 3 Degree of Freedom that can rotate, pan, anduse existing information as marker to provide 3 layered oftilt just like human head without torso and neck. The user would

information as same as object properties. Here is the table shows

be able to control robot head by using intuitive movement of

meaningful distance in human [4]user’s head. Oculus motion sensors detect the movement andⓒ2014 Information Processing Society of Japan2

send it to computer to generate the data that can control 3
desired motors on the robot.
Robot head uses Full HD camera with 1920×1080 pixels with
90-degree wide-angle field of view (FoV). On the software part,
displaying system was integrated with AR cameras that cancatch AR code and render 3D virtual object augmented into thereal world. User cannot recognize AR camera and user will

Figure 4: Object Properties Information implementation

know when he saw the marker and 3d virtual object was
rendered.
By using Arduino connect with network enabled Unity

Contents/TypeLow

MediumHigh

Content

1. Project Name

1. Project name

1. Project name

2. Project Picture

2. Project video

2. System Overview

500 - 230c.m.

230 – 75c.m.

75 - 0c.m.

platform [5], it can synchronize overlay information interface inreal time interaction. The implementation use Arduino as aprocessor to control the controllable object such as LED in thisDistance

Table 3: Object Properties Information Hierarchy

implementation. User can use IR controller to turn on and off
the light by click it when user saw the AR control panel that willshow when user look at specific objects.
3.2 Software System Overview

Figure 5: Personal information implementation
Contents/Type
Content

Public
1. Person nameSocialPersonal

1. Person name

1. Personal name

2. Status

2. Personality

3. Nationality

3. Favorite

4. Hobby

personality

5. Skill

Figure 3: Software System Overview
Above system is implemented using Unity Game engine, so

Distance

760 -120c.m.

120 – 75c.m.

Lower than 75c.m.

Table 4: Personal Information Hierarchy

that a community level can do any further development. This
will be greatly helpful when the system is being used incommon exhibitions and demonstrations where the designers
can easily design their own template and integrate into thesystem.
System uses 3 cameras to provide image data from robot touser display. 2 × Microsoft Lifecam Studio cameras mergedwith AR camera with 320×240 pixels with Metaio SDK [6] canprovide overlay information into front of video texture that

Figure 6: Virtual Controller Implementation

render real world image data then use Oculus camera lens

System uses overlay information as an interface to control

correction shader to filter and distort the image data from left

television by using AR marker [7]. It can provide real-timeand right eyes camera.

remote control by using Arduino to send data to AC voltage
controller and then control electricity flow to the television.

3.3 Categorize information based on distance
System categorizes information based on meaning distance

4. User Study

between human and object, human and human by using

To find out whether using layered information in Telexistence

information card as a marker in project information and using

robot can decrease time consumption on finding object

name card for provide personal information as following.

compared to paper based information, a user study wasⓒ2014 Information Processing Society of Japan3

conducted. The experiment was held in “Living Laboratory

resolution is not so good. And one of the subject feel sick by

Tokyo” at Miraikan. 5 subjects were participated with aged

stereoscopic. One of the subject said that with AR, Telexistence

around 20 – 40 years old with random background. Subjects

can be applied for using in different propose and the applicable

were given enough time to adjust to robot control before theuse would be wider. Some response said that having AR can get

experiment begins. The experiment was started with laying 3

attention and it is very easy to use.

object to let the subject find in the area of 5m× 5m in a normal
living environment. The position of each object is randomized

5. Conclusion

on each trial. The evaluation is based on the performance factor

Lack of information in Telexistence makes it difficult whenor time taken to find each objects. Each trial is repeated 2 times

achieving a specific goal such as finding object, getting personalwith AR contents and without AR contents as an aid to help theinformation and controlling home appliances. Moreover, tooparticipants.

much information confuses the user. By providing overlay

Time consumption insecondinformation in Telexistence, even resolution is low, FOV is

300

129

200

small, user still can easily achieving a specific goal by reducing
time, reduce physical action and knowing information without

8

asking. Moreover the information is categorized into layers and100

implements it into 3 type of information to display such as

0

1 st
user

2nd
user

3rd
user

4th
user

5th
user

Paper based

209

157

220

180

240

Layered AR

160

149

123

98

111

object properties, personal information and virtual controllers.
As a result from user study, using layered information in
Telexistence robot will decrease time consumption compare to
paper based information when finding object in living

Table 5: User Study Result
The result said that using Layered information in Telexistence
can reduce time consumption on finding object in average 73
seconds compare to paper based information. Furthermore,acccording to discusion, there were interesting points such as
one of the testers is very familiar with the room and knows theobject very well so the result becomes faster. The problem theyenvironment.

Acknowledgement
This project is supported by JSPS Grant-in-Aid for Scientific
Research A (23240017).

References[1]

Tachi,

Telexistence,

World

Scientific

Publishing

Company ISBN 978-981-283-633-5, 2009.

face is how to control the body. One subject said that it is very
convenience to use head movement for look around the

S.

[2]

C. Fernando, M. Furukawa, T. Kurogi, S. Kamuro, K. K.

environment but the resolution of the display is low so sometime,

Sato and S. Tachi, "Design of TELESAR V for transferring

if the subject found the object but he cannot read the detail orbodilyeven see it clearly. Another subject said that the sign of AR ispp.5112-5118.

also very small and hard to read but the color is fine. And tester[3]

also commented that it will be more fun if they can control

consciousnessintelexistence,"

2012,

pp.

R. OPPERMANN, User-interface design, Springer Berlin
Heidelberg: Handbook on information technologies forsomething more than just a light. For example, a fan, light of theeducation and training, 2002.

room or interact with a tree when tree want more water or thetemperature is not appropriate. The zoom function is important[4]

Distance,"[Online].

Available:

g-distance/. [Accessed 28 5 2014].

controller to control the locomotive movement. If there is zoom
[5]

"Unity3d,"

Unity3d,[Online].

Available:

http://japan.unity3d.com/. [Accessed 7 5 2014].

subjects also mentioned that if the head go beyond 180 degree
and up to 360 degree, subject would be able to look around theandhttp://qrworld.wordpress.com/2011/07/16/qr-codes-scannin

because subject doesn’t want to get close to the object by using
in and out function, it will be more convenient. Moreover, few"Scanning[6]

room without rotation the body, which may be useful for finding

K. Schiller, "Augmented Reality Comes to Market.," inInformation Today26, 11(2009), pp. 45-46.

the object. Somehow, the interaction is very new in the user
experience and subject are not familiar in the first place butwhen they learned and aware that if they get closer, they will get
more information, they tend to use it with different AR marker
to get information also. 2 out of 5 subjects said that theⓒ2014 Information Processing Society of Japan[7]

H. Kato, ""A city-planning system based on augmented
reality with a tangible interface." Mixed and Augmented
Reality, 2003," in The Second IEEE and ACM International
Symposium on., 2003.

4