ISWC '15, SEPTEMBER 7–11, 2015, OSAKA, JAPAN

Wearing Another’s Personality: A Human-Surrogate Systemwith a Telepresence Face
Kana Misawa
Interfaculty Initiative in Information Studies,
The University of Tokyo
7-3-1, Hongo, Bunkyo-ku Tokyo, JAPANk.misawa@acm.org

Jun Rekimoto
Interfaculty Initiative in Information Studies,
The University of Tokyo
Sony Computer Science Laboratories, Inc.
rekimoto@acm.org

ABSTRACT

ChameleonMask is a telepresence system that displays a remote user’s face on another user’s face. Whereas most telepresence systems are designed to provide the remote user withan existence via a teleoperated robot, the system uses a realhuman as a surrogate for the remote user. This is accomplished by the surrogate user wearing a mask-shaped displaythat shows the remote user’s live face, and a voice channeltransmitting the remote user’s voice. The surrogate user mimics the remote user by following the remote user’s directions.
In initial experiments conducted, the surrogate tended to be
regarded as the actual person (i.e., the remote user). We implemented applications the remote user gives the surrogate
directions visually. We conducted user studies to determine
how the remote user felt about giving directions to the surrogate and how the surrogate felt to be the body of the director.
In the studies conducted, the director had the confidence togo outside with ChameleonMask and the surrogate tended tofulfill the director’s requests and felt positive about being thesurrogate.

Figure 1.
The remote user aquires his/her surrogate wearing
ChameleonMask.

a fixed space. Many studies have been conducted with theobjective of overcoming these limitations and accomplishinga sense of “being there.” Autonomous robots[9] are telepresence systems that allow remote users to freely travel aroundan objective place by teleoperating the system. These systems
display a human face on their screens and replace wheels withlegs, which replicates the remote user’s presence and mobility
in a distant location. Autonomous robots have shown promisefor use in many scenarios. For example, some companies
have introduced these systems into their offices1 , and more
researchers are using these systems at academic conferences2 .
These systems produce satisfaction and a feeling of physicalpresence. In addition, local users also have a more realistic
sensation of the remote user’s presence when an autonomous
robot is used [17].

Author Keywords

telepresence; augmented human; wearable system;
ACM Classification Keywords

H.5.3. Group and Organization Interfaces:
supported cooperative work

Computer-

INTRODUCTION“If I had another body, I could go there.” When we are heavily occupied with matters or have an uncontrollable urge to go
somewhere, we often wish we had another body. Constrainedby time and location, it is impossible to travel to another placeanytime, anywhere. Therefore, teleconference systems areused in many companies. However, teleconference systems
are usually installed in a room; consequently, in such scenarios, a remote user can only communicate with people in

However, these systems are still imperfect. Substituting physical human existence in the real world is difficult, and many
limitations exist: 1) A difference in appearance influences human communication [18]; however, in conventional systems,
the physical appearance of the system and that of the remoteuser do not correspond. For example, although the system’sheight may be adjustable, in order to maintain its stability, themachine is often smaller than the user. 2) A mobile telepresence robot’s volume control system must be adjusted to theenvironment [5]. For example, when a remote user asked apresenter a question, the robot’s voice was too low, requiring

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
ISWC ’15, September 7-11, 2015, Osaka, Japan.
Copyright 2015 c ACM 978-1-4503-3578-2/15/09...$15.00.
http://dx.doi.org/10.1145/2802083.2808392

1

Double robot, http://www.doublerobotics.com/business/
Should I Attend a Conference Via a Telepresence
Robot?,
http://spectrum.ieee.org/automaton/robotics/industrialrobots/attending-conference-via-telepresence-robot
2

125

ISWC '15, SEPTEMBER 7–11, 2015, OSAKA, JAPANthe staff to repeat the remote user’s question instead. Anotheruser reported a volume level that was too loud owing to alack of available volume adjustment. 3) Even though a system is “autonomous,” it may still need human help every time
a problem occurs. Rae et al. [17] stated that tasks with highmobility requirements reduced the system’s performance. In
this scenario, special support is necessary in order to improvetask performance. Furthermore, controlling these robots in aremote place is difficult. Autonomous robots frequently encounter obstacles on the floor or stairs. As a result, the system does not blend well into local environments. If multiple
people simultaneously used autonomous robots, even large
spaces would become crowded.

to make eye contacts during video communication [13, 14,15]. Geminoid [21] is an android system that has a humanlike avatar. Its appearance is substantially real, but its mechanical structure is complex, such that the motion is limitedby the remote user’s movement. Although there are telepresence systems that convey the haptic senses, in addition tovision and audio [22], and enable a full existence experience,
they require an inordinate amount of resources. Our approach
uses innate human abilities in order to achieve telepresence.
Leveraging human skill via the Internet

The ChameleonMask system relies on a surrogate to relay theremote user’s presence. Some studies have also investigatedremote user controls in order to share experiences with local
users. Tele-Actor [4] is a system that is worn in order to communicate in a remote environment. Omnipresenz6 offers asocial service concept of visual sharing. Jackin is a system inwhich the local user transmits 360-degree vision to the remoteuser using a headset equipped with cameras [6]. ChameleonMask conveys not only the remote environment but also theremote user’s presence on a mask. Teroos is a wearable system, placed on the shoulders, in which a remote user communicates with a local user through a small robot [7]. Even
though, ChameleonMask is a wearable system in which thelocal user becomes a surrogate for the remote user and acts
in response to the director’s requests. Online marketplaces
that allow users to outsource small jobs or buy and sell expert
services 78 are increasing. We consider that ChameleonMask
can be a physical body marketplace that is analogous to Amazon Mechanical Turk 9 being a brain marketplace.

In order to address these problems, we propose a systemcalled ChameleonMask that a surrogate human utilizes to embody a remote user’s physical and social telepresence (Figure 1). Many telepresence systems use autonomous robots,or robots that behave like humans. By contrast, we propose
the use of an actual human.
In this paper, we introduce our concept and system design.
The main goal of this system is to have the surrogate regarded
as the director. However, there are some remaining issues to
be clarified beforehand: How do people react to the system?
How do people feel to be someone’s surrogate? Our initial
field tests indicated that people tended to regard the masked
person as the actual person. We hypothesized that the relationship between the surrogate and the director is importantand determines whether people think the surrogate is the actual person. Therefore, we decided to investigate two otherremaining key questions: (1) How does a remote user feel to
use this system in comparison to a telepresence robot? (2)How does the surrogate feel to lend his/her body to a director? We conducted experiments to determine the feelings in
the two kinds of roleshow remote users feel while using thissystem and how people feel to be someone’s surrogate after
experiencing being a surrogate.

CHAMELEONMASKChameleonMask is a wearable mask system that enables aremote user to be embodied through a surrogate. The face
expression of the remote user is streamed and the mask isused as a display. The remote user can communicate withlocal persons by sharing the surrogate’s eye sight and situation. The existence on the surrogate enables the remote user
to show his/her physical and social presence in the local place.

RELATED WORK

There are several art performances345 wears a display on
his/her body or face in an original context [19]; however; atelepresence study in which a local user becomes the remoteuser’s surrogate by wearing an iPad does not exist.

In the ChameleonMask framework, we define three primary
stakeholders: a remote user (the director), a local user (thesurrogate), and local party members. They might be affectedby the use of a telepresence system. The director is the person who wishes to participate from a remote place and gives
his/her surrogate directions concerning where to go and how
to act. The director acts as though he/she is present in the local place, but the surrogate operates within the location. We
call the local user the surrogate. The surrogate is the user
who receives directions from the director and acts on behalfof the director. The surrogate operates in accordance withthe director as the director’s remote body and shares the local
environment and situation through video and voice. The surrogate does not talk with local party members but lends his or
her body to the director. Therefore, the surrogate must mask

Human presence in telecommunication

Many telepresence systems for enhancing the remote user’s
presence have been proposed. Prior work has shown thatmovement [16], mobility [17] and gesture [1] are more engaging and enhanced in social telepresence. These systems
usually comprise a tablet, a speaker, and a microphone representing the remote user. While the displays of these systems are 2D, some researchers have also focused on using
3D face-shaped displays[11, 12]. One reason is that a faceshaped screen aids in effective communication by facilitating
eye contact [10]. Eye contact plays an important role in faceto-face communication; hence, many studies focused on how

6

Omnipresenz, http://www.omnipresenz.com/
Taskrabbit, https://www.taskrabbit.com/
8
Fiverr, https://www.fiverr.com/
9
Amazon Mechanical Turk, https://www.mturk.com

3

7

Tablet man, http://www.damndigital.com/archives/74517
4
Yamada Taro project., https://vimeo.com/82250584
5
iPad Head Girl, https://youtu.be/fLPMLJgGsiA

126

SESSION: TOWARDS NEW WEARABLE APPLICATIONS

Figure 2. System overview of two communication lines: 1) Public line is a channel between a director and local party members. Audio and detected
face Images are sent to the surrogate’s iPad. 2)Private line is a channel between a surrogate and a director. The director gives the surrogate direction
by some ways. Sharing surround environment from the surrogate to the director with eye sights and voices.

his or her character and presence in every way except bodily.
Local party members are people who see the surrogate andtalk with the director. For example, they could include the director’s colleagues or other participants at a conference. The
aim of ChameleonMask is for the remote user to communicate with them as if he/she was actually there.
Physical Presence and Face Display

Telecommunication must embody the remote user’s existence. Further, a wearable system should be incorporated into
fashion and fit the user in order for the degradation of social
interaction to not occur [20]. Accordingly, ChameleonMask
needs to balance technical function with the social weight oftechnology. We decided to show the director’s face on thedisplay worn by the surrogate to embody the remote user. In
our system, the remote director’s presence is produced using
the surrogate’s physical body. Then, we conducted field tests
in order to investigate how much that technology affects communication and social interaction.

Figure 3. The surrogate visited the city office on behalf of the director..

INITIAL FEASIBILITY STUDIES

We conducted two field tests to determine the feasibility of
our system’s distinctive features [3]. We wanted to ascertain
the natural reactions and feelings when people see the surrogate wearing a prototype system. The first field test involved
the surrogate going to city government’s office to obtain apublic document in place of the director(Figure 3). This test
examined whether a stranger could confirm the identity of thedirector. The second field test was conducted to determine
whether an acquaintance regarded the surrogate as the actual
person. In the test, the surrogate showed up at a grandmotheras her grandchild. The result was that people tended to regard
the surrogate as the director. Participants wondered whether
the actual person was the director wearing ChameleonMask
and communicating through the iPad. They believed the actual person was there.

Social Presence

Facial recognition is the primary method humans use to visually distinguish one another. Therefore, the mask-shaped
display must exhibit the director’s live face, and a voice channel should transmit the director’s voice. Methods for meeting these system requirements include face-mask projection.
There are several examples of a rear-projection systems thatproject an image onto a face-shaped screen [10, 11, 12].
Projection enables a high level of expression, such as facial
details and motion [2]. However, these techniques do notsuitably apply to wearable systems because of their lack ofprojector placement. Willis et al. [23] introduced a threedimensional (3D) printing custom optical element techniquein which a curved 3D optical display can be projected onto anarbitrary place without interfering with the surrogate’s face.
However, a flexible display that can be molded to the shape ofthe face, which is a topic for future research, would improve
this method. Before developing the mask, we constructed a
prototype system in order to experiment and ascertain people’s feelings regarding these kinds of systems.

In addition, we obtained user feedbacks and design improvements for the mask. One such improvement was systemweight. The prototype system was weighted to the front because the iPad was set on a monocular head-mounted display(HMD) consisting of a smartphone and a fresnel lens10 . The
structure of the HMD is made from cardboard. To counterbalance the weight, we utilized a full-face helmet. The mask
fit on the head, but it was heavy and claustrophobic to wear
for long time. Consequently, we needed to change the design of the mask. The second improvement needed was in
10

127

Hacosco, http://hacosco.com/en/.

ISWC '15, SEPTEMBER 7–11, 2015, OSAKA, JAPAN

Figure 4. The surrogate looks the landscape from iPhone outside camera
through HMD(Hacosco) and the director’s face on the corner. This viewis sent to the director. There is a fresnel lens on HMD.

the way how the surrogate is given the remote user’s direction. The director could give the directions via oral instruction, and the surrogate was able to understand what to do.
However, when the surrogate was in noisy surroundings, thedirection did not carry clearly. Therefore, we considered that
directions in script in combination with the oral one could be
helpful. Finally, the gesture direction needed improvement.
The surrogate could see the director in his/her peripheral vision on the HMD. The appearance of the director was small,but the surrogate said, ”The director’s appearance made me
feel relieved”. The director’s window showed his/her existence, but the size of the windows was too small to read any
gesture. Therefore, we designed another way to show the director’s gesture.

Figure 6. We use optical phenomenon on lcd display. A polarizing filter
is attached on camera to detect hand from black background. Detected
hand images are overlaid on the landscape the surrogate looks.

Public line

The remote director uses the public communication line inorder to communicate with local party members with a live
face image and voice communication (Figure 2). Further, we
implemented an application that sends the director’s face image at this size from the director’s PC to the iPad. The face
image is obtained via a USB camera with face recognition using Haar-like features. Voice communication is facilitated by
Skype, which runs as an iPad application. If the volume ofthe voices is too low, the surrogate can adjust the volume on
the iPad or use the private line to tell the director to raise hisor her voice.

SYSTEM DESIGN

ChameleonMask requires a display showing the director’s
face and a monocular HMD and camera to capture the outside view for the surrogate. At the points on the human facethat facilitate the size and weight for comfortable wear, weplaced an iPad mini, which is 7.9 in tall, to display the director’s face. We chose a Hacosco deluxe model HMD, which
is made from plastic, with a rubber band in order to reduce
the weight. An iPhone 6 was set in Hacosco in order to acquire the surrogates vision. The Hacosco HMD has a simple
structure; therefore, we combined a goggles frame with theHacosco HMD to add cushioning. Thus, the surrogate wasable to view the outside on the iPhone display, captured by
the rear-facing camera of the iPhone.

Private line

The private line relays communication between the surrogate
and the director (Figure 2). Video images and a voice channel are sent from the surrogate. The director has to mute thepublic audio line before using the private line. The surrogate
wears an earphone with a microphone in order to talk withthe director, but does not speak when the director is speaking
with local members. When the surrogate uses the private line,
he/she has to speak at a lower volume. In addition, when thesurrogate wears the mask, his/her vision is blocked. Consequently, to enable the surrogate to see, we attached a Hacosco
VR tool to the iPad. Surrogates can see immersive outside
images from an iPhone 6 camera placed on the Hacosco (Figure 4). The field of vision is narrower than that of humaneyesight, but the view from the iPhone is sufficiently clearthat the surrogate can see speech partners and walk around.
The iPhone video stream that the surrogate sees is sent to thedirector’s PC. Voice communication is conducted via Skype.

ChameleonMask has two communication lines. One is a public line and the second is a private line. The director switchesfrom private to public channel when responding to conversation partners.

Giving directions

We considered that the basic method of communicating directions should be simple. A human-robot system requires
preconfiguration and minute functions to accommodate highFigure 5. When the director send a message, the surrogate can see itoverlaid on landscape.

128

SESSION: TOWARDS NEW WEARABLE APPLICATIONS

level requests, whereas a human does not have such a needbecause a human can understand the circumstance and the remote user’s feelings via oral instructions. When the director
notices that his/her surrogate is about to make a mistake, or
misunderstands his direction, they can communicate. Second,
the manner in which the direction is communicated depends
on the contents of the task the director requests. Considering
these points, we implemented a basic application function for
directions.

Figure 7. Participants tried to control the telepresence robot and thehuman surrogate outdoor.

Directions in script mode: Script mode can be used whenpeople are in noisy environments where hearing is difficult.
In addition, it can be useful when persons locally are facing
the surrogate while the director is in private mode. When the
director uses the private line, the director’s voice is only heard
by the surrogate. The voice is muted on the public line, butpeople can still see the director’s lip moving, so they may feel
uncomfortable in front of the surrogate. Therefore, we developed an application that allows the director to enter his/her
directions via a script that overlays his/her surrogate’s view
(Figure 5).

Figure 8. Left: A director is talking with a store clerk. Right: A surrogate is entering into a fruit shop.

Directions in hand gesture mode: This mode conveys the
director’s hand gesture to the monitor (Figure 6). The director
sees the scene that the surrogate is viewing on his/her monitor. The surrogate can understand the director’s pointing, visually and imitate the gesture. This mode is useful in situations where the director cannot explain well how to act. For
example, oral directions such as “go straight” or “turn right”
are simple to understand, whereas directions such as “stroke
the horse” may be confusing because it does not state how to
stroke, so that the surrogate needs to interpret it. We use apolarizing filter to detect hand gestures. It is well-known thatthe light from an LCD display is polarized in the horizontal
direction. When the polarizing filter is attached to a camera
perpendicular to the LCD light, the image from the LCD isblocked by the filter [8]. By a technique using this opticalphenomenon, we can detect the director’s hand gestures fromthe black background. The extracted hand images are overlaid on the surrogate’s vision.

Experimental setup: We prepared the experimental test
course for the telepresence robot and the surrogate in theroom. When the telepresence robot went outside, we cleared
the way to prevent interference from passersby. The surrogate
wore a board with the word ”Testing” written on it during theexperiment and experienced no interference from passersby.
We explained to the shop owner that the participants would
come to buy some fruits in advance.
Task A: Teleoperation of the telepresence robot

Each participant teleoperated the Double robot11 to test its usability. The robot weighs 6.8 kg and has an adjustable heightin the range of 47-60 inches (101.6-152.4 cm). The Double
screen was an iPad 2 tablet computer with a diagonal screen
size of 9.7 inches (24.6 cm) and resolution 2048 ×1536 pixels. First, an operator instructed the participant on how tocontrol the system. Then, following several trials, the actual
tests were conducted. The route for the test course was prepared in advance. It started from the experimentation room,
went through the hallway, and outside the building to the gateof the university. The time each participant took to complete
the course was recorded to the second.

EX1: USER STUDY OF THE DIRECTOR ROLE

In the early field test, people tended to believe the surrogate
was the actual person. To prepare for future experiments,
we therefore needed to understand the experience and role ofthe director and the surrogate. Consequently, we conducteda user study to collect feedback from participants about thedirector’s experience. Participants also teleoperated a telepresence robot for comparison. The task involved guiding thesurrogate/telepresence robot through a pre-planned route inorder to buy a product at a local fruit shop.

Task B: Teleoperation of the human surrogate

Each participant acted as a director and teleoperated an experimental surrogate. Each surrogate followed the same test
course to the fruit shop and bought the fruit the director
wanted. First, an operator instructed the participant on how
to use and switch communication lines. Then, the operator
gave the participant a map showing the route to the shop. The
participant then guided the surrogate by reading the map.

Purpose: To investigate how the director feels about using
the body of the surrogate.

Experimental design: The results of our preparatory experiment indicated that the respective character of the director
and the surrogate has an effect on the impression of the experience. For example, the character of the director, such as
cordial or rude, can give the surrogate different impressions.

Participants: We recruited nine participants, four males andfive females, aged 22-28 years old, from a local university toparticipate in our study. The participants had no prior associations or experience with our projects. They were all beginners
of using telepresence system.

11

129

Double robot, http://www.doublerobotics.com/business/

ISWC '15, SEPTEMBER 7–11, 2015, OSAKA, JAPAN

Figure 10. Left: A director is pointing the cloth he wants to look closely.
Right: A surrogate is checking the clothes. The director’s face is shownon the display.
Figure 9. Results of Questionnairethan the telepresence robot. For example, the director did not
notice an approaching ambulance before the surrogate turned
her head toward the direction of the siren. There are large differences for Q5 and Q6. The telepresence robot has wheels,
but a typical comment such as this was given: “I don’t havethe confidence to go outside with a robot because I’m anxious
about not crashing into other people.”

Therefore, we trained a female experimental surrogate to act
at the same level of performance for all participants. In theexperiment, the surrogate wore black unisex outfits. The surrogate was not known to any participant; thus, the relationshipbetween the surrogate and the director was not affected byprior relationships. The user study consisted of four parts: 1)
system introduction, 2) system evaluation, 3) post-test questionnaire, and 4) interview. Each participant performed both
Tasks A and B, but the order was varied to exclude any learning effect.

EX2: USER STUDY OF THE SURROGATE ROLE

Our underlying concept is that human surrogates can replace
telepresence robots in their roles like actuators in their workfor their directors. However, there are fundamental questions.
Do they want to help others (i.e., the director)? How do they
feel when they lend their body as a surrogate? We consider
these to be important questions to answer in order to develop
our system framework for the future. Thus, we conducteda user study to collect feedback from participants acting as
surrogates in order to understand the surrogate experience.

Post-test questionnaire: We asked six questions via a questionnaire following the user study. The post-test questionnaire (seven point Likert scale, with seven being the best)comprised the following questions. Q1. Was it easy to teleoperate the robot/the surrogate? Q2. Did the robot/surrogate
act in accordance with your requests? Q3. Did you feel asense of immersion? Q4. Were you able to see the remote environment and communicate? Q5. Do you want to go outsideusing the robot/surrogate? Q6. Do you want to communicate
and send the robot/the surrogate to a remote place?

Experimental design: We designed a controlled laboratory
experiment in which experimental directors gave directions
to participants acting as surrogates. The user study consistedof four parts: 1) system introduction, 2) system evaluation, 3)
post-test questionnaire, and 4) interview.
Participants: We recruited eight participants, four males andfour females, aged 22-28 years, from the local university anda local office to participate in our study. The participants
recruited had no prior associations or experience with our
projects. In addition, we recruited a male director and a female director and trained them for the experiments.

Results and discussions: The results obtained are shown in
Figure 9. There are significant variabilities among the answers for the telepresence robot compared to those for thehuman surrogate. For operability, Q1, participants enjoyedcontrolling the robot, while their expressions were very serious in the tests. They were careful about not crashing into
walls and knocking over the system. The average time thatparticipants teleoperate the robot in experimental course was
315 s (human surrogate had an average of 115s). Female participants tended to exert more careful control than the males.
One participant made the following comment: “I felt relievedthat the surrogate walked along the street without detailed instructions from me.” For responsibility, Q2, participants had
trouble adjusting to moving and seeing underfoot. When the
director wanted to see the underfoot of Double, the director
had to switch the camera to underfoot. It was difficult for the
director to see underfoot for both cases, but the human surrogate avoided obstacles without the director’s instruction. For
the sense of immersion, Q3, there was no major difference.
One participant stated the following: “I felt as if I was teleoperating the robot and the surrogate in a control room.” Participants did not feel truly immersed. The answers to Q4 indicate that the human surrogate shared the environment more

Experimental setup: We wrote acting scripts so that the directors could give the same attitude to all surrogates. They
communicated with the surrogates using audio, directions
from the script and hand gestures. The male director wasplaced in charge of the male surrogate participants, while thefemale director was placed in charge of the female surrogate
participants. The directors were known to the surrogates.
Experimental scenario: Each surrogate assisted in buying
clothes for the director. The director was not able to go outside for some reason. He/she could buy the clothes on theInternet but he/she wanted to experience the pleasure of shopping in a store. Thus, the director asked the surrogate to assist
with the shopping. After walking around the store, they selected five kinds of clothes. Finally, the surrogate was asked
to help the director to make the final decision on what clothes
the director should buy.

130

SESSION: TOWARDS NEW WEARABLE APPLICATIONS

Post-test questionnaire: We asked eight questions via questionnaire after the user study. The post-test questionnaire results (seven point Likert scale) and mean and SD are shownin Table 1.

the city officer. The city officer eventually asked the surrogate to take the mask off, but he talked to the surrogate fora while. We believe that this is because of the facial images
displayed on the top of the surrogate’s body. Local participants acknowledged the surrogate’s humanity and thought
they confirmed her identity. Therefore, they did not ignore
the surrogate but treated her like a human being.

Results and discussion: All participants helped the directors
very well according to the request. Many participants said
hand gestures were easy to understand visually. We observedthat the some participants opened the zipper of a parka when
the director gave the gesture for unzipping. They seemed tounderstand what the director wanted to do. They did not feelthat the tasks were difficult. We hypothesized that it is difficult for people to accept becoming someone’s surrogate. Surprisingly, the participants tended to feel positive about beinga surrogate. In addition, many participants tried to work voluntarily and wanted the director to give them more requests.
One participant said, “I wanted to hear the requests about
what the director wanted to do.” “The director did not ask me
to, but I was willing to wear the parka to show the director
the fit.” In addition, many surrogates explained the texture to
the respective director because the director could not touch
the clothes and so they did not know the tactile sensation. In
this user test, most participants did not need to be preparedwith special training in advance. Some wanted to get accustomed to looking at the landscape through the VR becauseof its narrow field of vision. Finally, participants stated thatthey foresee the possibility of a job in which people become
surrogates for others.

While we were testing the operation of the telepresence robot
in EX1, we met some strangers who got interested in thetelepresence robot and took a picture spontaneously with asmartphone without authorization. However, they did nottake a picture of the surrogate wearing the mask, even though
they stared and seemed to find it interesting. This incident appears to indicate that the strangers regarded the teleoperated
target as a person or a thing.
Telepresence Robot vs Human Surrogate

Our comparison of the usability of the telepresence robot andthe human surrogate is not about winning or losing. We believe that different systems are suitable for different purposes
because various kinds of systems are developed. Therefore,
there are tasks that human surrogates are useful for, such as
the tasks in this study, and there are tasks that telepresence
robots are more useful for. For instance, a participant said
that he/she was ashamed to use the surrogate for easy tasks.
In addition, when a director wants to talk with a local person
about complicated private things, he/she should talk face-toface without the surrogate. We therefore need to investigate
what kinds of scenarios for which this system is most applicable.

Table 1. Results of Questionnaire

No.

Question

Mean

SD

1

Did you understand the director’s
intention?

6.13

0.64

2

Was it difficult to carry out director’s request?

3.38

2.39

3

Did you feel uneasy getting direction?

2

0.93

4

Were you willing to work for the director?

5.38

0.74

5

Did you prefer getting many requests from the director?

5.5

1.31

6

Did you feel like you were helping
the director?

5.88

0.35

7

Do you think you would need toprepare in order to be the director?

4.25

1.67

8

Do you think the job of lending the
body is possible as a part-time job?

5.63

0.52

What Happens When a Human Teleoperates a Human?

There is a concept that states that “using human is cheaper
than developing robot,” but a human is difficult and very sensitive to deal with. This is because some uncertainty elementsinteract in humans, such as character and the relationship between the director and the surrogate. We consider that it isbetter for the surrogate to look like the director, while thedifference between them can complement each other. People should become the surrogate specialized for their strong
points and abilities, or we should ask that of them. Tasks
for which the surrogate needs to know the director well in advance should be assigned to their mutual friends. Professionaltasks that require experts should be assigned to professionals.
ChameleonMask has the potential to create new employment.
People felt positive being surrogates in our test. Of course,
it depends on the tasks the directors wish to perform. However, participants were willing to help the director. They felt
uneasy when the director did not give directions. It was very
interesting that some participants felt easy and comfortablebeing surrogates. One participant stated, “I didn’t need tomake my own decisions, I just had to do what the director
wanted.”DISCUSSION
Physical and Social Telepresence

When do we use ChameleonMask?

In our system, the remote user teleoperates a human insteadof a robot. In early field tests, we hypothesized that people
would be skeptical of the mask, look at it in disbelief, andimmediately request that the surrogate remove it in the experiments. However, we did not observe this reaction from

The main goal of this study was to convey the director’s existence to a remote place using a human. This system is therefore useful for many occasions. Furthermore, this system enables persons to borrow not only a body but also someone’s

131

ISWC '15, SEPTEMBER 7–11, 2015, OSAKA, JAPAN8. Koike, H., Nishikawa, W., and Fukuchi, K. Transparent
2-d markers on an lcd tabletop system. Proc. of CHI ’09,
163–172.

face (in contrast to lending a face to someone). This is because a face and a body are separate. Example, borrowing
a body: When I want to make an impact at a presentation, Iborrow a bigger body to make a strong impression. Example, borrowing a face: When I want to have a lot of admirers
among the ladies, I borrow an idol’s face.

9. Lee, M. K., and Takayama, L. ”Now, I have a body”:
Uses and social norms for mobile remote presence in theworkplace. Proc. of CHI ’11, 33–42.

The present mask may be difficult to use in practical scenarios
because of the weight, field of vision, and appearance. However, this study is simply an initiation to judge this concept
prior to a 3D live face-shaped mask being produced.

10. Misawa, K., Ishiguro, Y., and Rekimoto, J. Livemask: A
telepresence surrogate system with a face-shaped screen
for supporting nonverbal communication. Proc. of AVI’12, 394–397.
11. Morishima, S., Yotsukura, T., Binsted, K., Nielsen, F.,
and Pinhanez, C. Hyper mask projecting a talking head
onto a real object. Proc. of MMM ’00, 403–412.

CONCLUSIONS AND FUTURE PLANS

We hypothesized that a human surrogate can embody a remote user. In early experiments, the surrogate tended to be
regarded as the actual person. As a precursor to making a 3D
live face-shaped mask, we introduced the concept and the system requirements. We also conducted user studies to understand the experience of persons carrying out the directors roleand the surrogate’s role. The results indicate that ChameleonMask is feasible for use. The director had the confidence to go
outside with ChameleonMask. Further, the surrogates tended
to want the director’s requests and felt positive about beinga surrogate. In the future, we would like to conduct experiments in practical scenarios. Finally, we would like to make3D live face-shaped masks to actualize this concept.

12. Lincoln, P., Welch, G., Nashel, A., Ilie, A., and Fuchs,
H. Animatronic shader lamps avatars. Virtual Reality’11,
15(2-3)225–238.
13. Yang, R.,and Zhang, Z Eye Gaze Correction with
Stereovision for Video-Teleconferencing. TPAMI’04,
26(7)956–960.
14. Kuster, C., Popa, T., Bazin, J-C., Gotsman, C., andGross, M. Gaze Correction for Home Video
Conferencing. Trans. Graph’12, 31(6)174:1–174:6.
15. Kjeldskov, J., Smedegard, K.Y., Nielsen, S.T., Skov,
B.M., and PaayLincoln, J. EyeGaze: Enabling EyeContact over Video. Proc. AVI’14, 15(2-3)225–238.

REFERENCES

1. Adalgeirsson, S. O., and Breazeal, C. Mebot: A robotic
platform for socially embodied presence. Proc. of HRI’10, 15–22.

16. Nakanishi, H., Kato, K., and Ishiguro, H. Zoom cameras
and movable displays enhance social telepresence. Proc.
of CHI ’11, 63–72.

2. Bermano, A., Brüschweiler, P., Grundhöfer, A., Iwai, D.,
Bickel, B., and Gross, M. Augmenting physical avatars
using projector-based illumination. ACM Trans. Graph.
32 ’13, 189:1–189:10.

17. Rae, I., Mutlu, B., and Takayama, L. Bodies in motion:
Mobility, presence, and task awareness in telepresence.
Proc. of CHI ’14, 2153–2162.

3. Kana, M., and Jun, R. ChameleonMask: Embodied
Physical and Social Telepresence using humansurrogates. Proc. of CHI ’15 EA,

18. Rae, I., Takayama, L., and Mutlu, B. The influence ofheight in robot-mediated communication. Proc. of HRI’13, 1–8.

4. Goldberg, K., Song, D., Khor, Y., Pescovitz, D.,
Levandowski, A., Himmelstein, J., Shih, J., Ho, A.,
Paulos, E., Donath, J., Berkeley, U., and Mit, J. D.
Collaborative online teleoperation with spatial dynamic
voting and a human ”tele-actor”. Proc. of ICRA ’02,
1179–1184.

19. Jennifer, G.S., Alan, D., Simon, L., and Alice, B.
Understanding interaction in ubiquitous guerrilla
performances in playful arenas. A framework forunderstanding and designing telepresence. ’05 , 3–17.
20. Toney, A., Mulley, B., Thomas, B.H., and Piekarski, W.
Minimal social weight user interactions for wearable
computers in business suits. Proc. of ISWC. ’02, 57–64.

5. Hayamizu, A., Imai, M., Nakamura, K., and Nakadai, K.
Volume adaptation and visualization by modeling thevolume level in noisy environments for telepresence
system. Proc of HAI ’14, 67–74.

21. Sakamoto, D., Kanda, T., Ono, T., Ishiguro, H., andHagita, N. Android as a telecommunication medium
with a human-like presence. Proc. of HRI ’07, 193–200.

6. Kasahara, S., and Rekimoto, J. Jackin: Integrating
first-person view with out-of-body vision generation forhuman-human augmentation. Proc. of AH ’14,
46:1–46:8.

22. Tadakuma, R., Asahara, Y., Kajimoto, H., Kawakami,N., and Tachi, S. Development of anthropomorphic
multi-d.o.f. master-slave arm for mutual telexistence.
IEEE Trans.on Graphics 11 ’05, 626–636.

7. Kashiwabara, T., Osawa, H., Shinozawa, K., and Imai,
M. Teroos: A wearable avatar to enhance joint activities.
Proceedings of CHI ’12, 2001–2004.

23. Willis, K., Brockmeyer, E., Hudson, S., and Poupyrev, I.
Printed optics: 3d printing of embedded optical elements
for interactive devices. Proc. of UIST ’12, 589–598.

132