# Qualitative assessment of an immersive teleoperation environment for collaborative professional activities in a "beaming" experiment

Guillermo Gomez<fnref n="1" />, Carole Plasson<fnref n="2" />, Frédéric Elisei<fnref n="2" />, Frédéric Noël<fnref n="1" />, Gérard Bailly<fnref n="2" />,

## Abstract

The current study assesses an immersive teleoperation platform for professional collaborative activities. This platform uses an iCub robot and virtual-reality hardware (VR headset and motion capture) to elicit an embodiment situation, where the pilot collaborates with remote actors through the robot ("beaming platform"): through the robot, the pilot can act and perceive using his natural affordances (head moving, vision and hearing) that are mapped to/from the robot effectors/sensors. One challenge is to measure how the robot expression capacity level can modulate the efficiency of the participation of the remote pilot with his collaborators. An experience where the pilot trains two subjects to assemble a mechanical system was organised: we observed the capacity of the two subjects to realize the task they discover and to collaborate with the pilot, with a more or less expressive robot. This experience clearly highlighted the importance of embodiment and the opportunities opened by beaming platforms, allowing the enhancement of the physical space by agents:i.e. social robots or conversational agents.

Categories and Subject Descriptors (according to ACM CCS): I.2.9 [Computer Graphics]: Robotics/Operator interfaces—H.1.2 [Information Systems]: User/Machine Systems/Human factors—H.5.3 [Information Systems]: Group and Organization Interfaces—

## 1. Introduction

The ever-growing technologies in the fields of telerobotics have led both the scientific and industrial community wonder when and how they can start to adapt and deploy these technologies in their respective fields of work. A particular application has turn out to be particularly interesting: gifting a person with the capacity to transport to a distant location, thanks to a humanoid robot that serves as a platform between the individual and another group of people on a determined physical location, for acting there as well as perceiving from there : it is a beaming [STB* 12] process where the individual is replicated by the robot, and common affordances are used (for example, moving one's head or hand to move the robot's head or hand) and sense the environment accordingly.

**Beaming** refers to technologies digitally transporting a representation of yourself to a distant place, where you can

interact with the people there as if you were there (acting, perceiving). The possibilities for such a technology are tremendous, and this research focuses on the industrial engineering application field, which is often a collaborative work.

Although humanoid robots are not expected to replace humans, they could be very helpful in some particular tasks. However there is still plenty of work to design robots that can be socially and industrially competent, in order to have a satisfying and successful interaction with humans in diverse environments and domains.

It is important to study how humans react when interacting with a robot instead of a real person, and the impact on their behaviour. Humans are very sensitive to verbal and non-verbal communication, and humanoid robots are still being developed in terms of their "human-likeness" and

<footnote n="1">¹Univ. Grenoble Alpes, G-SCOP, F-38000 Grenoble, France
CNRS, G-SCOP, F-38000 Grenoble, France</footnote>

<footnote n="2">²Univ. Grenoble Alpes, GIPSA, F-38000 Grenoble, France
CNRS, GIPSA, F-38000 Grenoble, France</footnote>
# Qualitative assessment of an immersive teleoperation environment for collaborative professional activities in a "beaming" experiment

Guillermo Gomez, Carole Plasson, Frédéric Elisei, Frédéric Noël, Gérard Bailly

► To cite this version:

Guillermo Gomez, Carole Plasson, Frédéric Elisei, Frédéric Noël, Gérard Bailly. Qualitative assessment of an immersive teleoperation environment for collaborative professional activities in a "beaming" experiment. EuroVR 2015 - European conference for Virtual Reality and Augmented Reality, Oct 2015, Milan, Italy. 8 p. hal-01228890

HAL Id: hal-01228890

https://hal.science/hal-01228890

Submitted on 7 Mar 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
"sociability". The evaluation of a beaming platform implies
many different features to study in order to provide a deep
analysis of the strengths and weaknesses.

**The current study** uses a system of interaction in a multi-site collaborative work, allowing collaborators to be "telerepresented" by a humanoid robot with an active and physical presence. An experience was organized to perform a remote assembly training scenario with the confrontation of different objects. It is expected to check the condition for an actor to participate intuitively from a remote location. The same experience is run while degrading the functions of the environment to identify the most important functions which are associated here with the robot degrees of freedom allowing more or less expression transmission.

The next section introduces the technical state of the art
about remote teleoperation. Then a short state of the art
presents the domain of application we are concerned with
(collaborative design activities). At this step the experience
built for the current study will be described and the final sec-
tion will introduce the first level of the results from this ex-
perience.

## 2. Remote teleoperation issues

**2.1. Beaming platforms**

Telerobotics [She89, She92] is the core technology of the
assessed environment. It allows to remotely drive a robotic
system. Robots are an extremely good alternative to humans
for multiple applications (medicine, military, nuclear plant
maintenance, construction) and also for collaborative distant
work.

The "Wizard of Oz" (WoZ) refers to a person remotely
operating a robot [Rie12], controlling its movement, nav-
igation, speech, gestures, for example by triggering pre-
defined actions by pressing keys or teleoperating movements
of some body segments from a mouse, a joystick or any other
motion capture device. WoZ controls a robot from fully au-
tonomous to fully tele-operated. The controlled robot serves
more as a proxy for humans than as an independent entity.
A valid WoZ simulation expects [Rie12, GPL*12] "to simu-
late the future system, given human limitations", "to specify
the future system's behaviour" and "to make the simulation
convincing".

Part of telerobotics, the beaming concept (Being in Aug-
mented Multimodal Naturally Networked Gatherings) was
introduced [STB*12]. Beaming is the name for the process
that allows someone - here the pilot - to instantaneously
transfer himself from one physical place to any given dis-
tant location, thanks to the monitoring of the pilot's actions,
physiological states and emotions which can be streamed
over the internet while simultaneously streaming visual, au-
dio, spatial, and context back from the distant destination.
This flow of data is synchronized and the result is an unified

virtual environment representing the physical space of the
destination in real time. The destination is thus inhabited by
both local and virtual users.

Beaming allows to be embodied in a virtual interface. The
destination/visitor interface is non-symmetrical, but beam-
ing is defined to support symmetric social interaction be-
tween the visitor and locals. Thus an intensive attention is
given to the quality of the visual immersion for the pilot,
copying the physical remote destination. But indeed visual-
isation is just a part of a successful beaming platform, since
all human senses should participate to the embodiment expe-
rience up to the "self-perception" of the avatar in the remote
location.

With a virtual filter, human senses may be tricked intro-
ducing a infinite range of applications where perception may
be faked [NSVW*12] and where awareness becomes a real
ethical issue. Gathering immersive virtual reality environ-
ment with tele-operated systems opens applications which
otherwise would be impossible to achieve. For example,
[BEFS15] uses a humanoid robot, tele-operated by a human
pilot to artificially provide the robot with social skills.

Such a WoZ platform was used to experience effects
of culture in human robot interaction and social robotics
[BV13] highlighting the role of head nods for social inter-
action. A basic analysis grid is used to assess a robot perfor-
mance through various subjective criteria: likeability, engag-
ing, satisfaction, useful, human-likeness, efficiency, credibil-
ity, human-presence.

[GPL*12] focused on the perception of gaze and head
movements through an iCub robot. It allows the interaction
to get closer to humans because it feels more "human-like",
with the help of another sensor for capturing gaze and head
motion.

**2.2. Human robot interaction**

With a telerobotic beaming platform, a human robot interac-
tion is taking place at the shared physical place. Many key
issues arise then [PDGA14]: mutual interaction learning, hu-
man presence undertaking, integrating social rules and pro-
tocols, as well as legible motion. There are also more specific
issues like: robustness and efficiency, comfort and intention-
ality to specify robot behaviour, the capacity of the robot to
adapt and plan according to its environment just like a hu-
man would do it, among many others.

[BCBJ*09, SLMF06, CKH*15] deeply analysed the ef-
fects of head nodding that can be simulated or reproduced
by a robot. They analysed how people use visual feedback
while conversing with robots. Humans apparently nod at the
robot during conversation.

[CKH*15] debates over social cues of interaction as well.
They assume that human robot interaction comes only when
the capacities of perception of the environment as much as
# Qualitative assessment of an immersive teleoperation environment for collaborative professional activities in a "beaming" experiment

Guillermo Gomez<fnref n="1" />, Carole Plasson<fnref n="2" />, Frédéric Elisei<fnref n="2" />, Frédéric Noël<fnref n="1" />, Gérard Bailly<fnref n="2" />,

## Abstract

The current study assesses an immersive teleoperation platform for professional collaborative activities. This platform uses an iCub robot and virtual-reality hardware (VR headset and motion capture) to elicit an embodiment situation, where the pilot collaborates with remote actors through the robot ("beaming platform"): through the robot, the pilot can act and perceive using his natural affordances (head moving, vision and hearing) that are mapped to/from the robot effectors/sensors. One challenge is to measure how the robot expression capacity level can modulate the efficiency of the participation of the remote pilot with his collaborators. An experience where the pilot trains two subjects to assemble a mechanical system was organised: we observed the capacity of the two subjects to realize the task they discover and to collaborate with the pilot, with a more or less expressive robot. This experience clearly highlighted the importance of embodiment and the opportunities opened by beaming platforms, allowing the enhancement of the physical space by agents:i.e. social robots or conversational agents.

Categories and Subject Descriptors (according to ACM CCS): I.2.9 [Computer Graphics]: Robotics/Operator interfaces—H.1.2 [Information Systems]: User/Machine Systems/Human factors—H.5.3 [Information Systems]: Group and Organization Interfaces—

## 1. Introduction

The ever-growing technologies in the fields of telerobotics have led both the scientific and industrial community wonder when and how they can start to adapt and deploy these technologies in their respective fields of work. A particular application has turn out to be particularly interesting: gifting a person with the capacity to transport to a distant location, thanks to a humanoid robot that serves as a platform between the individual and another group of people on a determined physical location, for acting there as well as perceiving from there : it is a beaming [STB* 12] process where the individual is replicated by the robot, and common affordances are used (for example, moving one's head or hand to move the robot's head or hand) and sense the environment accordingly.

**Beaming** refers to technologies digitally transporting a representation of yourself to a distant place, where you can

interact with the people there as if you were there (acting, perceiving). The possibilities for such a technology are tremendous, and this research focuses on the industrial engineering application field, which is often a collaborative work.

Although humanoid robots are not expected to replace humans, they could be very helpful in some particular tasks. However there is still plenty of work to design robots that can be socially and industrially competent, in order to have a satisfying and successful interaction with humans in diverse environments and domains.

It is important to study how humans react when interacting with a robot instead of a real person, and the impact on their behaviour. Humans are very sensitive to verbal and non-verbal communication, and humanoid robots are still being developed in terms of their "human-likeness" and

<footnote n="1">¹Univ. Grenoble Alpes, G-SCOP, F-38000 Grenoble, France
CNRS, G-SCOP, F-38000 Grenoble, France</footnote>

<footnote n="2">²Univ. Grenoble Alpes, GIPSA, F-38000 Grenoble, France
CNRS, GIPSA, F-38000 Grenoble, France</footnote>
movement of the robot, conjugate harmoniously to attain
the individual expectations. Likewise, signal and gestural
responses from the robot such as head nodding along with
other non-verbal behaviour, help individuals to know the in-
tentions of their interlocutor, and so react and adapt their
behaviour accordingly.

## 2.3. Multiparty dialog and situated interaction

The robot behaviour must integrate qualities that favour the communication between multiple parties. Gaze perception plays a crucial role in human-human interaction [Lan00, MEB12]. It has been proved that it affects several aspects of communication and dialogue. An embodied conversational avatar must integrate an accurate gaze model. Gaze remains one of the strongest nonverbal cues in human interaction and this result should be assessed for human-robot interaction.

Interaction with humans expects to consider also human
factors. Primary causes of malfunctions in processes are due
to human factors [Nar99]. Among all the human factors to
consider that might have an impact on repairing, problem
solving and effectiveness, we can cite a few: attitude, fatigue,
workload, schedule, etc. For [HC98] having eyes and hands
busy, drive decision about the usage or not of voice recogniz-
ing systems instead of classic systems like a keyboard. In ad-
dition, speech is a strongly personal activity which contains
more information than just the text produced: then voice acts
also as an important human factor.

## 2.4. Collaboration within design and manufacturing activities

The current study applies a beaming platform to support col-
laborative professional activity, specially in a design and/or
manufacturing context. Design and manufacturing engineer-
ing is a cognitive activity [Cho09, GK04]. There are several
design modes: the prescriptive approach, which describes
the process of design as a succession of stages. This ap-
proach masters processes, acknowledge the creative part,
and is teachable. Design is also viewed [Sim96] as a "prob-
lem solving" process where it becomes a non-linear and non-
deterministic process of resolution, and has multiple solu-
tions. [Sch83] presents design as a process of reflexive con-
versation: designers work in virtual worlds, and their talent
lies on the capacity to build and manipulate these worlds.
Then design is an art in the professional activity, it is a form
of intelligence that is not teachable but can be learned, where
identifying a problem and improvising is a necessary "art"
for designers.

To collaborate designers use intermediary objects [BB03].
Objects are a second essential component in design and man-
ufacturing processes. Design is often supported by visual
reasoning with object representations. [Kua06] built experi-
ences to analyse the links between cognitive micro-activities

during design and demonstrated the separation between de-
sign reasoning, intention of design, intention of representa-
tion, and modeling actions.

[OOC99] observed how design often takes place in a
context of a collaborative face-to-face meeting. They try to
analyse how designers use their time, what kind of activi-
ties they develop, how they organise these activities and if
there exist any similar "models" between different projects
and teams of people. They conclude that people expose their
problems, alternatives and criteria to evaluate them, and they
share their expertise. Results show that 40% of the time is
spent on design discussions and briefings account for 30%
of the time, which implies that reunions apparently have
a role of coordination, generating a discussion in order to
clarify the best way to proceed, and moreover that meetings
produce an important amount of clarifications, since partici-
pants make and answer questions on diverse subjects.

Design and manufacturing activities provide a good con-
text to assess the usability and utility of a beaming platform.

## 3. An experience for collaborative remote training

### 3.1. Research question

This study expects to assess the conditions for creating a
beaming context to provide an efficient support for design
and manufacturing professional activities. A basic beaming
platform is controlled and its functions depreciated to check
their importance. Especially the following research ques-
tions are submitted:

1. What is the impact of the different grades of movement and expression replicated by a humanoid robot in the interaction with individuals on a collaborative work environment?

2. How to assess this collaboration between individuals and the robot?

Increasing the freedom and expressiveness of the robot's
movements should raise the fluency of the avatar represent-
ing the distant person, and thus improve the collaboration
with a team of individuals, allowing a more meaningful in-
teraction and a greater implication of individuals in the task.
Such an outcome comes with the interest of building a plat-
form for interaction of people in a collaborative setting,
which has applications in the industrial sector within a re-
alistic use-case.

### 3.2. A beaming platform

The beaming platform used for the experience integrates an
iCub robot, which is 1 m tall. This robot has 53 degrees of
freedom, plus 5 extra DoF for the jaw and mouth [PRM*15].
In the used beaming platform version, only a few are mapped
from a real person : it can move the head using the three
degrees of freedom, and it can move the jaw and lips thanks
"sociability". The evaluation of a beaming platform implies
many different features to study in order to provide a deep
analysis of the strengths and weaknesses.

**The current study** uses a system of interaction in a multi-site collaborative work, allowing collaborators to be "telepresented" by a humanoid robot with an active and physical presence. An experience was organized to perform a remote assembly training scenario with the confrontation of different objects. It is expected to check the condition for an actor to participate intuitively from a remote location. The same experience is run while degrading the functions of the environment to identify the most important functions which are associated here with the robot degrees of freedom allowing more or less expression transmission.

The next section introduces the technical state of the art about remote teleoperation. Then a short state of the art presents the domain of application we are concerned with (collaborative design activities). At this step the experience built for the current study will be described and the final section will introduce the first level of the results from this experience.

## 2. Remote teleoperation issues

### 2.1. Beaming platforms

Telerobotics [She89, She92] is the core technology of the assessed environment. It allows to remotely drive a robotic system. Robots are an extremely good alternative to humans for multiple applications (medicine, military, nuclear plant maintenance, construction) and also for collaborative distant work.

The "Wizard of Oz" (WoZ) refers to a person remotely operating a robot [Rie12], controlling its movement, navigation, speech, gestures, for example by triggering pre-defined actions by pressing keys or teleoperating movements of some body segments from a mouse, a joystick or any other motion capture device. WoZ controls a robot from fully autonomous to fully tele-operated. The controlled robot serves more as a proxy for humans than as an independent entity. A valid WoZ simulation expects [Rie12, GPL*12] "to simulate the future system, given human limitations", "to specify the future system's behaviour" and "to make the simulation convincing".

Part of telerobotics, the beaming concept (Being in Augmented Multimodal Naturally Networked Gatherings) was introduced [STB*12]. Beaming is the name for the process that allows someone - here the pilot - to instantaneously transfer himself from one physical place to any given distant location, thanks to the monitoring of the pilot's actions, physiological states and emotions which can be streamed over the internet while simultaneously streaming visual, audio, spatial, and context back from the distant destination. This flow of data is synchronized and the result is an unified

virtual environment representing the physical space of the
destination in real time. The destination is thus inhabited by
both local and virtual users.

Beaming allows to be embodied in a virtual interface. The destination/visitor interface is non-symmetrical, but beaming is defined to support symmetric social interaction between the visitor and locals. Thus an intensive attention is given to the quality of the visual immersion for the pilot, copying the physical remote destination. But indeed visualisation is just a part of a successful beaming platform, since all human senses should participate to the embodiment experience up to the "self-perception" of the avatar in the remote location.

With a virtual filter, human senses may be tricked introducing a infinite range of applications where perception may be faked [NSVW*12] and where awareness becomes a real ethical issue. Gathering immersive virtual reality environment with tele-operated systems opens applications which otherwise would be impossible to achieve. For example, [BEFS15] uses a humanoid robot, tele-operated by a human pilot to artificially provide the robot with social skills.

Such a WoZ platform was used to experience effects of culture in human robot interaction and social robotics [BV13] highlighting the role of head nods for social interaction. A basic analysis grid is used to assess a robot performance through various subjective criteria: likeability, engaging, satisfaction, useful, human-likeness, efficiency, credibility, human-presence.

[GPL*12] focused on the perception of gaze and head movements through an iCub robot. It allows the interaction to get closer to humans because it feels more "human-like", with the help of another sensor for capturing gaze and head motion.

### 2.2. Human robot interaction

With a telerobotic beaming platform, a human robot interaction is taking place at the shared physical place. Many key issues arise then [PDGA14]: mutual interaction learning, human presence undertaking, integrating social rules and protocols, as well as legible motion. There are also more specific issues like: robustness and efficiency, comfort and intentionality to specify robot behaviour, the capacity of the robot to adapt and plan according to its environment just like a human would do it, among many others.

[BCBJ*09, SLMF06, CKH*15] deeply analysed the effects of head nodding that can be simulated or reproduced by a robot. They analysed how people use visual feedback while conversing with robots. Humans apparently nod at the robot during conversation.

[CKH*15] debates over social cues of interaction as well. They assume that human robot interaction comes only when the capacities of perception of the environment as much as
<figure><img src="image_1.png" /><figcaption>**Figure 1:** *The beaming platform: pilot side.*</figcaption></figure>

<figure><img src="image_2.png" /><figcaption>**Figure 2:** *The beaming platform: avatar side.*</figcaption></figure>

to the generation of a model adapted to track pilot's lips. To create the model for the lips movement, the pilot had to glue a few motion infra-red reflectors strategically placed around the mouth, and run a calibration before each experiment.

The robot eyes are instrumented with two cameras allowing a stereoscopic capture of the robot vision. Microphones were inserted at robot ears location, allowing the capture of the sound environment around the robot. A loudspeaker was also inserted in the robot mouth, creating audio source for verbalization reproduction.

A Sony© Head Mounted Device (HMD), like to the one from Figure 1, was used to immerse the pilot in the robot space that transmit the views from the two mounted cameras (Figure 2). The HMD earphones receive the sounds captured by the iCub ear microphones. Pilot's speech is captured by a close microphone and played back via the robot's mouth loudspeaker. The robot acts as the distant avatar of the pilot. A motion tracking system was used for capturing both global movements of the pilot and mouth articulations.

### 3.3. Use case selection about mechanism assembly

A scenario that could be played by actors in a reasonable amount of time was expected to make experiences acceptable by subjects. The proposed scenario should be of a bearable difficulty, considering that something too hard to achieve is not useful here. To seek with utility within design and manufacturing applications, a jigsaw system was selected. This system provides a realistic context; on the inside it is a traditional mechanical system, with simple components such as a rotor, a gear, a switch, a rod, etc. and it is a real system for real engineering context whether it is for design, maintenance, production, etc. Additionally, it is easy enough for people to quickly identify its main parts and even if they have never seen such a system before, to be able to handle it without requiring training or whatsoever. This system can be mounted in a reasonable time.

<figure><img src="image_3.png" /><figcaption>**Figure 3:** *Jigsaw system on the inside (left) and completely mounted (right).*</figcaption></figure>

Involved subjects were not familiar with it. The disassembled system is presented to two subjects. The two subjects are expected to collaborate to re-assemble the system (see Figure 3) under the supervision of the pilot. The role of the pilot is limited to supervision (our beaming platform does not have an exoskeleton to support arm control yet). The scenario can be viewed as a training situation where two persons discover and learn a professional task supported by a remote expert. This context is closed to a context of project review where the pilot plays the role of manager and moderator of a discussion between two experts.

The selection of the jigsaw system is a compromise between complexity of task to assemble it, realism of the system and level of details: indeed the pilot guides the robot with a vision replication which has a low resolution and quality; it is due to the resolution of camera installed in the robot eyes (two 640x480 cameras with Bayer filter) and to the HMD which reproduce vision with a better resolution than the camera but creates new sources of uncomfort: weight, vergence for eye accommodation, etc.
movement of the robot, conjugate harmoniously to attain
the individual expectations. Likewise, signal and gestural
responses from the robot such as head nodding along with
other non-verbal behaviour, help individuals to know the in-
tentions of their interlocutor, and so react and adapt their
behaviour accordingly.

## 2.3. Multiparty dialog and situated interaction

The robot behaviour must integrate qualities that favour the communication between multiple parties. Gaze perception plays a crucial role in human-human interaction [Lan00, MEB12]. It has been proved that it affects several aspects of communication and dialogue. An embodied conversational avatar must integrate an accurate gaze model. Gaze remains one of the strongest nonverbal cues in human interaction and this result should be assessed for human-robot interaction.

Interaction with humans expects to consider also human
factors. Primary causes of malfunctions in processes are due
to human factors [Nar99]. Among all the human factors to
consider that might have an impact on repairing, problem
solving and effectiveness, we can cite a few: attitude, fatigue,
workload, schedule, etc. For [HC98] having eyes and hands
busy, drive decision about the usage or not of voice recogniz-
ing systems instead of classic systems like a keyboard. In ad-
dition, speech is a strongly personal activity which contains
more information than just the text produced: then voice acts
also as an important human factor.

## 2.4. Collaboration within design and manufacturing activities

The current study applies a beaming platform to support col-
laborative professional activity, specially in a design and/or
manufacturing context. Design and manufacturing engineer-
ing is a cognitive activity [Cho09, GK04]. There are several
design modes: the prescriptive approach, which describes
the process of design as a succession of stages. This ap-
proach masters processes, acknowledge the creative part,
and is teachable. Design is also viewed [Sim96] as a "prob-
lem solving" process where it becomes a non-linear and non-
deterministic process of resolution, and has multiple solu-
tions. [Sch83] presents design as a process of reflexive con-
versation: designers work in virtual worlds, and their talent
lies on the capacity to build and manipulate these worlds.
Then design is an art in the professional activity, it is a form
of intelligence that is not teachable but can be learned, where
identifying a problem and improvising is a necessary "art"
for designers.

To collaborate designers use intermediary objects [BB03].
Objects are a second essential component in design and man-
ufacturing processes. Design is often supported by visual
reasoning with object representations. [Kua06] built experi-
ences to analyse the links between cognitive micro-activities

during design and demonstrated the separation between de-
sign reasoning, intention of design, intention of representa-
tion, and modeling actions.

[OOC99] observed how design often takes place in a
context of a collaborative face-to-face meeting. They try to
analyse how designers use their time, what kind of activi-
ties they develop, how they organise these activities and if
there exist any similar "models" between different projects
and teams of people. They conclude that people expose their
problems, alternatives and criteria to evaluate them, and they
share their expertise. Results show that 40% of the time is
spent on design discussions and briefings account for 30%
of the time, which implies that reunions apparently have
a role of coordination, generating a discussion in order to
clarify the best way to proceed, and moreover that meetings
produce an important amount of clarifications, since partici-
pants make and answer questions on diverse subjects.

Design and manufacturing activities provide a good con-
text to assess the usability and utility of a beaming platform.

## 3. An experience for collaborative remote training

### 3.1. Research question

This study expects to assess the conditions for creating a
beaming context to provide an efficient support for design
and manufacturing professional activities. A basic beaming
platform is controlled and its functions depreciated to check
their importance. Especially the following research ques-
tions are submitted:

1. What is the impact of the different grades of movement and expression replicated by a humanoid robot in the interaction with individuals on a collaborative work environment?

2. How to assess this collaboration between individuals and the robot?

Increasing the freedom and expressiveness of the robot's
movements should raise the fluency of the avatar represent-
ing the distant person, and thus improve the collaboration
with a team of individuals, allowing a more meaningful in-
teraction and a greater implication of individuals in the task.
Such an outcome comes with the interest of building a plat-
form for interaction of people in a collaborative setting,
which has applications in the industrial sector within a re-
alistic use-case.

### 3.2. A beaming platform

The beaming platform used for the experience integrates an
iCub robot, which is 1 m tall. This robot has 53 degrees of
freedom, plus 5 extra DoF for the jaw and mouth [PRM*15].
In the used beaming platform version, only a few are mapped
from a real person : it can move the head using the three
degrees of freedom, and it can move the jaw and lips thanks
**3.4. Experimental protocol**

The pilot is driving tasks within a predefined sequence of
actions. The pilot is expert of the system, knows every de-
tails about the mechanism and must have enough possible
situations in mind to react promptly as soon as there is a de-
viation from the predefined sequence of tasks to ensure to
make possible comparisons between tests. The sub-tasks are
thus organised into a series of short collaborations or paral-
lel actions of the two subjects to ensure that the predefined
process is followed.

The position of parts when the subjects enter the room
is pre-defined as organised in Figure 4. When entering the
room the jigsaw is masked.

<figure><img src="image_4.png" /><figcaption>Figure 4: Jigsaw presentation when initializing the process.</figcaption></figure>

Two configurations of robot behavior are created.

**Speaker configuration : A:** the robot is motionless, it transmits the voice of the pilot trough the high-speaker, but all his body articulations are off.

**Beaming configuration : B:** The robot has full motion of its head, with 3 degrees of freedom, and reproduced the movement of the lips from the pilot according to the model set up before the experiments. In this condition, the pilot has a partly blurred vision in the HMD, except in the center area. This vision mode forces the pilot to use explicit head movements to track or see the focus zone: the interest point he's gazing at is then clearly identified by the subjects.

Three questionnaires have been prepared which cannot be
fully reported here but we provide some question example to
fix ideas :

**Memory Questionnaire:** $Q_1$ : 10 basic questions to test what the person remind by himself even if no special attention was drawn to these details during the assembly process:

$Q_{1.1}$ What is the color of the main switch of the sigsaw ?
black☐ red☐ yellow☐ green☐

$Q_{1.2}$ How many screw sizes belongs to the system ? 1☐
2☐ 3☐ 4☐

$Q_{1.i}$ etc.

**Individual perception of collaboration:** $Q_2$ : 10 new questions were devoted to measure the perception of a more or less living robot:

$Q_{2.1}$ How will you qualify the robot from machine to al-
ife ? 1☐ 2☐ 3☐ 4☐ 5☐

$Q_{2.2}$ How will you qualify pilot/robot from stupid to clever ? 1☐ 2☐ 3☐ 4☐ 5☐

$Q_{2.i}$ etc.

**Individual comparison of configurations:** $Q_3$ : 5 questions to compare configurations 'A' or 'B'.

$Q_{3.1}$ Mark from 1 to 3 the importance of the following modalities ?

• voice transmission : 1☐ 2☐ 3☐

• head movement : 1☐ 2☐ 3☐

• lip movement : 1☐ 2☐ 3☐

$Q_{3.2}$ In which configuration the pilot specifications were more clear ? A☐ or B☐ or None☐

$Q_{3.3}$ In which configuration the pilot was the more attentive ? A☐ or B☐ or None☐

$Q_{3.4}$ In which configuration did you feel more easy when interacting with the pilot ? A☐ or B☐ or None☐

$Q_{3.5}$ What configuration did you prefer ? A☐ or B☐ or None☐

After filling a participation agreement form, both the
avatar and pilot are presented to subjects. The wizard of Oz
context is clearly highlighted: they know that the robot is
driven by a person. They are informed that they will have to
follow the directives of the pilot and that they will assemble
twice the same system following different sequences. Each
sequence will not overpass 20 minutes: the task is stopped
after this delay.

There are two assembly sequences, $S_1$ and $S_2$. For each $S_i$, assembly sub-tasks are executed in different orders and the roles of the two subjects are swapped. Sub-tasks are either individual or collaborative, and can be executed in parallel or sequentially.

For statistics issues configurations A and B are associated
to sequence $S_1$ and $S_2$ leading to four conditions that are
applied alternatively to the tuples of subjects.

Then, the mask over the mechanism is removed and the
pilot drives the first assembly sequence. At the end of this
sequence the questionnaire $Q_1$ is filled individually by the
two subjects. Then they also answer to the questionnaire $Q_2$
to collect their subjective feedback about the collaboration
with the pilot through the robot. While filling the question-
naire the system is disassembled and prepared again for the
second sequence. The second sequence is played with the re-
maining configuration. After the second sequence, the ques-
tionnaire $Q_2$ is answered again but in reference to the new
configuration. At last, the two subjects answer to the ques-
tionnaire $Q_3$.

This protocol was repeated for 9 couples of subjects and
indeed 18 different persons were involved. The distribution
of participants could be presented as follow: 10 females and
8 males, with 3 teams of women, 2 teams of men and 4 mixed
<figure><img src="image_1.png" /><figcaption>**Figure 1:** *The beaming platform: pilot side.*</figcaption></figure>

<figure><img src="image_2.png" /><figcaption>**Figure 2:** *The beaming platform: avatar side.*</figcaption></figure>

to the generation of a model adapted to track pilot's lips. To create the model for the lips movement, the pilot had to glue a few motion infra-red reflectors strategically placed around the mouth, and run a calibration before each experiment.

The robot eyes are instrumented with two cameras allowing a stereoscopic capture of the robot vision. Microphones were inserted at robot ears location, allowing the capture of the sound environment around the robot. A loudspeaker was also inserted in the robot mouth, creating audio source for verbalization reproduction.

A Sony© Head Mounted Device (HMD), like to the one from Figure 1, was used to immerse the pilot in the robot space that transmit the views from the two mounted cameras (Figure 2). The HMD earphones receive the sounds captured by the iCub ear microphones. Pilot's speech is captured by a close microphone and played back via the robot's mouth loudspeaker. The robot acts as the distant avatar of the pilot. A motion tracking system was used for capturing both global movements of the pilot and mouth articulations.

### 3.3. Use case selection about mechanism assembly

A scenario that could be played by actors in a reasonable amount of time was expected to make experiences acceptable by subjects. The proposed scenario should be of a bearable difficulty, considering that something too hard to achieve is not useful here. To seek with utility within design and manufacturing applications, a jigsaw system was selected. This system provides a realistic context; on the inside it is a traditional mechanical system, with simple components such as a rotor, a gear, a switch, a rod, etc. and it is a real system for real engineering context whether it is for design, maintenance, production, etc. Additionally, it is easy enough for people to quickly identify its main parts and even if they have never seen such a system before, to be able to handle it without requiring training or whatsoever. This system can be mounted in a reasonable time.

<figure><img src="image_3.png" /><figcaption>**Figure 3:** *Jigsaw system on the inside (left) and completely mounted (right).*</figcaption></figure>

Involved subjects were not familiar with it. The disassembled system is presented to two subjects. The two subjects are expected to collaborate to re-assemble the system (see Figure 3) under the supervision of the pilot. The role of the pilot is limited to supervision (our beaming platform does not have an exoskeleton to support arm control yet). The scenario can be viewed as a training situation where two persons discover and learn a professional task supported by a remote expert. This context is closed to a context of project review where the pilot plays the role of manager and moderator of a discussion between two experts.

The selection of the jigsaw system is a compromise between complexity of task to assemble it, realism of the system and level of details: indeed the pilot guides the robot with a vision replication which has a low resolution and quality; it is due to the resolution of camera installed in the robot eyes (two 640x480 cameras with Bayer filter) and to the HMD which reproduce vision with a better resolution than the camera but creates new sources of uncomfort: weight, vergence for eye accommodation, etc.
teams. The average age of the participants in the experiments was 25 years old, and the majority had an engineering background.

# 4. Analysis of collaboration between subjects and a remote drive

## 4.1. Analysis of questionnaire $Q_1$

The overall objective of the questionnaire $Q_1$ is to check if subjects have a better performance in configuration *B* than in configuration *A*. The average percentage of correct answers was medium to low in both cases: 45% of success rate for the actors that answered after experiencing the configuration *A*, and 62% for actors that answered after configuration *B*. It must be reminded that in both cases they answer after the first sequence of assembly. These values seem to indicate that actors in configuration *B* are more engaged in the task because they have a better communication whenever the robot presents a more human-like expression and record more information.

## 4.2. Analysis of questionnaire $Q_2$

<figure><img src="image_5.png" /><figcaption>**Figure 5:** *Average results for questionnaire 2.*</figcaption></figure>

The second questionnaire deals with the subject individual impression about the robot. The average mark of each question for all participants are representative numbers to analyse. For every question, the average mark is higher for configuration *B* than for configuration *A*. Figure 5 details results for every question and the configuration *B* is clearly appreciated but questions can be separated into two sets depending on a more or less difference between the two configurations. Then it seems that:

* Configuration *B* provides a real impact to understand who (between the two subjects) is concerned when the pilot interacts. This configuration also enhances the frequency of interaction and the demand for interaction with the pilot.

* Configuration *B* seems to be also better for interaction quality and when assessing the overall pleasure to interact through the avatar but the difference is less obvious.

The average and the standard deviation on the overall questions (Figure 6) demonstrate the tendency and the clear advantage of configuration *B* which is here much more obvious than for questionnaire $Q_1$. Indeed enhancing the expressiveness of the robot is a major asset in human-robot interaction, as well as a useful tool for communication between humans.

<figure><img src="image_6.png" /><figcaption>**Figure 6:** *Average results for questionnaire 2.*</figcaption></figure>

## 4.3. Analysis of questionnaire $Q_3$

The third questionnaire serves as a summary to confirm the analysis of the first questionnaires. The first question (Figure 7) provides a quite obvious result but which is clearly measured. The voice is the preferred interaction modality but the gaze identified by head movements is a major interaction modality. The mouth articulation are not unnecessary (indeed, they might play an important role for speech intelligibility), but from the conscious self-report task, they are perceived as less important.

<figure><img src="image_7.png" /><figcaption>**Figure 7:** *Results for question $Q_{3.1}$*</figcaption></figure>

The other questions of $Q_3$ were direct comparisons of the two configurations. The results described by Figure 8 are incontrovertible. The configuration *B* is clearly preferred even if 1 or 2 subjects reported that they did not care at all about the robot, which explains a few answers with no preference: video shows that these persons were concentrated on the task while listening the pilot but without watching the robot.
**3.4. Experimental protocol**

The pilot is driving tasks within a predefined sequence of actions. The pilot is expert of the system, knows every details about the mechanism and must have enough possible situations in mind to react promptly as soon as there is a deviation from the predefined sequence of tasks to ensure to make possible comparisons between tests. The sub-tasks are thus organised into a series of short collaborations or parallel actions of the two subjects to ensure that the predefined process is followed.

The position of parts when the subjects enter the room is pre-defined as organised in Figure 4. When entering the room the jigsaw is masked.

<figure><img src="image_4.png" /><figcaption>**Figure 4:** Jigsaw presentation when initializing the process.</figcaption></figure>

Two configurations of robot behavior are created.

**Speaker configuration : A:** the robot is motionless, it transmits the voice of the pilot trough the high-speaker, but all his body articulations are off.

**Beaming configuration : B:** The robot has full motion of its head, with 3 degrees of freedom, and reproduced the movement of the lips from the pilot according to the model set up before the experiments. In this condition, the pilot has a partly blurred vision in the HMD, except in the center area. This vision mode forces the pilot to use explicit head movements to track or see the focus zone: the interest point he's gazing at is then clearly identified by the subjects.

Three questionnaires have been prepared which cannot be fully reported here but we provide some question example to fix ideas :

**Memory Questionnaire:** $Q_1$ : 10 basic questions to test what the person remind by himself even if no special attention was drawn to these details during the assembly process:

$Q_{1.1}$ What is the color of the main switch of the sigsaw ?
black☐ red☐ yellow☐ green☐

$Q_{1.2}$ How many screw sizes belongs to the system ? 1☐
2☐ 3☐ 4☐

$Q_{1.i}$ etc.

**Individual perception of collaboration:** $Q_2$ : 10 new questions were devoted to measure the perception of a more or less living robot:

$Q_{2.1}$ How will you qualify the robot from machine to al-
ife ? 1☐ 2☐ 3☐ 4☐ 5☐

$Q_{2.2}$ How will you qualify pilot/robot from stupid to clever ? 1☐ 2☐ 3☐ 4☐ 5☐

$Q_{2.i}$ etc.

**Individual comparison of configurations:** $Q_3$ : 5 questions to compare configurations 'A' or 'B'.

$Q_{3.1}$ Mark from 1 to 3 the importance of the following modalities ?

• voice transmission : 1☐ 2☐ 3☐

• head movement : 1☐ 2☐ 3☐

• lip movement : 1☐ 2☐ 3☐

$Q_{3.2}$ In which configuration the pilot specifications were more clear ? A☐ or B☐ or None☐

$Q_{3.3}$ In which configuration the pilot was the more attentive ? A☐ or B☐ or None☐

$Q_{3.4}$ In which configuration did you feel more easy when interacting with the pilot ? A☐ or B☐ or None☐

$Q_{3.5}$ What configuration did you prefer ? A☐ or B☐ or None☐

After filling a participation agreement form, both the avatar and pilot are presented to subjects. The wizard of Oz context is clearly highlighted: they know that the robot is driven by a person. They are informed that they will have to follow the directives of the pilot and that they will assemble twice the same system following different sequences. Each sequence will not overpass 20 minutes: the task is stopped after this delay.

There are two assembly sequences, $S_1$ and $S_2$. For each $S_i$, assembly sub-tasks are executed in different orders and the roles of the two subjects are swapped. Sub-tasks are either individual or collaborative, and can be executed in parallel or sequentially.

For statistics issues configurations A and B are associated to sequence $S_1$ and $S_2$ leading to four conditions that are applied alternatively to the tuples of subjects.

Then, the mask over the mechanism is removed and the pilot drives the first assembly sequence. At the end of this sequence the questionnaire $Q_1$ is filled individually by the two subjects. Then they also answer to the questionnaire $Q_2$ to collect their subjective feedback about the collaboration with the pilot through the robot. While filling the questionnaire the system is disassembled and prepared again for the second sequence. The second sequence is played with the remaining configuration. After the second sequence, the questionnaire $Q_2$ is answered again but in reference to the new configuration. At last, the two subjects answer to the questionnaire $Q_3$.

This protocol was repeated for 9 couples of subjects and indeed 18 different persons were involved. The distribution of participants could be presented as follow: 10 females and 8 males, with 3 teams of women, 2 teams of men and 4 mixed
<figure><img src="image_8.png" /><figcaption>**Figure 8:** *Results for questionnaire Q₃, question 2 to 5.*</figcaption></figure>

**5. Conclusion and perspectives**

The current study clearly demonstrates again the importance of gaze and head movements within social interaction. It also demonstrates the capacity of a beaming system to telepresent a remote human coach as soon as the beaming system reproduces the good interaction modalities. It demonstrates also a beaming process within a realistic professional collaborative task. Usually, research about user interfaces focuses on usability demonstration. With the current realistic process, a first step towards utility assessment is passed.

Obviously the statistics sample should be extended but the first results seem incontrovertible. We need to explore which parts of the dialog benefit from gazing at head or lips movements, and how these gaze patterns evolve with the duration of the interaction. Another direction of assessment will be exploited soon to get objective measures: Indeed all the scenarios have been recorded (video, audio, motion capture of the pilot, as well as robot's sensors streams) providing a rich set of corpus which can be deeply analysed (video labeling of the eye contacts, of the number of head movements from the human partners...)

Another perspective is also to go ahead with this experience using other type of avatars, to check how presence filling may be produced on a distant location. A video of the pilot face may be streamed at the remote location but it can be expected to face the Mona-Lisa cue: a 2D image cannot propagate the gaze direction of the pilot. An image avatar could provide an immersive environment where the importance of gaze could be checked with respect to the quality of the avatar. The image could provide really realistic human face. To avoid the Mona-Lisa cue, stereoscopic displays or holographic displays could create an interesting 3D avatar; for a single remote subject, stereoscopy may be used while for multiple remote subjects holography will be expected. It will be worth exploring the respective properties of the streaming of 3D videos vs. the sensorimotor monitoring of virtual/robotic avatars in terms of users' experience and interaction efficiency.

**Aknowledgement**

* This paper was written within the scope of the LabEx PERSYVAL-Lab (ANR-11-LABX-0025-01).

* This work has also been partially supported by the VI-SIONAIR infrastructure project funded by the European Commission under grant agreement 262044.

* The experimental robotic platform has been supported by the ANR (Robotex ANR-10-EQPX-44-0 and SOM-BRERO ANR-14-CE27-0014) and the UJF (EMSOC 2012084RECF991).

**References**

[BB03] BOUJUT J.-F., BLANCO E.: Intermediary objects as a means to foster co-operation in engineering design. *Computer Supported Cooperative Work (CSCW) 12*, 2 (2003), 205–219. 3

[BCBJ*09] BOKER S., COHN J. F., BARRY-JOHN T., MATTHEWS I., BRICK T., SPIES J. R.: Effects of damping head movement and facial expression in dyadic conversation using real-time facial expression tracking and synthesized avatars. *Philosophical transactions of the royal society B*, 364 (2009), 3485–3495. 2

[BEFS15] BAILLY G., ELISEI F., F., SAUZE M.: Beaming the gaze of a humanoid robot. In *Human-Robot Interaction, Portland, United States* (2015). 2

[BV13] BADDOURA R., VENTURE G.: Human motion measures and adequacy of the response in relation to the robots sociable character. In *International Workshop on Developmental Social Robotics* (2013). 2

[Cho09] CHOULIER D.: Comprendre l'activité de conception : Partie I : Définir la conception. Pôle Éditorial de l'Université de Technologie de Belfort-Montbéliard, 2009. 3

[CKH*15] CAVEDON L., KROOS C., HERATH D., BURNHAM D., BISHOP L., LEUNG Y., STEVENS C.: C'mon dude!ÂÂÂÎ: Users adapt their behaviour to a robotic agent with an attention model. *International Journal of Human-Computer studies*, 80 (2015), 14–23. 2

[GK04] GERO J. S., KANNENGIESSER U.: The situated function behaviour structure framework. *Design Studies* 25 (2004), 373–391. 3

[GPL*12] GIBERT G., PETIT M., LANCE F., G. P., DOMINEY P. F.: Super wizard of oz technique to study human-robot interaction. In *Workshop Affect, Compagnon Artificiel, Grenoble, France* (2012). 2

[HC98] HOLLINGUM J., CASSFORD G.: Speech technology at work. *IFS* (1998), 65–74. 3

[Kua06] KUATE G.: Analyse d'activités de conception: contribution à la traçabilité des intentions de conception dans les modèles CAO. PhD thesis, thèse de doctorat, spécialité Sciences pour l'ingénieur, Université de Technologie de Belfort-Montbéliard, 2006. 3

[Lan00] LANGTON S. R.: The mutual influence of gaze and head orientation in the analysis of social attention direction. *Quarterly Journal of Experimental Psychology* 53A, 3 (2000), 825–845. 3

[MEB12] MOUBAYED S. A., EDLUND J., BESKOW J.: Taming mona lisa: Communicating gaze faithfully in 2d and 3d facial projections. *ACM Transactions on Interactive Intelligent Systems* 2, 1) (2012), 1 – 25. 3

[Nar99] NARDONE M. S.: Direct Digital Control Systems: Application, Commissioning. Springer Science & Business Media, 1999. 3
teams. The average age of the participants in the experiments was 25 years old, and the majority had an engineering background.

# 4. Analysis of collaboration between subjects and a remote drive

## 4.1. Analysis of questionnaire $Q_1$

The overall objective of the questionnaire $Q_1$ is to check if subjects have a better performance in configuration *B* than in configuration *A*. The average percentage of correct answers was medium to low in both cases: 45% of success rate for the actors that answered after experiencing the configuration *A*, and 62% for actors that answered after configuration *B*. It must be reminded that in both cases they answer after the first sequence of assembly. These values seem to indicate that actors in configuration *B* are more engaged in the task because they have a better communication whenever the robot presents a more human-like expression and record more information.

## 4.2. Analysis of questionnaire $Q_2$

<figure><img src="image_5.png" /><figcaption>**Figure 5:** *Average results for questionnaire 2.*</figcaption></figure>

The second questionnaire deals with the subject individual impression about the robot. The average mark of each question for all participants are representative numbers to analyse. For every question, the average mark is higher for configuration *B* than for configuration *A*. Figure 5 details results for every question and the configuration *B* is clearly appreciated but questions can be separated into two sets depending on a more or less difference between the two configurations. Then it seems that:

* Configuration *B* provides a real impact to understand who (between the two subjects) is concerned when the pilot interacts. This configuration also enhances the frequency of interaction and the demand for interaction with the pilot.

* Configuration *B* seems to be also better for interaction quality and when assessing the overall pleasure to interact through the avatar but the difference is less obvious.

The average and the standard deviation on the overall questions (Figure 6) demonstrate the tendency and the clear advantage of configuration *B* which is here much more obvious than for questionnaire $Q_1$. Indeed enhancing the expressiveness of the robot is a major asset in human-robot interaction, as well as a useful tool for communication between humans.

<figure><img src="image_6.png" /><figcaption>**Figure 6:** *Average results for questionnaire 2.*</figcaption></figure>

## 4.3. Analysis of questionnaire $Q_3$

The third questionnaire serves as a summary to confirm the analysis of the first questionnaires. The first question (Figure 7) provides a quite obvious result but which is clearly measured. The voice is the preferred interaction modality but the gaze identified by head movements is a major interaction modality. The mouth articulation are not unnecessary (indeed, they might play an important role for speech intelligibility), but from the conscious self-report task, they are perceived as less important.

<figure><img src="image_7.png" /><figcaption>**Figure 7:** *Results for question $Q_{3.1}$*</figcaption></figure>

The other questions of $Q_3$ were direct comparisons of the two configurations. The results described by Figure 8 are incontrovertible. The configuration *B* is clearly preferred even if 1 or 2 subjects reported that they did not care at all about the robot, which explains a few answers with no preference: video shows that these persons were concentrated on the task while listening the pilot but without watching the robot.
[NSVW*12] NORMAND J.-M., SANCHEZ-VIVES M. V., WAECHTER C., GIANNOPOULOS E., GROSSWINDHAGER B., SPANLANG B., GUGER C., KLINKER G., SRINIVASAN M. A., SLATER M.: Beaming into the rat world: enabling real-time interaction between rat and human each at their own scale. http://hdl.handle.net/1721.1/76264.2

[OOC$92$] OLSON G. M., OLSON J. S., CARTER M. R., STOR-ROSTEN M.: Small group design meetings: An analysis of col-laboration. *Human–Computer Interaction* 7, 4 (1992), 347–374.

[PDGA14] PANDEY A., DAUTENHAHN K., GELIN R., ALAMI R.: Report on workshop on societal applications of humanoid robots. In *Workshop on Societal Applications of Humanoid Robots* (2014). 2

[PRM*15] PARMIGGIANI, ALBERTO, RANDAZZO, MARCO, MAGGIALI, MARCO, METTA, GIORGIO, ELISEI, FRÉDÉRIC, BAILLY, GÉRARD: Design and validation of a talking face for the iCub. *International Journal of Humanoid Robotics* (2015). 3

[Rie12] RIEK L. D.: Wizard of oz studies in hri: a systematic review and new reporting guidelines. *Journal of Human-Robot Interaction* 1, 1 (2012). 2

[Sch83] SCHÖN D. A.: *The reflective practitioner: How profes-sionals think in action*, vol. 5126. Basic books, 1983. 3

[She89] SHERIDAN T. B.: Telerobotics. *Automatica* 25, 4 (1989), 487–507. 2

[She92] SHERIDAN T. B.: *Telerobotics, automation, and human supervisory control*. MIT press, 1992. 2

[Sim96] SIMON H. A.: *The sciences of the artificial*, vol. 136. MIT press, 1996. 3

[SLMF06] SIDNER C. L., LEE C., MORENCY L.-P., FORLINES C.: The effect of head-nod recognition in human-robot conversation. In *Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction* (2006), ACM, pp. 290–296. 2

[STB*12] STEED A., TECCHIA F., BERGAMASCO M., SLATER M., STEPTOE W., OYEKOYA W., PECE F., WEYRICH T., KAUTZ J., FRIEDMAN D., ET AL.: Beaming: an asymmetric telepresence system. *IEEE computer graphics and applications*, 6 (2012), 10–17. 1, 2
<figure><img src="image_8.png" /><figcaption>**Figure 8:** *Results for questionnaire Q₃, question 2 to 5.*</figcaption></figure>

**5. Conclusion and perspectives**

The current study clearly demonstrates again the importance of gaze and head movements within social interaction. It also demonstrates the capacity of a beaming system to telepresent a remote human coach as soon as the beaming system reproduces the good interaction modalities. It demonstrates also a beaming process within a realistic professional collaborative task. Usually, research about user interfaces focuses on usability demonstration. With the current realistic process, a first step towards utility assessment is passed.

Obviously the statistics sample should be extended but the first results seem incontrovertible. We need to explore which parts of the dialog benefit from gazing at head or lips movements, and how these gaze patterns evolve with the duration of the interaction. Another direction of assessment will be exploited soon to get objective measures: Indeed all the scenarios have been recorded (video, audio, motion capture of the pilot, as well as robot's sensors streams) providing a rich set of corpus which can be deeply analysed (video labeling of the eye contacts, of the number of head movements from the human partners...)

Another perspective is also to go ahead with this experience using other type of avatars, to check how presence filling may be produced on a distant location. A video of the pilot face may be streamed at the remote location but it can be expected to face the Mona-Lisa cue: a 2D image cannot propagate the gaze direction of the pilot. An image avatar could provide an immersive environment where the importance of gaze could be checked with respect to the quality of the avatar. The image could provide really realistic human face. To avoid the Mona-Lisa cue, stereoscopic displays or holographic displays could create an interesting 3D avatar; for a single remote subject, stereoscopy may be used while for multiple remote subjects holography will be expected. It will be worth exploring the respective properties of the streaming of 3D videos vs. the sensorimotor monitoring of virtual/robotic avatars in terms of users' experience and interaction efficiency.

**Aknowledgement**

* This paper was written within the scope of the LabEx PERSYVAL-Lab (ANR-11-LABX-0025-01).

* This work has also been partially supported by the VI-SIONAIR infrastructure project funded by the European Commission under grant agreement 262044.

* The experimental robotic platform has been supported by the ANR (Robotex ANR-10-EQPX-44-0 and SOM-BRERO ANR-14-CE27-0014) and the UJF (EMSOC 2012084RECF991).

**References**

[BB03] BOUJUT J.-F., BLANCO E.: Intermediary objects as a means to foster co-operation in engineering design. *Computer Supported Cooperative Work (CSCW) 12*, 2 (2003), 205–219. 3

[BCBJ*09] BOKER S., COHN J. F., BARRY-JOHN T., MATTHEWS I., BRICK T., SPIES J. R.: Effects of damping head movement and facial expression in dyadic conversation using real-time facial expression tracking and synthesized avatars. *Philosophical transactions of the royal society B*, 364 (2009), 3485–3495. 2

[BEFS15] BAILLY G., ELISEI F., F., SAUZE M.: Beaming the gaze of a humanoid robot. In *Human-Robot Interaction, Portland, United States* (2015). 2

[BV13] BADDOURA R., VENTURE G.: Human motion measures and adequacy of the response in relation to the robots sociable character. In *International Workshop on Developmental Social Robotics* (2013). 2

[Cho09] CHOULIER D.: Comprendre l'activité de conception : Partie I : Définir la conception. Pôle Éditorial de l'Université de Technologie de Belfort-Montbéliard, 2009. 3

[CKH*15] CAVEDON L., KROOS C., HERATH D., BURNHAM D., BISHOP L., LEUNG Y., STEVENS C.: C'mon dude!ÂÂÂÎ: Users adapt their behaviour to a robotic agent with an attention model. *International Journal of Human-Computer studies*, 80 (2015), 14–23. 2

[GK04] GERO J. S., KANNENGIESSER U.: The situated function behaviour structure framework. *Design Studies* 25 (2004), 373–391. 3

[GPL*12] GIBERT G., PETIT M., LANCE F., G. P., DOMINEY P. F.: Super wizard of oz technique to study human-robot interaction. In *Workshop Affect, Compagnon Artificiel, Grenoble, France* (2012). 2

[HC98] HOLLINGUM J., CASSFORD G.: Speech technology at work. *IFS* (1998), 65–74. 3

[Kua06] KUATE G.: Analyse d'activités de conception: contribution à la traçabilité des intentions de conception dans les modèles CAO. PhD thesis, thèse de doctorat, spécialité Sciences pour l'ingénieur, Université de Technologie de Belfort-Montbéliard, 2006. 3

[Lan00] LANGTON S. R.: The mutual influence of gaze and head orientation in the analysis of social attention direction. *Quarterly Journal of Experimental Psychology* 53A, 3 (2000), 825–845. 3

[MEB12] MOUBAYED S. A., EDLUND J., BESKOW J.: Taming mona lisa: Communicating gaze faithfully in 2d and 3d facial projections. *ACM Transactions on Interactive Intelligent Systems* 2, 1) (2012), 1 – 25. 3

[Nar99] NARDONE M. S.: Direct Digital Control Systems: Application, Commissioning. Springer Science & Business Media, 1999. 3
[NSVW*12] NORMAND J.-M., SANCHEZ-VIVES M. V., WAECHTER C., GIANNOPOULOS E., GROSSWINDHAGER B., SPANLANG B., GUGER C., KLINKER G., SRINIVASAN M. A., SLATER M.: Beaming into the rat world: enabling real-time interaction between rat and human each at their own scale. http://hdl.handle.net/1721.1/76264.2

[OOC$92$] OLSON G. M., OLSON J. S., CARTER M. R., STOR-ROSTEN M.: Small group design meetings: An analysis of col-laboration. *Human–Computer Interaction* 7, 4 (1992), 347–374.

[PDGA14] PANDEY A., DAUTENHAHN K., GELIN R., ALAMI R.: Report on workshop on societal applications of humanoid robots. In *Workshop on Societal Applications of Humanoid Robots* (2014). 2

[PRM*15] PARMIGGIANI, ALBERTO, RANDAZZO, MARCO, MAGGIALI, MARCO, METTA, GIORGIO, ELISEI, FRÉDÉRIC, BAILLY, GÉRARD: Design and validation of a talking face for the iCub. *International Journal of Humanoid Robotics* (2015). 3

[Rie12] RIEK L. D.: Wizard of oz studies in hri: a systematic review and new reporting guidelines. *Journal of Human-Robot Interaction* 1, 1 (2012). 2

[Sch83] SCHÖN D. A.: *The reflective practitioner: How profes-sionals think in action*, vol. 5126. Basic books, 1983. 3

[She89] SHERIDAN T. B.: Telerobotics. *Automatica* 25, 4 (1989), 487–507. 2

[She92] SHERIDAN T. B.: *Telerobotics, automation, and human supervisory control*. MIT press, 1992. 2

[Sim96] SIMON H. A.: *The sciences of the artificial*, vol. 136. MIT press, 1996. 3

[SLMF06] SIDNER C. L., LEE C., MORENCY L.-P., FORLINES C.: The effect of head-nod recognition in human-robot conversation. In *Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction* (2006), ACM, pp. 290–296. 2

[STB*12] STEED A., TECCHIA F., BERGAMASCO M., SLATER M., STEPTOE W., OYEKOYA W., PECE F., WEYRICH T., KAUTZ J., FRIEDMAN D., ET AL.: Beaming: an asymmetric telepresence system. *IEEE computer graphics and applications*, 6 (2012), 10–17. 1, 2