How Can Robot's Gaze Ratio and Body Direction Show an Awarenessof Priority to the People with whom it is Interacting?
Honoka Arai, Mitsuhiko Kimoto, Takamasa Iio, Katsunori Shimohara, Reo Matsumura, and Masahiro Shiomi

Abstract—When a person interacts with others who requiredifferent levels of authority, her gaze ratio and body direction
for each person are different, e.g., a salesperson who turns toand looks at a VIP more attentively than a member of that VIP’s
staff. In this study, we investigate the effects of gaze ratios andbody directions with which social robots demonstrate anawareness of authority when they are simultaneouslyinteracting with multiple persons. We develop a gaze-controller
system for a social robot and experimentally investigate theeffects of the gaze ratio and the body direction (priority-oriented:
body direction turns to a target and non-priority-oriented: body
direction does not turn to a target) when the robot salesperson is
describing items to two persons who are playing different roles:
a VIP and a follower (e.g., member of a VIP’s staff). Our
experiment results show that different advantages of gaze ratio
and body direction demonstrate an awareness of priority. With
the priority-oriented condition, participants more highly
evaluated the gaze at a 100:0 ratio between VIPs and followers.
But in the non-priority-oriented condition, the participants more
highly evaluated the gaze ratios at 80:20 or 90:10. These results
contribute to a gaze behavior design for social robots thatinteract with multiple persons who require different levels ofpriority.

I. INTRODUCTION
In information-providing situations with multiple people, aspeaker basically assigns similar gaze ratios to the addressees
when their levels of authority are similar, and when they are
different, the relationship is obviously also different. For
example, when a salesperson is describing new products to an
important customer and a member of that the customer’s staff,
she mainly looks at the valued customer during his
explanation. Since such social gaze behaviors often occur inreal settings, social robots also need to be equipped with anappropriate gaze-ratio control system when they act in suchdaily situations as museum guides [1-4], providing
information to visitors or customers [5-10], and educational
support for children [11-14].
Body direction also has essential roles during
conversations. In the above situation where the salesperson
interacts with the important customer and others, her body
direction also turns to the customer. A past study investigatedThis research work was supported in part by the Tateishi Science andTechnology Foundation, in part by JST CREST Grant Number
JPMJCR18A1, Japan, and in part by JSPS KAKENHI Grant Numbers
18H03311, and JP18J12552, and JST, PRESTO Grant Number JPMJPR1851,
Japan.
H. Arai, M. Kimoto, T. Iio, K.Shimohara, Reo Matsumura, and Masahiro
Shiomi are with ATR, Kyoto, Japan. (e-mail: m-shiomi@atr.jp).
H. Arai, M. Kimoto, and K. Shimohara are also with Doshisha Univ.,
Kyoto, Japan.
T. Iio is also with University of Tsukuba / JST PRESTO, Ibaraki, Japan.
Reo Matsumura is also with karakuri products Inc.

the effects of a virtual agent’s body direction during an
information-providing context and concluded that when the
body direction was aligned toward its gaze direction, theperceived interest toward the contents improved [15]. Other
studies focused on a robot's gaze ratio when the robot interacts
with multiple persons, but less on its body direction effects.
For instance, to maintain the O-space spatial relationship, paststudies fixed the robot’s body direction between the people
and/or the objects [11, 16].
How do social robots assign gaze ratios to interact withmultiple people with different authority levels and hence
deserve different levels of respect? How does a robot’s body
direction influence the perceived impression to such multiple
people? One simple answer is always turning and only looking
at a person with more authority, but such an approach might bedemeaning to another person with less prestige. If we can findan acceptable, balanced gaze ratio that influences the body
direction for both people, the robot might achieve morenatural and acceptable behaviors in such situations. Althoughseveral human-robot interaction studies modeled natural gazebehaviors [11, 16-19], they focused less on situations where arobot interacts with multiple people based on their priority
relationships. We discussed the details in the next section.
Based on these contexts, we investigate the effects of gazeratios and body direction when a robot simultaneouslyinteracts with multiple persons to build into robots the ability
to signal different levels of importance in conversations. First,
we employ an information-providing situation with two
persons: a VIP and a follower. We develop a gaze-controller
system for a social robot with a depth sensor and answer thefollowing two questions:- What well-balanced gaze ratio is appropriate when asocial robot interacts with people who have different levels of
authority?
- How does the robot’s body direction change theperceived impressions of the people who have different levelsof authority? If so, what is a better combination between its
body direction and gaze ratio?

Figure 1. Priority-oriented (left) and non- priority-oriented (right) body
direction during gazing behavior of a social robot

II. RELATED WORK
A. Gaze behavior design in human-robot interaction
Social gaze behavior is an active research topic inhuman-robot interaction. For instance, several researchersmodeled a natural gaze control mechanism in human-robot
interaction [11, 16-20]. Yamazaki et al. investigated theprecise timing for controlling a robot’s head (including gaze)and utterances for social robots [18]. Mutlu et al. pioneeredmodeling gaze behaviors for social robots in multi-party
conversation settings, scrutinized human gaze behaviors, andimplemented them in social robots [17, 19, 21]. Other
researchers also developed mechanisms to control a social
robot’s gaze behaviors for multiple children based on Mutlu’s
work [11, 16]. Vázquez et al. investigated the effects of bodyorientation and gaze direction in group conversations between
robots and people, but the robot’s role was mainly as anaddressee and priority relationships weren’t considered [22].
From another perspective, Pejsa et al. investigated the effectsof gaze and body direction and reported that affiliative gazeprovides more positive perceived impressions such as
likability and rapport than referential gaze [15].
These research works identified rich knowledge aboutmodeling natural gaze behaviors for social robots and theireffectiveness through various experiments on human-robot
interaction. However, their interaction targets all have thesame priorities. In other words, they focused less on gazeratios and body direction effects for social robots when theyinteract with multiple persons who have different power
relationships. One unique point of our study is its explorationof gaze behavior for a social robot by considering different
priorities between interacting people.
B. Relationship effects in human-robot interaction
Typical human-robot interaction generally involves
one-to-one interaction between a social robot and just one
person. In such situations, robotics researchers usually assign
lower authority to the robot than an interacting person, e.g.,
the robot always prioritizes the interaction person. For
example, past studies developed a social coordination
mechanism that prioritizes people for human-aware robot
navigation and/or social coordination [23, 24].

recognizes the most important person in working contexts,
appropriate behaviors and the effects of gaze ratio and body
direction remain unknown. Therefore, we investigated theeffects of gaze ratio and body direction to prioritize more
powerful individuals.
III. EXPERIMENT DESIGNIn this section, we describe our robot and its sensors, thedetails of our gaze-control system, and its experimental
settings. Fig. 2 shows an overview of our developed systemthat consists of the following components: a human-tracking
system, a behavior controller, and a robot. The details of eachcomponent are described as follows.
Gaze model
Depth
sensorHuman
tracking

PositionGazetargetSpeechsynthesis
Command

Behavior
controller

Gesturecontrol

SpeechRobot
MotionFigure 2. System overview

A. Human-tracking system
For multiple-person tracking, we used one depth sensor(Kinect V2) and a laptop PC to process the sensor data. The
area for recognizing a person ranged from 0.5 to 4.5 m, thefield’s horizontal angle was 70°, and its perpendicular angle
was 60°. This system sends the estimated head positions of thepeople using Kinect V2 libraries to the behavior selector
through a wired network connection.
B. Robot
We used Sota, a desktop-sized interactive humanoid robot
characterized by its humanlike physical expressions (Fig. 1). It
has eight degrees of freedom (DOF): three in its head, two for
its shoulders, one in each elbows, and another in its base. It is
28 cm tall with a voice synthesis function that can produce
childlike and non-gendered sounds. The LED on its mouthblinks depending on the sound level to indicate speaking. Itsgazes are autonomously controlled with a developed gazemodel.

Recently, due to installing social robots in actual
environments, robots simultaneously need to interact withmultiple persons. In such situations, they must consider the
relationships among the interacting people to achieve morenatural interactions. Researchers first focused on developing
sensing systems that understand the relationships of people
and used such information to decide a robot’s behaviors. For
instance, a robot approached parents to escape being bulliedby children in a mall environment [25]. Another study
estimated the social status of children by observing
interactions with robots and children [26]. Oliveira et al.
investigatedpeople’s
non-verbal
behaviorsandsocioemotional interactions toward robots with different
relationships, such as partners or as opponents [27].

C. Gaze model
Although several gaze models have already been proposedin multi-party conversations [11, 17, 19], they mainly focusedon situations where the interaction people share an identicalpower relationship. On the other hand, in this study the robot
needs to appropriately assign the gaze ratio to show anawareness of the people’s authority levels during
information-providing. First, we developed a basic gazecontrol mechanism based on a previous study [17] whosemechanism focused on an information-providing situation
where a robot interacts with two persons. That previous study
only focused on using gaze behaviors, not body direction. In
their model on average, the robot looked at the two addresses
and its environment 35.5% and 29% of the time.

However, even if these studies enable social robots tointeract with people by considering their relationships orrecognizing them, they focused less on situations where arobot simultaneously interacts with multiple people who sharedifferent level of priorities. In other words, even if a robot

In our setting, the gaze targets are the two addresses andthe environment. Based on a previously defined gaze-ratio
probability [17], we set the initial gaze ratios to the targets to
35.0, 35.0, and 30.0% for an addressee, another addressee, andthe environment. When the gaze target is the environment, thegaze angle was calculated from the points of the heads of thetwo participants. The robot’s yaw was the center of the twoparticipants, and the pitch was the center of the height of thetwo participants’ heads minus 20 cm. We changed the gazetarget every three seconds.
To investigate the gaze-ratio effects (Table I and Fig. 3),
we fixed the gaze ratio to the environment (i.e., 30%) and onlychanged the balances for the addresses between 100:0 to 50:50.
Thus, “100:0” indicates that the robot gazed at the VIP at 70%,
the environment at 30%, and did not gaze at the follower.
“50:50” indicates that the robot equally gazed at both the VIPand the follower at 35%, and the environment at 30%. We
used slice resolutions as 10% ratio differences tocomprehensively investigate the gaze-ratio effects, which weinvestigated with sliced resolutions, i.e., instead of gatheringgaze-ratio data from human-human interaction settings, toavoid local minimum solutions about the gaze ratio.
During the priority-oriented body direction, the robot
turned its face and base to the VIP when the gaze target was a
VIP and only turned its face to the follower when the follower
was the gaze target (Table II). During thenon-priority-oriented body direction, the robot’s base was
fixed between the VIP and the follower and only turned its
face to the gaze target (Table III).
IV. EXPERIMENT

70% (100:0) ~35% (50:50)

0% (100:0) ~35% (50:50)

Environment
VIP

Follower

Robot
Figure 3. Illustration of gaze ratio for each target
TABLE II.

GAZE BEHAVIOR IN PRIORITY-ORIENTED CONDITION
Gaze target
VIP

Environment

Follower

VIP’s
viewpoint

A. Hypothesis and prediction
In human-human interaction, we implicitly change our
behavior based on the authority relationships of those people
with whom we are interacting. For example, a salesperson willprobably turn to and look at a VIP more attentively than othercustomers. However, past human-robot interaction studies
focused less on such priority relationships when a robot
interacted with multiple people. By considering people’s
behaviors in such situations, we believe that gaze ratios andbody direction can change the perceived impressions ofpeople who have different authority levels and would
outperform a commonly used gaze behavior for social robots.
Based on these considerations, we made the following
hypotheses:

Follower’s
viewpoint

TABLE III.

Due to the difficulties of complex combinations between
all the gaze ratios and body directions, we cannot predict
beforehand which combination is better between them. If ourprediction is supported, we can identify better combinations
that maximize user satisfaction by comparing the baseline
group and other groups.

VIP

VIP’s
viewpoint

GAZE RATIOS FOR VIPS AND FOLLOWERS

Gaze ratio

100:0

90:10

80:20

70:30

60:40

50:50

VIPs

70%

63%

56%

49%

42%

35%

Followers

0%

7%

14%

21%

28%

35%

Environment

30%

30%

30%

30%

30%

30%

GAZE BEHAVIOR IN NON-PRIORITY-ORIENTED CONDITION
Gaze target

Prediction 1: Specific combinations between biased gazeratio and body direction will be evaluated more highly than
traditional gaze behavior (i.e., 50:50 gaze ratio andnon-priority-oriented body direction).

TABLE I.

30% (always)

Follower’s
viewpoint

Environment

FollowerD. Participants
Twenty participants (ten females/ males, whose averageages were 30.25) participated in our experiment. They did notmeet beforehand. All the trials included both genders. The
numbers of VIP/follower roles were evenly distributed bygender.

SensorRobot

VIP

Follower
Figure 4. Experiment settings

B. Conditions
To verify our predictions, we conducted an exploratory
experiment to investigate the effects of the gaze-ratio
balances described in Section III.B as well as the body
direction (priority-oriented/non-priority-oriented).
This study used a mixed factorial design that combined
within- and between-participant designs. The number ofwithin-participant factors was two: gaze-ratio andbody-direction. The between-participant factor was theparticipant’s roles: VIP and follower.
Gaze-ratio factor: This factor has six conditions. The
robot changed the gaze ratios as described in Table 1.
Body-direction factor: This factor has two conditions. In
the priority-oriented condition, the robot’s body direction isfixed on the VIP, and the robot changes its face direction tolook at the gaze target. In the non-priority-oriented condition,
the robot’s body direction is fixed toward the front (i.e.,environment), and the robot only changes its face direction tolook at the gaze target.
Role factor: This factor has two conditions: VIP andfollower, both of which were assigned randomly. During theexperiment, the participants did not change their roles.
C. Procedures
Before the experiment, the participants were given a brief
description of the experiment procedure. In this study, weassigned different roles to each participant (VIP or follower)
who worked together. Note that the assigned roles and theirposition information were manually inputted to the robot
system to control the behaviors. Both participants listened to
the robot and were told that the follower role has less authority
than the VIP role. For instance, we told the participants toimagine several situations, such as a subordinate (follower)
who is explaining her developed system to her superior (VIP)and where a follower guides a VIP at a shopping mall, and soon.
Participants joined 12 sessions. In each session, theylistened to the robot’s two-minute explanation about asmartphone. We prepared six conversation contents (e.g., therobot explains the item’s name, its specs, and so on) andassigned them randomly during the 12 sessions. Note that weequaled the importance of each content and set similar lengths
as much as possible because the importance of the contents
influenced the timing of the gaze changes. After each session,both participants filled out questionnaires.

E. Environment
Figure 4 shows an experiment scene where a robot
provides information to two participants and looks at thefollower. We prepared six kinds of two-minute contents about
smartphones to reproduce a salesperson role for the robot. The
participant playing the VIP sat in the left side chair in front of
the robot, and the participant playing the follower sat on theright side.

F. Measurements
We measured four subjective items related to theperceived impressions by questionnaires to investigate thegaze-ratio and body-direction effects. The items were
evaluated on a 1-to-7-point scale, where 1 is the most negativeand 7 is the most positive.
- Intention to use (ITU) was three previously modified items[28]. The Cronbach alpha [29], which was 0.964 in thisexperiment, showed acceptable values for analysis.
- We defined two items for politeness: “The robot is polite”
and “the robot is rude” (reversed item). The Cronbach alphawas 0.881.
- We defined two items for understandability: “this robot
understood the power dynamic between the participants” and“this robot behaved based on an understanding of the power
dynamic of the participants.” The Cronbach alpha was 0.791.
- We defined one item for the total impression: “the total
impression of the robot is good.”
V. RESULTS
A. Analysis of intention to use
Figure 5 shows the questionnaire results of intention to use.
We conducted a three-factor mixed ANOVA for each scale on
the gaze-ratio, body-direction, and role factors. The results(Table IV) showed a significant effect in the simple interaction
effects between the gaze-ratio and body-direction factors(F(5,90)=2.561, p=.033, partial η2=.125). We did not identify
any significant main effects in the body-direction factor, thegaze-ratio factor, the role factor, the simple interaction effectbetween the body-direction and the role factors, the simple
interaction effect between the gaze-ratio and the role factors,
or the two-way interaction effects.
For a multiple comparison, we conducted a Dunnett's test
to investigate what combinations between the gaze ratio andbody direction outperformed the traditional gaze behavior(Section IV.A). For this purpose, we set the 50:50 gaze ratio
and the non-priority-oriented body direction as the baseline
and conducted a pairwise test with other combinations. As aresult, four combinations were more highly evaluated (i.e.,p<.05) than the baseline: 80:20, 90:10, and 100:0 with thenon-priority-oriented body direction and 100:0 with thepriority-oriented body direction.

7*(Non-priority-oriented)

6* (Non-priority-oriented)

5*(Non-priority-oriented)* : p < .05

4

3
2* (Priority-oriented)

1
50:50

60:40

70:30

80:20

Non-priority-oriented

90:10

100:0

Priority-oriented

interaction effect between the body-direction and the role
factors, the simple interaction effect between the gaze-ratio
and the role factors, or the two-way interaction effects.
For a multiple comparison, we also conducted a Dunnett's
test for a pairwise test between the baseline and othercombinations. Ten combinations were more highly evaluated(i.e., p<.05) than the baseline: 70:30, 80:20, 90:10, and 100:0
with the non-priority-oriented body direction, and all the gazeratios with the priority-oriented body direction. Therefore,
we only showed the non-significant combination (60:40 gazeratio and non-priority-oriented body direction) in the Fig.7 forreadability.

Figure 5. Questionnaire results of intention to use. Marks indicate
significant differences compared to baseline (i.e., 50:50 gaze ratio andnon-priority-oriented body direction).

* (Non-priority-oriented)

7* : p < .05** (Non-priority-oriented)(Non-priority-oriented)

65

TABLE IV.
STATISTICAL RESULTS OF INTENTION TO USE(BOLD INDICATES THAT P-VALUE IS LESS THAN .05)

4

Partial η2

3

0.857

0.002

2

1.959

0.092

0.098

1

Role (R)

3.422

0.081

0.160

B*R

0.072

0.791

0.004

G*R

0.495

0.779

0.027

B*G

2.561

0.033

0.125

Figure 6. Questionnaire results of politeness. Marks indicate significant
differences compared to baseline (i.e., 50:50 gaze ratio andnon-priority-oriented body direction).

B*G*R

0.253

0.937

0.014

TABLE V.

SourceFp

Body direction (B)0.033

Gaze (G)50:50

B. Analysis of politeness
Figure 6 shows the politeness results of understandability.
We conducted a three-factor mixed ANOVA for each scale on
gaze-ratio, body-direction, and role factors. The results (TableV) showed a significant effect in the simple interaction effects
between the body-direction and gaze-ratio factors(F(1,18)=3.797, p=.004, partial η2=.174). We did not identify
any significant main effects in the body-direction factor, thegaze-ratio factor, the role factor, the simple interaction effectbetween the body-direction and the role factors, the simple
interaction effect between the gaze-ratio and the role factors,
or the two-way interaction effects.
For a multiple comparison, we also conducted a Dunnett's
test for the pairwise test between the baseline and othercombinations. Three combinations were more highly
evaluated (i.e., p<.05) than the baseline: 70:30, 80:20, and90:10 with a non-priority-oriented body direction.
C. Analysis of understandability
Figure 7 shows the questionnaire results ofunderstandability. We conducted a three-factor mixed
ANOVA for each scale on the gaze-ratio, body-direction, androle factors. The results (Table VI) showed significant main
effects in the gaze-ratio factor (F(1,18)=12.483, p=.001,
partial η2=.410) and simple interaction effects between thebody-direction and gaze-ratio factors (F(1,18)=4.749, p=.001,
partial η2=.209). We did not identify any significant effects inthe body-direction factor, the role factor, the simple

60:40

70:30

80:20

Non-priority-oriented

90:10

100:0

Priority-oriented

STATISTICAL RESULTS OF POLITENESS (BOLD
INDICATES THAT P-VALUE IS LESS THAN .05)

SourcePartial η2

Fp

Body direction (B)

1.913

0.184

0.096

Gaze (G)

1.151

0.339

0.060

Role (R)

0.043

0.838

0.002

B*R

0.478

0.498

0.026

G*R

0.784

0.564

0.042

B*G

3.797

0.004

0.174

B*G*R

2.201

0.061

0.109

80:20

90:10

7
6

5
4
3
2
n.s (Non-priority-oriented)

1
50:50

60:40

70:30

Non-priority-oriented

100:0

Priority-oriented

Figure 7. Questionnaire results of understandability. Except for “n.s” mark(60:40 gaze ratio and non-priority-oriented body direction), all comparisonswith baseline showed significant differences.

TABLE VI.

STATISTICAL RESULTS OF UNDERSTANDABILITY(BOLD INDICATES P-VALUE IS LESS THAN .05)

SourcePartial η2

Fp

Body direction (B)

3.508

0.077

0.163

Gaze (G)12.483

0.001

0.410

Role (R)

0.743

0.400

0.040

B*R

0.284

0.601

0.016

G*R

1.396

0.253

0.072

B*G

20.243

0.001

0.529

B*G*R

1.914

0.183

0.096

* : p < .05* (Non-priority-oriented)

7** (Non-priority-oriented)(Non-priority-oriented)

6

5
4

3

For a multiple comparison, we also conducted a Dunnett's
test for the pairwise test between the baseline and othercombinations. Three combinations were more highly
evaluated (i.e., p<.05) than the baseline: 70:30, 80:20, and90:10 with the non-priority-oriented body direction.
E. Summary of analysis
By considering the statistical analysis results, prediction 1
was supported. We found several combinations between thebiased gaze ratio and the body direction that are more highly
evaluated than the baseline. We also found that the better gazeratios are different based on body directions. In the
non-priority-oriented body direction, 80:20 or 90:10 gazeratios maximized participant satisfaction because they are
significantly better than the baseline in all the measurements.
In the priority-oriented body direction, the 100:0 gaze ratio
maximized participant satisfaction in some of themeasurements. Note that all the gaze ratios with thepriority-oriented body direction were better from theviewpoint of understandability. If a robot needs to keep a
non-biased gaze ratio (i.e., 50:50), a priority-oriented body
direction remains useful to imply its capability to people whohave different authority. We found no significant differencesin the role factor.
VI. DISCUSSION

2

1
50:50

60:40

70:30

80:20

Non-priority-oriented

90:10

100:0

Priority-oriented

Figure 8. Questionnaire results of total impression. Marks indicate
significant differences compared to baseline (i.e., 50:50 gaze ratio andnon-priority-oriented body direction)
TABLE VII.

STATISTICAL RESULTS OF TOTAL IMPRESSIONS(BOLD INDICATES P-VALUE IS LESS THAN .05)

SourcePartial η2

Fp

Body direction (B)

1.176

0.293

0.061

Gaze (G)

1.496

0.199

0.077

Role (R)

0.405

0.533

0.022

B*R

0.035

0.854

0.002

G*R

0.445

0.816

0.024

B*G

2.615

0.030

0.127

B*G*R

0.734

0.600

0.039

D. Analysis of total impressions
Figure 8 shows the questionnaire results of the total
impressions. We conducted a three-factor mixed ANOVA foreach scale on the gaze-ratio, body-direction, and role factors.
The results (Table VII) showed simple interaction effects
between the body-direction and gaze-ratio factors(F(1,18)=2.615, p=.030, partial η2=.137). We did not identify
any significant main effects in the body-direction, gaze-ratio,
and the role factors or in the simple interaction effects
between the body-direction and role factors, between thegaze-ratio and role factors, or between the two-way interaction
effects.

A. Design implications and contribution to HRI
One important contribution of this paper is that we
identified the importance of the biased gaze ratio for social
robots that must consider different power levels for the people
with whom they are interacting. Since past studies basicallyassumed that interacting people have identical status, theexisting gaze models assign similar gaze ratios to social robots.
However, situations where robots need to interact withmultiple people with different levels of authority must be
considered in daily environments. Our results show that abalanced gaze ratio (50:50) in such situations create relatively
negative impressions in people.
Based on these results, we discuss two guidelines and thedesign implications for the gaze behaviors of social robots toshow an awareness of authority. First, if a developer uses a
robot with the capability to change both its face and body
direction (like Pepper, Nao, and Sota), its gaze behaviors canbe controlled for information-providing tasks in two ways:
non-priority-oriented body direction or priority-oriented body
direction with biased gaze ratios. Although our study did not
find any significant differences between them, it did identify
advantages for the non-priority-oriented body direction with abalanced gaze ratio, which is the traditional way of
information-providing for social robots.
Second, if a developer uses robots with relatively simple
mechanisms that can only change their face direction (likePaPeRo I or static robots that use displays to show their eyes),i.e., can only use the non-priority-oriented body direction, abiased gaze ratio with 80:20 or 90:10 ratio is more suitable. In
other words, our study also suggests that only changing thegaze ratio can exhibit an awareness of authority, and thisknowledge is useful for such robots.
From another perspective, if a robot needs to show suchawareness of priority, it should consider its own available
modalities in interaction. In this study, the robot uses suchdifferent modalities as gaze direction and body direction. Our
results suggest that a biased ratio of either modality is usefulfor such purposes, i.e., participants more highly evaluated apriority-oriented body direction with a 50:50 gaze ratio (i.e.,
only its body direction is biased) and a 100:0 gaze ratio (boththe gaze ratio and body direction are biased), or a
non-priority-oriented body direction with a 80:20/90:10 ratio(only the gaze ratio is biased). Based on these considerations,if a developer uses a robot that can only change its body
direction (like Keepon), i.e., can only use a priority-oriented
body direction, a biased gaze ratio of 80:20/90:10 is probably
more suitable because such a robot can only use its body
direction as a modality. Our results suggest that completely
ignoring a follower might offend, as discussed above.
B. Gaze behavior design with more interacting people
In this study, we only focused on a situation with two
people. If a robot interacts with more people, what kinds ofgaze ratios and behaviors are appropriate? When the number
of VIPs/followers increases, we thought that simply
separating the gaze ratios based on their number isinappropriate. We suggested assigning a 80/90% ratio to themost important target and spreading the remaining 20/10%
among the other targets. During a lecture, the robot should
sweep its gaze over all the audience members and make eye
contact with VIPs.
Moreover, in such situations, accurate human-tracking
systems and person identification functions are needed. In
this study we manually assigned role information to the robot
system, but such recent image recognition functions as face
identification are useful for autonomous systems that identify
VIPs/followers from among interacting people. Several paststudies also used environmental sensor systems tosimultaneously track and identify multiple people [26, 30].
Such knowledge is critical for interacting with multiple
people based on priority relationships.
C. Different modalities to show awarenessIn this study, we focused on gaze ratios and behaviors toshow an awareness of power relationships, but of course therobot can use other modalities for this purpose. For example,
if it uses respectful language and/or gestures during
interaction with a specific interacting person, such behavior
conveys an awareness of authority. Combinations ofappropriate gaze behaviors and such different modalities arealso useful; as discussed above, some different modalities
should be used with a biased ratio to show such awareness.
On the other hand, if a robot’s gaze behavior and othermodalities are mismatched, i.e., if a robot uses respectfullanguage but a smaller gaze ratio, people might infer
awkward impressions. We did not investigate different levelsof power relationships. If a robot interacts with a person
whose status is much higher, such as a member of a royalfamily, the robot must show such awareness using all of its
modalities.

TABLE VIII. CORRELATION AMONG QUESTIONNAIRE ITEMS. *SIGNIFICANT CORRELATION AT 0.005 LEVEL (2-TAILED)

1. Intention to use
2. Politeness3. Understandability
4. Total impression

1
0.684*
0.573*
0.797*

2
0.598*
0.856*

3
0.645*

4-

D. Factor analysis among measurements
In this study, three measurements (i.e., intention to use,politeness, and total impression) show similar patterns basedon gaze ratios and body directions. Our correlation analysis
showed that all items are strongly correlated with significant
levels (Table VIII). Therefore, we conducted a factor analysis
to investigate the relationships among the measurements. It
yielded two factors that explain 68.4% of the variance for theentire set of variables. This first factor, hereinafter “perceived
usability”, which consists of intention to use, politeness, andtotal impressions, explained 59.95% of the variance, and thesecond factor (understandability) explained 8.53% of it.
These results suggest that the understandability did not
directly influence the perceived usability (i.e., intention to use,politeness, and total impressions).
E. Limitation and future work
Since we only used an existing robot (Sota), generality
about robot appearance is limited. To use our knowledge inactual environments, we need to investigate whether different
kinds of robots show an awareness of power relationships
using biased gaze ratios and body directions.
We experimented with a pair of participants with arepresentative of each gender, i.e., only cross-gender settings.
Therefore, investigating critical mixed gender effects while
analyzing gazing behaviors would provide richer knowledge.
In addition, such other factors as age and appearance effects
are important.
In this study, the robot’s body direction always faced the
VIPs as priority-oriented body direction settings, but theexperiment did not cover the effects of mixed gaze behavior
between the non-priority-oriented and priority-oriented body
directions.
Another future work will investigate the relationships
between the importance of the conversation contents and thetiming of gaze changes. For example, a salesperson will lookat and turn to a VIP person when she explains a critical pieceof information. Although we leveled the importance of the
explained contents with similar lengths to avoid this difficulty,
it would be another interesting future work.
VII. CONCLUSION
During interactions with people who have different levelsof authority, gaze ratios and body-directions are useful toconvey an awareness of such social power dynamics. In thisstudy we investigated which gaze ratios between VIPs andfollowers and which body direction (i.e., priority-oriented or
non-priority-oriented) are conducive for social robots that areinteracting with people of different power relationships in aninformation-providing task. We developed a gaze control
mechanism for a social robot and conducted experiments.
Our experiment results showed that a biased gaze ratio
and body direction while gazing are crucial to exhibit anawareness of authority and the perceived impressions of therobot from the participants. Our results suggest that if therobot uses a non-priority-oriented body direction, biased gazeratios (80:20/90:10) will be more positively inferred by thepeople. If the robot uses a priority-oriented body direction, acompletely biased gaze ratio (100:0) will be positively
perceived; a balanced gaze ratio (50:50) also has anadvantage to convey its capability to people who havedifferent levels of authority. These experiment results provide
knowledge about future gaze behavior design for social
robots.
REFERENCES[1] W. Burgard, A. B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D.
Schulz, W. Steiner, and S. Thrun, “Experiences with an interactive
museum tour-guide robot,” Artificial intelligence, vol. 114, no. 1-2, pp.
3-55, 1999.
[2] I. R. Nourbakhsh, C. Kunz, and T. Willeke, “The mobot museum robot
installations: A five year experiment,” in IEEE/RSJ International
Conference on Intelligent Robots and Systems, 2003 (IROS 2003), pp.
3636-3641, 2003.
[3] M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita, “Interactive
Humanoid Robots for a Science Museum,” IEEE Intelligent Systems,
no. 2, pp. 25-32, 2007.
[4] T. Iio, Y. Yoshikawa, and H. Ishiguro, “Retaining Human-Robots
Conversation: Comparing Single Robot to Multiple Robots in a RealEvent,” Journal of Advanced Computational Intelligence andIntelligent Informatics, vol. 21, no. 4, pp. 675-685, 2017.
[5] S. Satake, K. Hayashi, K. Nakatani, and T. Kanda, “Field trial of an
information-providing robot in a shopping mall,” in The IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS
2015) pp. 1832-1839, 2015.
[6] R. Triebel, K. Arras, R. Alami, L. Beyer, S. Breuers, R. Chatila, M.
Chetouani, D. Cremers, V. Evers, and M. Fiore, “Spencer: A sociallyaware service robot for passenger guidance and help in busy airports,”
in Field and Service Robotics, pp. 607-622, 2016.
[7] A. M. Sabelli and T. Kanda, “Robovie as a Mascot: A Qualitative Studyfor Long-Term Presence of Robots in a Shopping Mall,” International
Journal of Social Robotics, vol. 8, no. 2, pp. 211-221, 2016.
[8] M. Joosse and V. Evers, “A Guide Robot at the Airport: First
Impressions,” in Proceedings of the Companion of the 2017
ACM/IEEE International Conference on Human-Robot Interaction,
Vienna, Austria, pp. 149-150, 2017.
[9] D. Sakamoto, et al., “Humanoid Robots as a Broadcasting
Communication Medium in Open Public Spaces,” International Journalof Social Robotics, vol. 1, no. 2, pp. 157-169, 2009.
[10] R. Matsumura, M. Shiomi, and N. Hagita, “Does an Animation
Character Robot Increase Sales?,” in Proceedings of the 5thInternational Conference on Human Agent Interaction, pp. 479-482,
2017.
[11] T. Komatsubara, M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita,“Can a social robot help children's understanding of science inclassrooms?,” in Proceedings of the Second International Conferenceon Human-agent Interaction, Tsukuba, Japan, pp. 83-90, 2014.
[12] I. Leite et al., “Emotional Storytelling in the Classroom: Individual
versus Group Interaction between Children and Robots,” inProceedings of the Tenth Annual ACM/IEEE International Conferenceon Human-Robot Interaction, Portland, Oregon, USA, pp. 75-82, 2015.
[13] P. Alves-Oliveira, T. Ribeiro, S. Petisca, E. Di Tullio, F. S. Melo, and A.
Paiva, “An empathic robotic tutor for school classrooms: considering
expectation and satisfaction of children as end-users,” in International
Conference on Social Robotics, pp. 21-30, 2015.
[14] M. Shiomi, T. Kanda, I. Howley, K. Hayashi, and N. Hagita, “Can a
Social Robot Stimulate Science Curiosity in Classrooms?,”

International Journal of Social Robotics, vol. 7, no. 5, pp. 641-652,
2015.
[15] T. Pejsa, S. Andrist, M. Gleicher, and B. Mutlu, “Gaze and Attention
Management for Embodied Conversational Agents,” ACM Trans.
Interact. Intell. Syst, vol. 5, no. 1, pp. 1-34, 2015.
[16] Y. Tamura, M. Kimoto, M. Shiomi, T. Iio, K. Shimohara, and N. Hagita,“Effects of a Listener Robot with Children in Storytelling,” inProceedings of the 5th International Conference on Human AgentInteraction, Bielefeld, Germany, pp. 35-43, 2017.
[17] B. Mutlu, T. Kanda, J. Forlizzi, J. Hodgins, and H. Ishiguro,“Conversational gaze mechanisms for humanlike robots,” ACM
Transactions on Interactive Intelligent Systems (TiiS), vol. 1, no. 2, pp.
12, 2012.
[18] A. Yamazaki, K. Yamazaki, Y. Kuno, M. Burdelski, M. Kawashima,and H. Kuzuoka, “Precision timing in human-robot interaction:
coordination of head movement and utterance,” in Proceeding of thetwenty-sixth Annual SIGCHI Conference on Human factors in
Computing Systems, Florence, Italy, pp. 131-140, 2008.
[19] B. Mutlu, J. Forlizzi, and J. Hodgins, “A storytelling robot: Modelingand evaluation of human-like gaze behavior,” in The 6th IEEE-RAS
International Conference on Humanoid Robots, pp. 518-523, 2006.
[20] A. Moon, D. M. Troniak, B. Gleeson, M. K. Pan, M. Zheng, B. A.
Blumer, K. MacLean, and E. A. Croft, “Meet me where I'm gazing:
how shared attention gaze affects human-robot handover timing,” inProceedings of the 2014 ACM/IEEE International Conference on
Human-Robot Interaction, pp. 334-341, 2014.
[21] B. Mutlu, T. Shiwa, T. Kanda, H. Ishiguro, and N. Hagita, “Footing inhuman-robot conversations: how robots might shape participant roles
using gaze cues,” in Proceedings of the 4th ACM/IEEE International
Conference on Human-Robot Interaction, pp. 61-68, 2009.
[22] M. Vázquez, et al., “Towards robot autonomy in group conversations:
Understanding the effects of body orientation and gaze,” inProceedings of the 2017 ACM/IEEE International Conference on
Human-Robot Interaction, pp. 42-52, 2017.
[23] M. Kollmitz, K. Hsiao, J. Gaa, and W. Burgard, “Time dependent
planning on a layered social cost map for human-aware robot
navigation,” in Mobile Robots (ECMR), 2015 European Conference on,
pp. 1-6, 2015.
[24] S. Akita, S. Satake, M. Shiomi, M. Imai, and T. Kanda, “Social
Coordination for Looking-Together Situations,” in 2018 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS), pp.
834-841, 2018.
[25] D. Brscić, H. Kidokoro, Y. Suehiro, and T. Kanda, “Escaping fromChildren's Abuse of Social Robots,” in Proceedings of the Tenth
Annual ACM/IEEE International Conference on Human-Robot
Interaction, pp. 59-66, 2015.
[26] T. Komatsubara, M. Shiomi, T. Kaczmarek, T. Kanda, and H. Ishiguro,“Estimating Children’s Social Status Through Their Interaction
Activities in Classrooms with a Social Robot,” International Journal ofSocial Robotics, pp. 1-14, 2018.
[27] R. Oliveira, P. Arriaga, P. Alves-Oliveira, F. Correia, S. Petisca, and A.
Paiva, “Friends or foes?: Socioemotional support and gaze behaviors inmixed groups of humans and robots,” in Proceedings of the 2018
ACM/IEEE International Conference on Human-Robot Interaction, pp.
279-288, 2018.
[28] M. Heerink, K. Ben, V. Evers, and B. Wielinga, “The influence ofsocial presence on acceptance of a companion robot by older people,”
Journal of Physical Agents, vol. 2, no. 2, pp. 33-40, 2008.
[29] L. J. Cronbach, “Coefficient alpha and the internal structure of tests,”
psychometrika, vol. 16, no. 3, pp. 297-334, 1951.
[30] M. Shiomi and N. Hagita, “Finding a person with a wearable
acceleration sensor using a 3D position tracking system in daily
environments,” Advanced Robotics, vol. 29, no. 23, pp. 1563-1574,
2015.