Work-in-Progress: Emotions

CHI 2013: Changing Perspectives, Paris, France

Expressing a Robot’s Confidencewith Motion-based Artificial Subtle
Expressions
Seiji Yamada
National Institute ofInformatics / SOKENDAI
2-1-2 Hitotsubashi, Chiyoda,
Tokyo, 101-8430 Japanseiji@nii.ac.jp

Takanori Komatsu
Meiji University
1-1 Kandasurugadai, Chiyoda
Tokyo, 101-8301 Japantkomat@acm.org

Kazunori Terada
Gufu University
1-1 Yanagido
Gifu, 501-1193 Japan
terada@gifu-u.ac.jp

Kotaro Funakoshi
Honda Research InstituteJapan Co., Ltd.
8-1 Honcho, Wako
Saitama, 351-0188 Japan
funakoshi@jp.honda-ri.com

Abstract
In this paper, motion-based Artiﬁcial Subtle Expression(ASE) as a novel implementation of ASE is described, inwhich a robot expresses conﬁdence in its advice to ahuman. Conﬁdence in advice is one of robot’s usefulinternal states, and it is an important goal to develop apractical and inexpensive methodology to correctly express
it. To achieve this goal, we propose motion-based ASE in
which a robot slowly hesitates by turning to a humanbefore giving advice with low conﬁdence. We conducted
experiments to evaluate the eﬀectiveness of motion-based
ASE with participants, and obtained promising results.

Author Keywords
Kazuki Kobayash
Shinshu University
4-17-1 Wakasato
Nagano, 380-8553 Japan
kby@shinshu-u.ac.jp

Mikio Nakano
Honda Research InstituteJapan Co., Ltd.
8-1 Honcho, Wako
Saitama, 351-0188 Japannakano@jp.honda-ri.com

Expression of a robot; ASE; Human-Agent Interaction(HAI); HRIACM Classification Keywords
H.5.2 [[Information Interfaces And Presentation]]: User
Interfaces - Theory and methods;

General Terms
Experimentation; Design; Human Factors.
Copyright is held by the author/owner(s).
CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris,
France.
ACM 978-1-4503-1952-2/13/04.

Introduction
In Human-Agent Interaction (HAI) and Human-Robot
Interaction (HRI), it is important that a robot informs a

1023

Work-in-Progress: Emotions

CHI 2013: Changing Perspectives, Paris, France

human of its internal state, including its conﬁdence inprocessed results, emotion, and so on [7]. In this work, we
focus on a robot’s confidence in the information that it
informs to a human, and we develop a simple andeﬀective method for expressing this conﬁdence with lesscognitive load. We consider conﬁdence to be one of therobot’s most signiﬁcant internal states that can be
expressed along with various pieces of information that a
robot informs a human of, e.g., a robot should express
conﬁdence in its advice during human decision making.
In human communication, a speaker’s conﬁdence in thecontent of what they are saying is frequently expressed as
non-verbal information, such by facial expressions, pitch,and gestures, to complement the main verbal
communication [6]. In particular, subtle expression [1] ofsuch non-verbal information is known to enrich humancommunication, and the engineering implementation
methodology of such subtle expression for virtual agents,
robots, and appliances is called “Artiﬁcial Subtle
Expression” (ASE) [8].
In this paper, we apply ASE to a robot so that it is able
to express its conﬁdence in advice given to a humanduring decision making. We developed a turning-aroundbehavior and a motion-based ASE (motion-ASE for short)that can be implemented simply by changing the velocityof the behavior. Then, we conducted experiments with atreasure chest guessing task with participants to evaluate
the eﬀectiveness of the motion-ASE.
Various studies on expressing a robot’s internal state havebeen done thus far for HRI. With a mobile robot, facial
expressions are indicated on the top of the robot [11].
Most of these studies were an investigation of therelationship between LED color and the expressed

emotions [9]. Almost all of the studies dealt with theemotions of a robot, not its conﬁdence.
Subtle expressions and non-verbal communication havealso been implemented in robots. There are studies on theeﬀective head movements of humanoid robots [4] andexperiments to change the movement velocity of a
shopping cart robot [3] for human-robot communication.
One of these researchers worked on expressing conﬁdencewith robots. A lot of studies needed a humanoid robot
and such a robot is very expensive and hard to control
precisely.
ASEs have been applied to various domains [8, 5] thus far.
We believe this work signiﬁcantly extends ASE’s
applicability to new domains, robot motions excluding
beep sounds [8], and LED lighting [5].

Motion-ASE: Expressing Confidence with Motion
Robot’s Confidence and Hesitation Behavior
Since the processing in a robot contains various noisefrom the real environment, the information expressed fromthe robot to a human frequently has uncertainty. This
uncertainty is signiﬁcantly caused by a limitation in the
robot’s ability, not by the ambiguity of humaninterpretation [10].
Thus, a robot needs to inform a human of its conﬁdencein HRI situations. Since conﬁdence signiﬁcantly inﬂuences
human decision making, a robot’s expression of conﬁdence
is important. We propose motion-ASE for mobile robots
so that they can express conﬁdence with simple
representations and low cognitive load required for humaninterpretation.

1024

Work-in-Progress: Emotions

CHI 2013: Changing Perspectives, Paris, France

In human communication, information with low conﬁdence
is frequently expressed with slow and pausing motions as
hesitation [2]. Thus, we implemented motion-ASE byusing hesitation behavior and having a robot slowly turnaround to face a human as is done in HRI where a robot
turns to a human before advising him/her. We consider
this behavior to be executed in various HRI situations. We
assume that, since slowly turning around can be used toexpress a robot’s hesitation to a human, the human feels
the conﬁdence in the advice is lower than when the robot
turns fast. Although there are various ways to use ASE to
express hesitation, we consider this slow turn to be one of
the most suitable expressions because it is simple, does
not require any additional equipment, and the cost of
implementation is very low. Furthermore motion-ASE has
the possibility to be extended to various robots for thesame reason. The concrete implementation of thehesitation behavior and experimental evaluation will be
discussed in the next section.
ASE Design Policy
We applied ASE design policy [8] to implement the abilityof a robot to express its conﬁdence in its advice to ahuman during decision making because ASE is suitable fora mobile robot to inform a human of its binary internal
state (i.e., high/low conﬁdence in this work) in an
inexpensive way [8]. ASE has the following four
requirements as the design policy, and they should besatisﬁed to realize ASE. The ﬁrst two are design
requirements that should be satisﬁed in the design phase,and the remaining two are function requirements thatshould be experimentally evaluated after the
implementation of ASE [8]. We will discuss how
motion-ASE satisﬁes the above requirements in the later
sections.

(A1) Simple: ASE is implemented in a single modality.
(A2) Complementary : ASE does not interfere with
communication’s main protocol (e.g. verbal
communication).
(A3) Intuitive: ASE is intuitively understandable.
(A4) Accurate: ASE conveys the designer’s intended
meanings accurately.
Since motion-ASE has the single visual modality of arobot turning to a human, (A1) is satisﬁed, and (A2) isalso satisﬁed because turning to a human is executed
quietly. The remaining (A3) and (A4) will be evaluatedlater in the discussion.

Experiments
We conducted experiments with participants to evaluate
the eﬀectiveness of motion-ASE. In this work, we set a
mobile robot to slowly turn around to face a human as the
motion-ASE described before. We used the speed of theturning as a parameter for expressing the robot’s
conﬁdence in its advice. A human sometimes turns quickly
to another to intentionally express strong agreement andsometimes turns slowly to express hesitation. Thus, we
believe the speed of the turning can be a parameter to beinvestigated for motion-ASE. Thus we introduced two
conditions: fast-motion and slow-motion.
As another parameter to be investigated for expressing
conﬁdence, we introduced delay time, which is the delay
before the robot starts to give advice. Since the robot’s
advice is given to a human just after turning around, thedelay time of the fast-motion condition is shorter thanthat of the slow-motion condition. To investigate how
only the delay time might inﬂuence the expression of therobot’s conﬁdence without turning around, we introduce
short-wait and long-wait conditions without the turning

1025

Work-in-Progress: Emotions

around. To clarify the requirements for the most eﬀective
expression, we used four conditions: fast-motion,slow-motion, short-wait, and long-wait for theexperiments.

Curtain
Table
Red
box

Blue
box

2m

140

°

Robot

CHI 2013: Changing Perspectives, Paris, France

Start point
Participant

1m

Wall

Figure 1: Experimental
environment.

Figure 2: Robot used in theexperiments (yellow lighting).

Figure 3: Snapshot ofExperiment at the Robot’s
behavior (R2).

Experimental Task, Environment and Robot
We used a treasure chest guessing task as a representative
experimental task of human decision making in which
participants were required to repeatedly guess which of
two boxes (i.e., a red or blue box) on a table contained atreasure. In each trial, a robot gave advice to theparticipants by saying the color (i.e., red or blue) of thebox that it guessed contained the treasure by using speechsynthesis. The participants did not know how the robot
could guess the location of the treasure.
We used between-participant design. A participant
repeatedly tried to guess the location of the treasure
under one of the four experimental conditions so thathe/she could not compare the behaviors of the diﬀerent
conditions. Participants were not told about the correct
location of the treasure after each guess so that they
could not estimate the probability that the robot’s advice
was correct by using the feedback history.
Figure 1 shows the experimental environment. The
experimental room was surrounded by curtains. The
curtain in front of the boxes was closed at the beginningof every trial, and then an experimenter opened thecurtain. Figure 2 shows the robot used through all theexperiments. It had two wheels, and its two motors were
controlled by a laptop PC through a USB connection.
Dead reckoning and computer vision were used to correct
the localization of the robot and the turning-aroundbehavior. The robot autonomously behaved without
WOZ.

Procedure, Evaluation Indexes, and ParticipantsParticipants were told that a treasure was set in either thered or blue box at the beginning of the trial and asked tochoose which box they expected to contain the treasure
after hearing the robot’s advice because the robot andthey were cooperative partners. No knowledge of therobot’s behaviors except that of the robot’s advice wasinformed to the participants. Also, they were required toscore their estimated conﬁdence of the robot’s advice byusing a seven-point-scale value after every trial. Eachparticipant performed this trial ten times under one of thefour conditions. The participant’s and the robot’s
behaviors, (P) and (Rn) are described as follows.
First, an experimenter hid the two boxes from eachparticipant by closing the curtain and put a treasure into
one of them. Then, he/she would say “Next problem” tothe participant.
(R1) The robot went to the front of the table. The
moving speed was 25 cm/sec.
(R2) It stopped there for two seconds to think. (Figure 3
shows a snapshot.)(R3) It blinked yellow for one second after ﬁnishing
thinking.
(R4) It expressed conﬁdence in its advice. In the fast or
slow-motion condition, the robot quickly or slowly
turned 140◦ to the participant for two or eight
seconds, respectively. In the short or long-wait
condition, it only stopped in front of the table fortwo or eight seconds, respectively.
(R5) It said ”Red” or ”Blue” as advice to the participant.
(P) The participant answered “Red” or “Blue” toindicate the expected location of the treasure after
hearing the robot’s advice, and then he/she scored1026

Work-in-Progress: Emotions

CHI 2013: Changing Perspectives, Paris, France

the estimated conﬁdence by using aseven-point-scale value.
(R6) The robot returned to the initial position.
12

*

t
n
a 10
ip
ci s
rta ceip vd 8
a a
s 's
e
t
o
im
t b 6
f o
r
o d
r
e e
b ts 4
m u
u tr
n
e
2
h
T

0

fast-motion slow-motion

fast-stillshort-waitslow-still
long-wait

Figure 4: The number of times aparticipant accepted robot’s
advices.

7

t
o
b6
ro
a
f5
o
e
c
n4
e
d
fi
n3
o
c
d
2
te
a
im
ts 1
E
0

**

**

fast-motion slow-motion*

short-wait
fast-still

long-waitslow-still

Figure 5: Estimated confidence
on the robot’s advice.

We measured the number of times a participant accepted
the robot’s advice for behavioral evaluation. We alsomeasured the estimated conﬁdence of the robot’s advice
evaluated by the participants by using a seven-point-scalevalue as a subjective and more detailed evaluation. We
expected “excellent expression” to outperform the others
in both evaluations.
For this experiment, we used 60 Japanese undergraduate
students in the computer engineering department of Gifu
University in Japan. They were 18 to 24 years old, andequally divided into four groups with almost the same
average age. In regard to the gender balance of theparticipants, there were 44 males and 16 females. Thisbalance was similar to that in the department, and there
was no intended control for the gender balance.
Results
Figure 4 shows the number of times the robot’s advice
was accepted by the participants (averages and SD). The
mean score of the fast-motion condition was 8.93 (SD =1.28), that of the slow-motion condition was 7.27 (SD =1.98), that of the short-wait condition was 8.73 (SD =1.39), and that of the long-wait condition was 7.86 (SD =1.64). Figure 5 shows participants’ ratings of estimatedconﬁdence for each condition (averages and SD). The
mean score of the fast-motion condition was 5.21 (SD =.94), that of the slow-motion condition was 3.95 (SD =.75), that of the short-wait condition was 4.83 (SD =1.12), and that of the long-wait condition was 3.69 (SD =1.00).

We performed ANOVAs to ﬁnd the diﬀerence between theconditions in terms of the number of times a participant
accepted the robot’s advice and in terms of the estimatedconﬁdence in the robot’s advice. Signiﬁcant diﬀerenceswere found in the number of acceptances [F (3,56) =3.54, p < .05] with the eﬀect size η 2 = .16 and theestimated conﬁdence [F (3,56) = 8.33, p < .01] with η 2 =.31. Furthermore, multiple comparisons made by using
Tukey’s method found a signiﬁcant diﬀerence between the
fast-motion and slow-motion conditions (p < .05) interms of the number of acceptances, between the
fast-motion and slow-motion conditions (p <.01), between
the fast-motion and long-wait conditions (p <.01), andbetween the short-wait and long-wait conditions (p <.05)
in the estimated conﬁdence.

Discussion
In this paper, we introduced the behavior of a robot
slowly turning around to face a human as motion-ASE. As
seen from the number of times participants accepted therobot’s advice (Figure 4) and from the estimatedconﬁdence (Figure 5), signiﬁcant diﬀerences between the
fast-motion and slow-motion conditions were found inboth of the two evaluations. Thus, by using the
fast-motion and slow-motion that the mobile robot used
to express its conﬁdence to a human, the participants
could correctly interpret high/low-conﬁdence fromfast/slow-motion expressions and could act based on thisinterpretation. As a result, we experimentally conﬁrmed
the eﬀectiveness of motion-ASE.
Meanwhile, as seen from the results of the long/short-wait
conditions, although signiﬁcant diﬀerence was not foundin the number of times a participant accepted the robot’s
advice (Figure 4), it was found in the estimatedconﬁdence (Figure 5). In contrast to fast/slow-motion

1027

Work-in-Progress: Emotions

CHI 2013: Changing Perspectives, Paris, France

expressions having a diﬀerence both in the behavioral
evaluation and the subjective evaluation, long/short-wait
expressions had it only in the subjective evaluation. Thus,
we conclude that, although the long/short-wait
expressions might be applicable to expressing a robot’s
conﬁdence, fast/slow-motion as a motion-ASE is moresuitable for such expressions.
As described in the ASE Design Policy, while ASE
requirements (A1) and (A2) were satisﬁed in the designphases of motion-ASE, (A3) and (A4) needed to beevaluated after the experiments. In the experiments,
although participants had no knowledge of expressions
and the between-participant design made the evaluation
absolute, they could interpret the motion-ASE . Thus, we
believe that ASE requirement (A3) was satisﬁed. Also,
since their interpretation was found to be signiﬁcantly
correct from the experimental results, requirement (A4)was satisﬁed too.

Conclusion
In HAI and HRI, a robot’s conﬁdence in its advice is a
useful internal state that should be correctly expressed toa human, and it is an important goal to develop apractical methodology to express it. For this goal, we
proposed motion-ASE as a novel implementation of ASE
in which a robot hesitates to slowly turn-around to face ahuman before giving advice with low conﬁdence. We
conducted experiments to evaluate the eﬀectiveness withparticipants, and obtained results that support theeﬀectiveness.

References[2] E. S. Reed, et al. On the nature and signiﬁcance of
microslips in everyday activities. EcologicalPsychology Studies, 4(1):51–66, 2009.
[3] H. Hüttenrauch, et al. Evaluation of robot body
movements supporting communication. In Proc. of
2nd International Symposium on New Frontiers in
Human-Robot Interaction, pages 42–49, 2010.
[4] C. T. Ishi, C. Liu, H. Ishiguro, and N. Hagita. Headmotions during dialogue speech and nod timing
control in humanoid robots. In HRI’10, pages
293–300, 2010.
[5] K. Funakoshi, et al. Non-humanlike spoken dialogue:
a design perspective. In SIGDIAL’10, pages 176–184,
2010.
[6] A. Kendon. Do gestures communicate? a review.
Research on Language & Social Interaction,27(3):175–200, 1994.
[7] M. Scheutz, P. Schermerhorn, and J. Kramer. The
utility of aﬀect expression in natural language
interactions in joint human-robot tasks. In HRI’06,
pages 226–233, 2006.
[8] T. Komatsu, et al. Artiﬁcial subtle expressions:
Intuitive notiﬁcation methodology of artifacts
atlanta. In CHI’10, pages 1941–1944, 2010.
[9] K. Terada, A. Yamauchi, and A. Ito. Artiﬁcial
emotion expression for a robot by dynamic color
change. In RO-MAN’12, pages 314–321, 2012.
[10] S. Thrun. Probabilistic robotics. Communications of
the ACM, 45(3):52–57, 2002.
[11] J. E. Young, M. Xin, and E. Sharlin. Robot
expressionism through cartooning. In HRI’07, pages
309–316, 2007.

[1] C. Bartneck and J. Reichenbach. Subtle emotional
expressions of synthetic characters. Int. Journal of
Human-Computer Studies, 62(2):179–192, 2005.

1028