Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

Bioluminescence-Inspired Human-Robot Interaction
Designing Expressive Lights that Affect Human’s Willingness to Interact with a Robot
Sichao Song

Seiji Yamada

The Graduate University for Advanced Studies(SOKENDAI), Department of Informatics
2-1-2 Hitotsubashi, Chiyoda, Tokyo, Japansichaos@nii.ac.jp

National Institute of Informatics and SOKENDAI
2-1-2 Hitotsubashi, Chiyoda, Tokyo, Japanseiji@nii.ac.jp

ABSTRACT
Bioluminescence is the production and emission of light by a living organism. It, as a means of communication, is of importancefor the survival of various creatures. Inspired by bioluminescentlight behaviors, we explore the design of expressive lights and evaluate the effect of such expressions on a human’s perception ofand attitude toward an appearance-constrained robot. Such robots
are in urgent need of finding effective ways to present themselvesand communicate their intentions due to a lack of social expressivity. We particularly focus on the expression of attractivenessand hostility because a robot would need to be able to attract or
keep away human users in practical human-robot interaction (HRI)
scenarios. In this work, we installed an LED lighting system on a
Roomba robot and conducted a series of two experiments. We firstworked through a structured approach to determine the best light
expression designs for the robot to show attractiveness and hostility. This resulted in four recommended light expressions. Further,
we performed a verification study to examine the effectiveness ofsuch light expressions in a typical HRI context. On the basis of thefindings, we offer design guidelines for expressive lights that HRI
researchers and practitioners could readily employ.

Figure 1: Examples of bioluminescent organisms
the ocean as about 80% of the genera known to contain luminous
species are marine, including, for instance, luminescent fish, e.g.,
mycophids and hatchetfish, and crustaceans, e.g., copepods andkrill) [5]. In addition, bioluminescence occurs in some fungi andterrestrial invertebrates, such as fireflies, as well (Figure 1). A large
number of organisms retain functional eyes to detect bioluminescence in dark environments, which suggests the importance of
bioluminescence as a means of communication for the survival ofa vast variety of creatures [5].
Appearance-constrained robots, in comparison, reside in a similar situation as bioluminescent organisms. These robots are designed to be functional and lack expressive faces [30]. Although
they primarily work in bright places, their lack of social expressivity
makes it hard for them to be perceived and understood by humans.
As a result, people are in a way “blind” to appearance-constrainedrobots. In human-robot interaction (HRI) scenarios, this can lead to
unsmooth or even failed interaction [28]. As if they were bioluminescent organisms in the dark, appearance-constrained robots arein urgent need of finding effective ways to present themselves andcommunicate their intentions.
Because of the restricted interaction modality of appearanceconstrained robots, current approaches rely mainly on motion cues
[11, 12, 15, 34, 42]. Unfortunately, these approaches are limited in
expressivity and are hard to apply in many practical scenarios. For
instance, it can be impossible for a robot to use big movements, e.g.,
acceleration and moving in an arc, to interact with human users
when situated in a crowded room. To address such limitations, weinvestigate expressive lights as an alternative interaction modality.
By using expressive lights, we are enabling a robot to modify its
appearance as a means of communicating with humans.
Expressive lights as a dynamic visual cue have been exploredfor HRI applications. For instance, Sony’s robot dog AIBO1 andAldebaran’s NAO2 use LED lights to assist in affective expression.

KEYWORDS
Bioluminescence; expressive lights; attractiveness; hostility; color
psychology; appearance-constrained robot; human-robot interaction (HRI)ACM Reference Format:
Sichao Song and Seiji Yamada. 2018. Bioluminescence-Inspired HumanRobot Interaction. In Proceedings of 2018 ACM/IEEE International Conferenceon Human-Robot Interaction, Chicago, IL, USA, March 5–8, 2018 (HRI ’18),
9 pages.
https://doi.org/10.1145/3171221.3171249

1

INTRODUCTION

Bioluminescence is the production and emission of light by a livingorganism [4]. The majority of bioluminescent organisms reside in
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
HRI ’18, March 5–8, 2018, Chicago, IL, USA© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-4953-6/18/03. . . $15.00
https://doi.org/10.1145/3171221.3171249

1 http://www.sony-aibo.com
2 http://www.ald.softbankrobotics.com/en/robots/nao224

Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

In general, expressive lights have been shown to be effective invarious HRI contexts such as indicating internal states [7], communicating intent [43], and expressing emotion [13]. The goals ofusing expressive lights on a social robot could be summarized by
the three I’s: Inform, Influence, and Interact [7]. Specifically, Informis about showing a robot’s internal state, Influence is about changing human behavior to a robot’s advantage, and Interact is about
affective communication and interaction. Most current research has
mainly been relevant to Inform, and the other two components, Influence and Interact, have seldom been touched upon. It is therefore
important to encourage further exploration on the use of expressive
lights.
Bioluminescence provides good inspiration. Basically, luminescence can serve three purposes within a single organism: offense,
e.g., attracting prey, defense, e.g., warning a predator, and mate
attraction [19]. With regard to HRI, we think that a robot may needto possess two general social abilities: initialize or escape from aninteraction. The importance of a robot initializing an interaction isevident in the vast applications of social robots. For instance, a service/guide robot needs to proactively approach customers/visitors
to promote successful interactions as previous research suggeststhat potential interaction opportunities, e.g., with people that hesitate about whether or not they will engage a robot, may be missed
if the robot is inactive [24, 41]. In addition, a robot needs to escapefrom a potentially harmful interaction as well. As reported by a few
pieces of literature, robots may physically get abused by humans[8, 10, 40]. In such cases, we compare a potential target human toprey when a robot approaches him or her proactively. Similarly, we
compare a potentially dangerous human to a predator when he or
she approaches the robot. Inspired by the functions of bioluminescence, we accordingly consider that such a robot needs to either
attract its target humans (offense) or warn the dangerous ones tokeep their distance (defense).
In this work, we primarily explore the design of expressive lights
to achieve such goals. Specifically, we aim at designing expressive
lights that are either perceived as attractive or hostile. We presume that a robot that is perceived as attractive may do a better job
at initializing an interaction, whereas a robot that is perceived as
hostile may reduce a human’s willingness to interact with it. On
the basis of inspiration from bioluminescence and color psychology theories, we work through a structured process to determine
a set of effective light expressions. A follow-up video-based validation study is then performed to verify the effectiveness of thelight expressions in practical HRI scenarios. Our work can serve as
an extension of [7, 43] and suggests potential effects of expressive
lights on influencing people’s perception and behavior, therefore
increasing exploration on the use of expressive lights in HRI.

restricted in appearance, while there is a need for them to be capableof affective interaction [31].
A number of studies have been carried out to explore the designof non-facial/non-verbal affective expressions [28–31]. The authors
claimed that appearance-constrained robots are not engineered to
be anthropomorphic due to either there being limited applications
or for cost-saving reasons. They highlighted the importance forsuch robots to express emotions. For instance, rescue workers were
found to expect a small tank-like robot to follow social conventions[14]; man-packable robots were observed to be perceived as “creepy”
and not reassuring when they were operated close to simulated
victims [38].
Although these pieces of work provide insights into affective
social interaction between humans and appearance-constrainedrobots, there are several limitations with regard to the generality of
their findings. As they focused on application scenarios involving
assessing victims in the aftermath of a disaster, their results weremajorly based on simulated human victims interacting with two
types of search and rescue robots [28]. Therefore, their methods
can be hard to generalize to other types of robots, e.g., Roomba, andpractical scenarios. In addition, since they used only blue light as an
auxiliary expression to elicit a calming response, they thus offeredopen research questions such as “Can illuminated colored lighting
effects be used effectively to convey affect and for naturalistic social
human robot interactions?” [28]. These questions are ones that wetry to solve.

2.2

Initialize or Escape from Interaction

We think that a social robot needs to have two basic social abilities:
to initialize or escape from a potential interaction. How a robot
should approach humans and try to successfully establish an interaction has been an active topic. Scenarios involving public service,e.g., an information center in a shopping mall, have been particularly researched [24, 41]. In many cases, if a robot only passively
waits for a human user to initiate an interaction with it, people whoare hesitating or unsure of how to interact would be not served [24].
Thus, it is of importance to allow a robot to proactively approachtarget humans. To achieve such a goal, the robot needs methods toattract target people so as to increase the possibility of successfully
establishing interactions with them.
In addition, a handful of studies have investigated robot abuse[8, 10, 40]. It is reported that people, children in particular, tend toreact to robots with high curiosity and often treat them aggressively[10, 40]. Abuse behaviors include saying bad things to the robots
and sometimes even kicking or punching them [10]. Therefore,
robots in such situations need to be able to escape from humanabuse.

2.3

2 BACKGROUND
2.1 Appearance-Constrained Robot

Bioluminescence Functions

The many functions of bioluminescence reflect the unique natureof the environment in which a vast variety of bioluminescent organisms have evolved [5]. Basically, luminescence can serve three
purposes within a single organism: offense, e.g., attract prey, defense, e.g., warning predators, and mate attraction [19]. For example, bioluminescent creatures, e.g., dinoflagellates and squid, uselight to startle a predator in order to defend themselves from being

Affective interaction has become an active topic in social robotics
and HRI [9]. However, major studies on it have been focused onhuman- and animal-like robots [9, 16]. There is a lack of methods
that appearance-constrained robots can use to express affect. Suchmethods are in eager need, as many currently-in-use robots are225Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

ATTRACT

KEEP AWAY

Research on expressive lights for HRI is still in its infancy. It
is suggested that the goals of using expressive lights on social
robots can be summarized by the three I’s: Inform, Influence, andInteract [7]. Despite the many promising studies that have been
done in this area, they mainly touched on the first component only.
Therefore, both theoretical and empirical work regarding the designof expressive lights are needed to provide building blocks for moresophisticated and interaction-oriented HRI.

3 EXPRESSIVE LIGHT DESIGN
3.1 Exploring Design Space
3.1.1 Color. Although the RGB color space contains tens of
thousands of colors, we only consider categorical colors, e.g., green
and red, due to their simplicity and representativeness. In addition,
this is valid as the human color vision system processes color signals
in a categorically driven manner [21]. Color psychologists haveintensively investigated various aspects of color, including color
vision, color symbolism and association, and the effects of coloron psychological and biological functioning [21]. Basically, theirwork primarily focuses on red, blue, and green since such colors(especially red) have been considered to be special and have positive
links to the natural realm.
On one hand, associative learning theory provides a promising
explanation of color-emotion associations [22]. According to thistheory, color meanings are grounded in two basic sources: learned
associations that develop from repeated pairings of colors withparticular messages, concepts, or experiences and biologically based
proclivities to respond to particular colors in particular ways inparticular situations [22]. For example, red carries the meaningof danger and anger in life-threatening situations, such as whenviewing blood, an angry face, traffic lights, and/or warning signals
and sirens [23]. Similarly, green can be associated with positive
meanings due to traffic lights (green light indicates “go”) and an
image of being the color of nature. Blue can be associated withsadness due to the saying “I feel blue” [23].
On the other hand, mental alertness theory suggests that color
can affect a person’s level of mental alertness and therefore influence his or her psychological functioning [20]. Previous work
indicates that red makes people more alert and risk averse, whereas
blue encourages people to take risks and perform exploratory behaviors [17]. In addition, blue light is able to elicit pleasure [35]and creativity [39]. These findings indicate that the color blue canbe attractive to humans.
In addition, previous studies observed that exposure to green
and blue evokes lower feelings of anxiety and greater feelings ofcalmness [6]. It has long been proposed that being exposed to colors
with longer wavelengths such as red and yellow is stimulating andarousing, whereas shorter wavelength colors, such as green andblue, tend to evoke feelings of calmness and tranquility [6].

Figure 2: Two general functions of bioluminescence: attraction and keeping away (adapted from [19])preyed upon. Some predators, e.g., anglerfish, use light to lure theirprey. Many creatures, e.g., ostracodes and flashlight fish, rely on
bioluminescent light to attract and recognize their mates [19]. In
general, bioluminescence functions can be summarized into two
kinds: those that attract or keep others away (Figure 2).
With regard to the two HRI scenarios we focus on in this work,that is, initialize and escape from an interaction, we think that a
robot, similarly, should be able to attract or keep humans away. To
be specific, a robot needs to attract human users to successfully
establish an interaction, whereas it needs to keep unfriendly people
away to escape from abuse.

2.4

Expressive Lights for Robots

Expressive lights, as an explicit way of communication, have been
discussed in studies across various fields such as psychology [17, 35],
human-computer interaction (HCI) [18, 25, 36, 37], and humanrobot interaction [7, 26, 43]. To be specific, it is suggested that evensimple light expressions can be highly expressive [18] and are able
to evoke high-level social and emotional content [25, 37]. Such
artificial lights in different colors can implicitly affect human perception and psychological functioning and therefore may influence
human behavior [17, 35].
With regard to HRI scenarios, a majority of work focuses onhuman-oriented applications because one fundamental goal of social robots is to serve people. Expressive lights have been considered
as an effective approach for non-verbal communication, and suchan approach is considered to be particularly useful for appearanceconstrained robots, as such robots generally have very low social
expressivity [30]. Several studies have investigated potential functional uses of lights for robots. For instance, expressive light animations were applied to visualize a mobile service robot’s internal
state [7]. The authors designed different light patterns to indicatethat the robot is waiting for human input, is being blocked by ahuman, or is showing task progress. Another work [43] exploreddesign constrains to robot flight behaviors. Their designed light
expressions were able to significantly improve people’s responsetime and accuracy for predicting the flying direction of the robot.

3.1.2 Patterns. Due to the shape of the LED strip, the design
space of a light expression pattern is restricted to one dimension.
However, because each LED pixel can be individually controlled,
finding desirable designs is still challenging. To cover a wide rangeof possible light expressions, we primarily investigated two typesof patterns: strip-based patterns and pixel-based ones. The former

226

Session We-2A: Designing Robots and Interactions

static/blinking/breathing

HRI’18, March 5-8, 2018, Chicago, IL, USA

circlesplit

randomFigure 3: Demonstrations of candidate expressive light patterns. All, except static, were periodic. Each periodic expressive
light pattern consisted of five periods. For random pattern, singular random pixels in the LED strip were turned on and off
throughout the given time period
includes light expressions that make use of all LED pixels as awhole (the entire LED strip), whereas the latter consists of light
expressions that take advantage of individual LED pixel expressions.
During her speech at TED20113 , Edith Widder demonstrated
plenty of bioluminescent light behaviors performed by sea creatures. Inspired by such natural bioluminescent lights, we designed
a set of representative light patterns. To be specific, a pattern canhave two parameters: waveform and intensity (frequency). Existingfindings suggest that a rectangular waveform and high intensity
represent intense emotions, while a sinusoidal waveform and low
intensity represent weak emotions [44]. Therefore, with regard to
strip-based patterns, we explored both rectangular (named blinking) and sinusoidal (named breathing) waveforms with low andhigh frequencies, respectively. In addition, a static pattern, in which
the entire LED strip is always lit, was also chosen. With regard to
pixel-based patterns, the waveform parameters for individual LED
pixels were not considered. We chose three patterns: circle, split,
and random. It is notable that, for the random pattern, singularrandom pixels in the LED strip were turned on and off throughout
the given time period.

3.2

random) with two intensities (low, high), and one special pattern(static). All light expressions, except static, were periodic, and eachcorresponding expressive light contained five periods. We designed
a large number of expressive light patterns in the hope of covering
a wide range of potential light expressions.

3.3

Roomba Lighting System

We installed an LED lighting system on an iRobot Create 2 robot.
Roomba is a series of indoor autonomous robotic vacuum cleaners.

Candidate Expressive Lights

We decided on 44 light expressions in total as our expressive lightcandidates (Figure 3). In summary, they made use of four colors (red,green, blue, white), five patterns (blinking, breathing, circle, split,
Figure 4: Configuration of Roomba robot with LED lighting
system

3 See https://www.ted.com/talks/edith_widder_the_weird_and_wonderful_world_
of_bioluminescence#t-112245.

227

Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

All Roomba robots are disc-shaped, 34 cm in diameter, and less than
9 cm in height [3]. iRobot Create 2 is a programmable Roomba robot
for educators, students, and developers [2]. Therefore, it allows fora variety of programming methods and can be connected to a microcontroller. We think that such a robot perfectly meets the definitionof an appearance-constrained robot, and in addition, has limited
methods of expressing affect, e.g., moving forward/backward andspinning.
Figure 4 illustrates the configuration of the Roomba robot with
the LED strip. Following the design of [43], we attached a NeoPixel
LED strip (1 meter, 60 pixels) to the body of the robot in a ring.
The strip was controlled by an Arduino Uno R3 board, where thedata pin of the strip was connected to the digital output pin of theArduino board. Both the strip and board were powered by a 5-V,
3-A portable powerbank. In addition, the same board was used tocontrol the movements of the Roomba robot as well. iRobot Create
2 provides the Roomba Open Interface (OI) [1] which is a softwareinterface for controlling and manipulating Roomba’s behavior.

evaluate the participants’ perception of the robot: “This robot looks
attractive” and “This robot looks hostile.” A five-point Likert scale
was used for the statements (ranging from 1, strongly disagree, up
to 5, strongly agree). Each participant in the study viewed all 44
videos and rated them one at a time (within-participant design),
and the order of the videos was randomized.

4.2

4.3

4 EXPERIMENT
4.1 ProcedureMean

0.7 circle_blue_low
0.69 split_green_low
0.73 split_green_high0.74 split_blue_high
0.71 split_white_low
0.75 circle_white_low
0.78 random_blue_high
0.78 split_red_low
0.66 circle_blue_high
0.75 random_blue_low
0.8
0.76
0.74
0.73
0.74
0.78
0.82
0.78
0.79
0.79
0.71
0.7
0.82
0.76
0.83
0.75
0.82
0.75
0.75
0.79
0.77
0.75
0.72
0.78
0.72
0.72
0.74
0.68
0.8
0.69
0.77
0.72
0.69

split_blue_low
circle_blue_low
split_green_low
split_green_highsplit_blue_highsplit_white_low
circle_white_low
random_blue_highsplit_red_low
circle_blue_highrandom_blue_low
circle_white_highbreathing_green_low
circle_green_highcircle_green_low
split_white_highrandom_white_low
breathing_green_highbreathing_blue_low
blinking_white_low
static_white
blinking_blue_low
breathing_blue_highrandom_green_highcircle_red_low
circle_red_highblinking_green_low
random_red_low
random_white_highsplit_red_highbreathing_white_low
breathing_red_low
random_red_highbreathing_white_highstatic_blue
random_green_low
static_green
blinking_green_highblinking_blue_highbreathing_red_highblinking_red_low
blinking_white_highblinking_red_highstatic_red
0

1

Mean

Table 1
Code

Pattern0.68 split_blue_low

Results

We summarized the ratings from all 73 participants. Figures 5 and 6
show the mean Likert ratings for attractive and hostile, respectively.
Because we used a single Likert term rather than constructed scales
to assess human perception, statistic measures, such as ANOVA,
were not appropriate. Instead, we characterized the main trends
observed in the data and reported on them with regard to eachperception.
Attractive: We recommend split_blue_low (a low-intensity blue
split pattern) and circle_blue_low (a low-intensity blue circle pattern) for showing attractiveness. These are iconic and the top twohighest-rated light expressions (see Fig. 5). The commonality between the two light expressions is straightforward in that they bothfeature the color blue and have a low intensity. In general, it can beobserved that expressive lights with one or more of the following
three features received high ratings from the participants: 1) blue,2) low intensity, 3) pixel-based patterns (particularly split and circleones). On the other hand, expressive lights with one or more of theWe aimed at finding a set of appropriate expressive lights that
can allow the Roomba robot to be perceived as either attractive or
hostile by humans. To ensure the generality of the experimental
results, having a large and diverse set of participants was important.
Therefore, we employed a Japanese online crowdsourcing platform
Fastask to recruit participants for the experiment. Recent studies,e.g., [18, 27], have shown the validity and power of crowd-sourced
approaches. It allowed us to rapidly and inexpensively gather datafrom many more participants than would have been practical withother approaches. Data integrity was guaranteed by dropping participants whose answers had near-zero variances, e.g., all 3’s. As aresult, a total of 27 of them were discarded, leaving data from 73 participants (23 females, Maдe =48.04, SDaдe =14.12). All participants
were native Japanese speakers.
We used video recordings to demonstrate all the candidate expressive lights. For each video, we provided two statements to

Std

Criteria

We analyzed the candidate expressive lights with regard to the twoperceptions, attractive and hostile, separately. For the evaluation,
we introduced two criteria (adapted from [18]) to select good light
expressions: 1) an expression must have a strong interpretation
with regard to a perception (mean Likert rating in the top quartile)and 2) an expression must be iconic, meaning that it has only one
dominant perception among the two (an expression should not beperceived as both attractive and hostile). We assessed the iconicness of each candidate expression that meets criteria 1.

3.09

2.95

Mean

Std

Patternblinking_red_high

e31

4.02

0.65 blinking_red_highbreathing_red_high

e44

3.98

0.68 breathing_red_highblinking_red_low

e38

3.68

0.66 blinking_red_low

blinking_white_high

e24

3.65

0.7 blinking_white_highbreathing_white_high

e27

3.54

0.66 breathing_white_highcircle_red_high

e4

3.47

0.73 circle_red_highblinking_blue_high

e26

3.33

0.68 blinking_blue_highrandom_white_high

e36

3.22

0.7 random_white_highblinking_green_high

e3

3.18

0.73 blinking_green_highrandom_red_high

e42

3.13

0.73 random_red_highrandom_white_low

e34

3.09

0.7 random_white_low

random_red_low

e15

3.07

0.72 random_red_low

breathing_red_low

e39

3.01

0.75

static_red

e8

2.96

0.7

blinking_white_low

e21

2.95

0.72

split_red_high

e5

2.92

0.75

breathing_blue_high

e28

2.84

0.69

random_green_low

e2

2.83

0.71

breathing_green_high

e7

2.81

0.75

random_green_high

e17

2.81

0.73

blinking_blue_low

e22

2.77

0.76

random_blue_high

e9

2.76

0.73

random_blue_low

e6

2.75

0.73

blinking_green_low

e43

2.73

0.78

breathing_blue_low

e16

2.69

0.73

split_red_low

e37

2.68

0.71

breathing_green_low

e20

2.66

0.76

breathing_white_low

e18

2.65

0.75

split_green_high

e23

2.62

0.75

circle_blue_high

e40

2.61

0.72

circle_red_low

e25

2.57

0.74

circle_green_high

e41

2.56

0.72

static_green

e33

2.52

0.69

static_white

e10

2.49

0.77

split_white_high

e19

2.43

0.76

circle_green_low

e1

2.39

0.74

split_white_low

e32

2.32

0.7

static_blue

e29

2.28

0.71

circle_white_high
2

e11

2.21

0.68

split_blue_high

e12

2.16

0.72

split_blue_low

e35

2.11

0.7

split_green_low

e14

2.08

0.69

3

blinking_red_highbreathing_red_highblinking_red_low
blinking_white_highbreathing_white_highcircle_red_highblinking_blue_highrandom_white_highblinking_green_highrandom_red_highrandom_white_low
random_red_low
breathing_red_low
static_red
blinking_white_low
split_red_highbreathing_blue_highrandom_green_low
breathing_green_highrandom_green_highblinking_blue_low
random_blue_highrandom_blue_low
blinking_green_low
breathing_blue_low
split_red_low
breathing_green_low
breathing_white_low
split_green_highcircle_blue_highcircle_red_low
circle_green_highstatic_green
static_whitesplit_white_highcircle_green_low
split_white_low
static_blue
circle_white_highsplit_blue_highsplit_blue_low
split_green_low
circle_blue_low
circle_white_low
0

4

e30

1.96

0.71

circle_white_low

e13

1.87

0.68

2

3

Figure 6: Mean Likert ratings for hostile perception.

Figure 5: Mean Likert ratings for attractive perception.
circle_blue_low

1

228

4

Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

5.1

following three features received low ratings for attractiveness: 1)
red, 2) high intensity, 3) strip-based patterns.
Hostile: We recommend blinking_red_high (a high-intensity
red blinking pattern) and breathing_red_high (a high-intensity red
breathing pattern) for showing hostility. These are iconic and the
top two highest-rated light expressions (see Fig. 6). The commonality between the two light expressions is clear in that they bothfeature the color red and have a high intensity. In general, several
trends can be observed that show that expressive lights with one
or more of the following features received high ratings: 1) red, 2)high intensity, 3) strip-based patterns (particularly blinking andbreathing). Oppositely, expressive lights with one or more of the following features received low ratings for hostility: 1) low intensity,2) pixel-based patterns (particularly split and circle ones).

4.4

Discussion

Four expressive lights were selected as effective expressions for
the Roomba robot to show either attractiveness or hostility. The
results offer strong evidence, indicating that expressive lights caneffectively affect a human’s perception of a robot. To be specific,
we found that the robot was particularly attractive when showing
blue light at a low intensity. In addition, the split and circle patterns
turned out to be more absorbing compared with the other patterns.
We also found that the robot was perceived as particularly hostilewhen showing red light at a high intensity. Both blinking (rectangular) and breathing (sinusoidal) patterns were selected, suggestingthat waveform was a less important factor compared with color andintensity (frequency). However, it is noticeable that the two patterns are strip-based patterns. This indicates that high luminancemay be of importance for showing hostility as well.
Interestingly, features of these four expressive lights can be observed in a variety of bioluminescent ocean creatures. Edith Widderdemonstrated plenty of bioluminescent light behaviors performedby sea creatures, where many of them showed similar patterns. This
indicates that bioluminescent lights used by natural creatures may
have analogous effects on human perception and psychological
functioning. In addition, the results are in line with color psychology theories. As suggested by color psychology literature [17, 39],
red carries the meaning of danger and hostility and moreover makes
people more alert and risk averse, whereas blue elicits pleasure andencourages people to take risks. Therefore, it can be presumed thatblue light, in many situations, is more attractive than other colors,
while red light conveys a negative effect.

5

Procedure

We designed two HRI scenarios: robot approaches human (RH) andhuman approaches robot (HR). To be specific, in RH, the Roomba
robot moved toward a person (experimenter) proactively, whereas,in HR, the person moved toward the robot. Particularly in HR, therobot started to show expressive lights when the person was within
about 2 m. RH was designed to simulate a scenario in which a robot
tries to initiate an interaction with a person, and HR was to simulatea scenario in which a robot is approached by an unwanted person.
We employed the same Japanese online crowdsourcing platform
to recruit participants for this study. Video recordings were used todemonstrate the two HRI scenarios via an online survey. For eachscenario, we provided one synthetic video involving three conditions: the robot showing attractiveness, the robot showing hostility,
and the robot not showing any expressive light. With regard to theattractiveness condition, we picked split_blue_low as the representative light expression. Similarly, we chose blinking_red_high as
the representative light expression for the hostility condition.
Figure 7 shows screenshots of each video clip. It is notable that although the screenshots show a large difference in distance between
the camera and the robot regarding the two scenarios (RH and HR),
this was due to the different timings of taking these screenshots.
In RH, the robot is close to the person, which shows the effect of
the robot’s approach behavior (the robot has moved a long distance from its original position). Similarly in HR, the person is seen
standing close to the robot, which indicates the person’s approach
behavior (the person has moved a long distance from his original
position). The order of the three clips for each scenario was randomized. Participants whose responses had near-zero variances were
removed from the results evaluation. In total, 16 of them were discarded, leaving data from 205 participants (68 female, Maдe =50.97,
SDaдe =13.03). All the participants were Japanese.
With regard to each synthetic video, we prepared four statements
to evaluate the participants’ perception of and attitude toward

FOLLOW-UP VERIFICATION STUDY

As we worked through a structured process for designing affective
expressive lights, we were able to offer a set of four light expressions that can well show attractiveness and hostility. However, theeffectiveness of such lights had not yet been examined in practical HRI scenarios. Therefore, we further conducted a verification
experiment in which the same Roomba robot with LED lighting
system was employed. In the follow-up study, we evaluated theeffectiveness of the recommended light expressions by observing
whether people would be willing to approach and interact withthe robot when it showed attractiveness and whether they would
consider keeping away from it when it showed hostility.

(a) RH_none(b) RH_split(c) RH_blinking(d) HR_none(e) HR_split(f) HR_blinking

Figure 7: Screenshots of each video clip (condition)

229

Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

5.3

the robot: 1) “This robot looks attractive,” 2) “This robot looks
hostile,” 3) “I like this robot,” 4a) “I want to play with this robot”(RH scenario), and 4b) “I want to keep away from this robot” (HRscenario). For each statement, the participants were asked to choose
the robot presented in one of the three conditions that best fit thestatements. The selection rate (SR) was counted for each condition.
The SR number, ranging from 0 to 205 (total amount of participants),
indicated how many participants chose a robot for a particularstatement. Similar approaches were used in other studies, e.g., [7].

5.2

Results

We applied Pearson’s chi-square test to evaluate the effect of the
independent factor (expressive light) on the five statements as dependent factors. A post-hoc binomial test with Holm’s correction
was further applied if a significant difference was found. Because of
the three conditions, the hypothesized probability that each condition would be chosen at random with regard to a statement was set
to one-third (33.33%, which is the probability of a random guess).
Figure 8 shows the selection rates with regard to the participants’ perception of the robot. For each statement, a significant
difference was found for both the RH scenario (attractive: χ 2 (2) =157.09, p < 0.001; hostile: χ 2 (2) = 203.01, p < 0.001; likeable:χ 2 (2) = 123.11, p < 0.001; want-to-interact: χ 2 (2) = 113.98, p <0.001) and the HR scenario (attractive: χ 2 (2) = 129.02, p < 0.001;
hostile: χ 2 (2) = 211.5, p < 0.001; likeable: χ 2 (2) = 180.71, p <0.001; want-to-interact: χ 2 (2) = 71.22, p < 0.001). The post-hoc
tests suggest that the conditions with the top SR, e.g., split_blue_low
for attractive and blinking_red_high for hostile, were selected as
the ones that most fit the corresponding statements (significantly
above 33.33%, p < 0.001), while the SRs for the other conditions
were all significantly below 33.33% (p < 0.001).

t

none

blinking
136

32

29

164

143

26

140

26

145

30

28

166

159

20

35

125

split

6

blinking

135

I. Blue light is recommended to show attractiveness in a robot;
II. Red light is recommended to show hostility in a robot;
III. Patterns such as split and circle can be attractive to humans;
IV. A low intensity (frequency) can be used to support the expression of attractiveness, while a high one can be used tosupport the expression of hostility;
V. With regard to expressing strong or weak emotions, the typeof waveform, e.g. rectangular or sinusoidal, is less importantcompared with color and intensity.

90
45

attractive

hostilelikeablewant-to-interact

RH scenario

180

135

90

6.1

attractive

hostileLimitations and Future Work

Several limitations of this work should be recognized. First of all,the effects of expressive lights on human behavior (in practical HRI
contexts) remain purely potential as we did not explicitly investigate
dynamic human interaction behavior. In this study, we employeda video-based HRI method because such a crowd-sourcing-based
approach enabled us to access a large and diverse set of participants.
Compared with a live HRI method, previous studies (e.g., [45]) haveshown that a video-based method can provide comparable results in

45

0

GENERAL DISCUSSION

This work explored the design of expressive lights for a Roomba
robot to influence humans’ willingness to interact with it. The goal
was to allow robots to either attract people so that they can moreeasily initialize an interaction with them, or keep people away sothat they can escape from a potentially harmful interaction. Our
work expands on previous studies [7, 43] in terms of the following
three points. First, we took inspiration from bioluminescence andshowed that LED lights that simulate communication cues used byliving creatures may have analogous effects on human perception
and psychological functioning. Second, we proved that findings
from color science and color psychology can be referred to as theoretical groundings by HRI researchers to design effective light
expressions. Third, we tested a large design space that contained 44
candidate light expression combinations from a possibility of fourcolors, six patterns, and two levels of intensity. This allowed us toobserve common trends for the effects of the features of expressive
lights (color, pattern, and intensity) regarding two perceptions: attractiveness and hostility. To summarize, our findings suggest that
there are potential effects of expressive lights on influencing people’s perception and behavior, therefore we intend to delve further
into the exploration of the use of expressive lights in HRI.
On the basis of the results, we offer five design guidelines forthe design of affective expressive lights:

1800Discussion

The results verify the effectiveness of our recommended expressive
lights on people’s perception of a robot. The robot showed thatsplit_blue_low was particularly perceived as attractive and preferred by the participants, while it was perceived as hostile when
showing blinking_red_high. This meets our expectation as expressive light split_blue_low was recommended to show attractivenessand blinking_red_high was recommended to show hostility. Moreover, the participants’ attitudes toward the robot was influenced
as they preferred to play/interact with the robot when it showed
split_blue_low, whereas they considered keeping their distancefrom the robot when it showed blinking_red_high.

likeable want-to-keep-distance

HR scenario

Figure 8: Selection rates for each statement for RH scenario(above) and HR scenario (below)

230

Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA

certain contexts, such as when a robot approaches a person (similar
contexts are used in this work). However, people’s perception andespecially their behavior would possibly differ in contexts wheres/he were a participant of an interaction or were simply an onlooker.
Therefore, future work needs to apply live HRI to reveal direct
evidence for the influence of expressive lights on human behavior.
It should also be noted that the results of the first experiment
seemed to have fairly small effect sizes. With regard to the analyses
of both attractive and hostile perceptions, the majority of ratings
were given within the range of two to four. Therefore, the fact thatthe effect on people was not so large means that this work may
not be able to support strong recommendations for using the four
expressive lights. However, since both the trends observed in thefirst experiment and the results of the validation study support ourfindings, we consider our proposed general design guidelines to bevalid and useful.
Besides, due to the fact that color carries different meanings indifferent contexts [21], the generality of this work may be limited.
Therefore, research on expressive lights needs to be explored withvarious HRI contexts. Moreover, since the effects of color on humans
may depend on culture (in particular cases, for example, red has
positive meanings in Chinese culture) [21], it is also of importanceto test our findings (obtained with Japanese participants) on people
with different cultural backgrounds.
This work can be further explored. We decided upon our 44 candidate expressive light patterns by referring to color psychology
theories and bioluminescent light patterns in nature. Different design approaches (e.g., [32, 33, 42, 44]) can be applied to the early
design stage to guarantee that a set of highly representative candidate light patterns are prepared for the later experiments andanalysis. With regard to evaluation, interaction attributes [32, 33],
as a set of vocabulary for describing interaction experience, may
be useful for investigating people’s subjective experience of perceiving and interacting with the robot. We also suggest applying
constructed scales rather than single terms (attractiveness and hostility) to investigate people’s perception. In addition, future work is
required to investigate more design factors (e.g., a robot’s shape andarrangement of LEDs). Results from such studies would provide
further insights and be more generalized to various types of robots.

7

Effects on Mood and Perceived Exertion. Environmental Science and Technology
46, 16 (2012), 1–6. https://doi.org/Doi10.1021/Es301685g[7] Kim Baraka, Ana Paiva, and Manuela Veloso. 2016. Expressive Lights for Revealing
Mobile Service Robot State. Springer International Publishing, Cham, 107–119.
https://doi.org/10.1007/978-3-319-27146-0_9[8] Christoph Bartneck, Rosalia Chioke, Rutger Menges, and Inez Deckers. 2005.
Robot Abuse–A Limitation of the Media Equation. In Interact 2005 Workshop on
Abuse. Rome.
[9] Cynthia Breazeal. 2002. Designing Sociable Robots. MIT Press, Cambridge, MA,USA.
[10] Drazen Brscić., Hiroyuki Kidokoro, Yoshitaka Suehiro, and Takayuki Kanda. 2015.
Escaping from Children’s Abuse of Social Robots. In Proceedings of the Tenth
Annual ACM/IEEE International Conference on Human-Robot Interaction. ACM,
New York, NY, USA, 59–66. https://doi.org/10.1145/2696454.2696468[11] Jessica Rebecca Cauchard, Kevin Y. Zhai, Marco Spadafora, and James A. Landay.
2016. Emotion Encoding in Human-Drone Interaction. In The Eleventh ACM/IEEEInternational Conference on Human Robot Interaction. IEEE Press, Piscataway, NJ,
USA, 263–270. http://dl.acm.org/citation.cfm?id=2906831.2906878[12] Szafir Daniel, Bilge Mutlu, and Terrence Fong. 2014. Communication of Intent inAssistive Free Flyers. In Proceedings of the 9th ACM/IEEE International Conferenceon Human-robot Interaction. ACM, New York, NY, USA, 358–365. https://doi.org/
10.1145/2559636.2559672[13] Johannes Feldmaier, Tamara Marmat, Johannes Kuhn, and Klaus Diepold. 2016.
Evaluation of a RGB-LED-based Emotion Display for Affective Agents. CoRR
abs/1612.07303 (2016). arXiv:1612.07303 http://arxiv.org/abs/1612.07303[14] Thomas Fincannon, Laura Barnes, Murphy Robin R., and Dawn Riddle. 2004.
Evidence of the need for social intelligence in rescue robots. In Proceedings of the
14th IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 2.
1089–1095. https://doi.org/10.1109/IROS.2004.1389542[15] Levillain Florent, Zibetti Elisabetta, and Lefort Sébastien. 2017. Interactingwith Non-anthropomorphic Robotic Artworks and Interpreting Their Behaviour.
International Journal of Social Robotics 9, 1 (01 Jan 2017), 141–161. https://doi.
org/10.1007/s12369-016-0381-8[16] Terrence Fong, Illah Nourbakhsh, and Kerstin Dautenhahn. 2003. A survey ofsocially interactive robots. Robotics and Autonomous Systems 42, 3 (2003), 143–166.
https://doi.org/10.1016/S0921-8890(02)00372-X Socially Interactive Robots.
[17] Gianluigi Guido, Luigi Piper, M. Irene Prete, Antonio Mileti, and Carla M. Trisolini.
2017. Effects of Blue Lighting in Ambient and Mobile Settings on the Intention
to Buy Hedonic and Utilitarian Products. Psychology & Marketing 34, 2 (2017),
215–226. https://doi.org/10.1002/mar.20984[18] Chris Harrison, John Horstman, Gary Hsieh, and Scott Hudson. 2012. Unlocking
the Expressivity of Point Lights. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems. ACM, New York, NY, USA, 1683–1692.
https://doi.org/10.1145/2207676.2208296[19] Haddock Steven HD, Mark A. Moline, and James F. Case. 2010. Bioluminescencein the Sea. Annual Review of Marine Science 2, 1 (2010), 443–493. https://doi.org/
10.1146/annurev-marine-120308-081028 arXiv:https://doi.org/10.1146/annurevmarine-120308-081028 PMID: 21141672.
[20] Elliot Andrew J. 2015. Color and psychological functioning: A review oftheoretical and empirical work. Frontiers in Psychology 6, APR (2015), 1–8.
https://doi.org/10.3389/fpsyg.2015.00368[21] Elliot Andrew J., Fairchild Mark D., and Franklin Anna. 2015. Handbook of Color
Psychology. Cambridge University Press.
[22] Elliot Andrew J. and Markus A. Maier. 2007. Color and psychological functioning.
Current Directions in Psychological Science 16, 5 (2007), 250–254. https://doi.org/
10.1111/j.1467-8721.2007.00514.x[23] Elliot Andrew J. and Markus A. Maier. 2014. Color psychology: effects of perceiving color on psychological functioning in humans. Annual review of psychology
65 (2014), 95–120. https://doi.org/10.1146/annurev-psych-010213-115035[24] Yusuke Kato, Takayuki Kanda, and Hiroshi Ishiguro. 2015. May I Help You?:
Design of Human-like Polite Approaching Behavior. In Proceedings of the Tenth
Annual ACM/IEEE International Conference on Human-Robot Interaction. ACM,
New York, NY, USA, 35–42. https://doi.org/10.1145/2696454.2696463[25] Joseph ’Jofish’ Kaye. 2006. I Just Clicked to Say I Love You: Rich Evaluationsof Minimal Communication. In CHI ’06 Extended Abstracts on Human Factors in
Computing Systems. ACM, New York, NY, USA, 363–368. https://doi.org/10.1145/
1125451.1125530[26] Kazuki Kobayashi, Kotaro Funakoshi, Seiji Yamada, Mikio Nakano, Takanori
Komatsu, and Yasunori Saito. 2011. Blinking light patterns as artificial subtle
expressions in human-robot speech interaction. In Proceedings of the 20th IEEE
International Symposium on Robot and Human Interaction Communication. IEEE,
Atlanta, GA, USA, 181–186.
[27] Steven Komarov, Katharina Reinecke, and Krzysztof Z. Gajos. 2013. Crowdsourcing performance evaluations of user interfaces. In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). 207–216.
https://doi.org/10.1145/2470654.2470684[28] Bethel Cindy L. 2009. Robots without faces: non-verbal social human-robot interaction. [dissertation/Ph.D.’s thesis]. University of South Florida.

ACKNOWLEDGMENTS

This study was partially supported by JSPS KAKENHI “Cognitive
Interaction Design” (No. 26118005).

REFERENCES[1] 2015 (Accessed September 5, 2017). iRobot Create 2 Open Interface (OI): Specification based on the iRobot Roomba 600. https://cdn-shop.adafruit.com/datasheets/
create_2_Open_Interface_Spec.pdf[2] 2017 (Accessed September 5, 2017). iRobot Create 2 Programmable Robot. http://www.irobot.com/About-iRobot/STEM/Create-2.aspx[3] 2017 (Accessed September 5, 2017). Roomba. https://en.wikipedia.org/wiki/
Roomba[4] 2017 (Accessed September 5, 2017). What is bioluminescence? https://oceanservice.
noaa.gov/facts/biolum.html[5] Widder Edith. A. 2010.
Bioluminescence in the Ocean: Origins of Biological, Chemical, and Ecological Diversity.
Science
328, 5979 (2010), 704–708.
https://doi.org/10.1126/science.1174269
arXiv:http://science.sciencemag.org/content/328/5979/704.full.pdf[6] Adam Akers, Jo Barton, Rachel Cossey, Patrick Gainsford, Murray Griffin, andDominic Micklewright. 2012. Visual Colour Perception in Green Exercise: Positive231

Session We-2A: Designing Robots and Interactions

HRI’18, March 5-8, 2018, Chicago, IL, USA[38] Murphy Robin R., Dawn Riddle, and Eric Rasmussen. 2004. Robot-assisted medical
reachback: a survey of how medical personnel expect to interact with rescue
robots. In Proceedings of the 13th IEEE International Workshop on Robot andHuman Interactive Communication. 301–306. https://doi.org/10.1109/ROMAN.
2004.1374777[39] Mehta Ravi and Rui Juliet Zhu. 2009.
Blue or Red? Exploring
the Effect of Color on Cognitive Task Performances.
Science 323,
5918
(2009),
1226–1229.
https://doi.org/10.1126/science.1169144
arXiv:http://science.sciencemag.org/content/323/5918/1226.full.pdf[40] Pericle Salvini, Gaetano Ciaravella, Wantai Yu., Giuseppe Ferri, Alessandro Manzi,
Barbara Mazzolai, Cecilia Laschi, Sang-Rok Oh, and Paolo Dario. 2010. How safe
are service robots in urban environments? Bullying a robot. 19th International
Symposium in Robot and Human Interactive Communication (2010), 1–7. https://doi.org/10.1109/ROMAN.2010.5654677[41] Satoru Satake., Takayuki Kanda, Dylan F. Glas, Michita Imai, Hiroshi Ishiguro,
and Norihiro Hagita. 2009. How to approach humans?-strategies for social robots
to initiate interaction. In 2009 4th ACM/IEEE International Conference on HumanRobot Interaction (HRI). 109–116. https://doi.org/10.1145/1514095.1514117[42] David Sirkin, Brian Mok, Stephen Yang, and Wendy Ju. 2015. Mechanical Ottoman:
How Robotic Furniture Offers and Withdraws Support. In Proceedings of the Tenth
Annual ACM/IEEE International Conference on Human-Robot Interaction. ACM,
New York, NY, USA, 11–18. https://doi.org/10.1145/2696454.2696461[43] Daniel Szafir, Bilge Mutlu, and Terry Fong. 2015. Communicating Directionality
in Flying Robots. In Proceedings of the Tenth Annual ACM/IEEE International
Conference on Human-Robot Interaction. ACM, New York, NY, USA, 19–26. https://doi.org/10.1145/2696454.2696475[44] Kazunori Terada, Atsushi Yamauchi, and Akira Ito. 2012. Artificial emotion
expression for a robot by dynamic color change. In Proceedings of the 21st IEEE
International Symposium on Robot and Human Interactive Communication. 314–
321. https://doi.org/10.1109/ROMAN.2012.6343772[45] S. N. Woods, M. L. Walters, K. L. Koay, and K. Dautenhahn. 2006. Methodological
Issues in HRI: A Comparison of Live and Video-Based Methods in Robot to
Human Approach Direction Trials. In Proceedings of the 15th IEEE International
Symposium on Robot and Human Interactive Communication. 51–58. https://doi.
org/10.1109/ROMAN.2006.314394[29] Bethel Cindy L. and Robin R. Murphy. 2006. Affective Expression in Appearance
Constrained Robots. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on
Human-robot Interaction. ACM, New York, NY, USA, 327–328. https://doi.org/10.
1145/1121241.1121299[30] Bethel Cindy L. and Robin R. Murphy. 2007. Non-facial/Non-verbal Methods ofAffective Expression As Applied to Robot-assisted Victim Assessment. In Proceedings of the 2nd ACM/IEEE International Conference on Human-robot Interaction.
ACM, New York, NY, USA, 287–294. https://doi.org/10.1145/1228716.1228755[31] Bethel Cindy L. and Robin R. Murphy. 2008. Survey of Non-facial/Non-verbal
Affective Expressions for Appearance-Constrained Robots. IEEE Transactions on
Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38, 1 (Jan 2008),
83–92. https://doi.org/10.1109/TSMCC.2007.905845[32] Eva Lenz, Sarah Diefenbach, and Marc Hassenzahl. 2013. Exploring Relationships Between Interaction Attributes and Experience. In Proceedings of the 6thInternational Conference on Designing Pleasurable Products and Interfaces. ACM,
New York, NY, USA, 126–135. https://doi.org/10.1145/2513506.2513520[33] Youn-kyung Lim, Sang-Su Lee, and Da-jung Kim. 2011. Interactivity attributes
for expression-oriented interaction design. International Journal of Design 5, 3
(2011).
[34] Saerbeck Martin and Christoph Bartneck. 2010. Perception of affect elicited byrobot motion. In Proceedings of the 5th ACM/IEEE International Conference on
Human-Robot Interaction. 53–60. https://doi.org/10.1109/HRI.2010.5453269[35] Beaven C. Martyn and Johan Ekström. 2013. A Comparison of Blue Light andCaffeine Effects on Cognitive Function and Alertness in Humans. PLOS ONE 8,
10 (10 2013). https://doi.org/10.1371/journal.pone.0076707[36] Andrii Matviienko, Vanessa Cobus, Heiko Müller, Jutta Fortmann, Andreas
Löcken, Susanne Boll, Maria Rauschenberger, Janko Timmermann, Christoph
Trappe, and Wilko Heuten. 2015. Deriving Design Guidelines for Ambient Light
Systems. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia. ACM, New York, NY, USA, 267–277. https://doi.org/10.1145/
2836041.2836069[37] Bilge Mutlu, Jodi Forlizzi, Illah Nourbakhsh, and Jessica Hodgins. 2006. The Useof Abstraction and Motion in the Design of Social Interfaces. In Proceedings of
the 6th Conference on Designing Interactive Systems. ACM, New York, NY, USA,
251–260. https://doi.org/10.1145/1142405.1142444

232