Showing Robots How to Follow People using aBroomstick Interface
James E. Young1,3, Kentaro Ishii2, Takeo Igarashi2,3, Ehud Sharlin1
jim.young@ucalgary.ca, kenta@designinterface.jp, takeo@acm.org, ehud@cpsc.ucalgary.ca
1

University of Calgary
Calgary, Canada

2

3

Design Interface Project, JST Erato
Tokyo, JapanThe University of Tokyo
Tokyo, Japan

Figure 1 –During demonstration the primary person walks as they normally would in the given circumstance and the designer pushes
a robot (iRobot Roomba) on a broomstick to demonstrate the style in which the robot should follow the primary person. Following,
during generation the primary person walks normally and the interactive robot path and style is automatically generated in real-time
to react to the primary person in a way that matches the demonstration style.
Abstract—Robots are poised to enter our everyday environments
such as our homes and offices, contexts that present uniquequestions such as the style of the robot’s actions. Style-oriented
characteristics are difficult to define programmatically, a
problem that is particularly prominent for a robot’s interactive
behaviors, those that must react accordingly to dynamic actionsof people. In this paper, we present a technique for programming
the style of how a robot should follow a person by demonstration,such that non-technical designers and users can directly create
the style of following using their existing skill sets. We envisionthat simple physical interfaces like ours can be used by nontechnical people to design the style of a wide range of robotic
behaviors.

I.

INTRODUCTION

The design of robots that enter our everyday environments
must be considered beyond their utility and functionality, aspeople generally care a great deal about the style of objects andtechnologies that they possess [2]. Style and fashion are often
important factors of technology adoption, and people will want
attractive and pleasing robots the same as they want an
attractive table, wristwatch, or car – design is directly related touser experience and satisfaction [1, 2].
Advanced programming is generally required to get robots
to do even simple tasks in the real world, and the style of theresulting goal- and efficiency-oriented actions often tends to bevery mechanical. It is difficult to programmatically embedricher style into robot actions, e.g., to program robots that pickup objects, shake your hand, or follow you, to do these actions“in style”, for example, following an aggressive, timid, or
careful fashion. These problems are exacerbated for nonscripted interactive robot behaviors, where the robot must
interpret a changing environment or person’s actions, and react
appropriately, in real time.
We propose to use programming-by-demonstration to
directly design the style of robot interaction – we focus on thestyle of how a robot should interact with a person, rather thanany particular goal-oriented interaction with the environment.
Our current exploration focuses on enabling people todemonstrate to a robot how it should follow a person, apractical yet stylistically flexible and diverse interactive robotic
task. Our implementation is a robotic extension to Puppet
Master [3], a programming-by-demonstration system for thelocomotion-based interactive styles of animated avatars. We
choose this as it has been shown to capture style and emotionof interactive movements from an example demonstration.
II.

STYLE-BY-DEMONSTRATIONWe see our approach as a kind of style-by-demonstration,
where, through demonstration, the robot is programmed to act
in a particular style in reaction to the real-time actions of aperson. During the demonstration phase, example motions areprovided both for the person (primary) and for the robot(reactionary), showing by acting how a robot should respond tothe person’s movements. Both paths are given simultaneously
in real time (Figure 1).
After demonstration, the generation phase takes the style of
the reactionary characteristics shown in the demonstration andincorporates it into real-time interaction. Here, the person
moves freely while new, appropriately styled, robot actions aregenerated on the fly to respond to the person’s movements. Our
approach generally allows demonstration to be relatively short,from 30s to 2 minutes, and real-time generation occurs
automatically and immediately, with no need for preprocessing.
III.

BROOMSTICK INTERACTION

The primary goal in developing an interface to enable aperson to demonstrate stylistic motion to a robot was to make iteasy to use, and to enable a person to focus more on the motion
style and less on the mechanics of moving the robot. One noteis that, for demonstrating the robot input, we ruled out thepossibility of directly tracking a person’s movements: theresulting demonstration would be too expressive and would
contain motions and nuances not reproducible by the robot.
We developed a broomstick interface (Figure 2), where aregular aluminum broomstick is attached to an iRobot Roomba– using a real Roomba for demonstration allows us toencapsulate the robot’s model of movement, expressive
capabilities and limitations. The broomstick is attached to therobot via a two-axis swivel, allowing it to be freely moved
forward, backward, left and right. The Roomba itself can be
moved forward and backward by pushing and pulling, andtwisting the broomstick turns the Roomba.
The result is a natural and familiar mechanism (andsituation) for demonstrating following style to the robot. In ourimplementation (Figure 2), one person walks naturally while
another person uses the broomstick to demonstrate a following
style to the robot (reactionary entity). When demonstration isfinished, a real (non-broomstick) Roomba enters the interaction
space and follows the person in the style that was demonstrated.

IV.

IMPLEMENTATION

Our implementation was realized through the extension ofthe Puppet Master [2] animation system. The primary challengeof adopting the animation-only Puppet Master system to realrobots was to be flexible to the hard, real-world limitations ofrobots, as robots cannot be as easily moved, manipulated, andtheir movements cannot be smoothed or filtered in the same
flexible way that on-screen animated characters can. Our early
resulting robot output did not resemble the training – it was
jittery, movements appeared incoherent, and the robot was notable to maintain localization (its position in respect to the otherentity, the person).
Our solution surrounds the observation that the texture, or
high-frequency detail movements of the Puppet Master output,are very important to the quality of the resulting movement.
However, equally important is the robot’s position in relation to
the person, where the fine-detailed motions are intended tohappen. For example, a robot motion would have differentmeanings if it were performed away from the person, in frontof the person, or behind the person.
We filtered the Puppet Master output through a model ofthe robot’s movement capabilities to obtain robot commands
that would best reproduce the desired texture. Then, we
modified these texture commands such that the robot tends
toward the target location. The result is what we believe to be avery good representation of the Puppet Master output, such thatthe robot output matches well the demonstration given.
V.

DISCUSSION AND FUTURE WORK

We are currently in the process of conducting andanalyzing a multi-part user study surrounding ourimplementation, including comparisons to other systems aswell as more exploratory and qualitative evaluations.
We believe that programming robotic style bydemonstration is a viable approach to designing roboticinteraction. Our initial research through our implementation
presented here suggests that not only is programming style bydemonstration possible, but that it is a natural and intuitive way
for non-technical people to be involved in programming robots.
We intend to investigate these ideas further through extended
implementations and controlled user studies.
REFERENCES[1]
Donald A. Norman. Emotional Design: Why We Love(or Hate) Everyday Things. Basic Books, NY, 2004.
[2]
James E. Young, Richard Hawkins, Ehud Sharlin,
and Takeo Igarashi. Toward acceptable domestic robots:
Applying insights from social psychology. Inagural Issue ofthe Int. J. Social Robotics. January 2009, 1(1), 2009.

Figure 2 – demonstrating how a robot should follow a person(left) and generation, where the robot automatically mimics thedemonstration to follow the person (right)[3]
James E. Young, Takeo Igarashi, and Ehud Sharlin.
Puppet master: Designing reactive character behavior bydemonstration. In ACM SIGGRAPH / EG SCA ’08., pages
183–191, Germany, 2008. EG Press, EG Association.