A Dance Performance Environment in which
Performers Dance with Multiple Robotic Balls
Shuhei Tsuchida

Tsutomu Terada

Masahiko Tsukamoto

Grad. Sch. of Engineering,
Kobe University
1-1 Rokkodaicho, Nada
Kobe, 657-8501, Japan

Grad. Sch. of Engineering,
Kobe University
and JST PRESTO
1-1 Rokkodaicho, Nada
Kobe, 657-8501, Japan

Grad. Sch. of Engineering,
Kobe University
1-1 Rokkodaicho, Nada
Kobe, 657-8501, Japanshuhei.t@stu.kobeu.ac.jp

tsutomu@eedept.kobeu.ac.jp

ABSTRACTIn recent years, as robotics technology progresses, various
mobile robots have been developed to dance with humans.
However, up until now there have been no system for interactively creating a performance using multiple mobile robots.
Therefore, performance using multiple mobile robots is still
diﬃcult. In this study, we construct a mechanism by which
a performer can interactively create a performance while
he/she considers the correspondence between his/her motion
and the mobile robots’ movement and light. Specifically, wedeveloped a system that enables performers to freely create performances with multiple robotics balls that can moveomunidirectionnally and have full color LEDs. Performers
can design both the movements of the robotic balls and thecolors of the LEDs. To evaluate the eﬀectiveness of the system, we had four performers use the system to create anddemonstrate performances. Moreover, we confirmed thatthe system performed reliably in a real environment.

CCS Concepts•Human-centered computing → Human computer interaction (HCI);

Keywords
Dance; Performance; Mobile robot; Choreograghy; Ball;

1. INTRODUCTION
In recent years, as robotics technology progresses, various robots have been developed to dance with humans[1, 2,
3]. There have ever been attempts in which mobile robots
dance as the alternative of a human performer[4] and perform with another human[5]. However, up until now, there
has been no system for interactively creating a performance
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.

AH 2016, February 25-27, 2016, Geneva, Switzerlandc 2016 ACM. ISBN 978-1-4503-3680-2/16/02. . . $15.00⃝
DOI: http://dx.doi.org/10.1145/2875194.2875219

tuka@kobe-u.ac.jp

using multiple mobile robots. Therefore, performance using
multiple mobile robots is still quite diﬃcult. In this study,
we construct a mechanism by which a performer can interactively create a performance while he/she considers the correspondence between his/her motion and the mobile robots’
movement and light. Our system consists of a control system for the mobile robots and an application for creating
patterns of the movement and light of the mobile robots. A
user can create and confirm his/her choreography and the
movement and light of mobile robots on the application at
the same time. It is possible to program the mobile robots
to conform to the patterns that the user has created. In
this study, we applied Sphero2.0[6], a robotic ball, as a mobile robot. We felt that a type of sphere-shaped ball would
be compatible with body expression in many genres, as canbe seen from several artistic forms, such as MOVEment[7],Moon Beams[8], and Metamorphose(s)[9], sports gymnastics, freestyle football, mime, magic, and juggling. Additionally, a sphere shape can absorb and disperse the mostshock, and is durable in case of collision with a performer.
The remainder of this paper is organized as followed. Section 2 introduce related works and Section 3 explains thesystem requirements and configuration. We introduce two
applications, one for creating the movement and light ofrobotic balls and one for controlling the robotic balls, inthis section. Section 4 presents results we obtained fromexperiments and considerations on the evaluation system.
Section 5 presents two demonstrations and we conclude in
Section 6 with a brief summary and our plans for futurestudies.

2.

RELATED WORK

There have been some attempts at collaboration between
humans and mobile robots. Kosuge et al.[10] developed arobot that can be a social dance partner and moves by estimating human steps. Higuchi et al.[11] developed a systemthat decides on control parameters to move four quarters
or a rotation in concurrence with the body movements ofa human crouching and walking. Graether et al.[12] examined four themes—embodiment, control, personality, andcommunication—to provide an understanding of how to design robots for physical activities. Tominaga et al.[13] developed a mobile robot with a display that can provide arunner’s form in real time by filming and following a run-ner continuously from a certain distance. Futami et al.[14]developed a mobile robot with a Web camera to solve theproblem of self-haircut when both hands are unavailable foruse. PHENOX LAB[15] created “Phenox”, which can autonomously fly by determining the surrounding conditions
and recognizing the movement of a person. In this study,
we created a system that makes it easy for the user to dancewith multiple mobile robots.
Here, let us briefly list some attempts that cover the expansions of various things using a sphere. Izuta et al.[16]created “Bouncing Star”, a rubber ball for sports that is
equipped with electronic devices such a wireless module, acceleration sensor, sonic sensor and LED. The ball’s color
and the graphics of a playing field can be manipulated, suggesting exciting opportunities in the creation of new digital
sports. Yamaguchi et al.[17] implemented Digital Juggling,which stably projects an image on a sphere for toss juggling
by applying a Kalman filter and a movement model andmaking an orbit estimation. Michaud et al.[18] designed
Roball to for a variety of common household situations. For
example, Roball can attract the attention of children andmake them smile. Nitta et al.[19] proposed a flying ballbased on quadcopter technology. They developed some basic interactions such as the balls floating up when a player
raises a hand. TeamLab [20] developed “teamLabBall” thatcan change color and light and emit sound in accordancewith the movement of a person near it. Thus, new possibilities are created by adding power, sensors, visuals, and light.
In the same vein, body expression with the coordination of
persons can be expected with the robotic balls used in thisresearch.
Furthermore, in recent years, many directors and artists
have tried extending the body’s expression by using moving objects. Manabe et. al.[21] developed a system thatenables the performer to recognize an object intuitively byusing quadcopters controlled by the relationship between the
body and the data in real space. DreamWorks Animation
LLC.[22] was created performances in 3D space that cannot
be expressed by computer graphics. William Forsythe [23]prepared the space which has many pendulum which was
hanged on a ceiling. When the viewer enters this space, heor she must move and dodge in accordance with the pendulum engraving a periodic rhythm. This results in a situationthat looks like dancing. Daniel Wurtzel[24] created Pas deDeux, where a cloth is made to look as though it is dancing
by using a device placed on the floor that creates wind. The
performer freely creates a dance to fit the cloth dancing.
TOVIA Co.[25] developed Ninja Light, which can make asphere group move up and down to match the moving of aperformer’s body.
It is possible for the performer to express him or herself just like manipulating space. CONNECTED, made by
CHUNKEY MOVE[26] uses an expression similar to the
Ninja Light. Bot & Dolly[27] created Box, a combination ofhuman performance and the movement of two white plateswith a projected image. As mentioned above, there aremany approaches to the collaboration of mobile robots withthe body expression of people. We developed a system forusers to interactively create a performance while considering the correspondence between their body’s expression andthat of a mobile robot.

Table 1: Specs of Sphero2.0.
Size
74 mm
Weight168 g
Maximum speed
2 m/s
Communication distance max 30 m

3.

PROPOSED SYSTEM

3.1

System Design

The purpose of the proposed system is to enable users tocreate a performance with multiple robotic balls easily. We
developed a system by which a performer with no knowledgeof programming can create patterns of the movement andlight of robotic balls to certain music and choreography bymouse operation.
It is not easy for a performer to confirm his/her choreography by virtually moving robotic balls because it takes
a long time to charge robotic balls, set the communication
environment, and save a large place to set up an infrared
camera and them finally dance with the balls. Therefore,
it is necessary to create a mechanism to easily confirm thechoreography of dancing with robotic balls on a simulator.

3.2

System Configuration

The procedure the proposed system uses is as follows.
1. A user creates the movement and light of robotic balls
that synchronizes with music by using the developed
application.
2. The user confirms his/her choreography of dancing
with the robotic balls on a simulator by using Microsoft Kinect [28]. The user then modifies his/herchoreography by fitting the movement and light of theballs created in the first step.
3. The user saves the patterns of movement and light tomobile phones and a PC for control.
4. The user fits the movement and light in sync with themusic.
We used a robotic ball called Sphero2.0 as the hardware.
The specs are given in Table 1. We prepared two types ofexternal shells one made from ABS resin with a 3D printer(MakerBot, Replicator 2X) and one created by sharpening
commercially available plastic capsules. Examples of bothare given in Fig. 1.
We developed two following applications to realize theproposed system.

3.2.1

Aplication for Creating the Movement and theLight of Robotic Balls

The UI of the developed application is shown in Fig. 2.
We developed the application using openFrameworks andthe development environment was Xcode. This application
can change the number of robotic balls in accordance withthe scale of a performance. We will explain the functions of
this application shortly.
The display for creating formations is shown in Fig. 3.
It is possible to create formations of the robotic balls by7UDMHFWRU\WR
QH[WSRVWLRQ

1H[WSRVLWLRQ^ƌĞŝŶ&XUUHQWSRVLWLRQ&KDQJLQJWKHSRVLWLRQ
E\GUDJDQGGURSWůĂƐƚŝĐ

&KDQJLQJWKHSRVLWLRQE\XVLQJVOLGHU

Figure 3: Display for creating formations.
Figure 1: The two types of external shell.
&RQILUPLQJ SRVLWLRQVRIDOOURERWLFEDOOV&RQILUPLQJDIRUPDWLRQ

6HWWLQJDIRUPDWLRQ

0XVLFLQIRUPDWLRQ&RORUSDOOHW&KDQJLQJ
WKHFRORU.H\IUDPH)LJ
)LJ

0XVLFLQIRUPDWLRQ.H\IUDPH IRUUHDGLQJIRUPDWLRQV

5*% LQIRUPDWLRQ:ULWLQJWKHQXPEHURIDUHJLVWHUHGIRUPDWLRQ&KDQJLQJ
WKHVFDOHRIWLPHOLQH&RORU
LQIRUPDWLRQ

Figure 2: UI of developed application.

Figure 4: Timeline function.

fitting a parameter with a slider or dragging and dropping
the point indicated by the position of a robotic ball.
Regarding the formation created by adding Keyframe (Fig.
4) to the timeline on which music information is mounted,
the formation movement that is synced to music is madepossible. Added keyframes can be stored with reﬀrence tothe color information obtained by clicking a palette of the
CMYK color representation.
Furthermore, the confirmation of the movement and light
can be displayed in 3D. The overall performance can be confirmed while actually choreographing on the simulator byusing Kinect. The screen when confirming performance isshown in Fig. 5. A performer is represented by the skeleton created from the joint information acquired by Kinect.
When the confirmation of the performance is done, a file inwhich the patterns of the movement and light of the robotic
balls are contained is automatically created in CSV format.

to the ceiling by using projective transformation. Thus, theinfrared camera can be placed at a free angle and position.
The location the robotic ball moves to and the light havebeen determined by reading CSV formatted files created by
the application for the movement and light creation. The
lighting pattern is saved on the mobile phone in advance, so
the information sent from the main PC to the mobile phone
can be limited to the movement of the robotic balls.
The velocity and direction of a robotic ball are calculated
from the distance and the angle toward the next destination of the ball by using PID control. Moreover, to avoid
collision, a repulsion is put in place in accordance with distance, and the velocity and direction of the robotic ball arecorrected by the value. The system sends the informationof the velocity and direction of the robotic ball decided by
the controlling PC to the mobile phone via Wi-Fi and sends
commands from the mobile phone to the robotic ball connected via Bluetooth. A robotic ball corresponds to a mobile
phone to make the Bluetooth connection stable. To avoid
radio wave interference, the Wi-Fi uses the 5 GHz frequencyband.
One weak point of the current system is that it is diﬃcult
to track the robotic ball when the dancer is between thecamera and the robotic ball for more than a certain period.
Hence, a performance configuration that is not between the

3.2.2 Control System for Robotic Balls
The configuration of the robotic ball control system isgiven in Fig. 6. An infrared camera detects the positionof robotic balls mounted on an infrared LED from the difference between the current image and the previous image.
This is then transformed into the location information of
the robotic ball that is same as the environment attachedϭ͘Ϯŵ

5RERWLFEDOO

3HIRUPHU«V VNHOHWRQϮŵW<ŝŶĞĐƚ

ĂŶĐĞƌ

tĂůů

W

Ϯŵ

Ϯŵ

<ŝŶĞĐƚ

ĂŶĐĞƌϬ͘ϳŵϬ͘ϰϱŵ
ϮŵWƌŽũĞĐƚŽƌϮŵϮŵWƌŽũĞĐƚŽƌ

ϳŵ^ŝĚĞǀŝĞǁ

dŽƉǀŝĞǁ

Figure 5: 3D image.
Figure 7: Layout for procedure 3.
ĂŶĐĞƌ
ĂŵĞƌĂ,PDJH ,QIUDUHGFDPHUD

ϲŵ

Ϯ͘Ϯŵ
DŝƌƌŽƌ ϮŵDŝƌƌŽƌ

0DLQ3&'LUHFWLRQDQGVSHHG
RIURERWLFEDOOV

ϭ͘ϭŵ

Ϯŵ

ĞƐŬƚŽƉW ĂŵĞƌĂ
Ϯŵ

ϭ͘ϭŵ
ZŽďŽƚŝĐďĂůů ĞƐŬƚŽƉW

ĂŶĐĞƌ

ZŽďŽƚŝĐďĂůů'DQFHU^ŝĚĞǀŝĞǁ

0RELOHSKRQH'LUHFWLRQ
DQGVSHHG

dŽƉǀŝĞǁ

5RERWLFEDOO

Figure 8: Layout for procedure 4.
Figure 6: System configuration.
camera and the robotic ball is required.

4. EVALUATION EXPERIMENT
4.1 Experiment Procedure
In our evaluation experiment, participants created performances with robotic balls by using the proposed system.
We investigated how participants used the system and how
much time was needed for completion of a performance.
In terms of the performance accompanying the movement
robot, we investigated how participants took both their own
body expression and that of the robot into consideration tocreate the performance. Participants were three males withan average age of 22.3 who had been dancing for more thanfour years. This group included a system developer aged 25
had been dancing for more than four years and had someexperience in performance creation using robotic balls. The
procedure for the experiment is as follows.
(1) The participant receives a 20-minute explanation about
how to use the application.
(2) The participant uses the application to create choreography based on a song about 1 minutes, 10 seconds
long at 120 BPM.
(3) The participant confirms the choreography on the application.
(4) The participant carries out a performance using robotic
balls.

In this experiment, the number of robotic balls was three.
The participants created the movement and light of the mobile robot and choreographed while simultaneously creating performance. Moreover, the participants proceeded tocreate and modify the choreography while they confirmed
the movement and light of the mobile robot on the application. One B5 paper and a pen were also used whencreating the choreography. The experimenter received thequestion about how to operate the application during the experiment. The time taken for the experimental procedures
2 and 3 was noted and a 10-minute break was scheduled
every hour. At the end of the experiment, participants provided feedback on the application and commented on how
they considered the movement and the light of the robot
each eight beats. We defined a quarter note as one beat.
The layout of procedures 3 and 4 is shown in Fig. 7 and 8,
respectively. In this experiment, we used a laptop PC (Apple, Macbook Air 13-inch, mid-2011), a desktop PC (Apple,
iMac 27-inch, late 2013), projector (BenQ, MP522ST), Web
camera (iBUFFALO, BSW20KM11BK), and depth camera(Microsoft, Kinect v2). The experiment took place in a studio measuring 6 m×7 m, located inside a laboratory where
the experiment took place. The doors were all closed tominimize outside sounds. All lights were turned oﬀ whenusing the projector.

4.2

Results and Discussion

The time in took for participants to create a performanceis given in Table 2, how to show the expression of each eightis shown in Table 3, and the memos written by participants
1, 2 are shown in Fig. 9. It took an average of 2 hours 22
minutes for participants to create a performance with threerobotic balls.

Table 2: Time taken by participants for performance creation.
Participant Performance creation time
1
2 hour, 38 minutes
3 hour, 29 minutes
2
3
2 hour, 17 minutes
4
1 hour, 4 minutes

One favorable response about the application was that its
movements were immediately clear at a glance. However, on
the negative side, one participant said it was hard to determine the sense of distance with the robotic balls while confirming the choreography via simulator display. Therefore,
the participant would prefer to confirm the choreography
by actually using the robotic ball. Regarding this problem,
our future work will be to construct the system such thatit can give the sense of dancing with the robotic ball. Another negative piece of feedback was that the setting to takecolor gradient between keyframes made it hard to operate
when participant was deciding on the lighting. One participant suggested that it would be more convenient if thelocation of the robotic balls moves instantly when the participant moves the sequence bar on the timeline. We intend
to modify these points in the future.
Of all participants, participant 4 that has the experience
using the system created the performance most quickly. It
seems one reason for this was his familiarity with the system operation. However, the other three participants said
it was easy to operate as far as the operability of a systemgoes. Therefore, it was diﬃcult to consider that there wasmuch diﬀerence in the actual operating time. We considered
the important factor, as it follows more than the diﬀerencesin familiarity with operating. Participant 4 had experience
using the system and accumulated experience with how the
robotic balls were combined and the sense of dancing withthe robotic balls. It is then easier to imagine the expression
one wants to convey, and the ease of coming up with ways toachieve the expression contributed greatly to reducing thetime. Thus, we provided the performers with various examples of what they wants to express on the basis of the
response regarding how the expression was shown. For example, when the performer wants to express the robotic balls
connecting with him/her, we prepared various examples ofmaking it appear like they were connecting. If the prepared
expression can be easily previewed for confirmation, we feel
it will not only reduce the creation time but also widen therange of expression the performer creates. Here, each eightbeats, we confirmed from the memos in Fig. 9 that all participants created the choreography. We extract and list theways to show what the performer wants to express on thebasis of responses to how the expression was shown.
• The extracted expression
Enclose, Link a dancer, Emphasize, Gather, Enter aspace, Spread, Make a space, Rotate the entire formation, Work a force, Move a border, Intersection, Createa shape (such as a triangle).
• The extracted movement
Slide, Line up in sequence, Change alternately, Fix apoint, One by one action.

0HPRRISDUWLFLSDQW

0HPRRISDUWLFLSDQW

Figure 9: Memos of participants 1, 2.

If the movement corresponded with the expression thatmultiple people who had answered had equaled in each movement, the movement could be said to be frequently used by
performers. Therefore, those who frequently used the movement should be in the library programme. Here, we pickedup each movement that corresponded to the expression seen
within the expression extracted. We confirmed and summarized the characteristics of each movement.
All participants were found to have the expression of “entering the space (a triangle, etc.)”. Regarding participant
1, the robotic ball lined up in a row moved to a triangle
formation and the dancer slid into the space the robotic ballmade after the movement in eight beats. As for participant
2, the dancer had already slid into the triangular space therobotic ball drew in two beats. Simultaneously, the dancer
kept the spot of the space in accordance with the robotic
ball’s movement. With participant 3, the robotic ball lined
up in a row spread right and left into a triangle formationand the dancer slid into the space the robotic ball madein eight beats. Compared to participant 1, the dancer wasplaced slightly behind the space made. Regarding participant 4, the robotic ball had drawn a small triangle formation in advance and spread it in four beats for the dancer toslide into the space made in four beats. In consequence, inthe expression entering the space, there are two movementsof expression: the movement of the dancer sliding into thespace made by the robotic balls and the dancer sliding into
the space that the robotic ball formed in advance.

Beats
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8

Table 3: Ways to show what the performer wants to express: participant 1.
How to show a movement
How to show a lightStanding.

Showing the color change.

Posing. The audience’s attention is on the dancer.

Changing the line of color every four counts.

The dancer intersects robotic balls and changes a formation.
Stopping robotic balls. The audience’s attention is on the dancer.
The audience’s attention is on the dancer.
The dancer intersects robotic ball and enters created triangle.
Synchronizing the dancer and the robotic ball.
Creating a square by the dancer and the robotic ball.
Like controlling a robotic ball.
Line up robotic balls in a row in front of the dancer.
Sliding with robotic balls

Red line.
Blue and green lines.
From yellow to red.

Changing the color finely.

Robotic balls move in conjunction after the dancer moves.
Gathering the dancer and robotic balls.

All participants were found to have the expression of “Emphasize”. For all participants, the dancers were dancing
while the robotic balls were stopped. Regarding participant 1, the answer was turning the audience’s attention to
the dancer. This could be replaced as emphasizing. With
participant 2, the position of the dancer was almost fixed.
Moreover, the movement of robotic balls was changed ineight beats. Regarding participant 4, the state of a corresponding movement of the dancer and a robotic ball being emphasized, was expressed. Regarding participant 1,
the movement of controlling a robotic ball was confirmed.
It could be also considered as an expression of emphasizing: namely, the expression emphasizing movement. There
are two movements of expression about the dancer dancing
freely while the robotic balls were stopped and the dancer
moving simultaneously with a robotic ball.
All participants were found to have the expression of “Linka dancer”. With participant 1, robotic balls lined up in tandem and the dancer is moving toward the same direction ineight beats. Further, when the dancer comes near to therobotic balls in tandem, the robotic balls move away fromthe dancer. Regarding participant 2, the dancer stands inthe center of the triangle drawn by robotic balls and repeats
moving at the same time the direction is changed every eightbeats. For participant 3, the dancer stands behind robotic
balls that line up in a row and simultaneously move sidewaysin eight beats. Regarding participant 4, the dancer stands
behind robotic balls that line up in a row and simultaneously
move back and forth in eight beats. Furthermore, bodyparts such as feet are moving toward the same direction at
the same time as a robotic ball. In consequence, in the expression linking the dancer with the robotic balls, there are
two movements, one in which the dancer and robotic balls
move in the same direction simultaneously and one where
the dancer and robotic balls move in the same direction alternatively.
Three of the participants 1, 3, and 4 were found to havethe expression of “Gather”. Regarding participant 1, thedancer and the greatly expanding robotic balls are coming together in the center in four beats. For participant
3, the robotic balls expanding in a column move to the center in four beats and make the formation in tandem with thedancer. With participant 4, in accordance with the movement of the dancer turning, the expanding robotic balls
move to the center in eight beats. In consequence, in theexpression about gathering, there are two movements about
the robotic ball and the dancer: one where they are moving
to the center and one where the robotic balls move to thecenter in coordination with the dancer’s motion.
Three of the participants 1, 2, and 3 were found to havethe expression of “Create a shape”. For participant 1, thedancer who is in the center moves quickly in two beats andthe quadrangle is made by the dancer and three robotic
balls. Regarding participant 2, in the free comments of thequestionnaire, he created the performance with how it canbe expressed using a triangle drawn by three robotic balls
and the dancer in mind. Therefore, the ratio of a side andthe rotation were changed while the shape of a triangle wasmaintained. With participant 3, a robotic ball that was positioned far away from the rest moves in front of the dancer
in four beats and draws a triangle. In consequence, in theexpression of creating shapes, there are two instances about
the movement of shapes: one drawn by the dancer and therobotic balls, and one drawn by the robotic balls only.
Two of the participants 1 and 4 were found to have theexpression of “Intersection”. Regarding participant 4, hemoves toward the same direction with a robotic ball andthe other two robotic balls move toward the opposite direction simultaneously. As for participant 1, he made thesame movement as participant 4. Moreover, when the formation of robotic balls changes from in tandem to a triangular shape, a robotic ball and the dancer move as they
pass each other and their positions are exchanged. In consequence, in the expression of intersecting, there are two
movements about a robotic ball and a human: one moving
as they pass each other and one in which the position of the3HUIRUPHU

5RERWLFEDOO$XGLHQFH

Figure 11: Demonstration at ACE2015.
Figure 10: Demonstration at a domestic academic conference.

dancer and the position of a robotic ball switch.
On the basis of the above, our challenge for the future
is to create a list of what we wish to express and turn thecorresponding robot movement and lighting into a librarythat can be implemented in such a way that the movements
can be previewed on the application. We also intend to havemore performers use our system and enrich the expression
library. Moreover, we will investigate cases of more thanfour robotic balls. We will also create a correspondence
table of body expression and moving robots, and if we caninvestigate and clarify how the combination of these cangive a certain expression, we may be able to formulate an
expression that combines robot movements.

5. DEMONSTRATION
We confirmed that our proposed system could be used for
performance in a real environment by staging a demonstration with it at a domestic academic conference (Fig. 10)
held at the National Museum of Emerging Science and Innovation in Japan from March 6–8, 2015. We were able toreliably repeat the one-minute performance over a two-hourdemonstration. This academic conference boasts over 600
participants, many of whom had the opportunity to view theperformance. Viewers oﬀered feedback such as, “It looked
as if the performer were controlling the robots in real time”,“The wobbly robotic balls were irresistibly cute”, and “If themoving robots are not bigger then the viewers who are near
the back will not be able to tell that they are spherical”. As
a result, we won the award for an interactive presentationin the 2nd division.
We staged another demonstration using this system andsubmitted a short paper[29] as performance session at ACE
2015 (Fig. 11), which was held at the White Box and BlackBox arts complex Medini Mall in MALAYSIA from Nov.
16–19, 2015. We used our system with stage lighting andsucceeded in giving a one-minute performance in which no
mistakes are allowed and only one performance is permitted.
We received requests for an encore from the audience.
As demonstrated by, we were able to prove that the system
is reliable for use in real environments.

6.

CONCLUSION

In this paper, we constructed a system by which a performer can interactively create patterns of the movement
and light of robotic balls while he/she considers the correspondence between his/her motion and the mobile robot’s
movement and light. To evaluate the eﬀectiveness of thesystem, we had four performers use the system to createand demonstrate performances. We extracted and listed
the ways to show what the performer wants to express on
the basis of responses to the expression and confirmed thatthe movement corresponded with the expression by multiple
people whose answers matched each movement. Moreover,
we confirmed that the system performed reliably in real environment.
In the future, we will improve our system and to create
more performances. We will perform investigations into theeﬀect on body expression that comes from designing robots
that are non-spherical.

7.

ACKNOWLEDGMENT

This research is supported financially by IPA Exploratory
IT Human Resources Project 2014 (The MITOH Program2014)，the Japan Science and Technology Agency (JST) Super Cluster Program (Kyoto Area, Construction of HighlyEﬃcient Energy to Achieve a Clean and Low Environmental
Impact Society)，and Grant-in-Aid for Scientific Research(A)(23240010) from the Japanese Ministry of Education, Culture, Sports, Science and Technology.

8.

REFERENCES[1] F. Tanaka, J. R. Movellan, B. Fortenberry and K.
Aisaka: Daily HRI Evaluation at a Classroom
Environment -Reports from Dance Interaction
Experiments-, Proc. of the 1st ACM
SIGCHI/SIGART Conference on Human-Robot
Interaction (HRI ’07), pp. 3–9 (Mar. 2006).
[2] M. P. Minchalowski, S. Sabanovic and H. Kozima: A
Dancing Robot for Rythmic Social Interaction, Proc.
of the 2nd ACM/IEEE International Conference on
Human-Robot Interaction (HRI ’07), pp. 89–96 (Mar.
2007).
[3] S. Tsuchida, T. Terada and M. Tsukamoto: A System
for Practicing Formations in Dance PerformanceSupported by Self-Propelled Screen, Proc. of the 4th
Augmented Human International Conference (AH’13), pp. 178–185 (Mar. 2013).
[4] E. Jessop, P. A. Torpey and B. Bloomberg: Music andTechnology in Death and the Powers, Proc. of theInternational Conference on New Interfaces forMusical Expression (NIME ’11), pp. 349–354 (May2011).
[5] R. Barnelt et. al.: SERAPH(2011),
http://www.csail.mit.edu/node/1640.
[6] Orbotix: Sphero2.0, http://www.gosphero.com/ja/.
[7] Calvin Klein Collection x Jonah Bokaer and Julie
Kent: MOVEment, http://goo.gl/O1QZyk.
[8] MOMIX reMIX in Crete: Moon Beams,
https://goo.gl/HbrEQw.
[9] Compagnie de Danse l’Eventail: Metamorphose(s),http://goo.gl/kiW9mw.
[10] K. Kosuge, T. Takeda, Y. Hirata, M. Endo, M.
Nomura, K. Sakai, M. Koizumi and T. Oconogi:
Partner Ball-room Dance Robot-PBDR-, SICE
Journal of Control, Measurement, and System
Integration (SICE JCMSI), pp. 74–80 (Jan. 2008).
[11] K. HIguchi, K. Fuji, J. Rekimoto: Flying head: A
head-synchronization mechanism for flying
telepresence, Proc. of the 23rd International
Conference on Artificial Reality and Telexistence(ICAT ’13), pp. 28–34 (Dec. 2013).
[12] E. Graether: Joggobot: A Flying Robot as Jogging
Companion, Proc. of the Extended Abstracts on
Human Factors in Computing Systems (CHI ’12),
pp.1063-1066 (May 2012).
[13] J. Tominaga, K. Kawauchi and J. Rekimoto: Around
Me: A System with an Escort Robot Providing a
Sports Player’s Self-Images, Proc. of the 5th
Augmented Human International Conference (AH’14), No. 43, pp. 1–8 (Mar. 2014).
[14] K. Futami, T. Terada and M. Tsukamoto: A System
for Supporting self-haircuts Using Camera Equipped
Robot, Proc. of the 12th International Conference on
Advances in Mobile Computing and Multimedia(MoMM ’14), pp. 34–42 (Dec. 2014).
[15] PHENOX LAB: Phenox,
http://phenoxlab.com/?page id=601.
[16] O. Izuta, T. Sato, S. Kodama and H. Koike: Bouncing
Star project: design and development of augmented
sports application using a ball including electronic andwireless modules, Proc. of the 1st Augmented Human
International Conference (AH ’10), No. 22, (Apr.
2010).
[17] H. Yamaguchi and H. Koike: LumoSpheres: Real-Tme
Tracking of Flying Objects and Image Projection for a
Volumetric Display, Proc. of the 6th Augmented
Human International Conference (AH ’15), pp. 93–96,(Mar. 2015).
[18] F. Michaud and S. Caron: Roball, the Rolling Robot,
the Journal of Autonomous Robots, Vol. 12, pp.
221–222, (Mar. 2002).
[19] K. Nitta, K. Higuchi and J. Rekimoto: HoverBall:
Augmented Sports with a Flying Ball, Proc. of the 5th
Augmented Human International Conference (AH’14), No.13, (Mar. 2014).

[20] teamLab: teamLabBall, http://goo.gl/7xYxGu.
[21] D. Manabe: Dance with Drones,
http://www.daito.ws/work/elevenplay drones.html.
[22] DreamWorks Animation LLC: How To Train Your
Dragon Live Spectacular,
http://dreamworksdragonslive.com/.
[23] William Forsythe: Nowhere and Everywhere at theSame Time No.2, http://goo.gl/Pjxi8z.
[24] Daniel Wurtzel: Pas de Deux,
http://www.danielwurtzel.com/project.cfm?id=19.
[25] Tovia co.: NINJAR LIGHT, http://goo.gl/QGeWYe.
[26] CHUNKY MOVE: CONNECTED,
http://goo.gl/Q5mnRh.
[27] Bot & Dolly: Box,
http://www.philreyneri.com/#/box/.
[28] Microsoft: Kinect,
http://www.xbox.com/ja-JP/kinect.
[29] S. Tsuchida, T. Terada and M. Tsukamoto: A System
for Dancing with Robotic Balls, Proc. of the 12th
International Conference on Advances in ComputerEntertainment Technology (ACE ’15),