A System for Practicing Formations in Dance Performance
Supported by Self-Propelled Screen
Shuhei Tsuchida

Tsutomu Terada

Masahiko Tsukamoto

Grad. Sch. of Engineering,
Kobe University
1-1 Rokkoudaichou Nadaku,
Kobe, 657-8501, JAPAN

Grad. Sch. of Engineering,
Kobe University
PRESTO, JST
1-1 Rokkoudaichou Nadaku,
Kobe, 657-8501, JAPAN

Grad. Sch. of Engineering,
Kobe University
1-1 Rokkoudaichou Nadaku,
Kobe, 657-8501, JAPANshuhei.t@stu.kobeu.ac.jp

tsutomu@eedept.kobeu.ac.jp

ABSTRACTCollapsed formation in a group dance will greatly reduce thequality of the performance even if the dance in the group issynchronized with music. Therefore, learning the formationof a dance in a group is as important as learning its choreography. However, if someone cannot participate in practice,
it is diﬃcult for the rest of the members to gain a sense of
the proper formation in practice. We propose a practicesupport system for performing the formation smoothly using a self-propelled screen even if there is no dance partner.
We developed a prototype of the system and investigatedwhether a sense of presence provided by both methods ofpracticing formations was close to the sense we really obtain when we dance with humans. The result veriﬁed thatthe sense of dancing with a projected video was closest to
the sense of dancing with a dancer, and the trajectory information from dancing with a self-propelled robot was close
to the trajectory information from dancing with a dancer.
Practicing in situations similar to real ones is able to bedone by combining these two methods. Furthermore, weinvestigated whether the self-propelled screen obtained theadvantages of dancing with both methods and found that it
only obtained advantages of dancing with projected video.

General Terms
Human Factors, Experimentation, Performance

Keywords
Formation dance training, Mobile robot, Projector

1. INTRODUCTION
Improving physical expressions and the sense of rhythm indance performances has attached attention in recent years
due to the increase in child dancers and dance studios. Even
Permission to make digital or hard copies of part or all of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bear
this notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, to republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee.
AH’13, March 7-8, 2013, Stuttgart, Germany.
Copyright 2013 ACM 978-1-4503-1904-1/13/03 ...$15.00.

tuka@kobe-u.ac.jp

beginners in dance gain more opportunities to perform dancesin groups. When dancing in groups, collapsed formation will
greatly reduce the quality of dance performance, even if thechoreography is synchronized with the music. Therefore, it
is very important to keep formation in group dances. It isalso important to be aware of keeping the proper formationand moving smoothly into the next formation to performprofessional level group dances. However, it is diﬃcult toobtain the sense of a proper formation if some member ofthe dance cannot participate in practice. The presence ofa person dancing next to someone aﬀects his/her performance and senses.. When dancing in groups, training to do
the formation is an important element as well as learning
the choreography.
There have been various approaches that have supported
learning of the choreography. There have been methods oflearning skills using motion capture technology[1, 2] and inertial sensors have been employed[3]. This researches has
veriﬁed what kind of support is eﬀective to enable the choreography to be practiced. They also research has also been
done on the ways to display model dances and on visualizing the diﬀerences between model dances and one’s actual
movements.
However, there has not been any previous work that has
focused on training to do formations. Therefore, we proposea system of training to do formations when there a member
is about using a self-propelled robot to provide the presence of the missing member. We project an image of thedancer onto a self-propelled screen in this system that canmove as quickly as the dancer, making the other dancers
feel as though they were dancing with the missing dancer.
Furthermore, we evaluate which method of practicing formations gives the closest feeling of dancing with an actual
dancer.
The remainder of this paper is organized as follows. Section 2 outlines related work and Section 3 enumerates methods of practicing formations, explains system requirements,
presents the results, and provides considerations into evaluating the methods of practicing formations. it also describes
the design of our system. Section 4 explains the system implementation and Section 5 presents the results we obtained
from experiments and considerations into the methods ofevaluation. Section 6 presents our conclusions and plans wehave for future studies.

2. RELATED WORK
Various methods of having humanoid-robots dance haverecently been developed because of advances in roboticsrelated technologies [4, 5, 6, 7, 8]. Xia et al. have shownthat algorithms can automate the task of choreographingrobot dances to allow them to dance without detailed human planning [9]. Nakaoka et al. has developed a system
for robots to perform dance motions acquired from humandancers [10]. However, the robots using these methods cannot choreograph their movements or more as smoothly as
human beings. In addition, various autonomous robots thatcan dance with humans have been developed. For example,
the partner ballroom dance robot(PBDR) [11] developed by
Kosuge et al. has a variety of mechanisms to achieve thenecessary movements required in ballroom dancing. After
receiving forces and moments in three dimensions from a
6-axis force sensor built into it, the computer in PBDR controls its movements based on ballroom dance steps that havebeen stored in it in advance. Therefore, the robot can feelthe direction that the dance partner wants to lead and estimate the partner’s steps. Nakamura et al. proposed a system were students learned the choreography and the amountof translation eﬃciently by following a robot equipped with
an image display [12]. However, these training systems focused on ballroom dancing, and targeted beginners. Thesesystems did not consider that people made vigorous movements like those in street dancing in groups. The presenceof a person dancing next to them in a performance will
have various eﬀects on partners. Therefore, people can not
usually succeed in dancing by learning the choreography totranslate dances.
There are methods that support the learning of skills using
motion capture technology [13, 14, 15], inertial sensors, andﬁeld of the virtual reality [16]. Furthermore, Microsoft developed for a new type of motion controller for games called
KINECT. Much game software for learning dance has beenreleased. It is possible to obtain the position of joints in the
body in real time, and identify the movements of the entire
body with KINECT [17]. If users can learn useful skills in
the real world while playing video games through a more sophisticated virtual experience, this will greatly change theimportance of video games. Deng et al. proposed an interactive dancing game [18] in which a virtual character memorized one’s dance moves in advance, and a virtual avatar was
linked to one’s real time movements when dancing together
on a screen. By comparing the virtual character to one’s
avatar, he/she will be able to know if he/she is making thesame moves, and enjoy learning the choreography.
There are large numbers of studies that have supported
the learning of dance choreography. However, there havebeen no studies that have focused on supporting the practiceof dance formations. This paper proposes a training systemthat focuses on dance formations.

3. SYSTEM DESIGN
3.1 Methods of Practicing Formations
People in the normal practice of dance and look at a mirror
while practicing. Three possible approaches when dancers
are lacking in practicing formations have been enumerated
below.
• Methods of dancing by watching reference videos pre-viously created• Methods of learning the distance dances and the absence of dancers by projecting them onto a wall
• Methods of feeling as if dancers are absent using a selfpropelled screen that can move quickly like a person
We adopted methods of using a self-propelled robot that we
anticipate the strongest existence.

3.2

System Requirements

Although the related work that has been described so far
has been to primarily support the learning of choreography
and generating it, this paper proposes practicing support
with an emphasis on formation. The necessary conditions
to be included in a system to practice formations are given
below.
• The self-propelled robot moves smoothly like humans
do and moves exactly to the same position as thedancer.
The changing of formations will be disrupted if the robot
does not move smoothly. In addition, even though practice
involves learning of appropriate distances, we will obtain an
insigniﬁcant sense of distance if the robot is not able to movelike a human.
• The self-propelled robot makes it feel like one is actually dancing with a dancer.
It is necessary to share information that is obtained when
dancing with actual dancers, such as whether the choreography is synchronized or not, by providing a being that isclose to an actual dancer. The most necessary requirement
for a system is to practice formations when there is a lack ofdancers by providing the presence of a missing dancer using
a self-propelled robot.

3.3

Preliminary Experiment on Methods of Pr
acticing Formations

We evaluated which method was close to dancing withan actual dancer for each method of practicing formations(Figure 1). The subjects were nine dancers who had experience in dancing for more than three years. The subjects
learned the choreography for approximately 12 seconds thatconsisted of three times eight beats after checking a reference video. This choreography contained three elementsthat were considered to be greatly inﬂuenced by the presenceof the others.
• Intersection: Front and back intersection• Approaching: Quickly approaching to the dancer• Parallel translation: Moving while maintaining distance of side-by-side
Three experiments were carried out to compare the senseof dancing with a partner.
The dancer in Experiment 1 danced alone while looking in a mirror. We controlled OMNIKIT2010 (Figure 2)
with a wireless-controller attached to make the dancer feel
as if the missing dancer was there in Experiment 2. We
learned the appropriate distance between the missing dancer

Figure 1: Each method for practicing formations
with a dancer was termed N . The maximum number of data
for the other method was termed M . The two-dimensional
positions of subjects were termed (Xi, Y i) and (Xj ′ , Y j ′ ).
We calculated the average distances as Ave’ using the following algorithm:
Source Code 1: Calculating the average of distance

Figure 2: OMNIKIT2010

S=0;
f o r (m=0;m<3;m++){f o r ( n=0;n<3;n++){f o r ( i =0; i <N; i ++){
Min= ∞ ;f o r ( j =0; j <M; j ++){i f ( Min>| Ti−Tj | ) {Table 1: Question items
Question 1
Question 2

Question 3

Question 4
Question5

Was it close to the sense that you get when
you dance with a dancer, when you intersect?
Was it close to the sense that you get when
you dance with a dancer, when you quickly
approach to a dancer?
Was it close to the sense that you get when
you dance with a dancer, when you move
while keeping the distance of forming ranks?
Was it easy to do the choreography,
and the movement?
Was it easy to learn the sense of distance?

while checking a video of the dancer projected onto the wall
through the mirror in Experiment 3.
All subjects danced the choreography involving three times
eight beats for 12 times in total, i.e. three times each for experiments 1, 2, 3 and dancing with a dancer. We saved thelocation information obtained from a depth camera every
time we ﬁnished dancing the choreography that consisted ofthree times eight beats. We used Microsoft Corp’s Kinect
as the depth camera. The ﬁve question items (Table 1) took
into consideration three elements including the choreography. All subjects evaluated the methods in ﬁve stages. In
addition, free-comments were provided as a reference.

3.4 Results of the Experiment
Figure 3 plots the trajectories of movements for Subjects
1 and 2, Table 2 lists the results from a questionnaire, andTable 3 summarizes the average distances.The time at somepoint for the method of dancing with a dancer was termed
T i. The time at some point for the other was termed T j.
The maximum number of data for the method of dancing}

Min=| Ti−Tj | ; X’=Xj ’ ; Y’=Yj ’ ;}Di=s q r t ( ( Xi−X’ ) ˆ 2 + ( Yi−Y’ ) ˆ 2 ) ;S+=Di ;}
Ave+=S/N ;}}Ave’=Ave / ( 3 ∗ 3 ) ;We considered which method was closest to the sense andmovement of dancing with an actual dancer based on the
results obtained from an evaluation questionnaire.
For Question 1, in which the distributions of respondents
were identical between methods, we assessed the diﬀerence
between averages by using one-way analysis of variance (ANOVA). As a result, we found no signiﬁcant diﬀerences (p =0.112 > 0.05). However, the ratings were the highest when
the subjects danced with the self-propelled robot. The reasons were considered to be as follows. When dancing withthe projected video, subjects paid too much attention to
the video．Moreover, depth was hard to perceive from the
projected video. When dancing alone, it was diﬃcult for
subjects to establish the sense of distance because there wasnothing next to them to match the movements and depth
was hard to perceive from the mirror. When dancing withthe self-propelled robot, the distance between the robot andthe subjects made it easy for them to learn because they
did not have to worry about colliding with the robot. One
subject answered that it was easier to establish the senseof distance than when dancing alone. Another subject who
answered that it was easier to establish the distance dancing
alone said that they already had a clear image of intersect-Table 2: Results from a questionnaire

Alone
Robot
Projector

Question 1
Average
Variance
2.0
1.6
3.0
1.1
2.0
0.7

Question 2
Average
Variance
1.7
0.4
2.7
2.2
3.2
0.8

Question 3
Average
Variance
2.1
1.2
2.6
1.8
3.4
1.1

Question 4
Average
Variance
3.8
1.7
2.0
1.6
4.1
0.8

Question 5
Average
Variance
1.4
0.2
2.6
1.1
2.9
0.3

Table 3: The average of the distance between method of dancing with a dancer in each method
The subjects number
1
2
3
4
5
6
7
8
9
Dancing alone
207 185 353 415 363 320 570 337 281
Dancing with a self-propelled robot 202 203 469 360 278 196 481 257 317
254 218 532 552 306 256 280 237 469
Dancing with a projected video

ing, making it easy to imagine where the other dancer was
while dancing alone.
For Question 2, in which the respondents were identically
distributed between methods, we assessed the diﬀerence between the averages by using ANOVA. As a result, there wasa signiﬁcant diﬀerence (p = 0.026 < 0.05). Moreover, we assessed the diﬀerence between the averages for the methods
by using a Scheﬀe test. As a result, there were signiﬁcantdiﬀerences between when subjects danced alone and when
they danced with the projected video (p = 0.028 < 0.05). In
conclusion, the method of dancing with the projected video
obtained a closer sense of dancing with an actual dancer thanthe method of dancing alone. Additionally, the rating was
the highest as a number when subjects danced with the projected video. The reasons were considered to be as follows.
When subjects moved laterally, depth did not matter so they
tried to match the projected video through the mirror. It
was easy for them to retain the appropriate distance because
they did not have to worry about the colliding with anotherdancer or the robot. When dancing with the self-propelled
robot, it could only move at a constant speed making therobot move irregularly. Because of that and the robot not
being tall enough, the subjects had to pay attention to theirfeet to avoid stepping on the robot. However, variance washigh when subjects danced with the self-propelled robot.
Therefore, some subjects awarded them more points. The
reasons were that the subject felt like he/she was actually
dancing with another dancer because the robot actually approaching them made them aware of the distance between
them. The diﬀerence in the evaluations appeared because
the robot sometimes moved smoothly since it was manipulated with a controller. When dancing alone, subjects were
not able to learn the sense of being approached because
there was nothing coming up to them. Also, subjects did
not know how far they should move. For these reasons, theratings were low.
For Question 3, in which the distributions of the respondents were identical for the methods, we assessed the difference between the averages by using ANOVA. As a result,
there were no signiﬁcant diﬀerences (p = 0.090 > 0.05).
However, the ratings were also the highest when subjects
danced with the projected video. This occurred for the same
reasons as those in Question 2. When dancing alone, the ratings lowered because of the reasons mentioned in Question
2 and because the subjects paid less attention to theirs footsteps without the other dancer. However, Question 3 was

rated higher than Question 2. This is because that whileintersection and approaching were big movements paralleltranslation was a smaller movement and the subjects felt
more anxious when they took big steps alone. When dancing with the self-propelled robot, the robot was there as areference for dancing, so there was little diﬀerence between
parallel translation and approaching. As a result, the ratingsfor Question 3 were roughly equivalent to those for Question
2.
For Question 4, in which the distributions of respondents
were identical for the methods, we assessed the diﬀerence between the averages by using ANOVA. As a result, there wasa signiﬁcant diﬀerence (p = 0.003 < 0.05). Moreover, we
assessed the diﬀerence between the averages for the methods by using a Scheﬀe test. As a result, there were signiﬁcant diﬀerences between when subjects danced with theself-propelled robot and when they danced with the projected video (p = 0.024 < 0.05). In conclusion, the methodof dancing with the projected video obtained a closer senseof dancing with an actual dancer than the method of dancing with the self-propelled robot. The ratings were also thehighest as a number when the subjects danced with the projected video. The ratings were diﬀerent for the method using
a projector and dancing alone even though they were similarin that neither had anything next to the subject. The reason
for this is that it was easier to dance because the subject wasable to establish the sense of distance throughout the experiment as mentioned in Question 5. Moreover, it was easy forthe subjects to dance and to gain the sense of dancing with adancer because they could check the projected video throughthe mirror. When dancing alone, the variance in evaluations
was high and the evaluations were polarized. Subjects who
evaluated it highly strongly thought that they were able to
make big movements and engage in choreography because
they did not have to worry about collisions. Subjects who
evaluated it lowly strongly thought that it was diﬃcult forthem to learn the choreography because they did not havea role model.
For Question 5, in which the distributions of the respondents were identical for the methods, we assessed the difference between the averages by using ANOVA. As a result,
there were signiﬁcant diﬀerences (p = 0.002 < 0.05). Moreover, we assessed the diﬀerence between the averages in forthe methods by using a Scheﬀe test. As a result, there weresigniﬁcant diﬀerences when subjects danced alone and when
they danced with the projected video (p = 0.006 < 0.05). In

paring each items of trajectory information simultaneouslyis the most advantages. However, the timing when data arecollected diﬀer for each trial because there must be a distribution for the intervals when data are obtained. Suchdiﬀerences will accumulate if we compare individually datacollected from two diﬀerent trails from the start. Therefore, we regard the average distance between the positionof a model dancer on a time point and the position of atrial dancer on the nearest time point as the distance intime because we can minimize the error due to accumulation. The average distance data are calculated by comparing
trajectory information where subjects danced with a dancer
and trajectory information where they danced with anothermethod for all combinations of three trials for the former
and three trials for the latter. We could obtain nine distance data for each timing since the combination becomesnine, and we calculated the average for these nine data. Subjects whose average distances were the shortest in dancing
with the self-propelled robot were observed for four of thenine subjects from of these experiments. Dancing alone wasobserved for three subjects, and Dancing with the projectedvideo was observed for two subjects. Subjects whose average distances were the longest in dancing with the projectedvideo were observed for ﬁve of the nine subjects. Dancing
alone was observed for four subjects. The results revealed
that the trajectory information on dancing with the selfpropelled robot was the closest to the trajectory informationof dancing with a dancer. However, we could not determine
which method was best only from the results, i.e., closest to
dancing with a human dancer.
When subjects danced with the self-propelled robot, itmeans that they were the most accurate in actual positioning. The reasons were considered to be as follows. There
was nothing next to the subjects when they were dancing
alone or dancing with the projected video. However, the selfpropelled robot was next to the subjects when they were
dancing with it. Therefore, subjects obtained appropriate
trajectory information because they could use the robot asa marker. Moreover, the subjects’movements were restrictedunconsciously so that they did not collide with the robot as
they stated in their responses to Question 2.
Figure 3: Trajectories of movements of Subject 1 and 2

conclusion, the method of dancing with the projected video
provided a closer sense of dancing with an actual dancer
than the method of dancing alone. Additionally, the ratings
were the highest when subjects danced with the projectedvideo. It was easy for subjects to established the sense of
distance throughout in the case that the evaluations of Questions 2 and 3 were high even if it was diﬃcult for them toestablished the sense of distance. The ratings were low when
they danced alone. It was diﬃcult for them to establishedthe sense of distance because there were no references forthe dancer. When dancing with the self-propelled robot,
the evaluations were polarized reﬂecting the evaluation results in Questions 2 and 3.
These results from Questions 2, 4, and 5 for which there
were signiﬁcant diﬀerences indicated that dancing with the
projected video was the closest to dancing with a dancer.
Next, we investigated the trajectory information. Com-

3.5

System Functions

The results from the questionnaire suggested that dancing
with the projected video gave the subjects a close impression of similarity in dancing with a dancer. The results fromthe trajectories suggest that the trajectory information from
dancing with the self-propelled robot was the closest to the
trajectory information from dancing with a dancer in reproducing the positions. Based on these facts, we propose
the following system that meets the requirements describedin Section 3.2. We summarized the proposed methods thatmeet the system requirements below.
First, the self-propelled robot could be controlled in small
movements and was able to move in eight directions to movesmoothly like a human being.
Second, the real-time position of the dancer could be obtained and corrected by Kinect to accurately reproduce thepositions and the movements of the dancers. The directionof the robot was necessary to correct the position. We usedthe depth information from the screen to establish the direction of the robot. Moreover, we established the extent ofshaking from the screen by using the six-axis motion sensor.

Figure 5: System appearance
Figure 4: System Conﬁgurationfrom dancing formations with a self-propelled screen in formations with absent dancers by using the system.
Finally, the self-propelled robot had to be equipped witha screen to project the image of the dancer taken previously
to make it feel like the dancer was actually dancing with apartner. The screen had to face the same direction because
the video was accurately projected onto the screen. Therefore, we corrected the direction using depth information.
By implementing a system that met these requirements,
we propose a system that will make it possible to practiceformations when there is a lack of dancers.
There is an overview of the design of the proposed system
in Figure 4. The trajectory information and the choreography for the dancer as a reference were extracted and savedin advance. The robot was programmed to move based onthese data. The robot’s direction and position was correctedby obtaining the direction information of the initial position when the dancers were practicing, making it possible
to practice formations when there was a lack of dancers. An
overview of the procedure for the entire system is as follows.
(1) The choreography of a model dancer is captured onvideo. Simultaneously, the trajectory information ofa dancer practicing is obtained with Kinect and saved.
(2) The self-propelled screen moves matching the video playback.
The position and direction are corrected based on theinformation obtained for these. Thus, the self-propelledscreen is able to move to the designated position and toproject the image onto the screen accurately even if the selfpropelled screen deviates from the ordered movement because of changes in the environment. The sense of dancing
with another dancer is obtained by looking at the video projected onto the screen of the model dancer and checking thechoreography through the mirror. We will be able to obtaina feeling of being close to the presence of another dancerfrom the movements of the self-propelled robot based on
the trajectory information of the dancer. The program for
movement is adjusted so correction is minimized. The commands are sent to a small computer installed on the robot
by using socket transmission.
We will be able learn the appropriate senses of dancing

4.

IMPLEMENTATION

There is a photographer of the system being used in Figure 5. The OMNIKIT2010 made by Tosadenshi is a selfpropelled robot with a screen and a small PC (VGN-UX90PS,
Sony). The screen has a door to prevent the wind resistance.
The main PC that calculates the three-dimensional position
is connected to a depth camera (Kinect, Microsoft), a projector (MP522ST, BenQ), and a speaker (Computer MusicMonitor, BOSE).
We developed the application using C ♯, and the development environment was the Microsoft Visual Studio 2010. We
will explain the functions of this application shortly. First,
we start socket transmission between the small PC and main
PC and serial communication between the six-axis motion
sensor and main PC. Second, the application screen displays
a distance from the self-propelled screen to the destination,
the current progress of the program, the X value of the destination, and the Z value of the destination in real time.
The main PC calculates the S slope of the screen as the
X-axis is a plane parallel to Kinect obtained with Equation(1). The self-propelled screen rotates so that S is close to
zero. Therefore, the screen is parallel with Kinect. The
depth of the left end toward the screen is termed Z1 . The
depth of the right end toward the screen is termed Z2 . The
horizontal width of these two points is termed X12 .
Z2 − Z1(1)
X12
Furthermore, the system returns the self-propelled screen
to the destination automatically depending on the distance
between the destination and the current location.
S=

5.

EXPERIMENT ON SELF-PRO PELLEDSCREEN

5.1

Experimental Procedure

Next, we investigated whether or not the self-propelledscreen obtained the advantages of these methods. The subjects were nine dancers who had experience dancing for more

Table 4: Question items
Question 1
Question 2

Table 6: The average of the distance for 9 subjects
Experiment number
1
2
3
Approaching
109 63 128
Parallel translation 103 82 150
intersection
119 86 111

Was it close to the senseof dancing with a dancer?
Was it easy to learn the sense of distance?

Table 5: Results of the questionnaire
Approaching
Experiment number
Question 1
Question 2

1
1.7
2.4

2
3.5
3.4

3
3.1
3.3

Parallel
Translation
12
3
1.8 2.9 3.1
2.7 2.9 3.4

Intersection
1
2
2.8

2
3.3
3.5

3
3.1
3.7

than three years. They learned three simple choreographiesof approximately 8 seconds that consisted of two times eightbeats after checking the reference video. These three simple
choreographies corresponded to each of the three elements
of intersection, approaching, and parallel translation, mentioned in Section 3.3. By investigating the three choreographies that contained each of the elements between methods,
we could see the diﬀerence in eﬀects for the methods in eachof the elements.
The subjects in Experiment 1 danced with OMNIKIT2010
controlled by an attached wireless controller. The subjects
in Experiment 2 danced while checking the video of a dancer
projected onto the wall through the mirror. The subjects inExperiment 3 danced with the self-pro pelled screen thatprojected the video of the dancer. We considered that thesense of dancing was closest to the sense of dancing with adancer because subjects could check the video of the dancer,
and the trajectory information of dancing was close to the
trajectory information of dancing with a dancer because subjects danced with the robot as a marker.
All subjects danced the two times eight beats choreography for 36 times in total, 3 times each for experiments 1, 2,
3 and dancing with a dancer. We saved the location information obtained from Kinect every time we ﬁnished dancing
in the choreography that consisted of two times eight beats.
There were two question items (Table 4). All subjects evaluated the methods in ﬁve stages.

5.2 Results from Experiment
Table 5 summarized the results from the questionnaire.
Table 6 lists the average distances between the method of
dancing with a dancer in each method in approaching forthe nine subjects. It also lists the average of the maximum
width between the method of dancing with a dancer in eachmethod in parallel translation for the nine subjects and theaverage maximum longitudinal width between the methodof dancing with a dancer in each method in intersection for
the nine subjects.
We considered whether or not the self-propelled screen
would obtain the advantages of these methods based on the
results obtained from the evaluation questionnaire.
We found for both Questions 1 and 2 that the evaluationof dancing with the self-propelled screen was close to theevaluation of dancing with the projected video. As a result,the self-propelled screen was able to obtain the advantagesof dancing with the projected video for approaching, paralleltranslation, and intersection.
Next, the trajectory information needs to be considered.

For Experiment 1∼3, the average value for dancing with theself-propelled screen was the highest of the methods. As
a result, the self-propelled screen was not able to obtain
the advantages of dancing with the self-propelled robot for
approaching, parallel translation, and intersection. Furthermore, the values for dancing with the projected video was
the lowest for all experiments, i.e., the position information
on dancing with the projected video was closest to the position information on dancing with a dancer. The reasonsfor this were considered to be as follows. The choreography
for this experiment was simpler and shorter than it was in
the preliminary experiment for the methods of practicing
formations. Therefore, the self-pro pelled screen was notable to play the role of a marker. We considered that the
self-propelled screen would have been able to do it and obtain the advantages if the choreography had been longer andmore complex.
In conclusion, the self-propelled screen was able to obtain
the advantages of the projected video. However, it was notable to obtain the advantages of the self-propelled robot.
The reasons for this were considered to be as follows. The
self-propelled screen made it easier to do irregular motions
by shaking the screen. Therefore, it gave subjects a greater
sense of presence than necessary. Further, as the outer frameof the screen was made of aluminum, their fear of collision
was stronger than that with the other methods because there
was a possibility of injury. Furthermore, reliability withthe self-propelled screen was low because subjects did notpractice with it repeatedly. Therefore, they were not able
to move with complete freedom.
The improvements we need to make to the methods are
considered to be as follows. The self-propelled screen should
move more accurately. We need to reduce the risk of the collision by attaching a buﬀer material to the screen to prevent
injuries. Dancers should get used to dancing with the selfpropelled screen by practicing with it repeatedly.

6.

CONCLUSION

We proposed a system for practicing formations in dance
using a self-propelled screen instead of an actual dancer. By
dancing with a self-propelled screen, we were able to makepeople feel like they were dancing with the missing dancer.
We investigated which method for practicing formations was
closest to dancing with an actual dancer, and found that the
distance between a dancer and his/her partner in a formation dance changed according to the methods used. The
results from our experiments clearly revealed that dancing
with a projected video in terms of human feelings, and dancing with a self-propelled robot in terms of the trajectory
information were closest to the feelings and the movementsof dancing with a dancer. Therefore, our system that combines dancing with a projected video and dancing with aself-propelled screen is valid.
We intend to make a self-propelled screen in the futurethat has excellent mobility and is lighter to make the movements of the self-propelled screen smoother. Moreover, using
other sensors must be considered to obtain the appropriate
trajectory information. More appropriate corrections need
to be carried out for various situations to corrects positions
and directions. When we have optimized the self-propelledscreen, we intend to support the practice more complex formations with more than three dancers by using multiple
self-propelled screens.

7. REFERENCES[1] U. Yang and G. J. Kim: Just Follow Me: An Immersive
VR-Based Motion Tracking System, Proc. of theSeventh International Conference on Virtual Systems
and Multimedia (VSMM ’99), pp. 435–444 (1999).
[2] U. Yang and G. J. Kim: Implementation and
Evaluation of ”Just Follow Me”: an Immersive,
VR-Based, Motion Tracking System, Presence:
Teleoperators and Virtual Environments, MIT Press,
Vol. 11, No. 3, pp. 304–323 (June 2002).
[3] Y. J. Chen and Y. C. Hung: Using Real-time
Acceleration Data for Exercise Movement Training
with a Decision Tree Approach, Expert Systems with
Applications, Vol. 37, No. 12, pp. 7552–7556 (Dec.
2010).
[4] K. Shinozaki, et al.: Concept and Construction of a
Robot Dance System, International Journal of Virtual
Reality (IJVR), Vol. 6, No. 3, pp. 29–34 (Sep. 2007).
[5] M. Riley and C. G. Atkeson: Methods for Motion
generation and Interaction with a Humanoid Robot:
Case Studies of Dancing and Cdatching, Proc. 2000
Workshop on Interactive Robotics and Entertainmen(WIRE ’00), pp. 35–42 (2000).
[6] J. Or: A Control for a Flexible Spine Belly-Dancing
Humanoid, Journal of the International Society of
Artiﬁcial Life (ISAL), Vol. 12, No. 1, pp. 63–87 (Jan.
2006).
[7] K. Murata, et al.: A Robot Uses Its Own Microphone
to Synchronize Its Steps to Musical Beats while
Scatting and Singing, Proc. of the IEEERSJInternational Conference on Intelligent Robots andSystems (IROS’08), pp. 2459-2464 (Sep. 2008).
[8] J. L. Oliveira, et al.: Robot Dance based on Online
Automatic Rhythmic Perception, Proc. of the 3rdInternational Workshop on Intelligent Robotics(IROBOT’08) (Oct. 2008).
[9] G. Xia, et al.: Autonomous Robot Dancing Driven by
Beats and Emotions of Music, Proc. of the EleventhInternational Joint Conference on Autonomous Agentsand Multi-Agent Systems (AAMAS ’12) (June 2012).
[10] S. Nakaoka, et al.: Generating Whole Body Motions
for a Biped Humanoid Robot from Captured Human
Dances, Proc. IEEE International Conference on
Robotics and Automation (ICRA ’03), pp.3905–3910(Sep. 2003).
[11] K. Kosuge, et al.: Partner Ball-room Dance
Robot-PBDR-, SICE Journal of Control, Measurement,
and System Integration (SICE JCMSI), pp. 74–80 (Jan.
2008).
[12] A. Nakamura, et al.: Multimodal Presentation
Method for a Dance Training System, Proc. of the
SIGCHI Conference on Human Factors in Computing

Systems Extended Abstracts (CHI ’05), pp. 1685–1688(Apr. 2005).
[13] T. Shiratori, et al.: Rhythmic Motion Analysis Using
Motion Capture and Musical Information, Proc. of2003 IEEE International Conference on Multisensor
Fusion and Integration for Intelligent Systems (MFI
’03), pp. 89–94 (2011).
[14] K. Hachimura, et al.: A Prototype Dance Training
Support System with Motion Capture and Mixed
Reality Technologies, Proc. IEEE International
Workshop on Robot and Human Interactive
Communications (ROMAN 2004), pp. 217–222 (Sep.
2004).
[15] J. Chan, et al.: A Virtual Reality Dance Training
System Using Motion Capture Technology, IEEE
Transactions on Learning Technologies, Vol. 4, pp.
187–195 (Aug. 2010).
[16] R. Crivella, et al.: Training for Physical Tasks in
Virtual Environments: Tai Chi, Proc. IEEE Virtual
Reality (VR 2003), pp. 87–94 (Mar. 2003).
[17] J. Shotton, et al. Real-Time Human Pose Recognitionin Parts From Single Depth Images, Proc. of 2011 IEEE
Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR ’11) (June 2011).
[18] L. Deng, et al.: Real-Time Mocap Dance Recognition
for an Interactive Dancing Game, 2011 IEEE Computer
Society Conference on Computer Animation andVirtual Worlds (CASA ’11) Vol. 22, pp. 229–237 (Apr.
2011).