2023 27th International Conference Information Visualisation (IV)Visualization of the repetitive practice of dance
motion: Case study with multiple genres of dance
Mami Kawanishi

Shuhei Tsuchida

Takayuki itoh

Ochanomizu University
Tokyo, Japankawanishi.mami@is.ocha.ac.jp

Ochanomizu University
Tokyo, Japantsuchida.shuhei@ocha.ac.jp

Ochanomizu University
Tokyo, Japan
itot@is.ocha.ac.jp

can play the animation of user-specifed multiple motions toallow comparison and observation of the dancers’ motions.
This paper also presents the case study of motion datameasured for four different dance genres. In addition, theeffectiveness of the system is discussed by visualizing the
motions of three beginner dancers for three genres of dances.
The reminder of this paper is as follows: Section 2 describes
related work, Section 3 describes the proposed method, Section 4 shows the case study results, and Section 5 summarizes
the paper.

Abstract—With the enhancement of relatively inexpensive
human motion capture technology using cameras and infrared
sensors, the measurement of human body motion has been easierand its applications have expanded dramatically. We are using
this technology to develop a visualization system for the assistanceof dance skill improvement by visualizing differences and changes
in the motions of novice dancers while theur repetitive practiceof the same dance. This system applies a body size correction
between the dancer and the instructor, a time correction thataligns the timing of the motions, and a spatial correction thataligns the dancer’s position and body orientation. The systemthen applies a clustering process is to the body parts, andvisualizes the clustering results. Users can then select multiple
arbitrary motions and animate them to check the differences andchanges in the dancer’s motions. This paper ﬁrstly presents theprocessing ﬂow and functions of the visualization system, andthen introduces the results of the motion data of three novice
dancers in three different dance genres to verify the effectivenessof this method.
Index Terms—Visualization, Dance, Motion capture.

II. R ELATED W ORK
Developments of computer-powered assistance systems fordance lesson has been a well-discussed research topic [4]. Thissection introduces related studies on dance lesson systems andvisualization for dance lessons.
Some studies on dance support focused on the developmentof specialized hardware systems. Groβhauser et al. [5] presented a soniﬁcation system as feedback of motion and footpressure of a dancer. Nakamura et al. [9] presented a robotthat displays a reference dance motion and moves along withthe motion of a real dancer. Silveira et al. [11] applied softrobotics for the support of dance lessons.
Virtual reality (VR) and augmented reality (AR) technologies are also very helpful for the development of dance lesson
systems. Anderson et al. [1] developed “YouMove” that assists
the self-check of dancers’ own motions by the metaphor of
an augmented mirror. Chan et al. [3] presented a VR systemthat projects the motions of an instructor and captures the
movements of students. Senecal et al. [10] presented a VRbased practice system for salsa dance by displaying the danceof a virtual partner.
One of the authors also has a series of studies on dance
lesson systems, including a system for practicing formations
applying a self-propelled screen [12], a practice system forself-dance modeling [13], an online support system for ﬂippedclassroom [14], and experiments on effects on separated learning of acquiring movement skills [15].
Visualization techniques are very helpful tools for the analysis of human motion data and have been actually applied toa variety of motion capture data. Liu et al. [8] proposed a new
visualization method that represents gestures as vectors andplots them on a sphere. Keefe et al. [7] propose a method

I. I NTRODUCTION
Repeatation of the same motion is one of the most commonand effective practices for novice dancers. Here, there are
two main problems for beginners. Firstky, it is often difﬁcult
to feel the long-term progress. Also, it sometimes happen
that dancers feel the practices as boring routines. But still,dancers need to ﬁnd changes and modiﬁcations in their own
motions during the practice. In other words, identifying one’s
own changes and modiﬁcations will improve one’s dancing. A
deeper understanding of their own motions will lead to more
efﬁcient repetitive practice.
We are developing a visualization system to support dancers
in ﬁnding modiﬁcations in their unconscious motions. Specifically, the system improves the efﬁciency of repetitive dance
practice by visualizing the following points: when the dancer
shows improvement in dance, which parts of the body areprone to motion deviation, how often the dancer achieves thecorrect motion, and where the dancer’s habits tend to occur.
The procedure of this system is as follows. First, the samedance by the same dancer is measured multiple times using
a motion capture system. Then, scale, time, and position
corrections are applied to the motion data. After completingthe above processing, the system applies a clustering process
to the motion data, and visualizes the clustering results. Users

2375-0138/23/$31.00 ©2023 IEEE
DOI 10.1109/IV60283.2023.00068

362

3) Spatial correction: The acquired time-series data arecorrected for differences in standing position and body orientation. A simple afﬁne transformation (scaling and translation)is currently applied.

for interactive visualization of multiple motion data in thebiomechanical ﬁeld by applying pig chewing motion data.
A small number of studies have been focusing on visualization and visual analytics of dance motions. DanceMoves by
Arpatzoglou et al. [2] realizes the visual analytics to compare
the dance motions and visually search dance features.
As described above, there are many studies for teaching,analyzing, and comparing dance and visualization methods
for motion data. However, most of the conventional studies
compare and analyze experienced dancers and novice dancers,
and there are few studies that deeply analyze the repetitive
practice of a single dancer. Therefore, the purpose of this study
was to measure repetitive practice by a single dancer and tocompare and analyze them as time series data.

C. Clustering
Clustering is applied to the corrected motion data. Here, ourimplementation applies the clustering independently to the left
hand, right hand, left leg, right leg, and head. Each part iscomposed of the following joints in Figure 1.
Left arm: CLAVICLE LEFT, SHOULDER LEFT, ELBOW LEFT, WRIST LEFT
• Right arm: CLAVICLE RIGHT, SHOULDER RIGHT,
ELBOW RIGHT, WRIST RIGHT• Left leg: HIP LEFT, KNEE LEFT, ANKLE LEFT,
FOOT LEFT• Rightleg:
HIP RIGHT,
KNEE RIGHT,
ANKLE RIGHT, FOOT RIGHT• Head:HEAD, NOSE, EYE LEFT, EAR LEFT,EYE RIGHT, EAR RIGHT•

III. P ROCESSING F LOWThis section presents the processing ﬂow of the proposedsystem consisting of the following four steps: motion measurement, motion data correction, clustering, and visualization.
A. Motion measurement
1) Azure Kinect DK: Our current implementation uses
Azure Kinect DK to measure the dancers’ motion. It has abuilt-in distance image sensor using near-infrared light and avideo sensor (video camera), which can measure the positionsof 24 joints of human bodies. Users can archive motion
captures using Azure Kinect DK relatively easily withoutwearing special devices or markers.
We measured the repetition of the same dance by the samedancer over several repetitions (typically 10 to 20 repetitions)in this study. In addition, we measured the dance of an
instructor who has mastered exemplary movements once or
twice to visualize the difference from other dancers.
B. Motion data correction
Temporal and spatial correction for the motion data should
be applied prior to the clustering process. If dancers wantto compare their own motion data with the motion of the
instructor, body size correction between the dancer and the
instructor should also be applied prior to temporal and spatial
correction.
1) Body size correction: The difference in body size between the dancer and the instructor is compensated for. Ourcurrent implementation simply applies an afﬁne transformation.
2) Time correction: The acquired time series data may nothave the same relative time of the motion start to the recording
start time. Dynamic Time Warping (DTW) is a method for
calculating the similarity between time series data. It calculates
the similarity by ﬁnding the shortest path after calculating the
distance between points. Since each joint has position datain the XYZ coordinates, Multi-Dimensional Dynamic Time
Warping (MD-DTW) [6], which is an extension of DTW tomultiple dimensions, is used instead of ordinary DTW.

Fig. 1. Joints measured by Azure Kinect DK. 1

We experimented with the following three types of inputs
for the clustering process/The positions of the joints as vectors.
The movements between adjacent frames of the joints as
vectors.
• The angle of the joints as vectors.
•
•

363

”Waack,” ”Lock,” and ”Girls,” to verify the effectiveness ofthe proposed method for each genre of dance.
In this experiment, one of the authors choreographed all
dances and danced for approximately 10 seconds (2 x 8
minutes in length) to a metronome sound with a BPM of 110.
The dancers danced in a way that they themselves consideredcorrect from the ﬁrst to the third time, relaxed from the fourth
to the sixth time, considered only the arms correctly fromthe seventh to the ninth time, and considered only the legs
correctly from the tenth to the twelfth time.
In order to demonstrate the validity of the system, we
compared the data of the same dancer without the needfor body size correction. Therefore, in this experiment, theinstructor’s dance was not measured, but one of the dancesby the same dancer was selected and visualized as the dance
motions of a ﬁctitious instructor. In addition, we performed
visualizations using each of the three types of inputs forclustering described in Section III-C.
2) Visualization results: The clustering results for eachgenre of dance are described below. The number of clusters
was set to 5 in each case.
We assumed that for the left and right arms, the 1st to 3rd
and 7th to 9th times are stored in the upper cluster, and the 4th
to 6th and 10th to 12th times are stored in the lower cluster.
The left and right legs are assumed to be stored in the upper
cluster for the 1st to 3rd times and the 10th to 12th times, andin the lower cluster for the 4th to 9th times. We comparedthese assumptions with the actual results.

Fig. 2. A snapshot of the developed visualization system.

D. Visualization System
This system then visualizes the clustering results of the motion data. Figure 2 shows a capture of the visualization system.
Users can specify multiple dance motions by manipulating
this system to observe and compare the multiple motions. The”coloring” tab next to the ”main” tab on the left side of thewindow displays a list of the motion data.
This system also displays the clustering results described in
Section III-C on the right side of the window. The clusteringresults of the left arm, right arm, left leg, right leg, and headare displayed from the top. Users can specify the number ofclusters interactively on this panel. The ”play cluster” button
allows specifying which clusters of the motion data are played.
Or, the system can play representative dance motions of eachcluster simultaneously by pressing the ”Play Representative
Motions” button. Here, the representative dance motion is
selected based on the distance from the median of each cluster.
In the drawing areas for clustering results, the horizontal
axis represents a time series, with the leftmost cluster representing the ﬁrst dance and the rightmost cluster representing
the last dance. The horizontal lines represent the clusters,arranged in order of similarity to the instructor’s motion datafrom the top. The vertical lines in the display visualize thenumber of times the motion data is stored in each cluster.
Users can analyze their own dance in chronological orderby observing the clustering results. For example, users canobserve which habits are common in the early stages ofpractice, which habits appear suddenly and occasionally, andhow their motions change as practice progresses.

Fig. 3.
Clustering (1). (Upper-left)Jazz. (Upper-right)Waack. (Lowerleft)Lock. (Lower-right)Girls.

Figure 3 shows the results of the visualization using the
input (1) for clustering described in Section III-C.
For Jazz, only the right leg showed the visualization results
as expected by the authors. For the 7th to 12th motion, all
motions after the 7th motion were stored in the second cluster
for the left arm, whereas all motions for the right arm werestored in the fourth cluster, suggesting a large difference

IV. E XAMPLES
A. Veriﬁcation with four genres of dances
1) Measurement and visualization: The measured motion
data were visualized using the proposed system. In the ﬁrst experiment, we measured dance motions of four genres, ”Jazz,”
2 https://docs.microsoft.com/ja-jp/azure/kinect-dk/body-joints

364between the left and right arms. Waack’s results were as
expected for the left and right arms, as the seventh to ninth
motions were stored in the second to fourth clusters, and thetenth to twelfth motions were stored in the ﬁfth cluster. The
similarity was high for the left and right arms and the left
and right legs, respectively. Lock did not show the expected
results, as the 7th to 12th motions were classiﬁed into the
4th to 5th clusters for all sites. The results of the 7th to 12th
motions were highly similar for the left and right arms, andthe left and right legs, respectively. The Girls’ results wereas expected for the right leg, as the left leg was classiﬁed in
the 4th to 5th cluster for the 7th to 12th motion, while theright leg was classiﬁed in the 3rd cluster for the 11th to 12th
motion.

Fig. 5.
Clustering (3). (Upper-left)Jazz. (Upper-right)Waack. (Lowerleft)Lock. (Lower-right)Girls.

applying inputs (1) and (2). The results for Lock were different
from those while applying the inputs (1) and (2), but not as
expected for all regions. For Girls, the results for the legs were
somewhat similar to the expected results.
As a result, we determined that input (2) was the mosteffective in all three inputs. In addition, four of the six sites
while applying input (2) were arms, while all of the six sites
while applying input (3) were legs, indicating that input (3)is more effective for legs than arms. In both inputs, Lock did
not produce the expected results for all sites.
B. Visualization of practice process of beginners
Fig. 4.
Clustering (2). (Upper-left)Jazz. (Upper-right)Waack. (Lowerleft)Lock. (Lower-right)Girls.

1) Measurement: As the second experiment, we measured
the dances of novice dancers and visualized them using thepresented system. The choreographies used in the measurement were Jazz, Waack, and Girls, except Lock, for which no
sufﬁcient validity was observed in the previous experiment.
Three beginners with one to two years of dance experience
danced 12 times each for approximately 10 seconds (2 x 8
minutes in length) to a metronome at 110 BPM. In addition,
the author also measured one dance of each genre as the
instructor’s data.
2) Visualization results: The clustering results for eachdancer are introduced below. The number of clusters was set
to 5 for each dancer, and the input (2), which was validated
in Section IV-A, was used.
Figure 6 shows the visualization results for dancer 1.
All body parts had different clustering results for Jazz. In
particular, the left arm, right arm, and right leg had four to
ﬁve motions stored in the second cluster, whereas the left leg
had only one motion stored. The fact that the ﬁfth and latermotions were stored in the second to fourth clusters in all
regions suggests that the dance improved from the ﬁfth motion.
Waack showed that most of the motions were stored in thesecond cluster after the sixth time for the left arm, after the

Figure 4 shows the results of the visualization using the
input (2) for clustering described in Section III-C.
The visualization results of Jazz appeared as expected forthe right arm, left leg, and right leg. The left and right legs
showed the same results, while the left and right arms showed
different results, suggesting a large difference between thearms as applying the input (1). The results of Waack werealso as expected for the left and right arms as applying the
input (1). The results for Lock are exactly the same as applying
the input (1). For Girls, only the right arm was close to the
expected result. In addition, the difference between the arms
is larger than that of applying the input (1).
Figure 5 shows the results of the visualization using the
input (3) for clustering described in Section III-C.
Jazz’s results were somewhat close to the expected results
only for the left and right legs. In addition, the similarity
between the left and right arms was higher than applying
inputs (1) and (2). Waack did not produce the expected
results for all the regions. The similarity between the left andright arms and between the left and right legs was high asapplying inputs (1) and (2), but the similarity was lower than365

that the progress of the right arm, left leg, and right leg arehighly similar. In addition, motions of the left arm were stored
in the third cluster after the sixth motion, suggesting that thedance improved from the sixth motion.
Waack showed that the left and right arms had the mostmotions stored in the second cluster after the ﬁfth motion, theleft leg had the most motions stored in the second and third
clusters after the ﬁfth motion, and the right leg had the mostmotions stored in the ﬁfth cluster. This result suggests a highdegree of similarity between the left and right arms. Also, theresult suggests that the left arm, right arm, and left leg showed
improvement in dance from the ﬁfth motion.
Girls showed the highest number of motions stored in the
fourth cluster for all regions. In particular, the left and rightlegs are highly similar to each other since all motions after
the ﬁfth motion are stored in the fourth and ﬁfth clusters.

Fig. 6.
Result of dancer 1. (Upper-left)Jazz. (Upper-right)Waack.
(Lower)Girls.

ﬁfth time for the left and right legs, and after the ninth time
for the right arm. The results for the left and right arms weredifferent for the sixth to eighth trials, but the results for theﬁrst to ﬁfth trials and the ninth and later trials were exactly
the same.
The results of Girls for the left and right arms were different,
but the results for the ﬁrst, ﬁfth, and ninth results were exactly
the same. In particular, the results for the right arm, left leg,and right leg were identical.

Fig. 8.
Result of dancer 3. (Upper-left)Jazz. (Upper-right)Waack.
(Lower)Girls.

Figure 8 shows the visualization results for dancer 3.
Jazz showed that the left arm, right arm, and left leg had
the most motions stored in the ﬁfth cluster, while only theright leg had the most motions stored in the second cluster. In
particular, the results for the left and right arms were exactly
the same. This means that the progress of the left arm, rightArm, and left leg are highly similar.
Waack showed the highest number of motions stored in theﬁfth cluster for all parts of the body, resulting in a high degreeof similarity.
Similar to Waack, Girls showed the highest number ofmotions stored in the ﬁfth cluster for all the regions, resulting
in a high degree of similarity.
3) Discussion: This section discusses each subject and eachgenre based on the above visualization results.
For Jazz, dancer 1 showed improvement in dance from theﬁfth time, while dancers 2 and 3 stored more motions in theﬁfth cluster, which was signiﬁcantly different between dancer
1 and dancers 2 and 3. On the other hand, the left arm ofFig. 7.
Result of dancer 2. (Upper-left)Jazz. (Upper-right)Waack.
(Lower)Girls.

Figure 7 shows the visualization results for dancer 2.
In the Jazz data, the left arm was stored frequently in thethird cluster, while the right arm, left leg, and right leg motions
were stored most frequently in the ﬁfth cluster. This indicates366dancer 2 showed improvement in dancing from the sixth time,
and this timing is close to the timing of the improvementof dancer 1. This result suggests that Jazz tends to show
improvement from the ﬁfth to the sixth time. In addition, theresults showed that the left-right difference in arms was large
for Jazz, but this result was contradicted only for dancer 3.
Waack showed improvement from the ﬁfth to the sixth
session for dancer 1 and from the ﬁfth session for dancer 2,
suggesting that Waack, like Jazz, tends to show improvementfrom the ﬁfth to sixth session. In addition, the results in Section
IV-A showed that Waack had a high similarity between the twoarms and between the two legs, but the right arm and rightleg of dancers 1 and 2 contradicted this result. This suggeststhat the dancer 1 had more difﬁculty with the right arm andthe dancer 2 with the right leg than with the other parts.
The results for the Girls were more similar between thedancers than for the other genres, with more motions stored
in the fourth and ﬁfth clusters for all subjects. This suggeststhat the clustering accuracy is lower for Girls than for theother genres, since the input (2) in Section IV-A produced the
expected results only for the right arm. But even, the motionsof the dancers 1 and 3 were clustered in the ﬁfth cluster
more frequently, while those of the dancer 2 were clustered in
the fourth cluster more frequently, indicating slight individual
differences.
While comparing the results between dancers without ﬁxing
the genre, the dancers 1 and 2 showed variations in thevisualization results for each genre, while the dancer 3 showed
high similarity in all genres. This result may suggest thatclustering is not working correctly for the dancer 3. Possible
reasons for this include inaccurate body size correction andspatial correction.

with. In addition, the clustering accuracy was low for Girls,
and the clustering did not work correctly. This could be due tothe characteristics of each genre, or to insufﬁcient correction
for body size and space. We would like to further verify theseissues and improve the system as future work.
Another issue is that it is not easy to understand thedifferences in motion data and the characteristics of eachcluster from the visualization results. We would like to add
functions such as displaying joints that differ signiﬁcantly
from the instructor’s model motion in different colors asimportant joints, and making it possible to play back thechoreography for each movement.
R EFERENCES[1] F. Anderson, T. Grossman, J. Matejka, G. Fitzmaurice, YouMove:
Enhancing Movement Training with an Augmented Reality Mirror, 26th
Annual ACM Symposium on User Interface Software and Technology,
311-320 (2013).
[2] V. Arpatzoglou, A. Kardara, A. Diehl, B. Flueckiger, S. Helmer, R.
Pajarola, DanceMoves: A Visual Analytics Tool for Dance Movement
Analysis, EuroVis 2021 Short Papers (2021).
[3] J. C. P. Chan, H. Leung, J. K. T. Tang, T. Komura, A Virtual Reality
Dance Training System Using Motion Capture Technology, Journal ofIEEE Transactions on Learning Technologies, 4(2), 187-195 (2011).
[4] K. El Raheb, M. Stergiou, A. Katifori, Y. Ioannidis, Dance Interactive
Learning Systems: A Study on Interaction Workﬂow and Teaching
Approaches. ACM Computuer Survey 52(3), 50 (2020).
[5] T. Groβhauser, B. Blasing, C. Spieth, T. Hermann, Wearable sensorbased real-time soniﬁcation of motion and foot pressure in danceteaching and training. Journal of the Audio Engineering Society, 60(7/8),
580-589 (2012).
[6] G. A. ten Holt, M. J. Reinders, E. A. Hendriks, Multi-Dimensional
Dynamic Time Warping for Gesture Recognition, Thirteenth annual
conference of the Advanced School for Computing and Imaging (2007).
[7] D. Keefe, M. Ewert, W. Ribarsky, R. Chang, Interactive CoordinatedMultiple-View Visualization of Biomechanical Motion Data, IEEE
Transactions on Visualization and Computer Graphics, 15(6), 1383-1390(2009).
[8] L. Liu, MoViz: A Visualization Tool for Comparing Motion Capture
Data Clustering Algorithms. Proceedings of the 7th International Conference on Movement and Computing (2020).
[9] A. Nakamura, S. Tabata, T. Ueda, S. Kiyofuji, Y. Kuno, Dance Training
System with Active Vibro-Devices and a Mobile Image Display, Proceedings of the 2005 IEEE/RSJ International Conference onIntelligent
Robots and Systems, 3075–3080 (2005).
[10] S. Senecal, N. A. Nijdam, A. Aristidou, N. Magnenat-Thalmann, Salsa
dance learning evaluation and motion analysis in gamiﬁed virtual reality
environment, Journal of Multimedia Tools and Applications, 79(33),24621-24643 (2020).
[11] C. A. A. Silveira, O. K. Afsar, S. F. Alaoui, Wearable Choreographer:
Designing Soft-Robotics for Dance Practice. In Designing Interactive
Systems Conference, 1581–1596 (2022).
[12] S. Tsuchida, T. Terada, M. Tsukamoto, A System for Practicing Formations in Dance Performance Supported by Self-Propelled Screen,
Proceedings of the 4th Augmented Human International Conference,
178–185 (2013).
[13] S. Tsuchida, H. Mao, H. Okamoto, Y. Suzuki, R. Kanada, T. Hori, T.
Terada, M. Tsukamoto, Dance Practice System that Shows What You
Would Look Like if You Could Master the Dance, the 8th International
Conference on Movement and Computing (2022).
[14] S. Tsuchida, D. Shimizu, K. Shibasaki, T. Terada, M. Tsukamoto, Online
Dance Lesson Support System Using Flipped Classroom, Advances in
Mobile Computing and Multimedia Intelligence (2022).
[15] S. Tsuchida, T. Terada, M. Tsukamoto, Effects on Separated Learning of
Acquiring Physical Movement Skills Classiﬁed by Level of Difﬁculty.
Advances in Mobile Computing and Multimedia Intelligence (2022).

V. C ONCLUSION
This paper introduced the visualization system developed
for the dance practice process and discussed its effectivenessby showing the results of visualization of dance motions infour different genres and the results of visualization of dance
motions by three beginning dancers. This visualization systemapplies clustering to multiple dances by the same dancer
measured with a motion capture system after applying correction as a pre-processing step, and visualizes the classiﬁcation
results regarding the differences between motions. This system
makes it possible to simultaneously play back the skeletons ofdifferent time series of dances for comparison and observation.
It is also possible to examine the effects of repetitive practiceand the differences in clustering results by region.
The visualization results of dance motions of four different
genres indicate that clustering is effective when the movementsof joints between adjacent frames are used as the input ofclustering. The results were as expected for the right arm, left
leg, and right arm of Jazz, the left arm and right arm of Waack,and the right arm of Girls.
The visualization of dance motions by the novice dancers
revealed the timing of dance improvement in each genre, as
well as the parts of the body that each subject had difﬁculty367