Authoring System for Choreography
Using Dance Motion Retrieval and Synthesis
Ryo Kakitsuka∗1
Naoya Iwamoto1

Kosetsu Tsukuda2
Masataka Goto2

Satoru Fukayama2
Shigeo Morishima1

12

Waseda University
National Institute of Advanced Industrial Science and Technology (AIST)

Abstract

1 Introduction

Dance animations of a three-dimensional (3D)
character rendered with computer graphics(CG) have been created by using motion capturesystems or 3DCG animation editing tools. Since
these methods require skills and a large amountof work from the creator, automated choreography synthesis system has been developed tomake dance motions much easier. However,those systems are not designed to reflect artists’preferred context of choreography into thesynthesized result. Therefore, supporting theuser’s preference is still important. In this paper, we propose a novel dance creation systemthat supports the user to create choreography
with their preferences. A user firstly selects apreferred motion from the database, and thenassigns it to an arbitrary section of the music.
With a few mouse clicks, our system helps auser to search for alternative dance motion thatreflects his or her preference by using relevance
feedback. The system automatically synthesizes
a continuous choreography with the constraint
condition that the motions specified by the user
are fixed. We conduct user studies and showthat users can create new dance motions easily
with their preferences by using the system.
Keywords: relevance feedback,animation, motion capture, dance∗

character

Corresponding author:kakitsuka.99821@ruri.waseda.jp

Owing to the recent development of 3DCG
tools, 3D character dance animation has been
widely used in the entertainment field; for example, in films and computer games, and evenfor live performances in the form of “virtual
popstar”, a 3DCG character animation that projected on stage. Although there is a great necessity for choreographic authoring tools for character dance animation, using 3DCG animation
editing tools still requires a large amount ofwork, skill, and dance knowledge.
In light of the above, the automatic synthesis of dance motion has been the method used
to construct a 3D character dance animation
with ease. However, fully automated synthesis [1, 2] does not allow the animators to express their preferences. There are methods forediting timing and speed of exercise in a motion
sequence [3]. Although these methods are useful to reflect a user’s preference to choreography
or to edit the dance motion more specifically, it
is still insufficient when a user wants to replacemotion in a choreography with another one ina specific part of the music. In this case, motion retrieval technique enables the animators tochoose the alternative motion which matches thepreference from the database. However, existing motion retrieval methods were insufficientfor the purpose, since they often place a large
burden on animators by requiring input motions
as queries by means of figure drawing [4, 5] ormotion capture devices such as Microsoft Kinect

CASA 2017122

sensors1 [6, 7].
The goal of this study is to provide an authoring system for creating character dance animation with users’ preferences that does not require any knowledge of choreographic composition and variations, or skills for constructing CG
animations. Our approach consists of the following two steps: 1) the user assigns a preferredmotion obtained through our motion search algorithm to an arbitrary section of the given music. 2) our system automatically assigns motion segments from the database to the remaining sections of the music so that the sequenceof motions matches the music input considering
the connectivity of the motion segments and theclimax level of music.
To implement the system, we address twotechnical issues. The first one is how to enable
a user to easily search for the preferred motion
from the motion database. The second is how toautomatically synthesize a dance sequence under the condition where the motions selected by
the user are fixed. We solve the first problem
by visually presenting motion sequences thatcan be changed based on user’s relevance feedback [8] and the diversification framework [9].
It is difficult to judge whether a dance motion
is good or not without previewing a dance inconjunction with a corresponding musical piece.
In our system, the user can view candidate sequences on the screen and find a preferred alternative motion with only a few mouse clicks. We
address the second issue filling the time between
fixed parts with reasonable dance motions automatically selected from the database. We selectthe motions by minimizing the sum of the transition cost function using Dijkstra’s algorithm.
The main contributions in this paper are as
follows.
1. We introduce a relevance feedback framework in dance motion retrieval from thedance motion capture database.
Our
method enables a user to easily search forthe preferred dance motion.
2. Our method can automatically synthesize adance sequence considering the connectivity of the motion segments and how well acandidate motion segment matches a corresponding piece of music.
3. We propose a new system for constructing character dance animation considering
the animator’s preference. Results of auser study with the system showed that participants found the system to be suitablefor easily creating new dance motions with
their preferences.

2 Related Work
This section describes previous research relating to the two technical issues referred to above,
namely, motion retrieval methods and study of
automatic dance motion synthesis.

2.1 Motion Retrieval
In the past decade, motion retrieval technologies
have been advancing owing to the increasing
availability of large-scale motion databases2,3 .
There are several ways to retrieve a desired motion sequence from a large-scale human motion
database, the simplest being the use of textual
description key-words as queries; for example,“walking”, “jumping”, or more detailed sentences such as “a kick of the left foot followed
by a punch” [10]. While this approach is intuitive and efficient, it requires a large amountof manual effort to annotate all the motion sequences in the database, and it is not sufficient for retrieving certain desired dance motion sequences, because some dance motions
do not have appropriate names. To overcome
these limitations, many researchers have studied
the content-based human motion retrieval methods [6, 7, 11]. With those approaches, the desired motion sequences are retrieved by submitting a similar motion sequence as a query using
simple and easy motion capture devices such as
Microsoft Kinect sensors or the Perception Neuron4 . However, it is still difficult to acquire theappropriate motion sequences as a query, especially when a user wishes to retrieve a dance motion sequence, because these often include complex or difficult motions and it is necessary to2

http://mocap.cs.cmu.edu
http://smile.uta.edu/hmd/4
https://neuronmocap.com
3

1

www.xbox.com/en-US/kinect

CASA 2017123

be able to perform such movements. To acquire
the appropriate motion sequences as a query,a puppet interface [12] and sketch-based interfaces [4, 5, 13] have been proposed. However,submitting a motion sequence by using the puppet interface or sketching a person’s movement
is not easy for a general user.
Moreover, these approaches are premised on
the clarity of user’s information desire. They
cannot accommodate unclear requests of a user,
such as searching for a dance motion sequenceby matching an expected image to a piece ofmusic. Exploratory search [14] is effective foraccommodating such unclear requests. The exploratory search system [15, 16] needs to be designed so that it is tailored to a purpose and targeted users. For example, Bernard et al. [15]presented an exploratory search system in close
collaboration with the targeted users, who wereresearchers working on human motion synthesis
and analysis, including a summative field study.
However, an exploratory dance motion searchsystem for constructing character dance animation matching input music has not yet been established. Our dance motion search system enables the user to search a wide variety of motion
sequences without in-depth knowledge of dance.

2.2 Dance Motion Synthesis
The mainstream of generating dance motion
from input music is uncovering the relationships between existing dances and their accompanying music. Musical contents such as
tempo [17], acoustic beat features [1, 18], pitchand chord [19], and melodic contour [20], havebeen used to analyze these relationships, in addition to combinations of acoustic features [21]and structural similarity [22]. These approaches
construct a dance sequence by concatenating
segments of existing dances. A method to generate dance choreography automatically using machine learning is introduced [2] and it enables
various dance motions that are not included inexisting motion databases to be generated.
However, in our proposed system, these methods cannot automatically synthesize a sequence
under the constraint condition that the motions
chosen by the user are fixed. The dance synthesis method proposed by Xu et al. [23] has
the potential to overcome the problem described

Figure 1. Overview of the system
above. The authors applied Motion Graphs [24]to motion synthesis for synchronization withstreaming music. Motion Graphs is a method
for creating realistic, controllable motions using the structural expression of relations amongsmall bits of motion. The dance synthesis
method introduced Synchronization Capacity toensure that the duration of the generated motion
matches with the duration of the input music.
However, this constraint often causes the same
movement to be generated repeatedly, becauseof a transitable loop of nodes permanently being passed. We define the transition cost function between motion segments, considering theconnectivity of motion segments and how wella candidate motion segment matches a corresponding piece of music. Then, the filling dance
motions are selected by minimizing the transition cost function using Dijkstra’s algorithm. In
this way, our method automatically synthesizes
a dance sequence without a loop of nodes being made, under the constraint condition that the
motions selected by the user are fixed.

3 Choreographic Authoring
SystemFig. 1 provides an overview of the system.
Firstly, the user of the system assigns a preferredmotion to any part of the input music timeline
1(Fig. 1, ⃝).
Dance motions that correspond toother parts of the music are then synthesized

CASA 2017124

automatically, so that the movements are con2
nected naturally (Fig. 1, ⃝).
Secondly, the user
selects the parts he or she wishes to fix or change
3
in a generated sequence of dance (Fig. 1, ⃝).
The part the user wishes to change is then replaced with other preferred motions from the4
database (Fig. 1, ⃝).
Dance motions that correspond to other parts in the music are updated
5
automatically (Fig. 1, ⃝).
Finally, these steps
are repeated in order to compose a new dance
motion considering the user’s preference.
There are two technical challenges to realizing the proposed system, as described in Section 1. We describe the solution in Sections 4
and 5. Motion segments in the database are
obtained by separating existing dance motions5
at every four beats. Although automatic dance
motion segmentation methods based on choreographic primitives are proposed [25], we do not
use them owing to insufficient accuracy of them.

4 Dance Search System Using
Relevance Feedback

http://www.perfume-global.com/

mn+1 = argmax{ρ · rel(q, m)m∈U \Sn(1)+ (1 − ρ) · Φ(m, Sn , Ll )}.
Sn is the set of motion segments already recommended by the system and Ll is the set of motion
segments the user likes. U is the universal set of
motion segments in the database. ρ (0 ≤ ρ ≤ 1)
is the parameter that controls the tradeoff between rel(q, m) and Φ(m, Sn , Ll ). rel(q, m)
represents the relation between the candidate
motion segment m and the corresponding pieceof music q. We use the root mean square (RMS)
mean of each musical bar as the music featureand the motion intensity (as defined by Eq. (2))
mean of each motion segment as the motion feature. We define the motion intensity W as thelinear sum of the approximated instantaneous
speed calculated from the position of the joints:∑
W (f ) =αi · ∥ẋi ∥(2)i

Our approach described in Sections 4 and 5 isconstructed under the following two hypotheses built by Shiratori et al. [1]: 1) the rhythmof dance motions is synchronized to that of music. 2) the intensity of dance motions is synchronized to that of music.
In this section, we describe how our systemenables users to easily search for preferred motions from the motion database. For most users,
it is difficult to judge whether a dance motion isgood or not without previewing it with a corresponding musical piece. In our system, theuser can view candidate sequences on a screen
and simply choose the preferred one. Our system enables the user to re-retrieve motion datausing relevance feedback [8]. During the earlyphases of feedback, our system expands the variety of candidates, and gradually converges as
the number of re-retrievals increases. We utilize the diversification framework proposed by
Dou et al. [9] to select candidates with simultaneous consideration of a candidate’s relevanceto the corresponding piece of music and motions
already selected. The n + 1-th motion segment5

mn+1 is given by

where αi is the regularization factor for the ith joint. ∥ẋi ∥ is the Euclidean norm of theapproximated instantaneous velocity of the i-th
joint. These regularization parameters depend
on which parts we recognize as being importantfor dance expression. Φ(m, Sn , Ll ) controls thevariety of candidates. We define Φ(m, Sn , Ll )as follows:Φ(m,Sn , Ll ) = τ · min{D(m, mi )|mi ∈ Sn }+ (1 − τ ) · max{Sim(m, mj )|mj ∈ Ll }(3)
where D(m, mi ) and Sim(m, mi ) represent thedissimilarity and the similarity between m andmi , respectively. Sim(m, mi ) is calculated using the method proposed by Wang et al. [7].
D(m, mi ) is given by D(m, mi ) = 1 −Sim(m, mi ). τ is the parameter that controls thediversity of candidates and is given by{τ0 = 1(4)
τk = 1920 τk−1 (k = 1, 2, · · ·)where k is the number of feedback instances.
As the number of re-retrievals increases, τ decreases, thereby contracting the variety of candidates.

CASA 2017125

Cmusic (q, B). Cconnect (A, B) analyzes the connectivity from motion segments A to B, while
Cmusic (q, B) analyzes how well the intensity ofthe motion segment B matches that of the corresponding piece of music q. These evaluation
functions are described in detail in Sections 5.1
and 5.2.

5.1 Evaluating the Climax Level of Music

Figure 2. Automatic synthesis of a dance sequence using Dijkstra’s algorithm.
The red bold lines show the route minimizing the sum of the cost functions.

5 Automatic Synthesis of a Dance
Sequence
Our system automatically synthesizes a dance
sequence matched to input music, with the constraint condition that the motions selected by theuser are fixed. Therefore, the user is able to focus on the parts that he or she has a strong preference for of the input music timeline.
First, the user assigns a preferred motion
obtained through our dance search system described in Section 4 to the parts with strong preference. Subsequently the system assigns motions from the motion database to other partsof the music by considering the connectivity of
motion segments and how well a candidate motion segment matches a given piece of music.
The goal of this is to acquire the most appropriate motion segment sequence. The rhythmof the candidate dance motion segments is synchronized to that of the input music by resizing
the motion segments. The motion segment sequence that minimizes the sum of the transition
cost function C is obtained by using Dijkstra’salgorithm (Fig. 2). The transition cost function
from the motion segments A to B is given byC(A, B) = β·Cconnect (A, B)+ (1 − β) · Cmusic (q, B).

(5)

β (0 ≤ β ≤ 1) is the parameter that controls the tradeoff between Cconnect (A, B) andIn the evaluation function Cmusic (q, B), to analyze how well the intensity of the motion segment B matches that of the corresponding pieceof music q, we use the climax level analysis ofmusic proposed by Sato et al. [26]. The authors calculate the temporal climax transition ofa piece of music As (t) using music features such
as RMS energy, ∆RMS energy, roughness, andspectral flux. They select music features from
MIR toolbox Ver. 1.3 [27].
We use the temporal climax transition, normalized to values ranging from 0 to 1.
A′s (t) =As (t)
max As (t)(6)

Then, we calculate the means of A′s (t) in eachof the music bars, and Cmusic (q, B) is given by
Cmusic (q, B) = A′s (t, q) − A′s (t, q B ) .

(7)

A′s (t, q) represents the mean of A′s (t) in themusic bar q. A′s (t, q B ) represents the mean ofA′s (t) in the original music bar q B corresponding to the motion segment B.

5.2 Connectivity Analysis of Motion
Segments
In the evaluation function Cconnect , we utilize the connectivity analysis of motion segments [1]. We consider both the posture similarity Spose and the movement similarity Smove .
The posture similarity Spose between the iA -thframe of the motion segment A and the j B -thframe of the motion segment B is defined as theangular similarity of the link direction vectors v:∑
Spose (iA , j B ) =γl · (h(vl (iA )) · h(vl (j B )))CASA 2017

l

(8)

126

Figure 3. Example of the choreographic authoring system screen
where γl is the regularization factor for the l-th
link. Through h, an input vector a is converted
to the unit vector a/|a|. The movement similarity Smove is calculated as follows:
S move (iA , j B )∑=γl · g[h(vl (j B ) − vl (iA )) · h(v̇l (iA ))]l· g[h(vl (j B ) − vl (iA )) · h(v̇l (j B ))](9)where g[x] = x if x > 0, and g[x] =0 otherwise. Here, v̇ is calculated from thecandidate motion segment. Thus, connectivity from motion segments A to B is given by
Cconnect (A, B) = 1/(Spose + Smove )2 .

5.3 Transition Motion Generation
The resulting motion sequence is obtained byconnecting the motion segment sequence using
cubic interpolation for the posture of a character. For the position of the character, we pay
attention to the position relative to the ground,to avoid effects such as sliding or floating in theair.

6 Interface
The screenshot of the implemented user interface of our system is shown in Fig. 3. The interface includes basic viewing functions, such asa window showing the dance sequence, functions to load the input music, save, play andstop/pause the generated video, and the input

Figure 4. (a) Example of interactive sequence
selection. Six different dance candidate sequences are previewed. The
upper-right and lower-right candidates
are selected by the user. (b) By clicking the search button, the dance sequence candidates are updated using
relevance feedback.

music timeline. A musical bar is used as theunit to which the user assigns a preferred motion segment.
The interface also provides the following
functions for constructing a character dance animation reflecting the user’s preferences.
Searching for a user preference dance motion segment: Fig. 4 shows how a user searches
for a preferred motion segment from the motion database. Upon clicking a musical bar towhich the user wishes to assign a motion segment, the system visually presents the user withcandidate sequences selected on the basis of thesampling method described in Section 4. The
number of candidate sequences can be changed
from four to eight at an interval of two using
the “+” and “−” buttons (the default number issix). Users can update candidate sequences reflect their preferences by clicking the search button after selecting one or more motion segments
they like. When the users find a preferred motion segment, they can assign it to the musical

CASA 2017127

Figure 6. Results of (Q1) and (Q2) for eachdance synthesis method.
Figure 5. Diversity of the candidates
bar by clicking the set button.
Generating a dance sequence matched to input music: When the generate button is clicked,
the system automatically synthesizes a continuous choreography with the constraint condition
that the motion segments are selected by theuser. The user can view the generated dance
sequence in the upper-right section of the interface.

7 Experimental Results
7.1 Diversity of Candidates
We defined the diversity of candidates
as the mean of the dissimilarities between
each pair of six sequence candidates
1 ∑5 ∑6( 15 i=1 j=i+1 D(mi , mj )). Fig. 5 portrays
the relationship between the diversity of candidates and the number of feedback instances.
As expected, the variety of candidates graduallyconverges as the number of feedback iterations
is increased. However, the diversity of candidates begins to increase after the number of
feedback iterations is 3. This is because themotion segments selected as sequence candidates previously are neglected, that is to say,m ∈ U \Sn in Eq. (1). If m ∈ U \Sn is replacedwith m ∈ U , the user cannot update the dance
sequence candidates displayed in the screen byclicking the search button after the combinationof the minimum mean of the dissimilarities
between each pair of sequence candidates isdisplayed.

7.2 Resulting Dance Animation
To check the effectiveness of our dance synthesis method in the system, we conducted a user
study. Twenty-three computer science students

participated in this study. The participants wereshown six dance animations in total.
As input, we used three pieces of music (onepiece of “Perfume globalsite sound” and twopieces of “2U Night Drive”). Each piece ofmusic involved adding two types of generated
dance. These dance animations were generated
by using motion segments from the database,with the constraint condition that the motions
specified by the user are fixed. Other than using
our method, motion segments were randomly
selected from the database. The synthesized motion sequence is obtained by connecting the motion segment sequence using cubic interpolation
as with our method. To evaluate satisfaction andutility, after watching each of the video clips, theparticipants were asked the following questions,(Q1) Did you feel satisfied with the movementsof character body? (7-points Likert scale,
7: most satisfied; 1: least satisfied)(Q2) Did you feel that the dance is matched tothe music? (7-points Likert scale, 7: mostsatisfied; 1: least satisfied)(Q3) When did you feel satisfied or dissatisfied
when watching the video? (open ended)
Fig. 6 shows the results of the questionnaire.
The results statistically proved that our method
outperforms random selection in terms of theresulting movements quality of character body
(p-value was 2.80 × 10−9 using the Wilcoxon
signed-rank test) and how well a dance matches
to input music (p-value was 3.17 × 10−7 using
the Wilcoxon signed-rank test). In addition, theresult showed that our dance synthesis method
is effective in the system (the average score of
Q1 and Q2 were 5.67 and 5.39 respectively).
On the other hand, a participant who has experience of dancing provided a negative com-

CASA 2017128

Figure 7. An example result of generated dance motion. The motions that a user selects are withinorange frames. The system automatically synthesizes a continuous choreography with theconstraint condition that the motions selected by the user are fixed. The duration of theresulting dance is 32 beats (approximately 14.77 sec) and the screenshots are taken at twobeat (approximately 0.92 sec) intervals.

ment about the dance synthesized by our method
as follows.
“ I felt slightly dissatisfied with the quality ofthe dance owing to lack of choreographic
coherence on the same music structure.”
To improve the quality of a synthesized dance,
it is effective to consider how well a choreographic structure matches to a music structure.
The dance animations generated by our system
are shown in Fig. 7.

7.3 User Feedback about the System
In this section, we discuss the early reactions ofour targeted users who are interested in seeinga dance without any knowledge of choreography and any experience of making 3D animation. The number of them was about 20 people
or more. We asked them to use our user interface or to observe authoring choreography aftersufficiently describing the purpose of our systemand how to use our interface.
Most of them felt that the computing time ofgenerating a dance sequence is short enough tooperate without causing stress to the user. On
the other hand, some others presented improvements to our interface. They felt that the timeline without visualizing the music structure andthe state of being unable to browse a historyof past searches are inconvenient. Visualizing

the input music structure estimated automatically and implementation of the search history
function are our future work. In addition, another user presented new functions to import anarbitrary character model data, stage and background, since he felt that the suitable choreography varies by character or scene.
Many of them agree with our motivation todevelop this program. Moreover, they had aninterest in the system and told us they want to
use it. Some of them felt that it is magnificent
to enable people who cannot dance to create achoreography, since only someone who has anenough skill to dance is permitted to create thechoreography, previously.

8 Conclusion and Future WorkIn this paper, we have proposed a novel systemfor choreographic authoring for character dance
animation. To consider in implementing the system, we have provided the dance search systemusing relevance feedback and automatic synthesis of a dance sequence considering the connectivity of the dance motion segments and how
well a candidate motion segment matches a corresponding piece of music. With this system,
we can create new dance performances that consider the user’s preference in the virtual space.
However, since the variety of the choreog-

CASA 2017129

raphy of the synthesized dance depends on thedance motion database, it is important to expand
the database. In addition, to improve the qualityof a resulting dance, it is effective to consider
how well a choreographic structure matches to amusic structure. Furthermore, it is important toimprove our interface in ways such as the visualization of the music structure that is estimatedautomatically [28] and the implementation of
the search history function as future works.
This paper should be regarded as being the
new initiative addressing a challenging problem
in the animation community. It can enhance thequality of dance creation for the people in needand facilitate creative activities.

Acknowledgements
Hatsune Miku is copyrighted by Cripton Future
Media INC., and we used it in our research under PIAPRO license (www.piapro.net). 3D
model of Hatsune Miku used in our research iscreated by Lat. This work was supported in partby ACCEL, JST.

References[1] Takaaki Shiratori, Atsushi Nakazawa, andKatsushi Ikeuchi. Dancing-to-music character animation. In Computer Graphics
Forum, volume 25, pages 449–458. Wiley
Online Library, 2006.
[2] Satoru Fukayama and Masataka Goto.
Automated choreography synthesis using
a gaussian process leveraging consumergenerated dance motions. In Proc. of Advances in Computer Entertainment Technology, page 23. ACM, 2014.
[3] Tomohiko Mukai and Shigeru Kuriyama.
Pose-timeline for propagating motion
edits.
In Proc. of the ACM SIGGRAPH/Eurographics Symposium on
Computer Animation, pages 113–122.
ACM, 2009.
[4] Myung Geol Choi, Kyungyong Yang,
Takeo Igarashi, Jun Mitani, and Jehee Lee.
Retrieval and visualization of human motion data via stick figures. In Computer

Graphics Forum, volume 31, pages 2057–
2065. Wiley Online Library, 2012.
[5] Jun Xiao, Zhangpeng Tang, Yinfu Feng,and Zhidong Xiao. Sketch-based humanmotion retrieval via selected 2d geometric posture descriptor. Signal Processing,
113:1–8, 2015.
[6] Michalis Raptis, Darko Kirovski, andHugues Hoppe.
Real-time classification of dance gestures from skeleton animation.
In Proc. of the ACM SIGGRAPH/Eurographics symposium on computer animation, pages 147–156. ACM,
2011.
[7] Pengjie Wang, Rynson WH Lau, Zhigeng
Pan, Jiang Wang, and Haiyu Song. An
eigen-based motion retrieval method forreal-time animation. Computers & Graphics, 38:255–267, 2014.
[8] Joseph John Rocchio. Relevance feedback
in information retrieval. 1971.
[9] Zhicheng Dou, Sha Hu, Kun Chen, Ruihua Song, and Ji-Rong Wen.
Multidimensional search result diversification.
In Proc. of ACM international conferenceon Web search and data mining, pages
475–484. ACM, 2011.
[10] Atsuo Yoshitaka and Tadao Ichikawa. A
survey on content-based retrieval for multimedia databases. IEEE Trans. on Knowledge and Data Engineering, 11(1):81–93,
1999.
[11] Meinard Müller, Tido Röder, and Michael
Clausen. Efficient content-based retrievalof motion capture data. In ACM Trans.
on Graphics, volume 24, pages 677–685.
ACM, 2005.
[12] Naoki Numaguchi, Atsushi Nakazawa,
Takaaki Shiratori, and Jessica K Hodgins. A puppet interface for retrievalof motion capture data.
In Proc. of
ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 157–
166. ACM, 2011.

CASA 2017130

[13] Min-Wen Chao, Chao-Hung Lin, Jackie
Assa, and Tong-Yee Lee. Human motion retrieval from hand-drawn sketch.
IEEE Trans. on Visualization and Computer Graphics, 18(5):729–740, 2012.
[14] Ryen W White and Resa A Roth. Exploratory search: beyond the queryresponse paradigm (synthesis lectures on
information concepts, retrieval & services). Morgan and Claypool Publishers,
3, 2009.
[15] Jürgen Bernard, Nils Wilhelm, Björn
Krüger, Thorsten May, Tobias Schreck,
and Jörn Kohlhammer. Motionexplorer:
Exploratory search in human motion capture data based on hierarchical aggregation.
IEEE Trans. on visualizationand computer graphics, 19(12):2257–
2266, 2013.
[16] Songle Chen, Zhengxing Sun, Yan Zhang,and Qian Li. Relevance feedback for human motion retrieval using a boosting approach. Multimedia Tools and Applications, 75(2):787–817, 2016.
[17] Costas Panagiotakis, Andre Holzapfel,
Damien Michel, and Antonis A Argyros.
Beat synchronous dance animation basedon visual analysis of human motion andaudio analysis of music tempo. In International Symposium on Visual Computing,pages 118–127. Springer, 2013.
[18] Jae Woo Kim, Hesham Fouad, John L Sibert, and James K Hahn. Perceptually motivated automatic dance motion generation
for music. Computer Animation and Virtual Worlds, 20(2-3):375–384, 2009.
[19] Ferda Ofli, Engin Erzin, Yücel Yemez, andA Murat Tekalp. Learn2dance: Learning statistical music-to-dance mappingsfor choreography synthesis. IEEE Trans.
on Multimedia, 14(3):747–759, 2012.
[20] Sageev Oore and Yasushi Akiyama. Learning to synthesize arm motion to music by
example. 2006.

driven conventional dance motion synthesis. IEEE Trans. on visualization and computer graphics, 18(3):501–515, 2012.
[22] Minho Lee, Kyogu Lee, and Jaeheung
Park. Music similarity-based approach togenerating dance motion sequence. Multimedia tools and applications, 62(3):895–
912, 2013.
[23] Jianfeng Xu, Koichi Takagi, and Shigeyuki
Sakazawa. Motion synthesis for synchronizing with streaming music by segmentbased search on metadata motion graphs.
In Multimedia and Expo, IEEE International Conference on, pages 1–6. IEEE,
2011.
[24] Lucas Kovar, Michael Gleicher, andFrédéric Pighin. Motion graphs. In ACM
Trans. on graphics, volume 21, pages 473–
482. ACM, 2002.
[25] Narumi Okada, Naoya Iwamoto, Tsukasa
Fukusato, and Shigeo Morishima. Dancemotion segmentation method based on
choreographic primitives.
In GRAPP,
pages 332–339, 2015.
[26] Haruki Sato, Tatsunori Hirai, Tomoyasu
Nakano, Masataka Goto, and Shigeo Morishima. A soundtrack generation system tosynchronize the climax of a video clip withmusic. In Multimedia and Expo, IEEE
International Conference on, pages 1–6.
IEEE, 2016.
[27] Olivier Lartillot and Petri Toiviainen. A
matlab toolbox for musical feature extraction from audio. In International Conference on Digital Audio Effects, pages 237–
244, 2007.
[28] Masataka Goto. A chorus section detection method for musical audio signals andits application to a music listening station. IEEE Trans. on Audio, Speech, andLanguage Processing, 14(5):1783–1794,
2006.

[21] Rukun Fan, Songhua Xu, and Weidong
Geng. Example-based automatic music-

CASA 2017131