Online Dance Lesson Support System
Using Flipped Classroom
Shuhei Tsuchida1[0000−0003−1635−2257] , Daichi Shimizu1[0000−0001−8643−5440] ,Kanako Shibasaki2 , Tsutomu Terada1[0000−0003−2260−3788] , and Masahiko
Tsukamoto1[0000−0002−8662−4070]
1

Kobe University, 1-1, Rokkodai-cho, Nada-ku, Kobe, JAPANt.sway.tmpp@gmail.com
2
SPORTS ACADEMY TSUCHIURA, 1-11-5 Kawaguchi, Tsuchiura, Ibaraki,
JAPAN

Abstract. Contemporary online lessons that use web-conferencing systems suffer from several issues, mainly because instructors engage in thesame type of instruction as they do in conventional offline lessons. For example, it might be difficult for learners to grasp three-dimensional movements. Teaching through a video displayed on a screen is also limited, as
opposed to teaching through physical bodies and showing movements inperson. In addition, it is difficult for the instructors to teach one person
at a time during a lesson with many students. This study proposes anonline dance lesson support system that enables instructors to remotely
but effectively teach multiple learners. We initially focus on the framework of online dance lessons and subsequently propose a lesson style thatapplies to flipped classrooms. We aim to provide a new lesson style foron-demand lessons and real-time lessons using deep learning techniques.
Keywords: training · dance lesson · flipped classroom · online.

1

Introduction

Although the development of information technology has benefitted intellectuallabor, telecommunications, and digital content dissemination, it has reduced
the amount of physical activity in people’s daily lives and proliferated healthproblems caused by inactivity. In modern society, the lack of exercise needs tobe compensated for by incorporating sports and exercise into everyday life tomaintain normal bodily functions. Moderate exercise is also accompanied by asense of exhilaration and can improve mental health. Spaces where everyone canenjoy exercising to maintain a healthy state of mind and body, such as fitness
gyms, are crucial in modern society.
During the COVID-19 epidemic, demand for spaces where people can gather
shifted from the physical world to the virtual world. For example, the closure offitness gyms offering dance and other exercises opened opportunities for online
lessons using web conferencing systems, such as Zoom. However, current online
lessons frequently provide identical instruction to that of conventional offline

2

S. Tsuchida et al.

Fig. 1. Proposed online dance lesson applying flipped classroom.

lessons using an unmodified web conferencing system, causing diverse problems.
For example, compared to instructions that use the physical body to show movements in person, instruction via video on a PC or other display makes it difficult
to grasp three-dimensional movements and limits the methods of instruction. It
is also difficult for instructors to instruct one person at a time in a large group
lesson. Therefore, this study proposed an online dance lesson support systemthat enables instructors to remotely but effectively teach multiple learners. This
study initially focuses on the framework of online dance lessons and thereafter
proposes a lesson style that applies to flipped classrooms.
In particular, this study focuses on the flipped classroom as the basis of theframework. A flipped classroom is an educational method wherein learners work
individually before the class on basic learning such as explanatory lectures, andengage with the instructor during the class on learning that is necessary for theretention of knowledge and development of application skills such as tutorials
and project learning [2]. In conventional face-to-face and online dance lessons,“learning” (acquisition of individual movements) and “development/exercise”(acquisition and refinement of choreography with multiple movements, refinement of expressiveness to music) are usually interwoven. In our proposal, theflipped classroom structure is applied, and on-demand video lessons conduct the“learning” part of the lessons. In contrast, the “development/exercise” part is
conducted by real-time online lessons (Fig. 1). By applying the aforementioned
structure, the “learning” part, which requires repetition to understand the movements and make detailed corrections, can be studied repeatedly at the learners’
own pace. In contrast, the “development/exercise” part, which develops and expands the expressive power of learners according to their characteristics andmovements, can be conducted and explored through individual instruction andsharing with other participants. The advantage of this approach is that learners
can devote time to developmental and exploratory activities such as individual
instruction and sharing with other participants. We believe that the framework
composed of repeated “learning” - “development/exercise” lessons enables us toprovide effective lessons that develop the expressive ability of participants.

Online Dance Lesson Support System Using Flipped Classroom

2

3

Related Works

Many support systems for learning dance skills have been proposed [18]. They
vary from tactile [3,15] and auditory [9,14,23] systems to video presentations [1,8, 12], use of mobile robots [11, 16, 22], gamification [4, 20], etc. In this study,
we developed a dance lesson support system focusing on virtual reality(VR)technology to apply it to the flipped classroom of an online dance lesson.
Several VR-based support systems have been proposed [6, 17, 25]. Tsampounaris et al. [21] developed a system that allows users to change into different
avatars, visualize traces of the movements of various body parts, and interact
with virtual objects. Senecal et al. [19] developed a system for presenting virtual
partners in salsa dance, where practicing with a partner is important. This study
found that the motions of inexperienced participants converged with those ofskilled participants upon practicing with the system. Kasahara et al. [10] foundthat showing a slightly futuristic video from the motion information of users
made their bodies feel lighter. Various VR systems have been proposed; however, to the best of our knowledge, no systems apply flipped classrooms to an
online dance lesson.

3

Preliminary investigation

3.1

Lesson design

This study has planned and implemented an online dance lesson employing theabove framework and a dance instructor with immense teaching experience. The
lesson is summarized below. The dance lessons were held on July 4, 15, and 22,
2020, with detailed improvements. The first lesson was an on-demand video lesson of approximately 15 minutes (the participants could watch the video repeatedly, and the actual duration depended on the participants). A video lesson wasfilmed and distributed to the participants, briefly explaining and demonstrating
basic rhythms, dance steps, and simple choreographies combining them. The
instructor filmed each movement/step and provided a detailed verbal explanation of the demonstration. The participants listened to the instructor’s explanation and watched the demonstrated movements while imitating and learning thedance steps one by one. We sent the video via YouTube a few days before thereal-time lesson. Subsequently, the real-time online lesson was conducted, whichspanned approximately 45 minutes per lesson. In this lesson, all participants repeated the choreography that they had performed at the end of the on-demandlesson. The instructor commented on the quality, expressiveness, and areas for
improvement. The participants also had time to ask questions about what they
did not understand, share their impressions of the choreography in groups, andshare and discuss their impressions of the dance themselves. We designed thelessons to encourage information sharing and interaction between instructors andparticipants. The real-time lessons were conducted using the online conference
software “Zoom” (Fig. 2).

4

S. Tsuchida et al.

Fig. 2. A real-time lesson using Zoom. The participants attended from their own homes
and nearby studios.

3.2

Participants’ impressions

In the post-lesson discussions, the participants expressed their appreciation of
the framework’s value. Some participants felt that it was valuable to work repeatedly on the parts of the lesson that did not go well in the on-demand lesson.
Furthermore, they felt they could watch other participants dance in the real-time
lesson and share their impressions and awareness when dancing.
Considering the instructor’s findings and the participants’ feedback after the
preliminary investigation, we can discuss the establishment of teaching methods
tailored to individual situations in on-demand video lessons. The level of masteryof the basic steps varied greatly between participants, both in terms of type andpoints for improvement. Currently, there is no way to overcome this disparity
other than individual guidance in real-time lessons, and this shortened the time
allocated to development. The content needs to be improved to enable robust
learning of the basics. This can be done by preparing videos that include detailed
correction methods for the steps that the participants have difficulties with.

4

Proposed method

For preliminary investigation, we propose an online dance lesson style. The lessonstyle mainly consists of on-demand and real-time lessons (Fig. 3). This lessonstyle was structured considering a flipped classroom [13].
A flipped classroom is a teaching style that replaces the teaching traditionally conducted in the classroom (learning) with independent learning (development/exercise) modules assigned as homework. Usually, the learning occurs
in the class, followed by self-learning through exercises. However, in a flippedclassroom, the learning takes place through self-learning and the exercises areundertaken in the class. The advantages of this system are that learners canrepeat the learning at their own pace and have more opportunities for output;
further, the possibility of collaborative learning is increased, and instructors caneasily monitor learners’ progress. Owing to these advantages, we expected thatflipped classrooms could be applied to online dance lessons for effective teaching.

Online Dance Lesson Support System Using Flipped Classroom

5

Fig. 3. Our proposed lesson style.

The application of the flipped classroom to online dance lessons is illustrated
in Fig. 1. In conventional online lessons, real-time lessons using a web conferencing system such as Zoom are common. In the proposed method, the lessons are
divided into two categories: on-demand and real-time. In the on-demand lesson,
learners learn the basic dance steps on their own, and in the real-time lesson,
they perform dance steps that combine the basic steps learned in the on-demand
step. This lesson style allows detailed feedback to be returned to each learnerindividually, even when teaching many learners.
4.1

On-demand lesson: basic movement learning support tool

System design The preliminary investigation found that mastery of the basic steps, both in terms of type and improvement, varies between participants.
To address this, we need a system that can support participant-specific modifications in on-demand lessons. Specifically, a function that automatically scores
which parts of the learner’s body are similar to the reference movements and towhat extent, and a function that provides feedback on the scoring results, would
together solve this problem.
To achieve automatic scoring, we need to acquire the motion informationof the learner in real time. Moreover, these functions are also necessary forthe proposed system, based on related studies, such as the ability to practicewith a mirror [5], the desirability of the largest possible display when imitating
movements [7], and the ability to check movements from different angles in three
dimensions [24].
System Configuration Based on section 4.1 describing the system design,
we developed a support system for on-demand lessons. Fig. 4 shows a system
configuration. The system consists of a PC (G-Tune NM-S712SIR6SPZI), VIVE
Pro Eye, and three VIVE Trackers. In addition to these devices, our systemenables real-time motion tracking by utilizing the Final IK (Inverse Kinematics)
library.

6

S. Tsuchida et al.

Fig. 4. System configuration.

We configured the VR lesson space in Unity and implemented the automaticscoring functionality described later in Python. Specifically, the system sends
motion data from the VR lesson space to Python and automatically scores dance
motions by applying them to a learning model.
Automatic Scoring In the on-demand lessons, the learners learn dance movements using the proposed system that checks and supports the acquisition ofbasic steps, which is a fundamental part of improving one’s dancing. Specifically, we envisage the use of automatic scoring of dance steps. In basic dance
steps, each body part, for example, the hips and hands, needs to be moved accurately. Therefore, by presenting a score for each body part that is important
for achieving the basic dance steps, learners can check how accurately they are
moving their bodies, which supports self-learning.
To build the automatic scoring function, we first collected motions of basic dance steps from dancers. We adopted the motion “New Jack Swing”, abasic movement in hip-hop dance. We collected 900 movements spanning approximately two seconds each. We created a tool to visualize, edit, and scorethe collected motions using openFrameworks to create the training dataset. A
professional dance instructor subjectively scored the collected motions in threelevels. We used these data as training datasets with the correct answers labeled.
The learning model used the support vector machine (SVM) machine learning
method. The differences between frames of the joint positions of the 3D motion
were calculated for 120 frames, and we used the mean and standard deviationof the differences as a 40-dimensional feature vector. These vectors were divided
into parts for each region, each of which is trained using SVM. This process could
make models in which hand, waist, foot, and overall scores can be calculated on
three levels. The classification accuracy of the scores was approximately 89% for
the entire body, 84% for the feet, 89% for the hands, and 94% for the waist.
User flow This section describes the procedure for using the system from theuser’s perspective. The system utilization procedure is shown in Fig. 5. First,

Online Dance Lesson Support System Using Flipped Classroom

7

Fig. 5. User flow.

the user wears a headset and puts on trackers. Thereafter, the user enters the
VR space. The user can move freely in the space by pointing to the position
they want to move to while pressing a button on the controller. By pressing
another button on the controller, the user can watch an instructional video,
such as the one employed in the preliminary investigation. By practicing while
watching this video, the user can first receive the same level of practice as the
on-demand lessons in the preliminary investigation. Furthermore, the user canmove on to practice using the automatic scoring function. Pressing the trigger on
the controller starts the playback of the musical piece, and the reference avatar
starts dancing. The user dances to the avatar’s dance and watches the scoringchange in real time. The user can refer to this score to understand which bodyparts are not being imitated accurately. In addition to the scoring display, lowscoring areas are highlighted with arrows so that the user can visually see whichparts to focus on. Other advantages of VR include, for example, the possibility
to place a large mirror in the open air and concentrate on practice.
4.2

Real-time lesson

The real-time lessons corresponding to the “development/exercise” in the flippedclassroom focus mainly on the communication between the instructor and thelearners. Specifically, the lessons will be conducted using an online conferencing
system, such as Zoom, with the following flow: First, the learner demonstrates thebasic steps learned in the on-demand lesson. Thereafter, the instructor instructs
the learners one by one based on those basic steps. At this point, the instructor
also introduces a longer choreography combining the basic steps. Comments arethen shared among the participants. Finally, the instructor provides instructions
on the following task, and learners return to the on-demand lesson. We believethat learners will learn more advanced steps by repeating this lesson style.

8

S. Tsuchida et al.

Currently, we have conducted our proposed lesson with one beginner using
the proposed system. The instructor commented that the students had alreadylearned the basic steps, and could smoothly move on to the advanced steps.
They also commented the advantage of using the proposed system before thereal-time lesson was that the instructor could communicate with the learnerwith their concerns resolved. The learner commented that they enjoyed learning
as if they were in a room by themselves in the on-demand lesson. They could also
receive detailed explanations of any difficulties they encountered in the real-time
lesson. This feedback indicated that the proposed lesson style could be effective
for online dance lessons. In the future, we would like to conduct further researchwith more participants and clarify the effectiveness of our proposed lesson style.

5

Summary and future work

In this study, we proposed an online dance lesson support system that enabled
instructors to remotely but effectively teach multiple learners. We first focusedon the framework of online dance lessons and subsequently proposed a lessonstyle that applied to flipped classrooms.
Structuring “development/exercise” content in real-time lessons remains challenging. Currently, the content is being closely examined based on the rich teaching experience of the lecturers. Further refinement of the content into more effective content based on clear theory remains to be achieved. In the future, we aim
to construct more effective lessons for fostering expressive ability, considering
the interaction between on-demand lessons and real-time lessons.
When building the proposed system, motion-capturing devices were selectedconsidering device availability and accuracy. The minimum number of tracking devices required for learning dance steps has not been studied so far, andwe could share the knowledge (six-point tracking) to help build a dance-steplearning support system in the future. We developed the proposed system on
SteamVR, which can be released as an application.
Recently, VRChat has been widely used, and many users enjoy dancing on
VRChat. We can use our proposed system to develop lessons for such users.
There is a new type of user in VRChat that differs from those who attend
traditional studio lessons. For example, we expect that many people would liketo learn dancing but are embarrassed to attend a studio lesson because of thepresence of many people. Our proposed system and lesson style allow VRChat
users to take dance lessons and could be established as a new business modelusing machine learning techniques. Our system could also be used for purposes
other than dance lessons, including rehabilitation and weightlifting.

Acknowledgements This work was supported by Mitou Foundation, and JST,
CREST Grant Number JPMJCR18A3, Japan.

Online Dance Lesson Support System Using Flipped Classroom

9

References
1. Anderson, F., Grossman, T., Matejka, J., Fitzmaurice, G.: Youmove: Enhancing
movement training with an augmented reality mirror. In: Proceedings of the 26th
Annual ACM Symposium on User Interface Software and Technology, UIST ’13.
p. 311–320 (2013)
2. Bergmann, J., Aaron, S.: Flipped Learning: Gateway to Student Engagement. International Society for Technology in Education (2014)
3. Camarillo-Abad, H., Sánchez, A., Starostenko, O., Sandoval Esquivel, M.: A basic
tactile language to support leader-follower dancing. Journal of Intelligent & Fuzzy
Systems 36, 5011–5022 (2019)
4. Charbonneau, E., Miller, A., LaViola, J.J.: Teach me to dance: Exploring player
experience and performance in full body dance games. In: Proceedings of the 8thInternational Conference on Advances in Computer Entertainment Technology,
ACE ’2011. p. 8 (2011)
5. Dearborn, K., Ross, R.: Dance learning and the mirror: Comparison study of dancephrase learning with and without mirrors. Journal of Dance Education 6, 109–115(2006)
6. Eaves, D., Breslin, G., Schaik, P., Robinson, E., Spears, I.: The short-term effects ofreal-time virtual reality feedback on motor learning in dance. Journal of Presence
20, 62–77 (02 2011)
7. Elsayed, H., Hoffmann, P., Günther, S., Schmitz, M., Weigel, M., Mühlhäuser, M.,
Müller, F.: CameraReady: Assessing the Influence of Display Types and Visualizations on Posture Guidance, pp. 1046–1055 (2021)
8. Fujimoto, M., Terada, T., Tsukamoto, M.: A dance training system that maps
self-images onto an instruction video. In: Proceedings of the 5th International
Conference on Advances in Computer-Human Interactions, ACHI ’12. pp. 309–314(2012)
9. Großhauser, T., Bläsing, B., Spieth, C., Hermann, T.: Wearable sensor-based realtime sonification of motion and foot pressure in dance teaching and training. Journal of the Audio Engineering Society 60(7/8), 580–589 (2012)
10. Kasahara, S., Konno, K., Owaki, R., Nishi, T., Takeshita, A., Ito, T., Kasuga,
S., Ushiba, J.: Malleable embodiment: Changing sense of embodiment by spatialtemporal deformation of virtual human body. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17. p. 6438–6448 (2017)
11. Kosuge, K., Takeda, T., Hirata, Y., Endo, M., Nomura, M., Sakai, K., Koizumi,
M., Oconogi, T.: Partner ballroom dance robot -pbdr-. SICE Journal of Control,
Measurement, and System Integration 1(1), 74–80 (2008)
12. Kyan, M., Sun, G., Li, H., Zhong, L., Muneesawang, P., Dong, N., Elder, B., Guan,
L.: An approach to ballet dance training through ms kinect and visualization ina cave virtual reality environment. Journal of ACM Transactions on Intelligent
Systems and Technology 6(2), 37 (2015)
13. Lage, M.J., Glenn, J.P., Michael, T.: Inverting the classroom: A gateway to creating
an inclusive learning environment. Journal of Economic Education 31(1), 30–43(2000)
14. Landry, S., Jeon, M.: Interactive sonification strategies for the motion and emotionof dance performances. Journal of Multimodal User Interfaces 14, 167–186 (2020)
15. Nakamura, A., Tabata, S., Ueda, T., Kiyofuji, S., Kuno, Y.: Dance training systemwith active vibro-devices and a mobile image display. In: Proceedings of the 2005
IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS ’05.
pp. 3075–3080 (2005)

10

S. Tsuchida et al.

16. Nakamura, A., Tabata, S., Ueda, T., Kiyofuji, S., Kuno, Y.: Multimodal presentation method for a dance training system. In: Extended Abstracts Proceedings ofthe 2005 Conference on Human Factors in Computing Systems, CHI EA ’05. pp.
1685–1688 (2005)
17. Philo Tan Chua, Crivella, R., Daly, B., Ning Hu, Schaaf, R., Ventura, D., Camill,T., Hodgins, J., Pausch, R.: Training for physical tasks in virtual environments: Taichi. In: Proceedings of the Proceedings of the IEEE Virtual Reality Conference,
VR ’03. pp. 87–94 (2003)
18. Raheb, K.E., Stergiou, M., Katifori, A., Ioannidis, Y.: Dance interactive learning
systems: A study on interaction workflow and teaching approaches. Journal of
ACM Computing Surveys 52(3), 37 (2019)19. Senecal, S., Nijdam, N.A., Aristidou, A., Magnenat-Thalmann, N.: Salsa dance
learning evaluation and motion analysis in gamified virtual reality environment.
Journal of Multimedia Tools and Applications 79(33), 24621–24643 (2020)
20. Smith, S., Sherrington, C., Studenski, S., Schoene, D., Lord, S.: A novel dance
dance revolution (ddr) system for in-home training of stepping ability: Basic parameters of system use by older adults. Journal of Sports Medicine 45, 441–5(2009)
21. Tsampounaris, G., El Raheb, K., Katifori, V., Ioannidis, Y.: Exploring visualizations in real-time motion capture for dance education. In: Proceedings of the 20th
Pan-Hellenic Conference on Informatics, PCI ’16. p. 6 (2016)
22. Tsuchida, S., Terada, T., Tsukamoto, M.: A system for practicing formations indance performance supported by self-propelled screen. In: Proceedings of the 4th
Augmented Human International Conference, AH ’13. p. 178–185 (2013)
23. Yamaguchi, T., Kadone, H.: Supporting creative dance performance by graspingtype musical interface. In: Proceedings of the 2014 IEEE International Conferenceon Robotics and Biomimetics, ROBIO ’14. pp. 919–924 (2014)
24. Yan, S., Ding, G., Guan, Z., Sun, N., Li, H., Zhang, L.: Outsideme: Augmenting
dancer’s external self-image by using a mixed reality system. In: In Proceedingsof the 33rd Annual ACM Conference Extended Abstracts on Human Factors in
Computing Systems. pp. 965–970. CHI EA ’15 (2015)
25. Yang, U., Kim, G.J.: Implementation and evaluation of ”just follow me”: An immersive, vr-based, motion-training system. Journal of the PRESENCE: Virtualand Augmented Reality 11(3), 304–323 (2002)