## 3 Immersive Dance Training System
### System Overview  
This study developed a VR-based self-training system for dance that enables students to practice within a home setting without an instructor while simulating the feel of a real lesson. The system aids dance practitioners in understanding 3D movements and allows them to practice in an environment that mimics actual production settings, such as a stage. The VR Dance Training System consists of a VR application that provides dance training, a motion capture system that performs real-time full-body motion capture, and an evaluation server that performs automated dance evaluations. The hardware consists of a VIVE XR Elite headset, a PC for motion capture, two USB webcams, and a router for Wi-Fi connectivity (Figure 1). It is also capable of pass-through, hand tracking, and simple gesture recognition. The dance training system was developed as a standalone VR application using the Unity game engine. The PC used for motion capture was a MinisForum NAB6 with USB connectivity for multiple webcams. The motion capture PC and headset are connected to the same network via Wi-Fi, and data are sent and received in both directions using WebSocket communication (SimpleWebSocketServer Version 0.1.2 downloadable at [[https://pypi.org/project/simple-socket-server/](https://pypi.org/project/simple-socket-server/)]([https://pypi.org/project/simple-socket-server/](https://pypi.org/project/simple-socket-server/)) accessed on 10 June 2024). For dance evaluation, a dedicated server was built and designed to send dance motion data via HTTP and return the evaluation results to the system. The workflow of the system is depicted in Figure 2. The system is divided into three main phases: practice, evaluation, and reflection. The user materializes as an avatar within a VR dance studio, selects a lesson to practice from the VR panel, and transitions to the practice scene where a reference model avatar demonstrates a model dance. The user watches the model dance and imitates the moves step by step to learn the choreography and practice the dance. All the user's dances are automatically recorded and can be replayed at any time. In addition, pressing the \"Evaluate\" button will take the user to the evaluation scene, where the last dance practiced will be evaluated. The dance is assessed using the automatic evaluation model, which will be described in a subsequent section. In the reflection phase, the user can replay their dance moves and look back at their performance. The system visualizes the evaluation results, allowing the user to recognize points to improve and gain insights into their dance performance. Overall, this iterative process helps the user improve their dance skills effectively through continuous practice, evaluation, and reflection. The developed system is depicted in Figure 3. Within the virtual environment, an avatar serving as a model instructor is presented, allowing the user to engage in a dance routine alongside this avatar coach. Figure 1: System configuration of immersive dance training system. Figure 3: Dance training system in use (real scene on the **left;** virtual space scene on the **right**). Figure 2: Workflow of system. In the practice scene, it is possible to play, stop, and rewind the tune. In addition, the playback start and end positions can be specified, allowing the user to concentrate on a specific part of the choreography for repetitive practice. Playback speed can be adjusted from 0.5\(\times\) to 1.5\(\times\), allowing users to practice choreography in slow motion. The avatar is displayed in front of the user's left by default, but it can be moved to any position, allowing the user to observe it in three dimensions from any direction. Users can record and replay their own dances, which can then be replayed side by side with the model avatar for comparison as shown in Figure 4.  
In the evaluation scene, the user does a series of dance performances from the beginning to the end of the tune. The system automatically evaluates the user's dance performance as follows.