## 3 Immersive Dance Training System
### Automatic Evaluation and Feedback  
When conducting an evaluation of a dance, determining the appropriate evaluation items is crucial. The evaluation items should be coherent and independent of each other, not merely numerous. Furthermore, the content must be easily understandable to the users being evaluated. The evaluation items were determined based on the findings of previous studies and the opinions of experts obtained through data collection. The Laban body movement analysis theory proposed four Effort attributes--weight, space, time, and flow--to capture movement intensity, rhythm, direction, and continuity. In a previous study on emotion recognition and interaction in dance physical expression [58], factor analysis was applied to the impressions received from dance and three elements were defined: dynamics (intense change), expansiveness (spatial extent), and stability (balance of movement and harmony of expression). These factors capture the expressive diversity of dance and play an important role in the selection of evaluation items. The data collection included advice given by a dance expert who has been dancing for over ten years and is an instructor at a dance school, and the most frequent advice given during instruction was tabulated (Table 2). For example, the advice 'dance as hard as you can because you are nervous and your body is not moving' was related to strength and dynamics, while the advice 'dance bigger overall' was related to the size and spatial characteristics of the movement. These pieces of advice are considered to reflect important perspectives in the teaching of dance in practice. \begin{table}
\begin{tabular}{c l} \hline \hline
**Frequency** & **Advice Content** \\
**Order** & \\ \hline
1 & You are nervous and your body is not moving, so dance as hard as you can. \\
2 & Dance a whole lot bigger. \\
3 & Where you stop moving, stop tight. \\
3 & Move your hands smoothly and with awareness of the flow of your hands. \\
3 & Bend your knees and drop your center of gravity more. \\
6 & Move your hands more clearly. \\
6 & Go over your choreography again. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Professional advice. Figure 7: MR-based camera setup. This study proposes an automatic evaluation model for dance based on deep learning. Automatic evaluation is defined here as a problem of predicting the expert's evaluation of features related to expression such as dynamics, sharpness, scalability, timing, accuracy, and stability, where the prediction is based on the input of time-series information on the position, rotation, and speed of the major joint points of the human body as motion data. The automatic evaluation model was trained in two steps. First, an encoder was trained to convert motion data into features. Two methods were used to learn the encoder: supervised contrast learning and autoencoder. Then, the evaluation model was trained to predict the expert's evaluation using the obtained features as input. Following the automatic evaluation of the dance, users receive feedback to aid them in practicing their dances while being mindful of areas requiring improvement. The feedback comes in two forms: 3D feedback via dance replays and visual feedback through graphs. Dance replay feedback allows users to review their dance and compare it to a model dance. This facilitates the identification of specific movement areas for improvement, which can be applied to subsequent practice. Graphical feedback first presents the evaluation results on a radar chart. The evaluation is performed for each one-second segment, but the visualization results show the average of the evaluation values for the entire dance. This offers insights into the perspectives to concentrate on during practice. Moreover, it allows users to visually track their progress by comparing past results.