# A Diffusion Modeling-Based System for Teaching Dance to Digital Human
## 3 Theoretical Framework of Digital Human Dance Teaching System
### Dataset  
In this system, model training is critically dependent on the dataset. Significant improvements have been made to music and dance datasets through the efforts of previous researchers. The dataset is designed as a benchmark for motion generation and prediction tasks. It may also benefit other tasks, such as 2D/3D human pose estimation. To the best of our knowledge, AIST++ is the largest 3D human dance dataset, containing 1408 sequences, 30 subjects, and 10 dance genres, encompassing both basic and advanced choreography [14]. In contrast, FineDance is the largest dataset for music-dance pairing, encompassing a wide variety of dance genres [15]. These datasets are currently the most suitable for research on music-driven dance. As shown in Table 1. The evaluation metrics of the dataset are shown in bold in Table 1. Pos and Ros means 3D position and Rotation information respectively. Figure 2: System Technology Roadmap. The AIST++ dance movement dataset is derived from the AIST Dance Video DB and includes 1408 3D human dance movement sequences, represented as joint rotations and root trajectories. The dataset encompasses a diverse range of dance movements distributed across 10 dance genres and hundreds of choreographies. Movement durations vary from 7.4 to 48.0 s. Each dance sequence is paired with its corresponding music, making AIST++ a foundational dataset for this research. The AIST++ dataset is shown in Figure 3.  
The FineDance dataset comprises 14.6 h of music and dance pairing data, featuring detailed hand movements, a comprehensive categorization of 22 dance types, and precise pose information. In this system, FineDance is integrated with the AIST++ dataset, and the final data format for the dance sequences is SMPL. An illustration of the FineDance dataset is shown in Figure 4. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Pos/Rot**} & **Joints** & **Hand** & \multirow{2}{*}{**Genres**} & **Mocap** & **RGB** & \multirow{2}{*}{**SMPL**} & **Dancers** & **Total** & **Avg Sec** \\  & & **Num** & **Joint** & & & & **Views** & & **Hours** & **per Seq** \\ \hline GrooveNet & \(\surd\)/\(\times\) & 30 & \(\times\) & 1 & \(\surd\) & 1 & \(\times\) & \(\times\) & 1 & 0.38 & 690 \\ Dancew/ & \(\surd\)/\(\times\) & 21 & \(\times\) & 4 & \(\surd\) & 4 & \(\times\) & \(\times\) & - & 1.6 & 92.5 \\ Melody & \(\surd\)/\(\times\) & 55 & \(\surd\) & 2 & \(\surd\) & 2 & \(\times\) & \(\times\) & 2 & 0.96 & - \\ Musiel2 Dance & \(\surd\)/\(\surd\) & 24 & \(\times\) & 4 & \(\times\) & 4 & \(\times\) & \(\times\) & - & 0.35 & 73.8 \\ Phantom- & \(\surd\)/\(\times\) & 24 & \(\times\) & 13 & \(\times\) & 0 & \(\times\) & \(\times\) & - & 9.6 & 133.3 \\ Dance & \(\surd\)/\(\surd\) & 17/24 & \(\times\) & 10 & \(\times\) & 10 & \(\times\) & \(\surd\) & - & 5.2 & 13.3 \\ MMD & \(\surd\)/\(\surd\) & 52 & \(\surd\) & 4 & \(\surd\) & 0 & \(\surd\) & \(\times\) & - & 9.9 & - \\ \hline FineDance & \(\surd\)/\(\surd\) & 52 & \(\surd\) & 22 & \(\surd\) & 2 & \(\surd\) & \(\surd\) & 27 & 14.6 & 152.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison table for each music and dance dataset. Figure 3: Schematic of the AIST++ dataset. Data Extraction and Processing  
1. Data extraction: The AIST++ dataset is initially downloaded as BVH or FBX files, which contain skeletal motion capture data. After downloading, these files are converted into JSON format to facilitate the easy extraction and manipulation of joint positions, rotation angles, and frame data. 2. Data segmentation: Each dance sequence in AIST++ is divided into smaller segments based on dance phases or specific moves. For example, a single dance move can be divided into \"beginning\", \"middle\", and \"end\" phases or categorized by specific moves such as spins, jumps, or transitions. These segments are precisely aligned in time to ensure smooth transitions between different dance phases. This segmentation approach helps create seamless choreography when the segments are stitched together. 3.