# DanceCaps: Pseudo-Captioning for Dance Videos Using Large Language Models
## 4 Experiments
### Tag Evaluation  
This study aimed to quantitatively evaluate the effectiveness of self-defined tags that characterize choreography videos. Specifically, we focused on two types of self-defined tags: temporal motion tags and dance genre tags. For temporal motion tags, we assessed the accuracy of these tags in describing the actual temporal dance movements observed in the videos. This evaluation helped determine how well our defined tags captured the dynamic aspects of choreography. Regarding dance genre tags, in cases where genre information was not included in the video's metadata, we employed a trained model to infer the genre. The reliability of these tags was then evaluated based on the accuracy of the model's inference results. This two-pronged approach allowed us to comprehensively assess the efficacy of our tagging system in capturing both the temporal dynamics and stylistic categorization of dance videos. We first evaluated the similarity between annotations generated by human annotators (professional dancers) and LLMs for 877 frames extracted from 50 videos in the AIST++ dataset. As depicted in Figure 7, three professional dancers and the LLM independently annotated the movements in each frame, and the similarity was analyzed using cosine similarity and BERT Score. We first transformed the frame annotations into TF-IDF (Term Frequency-Inverse Document Frequency) vectors. This vectorization process allowed us to represent the textual annotations as numerical vectors, capturing the importance of each term within the context of the entire dataset. Subsequently, we computed the cosine similarity between these  
Figure 7: Examples of movements described using LLM and movements annotated by a human. TF-IDF vectors. This similarity measure enabled us to quantify the semantic relatedness between different frame annotations. BERT Score is a method for evaluating the semantic similarity between candidate and reference sentences using contextual embeddings from BERT. First, embeddings are generated for each token in both sentences using BERT. Next, the cosine similarity between each token in the candidate sentence and each token in the reference sentence is calculated. To compute precision, the maximum similarity for each token in the candidate sentence with respect to tokens in the reference sentence is used, while recall is calculated by taking the maximum similarity for each token in the reference sentence with respect to tokens in the candidate sentence. Finally, the F1 score is obtained by calculating the harmonic mean of precision and recall. This process allows the BERT Score to capture semantic similarity between sentences more accurately. The analysis revealed an average cosine similarity of \(0.81\pm 0.09\) and a BERT Score (F1 score) of \(0.88\pm 0.06\). These results clearly demonstrate that the choreography annotations generated by the LLM have a high level of semantic similarity to those of experts. The high similarity scores prove that LLMs can accurately capture the core elements and details of choreography, strongly suggesting the feasibility of automated choreography analysis and description systems. To assess the accuracy of the dance genre classification, we utilized a dataset comprising 1500 dance videos for training our proposed model and evaluated its performance on a dataset of 200 in-the-wild videos evenly distributed across 10 dance genres. The train dataset was constructed based on the AIST++ dataset, supplemented with additional in-the-wild videos collected from social media platforms to reflect real-world conditions. The proposed model achieved an average classification accuracy of 72% on the test set, demonstrating good performance in classifying diverse dance videos in in-the-wild environments. The confusion matrix shown in Figure 8 provides a details of the model's classification performance. Figure 8: Confusion matrix of the dance genre classification for 200 in-the-wild dance videos (20 videos per each genre).