# DanceCaps: Pseudo-Captioning for Dance Videos Using Large Language Models
## 3 Pseudo-Captioning Based Dance Caption Generation
### Caption Types  
In this study, three distinct types of captions were defined based on their purpose in dance captioning. The first type, known as the 'dance summary caption', was constructed primarily from the extracted dance tags to provide a basic description of the dance. This caption served as the fundamental form of description, capturing essential components of the dance. However, while the 'dance summary caption' provides a general overview, it may not encompass the full scope of the dance video, which includes temporal chore  
Figure 6: The processing of extracting other tags from the thumbnail of the dance video.  
ography and additional contextual components beyond the dance itself. To address this limitation, the second type of caption, termed 'time series motion caption', was designed to detail the sequence of dance movements over time. This type captured dynamic changes and intricate details that the 'dance summary caption' might have overlooked. The final type, referred to as the'multimodal caption', aimed to represent non-dance components, such as music, dancer characteristics, and environmental factors, as well as the interactions and harmonies between these components and the dance.  
Although it is feasible to create an all-encompassing caption that integrates all three types, such comprehensive captions may include extraneous information, leading to inefficiencies in both temporal and quality aspects during dataset generation. Consequently, tailoring captions to specific purposes can improve the efficiency and effectiveness of learning and research processes. The subsequent subsections will detail the tags necessary for each caption type and provide the instructions required for generating these captions using an LLM. This study employs GPT-4o [46] as the LLM, and the related instructions and results are derived from this model. Table 3 summarizes the contents of the following subsections, with detailed explanations provided therein.  
\begin{table}
**Caption Name** & **Required Tags** & **Instructions and Examples** \\ \hline Dance summary caption & Dance genre, choreography step, temporal motion & The tag information below includes a description of the dance and details on how the series of dance movements change over time. Based on the given information, please describe how the movements evolve over time in a detailed yet concise manner. Write it in one sentence, naturally as if a person were speaking, focusing solely on the dance movements and making sure to include the dance genre. \\ \hline \end{tabular} \
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline Time series motion & Temporal motion, average acceleration of movement, average velocity of movement, dance genre, choreography step & Please describe the dancer’s movements in detail in chronological order, using the tag information provided. Create a natural-flowing paragraph that captures the progression of the dance moves clearly, without additional explanations. \\ \hline \end{tabular}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline Multimedia captions & Music genre, tempo, instrument, average pitch level, mood, dance genre, choreography step, motion, dancer characteristics, spatial characteristics & Using the tag information, list and briefly describe the dancer’s movements in chronological order, ensuring they align with the music. Keep the paragraph concise and under 80 words, capturing the essence of the dance moves in relation to the music and environment tags. \\ \hline \end{tabular}
\end{table}
Table 3: Three types of captions, detailing the caption names, required tags, associated instructions, and examples for each type.