HFA-GTNet: Hierarchical Fusion Adaptive Graph Transformer network for dance action recognition

Authors:
raw_author_name: Ru Jia
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China
raw_author_name: Rui Yang
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China
raw_author_name: Honghong Yang
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China
raw_author_name: Xiaojun Wu
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China
raw_author_name: Yumei Zhang
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China
raw_author_name: Peng Li
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China
raw_author_name: Yuping Su
raw_affiliation_string: Key Laboratory of Intelligent Computing and Service Technology for Folk Song, Ministry of Culture and Tourism, Xi’an, 710119, China; Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China; School of Computer Science, Shaanxi Normal University, Xi’an, 710062, China

Abstract: Dance action recognition is a hot research topic in computer vision. However, current skeleton-based action recognition methods face difficulties in capturing the adequate spatial structure and temporal variations of dance actions, resulting in lower recognition accuracy. In this paper, we propose a Hierarchical Fusion Adaptive Graph Transformer network (HFA-GTNet) for dance action recognition. A Hierarchical Spatial Attention (HSAtt) module is designed to extract different levels of spatial feature information from joint to parts to group, it can effectively learn high-order dependency relationships from local joints to global poses in dance actions. Secondly, to extract the joint variations in dance actions at different speeds, we have designed a Temporal Fusion Attention (TFAtt) module. This module learns the short-term and long-term temporal dependencies among joints across frames. Additionally, to capture the variations in motion patterns and dance styles among different dancers, we introduce an Adaptive Component (AdaptC). Finally, we evaluate our model on two self-built dance datasets, MSDanceAction and InDanceAction, and demonstrate its superior performance compared to other state-of-the-art methods in dance action recognition.