# 3D-STARNET: Spatial-Temporal Attention Residual Network for Robust Action Recognition
## 1 Introduction  
With the development of deep learning technology, action recognition has become an important research topic in the field of video understanding. It has extremely wide application scenarios in the fields of autonomous driving, safety control, rail transportation, human-computer interaction, etc. Currently, most behavior recognition methods are based on RGB [1], skeleton [2], optical flow [3], depth map [4], radar [5], point cloud, and other modal data or the fusion of these modalities. Skeleton data are usually robust to lighting changes, background interference and occlusion, and their computational complexity is much smaller than that of other modal data. Skeleton sequences also have two significant advantages in the field of behavior recognition [6]: (1) The spatial information within the frame contains the correlation between different joint points, and rich structural information can be extracted. (2) The temporal information between frames preserves rich temporal correlations, so this article conducts research based on skeletal data.  
There are still many challenges in the field of action recognition using skeleton data. These challenges include the differences in the way different individuals perform the sameaction, the complexity of complex backgrounds, and the fluctuation of lighting conditions. One of the key issues is to effectively mine the spatiotemporal features inherent in skeleton sequences. Currently, in the field of skeleton-based action recognition, the most popular method is based on graph convolutional networks (GCNs) [7]. It is built based on a series of skeleton graph sequences, where each node corresponds to a joint of the human body and defines two types of edges: spatial edges, which conform to the natural connectivity of joints, and temporal edges, which connect the same joints in consecutive time steps and allow information to be integrated along the spatial and temporal dimensions to discover action patterns in spatial and temporal dimensions. This not only improves the expression ability of the model, but also facilitates generalization in different contexts. Although GCN can explore action patterns in the spatiotemporal dimension, its recognition ability is significantly affected by the coordinate distribution, so it is not robust.  
To address the above problems, we proposed the Spatial-Temporal Attention Residual Network for 3D human action recognition (3D-STARNET). In order to make the model robust, we convert the skeleton points into heat maps through Gaussian transformation, thus avoiding the strong dependence on the skeleton points. In order to enhance the network's ability to extract key frames and key regions between frames, we propose a spatiotemporal attention mechanism (STA). In order to improve the efficiency of signal propagation in the network, we introduced a phased residual structure (MS-Residual). Finally, we verified 3D-STARNET on the NTU-RGBD120 dataset (see Section 4 for details), achieving significant improvements in accuracy.  
To address the above issues, we proposed a Spatial-Temporal Attention Residual Network (3D-STARNET) for 3D human action recognition. Unlike GCN-based methods, we transform skeleton points into heat maps instead of skeleton sequences through Gaussian transformation. This change avoids a strong dependence on skeleton points, and this improvement significantly enhances the robustness of the algorithm. In order to enhance the network's ability to extract key frames and key areas between frames, we proposed a spatiotemporal attention mechanism (STA). In order to improve the efficiency of signal propagation in the network, we introduced a staged residual structure (MS-Residual). Finally, we verified 3D-STARNET on the NTU-RGBD120 dataset (see Section 4 for details), and the accuracy was significantly improved.