## 2 Related Works
### Depth-Based 3D Action Recognition  
For depth-based 3D action recognition, early approaches mainly represent depth videos by manual descriptors [19]. [PERSON] et al. [7] constructed depth motion maps (DMMs) by stacking the inter-frame differences of the projected depth frames. Then, they calculated the histogram of oriented gradients (HOGs) to represent the actions. Such methods have limited expressive power and thus usually need help in capturing spatial-temporal information. In recent years, deep learning methods have become mainstream with the development of neural networks. Most researchers attempted to compress deep video into images and analyzed motion patterns using CNNs [12]. [PERSON] et al. [27] input depth motion images (DMIs) and moving joint descriptors (MJDs) to CNNs for action recognition. To encode spatial-temporal information of depth sequences, [PERSON] et al. [28] proposed 3D-CNN to extract motion features. Furthermore, they proposed ConvLSTM [29] to accumulate discriminative motion patterns from long short-term units. [PERSON] et al. [3] rotated the virtual camera within the 3D space to densely project a raw depth video from different virtual imaging viewpoints and thus constructed multi-view dynamic images. For perspective view invariants, [PERSON] et al. [30] proposed an ActionNet based on CNNs and trained with a multi-view dataset collected using five depth cameras. [PERSON] et al. [31] computed multi-view depth descriptor edge-detected-motion history image (ED-MHI) as the input of a multi-stream CNN model. [PERSON] et al. [2] utilized segmented depth video sequences to generate three types of dynamic depth images. However, the 2D depth map still has difficulty in fully exploiting 3D motion patterns due to its compact spatial structure [10].  
Recently, the conversion of depth maps into point clouds for processing has achieved better results in both recognition and segmentation fields. Numerous studies have shown that point clouds have significant advantages in representing 3D spatial information due to their characteristics, such as disorder and rotation invariance. Deep learning for point clouds has not only been widely used in classification and segmentation tasks but has also demonstrated muscular strength in scene reconstruction [32] and target detection [33]. However, the above methods focus only on features within static point clouds. When using point clouds for 3D action recognition, it is necessary to extract dynamic features according to the time intervals and the appearance features of the whole action process. The key to an efficient processing of point cloud sequences lies in selecting a suitable point cloud analysis method. [PERSON] et al. [34] developed a method inspired by image-based convolution and employed a set of kernel points to distribute each kernel weight. As an efficient tool for analyzing and processing point sets, PointNet++ [11] is widely applied for 3D action recognition based on point cloud sequences. The first method is 3 DV [10], which executes 3D voxelization towards the point cloud sequences and describes the 3D appearance by spatial occupancy, and temporal rank pooling is utilized for 3 DV extraction. This method primarily focuses on an action's general motion and appearance changes. However, it ignores the details of the action, such as a subtle hand movement, which limits its ability to represent behavior accurately. Hence, we aim to capture the crucial parts of actions and their delicate information to recognize them as more robust human actions.