## 1 Introduction  
Human action recognition refers to classifying and understanding what humans are doing in the input data. It has a wide range of applications in real life. For example, human action recognition can be used for home monitoring to monitor the behavioral activities of the elderly and to detect dangerous actions such as falls in a timely manner [1], and it can help an automatic navigation system analyze and predict the action of pedestrians [2]. Commonly used inputs for human action recognition algorithms include RGB images and videos [3], skeleton [4], depth [5], point-cloud [6], and so on. Among these modalities, the RGB modality is the easiest to acquire and manipulate, using only the RGB cameras. However, action recognition algorithms based on RGB modality are often affected by appearance information, such as background, illumination, and color textures [7]. Besides RGB modality, skeleton modality is the most popular. As shown in Figure 1, skeleton modality refers to representing the human body through joint points and limb lines. Therefore, it is naturally concise but very precise in expressing motion information. However, in the actions of human-object interaction, the key appearance information cannot be captured, and the performance of skeleton-based human action recognition methods will be poor. By combining the skeleton modality with the RGB modality, the appearance information of the RGB modality can be utilized, and the precise dynamic information in the skeleton modality can also be utilized. Therefore, the methodsbased on both RGB and skeleton modalities are the best choice for action recognition tasks in practical applications. At present, some works [8; 9; 10; 11] have proposed action recognition algorithms based on both RGB and skeleton modalities. The two methods [8; 9] use the GCN networks to extract skeleton features, treat human joints as points, and presern human limbs as lines, as shown in Figure 1. However, it has been proved that the GCN-based methods still have many limitations in robustness, interoperability, and scalability [10]. Therefore, [PERSON] et al. [10] proposed PoseConv3D, which used the form of pseudo heatmaps to express skeletons and then fed the heatmaps into 3D CNN. However, each convolution operation can only process local information, which is a very small part of the entire input data. The size of the \"patch\" proposed in ViT [12] greatly increases the \"receptive field\" of the network. There have been many successful Transformer-based applications in computer vision, such as image classification [13; 14; 15], video detection [16; 17], and action recognition [18; 19]. Therefore, it is very meaningful to explore Transformer-based dual-modality action recognition methods. There is already a work [11], dubbed TP-ViT, trying to use Transformer to handle RGB and skeleton bimodal action recognition. However, TP-ViT still uses absolute coordinates to represent skeleton points, which will also face the problems of stability and poor performance in multi-person scenes. Hence, how to use Transformer to make the most of RGB and skeleton modalities is still a challenge. In this paper, in order to make the most of skeleton data and RGB data efficiently, we propose a two-stream Transformer framework. In this framework, inspired by SlowFast, inputting frames at a high temporal resolution, i.e., using fast refreshing frames, can effectively simulate potentially fast changing motion. In action recognition, motion information is fast-changing information compared to the almost constant appearance information. Therefore, in order to capture action information precisely, skeleton stream is input into Transformer with higher temporal resolution and lower spatial resolution compared with the RGB stream. To avoid the different coordinates obtained by different pose extractors from affecting the behavior recognition accuracy and to solve the problem of greatly increasing the computation of multi-person scenes caused by treating skeleton joints as points.