# Auxiliary Task Graph Convolution Network: A Skeleton-Based Action Recognition for Practical Use
## 1 Introduction  
Always-on cameras such as CCTV are widely used, and their purpose is mostly safety management. Recently, researchers have applied action recognition technology to the CCTV system, such that it can automatically recognize roamers' behavior and achieve better safety management [1, 2]. Additionally, action recognition could be used to understand people in the video, such as behavior in a class [3]. Graph convolution networks (GCNs) [4, 5, 6, 7, 8] are typical networks that recognize human action with information based on the human skeleton. This way of recognition has the benefit of removing noise from the image background, reducing the number of troubles and inference time. With this benefit, the GCN model shows accurate action recognition. However, the previous GCNs [9, 10, 11, 12] are not practical because of the sampling frame (image) process. The previous GCNs require the information of the video length prior to the sampling frame for recognition. The video from an always-on camera (i.e., CCTV) that action recognition with GCNs could be applied to does not have a video length; hence, it would be difficult to simply apply previous GCNs to it. A simple solution to this sampling issue is clipping a CCTV video to have a fixed length (e.g., clipping a video to have a four-second length). This simple solution, however, is not compatible with the previous GCNs. Most GCNs' datasets include videos of different lengths for training and testing; hence, simply clipping videos to a certain length may not include all the movements for human actions. Additionally, previous GCNs may not beeffective in recognizing real-world human actions because real-world actions have different pose sequences compared to the actions in the dataset. For example, human actions in the real world could have diverse speeds of movement, but the actions in the dataset are artificial and generally have a certain level of movement speed. Therefore, we propose an Auxiliary Task Graph Convolution Network (AT-GCN) structure for the practical use of action recognition. The AT-GCN structure implemented low and high pathways introduced in SlowFast Network [13] to parallelly sample the frame at low and high rates for slow and fast action recognition, respectively. This means that our AT-GCN structure could recognize diverse speeds of movement by parallelly capturing semantic information at low and high frame rates (Figure 1). There is a major difference between our AT-GCN and previous GCNs. Previous GCNs used only one frame rate for action recognition, but our AT-GCN uses two frame rates. Using one frame rate may not be suitable for recognizing a variety of actions, including slow and fast actions. To solve this problem, AT-GCN uses two frame rates for action recognition. Our AT-GCN is also different from SlowFast [13] as it has three losses, while SlowFast has only one fuse loss. Using one loss may not be suitable for learning a variety of features, such as different frame rate features. To solve this problem, AT-GCN uses two additional losses: low and high frame rate losses, in addition to the fuse loss. This structure is effective for learning more variety of features compared to using only one loss. Our AT-GCN includes three end branches to support three loss functions: a fuse loss function in the main task and a slow and a fast loss function in the auxiliary task [14; 15; 16]. In this paper, we compare two variations of GCNs--traditional GCNs and AT-GCNs --with regard to the NTU RGB+D 60 and 120 and NW-UCLA [17; 18; 19] datasets. The experimental results show that AT-GCNs did not increase the inference time with the learning process of slow and fast actions. Moreover, AT-GCNs outperform traditional GCNs on the three datasets. Figure 1: Auxiliary task part of AT-GCN. Low and high pathways defined by frame rates. Each pathway output features through the GCN backbone. Via a low header, high frame rate features had a score and slow, fast loss compared with the ground truth.