# Human Action Representation Learning Using an Attention-Driven Residual 3 DCNN Network
## 1 Introduction  
Convolutional neural networks (CNNs) have become one of the most widely used deep learning architectures in computer vision due to their ability to effectively capture the spatial features of image and video data. In recent years, CNNs have shown remarkable success in a variety of applications, including object detection and recognition [1], image segmentation [2], and scene understanding [3]. The design of deep neural networks is crucial for their efficiency, including the depth and structure of the network layers, depending on the task. In some cases, such as object recognition and video analytics, over-parameterization is necessary to ensure the model captures complex hidden patterns and generalizes well. However, this comes at the cost of increased computational complexity, making them unsuitable for real-time environments and resource-limited devices, and requiring high-end GPUs for training [4, 5]. The design of network architecture is task-specific which varies from problem-to-problem. For instance, recognizing objects in still images demands a plain 2 DCNN network composed of convolutional layers for spatial feature extraction and classification layer for classification task. Recognizing human actionsin video stream cannot be handled with single 2 DCNN networks, in view of the fact that videos are composed of large sequence of frames presenting the temporal flow of video across the frames in the temporal dimension (t). To cope with the challenge of human action recognition task in video streams, researchers have introduced different solutions that include two-stream 2D convolutional neural network (CNN) architectures [6], 3D CNN (2D + 1D) architectures [7], and CNN with a recurrent neural network (RNN) [8]. Typically, two-stream 2 DCNNs [6] use two different CNN architectures to extract two different kinds of features from the input video. The first CNN model extracts the spatial features from the input video frames, whereas the second network extracts the temporal optical flow features with respect to their corresponding spatial features. The extracted features from both the models can then be combined as a single latent representation vector for the activity recognition task. On the other hand, 3 DCNNs [7] use single CNN architecture having 3D convolution kernels, where the first two dimensions capture the spatial features and the last dimension of the 3D kernel captures temporal flow of the spatial features across the frames. The CNN with RNN architecture frameworks include two different type of models (i.e., CNN followed by RNN). The CNN model in CNN + RNN architectures extracts the spatial features from the video frames and converts it to a one-dimensional latent representation such as feature vectors. The extracted latent representation from the CNN models can then be fed to the RNN model for activity classification task using sequential pattern learning. Typically, an activity recognition framework with two different network architectures increases the parameters space (computational complexity) of the entire framework as well as the time complexity of the model for the task under the consideration. Considering the parameter space and time complexity of two-stream architectures, 3 DCNNs are considered a suitable candidate for human activity recognition task. Therefore, in this paper, we propose a computationally efficient residual 3 DCNN architecture called dual-attentional residual 3D convolutional neural network (DA-R3 DCNN) with channel and spatial attention for human activity recognition task. The proposed DA-R3 DCNN has channel and spatial attention layers after each residual block which helps our model to propagate salient features from the early layers to later layers. This propagation of salient information significantly improves the performance of our model for human activity recognition task. More precisely, the major contributions of this paper are as follows:  
1.