# Deep Learning-Based Human Action Recognition with Key-Frames Sampling Using Ranking Methods
## 1 Introduction  
The rapid development of electronic devices such as smartphones, televisions, notebooks, and personal computers plays an important role in our daily life. The ways of interacting with these devices have also improved dramatically over the past years. To provide easy, smart, and comfortable ways of communication, several devices and applications have been invented ranging from wired keyboards to wireless vision-based communication. Recently, vision-based communication has become very popular owing to time management, cost-effectiveness, and the pandemic. Several real-world vision-based applications have already been introduced, such as human-machine interaction [1], video surveillance systems [2], data retrieval [3], augmented reality [4; 5], virtual reality [5], medical care [6], autonomous driving systems [7], and gaming control [8]. Many methods have also been developed for pose estimation [9], facial expression recognition [10], behavior analysis [11], hand gesture recognition [12; 13; 14], and action recognition [15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48] to make vision-based decision or communication. Hand gesture recognition [12; 13; 14] is one of the most popular forms of vision-based interaction. It is limited to the classification of actions that are performed by only hands. Thus, it is necessary to develop a system that can understand the actions accomplished by different parts of the human body. Human action recognition focuses on the discriminationof human actions performed by hands (_'Clap'_, _'Catch'_, and _'SwipeLeft'_), leg (_'Kick'_, _'LegCurl'_), or whole-body (_'Jogging'_, _'Run'_, _'Walk'_). The advancement in modern data capturing devices allows researchers to capture multi-modal human action datasets such as inertial, skeleton, RGB, and depth [15]. The inertial dataset [16] has not gained much attention among researchers because the user needs to wear multiple inertial sensors on their body to collect inertial data. Sometimes, it becomes very uneasy and uncomfortable, such as when driving cars and for robotic operations in medical care systems. The inertial sensor is also sensitive to sensor location and angle on the body. Compared to the inertial dataset, it is easy and comfortable to capture skeleton information without intimate contact with sensors. The skeleton dataset [17; 18; 19; 20] contains only a limited amount of joint information about the human body, which is very little information for perfectly classifying an action. The shortage of information in the skeleton dataset sometimes degrades the performance of recognition systems. However, the RGB dataset [21; 22; 23] covers full information of the human body, but it is time-consuming, computationally expensive, and complex in practical usage. These data are also sensitive to illumination changes, camera calibration, lighting conditions, and background. By studying the drawbacks of the inertial, skeleton, and RGB data modalities, we use depth sequences for the proposed method that includes enough information including the whole body. The depth dataset is also not sensitive to illumination changes, lighting conditions, and background. In this article, we propose a novel approach for human action recognition with key-frames sampling. The key-frames are sampled using ranking metrics. By considering the objectives of the proposed method, we introduce two well-known similarity comparison metrics between images, namely, structural similarity index measure (SSIM) and correlation coefficient measure (CCM). The major contributions of the proposed method are summarized as follows:  
1. We introduce a novel ranking-based approach for human action recognition using 3D-CNN with raw depth sequence. 2. First, we use SSIM or CCM ranking metrics to select _k_-ranked frames that contain more spatial and temporal changes. This allows us to discard the redundant frames having the same or very similar information. 3.