## 5 Results and Discussion  
As discussed in Section 3.1, the proposed methodology began by first extracting key frames from the video clips in the datasets used in this study as described in Section 4. The key frames were extracted based upon the gradient of the total motion metric given in Equation (1) across consecutive frames in the video. The total motion metric was computed using the Gunnar Farneback algorithm and dense optical flow [40]. Before computing the changes in the intensity \(I(x,y,t)\) of pixels across successive frames, they were first converted into the hue, saturation and value (HSV) format, and then the pixels undergoing intensity changes in successive frames were highlighted for better visibility. From a set of flow vectors, the algorithm calculated the amount and direction of optical flow. The total motion metric \(M(t)\), as given in Equation (1), was then computed to quantify the motion over both the horizontal and vertical directions for a frame \(t\). The gradient of the optical flow function \(M(t)\) was then computed to select the key frames based upon the local extrema of the total motion in the video. Figure 5 shows the gradient of the total motion metric for a walking video from the UCF Sports dataset [48]. The number of local extrema depends upon the contents of the video (i.e., videos that record relatively more complex and abrupt movements or activities would have more local extrema, whereas videos that record relatively simpler and smoother activities and movements would have fewer local extrema). Therefore, to capture the most significant changes in motion that were important for the identification of an action, we selected \(k\) frames corresponding to the highest \(k\) local extrema. Figure 6 shows the selected key frames, which were \(k\) in number, for a video from the UCF Sports dataset, labeled as a _walking video_. Figure 5: The gradient of the total motion metric for a walking video from the UCF Sports dataset. The red dots are the local minima, whereas the green dots are the local maxima of the gradient function. The gradient of the total motion metric allowed the proposed algorithm to sum up the motion in successive frames of a video and to capture the motion information that better represented an action. Moreover, it also rendered a proposed method that was simpler and more efficient as motion information from the entire video sequence was squeezed into a few key frames (i.e., \(k\) number of key frames, which for the purpose of this study was taken to be 24). Twenty-four key frames amount to roughly 10-20% of the total frames in each video in the UCF Sports dataset. This is also helpful in reducing the dimensionality of the feature vectors comprising the human keypoints extracted from these key frames using pose estimation models, as we discarded 80-90% of the frames in each video. That translated into relatively faster training and more accurate classification of different actions by the DDNet in the latter stages. Once the key frames were extracted from a video, pose estimation models were applied to them to extract the human skeleton features, also known as human keypoints. These keypoints represent the coordinates of the different joints in a given frame that constitute a pose. In this study, two pose estimation models (i.e., HRNet-W48 [22] and OpenPose [12]) were used to extract the human keypoints from the extracted key frames. The HRNet with 48 channels extracted 17 keypoints from the input images, which were resized to \(384\times 288\) pixels, whereas OpenPose extracted 18 keypoints from the input images, which were resized to \(368\times 368\) pixels. Certain hyperparameters of the pose estimation models (i.e., the number of keypoints in the action capture model) were tuned to find out the values that would yield the best results. Different numbers of keypoints (i.e., 11, 13, 15 and 17) were extracted from the key frames using the pose estimation models, and the best results were obtained with 17 keypoints.