# Best Frame Selection to Enhance Training Step Efficiency in Video-Based Human Action Recognition
## 3 System Overview
### Adaptive Frame Cropping  
In the second part of the proposed method, the frames selected in the previous step must be resized or cropped to the input size of the HAR system. In some methods, the input video is resized using conventional image resizing methods. However, in these methods, the performance of the HAR system may be reduced due to the small size of the human image in the resultant frames. The proposed method was designed to estimate the movement region, i.e., the region related to a human action, in the input video with a low computational burden.  
Selecting the cropping region randomly is also a weak and too simple approach. On the other hand, the best approach is to identify the human location in the frame and to select the cropped region using human location and action-related features. However, this approach is not desirable due to a high computational burden. In the related step of the proposed method, which is described in Algorithm 2, all of the calculated differences between the normalized frames are transferred from the previous step to this step, which decreases the needed computations, and are used to build an energy map of the shortened video. In this map, the value of each pixel represents the sum of the pixel motion information in the entire frames. After this step, an average filter is applied to the built energy map. The window size of the average filter is defined as equal to the input size of the used HAR system, and the final image is obtained.  
```
Input : Shortened Video \(V\), The desired size of final video \(D_{s}\)  1- Separate Video to frames \(F_{i}\) (\(i\) is the number of frame),  2- Apply Gradient to all frames \(G_{F^{V}}\),  3- Compute absolute difference between frame gradients \(\Delta G_{i}=|G_{F^{V}_{i}}-G_{F^{V}_{i-1}}|\),  4- Add all \(\Delta G_{s}\) together and made an energy map for video, \(E_{map}=\sum_{i=1}^{(N_{frames})}\Delta G_{i}\),  5- Apply Average (or mean) filter with size \(D_{s}\) to \(E_{map}\),  6- Select the maximum pixel value index \(i_{S}\) of filtered image as the center of cropping area,  7- Crop frames using \(i_{s}\) as center and \(D_{s}\) as crop size, Output :Merge cropped frames and create the Shortened cropped Video
```  
**Algorithm 2**Adaptive Frame cropping  
Finally, the pixel that has the highest value in the filtered image is selected as the center of the cropping region. It is easy to argue that this window contains more relevant motion information, which was found based on the gradient difference between frames, than any other window than can be defined in the video.  
Figure 4 shows the overall energy map of a sample shortened video and the result of applying a average filter with dimensions equal to [111,111] on it. Figure 5 shows the original frames, the result of applying the proposed method and the result of a bad random selection. Hence, it is possible to conclude from Figure 5 that a randomly cropped video, in some cases, does not contain valuable information for training HAR systems due to the wrongly cropped regions. Contrary to random selection, the result of the proposed method contains complete information about the involved action.  
Figure 3: (**Left**): nine consecutive frames of a video; (**center**): the gradient of each frame; (**right**): the absolute value of the difference between the gradient of each frame and the next frame.