## 3 Proposed Methodology
### Integrated Spatial and Temporal Features Based on Weighted Mean  
\(FS^{N}\) is a collection of image data including time information sequentially listed according to any interval, and the model extracts \(FD^{N}\), which is the frame change rate of each image included in \(FS^{N}\), as a temporal feature. \(N\) means the number of frames that constitute \(FS^{N}\), and since they are extracted according to the same time interval, the frame change rate, \(FD^{N}\), that changes according to the time interval can be used as a temporal feature of motion representation. \(FD^{N}\) is calculated as the average of the frame change sequentially calculated according to the image sequence of \(FS^{N}\) and it is transformed to one-dimensional temporal feature vectors. As compared to optical flow applied to two-stream CNNs, LSTM applied to single-stream CNNs is simpler to operate and it is extracted at high speed. The frame change rate calculated by the frame change of the images constituting \(FS^{N}\) additionally calculates the frame change for the whole frame of the video and the first frame of \(FS^{N}\) because it has the length of \(N-1\). Therefore, \(FD^{N}\) has the same length as the data length \(N\), indicating the time information of \(FS^{N}\) extracted according to any interval in the video, and \(FD^{N}\) in \(N\) means the frame change rate of \(FM^{N-1}_{K}\) and \(FM^{N}_{K}\). \(FM^{N}_{K}\) has a size of \((N,\,K)\) over the entire time because \(K\) one-dimensional vectors are extracted at each time. The data in each row (\(N\)) of \(FM^{N}_{K}\) mean spatial features extracted from images at a specific time and are sorted in a column direction over time. In the proposed system, \(FV_{K}\) with spatiotemporal features is generated by weighted mean of \(FD^{N}\), which represents the degree of change of each frame over time in the spatial feature \(FM^{N}_{K}\) sorted according to time information, as shown in Equation (1).  
\[FV_{K}=\left[\frac{\sum_{i=1}^{N}FM^{i}_{1}\times FD^{i}}{\sum_{i=1}^{N}FD^{i} }\frac{\sum_{i=1}^{N}FM^{i}_{2}\times FD^{i}}{\sum_{i=1}^{N}FD^{i}}\ \ldots\ \frac{\sum_{i=1}^{N}FM^{i}_{K-1}\times FD^{i}}{\sum_{i=1}^{N}FD^{i}}\ \frac{\sum_{i=1}^{N}FM^{i}_{K}\times FD^{i}}{\sum_{i=1}^{N}FD^{i}}\right] \tag{1}\]  
The weighted mean is used to convert the spatial feature map, \(FM^{N}_{K}\), listed over time into a spatiotemporal feature during the entire time, and the elements of each feature map are averaged by weighting \(FD^{N}\) at the corresponding time according to size \(K\) of spatial feature maps. \(FD^{N}\) is a temporal feature extracted from spatial information represented by the motion information of an object and background information, which changes according to the camera's point of view, so it can be matched with spatial features simultaneously when \(FM^{N}_{K}\) is extracted. According to Equation (1), the system divides the sum of the result multiplying the \(K\)th element of each feature map by \(FD^{N}\) at the corresponding time and the sum of \(FD^{N}\) to generate spatiotemporal features \(FV\). The elements of feature maps are generated as spatiotemporal features for action recognition through weighted mean calculations considering frame change rate during the entire time. Figure 4 is a result of schematizing the operation process when \(K=1\) and generates \(FV_{K}\) by repeating the operation \(K\) times according to the size of the spatial feature map. The weighted mean is weighted to the feature point of video when the motion or background information of an object changes greatly according to the time information of \(FD^{N}\), thereby enabling the context of the spatial feature that changes throughout the whole time. Finally, \(FV_{K}\) generated through the weighted mean is inputted to the FC layer of the MLP structure, and the MLP outputs the class of the predefined videos.