# Temporal Modeling on Multi-Temporal-Scale Spatiotemporal Atoms for Action Recognition
## 4 Evaluation
### Qualitative Run-Time Analysis  
Compared with the unordered modeling in original Res3D work, our temporal method improves the recognition accuracy from 87.6% to 92.9% but due to the dense clip spits, two streams, multiple temporal scales and atoms mining involved, our method would spend much more time on recognition. In this section, we provide a qualitative run-time analysis of the proposed atoms temporal modeling method and compare it with unordered modeling method in the original Res3D work [13]. We only analyze the time consumption of the testing phase, which is a major focus in computer vision tasks. In our method, the testing phase contains optical flow extraction, feature extraction, atoms mining, temporal modeling and fusion. In the unordered modeling method, the testing phase contains optical flow extraction, feature extraction, clip merging and classification. For simplicity, we ignore the optical flow extraction which exists in both of our method and unordered method; we also ignore fusion in our temporal modeling and clip merging in unordered modeling, which take little time.  
We present the measurement of each step in Table 7. All measurements were conducted on a computer with an Intel Core i7-5930K 3.5 GHz 12 CUP, GeForce GTX TITAN, 62 GB RAM and Ubuntu 14.04 LTS. In the atoms mining for a video, each iteration requires \(O(N^{4})\) to evaluate Equation (5), where \(N\) is the number of dense clips. We only present several measurements of atom mining for the videos with special number of dense clips.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Sparse Clips** & **Atoms** \\ \hline Unordered Modeling & 87.1\% & 86.9\% \\ Temporal Modeling & – & 87.9 \\ \hline \hline \end{tabular}
\end{table}
Table 6: The results matrix of the performance evaluation experiment in the RGB stream of 8f temporal scale on UCF101 split-1.  
With these measurements, it is easy to obtain the qualitative time consumption. Take a video action with 256 frames (about 8500 ms for 30 fps video) as an example, the time consumption of the recognition can be calculated as follows:  
\[\text{Unordered modeling}:256\times 2.99+367.44=1132.88\text{ ms} \tag{12}\]  
\[\text{Our method}:2\times 2\times 256\times(3.96+2.99+1.91)+2\times(1197.43+282.62 +0)+2\times 3\times 0.46=12,035.5\text{ ms} \tag{13}\]  
In Equation (13), the first and second '2' denote the two streams and dense clips in feature extraction; the third and fourth '2' denote the two streams in atoms mining and temporal modeling. 1197.43, 282.62 and '0' denote 128 dense clips in 4f temporal scale, 64 dense clips in 8f temporal scale and 32 dense clips in 16f temporal scale, respectively. In this example, to recognize the action in an 8.5 s video with 30 fps, unordered modeling method spends 1.1 s and our temporal modeling method spends 12.04 s.  
This run-time analysis shows that our temporal modeling method improves the recognition accuracy at the cost of run-time. For a real-life application, this limitation of our method should be considered. And the rough time consumption of the recognition can be calculated using the measurements in Table 7.