# Fine-Grained Action Recognition by Motion Saliency and Mid-Level Patches
## 5 Experiments
### Comparison with State-of-the-Art Methods  
Table 4 presents the experimental results of our fine-grained action recognition algorithm against the state-of-the-art methods. From Table 4, we can see that the proposed method performs well. On the MPII cooking database, our method shows an overall mAP of 69.8%, which is significantly better than the baseline [1] (57.9). The proposed method outperforms the Hierarchical Mid-Level Actions [11], Interaction Part Mining (Max Pooling) [10], IDT-FV (Improved Dense Trajectory, Fisher Vector) [31], P-CNN (Pose-Based CNN) [31], Binary Hashing CNN Features [32], and the Order-Constrained Kernelized Feature [33]. Our algorithm is comparable to the Semantic Features [9] and Interaction \begin{table}
\begin{tabular}{c c} \hline \hline
**Color Space** & **mAP(\%)** \\ \hline RGB & 53.7 \\ nRGB & 54.4 \\ Lab & 53.5 \\ rgl & 52.1 \\ HSV & 47.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Recognition accuracies for different color spaces (%). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**With or Without Size Factor** & **Weight of Color Space** & **Weight of Saliency** & **Weight of Size** & **mAP(\%)** \\ \hline with size factor & 0.33 & 0.33 & 0.33 & 54.4 \\ without size factor & 0.5 & 0.5 & 0 & 49.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Recognition accuracies with or without region size factor (%). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Feature** & **Pr** & **Rc** & **mAP** \\ \hline HOG & 43.5 & 38.2 & 59 \\ HOF & 45.5 & 40.8 & 59.4 \\ MBHx & 45.2 & 39.7 & 61.2 \\ MBHy & 45.3 & 41.5 & 66.6 \\ MBH & 49.0 & 42.5 & 67.4 \\ Combined & 52.0 & 46.0 & 69.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Recognition accuracies of different features (%). Part Mining (Max-N Pooling) [10] methods. The Semantic Features method [9] requires object detection, while our method partitions object proposal regions in motion saliency maps without manual annotation. Interaction Part Mining (Max-N Pooling) [10] detects \(N\) instances for each patch detector to gain better results with abundant information, whereas our method only detects one instance for each detector. The per-class classification results for these three algorithms on the MPII cooking database are shown in Figure 6 (only 54 categories are shown). We compare our method with the Semantic Features [9] and Interaction Part Mining (Max-N Pooling) [10] methods. As shown in Figure 6, the proposed algorithm achieves better or comparable results for most categories of actions. For actions with visible operated objects, such as \"change temperature\", \"cut slices\", and \"open tin\", the recognition accuracies of our algorithm are the best. Meanwhile, our method and the Semantic Features method are comparable for other categories. Unlike the Semantic Features method, our algorithm does not require manual annotation, which shows that the segmentation of interaction components based on motion saliency is effective. For actions in which operated objects are difficult to detect, such as \"cut dice\", and \"cut in\", the results of the three algorithms are comparable. When the category of action has more samples, our method and the Semantic Features method perform well, such as \"take out from drawer\" and \"wash objects\". When the category has fewer samples, the recognition effect of Interaction Part Mining (Max-N Pooling) is good, such as \"cut dice\" and \"lid: remove\", but the advantage is not obvious. The reason may be that Max-N Pooling can extract more effective information for small sample categories, but our algorithm can obtain more effective information when the number of samples is enough. That is to say, our algorithm