# Body-Part-Aware and Multitask-Aware Single-Image-Based Action Recognition
## 4 Experiments
### Evaluation of the Action Recognition Task on the Stanford 40 Actions Dataset  
In this section, we present experimental results for action recognition by highlighting the importance of the feature streams and compare the performance with the existing methods on the Stanford 40 Actions Dataset. The action recognition task in the proposed method has three complementary streams. Each of these streams was trained with a loss function. Thus, we report the mAP on each of these streams and the concatenation-based feature in Table 3. The mAP obtained on individual feature streams are: 85.57% \begin{table}
\begin{tabular}{l l l l l l l l l} \hline \hline
**Arch** & **Head** & **Shoulder** & **Elbow** & **Wrist** & **Hip** & **Knee** & **Ankle** & **Mean** \\ \hline pose\_hrnet\_w32 [14] & 97.1 & 95.9 & 90.3 & **86.4** & 89.1 & **87.1** & **83.3** & 90.3 \\ \hline pose\_hrnet\_w32\_aug0 [14] & **97.3** & **96.1** & **90.9** & 86.2 & **89.2** & 87.0 & 82.8 & **90.4** \\ \hline proposed\_method & 96.7 & 95.4 & 88.7 & 83.3 & 87.5 & 84.4 & 79.4 & 88.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation of the pose estimation task in our multitask-based method versus pose estimation as a single task on the MPII Human Pose Dataset [17]. The performance metric used is [EMAIL_ADDRESS] [17]. Bold font indicates the best result. (image-based feature stream in Section 3.2), 85.38% (attention-image-based feature stream in Section 3.3), and 84.27% (part-image-based feature stream in Section 3.4). On concatenation-based action decision in Figure 1 (Image + Attention + Part), we obtained mAP of 91.76%. As depicted in Figure 1, each of three streams is a component of the entire end-to-end trainable network. Thus, the concatenation feature can learn the combination of features extracted by the three streams, achieving better performance than those by three streams. Additionally, we evaluated the ensemble of the estimated action recognition result from multiple streams of the proposed method because the ensemble technique [41] is a method commonly used to improve generalized performance. We denote it as \"Ensemble of (Image + Part + Concatenation)\" in Table 3. The obtained mAP is 91.91%. We report the ensemble of (Image + Part + Concatenation) as the proposed method for comparison. Figure 6 depicts the average precision (AP) of detailed action classes obtained from three feature streams and the result of the ensemble technique in Table 3. The features learned from the three feature streams are complementary; the performance of one is better than the others in some action classes, and vice-versa. Some notable classes are \"Blowing bubbles\" (95.00%), \"Brushing teeth\" (84.99%), \"Phoning\" (73.92%), and \"Smoking\" (73.11%) that perform better for the part-image-based feature stream in comparison to image-based feature stream and attention-image-based feature stream. One thing common to these action classes is that the major distinction is based on interacting objects and the parts of the human body. The part-image-based feature stream is designed to focus on body parts (head, right hand, and left hand) and objects around them. Such action classes have higher APs for part-image-based feature stream. However, on other actions such as \"Jumping\", \"Running\", and \"Walking a dog\", the part-image-based feature stream performs worse than other feature streams. The image-based feature stream obtained an AP of 97.62% on \"Jumping\" and 93.61% on \"Running\". Similarly, attention-image-based feature stream obtained an AP of 97.41% and 93.47%, respectively, on the same set of actions. For actions such as \"Jumping\", \"Running\", \"Walking the dog\", and \"Gardening\", the human pose and background information is much more relevant than the detailed body parts information. Overall, the \"Proposed: Ensemble of (Image + Part + Concatenation)\" is the best. It means that each of the feature streams learns complementary features for action recognition, as we intended.