# Body-Part-Aware and Multitask-Aware Single-Image-Based Action Recognition
## 4 Experiments
### Evaluation of the Action Recognition Task on the V-COCO Dataset  
To further validate the usefulness of our multitask-aware loss function, we performed experiments on the V-COCO Dataset [16]. Because the V-COCO Dataset is designed for human object interaction benchmark, a single image contains multiple human actions and multiple object labels. However, our focus was to perform action recognition for a single person. Thus, we created training and testing images by transforming each person at the center using a bounding box. We solved multi-labels action recognition task for the centered person on the V-COCO Dataset. We used the BCE [38] loss function for the multi-labels problem. This is different from the action loss function used on the Stanford 40 Actions Dataset. This shows the flexibility of multitask-aware loss function in Equation (3). Table 5 depicts comparison of the mAP obtained on the V-COCO Dataset through different feature streams and that of the of ensemble of multi-streams. The mAP obtained are 64.59% (image-based feature stream), 66.30% (attention-image-based feature stream), 62.50% (part-image-based feature stream) and 72.36% (Proposed: Ensemble of (Image + Part + Concatenation)). To analyze the effectiveness of different feature streams in our proposed method, we performed experiments by training the models independently with these feature streams excluded from the architecture and including them. In Table 6, \"Baseline 1 (trained with Image)\" represents the basic multitask learning framework with only one stream (image-based feature stream) for action recognition. \"Baseline 2 (trained with Image + Part)\" represents performance when including the part-image-based feature stream on training the deep network. \"Proposed (trained with Image + Attention + Part)\" is trained including three feature streams (image-based, attention-image-based, and part-image-based). As can be seen in Table 6, Baseline 1 achieved a mAP of 68.08%, which was improved by a large margin when trained with the part-image-based feature stream included, reaching a mAP of 69.15% (Baseline 1). Overall, the proposed training method that includes the three modules for action recognition (i.e., image-based, part-image-based, and attention-image-based feature streams) obtained the mAP of 72.36%. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **mAP** \\ \hline Action-Specific Detectors [26] & 75.4 \\ \hline VGG-16\&19 [42] & 77.8 \\ \hline TDP [29] & 80.6 \\ \hline ResNet-50 [23] & 81.2 \\ \hline Action Mask [28] & 82.6 \\ \hline Part-Based Network in [27] & 89.3 \\ \hline Part Action Network in [27] & 91.2 \\ \hline Proposed Method & **91.91** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of mAP with other methods on the Stanford 40 Actions Dataset [15]. Bold font indicates the best result. \begin{table}
\begin{tabular}{l c} \hline \hline
**Feature Representation** & **mAP** \\ \hline Image-based feature stream & 64.59 \\ \hline Attention-image-based feature stream & 66.30 \\ \hline Part-image-based feature stream & 62.50 \\ \hline Proposed: Ensemble of (Image + Part + Concatenation) & **72.36** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of mAP obtained through feature representation from multiple streams for action recognition on the V-COCO Dataset [16]. Bold font indicates the best result. In Figure 7, we compare the class activation mappings [43] of the image-based feature stream on Baseline 1 (trained with Image), Baseline 2 (trained with Image + Part), and Proposed (trained with Image + Attention + Part). According to the experimental results, the image-based feature stream in Baseline 1 can focus on both global and local information. Because Baseline 1 consists of only one feature stream for action recognition (the image-based feature stream), it is optimized to focus on both global and local information. On Baseline 2, a part-image-based feature stream is included in training, which is designed to focus on the local information.