# Open-Vocabulary Part-Level Detection and Segmentation for Human-Robot Interaction
## 4 Results
### Main Result  
In this study, relevant experiments are performed based on the PartImageNet dataset and the Pascal Part dataset to evaluate the performance of the proposed method. Additionally, the proposed method is compared with some representative methods by using the average precision (AP), mean average precision (mAP), and \(mAP_{50}\) metrics, with a higher value representing better performance in part-level detection and segmentation. AP is defined as the average value of 10 IOU (Intersection over Union) thresholds when the IOU takes 0.05 as a single step. mAP is obtained by mean computing the AP for each category separately. \(mAP_{50}\) refers to the mAP when the IOU threshold is 0.5. These calculation methods for the above evaluation indexes are shown in Equations (8) and (9). Given the difference in the number of parts per category, mAP is defined as the mean of AP for all categories in the dataset, which may mitigate this influence to some extent. Moreover, \(mAP_{50}\) explicitly defines the IOU threshold, facilitating a more precise evaluation of model performance. The results are presented in Table 1. The part-level segmentation using the proposed method based on PartImageNet and Pascal Part datasets is visualized in Figures 5 and 6, respectively. Although the proposed method is only trained via the ground-truth annotation from the PartImageNet and Pascal Part training datasets, it generates high-quality part segment masks that align well with ground-truth masks. In the application based on the PartImageNet dataset and the Pascal Part dataset, this method can learn better representation for part features whether in detection or segmentation, which corroborates its good performance.  
\[AP=\int_{0}^{1}P(r)dr \tag{8}\]  
\[mAP=\frac{\sum_{i=1}^{n}AP_{i}}{n} \tag{9}\]  
where \(P(r)\) represents precision when recall is \(r\); \(n\) represents the total number of parts classes in the dataset; \(AP_{i}\) represents the AP of the \(i\)-th class part.  
Figure 4: The distribution of each category in the two datasets.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**PartImageNet**} & \multicolumn{3}{c}{**[Part of the Object]**} & \multicolumn{3}{c}{**[An Object Part]**} \\  & **AP** & \(mAP_{\mathbf{50}}\) & **mAP** & **AP** & \(mAP_{\mathbf{50}}\) & **mAP** \\ \hline Detection & 47.63 & 75.46 & 46.04 & 47.40 & 74.92 & 46.42 \\ Segmentation & 44.36 & 72.39 & 42.05 & 44.29 & 72.80 & 42.53 \\ \hline \multirow{2}{*}{**Pascal Part**} & \multicolumn{3}{c}{**[Part of the Object]**} & \multicolumn{3}{c}{**[An Object Part]**} \\  & **AP** & \(mAP_{\mathbf{50}}\) & **mAP** & **AP** & \(mAP_{\mathbf{50}}\) & **mAP** \\ \hline Detection & 25.27 & 52.35 & 35.34 & 23.01 & 49.71 & 32.62 \\ Segmentation & 24.04 & 50.86 & 35.05 & 22.53 & 48.41 & 32.85 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The results of part-level detection and segmentation via Swin Transformer using two prompt templates in PartImageNet and Pascal Part.  
Figure 5: The qualitative results of part-level segmentation and part detection in Pascal Part. (**A**) The original images sourced from Pascal Part are used as the input for the model. (**B**) The prediction results from the model trained on Pascal Part illustrate the segmentation and detection effects.  
Figure 6: The qualitative results of part-level segmentation and part detection in PartImageNet. (**A**) The original images sourced from PartImageNet are used as the input for the model. (**B**) The prediction results generated by the model trained on PartImageNet demonstrate the segmentation and detection effects.