# Open-Vocabulary Part-Level Detection and Segmentation for Human-Robot Interaction
## 4 Results
### Ablation Study  
To verify whether part-level detection and segmentation are related to the text embedding, which contributes to promoting the component of part-level classification, the efficiency of different templates of text prompts is investigated in terms of part-level detection, and the capability of part segmentation without text prompts is also explored. More specifically, two familiar expressions are used in this experiment, including [part of the object] and [an object part], such as [a bottle cap] and [cap of the bottle]. As shown in Figure 7B and Table 3, the text embedding of both prompts can affect the accuracy of part location and the precision of part segmentation to some extent. The \(mAP_{50}\) of [part of the object] is slightly superior to [an object part] in PartImageNet. Similarly, the efficiency of different prompt templates in Pascal Part also exerts an impact. In this study, we simply demonstrate that the object information also affects the results. To draw a fair comparison, the quantity of parts categories in part segmentation without text prompts is related to labels both the part and the object, not only depend on the name of the part. Different prompt templates are the key factors influencing the performance of part-level detection and segmentation. Hence, a commonly better text prompt for part segmentation needs to be verified based on more datasets. Furthermore, it is an open problem that requires advanced prompt engineering in part segmentation. In the process of fusing image features from the image encoder, it can be verified that the multi-scale features generated from the FPN exert significant impacts on the results of part-level segmentation and detection, as shown in Tables 4 and 5. These results indicate that regardless of which FPN is employed in the image encoder, the results with the FPN are consistently better than those without the FPN. The structure of the FPN is crucial in part-level recognition, as it can effectively fuse semantic information at different scales. Without the FPN structure in the image encoder, the lowest AP and mAP of the part-level detection and segmentation can be observed in both Pascal Part and PartImageNet. In this study, the FPN with P6, the FPN without P6, and BiFPN are employed for a comparative analysis based on the PartImageNet and Pascal Part datasets. Additionally, the FPN with P6 and the FPN without P6 are also utilized in the PartImageNet and Pascal Part datasets to prove that P6 is not suitable for part-level recognition. Moreover, BiFPN is leveraged as a reference to compare the influence of different structures of the FPN. BiFPN involves a bidirectional feature propagation and fusion structure that not only has a top-down feature fusion path but also possesses a bottom-up path. Compared with the FPN with P6, the FPN without P6 raised 0.99% higher \(mAP_{50}\) on PartImageNet and 0.98% \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**PartImageNet** & \multicolumn{3}{c}{[**Part of the Object]**} & \multicolumn{3}{c}{[**An Object Part]**} \\  & **AP** & \(mAP_{50}\) & **mAP** & **AP** & \(mAP_{50}\) & **mAP** \\ \hline Detection & 35.99 & 65.68 & 35.83 & 36.36 & 65.82 & 36.07 \\ Segmentation & 35.16 & 61.91 & 33.39 & 35.54 & 62.12 & 33.78 \\ \hline \hline \multirow{2}{*}{**Pascal Part**} & \multicolumn{3}{c}{[**Part of the Object]**} & \multicolumn{3}{c}{[**An Object Part]**} \\  & **AP** & \(mAP_{50}\) & **mAP** & **AP** & \(mAP_{50}\) & **mAP** \\ \hline Detection & 18.62 & 43.46 & 25.98 & 17.44 & 41.28 & 24.36 \\ Segmentation & 18.96 & 42.06 & 27.43 & 18.09 & 40.26 & 26.64 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The results of part detection and segmentation via Retia using two prompt templates based on PartImageNet and Pascal Part.