# Open-Vocabulary Part-Level Detection and Segmentation for Human-Robot Interaction
## 5 Conclusions  
In this paper, a fine-grained segmentation method is proposed based on vision language models to ensure that robots can perceive surrounding environments during interaction with humans. To enhance the feature extraction capabilities of this method, Swin Transformer is introduced into the backbone of the image encoder. Additionally, the influence of different templates of text prompts on the detection and segmentation of parts is also analyzed. Compared with different prompt types and other methods, the fine-grained segmentation effect of the proposed method is better based on the two datasets. Additionally, the \(mAP_{50}\) of the proposed method reaches 75.46% (PartImageNet) and 52.35% (Pascal Part) in the detection task and 72.39% (PartImageNet) and 50.86% (Pascal Part) in the segmentation task using [part-of the object]. The results of this study are expected to lay a foundation for the interaction between robots and humans and the development of the intelligent agent. In the future, this model may be applied to various applications, such as robot manipulation, robot obstacle avoidance, and robotic crop picking.  
Conceptualization, S.Y.; methodology, S.Y.; formal analysis, S.Y. and X.L.; investigation, X.L.; data curation, S.Y.; writing--original draft preparation, S.Y.; writing--review and editing, X.L. and W.W.; visualization, S.Y.; supervision, W.W.; project administration, W.W.; funding acquisition, W.W. All authors have read and agreed to the published version of the manuscript.  
This work was supported by National Natural Science Foundation of China [grants No. 61573148 and 61603358] and Science and Technology Planning Project of Guangdong Province, China [grants No. 2015B010919007 and 2019A050520001].  
Not applicable.  
Data Availability Statement:Data is contained within the article.  
Conflicts of Interest:The authors declare no conflicts of interest.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**BiFPN**} & \multicolumn{3}{c}{**FPN with P6**} \\ \hline
**Detection** & **AP** & \(mAP_{50}\) & **mAP** & **AP** & \(mAP_{50}\) & **mAP** \\ \hline PartImageNet & 41.27 & 66.31 & 38.90 & 46.09 & 73.92 & 44.76 \\ Pascal Part & 23.84 & 50.03 & 34.49 & 24.26 & 51.43 & 34.21 \\ \hline
**Segmentation** & **AP** & \(mAP_{50}\) & **mAP** & **AP** & \(mAP_{50}\) & **mAP** \\ \hline PartImageNet & 38.81 & 64.26 & 35.90 & 43.15 & 71.40 & 41.12 \\ Pascal Part & 22.97 & 48.65 & 34.44 & 23.40 & 49.88 & 33.93 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Evaluation results of performance indicators for BiFPN and the original FPN using [part of the object].  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Detection** & **AP** & \(mAP_{50}\) & **mAP** \\ \hline PartImageNet & 39.07 & 62.81 & 38.16 \\ Pascal Part & 13.29 & 26.73 & 27.68 \\ \hline
**Segmentation** & **AP** & \(mAP_{50}\) & **mAP** \\ \hline PartImageNet & 37.03 & 60.46 & 34.90 \\ Pascal Part & 13.15 & 26.29 & 29.01 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Evaluation results of performance indicators without FPN using [part of the object].