SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure B.12: Qualitative results of cross-dataset part segmentation on Pascal-Part-116. Pascal-Part-116 provides more fine-grained part annotations for the "person" category, such as "hair" and "upper arm". The model trained on ADE20K-Part-234 demonstrates the ability to recognize "hair" but struggles to generalize from "arm" to "upper arm" and "lower arm" accurately. Moreover, the model exhibits potential in generalizing parts of the "airplane" category. Although ADE20K-Part-234 annotates the parts as "door", "fuselage", "landing gear", "propeller", "stabilizer", "turbine engine", and "wing", the model can generalize them to Pascal-Part-116's parts, including "body", "stern", "wing", "tail", "engine", and "wheel", despite the differences in vocabulary and granularity. Notably, ADE20K-Part-234 does not contain related classes to "bird", "sheep", and "potted plant", but the model demonstrates a certain level of generalization ability to segmenting these categories.
Figure B.13: Qualitative results of cross-dataset part segmentation on ADE20K-Part-234. For the categories "car" and "bus", the part annotations in Pascal-Part-116 are more coarse-grained. When tested on ADE20K-Part-234, the model trained on Pascal-Part-116 can predict novel parts like "logo", "wiper", "hood", and "bumper". However, the segments and part labels don't align accurately. For example, the model still segments the "bus's roof", which is annotated in Pascal-Part-116, but wrongly assigns it to "bus's bumper" in ADE20K-Part-234. This showcases the challenge of generalizing across different granular part definitions. For the novel object "swivel chair", the model adeptly delineates part boundaries even without relevant objects in Pascal-Part-116. But the category errors are still present. In the case of the "person" category, the model only segments the "upper arm", which demonstrates the difficulty of generalizing from "upper/lower arm" to "arm".
ADE20K-Part-234 are shown in Figure B.7 and Figure B.9 respectively. We report the qualitative results for the Few-Shot Part Segmentation on Pascal-Part-116 in Figure B.10 and on ADE20K-Part-234 in Figure B.11. And the results for the Cross-Dataset Part Segmentation on Pascal-Part-116 are shown in Figure B.12. Furthermore, we present the qualitative results for models trained on Pascal-Part-116 and then tested on ADE20K-Part-234 are shown in Figure B.13.
D Future Works and Negative Societal Impacts
Although part-level OVSS indeed presents more challenges compared to object-level OVSS, the OV-PARTS benchmark datasets have lower quality than existing object-level OVSS benchmark datasets. The original version of Pascal-Part and ADE20K-Part are annotated without considering the open vocabulary scenario especially the analogical reasoning ability and open granularity ability that we care about in a part-level OVSS model. The benchmark datasets need to be continuously expanded and improved to encompass more diverse and complex object-part annotations. There may be potential negative societal impacts associated with the OV-PART benchmark. The deployment of fine-grained part segmentation models in various real-world applications may lead to unintended consequences. We must ensure that the predictions be reliable and accurate in critical applications, such as medical diagnosis or autonomous vehicles. Also, there is a possibility of misuse of part segmentation technology for malicious purposes, such as creating deepfake images or spreading misinformation. Ensuring security measures and appropriate regulations to prevent such misuse is vital in the development and deployment process.