Open-Vocabulary Part Segmentation via Progressive and Boundary-Aware Strategy
Authors: Xinlong Li
Di Lin
Shaoyiyi Gao
Jiaxin Li
Ruonan Liu
Qing Guo
Venue: NeurIPS 2025 poster ()
Keywords: Open-Vocabulary Part Segmentation
Progressive Segmentation
Boundary-Aware Refinement
Hierarchical Part Connected Graph
Primary area: applications
TL;DR: We propose a novel training-free OVPS framework, PBAPS, using HPCGraph and BAR to improve structural part boundary segmentation.

Abstract: Open-vocabulary part segmentation (OVPS) struggles with structurally connected boundaries due to the inherent conflict between continuous image features and discrete classification mechanism. To address this, we propose PBAPS, a novel training-free framework specifically designed for OVPS. PBAPS leverages structural knowledge of object-part relationships to guide a progressive segmentation from objects to fine-grained parts. To further improve accuracy at challenging boundaries, we introduce a Boundary-Aware Refinement (BAR) module that identifies ambiguous boundary regions by quantifying classification uncertainty, enhances the discriminative features of these ambiguous regions using high-confidence context, and adaptively refines part prototypes to better align with the specific image. Experiments on Pascal-Part-116, ADE20K-Part-234, PartImageNet demonstrate that PBAPS significantly outperforms state-of-the-art methods, achieving 46.35\% mIoU and 34.46\% bIoU on Pascal-Part-116. Our code is available at https://github.com/TJU-IDVLab/PBAPS.

Open-Vocabulayy Part Segmentation via Progressive and Boundary-Aware Strategy
Xinlong Life Di Lin
Shaoyiyi Gao
Jiaxin Li
Ruonan Liu
Qing Guo
'Tianjin University, China
Southwest University, China *VCIP, CS, Nankai University, China
*Shanghai Jiao Tong University, China
1xl zs@jt........ jnd.uu
Abstract
Open-vocabulary part segmentation (OVPS) struggles with structurally connected boundaries due the inherent conflict between continuous image features and discrete classification mechanism. To address this, we propose PBAPS, a novel training-free framework specifically designed for OVPS, PBAPS leverages struc- tural knowledge of object-part relationships to guide a progressive segmenta- tion from objects to fine-graine fineggrinedd parts. To further improve accuracy at chal- lenging boundaries, we introduce a Boundary-Aware Refinement (BAR) module that identifies ambiguous boundary regions by quantifying classification uncer- tainty, enhances the discriminative features of these ambiguous regions using high-confidence context, and adaptively refines part prototypes to better align with the specific image. Experiments on Pascal-Part-1 116, -BF20K-Part-234, PartImageNet demonstrate that PBAPS significantly outperforms state-of-the-art methods, achieving 46.35% mloU and 34.46% bloU on Miscal-Part-111 Our code is available at
Introduction
Semantic segmentation aims to assign each pixel in an image to a predefined class. Traditional methods [
and have achieved significant progress in closed-set but exhibit limited zero-shot generalizability in open- scenarios. -pen-vocabulary semantic segmentation (OVSS) [10 11 addresses this limitation by leveraging pre-traine pre-trained lision-language models (VLM) 13
5 6 are based supervised learning with labeled data 7
8 9
14
15 to enable segmentation unseen
classes. Existing OVSS methods can be broadly categorized into two paradigms: (1) direct matching
via cross modal similarity [16] [2] [7] and (2) mask classification frameworks [18] [9 20 based
on feature clustering. Although effective for object-level segmentation, their part-level performance
degrades significantly (21] Open-vocabulan part segmentation (OVPS) 22 17 23 faces the
additional challenge of ambiguous part boundaries.
We subdivide boundaries into three types: object boundary, structurally connected part boundary, and non-structurally connected part boundary. Structurally connected parts refer to anatomically adjacent components with direct physical connections (e.g., cat head and neck). Existing OVPS methods, including multigranularity segmentation based on object-part modeling 24 25 26 and fine-graine feature enhancement approaches (27) (28), commonly exhibit inaccuracies in segmenting intra- intra-object structurally connected part boundaries (Figure I These approaches neglect the key distinction between OVSS and OVPS: object-level segmentation depends on distinct feature variations across boundaries, while pert-level segmentation encounters smooth and continuous feature transitions (e.g..
"Co-first authors.
t *Coresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

fur texture/color changes) at structurally connected boundaries, in contrast to the moderate local feature differences non-structurally connected part boundaries.
Boundary Issue
Ground Truth
Cat torso-leg
Sheep torso-neck
Bird head-neck
Figure 1: Boundary issues of existing methods on Pascal-Part-l [21]. pataset-specifie part boundary definition variances limit the validity of pixel-alignment metrics for segmentation evaluation. For example, although the prediction boundary between cat neck and torso (PanCLIIPSeg (28). blue dotted) deviates from annotations, it remains anatomically interpretable. Our method prioritizes anatomicalll//tructurally implausible boundaries (red dotted).
The fundamental challenge in segmenting structurally connected part boundaries arises from the incompatibility between continuous, smooth image features and discrete classification mechanisms. Discrete classification enforces mutually exclusive label assignments and fails to account for pixels structurally connected part boundaries, which exhibit hybrid characteristics of adjacent parts and lack clearly distinguishable features. To achieve accurate segmentation, is essential to enhance the discrimination of these ambiguous features. Furthermore, since label assignment relies category feature representations, adaptive optimization category features can synergistically improve classification precision.
We propose the Boundary-Aware Refinement (BAR) module to refine the structurally connected part boundary. The BAR (1) first locates ambiguous boundary regions by analyzing the matching cost. In this work, we use the cosine similarity between the pixel features and part prototypes as the matching cost [29 20 The ambiguous regions then separated from the original part masks, leaving the remaining deterministic regions as reliable references.
(2) BAR then optimizes the pixel features within ambiguous regions by leveraging the context from deterministic regions and adaptively updates the part prototypes to better align with the current image characteristics.
(3) Finally, ambiguous boundary regions reclassified using enhanced pixel features and adapted prototypes.
In this paper, we propose Progressive Boundary-Aware Part Segmentation (PBAPS), a training OVPS framework that refines structurally connected part boundaries. Specifically, we first generate visual prototypes for each part class using Stable Diffusion [30], SAM [31], and DINOv2 [32]. Then, based part structural relationships, Hicrarchical Part Connected Graph (HPCGraph) is constructed. Guided by this HPCGraph, progressive part segmentation is performed, where the BAR module mitigates boundary ambiguities and enhances segmentation precision.
In summary, the contribution of this paper is threefold: (i) We reveal the intrinsic cause of structurally connected part boundary errors: the conflict between continuous image features and discrete clas- sification. (ii) We propose the BAR module, which improves boundary precision through feature optimization and dynamic prototype adaptation. (iii) We propose PBAPS, novel and effective training-free OVPS method that integrates hierarchical reasoning with iterative boundary refinement, achieving state-of-the-- on Pascal-Part P1r (11, P0D20KK-Par-225 [21) and PartImageNet (33].

---

2
Related Work
Open Vocabulary Semantic Segmentation. OVSS aims overcome the limitations of predefined
classes in traditional segmentation [34, 33
36
38
37
38 by enabling zero-shot segmentation of unseen
classes, requiring integration of the semantic understanding of VLM (e.g., CLIP [13] BLIP (14)
with pixel-level localization. Existing methods can be broadly categorized into two paradigms. is based on feature clustering and mask classification. OVSegmentor [39 One [O 16 12 17
leverages slot-attention [4]] for pixel grouping and text alignment, RIM [4]] employs image-to-imag tomiaaee
matching for training- free segmentation, constructing visual references and enhancing robustness via a relation-awac ranking distribution strategy. Our method generates part prototypes based on
[42 balances embeddings between base/novel classes and supplements spatial cues with SAM [31. The second paradigm focuses on the
the process of constructing visual references in RIM. EBSeg
alignment of the pixel-level features [8 19 20). which directly establishes associations between the
vision and text features the pixel scale. ZegCLIP [19] extends CLIP image-level classification
capability to the pixel level. CAT-Seg [20] proposes a cost aggregation framework that optimizes CLIP image-text similarity through spatial and category aggregation.
Open Vocabulary Part Segmentation. Compared to OVSS, OVPS imposes higher demands on both model generalization and fine-grained recognition. Existing OVSS methods 43 20 21
often suffer significant performance degradation at the part level, mainly due insufficient object- part structural modeling and limited fine-grained feature extraction. Recent studies have made notable progress. OPS [24] introduces class-agnostic part segmentation via object-aware spatial constraints and self-supervised feature optimization. ViRReq [25] decomposes part segmentation into composable atomic requests, leveraging a knowledge base for multigranular parsing. ViRReq and TAPPS [4] have previously explored object-to-pare segmentation. In contrast, hierarchical reasoning in our method PBAPS introduces a cross cross-hierareee hicrarchy matching mechanism, fully exploiting hierarchical context to strengthen feature discrimination. VLPart (45) constructs joint vision- language embedding space through co- training,, using DINO (46) dense semantic correspondence to parse novel objects into known components. In terms of benchmark construction, OV-PARTS 21 establishes the first OVPS benchmark (Pascal-Part- 16, ADE20K-Part-255 with defined task scenarios. HIPIE [47] decouples hierarchical representations separate
features. WPS-SAM 48 proposes a weakly supervised framework based on SAM [31], which reduces annotation dependency through co-training of learnable part prompt tokens and bounding box/point supervision. In feature enhancement, OlParts (27] fuses DINOv2 (32] local features with Stable Diffusion 30 global representations via adaptive channel selection. ParCLIISeg (28] addresses fine-grat grained generalization by jointly optimizing separation and enhancement losses for part context modeling. ParCATTSeg 49 improves semantic discrimination by constructing part-
aware text embeddings combined with contrastive training, but lacks the ability to model structural relationships between parts.
3 Boundary Feature Gradient Analysis
To quantitatively analyze the feature variations across object boundaries, structurally connected part boundaries, and non-structurally connected part boundaries, we compute the spatial feature gradient. This feature gradient refers to the rate of spatial change in image feature vectors. Specifically, we extract pixel-wise features using DINOv2 (32. For the feature vector f(..j) E Ixdd of each pixel (i,j) in the image, we calculate the gradients in the horizontal (z) and vertical (y) directions for each channel c:
Cz(i..c) (s +,,,,) -.j, f(i,,,)), G,(i, + -
(1)
For each pixel (i,j), we aggregate the gradients across all channels using the Euclidean norm to obtain the overall spatial feature gradient: Grad(i, = VEL (G.(i.j, c)2 + (i,,,,))).. A higher value of Grad(i,j) corresponds to more pronounced local feature variation, whereas lower value indicates smoother feature transitions. We extract these gradient values for all pixels located within the ground-truth masks of each boundary type and perform a systematic statistical analysis.
As shown Table the mean gradients and standard deviations (SD) exhibit consistent patterns across all three datasets: the object boundary shows the highest mean and SD values, while the
3

---

Table 1: Quantitative analysis of boundary feature gradients on Pascal-Part-116 21 ADE20K-PARS 234 (2I), and PartimageNet 33 The mean reflects the overall intensity of feature changes the boundary. The standard deviation indicates the spatial consistency of boundary feature changes.
Pascal-Part 116
ADE20K ADE20K-Part-223 Part-234
PartImageNet
Boundary Type
Mean
SD
Mean
SD
Mean
SD
Object
0.5291
0.1300
0.5102
0.1253
0.5487
0.1352
Non-structurally connected part
0.5043
0.1164
0,4875
0.1108
0.5036
0.1204
Structurally connected part
0.4356
0.0762
0.4219
0.0721
0.4015
0.0658
Raw Image
Ground Truth
Feature Gradient
Figure 2: Heatmap of feature gradients. (1) Object boundary: feature gradients at the boundaries between human/dog and other objects are the highest. (2) Non-structurally connected part boundary (blue dotted): these have notably higher gradients than interior regions, indicating clear local vari- ations. (3) Structurally connected part boundary (red dotted): feature gradients are relatively low, reflecting smooth feature transitions between adjacent parts.
structurally connected part boundary presents the lowest values. This indicates that, regardless the scale and diversity of the dataset, the object boundary exhibits the most significant feature differences and the most drastic changes. The structurally connected part boundary exhibits smooth feature transitions, while the non-structurally connected part boundary shows moderate feature changes, reflecting local differences. These results are consistent with Figure [ provide theoretical support for optimizing structurally connected part boundaries in the BAR module.
The low feature gradients at structurally connected part boundaries reflect the conflict between the continuous feature space and the discrete semantic space. When pixel features exhibit smooth transitions between structurally connected parts, their feature vectors blend characteristics from both adjacent parts, making it difficult for discrete classification methods based thresholds similarity metrics to assign accurate labels. Moreover, the low feature gradients at these boundaries violate the common assumption that "boundaries have high gradients", rendering conventional boundary localization ineffective for such boundary.
To address these challenges, we design the Boundary-Awart Refinement (BAR) module, which explicitly locates ambiguous and deterministic regions at structurally connected part boundaries through cost-divergence maps. By leveraging context from deterministic regions, BAR enhances the discriminative features of ambiguous regions and adaptively optimizes part prototypes, therehy improving the segmentation accuracy of structurally connected part boundaries.

---

Method
Given an input image, the OVPS method is required to assign the correct part class label to each pixel. Unlike sett semantic segmentation, OVPS allows the test set Dtest to contain unseen part classes Cusseen that are not included the training set. our training- free framework, all classes in the test set belong to Canseen
present PBAPS, training-free OVPS framework that operates in three stages: (1) generating visual prototypes via foundation models, (2) constructing HPCGraph based on structural prior knowledge, and (3) performing hierarchical segmentation guided by HPCGraph. during which the BAR module refines the structurally connected part boundaries.
As illustrated in Figure
out head paw our mill shoup shou hrad
Adjacmary
Stable Diffission
SAM
1/2
Suppoot
nack
sule
Part classes
Synthesized images
Atieutionn maps
Part masks
Va
pillar
wondown
DINOv2
Ve
Cross-Model Prototype Induction
prolotypes
HPCGraph
Guide
DINOv2
BAR
imuge
Per-pixel features
Cat mask M.
Vi
V/
nnch
bool
HOME
Progressive Boundary-Aware Part Segmentation
Figure 3: The architecture of PBAPS. Object-level segmentation is first performed through feature- prototype matching in Vi, generating the object mask M, Makingg the cat mask Mj as an example, PBAPS then calculates part-level matching costs between the pixels and the corresponding part prototypes in V2, yielding the initial part mask M The BAR module subsequently refines structurally connected part boundaries in M.aww obtain the optimized part mask M2.
4.1 Cross-Model Prototype Induction
It is important to note that the paradigm of semantic segmentation via visual prototype construction has been adopted and validated as effective in existing studies, such as OVDiff [50] and RIM [41] The novelty of this work does not lie in the prototype construction itself, but rather in addressing the low segmentation accuracy of the structurally connected part boundaries in OVPS. Building upon the prototype-based paradigm, our method performs hierarchical segmentation guided by the HPCGraph, while the Boundary-Awe Aware Refinement (BAR) module further improves boundary precision through feature optimization and dynamic prototype adaptation.
To generate visual prototypes for part classes, we follow the proven pipeline 41 that integrates Stable Diffusion [30] SAM [31] and DINOv2 [32]. (1) First, Stable Diffusion generates synthetic images for each part e in Drest using text prompts (e.g., "a photo of c²), while extracting cross-attention maps to localize the target part across multiple layers and timesteps [0] 41 These attention maps are processed via normalization and thresholding to generate a high- nigh-confidence region mask M. (2) Within regions where M = 1, we randomly sample & foreground prompt points for SAM, which produces binary mask corresponding
part c. (3) Finally, DINOv2 extracts features from the masked regions of the synthetic images, and the visual prototype p. E R for e is computed as the average feature vector in these regions, serving as a global semantic representation for subsequent matching and boundary refinement.
5

---

4.2 Hierarchical Part Connected Graph
We construct the Hierarchical Part Connected Graph (HPCGraph) based on part structural relation- ships. The HPCGraph is defined as G = (V.E), where the node set V = (Vo. Vi.... V2 consists of L+1 hierarchical layers. Specifically, Vo {root} serves the starting symbol. denotes the object nodes, and Vi (1 > 1) represents the part nodes of different granularity. The edge set & comprises two types of relationships: (1) Hierarchical edge <p->c from the parent node Up Vi the child node tic Vi+1. representing the "composition" relationship (e.g.. "cat head" - "cat eye"). (2) Adjacency edge Ei-j between nodes 0i, V/ E Vi they have structural connection (e.g., "cat head" ++ "cat neck").
Cruss-hierarchy Matching Cost. To effectively exploit object-paut part and part-part relationships, we define the matching cost between pixel feature f E R. 1xd and node u V as the maximum cosine similarity within its dominance set D(v) = {u} {descendant nodes u):
. Pu [if] PP
S(S v)
max (ED(v)
4.3 Progressive Boundary-Award Part Segmentation
Given an input image E 1xxxx we extract pixel features F E hxxuxd using DINOv2, adopting wenddow strategy (52) to preserve fine spatial details. The progressive part segmentation is defined as Segment F, Vi) + Mr, where V denotes the node set the layer and M1 represents the corresponding segmentation mask produced for the layer
Object-level Segmentation (1 = 1). For each pixel feature f(i,j) in the feature map F, we calculate cross-hierarchy matching costs with all nodes in Vi. The resulting object-level segmentation mask M1 is obtained by assigning to each pixel (i,j) the node with the highest matching score: M1 (i.j) arg max S (f(i,)),
Part-level Segmentation (1 > 2). For each parent node E Vi-1, perform localized segmentation within its corresponding mask Mi- (p). Specifically, extract the parent-region feature F. F @ Mp) and calculate the cross-hierarchy matching costs between F, and its child nodes C' E Vi to generate raw mask M arg maX.eC S(Fp.c). c). The overall segmentation mask at layer is M UPEn Mip. Subsequently, the BAR module optimizes structurally connected part boundaries through feature optimization and prototype adaptation, generating the final refined masks Mr. The progressive part segmentation ends when the current layer contains only atomic parts (i.e., Vo Vi, C(v) = 0).
4.4 Boundary-Awar Refinement
The BAR module refines structurally connected part boundaries via four steps, as shown in Figure
Computing difffornne
larro
Matching
Cost-divergence
regions
Ope.
Adap.
Refined mask
Figure 4: Example of Boundary-Aware Refinement. When segmenting the cat neck and torso, the corresponding matching-cost maps are obtained. Taking their absolute difference quantifies the classification ambiguity, where salient regions indicate similar matching costs. Thresholding this difference map yields ambiguous regions (shaded areas), and the rest deterministic regions. Ambiguous-region feature optimization and prototype adaptation are then applied to refine the boundary between the neck and torso.

---

Ambiguous Boundary Region Localization. To address the uncertainty at structurally connected part boundaries, we first quantify the classification ambiguity using a cost-divergence map defined as DA,B = AAA SBl. where SA, SB E RR denote the matching cost maps of the structurally connected parts and B. The low divergence values correspond to regions where both parts exhibit similar matching costs, indicating semantic ambiguity. We normalize D and apply threshold to identify the ambiguous region threshold. The high-confidence deterministic regions are then obtained by excluding U from the initial part masks Araw, Braw'
1 (Norm((AAB) S A), where E [0, 1] is the ambiguity
Adet
Araw ① (1 U), Baet Braw O (1-U)
Ambiguous Pixel Feature Optimization. The deterministic regions serve as reliable context for refining ambiguous pixels, as their features are distinct and free from classification uncertainty. For each ambiguous pixel m E U. we enhance its discriminative characteristics by incorporating the context from deterministic regions Adet and Baet. Specifically, we first compute attention weights w wa(m,, a) by normalizing cosine similarities between the ambiguous pixel feature Jn and all pixels Adet. These weights quantify the relevance of each pixel in Adet to the ambiguous pixel m. Using (m, a) to aggregate contextual features from Adet:
exp(cos(fm: fa)) w (m, a) ExEAse exp(cos(fm:: fe)) cA(m)
Σ ade
wa(m,a)fa
Similarly, we compute (B(m,)) b) and aggregate the context of Baet into cn(m). The optimized feature fm is then derived by fusing the original feature fm with the contextual feature of both parts, this fusion mitigates the ambiguity of hybrid features by "pulling" fn toward the discrimina- tive characteristics of deterministic regions, thereby improving feature separability and boundary precision:
im 7 fm + (1 7)) CA(P) + cp)
2
Visual Prototype Adaptive Refinement. To adapt global part knowledge to the specific image context, we refine visual prototypes by integrating global priors with local image features. For the deterministic region of part A, we cluster its features Fict {fa | E A using K-means to obtain a local prototype QA. which captures the dominant appearance patterns of part A in the current image. The adaptive prototype is then computed by fusing the global prototype PA with its corresponding local prototype qAi
(6) where E [0, 1]. This adaptation allows P.A to preserve the universal characteristics of part A while incorporating image-specific variations (e.g., pose, texture), making it more relevant for matching ambiguous pixels in the current scene. Similarly, obtain Py for part B.
DA - a1- qA
Notably, the similarity between synthetic prototypes and real test images can affect segmentation performance. Our approach achieves a balance between effectiveness and generalization by employing synthetic prototypes with moderate similarity and combining them with the dynamic prototype adaptation mechanism. This design maintains high segmentation accuracy without overfitting to specific datasets, enabling robust performance across diverse domains. Moreover, even when the attention-baue based masks derived from Stable Diffusion contain minor inaccuracies (c.g.. small non-target regions), the prototype adaptation mechanism effectively mitigates such issues. By combining global prototypes with contextual features from deterministic regions, the resulting prototypes dynamically align with the true object parts present in the image, thus enhancing segmentation precision and robustness.
Ambiguous Region Reclassification For each ambiguous pixel m E U, we perform a reclassifica- tion using the optimized pixel feature
f. and the adaptive prototypes A.A, PB:
y(m) = arg max cos(Fm Po) of (A,)))
(7)
7

---

Experiments
5.1 Datasets and Evaluation
evaluate PBAPS on three benchmarks: (1) Pascal-Part-116 [21] which refines the Pascal-Part 5 by merging over-segmented parts and removing redundant descriptors. The validation set includes 17 object classes, 116 part classes, and 850 images.
[21], derived from ADE20K [54] via low-frequency class filtering and synonym merging, containing 44 object classes, 234 part classes, and 1016 validation images. (3) PartImageNet (33], which groups 1588 ImageNet
(2) ADE20K-Part
55 classes into 11 superclasses with uniform part structures, follows prior work (28] to evaluate 40 common object categories on 2957 validation images. In line with previous works [45] [28], we use the mean Intersection over Union (mloU) to assess overall segmentation quality. Additionally, the boundary Intersection over Union (bloU) [56] is introduced to specifically evaluate the accuracy of part boundary.
5.2 Implementation Details
Visual Prototype Generation. For each part class, we generate hundreds of 512x512 synthetic images using Stable Diffusion v1.4 (30). along with their corresponding cross-modal attention maps. After binarizing attention maps (threshold=0.7), 5 prompt points are randomly sampled for ViT-B SAM [3T to obtain the corresponding part masks. The features within the masked regions are then extracted using ViT-B DINOv2 [32]. Finally, K-means clustering (K 4) is applied to these features construct subcategory prototypes, thus capturing intra-class morphological diversity and improving the robustness of the part prototype.
Model Inference. We extract pixel-wise features from input images using VTT-B DINOv2 32
with Shiding-windowe strategy [52], using the "key" values from the final attention layer as feature representations [57]. The hyperparameters of the BAR module fixed follows: ambiguity threshold Aumb = 0.3, feature fusion weight y = 0.8, and prototype adaptation coefficient a = 0.7.
5.3 Comparison with Existing Methods
Table 2: Comparison with existing methods. * denotes re-implementation. Bold and underline indicate the best and second-best results, respectively.
Zero-shot transfer
Pascal-Parte 166
ADE20K Part-
PertImageNet
Method
Backbone
Supervision
mloU
bloU
mloU
bloU
mloU
bloU
7SSeg+
16.
21
ResNet-50
class label
X
24.91
18.18
19.84
12.89
VLPar
45
ResNet-50
class label
X
25.98
16.79
CLIPSeg
43
VIT-B/16
class label
X
24.23
15.98
5.88
4.87
26.98
18.54
CAT-Seg
ViT-B/16
class label
30.53
21.25
7.71
28.56
20.21
PartCATSSg
49
ViT-B/16
class label
X
29.54
20.96
14.68
11.99
30.18
21.29
PanCLIPSSg
28
VIT-B/16
class label x
35.96
26.72
12.64
9.69
30.38
21.04
OVDiil [50]
UNel
Truining-free
41.55
31,57
22.13
15.83
38.22
26.71
RIM* 41
VNT-+ViT-B/16
Truining-free
43.19
32.13
20.91
13.30
39.32
25.06
PBAPS (ours)
VN---V
Training-free
46.35
34.46
24.70
16.41
42.61
29.31
We compare PBAPS with 7SSeg+, VI.Par, CLIPSeg, CAT-Seg, PartCATSeg and PartCLIPSeg, which are fine-tuned target benchmarks. As shown Table [] PBAPS outperforms the state-of- the-art full-supervised method ParCCLISSeg (28) by more 10% in mloU across the three datasets, with bloU improvements of 7.74%, 6.72% and 8. 8.27% on Pascal-Part- 16, ADE20K-Part2255 and PartImageNet, respectively. We also compare PBAPS with training- free OVSS methods OVDiff [50] and RIM [1I. using identical visual prototype set for fairness. All three visual prototype matching methods outperform finetuning bassnd based methods, validating the effectiveness of this paradigm in OVPS. Our method achieves state-of-the-ar mloU and bloU scores on all three datasets, surpassing OVDiff visually demonstrates its superiority. Unlike RIM, which under-segments when target parts lack distinet boundaries, PBAPS captures subtle feature transitions between connected parts, enhancing segmentation accuracy.
and RIM. Figure 5

---

5.4 Analysis and Ablation Study
Table Ablation study for PBAPS on Pascal-Part-116. The baseline model (Ist row) excludes all components, directly matching pixel-wise features extracted by DINOv2 32
with visual prototypes.
HPCGraph
BAR
Pascal-Part-116
Hier. Seg.
X-Hier. Match
w/ Boun. Loc.
Feat. Opt.
Proto. Adapt.
mloU
bloU
40.74
31.09
43.37
32.66
44.08
32.99
43.86
32.89
44.83
33,42
45.58
34.04
45.79
34.16
46.35
34.46
Ablation study for HPCGraph. The HPCGraph enhances part-level semantic modeling via hierar- chical segmentation and cross-hierarchy matching. As shown in Table incorporating hierarchical segmentation (2nd row) increases mloU and bloU by 2.63% and 1.57%, respectively, indicating that modeling top-down part relationships effectively improves segmentation consistency. Further integration of cross-hierarchy matching (3rd row) raises mloU and bloU to 44.08% and 32.99% demonstrating that cross-level prototype comparison strengthens fine-grained part discrimination and improves boundary precision.
As shown in Table using feature optimization alone without ambiguous boundary localization (4th row) leads to slight performance degradation, mloU and bloU drop by 0.22% and 0.1% compared the 3rd row, indicating that indiscriminate context fusion weakens local discriminative features. Introducing boundary region localization (6th row) improves mloU and bloU by 1.5% and 1.05%, validating the importance of distinguishing deterministic from ambiguous regions. Moreover, the 5th and 7th rows demonstrate the synergistic effect between prototype adaptation and boundary localization, which jointly improve boundary precision. When all components are integrated (8th row), mloU and bloU increase by 5.61% % and 3.37% over the baseline, confirming the overall effectiveness of the BAR module in refining part boundaries.
Ablation study for BAR.
Effectiveness of boundary region localization. As shown in Table we assess the boundary localization capability the BAR module by computing boundary recall, which measures the proportion of true boundary pixels correctly covered by the detected ambiguous regions. When the ambiguity threshold set to Aamb = 0.3, the model achieves the optimal balance between the boundary recall (78.93%) and the segmentation precision (46.35% mloU). A strict threshold (Aumb = 0.1) filters out 64.78% of true boundary pixels, leading to insufficient refinement, while lenient threshold (Aamb 0.5) improves recall to 88.97% but introduces non-boundary noise, reducing mloU and bloU.
Table 4: Pascal-Part- 116 results with dif- ferent ambiguity threshold.
Table 5: Impact of part boundary optimization types on Pascal-Parte Part-116.
Aamb
Boundary Recall
mloU
bloU
Optimization Type
mloU
bloU
Inference Time (s)
0.1
35.22
44.48
33.42
Baseline
44.08
32,99
1.47
0.2
58.70
45.54
33.81
Non-struct. boundary
43.19
32.48
2.06
0.3
78.93
46.35
34.46
All part boundary
45.43
33.95
2.19
0.4
82.15
45.71
34.01
Struct. boundary
46.35
34.46
1.81
0.5
88,97
44.29
33.10
Impact of BAR for different boundary. As shown in Table 5] we analyze the impact of the BAR module on different types of part boundaries. The baseline (Ist row) employs the HPCGraph without BAR. For structurally connected part boundaries, BAR achieves the most significant improvements (2.27% mloU, 1.47% bloU). contrast, applying BAR to non-structurally connected part boundaries (2nd row) slightly decreases both mloU and bloU, while increasing inference time by 0.25s compared

---

to the 4th row. This degradation occurs because such boundaries are already well separated, and additional context fusion introduces noise that blurs the boundaries. Moreover, the greater number of boundaries increases the overall inference time. Although applying BAR to all part boundaries im- proves mloU by 1.35%, it increases the inference time by 48%, confirming that selective optimization of structurally connected boundaries offers the best balance between accuracy and efficiency.
Ground Truth
CAT-Seg
ParlCLIPSeg
RIM
PBAPS (Ours)
Figure 5: Qualitative results on Pascal-Pais Parlll-arttlll 16.
Conclusion
This study addresses the inaccurate segmentation of structurally connected part boundaries by in- troducing PBAPS, training-free free OVPS framework. PBAPS integrates progressive segmentation strategy guided by HPCGraph with BAR module enhance boundary precision. Extensive experiments demonstrate that PBAPS consistently outperforms state-of-the methods across mul- tiple benchmarks, confirming effectiveness and generalizability. This study offers a novel and interpretable solution for fine-grained part segmentation in open-world scenarios.
7 Acknowledgement
This research was supported by the National Natural Science Foundation China (No.62476192) and the Natural Science Foundation of Tianjin (No. NN.233JCNJC0020000 The support of these foundations has been instrumental advancing this study, for which we express our sincere gratitude.
10

---

References
[1] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmenta- tion, In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431-3440, 2015.
[2] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-asiisted Intervention-MICCA/ 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[3] Liang-Chich Chen, George Papandreou, lasonas Kokkinos, Kevin Murphy, and Alan I. Yuille. Deeplah: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected erfs. IEEE transactions on pattern analysis and machine intelligence, 40(4):834 848. 2017.
[4] Wei Jiang. Yan Wu, Linting Guan, and Junqiao Zhao. Dfnet: Semantic segmentation on panoramic images with dynamnc loss weights and residual fusion block. In 2019 International Conference on Robotics and Automation (ICRA), pages 5887-5892. IEEE, 2019.
[5] Nabil lbtehaz and Sohel Rahman. Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neural networks, 121:74-87, 2020.
[6] Di Lin, Dingguo Shen, Yuanfeng Ji, Siting Shen, Mingrui Xie, Wei Feng, and Hui Huang. Tagnet: Learning configurable context pathways for semantic segmentation. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(2):2475-2499 2022.
[7] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3213-3223, 2016.
[8] Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coca-stuff: Thing and stuff classes in context. In Proce Proceedings of the IEEE conference computer vision and pattern recognition, pages 1209-1221 1218, 2018.
Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: retrospective. International journal of computer vision, 111:98-136, 13088-13,,2220 2015.
[10] Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung- Yi Lin. Scaling open-vocabulary image segmentation with image-level labels. In European conference computer vision, pages 540-557. Springer, 2022.
[11] Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. Decoupling zero-shot semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11583-11592, 2022.
[12] Xiaoyi Dong, Jianmin Bao, Yinglin Zheng, Ting Zhang, Dongdong Chen, Hao Yang, Ming Zeng, Weiming Zhang, Lu Yuan. Dong Chen, Maskelip: Masked distlliit advances contrastive language-image pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 11995-11005,, 2023.
[13] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763. PmLR, 2021.
[14] Junnan Li. Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-in -magee--ieegeee pre- pre-training unified language understanding and generation. In International conference on machine learning. pages 12888-12900 198800 PMLR, 2022.
[15] Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung, Zhen Li, and Tom Ducrig, Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages 4904 4916. PMLR, 2021.
[16] Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open-vocabeeee vocabulary semantic segmentation with pre-trained vision- language model. In European Conference on Computer Vision, pages 353. Springer, 2022.

---

[17] Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, and Shalini De Mello. Open- vocabulary panoptic segmentation with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2955-2966, 2023.
[18] Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xiang Bai. San: side adapter network for open- vocabulary semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):15546-1555 2023.
[19] Ziqin Zhou, Yinjie Lei, Bowen Zhang, Linggiao Liu, and Yifan Liu. Zegclip: Towards adapting clip for zero-shot semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11175-11185, 2023.
[20] Seokju Cho, Heeseong Shin, Sunghwan Hong, Anurag Arnah, Paul Hongsuck Seo, and Seungryong Kim. Cat-seg: Cost aggregation for open- vocabulcyy semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4113-4122 2024.
[21] Meng Wei, Xiaoyu Yue, Wenwei Zhang, Shu Kong, Xihui Liu, and Jiangmiao Pang. Ov-parts: Towards open-vocabulary part segmentation. Advances in Neural Information Processing Systems, 36:70094-70114 14,
2023.
[22] Zheng Ding, Jieke Wang, and Zhuowen Tu. Open-vocahulary panoptic segmentation maskelip. 2022.
[23] Qihang Yu, Ju Hc, Xueqing Deng, Xiaohui Shen, and Liang-Chich Chen. Convolutions die hard: Open- vocabulary segmentation with single frozen convolutional clip. Advances in Neural Information Processing Systems, 36:32215-3222 -22224,,-.2244 2023.
[24] Ta Pan, Qing Liu, Wei-Lun Chao, and Brian Price. Towards open-world segmentation of parts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15392- 15401, 2023.
[25] Chufeng Tang, Lingxi Xie, Xiaopeng Zhang, Xiaolin Hu, and Qi Tian. Visual recognition by request. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15265- 15274, 2023.
[26] Bin Xie, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, and Yanwei Pang. Sed: A simple encoder-decoder for open-vocabulary semantic segmentation. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 3426-3436, 2024.
[27] Zhenqi Dai, Ting Liu, Xingxing Zhang, Yunchao Wei, and Yanning Zhang. One-shot in-context part segmentation. In ACM Multimedia, volume 2024, 2024.
[28] Jiho Choi, Seonho Lee, Scungho Lee, Minhyun Lee, and Hyunjung Shim. Understanding multi granularity for open-vocabulary part segmentation. arXiv preprint arXiv:2406. 1384, 2024.
[29] Biyang Liu, Huimin Yu, and Guodong Qi. Graftnet: Towards domain generalized stereo matching with broad-spectrum and task-oriented feature. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13012-13021, 2022.
[30] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[31] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Speneer Whitchead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2015-4020 2023.
[32] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXth:2304.007195 2023.
[33] Ju He, Shuo Yang. Shaokang Yang, Adam Kortlewski, Xiaoding Yuan, Jie-Neng Chen, Shuai Liu, Cheng Yang, Qihang Yu, and Alan Yuille. Partimagenet: large, high-quality dataset of parts. In European Conference on Computer Vision, pages 145. Springer, 2022.
[34] Di Lin, Dingguo Shen, Siting Shen, Yuanfeng Ji, Dani Lischinski, Daniel Cohen-Or Or. and Hui Huang. Zigzagnet: Fusing tow- and bottom-up context for object segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7490-7499, 2019.
12

---

[35] Di Lin and Hui Huang. Zig-zag network for semantic segmentation of rgb-d images. IEEE Transactions Pattern Analysis and Machine Intelligence, 42(10))2642-2655, 2019.
[36] Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmentation. Proceedings of the IEEE/CVF international conference on computer vision, pages 7262-7272 2021.
[37] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers, Advances in neural information processing systems, 14777771100090, 2021.
[38] Naisong Luo, Yuwen Pan, Rui Sun, Tianzhu Zhang, Zhiwei Xiong, and Feng Wu. Camoußlaged instance segmentation via explicit de-camouflaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 77918-17227 2023.
[39] Jilan Xu, Junlin Hou, Yuejic Zhang, Rui Feng, Yi Wang, Yu Qiao, and Weidi Xie. Learning open- vocabulary semantic segmentation models from natural language supervision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2935-2944, 2023.
[40] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention, Advances in neural information processing systems, 33:11525- 11538, 2020.
[41] Yuan Wang, Rui Sun, Naisong Luo, Yuwen Pan, and Tianzhu Zhang. Image-to-image matching via foundation models: A new perspective for open- vocabulcyy semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 3952-3963, 2024.
[42] Xiangheng Shan, Dongyue Wu, Guilin Zhu, Yuanjie Shao, Nong Sang, and Changxin Gao. Open- vocabulary semantic segmentation with image embedding balancing. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 28412 28412-28421, 2024.
[43] Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7086-7096, 2022.
[44] Daan De Geus and Gijs Dubhelman. Task-aligned part-aware panoptic segmentation through joint object- part representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3174-3183, 2024.
[45] Peize Sun, Shoufa Chen, Chenchen Zhu, Fanyi Xiao, Ping Luo, Saining Xie, and Zhicheng Yan. Going denser with open-vocabulary part segmentation. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 15453-15465, 2023.
[46] Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022.
[47] Xudong Wang. Shufan Li, Konstantinos Kallidromitis, Yusuke Kato, Kazuki Kozuka, and Trevor Darrell. Hicrarchical vocabuaayy universal image segmentation. Advances in Neural Information Processing Systems, 36:21429-21455 2023.
[48] Xin-Jian Wu, Ruisong Zhang, Jie Qin, Shijie Ma, and Cheng-Lin Liu. Wps-sam: Towards weakly- supervised part segmentation with foundation models. In European Conference on Computer Vision, pages 31 314-333. Springer, 2024.
[49] Jiho Choi, Sconho Lec, Minhyun Loc, Seungho Lec, and Hyunjung Shim. Fine-grained image-text corre- spondence with cost aggregation for open-vocabulary part segmentation. arXiv preprint arXiv: 09668, 2025.
[50] Laurynas Karazija, Iro Laina, Andrea Vedaldi, and Christian Rupprecht. Diffusion models for zero-shot open-vocabulary segmentation. arXiv e-prints, pages arXiv-2306, 2023.
[51] Weijia Wu, Yuzhong Zhao, Mike Zheng Shou, Hong Zhou, and Chunhua Shen. Diffumask: Synthesizing images with pixel-level annotations for semantic segmentation using diffusion models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1206-1217, 2023.
[52] Lukas Hoyer, Dengxin Dai, and Luc Van Gool. Hrda: Context-aware nigh-resolution domain- adaptive semantic segmentation. In European conference on computer vision, pages 372-391. Springer, 2022.
13

---

[53] Xianjie Chen, Roozbeh Mottaghi. Xiaobai Liu, Sanja Fidler, Raquel Urtasun. and Alan Yuille. Detect what you Detecting and representing objects using holistic models and body parts. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1971-1978. 2014.
[54] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torrala.. Scene parsing through ade20k dataset. Proceedings of the IEEE conference on computer vision and pattern recognition. pages 633 03-641,222 2017.
[55] Deng, Wei Dong, Richard Socher, Li-Jia Kai Li, Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248 255. leee, 2009.
[56] Bowen Cheng, Ross Girshick, Piotr Dollár, Alexander C Berg, and Alexander Kirillov. Boundary iou: Improving object-centric image segmentation evaluation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15334-13 15342, 2021.
[57] Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features dense visual descriptors. arXiv preprint arXiv:21 2(3):4, 2021.
[58] Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
[59] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference computer vision and patlern recognition, pages 16000-16009, 2022.
[60] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in supe--upe vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-9660. 2021.
[61] Bowen Cheng, Alex Schwing. and Alexander Kirillov. Per-pixel classification is not all you need for semantic segmentation. Advances in neural information processing systems, 34:17864 17866--78755, 2021.
[62] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
[63] Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár. Panoptic segmentation. In Proceedings of the JEEF/CVF conference on computer vision and pattern recognition, pages 9404-9413, 2019.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The Introduction (Section 1 explicitly states our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our method in Appendix A.I
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The mathematical formulations in Sections 3 and f are methodological com- ponents rather than theoretical propositions. The work focuses on algorithmic design and empirical validation.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The method PBAPS is detailed in Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer:[ Yes]
Justification:
We open-source the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
mripss.cc/ubllic/guidee CoodeSumeeieeeeee ssi cessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 5.2 comprehhenss specifies the datasets, the foundation models, the hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Statistical significance is not a focus of our experiments, as the emphasis lies comparing method effectiveness rather than statistical hypothesis testing.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: These can be found the Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research does not involve sensitive data and the datasets used are standard benchmarks in the field (5ection(5).
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss societal impacts in the Appendix A.2
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper focuses on segmentation framework using publicly available part segmentation datasets and does not involve the release of high-risk models
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The paper appropriately cites all assets used (Section ID, such datasets and models.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper introduces novel framework (PBAPS) and methodology but does not release new assets.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The study does not involve human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The PBAPS relies on computer vision techniques without employing LLMs as components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatp:s///oooriissccc///ooorrrree for what should or should not be described.
21

---

A Discussion
A.1 Limitations and Future Work
Although our method shows excellent performance the OVPS task, it still has several limitations. First, PBAPS focuses on part-level semantic segmentation and cannot distinguish different instances of the same category. For example, when multiple cats appear in an image, PBAPS segments all "cat head" without assigning each head its respective individual. Second, method relies the HPCGraph, which defines hierarchical and adjacency relationships between objects and parts via manual design external knowledge bases. However, when processing objects with complex, irregular structures (e.g., modular furniture), the statically predefined graph may fail to capture their intrinsic part topology and semantic associations.
To address these limitations, future work can explore two directions. On the one hand, introducing instance-aware mechanisms could enable simultancous semantic segmentation and instance separation. On the other hand, developing a dynamic hierarchical reasoning framework that can adaptively adjust part hierarchical structures based specific objects.
A.2 Social Impact
Our method offers significant societal value in multiple domains. healthcare, its precise segmenta- tion of structurally connected parts can improve the efficiency of pathological diagnosis and provide low-cost, annotation- diagnostic aid for remote regions, thereby promoting equitable access to medical resources. In industrial manufacturing, its hierarchical part reasoning capability supports automated quality inspection and disassembly of mechanical equipment.
Despite these advantages, PBAPS also has potential risks. In cafety-critical applications such as medical imaging and autonomous driving, reliance on predefined structural priors may lead to segmentation errors in complex scenarios. For example, misidentifying the boundaries of rare pathological organs, causing diagnostic inaccuracies.
B
Additional Experiments
B.1
Further Details
Stable Difffusion. A text-conditioned latent diffuusion model [30] that generates images via iterative denoising and has three main components: (I) a pre-trained VAE [58] for image encoding/decoding, (2) text encoder converting the prompt into embedding vector, and (3) time-conditional U- Net & that denoises an initial Gaussian noise to produce the image. During image synthesis, the corresponding cross-attention maps A [5]] extracted to locate the target part. Specifically, the prompt is encoded in text embedding r()) E R xd. where N denotes the length of the token sequence. At each U-Net o timestep visual features (zt) R from noisy image 2; flattened and projected to query Q, while text embedding T Tpp) produces the key K and the value V via learnable layers l and ty:
Q (((((2)), k= V=
(8)
The cross-attention weights A Softmax
QQK" Va
H To obtain robust class attention
maps, attention maps are aggregated over multiple layers and time steps:
A
A
B
max(AA) (A
BEBIET
where j is the index of the text token, B and denote the layers and timestep sets, respectively.
Segment Anything Model. SAM [1] is training-free image segmentation framework that enables prompt-based rapid segmentation of arbitrary objects using point, box, or mask input. Its architecture consists of: (1) an image encoder that extracts the global visual feature F; from the input image, (2) prompt encoder that encodes prompts into unified features Fps and (3) a mask decoder that generates candidate masks by integrating F; with Fp.
22

---

DINOv2. DINOv2 [32 is a discriminative self-supprvvsed VIT that distills general visual features from large-scale unlabeled data. Through joint global-imag image and local-patch level learning, its robust features excel in tasks including image classification, semantic segmentation, and patch matching.
For each part class c, we mainly adopt the generie prompt template "a photo of c" to guide Stable Diffusion [30] in synthesizing images of the part c. To further enrich visual diversity, we incorporate synonyms and subclasses of the original class names [51]. We also standardize class names (e.g.. "tymonitor" + "tv monitor") and refine them to resolve ambiguities (e.g., "cat hand" - "cat paw"),
During image generation, set an independent random seed for each class The image generation time scales linearly with the number of images: 124s for 32 images, 227s for 64 images, and 443s for 128 images. Each part prototype, extracted via DINOv2 32,, occupies 0.45 MB. During inference, PBAPS employs a sliding window of size 224x224 with a stride of 64 for feature extraction using DINOv2 (identical to all baseline methods), achieving an average processing time of 1.81s per image. All results are measured with a single NVIDIA A6000 GPU.
reproducibility.
B.2 Ablation on Feature Extractors
Table 6: Comprison of different feature ex- tractors Pascal-Part-I 16 [21).
Table 7: Comprison of mask generators on Pascal-Part- 116 (2]).
Feature Extractor
mloU
bloU
Mask Generator
mloU
bloU
MAE
37.94
27.47
DINO
60
41.20
30.79
MaskFormer 61
41.28
30.97
CLIP
13
42.39
30.41
SAM [3I
43.82
32.43
Stable Diffusion 30
44.12
33.31
None
46.35
34.46
DINOv2 332
46.35
34.46
Our method can integrate with any pretrained visual feature extractor construct visual prototypes and extract image features. As shown in Table to demonstrate the superiority of using DINOv2 32 for image feature extraction our framework, we compare it with several self-supervised ViT feature extractors. DINOv2 significantly outperforms other methods, benefiting from its pretraining based on image-level and patch-level discriminative learning, which empowers with fine-grained feature representation capabilities. CLIP exhibits limitations in fine ene-grained feature alignment, likely due to its contrastive learning that focuses on global feature alignment (Figure (6) MAE yields the weakest performance due to its lack of explicit semantic discriminative learning. Stable Diffusion performs secondarily by leveraging structural information implicitly learned through generative tasks.
Ground Truth
w/ CLIP
w/ DINOv2
Figure 6: Qualitative ablation on feature extractors in Pascal-Parts 116 21
23

---

B.3 Ablation on Mask Generators
Our PBAPS framework supports two segmentation paradigms: (1) direct per-pixel feature classifica- tion and (2) region-level classification following mask generation by pre-trained segmenter. To vali- date the superiority of our pixel-wise classification strategy, we compare against MaskFormer-baseee SAM-based variants. Note that when using MaskFormer [6]] or SAM [31 for mask generation, the segmentation process operates only under the structural constraints HPCGraph without boundary refinement via the BAR module, as these generators only produce binary masks, lacking class-specific matching cost maps.
shown Table introducing MaskFormer or SAM significantly degrades performance. This is because MaskFormer, trained on object-level COCO [62] g 63], tends to propose coarse object regions and fails to capture subtle part-level distinctions. Although SAM exhibits zero-shot generalization, its segmentation relies prominent visual changes (e.g.. texture, color). The part-level segmentation often involves only local, subtle variations, leading to frequent segee-segmnntation by SAM (Figure
Ground Truth
w/ SAM
w SAM
Figure 7: Qualitative ablation mask generators in Pascal-Part-I1 116 21
B.4
Sensitivity Analysis of Hyperparameters
mloU
mloU
mioU
46.5
A6.35
46.5
46.5
11:44
145.26
M5.99
35535
45
01.33
45.5
45.5
121
63.5
43.01
44.5
445
(a) The Feature fission weight 7
(b) The prototype aduptation coefficient &
(c) The number of subprootypps
Figure 8: The sensitivity analysis of y, a and on Pascal-Part-I1 16
21
To evaluate the impact of key hyperparameters in PBAPS, we conduct a sensitivity analysis on Pascal- Part-116 for three core parameters: feature fusion weight 7 prototype adaptation coefficient a, and the number of subprototypes K in visual prototype generation. Figure 8 shows that when 0.8, PBAPS achieves the highest mloU of 46.35%. moderate ? balances the retention of critical original feature information and the integration of discriminative context from deterministic regions. When 7 is low (0.5), excessive reliance on contextual features suppresses original discriminative information, degrading performance. When is high (1.0), the neglect of contextual guidance leaves

---

feature ambiguity unresolved. Similarly, moderate a (0.7) enables adaptive prototypes to retain both universal part knowledge and image-specific variations (e.g., pose, texture). When a is small, over-adaptation to local features may introduce noise or cause overfitting to image details. When approaches 1.0, the prototypes lack adaptability to variations in the current image. Regarding the number of subprototypes, K 4 allows the model to fully capture intra-class diversity while avoiding noise interference. When K O, the global prototype overly generic; when K > excessive subprototypes may introduce noisy clusters, leading to performance degradation.
B.5 Additional Qualitative Results
Ground Truth
PartCLIPSeg
PartCATSeg
RIM
PBAPS (Ours)
Figure Qualitative results on PartImageNet
+3
25

---

Ground Truth
ParlCLIPSeg
ParICATSeg
RIM
PBAPS (Ours)
Figure 10: Qualitative results on Pascal-Part- 16
21
26


Reviews:
[{"review_id": "rVVB4Sxzah", "reviewer": "Reviewer_Sy1A", "summary": "This paper proposes a novel training-free Open-Vocabulary Part Segmentation (OVPS) framework called PBAPS, addressing the inaccuracies in segmenting structurally connected part boundaries caused by the conflict between continuous image features and discrete classification mechanisms. PBAPS employs a progressive segmentation strategy guided by a Hierarchical Part Connected Graph (HPCGraph) and introduces a Boundary-Aware Refinement (BAR) module. The BAR module locates ambiguous boundary regions, optimizes pixel features, and adaptively refines visual prototypes for precise reclassification. Experimental results demonstrate that PBAPS significantly outperforms state-of-the-art methods on multiple benchmark datasets, showcasing its superior performance, especially in handling fine-grained part boundaries.", "questions": "see weakness above", "limitations": "see weakness above", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. This paper proposes a framework that successfully leverages pre-trained foundation models for a complex downstream task without any fine-tuning, demonstrating excellent zero-shot generalization and highlighting a promising research direction.\n2. The method achieves impressive SOTA results on three challenging benchmarks.\nWeakness:\n1. The paper only showcases successful examples. A discussion of failure cases would provide a more balanced perspective on the method's limitations and offer crucial insights for future work.\n2. The paper lacks a clear justification for how key hyperparameters were chosen without access to the test set. Furthermore, their robustness and generalizability across different datasets are not discussed.\n3. The method's reliance on a manually constructed Hierarchical Part Connected Graph (HPCGraph) is a major bottleneck, raising significant concerns about its scalability and applicability to novel object categories not defined in advance.", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedbacks as follows.\n\n---\n>**W1: The paper only showcases successful examples. A discussion of failure cases would provide a more balanced perspective on the method's limitations and offer crucial insights for future work.**\n\n**A1:** We fully agree with your comment. In the main paper, we primarily showcased representative successful cases to clearly demonstrate the advantages of our method. Although appendix A.1 includes a discussion on the limitations of our approach—such as difficulties in distinguishing multiple instances and handling complex or irregular object structures—it lacks visual examples that intuitively illustrate failure cases. Due to NeurIPS rebuttal constraints, we are unable to include images, PDFs, or external links at this stage. We briefly describe the failure cases here: when there are multiple animals of the same category in an image (e.g., three cats) with intertwined limbs, PBAPS fails to distinguish \"identical parts belonging to different individuals\" (e.g., the tail of a black cat vs. the tail of a white cat), resulting in cross-instance confusion. For modular furniture or mechanical parts, whose part hierarchies are ambiguous or topologies are variable, the static HPCGraph struggles to cover such scenarios, leading to part missed detections or over-merging (e.g., misclassifying folded table legs as part of the tabletop). We will include visualizations of typical failure cases in the revised version of the paper and provide an in-depth analysis of the underlying causes.\n\n>**W2: The paper lacks a clear justification for how key hyperparameters were chosen without access to the test set. Furthermore, their robustness and generalizability across different datasets are not discussed.**\n\n**A2:** Thank you for your valuable comment. Regarding hyperparameter selection, we strictly adhered to the **\"no access to test set\"** principle. Specifically, we evaluated model performance (mIoU) under different hyperparameter configurations on the training sets of the three target datasets. As shown in the table below, we selected a single set of hyperparameters ($\\gamma$ = 0.8, $\\alpha$ = 0.7, $K$ = 4) that consistently performed well across all three datasets. **This selection was made without any feedback from the test sets, thus avoiding dataset-specific tuning.** As for robustness, Table 2 demonstrates that PBAPS (ours), using the same hyperparameter configuration, achieves state-of-the-art performance on Pascal-Part-116, ADE20K-Part-234, and PartImageNet—datasets with significant differences in visual style and structural granularity—validating the robustness and generalizability of our hyperparameter choices across diverse domains.\n| &emsp;$\\gamma$&emsp; | **Pascal-Part-116** | **ADE20K-Part-234** | **PartImageNet**|\n|:-:|:-:|:-:|:-:|\n| 0.5 |43.82|23.76|40.21|\n| 0.6 |44.73|24.31|40.84|\n| 0.7 |45.51|**24.79**|42.39|\n| 0.8 |**46.21**|24.62|**42.75**|\n| 0.9 |45.40|24.03|41.42|\n| 1.0 |44.72|23.89|40.94|\n\n|&emsp; $\\alpha$&emsp; | **Pascal-Part-116** | **ADE20K-Part-234** | **PartImageNet**|\n|:-:|:-:|:-:|:-:|\n| 0.5 |45.67|24.08|41.62|\n| 0.6 |46.02|24.39|42.16|\n| 0.7 |**46.21**|**24.62**|**42.75**|\n| 0.8 |46.12|24.46|42.08|\n| 0.9 |45.83|24.25|41.86|\n| 1.0 |45.64|24.06|41.51|\n\n|&emsp; $K$&emsp; | **Pascal-Part-116** | **ADE20K-Part-234** | **PartImageNet**|\n|:-:|:-:|:-:|:-:|\n| 1 |45.21|24.15|41.43|\n| 2 |45.91|24.43|41.87|\n| 4 |**46.21**|**24.62**|**42.75**|\n| 6 |45.88|24.48|41.91|\n| 8 |45.62|24.22|41.54|\n\n>**W3: The method's reliance on a manually constructed Hierarchical Part Connected Graph (HPCGraph) is a major bottleneck, raising significant concerns about its scalability and applicability to novel object categories not defined in advance.**\n\n**A3:** Regarding the scalability of HPCGraph, we provide the following supplementary explanations:\n- HPCGraph is constructed based on commonsense structural knowledge (e.g., compositional relationships, anatomical structures) and does not rely on any manual annotations. As such, it incurs low construction cost and exhibits strong transferability and scalability.\n- **In large-scale object scenarios, both the topological structure and visual prototypes can be shared and reused.** For novel object categories with similar structures to existing ones (e.g., “zebra” as a novel category relative to the existing category “horse”), we can directly reuse existing topologies (e.g., “zebra head → zebra eye”, “zebra head ↔ zebra neck” correspond to the structure of \"horse\"). For PartImageNet (L198–200), we divided 158 object categories into 11 superclasses, where each superclass shares a unified topological graph. This strategy reduces the number of topology by **93%** (from 158 to 11) while maintaining high performance—PBAPS outperforms RIM by **8.4%** on PartImageNet (Table 2). When the number of object categories expands to tens of thousands, a large number of them are highly similar in form and structure (such as horses, donkeys, mules, zebras, etc.). We can use Large Language Models (LLMs) to analyze inter-category structural similarities and achieve cluster-level topology reuse (i.e., categories with similar structures share a set of topologies). Furthermore, part visual prototypes can also be reused (e.g., the prototype for “horse leg” can be applied to “donkey leg”).\n- In contrast to existing methods such as PartCLIPSeg [27] and PartCATSeg [53], which require extensive part-level annotations and retraining when introducing new object categories, our method is training-free and supervision-independent. It supports open-world scalability by simply extending the HPCGraph and adding a small number of visual prototypes, making it more suitable for large-scale and dynamic settings.\n- As noted in Appendix A.1, when dealing with objects with irregular structures (e.g., modular furniture), the static graph structure has insufficient adaptability. To address this, a dynamic graph generation mechanism can be considered. The core logic is to **automatically add adjacency edges for highly relevant parts based on the spatial adjacency and visual feature similarity between part response regions**. The following table compares performance using static and dynamic graphs. While static graphs yield better performance in well-structured domains, the dynamic version provides competitive results and significantly enhances flexibility for deployment in unconstrained real-world scenarios. \n| HPCGraph | **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n|  | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU |\n| Dynamic | 44.63&ensp;&emsp;33.12 |23.87&ensp;&emsp;15.63| 41.66&ensp;&emsp;28.72 |\n| Static | **46.35**&ensp;&emsp;**34.46** |**24.70**&ensp;&emsp;**16.41** | **42.61**&ensp;&emsp;**29.31** |\n\n---\nWe once again sincerely thank the reviewer for the insightful and constructive feedback. If there are any further concerns or points requiring clarification, we would be glad to provide additional details.\n\n[27] PartCLIPSeg: Understanding Multi-Granularityfor Open-Vocabulary Part Segmentation (NeurIPS 2024)\n\n[53] PartCATSeg: Fine-Grained Image-Text Correspondence withCost Aggregation for Open-Vocabulary Part Segmentation (CVPR 2025)"}, {"review_id": "zxZOW7pCPf", "reviewer": "Reviewer_BZzg", "summary": "This paper presents PBAPS, a training-free framework for open-vocabulary part segmentation (OVPS) that addresses the inherent challenge of segmenting structurally connected part boundaries-regions where anatomically adjacent parts exhibit smooth, continuous visual transitions. The authors identify that the core difficulty arises from the mismatch between continuous image features and discrete classification mechanisms, leading to ambiguous boundaries. To resolve this issue, PBAPS introduces a progressive boundary-aware strategy built upon Cross-Model Prototype Induction, Hierarchical Part-Connected Graph and Boundary-Aware Refinement. Extensive experiments on Pascal-Part-116, ADE20K-Part-234, and PartImageNet demonstrate state-of-the-art performance.", "questions": "1. Contamination from blended-part features in the “deterministic” pixels of BAR  \nBAR refines ambiguous boundaries by aggregating context from “deterministic” regions, yet gradual transitions may still introduce mixed-part features into these pixels. Could the authors provide  qualitative visualizations  that demonstrate this blending does **not** materially degrade final segmentation accuracy?\n2. Dynamic extensibility of a static HPCGraph  \nHand-crafted hierarchies (e.g., cat → head → eye) struggle with irregular or novel topologies (e.g., modular furniture). Have the authors considered a lightweight mechanism that allows the hierarchy to adapt at inference time while preserving the training-free constraint?\n3. Additional supervised open-vocabulary part-detection baseline  \nRecent supervised open-vocabulary detectors can be minimally adapted for part-level segmentation. Please report the  mIoU/bIoU after fine-tuning at least one such model on the same part vocabulary; if PBAPS still significantly outperforms, this will strengthen its training-free advantage.\n4. Ablation decoupling hierarchical reasoning from boundary refinement  \nThe current ablation (Table 3) does not disentangle the contributions of the hierarchy and BAR. Please add a  flat baseline that applies BAR directly to DINOv2 pixel features  without HPCGraph.", "limitations": "Please see weaknesses and questions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\nQuality: The paper presents a technically sound framework (PBAPS) for open-vocabulary part segmentation (OVPS). The methodology is well-structured, combining hierarchical reasoning (HPCGraph) with a novel Boundary-Aware Refinement (BAR) module. The experiments are comprehensive, covering three benchmarks (Pascal-Part-116, ADE20K-Part-234, PartImageNet) and demonstrating consistent outperformance over prior methods (e.g., +10% mIoU over PartCLIPSeg). The ablation studies validate the necessity of each component \nClarity: The paper is well-organized, with clear explanations of the BAR module’s four-step process (ambiguous region localization, feature optimization, prototype adaptation, reclassification). Figures effectively illustrate the boundary refinement pipeline. The distinction between structurally vs. non-structurally connected boundaries is rigorously analyzed via feature gradients, providing theoretical grounding.\n\nSignificance: The work addresses a critical gap in OVPS—structurally connected part boundaries—which prior methods struggle with due to smooth feature transitions.\nOriginality: While cross-model prototype induction resembles RIM’s pipeline, the novelty lies in adapting it for part-level segmentation and introducing BAR to resolve boundary ambiguities via cost-divergence maps and context-driven refinement. The HPCGraph’s hierarchical design (object → part → subpart) is tailored to OVPS, distinguishing it from prior OVSS methods.\n\n\nWeaknesses\nOver-smoothing in BAR \nBAR averages context from “certain” pixels to refine boundaries, but these pixels may themselves be ambiguous on gradual transitions, weakening discriminative cues.\n HPCGraph static dependency \nThe hand-crafted hierarchy (e.g., cat → head → eye) cannot generalize to objects with irregular or novel part topologies (e.g., modular furniture).\n\nMissing comparison to supervised open-vocabulary part detectors\nThe experimental baseline set omits recent strong supervised open-vocabulary detectors that can be adapted for part localization. Without this comparison, the paper cannot fully substantiate the added value of its training-free approach over competitive fine-tuned alternatives.\n\nInadequate ablation separating hierarchical reasoning and boundary refinement\nThe ablation study (Table 3) entangles the contributions of (i) the hierarchical HPCGraph and (ii) the BAR boundary-refinement module. A crucial baseline—BAR applied directly to flat DINOv2 features without the hierarchy—is absent, preventing an isolated assessment of each component’s impact.", "was_revised": true, "final_justification": "The rebuttal has well addressed my concerns. I am glad to arise my score.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedbacks as follows.\n\n---\n>**W1＆Q1: Contamination from blended-part features in the “deterministic” pixels of BAR.**\n\n**A1:** Regarding the question of whether BAR weakens the original discriminative features, we provide the following supplementary explanation:\n- In terms of design, BAR does not directly perform average context fusion on ambiguous regions. The attention weights in Equation 4 **only aggregate features from deterministic regions** that are most relevant to the current ambiguous pixel, avoiding the introduction of irrelevant or ambiguous contexts. Subsequently, a weighted fusion strategy of \"original features ($\\gamma$=0.8) + context (1-$\\gamma$=0.2)\" is adopted as specified in Equation 5, ensuring that **the discriminative features of the pixels remain dominant**. This design not only leverages context to resolve ambiguity but also prevents the loss of discriminative features.\n- In Appendix B.4, we evaluated the model performance under different $\\gamma$ values. As shown in Figure 8(a), using only the original ambiguous region features (without context fusion, $\\gamma$=1.0) results in 44.83% mIoU. Using only context features (without original features, $\\gamma$=0),  the mIoU drops to 43.31% (not included in the figure), indicating the loss of discriminative features and excessive boundary smoothing. With balanced fusion ($\\gamma$=0.8), the mIoU increases to 46.35%, which **retains discriminative features while eliminating ambiguity through context.**\n- Additionally, the ablation experiment in Table 3 effectively verifies the contribution of the BAR module. Figure 4 provides a visual comparison of the segmentation at the \"cat neck-torso boundary\" before and after refinement. Due to NeurIPS rebuttal constraints, we are unable to include images, PDFs, or external links at this stage. We will supplement additional qualitative visualization results in subsequent revisions of the paper to fully demonstrate the effectiveness of the BAR module.\n\n>**W2＆Q2: Extensibility of the HPCGraph.**\n\n**A2:** Regarding the Extensibility of HPCGraph, we provide the following supplementary explanations:\n- HPCGraph is constructed based on commonsense structural knowledge and does not rely on any manual annotations. As such, it incurs low construction cost and exhibits strong transferability and scalability. In large-scale object scenarios, **both the topological structure and visual prototypes can be shared and reused.** For novel object categories with similar structures to existing ones (e.g., “zebra” as a novel category relative to the existing category “horse”), we can directly reuse existing topologies (e.g., “zebra head ↔ zebra neck” correspond to the structure of \"horse\"). For PartImageNet (L198–200), we divided 158 object categories into 11 superclasses, where each superclass shares a unified topological graph. This strategy reduces the number of topology by **93%** (from 158 to 11) while maintaining high performance—PBAPS outperforms RIM by **8.4%** on PartImageNet (Table 2). When the number of object categories expands to tens of thousands, a large number of them are highly similar in form and structure (such as horses, donkeys, mules, zebras, etc.). We can use LLMs to analyze inter-category structural similarities and achieve cluster-level topology reuse (i.e., categories with similar structures share a set of topologies). Furthermore, part visual prototypes can also be reused (e.g., the prototype for “horse leg” can be applied to “donkey leg”).\n- In contrast to existing methods such as PartCLIPSeg [27] and PartCATSeg [53], which require extensive part-level annotations and retraining when introducing new object categories, our method is training-free and supervision-independent. It supports open-world scalability by simply extending the HPCGraph and adding a small number of visual prototypes.\n- As noted in Appendix A.1, when dealing with objects with irregular structures (e.g., modular furniture), the static graph structure has insufficient adaptability. To address this, a dynamic graph generation mechanism can be considered. The core logic is to **automatically add adjacency edges for highly relevant parts based on the spatial adjacency and visual feature similarity between part response regions**. The following table compares performance using static and dynamic graphs. While static graphs yield better performance in well-structured domains, the dynamic version provides competitive results and significantly enhances flexibility for deployment in unconstrained real-world scenarios. \n| HPCGraph | **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n|  | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU |\n| Dynamic | 44.63&ensp;&emsp;33.12 |23.87&ensp;&emsp;15.63| 41.66&ensp;&emsp;28.72 |\n| Static | **46.35**&ensp;&emsp;**34.46** |**24.70**&ensp;&emsp;**16.41** | **42.61**&ensp;&emsp;**29.31** |\n\n\n>**W3＆Q3: Additional supervised open-vocabulary part-detection baseline.**\n\n**A3:** Regarding the comparison with supervised open-vocabulary segmentation methods, as shown in Table 2, our method (PBAPS) has already compared with ZSSeg (ECCV 2022) [15] and CAT-Seg (CVPR 2024) [19], and outperformed these methods on all three datasets, effectively verifying the superiority of our training-free approach. Following your suggestion, we have further compared with two latest supervised open-vocabulary segmentation methods, SED (CVPR 2024) [60] and DPSeg (CVPR 2025) [61]. Both methods introduce a multi-scale feature fusion strategy based on CAT-Seg, achieving significant improvements in the segmentation of small objects and boundaries. We adjusted their decoders to support part-level prediction and fine-tuned them on the target datasets. As shown in the table below, **PBAPS still maintains superior performance**, further validating its advantages in OVPS.\n| Method | **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n|  | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU | \n| SED | 32.61&ensp;&emsp;21.76 |12.74&emsp;&emsp;8.58| 30.20&ensp;&emsp;19.45 |\n| DPSeg | 34.64&ensp;&emsp;22.49 |13.21&emsp;&emsp;8.87 | 32.57&ensp;&emsp;21.97 |\n| PBAPS (ours) | **46.35**&ensp;&emsp;**34.46** |**24.70**&ensp;&emsp;**16.41** | **42.61**&ensp;&emsp;**29.31** |\n\n>**W4＆Q4: Ablation decoupling hierarchical reasoning from boundary refinement.**\n\n**A4:** As shown in the table below, we directly applied BAR to optimize all structurally connected part boundaries based on the baseline (the 1st row, which excludes all components and directly matches pixel-wise features with visual prototypes). Furthermore, to further demonstrate the robustness and generalizability of our method, we supplement the same ablation experiment on ADE20K-Part-234 and PartImageNet. The results indicate that without introducing hierarchical structural constraints, BAR can still effectively model local contextual consistency and improve the segmentation accuracy of boundaries between structurally connected parts. Moreover, combining BAR with hierarchical reasoning yields further improvements, demonstrating their complementarity.\n| w/ HPCGraph |w/ BAR| **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-:|:-:|:-:|:-:|:-:|\n| | | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU | \n|  |  |40.74&ensp;&emsp;31.09 |20.59&ensp;&emsp;13.72 | 37.46&ensp;&emsp;25.78 |\n|  | √ |42.95&ensp;&emsp;32.34 |22.14&ensp;&emsp;14.53 | 39.54&ensp;&emsp;27.12 |\n| √ |  |44.08&ensp;&emsp;32.99 |23.62&ensp;&emsp;15.43 | 41.27&ensp;&emsp;28.58 |\n| √ | √ |46.35&ensp;&emsp;34.46 |24.70&ensp;&emsp;16.41 | 42.61&ensp;&emsp;29.31 |\n\n---\nWe once again sincerely thank the reviewer for the insightful and constructive feedback. If there are any further concerns or points requiring clarification, we would be glad to provide additional details.\n\n[15] ZSSeg: A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model (ECCV 2022)\n\n[19] CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation (CVPR 2024)\n\n[60] SED: ASimple Encoder-Decoder for Open-Vocabulary Semantic Segmentation (CVPR2024)\n\n[61] DPSeg: Dual-Prompt Cost Volume Learning for Open-Vocabulary Semantic Segmentation (CVPR2025)"}, {"review_id": "4WwZcdM5Hi", "reviewer": "Reviewer_LruU", "summary": "This paper addresses the problem of open-vocabulary part segmentation (OVPS), focusing on the challenge of segmenting structurally connected parts that lack clear visual boundaries. The authors argue that standard discrete classification struggles with these smooth transitions.\n\nTo mitigate this, they propose a Boundary-Aware Refinement (BAR) module that identifies ambiguous boundaries and refines them using nearby reliable regions and adaptive prototypes. They further design a training-free framework called PBAPS, which performs progressive refinement guided by a hierarchical graph of part relationships.\n\nThe method achieves strong performance on several benchmarks. While the use of multiple external models may introduce complexity, the approach is well-motivated and effectively tackles a core limitation in current OVPS systems.", "questions": "See weakness.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The method is entirely training-free, avoiding the need for large-scale annotated data or fine-tuning.\n\n- The Boundary-Aware Refinement (BAR) module is a novel design that specifically targets the challenge of ambiguous part boundaries, a key limitation in existing OVPS methods.\n\n- The use of a hierarchical part graph to guide progressive refinement introduces a structured and interpretable approach to part segmentation.\n\n- The framework is modular and leverages existing vision-language models effectively without additional training overhead.\n\nWeaknesses:\n\n- The method relies on generating hundreds of prototype images per part class, which may introduce a risk of overfitting or inadvertently aligning too closely with the test set, especially on relatively small or simple datasets. It would be helpful to demonstrate the method on more complex or diverse scenarios\n\n- The paper lacks an analysis of how the similarity between generated prototypes and dataset images affects performance, which would be important to clarify whether the method generalizes or exploits dataset biases.", "was_revised": true, "final_justification": "The authors have addressed my concern regarding the relationship between the prototype and the test set data and results. I am willing to raise my score accordingly.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedbacks as follows.\n\n---\n>**W1: The method relies on generating hundreds of prototype images per part class, which may introduce a risk of overfitting or inadvertently aligning too closely with the test set, especially on relatively small or simple datasets. It would be helpful to demonstrate the method on more complex or diverse scenarios.**\n\n**A1:** Thank you for pointing out this important concern regarding the potential proximity between generated prototypes and the test set. We provide the following clarifications:\n- The prototype images are entirely synthesized by Stable Diffusion without using any images from the target datasets. All part prototype images are generated using the publicly available pre-trained text-to-image model (Stable Diffusion v1.4). **These synthesized prototypes differ significantly from real-world test images in style, background, and composition**, thereby minimizing the risk of overfitting or unintended alignment with the test set.\n- To quantitatively assess the distinctness between the synthesized prototypes and the test set images, we conducted a **feature-level similarity analysis**. First, we input the prototype set and test set images into DINOv2 ViT-B/16 to extract global image features. Then, we performed L2 normalization on these image features and calculated the mean values of the features of the prototype set and test set respectively as their central vectors. The cosine similarity between these two central vectors was used to quantify the similarity between the prototype set and the test set. For comparison, we also calculated the similarity within the test set. We randomly split the test set into two halves: one half was used to generate a central vector $\\mathbf{V}_1$, whose similarity with the prototype set’s central vector $\\mathbf{V}$ was calculated. The other half was used to generate a central vector $\\mathbf{V}_2$, whose similarity with $\\mathbf{V}_1$ was calculated as a reference for the consistency of the test set. The results are shown in the table below. The similarity between the prototype set and the test set is significantly lower than the internal similarity of the test set, indicating that the synthesized prototypes are significantly different from the test set images and there is no risk of \"excessive proximity\".\n| Similarity Type | **Pascal-Part-116** | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n| Prototype set vs. Test set |0.63 ± 0.01|0.59 ± 0.01|0.62 ± 0.01|\n| Within Test set |0.87 ± 0.01|0.86 ± 0.02|0.89 ± 0.01|\n\n- In terms of generalizability, PBAPS has been comprehensively evaluated on three highly diverse datasets: Pascal-Part-116 (natural objects such as animals), ADE20K-Part-234 (indoor scenes with furniture and appliances), and PartImageNet (cross-domain categories like vehicles and instruments). These datasets vary significantly in part types, scene complexity, and domain coverage. As shown in Table 2, using a single prototype generation process and fixed hyperparameters, **PBAPS achieves state-of-the-art results across all three datasets**, validating its strong cross-domain generalization ability.\n\nIn summary, the prototype generation process in PBAPS is entirely independent of the target datasets and does not introduce any data leakage risk. Our quantitative analysis confirms that the synthesized prototypes are significantly distinct from the test images, alleviating concerns of overfitting or excessive similarity. Furthermore, the consistent performance across diverse datasets demonstrates the robustness and generalizability of our method, highlighting its suitability for large-scale and cross-domain open-vocabulary part segmentation tasks.\n\n>**W2: The paper lacks an analysis of how the similarity between generated prototypes and dataset images affects performance, which would be important to clarify whether the method generalizes or exploits dataset biases.**\n\n**A2:** Thank you for highlighting the potential impact of prototype-test image similarity on model generalization. We provide the following clarifications and additional analysis:\n- It should be emphasized that the approach of constructing visual prototypes using Stable Diffusion + SAM + DINO has been adopted and validated as effective in multiple existing works (e.g., OVDiff [46], RIM [38]). **Our contribution does not lie in the prototype construction, but rather in addressing the challenge of low segmentation accuracy at structurally connected part boundaries in OVPS.** Guided by HPCGraph, hierarchical segmentation is performed, and the BAR module improves boundary precision via feature optimization and dynamic prototype mechanisms. This design is the key reason why our method PBAPS outperforms the state-of-the-art method RIM by **7.3%, 18.1%**, and **8.4%** on Pascal-Part-116, ADE20K-Part-234, and PartImageNet, respectively (as shown in Table 2).\n- Although the prototype generation mechanism is not the contribution of this paper, we agree that this mechanism may affect model performance. Following your suggestion, we designed experiments to **analyze the relationship between \"prototype image-test set similarity\" and model performance (mIoU).** First, we synthesized prototype images with \"low similarity\" (<0.3) to real test set images by modifying the style prompts of Stable Diffusion (e.g., \"cartoon\"). Then, we slightly augmented and perturbed the training set images of the target datasets to generate prototype images with \"high similarity\" (>0.8). We evaluated the performance of PBAPS using prototype features generated from these two types of prototype images. The results are shown in the table below. Segmentation performance is positively correlated with prototype similarity, which is reasonable and expected. This is because prototypes with high similarity can more easily provide discriminative visual features, enabling more accurate segmentation. However, using prototypes with high similarity to the target dataset may weaken the model's cross-domain generalizability and make it less robust to style or distribution shifts.\n| Prototype-Test Similarity | **Pascal-Part-116** | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n| Low (<0.3)|43.20|21.87|38.62|\n| Our method (0.55~0.65) |46.35|24.70|42.61|\n| High (>0.8)| 50.83|29.04|47.06|\n- To address the discrepancies between the prototype set and the test set in terms of style, distribution, or visual details, relying solely on static visual prototypes may lead to inaccurate matching or misaligned responses, ultimately affecting the precision of part localization. To mitigate this issue, the BAR module introduces a **Dynamic Prototype Adaptation mechanism, which effectively enhances the adaptability of prototypes to test images** and improves segmentation performance. As described in Lines 182–189, this mechanism fuses the global prototype with high-confidence region features extracted from the current image to generate an adaptive prototype that better aligns with the actual content of the image. Compared to using only pre-generated static prototypes, this fusion strategy enables the model to capture instance-specific variations and contextual cues, making the prototype more consistent with the spatial distribution and structural characteristics of the target part. As shown in the ablation study (Table 3), removing the dynamic prototype adaptation mechanism leads to a significant drop in mIoU and bIoU across all three datasets, further confirming its effectiveness in improving segmentation accuracy and mitigating the impact of cross-image discrepancies.\n\nIn summary, while prototype similarity does influence segmentation performance, our method is carefully designed to **maintain a balance between effectiveness and generalizability.** The use of synthesized prototypes with moderate similarity, combined with the BAR module's dynamic prototype adaptation mechanism, ensures robust performance without overfitting to specific datasets. These design choices allow PBAPS to generalize well across diverse domains while preserving high segmentation accuracy.\n\n---\nWe once again sincerely thank the reviewer for the insightful and constructive feedback. If there are any further concerns or points requiring clarification, we would be glad to provide additional details.\n\n[38] Image-to-Image Matching via Foundation Models: A New Perspective for Open-Vocabulary Semantic Segmentation (CVPR 2024)\n\n[46] OVDiff: Diffusion Models for Open-Vocabulary Segmentation (ECCV 2024)"}, {"review_id": "rJj7VEBGKv", "reviewer": "Reviewer_76j9", "summary": "This paper addresses a critical challenge in Open Vocabulary Part Segmentation (OVPS): the inaccurate delineation of boundaries between structurally connected parts. To solve this, the authors propose PBAPS, a novel training free framework capable of performing OVPS without any additional learning.\n\nPBAPS consists of three main components:\n\n- HPCGraph, which models the hierarchical relationships between objects and their parts\n\n- BAR (Boundary Aware Refinement), a module that detects ambiguous boundary regions and uses surrounding context to enhance feature representations and adapt visual prototypes\n\n- A training free framework that leverages pre-trained models such as Stable Diffusion, SAM, and DINOv2 to generate visual prototypes and perform segmentation purely through inference\n\nThe proposed method achieves state-of-the-art performance on three benchmarks: Pascal Part 116, ADE20K Part 234, and PartImageNet. It shows significant improvement over existing methods, especially in accurately segmenting structurally connected part boundaries.", "questions": "- In the Cross-Model Prototype Induction stage, the visual prototypes are entirely dependent on the localization capability of Stable Diffusion. However, Stable Diffusion does not always highlight regions that accurately correspond to the given input prompt. What is the likelihood that the resulting visual prototype actually represents the region corresponding to the prompt?\n- Additionally, the cross-attention maps in Stable Diffusion vary significantly depending on the timestep and the layer of the U-Net. From which timestep and which specific layer of the U-Net do you extract the attention maps for localization purposes?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper clearly identifies the fundamental cause of segmentation errors at structurally connected part boundaries, focusing on the intrinsic conflict between continuous visual features and discrete classification mechanisms, and proposes a solution to address this issue.\n\n- PBAPS is the first training-free framework in the OVPS field, enabling segmentation without any additional training. This approach demonstrates strong zero-shot generalization to unseen part classes.\n\n- The method achieves state-of-the-art mIoU and bIoU scores on three benchmarks (Pascal-Part-116, ADE20K-Part-234, and PartImageNet) surpassing both supervised and existing training-free methods by a significant margin.\n\nWeakness\n- Dependence on foundation models: Despite the advantage of being training-free, PBAPS heavily relies on large-scale pre-trained foundation models such as Stable Diffusion, SAM, and DINOv2 for visual prototype generation and feature extraction. As a result, its performance and applicability may be influenced by the quality, availability, and domain compatibility of these models.\n\n- Sensitivity to hyperparameters: The performance of the BAR module is sensitive to hyperparameters such as feature fusion weight, prototype adaptation coefficient, and number of sub-prototypes. Achieving optimal performance requires careful tuning, which can hinder the model’s generalizability and ease of deployment across diverse scenarios.\n\n- Manual construction of HPCGraph: As acknowledged by the authors, the construction of HPCGraph is not an automated process but rather a manual one, relying on predefined object-part hierarchies or external knowledge. This limits the direct applicability of PBAPS to new domains or complex object categories where such structural priors are not readily available.\n\n- Incomplete ablation study: The ablation study for the proposed method is conducted solely on the Pascal-Part-116 dataset. This presents a limitation in assessing the generalizability and robustness of individual components, as it remains unclear whether similar trends would hold across other datasets like ADE20K-Part-234 or PartImageNet.", "was_revised": true, "final_justification": "The paper identifies the root cause of segmentation errors at structural boundaries and introduces PBAPS, the first training-free OVPS framework with strong zero-shot generalization. Since the authors have addressed my concern well, I am raising my rating.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedbacks as follows.\n\n---\n>**W1: Dependence on foundation models.**\n\n**A1:** Regarding PBAPS's dependence on foundation models, we need to clarify: this dependence is not a flaw but rather a best practice in cutting-edge research. The value of this paper lies in **how to leverage the capabilities of foundation models to solve the core challenge of OVPS, namely \"low segmentation accuracy for structurally connected part boundaries.\"**\n\nFoundation models such as Stable Diffusion, SAM, and DINOv2, having been trained on massive amounts of data, possess strong general visual understanding capabilities, which serve as the premise for PBAPS's training-free design. The outputs of foundation models (such as SAM's masks and DINOv2's features) are essentially general visual signals, whereas OVPS requires structured cognitive understanding of part relationships: How to locate part boundaries from diffuse attention maps? How to adapt global prototypes to instance-specific differences in specific images? Our HPCGraph, through structured knowledge constraints, guides the model in hierarchical reasoning; the dynamic prototype mechanism within the BAR addresses the misalignment between foundation model outputs and the actual distribution of parts. These designs are our innovation—not relying on the foundation models, but constructing the paradigm for applying their capabilities. When more powerful foundation models emerge, our method can directly reuse their capabilities without any modifications.\n>**W2: Sensitivity to hyperparameters.**\n\n**A2:** Due to the space constraints of the rebuttal, we kindly refer you to our detailed response to Reviewer Sy1A’s Weakness 2, where we thoroughly address the selection rationale, robustness, and generalizability of our hyperparameter settings across diverse datasets.\n\n>**W3: Manual construction of HPCGraph.**\n\n**A3:** Regarding the scalability of HPCGraph, we provide the following supplementary explanations:\n- HPCGraph is constructed based on commonsense structural knowledge and does not rely on any manual annotations. As such, it incurs low construction cost and exhibits strong transferability and scalability. In large-scale object scenarios, **both the topological structure and visual prototypes can be shared and reused.** For novel object categories with similar structures to existing ones (e.g., “zebra” as a novel category relative to the existing category “horse”), we can directly reuse existing topologies (e.g., “zebra head ↔ zebra neck” correspond to the structure of \"horse\"). For PartImageNet (L198–200), we divided 158 object categories into 11 superclasses, where each superclass shares a unified topological graph. This strategy reduces the number of topology by **93%** (from 158 to 11) while maintaining high performance—PBAPS outperforms RIM by **8.4%** on PartImageNet (Table 2). When the number of object categories expands to tens of thousands, a large number of them are highly similar in form and structure (such as horses, donkeys, mules, zebras, etc.). We can use LLMs to analyze inter-category structural similarities and achieve cluster-level topology reuse (i.e., categories with similar structures share a set of topologies). Furthermore, part visual prototypes can also be reused (e.g., the prototype for “horse leg” can be applied to “donkey leg”).\n- In contrast to existing methods such as PartCLIPSeg [27] and PartCATSeg [53], which require extensive part-level annotations and retraining when introducing new object categories, our method is training-free and supervision-independent. It supports open-world scalability by simply extending the HPCGraph and adding a small number of visual prototypes.\n- As noted in Appendix A.1, when dealing with objects with irregular structures (e.g., modular furniture), the static graph structure has insufficient adaptability. To address this, a dynamic graph generation mechanism can be considered. The core logic is to **automatically add adjacency edges for highly relevant parts based on the spatial adjacency and visual feature similarity between part response regions**. The following table compares performance using static and dynamic graphs. While static graphs yield better performance in well-structured domains, the dynamic version provides competitive results and significantly enhances flexibility for deployment in unconstrained real-world scenarios. \n| HPCGraph | **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n|  | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU |\n| Dynamic | 44.63&ensp;&emsp;33.12 |23.87&ensp;&emsp;15.63| 41.66&ensp;&emsp;28.72 |\n| Static | **46.35**&ensp;&emsp;**34.46** |**24.70**&ensp;&emsp;**16.41** | **42.61**&ensp;&emsp;**29.31** |\n\n>**W4: Incomplete ablation study.**\n\n**A4:** In response to your comment, we supplement the ablation experiment results of PBAPS on the ADE20K-Part-234 and PartImageNet datasets. As shown in the table below, on three datasets with significant differences (natural objects, indoor scenes, and cross-domain complex objects), each component significantly improves the model performance (mIoU). This not only verifies the effectiveness of each component design but also demonstrates that our method has good generalizability and robustness in diverse data environments.\n\n| w/ HPCGraph |w/ BAR| **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-:|:-:|:-:|:-:|:-:|\n| | | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU | \n|  |  |40.74&ensp;&emsp;31.09 |20.59&ensp;&emsp;13.72 | 37.46&ensp;&emsp;25.78 |\n|  | √ |42.95&ensp;&emsp;32.34 |22.14&ensp;&emsp;14.53 | 39.54&ensp;&emsp;27.12 |\n| √ |  |44.08&ensp;&emsp;32.99 |23.62&ensp;&emsp;15.43 | 41.27&ensp;&emsp;28.58 |\n| √ | √ |46.35&ensp;&emsp;34.46 |24.70&ensp;&emsp;16.41 | 42.61&ensp;&emsp;29.31 |\n\n>**Q1: What is the likelihood that the resulting visual prototype actually represents the region corresponding to the prompt?**\n\n**A5:** Thank you for your valuable comment, we provide the following supplementary explanation:\n- It should be emphasized that the approach of constructing visual prototypes using Stable Diffusion + SAM + DINO has been adopted and validated as effective in multiple existing works (e.g., OVDiff [46], RIM [38]). **Our contribution does not lie in the prototype construction, but rather in addressing the challenge of low segmentation accuracy at structurally connected part boundaries in OVPS.** Guided by HPCGraph, hierarchical segmentation is performed, and the BAR module improves boundary precision via feature optimization and dynamic prototype mechanisms. This design is the key reason why our method outperforms OVDiff, RIM, and other methods on the three datasets (Table 2).\n- Regarding whether Stable Diffusion can accurately localize prompt regions, **we conducted a quantitative verification using CLIP cross-modal similarity.** For the synthetic images of each part category, SAM generates target masks based on the attention maps of Stable Diffusion. Subsequently, we used CLIP to calculate the similarity between the masked regions, non-masked regions, and the entire image with the target text prompts, respectively. The results are shown in the table below. The similarity score of the target masked regions is significantly higher than that of the entire image, verifying that Stable Diffusion can accurately focus on the target parts. In addition, the low similarity score of non-masked regions also indirectly confirms the accuracy of the masked regions.\n\n| Source of Image Feature | **Pascal-Part-116** | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n| Masked Region |0.75±0.04|0.74±0.03|0.72±0.04|\n| Entire Image |0.61±0.05|0.58±0.03|0.56±0.04|\n| Non-masked Region |0.21±0.03|0.20±0.03|0.22±0.02|\n- Furthermore, even if there is a certain deviation in the target masks (e.g., including a small number of non-target regions), leading to ambiguity in the generated part visual prototypes, the \"dynamic prototype adaptation\" mechanism of the BAR module can effectively alleviate this issue. **By weighted fusion of global prototypes with the features of deterministic regions in the current image, the prototypes are dynamically adapted to the real target parts in the image**, thereby improving segmentation accuracy. The ablation experiment in Table 3 also confirms the effectiveness of the \"dynamic prototype adaptation\" mechanism.\n\n>**Q2: From which timestep and which specific layer of the U-Net do you extract the attention maps for localization purposes?**\n\n**A6:** As shown in L129-131 and L456-467, we follow the prior works such as DiffuMask [45], OVDiff [46], and RIM [38], obtaining robust localization signals through cross-timestep and cross-layer aggregation. Cross-attention maps are extracted from all **50** diffusion timesteps, and noise from a single step is suppressed through an averaging operation. The attention maps come from the **16×16** resolution cross-attention layers generated during the downsampling process of the U-Net, as well as the **32×32** and **64×64** resolution cross-attention layers generated during the upsampling process. Finally, attention maps of different resolutions are scaled to 512×512 through interpolation and weighted averaged to generate the final semantic localization heatmap.\n\n---\n\n\n[27] PartCLIPSeg: Understanding Multi-Granularityfor Open-Vocabulary Part Segmentation (NeurIPS 2024)\n\n[38] Image-to-Image Matching via Foundation Models: A New Perspective for Open-Vocabulary Semantic Segmentation (CVPR 2024)\n\n[45] Diffumask: Synthesizing391 images with pixel-level annotations for semantic segmentation using diffusion models (ICCV 2023)\n\n[46] OVDiff: Diffusion Models for Open-Vocabulary Segmentation (ECCV 2024)\n\n[53] PartCATSeg: Fine-Grained Image-Text Correspondence withCost Aggregation for Open-Vocabulary Part Segmentation (CVPR 2025)"}, {"review_id": "8nEPQ0UNJO", "reviewer": "Reviewer_4fTr", "summary": "This paper presents Progressive Boundary-Aware Part Segmentation (PBAPS), a training-free model for open-vocabulary part segmentation (OVPS). This model takes the target part class vocabulary, and uses Stable Diffusion, SAM and DINOv2 to get “prototype” embeddings for all classes. Then, it uses a class hierarchy graph (called HPCGraph) and the generated prototype embeddings to segment an image in a top-down manner: first the object-level classes, then the part-level classes, then the subparts, etc. When segmenting the parts and subparts, the model applies a Boundary-Aware Refinement (BAR) module to refine initial segmentation predictions at boundaries between parts that are structurally connected (e.g., “dog-head” and “dog-neck”), as the paper observes that this is where many existing OVPS models struggle. Through experiments, PBAPS is shown to be more effective than existing OVPS models, achieving new state-of-the-art results. Ablation experiments show that the use of the HPCGraph for hierarchical segmentation and the BAR module are essential in achieving this performance.", "questions": "My initial rating of this paper is “Borderline Reject”, but I am leaning towards “Reject”. Although the presented method obtains new state-of-the-art results for OVPS, I believe the paper contains too many flaws to be accepted in its current state. Importantly, the paper does not acknowledge the similarity of aspects of the presented method with existing work, making it difficult to judge how significant the contributions are. Moreover, the clarity of several parts of the paper is poor, and an important limitation is not addressed.\n\nIn the rebuttal, I would encourage the authors to attempt to address my concerns as stated in the “weaknesses” section, and specifically to (a) explain how the proposed method compares to existing work and (b) clarify the operation of the method.", "limitations": "The paper lists some limitations of the work in the appendix, but does not mention the limitation of scalability. See weakness 7 in the 'Strengths and Weaknesses' part for more details.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1.\tThe presented model, PBAPS, is shown to be effective. On various OVPS benchmarks, it significantly outperforms existing models. This demonstrates the value of the presented model.\n2.\tThe proposed HPCGraph and BAR module are shown to be effective and essential to achieve the state-of-the-art performance that PBAPS obtains, through ablation experiments in Tab. 3. This validates the design of the proposed components.\n3.\tThe qualitative results shown in Fig. 5 provide relevant insights into the actual quality of the predictions by the different models. They show that PBAPS is better able to identify the correct boundaries between parts of an object, which was one of the objectives of this work.\n\n**Weaknesses**\n\n1.\tThe overall method – i.e., the use of Stable Diffusion, SAM, and DINOv2 to obtain class prototypes that can be used for open-vocabulary segmentation – is largely based on existing method RIM [a], but the paper does not acknowledge this in the paper nor provides proper attribution to this work. The paper does cite RIM as one of several open-vocabulary segmentation models in the related work, and compares to it experimentally in Tab. 2, but it does not mention that the overall method is largely based on it. The submitted paper should mention that the overall operation of PBAPS is largely based on RIM, and explain how it differs from this existing work. Besides being good scientific practice, this enables readers to properly judge the significance of the contributions of the work, which is not possible in the current version.\n2.\tThe concept of top-down, hierarchical object-to-part segmentation – called “progressive” segmentation in this work – is not new, e.g., it is also done by ViRReq [b] and TAPPS [c]. The paper should acknowledge these works, and explain how the proposed top-down segmentation approach differs from these existing approaches, to clarify what the significance is of the presented contribution. This is currently not done.\n3.\tIn general, the paper does not clearly describe how the proposed method compares to existing OVPS methods. The paper does list several existing OVPS methods in L72-L87, and briefly summarizes how they work, but it does not contrast the overall operation of PBAPS to that of existing methods. As such, again, the significance of this work’s contributions cannot be judged fully.\n4.\tThe abstract (L4) claims that this work proposes the first training-free OVPS framework. However, in Tab. 2, there are two other methods (OVDiff and RIM) that are also training-free. Although these methods were not specifically designed for OVPS but for open-vocabulary semantic segmentation, they do work for the OVPS task. Therefore, the claim in L4 is not fully justified, and it would be better if it were removed or altered.\n5.\tSeveral sections of the paper are hard to understand and follow because they use (technical) terms that have not been defined or explained yet. This limits the readability and clarify of the paper. For example:\n\n   \n    -\tL6-L10 of the paper uses many technical terms to describe the method, e.g., “cross-hierarchy matching”, “matching cost divergence”, “deterministic-region context”. However, these are not commonly used terms, but terms that are introduced in this paper, in later sections. As a result, it is not clear what these terms mean, and the abstract is difficult to understand. The abstract would be much clearer if (a) the method were explained at a higher abstraction level or (b) if these terms were explained.\n\n   -\tSimilarly, L40-L46 of the introduction describes the BAR module, but this description refers to concepts from the overall method that have not been explained yet, e.g., “part prototypes”, “matching cost”, “original part masks”. As a result, this description cannot be properly understood yet. The clarity of the introduction could be improved by first describing the overall operation of the method (currently done in L47-L52) and introducing these concepts, before explaining the operation of the BAR module.\n\n   -\tSimilarly, L114-L118 refers to “deterministic regions” and “part prototypes”, concepts that have not been explained yet.\n\n6.\tAlthough the rough, overall operation of the method becomes clear when reading the paper, several aspects of the model are not explained in sufficient detail. This limits understandability and reproducibility of the work. For example:\n\n   -\tL130-L131 mentions that cross-attention maps are extracted. However, the paper does not explain how these are extracted. The submitted paper cites 3 other works in this sentence, but it is not clear what exact operation is used. \n   -\tL134 describes that DINOv2 extracts features from masked regions, but it is not clear if masked image is fed through DINOv2, or if the full image is fed, after which only the per-pixel DINOv2 features that belong to the part masks are used to generate the visual prototype.\n   -\tL135 makes it seem like there is only one visual prototype per class $c$, but L208 and L522 suggest that there are multiple subcategory prototypes per class. This is inconsistent. Are there multiple prototypes per class, or just a single one? This should be clarified. \n   -\tIn the cost divergence map $D_ {A,B}$ (L165-L166), pixels that have a low matching cost with *both* part $A$ and $B$ will also have a low divergence. This could happen at pixels outside of the mask of the parent class, where it is quite certain that both part classes are not present. In this case, there is clearly no semantic ambiguity, but there is still a low cost divergence. Hence, the statement that “low divergence values indicate regions where both parts exhibit similar matching costs, leading to semantic ambiguity” is not fully true. I assume, therefore, that there is some masking with the parent segmentation mask or initial part segmentation mask, but this is not mentioned or explained. This masking is only done for the deterministic regions $A_ {\\textrm{det}}$ and $B_ {\\textrm{det}}$ (Eq. 3), but not for the ambiguous regions $U$. This should be clarified.\n   -\tL204 mentions that “hundreds” of images are generated for each part class, but this is quite vague. Without knowing the exact number, the work is not reproducible.\n\n7.\tA limitation of the method, which is not acknowledged in the paper, is that the method does not scale well to a larger vocabulary. If the objective is to do segmentation with a truly 'open' vocabulary, then we would like to have a vocabulary of perhaps millions of concepts. CLIP-based (or other text-embedding-based) methods scale to this relatively well, as we would only have to feed the text description of these concepts through the text encoder, and we could use them directly for segmentation. In the presented PBAPS method, however, we would have to feed all these millions of concepts through Stable Diffusion, SAM, and DINOv2, generating hundreds of millions of images to obtain the visual prototypes, which is rather inefficient. Moreover, we would have to manually construct the hierarchical HPCGraph, which would become intractable at such a scale. I would encourage the authors to acknowledge and reflect on this limitation in the submitted work.\n\nThere are some minor additional weaknesses that do not significantly impact my rating:\n\n-\tL132 and L167: The symbol $A$ is used for two different purposes, which is confusing.\n-\tL217: the spaces should be removed, “7. 74” should be “7.74”, etc.\n-\tL238: “effectivenes” should be “effectiveness”\n\n[a] Wang et al., “Image-to-Image Matching via Foundation Models: A New Perspective for Open-Vocabulary Semantic Segmentation,” CVPR 2024.\n\n[b] Tang et al., “Visual Recognition by Request,” CVPR 2023.\n\n[c] De Geus et al., “Task-aligned Part-aware Panoptic Segmentation through Joint Object-Part Representations,” CVPR 2024.", "was_revised": true, "final_justification": "Based on the different reviews, rebuttals, and the discussion with the authors, I upgrade my rating to 4 (borderline accept), but I remain concerned about some aspects of the paper.\n\nIn my initial review, I was mostly concerned about (a) improper attribution to existing work regarding both the prototype generation and hierarchical segmentation, (b) missing conceptual comparisons with existing methods, (c) the readability of the paper, (d) missing explanations, and (e) the limited scalability of the HPCGraph.\n\nIn the rebuttal, the authors have addressed most of these concerns by providing additional explanations and promising to revise the paper. Therefore, I am inclined to upgrade my rating. However, this upgrade is contingent on two things:\n\n1.\tI remain concerned about the scalability of the HPCGraph, as it will require much manual effort to scale it to a much larger vocabulary with nontrivial object-part hierarchies. However, the authors acknowledge this limitation in the rebuttal, and they explain that there are manners to mitigate it. As such, I am okay with this paper being accepted as long as the authors clearly acknowledge this limitation in the paper, and explain how it can be mitigated.\n2.\tAs highlighted in the review and during the discussion: if the paper is accepted, then the text of the paper needs to be updated significantly to improve readability and understandability. However, there is no way to verify or guarantee that the authors will truly resolve all the issues in the text in a revision, as it is not possible to provide an updated manuscript during the rebuttal period. However, I am willing to give the authors the benefit of the doubt on this matter, because they have provided an example of what the updated abstract will look like and explained how the other parts of the paper will be revised.\n\nRegarding the reviews by the other reviewers, I think the authors have properly addressed most of the identified concerns.", "rebuttal": "We thank reviewer for the constructive comments. We provide our feedbacks as follows.\n\n---\n>**W1-W3: Comparison with prior works.**\n\n**A1:** We agree that distinguishing our work from prior methods is essential for clarifying its contributions. In response, we provide the following explanation:\n\n- DiffuMask [45] uses attention maps from Stable Diffusion to obtain category masks for generating semantic segmentation datasets. OVDiff [46] combines diffusion models, segmenters, and feature extraction models to achieve OVSS. RIM [38], building on OVDiff, introduces a stronger feature encoder (DINOv2) and segmenter (SAM), and then conducts relation-aware matching. We highly recognize the contributions of these works, among which the visual prototype construction module provides an important foundation for subsequent research. Our paper follows this technical route, but **our innovation does not lie in prototype construction. We focus on the problem of \"low segmentation accuracy for structurally connected part boundaries\" in OVPS.** Guided by HPCGraph, hierarchical segmentation is performed, and the BAR module improves boundary precision via feature optimization and dynamic prototype mechanisms. This is also the key reason why our method outperforms OVDiff, RIM, and other methods on the three datasets (as shown in Table 2).\n\n- The progressive segmentation guided by HPCGraph also differs significantly from the top-down fine-grained segmentation in ViRReq [24] and TAPPS [62]. Firstly, when calculating the matching score between pixel features and visual prototypes (L145-147), we fully utilize object-part and part-part relationships. The matching cost between a pixel feature f and a node $v ∈ V$ is defined as the maximum cosine similarity within its dominance set $D(v)$ = {$v$} $\\cup$ { descendant nodes of $v$ }. **This combines the current node with all its child nodes into a joint semantic space, thereby fully leveraging hierarchical context to enhance discriminative power.** In addition, the adjacency edges in HPCGraph representing the structural connection between two parts (L141-144). These adjacency edges guide BAR to refine the boundaries of structurally connected parts.\n\n- It should be emphasized that we have never intentionally avoided referencing existing works, nor have we claimed visual prototype construction as our innovative contribution. In Section 2 (Related Work), we provided a brief overview of the aforementioned related works. In Table 2, our method was also fairly compared with these works. Meanwhile, to avoid misleading readers, Section 4.1 only briefly describes the prototype generation process and cites relevant works. Due to the space constraints of the main text, we did not elaborate on the differences between our method and existing works. In the subsequent revision of the paper, we will add a separate section to this end, helping readers more intuitively understand the core contributions and research significance of our work.\n\n>**W4: The statement \"propose the first training-free OVPS framework\" in the abstract may cause ambiguity.**\n\n**A2:** Thank you for your valuable comment, and we apologize for any confusion caused by the original wording regarding “the first training-free OVPS framework.” We provide the following clarification:\n- To avoid any potential ambiguity, we will revise the original abstract statement to: “We propose a novel and effective training-free framework specifically designed for OVPS...” This revision more accurately convey the positioning and contributions of this work.\n- Although OVDiff and RIM can indeed be applied to OVPS, they are not inherently designed for part-level segmentation tasks. Specifically, these methods focus on object-level OVSS, and their usage in OVPS represents only an indirect adaptation. **They do not address OVPS-specific challenges such as structural dependencies between parts and boundary ambiguity.**\n- In contrast, our method PBAPS is a novel and effective training-free framework explicitly designed for the OVPS task. It targets the core challenge of “low segmentation accuracy at structurally connected part boundaries.” Under the guidance of HPCGraph, PBAPS performs hierarchical segmentation and refines boundaries via the BAR module. As shown in Table 2, under the same prototype generation setup, PBAPS outperforms the state-of-the-art method RIM by **7.3%, 18.1%**, and **8.4%** on Pascal-Part-116, ADE20K-Part-234, and PartImageNet, respectively.\n\n\n>**W5: Readability of the paper.**\n\n**A3:** Thank you for your suggestions regarding the readability of the paper. We will make the following specific revisions:\n- For non-generic terms such as \"cross-hierarchy matching\" and \"deterministic region\" that appear early in the abstract and introduction, we will add brief, context-rich explanations when they first appear  (e.g., explaining how \"deterministic regions\" serve as reliable context for refining ambiguous boundaries), ensuring readers grasp their core meaning upfront.\n- We will restructure the abstract and introduction to first outline the overall goal of OVPS and highlight the key challenge of structurally connected part boundaries. The PBAPS framework and its core modules (HPCGraph, BAR) will then be introduced progressively in a logical flow, with technical details deferred until the basic concepts are established. Additionally, we will simplify overly technical language to better emphasize the motivation, main contributions, and high-level workflow, thereby improving accessibility for readers without a specialized background.\n\n\n>**W6: Some operations in the paper lack sufficiently detailed explanations.**\n\n**A4:** Thank you for pointing out that some operations in this paper lack detailed explanations. In response, we provide the following supplementary notes:\n- Regarding the extraction of cross-attention maps in L130–131. To avoid misleading readers into thinking this is an innovation of our work, we only give a brief description of this part in the main text (L130–131) and provide more details in Appendix B.1. We further supplement as follows: we follow the prior works such as DiffuMask [45], OVDiff [46], and RIM [38], obtaining robust localization signals through cross-timestep and cross-layer aggregation. Cross-attention maps are extracted from all 50 diffusion timesteps, and noise from a single step is suppressed through an averaging operation. The attention maps come from the 16×16 resolution cross-attention layers generated during the downsampling process of the U-Net, as well as the 32×32 and 64×64 resolution cross-attention layers generated during the upsampling process. Finally, attention maps of different resolutions are scaled to 512×512 through interpolation and weighted averaged to generate the final semantic localization heatmap.\n- Regarding the feature extraction for masked regions in L134. We use the masked image as input to DINOv2 for pixel feature extraction, instead of extracting features from the entire image first and then applying masking. This pre-masking operation enhances the accuracy of prototype construction.\n- Regarding the inconsistent descriptions of the number of visual prototypes (L135, L208, L522). Each part category corresponds to multiple subcategory prototypes, which is the configuration of our method. However, in L135, for the sake of narrative simplicity and to help readers understand, we described the visual prototype as \"one\", and this wording may have caused misunderstandings. We apologize for this.\n- Regarding the calculation range of the cost divergence map D (L165–166). Your understanding is correct. The cost divergence map is calculated only within the parent mask region. When constructing the cost divergence map, we only consider pixels that belong to the upper-level part segmentation mask (parent mask) to avoid misjudgment caused by irrelevant regions.\n- Regarding the number of synthetic images (L204). For most part categories, we generated 120 prototype images; only for a few categories with poor generation results, an additional 30 images were generated to improve prototype quality. Since we cannot add images, PDFs, or links during this NeurIPS rebuttal stage, we will provide a complete list of image quantity configurations for all categories in subsequent revisions of the paper and make the corresponding image data publicly available.\n\n\n>**W7: The scalability of HPCGraph.**\n\n**A5:** Due to the space constraints of the rebuttal, we kindly refer you to our detailed response to Reviewer 76j9’s Weakness 3, where we provide an in-depth explanation of the scalability of HPCGraph, including its construction principles, reuse mechanism across structurally similar categories (group 158 classes into 11 superclasses, topology count drops by **93 %** while PBAPS still leads RIM by **8.4 %**), and dynamic graph generation mechanism (**automatically add adjacency edges for highly relevant parts based on the spatial adjacency and visual feature similarity between part response regions**). The comparative results between dynamic and static graphs are as follows:\n| HPCGraph | **Pascal-Part-116**  | **ADE20K-Part-234** | **PartImageNet**|\n|:-|:-:|:-:|:-:|\n|  | mIoU&ensp;&emsp;bIoU | mIoU&ensp;&emsp;bIoU| mIoU&ensp;&emsp;bIoU |\n| Dynamic | 44.63&ensp;&emsp;33.12 |23.87&ensp;&emsp;15.63| 41.66&ensp;&emsp;28.72 |\n| Static | **46.35**&ensp;&emsp;**34.46** |**24.70**&ensp;&emsp;**16.41** | **42.61**&ensp;&emsp;**29.31** |\n\n---\n\n\n[24] Visual recognition by request (CVPR 2023)\n\n[38] Image-to-Image Matching via Foundation Models: A New Perspective for Open-Vocabulary Semantic Segmentation (CVPR 2024)\n\n[45] Diffumask: Synthesizing391 images with pixel-level annotations for semantic segmentation using diffusion models (ICCV 2023)\n\n[46] OVDiff: Diffusion Models for Open-Vocabulary Segmentation (ECCV 2024)\n\n[62] Task-aligned Part-aware Panoptic Segmentation through Joint Object-Part Representations (CVPR 2024)"}]

Author rebuttal:

Meta-review: This paper introduces PBAPS, a novel and effective training-free framework for open-vocabulary part segmentation that achieves state-of-the-art results on several benchmarks. The core contribution includes identifying and addressing the inaccurate segmentation of structurally connected part boundaries, which is a key weakness in prior methods. The authors propose a progressive, hierarchically-guided segmentation strategy using a Hierarchical Part Connected Graph (HPCGraph) and a Boundary-Aware Refinement (BAR) module, which intelligently identifies and refines ambiguous boundary regions. The primary strengths of the paper are its novel training-free approach, the technical soundness of the proposed modules, and the significant performance improvements demonstrated through comprehensive experiments. The main weaknesses initially identified by reviewers (e.g., Sy1A, BZzg, 4fTr) centered on the scalability of the manually constructed HPCGraph, the clarity of the writing, and proper attribution to prior work for prototype generation. However, the authors provided a thorough rebuttal that convincingly addressed these concerns. They demonstrated the feasibility of scaling the HPCGraph through topology and prototype reuse, supported by experiments, and proposed leveraging knowledge bases like WordNet. They also committed to significant revisions to improve readability and clarify the relationship with existing methods, even providing a revised abstract to reviewer 4fTr. Furthermore, they supplied additional ablation studies and comparisons to the latest supervised methods as requested by reviewers BZzg and 76j9, leading reviewers BZzg and LruU to raise their scores. The constructive discussion period has significantly strengthened the paper, and the proposed method stands as a solid contribution to the field. Therefore, AC recommends accept.
Decision: Accept (poster)