Title: LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Object Part Segmentation; Language Grounding; Multi-modality; Multimodal Large Language Model
Abstract: We propose LangHOPS, the first Multimodal Large Language Model (MLLM)-based framework for open-vocabulary object–part instance segmentation. Given an image, LangHOPS can jointly detect and segment hierarchical object and part instances from open-vocabulary candidate categories. Unlike prior approaches that rely on heuristic or learnable visual grouping, our approach grounds object–part hierarchies in language space. It integrates the MLLM into the object-part parsing pipeline to leverage rich knowledge and reasoning capabilities, and link multi-granularity concepts within the hierarchies. We evaluate LangHOPS across multiple challenging scenarios, including in-domain and cross-dataset object-part instance segmentation, and zero-shot semantic segmentation. LangHOPS achieves state-of-the-art results, surpassing previous methods by 5.5% Average Precision(AP) (in-domain) and 4.8% (cross-dataset) on the PartImageNet dataset and by 2.5% mIOU on unseen object parts in ADE20K (zero-shot). Ablation studies further validate the effectiveness of the language-grounded hierarchy and MLLM-driven part query refinement strategy.


Peer reviews:
comments: SUMMARY:
The authors propose a new method for Open-vocabulary Part Instance Segmentation (OVPIS) called LangHOPS by using hierarchical modeling in language space to encode of object parts. LangHOPS performs better than existing state-of-the-art methods across multiple benchmarks.

STRENGTHS AND WEAKNESSES:
Strengths: 

1. LangHOPS integrates MLLMs for OVPIS and grounds object-part hierarchies in language space. This is novel and superior to prior methods that struggle with modeling hierarchical relationships or handling part granularity variations.
2. The method achieves strong empirical results over baselines.

Weaknesses:

1. The object part granularity can be at different levels (as also seen from the supplementary figure 4). How to account for such ambiguities during evaluation? For example, if face is a sub-part for a human, what happens if the module parts with higher granularity like eyes, nose, mouth, etc?
2. The paper's math notations use incorrect formatting throughout. For instance, $C_{obj}$ should be $C_\text{obj}$ and $C_{part}$ should be $C_\text{part}$ and so forth. This inconsistency hampers readability, and I strongly encourage the authors to correct these notation errors.

QUESTIONS:
Please see weaknesses

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n* This paper presents LangHOPS, a new framework for Open-Vocabulary Part Instance Segmentation (OVPIS).\n* LangHOPS explicitly embeds object–part hierarchies in language space, providing a structured representation of object-part relationships.\n* The method integrates a Multimodal Large Language Model (MLLM) to enable context-aware and granularity-adaptive object-part parsing.\n* Compared to previous works based on heuristics or constrained CLIP embedding space reasoning, LangHOPS introduces language-grounded part queries for accurate and scalable instance-level segmentation.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n* The paper introduces the first integration of Multimodal Large Language Models (MLLMs) for the task of object-part instance segmentation, demonstrating originality.\n* By explicitly modeling language-grounded object–part hierarchies, LangHOPS improves part query representation, enabling enhanced multi-granularity reasoning and context awareness compared to prior CLIP-based methods.\n* The experimental evaluation is covering major benchmarks, including PartImageNet, Pascal-Part-116, and ADE20K, and demonstrates scalability, with further performance gains observed by incorporating additional training data.\n\n### Weaknesses\n* The proposed approach appears to be an extension of PartGLEE with an integrated MLLM, but the architectural or training-level novelties beyond this integration are not clearly articulated. A more precise explanation of additional contributions, if any, would strengthen the claim of originality.\n* Since LangHOPS relies on an MLLM (specifically, PaliGemma2), it would strengthen the work to provide quantitative comparisons regarding computational resources, such as model parameters and inference costs, relative to baseline methods.\n* In Section 3.2 Method Overview, the sub-section title \"Object Segmentation\" is redundantly repeated in Lines 130 and 133, which should be clarified. Furthermore, inconsistencies in terminology between Figure 2, Section 3.2, and subsequent subsections (e.g., Object-Part Parsing, Part Segmentation) could lead to confusion and should be aligned.\n* As noted in the NeurIPS Paper Checklist [7. Experiment statistical significance], the experimental results lack information on error bars or measures of variability, limiting the assessment of result consistency and robustness.\n* The NeurIPS Paper Checklist indicates \"[5. Open access to data and code] Yes,\" but I could not find a clear link or reference for accessing the released data and code at the time of reviewing.\n* Minor typos present in the manuscript:\n\t* Line 50: produing → producing\n\t* Line 131: featrues → features\n\t* Line 243: dateset → dataset\n\t* Table 1: artImageNet → PartImageNet\n\nQUESTIONS:\n* In Table 1 and Table 2, experiments are conducted using Pascal-Part-116, which, to my understanding, only provides semantic part segmentation annotations (e.g., \"bus's headlight\"). Could the authors clarify how part instance segmentation (e.g., \"bus 1's headlight 1\") is performed using this dataset? Specifically, how was the OVPIS task implemented on PPS-116, which lacks instance-level part annotations?\n* In Figure 1, the second example (two people walking) appears to demonstrate semantic segmentation, rather than instance-level segmentation, particularly for the parts. Could the authors clarify whether this example illustrates part instance segmentation or standard semantic segmentation?\n* In Line 192, the paper provides an example of the structured prompt input for the MLLM-based object-part parsing. Could the authors elaborate on what specific outputs are produced by the MLLM at this stage?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces LangHOPS, the first framework that tackles open-vocabulary object–part instance segmentation (OVPIS) by (i) explicitly encoding object–part hierarchies in CLIP language space and (ii) passing these language-grounded queries through a lightweight MLLM (PaliGemma-2) to refine part queries before decoding. Experiments on three benchmarks (PartImageNet, PascalPart-116, ADE20K) show consistent state-of-the-art gains: +5.5 AP (in-domain) and +4.8 AP (cross-dataset) on PartImageNet, and +2.5 mIoU on unseen ADE20K parts. Ablations confirm that both the language-grounded hierarchy and the MLLM parser are indispensable and that joint training yields positive object–part synergy.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths:\n\n  1). Solid empirical evidence: thorough in-domain, cross-dataset, zero-shot, scalability and ablation studies.\n\n  2). Clear Implementation details: two-stage training, clear loss formulations and hyper-parameters.\n\n  3). Significance and originality: First to integrate an MLLM for OVPIS; shows that language-grounded hierarchies boost both part and object quality; could inspire similar hierarchical designs in other dense tasks. Combines hierarchical language modelling with MLLM-driven query refinement; introduces explicit synergy analysis.\n\n\n- Weaknesses\n\n  1). Error bars / statistical significance are missing; some ablations (e.g., prompt sensitivity, MLLM size) are limited; compute cost for full training set (INS + PART) not fully quantified.\n\n  2). Impact depends on availability of strong MLLMs; benefit over strong vision-only baselines (e.g., clipped Q-Former) is moderate on ADE20K (49.5 hIoU vs 50.0 of PartCatSeg).\n\n  3). Repetition/typos (e.g., duplicated “Object Segmentation” heading), table captions compressed, and some notation (Np, H) introduced late.\n\nQUESTIONS:\n1). Statistical robustness – Please report confidence intervals (e.g., ± std over three seeds) for main tables. A ≥ 3 pt drop might change my rating.\n2). Prompt sensitivity – How stable is performance to wording or ordering of the structured prompt? A study on a held-out set would strengthen claims of robustness.\n3). Compute footprint – Provide GPU hours and carbon cost for each training stage and for the large INS + PART run; if substantially larger than baselines, discuss trade-offs.\n4). Hierarchy construction – The method assumes a clean object→part taxonomy. How does LangHOPS behave when candidate lists are noisy or overlapping? An experiment with automatically mined candidates would be helpful.\n5). Synergy mechanism – The paper shows gains but not why gradients from parts help objects. Visualizing attention maps before/after joint training could clarify this; stronger evidence could raise my score.\n\nLIMITATIONS:\nThe paper openly notes that synergy is implicit and that the role of language-guided hierarchies on object quality is not fully characterized (Sec. 4.4). However, potential societal impacts (e.g., misuse of fine-grained segmentation for surveillance) are not discussed; consider adding a Broader-Impacts paragraph.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an object-level instance and object-specific part-level instance segmentation method. The results on several datasets surpass previous methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThis paper provides a detailed definition of the problem associated with this \"new\" task. The experiments conducted on different benchmarks verify the effeteness of the proposed method, particularly in the cross-dataset experiment.\n\nWeaknesses:\n\nHowever, the motivation for this task is not explained well. The object-specific part-level instance segmentation is more like a combination of part-segmentation and instance-segmentation tasks. It is hard for readers to get the motivation in the beginning.\n\nIn addition, the proposed method includes several components while the connection is not strong. The part segmentation module and the object segmentation have overlaps for a specific object category. For instance, both parts can learn the information for \"bus\". They are no relations with this information when building the structure. \n\nAnother critical issue is why the authors refer to it as language-grounded. The processing of the textual part is limited. The prompts proved from neither object segmentation nor part segmentation are passed through a frozen text-encoder, which is followed by a concatenation. The additional information plays limited role as \"language-grounded\".\n\nThe paper requires careful proofreading, particularly in image captions and Section 3.2, 4.3.\n\nQUESTIONS:\n1.  Line 42-47, can you explain the limitations of previous methods? I do not entirely agree with your points.\n2. Why do the authors train the visual and pixel backbones, which is very rare? \n3. What is the difference between the proposed method and a set of learnable, universal queries in PartGLEE in Section 3.4?\n\nLIMITATIONS:\nSome parts are insufficiently explained in section 3.2.\nThe improvements in Tables 2 and 3 are not significant.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}