DeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection
Authors: Siheng Wang
Yanshu Li
Bohan Hu
Zhengdao Li
HaiboZhan
Linshan Li
Weiming Liu
Ruizhi Qian
Guangxin Wu
Hao Zhang
Jifeng Shen
Piotr Koniusz
Zhengtao Yao
Junhao Dong
Qiang Sun
Venue: ICLR 2026 Poster (2026)
Keywords: Open-Vocabulary Object Detection
Knowledge Distillation
Multi-modal
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Open-Vocabulary Object Detection (OVOD) plays a critical role in autonomous driving and human-computer interaction by enabling perception beyond closed-set categories. However, current approaches predominantly rely on multimodal fusion, facing dual limitations: multimodal fusion methods incur heavy computational overhead from text encoders, while task-coupled designs compromise between detection precision and open-world generalization. To address these challenges, we propose Decoupled Cognition DETR, a vision framework featuring a three-stage cognitive distillation mechanism: Dynamic Hierarchical Concept Pool constructs self-evolving concept prototypes using LLaVA-generated region descriptions filtered by CLIP alignment, aiming to replace costly text encoders and reduce computational overhead; Hierarchical Knowledge Distillation decouples visual-semantic space mapping via prototype-centric projection, avoiding task coupling to enhance open-world generalization; Parametric Decoupling Training coordinates localization and cognition through dual-stream gradient isolation, further optimizing detection precision. Extensive experiments on the common OVOD evaluation protocol demonstrated that DeCo-DETR achieves state-of-the-art performance compared to existing OVOD methods. It provides a new paradigm for extending OVOD to more real-world applications.

Published as a conference paper at ICLR 2026
DECO-DETR: DECOUPLED COGNITION DETR FOR EFFICIENT OPEN- VOCABULARY OBJECT DETECTION
Siheng WanglA., Yanshu Li Bohann Hut Zhengdao Li Haibo Zhan', Linshan Li' Weiming Liu', Ruizhi Qian', Guangxin Wn', Hao Zhang,, Jifeng Shen Piotr Koniusz Zhengtao Yao",, Junhao Dong* Qiang Sun
48=
Jiangsu University 2 Brown University Nanyang Technological University" MBZUAI
5 University of New South Wales USC 'Bytedance UUniversity of Toronto Data6 C
qsunstat s ggmaiia.ttmmmmmmmmmmmmmmmm.....m gmw.l.ma...mm com
ABSTRACT
Open-Vocabee Vocabulary Object Detection (OVOD) plays a critical role in many mod- ern computer vision applications by enabling perception that generalizes beyond closed-set categories. However, existing OVOD approaches face dual limitations that restrict their adoption in real-world scenarios: multimodal fusion methods incur substantial computational overhead due to reliance on text encoders, while task-coupled designs introduce a trade-off between detection accuracy and open world generalization. To address these challenges, we propose Decoupled Cog- nition DETR (DeCo-DETR), a novel vision-centric framework with three stage cognitive distillation mechanism. We first construct self-evolving concept proto- types using LLaVA-generated region descriptions filtered via CLIP-based align- ment to reduce computational overhead. We then enhance open- goon-world general- ization by decoupling visual semantic space mapping through prototype-centric projection. Finally, we design a targeted training strategy that coordinates local- ization and cognition via dual stream gradient isolation. Extensive experiments under the standard OVOD evaluation protocol show that DeCo DETR achieves competitive performance compared to existing methods while greatly improving efficiency. It provides meaningful insights toward more practical OVOD systems.
INTRODUCTION
Open-vocabulary object detection (OVOD) enables inference-time localization and classification of both seen and unseen classes during training, thus overcoming the outtof-diisributiinn limitations of traditional object detectors (Minderer et al. 2023 Zareian et al,, 2021a Gu el al., 202T). This ca- pability for real-time novelty recognition essential for various world applications, including autonomous driving (Cao et al.] (2023), biometric security (Bansal et al.] 2021), and interactive as-
CL.Pe-ttyle mroas alignment to extract textual cues to recognize unseen categories (Radford et al. 2021a).
sistants (Zou et al. 2023). A natural and early strategy for OVOD is
The emergence of large language models (LLMs) has introduced new perspectives for OVOD by enabling detectors to benefit from richer and more expressive semantic supervision (Xu et al., 2023 Fu al., 2025). Early attempts rely on prompt engineering to directly incorporate LLM- derived knowledge, but such designs require both the LLM and the detector to operate during in- ference, which significantly undermines efficiency. To mitigate this issue. knowledge distillation (KD) has gained increasing attention as practical mechanism for transferring semantic knowl- edge from large-scale multimodal L.LMs (MLLMM)) into lightweight detectors, thereby maintaining open-vocabulary recognition capability while reducing inference cost. A representative example is ViLD (Gu et al. 2021), which employs a VLM to produce text embeddings of category names and distills their alignment with visual features into the detector. Following this paradigm, subsequent works such as DK DETR (Li et al. 2023) and DetCLIP (Yoo et al., (002a)) further improve visual semantic alignment enhance detection of novel categories. Despite their effectiveness on standard benchmarks, existing distillation based approaches remain closely coupled with textual encoders,
Equal contribution. Corresponding authors.

---

Published as a conference paper at ICLR 2026
which leaves unresolved trade-offs between inference latency and open- spendworld generalization MMa et al. 2025a). These limitations become more evident in complex scenarios and expose two critical challenges
First, heavy computational overhead arises from the reliance on large text encoders or LLM-base prompt engineering during inference to generate textual cues for novel classes, hindering real-time deployment(Liu et al. (202cc). Second, a task compromise inherent in multimodal fusion designs of- forces difficult balance between achieving high closed-set detection precision and robust open- world generalization capability Zarrian et al. (2021b); Gu et al. (202T). This trade-off stems from the optimization conflict where aggressively tuning features for seen categories can bias the model, thereby degrading the vision- language alignment required for recognizing unseen classes (Zhang al., 2024; Fang et al. 2025). Consequently, existing methods often sacrifice performance on one
front to optimize the other.
To address the first challenge of computational bottlenecks caused by online text encoders, we propose the Dynamic Hierarchical Concept Pool (DHCP). Instead of repeatedly invoking heavy text encoders for each query, DHCP constructs self-evolving library visual-text prototypes that acts lightweight proxy for semantic knowledge. This process involves three stages: utilizing the Region Proposal Network (RPN) and LLa LLaVAiLiu et al. 2024a 2223a b) to generate rich region-tess textt pairs, filtering them via CLIP-based modsl alignment. and employing spectral clustering (K- Means(Ikotun et a 2023 for coarse concepts and DBSCAN (Deng, 2020) for fine details) to build hierarchical anchors. Crucially, to ensure these cached prototypes remain robust to distribution shifts without costly re-encoding, we introduce momentum updates with attention weighting. This mechanism drives the online refinement of the concept pool, effectively decoupling the detector from the text encoder during inference and significantly reducing latency.
To tackle the second challenge of task compromise between closed-set precision and open-world world generalization, we introduce a decoupled cognition framework consisting of two synergistic mech- anisms. First, the Hierarchical Knowledge Distillation (Hi-Know DPA) bridges the visual- semantic gap. It employs trainable projection networks to align detector features with CLIP's em- bedding space, using cosine similarity to generate semantic-enhanee queries that preserve spa- tial structure. Second, to fundamentally resolve the optimization conflict between localization and alignment, we propose Parametric Decoupling Training (PD-DuGi). This strategy enforces dual- stream gradient isolation via differentiable stop-gradient operators, which confine detection loss to localization parameters and semantic aligmment loss to cognition networks. weighting strategy further coordinates these objectives, prioritizing detection stability in early train- ing before progressively enhancing semantic alignment, thus achieving both high precision and ro- bust generalization.)
The contributions can be summarized as follows:
We reveal two critical flaws in existing vocabuucyyl detection: 1) Heavy reliance on text encoders and L.M prompting causes high inference latency; 2) Multimodal fusion forces painful trade-offs between closed-set precision and open-world generalization.
To address these issues, we propose the DeCo-DETR framework: It climinates text encoder dependency via the Dynamic Hierarchical Concept Pool (DHCP), solving computational bottlenecks in multimodal fusion during inference time; and enhances generalization in open scenarios through Hierarchical Knowledge Distillation (Hi-Know DPA) and Para- metric Decoupling Training (PD-DuGi).
We conduct extensive experiments on multiple open- opca-bacay detection benchmarks, including OV-COCO and OV-LVIS. DeCo-DETR achieves advanced performance on all benchmarks, delivering significant improvements of +3.1 to 5.8 points novel class APs while maintaining efficient 135ms inference. These results demonstrate DeCo's superior performance and generalization capabilities. Comprehensive ablation studies further vali- date the advantages of our novel design, providing transformative insights for the DETR- based detection paradigm and establishing a new foundation for future open-vocabulary research.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Open-vocabulary Object Detection (OVOD). OVOD, formalized in (Zareian et al., 202 uses image-caption data and base-class annotations to detect arbitrary categories, outperforming both zero-shot and weakly supervised methods (Cai et al., 2022b Yao et al., 2021 Advances in vi-
language models (VLMs) pre-trained on web-scale data (Radford et al. 202 la Jia et al. (021T)
significantly improved OVOD. One approach leverages VIM knowledge to generate pseudo- labels
for novel classes (Zhou et al. 2022b; Liu et al.] 2024b). using external sources or existing datasets
like LVIS (Gupta al., 2019), VL-PLM (Zhao et 2022a). Another refines VLM interaction through learnable prompts (DetPro (Khattak et al. 2024). PromptDet (Feng et al. 2022bl), surpass- ing static CLIP templates. However, these strategies incur high computational costs and incomplete
knowledge transfer (Zhu & Chen 2024). Thus, how to embed rich ppen-vocabulary semantics into
lightweight detectors has become a focus (Rasheed et al. 2022a Ma et al., 2022a Gu et al. (021).
Knowledge Distillation for VLMss Knowledge distillation (KD) effectively transfers capabilities
from large teacher models into compact student models (Xu al. 2024), addressing the growing demand for efficient vision-language functionality resource-constrained devices (Larrodie et al.,
2023). For instance, TinyCLIP (Wu et al. 2023a) boosts open- vocabulaay performance through advanced affinity mimicking and weight inheritance derived from CLIP. Subsequent research further expands specialized KD strategies to lightweight detectors, enabling more practical deployment of VLMs while preserving their generalization capabilities (Pei et al. 2023 Li et al. 2024b).
Knowledge Distillation for OVOD. KD is highly effective beyond tasks like semantic segmenta- tion (Ji et al. (025) and visual reasoning (Aditya al. 2019), showing significant impact in OVOD. VL.M into a two-stage detector, en- hancing generalization. DK-DETR (Li al.) 2023) further improves OVOD precision by distilling VLM knowledge into DETR which is transformer-basee architecture specifically designed for ob- ject detection (Carion et al. (020). architectures. KD has thus become mainstream in OVOD (Wang et al. 2023b Wu et al. 2023b; Rasheed et al., 2022b). However, reliance on textual cues from large models limits generalization and efficiency. CAKE (Ma al. 2025a) mitigates textual dependence but struggles with fine- fineigrained detection. DeCo-DETR addresses these gaps by implementing vision-centric mechanism that enhances perception without heavy textual dependency.
ViLDD ( Gu et al. successfully distills a classification--an
3 METHOD
3.1 FRAMEWORK OVERVIEW
Current multimodal fusion methods suffer from high computational overhead and task compromise. To mitigate this issue, we propose DeCo-DETR. DeCo-DETR aims to efficiently transfer open-set knowledge from LVLMSs compact detector without text encoders the test time. The over- all framework is illustrated in Figure I DeCo-DETR mainly consists of the following modules: Dynamic Hierarchical Concept Pool (DHCP): Constructs self-evolving concept prototypes using LLaVA- region descriptions filtered by CLIP alignment, replacing costly text encoders. Its two-level hierarchy simulates human cognition from coarse- coarse-grained (e.g.. "vehicle") to fine- ane-grained (e.g.. "hexagona wheels") granularity. Hierarchical Knowledge Distillation (Hi-Know DPA): De- couples visual-semantic space mapping via prototype-centric projection, aligning CLIP visual proto- types while disentangling semantic spaces of similar categories. Parametric Decoupling Training (PD-DuGi): Coordinates localization and cognition tasks through dual-stream gradient isolation. During inference, the dynamic prototype pool provides semantic knowledge while the dual-stream decoder processes spatial localization and semantic alignment in parallel.
3.2 DYNAMIC HIERARCHICAL CONCEPT POOL
To model hierarchical semantic spaces for open-vocabulary detection, we propose Self-Evolving Concept Pool framework (DHCP), which dynamically constructs and refines vasion-language joint spaces via cross cross-modal alignment and prototype distillation.
Moos- Feature Alignment: To build the concept pool, we extract multi-scale regions { N from the training images using pretrained backhone (e.g., ResNet) and Region Proposal Network
3

---

Published as a conference paper at ICLR 2026
LLaVA Generation(t))
CLIP Text Enc.
Momentum Update
BGG M
RPN Regions R:
Hierarchical Clustering
Concept
B
Alignment Filter
K. K-Means
BBSCAN
Coarse
Fine
Prototypes
CLIP Visual Enc.
Dynamic Hierarchical Concept Pool (DHCP)
Hierarchical Knowledge Distillation (Hi-Know DPA)
EGG
Backboms (ResNet/ ViT/Swin)
Attention-Awars Aggregation A)
Object Queries Q
MLP by (4.)
Encoder
B
Parametric Decoupling Training (PD-DuGi)
Localization stream BBox Objectness
Trandble
Enhanced
Query
Frezon
Teacher Target (Frozen CLIP)
((t)aai
Cognition stream (PDT Network g.)
Erhancod Querry cortains
Figure 1: Pipeline of DeCo-DETR. (a) Dynamic Hierarchical Concept Pool (DHCP): Con- structs self-evolving concept prototypes (covering coarse-grained e.g. "vehicle" to fine-grained e.g.. "hexagonal wheels") using LLaVA-generated region descriptions filtered by CLIP feature align- ment, aiming to replace costly text encoders. (b) Hierarchical Knowledge Distillation (Hi-Know DPA): Decouples visual-semantic space mapping via prototype-centric projection, aligning CLIP visual prototypes while disentangling semantic spaces of similar categories. (c) Decoupling Train- ing (PD-DuGi): Coordinates localization and cognition tasks through dual-stream (Obj Layer/Reg Layer for localization, Feature Alignment for cogmition) gradient isolation. During inference, the dynamic prototype pool provides semantic knowledge while the dual-stream decoder processes spa- tial localization (via BoxDelta and Objectness) and semantic alignment in parallel.
(RPN). Each region is then processed by LLaVA to generate free-form textual descriptions 4; ALaVA(R;) To climinate modality gaps, we project both the image regions and their corresponding text descriptions into a joint embedding space using CLIP's dual encoders:
Vi Sene (R:), === (4),
(1)
with high-confidence aligned pairs selected via cosine similarity thresholding initialized to be 0.7:
T = {R;;4t) 42) cos(u;, u:) > 8}-
(2)
Hierarchical Prototype Distillation: For aligned text embeddings {ej design spec- tral clustering-based hierarchical compression algorithm: Coarse-grained Anchors: Global Mcans[Ikotun al. 2023) clustering (with & M2 1203) extracts base prototypes (e.g.. "vehi- cle", "texture patter" ") from the aligned text embeddings {ey}. capturing broad semantic concepts and ensuring global connectivity. Fine-grained Units: Local DBSCAN(Deng 2020) clustering (with e = 0.5 and min_samples = 5) is applied to the embeddings within each coarse cluster. This further partitions them into an average of 4 fine-grained units per cluster (e.g., "sedan", "hor- izontal stripes")), resulting in a total of M2 = 4800 fine-grained prototypes. Together, they form multi-scale prototype matrix A RdxM where M M₁ + M2 and d is the CLIP embedding dimension. Details regarding the shared nature of M and the mapping relationship can be found in the Appendix A.4
Dynamic Memory Update: To adapt to semantic distribution shifts, we propose an attention-guide momentum memory update:
exp(r- cos(e;, Az))
Dr.j Σ 1= exp(7-1 cos(e;, A, A&))
(3)
4

---

Published as a conference paper at ICLR 2026
Algorithm Dynamic Hierarchical Concept Pool (DHCP)
Require: Training images D, Pretrained Backbone & RPN, LLaVA. CLIP (frozen).
Require: Hyperparameters: Similarity threshold 6, Momentum 7, Temperature т.
Ensure: Hierarchical Prototype Matrix A E R
1: Stage 1: Initialization (Offline)
2: Initialize alignment set <
3: for each image / С "D do
4: Extract regions { via Backbone and RPN
5: Generate descriptions: t, < LLa (R,)
6: Extract embeddings: V; f- CLIP ing (Ra). u, t (L
7: Filter pairs: if cos(e; uz) 5 then T + T [u]}
8: end for
9: // Hierarchical Clustering
10: Ceoarse + K-Means Initialize +
Mi) {e.g.. 1203 coarse concepts)
12: for each cluster E Ceoare do Cine DBSCAN (Fine-gran siine-grained discovery)
14: Append centroids of Cine to A
end for
16: Stage 2: Online Update (During Training)
17: while training do
18:
Receive batch aligned text embeddings {es} from current iteration
19:
Compute similarity matrix: Dij exp(r cos(ei.Az))
exp(r (,)))
20:
Update prototypes with momentum:
21:
A A (1 LayyrrrN)) Diye))
end while
Aj + >A, LayerNormn
Σ Dijez
where is learnable temperature parameter, E [0, controls memory decay rate, and Layer- Norm ensures numerical stability. This mechanism enables continuous semantic evolution through online adaptation, the full details can be found in Algorithm [
3.3 HIERARCHICAL KNOWLEDGE DISTILLATION
DHCP provides a bank of visual prototypes. To bridge the gap between visual features and semantic embeddings, we propose Hierarchical Knowledge Distillation Hni-Know DPA), which decouples visual-semantic mapping through trainable projection networks aligned with hierarchical concept prototypes. This mechanism operates through two synergistic phases:
Phase I: Cross-Modal Feature Projection: Given backbone feature map (07) R the transformer decoder produces object queries Q = { via multi-head attention mechanisms. To establish semantic grounding, we introduce trainable projection network he RC + Rd which aligns visual features to the CLIP embedding space:
án = ho(g).. Vgn E º
(5)
where d denotes the joint embedding dimension, and On denotes decoder query token. This para- metric mapping enables explicit modality alignment while preserving spatial-semane semantic relationships.
Phase Attention-Awan Aware Prototype Aggregation: To exploit the hierarchical concept prototypes (A E dd multi-granularity semantics, we compute prototype relevance using temperature- scaled cosine similarity:
exp(a cos(an. A,))
Wo.j
(6)
k= exp(a-1 cos(gn, Av))
5

---

Published as a conference paper at ICLR 2026
where is learnable temperature parameter controlling distribution sharpness. The resultant semantic-enhancee query In is computed as:
Ta Σ W + (L)(
where the residual connection with M.P-processed original features ensures stability during early training phases. This computational design emulates human perceptual mechanisms-first activat- ing coarse semantic anchors, then refining through detailed visual evidence.
Optimization Strategy: The entire framework is trained end-to-end using composite loss:
f= AKL Σ KL (...w//m))) Aalign Calign
where Lder denotes the standard DETR loss (Carion et al. 2020 Sh denotes the target attention distribution derived from the frozen CLIP teacher model's cross-modal matching between image features and text prototypes P, where P IR Mxd contains CLIP text embeddings of category names and
phrases. Wn denotes the prototype assignment weight vector generated by the student model (DeCo-DETR). The weighting coefficients ÅKI. and Aalign follow cosine annealing schedules to prioritize detection stability initially and semantic alignment subsequently, the full details can be found in Algorithm 2
Algorithm Hierarchical Knowledge Distillation (Hi-Know DPA)
Require:
Training set D, image 7
Require: Student: Backbone, proj. net hg, prototypes E RR
Require: Teacher: Pretrained CLIP (frozen), text prototypes P Mxd (from category names
LLaVA phrases)
1: while not converged do
2: Sample batch B C DD
3: for each / B do
4:
(() + Backbonee 7)
(Feature extraction]
5:
Q Decoder ( (f)) (Geneaate object queries)
6:
{ + h
PPooject features]
7:
Wn -- Softmax (a cos (m. A))
8:
Ta + E, Wn.j MLP((g)) (Semantic enhancement]
9:
ET + Softmax(T cos(ân: P)) {Target distribution from teacher)
10:
t Laa + AKL Σn, KL(W,, [,,,, + Aalign
end for
12:
Update student parameters 9 using L
Adjust AKL, Aulign (Cosine annealing)
14: end while
3.4 PARAMETRIC DECOUPLING TRAINING
To resolve potential representation conflicts between base detection and open-vocabulary alignment, we propose Parametric Decoupling Transformer (PDT) framework based on structured feature space orthogonalization.
Given the semantically enhanced query features r, Rd the PDT network 96 : Rd - R Crea generates pseudo-semantic probability distributions through hierarchical mapping:
i (oftmax(gg(r.)),,
where Do employs multi-layer cross-attention blocks to model prototype-category correlations. To suppress inter-modal interference, the Dual-stream Gradient Isolation Mechanism is designed: The detection loss Lda propagates solely to the DETR encoder-decoder parameters, while the semantic alignment loss
Lalign Σ T.. log(LinearHead( n
(10)

---

Published as a conference paper at ICLR 2026
Algorithm 3 Parametric Decoupling Training (PD-DuGi)
Require: Image I, Ground Truth Y, Student Model (Backbone, Decoder, PDT), Teacher CLIP.
Require: Learning rate scheduler Aatign (t).
Ensure: Optimized model parameters B.
1: Forward Pass:
2: (1)) <- Backbonee())
3: Q {1m <- Decoder( (7)) (Object queries)
4: Stream 1: Localization (Detection)
5: Y Ypred DeDeeetionHead((ge
6: Ldet + LoSSguuuaiia (Xpred.
7: // Gradients from Ldet update Backbone & Decoder
8: Stream 2: Cognition (Semantic Alignment)
9: E/2 StopGradient( {Isolate gradients from alignment loss}
10: On - he (4k) (Project to CLIP space)
n ProrotypeAggregattioo () {See Sec 3.3}
12: in Softmax (FDTTrrr (Parametric Decoupling Transformer)
13: T teacher + CLIP ClIPPreacher( (T, Prompts)
14: Lalign & CrossEntropy(fn Tteacher)
15: // Gradients from Lalign update only PDT & Projection he
16: Optimization:
Get current annealing weight: À (Ilign(I) {Cosine schedule)
18: Fiotal + Laet + A. Lulign
Update parameters 0 t Optimizer( V Leotal)
updates only the PDT parameters ø and the classifier head parameters. We implement explicit gra- dient stopping: gradients from Lalign are prevented from flowing the detection backbone and decoder, and vice versa for Laet- This architecture ensures: Knowledge Preservation: Cartesian product mapping y S > y between visual (V) and semantic (S) manifolds Dynamic Adaptabil- ity: Online prototype clustering enables extrapolation unseen semantic spaces
The unified objective function combines both streams through curriculum learning:
Listal Liee + Aaliga (t) Calign.
(11)
where >atign()) (f) follows cosine annealing schedule (increasing from 1) to prioritize detection stability initially and strengthen semantic alignment as training progresses. Inference requires only single-pass forward computation without post-processing, full details can be found in Algorithm
EXPERIMENT
4.1 DATASETS AND EVALUATION METRICS
Following standard protocols in the OVOD literature (Jin et al. 2024 Zhou et al., 2022b Ma et al. 2025a), evaluate the effectiveness of DeCo-DETR OV-COCO (Bansal et al.) 2018b) and OV-LVIS (Gu et al. 2021). These benchmarks are open- vocabulary variants derived from the popular MSCOCO (Lin al. 2015) and LVIS datasets, spectively. OV-COCO utilizes I 118,000 images from MSCOCO, designating 48 common categories as base classes and holding out categories as novel classes for zero-shot generalization. OV-LVIS reuses the same image set but applies LVIS annotations; among 1 1,203 categories, the 866 frequent and common categories form the base set, while the 337 rare categories are treated as novel. This long-tail distribution better reflects real-world category imbalance and presents greater challenge for OVOD methods. For OV-COCO, we report A5500 at an loU threshold of 0.5 for novel categories-as the primary metric. Additionally, we provide performance on base categories (AP50base) and overall performance across all categories (AP50). For OV-LVIS, report AP,. APc. and AP, denoting mAP rare, common, and frequent cat- egories, sospeciveey-alonn with the overall AP, all computed using standard box-based mAP. About the V-OVD, G-OVD, C-OVD and WS-OVD, more details can be found in Appendix A.5
widely adopted benchmarks:
mean Average Precision (mAP)

---

Published as a conference paper at ICLR 2026
Table 1: OV-COCO comparison (AP3o) across wide range of open-vocabulary object detection
(OVOD) methods.
Benchmark
Method
A pnovel
A
A 50
ViLD (Gu et
2021)
29.4
52.6
48.9
OADP TeDPTWang (Wang et al.
2023c)
30.0
53.3
47.2
DK-DETR et 2023
32.3
61.1
53.6
BARON Wu et al. 2023c
33.1
54.8
49.1
V-OVD
LBP (Li al. 2024a)
37.8
58.7
53.2
OC-OVD (Bangalan gath
h et al.
2022
36.6
54.0
49.4
GOAT (Wang et al. 2023a
36.4
53.0
48.6
CAKE (M et al. 2025b)
38.2
58.0
52.8
DeCo-DETR (Ours)
41.3
56.7
53.1
OV-DETR (Zang
2022)
29.4
61.0
52.7
VL-PLM (Zhao et al.
2022a
32.3
54.0
48.3
OADP (Wang et 2023cc
35.6
55.8
50.5
LP-OVOD (Pham 2024)
40.5
60.5
$5.2
G-OVD
CLIM Wu al. 2024
25.7
42.5
CCKT SerT-DerrZhang et al.
2025
53.2
RALF (Kim et al. 2024)
41.3
54.3
50.9
CAKE (Ma et al. 2025b)
39.1
58.1
53.1
DeCo-DETR (Ours)
47.1
60.2
55.0
RegionCLIP (Zhong al.
2022
26.8
54.8
47.5
CoDet (Ma et al. (0233)
30.6
52.3
46.6
C-OVD
BARON (w al. (202cc
35.8
58.2
52.3
BIRDet Zeng ct al. 2024
46.2
63.0
$8.6
CAKE (Ma et al.) 20255)
41.3
60.2
55.3
DeCo-DETR (Ours)
44.9
59.8
56.3
Detic (Zhou et al. 2022a)
28.4
53.8
47.2
GOAT (Wang et al. 2023a
36.4
53.0
48.6
WS-OVD
OC-OVD (Bangalati et al.
2022
36.6
54.0
49.4
CAKE (Ma et 2025b)
41.8
60.6
55.7
DeCo-DE: (Ours)
45.5
60.5
57.1
4.2 IMPLEMENTATION DETAILS
To validate the effectiveness of our method, we build the model upon DETR with ResNet-50, ViT- B/16, and Swin-T T backbones. The Dynamic Hierarchical Concept Pool (DHCP) comprises 1 1,203 coarse-grained and 4,800 fine-grained prototypes, which are iteratively updated via self-supervised contrastive learning. The dual-stream decoder consists of six Transformer layers, each with eight attention heads, and employs cosine-anneale fusion weights A(t) to balance the classification and regression objectives. The batch size fixed at 64 samples per GPU across 8xNVIDIA A100). Considering the hardware constraints, composite loss defined as L Laet + X())Aatign with X(I) annealed from 0.5 to 0.1 over training. The Dynamic Hierarchical Concept Pool is updated online with momentum Y 0.99, and the temperature = 0.07 used to sharpen similarity distributions. During inference, the experiments conducted single NVIDIA RTX 4090.
cropped the dataset for some ablation experiments. The
4.3
MAIN RESULTS
Benchmark. DeCo-DETR achieves advanced zero-shot detection performance on both OV-COCO and OV-LVIS benchmarks. As shown in Table 1 DeCo-DETR attains 41.3% APs0 on OV- COCO, surpassing the strongest baseline LBP (37.8%) by +3.5 points, while the overall APs0 (56.7%) outperforms all competitors (e.g., 53.6% for DK-DETR), On the challenging long-tailed OV-LVIS dataset (Table 2 DeCo-DETR achieves 29.4% AP, for rare classes, and sets a new record with overall AP of 35.2 %. These results demonstrate DeCo-DETR 's capability to mitigate classification bias in long-tailed distributions while maintaining high accuracy for common and frequent classes.
8

---

Published as a conference paper at ICLR 2026
Table 2:
OV-LVIS comparison (AP) across multiple open-vocabulary object detection methods.
Method
AP,
AP.
APr
AP
DetPro Du al.
2022
20.8
27.8
32.4
28.4
VLDet et al.
2022
21.7
29.8
34.3
30.1
OC-OVD BBangalatt att
al.
2022
21.1
25.0
29.1
25.9
OADP Wang et al. 2023c
21.9
28.4
32.0
28.7
CORA Wu et al.
2023d
22.2
32.0
40.2
33.5
BARON (Wu et al.. 2023c
23.2
29.3
32.5
29.5
CoDet (M et al.
20233
23.4
30.0
34.6
30.7
LBP (Li et al. 2024a)
24.1
29.5
32.8
29.9
LP-OVOD (Pham
(2024
19.3
26.1
29.4
26.2
Mamba (Wang et al.,
2025
29.3
34.2
36.8
35.0
BIRDet Zeng et al.
2024
26.0
21.7
29.5
25.5
RALF (um et al. 2024
21.9
26.2
29.1
26.6
DeCo-DETR (Ours)
29.4
33.1
38.9
35.2
Table 3: Inference latency, GFLOPs, and parameter size across three backbone architectures.
Method
Latency (ms/img)
GFLOPs
Params (M)
R50
ViT
Swin
R50
ViI
Swin
R50
ViT
Swin
Deformable DETR (Zu 2020b
120
210
220
220
320
325
41
87
DetPro (Du et al. 2022
140
250
260
240
340
345
45
91
100
UP-DETK (Da al. 2021
115
205
215
215
315
320
40
85
92
DeCo-DETR (Ours)
135
240
250
235
335
340
44
90
97
DeCo-DETR balances accuracy and efficiency. With ResNet-50 backbone (Table 3). inference la-
tency increases by only 5-15ms, computation (GFLOPs) by 5%. and parameters by 3% (44M vs. 41M). Compared to ViLDD (140ms/img) and DetPro (250ms/img), DeCo-DETR (135ms/img) achieves accuracy-efficiene trade-offs.
4.4 ABLATION STUDY
In this section, we adopt DETR as the base model. Table 4 presents an ablation study validating the contribution each component: Dynamic Hierarchical Concept Pool. Incorporating multi- granular prototypes (1,203 coarse 4,800 fine) improves APso novel by 2.5% compared to using single-level prototype pool. This result underscores the effectiveness of hierarchical semantic ab- straction: coarse-level prototypes capture broad inter-class distinctions, while fine-level prototypes model subtle intra-class variations. By jointly leveraging these multiscale semantic features, model is better positioned to generalize to novel categories under limited supervision, leading to notable performance gain. PD-DuGi. The integration of the PD-DuGi mechanism yields a com- prehensive improvement across all metrics, validating the necessity of resolving task conflicts in open-vocabulary detection. The introduction of dual-stream gradient isolation boosts A APpooel from 36.6% to 37.5% (+0.9%) and, notably, increases APfase from 54.0% to 55.1% (+1.1%). This si- multaneous gain suggests that sharing a unified feature space for both localization and semantic alignment often leads to optimization interference, where the gradients for semantic adaptation may degrade the spatial features required for precise bounding box regression. PD-DuGi effectively mitigates this issue by explicitly isolating the optimization paths; it allows the cognition branch to leam robust semantic representations for novel categories without distorting the structural features essential for base category localization, thereby achieving a superior trade-off between open-wond world generalization and closed-set precision. Cosine Annealing Weights. Dynamically balancing detec- tion and alignment losses using a cosine annealing schedule improves AP30 by an additional 1.6 %. The -ime-dependent coefficient X(t) initially emphasizes the alignment loss to encourage robust feature embedding early training, and gradually shifts focus toward the detection loss to refine localization and classification boundaries. This smooth transition alleviates potential conflicts be- tween the two objectives, promoting more stable convergence and improved detection accuracy on novel categories.

---

Published as a conference paper at ICLR 2026
Table 4: Extended Ablation Study.
Configuration
AP novel
AP base
AP 50
1. Baseline only
30.4
52.6
46.8
2. Hicrarchical DHCP
36.6
54.0
49.4
3. PD-DuGi (Gradient Isolation)
37.5
55.1
50.5
4. Cosine A(t) (Full Model)
38.2
55.5
51.0
Efficiency Analysis. Table 7 presents a comprehensive comparison of inference latency and de- tection performance. Compared to fusion-based methods like Grounding DINO, which rely computationally heavy text encoders (c.g.. BERT-Base Base) and suffer from high latency (~2800ms), our DeCo-DETR eliminates the text encoder dependency during inference. This architectural advantage results in significant speedup of approximately 2 (135ms vs. 280ms) while maintaining compet- itive accuracy 41.%% vs. 42.1% novel). Furthermore, among distillation-baser and decoupled methods, DeCo-DEL DETR has good performance, by increasing AP to +41.3 points while reduc- ing latency to 135ms (7.4 FPS). These results demonstrate that DeCo-DETR establishes a superior efficiency-accuracy trade-off, making it highly suitable for real-time open- ppe-vvocbbllary applications. Impact Different VLMs: We further investigate the impact of varying scales of vision-language models (VLMs) detection performance (see Table (). Experimental results indicate that when using smaller models (e.g.. 1.5 7B), there a noticeable limitation on the detection perfor- mance for novel classes (AP30 ), which is only 30.1%. %. However, when the model scale increases to 13B or larger (e.g.. LLaVA-1.5 13B, LLaVA-NEXT 13B, Qwen2.5. VL 32B), AP stabilizes between 38.2%-38.9%% showing significant improvement over the 7B model. This suggests that once the model parameter count exceeds a certain threshold (around 13B), further increases in pa- rameters have negligible impact detection accuracy. Therefore, in practical deployment, moderately sized VLM can be selected to balance performance and computational cost. Ablation on Queries and Prototypes. Table investigates the impact of decoder query quantity (N) and prototype granularity (M2). Regarding the number of queries, increasing N from 300 to 2000 yields a substantial performance gain of +4.8 APnooel- Thanks the parallel nature of the Transformer decoder, this improvement incurs only marginal latency overhead (~ (0mm). Notably, even with reduced set of N 300, our method achieves 36.5% AP Aonoett significantly outperforming previous state-of-the-art methods like ViLD (29.4%). Regarding prototype scale, the fine-grained units (M2) prove critical for open-vocabulary generalization. Removing them (M2 = 0) causes a sharp drop of 10.5 points in AProvel. validating the effectiveness of our Dynamic Hierarchical Concept Pool (DHCP). Conversely, doubling the fine-grained units to 9600 yields diminishing returns (+0..2% APnovet) while increasing memory usage and latency, confirming that M2 4800 is the optimal configuration.
5 CONCLUSION
In this work, we present DeCo-DETR, a novel open-vocabulary object detection framework. Our approach introduces DHCP (Dynamic Hierarchical Concept Prototypes) mine visual prototypes from DETR's attention mechanisms, enabling seamless alignment between image features and se- mantic concepts. A decoupled two-stage training strategy effectively separates detection objec- tives from semantic learning. minimizing task interference while preserving detection performance. Experiments demonstrate otate-of-the-art zero-shot detection results on LVIS and COCO bench- marks. Notably, our framework eliminates dependency on text encoders during inference, sig- nificantly accelerating deployment speed. The proposed architecture establishes a versatile plug- and-play foundation for open-environment perception. Its modular design readily supports self- supervised alternatives to CLIP embeddings and enables effortless extension of the DHCP frame- work to video analysis or 3D perception tasks. By advancing autonomous semantic understanding in vision systems, DeCo-DETR provides a scalable pathway for next sext-gennrattin adaptive perception in real- real-world applications.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We acknowledge the computing resources provided by USC, and the discussion between Professor Shen and Professor Sun.
REFERENCES
Somak Aditya, Rudra Saha, Yezhou Yang, and Chitta Baral. Spatial knowledge distillation to aid visual reasoning, In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 227-235. IEEE, 2019.
Hanoona Bangalath, Muhammad Maaz, Muhammad Uzair Khattak, Salman Khan, and Fahad Shahbaz Khan. Bridging the gap between object and image-level representations for open- vocabulary detection. Advances in Neural Information Processing Systems, 35:33781-33794, 2022.
Ankan Bansal, Karan Sikka, Gaurav Sharma, Rama Chellappa, and Ajay Divakaran. Zero-shot object detection. ECCV, 2018a.
Ankan Bansal, Karan Sikka, Gaurav Sharma, Rama Chellappa, and Ajay Divakaran. Zero-shot object detection. In Proceedings of the European conference computer vision (ECCV), pp. 384-400, 2018b.
Monika Bansal, Munish Kumar, and Manish Kumar. object recognition techniques: state-of-the art work. Archives of Computational Methods in Engineering, (663)):147-1111 2021.
Hakan Bilen and Andrea Vedaldi. Weakly supervised deep detection networks. In CVPR, 2016.
Maria Bravo, Sudhanshu Mittal, and Thomas Brox. Localized vision-language matching for open-vocabulary object detection. In GCPR, 2022.
Zhaowei Cai, Gukyeong Kwon, Avinash Ravichandran, Erhan Bas, Zhuowen Tu, Rahul Bhotika, and Stefano Soatto. X-detr: A versatile architecture for instance-wise vision-language tasks. In ECCV, 2022a.
Zhaowei Cai, Gukyeong Kwon, Avinash Ravichandran, Erhan Bas, Zhuowen Tu, Rahul Bhotika, and Stefano Soatto. X-detr: versatile architecture instance-wise vision-language tasks, 2022b. hhttps://arxt wwt.voorggabbb////ee 2204. 05626
Weipeng Cao, Yuhao Wu, Chinmay Chakraborty, Dachuan Li, Liang Zhao, and Soumya Kanti Ghosh. Sustainable and transferable traffic sign recognition for intelligent transportation sys- tems. IEEE Transactions Intelligent Transportation Systems, 24(12):15784-1577) 2023. doi:
N0.11009/ITSS.2222225555
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and
Sergey Zagoruyko. End-to-end object detection with transformers, 2020. URL https:
7/ //arxiv. //aaxiv.rrg//aaaa.aaaaaaa wtt.oorg/abbba//ooeeeeee 20872
Xinlei Chen, Hao Fang, Tsung-yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollar, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:/504.000 50.000355, 2015.
Zhigang Dai, Bolun Cai, Yugeng Lin, and Junying Chen. Up-detr: Unsupervised pre-training for object detection with transformers. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 1601-1610, 2021.
Dingsheng Deng. Dbscan clustering algorithm based on density. In 2020 7th International Forum on Electrical Engineering and Automation (IFEEA), pp. 943, 2020. doi: 10.1109//FE 2020.00199.
Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. Learning to prompt for open- vocabulary object detection with Aision-language model. In Proceedings of the IEEÉ/CVF conference on computer vision and pattern recognition, pp. 14084-14099 2022.

---

Published as a conference paper at ICLR 2026
Kai Fang, Anqi Zhang, Guangyu Gao, Jianbo Jiao, Chiharold Liu, and yunchao Wei. Combo: Conflict mitigation via branched optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Chengjinn Feng, Yujie Zhong, Zequn Jie, Xiangxiang Chu, Haibing Ren, Xiaolin Wei, Weidi Xie, and Lin Ma. Promptdet: Expand your detector vocabulary with uncurated images. In ECCV, 2022a.
Chengjian Feng, Yujie Zhong, Zequn Jie, Xiangxiang Chu, Haibing Ren, Xiaolin Wei, Weidi Xic, and Lin Ma. Promptdet: Towards open-vocabulary detection using uncurated images, 2022b. URL
http://w...r./a////.......2 16113
Shenghao Fu, Qize Yang, Qijie Mo, Junkai Yan, Xihan Wei, Jingke Meng, Xiaohua Xie, and Wei- Shi Zheng. Limdet: Learning strong open-vocabulary object detectors under the supervision of large language models. arXiv preprint arXiv:2501.1895. 2025.
Mingfei Gao, Chen Xing, Juan Carlos Niebles, Junnan Li, Ran Xu, Wenhao Liu, and Caiming Xiong. Open vocabulary object detection with pseudo bounding-boo labels. In ECCV, 2022.
Xiuye Gu, Tsung-Yi Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104. 13921,, 2021.
Agrim Gupta, Piotr Dollár, and Ross Girshick. Lvis: A dataset for large vocabulary instance seg- mentation, 2019.
ats//bbs//// 1088.90..0 031 95
Abiodun M. Ikotun, Absalom E. Ezugwu, Laith Abualigah, Belal Abuhaija, and Jia Heming. K- means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data. Information Sciences, 122:178-210, 2023. ISSN 0020-0255. doi: https://doi.
htg/10.1016/1/ 1tt.100016/..ss2222111..33 URL
srriclee/A1/502202555220022
Deyi Ji, Haoran Wang. Mingyuan Tao, Jianqiang Huang, Xian-Sheng Hua, and Hongtao Lu. Structural and statistical texture knowledge distillation for semantic segmentation, 2025. URL
https:// wwtpaarww....gaa.oee.eoeeeeeeeee wttp:or//g/.....oooo 03944
Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, and Tom Duerig, Scaling up visual and vision-language representation learning with noisy text supervision, 2021. Utt.httss://aaxt org. abs wwwp://pwp....o.ooooooooo 05918
Sheng Jin, Xueying Jiang, Jiaxing Huang, Lewei Lu, and Shijian Lu. Llms meet vlms: Boost open vocabulary object detection with fine-grained descriptors. urXiv preprint arXiv:2402.04630 2024.
Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Car- ion. Mdetr modulated detection for end-to-end multi-modal understanding. In ICCV, 2021.
Muhammad Uzair Khattak, Muhammad Ferjad Naeem, Muzammal Naseer, Lue Van Gool, and Federico Tombari. Learning to prompt with text only supervision for lasguaelanguage models, 2024.
http:v//./xxxvvoooooooooo www./voorr/aaaa abs 2401. 02418
Jooyeon Kim, Eulrang Cho, Sehyung Kim, and Hyunwoo J Kim. Retrieval-augmented open- vocabulary object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17427-17436, 2024.
Clement Laroudie, Andrei Bursue, Mai Lan Ha, and Gianni Franchi. Improving clip robustness with knowledge distillation and self-training. arXiv preprint arXiv:2309.1036) 2023.
Jiaming Li, Jiacheng Zhang, Jichang Li, Ge Li, Si Liu, Liang Lin, and Guanbin Li. Learning back- ground prompts to discover implicit knowledge for open vocabulary object detection. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16678- 16687, 2024a.
12

---

Published as a conference paper at ICLR 2026
Liangqi Li, Jiaxu Miao, Dahu Shi, Wenming Tan, Ye Ren, Yi Yang, and Shiliang Pu. Distilling detr with iisual-linguiste knowledge for open- vocabuocyyl object detection. Proceedings of the IEEE/CVF International Conference Computer Viston. pp. 6501-6510, 2023.
Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang. Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre- training. In CVPR, 2022.
Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, and Jian Yang. Promptkd: Unsupervised prompt distillation for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16617-26626, 2024b.
Chuang Lin, Peize Sun, Yi Jiang, Ping Luo, Lizhen Qu, Gholamreza Haffari, Zehuan Yuan, and Jianfei Cai. Learning lbject-language alignments for open-vocabulary object detection. arXiv preprint arXiv:2211. 14843, 2022.
Tsung- Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitmick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015, URL http::t///x/xi...ootmt org/ abs// /tw.a////......... 0312
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023b.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL httpp::/
llava ww..itt.tiii.. htt.giittubb.ooo/lc//2220
01 30 1lava -ewatnnee//
Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303. 05499, 2023c.
Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding dino: Marrying dino with grounded pre-training for open-set object detection, 2024b. URL
05499
Chuofan Ma, Yi Jiang, Xin Wen, Zehuan Yuan, and Xiaojuan Qi. Codet: Co-occurrence guided region-word alignment for open- vocabulary object detection. Advances in neural information 2023.
processing systems,
Shiyuan Ma. Donglin Qian, Kai Ye, and Shengchuan Zhang. Cake: Category aware knowledge traction for open-vocabulary object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5982-5990, 2025a.
Shiyuan Ma. Donglin Qian, Kai Ye, and Shengchuan Zhang. Cake: Category aware knowledge ex- traction for open-vocabulary object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5982-5990. 2025b.
Zongyang Ma, Guan Luo, Jin Gao, Liang Li, Yuxin Chen, Shaoru Wang, Congxuan Zhang, and Weiming Hu. Open-vocabulary one-stage detection with hierarchical visual-languag knowledge distillation, 2022a. URL htt.t::://axxv..o/////////
10593
Zongyang Ma, Guan Luo, Jin Gao, Liang Li, Yuxin Chen, Shaoru Wang, Congxuan Zhang, and Weiming Hu. Open-vocabulary one-stage detection with hierarchical visual-language knowledge distillation. In CVPR, 2022b.
Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Amab, Mostafa Dehghani. Zhuoran Shen, et al. Sim- ple open-vocabulary object detection with vision transformers. In ECCV, 2022.
Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open- vocabulay object detection. Advances in Neural Information Processing Systems, 36:72983-73007, 2023.
13

---

Published as a conference paper at ICLR 2026
Renjing Pei, Jianzhuang Liu, Weimian Li, Bin Shao, Songeen Xu, Peng Dai, Juwei Lu, and Youliang Distilling clip-based models with a student base for video-language retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
Yan. Clipping:
18983-18992, 2023.
Pham. Lp-ovod: Open-vocabulary object detection by linear probing. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 779-788, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021a. URL https:://axxtt
org/ wttgg//////////
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, volume 139, 202
Shafin Rahman, Salman Khan, and Nick Barnes. Transductive learning for zero-shot object detec- tion. In ICCV, 2019.
Hanoona Rasheed, Muhammad Maaz, Muhammad Uzair Khattak, Salman Khan, and Fahad Shah- baz Khan. Bridging the gap between object and image-level representations for apen-vocabulary detection, 2022a. URL httpsts:: www.iriiiivv..... org. abts///b////// 2207. 03482
Hanoona Rasheed, Muhammad Maaz, Muhammad Uzair Khattak, Salman Khan, and Fahad Shah- baz Khan. Bridging the gap between object and image-level representattin for open-vocabulary detection, 2022b. URL /ttp:tts////aaa..c////////////
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypemymed, image alt-text dataset for automatic image captioning. In ACL, 2018
Haoxuan Wang, Qingdong He, Jinlong Peng, Hao Yang, Mingmin Chi, and Yabiao Wang. Mamba- yolo-world: marrying yolo-world with mamba for open-vocabulary detection. In ICASSP 2025- 2025 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1-5. IEEE, 2025.
Jiong Wang, Huiming Zhang, Haiwen Hong, Xuan Jin, Yuan He, Hui Xue, and Zhou Zhao. Open- vocabulary object detection with an open corpus. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6759-6769, 2023a.
Luting Wang. Yi Liu, Penghui Du, Zihan Ding, Yue Liao, Qiaosong Qi, Biaolong Chen, and Si Liu. Object-aware distillation pyramid for ooen-vocayulary object detection, 2023b. URL https: arxiv. owg. 223/ 05892
Luting Wang. Yi Liu, Penghui Du, Zihan Ding, Yue Liao, Qiaosong Qi, Biaolong Chen, and Si Liu. Object-aware distillation pyramid for open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11
2023c.
Kan Wu, Houwen Peng, Zhenghong Zhou, Bin Xiao, Mengchen Liu, Lu Yuan, Hong Xuan, Michael Valenzuela, Xi Stephen Chen, Xinggang Wang, et al. Tinyclip: Clip distillation via affinity mim- icking and weight inheritance. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11970-219980 2023a.
Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. Aligning bag of regions for open-vocabulary object detection, 2023b. URL
abtp//aa..222.... 13996
Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. Aligning bag of regions for open-vocabulary object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15254-15264, 2023c.
Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Wentao Liu, and Chen Change Loy. Clim: Con- trastive eegguage-image mosaic for region representation. In Proceedings of the AAAI conference artificial intelligence, volume 38, pp. 61 17-6155, 2024.
14

---

Published as a conference paper at ICLR 2026
Xiaoshi Wu, Feng Zhu, Rui Zhao, and Hongsheng Li. Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre- mre-matchnng. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7031-7040, 2023d.
Han Xu, Jie Ren, Pengfei He, Shenglai Zeng, Yingqian Cui, Amy Liu. Hui Liu, and Jiliang Tang. On the generalization of training- based chatgpt detection methods, 2023.
Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng
Tao, and Tianyi Zhou. A survey knowledge distillation of large language models, 2024. URL https:
wtwgg//b///aaaa 13116
Caixia Yan, Xiaojun Chang, Minnan Luo, Huan Liu, Xiaoqin Zhang, and Qinghua Zheng. Semantics-guides contrastive network for zero-shot object detection. TPAMI, 2022.
Lewei Yao, Renjie Pi, Hang Xu, Wei Zhang, Zhenguo Li, and Tong Zhang. G-detkd: Towards general distillation framework for object detectors via contrastive and semantic-guided feature imitation, 2021. URL
www.oorg/////./..o.ooo.../...e// 2108 07482.
Lewei Yao, Jianhua Han, Youpeng Wen, Xiaodan Liang, Dan Xu, Wei Zhang, Zhenguo Li, Chunjing Xu, and Hang Xu. Detelip: Dictionary-enriched visual-concept paralleled pre-training for open- world detection, 2022a.
2209. 09407
Lewei Yao, Jianhua Han, Youpeng Wen, Xiaodan Liang. Dan Xu, Wei Zhang, Zhenguo Li, Chunjing Xu, and Hang Xu. Detclip: Dictionary-enricen visual-concept paralleled pre-training for open- world detection. In NeurlPS, 2022b.
Keren Ye, Mingda Zhang, Adriana Kovashka, Wei Li, Danfeng Qin, and Jesse Berent. Cap2det: Learning amplify weak caption supervision for object detection. In ICCV, 2019.
Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-vocabulary detr with conditional matching. European conference on computer vision, pp. 106-122. Springer, 2022.
Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. Open-vocabulary object detection using captions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2021a.
Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. Open-vocabulary object detection using captions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021b.
Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. Open-vocabulary object detection using captions. In CVPR, 2021c.
Ruizhe Zeng, Lu Zhang, Xu Yang, and Zhiyong Liu. Boosting open-vocabulay object detection by handling background samples. In International Conference Neural Information Processing, pp. 274-289. Springer, 2024.
Chuhan Zhang, Chaoyang Zhu, Pingcheng Dong, Long Chen, and Dong Zhang. Cyclic contrastive knowledge transfer for open- open-vocabulary object detection. arXiv preprint arXiv:2503. 11005, 2025.
Yanan Zhang, Jiangmeng Li, Lixiang Liu, and Wenwen Qiang. Rethinking misalignment in vision- language model adaptation from a causal perspective. 2024.
Shiyu Zhao, Zhixing Zhang, Samuel Schulter, Long Zhao, Vijay Kumar B. G, Anastasis Stathopou- los, Manmohan Chandraker, and Dimitris Metaxas. Exploiting unlabeled data with vision and lan- guage models for object detection, 2022a. URL
http:///axvvvv.. org abs 20..222222222
Shiyu Zhao, Zhixing Zhang, Samuel Schulter, Long Zhao, BG Vijay Kumar, Anastasis Stathopoulos, Manmohan Chandraker, and Dimitris N Metaxas. Exploiting unlabeled data with vision and language models for object detection. In European conference on computer vision, pp. 175-175. Springer, 2022b.
15

---

Published as a conference paper at ICLR 2026
Yiwu Zhong, Jianwei Yang, Pengchuan Zhang. Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region- basad language-iman -mage pretraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pp. 16793-16803, 2022.
Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting (wenty-thousane classes using image-level supervision. In European conference on computer vision, pp. 350-368. Springer, 2022a.
Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting twenty-thousand classes using image-level supervision, 2022b. URL httpp:://axiv..rrtt abs/2200 02605
Chaoyang Zhu and Long Chen. A survey open-vocabulary detection and segmentation: Past, present, and future, 2024. URL /tl.httcpss//arrri.orr/////////s 09220
Pengkai Zhu, Hanxiao Wang, and Venkatesh Saligrama. Zero shot detection. TCSVT, 30(4), 2020a.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: De- formable transformers for end-to-end object detection. arXiv preprint arX11:2010000455, 2020b.
Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. Object detection in 20 years: A survey, 2023. URL aatppa://aat //taxi//aiiv..//
16

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 USE OF LLM
We use LLM aid polish writing. Details are described in the paper.
A.2 ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. Our study does not involve human subjects, per- sonal or sensitive data. All datasets used in this paper (e.g.. COCO, LVIS) are publicly available and widely adopted in the research community, and we strictly follow their licenses and intended usage. The proposed DeCo-DETR framework is designed for academic exploration of open-vocabulary ob- ject detection. Potential misuse model in safety-critical or surveillance scenarios outside scope of this research, and strongly encourage responsible and ethical use in line with research integrity principles.
A.3 REPRODUCIBILITY STATEMENT
We make every effort to ensure the reproducibility of our results. Full training details, including model architectures, hyperparameters, and optimization schedules, are provided in the main paper and appendix. The experimental settings cover key modules such as Dynamic Hierarchical Con- cept Pool construction, Hierarchical Knowledge Distillation, and Parametric Decoupling Training, with clear descriptions of dataset preprocessing and evaluation protocols. Our implementation is based PyTTrrh and standard detection frameworks. To facilitate replication, will release the source code, configuration files, and pre-trained models upon publication. All reported results can be reproduced using the provided settings and supplementary material.
A.4 METHIODOLOGY DETAILS
The student prototypes A and teacher prototypes P share the same index dimension M because they are derived from the same set of multi-modal clusters in the joint CLIP embedding space. This correspondence established follows:
Joint Clustering:
We perform clustering on the aligned pairs of region visual features and text embeddings (filtered by CLIP). This partitions the data into M clusters, where each cluster j E MM represents a specific shared semantic concept.
2. Definition of Prototypes: For each cluster j, the Teacher Prototype Pj is defined as the centroid of the text embeddings in that cluster, while the Student Prototype A₁ is initialized as the centroid of the visual embeddings in the same cluster.
Alignment Mechanism: Since Pj and Aj originate from same multi-modal cluster j. they naturally paired. The distillation loss aligns the student's 's distribution (calculated via A) with the teacher's 's distribution (calculated via P), ensuring the student learns the corresponding semantic structure.
A.5 OVD BENCHMARKS
According the training data, existing Open-Vocabulat Object Detection (OVD) methods summarized into four types of benchmarks: Vanilla OVD (V-OVD), Caption-based OVD (C-OVD), Generalized OVD (G-OVD), and Weakly Supervised OVD (WS-OVD). All benchmarks rely instance-level annotations and large-scale image-text pairs to learn OVD.
For clarity, base categories defined those included in the instance-level annotations, while novel categories are the others.
A.5.1
VANILLA OVD (V-OVD)
V-ODD
Cai et al.
(002a); Du et al. (2022) Kamath et al. (202T): Li et al. (2022); Minderer et al.
2022
Yao al. (002bh Zhong al. (2022 pure OVD benchmark setting. It requires the
17

---

Published as a conference paper at ICLR 2026
Table 5: Ablation study the number of Decoder Queries (N) and Fine-grained Prototypes (M2).
The default setting is highlighted in bold.
Configuration Queries (N)
Fine Units (M2)
AP novel
base
Latency (ms)
DeCo-DETR
300
4800
36.5
53.8
125
DeCo-DETR
1000
4800
39.1
54.5
130
DeCo-DETR
2000
4800
41.3
55.5
135
DeCo-DETR
2000
(Coarse only)
30.8
54.1
131
DeCo-DETR
2000
9600 (Double)
41.5
55.6
142
detector to train only on an object detection dataset with fixed set of categories. Information about novel categories is unavailable, but unannotated data is allowed. A common practice for this benchmark to learn open vocabulary knowledge from image-text pairs and transfer the knowledge to detectors through transfer learning knowledge distillation. V-OVD is similar to Zero-Shot Detection (ZSD) Bansal et al. (2018a); |aahman et al.] (2019); Yan et al. (2022); Zhu et al. (2020a), except that V-OVD relies large-scale image-text pairs acquire open- vocabulary knowledge.
A.5.2 CAPTION-BASEI OVD (C-OVD)
C-OVD Bravo et al. 20222 Gao et al. (2022); Ma et al. (202bb): Zareian et al. (2021c) adds addi- tional image caption annotations to the V-OVD benchmark. This refers in-domain captions the instance-level annotations (e.g.. COCO-Captions Chen et al. (2015)) rather than large-scale image- text pairs like CC3M Sharma et al. (2018) or CLIP400M et al. (0021b), In-domain captions enrich annotations and imply distribution of potential novel categories. C-OVD is expected to perform better than V-OVD due to slightly more annotations.
A.5.3 GENERALIZED OVD (G-OVD)
G-OVD al. 2002a Zang et (2022) Zhao al. (2022b) introduces human priors on novel categories to the V-OVD benchmark. It assumes that if specific novel categories are likely to appear during inference, it is beneficial prepare for them during training. Most existing methods assume all dataset category names (including novel ones) are known during training. A typical solution involves generating instance-level pseudo annotations.
A.5.4 WEAKLY SUPERVISED OVD (WS-OVD)
WS-OVD Zhou et al. 2022a utilizes image-level category labels beyond G-OVD, Similar to Weakly Supervised Detection (WSD) [Bllen & Vedaldi (2016): Ye et al. (2019), image-level labels reflect the presence of base and novel categories. The annotation cost is significantly higher than the benchmarks mentioned above, giving WS-OVD methods the greatest potential to push the limits of OVD.
Table 6: Summary of OVD benchmarks. "Caption": in-domain captions like COCO-Captions..
"Category Prior": human priors on novel categories. "Image Label": image-level category labels.
Benchmark
Caption
Category Prior Image Label
V-OVD
C-OVD
G-OVD
WS-OVD
V
A.6 ABLATION STUDY
This section contains some tables of ablation experiments.
18

---

Published as a conference paper at ICLR 2026
Table Comparison of inference efficiency and open-vocabulary detection performance COCO. DeCo-DETR achieves the best trade-off between accuracy and speed.
Method
Backhone
AP novel
Text Enc.
Latency (ms)
FPS
Fusion-based:
Grounding DINO-T
Swin-T ResNet-50
42.1
BERT-Base RoBERTa
~28
3.5
VL-PLM
32.3
~210
4.7
Distillaton/Deccoppee::
DetPro
ResNet-50
29.4
250
4.0
CAKE
ResNet-50
38.2
145
6.9
DeCo-DETR (Ours)
ResNet-50
41.3
135
7.4
Table 8: Comparison performance of different models on the OV-COCO dataset
Model
APbase A
50
1L. 7B
30.1
52.1
LLaVA-1.5 13B(Ours)
38.2
55.5
LLaVA-NEXT 7B
32.1
53.3
LLaVA-NEXT 13B
38.6
55.8
Qwen2.5-VL 7B
33.1
53.9
Qwen2.5-VL 32B
38.9
55.9
A.7 LIMITATIONS
Although DeCo-DETR achieves state-of-the-ie performance in open-vocabulary object detection, several limitations remain. First, the construction the Dynamic Hierarchical Concept Pool (DHCP) relies large vision-language models such LLaVA and CLIP, which may hinder de- ployment resource-constranl
environments. Second, despite mitigating task conflicts via para- metric decoupling training, the model's generalization ability on extreme long-tailed distributions or fine-grained categories with high similarity still requires further improvement. Additionally, the current method primarily designed for static image detection and has not yet been extended to real-time open-vocabulary detection in video sequences or dynamic scenarios.
A.8 SOCIAL IMPACT
DeCo-DETR has broad application potential in autonomous driving, human-computer interaction, and intelligent security systems. By enhancing the model's ability to recognize unseen categories, can improve the adaptability and safety of intelligent systems in open-world environments. How- ever, also recognize that efficient object detection technology could be misused for privacy in- fringement or large-scale surveillance. Therefore, we encourage the research community to adhere ethical guidelines, ensure that applications align with social responsibility and legal standards, and promote the development of transparent, trustworthy, and controllable Al systems.
19


Reviews:
[{"review_id": "kslM9od73j", "reviewer": "Reviewer_Ttz9", "summary": "This paper proposes DeCo-DETR, an open-vocabulary object detector that removes dependence on text encoders and improves both precision and generalization. It introduces a three-stage cognitive distillation mechanism—a dynamic concept pool from LLaVA-CLIP filtering, hierarchical knowledge distillation for decoupled visual-semantic mapping, and parametric dual-stream training for coordinated localization and recognition.", "questions": "1.\tThe manuscript implicitly assumes the student prototypes A and teacher prototypes P share a one-to-one correspondence via the same M, but it is unclear how this mapping is defined or established, since A results from unsupervised clustering and P is text-derived.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThe paper introduces a three-stage cognitive distillation framework (DHCP, Hi-Know DPA, PD-DuGi) that provides a conceptually coherent and interpretable alternative to conventional multimodal fusion.\n2.\tThe model achieves improvements in both detection accuracy and inference efficiency, effectively reducing computational cost while maintaining strong open-vocabulary generalization.", "weaknesses": "1.\tMain weakness – Metric inconsistency (Table 1). The reported AP50 values are higher than both APNovel50 and APBase50, which violates standard OVOD evaluation logic. Since AP50 includes both base and novel categories, its score should theoretically lie between them. This inconsistency casts doubt on the correctness of the evaluation protocol or result reporting, and substantially weakens the empirical credibility of the paper’s main claims.\n2.\tIncomplete ablation (Table 4). Table 4 is missing key variants and does not include an ablation isolating the proposed Dual-stream Gradient Isolation Mechanism, leaving its effectiveness unverified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "76bhpcH0kv", "reviewer": "Reviewer_pDPC", "summary": "This manuscript targets open-vocabulary object detection (OVOD) and proposes DeCo-DETR, a three-stage decoupled cognition pipeline.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "[1] DHCP: a dynamic, hierarchical concept pool that generates region descriptions with LLaVA and filters them with CLIP to build semantic prototypes (thus removing the text encoder at test time); \n\n[2] Hi-Know DPA: hierarchical knowledge distillation that projects decoder queries into the prototype space for prototype attention/alignment; \n\n[3] Parametric Decoupling Training: a dual-stream gradient isolation scheme that routes localization and semantic-alignment gradients separately. \n\n[4] DeCo-DETR reports strong gains (+3.5 to +7.2 AP on novel classes) while keeping inference at 135 ms/image.", "weaknesses": "[1 ]Missing scale ablation on queries and prototypes：There is no systematic ablation for the number of decoder queries N=2000and the total number of prototypes M=M_1+M_2(with M_1=1203 coarse anchors and M_2=4800 fine units). In DETR-style models, increasing the number of queries and prototypes generally improves accuracy, but drives memory usage up linearly;\n\n[2] Fairness of the efficiency comparison (Table 3)：One of the paper’s main claims is removing multimodal computation at test time. For fairness, Table 3 should include methods with similar accuracy and comparable settings. As it stands, Table 3 under-represents strong multimodal fusion baselines near DeCo-DETR accuracy, making the efficiency story less convincing;\n\n[3] Isolating the independent contribution of Parametric Decoupling Training：Section 4.4 shows that the cosine annealing weight adds ~+1.6 AP_50, but this does not quantify the benefit of Parametric Decoupling Training itself.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VUQacWJOrD", "reviewer": "Reviewer_G9cE", "summary": "This paper proposes DeCo-DETR, a framework for open-vocabulary object detection that aims to eliminate text encoder dependency during inference while improving generalization. The approach introduces three main components: (1) Dynamic Hierarchical Concept Pool (DHCP) that constructs visual prototypes using LLaVA-generated descriptions filtered by CLIP, (2) Hierarchical Knowledge Distillation (Hi-Know DPA) for visual-semantic alignment, and (3) Parametric Decoupling Training (PD-DuGi) with gradient isolation. The method achieves competitive results on OV-COCO and OV-LVIS benchmarks with low inference latency.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel approach to eliminating text encoder dependency. The Dynamic Hierarchical Concept Pool is an interesting idea that pre-computes and maintains visual prototypes, eliminating the need for text encoders at inference time. \n- The results show strong empirical performance. The experiments demonstrates consistent improvements across multiple benchmarks and settings, achieving state-of-the-art result while maintaining reasonable computational cost.", "weaknesses": "- There exists several misleading claims. 1) The paper claims to eliminate \"multimodal fusion\" but DHCP construction still heavily relies on LLaVA and CLIP. 2) The framework is only vision-only at inference, not overall.\n- Lack in-depth ablations. The ablation study in table 4 only compares 2 configurations, not isolating individual component contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K8T3j076r5", "reviewer": "Reviewer_7QW3", "summary": "This article introduces DeCo-DETR (Decoupled Cognition DETR), a novel open-vocabulary object detection (OVOD) framework designed to overcome the limitations of existing methods, particularly high computational overhead from text encoders and a trade-off between detection precision and open-world generalization.", "questions": "1. In Section 1, Paragraph 1, the authors assert that “the emergence of large language models (LLMs) has significantly enhanced detector generalization by providing richer and more nuanced semantic supervision,” yet no supporting citation is provided. Similarly, the authors claim that existing distillation methods face latency and generalization trade-offs, again without citing any references. It is recommended that relevant citations or quantitative results be added to substantiate these claims.\n\n2. In Section 1, Paragraph 2, the authors state that multimodal fusion designs inherently involve compromises, but they provide no explanation, citation, or quantitative evidence to support this claim. As a result, the argument lacks persuasiveness.\n\n3. In Section 1, Paragraph 3, the DHCP module is introduced without clarifying how it addresses the challenge outlined in the preceding paragraph. Moreover, the mention of “momentum updates with attention weighting” is confusing, and the authors should elaborate on its relationship with DHCP.\n\n4. A similar issue arises in Section 1, Paragraph 4, where the connection between the proposed design and the previously mentioned challenges is not clearly articulated.\n\n5. Due to the issues raised in points 1–4, the first contribution claim in Section 1—“We reveal two critical flaws in existing open-vocabulary detection”—does not appear sufficiently supported.\n\n6. In the second contribution statement in Section 1, the authors mention “dynamic concept pooling” and “hierarchical distillation and parametric isolation mechanisms” as solutions to the identified challenges. However, these terms appear only in this section and are likely referring to “DHCP” and “Hi-Know PDA” introduced later in the manuscript. The current wording is ambiguous and may cause confusion.\n\n7. In Figure 1, the bottom-left subfigure seems unnecessary. Additionally, the caption indicates that Hi-Know PDA is part of the framework diagram, but it is not visually represented.\n\n8. Several issues are present in Section 3. For instance, the “spectral clustering-based hierarchical compression algorithm” is mentioned for the first time in Section 3.1, yet its specific operation is not explained, nor is it illustrated in any figure or pseudocode.\n\n9. Certain references are missing throughout the manuscript. For example, LLaVA and DBSCAN are not cited.\n\n10. The results in Table 3 are unclear. DeCo-DETR does not appear to be the most efficient method according to the table. Moreover, comparing it with Deformable-DETR is confusing, as Deformable-DETR is not an open-vocabulary object detection (OVOD) method nor serves as the baseline for DeCo-DETR.\n\n11. In Section 4.3, Table 2, the benchmarks V-OVD, G-OVD, C-OVD, and WS-OVD are neither introduced nor cited. While these may originate from OVCOCO (Bansal et al., 2018), they are not referenced in the paper.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper identifies certain issues in current large model-based OVD models, proposes a viable approach to address them, and conducts comprehensive experiments that can serve as a valuable reference for future research.", "weaknesses": "The description on issues and challenges lacks persuasiveness, and the connection between the proposed method and the challenges it aims to address is weak. Furthermore, the explanation of the methodology lacks clarity, and the presentation of figures, tables, as well as certain parts of the writing, appears somewhat casual. Overall, the rigor of the work needs to be enhanced.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)