Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The work proposes to build a concept dictionary which enables to perform visual-concept pretraining in a unified way across multiple heterogeneous datasets for open-world detection. With the proposed dictionary design, the proposed method can even perform partial annotation enrichment for label completion and negative concept sampling to further improve the performance.
strengths_and_weaknesses: Strength:

1. The built concept dictionary can be used to convert the annotations of heterogeneous datasets, including detection, visual grounding, etc, into a unified one.
2. With the concept dictionary, the proposed method can perform concept enrichment with the assistance of other pretrained text encoder, FILIP in addition to negative concept sampling and partial annotation enrichment for those labels which are not well annotated in the existing datasets.
3. The performance improvement in LVIS for rare and frequent categories is significant as compared with other previous state-of-the-art method.

Weakness:

1. The construction of the concept dictionaries rely on the wordNet, and thus  the concepts are mainly composed of those which are well-defined in WordNet. 
2. For the concept enrichment, it still requires a pre-trained text encoder, like FILIP, to match related concepts. The quality will depends on the reliability of the pretrained text-encoder used.
3. When resolving the name ambiguity among different datasets, only the text part is used without exploiting the corresponding image information.
questions: The experimental results are very thorough covering different open-vocabulary datasets, and also show promising performance improvement. I have few questions.

1. Is the definition from the wordNet necessary to be used? If we only use the unified label, how much will it affect the performance?
The following work is not targeted at open-vocabulary detection, but it also proposes to perform contrastive learning using the unified space.

Yang, Jianwei, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, and Jianfeng Gao. "Unified contrastive learning in image-text-label space." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19163-19173. 2022.


2. How is the pseudo labeling for label completion of using the all the concepts in the dictionary conducted? The dictionary size is usually very large (e.g., 14k). The paper describes to add the concepts in the dictionary as the addition category inputs. 
3. For concept enrichment and partial concept enrichment, how would the pretrained encoder FILIP affect the final performance? Do the authors try other text-encoders?
4. How is the proposed method compared with recently released work, GLIPv2 (optional)?
https://arxiv.org/abs/2206.05836
limitations: The authors do address the limitations and why there is no potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper explores a parallel visual-concept pretraining method for open vocabulary object detection by resorting to knowledge enrichment from a designed concept dictionary.
They make the following contributions:
1.a novel paralleled concept formulation to improve learning efficiency.
2.a novel concept dictionary to enrich the prompt text concepts during joint pretraining.
3.DetCLIP outperforms the state-of-the-art GLIP.
strengths_and_weaknesses: Strengths
1.  The method is simple and effective and I like the examples author offered here and there.
2.  The model significantly improves the learning efficiency.
3.  The experiment results seem solid. They evaluate the method on LVIS and transfer the model to 13 downstream detection datasets. 

Weaknesses
I think the writing of the abstract and introduction needs to be significantly improved: they are too long especially for the abstract. And also many concepts were used directly without explanation/definitions/references at the first hand or not at all. For instance, what is GLIP, DetCLIP-T, GLIP-T, positive samples v.s. negative samples ? What is the grounding, what's its difference to image-text pair data ? what's the partial label problem ?   

Authors claimed their method is much more efficient and effective than GLIP in terms of data usage. However I notice that they have also utilized other models such as CLIP , FILIP. Are these models considered as part of the proposed method (are they multi-stage) ? How would this compare to GLIP ? 

L157 Can you provide further explanation that why interaction between categories is unnecessary?

L169: Why context information is just removed. I know the context cannot fed into the text encoder individually. Maybe it can be added to every individual concept phrase. Any study for this  ?

L243 : What is partial positive concept ?  Is it derived from the partial labelling problem, I don't see a clue.  

Last, bold text is normally for title/subtitle as a standard practice. Be using it properly!  

questions: See above 
limitations: See above
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper is developed based on GLIP:

- To solve the problems of inefficient interaction between categories and the restriction of max input sentence length, it proposes a parallel method. Compared to the sequential input in GLIP, the paper extracts phrases as the input in a parallel manner.
- To build implicit relationships between input phrases, it proposes a concept dictionary to provide the prior knowledge.

The paper is verified in the task of zero-shot detection on LVIS.

strengths_and_weaknesses: Strengths:
- The framework does make sense. I agree with the point that extracting the noun phrase is efficient for alignment between regions and concept words.
- The proposed large-scale concept dictionary could be a future valuable toolkit for future open-vocabulary or vision-language research if it can be open-released. 
- Compared to GLIP, improvements are large.

questions: - The notations in Formula (1) are odd due to the upper letter T and Transpose. Please consider changing it.
- Experiments mainly use Swin-Transformer as the backbone. Can the backbone be ViT?
limitations: None.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes an extension of GLIP for Open World Detection. The extension comprises two parts: one methodological by introducing a so-called parallel processing of the text data, and one more data related that allows the unification of many heterogeneous datasets for training. The latter also proposes a richer description of the categories -- deemed concepts in the paper -- through wordnet.
The paper reports significant improvements of GLIP. It is mainly evaluated on LVIS dataset
strengths_and_weaknesses: On the positive side:
- the authors study an important problem in literature that of open-world object detection where not many solutions are available
- the parallel extension to GLIP, although straightforward, makes sense. The original GLIP sequentual formulation was problematic and did not scale well with a large number of categories
- the authors did a good job in terms of proposing a framework for unifying many established and heterogeneous datasets including standard detection dataset, grounding data, and image/caption pairs. Actually, in my opinion, this is the biggest contribution of the paper. 
- the improvements over GLIP are significant. Even when trained on the same data, +5.1% is reported (table 3).
questions: On the negative side:
- the methodological contribution of the parallel processing is rather thin and is actually the first thing that would come to mind when reading the original GLIP paper.
- most of the improvements and contributions come from the dataset side. Although the framework for creating the dataset is worthy, it is more an engineering approach rather than an actual methodological contribution. After reading the paper you get mostly the impression that the authors created a much better dataset and then trained a (straightforward) extension of GLIP on it.
- The evaluation on LVIS is sound and convincing. However with such big and heterogeneous data used, the zero shot capabilities of the method are not clear. Can the authors also evaluate following the VILD protocol (https://arxiv.org/pdf/2104.13921.pdf) where the method is trained on base classes and then explicitly evaluated on unseen classes?

**Final Recommendation**
As said I will stick to my original score which is line with the score from other Rs.
limitations: I didn't see a paragraph for this in the paper.
review_title: null

Meta-review: metareview: The paper receives overall positive reviews and rebuttal has resolved the reviewer's concerns. Reviewers agree that the paper proposes a simple yet effective approach to enrich language concepts to learn better region-concept alignment for object detection. The approach is supported by solid empirical evidence on the LVIS dataset and 13 DATA. AC agrees the methodology of processing data is worth sharing to a broader audience and recommends to accept the paper.
confidence: Certain
Decision: Accept