Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper propose a novel method for knowledge distillation on object detection by measuring the correlation between each object and human observed instances. They have introduced a conditional modeling paradigm to find the crucial regions for distillation and apply the auxiliary tasks to supervise its training. Good experiment results have been achieved on COCO.
main_review: Originality. The ideas that applying KD on object detection and find the important regions for KD are not new. But the ideas that applying transformer decoder and conditional computation to find the KD regions are very interesting and novel.

Quality. This paper has sufficient experiments and ablations to show the effectiveness of their methods. Although there is no theoretical proof about their methods, the visualization results provided in the paper show that they can find better KD regions in object detection. I think it's a new technical sound in this domain.

Clarity. This paper is easy to follow but some details need to be improved.

Significance. They have outperform the existing object detection KD methods by a clear margin in Table.1. And good results have been achieved on the state-of-the-art detectors such as SOLOv2 and CondInst and efficient detectors such as EfficientNet-B0.


limitations_and_societal_impact: 1. All the experiments are conducted on COCO, it will be better with experiments on more datasets such as Cityscpaes.
2. In table 2, it will be better to report the inference time and parameters of each model.
3. Figure 1 "Previous studies for object detection distill discriminate regions by sampling proposals or center areas". This is not correct. There have been some methods which find the regions without proposals such as [47].
4. Figure 2 needs to be explained better.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a target-oriented solution towards knowledge distillation for object detection. Unlike prior works which focus on distilling the global representations or foreground-region representations, the paper aims to find relevant regions for distillation based on the instance-conditional information. The auxiliary task optimization helps in finding the related knowledge. The paper performs extensive experiments on MS-COCO to show the efficacy of their method.
main_review: - The idea of instance-aware knowledge distillation is novel.

- The paper achieves respectable performance improvement over the previous approaches even beats the teacher in
some cases.

- In Table 1, the backbone of the teacher network used is Resnet-101. For a fair comparison with [47],  the backbone
that should be used is Cascade Mask RCNN with ResNetXt101. [47] obtains 41.5% mAP on Resnet-50 Faster-RCNN student in this setting. Similar setting for RetinaNet would be appreciated. The method beats Wang et al. [42] with a good margin though.

- Although the paper argues that the cost associated with the additional training is negligible, the concatenation of the feature maps across different scales (L) seems to add more parameters for training.

- The method seems to depend on FPN. Is that the case ? This will limit the usage of the method significantly.

- In Figure 4, the learned attention over the heads is degenerate for some heads (v). Better techniques for learning
attention over each head should be explored.

- Correction: Line no. 229, I think it should be “a very recent work with strong performance from Zhang et al. [47]”.
limitations_and_societal_impact: The authors have addressed the limitations for their method in the discussion section. 

1. Fail cases in the form of learning some degenerate heads.

2. Learning from a teacher can be costly.

3. Student might inherit some bias from the teacher.

However, Points 2 and 3 are the limitations of the Knowledge Distillation setup in general, not particular to this method.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This submission proposes a novel knowledge distillation approach for object detection. The authors attempt to use an auxiliary task with the key-query (transformer-liked) method to determine where and how the feature map can be transferred from a teacher to a student. In the paper, the target instances are encoded as queries to determine the knowledge distilled from the teacher. Extensive experiments show the effectiveness of the proposed approach.  
main_review: Strength:

+ Novel idea. The authors propose to make use of the transformer-liked method to generate the attention mask based on real instances such that the approach can locate the informative knowledge to guide the student network's training.
+ Promising results and convincing ablation study.

Weakness:

- The English writing of this paper should be improved. Several typos in the paper, such as 'wildly' in line 99, 'Direc' in line 201. Please check the paper carefully.
- Figures 1, 2, and 3 are hard to follow. Please add related notations along with the text and some sub captions to refine them.
- I believe that generating fake instances is crucial in the method, how and why 5? It would be better to discuss more on this part.
- The inheriting parameters strategy, which improves the performance significantly, can only be used in the scenario where the student and teacher share the same architecture in detection head and FPN. It is more or less a big limitation in the knowledge distillation domain where the student and teacher may vary a lot in the network architecture.
limitations_and_societal_impact: The proposed distillation method only explores the situation where teacher and student have the same framework but different backbones, such that from Resnet101-based teachers to ResNet50-based students. The inheriting parameters strategy is not possible to use if the teacher and student have different detection heads and FPNs.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new method named Instance-Conditional knowledge Distillation (ICD) to deal with the knowledge distillation problem in image object detection task. Motivated by the fact the traditional classification-level distillation methods cannot focus on the locations of objects, they insert additional instance information to the distillation training together with auxiliary tasks, i.e. recognition and localization. Their proposed distillation framework improves the performance of the student network, and even outperforms the teacher network under some specific settings.
main_review: **Originality**: This method is original in this area, although it combines several well-known techniques. Related works are also well-summarized and discussed. 

**Quality**: With my experience, this method is technically correct. The motivation of is clear and straightforward. Their experiment supports their statements in their methodology. There are two things that might improve the quality of this paper:

(1)	The analysis of why the student network outperforms the teacher network. Is it possible that the teacher network is not well-trained, and the new modules and tasks added in the KD framework make up for the previous training deficiencies?

(2)	The analysis of the hyper-parameter $\lambda$ in equation (12).

**Clarity**: In general, the paper is well-organized and easy to follow. However, I have some suggestions about the figures:

(1)	In Figure 1, the last column uses ”object-oriented“ as its title, but this term does not appear in the introduction part. I only find ”task-oriented“ described in the context. I am not sure if the last column represents their proposed method. 

(2)	In Figure 1, there is no description of the blue and gray blocks. I assume the blue one is the student network and the gray one is the teacher network.

(3)	In Figure 2, there are a lot of different kinds of arrows and rectangles. Without any descriptions, it is really hard to understand the details of the proposed method.

**Significance**: The significance and novelty are the main reason for my current rating. Although their experiment indeed shows improvement compared with the 4 previous methods, the contribution of the proposed method is still limited. If I understand the method correctly, their improvement mainly comes from the auxiliary tasks of instance information, which is not a fundamental modification.

limitations_and_societal_impact: As the authors mention in Section 5, one limitation of their method is that the student might inherit biases from the teacher which could be hard to identify. Another limitation I can imagine is the access to instance information. What if we cannot access the training label? Where can we get the instance information?
review_title: null

Meta-review: Although the paper originally received slightly mixed ratings, with one reviewer recommending rejection and 3 acceptance, the authors' feedback eventually convinced the most negative reviewer to update their score. Altogether, the reviewers acknowledge the novelty of the proposed method and that the empirical results are convincing. The authors' feedback also provided additional results and clarifications, which we strongly encourage the authors to incorporate in the final version.
Decision: Accept (Poster)