Distilling Object Detectors with Feature Richness
Authors: Zhixing Du
Rui Zhang
Ming Chang
Xishan Zhang
Shaoli Liu
Tianshi Chen
Yunji Chen
Venue: NeurIPS 2021 Poster (2021)
Keywords: Knowledge Distillation； Object Detection

Abstract: In recent years, large-scale deep models have achieved great success, but the huge computational complexity and massive storage requirements make it a great challenge to deploy them in resource-limited devices. As a model compression and acceleration method, knowledge distillation effectively improves the performance of small models by transferring the dark knowledge from the teacher detector. However, most of the existing distillation-based detection methods mainly imitating features near bounding boxes, which suffer from two limitations. First, they ignore the beneficial features outside the bounding boxes. Second, these methods imitate some features which are mistakenly regarded as the background by the teacher detector. To address the above issues, we propose a novel Feature-Richness Score (FRS) method to choose important features that improve generalized detectability during distilling. The proposed method effectively retrieves the important features outside the bounding boxes and removes the detrimental features within the bounding boxes. Extensive experiments show that our methods achieve excellent performance on both anchor-based and anchor-free detectors. For example, RetinaNet with ResNet-50 achieves 39.7% in mAP on the COCO2017 dataset, which even surpasses the ResNet-101 based teacher detector 38.9% by 0.8%. Our implementation is available at https://github.com/duzhixing/FRS.

Distilling Object Detectors with Feature Richness
Zhixing Du 1,2,3
Rui Zhang 23 Ming Chang
3
Xishan Zhang 233
Shaoli Liu
Tianshi Chen
Yunji Chen 2,4
University of Science and Technology of China
SKL of Computer Architecture, Institute of Computing Technology, CAS, Beijing, China Cambbrico Technologies, China
University of Chinese Academy of Sciences, China dzx1 @mail www..uttc.... edu.cn
wha.ggisiiiaaagiissaanan wwcetiittt......e..eeeeeeeeeeeee ac cn {changming, Baatnng,iuue liushaoli, tcama com
Abstract
In recent years, large-scale deep models have achieved great success, but the huge computational complexity and massive storage requirements make it a great chal- lenge deploy them in lesttrce-limited devices. As a model compression and acceleration method, knowledge distillation effectively improves the performance of small models by transferring the dark knowledge from the teacher detector. How- ever, most of the existing distillation-basee detection methods mainly imitating features near bounding boxes, which suffer from two limitations. First, they ignore the beneficial features outside the bounding boxes. Second, these methods imitate some features which are mistakenly regarded as the background by the teacher detector. To address the above issues, propose novel Ceature-Richnese Score (FRS) method to choose important features that improve generalized detectability during distilling. The proposed method effectively retrieves the important fea- tures outside the bounding boxes and removes the detrimental features within the bounding boxes. Extensive experiments show that our methods achieve excel- lent performance on both anchor-based and anchor-free detectors. For example, RetinaNet with ResNet-50 achieves 39.7% in mAP the COC02017 dataset, which even surpasses the ResNet-101 101 based teacher detector 38.9% by 0.8%. Our implementation is available https://gitubb.com/duuziiiii
1 Introduction
Owe to the widespread use of deep learning, object detection method have developed relatively rapidly. Large-scale deep models have achieved overwhelming success, but the huge computational complexity and storage requirements limit their deployment real-time applications, such as video surveillance, autonomous vehicles. Therefore, how to find better balance between accuracy and efficiency has become a key issue. Knowledge Distillation 12 promising solution for the above problem. is model compression and acceleration method that effectively improve the performance of small models under the guidance of the teacher model.
For object detection, distilling detectors through imitating all features is inefficient, because the object- irrelevant area would unavoidably introduce lot of noises. Therefore, how to choose the important features beneficial to distillation remains unsolved issue. Most of the previous distillation based detection methods mainly imitate features that overlap with bounding boxes (i.e. ground truth objects), because they believe the features of foreground which can be selected from bounding boxes are
"Correspondngg author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

(a) Images and bounding boxes (b) Bounding box based method
(c) Ours
Figure 1: Visualization of feature masks used in distillation of bounding box based method and ours.
important. However, these methods suffer from two limitations. First, foreground features selected from bounding boxes only contain categories in the dataset, but ignore the categories of objects outside the dataset, which leads to the omission of some important features, as shown in Figure [ (b)-bottom. For example, the mannequin category is not included in the COCO dataset, but the person category is included. Since mannequins are visually similar to persons, the features of mannequins contain many useful characteristics of persons which are beneficial for improving the detectability of person for detectors in distillation. Second, only using the prior knowledge of bounding boxes to select features for distillation ignores the defects of the teacher detector. Imitating features that are mistakenly regarded as the background by the teacher detector will mislead and damage the distillation, as shown Figure (b)-top.
To handle the above problem, we propose a novel Feature- Score (FRS) method to choose important features that beneficial to distillation. Feature richness refers the amount of object information contained in the features, meanwhile it can be represented by the probability that these features are objects. Distilling features that have high feature richness instead of features in bounding boxes can effectively solve the above two limitations. First, features of objects whose categories included in the dataset have high feature richness. Thus, using feature richness can retrieve the important features outside the bounding boxes, which can guide the student detector to learn the generalized detectability of the teacher detector. For example, features of mannequins that have high feature richness can promote student detector to improve its generalized detectability of persons, as -cbbotomm. Second, features in the bounding boxes but are misclassified with teacher detector have low feature richness. Thus, using feature richness can remove the misleading features of the teacher detector in the bounding boxes, as shown in Figure (c)-top. Consequently, the importance of features is strongly correlated with the feature richness, namely feature richness is appropriate to choose important features for distillation. Since the classification score aggregating of all categories is an approximation of probability that the features are objects, we use the aggregated classification score as the criterion for feature richness. In practice, we utilize the aggregated classification score corresponding to each FPN level in teacher detector as the feature mask which is used as feature richness map to guide student detector, in both the FPN feature and the subsequent classification head.
shown in Figure I
Compared with the previous methods which use prior knowledge of bounding box information, our method uses aggregated classification score of the feature map the mask of feature richness, which related to the objects and teacher detector. Our method offers the following advantages. First of all, the mask in our method is pixel-wise and fine inn-ggrined, so we can distill the student detector in a more refined approach to promote effective features and suppress the influence of ineffective features of teacher detector. Besides, our method is more suitable for detector with the FPN module, because our method can generate corresponding feature richness masks for each FPN layer of student detector based on the features extracted from each FPN layer of teacher detector. Finally, our method

---

is a plug-and -nd- block to any architecture. We implement our approach in multiple popular object detection frameworks, including one-stage, two-stage methods and anchor- free methods.
To demonstrate the advantages of the proposed FRS, we evaluate it on the COCO dataset on various framework: Faster-RCNN [24], Retinanet [18) GFL. 17 and FCOS (27]. With FRS, we have out- performed state-of-the tatt-- methods on all distillation-base detection frameworks. This achievement shows that the proposed FRS effectively chooses the important features extracted from the teacher.
2
Related Works
2.1 Object Detection
In recent years, noticeable improvements in aceuracy have been made in object detection. The current mainstream object detection algorithms are mainly divided into two-stage detectors [24 2 9
[8] 23 Two-stage methods such as FasterRCNN [24] region proposal network and refinement procedure of classification and location. Although detectors fitted with very deep backbone have better detection accuracy, they are expensive in terms of computation cost and hard to deploy to real-time applications. While one-stage methods, such Retinanet [18] and FCOS (27), maintain an advantage in speed that can directly generate the category probability and position coordinate value of the object. In object detection, the most anchor- models [18 24 have achieved good results by using predefined anchors, but the predefined anchors will bring a large amount of output and adjustments to the positions and sizes corresponding to different anchors. The anchor-free free methods [27] 14 6 is lighter, getting rid of redundant Syper-parameters by detecting an object bounding box as a pair of key points such as center and distance to boundaries. Although large-scale deep neural networks have achieved great success, the huge computational complexity and storage requirements make it a great challenge deploy in real-time applications, especially on video surveillance and other resource-limited devices. The model compression and acceleration techniques, such as quantized 34 and knowledge distillation [12] become an important way to solve this problem.
and one-stage detectors [22 17
27
2.2 Knowledge Distillation
Knowledge distillation is an effective method of model compression, which can improve the perfor-
mance of small models. Hinton et al. [12 proposed to transfer the dark knowledge from teacher network to student network through the soft outputs which opens the door the rapid development of knowledge distillation. There are three categories of knowledge for knowledge distillation (7). Response- Based methods use the logits of large deep model as teacher knowledge [12 [3 21 33).
Feature-Basse methods think that the features of intermediate layers also play an important role in guiding the learning of the student detector 25 11 And Relation- Bassd methods believe that the relationship between different activations, neurons or sample pairs contains rich information
learned by the teacher model
29 30 20
Recently, there are several methods which are used to compress object detectors by knowledge distillation. Chen et al. 3 proposed to distills the student detector through all components that ignore the imbalance in foreground and background, leading to poor results. Wang et al. (28) pay more attention to the near ground truth area. Li et al. [16] address the problem by distilling the positive and negative instances in certain proportion sampled by RPN. Ruoyu et al. (26) propose to use wwo-dimensional Gaussian mask suppress irrelevant redundant background while emphasizing the target information. Dai et al. (5] propose distillation method for detection tasks based on discriminative instances without considering the positive or negative distinguished by ground truth. Guo et al. [ point out that the information of features derived from regions excluding objects is also essential for distilling the student detector, which did not explore the background area in more detail. However, we find the feature-rich areas outside the bounding box regions also play an important role in knowledge distillation. It unreasonable to extract features by simply distinguishing positive and negative samples.
3

---

bboxt_eey
Backbone
PPN
Det Head
els soooe
Teacher
FRS Mosk
Tevel0
Distillation Feuture
Distilaation
Head Loss
Teve12
Texe12
Teve13
Teve14
Student
cls score
Backbone
PPN
Det Head
bbox reg
Figure 2: Overview of the proposed distillation via Feature Richness Score (FRS) framework.
Each FPN layer generates a distillation mask which represents feature richness, and we use the corresponding mask on distilling both the FPN layers and classification head.
3 Distilling Object Detectors with Feature Richness Score
previous works, most knowledge distillation methods pay more attention to the features bounding box areas, while ignoring the important feature outside the bounding box and detrimental feature within the bounding box. address this issue, we propose the Feature Richness Score as the criterion for selecting distillation areas. As shown in Figure 2 we use the aggregated classification score to represent the feature richness and then perform distillation on both FPN layers and the classification head.
3.1 Feature Richness Score Module
Given feature f, the probability that these features are objects can be described as:
Vc P(cl)),
(1)
where represents object category whether included in the dataset not, ye is the probability that feature if is an object of category c. This conditional probability can be modeled with deep neural network, which can be expressed as parameter model 0, described as:
Ye P)).f.0).
The proposed feature richness is defined as probability that these features are objects, whether categories of objects included in the dataset not. To make summary of all categories, get the feature richness by aggregating the probability of all categories. Here we use maximum to perform the aggregating:
S = max ye = max ().
c
In the framework distillation, we use the trained classification head teacher detector as an instantiation of 0. At the same time, since there are objects of various categories, we use the categories E [1, C*] in the dataset as the samples to represent all objects c, where C the number of categories in the dataset. Therefore, the feature richness can be approximated classification probability of teacher detector:
== max v = max 15e'5c Ise'Sc
P(d'lft,0')
where y the classification probability teacher detector, namely classification score map which is the output of classification head from teacher detector, f' is the features fed into the classification head teacher detector. practice, we can get the feature richness by integrating the classification score map output of the teacher detector in the direction of the channel.

---

3.2 EPN layers
Nowadays, the FPN module is an important part of the object detection frameworks. Since FPN integrates multiple layers of the backbone, performing distillation on FPN can benefit both FPN itself and the backbone. FPN module proposed to handle objects of various scales with multiple FPN layers. The shallow layers have high resolution to detect objects of small scales, while deep layers have a low resolution to detect objects of large scales. Therefore, it is necessary that FPN layers should use separate feature richness masks which represent the probability of objects of different scales and are different resolutions. We use the aggregated classification score map from the corresponding FPN layer as the feature richness:
S = max Vie:
(5)
15e50
where Sj is the feature richness mask of the 1-th FPN layer, of is the classification score map of the 1-th FPN layer, C represents the number of channels. Then the distillation loss of FPN is:
W
LFPN
Snj
E(rige - Dadape (Fagee Figee)
(6)
j=1
where M is the number of FPN layers, represents the 1-th FPN layer, i,j are the location of corresponding feature map with width W and height H. N; = = Σド= Suy- Fj and Fi are the 1-th FPN features of student and teacher detectors, respectively. Function Oudapt is the convolutional adaptation function adapt Ff to the same dimension as F/.
3.3 Classification Head
For distilling the classification head, is also necessary to use the proposed feature richness mask to effectively retrieve the important features outside the bounding boxes and remove the detrimental features within the bounding boxes. The feature richness mask used for classification head the same as that of FPN in (5) The distillation loss of the classification head is described as:
M
W
H
Lhead
Σ 1=1
Sug
Myyikee Virge):
where represents the 1-th FPN layer, y/ yf the output of classification head of the I-th FPN layer from teacher model and student model respectively, & refers to the Binary Cross-Entropy function used in classification head.
3.4 Overall loss function
The overall training targets of the student can be formulated as the weighted sum of the above- mentioned distillation losses alone with the conventional detection loss, described as:
L = Ler + aLFPN + L
(8)
where I-rT is the detection training loss, 3 are hyper-parameters to balance different distillation losses in our method.
Experiments
In order to verify the effectiveness and robustness of our method, conduct experiments different detection frameworks and heterogeneous backbones, and on the COCO dataset. We choose the default 120k train images split for training and 5k val images split for the test. Meanwhile, consider Average Precision as evaluation metric, i.e., mAP, APsu. AP7s, APs, APMandAP,. The last three measure performance with respect to objects with different scales.
with Pytorch framework. we use 2x learning schedule to train 24 epochs or the Ix learning schedule to train 12 epochs on COCO dataset. And the learning rate is divided by 10 the 8-th and I 11th epochs for 1x schedule and the 16-th and 22-th epochs for 2x schedule. We set momentum as 0.9 and weight decay 0.0001.
Our implementation based on mmdetection
5

---

mode
mAP
AP50
AP75
AP S
AP M
AP L
Retina- Renaa- (11 (teacher) Retina-Res50( ((
2x
38.9
58.0
41.5
21.0
42.8
52.4
2x
37.4
56.7
39.6
20.0
40.7
49.7
ours
2x
39.7
58.6
42.4
21.8
43.5
52.4
gain
42.3
+1.9
+2.8
+1.8
+2.8
+2.7
GFL-Resnet 01 (teacher)
2x
44.9
63.1
49.0
28.0
49.1
57.2
GFL- (eLL-Resne50(stuudee
1x
40.2
58.4
43.3
23.3
44.0
52.2
1x
43,6
61.9
47.5
25,9
47.7
56.4
gain
43.4
+3.5
+4.22
+2.6
+3.7
+4.2
GFL--Resnet (teacher)) (stuuet
2x
44.9
63.1
49.0
28.0
49.1
57.2
42.9
61.2
46.5
27.3
46.9
53.3
ours
2x
44.7
63.0
48.4
28.7
49.0
56.7
gains
+1.8
+1.8
+1.9
+1,4
+2.1
+3.4
Faster-Res 10 (teacher)
1x
39.4
60.1
43.1
22.4
43.7
51.1
Seaier-RRe50(studdeee
1x
37.4
58.1
40.4
21.2
41.0
48,1
1x
39.5
60.1
43.3
22.3
43,6
51.7
gains
+2.1
+2.0
+2.9
+1.1
+2.6
+3.6
Resnet101
2x
40.8
60.0
44.0
24.2
44.3
52.4
FCOS- Resnet50(student)
2x
38.5
57.7
41.0
21.9
42.8
48.6
ours
2x
40.9
60.3
43.6
25.7
45.2
51.2
gains
+2.4
+2.6
+2.6
+3.8
+2.4
+2.6
Table 1: Results of the proposed method with different detection frameworks. schedule to train 24 epochs or the 1x learning schedule to train 12 epochs on COCO dataset.
2x Tearning
model
mode
mAP
AP50
AP75
APS
APM
APL
Retina-ResX 101 (teacher)
2x
40.8
60.5
43.7
22.9
44.5
54.6
Retina-Res (estna--ReSS(studenn)
2x
37.4
56.7
39.6
20
40.7
49.7
TO
2x
37.8
58.3
41.1
21.6
41.2
48.3
39.6
42.1
22.7
52.5
our'
2x
40.1
59.5
42.5
21.9
43.7
54.3
Retina-Rees (teacer))
38.1
58.3
40.9
21.2
42.3
51.1
Getina-Res50(stuuee
2x
36.2
55.8
38.8
20.7
39.5
48.7
Fitnet 25
2x
37.4
57.1
40
20.8
40.8
50.9
General Instance
2x
39.1
59
42.3
22.8
43.1
52.3
ours
2x
39.3
58.8
42
21.5
43.3
52.6
Table 2: Comparison with previous work with different detection frameworks.
4.1 Different defection frameworks
We verify the effectiveness of our proposed FRS on multiple detection frameworks, including anchor- based one-stage detector RetinaNet [8] and GFL (17), anchor-free one-stage detector FCOS (27).
and two-stage detection framework Faster-RCNN [24] on COCO dataset [19]. As shown in Table I ResNet50 is chosen the student detector, ResNet101 is chosen as the teacher detector. The results clearly show that our method gets significant performance gains from the teacher and reaches a comparable even better result the teacher detectors. RetinaNet with ResNet50 achieves 39.7 in mAP, meanwhile surpasses the teacher detector by 0.8% mAP. The ResNet50 1x based GFL surpasses baseline by 3.4% mAP. The ResNet50 based on FasterRCNN gain 2.4% mAP. FCOS with ResNet50 achieves 40.9%, which also exceeds the teacher detector. These results clearly indicate effectiveness and generality of our method in both one-stage and two-stage detectors.
4.2 Comparison with State-of-the-arts
Table 2 shows the comparison of the results of -fate-of-the-ar distillation methods on the COCO 19 benchmark. including bounding boxes based methods (28 As shown in Table the performance
For fair comparison, we run the experiment using the teacher and student model the same accuracy, the implementation is based on the mmdetection 1.2. In addition to this set of experiments, our other experiments based mmdetection2.

---

Retina-Res50 Retina-Res 101
FPN layers Classification Head
MAP
37.4
38.9
39.4
38.4
39.7
Table 3: Ablation Study for various distillation modules on COCO dataset
Figure 3: From left to right, they correspond to the feature masks of different FPN layers. The leftmost corresponds to the lowest level of FPN, which identifies small objects, and the rightmost corresponds to the highest level of FPN, which used identify large objects.
of the student detector has been greatly improved. For example, under the guidance of RseNext101 teacher detector, the ResNet50 with RetinaNet student detector achieves 40. % in mAP, surpassing 32] by 0.5% mAP. With the help of ResNet101 teacher detector, the ResNet50 based RetinaNet gets 39.3% mAP, which also exceeds the State-of-the-att methods General Instance [5]. The results demonstrate that our method has achieved a good performance improvement, surpassing the previous SOTA methods.
4.3 Ablation Study
4.3.1 Impact of different modules
In this part, we conducted some ablation experiments with different distillation modules to better reflect the impact of each module on knowledge distillation. As shown Table each distillation module can improve the performance of the student network. Distillation on the FPN module can bring an improvement of 2.0% mAP, and distillation on the classification head module can gain 1.0% mAP. The combination of both can further improve performance, which brings up to 2.3% mAP improvement. can be seen that distillation on each module has a certain improvement for knowledge distillation.
4.3.2 Varying teacher detector for FRS
In this subsection, we investigate the influence of different teacher detectors for distillation. The student detector is RetinaNet (whose backbone is RegNet). shown Figure it obvious that with the improvement of the accuracy of the teacher's network, the performance of the students has also been improved.
4.4 Analysis
4.4.1 Qualitative performance gain from imitation
We qualitatively compare the results of student detector and our FRS. In Figure the upper row images show the results of student detector without distillation, while the lower shows the

---

Bescling lino Dstector
PRS Distilled
Detector
(b)
(c)
(d)
Figure 4: Qualitative analysis on COC02017 dataset with distilled and baseline Retianet-RRsNedl The blue boxes mean the undetected or wrongly detected objects of the baseline detector
distillation results of the FRS method. Obviously, the detectability of the student network has been improved by the FRS distillation method. As shown Figure 4 a)(c), The student detector generates multiple bounding boxes for the same object which are unfortunately not able to be suppressed by NMS. while our method can accurately locate the location of the same objects that are close each other, avoiding generating incorrect bounding box. Figure shows the student model mistakenly detects a boat, while our method avoids this error and outputs rather accurate bounding box for the trunk instance. As shown in Figure the raw student model mistakes the cat for dog. In Figure c). the student model wrongly predicts an area of background as a cat instance, while the imitated student avoids the error, indicating that the FRS method has a better ability to discriminate.
4.4.2 Visualization of FRS
As shown in Figure we visualize the heat map from each FPN level by the FRS method to better understand the importance of different regions. It can be clearly FRS obviously covers the objects of the corresponding scale. As for the edge area of objects, the corresponding feature weight is smaller, while in the regions with rich feature information, such as the central area of the object, the corresponding feature weight is relatively high. Meanwhile, the highlight areas of different FPN levels are different and have different semantic information. The low-level FPN is more focused on finding the characteristics of small objects, while the high-level FPN level pays more attention to large objects. Besides, the distance also affects the visual size. The FRS method can better extract subtle information and find the most suitable one from different FPN levels.
that the region extracted by
FRS method is pixel-wise and each pixel is in a form of soft-label which can carry more information than the hard-label. In foreground areas, shown in Figure the closer the center of the object, the richer the features and the higher the corresponding value. In the edge area of the object, the detector can only observe the local features of the object, which leads to lower feature richness score.
4.4.3 Comparison of different distillation areas
To further analyze the contribution of FRS methods, we investigate the influence of the different regions. In order to compare the difference between the FRS method and the previous ground truth method in detail, divide the image area into four parts according to the feature extraction situation of the teacher network, namely TP(Tree Positive), FP(False Positive), FN(False Negative), TN(True Negative). As shown in Table 4 distilling the TP+FP regions performs the highest mAP, also higher than using TP+FN regions. The area selected by the previous ground truth method corresponds to TP+FN regions, while the FRS method corresponds TP+FP regions, since thinks the FP regions are as important as TP regions but ignores the FN regions. These results show that the proposed FRS outperforms than existing methods distilling based on bounding boxes in different aspect. Moreover, the FRS method which focuses on TP+FP regions is better than distilling only the TP area, which proves that the FP region effective for distillation. Besides, distilling on TP+FN regions shows lower than that on only TP regions, which shows distilling FN may damage the results.
8

---

mAP
AP50
AP75
AP_S
AP_M P_MM
AP
L
TP 39.1
58.2
41.8
21.3
43.0
51.7
TP+FP
39.4
58.4
42.1
21.6
43.7
52.1
TP+FN
39.0
58.0
41.9
21.5
43.1
51.9
TP+TN 38.7
57.8
41.2
20.6
42.3
52.3
Table 4: Based on the ground truth and classification score of the teacher detector, we have made more detailed division of the foreground and background area. TP (True Positive) refers to the foreground area with high feature richness scores, FN (False Negative) means the foreground area with low feature richness scores, FP (False Positive) is the background area with high scores, and TN (True Negative) denotes the background area with low scores. For clarity, we only distill the FPN layers Resiaee model, under the guidance of ResNet101 detector
teadine nole Rog
54-
30.5
37.
37.0
20.
4.4
36.6
Teacher male
Figure Information entropy of different regions. Figure Relation between teacher and student detec- tor on Regnet based Retinanet in COCO2017 dataset
Meanwhile, we also compared the distillation on the TP+TN regions, and the result is no better than that on only the TP area, which shows that TN has no effect on distillation.
We analyzed the FRS feature extraction method from the perspective of information entropy. We find that the FRS method can extract more feature-rich areas from the background area, which are the areas with higher information entropy. As shown in Table the information entropy of the TP area is the highest, and the information entropy corresponding to the FP area is also relatively high. The rest of TN and FN have the lowest information entropy. Therefore, the area with higher information entropy corresponds to more feature information, which can promote the distillation effect.
5 Conclusion
In this paper, we propose an effective distillation method for object detection. We analyze and demonstrate the beneficial features outside the bounding box and the detrimental features within the bounding boxes. Based on this finding, we propose the FRS method to distill important features that improve the detectability of the student detector. FRS method effectively improves the performance of modern detection frameworks and can be universally applicable for both two-stage and one-stage detection frameworks.
Acknowledgments
This work is partially supported by the Beijing Natural Science Foundation (JQ18013), the NSF of China(under Grants 61925208, 61906179, 62002338, 61732007, 11732002,62102399 Strategic Priority Research Program of Chinese Academy of Science (XDB32050200) Beijing Academy of Artificial Intelligence (BAAI), CAS Project for Young Scientists in Basic Research( SBR-029) and Xplore Prize.

---

References
[1] Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q Weinberger, editors, Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 2654-2662, 2014.
[2] Zhaowei Cai and Nuno Vasconcelos. Cascade r-enn: High quality object detection and instance segmentation. IEEE Transactions on Patlern Analysis and Machine Intelligence, page 1-1, 2019.
[3] Guobin Chen, Wongun Choi, Xiang Yu, Tony X. Han, and Manmohan Chandraker. Learning efficient object detection models with knowledge distillation. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 742-751, 2017.
[4] Kai Chen, Jiaqi Wang, Jiangmiao Pang. Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. Mmdetection: Open mmlab detection toolbox and benchmark. CoRR, abs/ 1906999067700 07155, 2019.
[5] Xing Dai, Zeren Jiang, Zhao Wu, Yiping Bao, Zhicheng Wang. Si Liu, and Erjin Zhou. General instance distillation for object detection. CoRR, abs/2103.02 0234000,03 2021.
[6] Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. Centernet: Keypoint triplets for object detection. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - -November 2, 2019, pages 6568-6577. IEEE, 2019.
[7] Jianping Gou, Baosheng Yu, Stephen John Maybank, and Dacheng Tao. Knowledge distillation: A survey. CoRR, abs/2006.05525, 2020.
[8] Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, and Chang Xu. Distilling object detectors via decoupled features. CoRR, Abs/22003.144755 2021.
[9] Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. Mask r-cnn. 2017 IEEE International Conference on Computer Vision (ICCV), 2017.
[10] Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi. A comprehensive overhaul of feature distillation. In 2019 IEEE/CVE International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 -Nvember 2, 2019, pages 1921-1930. IEEE, 2019.
(11) Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi. Knowledge transfer via distillation of activation boundaries formed by hidden neurons. In The Thirty -hird AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 February 1, 2019, pages 3779-3787. AAAI Press, 2019.
[12] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. CoRR, 0ts//1503.0253/ 2015.
[13] Jangho Kim, Seonguk Park, and Nojun Kwak. Paraphrasing complex network: Network compression via factor transfer. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NewrlPS 2018, December 3-8, 2018, Montréal, Canada, pages 2765-2773 2018.
10

---

[14] Hei Law and Jia Deng. Comernet: Detecting objects as paired keypoints. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision ECCV 2018 15th European Conference, Munich, Germany, September 8- 14, 2018, Proceedings, Part XIV, volume 11218 of Lecture Notes in Computer Science, pages 765-781 Springer, 2018.
[15] Seunghyun Lee and Byung Cheol Song. Graph-based knowledge distillation by multi-head attention network. In 30th British Machine Vision Conference 2019, BMVC 2019, Cardiff, UK, September 9-12, 2019, page 141. BMVA Press, 2019.
[16] Quanquan Li, Shengying Jin, and Junjie Yan. Mimicking very efficient network for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI. USA, July 21-26, 2017, pages 7341-7349. IEEE Computer Society, 2017.
[17] Xiang Li. Wenhai Wang. Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li. Jinhui Tang, and Jian Yang. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection, arXiv preprint arXiv:2006. 04388, 2020.
[18] Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2017.
[19] Tsung- Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision ECCV 2014 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V, volume 8693 of Lecture Notes in Computer Science, pages 740-755. Springer, 2014.
[20] Yufan Liu, Jiajiong Cao, Bing Li, Chunfeng Yuan, Weiming Hu, Yangxi Li, and Yunqiang Duan. Knowledge distillation via instance relationship graph. In IEEE Conference Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 7096-7104. Computer Vision Foundation / IEEE, 2019.
[21] Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant: Bridging the gap between student and teacher. CoRR, abs/1992.00339 2019.
[22] Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. You only look once:
Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 779-788. IEFE Computer Society, 2016.
[23] Joseph Redmon and Ali Farhadi. YOLO9000: better, faster, stronger. In 2017 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 6517-6525. IEEE Computer Society, 2017.
[24] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real- time object detection with region proposal networks. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett, editors, Advances in Neural Information Processing Systems 28: Annual Conference Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 91-99, 2015.
[25] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[26] Ruoyu Sun, Fuhui Tang, Xiaopeng Zhang, Hongkai Xiong, and Qi Tian. Distilling object detectors with task adaptive regularization. CoRR, 10s/22006.1108, 2020.
[27] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection, arXiv preprint 104 2019.

---

[28] Tao Wang, Li Yuan, Xiaopeng Zhang, and Jiashi Feng. Distilling object detectors with fine- grained feature imitation. In JEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 4933 4942. Computer Vision Foundation TERE, 2019.
[29] Junho Yim, Donggyu Joo, Ji-Hoon Bae, and Junmo Kim. gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In 2017 IEEE Conference Computer Vision and Patter Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 7130-7138. IEEE Computer Society, 2017.
[30] Lu Yu, Vacit Oguz Yazici, Xialei Liu, Joost de Weijer, Yongmei Cheng, and Amau Ramisa. Learning metrics from teachers: Compact networks for image embedding. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 2907-2916. Computer Vision Foundation IEEE, 2019.
[31] Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
[32] Linfeng Zhang and Kaisheng Ma. Improve object detection with cadture-based knowledge distillation: Towards accurate and efficient detectors. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[33] Rui Zhang, Sheng Tang, Luoqi Liu, Yongdong Zhang, Jintao Li, and Shuicheng Yan. High resolution feature recovering for accelerating urban scene parsing. In Jérôme Lang, editor, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pages 1156-1162. ijeai.org, 2018.
[34] Xishan Zhang, Shaoli Liu, Rui Zhang, Chang Liu, Di Huang, Shiyi Zhou, Jiaming Guo, Qi Guo, Zidong Du, Tian Zhi, et al. Fixed-point back- packppropagation training. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 2330-2333 2020.
12


Reviews:
[{"review_id": "lRbYsV3mqje", "reviewer": "Reviewer_iJCP", "summary": "One of the challenges in object detection is to train models with reduced computational and memory complexity for deployment on the resource-limited devices. One of the central methods is knowledge distillation which transfers knowledge from large teacher detector into small one with almost preserving the quality of the teacher model. However, existing methods have several issues: i) they ignore feature representation outside bounding boxes which can be helpful (e.g. similar but absent from the class categories objects which detection can improve target class detection)  ii) they use some detrimental representations within bounding boxes (ignore teacher model mistakes). To overcome these issues the paper proposes a novel method (FRS: Feature-Richness Score) to choose important features which can improve detection during distillation process. With experiments on anchor-based/anchor-free and one-stage/two-stage detectors authors demonstrate significant improvement in detection with FRS: FRS outperformes state-of-the-art methods on all distillation-based detection frameworks, Faster-RCNN, Retinanet, GFL and FCOS.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality** \n\nKnowledge distillation is a well known approach and widely used for different tasks including object detection. Specifically for object detection task imitating all features is inefficient because there is a lot of object-irrelevant information. So the main focus of research for knowledge distillation in object detection is how to select proper representations to distill. Instead of following common approach to select features from the ground truth bounding boxes, in this paper a novel approach how to select/define important features for distillation is proposed: features importance can be seen as probability that these features are objects (pixel-wise aggregated classification score is used as this importance measure). Proposed method generates pixel-wise importance masks for each FPN layer and applicable to any detector type: one-stage, two-stage, anchor-free. Section 2 well covers related works and comparison between existed methods and the one proposed in the paper.\n\n**Quality**\n\nThere is no any theoretical analysis and work is purely experimental. Experiments are performed\n- across different detectors with standard distillation approach; proposed method demonstrates significant gains and even sometimes outperforms the teacher (Table 1)\n- across different SOTA distillation approaches for different detectors (Table 2) - here some clarification on the Table is needed (see comment below)\n- for ablations on each distillation loss component (Table 3)\n- for ablation on varying the teacher detector performance (better teacher -> better student)\n- for detailed analysis of the proposed method, demonstrating that indeed usage of the most informative parts (TP and FP) improves distillation.\n\nThe following additional information on experiments is needed to conclude if comparison is technically correct:\n- What will happen if learning schedule will be longer for student in table 1?\n- Table 2: It seems results for all methods except teacher and student are taken from [5], and in [5] teacher and student baselines are reported to be worse than ones in the paper. Did authors retrain teacher and student on their own? Did they retrain all other state-of-the-art methods? For example in [5] teacher is reported 38.1 (-0.8 from the number in Table 2) which can be the source of inconsistent results in Table 2 due to better teacher used for the paper's method (could be that [5] outperforms proposed method with the same teacher). This place should be clarified and fixed in the text.\n\nOther comments/questions:\n- YOLO is also one of the leading detectors, what about it?\n- line 133-135: sentence is not clear. Could author explain what they mean?\n- It is not clear from the text how categories/classes out of training dataset can be used, how the feature richness score is computed in this case?\n- Eq. (6) why is $\\phi_{adapt}$ applied to the student, not to the teacher? I assume teacher has larger dimension, so it is more reliable to perform downsampling rather than upsampling.\n- line 203: 1.9% should be 2.0% looking at Table 3; line 204: 1.1% should be 1.0% looking at Table 3.\n- Table 3 is possible incorrect for first two columns. In table 1 Retine-Res101 is reported as 38.9 (37.4 in Table 3) and Retina-Res50 37.4 (38.5 in Table 3).\n\n**Clarity**\n\nPaper is mostly clear written, however there are some misleading sentences/formulations, a lot of typos and probably it is worth to have some native-speaker proof-reading for the text to improve readability. Below the list (not the full one) of typos and additional comments/suggestions on improving the text:\n- line 55: \"the importance of features is strongly correlated with the feature richness\". I would say importance of features by paper's definition is feature richness. Probably this sentence should be removed or reformulated.\n- Please introduce FPN notation before its usage\n- line 68: \"Finally, Our\" -> \"Finally, our\"\n- line 71: \" FRS, We\" -> \" FRS, we\", \"on various framework\" -> \"on various framework**s**\"\n- throughout the text fix listing several references together, for example line 78 \"[23]  [2][9]\" -> \"[23,2,9]\" \n- line 79-80: reformulate sentence as the verb is absent in it.\n- line 82: fix is needed in \"applications. while\"\n- line 87-88: fix English in the sentence - either remove first \"while\" or put some verb in the second sentence part.\n- line 95: \"Hinton [11] proposal\" -> \"Hinton [11] proposed\"\n- line 96: \"which open\" -> \"which open**s**\"\n- line 99: \"a important role\" -> \"an important role\"\n- line 101: \"The relationship\" -> \"the relationship\"\n- line 104: \"[3] proposal to distills\" -> \"[3] proposed to distill\"\n- be consistent with \"et al\"/ \"et al.\" usage \n- lines 107, 109: two spaces before references\n- line 143: \"perform distillation\" -> \"performing distillation\"\n- line 154: \"student and teacher detector\" -> \"teacher and student detectors\"\n- line 172-173 \", We\" -> \", we\"; fix \"and\" in the formula\n- line 175: \". we\" -> \". We\"\n- Template guidelines: tables captions should be at the top of the tables not at the bottom\n- line 191: \"The table 2\" -> \"Table 2\"\n- Table 2: Fater-Res101 -> Faster-RCNN? R-CNN?\n- line 204 \"can further improved\" -> \"can further improve\"\n- Figure 5 - text size is different from other\n- line 214: \"In Figure 4, The\" -> \"In Figure 4, the\"\n- line 219: \". while Our\" -> \", while our\"\n- line 220: \"bbox\" -> \"bounding box\" or introduce the abbreviation\n\nNecessary details on the training configuration are provided, so should be reproducible.\n\n**Significance** \n\nResults on distillation are important as we can further improve and simplify the student network training for deployment purposes.  In this paper authors propose a novel method how to define important regions for distillation and demonstrated with in-depth analysis its effectiveness. Proposed method is simple in its implementation, applicable to different detectors and outperforms state-of-the-art-models (should be rechecked with Table 2 + more info from authors).\n\n**Update**: As authors addressed to all my concerns and fixed the Table 2, I change the final decision from 5 to 6", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors did not specify any limitations or social impact throughout the text and in the supplementary, however in the checklist it is marked as \"Yes\". From my point I don't see any potential problem for societal impact except that detection can be further improved and additional bias with new training method should be later investigated. As limitations - obviously it will worse work with detectors without FPN layers as we cannot use extra info to define important regions.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}, {"review_id": "GKERhtFWy8k", "reviewer": "Reviewer_uYvK", "summary": "This paper introduces a novel knowledge distillation method for object detection, named Feature-Richness Score(FRS), which makes good use of aggregated classification score of the feature map as the mask of FRS. This pixel-wise FRS mask would be used as the weighted matrix to guide the distillation from teacher to student within selected layers, such as FPN and head. Plenty of well-designed experiments could show the effectiveness and availability of their method on serval mainstream detection framework, also prove it surpasses other recent methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1. The proposed method is quite clean and simple, but effective, compared with other recent methods with complicated design. It just use the aggregated classification score, take maximum operation in channel direction, of the feature map as a weighted score mask, named Feature-Richness Score(FRS) to guide the distillation, which is easy to implement and plug into any layers.\n2. Unlike the prior methods, the proposed method could make use of whole feature map, not only features around bbox area, which most methods usually do. That makes the authors claim, avoiding the defected results from teacher model, sounds reasonable and understandable.\n3. The authors had done extensive experiments on serval mainstream detection frameworks.\n\nCons:\n1. In Table 2, the accuracy of teacher model (Retina-res101, mAP=38.9) is higher than the accuracy (mAP=38.1) of the same model used by General Instance method, but the compared accuracy of General Instance seems like to be just copied from their work, which makes the comparison is not fair. Same issue on the results of faster-RCNN framework.\n2. In Table 4, the improvement (+0.3) from TP to TP+FP may prove the potential gain from make use of out of bbox features is limited, combining with Cons#1, the gap between the proposed method and previous sota might be shorten.\n3. The FRS design is highly related to the category size. what if the dataset has limited classes, which is a usual case in autonomous drive scenario. Think about an extreme case with just one class, it seems it would fall back to some kind of trivial baseline.\n4. The Table 3 is quite confusing, why the Res50 surpasses Res101? Also, the results of the last three columns didn't mention the backbone they used.\n\nOriginality: The motivation of the proposed method differs from previous contributions. And the design is simple and clear.\n\nQuality: The submission has extensive experiments to support their claims, but the results presentation could be improved.\n\nClarity: The submission is easy to read and understand.\n\nSignificance: The submission would be a meaningful contribution to this research domain and bring a new point of view to make use of features out of bbox.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. Typo: In Figure 2, the second level 1 in graph.\n2. Correction might be needed in Table 2.\n3. The readability of Section 4.3.1 and Table 3 could be improved.", "review_text": "", "extra_scores": {"ethical_concerns": "Don't have any Ethical Concerns with this submission.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3~4"}, "extra_text": {}}, {"review_id": "DsNGoLJNowZ", "reviewer": "Reviewer_KFUi", "summary": "In this paper, the authors proposed feature-richness score to choose important features for knowledge distillation. Specifically, the proposed method uses the classification score from teacher model as the feature-richness masks. The feature-richness masks are used as indicators that show where the import feature is and what the importance of this feature. Based on these masks, the proposed method can focus on distilling import features in FPN and classification head.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n- The proposed method is simple but effective. In the detectors with FPN, the improvements is great.\n- This paper is well written and readable.\n\nWeaknesses\n- In the experiments, the authors valid the effectiveness of the proposed method in detectors with FPN. But there are also many detectors without FPN, such as [1] and [2]. Could you show the improvements in these detectors with the proposed method?\n- This paper should compare with other recent state-of-the-art methods, e.g. [3]. \n- The distillation losses in the proposed method are related to two hyper-parameters: $\\alpha$ and $\\beta$. How these hyper-parameters affect the performance? Does the values of these hyper-parameters depend on datasets or detectors?\n- For Table 3, the performance Retina-Res50 is 38.5 mAP while  Retina-Res101 has 37.4 mAP. Is it fault? Besides, even if these two values exchange, for Retina-Res101, 38.5 mAP in Table 3 is different from 38.9 mAP in Table 1. In addition, in line 203 and line 204, the mAP improvements (1.9% and 1.1%) in text are different from ones, as shown in Table 3. Can you explain these mismatches?\n- In equation (5) and equation (6), the $c'$ and $C$ should denote different things, one for classes and the other for channels. It need to be clarified in the paper.\n- The proposed method is only valid on COCO dataset, while other methods, such as General Instance[5] and Fine-grained[27], also conduct experiments on PASCAL VOC. Could you show the results on PASCAL VOC with the proposed method?\n\n[1] Duan, Kaiwen, et al. \"Centernet: Keypoint triplets for object detection.\"Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.\\\n[2] Chen, Qiang, et al. \"You only look one-level feature.\"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. \\\n[3] Zhang, Linfeng, and Kaisheng Ma. \"Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors.\"International Conference on Learning Representations. 2021.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "24"}, "extra_text": {}}, {"review_id": "3PcnWE32q7A", "reviewer": "Reviewer_FmCT", "summary": "This paper proposes to perform knowledge distillation on object detectors with feature richness. They use the feature-richness score to decide how to choose the important features. Experiments and qualitative analysis on COCO have been provided to evaluate their method.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:  There have already been some methods that try to distill knowledge outside the anchors such as [1] and [2] so this is not a very novel topic. The authors propose to use the feature richness score to decide which region to be distilled, which is a new solution for this topic. \n\nQuality:  The author does not provide enough comparison with some very related works such as [1]. Comparisons on both experiments and methodology should be provided. 2. the experiments are also not enough. More detectors such as DERT and more datasets such as Cityscapes should be considered.  3. Analysis in 4.4.3 is interesting. \n\nClarity. The writing of this paper is ok. But some figures (fig5, fig6, fig7) are blurry. Authors should provide figures in vectorgraph. And the fonts in these figures are too small.\n\nSignificance. The results of this paper are not very significant compared with previous papers such as [1].\n\n[1] Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors, ICLR2021\n[2] General instance distillation for object detection, CVPR2021\n\n------------\nAfter Rebuttal, Rate 6 -> 7.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Please refer to main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a method (Feature-Richness Score) to choose important features to distill for object detection, leveraging a feature pyramid network. There is significant prior work in the domain but the method is different enough. The experimental evaluation is convincing as it encompasses several detectors, backbones, and datasets. A limit of the method is that it does not work out of the box with approaches without FPN (e.g. DETR). Overall, the paper can be improved easily and the authors' response convinced 3 reviewers (out of 4) to raise their score by one point. The outcome makes this paper fitting as poster at NeurIPS.

Decision: Accept (Poster)