Distilling Image Classifiers in Object Detectors
Authors: Shuxuan Guo
Jose M. Alvarez
Mathieu Salzmann
Venue: NeurIPS 2021 Poster (2021)
Keywords: knowledge distillation
classifier-to-detector
across tasks
object detection

Abstract: Knowledge distillation constitutes a simple yet effective way to improve the performance of a compact student network by exploiting the knowledge of a more powerful teacher. Nevertheless, the knowledge distillation literature remains limited to the scenario where the student and the teacher tackle the same task. Here, we investigate the problem of transferring knowledge not only across architectures but also across tasks. To this end, we study the case of object detection and, instead of following the standard detector-to-detector distillation approach, introduce a classifier-to-detector knowledge transfer framework. In particular, we propose strategies to exploit the classification teacher to improve both the detector's recognition accuracy and localization performance. Our experiments on several detectors with different backbones demonstrate the effectiveness of our approach, allowing us to outperform the state-of-the-art detector-to-detector distillation methods.

Distilling Image Classifiers in Object Detectors
Shuxuan Guo
Jose M. Alvarez?
Mathieu Salzmann'
CCVLab, EPFL, Lausanne 1015, Switzerland NVIDIA, Santa Clara, CA 95051, USA
shuxuan, Shuxuan.gu@@epfl. josealnvidia.com com, com, mathi eu salzmann@epfl.cd
Abstract
Knowledge distillation constitutes a simple yet effective way to improve the per- formance of compact student network by exploiting the knowledge of more powerful teacher. Nevertheless, the knowledge distillation literature remains lim- ited to the scenario where the student and the teacher tackle the same task. Here, we investigate the problem of transferring knowledge not only across architectures but also across tasks. To this end, study the case of object detection and, instead of following the standard detector-to-deteccor distillation approach, introduce a propose knowledge transfer framework. In particular, strategies to exploit the classification teacher to improve both the detector's recog- nition accuracy and localization performance. Our experiments on several detectors with different backbones demonstrate the effectiveness of our approach, allowing to outperform the sfateoffthhe-a detector-to -e-ttcto distillation methods.
classifiertt-ddetenee
Introduction
Object detection plays critical role in many real-world applications, such as autonomous driving and video surveillance. While deep learning has achieved tremendous success in this task [25] 26 31 32 40),
the speed-accuracy trade-off of the resulting models remains challenge. This is particularly important for real-time prediction on embedded platforms, whose limited memory and computation power impose strict constraints on the deep network architecture.
To address this, much progress has recently been made to obtain compact deep networks. Existing methods include pruning 2 3 22 38] and quantization 7 30 44 both of which aim to reduce the size of an initial deep architecture, as well as knowledge distillation, whose goal is to exploit deep teacher network to improve the training of given compact student one. In this paper, introduce a knowledge distillation approach for object detection.
focused on the task of image classification. several attempts have nonetheless been made for object detection. To this end, existing techniques (5] typically leverage the fact that object detection frameworks consist of three main stages depicted by Figure Ta): A backbone to extract features; a neck to fuse the extracted features; and heads to predict classes and bounding boxes. Knowledge distillation is then achieved using teacher with the same architecture as the student but deeper and wider backbone, such as a Faster RCNN [32 with ResNet 52 [14] teacher for Faster RCNN with ResNet50 student, thus facilitating knowledge transfer at all three stages of the frameworks. To the best of our knowledge,
While early knowledge distillation techniques [8
33 36
12
39]
4 constitutes the only
exception to this strategy, demonstrating distillation across different detection frameworks, such as from a RetinaNet (25) teacher to RepPoints (40) student. This method, however, requires the teacher and the student to rely on similar detection strategy, i.e., both must be either one-stage detectors or two-stage ones, and, more importantly, still follows a detector-to-detector approach to distillation.
'The work is done during an internship at NVIDIA.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In other words, the study of knowledge distillation remains limited to transfer across architectures tackling the same task. Our classification teacher tackles a different task from the detection student and is trained in a different manner but on the same dataset. Therefore, the classification teacher is capable of providing a different knowledge to the student, for both classification and localization, than that extracted by detection teacher.
In this paper, we investigate the problem of transferring knowledge not only across architectures but also across tasks. In particular, we observed that the classification head of state-of-the-art object detectors still typically yields inferior performance compared to what can be expected from an image classifier. Thus, as depicted by Figure [T], we focus on the scenario where the teacher is an image classifier while the student is an object detector. We then develop distillation strategies improve both the recognition accuracy and the localization ability of the student.
Our contributions can thus be summarized as follows:
We introduce the idea of classifier-to-detctorr knowledge distillation to improve the perfor- mance of student detector using a classification teacher.
We propose distillation method to improve the student's classification accuracy, applicable when the student uses either a categorical cross-entropy loss or binary cross-entropy one.
We develop distillation strategy to improve the localization performance of the student be exploiting the feature maps from the classification teacher.
We demonstrate the effectiveness of our approach the COC02017 benchmark 24 detectors, including the relatively large two-stage Faster RCNN and single-stage RetinaNet used in previous knowledge distillation works, well as more compact detectors, such SSD300, SSD512 (26] and Faster RCNNs 32) with lightweight backbones. Our classifier-to-detcttoo distil- lation approach outperforms the to-eteroto distillation ones in the presence of compact students, and helps to further boost the performance of ditector-todetectte distillation techniques for larger ones, such as Faster RCNN and RetinaNet with ResNet50 backbone. Our code is avlaible at:
using diverse
httpp://gii wtw.a/..oo///lbbDDo///t
2 Related work
Object detection is one of the fundamental tasks in computer vision, aiming to localize the objects observed in an image and classify them. Recently, much progress has been made via the development of both one-stage 9 [1] 26 (7) and two-stage 4 [5] 24 32] deep object detection frameworks, significantly improving the mean average precision (mAP) on standard benchmarks [00 11 23). However, the performance of these models typically increases with their size, and so does their inference runtime. This conflicts with their deployment on embedded platforms, such as mobile phones, drones, and autonomous vehicles, which involve computation and memory constraints. While some efforts have been made to design smaller detectors, such as SSD (26). YOLO [31] and detectors with lightweight backbones [9 35), the performance of these methods does not match that of deeper ones.
Knowledge distillation offers the promise to boost the performance of such compact networks by exploiting deeper teacher architectures. Early work in this space focused on the task of image classification. In particular, Hinton et al. [18 proposed to distill the teacher's class probability distribution into the student, and Romero et al. 33 encouraged the student's intermediate feature maps to mimic the teacher's ones. These initial works were followed by rapid growth in the number of knowledge distillation strategies, including methods based on attention maps [42]. on transferring feature flows defined by the inner product of features [41] and contrastive learning to structure the knowledge distilled from teacher to the student [36]. Heo et al. [17] proposed a synergistic distillation strategy aiming to jointly leverage teacher feature transform, student feature transform, the distillation feature position and a better distance function.
Compared to image classification, object detection poses the challenge of involving both recognition and localization. As such, several works have introduced knowledge distillation methods specifically tailored to this task. This trend was initiated by Chen et al. (5) which proposed to distill knowledge from teacher detector to student detector both the backbone and head stages. Then, Wang et al. 39] proposed to restrict the teacher-student feature imitation to regions around positive anchor

---

Ground truth
Object Ou!
Box Prediction
Detector
Teacher
Classitier Teacher
Teacher Backbone
Distilltion
Student Backbone
Student Backbone
Qai
Prodicted Object OP
Teacher Backbone
Teacher Backbone
Classifier Teacher
Teacher
Distillation
Student Neck
Cis Head
Student Neck
Neck
Teacher
Distillaionn
Student Head
Student Head
Festure
Cls
Cle
Localization
Distillatonn
Classification Distillation
Box
Box
Distillation
(a) Dinector-to-Detacii
Distillation
(b) Our Classifier-to to-ttecto Distillation
Figure 1: Overview of our classifier-oo-detccorr distillation framework, (a) Existing methods
perform distillation across corresponding stages in the teacher and student, which restricts their applicability to detector-to-detecte distillation. (b) By contrast, we introduce strategies to transfer the knowledge from an image classification teacher an object detection student, improving both its recognition and localization accuracy.
boxes; Dai et al. 8 produced general instances based both the teacher's 's and student's outputs, and distilled feature-based, relation-based and response-based knowledge in these general instances; Guo al. 12 proposed to decouple the intermediate features and classification predictions of the positive and negative regions during knowledge distillation. All the aforementioned knowledge distillation methods require the student and the teacher follow the same kind of detection framework, and thus typically transfer knowledge between models that only differ in terms of backbone, such as from RetinaNNt-RRssNtt1..
43] such a constraint able to transfer knowledge across the feature maps of different frameworks. This allowed the authors leverage the best one-stage, resp. two-stage, teacher model to perform distillation any one-stage, resp. two-stage, student. This method, however, still assumes that the teacher is detector.
RetinaNNetRReNNe150..
relaxed via method
In short, existing knowledge distillation methods for object detection all follow detector-to-detector transfer strategy. fact, to the best of our knowledge, distillation has only been studied across two architectures that tackle the same task, may it be image classification, object detection, semantic segmentation [16] [27]. In this paper, by contrast, we investigate the use of knowledge distillation across tasks and develop strategies distill the knowledge of image classification teacher to an object detection student.
Our Approach
Our goal is to investigate the transfer knowledge from an image classifier to an object detector. As illustrated in Figure [ this contrasts with existing knowledge distillation techniques for object detection, which typically assume that the teacher and the student both follow a similar three-stage detection pipeline. For our classifier-to-detectorr knowledge distillation be effective, we nonetheless need the student and teacher to process the same data and use the same loss for classification. To this end, given a detection dataset Dået depicting C' foreground object categories, we construct classification dataset Data by extracting all objects from Ddttt according to their ground bounding boxes and labels. We then train our classification teacher J,, with parameters 0', on Das in standard classification manner. In the remainder of this section, we introduce our strategies to exploit the resulting teacher to improve both the classification and localization accuracy of the student detector F., with parameters 8*

---

3.1 KD Knowledge Distillation for Classification
Our first approach to caassffie-tt-detectte distillation focuses on the classification accuracy of the student network. To this end, we make use of the class-wise probability distributions obtained by the teacher and the student, softened by making use of a temperature parameter T. Below, we first derive our general formulation for distillation for classification, and then discuss in more detail how we obtain the teacher and student class distributions for the two types of classification losses commonly used by object detection frameworks.
Formally, given positive anchor boxes or object proposals, which are assigned with one of the ground-trune truth labels and bounding boxes during training, let f denote the vector of softened class probabilities for box from the student network, obtained at temperature T, and let Ps f.7' denote the corresponding softened probability vector from the teacher network. We express knowledge distillation for classification as a loss function measuring the Kullback-Leibler (KL) divergence between the teacher and student softened distributions. This can be written as
Ckd-cls K
Σ
KL(PK" 1.7
The specific way define the probability vectors Pk
eT and Pkc I. then depends the loss function that the student detector uses for classification. Indeed, existing detectors follow two main trends: some, such Faster RCNN and SSD, exploit the categorical cross-entropy loss with softmax, accouting for the C' foreground classes and background one; others, such as RetinaNet, employ a form of binary cross-entropy loss with sigmoid focusing only the C' foreground classes, Let us now discuss these two cases in more detail.
Categorical cross-entropy. In this case, for each positive object bounding box k, the student detector outputs logits zt E (C + 1). We then compute the corresponding softened probability for class c with temperature T
Pk TT (c
2j=1 FighT
where =R, denote the logit corresponding to class By contrast, as our teacher is C-way classifier, it produces logits zf, C. We thus compute its softened probability for class
p (cl0*)
(3)
ES /T
and, assuming that all true objects should be classified as background with 0 probability, augment the resulting distribution account for the background class as p [ O,.
The KL-divergence between the teacher and student softened distributions for object & can then be written
C+1
KL(P'T TT Il P (.TT) = T Σ
TT - Pk.c logrk.c Pk.c log
TT
1. a, Pk.c
(4)
Binary cross-entropy. The detectors that rely on the binary cross-entropy output a score between and for each of the C' foreground classes, but, together, these scores do not form valid distribution over the C' classes as they do not sum to 1. To nonetheless use them in a KL-divergence measure between the teacher and student, we rely the following strategy. Given the student and teacher C-dimensional logit vectors for an object k, we compute softened probabilities as
Ph (c(0*) )- = (1 + 1 / T 1 (c|0') (1 (1
(5)
We then build a 2-class (False-True) probability distribution for each category according to the ground-truth label 1 of object k. Specifically, for each category e, we write
Pk. = Pi. Pk.c
(6)
In essence, the RetinaNet focal loss follows binary cross-entropy formulation.

---

for the student, and similarly for the teacher. This lets us express the KL-divergence for object & as
T C C
KLIPE LT I1 Pk
log toggh/
1.7 Pk.c (i) log Pk.c (i)
Pk.c
t.7" where Pkc (t) indicates the i-th element of the 2-class distribution Pk.c
3.2
KD Joci Knowledge Distillation for Localization
While, as will be shown by our experiments, knowledge distillation for classification already helps the student detector, it does not aim to improve its localization performance. Nevertheless, localization, bounding box regression, is critical for the success of a detector and is typically addressed by existing setector-to-detector distillation frameworks 5 3 To also tackle this in
dlssiffer-to-detectne approach, we develop feature-level distillation strategy, exploiting the intuition that the intermediate features extracted by the classification teacher from a bounding box produced by the student should match those of the ground-truth bounding box.
Formally, given input image / of size h, let denote by By (#1 1/1 22, 122) the top-left and bottom-right corners of the k-th bounding box produced by the student network. Typically, this is achieved by regressing the offset of anchor box or object proposal. We then make use of Spatial Transformer (20) unit to extract the image region corresponding to Bμ. It is parammmmet differentiable module that links the regressed bounding boxes with the classification teacher to yield an end-to-end model during training. Specifically, we compute the transformer matrix
(I2
21)/w
-1+ (01 + 22)/2
A
(92 y1)/h
-1+ (311 + /2)/h
which allows to extract the predicted object region O with a grid sampling size as
- (sk,,,,,
where fsT denotes the spatial transformer function. As illustrated the right portion of Figure [b). we then perform distillation by comparing the teacher's intermediate features within the predicted object region 0% to those within its assigned ground-truth one 0%
Specifically, for given layer É, we seek to compare the features F (O and (O of the positive box k. To relax the pixel-wise difference between the features, we make use of the adaptive pooling strategy of (28), which produces feature map APFF(()) of fixed size M x W H from the features extracted within region O. We therefore write our localization distillation loss as
K
Cka-ioc loc KLMHW
1
LILAP((((O)) AP(F((
(10)
A=
where K is the number of positive anchor boxes or proposals, L is the number of layers at which perform distillation. 1; is the indicator function to denote whether the layer is used or not to distill knowledge, and ||- |/1 denotes the L1 norm. As both the spatial transformer and the adaptive pooling operation are diffcrentiable, this loss can be backpropagated through the student detector.
Note that, special case, our localization distillation strategy be employed not only intermediate feature maps but on the object region itself (fg), encouraging the student to produce bounding boxes whose underlying image pixels match those of the ground-truth box. This translates to loss function that does not exploit the teacher and can be expressed as
(PP KMHW Σ [AA((O)) AP(O
(11)
Depending on the output size of the adaptive pooling operation, this loss function encodes a more- or-less relaxed localization error. As will be shown by our experiments, it can serve an attractive complement to the standard bounding box regression loss of existing object detectors, whether using distillation not.
5

---

Table Analysis our classifierto-dettector distillation method with compact students on the COC02017 validation set. R50 is ResNet50, MV2 is MobileNetV2, QR50 is quartered ResNet50.
Method
mAP
APso
AP
AP,
AP, m
AP;
mAR
AR.
ARm
AR;
SSD300-VGG16
25.6
43.8
26.3
6.8
27.8
42.2
37.6
12.5
41.7
58.6
KDr
(1
45.2
27.2
7.3
28.5
43.6
38.4
12.8
42.6
59.1
KD
27 1.5)
43.2
28.4
7.5
29.4
43.3
40.0
13.4
44.4
60.6
KDtte
27.2 ( 1.6)
43.3
28.5
7.5
29.5
43.5
40.2
13.2
44.7
61.5
+ KData K.D
27.9 2.3)
45.1
29.2
8.1
30.1
45.4
40.4
13.9
44.7
61.4
SSDS12-VGG16
29.4
49.3
31.0
11.7
34.1
44.9
42.7
17.6
48.7
60.6
KD
(1
51.1
31.7
12.7
34.6
45.5
43.3
19.4
49.0
60.4
KD
30.8 1.4)
48.8
32.9
12.8
35.8
46.2
44.7
18.8
51.1
63.4
KDoee
31
32.8
12.6
35.8
46.2
45.0
18.9
51.6
63.2
KDals KiDto
32.1 217)
51.0
34.0
13.3
36.6
47.9
45.3
20.1
51.2
Faster RCNN-QR50
23.3
40.7
23.9
13.1
25.0
30.7
40.2
22.7
42.8
51.8
+ -DD,a
21.9
45.5
26.2
15.3
27.9
34.0
42.8
25.5
46.0
54.9
+ KD
(1
41.1
25.0
13.7
25.8
32.1
41.7
23.8
44.3
54.8
+ -KDiee
24.3 (1 1.0)
41.0
25.1
13.0
25.9
32.5
41.6
22.7
44.6
54.7
+ KDD K.Doe
27.2 (1.))
46.0
27.7
15.2
29.3
36.2
44.5
25.9
48.1
58.3
Faster RCNN-MV2
31.9
52.0
34.0
18.5
34.4
41.0
47.5
29.7
50.9
60.4
-D
(2
53.3
34.6
18.9
34.8
42.3
48.1
29.7
51.2
61.5
KD
0.33)
51.9
34.2
34.4
47.9
29.0
50.8
KDDoe
32.3 0.4)
52.0
34.7
34.8
48.0
28.7
51.3
KD,,, KDooo
32.7 (1 0.8)
52.9
35.0
19.0
35.0
42.9
48.4
29.9
51.8
61.9
3.3 Overall Training Loss
To train the student detector given the image classification teacher, then seek minimize the overall loss
C Ldet + AkcCkd-ols + >ggCad-toc
(12)
where Latet encompasses the standard classification and localization losses used to train the student detector of interest. Ake and drt hypee--arameters setting the influence of each loss.
Experiments
In this section, first conduct full study of our classification and localization distillation methods on several compact detectors, and then compare our classifier-to-des
approach to the state-of- the-art ditector-to-detecte ones. Finally, perform extensive ablation study of our method and analyze how improves the class recognition and localization in object detection. All models are trained and evaluated on MS COC02017 (23), which contains 1 118k images for training and 5k images for validation (minival) depicting 80 foreground object classes. Our implementation is based on MMDetection 6 with Pytorch [29] Otherwise specified. we take the ResNet50 as the classification teacher. We will use the same teacher for all two-stage Faster RCNNs and one-stage RetinaNets in our Maussifer-to-detectt our method, since it lets us use the same teacher for multiple detectors. To train this classification teacher, we use the losses from Faster RCNN and RetinaNet frameworks jointly. Since SSDs use different data augmentation, we train another ResNet50 classification teacher for them. Additional experimental details on how to train our classification teachers are provided in supplementary material.
distillation method. We consider this be an advantage of
4.1 Classifierto-Deetccte Distillation Compact Students
We first demonstrate the effectiveness of our classifier-to-detedd distillation method on compact de- tectors, namely, SSD300. SSD512 26 and the woostage Faster RCNN (32 detector with lightweight backbones, i..., MobileNetV2 B51 and Quartered-Rer ResNet50 (QR50), obtained by dividing the number
of channels by 4 in every layer of ResNet50, reaching 66.33% top-1 accuracy on ImageNet (34
Experimental setting. All object detectors are trained in their default settings on Tesla V100 GPUs. The SSDs follows the basic training recipe in MMDetection (6). The lightweight Faster RCNNs

---

trained with x training schedule for 12 epochs. The details for the training settings of each model are provided in the supplementary material. We use ResNet50 with imput resolution 112 112 as classification teacher for all student detectors. We report the mean average precision (mAP) and mean average recall (mAR) for intersection over unions (IoUs) in [0.5:0.95], the APs loU=0.5 and 0.75, and the APs and ARs for small, medium and large objects.
Results. The results are shown in Table Our classification distillation yields improvements of at least 0.7 mAP for all student detectors. It reaches 2.6 mAP improvement for Faster RCNN-QR50, which indicates that the classification in this model is much weaker. The classification distillation improves AP more than APrs, while the localization distillation improves AP7s more than APs0. As increasing AP75 requires more precise localization, these results indicate that each of our distillation losses plays its expected role. Note that the SSDs benefit more from the localization method than the Faster RCNNs. We conjecture this be due to the denser, more accurate proposals of the Faster RCNNs compared to the generic anchors of the SSDs. Note also that Faster RCNNs with smaller backbone benefits more from our distillation than larger one.
4.2 Comparison with Delector-to-detector Distillation
We then compare our classifier-to-detcctor distillation approach with the of-th-of- detector-to. detector ones, such KD [5] FGFI (39]. GID [8] and FKD 43). Here, in addition to the compact students used in Section 4.1 we also report results on the larger students that are commonly used in the literature, i.e., Faster RCNN and RetinaNet with deeper ResNet50 (R50) backbones.
Experimental setting. Following 43,, the Faster RCNN-R50 and RetinaNet-R50 are trained with 2x schedule for 24 epochs. To illustrate the generality of our approach, we also report the results of our distillation strategy used in conjunction with FKD (3), one of the current best detector-to- detector distillation methods. Note that, while preparing this work, we also noticed the concurrent work of [12). whose DeFeat method also follows detector-to-deccen distillation approach, and thus could also be complemented with out strategy.
Results. We report the results in Table 2 For compact student detectors, such as Faster RCNN-QR50 and SSD512, our classifier-to- detector distillation surpasses the best detector- to-detector one by 1.1 and 0.9 mAP points, re- spectively. For student detectors with deeper backbones, our method improves the baseline by 0.8, 0.4 and 0.5 points. Furthermore, us- ing in conjunction with the FKD detector-to- detector distillation method boosts the perfor- mance to the state-of-the-ar of 28.0, 32.6, 34.2, 41.9 and 40.7 mAP. Overall, these results ev- idence that our approach is orthogonal to the setector-to-detecte distillation methods, allow- ing us to achieve state-of-the performance by itself by combining it with detector-to- detector distillation strategy.
Table 2: Comparison to detector-to-detector distil- lation methods on the COC02017 validation set.
Method
mAP
AP.
AP
AP:
Faster RCNN-QR50
23.3
13.1
25.0
30.7
+ +KKD [43
26.1
14.6
27.3
35.0
+ Ours
27.2
15.2
29.3
36.2
+ Ours FKD
28.0
15.4
29.8
38.5
SSD512-VGG16
29.4
11.7
34.1
44.9
FKK 43
31.2
12.6
37.4
46.2
Ours
32.1
13.3
36.6
47.9
Ours FKD
32.6
13.5
37.6
48.3
Faster RCNN-MV2
31.9
18.5
34.4
41.0
+ FKD 433
33.9
18.3
36.3
45.4
+ Ours
32.7
19.0
35.0
42.9
+ Ours PKD
34.2
18.5
36.3
45.9
Faster RCNN-R50
38.4
21.5
42.1
50.3
KD 5
38.7
22.0
41.9
51.0
4.3 Ablation Study
FGFI (3
39.1
22.2
42,9
51.1
GID S
40.2
22.7
44.0
53.2
FKD
41.5
23.5
45.0
55.3
In this section, we investigate the influence of the hyper-parameters and of different classifica- tion teachers in our approach. To this end, we use the SSD300 student detector.
Ours
38.8
22.5
42.5
50.8
Ours FKD
41.9
23.8
45.2
56.0
RetinaNet-R500
37.4
20.0
40.7
49.7
+ FGFI 59
38.6
21.4
42.5
51.5
Ablation study of KD We first study the effect of the loss weight Aie and the temperature for classification distillation. As shown in Table 3al these two hyper-parameters have mild impact on the results, and we obtain the best results with Age = 0.4 and T used for all other experiments with SSDs.
GID 8
39.1
22.8
43.1
52.3
FKD 4R3
39,6
22.7
43.3
52.5
Ours
37.9
20.5
41.3
50.5
Ours +FKD
40.7
23.1
44.7
53.8
2, which were
7

---

Table Ablation study of cisss We evaluate the impact of the classification teachers on our classification distillation.
and of various
(a) Varying Ake and T.
(b) Varying the teacher network.
Ake
T
mAP
AP
AP75
Teacher
Top-1
mAP
AP50
AP1s
baseline
/
25.6
43.8
26.3
ResNer18
75.78
25.9
44.4
26.4
ResNet50
80.30
26,3
45,2
27.2
0.1
25.8
44.2
26.6
ResNeXt101
83.35
25.3
43.3
25.8
0.1
2
25.4
44.4
25.7
0.2
25.8
44.2
26.6
Input size
Top-1
mAP
AP5o
AP1s
0.3
26.0
44.6
26.7
56 56
76.26
26.2
44.8
26.9
0.4
26.1
44.8
26.6
112 x 112
80.30
26.3
45.2
27.2
0.4
26.3
45.2
27.2
224 x 224
80.41
26.2
44.9
26.9
0.4
26.0
45.2
26.7
Table 4: Ablation study of KD loc. We investigate the effect of the sampling size, the pooling size and the choice of distilled layers on our localization distillation.
(a) Varying the sampling size.
(b) Varying the pooling size.
(c) Varying distilled layers.
Sampling size
mAP
AP
AP7s
Pooling size
mAP
APso
AP7a
Eo
mAP
14 x 14
26.4
43.0
27.0
2
26,6
43.5
27.5
27.1
28 x 28
26.7
43.2
27.8
4
27.0
43.5
28.1
26.8
56 x 56
26.8
43.3
28.0
8 x 8
27.1
43.2
28.4
27.2
112 x 112
27.0
43.5
28.1
16 x 16
26.9
42.8
28.1
/
/
/
26.9
224 x 224
27.0
43.4
28.2
We then investigate the impact of different classification teacher networks. To this end, we trained three teacher networks ranging from shallow deep: ResNet1 8, ResNet50 and ResNext101-32xx 8d. We further study the impact of the input size to these teachers on classification distillation, using the three sizes [56 56, 112 112, 224 224]. As shown in Table 3bb even the shallow ResNet18 classification teacher can improve the performance of the student detector by 0.3 points, and the improvement increases by another 0.4 points with the deeper ResNet50 teacher. However, the performance drops with the ResNeXt101 teacher, which is the teacher with the highest top- 1 accuracy. This indicates that deeper teacher is not always helpful, additional information compared to the ground ground-truth truth labels. As for the input size, we observe only small variations across the different sizes, and thus use size of 1 in all other experiments.
might be overconfident bring much
Ablation study of KD Dooce We then evaluate the influence of the two main hyper-parameter of localization distillation, i.e., the grid sampling size of the spatial transformer and the adaptive pooling size of the feature maps. To this end, we vary the sampling size in [14, 28, 56, 112, 224] and the pooling size in 2, 4,8 8 8, 16 x 16).
As shown in Table 4a our localization distillation method benefits from a larger sampling size, although the improvement saturates after size of 112. This lets us the same classification teacher, with input size 112, for both classification and localization distillation. The adaptive pooling size has milder effect the performance, as shown in Table 4b, with size of 8 yielding the best mAP. In our experiments, we adopt either 4 or 8, according to the best performance on the validation set.
We further study the layers to be distilled in our localization distillation. To this end, we extract features from the first convolutional layer é, and from the following bottleneck block & of the ResNet50 teacher. As shown in Table 4c| distilling the knowledge of only the object regions (En) yields better mAP than using the & features. However, combining the object regions (fn) with feature maps from l improves the results. Adding more layers does not help, which conjecture to be due to the fact that these layers extract higher-level features that are thus less localized.
4.4 Analysis
To further understand how our classifier-to-detector distillation method affects the quality of the classification and localization, in Table we report the APs obtained with loUs in [0.5, 0.95 with a step of 0.05. These results highlight that our classification and localization distillation strategies behave differently for different IoU thresholds. Specifically, KD yields larger improvements

---

Table APs for IoUs ranging from 0.5 to 0.95 on the COC02017 validation set.
Method
mAP
50
55
APai
AP 66
AP
AP7s
APsu
APss
AP
APgs
SSD300
25.6
43.8
41.3
38.4
35.1
31.2
26.3
20.3
13.0
5.2
0.5
K.Dds
26.3
45.2
42.6
39.9
36.1
31.6
27.2
21.0
13.5
5.1
0.5
KD
27.2
43.3
41.3
38.8
36.0
32,9
28.5
23.0
16.5
8,4
1.3
KDats + KDicc
27.9
45.1
42.8
40.2
37.0
34.0
29.2
23.9
17.0
8.8
1.2
Baseline
Ours
(a)
(d)
Figure 3: Qualitative analysis (better viewed in color). The ground-truth bounding boxes are in blue with their labels, and the predictions are red with predicted labels and confidence.
for smaller IoUs, whereas KD1oe more effective with IoUs larger than 0.75. This indicates that KDioe indeed focuses on precise localization, while KD distills category information. The complementarity of both terms further evidenced by the fact that all APs increase when using both of them jointly.
Detection error analysis. We analyze the dif- ferent types of detection errors using the tool proposed by Bolya al. 3 for the baseline SSD300 and the distilled models with our KD and KD We focus on the classification and localization errors, which are the main errors in object detection. The details of all error types are provided in the supplementary material. As shown in Figure 2 KDeis decreases the clas- sification eтror especially for IoUs smaller than 0.65. By contrast, as shown in Figure [p, the effect of KD increases with the loU. This again shows the complementary nature of these terms.
Baseline
30
0.5
0.6
0.8
0.5
0.7
100
lou
(a) Classification error.
(b) Localization error.
Figure 2: Detection error analysis.
Qualitative analysis. Figure compares the detection results of the baseline model and of distilled model on a few images. We observe that (i) the bounding box predictions of the distilled model are more precise than those of the baseline; (ii) the distilled model generates higher confidences for the correct predictions and is thus able to detect objects that were missed by the baseline, such as the boat in Figure and the giraffe in Figure 3..
5 Conclusion
We have introduced a novel approach to knowledge distillation for object detection, replacing the standard detector-to-detector strategy with a classifierto-ddetcttor one. To this end, have developed classification distillation loss function and a localization distillation one, allowing to exploit the classification teacher in two complementary manners. Our approach outperforms the state-of-the oh- art peneetoo-to--etectoe ones on compact student detectors. While the improvement decreases for larger student networks, our approach can nonetheless boost the performance detector-to-detectoo distillation. We have further shown that the same classification teacher could be used for all student detectors if they employ the same data augmentation strategy, thus reducing the burden of training separate teacher for every student detector. Ultimately, believe that work opens the door to

---

new approach to distillation beyond object detection: Knowledge should be transferred not only across architectures, but also across tasks.
Broader impact
Knowledge distillation is a simple yet effective method to improve the performance of a compact neural network by exploiting the knowledge of more powerful teacher model. Our work introduces a general approach to knowledge distillation for object detection to transfer knowledge across architectures and tasks. Our approach enables distilling knowledge from single classification teacher into different student detectors. As such, our work reduces the need for separate deep teacher detector for each student networks; therefore, we reduce training resources and memory footprint. As we focus on compact networks, our work could significantly impact applications in resource-constrained environments, such as mobile phones, drones, or autonomous vehicles. We do not foresee any obvious undesirable ethical/social impact at this moment.
Acknowledgments and Disclosure of Funding
This work was supported in part by the Swiss National Science Foundation and by NVIDIA during an internship. We would like to thank Maying Shen at NVIDIA for the valuable discussions and for rumning some of the experiments. We also thank NVIDIA for providing their excellent experiment platform and computing resources.
References
[1] J. M. Alvarez and M. Salzmann. Learning the number of neurons in deep networks. In Advances in Neural Information Processing Systems. 2016.
[2] M. Alvarez and M. Salzmann. Compression-awee aware training of deep networks. In Advances in Neural Information Processing Systems. 2017.
[3] D. Bolya, S. Foley, J. Hays, and J. Hoffman. TIDE: A general toolbox for identifying object detection errors. In European Conference Computer Vision, 2020.
[4] Z. Cai and N. Vasconcelos. Cascade R-CNN: Delving into high quality object detection. In Conference Computer Vision and Pattern Recognition, 2018.
[5] G. Chen, W. Choi, X. Yu, T. Han, and M. Chandraker. Learning efficient object detection models with knowledge distillation. In Advances in Neural Information Processing Systems, 2017.
[6] K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, Shi, W. Ouyang, C. C. Loy, and D. Lin. MMDetection: Open mmlab detection toolbox and benchmark. arXiv Preprint, 2019.
[7] M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1, arXiv Preprint, 2016.
[8] X. Dai, Z. Jiang, Z. Wu, Y. Bao, Z. Wang, Liu, and Zhou. General instance distillation for object detection. arXiv Preprint, 2021.
[9] K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, and Q. Tian. CenterNet: Keypoint triplets for object detection. International Conference on Computer Vision, 2019.
[10] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal 2007.
10

---

[11] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.paseal. www.oot.orgccallegss//////o22000 2/wrsssho//wwwksssonnddnee html, 2012.
[12] Guo, K. Han, Y. Wang, H. Wu, Chen, C. Xu, and C. Xu. Distilling object detectors via decoupled features. arXiv Preprint, 2021.
[13] S. Han, H. Mao, and W. J. Dally. Deep Compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In International Conference on Learning Representations, 2016.
[14] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition, 2016.
[15] K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask R-CNN. In International Conference on Computer Vision, 2017.
[16] T. He, C. Shen, Z. Tian, D. Gong, Sun, and Y. Yan. Knowledge adaptation for efficient semantic segmentation. In Conference on Computer Vision and Pattern Recognition, 2019.
[17] B. Heo, J. Kim, S. Yun, Park, N. Kwak, and Choi. comprehensive overhaul of feature distillation. In International Conference on Computer Vision, 2019.
[18] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv Preprint, 2015.
[19] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv Preprint, 2017.
[20] M. Jaderberg. K. Simonyan, A. Zisserman, and k. kavukcuoglu. Spatial transformer networks.
In Advances in Neural Information Processing Systems, 2015.
[21] H. Law and Deng. CornerNet: Detecting objects as paired keypoints. In European Conference on Computer Vision, 2018.
[22] N. Lee, T. Ajanthan, and H. Torr. SNIP: Single-shot network pruning based on connection
sensitivity. In International Conference on Learning Representations, 2019.
[23] T.-Y. Lin, M. Maire, Belongie, Hays, P. Perona, Ramanan, Dollar, and L. Ziticck. Microsoft COCO: Common objects in context. In European Conference Computer Vision, 2014.
[24] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In Conference on Computer Vision and Pattern Recognition, 2017.
[25] T.-Y. Lin, P. Goyal, R. B. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. International Conference Computer Vision, 2017.
[26] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. SSD: Single shot multibox detector. In European Conference on Computer Vision, 2016.
[27] Y. Liu, K. Chen, C. Liu, Z. Qin. Z. Luo, and J. Wang. Structured knowledge distillation for semantic segmentation. In Conference on Computer Vision and Pattern Recognition, 2019.
[28] B. McFee, J. Salamon, and J. Bello. Adaptive pooling operators for weakly labeled sound event detection. JEEF/ACM Transactions on Speech and Language Processing, 26(1 1)22000-11 2018. ISSN 2329-9290.
[29] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, 7. DeVito, Z. Lin, Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in PoTorch. 2017.
11

---

[30] M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. XNOR-Net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision, 2016.
[31] J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv Preprint, 2018.
[32] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems, 2015.
[33] A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio. FitNets: Hints for thin deep nets. arXiv Preprint, 2014.
[34] O. Russakovsky, J. Deng, H. Su, Krause, S. Satheesh, Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg. and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 2015.
[35] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. MobileNerV2: Inverted residuals and linear bottlenecks. In Conference on Computer Vision and Pattern Recognition, 2018.
[36] Y. Tian, D. Krishnan, and P. Isola. Contrastive representation distillation. In International Conference on Learning Representations, 2020.
[37] Z. Tian, Shen, H. Chen, and T. He. FCOS: Fully convolutional one-stage object detection. In International Conference on Computer Vision, October 2019.
[38] K. Ullrich, E. Meeds, and M. Welling. Soft weight-sharing for neural network compression. In International Conference on Learning Representations, 2017.
[39] Wang, L. Yuan, X. Zhang, and J. Feng. Distilling object detectors with fine-grained feature imitation. In Conference Computer Vision and Pattern Recognition, 2019.
[40] Z. Yang, S. Liu, H. Hu, L. Wang, and S. Lin. RepPoints: Point set representation for object detection. In International Conference on Computer Vision, 2019.
[41] J. Yim, D. Joo, J. Bae, and J. Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In Conference on Computer Vision and Pattern Recognition, 2017.
[42] S. Zagoruyko and N. Komodakis. Paying More Attention to Attention: Improving the perfor- mance of convolutional neural networks via attention transfer. In International Conference on Learning Representations, 2017.
[43] L. Zhang and K. Ma. Improve object detection with feature-based knowledge distillation: Towards accurate and efficient detectors. 2021.
[44] R. Zhao, Y. Hu, J. Dotzel, C. De Sa, and Z. Zhang. Improving neural network quantization without retraining using outlier channel splitting. In International Conference Machine Learning, 2019.
12


Reviews:
[{"review_id": "wMtNrRywJC0", "reviewer": "Reviewer_ML2n", "summary": "This paper designs a new framework for distillation in object detection tasks. It claimed itself as a new method for opening the cross-task distillation directions. However, it is a bit overstated.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a nice paper with clear illustrations and structures. I enjoy reading it.\n\nA big issue of this work is that its significance is overstated (\"Ultimately, we believe that our work opens the door to a new approach to distillation beyond object detection: Knowledge should be transferred not only across architectures, but also across tasks.\") because we can see from table 1, the localization distillation loss almost does not work (mostly only 0.1% mAP improved). What works for the final performance boost is the classification distillation loss which is basically applied between the classifier of the student and the teacher (which is also a classifier) --- the meaningful distillation happens only between the teacher classifier and the student classifier but not between the teacher classifier and student detector. This means that the experimental results in this paper do not support its assumption or motivation. \n\nOther detailed comments:\n\n1. Performance is not impressive compared to related works. From table 2, we can see that \"ours\" exceeds FKD for 1% when feature plugin is not possible (in below two blocks) and all its other performances (+Ours in the top three blocks) are lower than FKD. \n\n2. Lines 107-109. This is not very interesting actually, as it trains a classifier and distill the knowledge to another classifer (the loc distill not working as mentioned above). While, it would be interesting to see how can apply a normally trained classifier (does not require such a cropping in the images using bounding box labels) for distilling a good detector. The way is to use a pre-trained classifiers (on ImageNet) or a normally trained classifier on target dataset such as multilabel classifiers trained on the weakly supervised detection dataset (i.e., bbox can not be used in training classifiers). \n\n3. Lines 38, 39. It is not significant if distilling is only between classification and detection models as the only arch difference is a regression head in the last layer (output layer). I would be more agreed on the significance if it is from a classification model to a segmentation model where the latter needs a very different arch (an additional multi-layer net to decode a mask).", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5 hours"}, "extra_text": {}}, {"review_id": "HJmTSB2xLI4", "reviewer": "Reviewer_q5n7", "summary": "The paper proposes a method for distilling a low-compute object detector. Contrary to prior work, which used a more standard object detector-to-object detector distillation approach (where object detectors were most often restricted to be of the same category), the authors propose to use a classification backbone as teacher. To do this, the classifier is trained on the ground truth bounding boxes, and then used to impose both a classification head distillation, and a localization head distillation. The classification distillation is rather straightforward (using a logit matching approach), while the localization distillation is more involved. A spatial transformer is used on top of the bounding box prediction, and a feature matching loss is imposed so that the predicted bbox -> transfomer -> adaptive pooling branch, and the ground truth bbox -> classifier teacher -> adaptive pooling branch are similar to each other.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The premise of the paper is clear, and proposes a scenario that is novel (classifier to detector distillation). The proposed scenario might also have some interesting practical implications, though this is not really discussed much in the paper. In the end, the classifier is trained on the dataset itself, why is this better than using an integrated localization and classifier model? I understand being able to use a large-scale classification dataset would be an advantage, but why does it work better given that the method trains on the detection dataset?\n\nThe technical approach is split into two: the classifier distillation itself and the localization distillation (which uses two losses). The first bit is very standard and it does not have any novelty (classifer-to-detector scenario aside). The second is more novel, but I have questions about it (see comments bellow). \n\nThe paper is mostly well written, but in my opinion would benefit from a re-balancing of what the limited space is devoted to: the details of the classifier distillation are very standard and in some cases could be omitted altogether (is ok to leave softmax computation out, even with temperature, eq. 5 and 6 could occupy less space - particularly 6 could be removed, etc.). Instead, the localization distillation is much more vague. For example, the spatial transformer bit will not make the idea clear to someone that is not familiar with the spatial transformer, and the adaptive pooling is simply not introduced at all - and this is far from a standard technique.\n\nTechnically speaking, the method mixes simple concepts with some that are more obscure to me. The localization distillation is very unclear actually. The spatial transformer is introduced in-between the output of the localization of the student network, and the second bit of the distillation strategy, that compares the output of the spatial transformer to the classification feature. But what confuses me the most is, unless you actually keep the spatial transformer at test time on top of the bounding box prediction (like an alternative bounding box regression), then the only way it can influence the parameters involved at test time is very limited. This had me quite confused during section 3.2, thinking I misunderstood what the authors were proposing, but then I saw that the impact is very limited (ablation study on Table 1). Instead, the localization branch learns through the second localization loss (termed \"loc\" in the paper), which does not involve a teacher at all. So, to sum up, I'm not sure the contributions as claimed are the relevant bits, and I'm a bit confused of why a term that does not involve a teacher at all has such positive effect - this sounds like a general-purpose module, is it novel? \n\nIn terms of results, Ours + FKD offers good improvements over \"Ours\". Why is it not shown for every student network considered?\n\nI'm open to upgrading the score if the rebuttal clarifies the methodological doubts I expressed above.\n\nIt would be nice to see the results when using 1x, 2x, 4x scheduler. KD can make models train faster, but maybe not better if trained for long enough?\n\n\nSmall issues (no need to address them on rebuttal):\n\n\"Heo et al. [17] provides a comprehensive study of feature distillation.\" This is not literature review, it is taking the title of the paper and making it a phrase. No insight provided.\n\nSome details on how the classifier is trained would be welcome. e.g. the heuristics of patch cropping, augmentations, schedule, hyperparameters, etc\n\nReferences need a bit more care. For example, Sec. 4.1 the reference provided for ImageNet has nothing to do with the dataset creation (it is \"just\" a method to classify on imagenet). SSD and Faster RCNN are introduced before but it would not be bad to give the reference here too, but MobileNetV2 and QR50 do not have a reference (not sure where the QR50 comes from either). And for example it's MMDetection, not mmdetection.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors claim that they discuss the limitations on Section 5. This is not really the case, no such discussion actually happens.\n\nSocietal impact is discussed... although it is more of a variant of the conclusions. I am however of the opinion that no specific negative societal impact should be highlighted for this work as it is a rather generic approach with applicability to any object detection method.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "C6_qTEUkIBd", "reviewer": "Reviewer_twvd", "summary": "The paper aims to transfer knowledge from image classifiers to object detectors using distillation. The teacher and student models are trained on the same dataset, and distillation is performed from teacher to student via classification and localization components. The said model improves on the non-distilled baseline student detectors, and also compete with prior related detector-to-detector distillation works. The ablation study demonstrates the effectiveness of the loss terms and also shows the complementary nature of the classification and localization distillation loss terms.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\n- The claims made in the paper are appropriate and are substantiated by the experimental results and ablation study\n\n\n### Weaknesses\n\nThe problem setup\n- Although not exactly a weakness, but the idea of transferring knowledge from image classifiers to object detectors touches upon another line of work where an explicit transformation / mapping function is learned from classifiers to detectors [1, 2], which can then be readily deployed on novel classes for which the detection data is scarce, making the problem highly desirable. However the problem explored in this paper is distillation primarily and obtaining lower memory and compute intensive models (student) that are able to deliver better accuracy. My question is did the authors think about using less detection data and still be able to manage comparable performance to SOTA? That, in my opinion, would be even more impactful.\n\n### Typo\n\n- Line 49: ’be’ —> ‘by’\n- Line 215: ’reaches a 2.3 mAP improvement’. Shouldn’t it be 2.6?\n\n### Justification for the rating\n\nAlthough I’m not super familiar with this line of research, however in my opinion the primary incentive for distillation is to obtain better performing compact students which the paper has empirically demonstrated. I’m leaning towards accept, however I’m keen to hear what the other reviewers might have to say to inform my final rating.\n\n\n[1] Hoffman, Judy, et al. \"LSDA: Large scale detection through adaptation.\" Advances in neural information processing systems 27 (2014): 3536-3544.\n\n[2] Wang, Yu-Xiong, Deva Ramanan, and Martial Hebert. \"Meta-learning to detect rare objects.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Two of the reviewers agreed that the paper is novel and can result in cheaper, higher-performing object detectors. The third reviewer misunderstood the term "spatial transformer" and was confused by its use, not responding to the authors' clarification. Given the strong experimental result and interesting approach, I support the acceptance of this paper.
Decision: Accept (Poster)