Training Domain Speciﬁc Models for Energy-Efﬁcient Object Detection
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We propose an end-to-end framework for training domain speciﬁc models (DSMs)to obtain both high accuracy and computational efﬁciency for object detectiontasks.
DSMs are trained with distillation [1] and focus on achieving high accuracyat a limited domain (e g. ﬁxed view of an intersection).
We argue that DSMscan capture essential features well even with a small model size, enabling higheraccuracy and efﬁciency than traditional techniques. In addition, we improve thetraining efﬁciency by reducing the dataset size by culling easy to classify imagesfrom the training set.
For the limited domain, we observed that compact DSMssigniﬁcantly surpass the accuracy of COCO trained models of the same size.
Bytraining on a compact dataset, we show that with an accuracy drop of only 3.6%,the training time can be reduced by 93%.
Introduction
There has recently been a signiﬁcant improvement in accuracy for object detection tasks due tothe development of convolutional neural networks (CNN).
Such advances motivate implementing
CNN-based object detection in surveillance cameras to enhance safety of cities, homes, ofﬁces andfactories.
However, the system energy efﬁciency is a key requirement because modern surveillancecameras demand battery operation, to ease installion.
While deeper CNNs achieve better accuracyin computer vision [2], but the increased layers increases inference energy; a clear tradeoff betweenaccuracy and efﬁciency exists.
Many successful approaches to improve the efﬁciency of image classiﬁcation have been proposed,such as model compression[3] and model cascades with domain speciﬁc models [4].
However,object detection is more complex than imagine classiﬁcation, and while these techniques are likely toremain effective, there is need for additional methods.
This papers describes a distillation approach[1][5] for object detection which reduces the accuracy gap between student and teacher models bytraining the student using a restricted class of domain speciﬁc images.
This paper’s contribution is summarized below.
(cid:15)
We propose an end-to-end framework for training domain speciﬁc models (DSMs) to mit-
igate the tradeoff between object-detection accuracy and computational efﬁciency.
The
framework consists of dataset setup and training, and DSMs are small object detectionmodels trained to achieve high accuracy only for a limited domain (e.g. ﬁxed view of anintersection).
To the best of our knowledge, this is the ﬁrst successful demonstration of training DSMs for object detection tasks.
(cid:15)
By training resnet18-based Faster-RCNN DSMs, we observed a 19.7% accuracy (relativemAP) improvement compared to COCO trained model of the same size, tested on a cus-
tomized Youtube Live dataset.
Require: Domain Speciﬁc Model (DSM), Teacher Model 1: procedure 1.
P REPARE D ATASET
Given domain images x i for i = f 0 ; : : : ; N train (cid:0) 1 g 2: for i < N train do 3: label( i )
Teacher .predict ( x i ) .
4: pred( i )
DSM .predict ( x i ) .
5: compute L train ( i ) from label( i ) and pred( i ) .
Collect [ x i ; label( i )] pairs with n largest values of L train .
7: Compile Difﬁcult Dataset (DDS) : Ω =
[ x 0 ; label(0)]) ; : : : ; [( x n (cid:0) 1 ; label( n (cid:0) 1)]) .
8: procedure 2.
T RAIN DSM 9:
DSM .train (Ω) 10: procedure 3.
I NFERENCE 11:
Detection
DSM .predict (image) (cid:15)
We propose culling the training dataset to signiﬁcantly reduce the computation resourcerequired for domain speciﬁc training. Only training data that has high utility in trainingis added. This ﬁltering allows us to reduce training time by 93% with an accuracy loss ofonly 3.6%.
Training Domain Speciﬁc Models2.1 Overview of the framework
Large scale object detection datasets such as COCO[6] contain a large and diverse set of naturalimages.
Using a small model on such a large dataset would typically yield higher misdetectionsthan a large model.
Furthermore, [5] showed that misdetections usually occur between foregroundand background (false positives + true negatives); rarely do misdetections occur as a result of interclass errors.
In video surveillance, because frames in a video stream share a stationary background,a compact model can be good enough to detect foreground and background.
This motivates our
DSM framework to train compact models with dataset constructed by domain-speciﬁc data.
As illustrated in Algorithm 1, our DSM framework consists of preparation of the data and trainingof the DSM.
A large challenge when deploying models in surveillance is preparing the training datasince manually labelling frames in videos is cumbersome.
To overcome this, we label the datasetused to train the DSM by using the predictions of a much larger teacher model with higher accuracyand treating these predictions as ground truth labels.
Furthermore, we compare the prediction onimage x i made by the teacher to that of the DSM; we determine whether to store the x i and label Teacher : predict ( x i ) in our compiled dataset Ω .
After the training set is compiled, it is used to trainthe DSM.
2.2 Constructing the difﬁcult dataset (DDS)
Training a object detection model can take hours even with a GPU and can be challenging forapplications requiring frequent retraining.
We exploit the fact that when the DSM is pretrained onlarge-scale general dataset, it can already provide good predictions for a large chunk of the domainspeciﬁc data.
This procedure develops a compact dataset Ω that is only composed of data that the DSM ﬁnds inconsistent with the prediction made by the teacher model. Keeping data x
j that both
[Table] [Table]
Training Time [min] - - 1.
the DSM and teacher detections are consistent is computationally redundant because it does notcontribute to gradient signals.
We deﬁne L train to quantify the consistency between teacher and DSM:
[Equation]
where TP, FP, FN represents the number of true positive, false positive and false negative boundingbox (BB) detections of the image and ϵ = 0 : 5 .
Signiﬁcantly fewer training data and steps arerequired with only a minimal penalty in accuracy.
ExperimentsModels.
We develop Faster-RCNN object detection models on PyTorch pretrained on MSCOCO[7][8][6].
We use 3 models: resnet101(res101), resnet(res18), and squeezenet(squeeze) asthe backbone region proposal network (RPN)
During training, res101 is used as the teacher,while res18 and squeeze are used as DSMs.
Dataset.
We obtain 5 ﬁxed-angle videos from YouTubeLive .
The video is 2 hours each with 1frames-per-second (fps), consisting of 7200 images.
We split the images evenly: the ﬁrst 3600images are for training and the later 3600 images for testing.
Results.
As shown in table 1, we ﬁrst train our res18 DSM using the full N train = 3600 trainingimages for 10 epochs using stochastic-gradient descent with a learn rate of 10 (cid:0) 4 .
As comparedto the res18 model pretrained on MS-COCO but without domain speciﬁc training, we achieved anaverage of 19.7% accuracy improvement.
Table 2 shows the effectiveness of DDS on res18.
Using DDS, we were able to reduce the trainingtime by 93% (256 images) with only 3.2% accuracy penalty.
If we simply picked 256 trainingimages sequentially (strategy simple on table), the accuracy worsens 10.2% compared to DDS.