Training Domain Speciﬁc Models for Energy-Efﬁcient Object Detection
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We propose an end-to-end framework for training domain speciﬁc models (DSMs)to obtain both high accuracy and computational efﬁciency for object detectiontasks.
DSMs are trained with distillation [1] and focus on achieving high accuracyat a limited domain (e.g. ﬁxed view of an intersection).
We argue that DSMscan capture essential features well even with a small model size, enabling higheraccuracy and efﬁciency than traditional techniques.
In addition, we improve thetraining efﬁciency by reducing the dataset size by culling easy to classify imagesfrom the training set.
For the limited domain, we observed that compact DSMssigniﬁcantly surpass the accuracy of COCO trained models of the same size.
Bytraining on a compact dataset, we show that with an accuracy drop of only 3.6%,the training time can be reduced by 93%
Introduction
Implementing CNN based object detection on stationary surveillance cameras can lead to enhancingthe safety of cities, homes, ofﬁces and factories by detecting unauthorized substances or discoveringanomaly events (e.g. a collapsed person).
However, the computation-efﬁciency is a key requirementsince such devices demand battery operation to ease installation.
Many successful approaches toimprove the efﬁciency of image classiﬁcation have been proposed, such as model compression[2]and model cascades with domain speciﬁc models [3].
However, object detection is more complexthan image classiﬁcation, and while these techniques are likely to remain effective, there is need foradditional methods.
Instead of compressing large models, we target to train a computation-efﬁcient model for each speciﬁc surveillance camera and a framework is proposed to train such domain speciﬁc models (DSM).
The framework is based on knowledge distillation [1][4][5] but targets to reduce the accuracy gapbetween student and teacher models by training the student using a restricted class of domain speciﬁcimages. Since such training may be conducted on edge-devices, we improve the training efﬁciencyby culling easy-to-classify images with small accuracy penalty.
This paper’s contribution is summarized below.
(cid:15)
We propose an end-to-end framework for training domain speciﬁc models (DSMs) to mit-
igate the tradeoff between object-detection accuracy and computational efﬁciency.
To the
best of our knowledge, this is the ﬁrst successful demonstration of training DSMs for objectdetection tasks.
(cid:15)
By training resnet18-based Faster-RCNN DSMs, we observed a 19.7% accuracy (relativemAP) improvement compared to COCO trained model of the same size, tested on a customized YoutubeLive dataset.
Require: Domain Speciﬁc Model (DSM), Teacher Model 1: procedure 1.
P REPARE D ATASET
Given domain images x i for i = f 0 ; : : : ; N train (cid:0) 1 g 2: for i < N train do 3: label( i )
Teacher .predict ( x i ) .
4: pred( i )
DSM .predict ( x i ) .
5: compute L train ( i ) from label( i ) and pred( i ) .
Collect [ x i ; label( i )] pairs with n largest values of L train .
7: Compile Difﬁcult Dataset (DDS) : Ω =
[ x 0 ; label(0)]) ; : : : ; [( x n (cid:0) 1 ; label( n (cid:0) 1)]) .
8: procedure 2.
T RAIN DSM 9:
DSM .train (Ω) 10: procedure 3.
I NFERENCE 11:
Detection
DSM .predict (image)
(cid:15)
Since edge devices will have limited resources, we propose culling the training dataset tosigniﬁcantly reduce the computation resource required for the training.
Only training datathat has high utility in training is added.
This ﬁltering allows us to reduce training time by93% with an accuracy loss of only 3.6%.
Training Domain Speciﬁc Models
Large scale object detection datasets such as COCO[6] contain a large and diverse set of naturalimages.
Using a small model on such a large dataset would typically yield higher misdetectionsthan a large model.
Furthermore, [4] showed that misdetections usually occur between foregroundand background (false positives + true negatives); rarely do misdetections occur as a result of interclass errors.
In video surveillance, because frames in a video stream share a stationary background,a compact model can be good enough to detect foreground and background.
This motivates our
DSM framework to train compact models with dataset constructed by domain-speciﬁc data.
As illustrated in Algorithm 1, our DSM framework consists of preparation of the data and trainingof the DSM.
A large challenge when deploying models in surveillance is preparing the training datasince manually labelling frames in videos is cumbersome.
To overcome this, we label the datasetused to train the DSM by using the predictions of a much larger teacher model with higher accuracyand treating these predictions as ground truth labels.
Furthermore, we compare the prediction onimage x i made by the teacher to that of the DSM; we determine whether to store the x i and label
Teacher : predict ( x i ) in our compiled dataset Ω .
After the training set is compiled, it is used to trainthe DSM.
Training a object detection model can take hours even with a GPU and can be challenging forapplications requiring frequent retraining.
We exploit the fact that when the DSM is pretrained onlarge-scale general dataset, it can already provide good predictions for a large chunk of the domainspeciﬁc data.
This procedure develops a compact dataset Ω that is only composed of data
that the
DSM ﬁnds inconsistent with the prediction made by the teacher model.
Keeping data x j that boththe DSM and teacher detections are consistent is computationally redundant because it does notcontribute to gradient signals.
We deﬁne L train to quantify the consistency between teacher and
DSM:
[Equation]
[Table] [Table] Training Time [min] - - 1. where TP, FP, FN represents the number of true positive, false positive and false negative boundingbox (BB) detections of the image and ϵ = 0 : 5 .
Signiﬁcantly fewer training data and steps arerequired with only a minimal penalty in accuracy.
Experiments
Models.
We develop Faster-RCNN object detection models on PyTorch pretrained on MSCOCO[7][8][6].
We use 3 models: resnet101(res101), resnet(res18), and squeezenet(squeeze) asthe backbone region proposal network (RPN) [9][10].
Res18 and squeeze holds 10% and 19% TOP5 Imagenet error, which is a common accuracy range for compact CNN models like MobileNet [11].
During training, res101 is used as the teacher, while res18 and squeeze are used as DSMs.
Whilewe chose Faster-RCNN for its accuracy on YoutubeLive, we can also use YOLO/SSD detectorsfor improved efﬁciency with this framework because the training requires the bounding box labels[12][13].
Dataset.
We obtain 5 ﬁxed-angle videos from YouTubeLive .
The video is 2 hours each with1 frames-per-second (fps), consisting of 7200 images.
We split the images evenly: the ﬁrst 3600images are for training and the later 3600 images for testing.
Results.
As shown in table 1, we ﬁrst train our res18 DSM using the full N train = 3600 trainingimages for 10 epochs using stochastic-gradient descent with a learn rate of 10 (cid:0) 4 .
As comparedto the res18 model pretrained on MS-COCO but without domain speciﬁc training, we achieved anaverage of 19.7% accuracy improvement.
Table 2 shows the effectiveness of DDS on res18.
Using DDS, we were able to reduce the trainingtime by 93% (256 images) with only 3.6% accuracy penalty.
If we simply picked 256 trainingimages sequentially (strategy simple on table), the accuracy worsens 10.2% compared to DDS.  Appendix4.1 Comparison against Data Distillation [Table]
Data Distillation [5] is fundamentally different from our application setting and methods.
Modelswith large network capacities were shown to achieve higher accuracy by bootstrapping the datasetwith [5].
On the other hand in our framework, in order to fully utilize the small network capacity,we aim to train the models with only the domain speciﬁc data.
We show on Table 3 that following a traditional method of data distillation (i.e. aggregating PASCALwith YoutubeLive data) yields lower rmAP improvement than with our approach of training withonly domain speciﬁc data.
In addition, training the small models with the entire YoutubeLive datasetalso yields lower improvements as well.
This is fundamentally because of the limited model capacityof the compact, but computationally-efﬁcient model.
We observe that for training small models,utilizing a larger dataset do not always obtain better results but restricting the data domain can dobetter.