Most object detectors contain two important components: a feature extractor and an object classifier. The feature extractor has rapidly evolved with significant research efforts leading to better deep convolutional architectures. The object classifier, however, has not received much attention and many recent systems (like SPPnet and Fast/Faster R-CNN) use simple multi-layer perceptrons. This paper demonstrates that carefully designing deep networks for object classification is just as important. We experiment with region-wise classifier networks that use shared, region-independent convolutional features. We call them "Networks on Convolutional feature maps" (NoCs). We discover that aside from deep feature maps, a deep and convolutional per-region classifier is of particular importance for object detection, whereas latest superior image classification models (such as ResNets and GoogLeNets) do not directly lead to good detection accuracy without using such a per-region classifier. We show by experiments that despite the effective ResNets and Faster R-CNN systems, the design of NoCs is an essential element for the 1st-place winning entries in ImageNet and MS COCO challenges 2015. Index Terms-Object detection, CNN, convolutional feature map Ç 1 INTRODUCTION MOST object detectors contain two important components: a feature extractor and an object classifier. The feature extractor in traditional object detection methods is a hand-engineered module, such as HOG [1]. The classifier is often a linear SVM (possibly with a latent structure over the features) [2], a non-linear boosted classifier [3], or an additive kernel SVM [4]. Large performance improvements have been realized by training deep ConvNets [5] for object detection. R-CNN [6], one particularly successful approach, starts with a pre-trained ImageNet [7] classification network and then fine-tunes the Con-vNet, end-to-end, for detection. Although the distinction between the feature extractor and the classifier becomes blurry, a logical division can still be imposed. For example, an R-CNN can be thought of as a convolutional feature extractor, ending at the last pooling layer, followed by a multi-layer perceptron (MLP) classifier. This methodology, however, appears rather different from traditional methods. A research stream [8], [9], [10], [11] attempting to bridge the gap between traditional detectors and deep ConvNets creates a hybrid of the two: the feature extractor is "upgraded" to a pre-trained deep ConvNet, but the classifier is left as a traditional model, such as a DPM [8], [9], [10] or a boosted classifier [11] . These hybrid approaches outperform their HOG-based counterparts [2], [3], but still lag far behind R-CNN, even when the hybrid model is trained end-to-end [10] . Interestingly, the detection accuracy of these hybrid methods is S. Ren is with the
Networks on the convolutional feature maps(NoCs) REF extracts features with a fixed pre-trained deep CNN and explores different networks on the convolutional feature maps as object classifiers.
Object Detection Networks on Convolutional Feature Maps
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Pattern Analysis and Machine Intelligence
Computer Science Medicine