OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Authors: Michael Mathieu, Yann LeCun, Rob Fergus, David Eigen, Pierre Sermanet, Xiang Zhang
ICLR 2014

Abstract: We present an integrated framework for using Convolutional Networks for classification, localization and detection. We show how a multiscale and sliding window approach can be efficiently implemented within a ConvNet. We also introduce a novel deep learning approach to localization by learning to predict object boundaries. Bounding boxes are then accumulated rather than suppressed in order to increase detection confidence. We show that different tasks can be learnt simultaneously using a single shared network. This integrated framework is the winner of the localization task of the ImageNet Large Scale Visual Recognition Challenge 2013 (ILSVRC2013), and produced near state of the art results for the detection and classifications tasks. Finally, we release a feature extractor from our best model called OverFeat.


Peer review:
[Review]:
The paper presents a method for object recognition, localization and detection that uses a single convolutional neural network to locate and classify objects in images.  The basic architecture is similar to Krizhevsky’s ImageNet 2012 system, but with modifications to apply it efficiently in a “sliding window” fashion and to produce accurate bounding boxes by training a regressor to predict the precise position of the bounding box relative to the sliding window detector.  Numerous tweaks are documented for making this system work well including:  multi-scale evaluation over widely-spaced scales, custom shifting/pooling at the top layers to help compensate for spatial subsampling of the network, per-class regressors to localize objects relative to the input window, and a simple merging procedure to combine the regressed boxes into final detections.  This method is shown to achieve state-of-the-art results on ImageNet localization and detection tasks while also being relatively fast.

Overall, this paper presents a very thorough accounting of a fully functioning detection pipeline based on convnets that is the top performer on one of the toughest vision tasks around.  One of the challenges with reporting results like this is to make them reproducible, and I think this paper includes all of the details that a researcher would need to do so, which is really excellent.  

There is currently a lot of work on detection architectures (e.g., from Erhan et al.) but this one is fairly complete and high-performing.  So, while there aren’t huge new ideas here, considering the depth of experiments and the cornucopia of tricks for maximizing performance the work looks very worthwhile.

Pros:

End-to-end training of the entire detection and localization pipeline.  The decomposition into 3 clean stepping stones (classifier, localizer, detector) is a nice strategy.

State-of-the-art detection performance on Image-Net.

Cons:

Somewhat “specialized” convnet architecture to deal with subsampling issues and multi-scale (e.g., it is mentioned that the detector of Fig 11 also uses multiple scales for context)

Other:
The text is very detailed in order to make the system reproducible.  This is great, but perhaps some of the tables and minor notes [parameter settings, etc.] could be moved to an appendix to tighten up the text.