Some Improvements on Deep Convolutional Neural Network Based Image Classification
Authors: Andrew Howard
ICLR 2014

Abstract: We investigate multiple techniques to improve upon the current state of the art deep convolutional neural network based image classification pipeline. The techiques include adding more image transformations to training data, adding more transformations to generate additional predictions at test time and using complementary models applied to higher resolution images. This paper summarizes our entry in the Imagenet Large Scale Visual Recognition Challenge 2013. Our system achieved a top 5 classification error rate of 13.55% using no external data which is over a 20% relative improvement on the previous year's winner.


Peer review:
[Review]:
This paper presents a few simple tricks to improve over Krizhevsky's model:
- use full image instead of cropped square by Krizhevsky.
- jitter in image contrast, brightness and color.
- add scales at testing time (+ greedy algorithm to keep it fast).
- extra high-resolution model trained on scaled up patches (essentially adding scales at training time).
- turning off dropout for the last learning rate decrease.

Novelty: nothing new here, just the application of existing techniques.

Pros:
- very good result on ILSVRC13 classification.

Cons:
- the improvements proposed are not novel but the paper can be made interesting if details are given. For example, instead of just mentioning contrast, brightness and color jitter, explain precisely what operations are taking place here. And what do the .5 and 1.5 values correspond to?
- the greedy algorithm is missing some details: 'starts with the best prediction', how is best defined here? highest confidence? out of which views? how many views are initially computed?
- the training images for the high-res model differs from run-time, scaling up patches of 128x128 is in my opinion not as good as using the full-resolution image directly like at run-time. Although this approach possibly introduces noise that is beneficial for the case of images smaller than 224x224 which need to be scaled up at test time. And as mentioned, this augments the data considerably so it is a good thing.