Network In Network
Authors: Min Lin, Qiang Chen, Shuicheng Yan
ICLR 2014

Abstract: We propose a novel network structure called 'Network In Network' (NIN) to enhance the model discriminability for local receptive fields. The conventional convolutional layer uses linear filters followed by a nonlinear activation function to scan the input. Instead, we build micro neural networks with more complex structures to handle the variance of the local receptive fields. We instantiate the micro neural network with a nonlinear multiple layer structure which is a potent function approximator. The feature maps are obtained by sliding the micro networks over the input in a similar manner of CNN and then fed into the next layer. The deep NIN is thus implemented as stacking of multiple sliding micro neural networks. With the enhanced local modeling via micro network, we are able to utilize global average pooling over feature maps in the classification layer, which is more interpretable and less prone to overfitting than traditional fully connected layers. We demonstrated state-of-the-art classification performances with NIN on CIFAR-10, CIFAR-100 and SVHN datasets.


Peer review:
[Review]:
I doubt that the mlpconv layers can be easily implemented by successive 1x1 conv layers. In a 1x1 conv layer, the lower feature maps and the upper feature maps at each location are fully-connected.  For examples, 5x5 conv - 1x1 conv - 1x1 conv is equivalent to 5x5 mlpconv layers with 3 local layers.

Of course, this work is still interesting and valuable even though my thinking is correct. But in that case, it can be easily implemented by the ordinary CNN packages.