Paper:
BIG-LITTLE NET: AN EFFICIENT MULTI-SCALE FEATURE REPRESENTATION FOR VISUAL AND SPEECH RECOGNITION
In this paper, we propose a novel Convolutional Neural Network (CNN) architecture for learning multi-scale feature representations with good tradeoffs between speed and accuracy. This is achieved by using a multi-branch network, which has different computational complexity at different branches with different resolutions. Through frequent merging of features from branches at distinct scales, our model obtains multi-scale features while using less computation. The proposed approach demonstrates improvement of model efficiency and performance on both object recognition and speech recognition tasks, using popular architectures including ResNet, ResNeXt and SEResNeXt. For object recognition, our approach reduces computation by 1/3 while improving accuracy significantly over 1% point than the baselines, and the computational savings can be higher up to 1/2 without compromising the accuracy. Our model also surpasses state-of-the-art CNN acceleration approaches by a large margin in terms of accuracy and FLOPs. On the task of speech recognition, our proposed multi-scale CNNs save 30% FLOPs with slightly better word error rates, showing good generalization across domains.

Review:
This paper presents a novel multi-scale architecture that achieves a better trade-off speed/accuracy than most of the previous models. The main idea is to decompose a convolution block into multiple resolutions and trade computation for resolution, i.e. low computation for high resolution representations and higher computation for low resolution representations. In this way the low resolution can focus on having more layers and channels, but coarsely, while the high resolution can keep all the image details, but with a smaller representation. The branches (normally two) are merged at the end of each block with linear combination at high resolution. Results for image classification on ImageNet with different network architectures and for speech recognition on Switchboard show the accuracy and speed of the proposed model.
Pros:
- The idea makes sense and it seems GPU friendly in the sense that the FLOPs reduction can be easily converted in a real speed-up
- Results show that the joint use of two resolution can provide better accuracy and lower computational cost, which is normally quite difficult to obtain
- The paper is well written and experiments are well presented.
- The appendix shows many interesting additional experiments
Cons:
- The improvement in performance and speed is not exceptional, but steady on all models.
- Alpha and beta seem to be two hyper-parameters that need to be tuned for each layer.
Overall evaluation:
Globally the paper seems well presented, with an interesting idea and many thorough experiments that show the validity of the approach. In my opinion this paper deserves to be published.
Additional comments:
- - In the introduction (top of pag. 2) and in the contributions, the advantages of this approach are explained in a different manner that can be confusing. More precisely in the introduction the authors say that bL-Net yeald 2x computational saving with better accuracy. In the contributions they say that the savings in computation can be up to 1/2 with no loss in accuracy.