Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: The SqueezeNet seems to be an application of ideas from inception modules (CNN Microarchitecture) and ResNet modules (CNN Macroarchitecture). Question: how does SqueezeNet performs compared to GoogLeNet and ResNet in terms of accuracy?
review_title: SqueezeNet versus other models than AlexNet

final_score: rating: 7: Good paper, accept
review_content: Summary: The paper presents a smaller CNN architecture called SqueezeNet for embedded deployment. The paper explores CNN macroarchitecture and microarchitecture to develop SqueezeNet, which is composed of fire modules.

Pros: 
Achieves x50 less memory usage than AlexNet while keeping similar accuracy.

Cons & Questions:
Complex by-pass has less accuracy than simple by-pass. And simple by-pass is like ResNet bottlenecks and complex by-pass is like inception modules in GoogLeNet. Can we say that these two valiants of SqueezeNet are adaptation of concepts seen in GoogLeNet and ResNet? If so, then shouldn’t be there a SqueezeNet like model that achieves similar accuracy compared with GoogLeNet and ResNet?

review_title: Are fire modules an application of concepts from GoogLeNet and ResNet?

final_score: rating: 5: Marginally below acceptance threshold
review_content: Strengths
-- An interesting proposal for a smaller CNN architecture designed for embedded CNN applications. 
-- Balanced exploration of CNN macroarchitecture and microarchitecture with fire modules.
-- x50 less memory usage than AlexNet, keeping similar accuracy 
-- strong experimental results

Weaknesses
--Would be nice to test Sqeezenet on multiple tasks

--lack of insights and rigorous analysis into what factors are responsible for the success of SqueezeNet. For example, how are ResNet and GoogleNet connected to the current architecture? Another old paper (Analysis of correlation structure for a neural predictive model with application to speech recognition, Neural Networks, 1994) also showed that the “by-pass” architecture by mixing linear and nonlinear prediction terms improves long term dependency in NN based on rigorous perturbation analysis. Can the current work be placed more rigorously on theoretical analysis?

review_title: excellent empirical work with potential impact, but the impact would be greater if insights and analysis can be provided by reading back earlier work aimed to analyse the “by-pass” architecture by mixing linear and nonlinear predictions similar to the proposed architecture 

final_score: rating: 7: Good paper, accept
review_content: The Squeezenet paper came out in Feb 2016, and I read it with interest. It has a series of completely reasonable engineering suggestions for how to save parameter memory for CNNs for object recognition (imagenet). The suggestions make a lot of sense, and provide an excellent compression of about 50x versus AlexNet. (Looks like ~500x if combined with Han, 2015). So, very nice results, definitely worth publishing.

Since the arxiv paper came out, people have noticed and worked to extend the paper. This is already evidence that this paper will have impact --- and deserves to have a permanent published home.

On the negative side, the architecture was only tested on ImageNet -- unclear whether the ideas transfer to other tasks (e.g., audio or text recognition). And, as with many other architecture-tweaking papers, there is no real mathematical or theoretical support for the ideas: they are just sensible and empirically work.

Oh the whole, I think the paper deserves to appear at ICLR, being in the mainline of work on deep learning architectures.
review_title: Reasonable convnet engineering paper

final_score: rating: null
review_content: How does your method compare with Knowledge Distillation method?

review_title: Comparisons with Knowledge Distillation

final_score: rating: null
review_content: The authors analyze the amount of space required for storing the parameters of SqueezeNet (which is valuable to estimate for downloading speed, etc.) But, the Gschwend thesis also analyzes the amount of space required to store the activations during run time. Gschwend's results seem to show that the activation memory is much larger than the parameter memory.

Comments? Is Gschwend right? If so, can you add this to the paper?
review_title: Memory for storing activations?

Meta-review: The paper proposes a ConvNet architecture ("SqueezeNet") and a building block ("Fire module") aimed at reducing the model size while maintaining the AlexNet level of accuracy. The novelty of the submission is very limited as very similar design choices have already been used for model complexity reduction in Inception and ResNet. Because of this, we recommend rejection and invite the authors to further develop their method.
Decision: Reject