Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: quality: this paper is of good quality
clarity: this paper is very clear but contains a few minor typos/grammatical mistakes (missing -s for plurals, etc.)
originality: this paper is original
significance: this paper is significant

PROS
- Using ECC theory for reducing the memory footprint of a neural network seems both intuitive and innovative, while being grounded in well-understood theory.
- The authors address a consequence of current approaches to neural network pruning, i.e., the high cost of sparse matrix index storage.
- The results are extensive and convincing.

CONS
- The authors mention in the introduction that this encoding can speed up inference by allowing efficient parallel sparse-to-dense matrix conversion, and hence batch inference, but do not provide any experimental confirmation.

Main questions
- It is not immediately clear to me why the objective function (2) correlates to a good accuracy of the pruned network. Did you try out other functions before settling on this one, or is there a larger reason for which (2) is a logical choice? 
- On a related note, I would find a plot of the final objective value assigned to a pruning scheme compared to the true network accuracy very helpful in understanding how these two correlate.
- Could this approach be generalized to RNNs?
- How long does the Viterbi pruning algorithm take, as it explores all 2^p possible prunings?
- How difficult is it to tune the pruning algorithm hyper-parameters?
review_title: The authors use Viterbi encoding to dramatically compress the sparse matrix index of a pruned network, reducing one of the main memory overheads of a pruned neural network and speeding up inference in the parallel setting.

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes VCM, a novel way to store sparse matrices that is based on the Viterbi Decompressor. Only a subset of sparse matrices can be represented in the VCM format, however, unlike CSR format, it allows for faster parallel decoding and requires much less index space. The authors also propose a novel method of pruning of neural network that constructs an (sub)optimal (w.r.t. a weight magnitude based loss) Viterbi-compressed matrix given the weights of a pretrained DNN.
VCM is an interesting analog to the conventional CSR format that may be more computationally efficient given particular software and/or hardware implementations of the Viterbi Decompressor. However, the empirical study of possible acceleration remains as an open question.
However, I have a major concern regarding the efficiency of the pruning procedure. Authors report practically the same level of sparsity, as the pruning procedure from the Deep Compression paper. Both the proposed Viterbi-based pruning, and Deep Compression pruning belong to the previous era of pruning methods. They separate the pruning procedure and the training procedure, so that the model is not trained end-to-end. However, during the last two years a lot of new adaptive pruning methods have been developed, e.g. Dynamic Network Surgery, Soft Weight Sharing, and Sparse Variational DropOut. All of them in some sense incorporate the pruning procedure into the training procedure and achieve a much higher level of sparsity (e.g. DC achieves ~13x compression of LeNet5, and SVDO achieves ~280x compression of the same network). Therefore the reported 35-50% compression of the index storage is not very significant.
It is not clear whether it is possible to take a very sparse matrix and transform it into the VCM format without a high accuracy degradation. It is also not clear whether the VCM format would be efficient for storage of extremely sparse matrices, as they would likely be more sensitive to the mismatch of the original sparsity mask, and the best possible VCM sparsity mask. Therefore I’m concerned whether it would be possible to achieve a close-to-SotA level of compression using this method, and it is not yet clear whether this method can be used for practical acceleration or not.
The paper presents an interesting idea that potentially has useful applications, however the experiments are not convincing enough.
review_title: Viterbi-based pruning review

final_score: rating: 6: Marginally above acceptance threshold
review_content: It seems like the authors have carefully thought about this problem, and have come up with some elegant solutions, but I am not sold on whether it's an appropriate match for this conference, mainly because it's not clear how many machine learning people will be interested in this approach.

There was a time about 2 or 3 years ago when sparse-matrix approaches seemed to have a lot of promise, but I get the impression that a lot of people have moved on.  The issue is that it's hard to construct a scenario where it makes sense from a speed or memory standpoint to do this.  The authors seem to have found a way to substantially compress the indexes, but it's not clear to me that this really ends up solving any practical problem.  Towards the end of the paper I see mention of a 38.1% reduction in matrix size.  That is way too little to make sense in any practical application, especially when you consider the overhead of decompression.   It seems to me that you could easily get a factor of 4 to 8 of compression just by finding a suitable way to encode the floating-point numbers in many fewer bits (since the weight parameters are quite Gaussian-distributed and don't need to be that accurate).

review_title: Solid work but question its usefulness

Meta-review: The paper proposes a new sparse matrix representation based on Viterbi algorithm with high and fixed index compression ratio regardless of the pruning rate.  The method allows for faster parallel decoding and achieves improved compression of index data storage requirement over existing methods (e.g., magnitude-based pruning) while maintaining the pruning rate. The quality of paper seems solid and of interest to a subset of the ICLR audience.
Decision: Accept (Poster)