Title: No title found

Abstract: No abstract found

:



Figures and Tables:

Peer review (iclr2020):
This paper proposes to optimize for a sparse and quantized network under certain memory constraints. Experiments are performed on typical image classification data sets CIFAR-10 and ImageNet. The authors show that the proposed method can achieve very high compression rate without accuracy degradation. The paper is clearly written, and the compression rate is very impressive. One of my main concerns is that the training can be very expensive due to the expensive alternating steps in the ADMM solver. For instance, in section 3.3 which describes the solver for problem (5), though the authors claim that the sorting operation involved can be efficiently done in GPU, the O(n(log n)) complexity can be even larger than the multiplication between the weights and intermediate activations. From steps 13 - 15, it is possible that the proposed ADMM solver does not converge. Can the authors discuss the convergence properties? How does the solution of each alternating step affects the final convergence of the whole algorithm? How often does the algorithm not converge? Will brutely quantize the weights result in undesirable performance? Another concern is that though combining pruning and quantization can greatly increase the compression rate, the actual speedup of the inference highly depends on the hardware or operation design. What is the inference speed of the proposed method?