From: NIPS.cc/2018/Workshop/CDNNRIA/Paper14/Authors
comment: Thanks for the review and pointing out the relevant work [1,2]. 

Our proximal gradient method and the proximal Newton method proposed in [1,2] are nicely complementary. Algorithmically, each iterate of our ProxQuant is slightly less expensive because we don't need to store and access the (diagonal) approximate Hessian (but it may be interesting to compare them in wall time). Experimentally, we have results on word-level LSTMs whereas [1,2] focuses on character-level LSTMs for recurrent nets. Despite these differences, we think that both papers point to the direction that straight-through gradient method may not (and should not!) be the choice by default; significant improvement can be obtained when the algorithm takes more gradient (and Hessian) information into account.

In addition to measuring performance, our paper characterizes of the behavior between prox-gradient method and the straight-through gradient method, demonstrating that prox-gradient has more optimization stability than BinaryConnect for binary quantization (Section 4 and Appendix G). We conjecture that such a better optimization stability may also be enjoyed by the proximal Newton method in [1,2]. 

We have added a prior work section in Appendix B and addressed this work. We would be glad to perform an experimental comparison between our method and [1,2] in the camera ready version.

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper14/Authors
comment: Thanks for the thoughtful review and suggestions. 

We have substantially expanded the appendix to make the submission more self-consistent. In particular, Appendices A - D now provides a thorough theoretical motivation of why STE might be bad (A), prior work review (B), notation (C), preliminary derivations of our method (D), while Appendix E describes in detail the ProxQuant algorithm (choice of regularizers and derivation of prox operators).

Specifically:
(1) ProxQuant is now in Appendix E;
(2) \tilde{\nabla}L stands for the mini-batch stochastic gradient (as opposed to \nabla L, the full gradient). We explain this in Appendix C;
(3) The connection STE=dual-averaging, as well as preliminaries for our method, is presented the Appendix D.

We will integrate these appendices back to the main text in our camera ready version upon acceptance.

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper14/AnonReviewer1
title: Major problem of this paper is that it is not self-consistent in this submission status
review: This paper proposes ProxQuant to replace STE for training quantized neural networks, and demonstrate the effectiveness on ResNet and LSTM. 

The major problem of this paper is that it is not self-consistent in this submission status. 
A lot of theoretic part are missing. For instance,
(1) We even do not know what exact the ProxQuant is in the submission. 
(2) the idea of the notion "\tilde{\nabla}L" in Eq(1) and (2)
(3) The author claims STE is identical to "Nesterov’s dual-averaging" in the abstract, but we did not see anything in the paper and appendix. 

The author should provide sufficient information for the theoretic part in the appendix even though there is page space limitation. 
Although there might be great theoretic contribution in the work itself, the current submission is too rush to make me give a higher rating. 
rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/CDNNRIA/Paper14/AnonReviewer2
title: Training binary/ternary neural networks with a proximal gradient method is not new.
review: This paper proposes to train quantized neural network with a proximal gradient method. Both binary and ternary quantization tasks are considered in the experiments.

The main issue of this paper is the novelty. Training binary/ternary neural networks with proximal Newton methods has already been explored in [1]/[2] respectively.

Please clarify the difference of your method against them theoretically and experimentally.

Besides, the algorithm part is very coarse. A lot of details (such as notation definition and how a network is quantized into its binary/ternary version) are missing.


Ref:
[1] L. Hou, Q. Yao, and J. T. Kwok. Loss-aware binarization of deep networks. In ICLR, 2017
[2] L. Hou and J. T. Kwok. Loss-aware weight quantization of deep networks. In ICLR, 2018

rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct