Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes an alternative to straight-through estimator by considering the geometry of the likelihood. The proposed method can be viewed as a natural gradient descent algorithm on the Riemannian manifold. Compared to STE, the proposed estimator seems to penalize the quantities that are far from the quantization boundary. Slightly better accuracy results are reported for training 1-bit weight and activation neural networks. 
main_review: Strengths
- Investigating principled gradient estimators for the discrete quantization is an important problem to study. 
- The idea of utilizing manifold geometry makes sense.

Weaknesses
- the actual used manifold in Sec. 3.3 does not seem to use the Fisher information. Why is the weak curvature metric better than the Euclidean distance?
- It looks like that the proposed estimator in Sec. 3.3 only applies to 1-bit quantization. How does the proposed approach generalize to multi-bit quantization?
- Some convergence theorems are required to justify that learning with the proposed estimator can indeed converge to better models.
- Is the vanilla straight-through estimator compared in the experiments?
summary_of_the_review: The idea of the paper makes sense, but the theoretical and experimental results cannot support the usefulness of the proposed method well.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The main motivation for this paper is the gradient mismatch problem, which emerges from using the Straight-Through Estimator (STE) in training quantized neural networks and leads to an unstable training process. To deal with that gradient mismatch problem, the authors introduce the Manifold Quantization (ManiQuant) that embeds Riemannian manifolds into the STE. Specifically, the ManiQuant associates the gradient mismatch problem with Fisher information, which can then be exploited to alleviate that problem. Considering the high cost when inverting the Fisher information, the authors present an alternative simpler method related to Hyperbolic divergence and weak curvature manifold.


main_review: In Theorem 1 (more specifically in the corresponding proofs in Appendix A), it seems Eq (8) is only $\textbf{approximately}$ unbiased, because of the Taylor expansion in Eq (21) in Appendix A. It's not rigorous to state the estimator in Eq (8) is unbiased. At least, this should be discussed carefully.

Honestly, I am confused about the underlying logic in Section 3.1. Of course, the gradient mismatch problem is caused by using the STE. But why does the root of that problem ONLY come from the STE variance (or its Cramer-Rao Lower Bound)?

In Eq (10), from the second row to the third row, you cannot simply replace the log-likelihood with the loss function L in general. Please elaborate on it.

In Figure 2, it seems the demonstrated results are from a single run. Please add the error bar that is shown in the tables.

Theoretically, why using the weak curvature gradient could deliver better performance?
summary_of_the_review: The underlying logic is not clear (see the detailed comments above). 
Some statements/derivations are not rigorous and thus are not convincing.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a new gradient estimation method for training quantized neural networks (QNN). It alleviates the gradient mismatch problem that occurred in previous quantization methods that use the Straight-Through Estimator.

main_review: I find the paper clearly written. As a non-expert, I think that the questions that the gradient mismatch problem that the author aims to address is significant and the experiment results convincingly demonstrate the superior performance of the proposed gradient estimation method.

My main suggestion for the authors is to provide more context for the benefits of training QNN and position their contribution in this context. For instance, is the goal of training QNN to gain training speed, reduce GPU memory, or reduce the size of the network for storage purposes? How much can the proposed method improve upon these metrics (other than just accuracy), compared to full-precision networks? Indeed, the proposed method makes use of the natural gradients, which, to my knowledge, are much more computationally expensive to evaluate than first-order optimization methods such as the plain SGD. Can the proposed method improve training speed or memory usage upon a full-precision network trained with vanilla SGD? It will be great to have a table (similar to Table 2 and Table 3) to list the acceleration or storage gain of different methods and compare them with vanilla training of full-precision networks. There is a small subsection (subsection 4.5) that discusses the training time. This subsection, in my opinion, can be expanded, for instance, to include training time required for a full-precision network using SGD. 

Minor: In the first paragraph of Section 3 on page 3, the authors mentioned that the quantization function is a **one-to-one** mapping from full-precision values to quantized values. Why is the quantization function a one-on-one map? I supposed that a single quantized value may correspond to multiple pre-quantized, full-precision values. 
summary_of_the_review: As a non-expert, I find that the method proposed in the paper is novel and empirically well-tested. However, the proposed approach for training QNNs seems to be computationally expensive on its own, and I am not sure if the required computational budget defeats the purpose of training QNN -- it will be great if the authors can help me understand in this regard.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed to improve the gradient of quantization operator in quantization-aware training (QAT). Basically, it pointed out that the Euclidean assumption in general gradient (used by Straight-Through Estimator) can not reflect the curvature in the loss surface. Instead, it revised the gradient (natural gradient) based on mainfold learning. Due to the huge cost of Fisher Information Matrix (FIM) used to achieve natural gradient, it proposed to approximate it by weak curvature embedding. 
main_review: Pros:
1. The motivation of this paper is straight-forward and interesting: Basically, it first pointed out that quantization error is introduced by variance of STE, which can be upper bounded by the FIM. Then it related FIM with STE and kl-divergence.

Questions:
1. Author mentioned that FIM should be increased to alleviate the gradient mismatch problem. But the natural gradient introduced in Eq.10 does not seems to contribute to this goal. How the increase of FIM is related to the training ojective of model quantization?
2. How does optimizer interact with the proposed gradient? In experiments, author used SGD with momentum (SGD-M). Does that mean the natural gradient is firstly attained by Eq.18, then SGD-M is applied to the gradient to achieve gradient used in parameters update ? How if Adam is applied in the experiments since Adamm affects much to the initial gradient.
3. Besides, is SGD-M still applied in ImageNet experiments? Accuracy in table 3 is too high for SGD-M in ResNet18.
summary_of_the_review: It is interesting to incorporate manifold learning in QAT, but more concerete analysis (experiments) in deep learning field should be conducted.
review_title: null

Meta-review: The paper seeks to improve straight-through estimators by combining them with the ideas for correcting the step direction to be closer to a natural gradient.
 
While some (modest) improvements are demonstrated experimentally, the paper critically lacks technical correctness and has quite some gaps when trying to derive the algorithm from the natural gradient and Rao-Cramer bound. See public comments by reviewers and AC. The algorithm ends up to be a mirror descent with a mirror map, which is cheap to compute but not particularly well motivated. Moreover application of mirror descent to the activations (unlike the weights) is not well justified. The paper is rather unclear and hard to read also language-wise. Please proofread _before_ submitting.
Decision: Reject