Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper proposed a novel network quantization method to reduce the bit-lengths of the network weights
and activations, which is one of the most important problem for resource-limited devices.
The presentation of this paper is well written and organized.
The problem definition is clear, i.e., relaxed quantization with the Gumbel-Softmax relaxation suffers from bias-variance trade-off depending on the temperature parameter of Gumbel-Softmax.
The proposed method to solve this problem is reasonable.
The experimental results are also convincing.

Minor comments:
1. In Table 3, "MNIST" should be LeNet-5.
2. Section 3.4 is a little dense.
3. I understand that there is little space, but embedding formulas in sentences is difficult  to read.
4. I recommend you to show the result only using SQR for  ImageNet  in Table 2 as in MNIST and CIFER10 in Table 1.
review_title: Well written and organized paper

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary
This work presents 1) Semi-Relaxed Quantization (SRQ), a method that targets learning low-bit neural networks, 2) DropBits, a method that performs dropout-like regularization on the bit width of the quantizers with an option to also automatically optimise the bit-width per layer according to the data, and 3) quantised lottery ticket hypothesis. SRQ is an extension of Relaxed Quantization (RQ), which is prior work, in two ways; firstly the authors replace the sampling from the concrete relaxation during training to deterministically selecting the mode (which is non-differentiable) and, secondly, they propose a specific straight-through gradient estimator (STE) than only propagates the gradient backwards for the elements that were selected in the forward pass. DropBits is motivated from the perspective of reducing the bias of the STE gradient estimator by randomly dropping grid points associated with a specific bit-width and then renormalising the SRQ distribution over the grid. This essentially induces stochasticity in the sampling distribution for the quantised value (which was removed before by selecting the mode in SRQ). The authors further extend DropBits in a way that allows for learning the drop probabilities for each bit-width, thus allowing for learning mixed-precision networks. Finally the authors postulate the quantised lottery ticket hypothesis, which refers to that “one can find the learned bit-width network which can perform better than the network with the same but fixed bit-widths from scratch”.

### Pros
- This work provides a set of additions that improves upon prior work
- The DropBits method is novel and allows for learning the bit-width in a straightforward manner
- The results improve upon recent works that learn quantised neural networks

### Cons
- Some claims from the authors are misleading while others are not precise
- The computational complexity of the method is not discussed 
- Some experimental settings might not be consistent with some of the baselines

### Detailed feedback
This work tackles the problem of learning quantized neural networks and the authors show empirically that their proposed method achieves good results. The DropBits extension is particularly interesting in that it allows for learning the appropriate bit-width of a given tensor via traditional pruning approaches. I also like the fact that the authors explain illustratively their proposed approach via several figures, which provide a nice boost to clarity. Nevertheless, I believe that there are still some important aspects that need to be addressed before I recommend acceptance for this work.

First of all, the comparison against the prior work on figure 1 is misleading; the authors compare the *entire categorical distribution* (i.e., the one obtained after discretising the logistic onto the quantization grid) of SRQ at Fig 1(b) with a *single sample* from the concrete relaxation of the same distribution at Fig. 1(a) right for RQ. In fact, the underlying categorical distribution will be the same for both SRQ and RQ in the specific example of figure 1. Furthermore, it is worthwhile to notice that the underlying categorical distribution (i.e., pre relaxation) does have support for the value of -a (as the p(g_i = -a) is nonzero at Fig.1 (b)), thus it is not unreasonable that there are specific samples which lead to the quantised value being -a, thus incurring larger quantization loss.

Furthermore, I believe that the discussion about SRQ misses some important points that would improve the clarity of the work if they are addressed. Selecting the most probable point in the categorical distribution for the forward pass is equivalent to rounding to the nearest grid point, which can be done much more efficiently than computing the entire categorical over the grid and then taking the argmax. In addition, this is also the same as taking sigma -> 0 for the logistic distribution that is to be discretized in the forward pass. Finally the authors argue that their novel multi-class STE reduces the variance of the gradient estimator but no formal justification is given apart from some hand-wavy arguments. Why is the variance lower if the gradient only flows through to r_imax? Furthermore, for the second point with respect to the benefits of the multi-class STE; while it does seem desirable that it aggressively clusters the weights and activations around the grid-points, I wonder how much can that hinder convergence. Do you ever observe that the weights can be prematurely “stuck” (and thus lead to a bad local minimum) and do the weights ever move further away than just the closest grid point? DropBits could potentially help with the latter part, but it would be interesting to see what happens without it.

DropBits in my opinion is the main novel idea of this work, and it is an interesting way to learn the bit-precision of each tensor in the network. I have two main points for this section in general and DropBits in particular. The main motivation behind DropBits seems to be converting the sampling distribution of SRQ (which is deterministic) to a stochastic one. If this is the case, then why have it be deterministic in the first place? You could just sample from the original categorical distribution (by, e.g., using the Gumbel-Softmax STE which gives samples exactly on the grid) in the forward pass and use your multi-class STE approximation in the backward pass.  It would be interesting to see how that fares with SRQ + DropBits and would highlight whether the main benefit of DropBits was the regularization aspect (and not that of improving the sampling distribution). As for DropBits in particular; it seems that you drop bits independently with each of the gates z_1, z_2, … (i.e., figure 3). If this is the case, then you could end up with a non-uniform grid that cannot be exactly represented as a fixed point tensor (e.g., on figure 3b you could have z_1 = 0 and z_2 = 1). If this is the case, then comparison against other approaches that use uniform quantization is not apples-to-apples.

Finally, a couple of other things that I believe should be addressed; the authors don’t make any discussions about the computational and memory complexity of the resulting algorithm. It seems that for every individual weight and activation they first have to construct the categorical distribution over the entire grid (which can quickly become very large, e.g., for 8 bits there are 256 categories), in order to take the weighted sum. This doesn’t seem to scale very well. How expensive is something like this in practice and how long do experiments take on, e.g., Imagenet? Furthermore, the quantised lottery ticket hypothesis (QLTH) is a bit peculiar. The original lottery ticket hypothesis (LTH) was about finding sparse networks at initialisation that can be trained from scratch and achieve the same accuracy as the original dense equivalents. This is different than what the authors articulate here, specifically that it is about finding sparse networks that are easier to train compared to sparse networks obtained from pruning. As a result, their QLTH seems to state the opposite than what the original LTH was about; it states that a QLT is obtained when you manage to find a network X that a.) has smaller bit width than the original network and b.) has better performance than a network initialised to the bit-width of X and trained to convergence. Following the arguments of the LTH, I would expect a QLT to be obtained when you can quantise a neural network to a specific bit-width at initialisation and when you train from scratch that particular quantised network, you obtain the same performance as the full precision equivalent. I would thus encourage the authors to clarify this point and better align with the original LTH. 

Based on the aforementioned points, I cannot at the moment recommend acceptance for this work. Nevertheless, as I believe DropBits is an interesting idea, I would encourage the authors to put in the effort and rework the paper by addressing these points over the rebuttal.

review_title: Interesting ideas, but needs some more work

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper deals with network quantization. It proposes Semi-Relaxed Quantization (SRQ) that uses a multi-class straight-through estimator to effectively reduce the bias and variance, along with a new regularization technique, DropBits that replaces dropout regularization to randomly drop the bits.  Extensive experiments are conducted to validate our method on various benchmark datasets and network architectures.

Pros:
- Reducing variance of gradient estimator in Gumbel of RQ using multi-class STE.
- A novel dropbits technique to reduce bias in gradient estimator of SRQ, as well as supporting the mixed-precision scheme.
- A "quantized winning tickets" is introduced to train probs of binary masks so that learning proper bitwise for each layer. 

Cons:
- The SRQ process seems not to be novel enough. Actually here pi is just a multi-class sigmoid-output, and y is derived directly by argmax(prob_pi) in forward process (and calculate gradients using STE.). This
is quite similar to existing quantization methods using softmax + STE. [1]
- Only 3/4 bit results of Res18/MobileNetV2 showed in ImageNet. I'd appreciate it if authors could offer more quantitive analysis on more architectures and tasks.
- More comparisons on latency/energy/flops during the training and evaluation process should be provided to validate the SRQ + DropBits.

[1] Hardware-aware Softmax Approximation for Deep Neural Networks. Xue Geng et al.

***After rebuttal and discussion

 This paper proposes a new network quantization framework. In particular, the proposed DropBits is somewhat novel. However, it lacks sufficient and accurate analysis of SRQ+DropBits.  For example,  why SRQ can reduce quantization error has not been well motivated and explained.  The definition of distribution bias is still unclear.  I think that an accurate description of terminology is crucial and required for scientific research. Hence, the paper still needs minor polishing for publishing. I would like to decrease my rating to 5.
review_title: Review of Paper3191

Meta-review: # Summary
The paper was initially well received by reviewers, remarking the new gradient estimator, a new dropbits technique and an interesting observations of better performance when the bitwidth is learned. The experimental results also look promising: showing improved training performance and test performance (including on ImageNet with ResNet-18), properties to reduce quantization error of learned weights, possibility to learn number of bits via learning stochastic bit-dropping masks.

A deeper verification of the specific methods proposed however showed principal issues:
- The methods proposed in the paper are not sufficiently justified by verifiable formal arguments. The proposed intuitive explanations are entangled and actually lead to wrong conclusions. In particular a main claim of the paper that the proposed estimator reduces bias and variance of Gumbel-Softmax estimator was shown wrong and was removed in the revision. The remaining claim that the estimator reduces quantization error is also wrong (see below). With these issues, the gradient part of the paper is largely incorrect, which is in a strong discrepancy with good experimental results.

- Other parts of the paper, comprising the remaining technical contributions are not properly positioned with respect to the SOTA and thus are not necessary novel / improving.

The main technical issues were discussed with all reviewers and were either supported or not objected. Therefore, I am confident that the submission has critical problems and must be rejected. I recommend the authors to thoroughly investigate all the raised issues (by all reviewers) before resubmitting to other venues. 

# Details

## Gradient

The overclaim of reducing bias and variance / resolving bias/variance tradeoff has been removed in the revision, but the new gradient estimator remains a central innovation proposed. It is however not justified and cannot indeed be regarded as a good estimator:

* The justification argues about the bias of the Gumbel-Softmax sampling distribution, but the proposed estimator does not use a sampling distribution in the forward pass, and thus by design cannot address this problem.

* The backward pass to use gradient in i_max only (Eq. 3) is not based on any justification at all. 

* The remaining claimed good property: "to reduce the quantization error" is, according to the definition in sect. 3.4, not a property of a gradient estimator, but of the stochastic relaxation alone. There is an experimental evidence Fig.2 that the estimator _leads_ to lowering the quantization error. This is however in a contradiction with a direct verification of the proposed estimator that was conducted:
The verification inspects gradient in a single variable $x$ and a linear loss function of the quantized variable $\hat x$.  It shows that the gradient is zero at grid points and discontinuously reverses the direction at half-grid points. Because of such zigzagging, *it does not correspond to minimizing the loss function*, i.e. not a reasonable estimator. The grid points, where the gradient vanishes, may correspond to either local minima or to local maxima of the estimator. Which of the two cases occurs depends exclusively on the sign of the incoming gradient from the loss function. For $L(\hat x) = \hat x$ we observe that the negative gradient points towards nearest grid point, but for $L(\hat x) = -\hat x$ it points away from the nearest grid point, i.e. a step would *increase the quantization error*. The implementation of this verification is attached anonymously:
https://colab.research.google.com/drive/1PibzRMXQ-NVZMUdfgTIK0Q5FxUKyxfqI?usp=sharing

* Alternative existing estimators are not sufficiently discussed: e.g. common deterministic STE, as used in quantization papers: to just treat the quantization operation as identity on the backward pass. Estimator used by Shayer et al. (2018),  Ajanthan et al. 2019 “Mirror descent view for neural network quantization”, Unbiased estimators (e.g. Yin et al. 2019 “ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables”). While unbiased estimators may still have too high variance and or be too computationally demanding for deep networks, they can be used for verification purposes. 

* The claim that it is not possible to apply unbiased estimators, in particular score function estimator, because of dependency on x is incorrect. See e.g. Schulman et al. 2015 “Gradient Estimation Using Stochastic Computation Graphs”. Many works on advanced unbiased estimators also demonstrate experiments with 2 or more layers of hidden discrete stochastic variables. From this and technical discussion with authors, it is seen that the experimental study is Sec 3.4 is very limited and erroneous. 

* The rule by which the probability mass of the dropped bits is uniformly spread over the remaining bits is not justified and appears methodologically incorrect. In Fig.4 it is not clear what bits were dropped and why the mass at $-2\alpha$ has decreased.

## Gradient and Other Techniques relative to SOTA

* The bias problem of GS estimator, detailed in Fig.1. is not novel to me, it is in fact known that the mean under the concrete distribution (of linear or non-linear objective) differs from the mean under the categorical distribution, see e.g.

Lorberbom et al. (2018) Direct Optimization through argmax for Discrete Variational Auto-Encoder (Fig.1)

Andriyash et al. (2018) Improved Gradient-Based Optimization Over Discrete Distributions

Thus analysis of individual samples in Fig.1 appears unnecessary detailed. The issue that the relaxed distribution of Gumbel-Softmax may cause a large estimation error for gradients downstream is already discussed by Louizos et al. (2018) and other works, e.g. 

Choi 2017, "Unsupervised Learning of Task-Specific Tree Structures with Tree-LSTMs" Sec 3.2

and Andriyash (2018). This later problem was previously addressed in many cases by the ST Gumbel-Softmax heuristic. This heuristic indeed performs better in CIFAR-10 experiments in the submission / Louizos (2018), which is likely to be a better tuned and more controlled experiment than ImageNet.
* More methods should be discussed that reduce the quantization error during learning. E.g.

Cong et al. (2018): “Extremely low bit neural network: Squeeze the last bit out with ADMM”, 

who include terms explicitly minimizing the 
quantization error. In fact most works quantizing network weights primarily focus on reducing the quantization error, e.g.

Nagel et al. (2019)" Data-Free Quantization Through Weight Equalization and Bias Correction

* The prior works on learning bit width should be more extensively discussed / compared to, especially if this part becomes central to the submission. E.g.

Baalen et al. (2020) “Bayesian Bits: Unifying Quantization and Pruning” (or references therein if this is considered contemporaneous).  

Courbariaux & David (2015): Training deep neural networks with low precision multiplications

* The new hypothesis for quantization is in fact similar to the effect observed elsewhere that quantizing neural networks progressively leads to better results.  E.g. 

Zhou et al. (2017) Incremental Network Quantization- Towards Lossless CNNs with Low-Precision Weights.

It is questionable whether the link to the lottery ticket hypothesis is justified, since the latter shows quite the opposite, as reviewers have pointed.
Decision: Reject