Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In the manuscript, the authors propose SQuant, which is a data-free quantization method that can apply post-training quantization (PTQ) without any backpropagation.

Specifically, SQuant is taking advantage of approximated Hessian information. Based on the assumptions and deductions in the paper, SQuant tries to optimize constrained absolute sum of error (CASE) instead of MSE.

The authors show many experimental results to validate the effectiveness of SQuant.
main_review: Strengths:
1. I think the paper is very well written, with good illustrations.
2. The related work section is thorough.
3. The notations in Section 2.1 are necessary and helpful, so as the Algorithm 1 and 2. Otherwise, the notion of blocks, layers, elements, input/output channels, kernels will make the math become super confusing.
4. The experimental results are state-of-the-art, with evaluations on various neural networks.
5. I go through the math and find them generally correct. The decomposition/approximation idea is decent.

Weaknesses:
1. SQuant is for weight and activation quantization. I think the weight quantization part is well supported, but there lacks discussion about activation quantization. Especially, how it affects the accuracy of approximations/Hessian representations used in SQuant.
2. Extra clarifications are needed for Equation 7 to 8. As I understand it, equation 7 to 8 works well for SQuant because of its iterative optimization on the three components. But generally, those two are not equivalent.
3. How is the quantization range (min, max) determined in SQuant? Would SQuant work better if combined with other orthogonal PTQ methods?
4. For completeness, I wonder about 4-bit quantization results of Shufflenet?

Small Issues:
1. In supp A, extra W

======Post Rebuttal======

I think the rebuttal from the authors makes sense and can address my concerns (excluding non-technical issues such as novelty and potential influence). As a result, I maintain my score tending to accept.
summary_of_the_review: I think the manuscript is technically sound with a solid understanding of the PTQ problem. The experimental results are quite good. Since there are still clarifications to be made, and the fact that similar deductions and flipping methods exist in previous works, I recommend a weak acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a data-free quantization method based on the second-order Taylor expansion of  loss, where the Hessian matrix is approximated with different levels: element-wise, kernel-wise and channel-wise. The authors progressively determine the quantized weights from element-wise to kernel-wise and then to channel-wise. The derivation and solution of the quantization are novel. Empirical results show that the proposed method outperforms recent data-free methods.


main_review: The paper is overall clearly structured and written. The proposed method is well-motivated, and the resultant flipping solution is novel as far as I know. This paper provides a brilliant way to directly use discrete optimization for quantization instead of conventional training with gradient backpropagation. However, the following points still require some more clarifications.
- It is not clear why the first-order term is omitted in equation (1).
- It is not clear why the scaling parameters like e_{n,i}, k_n and c_m
are omitted in equation (7). The solution from the proposed progressive optimization is not the only solution, and other solutions may still depend on these scaling parameters.
summary_of_the_review: This paper is well written, and the proposed method is novel.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new data-free quantization method that does not require back-propagation nor fine-tuning. The key idea is adopting Hessian-based optimization that can be decomposed into three parts (SQuant-E, K, and C) corresponding to the three dimensions of the weight tensor (using a few approximations, such as cross-layer independence to simplify the optimization). Then, instead of MSE, the authors introduce CASE (constrained ASE) of weight perturbation. The experimental results show that the proposed DFQ method outperforms even GDFQ that is basically a kind of QAT. The proposed technique is especially useful for a low-bit quantization.
main_review: This paper is clearly written and well-motivated. After a successful approximation of the Hessian-based approach, the authors discuss how to minimize the necessity of activation distribution information in Section 3 (assuming that input feature maps auto-correlate with each other in a similar way). The experimental results are impressive in Section 4.

(1) What are the limitations of this work? The quality of the proposed method would depend on the validity of the assumptions to ignore activation distribution. If a summary of limitations and assumptions of input distribution is provided, it would be helpful to understand why this work is only limited to certain CNN models as written in the paper. Since Transformers and MLPs are increasingly utilized for the area that has been dominated by CNN models, it would be necessary to describe which dataset or model architecture is best performed by the proposed method.

(2) The flipping approach is limited to up or down with a step size of 1. Is there any chance to improve the accuracy further if we increase the step size (i.e., allowing wider flipping approaches)?

(3) Flipping approach has been introduced to a few previous PTQ techniques (while such techniques usually assume that a calibration set is provided). How close is the proposed work to those previous flipping-based PTQ methods in terms of model accuracy? If the difference is marginal, does it mean that investigating input data distribution is inherently less important for CNN models? Is there any chance such a difference (between data-free PTQ and calibration-based PTQ) can be larger depending on the characteristics of the dataset? Since all experimental data in this paper is given for ImageNet only, this paper might present an impression that the proposed method might be optimized for ImageNet-like datasets.
summary_of_the_review: This paper introduces a few reasonable assumptions to alleviate the efforts to estimate the input data distribution. Successful approximations lead to improved model accuracy even with a very small number of bits to represent weights.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose a data-free quantization algorithm of deep neural networks called SQuant. The main idea of SQuant is to decompose the Hessian-based optimization objective into three components: element-wise, kernel-wise and channel-wise components. In order to jointly optimize these three objective functions, a constrained absolute sum of error (CASE) is studied and a progressive algorithm is used. Experiment results show that SQuant algorithm is able to keep higher accuracy with the same number of bits, compared to several baseline algorithms.
main_review: Strength: The main strength of this paper is the introduction of multi-scale approximation of Hessian matrix. Due to the scalability issue, it is impossible to store the entire matrix. Traditional methods usually only keep track of the diagonal terms. Inspired by matrix decomposition ideas, this paper adopt a diagonal + block-wise diagonal +  low rank approximation, which is a good idea. Experiment results also show that the matrix decomposition idea work well.

Weakness: I think there are some flaws in the derivation of the optimization objective.

1. In formula (2), H^{W_m^l} = l_m x^l (x^l)^T. But from formula (3) to formula (4), E[H^{W_m^l}] is replaced by E[x^l (x^l)^T], where l_m disappears. Why this step hold true?
2. In formula (7), the objective is \sum_{n, i} e_{n, i} (\Delta W_{m,n,i}^l)^2 + \sum_n k_n \Delta W_{m,n,:}^l J_k \Delta W_{m,n,:}^l^T + c_m \Delta W_{m,:}^l J_NK W_{m,:}^l^T, but in formula (8), the coefficients e_{n, i}, k_n and c_m disappear. I understand that all the coefficients are positive from Appendix A.2., but as long as they are not all equal, the objective should be different without these coefficients. I don't quite understand how we get (8) from (7). 
summary_of_the_review: The motivation to decompose the Hessian matrix into diagonal, block-wise diagonal and low-rank components is the crucial contribution of this paper. Experiment results show that the multi-scale optimization objective lead to good performance after quantization. I don't follow some steps in the derivations of the optimization objective. This paper will be better if the authors can illustrate these steps well.
review_title: null

Meta-review: The authors propose a data-free quantization method that can be applied post-training quantization without backpropagation.  The method takes advantage of approximate Hessian information in a certain scalable approximate way. Based on the assumptions and deductions in the paper, SQuant tries to optimize constrained absolute sum of error (CASE) instead of MSE.  There are good empirical results showing the effectiveness of the method, and the paper is well written, and the method should be of broad interest.
Decision: Accept (Poster)