Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposed a novel non-uniform data-free quantization method.
The main Idea (which is very interesting) is to find the best non-uniform quantization intervals for both weights and activations by minimizing the reconstruction error using the proposed quantization and de-quantization method. 
The paper provides extensive experimentation on various datasets and tasks (including Image-classification and NLP) and archives promising results (outperforms  SOTA). 
strength_and_weaknesses: Strength: 

- Novelty 
- The claims are backed by mathematical explanation 
- promising results

Weaknesses: 

- The paper is heavy and very hard to read. To someone like me who is a non-native English speaker, the paper looks like a poorly translated novel that consists of unnecessary fancy words and grammars. It was hard to keep reading the paper and not getting distracted. 
- Section "3.3 FUSED DE-QUANTIZATION AND ACTIVATION FUNCTION" needs more clarification. I could not understand how the quantization works for the activations. Do we calculate $x^{\alpha}$ online during the inference for the activations? If yes (and I think the answer is yes) then how is it calculated? 
clarity,_quality,_novelty_and_reproducibility: Please read "Strength And Weaknesses" section 
summary_of_the_review: As mentioned in the Strength And Weaknesses section, I did not understand the quantization of activations. Hence, I need to rely on the answer from the authors and other reviewers comments. I am open to change my score in future.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to transform weights with power operation before quantization to improve the quantized accuracy. The motivation behind this is the bell-shaped distribution of weights, indicating that fine-grained resolution should be allocated around zero. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method against several strong baselines. 
strength_and_weaknesses: ### Strength 
* This paper is well-motivated. Bell-shaped distribution of weights is widely observed. Power transformation addresses this issue well. 
* Continuous automorphisms are highly appreciated by quantization and hardware implementation. This method leverages continuous automorphisms in a smart way. 
* Extensive experiments are conducted on various architectures and datasets with significant improvements. 


### Weaknesses
* Is $W$ in Eq. 2 a matrix? What is the precise definition of $|\cdot|^a$?
* "for instance, for both SiLU (EfficientNets) and GeLU (Image Transformers), the activations are bounded below" Are they empirical observations? How many examples are tested?
* Although the authors mentioned the overhead of power transformation (against uniform quantization) in Eq (5), it should be further analyzed in detail to help understand the trade-off. 
* Missing reference on power-based quantization [1] [2]

[1] Zhang, Sai Qian, et al. "Training for multi-resolution inference using reusable quantization terms." Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2021.
[2] Li, Yuhang, Xin Dong, and Wei Wang. "Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks." ICLR (2020).
clarity,_quality,_novelty_and_reproducibility: see above
summary_of_the_review: Both methods and results are interesting and significant. I am looking forward to feedback from the authors on my questions.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper looks at data free techniques for quantization. The framework is to look for automorphisms on the (R,x) which the authors show is essentially power functions. Then exponential representation are optimized to minimize reconstruction error. Finally, experimental results are presented to show the usefulness of the work.
strength_and_weaknesses: Strengths: - The paper proposes a novel quantization technique based on power functions. - Some theoretical analysis is performed to describe the method. - Extensive experiments are performed to evaluate the method. Weaknesses: - The mathematical derivations are hard to follow. I think there is room for improvement in terms of clarity. 
clarity,_quality,_novelty_and_reproducibility: I understand this field is moving very fast and it is hard to keep track to the most recent related works. But most of the works that were compared to are already dated by a few years. Can the authors compare to more recent works such as [1]

[1] Sakr, Charbel, et al. "Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Training." International Conference on Machine Learning. PMLR, 2022.
summary_of_the_review: The paper proposes a new quantization method, provides analyses and empirical evaluations. 

During the rebuttal and through discussions with the authors, I have raised my score to a 6.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies data-free non-uniform quantization for deep neural networks, which can help lower the number of bits that are used for encoding weights and activations. The framework is to search among automorphisms which are essentially power functions. All reviewers agreed that this work is novel and mathematically rigorous. Moreover, the extensive experiments show the good performance of the proposed method. That being said, most reviewers complained that this work was poorly written and hard to follow. I strongly suggest that the authors polish this paper in the revised version.  I suggest acceptance.
justification_for_why_not_higher_score: Its contribution is not strong enough for a spoylight paper. The writing is bad.
justification_for_why_not_lower_score: All reviewers agreed that this work is novel and mathematically rigorous. Moreover, the extensive experiments show the good performance of the proposed method.
Decision: Accept: poster